{"slug": "i-built-a-simple-rag-app-with-langchain-openai-and-pinecone", "title": "I Built a Simple RAG App with LangChain, OpenAI, and Pinecone", "summary": "A developer built a simple Retrieval-Augmented Generation (RAG) application using LangChain, OpenAI embeddings, and Pinecone. The app ingests text documents by splitting them into chunks, converting them to embeddings, and storing them in Pinecone. When a user asks a question, the app retrieves relevant chunks and feeds them to an AI model to generate grounded answers.", "body_md": "Large language models know a lot, but they do not automatically know the contents of our private files, company documents, notes, or recently written articles.\n\nThis is where Retrieval-Augmented Generation, commonly called RAG, becomes useful.\n\nIn this project, I built a small RAG application that:\n\nReads a text document.\n\nBreaks the document into smaller sections.\n\nConverts those sections into numerical representations called embeddings.\n\nStores the embeddings in Pinecone.\n\nFinds the most relevant sections when a user asks a question.\n\nGives those sections to an AI model so it can produce a grounded answer. A grounded answer is an answer based on the retrieved source material. RAG reduces unsupported answers, but it does not guarantee that every response will be correct\n\nI used:\n\nPython for the application\n\nLangChain to connect the different steps\n\nOpenAI embeddings to represent text as numbers\n\nPinecone to store and search those embeddings\n\nuv to manage the Python project and dependencies\n\nIn this article, I will explain how the application works, how the different components fit together, and what I learned while debugging it.\n\nWhat Is RAG?\n\nPress enter or click to view image in full size\n\nRAG Pipeline\n\nImagine that you are taking an open-book examination.\n\nWithout a book, you answer only from memory. You may know the answer, but you may also forget something or provide incorrect information.\n\nPress enter or click to view image in full size\n\nDebugging using LangSmith\n\nWith an open book, you can:\n\nRead the question.\n\nSearch for the relevant page.\n\nUse that information to write the answer.\n\nA RAG application works in a similar way.\n\nThe language model is the person answering the question. The vector database acts like a searchable book. Before answering, the application finds the most relevant information and gives it to the model.\n\nThe model does not need to memorize the entire document. It only receives the sections that are likely to contain the answer.\n\nThe Two Main Parts of a RAG Application\n\nA basic RAG application has two phases.\n\nPhase 1: Ingestion\n\nIngestion means preparing the documents so that they can be searched later.\n\nText document\n\n↓\n\nLoad the document\n\n↓\n\nSplit it into smaller chunks\n\n↓\n\nConvert each chunk into an embedding\n\n↓\n\nStore the embeddings in Pinecone\n\nThis process usually runs when documents are added or updated.\n\nPhase 2: Retrieval and Answer Generation\n\nThis phase runs whenever a user asks a question.\n\nUser question\n\n↓\n\nConvert the question into an embedding\n\n↓\n\nSearch Pinecone for similar document chunks\n\n↓\n\nAdd those chunks to the prompt\n\n↓\n\nAsk the language model to answer\n\nKeeping these phases separate makes the application easier to understand.\n\nIngestion prepares the knowledge. Retrieval uses that knowledge.\n\nWhat Is an Embedding?\n\nPress enter or click to view image in full size\n\nComputers cannot directly compare the meaning of sentences in the same way people do.\n\nAn embedding model converts text into a list of numbers called a vector.\n\nFor example:\n\n\"How do I reset my password?\"\n\nmight become something that looks like:\n\n[0.12, -0.48, 0.77, 0.03, ...]\n\nThe actual vector contains hundreds or thousands of numbers.\n\nSentences with similar meanings usually receive vectors that are close to each other. This allows the application to search by meaning rather than only by exact keywords.\n\nPress enter or click to view image in full size\n\nHow embeddings work in real.\n\nLangChain provides a standard interface for embedding models, while OpenAI’s text-embedding-3 models allow developers to control the number of dimensions returned.\n\nWhat Is Pinecone?\n\nPress enter or click to view image in full size\n\nPinecone is a vector database.\n\nA normal database is excellent at questions such as:\n\nFind the user whose ID is 123.\n\nA vector database is useful for questions such as:\n\nFind the document sections whose meaning is closest to this question.\n\nPinecone stores the document vectors and searches for the vectors that are closest to the user’s question.\n\nThe Pinecone index and the embedding model must use the same vector dimension. Otherwise, Pinecone cannot store or compare the vectors.\n\nProject Structure\n\nNow that the basic idea is clear, let us look at how the application is structured and how each part of the pipeline works.\n\nMy project contains two main Python files:\n\nrag-project/\n\n├── rag-gist/\n\n│ ├── ingest.py\n\n│ ├── main.py\n\n│ └── mediumblog.txt\n\n├── .env\n\n├── .gitignore\n\n└── pyproject.toml\n\nHere is what each file does:\n\nmediumblog.txt contains the source information.\n\ningest.py loads and stores the document in Pinecone.\n\nmain.py accepts a question and generates an answer.\n\n.env stores API keys.\n\n.gitignore prevents secrets from being committed.\n\npyproject.toml contains the project dependencies.\n\nThe project contains separate scripts for document ingestion and question answering.\n\nStep 1: Install the Dependencies\n\nI used uv to manage the project.\n\nThe required packages can be installed with:\n\nuv add langchain \\\n\nlangchain-community \\\n\nlangchain-openai \\\n\nlangchain-pinecone \\\n\nlangchain-text-splitters \\\n\npython-dotenv\n\nThe main packages have different responsibilities:\n\nlangchain-openai connects LangChain to OpenAI models.\n\nlangchain-pinecone connects LangChain to Pinecone.\n\nlangchain-community provides document loaders.\n\nlangchain-text-splitters provides tools for dividing documents into chunks.\n\npython-dotenv loads environment variables from a .env file.\n\nLLM libraries change quickly, so it is better to keep the generated lock file in the repository instead of relying only on version numbers written in an article.\n\nStep 2: Add the API Keys\n\nCreate a .env file in the project root:\n\nOPENAI_API_KEY=your_openai_api_key\n\nPINECONE_API_KEY=your_pinecone_api_key\n\nNever upload this file to GitHub.\n\nAdd it to .gitignore:\n\n.env\n\n.venv/\n\n**pycache**/\n\nYou can also create a safe example file called .env.example:\n\nOPENAI_API_KEY=\n\nPINECONE_API_KEY=\n\nThis tells other developers which variables are required without exposing the real values.\n\nStep 3: Create the Pinecone Index\n\nBefore running the ingestion script, create a dense Pinecone index.\n\nPinecone does not generate the answer. Its job is to find the document chunks that are most relevant to the user’s question.\n\nFor this project, I used:\n\nIndex name: medium-blogs-embedding-index\n\nDimension: 512\n\nMetric: cosine\n\nThe dimension is important.\n\nThe embedding model and the Pinecone index must use the same number of dimensions. Otherwise, Pinecone cannot store or compare the vectors.\n\nOpenAI embeddings can be used for semantic search by comparing the similarity between document and query vectors.\n\nPress enter or click to view image in full size\n\nThe Pinecone index uses 512 dimensions, matching the embedding configuration in the Python code\n\nStep 4: Load and Store the Document\n\nThe ingestion script reads the text file, divides it into chunks, converts each chunk into an embedding, and stores the result in Pinecone.\n\nHere is the ingestion script:\n\nfrom pathlib import Path\n\nfrom dotenv import load_dotenv\n\nfrom langchain_community.document_loaders import TextLoader\n\nfrom langchain_openai import OpenAIEmbeddings\n\nfrom langchain_pinecone import PineconeVectorStore\n\nfrom langchain_text_splitters import CharacterTextSplitter\n\nload_dotenv()\n\nINDEX_NAME = \"medium-blogs-embedding-index\"\n\nEMBEDDING_MODEL = \"text-embedding-3-small\"\n\nEMBEDDING_DIMENSION = 512\n\ndef main() -> None:\n\nprint(\"Ingesting...\")\n\nfile_path = Path(**file**).with_name(\"mediumblog.txt\")\n\nif not file_path.exists():\n\nraise FileNotFoundError(\n\nf\"Knowledge file was not found: {file_path}\"\n\n)\n\nprint(\"Loading document...\")\n\nloader = TextLoader(str(file_path), encoding=\"utf-8\")\n\ndocuments = loader.load()\n\ntext_splitter = CharacterTextSplitter(\n\nchunk_size=1000,\n\nchunk_overlap=0,\n\n)\n\nchunks = text_splitter.split_documents(documents)\n\nprint(f\"Created {len(chunks)} text chunks\")\n\nembeddings = OpenAIEmbeddings(\n\nmodel=EMBEDDING_MODEL,\n\ndimensions=EMBEDDING_DIMENSION,\n\n)\n\nPineconeVectorStore.from_documents(\n\ndocuments=chunks,\n\nembedding=embeddings,\n\nindex_name=INDEX_NAME,\n\n)\n\nprint(\"Ingestion complete.\")\n\nif **name** == \"**main**\":\n\nmain()\n\nRun the script with:\n\nuv run python rag-gist/ingest.py\n\nThe output should look similar to this:\n\nIngesting...\n\nLoading document...\n\nCreated 31 text chunks\n\nIngestion complete.\n\nPress enter or click to view image in full size\n\nThe ingestion script loaded the document, created 31 chunks, and stored them in Pinecone.\n\nUnderstanding the Ingestion Code\n\nLet us look at what the script is doing.\n\nLoading the Document\n\nloader = TextLoader(str(file_path), encoding=\"utf-8\")\n\ndocuments = loader.load()\n\nThe loader turns the text file into LangChain Document objects.\n\nA document usually contains:\n\npage_content: the actual text\n\nmetadata: information such as the source filename\n\nSplitting the Document\n\ntext_splitter = CharacterTextSplitter(\n\nchunk_size=1000,\n\nchunk_overlap=0,\n\n)\n\nInstead of embedding the entire document as one large block, we divide it into smaller chunks.\n\nPress enter or click to view image in full size\n\nChunking is essential for context window : and token optimization. Instead of sending the entire document to the language model, the application sends only the sections that are most relevant to the question.\n\nThis is important because a user’s question may only be related to one small part of the document.\n\nA chunk size of 1000 means that each section contains approximately 1,000 characters.\n\nFor this first version, I used no overlap. In a stronger version, I would add some overlap so that information near the end of one chunk is also available at the beginning of the next chunk.\n\nCreating Embeddings\n\nembeddings = OpenAIEmbeddings(\n\nmodel=\"text-embedding-3-small\",\n\ndimensions=512,\n\n)\n\nThis code tells OpenAI to return embeddings with 512 dimensions.\n\nThe dimensions option is supported by OpenAI’s text-embedding-3 family of models.\n\nStoring the Chunks\n\nPineconeVectorStore.from_documents(\n\ndocuments=chunks,\n\nembedding=embeddings,\n\nindex_name=INDEX_NAME,\n\n)\n\nThis method:\n\nReads the text from each document chunk.\n\nSends the text to the embedding model.\n\nReceives a vector for each chunk.\n\nStores the vector and document metadata in Pinecone.\n\nStep 5: Retrieve Information and Generate an Answer\n\nAfter ingestion is complete, we can ask questions.\n\nBecome a Medium member\n\nHere is the retrieval script:\n\nfrom dotenv import load_dotenv\n\nfrom langchain_core.prompts import ChatPromptTemplate\n\nfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings\n\nfrom langchain_pinecone import PineconeVectorStore\n\nload_dotenv()\n\nINDEX_NAME = \"medium-blogs-embedding-index\"\n\nEMBEDDING_MODEL = \"text-embedding-3-small\"\n\nEMBEDDING_DIMENSION = 512\n\nembeddings = OpenAIEmbeddings(\n\nmodel=EMBEDDING_MODEL,\n\ndimensions=EMBEDDING_DIMENSION,\n\n)\n\nllm = ChatOpenAI(\n\nmodel=\"gpt-4o-mini\",\n\ntemperature=0,\n\n)\n\nvectorstore = PineconeVectorStore(\n\nindex_name=INDEX_NAME,\n\nembedding=embeddings,\n\n)\n\nretriever = vectorstore.as_retriever(\n\nsearch_kwargs={\"k\": 5}\n\n)\n\nprompt_template = ChatPromptTemplate.from_template(\n\n\"\"\"\n\nYou are a helpful assistant.\n\nAnswer the question using only the context provided below.\n\nIf the answer is not present in the context, say:\n\n\"I could not find that information in the provided document.\"\n\nContext:\n\n{context}\n\nQuestion:\n\n{question}\n\nAnswer:\n\n\"\"\"\n\n)\n\ndef format_documents(documents) -> str:\n\nreturn \"\\n\\n\".join(\n\ndocument.page_content for document in documents\n\n)\n\ndef answer_question(question: str) -> str:\n\ndocuments = retriever.invoke(question)\n\ncontext = format_documents(documents)\n\nmessages = prompt_template.format_messages(\n\ncontext=context,\n\nquestion=question,\n\n)\n\nresponse = llm.invoke(messages)\n\nreturn response.content\n\nif **name** == \"**main**\":\n\nuser_question = input(\"Ask a question: \").strip()\n\nif not user_question:\n\nraise ValueError(\"The question cannot be empty.\")\n\nanswer = answer_question(user_question)\n\nprint(\"\\nAnswer:\\n\")\n\nprint(answer)\n\nRun it with:\n\nuv run python rag-gist/main.py\n\nThe program asks for a question:\n\nAsk a question: What is the main idea of the document?\n\nIt then searches Pinecone and sends the most relevant sections to the language model.\n\nPress enter or click to view image in full size\n\nQuery for simple RAG System\n\nCaption: The application retrieved relevant document sections and used them to generate an answer.\n\nWhy Use a Retriever?\n\nThe vector store handles storage and similarity search.\n\nA retriever provides a simpler interface for requesting relevant documents:\n\nretriever = vectorstore.as_retriever(\n\nsearch_kwargs={\"k\": 5}\n\n)\n\nThe value k=5 means:\n\nReturn the five document chunks that are most closely related to the question.\n\nWhen this line runs:\n\ndocuments = retriever.invoke(question)\n\nthe application:\n\nConverts the question into an embedding,\n\nSearches Pinecone,\n\nFinds the closest vectors.\n\nReturns their original text.\n\nHow the Prompt Reduces Unsupported Answers\n\nThe prompt contains an important instruction:\n\nAnswer the question using only the context provided below.\n\nIt also tells the model what to do when the answer is missing:\n\nIf the answer is not present in the context, say:\n\n\"I could not find that information in the provided document.\"\n\nThis does not guarantee perfect accuracy, but it gives the model a clear boundary.\n\nA RAG system should not only retrieve information. It should also tell the model how that information must be used.\n\nWhat Is a LangChain Runnable?\n\nLangChain uses a common interface called a Runnable.\n\nA runnable is something that accepts input and produces output.\n\nFor example:\n\ndocuments = retriever.invoke(question)\n\nand:\n\nresponse = llm.invoke(messages)\n\nBoth the retriever and the language model support .invoke().\n\nThat common interface makes it easier to connect prompts, retrievers, models, and custom functions into larger chains.\n\nProblems I Encountered\n\nThe most useful part of building this project was understanding the errors.\n\nIndex 'mediumblog' not found.\n\nDid you mean: medium-blogs-embedding-index\n\nThe name in the Python script did not match the name in Pinecone.\n\nI fixed the constant:\n\nINDEX_NAME = \"medium-blogs-embedding-index\"\n\nA useful improvement is to keep shared settings in one configuration file so that ingestion and retrieval cannot accidentally use different index names.\n\nVector dimension 1536 does not match\n\nthe dimension of the index 512\n\nThe Pinecone index expected vectors containing 512 values, while the application was sending vectors of a different size.\n\nI fixed it by explicitly setting:\n\ndimensions=512\n\nThe same embedding model and dimension must be used during both ingestion and retrieval.\n\nPineconeVectorStore(\n\nindex_name=INDEX_NAME,\n\nembeddings=embeddings,\n\n)\n\nHowever, the constructor expected embedding, singular:\n\nPineconeVectorStore(\n\nindex_name=INDEX_NAME,\n\nembedding=embeddings,\n\n)\n\nCurrent LangChain-Pinecone examples also initialize the vector store with an embedding object.\n\nThis was a small naming difference, but it prevented the application from starting.\n\n{question}\n\nBut the code passed:\n\nquery=query\n\nLangChain raised:\n\nKeyError: 'question'\n\nThe variable name passed to the template must exactly match the placeholder:\n\nmessages = prompt_template.format_messages(\n\ncontext=context,\n\nquestion=query,\n\n)\n\nThis error also reminded me to keep naming consistent throughout the application.\n\nWhat I Would Improve Next\n\nThis project is a good learning version, but it is not yet a production-ready RAG system.\n\nMy next improvements would be:\n\nUse a Better Text Splitter\n\nI would replace CharacterTextSplitter with RecursiveCharacterTextSplitter.\n\nIt tries to keep paragraphs and sentences together instead of cutting the text only by character count.\n\nAdd Chunk Overlap\n\nFor example:\n\nchunk_size=1000\n\nchunk_overlap=150\n\nThe overlap helps preserve information that appears near chunk boundaries.\n\nPrevent Duplicate Ingestion\n\nRunning the ingestion script repeatedly can store the same content more than once.\n\nA production version should create stable document IDs and update existing records instead of always inserting new ones.\n\nReturn Source Information\n\nThe answer should show which chunks or source files were used.\n\nFor example:\n\nSource: mediumblog.txt, chunk 8\n\nThis would make the application easier to trust and debug.\n\nAdd a Relevance Threshold\n\nThe application currently returns the top five chunks even when none of them is strongly related to the question.\n\nA similarity threshold could help reject unrelated results.\n\nAdd Logging and Tracing\n\nTracing tools can show:\n\nwhich question was asked\n\nwhich chunks were retrieved\n\nhow long each step took\n\nhow many tokens were used\n\nwhat answer was generated\n\nCreate an Evaluation Set\n\nI would prepare a small collection of:\n\ntest questions\n\nexpected sources\n\nexpected answer points\n\nThis would make it easier to compare changes to chunk size, retrieval count, prompts, and embedding models.\n\nWhat I Learned\n\nBefore building this project, RAG looked like one complicated AI feature.\n\nAfter implementing it manually, I understood that it is simply a pipeline:\n\nDocuments become vectors.\n\nQuestions become vectors.\n\nVector search finds related text.\n\nThe language model turns that text into an answer.\n\nThe difficult part is not calling one magical RAG function.\n\nThe important decisions are:\n\nhow the document is divided,\n\nwhich embedding model is used\n\nwhether vector dimensions match\n\nhow many chunks are retrieved\n\nhow irrelevant results are handled\n\nwhat instructions are given to the language model\n\nhow sources are shown to the user\n\nOne of the most useful lessons from this project was to inspect the retrieved chunks before changing the prompt.\n\nIf the correct information is not retrieved, the language model cannot use it, no matter how well the prompt is written.\n\nBuilding each step separately made the architecture much easier to understand.\n\nFinal Thoughts\n\nA small RAG application is a useful project for learning how modern AI applications work with external knowledge.\n\nThe best way to learn RAG is to begin with one document, one question, and one successful retrieval.\n\nIt teaches more than prompt writing. It introduces document processing, embeddings, vector search, retrieval, prompt design, configuration management, and debugging.\n\nThe first version does not need to be perfect.\n\nStart with one document, one ingestion script, one retrieval script, and one question.\n\nOnce the basic pipeline works reliably, improve one part at a time and observe how each change affects retrieval quality.", "url": "https://wpnews.pro/news/i-built-a-simple-rag-app-with-langchain-openai-and-pinecone", "canonical_source": "https://dev.to/officialbidisha/i-built-a-simple-rag-app-with-langchain-openai-and-pinecone-j73", "published_at": "2026-07-12 19:22:27+00:00", "updated_at": "2026-07-12 19:44:10.176845+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "developer-tools", "machine-learning"], "entities": ["LangChain", "OpenAI", "Pinecone", "Python"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-simple-rag-app-with-langchain-openai-and-pinecone", "markdown": "https://wpnews.pro/news/i-built-a-simple-rag-app-with-langchain-openai-and-pinecone.md", "text": "https://wpnews.pro/news/i-built-a-simple-rag-app-with-langchain-openai-and-pinecone.txt", "jsonld": "https://wpnews.pro/news/i-built-a-simple-rag-app-with-langchain-openai-and-pinecone.jsonld"}}