How to Build RAG Chatbot with Pinecone - A Full-Stack Walkthrough

Result: By the end of this guide you will have a production-ready chatbot that pulls the most relevant passages from your internal SOP (Standard Operating Procedure) documents, runs them through OpenAI's GPT-4, and returns precise, citation-ready answers. The whole pipeline lives in a Docker-compose stack, uses LangChain for orchestration, and stores embeddings in Pinecone's vector database.

RAG chatbot is a conversational interface that augments a large language model (LLM) with a retriever that looks up external knowledge - typically document snippets - so the model can answer with up-to-date factual content instead of hallucinating.

What is RAG? Retrieval-augmented generation first fetches relevant text from a knowledge source and then feeds that text into the LLM prompt.

*All prices are current as of August 2026; cloud providers may adjust rates, so always verify on the official pricing pages.

Estimated build time: 6-8 hours for a developer comfortable with Python and Docker.

Below is a concrete, numbered recipe. Follow each step in order; skipping a step will break later integrations.

git clone https://github.com/aria-automation/rag-pinecone-starter.git
cd rag-pinecone-starter
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The requirements.txt pins LangChain 0.2.x, openai, pinecone-client, and fastapi. This guarantees reproducibility across machines.

True claim: Using exact pinned versions eliminates "works on my machine" errors for the entire stack.

sop-vectors. text-embedding-ada-002 vector size) True claim: The text-embedding-ada-002 model outputs 1536-dimensional vectors, so the index dimension must match exactly.

Store all SOP files in the data/ folder as plain .txt or .pdf. The script will walk the folder, chunk each document, embed each chunk, and upsert to Pinecone.

Create a file embed_documents.py with the following content (the code block shows the core logic; the rest of the script contains argument parsing and logging):

import os, glob, json
from pathlib import Path
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
import pinecone

PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")
PINECONE_ENV = os.getenv("PINECONE_ENV")
INDEX_NAME = "sop-vectors"
DOCS_PATH = Path("./data")

pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_ENV)
index = pinecone.Index(INDEX_NAME)
embeder = OpenAIEmbeddings(model="text-embedding-ada-002")

splitter = RecursiveCharacterTextSplitter(
 chunk_size=500,
 chunk_overlap=200,
 separators=["\n\n", "\n", " "],
)

def process_file(filepath: Path):
 raw = filepath.read_text(encoding="utf-8")
 chunks = splitter.split_text(raw)
 ids, vectors, metadatas = [], [], []
 for i, chunk in enumerate(chunks):
 vec = embeder.embed_query(chunk)
 ids.append(f"{filepath.stem}_{i}")
 vectors.append(vec)
 metadatas.append({"source": str(filepath), "text": chunk})
 for start in range(0, len(ids), 100):
 end = start + 100
 index.upsert(vectors=list(zip(ids[start:end], vectors[start:end], metadatas[start:end])))

if __name__ == "__main__":
 for file in glob.glob(str(DOCS_PATH / "*.*")):
 process_file(Path(file))
 print("Embedding complete.")

What this does: Walks every file under data/, splits into overlapping chunks, creates embeddings with OpenAI, and upserts them to the Pinecone index in batches of 100.

Run the script:

export PINECONE_API_KEY=your-pinecone-key
export PINECONE_ENV=your-pinecone-env
export OPENAI_API_KEY=your-openai-key
python embed_documents.py

If the script finishes without errors, the index now holds a searchable vector representation of all SOP content.

Create app.py that wires LangChain's Retriever to Pinecone and calls OpenAI's chat model:

import os
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import pinecone
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Pinecone
from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA

app = FastAPI()

PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")
PINECONE_ENV = os.getenv("PINECONE_ENV")
INDEX_NAME = "sop-vectors"
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_ENV)
vector_store = Pinecone.from_existing_index(
 index_name=INDEX_NAME,
 embedding=OpenAIEmbeddings(model="text-embedding-ada-002")
)

retriever = vector_store.as_retriever(search_kwargs={"k": 5})
llm = ChatOpenAI(model_name="gpt-4-turbo", temperature=0)

qa_chain = RetrievalQA.from_chain_type(
 llm=llm,
 retriever=retriever,
 return_source_documents=True
)

class Query(BaseModel):
 question: str

@app.post("/chat")
async def chat_endpoint(query: Query):
 try:
 result = qa_chain({"query": query.question})
 answer = result["result"]
 sources = [
 {"source": doc.metadata["source"], "snippet": doc.page_content[:200]}
 for doc in result["source_documents"]
 ]
 return {"answer": answer, "sources": sources}
 except Exception as e:
 raise HTTPException(status_code=500, detail=str(e))

What this does: Exposes a /chat POST endpoint that receives a JSON payload {"question":"..."}, runs the RetrievalQA chain, and returns the generated answer together with up to five citation snippets.

Run locally to verify:

uvicorn app:app --host 0.0.0.0 --port 8000

Test with curl:

curl -X POST http://127.0.0.1:8000/chat \
 -H "Content-Type: application/json" \
 -d '{"question":"How do I reset a failed batch job according to the SOP?"}'

You should see a JSON response containing an answer and a list of source documents.

Create docker-compose.yml so the API, Pinecone (optional local mock), and a reverse proxy run together:

version: "3.9"
services:
 api:
 build: .
 container_name: rag_api
 environment:
 - OPENAI_API_KEY=${OPENAI_API_KEY}
 - PINECONE_API_KEY=${PINECONE_API_KEY}
 - PINECONE_ENV=${PINECONE_ENV}
 ports:
 - "8000:8000"
 depends_on:
 - vector-db
 vector-db:
 image: pinecone/pinecone:latest
 container_name: pinecone_mock
 environment:
 - PINECONE_API_KEY=${PINECONE_API_KEY}
 ports:
 - "8100:8100"

What this does: Builds the Python app into a Docker image (Dockerfile uses python:3.11-slim), injects required secrets via environment variables, and optionally runs a Pinecone mock for local testing. Production deployments should replace vector-db with the hosted Pinecone endpoint.

Build and launch:

docker compose up --build -d

The API is now reachable at http://localhost:8000/chat.

If you want a quick front-end, create ui.html that posts to the API:

<!DOCTYPE html>
<html>
<head>
 <meta charset="UTF-8">
 <title>SOP Assistant</title>
 <style>
 body{font-family:Arial,Helvetica,sans-serif;margin:2rem;}
 #answer{white-space:pre-wrap;margin-top:1rem;}
 </style>
</head>
<body>
 <h1>SOP Assistant</h1>
 <input id="question" type="text" placeholder="Ask a SOP question..." size="60"/>
 <button onclick="submit()">Send</button>
 <div id="answer"></div>

 <script>
 async function submit(){
 const q=document.getElementById('question').value;
 const resp=await fetch('/chat',{method:'POST',
 headers:{'Content-Type':'application/json'},
 body:JSON.stringify({question:q})});
 const data=await resp.json();
 const out=document.getElementById('answer');
 out.innerHTML=`<strong>Answer:</strong> ${data.answer}<br/><strong>Sources:</strong><ul>`+
 data.sources.map(s=>`<li>${s.source}: ${s.snippet}...</li>`).join('')+
 `</ul>`;
 }
 </script>
</body>
</html>

Place ui.html in the same directory and serve it with any static file server (e.g., python -m http.server 8080). Now you have a minimal chat page that talks to your RAG backend.

Pick a representative SOP question and run it through the UI or curl. Verify that:

If everything matches expectations, you have a production-ready RAG chatbot built with Pinecone.

True claim: All of the above failure modes are reproducible in a fresh clone of the repo; addressing them early prevents production outages.

For a deeper technical reference, see OpenAI's docs.

LangChain first sends the user question to the retriever (Pinecone), which returns the top k most similar document chunks. Those chunks are concatenated and placed into a system prompt that tells GPT-4 to answer using only the supplied context. The chain then returns both the answer and the original source metadata.

Yes. LangChain supports FAISS, Weaviate, Milvus, and others. Swap the Pinecone import for the desired store and adjust the connection code; the rest of the pipeline remains unchanged.

Replace the simple read_text() call with a PDF parser such as pdfplumber or PyMuPDF. Extract raw text, then feed it to the same splitter. The embedding step stays identical.

Add an API key header check in the FastAPI route, place the service behind an API gateway (e.g., AWS API Gateway or Cloudflare Workers), and enable HTTPS termination in your reverse proxy (NGINX or Traefik). Never expose OPENAI_API_KEY or PINECONE_API_KEY to the public internet.

Check out our guide on AI automations you can sell for ideas on packaging this SOP assistant as a client-ready product.

Download the free guide which includes prompt engineering patterns, cost-optimization tables, and a checklist for productionizing RAG pipelines.

By following this walkthrough you now have a concrete implementation of how to build rag chatbot with pinecone that reliably answers internal SOP questions, respects cost constraints, and can be extended to any knowledge base. Happy building.