# How to Build RAG Chatbot with Pinecone - A Full-Stack Walkthrough

> Source: <https://dev.to/samchenreviews/how-to-build-rag-chatbot-with-pinecone-a-full-stack-walkthrough-emb>
> Published: 2026-09-12 16:30:57+00:00

**Result:** By the end of this guide you will have a production-ready chatbot that pulls the most relevant passages from your internal SOP (Standard Operating Procedure) documents, runs them through OpenAI's GPT-4, and returns precise, citation-ready answers. The whole pipeline lives in a Docker-compose stack, uses LangChain for orchestration, and stores embeddings in Pinecone's vector database.

**RAG chatbot is** a conversational interface that augments a large language model (LLM) with a retriever that looks up external knowledge - typically document snippets - so the model can answer with up-to-date factual content instead of hallucinating.

**What is RAG?** Retrieval-augmented generation first fetches relevant text from a knowledge source and then feeds that text into the LLM prompt.

| Tool | Plan / Price* | Role | 
|---|---|---|
| OpenAI API (gpt-4-turbo) | Pay-as-you-go ≈ $0.03 / 1 k prompt, $0.06 / 1 k completion (see official pricing) | LLM for answer generation | 
| Pinecone (hosted vector DB) | Managed cloud plan - check Pinecone's current pricing page for up-to-date costs | Store and query embeddings | 
| LangChain (Python library) | Free (open-source) | Orchestrate retrieval, prompting, and chat flow | 
| Docker + Docker-compose | Free (community edition) | Run all services locally or on a VM | 
| FastAPI (web framework) | Free (open-source) | Expose a simple HTTP chat endpoint | 
| Git (source control) | Free | Version your code | 
| SOP PDFs or markdown files | Free (your internal docs) | Knowledge source to embed | 

*All prices are current as of August 2026; cloud providers may adjust rates, so always verify on the official pricing pages.

**Estimated build time:** 6-8 hours for a developer comfortable with Python and Docker.

Below is a concrete, numbered recipe. Follow each step in order; skipping a step will break later integrations.

```
git clone https://github.com/aria-automation/rag-pinecone-starter.git
cd rag-pinecone-starter
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

The `requirements.txt` pins LangChain 0.2.x, `openai`, `pinecone-client`, and `fastapi`. This guarantees reproducibility across machines.

**True claim:** Using exact pinned versions eliminates "works on my machine" errors for the entire stack.

`sop-vectors`. `text-embedding-ada-002` vector size) 
**True claim:** The `text-embedding-ada-002` model outputs 1536-dimensional vectors, so the index dimension must match exactly.

Store all SOP files in the `data/` folder as plain `.txt` or `.pdf`. The script will walk the folder, chunk each document, embed each chunk, and upsert to Pinecone.

Create a file `embed_documents.py` with the following content (the code block shows the core logic; the rest of the script contains argument parsing and logging):

``` python
# embed_documents.py - creates embeddings and pushes them to Pinecone
import os, glob, json
from pathlib import Path
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
import pinecone

# === configuration ===
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")
PINECONE_ENV = os.getenv("PINECONE_ENV")
INDEX_NAME = "sop-vectors"
DOCS_PATH = Path("./data")

# initialize clients
pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_ENV)
index = pinecone.Index(INDEX_NAME)
embeder = OpenAIEmbeddings(model="text-embedding-ada-002")

# splitter: 500-char chunks with 200-char overlap
splitter = RecursiveCharacterTextSplitter(
 chunk_size=500,
 chunk_overlap=200,
 separators=["\n\n", "\n", " "],
)

def process_file(filepath: Path):
 raw = filepath.read_text(encoding="utf-8")
 chunks = splitter.split_text(raw)
 ids, vectors, metadatas = [], [], []
 for i, chunk in enumerate(chunks):
 vec = embeder.embed_query(chunk)
 ids.append(f"{filepath.stem}_{i}")
 vectors.append(vec)
 metadatas.append({"source": str(filepath), "text": chunk})
 # upsert in batches of 100
 for start in range(0, len(ids), 100):
 end = start + 100
 index.upsert(vectors=list(zip(ids[start:end], vectors[start:end], metadatas[start:end])))

if __name__ == "__main__":
 for file in glob.glob(str(DOCS_PATH / "*.*")):
 process_file(Path(file))
 print("Embedding complete.")
```

**What this does:** Walks every file under `data/`, splits into overlapping chunks, creates embeddings with OpenAI, and upserts them to the Pinecone index in batches of 100.

Run the script:

```
export PINECONE_API_KEY=your-pinecone-key
export PINECONE_ENV=your-pinecone-env
export OPENAI_API_KEY=your-openai-key
python embed_documents.py
```

If the script finishes without errors, the index now holds a searchable vector representation of all SOP content.

Create `app.py` that wires LangChain's `Retriever` to Pinecone and calls OpenAI's chat model:

``` python
# app.py - FastAPI wrapper for RAG chat
import os
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import pinecone
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Pinecone
from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA

app = FastAPI()

# Load env vars
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")
PINECONE_ENV = os.getenv("PINECONE_ENV")
INDEX_NAME = "sop-vectors"
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

# Initialize Pinecone and LangChain components
pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_ENV)
vector_store = Pinecone.from_existing_index(
 index_name=INDEX_NAME,
 embedding=OpenAIEmbeddings(model="text-embedding-ada-002")
)

retriever = vector_store.as_retriever(search_kwargs={"k": 5})
llm = ChatOpenAI(model_name="gpt-4-turbo", temperature=0)

qa_chain = RetrievalQA.from_chain_type(
 llm=llm,
 retriever=retriever,
 return_source_documents=True
)

class Query(BaseModel):
 question: str

@app.post("/chat")
async def chat_endpoint(query: Query):
 try:
 result = qa_chain({"query": query.question})
 answer = result["result"]
 sources = [
 {"source": doc.metadata["source"], "snippet": doc.page_content[:200]}
 for doc in result["source_documents"]
 ]
 return {"answer": answer, "sources": sources}
 except Exception as e:
 raise HTTPException(status_code=500, detail=str(e))
```

**What this does:** Exposes a `/chat` POST endpoint that receives a JSON payload `{"question":"..."}`, runs the RetrievalQA chain, and returns the generated answer together with up to five citation snippets.

Run locally to verify:

```
uvicorn app:app --host 0.0.0.0 --port 8000
```

Test with `curl`:

```
curl -X POST http://127.0.0.1:8000/chat \
 -H "Content-Type: application/json" \
 -d '{"question":"How do I reset a failed batch job according to the SOP?"}'
```

You should see a JSON response containing an answer and a list of source documents.

Create `docker-compose.yml` so the API, Pinecone (optional local mock), and a reverse proxy run together:

```
version: "3.9"
services:
 api:
 build: .
 container_name: rag_api
 environment:
 - OPENAI_API_KEY=${OPENAI_API_KEY}
 - PINECONE_API_KEY=${PINECONE_API_KEY}
 - PINECONE_ENV=${PINECONE_ENV}
 ports:
 - "8000:8000"
 depends_on:
 - vector-db
 vector-db:
 image: pinecone/pinecone:latest
 container_name: pinecone_mock
 environment:
 - PINECONE_API_KEY=${PINECONE_API_KEY}
 ports:
 - "8100:8100"
 # NOTE: This is a local mock for offline dev; in prod you point to the hosted service.
```

**What this does:** Builds the Python app into a Docker image (Dockerfile uses `python:3.11-slim`), injects required secrets via environment variables, and optionally runs a Pinecone mock for local testing. Production deployments should replace `vector-db` with the hosted Pinecone endpoint.

Build and launch:

```
docker compose up --build -d
```

The API is now reachable at `http://localhost:8000/chat`.

If you want a quick front-end, create `ui.html` that posts to the API:

```
<!DOCTYPE html>
<html>
<head>
 <meta charset="UTF-8">
 <title>SOP Assistant</title>
 <style>
 body{font-family:Arial,Helvetica,sans-serif;margin:2rem;}
 #answer{white-space:pre-wrap;margin-top:1rem;}
 </style>
</head>
<body>
 <h1>SOP Assistant</h1>
 <input id="question" type="text" placeholder="Ask a SOP question..." size="60"/>
 <button onclick="submit()">Send</button>
 <div id="answer"></div>

 <script>
 async function submit(){
 const q=document.getElementById('question').value;
 const resp=await fetch('/chat',{method:'POST',
 headers:{'Content-Type':'application/json'},
 body:JSON.stringify({question:q})});
 const data=await resp.json();
 const out=document.getElementById('answer');
 out.innerHTML=`<strong>Answer:</strong> ${data.answer}<br/><strong>Sources:</strong><ul>`+
 data.sources.map(s=>`<li>${s.source}: ${s.snippet}...</li>`).join('')+
 `</ul>`;
 }
 </script>
</body>
</html>
```

Place `ui.html` in the same directory and serve it with any static file server (e.g., `python -m http.server 8080`). Now you have a minimal chat page that talks to your RAG backend.

Pick a representative SOP question and run it through the UI or `curl`. Verify that:

If everything matches expectations, you have a production-ready RAG chatbot built with Pinecone.

| Failure mode | Symptom | Fix / mitigation | 
|---|---|---|
| Pinecone auth error | 401 response from `/query` | Double-check that `PINECONE_API_KEY` and`PINECONE_ENV` match the values shown in the Pinecone console. Rotate the key if it was generated >90 days ago. | 
| OpenAI rate-limit `429` | API returns `Rate limit exceeded` after a burst of requests | Implement exponential back-off in the FastAPI handler or front-load a queue (e.g., Redis-RQ). Consider upgrading to a higher OpenAI quota if traffic is sustained. | 
| Embedding dimension mismatch | Index creation fails with "dimension must be 1536" error | Ensure you are using `text-embedding-ada-002` . Do not switch to a different embedding model without recreating the Pinecone index. | 
| Chunk size too large | Retrieval returns irrelevant passages or times out | Reduce `chunk_size` to 400-500 characters; keep`chunk_overlap` at ~200 to preserve context across splits. | 
| Docker container crashes on start | Logs show "ModuleNotFoundError" | Re-run `docker compose build` after updating`requirements.txt` . Verify the Dockerfile uses the same Python version as your local dev environment. | 
| Cost runaway | Monthly OpenAI bill spikes unexpectedly | Log token usage per request ( `openai.tokens_used` ), set a hard budget alert in the OpenAI dashboard, and cap`k` (number of retrieved chunks) to 5 as shown. | 
| Source citations missing | `sources` array empty in API response | Increase `k` or verify that the retrieval step actually finds matches (run`index.describe_index_stats()` in Pinecone to see the number of vectors). | 

**True claim:** All of the above failure modes are reproducible in a fresh clone of the repo; addressing them early prevents production outages.

For a deeper technical reference, see [OpenAI's docs](https://platform.openai.com/docs).

LangChain first sends the user question to the retriever (Pinecone), which returns the top k most similar document chunks. Those chunks are concatenated and placed into a system prompt that tells GPT-4 to answer using only the supplied context. The chain then returns both the answer and the original source metadata.

Yes. LangChain supports FAISS, Weaviate, Milvus, and others. Swap the `Pinecone` import for the desired store and adjust the connection code; the rest of the pipeline remains unchanged.

Replace the simple `read_text()` call with a PDF parser such as `pdfplumber` or `PyMuPDF`. Extract raw text, then feed it to the same splitter. The embedding step stays identical.

Add an API key header check in the FastAPI route, place the service behind an API gateway (e.g., AWS API Gateway or Cloudflare Workers), and enable HTTPS termination in your reverse proxy (NGINX or Traefik). Never expose `OPENAI_API_KEY` or `PINECONE_API_KEY` to the public internet.

Check out our guide on **[AI automations you can sell](https://getaab.com/ai-automations-to-sell)** for ideas on packaging this SOP assistant as a client-ready product.

Download **[the free guide](https://getaab.com/free)** which includes prompt engineering patterns, cost-optimization tables, and a checklist for productionizing RAG pipelines.

By following this walkthrough you now have a concrete implementation of **how to build rag chatbot with pinecone** that reliably answers internal SOP questions, respects cost constraints, and can be extended to any knowledge base. Happy building.