Build a Q&A Bot over Your Company Docs with Pinecone + Claude (Python, SDK v10)

A developer published a walkthrough for building a command-line retrieval-augmented generation Q&A bot over internal company documents using Pinecone's Python SDK v10.3.0 and Anthropic's anthropic 2.4.0 with the claude-sonnet-5-5 model. The pipeline chunks documents, embeds and stores them in a Pinecone index via hosted llama-text-embed-v2, retrieves and reranks results with bge-reranker-v2-m3, then prompts Claude to answer only from retrieved chunks with numbered citations, and it maps deprecated v5–v7 SDK calls to their v10 equivalents.

AI disclosure: This article was written with the help of AI and reviewed by me. Every API call in it was checked against the SDK source of pinecone==10.3.0 and anthropic==2.4.0 , and prices and limits were taken from the official pages linked inline as of November 2026 . Treat the code as a starting point: run it against your own keys before relying on it. Once a company has enough internal docs, the same problems show up: new hires can't find the leave policy, nobody remembers which version of the expense rules is current, and wiki search only works if you guess the exact words the author used "PTO" won't find the page that says "annual leave" . Retrieval-augmented generation RAG is the usual fix: split documents into chunks, embed them into a vector database, retrieve the most relevant chunks for each question, and have an LLM answer only from those chunks, with citations. This walkthrough builds a command-line Q&A bot with Pinecone managed vector database with hosted embedding and rerank models and Claude Anthropic's LLM, for answer generation : Versions used latest on PyPI in November 2026 : | Component | Version | |---|---| | Python | 3.10+ Pinecone SDK v10 supports 3.10–3.14 | | pinecone Pinecone Python SDK | 10.3.0, API version 2026-07 | | anthropic Anthropic Python SDK | 2.4.0 | | Claude model | claude-sonnet-5-5 | The Pinecone SDK changed a lot in v10. Many tutorials still use the v5–v7 style create index for model , upsert records , ServerlessSpec , which may fail or emit deprecation warnings. The pitfalls section maps old calls to new ones. ┌──────────── Offline ingest ─────────────────┐ .md/.txt ── chunk ── documents.upsert ── Pinecone index │ llama-text-embed-v2 │ embeds on write └─────────────────────────────────────────────┘ ┌──────────── Online ask ─────────────────────┐ question ── documents.search embed scoring, recall 12 ── inference.rerank bge-reranker-v2-m3, keep 4 ── build prompt ── Claude Messages API ── answer with n citations └─────────────────────────────────────────────┘ Embedding, storage and reranking all happen on Pinecone's side, so you don't install PyTorch or host an embedding model. The trade-off: your text is processed by Pinecone's hosted models more on that under data residency . Create an account at pinecone.io https://www.pinecone.io/ , then create a key on the API Keys page of the console https://app.pinecone.io https://app.pinecone.io . The free Starter plan is enough for everything here, but note its limits as of November 2026, see Pricing https://www.pinecone.io/pricing/ and Rate limits https://docs.pinecone.io/reference/api/database-limits/rate-limits.md : us-east-1 bge-reranker-v2-m3 : Create a key in the Claude Console https://platform.claude.com https://platform.claude.com . Check Anthropic's supported countries list https://www.anthropic.com/supported-countries for your region first. requirements.txt : pinecone==10.3.0 anthropic==2.4.0 python -m venv .venv source .venv/bin/activate pip install -r requirements.txt The package is pinecone , not the old pinecone-client . | Variable | Required | Notes | |---|---|---| | PINECONE API KEY | yes | From the Pinecone console | | ANTHROPIC API KEY | yes | From the Claude Console; anthropic.Anthropic reads it by default | | PINECONE INDEX | no | Index name, default company-docs | | PINECONE NAMESPACE | no | Namespace, default default-tenant | | USE RERANK | no | 1 to rerank default , 0 to skip and save Starter rerank quota | export PINECONE API KEY="your-pinecone-key" export ANTHROPIC API KEY="your-anthropic-key" Don't hard-code keys or commit them to Git. Chunking is the first thing that makes or breaks RAG quality. Chunks that are too large mix several topics, so the embedding gets "averaged out" and you waste prompt tokens. Chunks that are too small lose context and the model gets half a sentence. A simple strategy that works for Markdown or plain-text policy docs: php def split text text: str, size: int = 500, overlap: int = 80 - list str : paragraphs = p.strip for p in text.split "\n\n" if p.strip chunks, buf = , "" for p in paragraphs: if len p size: if buf: chunks.append buf buf = "" step = size - overlap chunks.extend p i:i + size for i in range 0, len p , step continue if len buf + len p + 1 <= size: buf = f"{buf}\n{p}" if buf else p else: chunks.append buf buf = buf -overlap: + "\n" + p if overlap else p if buf: chunks.append buf return chunks This counts characters, not tokens, to avoid depending on a tokenizer. 500 characters of English is roughly 100–150 tokens, well under both limits that matter here: llama-text-embed-v2 : max 2048 tokens per input, truncated by default beyond that For finer strategies by sentence, by token, by heading , see Pinecone's Chunking Strategies https://www.pinecone.io/learn/chunking-strategies/ . Each chunk carries metadata: source relative path , category top-level folder such as hr/ or finance/ and chunk index . category is used for filtering, source for citations and for cleaning up on re-ingest. Anthropic doesn't offer an embedding model Embeddings https://platform.claude.com/docs/en/build-with-claude/embeddings , so you need one from somewhere else. Three common options: | Option | How | Pros | Cons | |---|---|---|---| | A. Pinecone integrated embedding used here | Declare embed on a text field in the index schema; write and query plain text | Least code, no dimension mismatches | Limited to Pinecone-hosted models; text is processed by those models | | B. Pinecone Inference, called separately | pc.inference.embed ... returns vectors you write to a dense vector field | You own the vectors and can reuse them | You must keep the model and dimension consistent between write and query | | C. Third-party or self-hosted model | Voyage AI one of the providers Anthropic's docs point to , another cloud API, or an open-source model | Any model, can be fully private | Another service to run; you align dimensions and normalization yourself | Pinecone-hosted dense embedding models docs as of November 2026 : | Model | Dimensions | Max input | Notes | |---|---|---|---| | llama-text-embed-v2 | 1024 default / 2048 / 768 / 512 / 384 | 2048 tokens | Model page lists 26 supported languages | | multilingual-e5-large | 1024 | 507 tokens | Multilingual; good for short queries + medium passages | | text-embedding-3-small / -large | up to 1536 / 3072, configurable | 8192 tokens | OpenAI models processed via Microsoft Azure; document indexes only | | gemini-embedding-2 | 128–3072 | 8192 tokens | Google model; not available on Starter | On the Standard plan, llama-text-embed-v2 is listed at $0.16 per million tokens and multilingual-e5-large at $0.08 Pricing https://www.pinecone.io/pricing/ . This walkthrough uses llama-text-embed-v2 : multilingual, a roomier input limit than e5, and available on Starter. If your docs are mostly short Q&A pairs, try multilingual-e5-large : change the model name in the schema and rebuild the index. In SDK v10 the recommended index type is the document index . You declare the scoring fields with SchemaBuilder ; every other field you write is stored as metadata and indexed for filtering automatically. Adding an embed config to a string field makes Pinecone embed that field's text on write, and embed the query text at search time Integrated embedding https://docs.pinecone.io/guides/search/integrated-embedding.md . python import os from pinecone import Pinecone, SchemaBuilder pc = Pinecone api key=os.environ "PINECONE API KEY" schema = SchemaBuilder .add string field name="chunk text", embed={"model": "llama-text-embed-v2", "dimension": 1024}, .build if not pc.indexes.exists name="company-docs" : pc.indexes.create name="company-docs", schema=schema, deployment={"deployment type": "managed", "cloud": "aws", "region": "us-east-1"}, Notes: deployment type: "managed" means serverless. spec=ServerlessSpec ... is deprecated the SDK source says it's translated into deployment . If you omit pc.indexes.create timeout=-1 to return immediately. source or category in it returns a 400 at creation. embed field also stores the original text and enables full-text search by default store text defaults to true , so search results can return chunk text . Set store text=False if you don't want the raw text stored in Pinecone, but then you'll have to look it up by ID in your own storage. index = pc.index name="company-docs" documents = { " id": "3be9998a6b0d7b83 0", ASCII only "chunk text": "Employees get 5 days of paid annual leave after one year of service...", "source": "hr/annual-leave.md", metadata, filterable automatically "category": "hr", "chunk index": 0, }, ... for i in range 0, len documents , 96 : index.documents.upsert namespace="default-tenant", documents=documents i:i + 96 Key points: id must be 1–512 ASCII characters. usage.embed total tokens in the response tells you how many embedding tokens the write used, which helps reconcile costs. Re-ingesting: if a file shrinks and produces fewer chunks, the old extra chunks linger. The script calls documents.delete filter=... by source before writing, to clear a file's old chunks. resp = index.documents.search namespace="default-tenant", top k=12, score by= {"type": "embed", "fields": "chunk text" , "query": "How much annual leave do I get?"} , include fields= "chunk text", "source", "chunk index" , filter={"category": {"$eq": "hr"}}, optional metadata filter for m in resp.matches: print m.id, m.score, m.get "source" include fields you only get score . This is a classic beginner trap: the search "works" but returns no text. " " returns every field. score by ; it can't be combined with a BM25 text clause. For hybrid search, the docs suggest running both searches and merging with RRF Vector recall uses a bi-encoder: question and documents are embedded separately and compared by distance. It's fast but coarse. A rerank model cross-encoder scores the question and each document together, which is more accurate but slower. The usual pattern is to over-recall with vectors 12 here and rerank down to the best few 4 here for the prompt. ranked = pc.inference.rerank model="bge-reranker-v2-m3", query="How much annual leave do I get?", documents= {"id": m.id, "chunk text": m.get "chunk text", "" } for m in resp.matches , rank fields= "chunk text" , top n=4, return documents=False, parameters={"truncate": "END"}, for r in ranked.data: print r.index, r.score r.index is the position in the documents you passed in Model choice: Pinecone currently hosts three rerank models: bge-reranker-v2-m3 multilingual, available on Starter , cohere-rerank-4-fast Standard and above, processed via Azure, possibly outside the US , and pinecone-rerank-v0 deprecated; don't use it in new projects . bge-reranker-v2-m3 defaults to truncate: NONE , which errors on over-long input, so END is set explicitly. Older tutorials often do recall + rerank in one call via index.search ..., rerank={...} . That's for legacy integrated indexes created with create index for model , using the Records API . With document indexes, rerank is a separate pc.inference.rerank call. Wrap each retrieved chunk in numbered XML tags so the model has anchors to cite: SYSTEM PROMPT = """You are an internal knowledge-base assistant. Answer employees' questions using ONLY the material inside <documents . Rules: 1. If the answer isn't in the material, say "I couldn't find this in the knowledge base." Don't make anything up. 2. After each key statement, cite the source as n , where n matches <document index="n" . 3. If sources conflict, list the conflict instead of picking one. 4. Answer concisely.""" def build user message question, hits : blocks = "\n".join f'<document index="{i}" source="{h "source" }" \n{h "text" }\n</document ' for i, h in enumerate hits, start=1 return f"<documents \n{blocks}\n</documents \n\nQuestion: {question}" Calling the Messages API: python import anthropic client = anthropic.Anthropic reads ANTHROPIC API KEY response = client.beta.messages.create model="claude-sonnet-5-5", max tokens=16000, system=SYSTEM PROMPT, messages= {"role": "user", "content": build user message question, hits } , output config={"effort": "medium"}, fallbacks="default", betas= "server-side-fallback-2026-07-01" , if response.stop reason == "refusal": print "The model declined to answer." else: print "".join b.text for b in response.content if b.type == "text" What's different from older code: response.content is a list of blocks thinking blocks. Filter on type == "text" instead of reading content 0 .text . output config.effort low / medium / high / xhigh / max . Sonnet 5.5 defaults to temperature . 2026-10-21 , the default in Anthropic SDK 2.x, rejects any request containing top p or top k with a 400. Delete the temperature=0 that old code often has. stop reason == "refusal" fallbacks="default" plus the beta header server-side-fallback-2026-07-01 lets the server retry a refused request on a recommended fallback model client.beta.messages.create . To skip the beta, drop the last two arguments and use client.messages.create ; keep the refusal check either way. max tokens Save as rag bot.py : """ rag bot.py - company-docs Q&A bot on Pinecone integrated embedding + rerank + Claude Usage: python rag bot.py ingest ./docs chunk and upsert python rag bot.py ask "How much annual leave do I get?" python rag bot.py ask "expense process" --category finance search one category only Versions at time of writing: pinecone==10.3.0, anthropic==2.4.0, python =3.10 """ import argparse import hashlib import os import sys import time from pathlib import Path import anthropic from pinecone import Pinecone, SchemaBuilder from pinecone.errors import NotFoundError ---------- config ---------- INDEX NAME = os.environ.get "PINECONE INDEX", "company-docs" NAMESPACE = os.environ.get "PINECONE NAMESPACE", "default-tenant" EMBED MODEL = "llama-text-embed-v2" Pinecone-hosted embedding model EMBED DIM = 1024 this model supports 384/512/768/1024/2048 RERANK MODEL = "bge-reranker-v2-m3" Pinecone-hosted rerank model multilingual USE RERANK = os.environ.get "USE RERANK", "1" == "1" CLAUDE MODEL = "claude-sonnet-5-5" CHUNK SIZE = 500 target characters per chunk CHUNK OVERLAP = 80 overlap between adjacent chunks UPSERT BATCH = 96 max docs per upsert for indexes with an embed field RECALL TOP K = 12 vector recall size FINAL TOP N = 4 chunks sent to the prompt after rerank pc = Pinecone api key=os.environ "PINECONE API KEY" claude = anthropic.Anthropic reads ANTHROPIC API KEY ---------- 1. chunking ---------- def split text text: str, size: int = CHUNK SIZE, overlap: int = CHUNK OVERLAP - list str : """Split on blank lines, pack paragraphs up to size , hard-split long paragraphs with overlap.""" paragraphs = p.strip for p in text.split "\n\n" if p.strip chunks, buf = , "" for p in paragraphs: if len p size: long paragraph: flush buffer, then sliding window if buf: chunks.append buf buf = "" step = size - overlap chunks.extend p i:i + size for i in range 0, len p , step continue if len buf + len p + 1 <= size: buf = f"{buf}\n{p}" if buf else p else: chunks.append buf carry the tail of the previous chunk to reduce answers cut in half buf = buf -overlap: + "\n" + p if overlap else p if buf: chunks.append buf return chunks def load documents root: Path - list dict : """Read .md / .txt files under root and build the documents to upsert.""" docs = for path in sorted root.rglob " " : if path.suffix.lower not in {".md", ".txt"} or not path.is file : continue rel = path.relative to root .as posix text = path.read text encoding="utf-8" id must be ASCII; hash the path so non-ASCII file names work file key = hashlib.sha1 rel.encode "utf-8" .hexdigest :16 category = rel.split "/" 0 if "/" in rel else "general" for i, chunk in enumerate split text text : docs.append { " id": f"{file key} {i}", "chunk text": chunk, Pinecone embeds this field "source": rel, the rest is metadata, filterable automatically "category": category, "chunk index": i, } return docs ---------- 2. index ---------- def ensure index - None: if pc.indexes.exists name=INDEX NAME : return schema = SchemaBuilder .add string field name="chunk text", embed={"model": EMBED MODEL, "dimension": EMBED DIM}, .build print f"Creating index {INDEX NAME} ..." pc.indexes.create name=INDEX NAME, schema=schema, managed = serverless. The free Starter plan only allows aws / us-east-1 deployment={"deployment type": "managed", "cloud": "aws", "region": "us-east-1"}, blocks until the index is ready by default ---------- 3. ingest ---------- def ingest root: Path - None: ensure index index = pc.index name=INDEX NAME docs = load documents root if not docs: print "No .md / .txt files found" return on re-ingest, delete each file's old chunks first so none are left behind for source in sorted {d "source" for d in docs} : try: index.documents.delete namespace=NAMESPACE, filter={"source": {"$eq": source}} except NotFoundError: pass namespace doesn't exist yet first ingest for i in range 0, len docs , UPSERT BATCH : batch = docs i:i + UPSERT BATCH resp = index.documents.upsert namespace=NAMESPACE, documents=batch tokens = resp.usage.embed total tokens if resp.usage else "?" print f"Upserted {i + len batch }/{len docs } chunks, embedding tokens this batch: {tokens}" time.sleep 5 indexing is asynchronous; wait a moment before querying print "Ingest complete" ---------- 4. retrieve + rerank ---------- def retrieve question: str, category: str | None = None - list dict : index = pc.index name=INDEX NAME resp = index.documents.search namespace=NAMESPACE, top k=RECALL TOP K, score by= {"type": "embed", "fields": "chunk text" , "query": question} , include fields= "chunk text", "source", "chunk index" , filter={"category": {"$eq": category}} if category else None, hits = { "id": m.id, "score": m.score, "text": m.get "chunk text", "" , "source": m.get "source", "" , } for m in resp.matches if not hits or not USE RERANK: return hits :FINAL TOP N ranked = pc.inference.rerank model=RERANK MODEL, query=question, documents= {"id": h "id" , "chunk text": h "text" } for h in hits , rank fields= "chunk text" , top n=FINAL TOP N, return documents=False, parameters={"truncate": "END"}, r.index is the document's position in the list we passed in return dict hits r.index , score=r.score for r in ranked.data ---------- 5. prompt Claude ---------- SYSTEM PROMPT = """You are an internal knowledge-base assistant. Answer employees' questions using ONLY the material inside <documents . Rules: 1. If the answer isn't in the material, say "I couldn't find this in the knowledge base." Don't make anything up. 2. After each key statement, cite the source as n , where n matches <document index="n" . 3. If sources conflict, list the conflict instead of picking one. 4. Answer concisely.""" def build user message question: str, hits: list dict - str: blocks = "\n".join f'<document index="{i}" source="{h "source" }" \n{h "text" }\n</document ' for i, h in enumerate hits, start=1 return f"<documents \n{blocks}\n</documents \n\nQuestion: {question}" def answer question: str, category: str | None = None - None: hits = retrieve question, category if not hits: print "Nothing was retrieved from the knowledge base." return response = claude.beta.messages.create model=CLAUDE MODEL, max tokens=16000, system=SYSTEM PROMPT, messages= {"role": "user", "content": build user message question, hits } , output config={"effort": "medium"}, if a safety classifier refuses, the server retries on a recommended model beta fallbacks="default", betas= "server-side-fallback-2026-07-01" , if response.stop reason == "refusal": print "The model declined to answer." return text = "".join b.text for b in response.content if b.type == "text" print text print "\n---- Sources ----" for i, h in enumerate hits, start=1 : score = f"{h 'score' :.4f}" if h "score" is not None else "n/a" print f" {i} {h 'source' } score={score} " u = response.usage print f"\n model={response.model}, input tokens={u.input tokens}, output tokens={u.output tokens} " ---------- entry point ---------- def main - None: parser = argparse.ArgumentParser description="Pinecone + Claude docs Q&A" sub = parser.add subparsers dest="cmd", required=True p ingest = sub.add parser "ingest" p ingest.add argument "path", type=Path p ask = sub.add parser "ask" p ask.add argument "question" p ask.add argument "--category", default=None args = parser.parse args if args.cmd == "ingest": ingest args.path else: answer args.question, args.category if name == " main ": try: main except anthropic.APIStatusError as e: sys.exit f"Claude API error {e.status code}: {e.message}" Prepare a folder whose top-level subfolders act as categories: docs/ ├── hr/ │ ├── annual-leave.md │ └── onboarding.md └── finance/ └── expenses.md python rag bot.py ingest ./docs python rag bot.py ask "Do I get annual leave in my first year?" python rag bot.py ask "What's the hotel limit on business trips?" --category finance USE RERANK=0 python rag bot.py ask "How far ahead do I need to request leave?" no rerank quota used The output shows the answer, the source file behind each n with its score, and the token usage for the call so you can check it against your bill. These are list-price estimates, not a measured bill; your console usage is the source of truth. Pinecone as of November 2026, Pricing https://www.pinecone.io/pricing/ : | Plan | Price | Notes | |---|---|---| | Starter | Free | Limits in 2.1; over the limit requests get 429, never billed | | Builder | $20/month flat | Higher limits; overages are also blocked, not billed | | Standard | $50/month minimum, then usage | Storage $0.33/GB/month; reads $16–18 per million RU; writes $4–4.5 per million WU varies by cloud/region ; llama-text-embed-v2 $0.16 per million tokens; bge-reranker-v2-m3 $2 per 1,000 requests; egress $0.10/GB 100 GB/month included | | Enterprise | $500/month minimum | 99.95% uptime SLA | An internal knowledge base of a few hundred policy docs usually fits within Starter's 2 GB and 5M embedding tokens a month. The first limit you'll hit is usually 500 reranks a month . Either disable rerank on Starter, call it only when recall scores are too close to separate, or upgrade. Read/write unit accounting is detailed in Understanding cost https://docs.pinecone.io/guides/manage-cost/understanding-cost.md . Claude as of November 2026, Pricing https://platform.claude.com/docs/en/about-claude/pricing : claude-sonnet-5-5 is $2 per million input tokens, $10 per million output tokens, and $0.10 per million cache-hit tokens. Rough math for one question with these settings: four chunks plus system prompt and question is a few thousand input tokens; the answer is a few hundred. Input cost is well under a cent; output cost depends on answer length, and thinking tokens are billed as output , which is why effort is worth tuning. For real numbers, multiply the input tokens / output tokens the script prints by the unit prices. Ways to save: FINAL TOP N aren't automatically better: 10 chunks instead of 4 is more than double the input tokens. If you generate vectors yourself options B/C , the index dimension must match the model output. Writing with model A and querying with model B gives essentially random results even if the dimensions happen to match. Using an embed field with embed scoring avoids this structurally, because Pinecone uses the model fixed at creation for both. Remember: the schema and embedding config can't change after creation ; a new model means a new index and a full re-ingest. With option B, the quickstart pattern is pc.inference.embed model=..., inputs= ... , parameters={"input type": "passage"} , written into a field declared with add dense vector field , and input type="query" at search time. Omitting or swapping input type won't error, but retrieval quality drops. If the write and query namespaces differ, you get no error, just empty results. The script keeps the namespace in one env var. Namespaces are good for multi-tenant isolation one per department or customer , but Starter caps an index at 100 namespaces, so many tenants need a higher plan or metadata filtering instead. $eq , $in , $gte and friends. $match phrase / $match all / $match any . chunk index always an integer. Metadata filters are not access control. If finance policies should only be visible to finance staff, the server must add the filter based on the logged-in user; never trust a category sent from the client. For stricter isolation, use separate namespaces or even separate indexes. truncate=END , so the tail never makes it into the vector. Indexing is asynchronous, so there's a short eventual-consistency window. A few seconds of sleep is enough in an ingest script; if your app needs read-after-write, handle it with a retry or a UI hint. | What old tutorials show | In v10 | |---|---| | pip install pinecone-client | Package renamed to pinecone | | pc.create index ..., spec=ServerlessSpec ... | spec deprecated; use pc.indexes.create ..., deployment={...} | | pc.create index for model ... + index.upsert records ... + index.search ..., rerank=... | Creates a "legacy integrated index" on the old Records API; still works, but no text updates or import | | from pinecone import ...preview... | pinecone.preview removed in v10; document indexes and SchemaBuilder import from pinecone directly | | Testing locally with Pinecone Local | Pinecone Local reached end of life in 2026 Release notes https://docs.pinecone.io/release-notes/2026.md | messages to steer formatting; current models don't support prefill and return 400. Use output config.format for structured output. stop reason == "max tokens" . Check these before rolling this out inside a company: ap-southeast-1 and Europe need a paid plan global , possibly outside the US If your docs contain personal data or sensitive business information, get legal and security review first. If data can't leave your environment, this managed stack isn't the right fit; look at self-hostable options like pgvector or Milvus with a model you can run or contract for in-region. string field of a document index, write plain text, let Pinecone embed it with documents.search with pc.inference.rerank with <documents tags; Claude Sonnet 5.5 answers only from them, with citations. The upside is very little code and no models or databases to operate, which makes it a good way to test whether "Q&A over company docs" is worth building. The downsides are just as clear: Pinecone is a closed managed service and your data lives in its cloud; Starter's rerank and region limits are tight; and costs grow linearly with data and query volume, so reconcile your bill regularly. Where to go next: CHUNK SIZE , effort . Pinecone and Anthropic iterate quickly; check versions before running, and where this article and the official docs disagree, trust the docs.