Models are trained on a massive amount of data. That doesn't mean that they know all the details of your specific situation. The training data might contain all chargeback fee schedules for Mastercard up until 2025, but you live in 2026 and actually ask about chargebacks on Wero. The model's knowledge is broad, generic and frozen at training time. You cannot just retrain Claude to include your data (except if you own Anthropic). So how can we pass knowledge that is specific to your situation or company? Well, we can use a tool. As we learned in Part 3, tools are simply functions that the model can request to invoke with the output of the tool being pasted back into the model's context window.
Imagine that we have a collection of documents and we create a search engine tool. The model can request the tool with a specific question like "what are the chargeback fees for Wero in 2026?" and our Python code browses the document collection to return the relevant sections. This pattern is so common that it even got a dedicated AI name: Retrieval-Augmented Generation, or RAG. This article shows you the basics.
Note: this article is about the agentic variant of RAG, in which the model decides what it wants to search to answer the question at hand. This is different from traditional RAG where relevant (parts of) documents are retrieved and included into the first prompt to the model.
Note 2: the examples in this article show processing fees of different payment methods. The fees and all other information are purely fictional. These are not the actual numbers.
The simplest way to implement this search engine tool is to do a full-text search over all documents based on the search query and then copy-paste matching documents into the context window. However, this might not even yield the best results. RAG uses two concepts to do better:
Full-text search is not only slow, it can also easily miss relevant sections. Synonyms, jargon words, implicit descriptions. Instead of comparing the text itself, we convert the text into so-called embeddings. An embedding is a fancy word for a vector (yes, from mathematics) that represents the meaning of that text. These embedding vectors are calculated in such a way that text with similar meaning maps to similar vectors (think similar as in cosine similarity or Euclidian distance), even if the words are not exactly the same. "Initiating a chargeback" and "How to get my money back?" have different words but similar meaning. Their vectors should therefore be more similar. Calculating similarity between vectors is also a lot faster than full-text comparison. Mapping text to an embedding is done by an embedding model, which itself is an AI model that was trained on millions of text pairs that do or do not belong together. The embeddings are stored in a vector database that can perform the similarity calculations.
A document is rarely about just one topic. A single article might explain processing fees, refund rules, chargeback procedures, settlement times and more. The embedding of that multi-topic document will also be less similar to text about any of the topics. The different topics all steer the vector in a different direction and it ends up no where close to the individual topics. And even if the document is still selected, you'd risk pasting a 30-page document into the model's context where you only need three sentences. This will rot your context and cost your tokens. That is why documents are split into chunks by a splitter before the embeddings are calculated. Many splitters exist: some split every 500 tokens, while others try to be smart and split according to the document type's structure. Later in this article, we see a Markdown-specific chunking approach.
If the above sounds a bit abstract and mathematical, then you are completely right. Embedding models and splitters are academic topics. They are to AI engineers what databases and search algorithms are to software engineers. You should understand fundamentals and be able to apply them in practice, but there's no need to understand their full academic depth. Therefore, let's leave the theory and look at a concrete example document:
## Card Processing Fees
Mastercard transactions incur a 2.2% processing fee plus €0.30, while Wero charges €0.32 flat.
## Refund Policy
Customers may request a refund within 30 days of purchase.
## Chargebacks
Chargeback disputes must be answered within 7 days.
We want to calculate its embedding so that the model can search for it later. First, the splitter splits the document into three chunks:
| Chunk | Content |
|:------|:----------------------------------------------------------------------|
| C1 | Card Processing Fees + the paragraph describing processing fees |
| C2 | Refund Policy + the refund rules |
| C3 | Chargebacks + the chargeback policy |
Each chunk is passed to the embedding model which converts it into a vector. The embedding model that we'll use in the code example later (all-MiniLM-L6-v2
) produces a vector of 384 numbers for every chunk, which is still small compared to production models that go over a thousand numbers per vector. For our example here, let's pretend the model produces only vectors of 2. The embedding model produces the following vectors:
| Chunk | Embedding vector |
|:------|:-----------------|
| C1 | (0.91, 0.15) |
| C2 | (0.23, 0.84) |
| C3 | (0.88, 0.11) |
The numbers themselves are meaningless to us humans. We cannot look at (0.91, 0.15)
and conclude that it represents payment fees. What matters is the similarity of the vectors. Chunks with similar meaning end up close together in the vector space, while unrelated chunks end up further apart.
Suppose the model uses the tool and asks: "What are the card processing fees?". The search engine tool first embeds the query using the same embedding model:
| Query | Embedding vector |
|:-------------------------------------|:-----------------|
| "What are the card processing fees?" | (0.90, 0.14) |
The query's embedding is passed to the vector database and compared with every other embedding. One common similarity measure is cosine similarity, which measures the extend to which two vectors point in the same direction.
| Chunk | Cosine similarity | Most similar vector |
|:------|:------------------|:--------------------|
| C1 | 0.99 | ✅ |
| C2 | 0.28 | |
| C3 | 0.74 | |
Because Chunk 1 has the highest similarity score, it is retrieved and returned to the model via it's context window and used to answer the user's question.
Now lets build this in code. Create data/payment_ops_notes.md
and fill it with Markdown:
## Card Processing Fees
European consumer cards cost 1.8% + €0.25 per transaction. Non-European and
commercial cards run higher, around 2.9% + €0.25. Fees are charged on the
original transaction and are NOT returned when you refund. This means that
a refunded sale still costs you the full processing fee.
## Refunds
Processing fees from the original charge are not returned, and the processor
charges a €0.25 admin fee per refund on top. Refunds are possible up to 180
days after the original charge; after that, use a manual bank transfer.
Card refunds take 5-10 business days to reach the customer, which is the
single most common cause of "where is my refund" tickets. Set that
expectation in the first reply.
## Chargebacks (cards)
The processor charges a €15 dispute fee per chargeback, win or lose. Respond
before the deadline in the dispute notification - typically 20-30 days
depending on the card network; a missed deadline is an automatic loss.
Common reason codes: 10.4 (fraud, card-absent), 13.1 (goods not received),
13.6 (credit not processed). Evidence that wins fraud disputes: a 3-D Secure
authentication record, AVS/CVV match, delivery confirmation to the
cardholder's billing address, and prior undisputed orders from the same
customer.
## Dispute Economics Rule
Only fight a chargeback when (disputed amount × realistic win probability)
clearly exceeds the dispute fee plus internal handling cost (budget ~€25 of
staff time per response). Below roughly €50 disputed, accepting is almost
always cheaper than fighting.
## Chargeback-Rate Monitoring
Card networks track your chargeback rate, not just your losses. Above
roughly 0.9% of transactions, merchants enter monitoring programs with
monthly fines and, eventually, termination of card acceptance. This is why
refunding a plausible fraud complaint fast is often cheaper than "winning"
the dispute: a refund does not count toward the chargeback rate, a
chargeback does - even one you win.
And also create the RAG tool in app/rag.py
:
from pathlib import Path
from langchain_core.documents import Document
from langchain_core.tools import tool
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_text_splitters import MarkdownHeaderTextSplitter
DATA_PATH = Path(__file__).parent.parent / "data" / "payment_ops_notes.md"
def build_vectorstore() -> InMemoryVectorStore:
text = DATA_PATH.read_text()
splitter = MarkdownHeaderTextSplitter(
headers_to_split_on=[("##", "section")]
)
chunks: list[Document] = splitter.split_text(text)
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
return InMemoryVectorStore.from_documents(chunks, embeddings)
_vectorstore: InMemoryVectorStore | None = None
def get_vectorstore() -> InMemoryVectorStore:
global _vectorstore
if _vectorstore is None:
_vectorstore = build_vectorstore()
return _vectorstore
@tool
def search_payments_knowledge_base(query: str) -> str:
"""Search internal notes on processing fees, refund policy, and disputes.
Use this before quoting fee figures or policy deadlines yourself, and
before calling calculate_refund_cost if the user hasn't already given you
the payment method's fee structure. Ground your answer in these notes
rather than guessing.
Args:
query: What you need to know. Examples: "card processing fees" or
"chargeback response deadline".
"""
results = get_vectorstore().similarity_search(query, k=2)
if not results:
return "No relevant notes found."
return "\n\n---\n\n".join(doc.page_content for doc in results)
Some notes about the above:
##
headers, so every chunk is one coherent section with its heading attached. Such structural splitting can be applied if you know that your documents are written in Markdown. As mentioned before, more general splitters also exist.all-MiniLM-L6-v2
is fine for examples and small applications. Its small enough to fit in your laptops memory. Use it for developing your agent. Once you reach serious scale, you can always switch to hosted embedding APIs from OpenAI and others. Although, do note that the embedding model used on the query must be the same as the model you used to fill the vector store. Compare it to hashing: if all passwords in the database are hashed with SHA256 and you has the given password with SHA3 then it will never match.InMemoryVectorStore
is really the sqlite of AI engineering: a simple in-memory database for tutorials and local development only. Beyond that, I recommend the pgvector
extension for Postgresql. Now we just need to import our tool and run the application. Create 04_tool_rag.py
:
from dotenv import load_dotenv
from langchain.chat_models import init_chat_model
from langchain_core.messages import ToolMessage
from app.rag import search_payments_knowledge_base
load_dotenv()
model = init_chat_model("anthropic:claude-sonnet-5")
model_with_tools = model.bind_tools([search_payments_knowledge_base])
question = "What is the transaction fee for European consumer cards? Can they do chargebacks?"
msg = model_with_tools.invoke(question)
print(msg.tool_calls)
tool_map = {
"search_payments_knowledge_base": search_payments_knowledge_base
}
tool_messages = []
for tool_call in msg.tool_calls:
tool = tool_map[tool_call["name"]]
result = tool.invoke(tool_call["args"])
tool_messages.append(
ToolMessage(content=str(result), tool_call_id=tool_call["id"])
)
next_msg = model_with_tools.invoke(
[{"role": "user", "content": question}, msg, *tool_messages]
)
print(next_msg.content)
Run it with python 04_tool_rag.py
and you'll see something like:
$ python 04_tool_rag.py
[
{'name': 'search_payments_knowledge_base', 'args': {'query': 'European consumer card transaction fee'}, 'id': 'toolu_01JzAV1GyQEyuu6zUoEne4EM', 'type': 'tool_call'},
{'name': 'search_payments_knowledge_base', 'args': {'query': 'chargeback eligibility European consumer cards'}, 'id': 'toolu_01B4kA41umG1UHUV3XRmefTX', 'type': 'tool_call'}
]
**Transaction fee:** European consumer cards cost **1.8% + €0.25** per transaction. (For comparison, non-European or commercial cards run higher, around 2.9% + €0.25.) Note that this fee is charged on the original sale and is **not refunded** even if you later issue a refund — so a refunded transaction still costs you the full processing fee.
**Chargebacks:** Yes, European consumer cardholders can file chargebacks like any other card network transaction. A couple of important points from a risk-management perspective:
- Card networks monitor your **chargeback rate** (not just dollar losses). If it exceeds roughly **0.9% of transactions**, you can be placed into a monitoring program with monthly fines, and eventually risk losing card acceptance entirely.
- Chargebacks count against that rate **even if you win the dispute** — but refunds do **not** count against it.
- So for plausible fraud/dispute complaints, it's often cheaper overall to issue a refund proactively rather than fight (and "win") a chargeback, since winning still hurts your chargeback ratio while a refund doesn't.
Let me know if you'd like me to calculate the actual cost of a refund vs. a chargeback for a specific transaction amount.
Above you see the tool calls and then the final answer. What I find pretty exciting is that the model decides by itself to perform two separate queries. After using the RAG tool, the model answers based on our document instead of from its training data. It flagged where the figures came from because the docstring told it to. Nice, right? Nothing in our code explicitly enforced any of that.
This article showed how we can make our own internal documents available to the agent. Of course, there is much more to explore with RAG. One advanced concept worth looking up is reranking, where the chunks from the vector store are 'reranked' so that only the most relevant subset is passed to the agent.
As you might have realised, tools make the model pretty powerful. Real use cases often involve many tools and, like libraries and frameworks, you do not want to hand-roll that for every project. The next article shows how to publish tool catalogs through a protocol called MCP for others to use. Then, the article after that (Part 6) finally ties everything together when we build a full-fledged agent like the ones we're used to from Claude Code or Codex.
Find all code samples in the companion repo here:
[https://github.com/BjornvdLaan/ai-engineering-articles-code-samples]