How I built a tool-calling ReAct agent over a private knowledge base, packaged it as a Lambda container, and provisioned all of it as code.
Most retrieval-augmented generation (RAG) demos follow one fixed path: embed the question, fetch the top-k chunks, put them in a prompt, and return whatever the model says. That works until a question needs more than one lookup, or needs arithmetic the model shouldn't guess at, or doesn't need retrieval at all.
I wanted an assistant that could decide what to do: search the knowledge base, search again with a better query, call a calculator, or just answer. I also wanted to deploy it with no hand-configured infrastructure. This post walks through the design: a LangGraph ReAct agent, FAISS for vector search, a FastAPI interface, a provider-agnostic LLM layer, and an AWS Lambda container image provisioned end-to-end with Terraform.
In classic RAG, retrieval is a step that always runs once. In agentic RAG, retrieval is a tool. The model works in a loop (the ReAct pattern: reason, act, observe) and picks which tool to call based on what it has learned so far.
That gets you three things:
The cost is more LLM calls per question and a bit of extra complexity. LangGraph keeps that complexity manageable.
Client ──HTTPS──▶ Lambda (container image)
│
├─ FastAPI app (via ASGI adapter)
│
└─ LangGraph agent
├─ LLM node (OpenAI or Anthropic)
└─ Tool node
├─ retriever → FAISS index
└─ calculator
I split the codebase into small modules so each part could be tested on its own:
config holds typed settings (model provider, model name, top-k, index path), loaded from environment variables.retriever loads the FAISS index and returns ranked chunks with their source metadata.tools defines the retriever and calculator as tools the LLM can call.llm is a thin, provider-agnostic wrapper.graph wires the agent.api is the FastAPI layer.
LangGraph models an agent as a state machine. The state is mainly the list of messages; nodes read the state and add to it; edges decide what runs next.
The ReAct loop needs two nodes and one conditional edge:
from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import ToolNode
tools = [retrieve_docs, calculator]
llm_with_tools = llm.bind_tools(tools)
def call_model(state: MessagesState):
return {"messages": [llm_with_tools.invoke(state["messages"])]}
def should_continue(state: MessagesState):
last = state["messages"][-1]
return "tools" if last.tool_calls else END
graph = StateGraph(MessagesState)
graph.add_node("agent", call_model)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "agent")
graph.add_conditional_edges("agent", should_continue, ["tools", END])
graph.add_edge("tools", "agent")
app_graph = graph.compile()
The model either asks for tool calls or produces a final answer. If it asks for tools, ToolNode runs them, appends the results as tool messages, and control goes back to the model. I also set a recursion limit when invoking the graph so a confused model can't loop forever.
Writing the loop as an explicit graph, rather than hiding it inside an agent class, paid off quickly. I could see every transition, test should_continue with plain unit tests, and add nodes later (a query-rewriting step, for example) without touching the rest.
The retriever tool's docstring matters as much as its code, because the model reads it to decide when to call it:
@tool
def retrieve_docs(query: str) -> str:
"""Search the private knowledge base. Use for any question about
internal documents. Returns the most relevant passages with sources."""
hits = retriever.search(query, k=settings.top_k)
return "\n\n".join(f"[{h.source}] {h.text}" for h in hits)
Returning the source with each passage lets the model cite where an answer came from, and makes bad retrievals easy to spot while debugging.
The calculator is deliberately narrow. It evaluates arithmetic expressions with a small safe parser rather than Python's eval, because tool inputs are model-generated text and should be treated as untrusted.
For a private knowledge base of modest size, FAISS is a good fit: no separate database to run, and the index is just a file. I build the index offline (chunk documents, embed them, write the index plus a metadata file mapping vector IDs to text and source), then ship it with the application.
On Lambda that has a nice property: the index loads from the container's filesystem at cold start and stays in memory for warm invocations. I load it once at module level, not per request, so warm requests skip that cost.
The trade-off is that updating the knowledge base means rebuilding the index and redeploying. For a knowledge base that changes rarely, I'll take that over running a vector database.
I didn't want the agent tied to one vendor. The llm module exposes one factory that returns a chat model supporting tool binding, chosen by config:
def get_llm(settings: Settings):
if settings.provider == "openai":
return ChatOpenAI(model=settings.model, temperature=0)
if settings.provider == "anthropic":
return ChatAnthropic(model=settings.model, temperature=0)
raise ValueError(f"Unknown provider: {settings.provider}")
The rest of the code depends only on this interface. Switching providers is a config change, which also makes it easy to compare how different models behave with the same tools and prompts. API keys come from the environment and never live in the code or the image.
FastAPI gives a typed request and response schema and automatic docs during local development. To run the same app on Lambda, I wrap it with an ASGI-to-Lambda adapter such as Mangum, so one codebase serves both uvicorn locally and Lambda in production.
app = FastAPI()
class Ask(BaseModel):
question: str
@app.post("/ask")
def ask(body: Ask):
result = app_graph.invoke(
{"messages": [("user", body.question)]},
config={"recursion_limit": 10},
)
return {"answer": result["messages"][-1].content}
handler = Mangum(app)
Lambda's zip packages have tight size limits, and FAISS plus the LLM SDKs add up. A container image avoids that. The Dockerfile starts from the AWS Lambda Python base image, installs pinned dependencies, copies the code and the FAISS index, and points the command at the handler:
FROM public.ecr.aws/lambda/python:3.11
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ ${LAMBDA_TASK_ROOT}/app/
COPY index/ ${LAMBDA_TASK_ROOT}/index/
CMD ["app.api.handler"]
I can run the same image locally with Lambda's runtime interface emulator, which caught packaging mistakes before they reached AWS.
The goal was that a fresh AWS account could go from nothing to a working HTTPS endpoint with terraform apply. The configuration provisions:
resource "aws_lambda_function" "agent" {
function_name = "agentic-rag"
package_type = "Image"
image_uri = "${aws_ecr_repository.agent.repository_url}:${var.image_tag}"
role = aws_iam_role.lambda.arn
timeout = 60
memory_size = 1024
environment {
variables = { LLM_PROVIDER = var.llm_provider }
}
}
Two settings deserve attention. The timeout has to cover several LLM round trips, because an agentic question can take more than one model call. Memory on Lambda also scales CPU, which affects how quickly the FAISS index loads. I treat both as variables and tune them rather than hard-coding guesses. Secrets such as API keys should come from a secrets service, not plain Terraform variables in version control.
Having everything in Terraform also makes teardown trivial, which matters for a side project you don't want billing you quietly.
Because each component is separate, most tests don't need an LLM or AWS:
That last test is the most useful: it checks the wiring of the loop deterministically, without paying for or depending on a real model.
I'd like to add a query-rewriting node before retrieval, evaluation on a fixed question set to compare providers and chunking strategies, and response streaming. If you're building something similar, I'd be glad to compare notes.
Code and setup details: github.com/Ved210105