Shipping an Agentic RAG Assistant on AWS Lambda with LangGraph and Terraform

A developer built an agentic RAG assistant using a LangGraph ReAct agent that treats retrieval as a callable tool alongside a calculator, backed by a FAISS vector index and a provider-agnostic LLM layer for OpenAI or Anthropic models. The system is served through FastAPI inside an AWS Lambda container image and provisioned end-to-end with Terraform, with the agent modeled as an explicit state graph so transitions like the tool-call decision can be unit tested and new nodes added without reworking the rest.

How I built a tool-calling ReAct agent over a private knowledge base, packaged it as a Lambda container, and provisioned all of it as code. Most retrieval-augmented generation RAG demos follow one fixed path: embed the question, fetch the top-k chunks, put them in a prompt, and return whatever the model says. That works until a question needs more than one lookup, or needs arithmetic the model shouldn't guess at, or doesn't need retrieval at all. I wanted an assistant that could decide what to do: search the knowledge base, search again with a better query, call a calculator, or just answer. I also wanted to deploy it with no hand-configured infrastructure. This post walks through the design: a LangGraph ReAct agent, FAISS for vector search, a FastAPI interface, a provider-agnostic LLM layer, and an AWS Lambda container image provisioned end-to-end with Terraform. In classic RAG, retrieval is a step that always runs once. In agentic RAG, retrieval is a tool . The model works in a loop the ReAct pattern: reason, act, observe and picks which tool to call based on what it has learned so far. That gets you three things: The cost is more LLM calls per question and a bit of extra complexity. LangGraph keeps that complexity manageable. Client ──HTTPS──▶ Lambda container image │ ├─ FastAPI app via ASGI adapter │ └─ LangGraph agent ├─ LLM node OpenAI or Anthropic └─ Tool node ├─ retriever → FAISS index └─ calculator I split the codebase into small modules so each part could be tested on its own: config holds typed settings model provider, model name, top-k, index path , loaded from environment variables. retriever loads the FAISS index and returns ranked chunks with their source metadata. tools defines the retriever and calculator as tools the LLM can call. llm is a thin, provider-agnostic wrapper. graph wires the agent. api is the FastAPI layer. LangGraph models an agent as a state machine. The state is mainly the list of messages; nodes read the state and add to it; edges decide what runs next. The ReAct loop needs two nodes and one conditional edge: python from langgraph.graph import StateGraph, MessagesState, START, END from langgraph.prebuilt import ToolNode tools = retrieve docs, calculator llm with tools = llm.bind tools tools def call model state: MessagesState : return {"messages": llm with tools.invoke state "messages" } def should continue state: MessagesState : last = state "messages" -1 return "tools" if last.tool calls else END graph = StateGraph MessagesState graph.add node "agent", call model graph.add node "tools", ToolNode tools graph.add edge START, "agent" graph.add conditional edges "agent", should continue, "tools", END graph.add edge "tools", "agent" app graph = graph.compile The model either asks for tool calls or produces a final answer. If it asks for tools, ToolNode runs them, appends the results as tool messages, and control goes back to the model. I also set a recursion limit when invoking the graph so a confused model can't loop forever. Writing the loop as an explicit graph, rather than hiding it inside an agent class, paid off quickly. I could see every transition, test should continue with plain unit tests, and add nodes later a query-rewriting step, for example without touching the rest. The retriever tool's docstring matters as much as its code, because the model reads it to decide when to call it: php @tool def retrieve docs query: str - str: """Search the private knowledge base. Use for any question about internal documents. Returns the most relevant passages with sources.""" hits = retriever.search query, k=settings.top k return "\n\n".join f" {h.source} {h.text}" for h in hits Returning the source with each passage lets the model cite where an answer came from, and makes bad retrievals easy to spot while debugging. The calculator is deliberately narrow. It evaluates arithmetic expressions with a small safe parser rather than Python's eval , because tool inputs are model-generated text and should be treated as untrusted. For a private knowledge base of modest size, FAISS is a good fit: no separate database to run, and the index is just a file. I build the index offline chunk documents, embed them, write the index plus a metadata file mapping vector IDs to text and source , then ship it with the application. On Lambda that has a nice property: the index loads from the container's filesystem at cold start and stays in memory for warm invocations. I load it once at module level, not per request, so warm requests skip that cost. The trade-off is that updating the knowledge base means rebuilding the index and redeploying. For a knowledge base that changes rarely, I'll take that over running a vector database. I didn't want the agent tied to one vendor. The llm module exposes one factory that returns a chat model supporting tool binding, chosen by config: python def get llm settings: Settings : if settings.provider == "openai": return ChatOpenAI model=settings.model, temperature=0 if settings.provider == "anthropic": return ChatAnthropic model=settings.model, temperature=0 raise ValueError f"Unknown provider: {settings.provider}" The rest of the code depends only on this interface. Switching providers is a config change, which also makes it easy to compare how different models behave with the same tools and prompts. API keys come from the environment and never live in the code or the image. FastAPI gives a typed request and response schema and automatic docs during local development. To run the same app on Lambda, I wrap it with an ASGI-to-Lambda adapter such as Mangum, so one codebase serves both uvicorn locally and Lambda in production. app = FastAPI class Ask BaseModel : question: str @app.post "/ask" def ask body: Ask : result = app graph.invoke {"messages": "user", body.question }, config={"recursion limit": 10}, return {"answer": result "messages" -1 .content} handler = Mangum app Lambda's zip packages have tight size limits, and FAISS plus the LLM SDKs add up. A container image avoids that. The Dockerfile starts from the AWS Lambda Python base image, installs pinned dependencies, copies the code and the FAISS index, and points the command at the handler: FROM public.ecr.aws/lambda/python:3.11 COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY app/ ${LAMBDA TASK ROOT}/app/ COPY index/ ${LAMBDA TASK ROOT}/index/ CMD "app.api.handler" I can run the same image locally with Lambda's runtime interface emulator, which caught packaging mistakes before they reached AWS. The goal was that a fresh AWS account could go from nothing to a working HTTPS endpoint with terraform apply . The configuration provisions: resource "aws lambda function" "agent" { function name = "agentic-rag" package type = "Image" image uri = "${aws ecr repository.agent.repository url}:${var.image tag}" role = aws iam role.lambda.arn timeout = 60 memory size = 1024 environment { variables = { LLM PROVIDER = var.llm provider } } } Two settings deserve attention. The timeout has to cover several LLM round trips, because an agentic question can take more than one model call. Memory on Lambda also scales CPU, which affects how quickly the FAISS index loads. I treat both as variables and tune them rather than hard-coding guesses. Secrets such as API keys should come from a secrets service, not plain Terraform variables in version control. Having everything in Terraform also makes teardown trivial, which matters for a side project you don't want billing you quietly. Because each component is separate, most tests don't need an LLM or AWS: That last test is the most useful: it checks the wiring of the loop deterministically, without paying for or depending on a real model. I'd like to add a query-rewriting node before retrieval, evaluation on a fixed question set to compare providers and chunking strategies, and response streaming. If you're building something similar, I'd be glad to compare notes. Code and setup details: github.com/Ved210105 https://github.com/Ved210105