{"slug": "shipping-an-agentic-rag-assistant-on-aws-lambda-with-langgraph-and-terraform", "title": "Shipping an Agentic RAG Assistant on AWS Lambda with LangGraph and Terraform", "summary": "A developer built an agentic RAG assistant using a LangGraph ReAct agent that treats retrieval as a callable tool alongside a calculator, backed by a FAISS vector index and a provider-agnostic LLM layer for OpenAI or Anthropic models. The system is served through FastAPI inside an AWS Lambda container image and provisioned end-to-end with Terraform, with the agent modeled as an explicit state graph so transitions like the tool-call decision can be unit tested and new nodes added without reworking the rest.", "body_md": "*How I built a tool-calling ReAct agent over a private knowledge base, packaged it as a Lambda container, and provisioned all of it as code.*\n\nMost retrieval-augmented generation (RAG) demos follow one fixed path: embed the question, fetch the top-k chunks, put them in a prompt, and return whatever the model says. That works until a question needs more than one lookup, or needs arithmetic the model shouldn't guess at, or doesn't need retrieval at all.\n\nI wanted an assistant that could decide what to do: search the knowledge base, search again with a better query, call a calculator, or just answer. I also wanted to deploy it with no hand-configured infrastructure. This post walks through the design: a LangGraph ReAct agent, FAISS for vector search, a FastAPI interface, a provider-agnostic LLM layer, and an AWS Lambda container image provisioned end-to-end with Terraform.\n\nIn classic RAG, retrieval is a step that always runs once. In agentic RAG, retrieval is a **tool**. The model works in a loop (the ReAct pattern: reason, act, observe) and picks which tool to call based on what it has learned so far.\n\nThat gets you three things:\n\nThe cost is more LLM calls per question and a bit of extra complexity. LangGraph keeps that complexity manageable.\n\n```\nClient ──HTTPS──▶ Lambda (container image)\n                     │\n                     ├─ FastAPI app (via ASGI adapter)\n                     │\n                     └─ LangGraph agent\n                          ├─ LLM node (OpenAI or Anthropic)\n                          └─ Tool node\n                               ├─ retriever → FAISS index\n                               └─ calculator\n```\n\nI split the codebase into small modules so each part could be tested on its own:\n\n`config` holds typed settings (model provider, model name, top-k, index path), loaded from environment variables.`retriever` loads the FAISS index and returns ranked chunks with their source metadata.`tools` defines the retriever and calculator as tools the LLM can call.`llm` is a thin, provider-agnostic wrapper.`graph` wires the agent.`api` is the FastAPI layer.\nLangGraph models an agent as a state machine. The state is mainly the list of messages; nodes read the state and add to it; edges decide what runs next.\n\nThe ReAct loop needs two nodes and one conditional edge:\n\n``` python\nfrom langgraph.graph import StateGraph, MessagesState, START, END\nfrom langgraph.prebuilt import ToolNode\n\ntools = [retrieve_docs, calculator]\nllm_with_tools = llm.bind_tools(tools)\n\ndef call_model(state: MessagesState):\n    return {\"messages\": [llm_with_tools.invoke(state[\"messages\"])]}\n\ndef should_continue(state: MessagesState):\n    last = state[\"messages\"][-1]\n    return \"tools\" if last.tool_calls else END\n\ngraph = StateGraph(MessagesState)\ngraph.add_node(\"agent\", call_model)\ngraph.add_node(\"tools\", ToolNode(tools))\ngraph.add_edge(START, \"agent\")\ngraph.add_conditional_edges(\"agent\", should_continue, [\"tools\", END])\ngraph.add_edge(\"tools\", \"agent\")\napp_graph = graph.compile()\n```\n\nThe model either asks for tool calls or produces a final answer. If it asks for tools, `ToolNode` runs them, appends the results as tool messages, and control goes back to the model. I also set a recursion limit when invoking the graph so a confused model can't loop forever.\n\nWriting the loop as an explicit graph, rather than hiding it inside an agent class, paid off quickly. I could see every transition, test `should_continue` with plain unit tests, and add nodes later (a query-rewriting step, for example) without touching the rest.\n\nThe retriever tool's docstring matters as much as its code, because the model reads it to decide when to call it:\n\n``` php\n@tool\ndef retrieve_docs(query: str) -> str:\n    \"\"\"Search the private knowledge base. Use for any question about\n    internal documents. Returns the most relevant passages with sources.\"\"\"\n    hits = retriever.search(query, k=settings.top_k)\n    return \"\\n\\n\".join(f\"[{h.source}] {h.text}\" for h in hits)\n```\n\nReturning the source with each passage lets the model cite where an answer came from, and makes bad retrievals easy to spot while debugging.\n\nThe calculator is deliberately narrow. It evaluates arithmetic expressions with a small safe parser rather than Python's `eval`, because tool inputs are model-generated text and should be treated as untrusted.\n\nFor a private knowledge base of modest size, FAISS is a good fit: no separate database to run, and the index is just a file. I build the index offline (chunk documents, embed them, write the index plus a metadata file mapping vector IDs to text and source), then ship it with the application.\n\nOn Lambda that has a nice property: the index loads from the container's filesystem at cold start and stays in memory for warm invocations. I load it once at module level, not per request, so warm requests skip that cost.\n\nThe trade-off is that updating the knowledge base means rebuilding the index and redeploying. For a knowledge base that changes rarely, I'll take that over running a vector database.\n\nI didn't want the agent tied to one vendor. The `llm` module exposes one factory that returns a chat model supporting tool binding, chosen by config:\n\n``` python\ndef get_llm(settings: Settings):\n    if settings.provider == \"openai\":\n        return ChatOpenAI(model=settings.model, temperature=0)\n    if settings.provider == \"anthropic\":\n        return ChatAnthropic(model=settings.model, temperature=0)\n    raise ValueError(f\"Unknown provider: {settings.provider}\")\n```\n\nThe rest of the code depends only on this interface. Switching providers is a config change, which also makes it easy to compare how different models behave with the same tools and prompts. API keys come from the environment and never live in the code or the image.\n\nFastAPI gives a typed request and response schema and automatic docs during local development. To run the same app on Lambda, I wrap it with an ASGI-to-Lambda adapter such as Mangum, so one codebase serves both `uvicorn` locally and Lambda in production.\n\n```\napp = FastAPI()\n\nclass Ask(BaseModel):\n    question: str\n\n@app.post(\"/ask\")\ndef ask(body: Ask):\n    result = app_graph.invoke(\n        {\"messages\": [(\"user\", body.question)]},\n        config={\"recursion_limit\": 10},\n    )\n    return {\"answer\": result[\"messages\"][-1].content}\n\nhandler = Mangum(app)\n```\n\nLambda's zip packages have tight size limits, and FAISS plus the LLM SDKs add up. A container image avoids that. The Dockerfile starts from the AWS Lambda Python base image, installs pinned dependencies, copies the code and the FAISS index, and points the command at the handler:\n\n```\nFROM public.ecr.aws/lambda/python:3.11\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\nCOPY app/ ${LAMBDA_TASK_ROOT}/app/\nCOPY index/ ${LAMBDA_TASK_ROOT}/index/\nCMD [\"app.api.handler\"]\n```\n\nI can run the same image locally with Lambda's runtime interface emulator, which caught packaging mistakes before they reached AWS.\n\nThe goal was that a fresh AWS account could go from nothing to a working HTTPS endpoint with `terraform apply`. The configuration provisions:\n\n```\nresource \"aws_lambda_function\" \"agent\" {\n  function_name = \"agentic-rag\"\n  package_type  = \"Image\"\n  image_uri     = \"${aws_ecr_repository.agent.repository_url}:${var.image_tag}\"\n  role          = aws_iam_role.lambda.arn\n  timeout       = 60\n  memory_size   = 1024\n  environment {\n    variables = { LLM_PROVIDER = var.llm_provider }\n  }\n}\n```\n\nTwo settings deserve attention. The **timeout** has to cover several LLM round trips, because an agentic question can take more than one model call. **Memory** on Lambda also scales CPU, which affects how quickly the FAISS index loads. I treat both as variables and tune them rather than hard-coding guesses. Secrets such as API keys should come from a secrets service, not plain Terraform variables in version control.\n\nHaving everything in Terraform also makes teardown trivial, which matters for a side project you don't want billing you quietly.\n\nBecause each component is separate, most tests don't need an LLM or AWS:\n\nThat last test is the most useful: it checks the wiring of the loop deterministically, without paying for or depending on a real model.\n\nI'd like to add a query-rewriting node before retrieval, evaluation on a fixed question set to compare providers and chunking strategies, and response streaming. If you're building something similar, I'd be glad to compare notes.\n\n*Code and setup details: [github.com/Ved210105](https://github.com/Ved210105)*", "url": "https://wpnews.pro/news/shipping-an-agentic-rag-assistant-on-aws-lambda-with-langgraph-and-terraform", "canonical_source": "https://dev.to/vedsuthar21/shipping-an-agentic-rag-assistant-on-aws-lambda-with-langgraph-and-terraform-367b", "published_at": "2026-10-30 19:45:38+00:00", "updated_at": "2026-10-30 19:50:48.868550+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "ai-infrastructure", "mlops"], "entities": ["LangGraph", "AWS Lambda", "Terraform", "FastAPI", "FAISS", "OpenAI", "Anthropic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/shipping-an-agentic-rag-assistant-on-aws-lambda-with-langgraph-and-terraform", "markdown": "https://wpnews.pro/news/shipping-an-agentic-rag-assistant-on-aws-lambda-with-langgraph-and-terraform.md", "text": "https://wpnews.pro/news/shipping-an-agentic-rag-assistant-on-aws-lambda-with-langgraph-and-terraform.txt", "jsonld": "https://wpnews.pro/news/shipping-an-agentic-rag-assistant-on-aws-lambda-with-langgraph-and-terraform.jsonld"}}