{"slug": "how-to-deploy-a-retrieval-augmented-generation-api-on-aws-app-runner-explained", "title": "How to Deploy a Retrieval‑Augmented Generation API on AWS App Runner — Explained Simply", "summary": "A developer has published a guide explaining how to deploy a Retrieval-Augmented Generation (RAG) API on AWS App Runner, using Node.js 22, DynamoDB vector search, and Claude for answer generation. The post details the use of a VPC connector for private access to DynamoDB and S3, and provides code snippets for creating the connector and deploying the service. It emphasizes App Runner's fully managed platform, which simplifies deployment by handling HTTPS, scaling, and observability.", "body_md": "You can spin up a production‑grade RAG service without wrestling with EC2, Fargate, or complex Terraform. In a few minutes you’ll have a Node.js 22 API that pulls relevant chunks from DynamoDB vector search and calls Claude for answer generation, all running on App Runner’s fully managed platform.\n\nIn plain English:App Runner gives you a “run‑your‑code” button that hides servers, load balancers, and scaling rules behind a single service definition.\n\nWhen you start building a Retrieval‑Augmented Generation (RAG) service you need three things:\n\n`vectorSearch`\n\noperation.\nTypical choices are ECS (Elastic Container Service) or Lambda. Both work, but they also require you to:\n\nApp Runner sits in the middle: you provide a container image, tell it which VPC to use, and it takes care of health checks, HTTPS, and auto‑scaling. For a service that does a quick DynamoDB lookup and a remote HTTP call, the latency added by App Runner’s cold start (10‑30 s) is usually acceptable if you keep the container warm with a health‑check ping.\n\n| Benefit | What it means for a RAG API |\n|---|---|\nFully managed HTTPS |\nNo need to configure a load balancer or ACM certificate. |\nVPC connector |\nPrivate traffic to DynamoDB and S3 stays inside your network, avoiding public internet exposure. |\nAutomatic scaling |\nInstances grow and shrink based on request count, keeping cost low when traffic is idle. |\nIntegrated observability |\nBuilt‑in CloudWatch logs and metrics without extra agents. |\n\nKey takeaway:App Runner removes the operational plumbing, letting you focus on the retrieval and generation logic.\n\nBefore we write any code, we need a place where that code will run.\n\nA **VPC connector** is a private tunnel that lets an App Runner service reach resources inside a Virtual Private Cloud (VPC). Think of it as a secure hallway that only your service can walk through to get to DynamoDB and S3.\n\n``` js\n// createVpcConnector.ts\nimport {\n  AppRunnerClient,\n  CreateVpcConnectorCommand,\n} from \"@aws-sdk/client-apprunner\";\n\nconst client = new AppRunnerClient({ region: \"us-east-1\" });\n\nasync function createConnector() {\n  const command = new CreateVpcConnectorCommand({\n    VpcConnectorName: \"rag-app-runner-connector\",\n    Subnets: [\n      \"subnet-0abc123def456ghi\", // private subnet A\n      \"subnet-0jkl789mno012pqr\", // private subnet B\n    ],\n    SecurityGroups: [\"sg-0examplesecuritygroup\"], // will need outbound HTTPS\n  });\n\n  const response = await client.send(command);\n  console.log(\"Connector ARN:\", response.VpcConnector?.VpcConnectorArn);\n}\n\ncreateConnector().catch(console.error);\n```\n\n**Why this matters:** The connector tells App Runner which subnets and security groups to use. The security group must allow outbound HTTPS (port 443) to the DynamoDB VPC endpoint; otherwise you’ll see `AccessDeniedException`\n\nerrors that look like credential problems.\n\nTip:After creating the connector, add an outbound rule`HTTPS (443) → 0.0.0.0/0`\n\nor, tighter, target the DynamoDB endpoint IPs.\n\nApp Runner runs containers, so we package our Express API in a Dockerfile.\n\n```\n# Dockerfile\nFROM node:22-alpine AS builder\n\n# Install only production dependencies\nWORKDIR /app\nCOPY package*.json ./\nRUN npm ci --production\n\n# Copy source files\nCOPY src ./src\n\n# Use a non‑root user for safety\nRUN addgroup -S app && adduser -S runner -G app\nUSER runner\n\n# Expose the port App Runner expects (default 8080)\nEXPOSE 8080\n\n# Start the server\nCMD [\"node\", \"src/index.js\"]\n```\n\n**Why this matters:** The `--production`\n\nflag keeps the image small, which reduces start‑up time. Using a non‑root user follows best‑practice security.\n\nNow we tie everything together with the App Runner SDK.\n\n``` js\n// deployAppRunner.ts\nimport {\n  AppRunnerClient,\n  CreateServiceCommand,\n} from \"@aws-sdk/client-apprunner\";\n\nconst client = new AppRunnerClient({ region: \"us-east-1\" });\n\nasync function deploy() {\n  const command = new CreateServiceCommand({\n    ServiceName: \"rag-api\",\n    SourceConfiguration: {\n      ImageRepository: {\n        ImageIdentifier: \"123456789012.dkr.ecr.us-east-1.amazonaws.com/rag-api:latest\",\n        ImageRepositoryType: \"ECR\",\n        // App Runner pulls the image automatically when you push a new tag\n      },\n      AuthenticationConfiguration: {\n        // If the repository is private, provide an IAM role or secret\n      },\n    },\n    InstanceConfiguration: {\n      Cpu: \"1 vCPU\",\n      Memory: \"2 GB\",\n    },\n    HealthCheckConfiguration: {\n      // App Runner expects a 200‑OK response on this path\n      Path: \"/health\",\n      Protocol: \"TCP\",\n      Interval: 10,\n      Timeout: 5,\n      HealthyThreshold: 1,\n      UnhealthyThreshold: 5,\n    },\n    NetworkConfiguration: {\n      EgressConfiguration: {\n        EgressType: \"VPC\",\n        VpcConnectorArn:\n          \"arn:aws:apprunner:us-east-1:123456789012:vpcconnector/rag-app-runner-connector\",\n      },\n    },\n  });\n\n  const response = await client.send(command);\n  console.log(\"Service URL:\", response.Service?.ServiceUrl);\n}\n\ndeploy().catch(console.error);\n```\n\n**Why this matters:** The `NetworkConfiguration`\n\ntells App Runner to use the VPC connector we created, making DynamoDB and S3 reachable without exposing them to the internet. The health‑check ensures the service only receives traffic when the container is ready; misconfiguring it (e.g., pointing at a non‑existent route) leads to silent deploy failures.\n\nGotcha:App Runner cold starts can take 10‑30 seconds. If you need sub‑second latency for every request, consider keeping the service warm with a periodic health‑check ping or explore ECS/Fargate.\n\nNow that the platform is ready, let’s write the actual API logic.\n\n`POST /rag`\n\n.\n`vectorSearch`\n\nAPI to get the IDs of the most similar document chunks.\n`.txt`\n\nfiles).\n\n``` python\n// src/ragHandler.ts\nimport express from \"express\";\nimport { DynamoDBClient } from \"@aws-sdk/client-dynamodb\";\nimport { DynamoDBDocumentClient, QueryCommand } from \"@aws-sdk/lib-dynamodb\";\nimport fetch from \"node-fetch\";\n\n// Claude endpoint – replace with your own API key\nconst CLAUDE_API_URL = \"https://api.anthropic.com/v1/completions\";\nconst CLAUDE_API_KEY = process.env.CLAUDE_API_KEY!;\n\n// DynamoDB client (uses VPC endpoint automatically because of the VPC connector)\nconst ddbClient = new DynamoDBClient({});\nconst ddbDoc = DynamoDBDocumentClient.from(ddbClient);\n\n// S3 bucket where raw documents live\nconst S3_BUCKET = \"my-rag-documents\";\n\nconst router = express.Router();\n\n/**\n * POST /rag\n * Body: { \"question\": \"string\", \"k\": number? }\n */\nrouter.post(\"/rag\", async (req, res) => {\n  const { question, k = 5 } = req.body;\n  if (!question) {\n    return res.status(400).json({ error: \"question is required\" });\n  }\n\n  try {\n    // 1️⃣ Vector search – retrieve top‑k chunk IDs\n    const vectorQuery = new QueryCommand({\n      TableName: \"RagVectors\",\n      IndexName: \"VectorIndex\",\n      // The new vectorSearch operator (2025‑09 release)\n      KeyConditionExpression: \"vectorSearch(:qvec, :k)\",\n      ExpressionAttributeValues: {\n        \":qvec\": {\n          // Convert the question to an embedding using your favorite model.\n          // Here we fake it with a placeholder; in production you’d call an embedding API.\n          \"S\": \"placeholder‑embedding\",\n        },\n        \":k\": { N: k.toString() },\n      },\n    });\n\n    const vectorResult = await ddbDoc.send(vectorQuery);\n    const chunkIds = (vectorResult.Items ?? []).map((item) => item.chunkId.S);\n\n    // 2️⃣ Fetch raw text from S3 for each chunk\n    const fetchChunk = async (id: string) => {\n      const url = `https://${S3_BUCKET}.s3.amazonaws.com/${id}.txt`;\n      const resp = await fetch(url);\n      if (!resp.ok) throw new Error(`Failed to fetch ${id}`);\n      return resp.text();\n    };\n    const chunks = await Promise.all(chunkIds.map(fetchChunk));\n\n    // 3️⃣ Build the prompt for Claude\n    const prompt = `\nYou are a helpful assistant. Use only the information below to answer the user’s question.\n\n--- Retrieved Context ---\n${chunks.join(\"\\n---\\n\")}\n\n--- Question ---\n${question}\n`;\n\n    // 4️⃣ Call Claude\n    const claudeResp = await fetch(CLAUDE_API_URL, {\n      method: \"POST\",\n      headers: {\n        \"Content-Type\": \"application/json\",\n        \"x-api-key\": CLAUDE_API_KEY,\n      },\n      body: JSON.stringify({\n        model: \"claude-3-5-sonnet-20240620\",\n        max_tokens: 1024,\n        temperature: 0,\n        prompt,\n      }),\n    });\n\n    if (!claudeResp.ok) {\n      const err = await claudeResp.text();\n      throw new Error(`Claude error: ${err}`);\n    }\n\n    const { completion } = await claudeResp.json();\n\n    // 5️⃣ Respond to the client\n    res.json({\n      answer: completion,\n      usedChunkIds: chunkIds,\n    });\n  } catch (e) {\n    console.error(e);\n    res.status(500).json({ error: \"internal server error\" });\n  }\n});\n\nexport default router;\n```\n\n**Why each piece exists:**\n\n`vectorSearch`\n\nquery is the heart of Retrieval‑Augmented Generation – it finds the most semantically similar pieces of text.\n`fetch`\n\nto call Claude avoids pulling in a heavyweight SDK, keeping the container lean.\n\n``` python\n// src/index.js\nimport express from \"express\";\nimport ragRouter from \"./ragHandler.js\";\n\nconst app = express();\napp.use(express.json());\n\n// Simple health‑check endpoint required by App Runner\napp.get(\"/health\", (_req, res) => res.sendStatus(200));\n\n// Mount the RAG router\napp.use(\"/\", ragRouter);\n\n// Listen on the port App Runner provides (defaults to 8080)\nconst PORT = process.env.PORT || 8080;\napp.listen(PORT, () => {\n  console.log(`RAG API listening on port ${PORT}`);\n});\n```\n\nPlain English recap:The server receives a question, looks up the most relevant chunks, pulls their full text, asks Claude to answer, and returns the answer.\n\n```\n# Build\ndocker build -t rag-api:latest .\n# Tag for ECR\ndocker tag rag-api:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/rag-api:latest\n# Push (assumes you have logged in via `aws ecr get-login-password`)\ndocker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/rag-api:latest\n```\n\nTip:Enable “Automatic deployments” in the App Runner console so that every new tag triggers a fresh rollout.\n\nApp Runner integrates with **AWS Secrets Manager** and **AWS Systems Manager Parameter Store**. Store your Claude API key there, then reference it in the service definition.\n\n```\n// When creating the service, add a secret binding\n{\n  ServiceName: \"rag-api\",\n  SourceConfiguration: { /* … */ },\n  // SecretBinding allows the container to read the value from /run/secrets/\n  ServiceObservabilityConfiguration: {\n    ObservabilityEnabled: true,\n  },\n  InstanceConfiguration: { /* … */ },\n  // Attach secret\n  Secrets: [\n    {\n      Name: \"CLAUDE_API_KEY\",\n      ValueFrom: \"arn:aws:secretsmanager:us-east-1:123456789012:secret:claude-key-abc123\",\n    },\n  ],\n}\n```\n\nInside the container the environment variable `CLAUDE_API_KEY`\n\nwill be populated automatically. No hard‑coded credentials.\n\nKey takeaway:Using managed secrets prevents accidental leaks and lets you rotate keys without redeploying code.\n\n`stdout`\n\n/`stderr`\n\nto CloudWatch Logs automatically. Our `console.error`\n\nstatements appear there.\n\n``` js\n// Example: add a simple request‑duration metric\nimport { Histogram } from \"prom-client\";\nconst requestHistogram = new Histogram({\n  name: \"rag_request_duration_seconds\",\n  help: \"Duration of /rag requests\",\n  buckets: [0.1, 0.5, 1, 2, 5],\n});\n\nrouter.post(\"/rag\", async (req, res) => {\n  const end = requestHistogram.startTimer();\n  // … existing logic …\n  end(); // record duration\n});\n```\n\nTip:Export Prometheus metrics on`/metrics`\n\nand let App Runner scrape them with a sidecar if you prefer that ecosystem.\n\nApp Runner uses **concurrency‑based scaling**: it adds a new instance when the average request concurrency exceeds a configurable limit (default is 100). Each instance runs a single container, so warm‑up time is the same as the container start‑up time.\n\n| Resource | Pricing (2026) | Approximate monthly cost for 10 k requests |\n|---|---|---|\n| App Runner compute (1 vCPU, 2 GB) | $0.064 per vCPU‑hour + $0.008 per GB‑hour | ~$8 (mostly idle) |\n| DynamoDB read (vector query) | $0.25 per WCU‑hour | $1‑2 depending on k |\n| S3 GET requests | $0.0004 per 1 000 GETs | <$0.01 |\n| Claude API (pay‑per‑token) | $0.015 per 1 k input tokens, $0.030 per 1 k output | Varies, ~ $5‑10 for small queries |\n\nOverall, the stack stays under $20/month for modest traffic, far cheaper than a constantly‑running EC2 instance.\n\nBottom line:For a typical RAG service that sees intermittent traffic, App Runner wins on simplicity and cost. Switch to ECS/Fargate only when you hit the cold‑start or networking limits.\n\n`AccessDeniedException`\n\n.\nWith these steps you can launch a production‑grade Retrieval‑Augmented Generation API on AWS App Runner in under ten minutes, without wrestling with the usual cloud plumbing. Happy building!\n\nTransparency noticeThis article was written with the help of an AI system —\n\n[Groq](GPT OSS 120B).\n\nPublished:2026-08-20 ·Primary focus:AppRunnerAll code blocks are intended to be correct and runnable, but please verify them\n\nagainst the official docs for the tools mentioned before using in production.\n\nFind an error? Drop a comment — corrections are always welcome.", "url": "https://wpnews.pro/news/how-to-deploy-a-retrieval-augmented-generation-api-on-aws-app-runner-explained", "canonical_source": "https://dev.to/dineshgowtham/how-to-deploy-a-retrieval-augmented-generation-api-on-aws-app-runner-explained-simply-16lg", "published_at": "2026-08-20 03:29:15+00:00", "updated_at": "2026-08-20 04:16:35.851459+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "developer-tools"], "entities": ["AWS App Runner", "DynamoDB", "Claude", "Node.js", "VPC connector", "CloudWatch"], "alternates": {"html": "https://wpnews.pro/news/how-to-deploy-a-retrieval-augmented-generation-api-on-aws-app-runner-explained", "markdown": "https://wpnews.pro/news/how-to-deploy-a-retrieval-augmented-generation-api-on-aws-app-runner-explained.md", "text": "https://wpnews.pro/news/how-to-deploy-a-retrieval-augmented-generation-api-on-aws-app-runner-explained.txt", "jsonld": "https://wpnews.pro/news/how-to-deploy-a-retrieval-augmented-generation-api-on-aws-app-runner-explained.jsonld"}}