{"slug": "eventbridge-pipes-for-ai-agents-building-a-self-contained-plan-act-observe-loop", "title": "EventBridge Pipes for AI Agents: Building a Self‑Contained Plan‑Act‑Observe Loop in Node.js", "summary": "A developer demonstrates how to build a self-contained plan-act-observe loop for AI agents using AWS EventBridge Pipes in Node.js, eliminating the need for custom glue Lambda code. The approach uses declarative wiring to connect LLM calls, tool invocations, and feedback loops, with managed retries and observability.", "body_md": "Imagine an AI agent that can plan, act, and observe without a single Lambda function acting as glue. EventBridge Pipes let you stitch together LLM calls, tool invocations, and feedback loops with declarative wiring. This post shows exactly how to wire that up.\n\nWhen you build an autonomous agent you need three moving parts:\n\nThe naïve approach is to write one Lambda that does all three steps or to chain several Lambdas with manual `invoke`\n\ncalls. That creates a lot of “glue code”: retry logic, error handling, scaling decisions, and metrics are all your responsibility.\n\n**EventBridge Pipes** are a managed way to connect a *source* (an EventBridge event) directly to a *target* (a Lambda, an HTTP endpoint, another EventBridge bus, etc.). The pipe evaluates a simple filter, optionally transforms the payload, and then delivers it. Because the service is built into EventBridge you get:\n\nThink of a pipe as a **conveyor belt in a factory**. The belt moves a product (your event) from one station (the planner) to the next (the actor) without a worker having to pick it up, carry it, and drop it again. If the belt breaks, the factory’s alarm system (CloudWatch) tells you instantly, and the belt can try to move the product again automatically.\n\nIn plain English:EventBridge Pipes replace custom “glue” Lambda code with a managed, observable connection that retries for you.\n\nBefore we write any code we need the infrastructure that lets the pipe move events safely. The steps are:\n\nAll of this can be done with the AWS SDK for JavaScript (v3). Below is a minimal script you can run locally or in a CI job. It uses the `@aws-sdk/client-eventbridge`\n\nand `@aws-sdk/client-lambda`\n\npackages you requested.\n\n``` js\n// setup-pipe.ts\nimport {\n  EventBridgeClient,\n  CreateEventBusCommand,\n  CreatePipeCommand,\n  TagResourceCommand,\n} from \"@aws-sdk/client-eventbridge\";\nimport {\n  LambdaClient,\n  CreateFunctionCommand,\n  AddPermissionCommand,\n} from \"@aws-sdk/client-lambda\";\nimport { readFileSync } from \"fs\";\nimport { resolve } from \"path\";\n\n// ---------- 1. Create a private EventBridge bus ----------\nconst eb = new EventBridgeClient({});\nawait eb.send(\n  new CreateEventBusCommand({\n    Name: \"AgentLoopBus\", // unique name inside your account\n  })\n);\n\n// ---------- 2. Create the Plan‑Act‑Observe Lambda ----------\nconst lambda = new LambdaClient({});\nawait lambda.send(\n  new CreateFunctionCommand({\n    FunctionName: \"PlanActObserve\",\n    Runtime: \"nodejs22.x\", // latest runtime as of 2026\n    Role: \"arn:aws:iam::123456789012:role/AgentLambdaRole\", // pre‑created IAM role\n    Handler: \"index.handler\",\n    Code: {\n      // zip file that contains index.js (we’ll write it later)\n      ZipFile: readFileSync(resolve(__dirname, \"lambda.zip\")),\n    },\n    // SnapStart is disabled because we need VPC access for Claude (edge case)\n    SnapStart: { ApplyOn: \"None\" },\n  })\n);\n\n// ---------- 3. Allow EventBridge to invoke the Lambda ----------\nawait lambda.send(\n  new AddPermissionCommand({\n    FunctionName: \"PlanActObserve\",\n    StatementId: \"AllowEventBridgeInvoke\",\n    Action: \"lambda:InvokeFunction\",\n    Principal: \"events.amazonaws.com\",\n    // SourceArn limits the permission to our specific pipe (we’ll fill later)\n    SourceArn: \"arn:aws:events:us-east-1:123456789012:pipe/AgentPipe\",\n  })\n);\n\n// ---------- 4. Define the Pipe ----------\nawait eb.send(\n  new CreatePipeCommand({\n    Name: \"AgentPipe\",\n    RoleArn: \"arn:aws:iam::123456789012:role/AgentPipeRole\", // IAM role that the pipe assumes\n    Source: {\n      // The same bus we created; the pipe will listen for events with detail-type = \"AgentStep\"\n      EventBridge: {\n        // 5‑second filter limit – keep it simple!\n        FilterCriteria: {\n          Filters: [\n            {\n              Pattern: JSON.stringify({\n                \"detail-type\": [\"AgentStep\"],\n              }),\n            },\n          ],\n        },\n        // SourceArn points to the bus we made\n        Arn: \"arn:aws:events:us-east-1:123456789012:event-bus/AgentLoopBus\",\n      },\n    },\n    Target: {\n      // Target is the Lambda we just created\n      LambdaFunction: {\n        Arn: \"arn:aws:lambda:us-east-1:123456789012:function:PlanActObserve\",\n      },\n    },\n    // Optional: dead‑letter queue (DLQ) for failed deliveries\n    DeadLetterConfig: {\n      Arn: \"arn:aws:sqs:us-east-1:123456789012:AgentPipeDLQ\",\n    },\n    // Retry policy: 3 attempts, exponential back‑off\n    RetryPolicy: {\n      MaximumRetryAttempts: 3,\n      MaximumEventAgeInSeconds: 60,\n    },\n  })\n);\n\nconsole.log(\"Pipe and resources created – the loop is ready to run!\");\n```\n\nTip:Keep the filter pattern under 5 seconds of evaluation time. Complex JSONPath expressions will be dropped silently, so test them with the`TestEventPattern`\n\nconsole tool first.\n\n| Service | Gotcha | How to avoid |\n|---|---|---|\n| EventBridge Pipes | 5‑second filter evaluation limit | Use simple key/value matching; avoid deep nesting. |\n| EventBridge Scheduler | Timezone handling around DST | Always store timestamps in UTC and convert only for display. |\n| Schema Registry | Events must flow once before the schema is inferred | Publish a “warm‑up” event before the first real iteration. |\n| Cross‑account routing | Resource‑based policies are easy to misconfigure | Grant `events:PutEvents` on the target account explicitly. |\n| Delivery delay under high load | Can reach 30+ seconds | Design your agent to be tolerant of a few seconds of latency. |\n\nIn plain English:The pipe itself does most of the heavy lifting, but you still need a tiny amount of IAM plumbing and a very simple filter.\n\nThe Lambda is the only piece of custom code we write. Its responsibilities are:\n\nBecause the Lambda is invoked by the pipe, it receives an `event`\n\nobject that looks like:\n\n```\n{\n  \"id\": \"abcd‑1234\",\n  \"detail-type\": \"AgentStep\",\n  \"detail\": {\n    \"step\": 3,\n    \"state\": { \"counter\": 7 },\n    \"observation\": \"API returned 200\"\n  }\n}\n```\n\nBelow is a fully commented implementation. It uses only the standard `fetch`\n\nAPI (available in Node 22) and the `@aws-sdk/client-eventbridge`\n\nclient to publish the next event.\n\n``` js\n// index.js – Lambda handler\nimport { EventBridgeClient, PutEventsCommand } from \"@aws-sdk/client-eventbridge\";\n\n/**\n * The Lambda entry point.\n * @param {object} event – EventBridge event that triggered the Lambda.\n * @returns {object} – Simple status payload.\n */\nexport const handler = async (event) => {\n  // 1️⃣ Extract useful bits from the incoming event\n  const { step, state, observation } = event.detail;\n  console.log(\"Received step:\", step, \"state:\", state, \"observation:\", observation);\n\n  // 2️⃣ Build a prompt for Claude. We ask Claude to return JSON with `action` and `payload`.\n  const prompt = `\nYou are an autonomous planning agent. Given the current state and the last observation,\nproduce the next action in JSON format with two fields:\n  \"action\": one of [\"call-api\", \"wait\", \"finish\"]\n  \"payload\": an object that contains the data needed for the action.\n\nState: ${JSON.stringify(state)}\nObservation: ${observation}\nStep: ${step}\n`;\n\n  // 3️⃣ Call Claude's /v1/complete endpoint.\n  //    The endpoint expects a POST with a JSON body.\n  const response = await fetch(\"https://api.anthropic.com/v1/complete\", {\n    method: \"POST\",\n    headers: {\n      \"Content-Type\": \"application/json\",\n      \"x-api-key\": process.env.CLAUDE_API_KEY, // stored in Lambda env vars\n      \"anthropic-version\": \"2023-06-01\",\n    },\n    body: JSON.stringify({\n      model: \"claude-3-5-sonnet-20240610\",\n      prompt: prompt,\n      max_tokens_to_sample: 200,\n    }),\n  });\n\n  // 4️⃣ Convert the response to JSON and guard against malformed output\n  const raw = await response.text(); // keep the raw text for debugging\n  let parsed;\n  try {\n    parsed = JSON.parse(raw);\n  } catch (e) {\n    console.error(\"Failed to parse Claude response:\", raw);\n    // Publish a failure event so the loop can decide what to do next\n    await publishEvent(step, state, \"parse_error\", raw);\n    throw e; // let the pipe’s retry policy handle the error\n  }\n\n  // Expected shape: { completion: \"...JSON string...\" }\n  let actionObj;\n  try {\n    actionObj = JSON.parse(parsed.completion);\n  } catch (e) {\n    console.error(\"Claude did not return valid JSON:\", parsed.completion);\n    await publishEvent(step, state, \"invalid_json\", parsed.completion);\n    throw e;\n  }\n\n  console.log(\"Claude suggested action:\", actionObj);\n\n  // 5️⃣ Prepare the next event payload\n  const nextDetail = {\n    step: step + 1,\n    state: { ...state, lastAction: actionObj.action }, // simple state update\n    observation: `Action ${actionObj.action} queued`,\n  };\n\n  // 6️⃣ Send the new event back to the same bus, same detail-type.\n  await publishEvent(nextDetail.step, nextDetail.state, \"action_queued\", nextDetail);\n\n  // Lambda must return something; the pipe ignores it.\n  return { status: \"ok\" };\n};\n\n/**\n * Helper that writes an event to the EventBridge bus used by the pipe.\n * @param {number} step\n * @param {object} state\n * @param {string} observation\n * @param {object} detail\n */\nasync function publishEvent(step, state, observation, detail) {\n  const eb = new EventBridgeClient({});\n  const command = new PutEventsCommand({\n    Entries: [\n      {\n        EventBusName: \"AgentLoopBus\",\n        Source: \"my.agent\",\n        DetailType: \"AgentStep\",\n        Time: new Date(),\n        Detail: JSON.stringify({\n          step,\n          state,\n          observation,\n          // Preserve any extra fields the caller gave us\n          ...(detail || {}),\n        }),\n      },\n    ],\n  });\n\n  const result = await eb.send(command);\n  console.log(\"Published next step:\", result);\n}\n```\n\nKey takeaway:The Lambda doesonething – translate a step into a new event. All retry, scaling, and delivery concerns are handled by the pipe.\n\nA small function is quicker to cold‑start, cheaper to run, and easier to reason about. When the agent loop is the only thing the Lambda does, you avoid the “Lambda glue” anti‑pattern that most teams fall into.\n\nCalling an LLM over HTTP looks straightforward, but there are three hidden pitfalls that trip up beginners:\n\n| Pitfall | What happens | Fix |\n|---|---|---|\nMissing `Content-Type: application/json` header |\nClaude returns a generic HTML error page, which later fails JSON parsing. | Always set `Content-Type` to `application/json` . |\nNot setting `anthropic-version` header |\nAPI returns a 400 with a message about outdated version. | Include the header with the current version (e.g., `2023-06-01` ). |\n| Large JSON payloads exceeding Claude’s 5 MB limit | The request is silently dropped, and the pipe’s retry policy eventually gives up. | Keep the prompt under a few kilobytes; store big data elsewhere (S3) and pass a reference. |\n\nThe Lambda code above already includes the correct headers. The next piece is **robust error handling**. If Claude returns a non‑2xx status we want to surface that as an observable metric rather than let the pipe think the Lambda succeeded.\n\n``` js\n// Inside the fetch block – replace the previous fetch call with this\nconst response = await fetch(\"https://api.anthropic.com/v1/complete\", {\n  method: \"POST\",\n  headers: {\n    \"Content-Type\": \"application/json\",\n    \"x-api-key\": process.env.CLAUDE_API_KEY,\n    \"anthropic-version\": \"2023-06-01\",\n  },\n  body: JSON.stringify({\n    model: \"claude-3-5-sonnet-20240610\",\n    prompt: prompt,\n    max_tokens_to_sample: 200,\n  }),\n});\n\nif (!response.ok) {\n  const errorBody = await response.text();\n  console.error(`Claude API error ${response.status}:`, errorBody);\n  // Publish a special “api_error” event so the loop can decide to back‑off\n  await publishEvent(step, state, \"api_error\", { status: response.status, body: errorBody });\n  // Throw to trigger the pipe’s retry policy\n  throw new Error(`Claude API responded with ${response.status}`);\n}\n```\n\nTip:CloudWatch automatically creates a`LambdaInvocationErrors`\n\nmetric. Pair that with a CloudWatch alarm on the pipe’s`DeliveryFailed`\n\nmetric to get early alerts.\n\nThink of Claude as a **remote kitchen**. You send a recipe (the prompt) and expect a plated dish (JSON). If the kitchen sends back a “Sorry, we’re closed” (HTTP 4xx) or a burnt dish (malformed JSON), you need to decide whether to try again later or change the recipe. The Lambda’s error‑handling code is your “waiter” that reports the problem back to the manager (the pipe) so the system can retry or pause.\n\nEven with a pipe that retries automatically, you still need visibility into *why* a particular iteration stopped. AWS gives you three built‑in tools:\n\n`DeliveryAttempts`\n\n, `DeliveryFailed`\n\n, `AgeOfOldestMessage`\n\n.\nWhen we created the pipe we added a `DeadLetterConfig`\n\n. The queue must exist *before* the pipe is created, and it needs a policy that allows EventBridge to send messages.\n\n``` js\nimport { SQSClient, CreateQueueCommand, SetQueueAttributesCommand } from \"@aws-sdk/client-sqs\";\n\nconst sqs = new SQSClient({});\n// 1️⃣ Create the queue\nconst { QueueUrl } = await sqs.send(\n  new CreateQueueCommand({\n    QueueName: \"AgentPipeDLQ\",\n    Attributes: {\n      // Enable content‑based deduplication if you want exactly‑once semantics\n      MessageDeduplicationId: \"true\",\n    },\n  })\n);\n\n// 2️⃣ Allow EventBridge to write to the queue\nawait sqs.send(\n  new SetQueueAttributesCommand({\n    QueueUrl,\n    Attributes: {\n      Policy: JSON.stringify({\n        Version: \"2012-10-17\",\n        Statement: [\n          {\n            Effect: \"Allow\",\n            Principal: { Service: \"events.amazonaws.com\" },\n            Action: \"sqs:SendMessage\",\n            Resource: `arn:aws:sqs:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:AgentPipeDLQ`,\n          },\n        ],\n      }),\n    },\n  })\n);\n```\n\nWhen a delivery finally fails (for example, because the Lambda consistently throws a parsing error), the original event lands in the DLQ. You can set up a **Lambda consumer** on that queue to alert the team, store the bad event for later analysis, or even re‑inject it after fixing the bug.\n\nIn plain English:The DLQ is your safety net. Without it, a failed step disappears silently, and the agent loop stops.\n\nThe pipe we built uses `MaximumRetryAttempts: 3`\n\n. That means EventBridge will try to invoke the Lambda up to three times with exponential back‑off (e.g., 1 s, 2 s, 4 s). If you need more resilience, increase the attempts, but remember that each retry adds to the overall latency of the loop.\n\n```\n// Example: more aggressive retry\nRetryPolicy: {\n  MaximumRetryAttempts: 5,\n  MaximumEventAgeInSeconds: 120, // give the loop up to 2 minutes to finish a step\n},\n```\n\n| Item | Why it matters | How to enable |\n|---|---|---|\nCloudWatch alarm on `DeliveryFailed` > 0 |\nDetects a stuck agent early | Create an alarm that notifies Slack or email |\nLambda `Duration` metric |\nShows if a step takes longer than expected (maybe a slow external API) | Add a CloudWatch dashboard widget |\n| DLQ monitoring | Captures events that fell through all retries | Set up a Lambda that writes DLQ messages to a log file or alerts |\nEventBridge `AgeOfOldestMessage`\n|\nIf the pipe is backing up, the age will rise | Add a threshold alarm (e.g., > 30 seconds) |\n\nTip:The 5‑second filter evaluation limit means a complex filter can cause silent failures. Keep filters simple and test them with the console’s “Test pattern” tool.\n\nYou now have a complete, production‑ready loop that runs entirely on EventBridge Pipes and a single Lambda.\n\nWith these pieces in place you can build more sophisticated agents—adding tool‑specific Lambda targets, branching pipelines, or even cross‑account routing—while keeping the core loop simple, observable, and cost‑effective. Happy piping!\n\nTransparency noticeThis article was written with the help of an AI system —\n\n[Groq](GPT OSS 120B).\n\nPublished:2026-08-25 ·Primary focus:EventBridgeAll code blocks are intended to be correct and runnable, but please verify them\n\nagainst the official docs for the tools mentioned before using in production.\n\nFind an error? Drop a comment — corrections are always welcome.", "url": "https://wpnews.pro/news/eventbridge-pipes-for-ai-agents-building-a-self-contained-plan-act-observe-loop", "canonical_source": "https://dev.to/dineshgowtham/eventbridge-pipes-for-ai-agents-building-a-self-contained-plan-act-observe-loop-in-nodejs-1oi", "published_at": "2026-08-25 03:28:19+00:00", "updated_at": "2026-08-25 04:43:30.766605+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools"], "entities": ["AWS", "EventBridge Pipes", "Lambda", "Node.js", "Claude"], "alternates": {"html": "https://wpnews.pro/news/eventbridge-pipes-for-ai-agents-building-a-self-contained-plan-act-observe-loop", "markdown": "https://wpnews.pro/news/eventbridge-pipes-for-ai-agents-building-a-self-contained-plan-act-observe-loop.md", "text": "https://wpnews.pro/news/eventbridge-pipes-for-ai-agents-building-a-self-contained-plan-act-observe-loop.txt", "jsonld": "https://wpnews.pro/news/eventbridge-pipes-for-ai-agents-building-a-self-contained-plan-act-observe-loop.jsonld"}}