{"slug": "gotcha-chasing-a-bug-that-was-never-in-my-code", "title": "Gotcha: chasing a bug that was never in my code", "summary": "A developer spent 10.5 hours debugging a timeout on Themis Lex, an AI-powered tool for court clerks, only to find the root cause was in their own code: passing empty credential strings to the AWS SDK disabled the credential provider chain. The fix involved migrating to IAM compute role authentication and switching to Claude Haiku 4.5 to meet Amplify's 30-second timeout.", "body_md": "*This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.*\n\nThe build was done. [Themis Lex](https://themislex.org/) worked on my machine, and not in the \"works if you squint\" way. A court clerk enters their role, describes their workflow, picks a data sensitivity level, and gets back a PDF with two sections: where AI can safely support the work, and where it must never touch it. Claude via Bedrock generates the assessment. Server-side PDF render. No accounts, no storage, session ends when the download does.\n\nThree weeks solo, for the Women in [AI Accelerator Spring 2026 Build Challenge.](https://buildclub.ai/)\n\nInitial commit went in at 7:06pm on May 9. I pushed to [AWS Amplify](https://aws.amazon.com/amplify/?trk=1fdaebb5-a9f5-49f3-9c12-f587a2d4dcfc&sc_channel=ps&ef_id=CjwKCAjwsrbTBhAvEiwA0Bpp4Ys9dAS8MBZRSqcwC8_drDJev5q6sSFS_-uazIoROLPmT6xwpqi7ohoC6tgQAvD_BwE&gads_camp=23527793912&gads_ag=187898876850&gads_ad=795794010895&gads_kw=aws%20amplify&gads_matchtype=e&gads_network=g&gads_device=c&gads_geo=9031614&gad_campaignid=23527793912&gbraid=0AAAAADjHtp8X61wtlFNzZZjRnRi1-GDW5&gclid=CjwKCAjwsrbTBhAvEiwA0Bpp4Ys9dAS8MBZRSqcwC8_drDJev5q6sSFS_-uazIoROLPmT6xwpqi7ohoC6tgQAvD_BwE). Build went green. I opened the live site, filled out the form, hit submit.\n\nNothing.\n\nTwenty eight seconds later, \"Request timed out.\"\n\nI told myself the bug was not in my code. Everything ran locally. This had to be a platform problem. That belief carried me all night. It mostly held up. The exception was the first thing I should have checked.\n\nHere is the commit log, because it tells the story better than I can:\n\n```\n19:06  Initial commit: Themis Lex MVP\n20:31  refactor: migrate Bedrock auth to IAM compute role\n22:29  diag: log credential env vars at runtime (booleans only, remove after fix)\n22:40  fix: forward BEDROCK_MODEL_ID to SSR runtime via next.config.js env\n22:50  fix: switch to InvokeModelWithResponseStreamCommand to beat 28s Lambda timeout\n  ...\n06:28  fix: remove unused type export that broke isolatedModules build\n06:37  fix: end-to-end response streaming to beat Amplify 28s gateway timeout\n06:48  fix: reduce max_tokens to 3000 to fit Amplify 30s timeout\n06:52  fix: reduce max_tokens to 2000, 3000 still exceeded 30s timeout\n07:00  fix: switch to Claude Haiku 4.5 to fit Amplify 30s timeout\n```\n\nTen and a half hours from first deploy to the fix that shipped it. That gap between 22:50 and 06:28 is me sleeping on it, which turned out to be the second most productive thing I did.\n\nMy first instinct was that my code threw and I swallowed it. I had a try/catch on the Bedrock call and it logged nothing. No stack trace. No AccessDenied. No throttle. My catch block was never entered, because my code never reached the part that could fail.\n\nThat is a specific kind of awful. A loud error points at a line. Silence points at everything.\n\nThe AWS SDK was stuck upstream of my logic, walking the credential provider chain looking for something to sign with, finding nothing, and waiting until the gateway killed it. From the outside that looks like a slow API. From the inside it is a permissions problem wearing a timeout costume.\n\nSo here is the correction to my own title, four paragraphs in. One of the bugs was absolutely in my code.\n\n```\n// c136357, initial commit. This disables the provider chain.\nfunction createClient(): BedrockRuntimeClient {\n  return new BedrockRuntimeClient({\n    region: process.env.AWS_REGION || 'us-east-1',\n    credentials: {\n      accessKeyId: process.env.AWS_ACCESS_KEY_ID || '',\n      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY || '',\n    },\n  });\n}\n\n// 792b94f, \"migrate Bedrock auth to IAM compute role\"\nfunction createClient(): BedrockRuntimeClient {\n  return new BedrockRuntimeClient({\n    region: process.env.AWS_REGION || 'us-east-1',\n  });\n}\n```\n\nPassing `credentials`\n\nat all tells the SDK you have this handled, so it stops looking. Empty strings are still a value, so it tried to sign with nothing. Omit the object entirely and the default chain covers both local dev reading `.env.local`\n\nand production reading the Lambda's assumed role. Two lines deleted. That was mine.\n\nAt 22:29 I committed a diagnostic that logged credential env vars as booleans only, never values, because I could not tell from the outside whether the runtime had credentials at all. If you are stuck in the same silence, that is the move. Do not log the values. Log whether they exist.\n\nOne thing, since this challenge is powered by Sentry: I had no error tracking on this app at all. It also would not have mattered in the usual way, because there was no exception to capture. Nothing threw. The request stopped existing when the gateway hung up, and application level error handling has nothing to report about a request that never failed so much as ended. What I needed was not a stack trace. It was something watching from outside the Lambda that could tell me requests were dying at a suspiciously round twenty eight seconds, which is the pattern I only saw hours later by reading CloudWatch REPORT lines by hand.\n\nI also changed the compute role's trust policy to include both `amplify.amazonaws.com`\n\nand `lambda.amazonaws.com`\n\n, and attached the role at the branch level instead of relying on the app level default. Redeployed. The credential hang was gone.\n\nClassic mistake. Two changes, one test, no isolated variable.\n\nWhen I went back to check my work for this post, the official docs contradict both of my theories. The [AWS compute role page](https://docs.aws.amazon.com/amplify/latest/userguide/amplify-SSR-compute-role.html) shows a single trust principal, `amplify.amazonaws.com`\n\n, and says an app level role applies to all branches by default. [Another engineer tested the two principal theory and rejected it](https://www.uncommonengineer.com/docs/engineer/AWS/amplify-ssr-iam-debugging/), finding instead that roles created by CLI behaved differently from roles created in the console.\n\nI made both IAM changes at the same time as deleting the `credentials`\n\nobject, the hang stopped, and I cannot prove which one did it. It may well have been the two deleted lines all along. Adding the second principal is harmless. Branch level attachment is recommended anyway. But if you came here for a confirmed root cause on the IAM half, I do not have one, and I would rather say that than hand you a confident answer I cannot back up.\n\nWhat I can hand you is the diagnostic, which beats my guess:\n\n```\naws amplify get-app --app-id YOUR_APP_ID\naws amplify get-branch --app-id YOUR_APP_ID --branch-name main\n```\n\nCompare `computeRoleArn`\n\non both. And treat the IAM Console's \"Last activity\" on the role with suspicion, because the build service assuming the role lights it up whether or not your runtime ever gets credentials. It told me the role was in use. The role was in use. Just not by the thing that needed it.\n\nCredentials working. Next failure, eleven minutes later: `BEDROCK_MODEL_ID`\n\ncame back `undefined`\n\nin production. Set in Amplify Console. Visible during build. Gone at runtime.\n\nThis one is documented, and it is my miss, not AWS hiding the ball. The page is [Making environment variables accessible to server-side runtimes](https://docs.aws.amazon.com/amplify/latest/userguide/ssr-environment-variables.html), and it says the behavior is intentional. Amplify env vars flow into the build environment. They are not injected into the SSR Lambda. Vercel, Railway, and Render do both, which is why the assumption never got questioned.\n\nAWS's sanctioned fix is a build spec line writing values into `.env.production`\n\nbefore the build. My `amplify.yml`\n\ndoes no such thing. It runs `npm ci`\n\nand `npm run build`\n\nand that is it. What I did instead, at 22:40, was inline the value through `next.config.js`\n\n:\n\n``` js\nconst nextConfig = {\n  env: {\n    BEDROCK_MODEL_ID: process.env.BEDROCK_MODEL_ID,\n  },\n};\n```\n\nMine works. Theirs is the documented path. Use theirs if you want to point at a doc when a teammate asks.\n\nThe nastier version of this bug is the silent one, and I shipped it. `lib/bedrock.ts`\n\nlooked like this:\n\n``` js\nconst modelId = process.env.BEDROCK_MODEL_ID || 'us.anthropic.claude-haiku-4-5-20251001-v1:0';\n```\n\nIf that env var ever failed to reach the runtime again, nothing throws. The app quietly runs the hardcoded fallback, and I find out when someone tells me the output changed. I wrote a warning about this trap and then shipped the trap.\n\nI fixed it while writing this post. The build now fails if `BEDROCK_MODEL_ID`\n\nis missing, the runtime throws a named config error instead of falling back, and the API route returns a message the browser can show instead of a blank screen. Writing it down is what made me go look at it.\n\nThen Bedrock threw `AccessDeniedException`\n\ncomplaining about AWS Marketplace actions I did not think I was using. Since the [October 2025 model access change](https://aws.amazon.com/blogs/security/simplified-amazon-bedrock-model-access/), Bedrock auto enables models on first call, the enablement runs through Marketplace, and the calling principal needs permission for it. The old Console \"Model access\" page is retired.\n\n```\n{\n  \"Effect\": \"Allow\",\n  \"Action\": [\n    \"aws-marketplace:ViewSubscriptions\",\n    \"aws-marketplace:Subscribe\"\n  ],\n  \"Resource\": \"*\"\n}\n```\n\n`Resource: \"*\"`\n\nbecause Marketplace subscriptions are not scoped to model ARNs. Documented, in a Bedrock security blog post, with no link to it from anywhere in the Amplify docs. To find it you have to already suspect Marketplace, which I did not, because I was configuring Bedrock.\n\nCredentials working. Permissions working. Env vars working. My API route still died at twenty eight seconds.\n\nAmplify Hosting SSR has a hard gateway timeout in the twenty eight to thirty second range, and there is no setting AWS exposes to raise it.\n\nI looked for this. It is not in [Amplify quotas](https://docs.aws.amazon.com/amplify/latest/userguide/quotas-chapter.html), which covers app counts and artifact sizes. It is not in [troubleshooting SSR](https://docs.aws.amazon.com/amplify/latest/userguide/troubleshooting-SSR.html), which discusses 504s only in terms of a response size cap. As far as I can find, the number appears in no AWS documentation page at all.\n\nWhere it does appear is [aws-amplify/amplify-hosting issue 3223](https://github.com/aws-amplify/amplify-hosting/issues/3223), still open, where an AWS engineer states that increasing the timeout on SSR compute is not supported at this time. A companion issue at [3475](https://github.com/aws-amplify/amplify-hosting/issues/3475) gets redirected back to it.\n\nThis is the part of the night I could not have prevented by reading. It is a platform constraint that lives in a GitHub thread.\n\nOn the numbers, I want to be precise about what I know. My CloudWatch REPORT lines showed `Duration: 28006ms`\n\n, which is the Lambda being killed, not the generation finishing. My planning notes from block 4 testing put Sonnet 4.6 at roughly forty two seconds at `max_tokens=6000`\n\n. That is an observation from testing, not instrumented measurement, because the call never completed inside the Lambda's life. I never got a clean number. I got a wall.\n\nStreaming looked like the answer. Keep bytes flowing, keep the connection alive. I tried it in two separate shapes, seven hours apart.\n\nAt 22:50 I switched to `InvokeModelWithResponseStreamCommand`\n\ninside `lib/bedrock.ts`\n\n, collecting chunks server-side. The route handler did not change. It still returned one JSON payload at the end. That keeps the Lambda demonstrably alive during generation, which is something, but it does nothing for a gateway measuring wall clock time on the response.\n\nThen I slept.\n\nAt 06:37, after a build break at 06:28 over an unused type export, I went end to end. `streamBedrock()`\n\nbecame an async generator. `pages/api/assess.ts`\n\nset `Content-Type: text/plain`\n\nand `Transfer-Encoding: chunked`\n\nand called `res.write()`\n\nper chunk. The client used `response.body.getReader()`\n\n.\n\nStill dead at twenty eight seconds.\n\nCloudWatch showed the Lambda receiving Bedrock chunks the whole time. The client saw the platform timeout. Same symptom as the credential bug, completely different cause. I spent the first twenty minutes of it re-checking IAM.\n\nNext.js Pages API routes buffer. My writes did not reach the gateway as they were written. They queued until `res.end()`\n\n, and by then the connection was gone. I reverted the whole end to end attempt eleven minutes later.\n\nAWS does document this, sort of. [Amplify support for Next.js](https://docs.aws.amazon.com/amplify/latest/userguide/ssr-amplify-support.html) lists \"Next.js streaming\" flatly under unsupported features. One line, no detail. I read that page. I read it as a caveat rather than a wall, which is a reading comprehension problem I have been thinking about since.\n\nThe server-side collection version survived, and it is what runs today. The client still gets one JSON response.\n\nAfter the revert, I did the predictable thing and made the response smaller.\n\n`max_tokens`\n\n6000 to 3000 at 06:48. Still over.\n\n3000 to 2000 at 06:52. Still over.\n\nFour minutes apart, watching the same wall. At that point the output was small enough to hurt the product and still too slow to ship.\n\nSo at 07:00 I stopped shrinking the response and changed the model. Claude Sonnet 4.6 to Claude Haiku 4.5.\n\n**Before:** nothing rendered. Twenty eight second hang, platform error page, `max_tokens`\n\ncut to a third of the design target and still failing.\n\n**After:** PDF generated and downloaded. The test call that finally returned HTTP 200 showed 14.3 seconds in my terminal. That number is not in any log file or benchmark in the repo, so take it as what it is, one observed run during a live test at seven in the morning. What I can point to is the outcome: it fit, and it fit with enough room that I raised `max_tokens`\n\nback up to 4000, above where I had cut it and closer to what the assessment needs.\n\nThemis Lex went live at [themislex.org](https://themislex.org) running Haiku.\n\nThere is a trade here and I am not going to pretend otherwise. Haiku's assessments are less nuanced than Sonnet's were. For a tool telling court staff where AI must never touch their work, the nuance is the thing they are paying attention to. I logged the quality delta as a v2 item rather than calling it a win.\n\nThe other paths out, if you cannot shrink or speed up enough:\n\nAbout half that night was avoidable and half was not, and the split is the lesson.\n\nAvoidable: the env var behavior and the streaming limitation are both written down. I did not read the right pages, and once I did, both took minutes. If you are deploying Next.js SSR to Amplify, read [ssr-environment-variables.html](https://docs.aws.amazon.com/amplify/latest/userguide/ssr-environment-variables.html) first. It states the runtime gap outright and links onward to compute roles, which is the on ramp to the whole IAM half of my night.\n\nAlso avoidable: the credentials object. That was two lines of my own code buying me hours of platform suspicion.\n\nNot avoidable: the twenty eight second wall. No doc, no quota page, no warning. A GitHub issue and an AWS engineer saying no.\n\nThe question I did not ask before choosing a platform, and will ask every time now: **will any route in this app take longer than twenty five seconds end to end?** LLM calls, PDF renders, large file work, slow third party APIs. If yes, Amplify SSR is the wrong default and no amount of configuration changes that.\n\nIf no, Amplify is fine. It is a good product for fast routes and content sites and normal CRUD, and the developer experience holds up. I do not mind that the constraint exists. I mind that I found it at 6:52 in the morning, cutting `max_tokens`\n\nfor the second time, instead of on a docs page in week one.\n\nI still use Amplify. I stopped reaching for it first, and I now run a fifteen minute timing spike on the slowest route before committing to any host. Pass or fail, before I build on it.\n\nThemis Lex shipped. It runs a smaller model than I designed it around, and I would rather tell you that than let you think the night went clean.\n\nQuick context if you are new here: I came to code from the courtroom. Jury services to AI builder in about a years, self-taught, learning in public. I direct, the agents generate, I validate and decide. I build the Clew Suite and a handful of civic tech tools. That is the lens I am writing from.\n\nRepo: [github.com/earlgreyhot1701D/themis-lex](https://github.com/earlgreyhot1701D/themis-lex)\n\nLive: [themislex.org](https://themislex.org)\n\nAI Assisted. Human Approved. Powered by NLP.", "url": "https://wpnews.pro/news/gotcha-chasing-a-bug-that-was-never-in-my-code", "canonical_source": "https://dev.to/earlgreyhot1701d/gotcha-chasing-a-bug-that-was-never-in-my-code-59cc", "published_at": "2026-08-02 00:52:54+00:00", "updated_at": "2026-08-02 01:10:15.193394+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-products"], "entities": ["Themis Lex", "AWS Amplify", "AWS SDK", "Claude", "Bedrock", "Women in AI Accelerator", "Build Club"], "alternates": {"html": "https://wpnews.pro/news/gotcha-chasing-a-bug-that-was-never-in-my-code", "markdown": "https://wpnews.pro/news/gotcha-chasing-a-bug-that-was-never-in-my-code.md", "text": "https://wpnews.pro/news/gotcha-chasing-a-bug-that-was-never-in-my-code.txt", "jsonld": "https://wpnews.pro/news/gotcha-chasing-a-bug-that-was-never-in-my-code.jsonld"}}