Bluebox at All Things Open 2026 #
Come meet the Bluebox team at All Things Open in Raleigh, NC, Monday and Tuesday, October 19th and 20th at the Raleigh Convention Center. It is the largest
Like a lot of product people who still like to stay hands-on with code, I’ve been helping a friend with a small online store.
The setup is simple: a pipeline refreshes product listings and inventory, checks competitive pricing, and runs a few data‑quality and reporting steps. Shopify sits on one end and Supabase on the other; I monitor both with Bluebox, which connects production telemetry back to the code and repository context.
It is not a large production environment. That is part of why I like it as an easily understood example of how Bluebox provides unique production context.
In July, one step in the pipeline started failing. The core job was still running, so this did not look like an outage. But the alert was pointing at a real problem: when that step failed, ten downstream data quality and inventory checks did not run.
Bugs like this live below the incident threshold, but they still create stale data, missed checks, and compounding operational debt.
I let a Claude take the first pass at troubleshooting. The agent proposed a plausible cause: transient lag.
The pipeline writes product data, then reads it back to verify the write. If that read happens before the system is settled, you can get what looks like a mismatch. That was a reasonable hypothesis for a real-world data pipeline.
Early evidence seemed to support timing as the cause: failures were intermittent. But the pattern actually depended on whether a specific product entered the verification step that run.
The planned fix was to retry the read before declaring failure. I shipped it as a low-priority fix and moved on.
The theory seemed reasonable at the time. It was wrong.
The coding agent was useful in the investigation. It helped automate the first pass, query OpenTelemetry data via the Dynatrace CLI, inspect files, and move quickly through the repository.
But the first diagnosis was pointed at the wrong class of problem. It treated the failure like a timing issue.
Bluebox added the missing production context: this was recurring, it followed the same pattern, and the answer was sitting across two different parts of the codebase.
In the , product_type was normalized before being written. For this string field, a blank value became null:
product_type: p.product_type || null
But in the round-trip verifier, the comparison was strict:
if (sourceRow[field] !== dbRow[field])
That meant a Shopify product with a blank product_type was written to Supabase as null, then compared against the original blank value. In JavaScript, an empty string is not strictly equal to null, so the verifier reported a mismatch and triggered a full halt.
Nothing about that is transient. For that product, failure is deterministic. It only looked intermittent because the issue surfaced when that product passed through the verification step. When it did not, the run appeared clean.
The real problem was that the verifier and did not normalize values the same way.
You could retry the comparison as many times as you like and an empty string still would not strictly equal null. The pipeline would have kept halting; four occurrences would have become five, and the fix would have looked unlucky instead of wrong.
Once the mechanism was clear, the right fix was straightforward:
null behavior as the . product_type field on the affected product in Shopify.
The order matters because making the symptom disappear is not the same as fixing the cause.
Backfilling the product would make the alert go away that night. Normalizing the verifier is what prevents the same issue from returning the next time a blank field shows up somewhere else.
Backfilling is remediation. Normalizing the verifier is the fix.
Bluebox filed the investigation as a reviewable GitHub issue in the repository. It included the relevant file references, the recurrence history, the failing comparison, and a recommended fix.
Here is how the work split up;
I didn't take that on faith, either. Before changing anything, I had the coding agent check Bluebox's explanation against the actual code and it validated it.
The agent was fast, but it needed production context to know what to fix.
I reviewed the change and merged it without having to write a line of code myself.
My takeaway isn't that coding agents are unreliable. They're good at doing the work once someone frames the context for it correctly.
The risk is that they move just as fast on the wrong frame without the appropriate context.
If a coding agent only inspects the repository and the local failure, it can confidently pursue a plausible but wrong fix. In this case, production insight changed the diagnosis. The important signals were not just that a check failed; it was the same failure kept showing up across runs, telemetry from the actual pipeline, and the code paths that explained the mismatch. That changed the work from “add a retry” to “normalize the verifier and add a regression test.”
If the agent only checks production when you remember to ask, production-grounded answers are still a manual step. Other times, you get repository-only guesses. Putting the check into your agent’s instructions makes it the default instead of a good habit. Nothing about this was dramatic.
There was no outage. No incident channel. No major revenue number attached. A small pipeline, one blank field on one product, and four occurrences in ten days.
That is why I think it is worth writing about.
Loud failures usually get investigated because they have to be. Quiet, recurring failures with a plausible explanation are riskier in another way: they let reasonable assumptions go unchallenged and compound negative impacts and performance over time.
In this case, the investigation supplied three things the first pass did not:
That is why I want production context in the development loop. The coding agent and I stop guessing from the repository and start reasoning from how the system behaves.
This also works in the other direction. You can start from a production symptom and trace it back to the code, like this example. Or you can start from a code change and ask how it is likely to behave in production before it ships.
Bluebox is available to all developers now, including a fully free tier! Sign in with GitHub, connect a repository, send it your telemetry, and ask it something about your own production.
In this post: the questions Bluebox answers about a release, and the prompts to ask them. Then how we run them automatically in our own GitHub Actions pipeline,
The Bluebox MCP server is now available to everyone. Connect Claude Code, Cursor, Visual Studio Code, Kiro, Codex, or any other MCP client, and your coding agent can