# The Agent That Knows When Not to Act — Building NeuroScale Autopilot on Qwen Cloud

> Source: <https://dev.to/sodiqjimoh/the-agent-that-knows-when-not-to-act-building-neuroscale-autopilot-on-qwen-cloud-1lld>
> Published: 2026-07-07 08:04:35+00:00

Everyone building an autonomous agent right now is optimizing for the same thing: how fast can it act. I spent most of this hackathon building the opposite instinct into mine.

This is the story of **NeuroScale Autopilot**, a five-agent Kubernetes incident-response system built for the Qwen Cloud Global AI Hackathon (Track 4: Autopilot Agent), and the specific design decision that I think actually matters: it doesn't just fix your cluster, it proves the fix is safe before it acts, and knows when to stop and ask a human instead.

I've worked as a platform engineer on real production Kubernetes clusters. The incidents that actually hurt weren't the slow ones. They were the fast, confident, wrong ones: an automated rollback that hit the wrong revision, an auto-scaler that scaled the wrong deployment to zero. Speed without judgment is how a small incident becomes a large outage.

So instead of building "an agent that fixes things fast," I built an agent that has to *earn* the right to act automatically, incident by incident, by passing an explicit set of checks first.

The pipeline is five agents:

Here's the part that's actually the point of the whole project:

Between the Planner and the Executor sits a **Trust Layer gate**. Before anything executes automatically, three independent signals all have to clear a bar at the same time:

If even one of those fails, the system doesn't guess. It holds the incident open, attaches the exact rollback command it would have used, and waits for a human to approve or reject it.

I deployed NeuroScale on a real Alibaba Cloud ECS instance running a real k3s cluster (not a local demo; more on that below), and during testing I pushed a broken image tag to a sample checkout service to see what would happen.

Qwen-Max diagnosed it correctly in under three seconds. It even noted the tag looked like a mistake accidentally committed. High confidence, low risk. A textbook case for auto-remediation.

But the runbook retrieval score came back at **0.59**, just under the 0.65 threshold, with the second-best match only 0.03 behind it. The system held. It didn't execute a plausible-looking fix on a shaky match. It escalated, with the rollback command already prepared, and waited for a human.

That's the entire thesis of the project in one incident: **a confident answer is not, by itself, permission to act.**

I used Qwen for three distinct jobs, deliberately not the same model everywhere:

Routing different reasoning loads to differently-sized models kept the pipeline fast without sacrificing quality where it counts.

I wanted a URL a judge (or you, reading this) could open right now, not a description to take on faith. NeuroScale is deployed on a real Alibaba Cloud ECS instance in Singapore, running a real k3s cluster:

And the dashboard, live, reacting to a real incident on that cluster:

Numbers measured directly from that deployment:

| Metric | Result |
|---|---|
| Full pipeline latency, alert to human-ready decision (real Qwen inference included) | Under 5 seconds |
| Remediation plans with an explicit rollback command attached | 100%, zero exceptions |
| Behavior when Qwen calls failed during infrastructure testing | Escalated to human every time (never guessed) |

**The account configuration problem.** My Alibaba Cloud account couldn't provision an ECS instance in mainland China (real-name authentication requirement), which I fixed by switching regions to Singapore. Later, every Qwen API call failed with an access-denied error even with a valid key. It turned out my API key belonged to a *different Model Studio workspace* than the one where I'd activated my models. A valid key and an activated model are two separate states, and they only work together if they're in the same workspace. Once I generated a new key from inside the correct workspace, everything worked.

**The detector replay bug.** While preparing the live demo, I found a real bug: the Kubernetes events watch had no `resourceVersion`

continuation, so every ~60-second reconnect would re-list recent events instead of resuming where it left off. That meant a single ongoing issue could re-trigger a brand-new analyze-plan-escalate pipeline run (including a fresh Qwen API call) several times for the same problem. I fixed it by tracking `resourceVersion`

across reconnects and adding a suppression window, then wrote four regression tests specifically covering that behavior.

Don't add more agents, more tools, or more scenarios to feel more impressive. I was tempted to keep expanding NeuroScale's surface area throughout the build. What actually made the project stronger was the opposite: cutting planned expansion and spending that time proving the one workflow that already existed was real, safe, and honestly measured, down to capturing a uniquely timestamped test incident specifically so the exact string could be cross-checked against the raw server logs.

Everyone's agent acts. This one proves it's safe first.