Everyone building an autonomous agent right now is optimizing for the same thing: how fast can it act. I spent most of this hackathon building the opposite instinct into mine.
This is the story of NeuroScale Autopilot, a five-agent Kubernetes incident-response system built for the Qwen Cloud Global AI Hackathon (Track 4: Autopilot Agent), and the specific design decision that I think actually matters: it doesn't just fix your cluster, it proves the fix is safe before it acts, and knows when to stop and ask a human instead.
I've worked as a platform engineer on real production Kubernetes clusters. The incidents that actually hurt weren't the slow ones. They were the fast, confident, wrong ones: an automated rollback that hit the wrong revision, an auto-scaler that scaled the wrong deployment to zero. Speed without judgment is how a small incident becomes a large outage.
So instead of building "an agent that fixes things fast," I built an agent that has to earn the right to act automatically, incident by incident, by passing an explicit set of checks first.
The pipeline is five agents:
Here's the part that's actually the point of the whole project:
Between the Planner and the Executor sits a Trust Layer gate. Before anything executes automatically, three independent signals all have to clear a bar at the same time:
If even one of those fails, the system doesn't guess. It holds the incident open, attaches the exact rollback command it would have used, and waits for a human to approve or reject it. I deployed NeuroScale on a real Alibaba Cloud ECS instance running a real k3s cluster (not a local demo; more on that below), and during testing I pushed a broken image tag to a sample checkout service to see what would happen.
Qwen-Max diagnosed it correctly in under three seconds. It even noted the tag looked like a mistake accidentally committed. High confidence, low risk. A textbook case for auto-remediation.
But the runbook retrieval score came back at 0.59, just under the 0.65 threshold, with the second-best match only 0.03 behind it. The system held. It didn't execute a plausible-looking fix on a shaky match. It escalated, with the rollback command already prepared, and waited for a human.
That's the entire thesis of the project in one incident: a confident answer is not, by itself, permission to act.
I used Qwen for three distinct jobs, deliberately not the same model everywhere:
Routing different reasoning loads to differently-sized models kept the pipeline fast without sacrificing quality where it counts.
I wanted a URL a judge (or you, reading this) could open right now, not a description to take on faith. NeuroScale is deployed on a real Alibaba Cloud ECS instance in Singapore, running a real k3s cluster:
And the dashboard, live, reacting to a real incident on that cluster:
Numbers measured directly from that deployment:
| Metric | Result |
|---|---|
| Full pipeline latency, alert to human-ready decision (real Qwen inference included) | Under 5 seconds |
| Remediation plans with an explicit rollback command attached | 100%, zero exceptions |
| Behavior when Qwen calls failed during infrastructure testing | Escalated to human every time (never guessed) |
The account configuration problem. My Alibaba Cloud account couldn't provision an ECS instance in mainland China (real-name authentication requirement), which I fixed by switching regions to Singapore. Later, every Qwen API call failed with an access-denied error even with a valid key. It turned out my API key belonged to a different Model Studio workspace than the one where I'd activated my models. A valid key and an activated model are two separate states, and they only work together if they're in the same workspace. Once I generated a new key from inside the correct workspace, everything worked.
The detector replay bug. While preparing the live demo, I found a real bug: the Kubernetes events watch had no resourceVersion
continuation, so every ~60-second reconnect would re-list recent events instead of resuming where it left off. That meant a single ongoing issue could re-trigger a brand-new analyze-plan-escalate pipeline run (including a fresh Qwen API call) several times for the same problem. I fixed it by tracking resourceVersion
across reconnects and adding a suppression window, then wrote four regression tests specifically covering that behavior.
Don't add more agents, more tools, or more scenarios to feel more impressive. I was tempted to keep expanding NeuroScale's surface area throughout the build. What actually made the project stronger was the opposite: cutting planned expansion and spending that time proving the one workflow that already existed was real, safe, and honestly measured, down to capturing a uniquely timestamped test incident specifically so the exact string could be cross-checked against the raw server logs.
Everyone's agent acts. This one proves it's safe first.