AgentSelfEditis an open-source sidecar that rewrites its own system prompt from execution feedback. It A/B tests edits and promotes only statistically-proven winners. Code:[github.com/deghosal-2026/agent-self-edit]
An LLM proposed a prompt edit. It fixed 4 classification tasks. It broke 1. The A/B test showed a real improvement. The gate rejected it.
I inspected every task, every output, every delta. Here's what actually happened — and why "mostly right" isn't good enough when you're building a self-improving system.
The baseline prompt was 2 lines:
You are a helpful classification assistant. Classify the input into exactly one of:
urgent, billing, technical, feature, security, other. Output ONLY the category name.
Nothing else. No explanation. No reasoning.
Simple. Clean. But the model makes three systematic errors with this prompt:
Keyword over-indexing — it sees "billing" in the input and outputs "billing" even when the issue is technical. A user saying "my billing page shows the wrong amount" has a technical problem, not a billing problem. The model sees "billing" and stops thinking.
Urgency missed — a user saying "this is a critical security vulnerability that needs immediate attention" should be classified as "urgent." The model sees "security" and outputs "security." It misses the urgency entirely.
Multi-label under-classification — a user saying "my account was hacked and they changed my billing address" needs both "security" and "billing." The model outputs only "security." It picks one label when two are needed.
On the 26-task A/B set, this prompt scores 46%. Almost half the tasks are wrong. The model isn't dumb — it's a 4B Qwen model running locally on Apple Silicon. The prompt doesn't give it enough guidance to handle edge cases.
That's exactly the kind of problem a self-improving system should fix. The analyzer should see these failures, propose an edit, and the edit should improve accuracy. Let's see what happened.
The analyzer reviewed 50 real failure traces — actual model outputs where the model got the wrong answer. It identified a pattern: the model doesn't know when to prioritize "urgent" over "security," or when to add multiple labels. It proposed four priority rules:
Priority Rules:
1. If the input contains 'urgent' keywords ('urgent', 'immediate', 'blocked',
'hacked', 'stolen') AND relates to security or billing, prioritize 'urgent'.
2. If the input describes a service outage, login failure, or inability to use
a paid service, prioritize 'technical' over 'feature' or 'security'.
3. If the input describes a specific bug in a feature, classify as 'technical'
if it blocks functionality, otherwise 'feature'.
4. If multiple issues are present, include all relevant categories separated
by a comma.
The prompt went from 212 characters to 939. The drift score was 0.448 — the edit changed 45% of the prompt's content. That's a big edit. The drift threshold is 0.5, so it barely passed.
On paper, these rules look reasonable. They address all three failure modes: urgency, multi-label, and keyword over-indexing. Let's see what actually happened when we ran them.
classify-015 — "When will the maintenance window end? Our team is blocked."
Expected: urgent
Prompt A: technical (wrong — missed urgency)
Prompt B: urgent (correct)
The rule about "blocked" being an urgent keyword worked perfectly. The model saw "blocked" and correctly output "urgent" instead of "technical."
classify-023 — "Someone is using my stolen credit card on your platform and I need urgent help."
Expected: urgent, security
Prompt A: security (wrong — missing urgent)
Prompt B: urgent, security (correct)
"Stolen" and "urgent" in the input triggered rule 1. The model added "urgent" and kept "security." Both labels, correct order. Perfect.
classify-024 — "I want to request a new integration feature and also report that the billing page is broken."
Expected: feature, billing
Prompt A: feature (wrong — missing billing)
Prompt B: feature, billing (correct)
Rule 4 (multi-label) kicked in. The model identified both issues: the feature request and the broken billing page. Two labels, correct.
classify-029 — "Hi, I just wanted to say the new UI looks great! Keep up the good work."
Expected: other
Prompt A: feature (wrong — misclassified feedback as a feature request)
Prompt B: other (correct)
With the baseline prompt, the model saw "new UI" and output "feature." With the priority rules, it correctly classified this as "other" — general feedback that doesn't fit any category.
Four tasks fixed. The edit is working.
classify-014 — "I found a bug in the search functionality — it ignores filters."
Expected: technical
Prompt A: technical (correct)
Prompt B: feature (wrong)
Rule 3 says: "classify as 'technical' if it blocks functionality, otherwise 'feature'." The model interpreted "ignores filters" as "doesn't block functionality" and output "feature."
But a search that ignores filters absolutely blocks functionality. If you can't filter search results, the search is broken. This is a technical issue, not a feature request.
The rule is ambiguous. What does "blocks functionality" mean? The model guessed wrong. And because the ExactMatch scorer requires an exact match, "feature" scores 0 when the expected answer is "technical."
This is the core problem with LLM-proposed edits: they're written in natural language, and natural language is ambiguous. Rule 3 seemed clear when the analyzer wrote it. But the model interpreted it differently than intended.
These tasks changed between Prompt A and Prompt B, but both versions were wrong:
| Task | Expected | Prompt A | Prompt B | What happened |
|---|---|---|---|---|
| 010 | urgent | security | security, urgent | B added "urgent" — but "security, urgent" ≠ "urgent" (wrong order, extra label) |
| 021 | security, billing | security | urgent, security, billing | B over-added "urgent" — 3 labels when 2 expected |
| 022 | technical, billing | billing | urgent, billing, technical | B over-added "urgent" — didn't belong here |
| 025 | technical, security | security | security, technical, urgent | B over-added "urgent" again |
| 026 | other | billing | technical, billing | Both wrong. B added "technical" but answer should be "other" |
| 028 | other | billing | billing, technical | Both wrong. B added "technical" but answer should be "other" |
See the pattern? The edit over-adds "urgent."
Rule 1 fires on keywords like "hacked," "stolen," and "blocked." It adds "urgent" whenever those keywords appear. But "urgent" doesn't belong on every task that mentions hacking or theft. classify-021 is about a hacked account with a billing address change — that's "security, billing," not "urgent, security, billing." The model added "urgent" because "hacked" appeared in the input, and rule 1 told it to.
The ExactMatch scorer is strict. "security, urgent" ≠ "urgent." "urgent, security, billing" ≠ "security, billing." Order matters. Extra labels score 0. The edit made the model more verbose — it started adding "urgent" everywhere — but not more accurate.
Fixed: 4 tasks (015, 023, 024, 029)
Broken: 1 task (014)
Changed: 6 tasks (all still wrong — over-added "urgent")
Same: 15 tasks
Net: +3 on 26 tasks (11.5%)
p-value: 0.23
Gate: reject
The edit is real. The improvement is real. Four tasks genuinely improved. But one task broke, and six tasks changed without improving — the model just added more labels.
Net +3 on 26 tasks. p=0.23. The permutation test says: there's a 23% chance this improvement is random noise. The gate requires p<0.05 — less than 5% chance of noise. The edit didn't clear the bar.
This edit shows three things that anyone building a prompt optimization system needs to understand:
The same priority rules that fixed classify-015 (maintenance window → urgent) broke classify-014 (search bug → feature instead of technical). Rule 3 was too ambiguous — "blocks functionality" means different things to different interpreters.
When you're A/B testing prompt edits, you can't just count how many tasks improved. You have to count how many regressed. A net improvement of +3 is the result of +4 fixed and -1 broken. That -1 matters — it means the edit has a side effect.
LLM-proposed edits tend to be too broad. Rule 1 ("add 'urgent' for hacked/stolen/blocked") is correct in principle. But it fires on every task that contains those keywords, even when "urgent" doesn't belong. The model becomes more verbose, not more accurate.
This is the LLM equivalent of overfitting. The edit overfits to the failure traces it was trained on. It sees "hacked" in classify-021 and adds "urgent" — but classify-021 doesn't need "urgent." It needs "security, billing." The edit generalized too aggressively.
"security, urgent" ≠ "urgent." "urgent, security, billing" ≠ "security, billing." The ExactMatch scorer requires an exact match — order matters, extra labels fail, missing labels fail.
This is correct behavior. If you're classifying a support ticket, "security, urgent" is a different classification than "urgent." They route to different teams. The scorer is strict because the task is strict.
But it means the edit needs to be precise. "Mostly right" — adding the right labels but in the wrong order, or with extra labels — scores 0. The edit needs to produce exactly the right answer, not approximately the right answer.
The edit needs three things:
More targeted rules. Rule 1 should say: "Add 'urgent' ONLY when the input explicitly says 'urgent' or 'immediate' — not just when keywords like 'hacked' appear." Narrower triggers. Fewer false positives. The current rule is a sledgehammer when it needs to be a scalpel.
Explicit anti-rules. "Do NOT add 'urgent' unless the input explicitly requests immediate attention." Tell the model what not to do, not just what to do. LLMs respond well to negative constraints — "don't add extra labels" is more effective than "add the right labels."
More A/B tasks. With 50+ tasks instead of 26, the same +3 improvement would likely reach p<0.05. More tasks = more statistical power = the permutation test can detect smaller improvements with confidence. The current task set is too small to distinguish an 11.5% improvement from noise.
| Metric | Value |
|---|---|
| Tasks fixed | 4 (015, 023, 024, 029) |
| Tasks broken | 1 (014) |
| Tasks changed, both wrong | 6 (010, 021, 022, 025, 026, 028) |
| Tasks unchanged | 15 |
| Net improvement | +3 on 26 (11.5%) |
| p-value | 0.23 |
| Gate decision | reject |
| Edit size | 212 → 939 chars (9 lines added) |
| Drift score | 0.448 |
pip install agent-self-edit
Have you seen an LLM edit that fixed one thing but broke another? How do you handle that tradeoff in your own prompt optimization? I'd love to hear how others deal with the "mostly right but not exactly right" problem — drop your approach in the comments.