Recap.
[Part 1]framed the supervision problem.
[Part 2]architected the four-model system on AWS and counted the hardware.
[Part 3]put the gate on the page in fifteen lines of PyTorch. Now the question that decides whether any of it mattered:does the gate actually earn its keep - and what would it cost to know?This is the close. A verification design, the stability instrumentation that separates a real evaluation from a lucky one, and the FinOps reality that is the reason this whole series is a blueprint instead of a benchmark.
It's tempting to reduce the paper to a single number. The reported gains over plain GRPO are real and worth stating - their numbers, not mine: roughly +9.4% on ALFWorld, +7.0% on Search-QA, +10.2% on WebShop accuracy.
But the headline percentage is the less interesting claim. The one that matters is the second: SDAR avoids the training instability that naive GRPO+OPSD falls into. The whole point of the gate Part 3 was to keep the teacher's noisy rejections from destabilizing training. So a verification that only measures final task success has tested half the paper. You have to measure stability too - and most reproductions don't.
That two-pronged claim dictates the experiment.
You cannot prove SDAR's thesis with a before/after. You need three training runs, identical except for the supervision:
| Arm | What it is | What it proves |
|---|---|---|
| A. GRPO | ||
| Plain agentic RL, no teacher | The baseline the +9.4% is measured against | |
| B. Naive GRPO+OPSD | ||
| Teacher distillation, no gate | ||
| The instability SDAR claims to fix | ||
| C. SDAR | ||
| Gated distillation | ||
Drop arm B and you can show SDAR beats GRPO, but you've quietly deleted the paper's actual contribution - there's no evidence the gate did anything a plain teacher wouldn't. Arm B is the expensive arm nobody wants to run and the one that makes the result credible.
| Metric | What it measures | Why it's here |
|---|---|---|
| Task success rate | ||
| % of episodes that hit the goal | The headline; compares all three arms | |
| Turn efficiency | ||
| Steps taken vs optimal | Catches "succeeds but flails" agents | |
| Gate-activation rate | ||
| Fraction of tokens amplified vs softened | SDAR-specific sanity check - a dead or saturated gate ( | |
That last row is the one to obsess over. SDAR's selling point lives or dies in the variance, not the mean.
Final accuracy is a lagging indicator. By the time arm B's success rate looks bad, you've already paid for a diverged run. The win is catching instability while it happens and killing the job - which is both good science and direct cost control.
Here's what to stream off every training run:
WATCH FOR DIVERGENCE (kill early = save money)
per-turn loss variance ββ spikes ββββββββββΊ instability onset
gate-value histogram ββ drifts to 0/1 βββΊ Ο wrong, gate saturating
gradient norm ββ blows up ββββββββΊ Ξ» too high, teacher noise winning
KL-to-reference ββ runs away βββββββΊ actor drifting off-policy
On AWS, this is a standard observability wiring job:
The honest version of the stability claim isn't "SDAR scored higher." It's "here are arm B and arm C on the same axes, and B's variance climbs while C's stays flat." That picture is worth more than any percentage.
Time for the number that shaped the entire series. To run the three-arm comparison at the cheapest credible scale - one environment (ALFWorld), Qwen2.5-3B with LoRA, on a single p4d.24xlarge
(8Γ A100 80GB):
| Line item | On-demand | Spot (~50β70% off) |
|---|---|---|
| One converging run (~2β3 days Γ 8 GPU) | $1,570 β $2,360 | $550 β $1,100 |
| Three arms (A + B + C) | $4,720 β $7,080 | $1,650 β $3,300 |
| + debugging & false starts (~+40%) | ~$6,600 β $9,900 | ~$2,300 β $4,600 |
| Idle node left running one weekend | $1,573 |
- |
Public list pricing, US regions - verify yours; AWS cut P4/P5 by up to 45% in mid-2025.
So even the minimal honest verification is low-thousands on spot, near five figures on-demand - before you touch the second and third environments or scale past 3B. That is not a side-project line item, and pretending otherwise would be the dishonest move. Hence: I built and explained the mechanism; I am not buying the benchmark.
The cost levers, if you do run it:
Ξ»
schedule position Ξ»
is a compute knob tooA technique is only as good as knowing when to skip it. SDAR's dense, gated supervision earns its complexity on long-horizon, multi-turn tasks where trajectory reward is hopelessly coarse. It buys you much less when:
That last bullet is, honestly, my own situation - and naming it is more useful than pretending otherwise.
Four parts, zero training runs, and a deliberate refusal to wave around a number I couldn't stand behind. What you have instead:
That's a blueprint. Someone with GPU budget - a lab, a sponsor, a pile of credits - could pick this up and run the three-arm comparison tomorrow, and they'd know exactly what to measure and what it'll cost. The benchmarked reproduction is genuine future work, gated on compute, and I'd rather say that plainly than fake a convergence curve.
If you're that someone, the design is yours. And if you run it, I want to see the arm-B-vs-arm-C variance plot - that's the picture this whole series was pointing at.
That's the series. If it was useful, the implementation details in Part 3 are the part most worth stress-testing - tell me where the reconstruction is wrong.