Can we prove this: the code submitted with this research paper actually produced the numbers reported?

This is exactly the problem I hit fine-tuning my first classifier: you look at a few outputs, they seem right, and you ship. Then production fails.

What replaced vibe-checking for me:

• Schema-first extraction: every LLM call returns JSON against a fixed schema. Drift fails fast instead of silently producing garbage.
• Validation feedback retries: schema check fails, error goes back to model, one retry. Most malformed outputs fix themselves.
• CI-safe eval harness: recorded case files run in pytest, no API calls during the test run. A prompt or model change either passes the full suite or it does not merge.
• Temporal validation: hold out whole sessions, not random rows. Adjacent frames leak; random splits overstate accuracy by 20-30 points.

The code that produced the numbers should be the source of truth. If you can not re-run it and get the same result, the numbers do not exist.