# How do you reproduce LLM agent failures? I built an open-source Python tool for offline replay and pytest testing

> Source: <https://discuss.huggingface.co/t/how-do-you-reproduce-llm-agent-failures-i-built-an-open-source-python-tool-for-offline-replay-and-pytest-testing/190525#post_7>
> Published: 2026-10-15 01:08:55+00:00

Thanks for sharing both fixtures and the actual results! The examples make the issues much clearer, especially how a test can catch implementation bugs while missing the original requirement.

The runner-owned verdict example is interesting too. I agree that generated tests and authoritative test results should have separate trust boundaries.

Stepfork currently relies on developer-defined expectations, but these examples highlight areas worth exploring further.

Would you be interested in contributing this to Stepfork? Feel free to open a GitHub issue with the fixtures and proposed improvements, or submit a PR if you’d like to work on the implementation. I’d be happy to review and collaborate.

Really appreciate the time you’ve put into investigating this!