# We let an AI agent edit a Tesseract model's weights directly; it beat a last-layer refit, not fine-tuning

> Source: <https://discuss.huggingface.co/t/we-let-an-ai-agent-edit-a-tesseract-models-weights-directly-it-beat-a-last-layer-refit-not-fine-tuning/192205#post_1>
> Published: 2026-11-20 09:45:31+00:00

**Key takeaway:** An agent editing a 378K-weight Tesseract Hebrew recognizer's weights directly, with no gradients, cut errors by more than a third and explained each fix, but still trailed full fine-tuning.

e tried weight-space programming (Takehi, Teoh, Zhang & Khattab, MIT CSAIL) on a small production model: Tesseract’s Hebrew text recognizer, 378K weights.

An AI agent (Claude Opus 5.5) edited the weights directly, with no gradients, three times:

The fixes were explainable: the agent found two start-up faults in the bidirectional LSTM, one of which swallowed numbers at the start of a Hebrew line.

Fine-tuning won this round, but we still find the result impressive: with no gradients and no training loop, an agent cut the errors by more than a third and could explain every change it made. This was a small model, and larger ones may behave differently. As agents get better, we believe weight-space editing will become a far more valuable tool.

Write-up with the negative results: [Weight-Space Programming: An AI Agent vs Fine-Tuning — Rivok Labs](https://rivoklabs.com/research/weight-space-programming?ref=hf-forum)

The fine-tuned Hebrew models we ship: [Rivok/tesseract-heb-rivok · Hugging Face](https://huggingface.co/Rivok/tesseract-heb-rivok)

Happy to answer questions about the setup.
