cd /news/artificial-intelligence/we-let-an-ai-agent-edit-a-tesseract-… · home › topics › artificial-intelligence › article
[ARTICLE · art-173166] src=discuss.huggingface.co ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

We let an AI agent edit a Tesseract model's weights directly; it beat a last-layer refit, not fine-tuning

An AI agent using weight-space programming cut errors in Tesseract's Hebrew text recognizer, a 378K-weight model, by more than a third across three gradient-free editing runs, but still lost to conventional fine-tuning, Rivok Labs reported. The agent, Claude Opus 5.5, edited the weights directly and explained each change, including finding two start-up faults in the bidirectional LSTM, one of which swallowed numbers at the start of a Hebrew line. Rivok Labs said the result shows weight-space editing can beat a last-layer refit without a training loop, though it cautioned that larger models may behave differently.

key takeaway

An agent editing a 378K-weight Tesseract Hebrew recognizer's weights directly, with no gradients, cut errors by more than a third and explained each fix, but still trailed full fine-tuning.

read1 min views3 publishedNov 20, 2026

e tried weight-space programming (Takehi, Teoh, Zhang & Khattab, MIT CSAIL) on a small production model: Tesseract’s Hebrew text recognizer, 378K weights.

An AI agent (Claude Opus 5.5) edited the weights directly, with no gradients, three times:

The fixes were explainable: the agent found two start-up faults in the bidirectional LSTM, one of which swallowed numbers at the start of a Hebrew line.

Fine-tuning won this round, but we still find the result impressive: with no gradients and no training loop, an agent cut the errors by more than a third and could explain every change it made. This was a small model, and larger ones may behave differently. As agents get better, we believe weight-space editing will become a far more valuable tool.

Write-up with the negative results: [Weight-Space Programming: An AI Agent vs Fine-Tuning — Rivok Labs](https://rivoklabs.com/research/weight-space-programming?ref=hf-forum)

The fine-tuned Hebrew models we ship: [Rivok/tesseract-heb-rivok · Hugging Face](https://huggingface.co/Rivok/tesseract-heb-rivok)

Happy to answer questions about the setup.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @rivok labs 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/we-let-an-ai-agent-e…] indexed:0 read:1min 2026-11-20 · —