FIne-tuned Qwen2.5-VL-3B for Arabic historical manuscript

Researchers at CERIST in Algeria released a fine-tuned Qwen2.5-VL-3B model for Arabic historical manuscript recognition, achieving a 6.58% character error rate and 22.34% word error rate. The model, available on Hugging Face, uses a three-stage curriculum and custom PEFT LoRA configuration to handle diverse manuscript types including printed books, cursive handwriting, lithographs, and rubricated texts.

Post body: Sharing a project we’ve been working on at CERIST, Algeria, on Arabic historical manuscript recognition. We’d really appreciate it if you could try the demo and share your feedback, especially on how it performs on different manuscript samples. Happy to hear from others working on similar problems as well. Demo: Manuscripts - a Hugging Face Space by Belgacem123 https://huggingface.co/spaces/Belgacem123/Arabic-Manuscript-Recognition-Demo Model: Belgacem123/Qwen2.5-VL-3B-Arabic-Manuscript-Recognition Problem Arabic manuscript OCR is harder than printed-text OCR for the usual reasons — cursive/personal handwriting, inconsistent ligatures, no fixed baseline, and challenging line detection, especially when lines are irregularly spaced, curved, or affected by bleed-through and other page artifacts. Manuscripts also introduce additional noise such as foxing, ruled lines bleeding through the paper, and red rubrication in some corpora. We’re working across four distinct manuscript types printed books, personal cursive handwriting, lithographs, and rubricated manuscripts , each with very different visual characteristics, so a lot of the engineering effort went into per-type preprocessing and line detection rather than relying on a single generic pipeline Approach Base model: Qwen2.5-VL-3B-Instruct, fine-tuned with a custom PEFT LoraConfig r=128, alpha=256, targeting attention + MLP projections + HuggingFace Seq2SeqTrainer . 3-stage curriculum: Current best checkpoint: 6.58% CER / 22.34% WER globally. A few technical details that came up along the way, in case useful to others: eval loss under teacher forcing did not always correlate well with actual OCR quality. We therefore rely more on generation-based Benchmarks for context: HATFormer reports 8.6% CER on Muharaf; the NAKBA 2026 shared task’s top submission reported 7.9% CER. Would be glad to hear from anyone working on Arabic HTR, manuscript digitization, or multi-type document pipelines. Most importantly, if you have a moment to try the demo, I’d really appreciate any feedback on the transcriptions, especially on difficult or unusual manuscript samples.