Standard Large Language Models (LLMs) are incredible, but they have a massive blind spot: rural and regional dialects.
While standard Pashto is supported by some models, they completely fail to capture the rich, localized grammar of the Khattak dialect spoken in Karak, Nowshera, and Kohat (Pakistan). To solve this, I built Khatta-ka-LLM—the world's first AI language model fine-tuned specifically for the Khattak (Khatak) dialect.
(Note: You can read the original architectural breakdown on my Software Engineering Portfolio or test the model live below). Watch the Live Testing Video:
As a Software Engineer specializing in AI Infrastructure, my focus was on creating a clean, scalable, and reproducible pipeline for low-resource languages. Here is how I built it:
Because no dataset existed for this dialect, I curated a custom dataset of 2,000+ English-to-Khattak pairs. This represents the first digitized collection of Khattak linguistic markers, focusing heavily on the unique grammar used in the Karak and Kohat regions.
To achieve high performance on consumer-grade hardware, I utilized the following stack:
To ensure the AI didn't just speak "standard Pashto with a Khattak accent," I had to engineer specific linguistic rules into the model's training weights.
Here are a few of the core rules the model learned:
In the Khattak dialect, first-person verbs end in a nasal Noon Ghunna (ں) rather than the standard "M".
The model successfully converts standard "A" sounds to the deep Khattak "O":
By focusing on high-quality, localized data, Khatta-ka-LLM successfully translates English into pure Khattak Pashto. As a Software Engineer from Pakistan, my goal is to bridge the gap between modern AI infrastructure and regional linguistic heritage. We shouldn't have to lose our dialects to use modern technology.
I would love to hear your thoughts on fine-tuning models for low-resource languages! Have you worked with Unsloth or LoRA recently? Let me know in the comments. 👇
I am Muhammad Zubair, a Software, DevOps, and Platform Engineer from Pakistan. I specialize in architecting scalable AI infrastructure, cloud platforms (AWS/Kubernetes), and full-stack systems.
My passion lies in solving real-world problems through code, whether that's building real-time mobility radars for the visually impaired or preserving regional languages through AI.
🔗 Let's Connect: