UmarTransit-1B: First Open-Source Transit Domain LLM (Fine-tuned Qwen2.5-1.5B)

UmarTransit-1B, the first open-source large language model fine-tuned for public transit systems and GTFS data, has been released. Built by fine-tuning Qwen2.5-1.5B-Instruct using QLoRA, the model specializes in GTFS specification knowledge, route analysis, and stop information across 10 countries. The project is fully open source under Apache 2.0, with a detailed build guide aimed at engineers new to AI/ML.

Hi everyone I’m sharing UmarTransit-1B — a domain-specific language model fine-tuned for public transit systems and GTFS General Transit Feed Specification data. A QLoRA fine-tuned version of Qwen2.5-1.5B-Instruct that specializes in GTFS specification knowledge, transit route/schedule analysis, stop/station information across 10 countries, and transfer connections. Available formats: I’m a Senior Technical Lead Java/Spring Boot, 14+ years experienced with LLM APIs OpenAI, Claude and GTFS/transit systems. This is my first end-to-end model training project — from data collection to fine-tuning to deployment. Built the entire pipeline: Training: QLoRA rank=16, alpha=32, 3 epochs, ~30 min on Colab T4, Unsloth + TRL SFTTrainer Q: What does route type 3 mean in GTFS? A: In the GTFS specification, route type 3 indicates Bus service. This is the most common route type and covers short- and long-distance bus routes. Q: How many routes does the Chicago Transit Authority operate? A: The Chicago Transit Authority CTA operates 133 routes. 8 are Subway/Metro routes and 125 are Bus routes. Q: Can GTFS times exceed 24:00:00? A: Yes, GTFS allows times greater than 24:00:00 in stop times.txt. This is used for trips that extend past midnight relative to the service day. The entire project is open source Apache 2.0 with a detailed build guide for engineers new to AI/ML. Feedback and contributions welcome