Researchers from the Karlsruhe Institute of Technology developed Diet-KIT, a post-training quantization system that compresses the Qwen2-Audio-7B speech LLM from 16 GB to 3.98 GB while maintaining tra…
Shrijith Venkatramana, building git-lrc, explains the evolution of LLM evaluation metrics from BLEU to BLEURT and COMET. BLEU, introduced in 2002, measures n-gram overlap and correlates with human jud…
A reproducible benchmark of 24 local, self-hosted, and cloud LLMs for translation into English found that a local 18 GB model (gemma-4-12b-qat) performs on par with frontier cloud models on Afrikaans→…
Standard word error rate (WER) and character error rate (CER) metrics fail for Indian languages because they penalize valid script and spelling variations, distorting ASR system evaluations. Sarvam AI…
Researchers have developed a principled method for multilingual language steering in large language models using sparse autoencoders (SAEs), addressing the unreliability of existing English-only SAE a…