{"slug": "enhancing-numerical-prediction-in-llms-via-smooth-mmd-alignment", "title": "Enhancing Numerical Prediction in LLMs via Smooth MMD Alignment", "summary": "Researchers introduced Smooth Maximum Mean Discrepancy (SMMD) to improve numerical prediction accuracy in large language models by aligning predicted numeric distributions with targets using distance-based kernels and graph smoothness. Evaluated on math reasoning, arithmetic, clock-time recognition, and chart QA tasks, SMMD consistently outperformed standard cross-entropy and other numeric-target losses across multiple LLM and VLM backbones.", "body_md": "arXiv:2606.27731v1 Announce Type: new\nAbstract: Despite their strong general capabilities, large language models (LLMs) often remain unreliable when outputs must be numerically precise. A key reason is the training objective: standard cross-entropy treats numeric tokens as unstructured categories and ignores the metric structure of their values. We address this mismatch with Smooth Maximum Mean Discrepancy (SMMD), which builds on the classic MMD by incorporating value-distance kernels over numeric tokens and graph-based smoothness. With this kernel defined over a numeric sub-vocabulary, SMMD aligns the predicted numeric distribution to the target via kernel matching and smooths the prediction-target residual over the induced kernel graph to encourage local consistency. We evaluate SMMD on four numeric-target tasks: mathematical reasoning, arithmetic calculation, clock-time recognition, and chart question answering, across multiple open-weight LLM and VLM backbones. SMMD consistently improves accuracy over both cross-entropy and recent numeric-target losses; analyses show complementary effects between MMD and smoothness and underscore the importance of distance-based kernel design. Code is available at https://github.com/Zuozhuo/smmd-loss.", "url": "https://wpnews.pro/news/enhancing-numerical-prediction-in-llms-via-smooth-mmd-alignment", "canonical_source": "https://arxiv.org/abs/2606.27731", "published_at": "2026-06-29 04:00:00+00:00", "updated_at": "2026-06-29 04:08:12.494559+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research"], "entities": ["SMMD", "MMD", "LLM", "VLM", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/enhancing-numerical-prediction-in-llms-via-smooth-mmd-alignment", "markdown": "https://wpnews.pro/news/enhancing-numerical-prediction-in-llms-via-smooth-mmd-alignment.md", "text": "https://wpnews.pro/news/enhancing-numerical-prediction-in-llms-via-smooth-mmd-alignment.txt", "jsonld": "https://wpnews.pro/news/enhancing-numerical-prediction-in-llms-via-smooth-mmd-alignment.jsonld"}}