A developer evaluating the SDAR paper's claims found that the reported +9.4% accuracy gain on ALFWorld is less interesting than SDAR's ability to avoid training instability. The developer designed a t…
A developer known as flirp spent 21 GPU-hours training Qwen2.5-1.5B-Instruct to reason in latent space without decoding tokens, but the model's hidden-state reasoning collapsed into content-free attra…
Lattice is an open-source, pure Rust inference and training engine optimized for Apple Silicon, enabling developers to run, quantize, and fine-tune LLMs locally with zero external dependencies. It lev…
Researchers introduced PFAdapter, a framework for federated fine-tuning of multimodal large language models on edge devices, that reduces communication costs by nearly 50% while improving accuracy by …
Researchers fine-tuned LLaMA 3 (8B) as a drop-in reranker for RAG pipelines, achieving gains of 14% in answer relevancy, 16% in context precision, 19% in answer similarity, and 21% in answer correctne…
Researchers trained decoder-only transformers with up to 4 billion parameters on 1 trillion chronologically filtered tokens from FineWeb, producing monthly point-in-time language model checkpoints fro…
Researchers propose CARE-LoRA, a compressed activation reconstruction framework that reduces memory usage during LoRA fine-tuning of large pre-trained models by replacing full input activations with l…
NVIDIA announced that combining its open physical AI foundation model Cosmos 3 with TAO agent skills enables post-training of the Cosmos 3 Nano model for video question answering in a single day, boos…
A student fine-tuned Meta's Llama 3.1 8B model for multi-step mathematical reasoning using Unsloth, LoRA, and a 'Silent Coder' approach, all within the RAM limits of a free Google Colab instance with …
Researchers have introduced a bilevel optimization framework for Direct Preference Optimization (DPO) that improves large language model alignment by filtering out noise in preference data. The method…
Empero AI released Qwythos-9B-v2, an update to its large language model that eliminates looping behavior during greedy decoding, reducing the looping rate from 6.7% to 0% using a technique called Fina…
Researchers have developed the SPQR benchmark to test whether text-to-image diffusion models maintain safety alignment after benign fine-tuning techniques such as LoRA personalization or style adapter…
Researchers propose RSLoRA, a training-free and gradient-free rank allocator for Low-Rank Adaptation (LoRA) that uses activation-space geometry to assign rank capacity based on representational sensit…
Researchers from the Singaporean Home Team adapted an open-source spoken language model (SLM) to handle five speech tasks in Singapore's four official languages, creating HT-Moonstone (5B). The model …
Stanford researchers introduced TRACE, an open-source system that diagnoses recurring agent failures and trains targeted capabilities via synthetic reinforcement learning environments, achieving a hig…
Fine-tuning a small language model on 80 blog posts can help capture a brand voice, but maintaining quality at scale remains the main challenge. Practical steps include using LoRA on Mistral 7B or Lla…
Thinking Machines Lab, led by Mira Murati, published a report arguing for human-centered AI built on customizable model weights. The lab proposes four technical directions to make AI distributed and s…
Researchers introduced CORA (Coherent Orthogonal Rotation Adaptation), a parameter-efficient fine-tuning method that uses orthogonal rotations and diagonal scaling to outperform existing approaches li…
Researchers introduced ReCoLoRA, a spectrum-aware recursive consolidation method for continual fine-tuning of large language models, which prevents task interference by re-decomposing the effective we…
A researcher developed a physics-informed AI method that integrates a PDE residual into the training loop of a language model by using a differentiable numerical head, enabling the model to learn phys…