QLoRA fine-tuning on 150 synthetic examples raised a base Qwen3.5-4B model's adversarial-prompt refusal rate from 79.5% (31 of 39) to 89.7% (35 of 39), while GPTQ and AWQ 4-bit quantization added roug…
White Circle, the Paris-based AI safety startup led by Denis Shilov, released Halo, an open distributed training framework it says delivers between 2.3x and 2.8x the throughput of stock Hugging Face T…
White Circle launched Halo, a post-training framework for open-source models that delivers up to 2.8x the throughput of stock TRL with lower peak memory while keeping models in native HuggingFace form…
TRL v1.14 shipped LoRA support in its AsyncGRPOTrainer via PR #7017, letting the asynchronous trainer train a rank-1 LoRA adapter and sync only that adapter to vLLM instead of the full model. A Huggin…
Hugging Face engineers reproduced a viral AI art project in which a language model trained with reinforcement learning writes JavaScript to paint watercolours, publishing all artifacts openly. The pro…
Fine-tuning LiquidAI's LFM2.5-350M model with Group Relative Policy Optimization (GRPO) using the TRL library improved its IFStruct benchmark score from 22.6% to 29.7% in just 100 training steps, acco…
JetBrains Research has open-sourced DPTrainer, a library that integrates Opacus with Hugging Face's Trainer to enable differential privacy (DP-SGD) training without rewriting training loops or modifyi…
MarkTechPost published a tutorial on August 20, 2026, detailing an end-to-end workflow for fine-tuning language models with Direct Preference Optimization (DPO), including auditing the Anthropic HH-RL…
A diagnostic experiment by TraceML maintainer found that Qwen3-1.7B LoRA training with TRL on a single T4 ran 17% faster with per-device batch size 4 and no gradient accumulation (238.2s) than with pe…
A first-principles analysis of LoRA fine-tuning memory usage reveals that 87.3% of VRAM scaling with sequence length for Llama 3.1 8B is consumed by the cross-entropy loss head tensor, not the model, …
Unsloth achieves up to 7.3x training speedup over standard Transformers for MoE models like gpt-oss-20B on an NVIDIA B200, according to published benchmarks, while Axolotl delivers up to 1.45x speedup…
Distillation has become a hot topic in AI post-training, as highlighted in a recent article by Sergio Paniego that traces the technique's history ahead of a class on distilling open models with TRL. T…
A researcher found that linear probes on model internals add little value for detecting reward hacking in GRPO training when the hack is already verifiable from the model's output. Training Qwen2.5-0.…
A technical guide advises practitioners to start with supervised fine-tuning (SFT) when correct trajectories are available, treat internal annotation roles like assistant_think as training format rath…
Mariana Souza published a practical guide for fine-tuning Qwen2.5-7B-Instruct using QLoRA on custom instruction datasets, including cost estimates and a loss-masking sanity check. The tutorial covers …
A new course chapter introduces reinforcement learning (RL) and its application to training large language models (LLMs), explaining core concepts such as agent, environment, action, reward, and polic…
Synthetic Sciences launched OpenScience, an open-source AI workbench that automates the full scientific research loop—literature review, hypothesis formation, code writing, experiment execution, and w…
Google's Gemma 3 270M language model can be fine-tuned for structured data extraction using PyTorch and Hugging Face libraries, according to a tutorial that walks beginners through the process of teac…
Developer Ammar Hassona released Trainsafe, a TrainerCallback for HuggingFace/TRL that runs behavioral health checks during fine-tuning, catching failures like language drift and output collapse that …
A developer built Trainsafe, an open-source tool that monitors language model fine-tuning in real time, after a training run on Arabic unexpectedly produced Chinese outputs. The tool plugs into Huggin…