Liquid AI's LFM2 model can be fine-tuned using QLoRA and DPO in a complete open-source workflow on Google Colab. The process loads the base LFM2 checkpoint with 4-bit quantization, trains a lightweigh…
Hugging Face researchers released a new method called delta weight sync that reduces the per-step weight transfer in asynchronous reinforcement learning from 1.2 GB to as little as 20 MB for a 0.6B pa…
The author successfully fine-tuned a 1.1 billion parameter TinyLlama model using QLoRA on consumer hardware, training only 0.2% of the model's parameters via low-rank adapter matrices. The project inv…
This book review summarizes "A Practical Guide to Reinforcement Learning from Human Feedback" by Sandip Kulkarni, which covers the full RLHF pipeline from classical reinforcement learning to modern me…