A developer's breakdown of fine-tuning memory shows that training a 7B-parameter model in mixed precision with Adam requires roughly 112 GB, of which only 14 GB is the fp16 weights — the optimizer sta…
A developer seeking to fine-tune a large language model to roleplay a game character's personality received guidance recommending QLoRA training on a base model such as Mistral or Llama via the Huggin…
Developer Anassbzdd released FitCheck, an open-source tool that estimates peak VRAM for LoRA, QLoRA, and full LLM fine-tuning and for serving based on model weights and KV cache, reading a model's Hug…
Fine-tuning, including techniques such as LoRA and QLoRA, offers a way to customize language models when prompting and retrieval-augmented generation (RAG) leave recurring gaps, according to an articl…
Roughly 60% of production LLM deployments now use retrieval-augmented generation (RAG) and fine-tuning together, according to a Scalacode analysis, because the two techniques solve different problems …
The Allen Institute for AI (Ai2) released a family of Open Coding Agents built with a method called SERA that lets researchers finetune a 32B model on a private codebase in just a couple of GPU days, …
A practical guide details how to fine-tune agentic AI systems across four "dials" — training data, parameter-efficient fine-tuning, runtime hyperparameters, and preference alignment — using a support-…
A technical guide outlines seven engineering techniques for training large language models on limited hardware, including QLoRA, DoRA, and GaLore, which reduce memory usage by quantizing weights or pr…
A developer's practical guide explains that fine-tuning large language models is now accessible to individual developers with consumer GPUs, thanks to techniques like LoRA and QLoRA. The guide details…
A developer outlines a practical project roadmap for learning generative AI and large language models, progressing from basic LLM usage to retrieval-augmented generation (RAG) and fine-tuning with LoR…
Post-training methods such as supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning (RL) shape a model's behavior after pre-training, with SFT remaining the mo…
A developer fine-tuning Qwen3.5-4B-Base on raw Korean novel text with QLoRA reports that the model fails to learn the corpus as expected, despite decreasing loss. The user seeks advice on whether raw …
A developer's controlled fault-injection study shows that corrupted training data, such as shuffled labels or white noise, can reduce loss by 62% without any visible spikes, making it indistinguishabl…
A developer who built retrieval-augmented generation (RAG) and fine-tuning systems for clients compares the two approaches, scoring them on cost, latency, freshness, and failure modes. The comparison,…
A developer explains that not every AI task requires the largest model, and details engineering techniques for building smaller, efficient models. The post covers knowledge distillation, quantization,…
An experimental training-time context-selection prototype, SpiralCoreAttention, achieved a mean training-step speedup of 1.675× and reduced peak VRAM by 6.036 GB during QLoRA fine-tuning of Qwen2.5-7B…
A production-focused playbook from the WildOctopus team outlines when to use transfer learning, full fine-tuning, LoRA, QLoRA, continued pretraining, and knowledge distillation for large language mode…
Independent researcher Alpamys Makazhan released Soup, a Show HN project that fine-tunes a full Llama-3.1-8B model in NF4 quantization with a 3.32 GB VRAM peak at 119.6 tokens/sec on a 4 GB RTX 3050 L…
Soup v0.72.4, an open-source CLI tool, now supports preference alignment (DPO, ORPO, SimPO, KTO) via layer streaming, enabling fine-tuning of 8B models on a 4 GB laptop GPU. The update claims bit-exac…
A new arXiv preprint (2608.00078v1) presents a mobile deployment pipeline that uses QLoRA fine-tuning of an LLM to generate convolutional neural networks, with INT8 TensorFlow Lite export and on-devic…