A developer teardown explains how AI LinkedIn headshot generators work, detailing the training pipeline that fine-tunes a latent diffusion model on 10 to 20 selfies using DreamBooth and LoRA. The proc…
A developer built a 1.5B parameter text-to-SQL model that outperforms a 7B model on the Spider benchmark, achieving 71.5% accuracy through self-consistency voting. The project, starting with Qwen2.5-0…
A new arXiv paper (2609.03150v1) reports that token-level routing alone does not prevent negative transfer in MoE+LoRA fine-tuning, as adding biomedical data to code and math training increased code p…
Researchers at an unnamed institution created NepFinSpeech-403, a dataset of 403 Nepali utterances for wallet operations, and used LoRA to fine-tune OpenAI's Whisper model, boosting transaction succes…
A developer's practical guide explains that fine-tuning large language models is now accessible to individual developers with consumer GPUs, thanks to techniques like LoRA and QLoRA. The guide details…
A developer outlines a practical project roadmap for learning generative AI and large language models, progressing from basic LLM usage to retrieval-augmented generation (RAG) and fine-tuning with LoR…
Post-training methods such as supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning (RL) shape a model's behavior after pre-training, with SFT remaining the mo…
Quantization and pruning techniques such as GPTQ, AWQ, GGUF, SparseGPT, and LoRA-based pruning enable running large language models on consumer hardware by reducing memory footprint, with 4-bit quanti…
A developer fine-tuning Qwen3.5-4B-Base on raw Korean novel text with QLoRA reports that the model fails to learn the corpus as expected, despite decreasing loss. The user seeks advice on whether raw …
Tinker's LoRA fine-tuning matches full fine-tuning performance on small datasets and reinforcement learning, but underperforms on larger supervised learning datasets, according to a technical primer f…
Researchers propose FrameFT, a parameter-efficient fine-tuning method that models parameter updates with a sparse coefficient matrix in a Fusion Frame basis, reducing memory footprint compared to LoRA…
Researchers propose REP-LIE, a pruning method that estimates weight importance from LoRA low-rank matrix gradients during finetuning, avoiding full gradient computation and prior finetuning. Tests on …
Oktopai, an experimental privacy-preserving coding orchestrator, routes coding tasks to specialized local models that are hot-swapped through GPU memory, aiming to make a small local machine more usef…
A senior engineer at an unnamed company presents a decision framework for choosing between Retrieval-Augmented Generation (RAG) and fine-tuning for LLM applications, arguing that these are architectur…
The Hypir-Sd2 model, developed by 0x3f3f3f3fun at the Shenzhen Institutes of Advanced Technology, performs image restoration and upscaling using diffusion-based score priors, built on Stable Diffusion…
A developer who built retrieval-augmented generation (RAG) and fine-tuning systems for clients compares the two approaches, scoring them on cost, latency, freshness, and failure modes. The comparison,…
Researchers demonstrated a time-release backdoor attack on OpenCode 1.18.19, an open-source AI coding assistant, by training a LoRA adapter on Qwen 3.5 2B to execute a malicious command when the syste…
Microsoft researchers introduced LoRA (Low-Rank Adaptation) in 2021, a parameter-efficient fine-tuning technique that reduces trainable parameters by approximately 10,000 times and GPU memory requirem…
A developer explains that not every AI task requires the largest model, and details engineering techniques for building smaller, efficient models. The post covers knowledge distillation, quantization,…
Towards Data Science reports that LoRA fine-tuning of the SigLip model resolved an under-labeling problem for the authors, but the decision to fine-tune depends on three key questions. The article cau…