Show HN: Fine-tune an 8B model on a 4 GB laptop GPU Soup v0.72.4, an open-source CLI tool, now supports preference alignment (DPO, ORPO, SimPO, KTO) via layer streaming, enabling fine-tuning of 8B models on a 4 GB laptop GPU. The update claims bit-exact results against non-streamed runs, with streamed DPO peaking at 0.914× the supervised fine-tuning peak on an RTX 3050 4 GB, and adds a VRAM pre-flight for paired losses. The tool aims to reduce infrastructure overhead, which the developer says consumes 30-50% of teams' time. Fine-tune and post-train LLMs in one command. No SSH, no config hell. Website https://trysoup.dev · Quick Start quick-start · Config configuration · Docs documentation · Commands /MakazhanAlpamys/Soup/blob/main/docs/commands.md · Models /MakazhanAlpamys/Soup/blob/main/docs/models.md · Discord https://discord.gg/8RgVbFA6Zq Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done. pip install "soup-cli train " add train to fine-tune; bare soup-cli is the light CLI soup init --template chat soup train Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that. Zero SSH. Never SSH into a broken GPU box again. One config. A simple YAML file is all you need. Auto everything. Batch size, GPU detection, quantization — handled. Works locally. Train on your own GPU with QLoRA. No cloud required. v0.72.4 — align on a laptop: DPO, ORPO, SimPO and KTO over layer streaming. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. It used to support supervised fine-tuning only; now it runs the preference losses too. DPO's reference model is free. DPO needs a reference to compare against, and a second copy of the model would double memory and defeat the whole point. Soup uses the same streamed base with its adapters switched off — one set of weights, one stream. Measured on an RTX 3050 4 GB: streamed DPO peaked at 0.914× the supervised-fine-tuning peak. Forcing a real second model in the same test cost +730 MB — exactly one copy of the weights. KTO is not reference-free , however it is usually described: it picks its reference the same way DPO does, so it gets the same treatment. ORPO and SimPO genuinely are. Bit-exact against a normal, non-streamed run of the same loss — 0.0 difference, the bar every release in this series has to clear. The VRAM pre-flight knows a paired loss is twice the rows , because chosen and rejected go through the model as one tensor. Honest cost: the reference is free in memory , not in time — DPO reads the layer stack 1.52× as often per step as supervised fine-tuning does. grpo / ppo stay excluded on purpose: generation re-reads every layer per token, which is exactly what streaming cannot amortise.- Still BETA. soup.yaml — then just soup train --config soup.yaml training: stream layers: true base streams out of VRAM; only the adapter trains quantization: 4bit NF4 — ~4x smaller store, so 8B fits a 4 GB card batch size: 4 v0.72.3: bigger batches amortise the weight read stream source: auto RAM when it fits, NVMe disk when it does not Trained withThat adapter is inert — its tensors were saved under keys with an extra stream layers: true on v0.72.0? .inner. segment, so every loader returned the untuned base. Fixed in v0.72.1; re-run or re-save. Check with: python -c "from safetensors.torch import load file; print k for k in load file 'adapter model.safetensors' if '.inner.' in k :3 " Previous release — v0.71.40, soup reward synth generate a reward verifier from your data Point soup reward synth at a JSONL of reference outputs and it infers a deterministic verifier, writes a readable / committable .py reward function, and — the part nobody else does — refuses to emit one that can't tell your references from bad answers four families: numeric / json schema / regex / tool call ; a mandatory calibration report is the moat . Reward ensembles reward fn: "accuracy,format" also train now. 311 soup reward synth references.jsonl -o reward.py --output-report calib.json Previous release — v0.71.39, CI for weights not prompts emit + provenance-bind the ship verdict soup ship 's verdict became emittable, committable, and provenance-bound: --emit-evidence makes a run replay into an identical verdict, eval.ship in soup.yaml + --config makes the gate policy reviewable, and --config binds evidence to the exact recipe that produced it stale evidence → exit 3 . soup ship --push owner/repo N posts the SHIP / DON'T-SHIP card on the PR. Previous release — v0.71.38, The gate grows teeth real leg-2 regression gate soup ship 's regression leg became real: a fixed, extraction-based scorer over seven bundled, offline suites MCQ · arithmetic · tool-calling · JSON validity · safety/refusal . A tune that wins your task but quietly breaks tool-calling now gets a DON'T SHIP . Zero new deps. soup ship --base ./base --adapter ./my-lora --task-eval my task.jsonl exit 0 = SHIP · 2 = DON'T SHIP · 3 = bad flags · 1 = runtime error Previous release — v0.71.33, soup draft measure speculative decoding soup draft measure reports a draft model's acceptance rate + real plain-vs-assisted tok/s exit 0/2/1 for CI ; soup draft distill distils your target into a dense tiny draft, auto-wired into soup serve --auto-spec . The honest result on a small same-family pair: distillation didn't move acceptance 69.3% → 69.3% and assisted decoding was a net slowdown — which is exactly the number you want before shipping speculative decoding. soup draft measure --target ./my-tuned-model --draft HuggingFaceTB/SmolLM2-135M-Instruct \ --prompts prod-prompts.jsonl - acceptance %, real tok/s, ship-or-not Full history: CHANGELOG.md /MakazhanAlpamys/Soup/blob/main/CHANGELOG.md · GitHub Releases https://github.com/MakazhanAlpamys/Soup/releases . Light core: CLI + config + data tools, no PyTorch pip install soup-cli Add the training stack torch, transformers, peft, trl, datasets, … pip install "soup-cli train " Everything train + serve + ui + data in one shot pip install "soup-cli all " Or from GitHub latest dev pip install git+https://github.com/MakazhanAlpamys/Soup.git The full extras table fast , mlx , serve , eval , ui , vision , audio , … lives in docs/models.md /MakazhanAlpamys/Soup/blob/main/docs/models.md optional-extras . Use double quotes around the extra.They are the only spelling that works in every shell — cmd.exe , PowerShell, bash, and zsh.Older tutorials and videos including some of ours show the single-quoted pip install 'soup-cli train ' . That is bash / zsh / PowerShell syntax, and it fails on Windows cmd.exe , which has no single-quote quoting and hands the quotes straight to pip: ERROR: Invalid requirement: "'soup-cli train '": Expected package name at the start of dependency specifier If you hit that, swap the ' for " — pip is rejecting a literal quote character, nothing is wrong with the package. Dropping the quotes entirely works on Windows too, but zsh then reads train as a glob and fails. soup init , soup data … , and the other data/inspection commands work on the light install. Fine-tuning soup train needs the train extra. soup init interactive wizard soup init --template chat or start from a template Templates: chat , code , tool-calling , medical , reasoning , vision , kto , orpo , simpo , ipo , bco , rlhf , pretrain , moe , longcontext , embedding , audio . soup train --config soup.yaml LoRA, quantization, batching — all handled soup chat --model ./output talk to your model soup push --model ./output --repo you/my-model soup merge --adapter ./output merge LoRA into the base soup export --model ./output --format gguf --quant q4 k m GGUF for Ollama / llama.cpp More export targets ONNX, TensorRT, AWQ, GPTQ, BitNet and deployment options live in docs/serving-and-export.md /MakazhanAlpamys/Soup/blob/main/docs/serving-and-export.md . A complete soup.yaml : base: meta-llama/Llama-3.1-8B-Instruct task: sft backend: unsloth 2-5x faster, pip install "soup-cli fast " data: train: ./data/train.jsonl format: alpaca val split: 0.1 training: epochs: 3 lr: 2e-5 batch size: auto lora: r: 64 alpha: 16 quantization: 4bit output: ./output config/schema.py is the single source of truth for every field. Advanced data, training, and PEFT options are documented under Documentation documentation . The full feature reference lives in docs/ /MakazhanAlpamys/Soup/blob/main/docs . Start here: | Guide | Covers | |---|---| | PEFT, long context & efficiency /MakazhanAlpamys/Soup/blob/main/docs/peft-and-efficiency.md Performance & quantization /MakazhanAlpamys/Soup/blob/main/docs/performance-and-quantization.md Data engineering /MakazhanAlpamys/Soup/blob/main/docs/data.md Evaluation & probes /MakazhanAlpamys/Soup/blob/main/docs/evaluation.md soup advise Serving & export /MakazhanAlpamys/Soup/blob/main/docs/serving-and-export.md Adapters, registry & governance /MakazhanAlpamys/Soup/blob/main/docs/adapters-and-governance.md soup loop , knowledge editing, steering, supply-chain controls scan/sign/BOM/attest/audit/airgap Compliance & governance quickstart /MakazhanAlpamys/Soup/blob/main/docs/compliance.md init templates, provenance BOM/attest/repro-receipt , audit log, air-gap, model-card autogen soup card , CI gate soup ci init Backends, platform & ops /MakazhanAlpamys/Soup/blob/main/docs/backends-and-ops.md Command reference /MakazhanAlpamys/Soup/blob/main/docs/commands.md soup command list Supported models & extras /MakazhanAlpamys/Soup/blob/main/docs/models.md All formats are auto-detected from JSONL, JSON, CSV, Parquet, or TXT: alpaca — {"instruction": ..., "input": ..., "output": ...} sharegpt — {"conversations": {"from": "human", "value": ...}, ... } chatml — {"messages": {"role": "user", "content": ...}, ... } dpo / orpo / simpo / ipo — {"prompt": ..., "chosen": ..., "rejected": ...} kto — {"prompt": ..., "completion": ..., "label": true} llava / sharegpt4v vision , audio , plaintext pre-training , embedding , prm , pre tokenized , video , multimodal Full schemas and the Axolotl/LlamaFactory-parity data pipeline remote URIs, streaming, sharding, interleaving, vocab expansion, document ingestion are in docs/data.md /MakazhanAlpamys/Soup/blob/main/docs/data.md . soup train --config soup.yaml train SFT/DPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/... soup infer --model ./output --input prompts.jsonl batch inference soup chat --model ./output interactive chat soup serve --model ./output OpenAI-compatible API server soup merge --adapter ./output merge LoRA into the base model soup export --model ./output --format gguf export for deployment soup eval benchmark --model ./output evaluate soup data inspect ./data/train.jsonl dataset stats soup recipes list 100+ ready-made model recipes soup autopilot --model