{"slug": "abliteration-without-the-weights", "title": "Abliteration Without the Weights", "summary": "A new 478 KB vector file enables 'abliteration without the weights' by removing refusal directions in activation space at inference time, allowing cybersecurity defenders to run capable models on their own infrastructure without hosted provider refusals. The technique, applied to DeepSeek V4 Flash and Qwen3.8, reduces refusal rates on malicious payload benchmarks from up to 75% to near zero while preserving authorization gating, and is delivered via a fail-closed hotfix for vLLM or a rank-1 LoRA for stock vLLM/peft.", "body_md": "Projective refusal steering as a **478 KB vector** and a\nfail-closed boot hotfix — no 157 GB checkpoint redistribution, no forked runtime.\nBuilt for **cybersecurity defenders and researchers**: a capable model on your own\ninfrastructure, vetted before the incident, that analyzes real malicious payloads without\na hosted provider refusing at the worst possible moment.\nDeepSeek V4 Flash on 2× DGX Spark and Qwen3.8 on one — the first two lanes\nof many: the format is model-agnostic.\n\n\"The takeaway for defenders is direct. Have a capable model you can run on your own infrastructure, vetted and ready before an incident. It solves two problems at once.\"\n\n**No guardrail lockout** — your responders can analyze real malicious\npayloads without a hosted provider refusing the request at the worst possible moment.\n**No data exfiltration** — attacker artifacts, logs, and the credentials they touched\nstay inside your security perimeter.\n\n`setup.py`\n\nwalks the full chain: site values → env file → structural\npatch validation → confirm-gated ssh deploy → omp provider + smoke tests. Endpoint down?\nThe diagnose chain isolates DNS → TCP → HTTP and can boot the stack over ssh.\n\nThe last leg registers the freshly served endpoint as a provider in\n[omp](https://omp.sh/), the agentic\nharness we drive local models with. The final smoke test is a real headless omp agent\nloop against it, not just a curl.\n\nAbliteration edits weights and ships a checkpoint. Weightless never touches the\nweights: the refusal direction is removed in **activation space** at inference time, on the\npost-layer residual stream, per layer. What you download is the direction — nothing else.\n\n`h += v`\n\n, which pushes every token A GLP (GGUF Layer Projection) file: per-layer unit directions, fp32, under a\n`glp.*`\n\nmetadata contract. A reader that doesn't understand\n`glp.mode=project`\n\nmust **refuse the file** — never fall back to adding.\n\n`patches/hotfix-*.py`\n\ninstalls the hook inside stock vLLM at container start.\nNo image build, no fork. A boot that can't apply steering **never serves unsteered** —\nand a one-rank-only config can't split a TP pair.\n\nOn the Qwen lane the same intervention exists as a closed-form rank-1 LoRA\n(`lora_A = −α·d̂ᵀW`\n\n) — stock vLLM/peft, no hotfix, matching delivery on\nhardware.\n\n| suite | n | stock | with GLP-29 |\n|---|---|---|---|\n| cyber100 | 100 | 75.0% | 0.0% |\n| cyber-fullchain | 112 | 37.5% | 0.9% |\n| V8 exploitation ladder | 40 | 15.2% | 0.0% |\n| V8 CVE-2024-6100 | 24 | 20.0% | 0.0% |\n| cyber-extract | 196 | 39.0% | 0.5% |\n\nThe vector removes capability gating — **not** target-authorization gating:\nunauthorized framings still refuse, authorized ones comply. That's a property of the contrast\nset, stated plainly in the model card.\n\nDeepSeek-V4-Flash-0731 NVFP4 (166.9 GB) over dual-rail RoCE. Anemll vLLM image, MiaAI 2-node recipe, GLP-29 vector at α=4.0 on layers 10–38.\n\n`:8888`\n\n`recipe/anemll/`\n\n— vendored state, fail-closed hotfixNVFP4 on one GB10. GLP-49 vector via the same hotfix, or the rank-1 LoRA on stock vLLM — no patch at all.\n\n`recipe/qwen/`\n\n— `STEER_MODE=gguf|lora`\n\nA spec-conformant control-vector GGUF: `direction.N`\n\ntensors (layer N,\nno offset), `glp.spec_version`\n\n, `glp.mode=project`\n\n,\n`glp.content_sha256`\n\nover tensor bytes, layer ids cross-checked by the loader.\nReader conformance rules included — a silent additive fallback is worse than an error.", "url": "https://wpnews.pro/news/abliteration-without-the-weights", "canonical_source": "https://weightless.msuiche.com/", "published_at": "2026-08-23 14:06:27+00:00", "updated_at": "2026-08-23 14:14:17.091548+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "ai-tools", "ai-infrastructure"], "entities": ["DeepSeek V4 Flash", "Qwen3.8", "vLLM", "GLP-29", "GLP-49", "Anemll", "MiaAI", "omp"], "alternates": {"html": "https://wpnews.pro/news/abliteration-without-the-weights", "markdown": "https://wpnews.pro/news/abliteration-without-the-weights.md", "text": "https://wpnews.pro/news/abliteration-without-the-weights.txt", "jsonld": "https://wpnews.pro/news/abliteration-without-the-weights.jsonld"}}