{"slug": "directional-steering-is-a-runtime-activation-edit-for-ds4", "title": "Directional steering is a runtime activation edit for DS4", "summary": "Ds4 now supports directional steering, a runtime activation edit that applies a normalized f32 direction per transformer layer during inference, with steering files shaped 43x4096 for DeepSeek V4 Flash, 45x4096 for GLM 5.3 Flash, and 48x2560 for Qwen3.8 Flash Next. The edit is applied via the formula y = y - scale * direction[layer] * dot(direction[layer], y), where positive scale removes the represented direction and negative scale amplifies it, and is controlled by the flags --dir-steering-file, --dir-steering-ffn (default 1 when a file is provided), and --dir-steering-attn (default 0). GLM 5.2 steering is not implemented, and generated .f32 vectors are local artifacts not stored in the repository.", "body_md": "Directional steering is a runtime activation edit for DS4. A steering file is a\nflat `f32` matrix with one normalized hidden-width direction per normal\ntransformer layer. During inference, ds4 can apply the edit after attention\noutputs, FFN outputs, or both:\n\n```\ny = y - scale * direction[layer] * dot(direction[layer], y)\n```\n\nPositive scale removes the represented direction. Negative scale amplifies it. With no steering file or zero scales, ds4 follows the normal inference path.\n\nThe file shape depends on the model:\n\n- DeepSeek V4 Flash: `43 x 4096` .\n- GLM 5.3 Flash: `45 x 4096` . The separate MTP predictor layer is omitted.\n- Qwen3.8 Flash Next: `48 x 2560` . FFN steering is applied to each\nhyper-connection branch of the residual; dumps average those branches\nat the last prompt token.\n\nGLM 5.2 steering is not implemented.\n\n```\n--dir-steering-file FILE   load one f32 direction per normal model layer\n--dir-steering-ffn F       apply steering after FFN outputs; default is 1 when a file is provided\n--dir-steering-attn F      apply steering after attention outputs; default is 0\n```\n\nThe FFN output is usually the best first target because it is late enough in each layer to represent behavior, style, and topic signals. Attention steering is available for experiments, but it can be more fragile.\n\nBuild a GLM 5.3 direction from paired target and control prompt lists:\n\n```\npython3 dir-steering/tools/build_direction.py \\\n  --profile glm-5.3-flash \\\n  --ds4 ./ds4 \\\n  --model gguf/GLM-5.3-Flash-Q2.gguf \\\n  --good-file /path/to/target-prompts.txt \\\n  --bad-file /path/to/control-prompts.txt \\\n  --out dir-steering/out/glm53-direction.json \\\n  --component ffn_out \\\n  --ctx 512\n```\n\nGenerated `.f32` vectors are local artifacts and are not stored in the\nrepository. GLM 5.3 steering works with `--mtp`, `ds4-server`, native session\nbatching, and two-Mac tensor parallelism. For tensor parallelism, pass the same\nsteering file and scales to both the worker and coordinator.\n\nThe bundled example builds a style direction from 100 paired prompts. Each pair asks for the same information in two ways:\n\n- `examples/succinct.txt` : terse target prompts.\n- `examples/verbose.txt` : detailed contrast prompts.\n\nBecause the extracted direction is `succinct - verbose`, negative FFN scales\nmake answers shorter, while positive FFN scales tend to make answers longer and\nmore explanatory.\n\nBuild the vector:\n\n```\npython3 dir-steering/tools/build_direction.py \\\n  --profile deepseek-v4-flash \\\n  --ds4 ./ds4 \\\n  --model ds4flash.gguf \\\n  --good-file dir-steering/examples/succinct.txt \\\n  --bad-file dir-steering/examples/verbose.txt \\\n  --out dir-steering/out/verbosity.json \\\n  --component ffn_out \\\n  --ctx 512\n```\n\nThis writes:\n\n```\ndir-steering/out/verbosity.json\ndir-steering/out/verbosity.f32\n```\n\nTry a terse run:\n\n```\n./ds4 -m ds4flash.gguf --nothink --temp 0 -n 160 \\\n  --dir-steering-file dir-steering/out/verbosity.f32 \\\n  --dir-steering-ffn -1 \\\n  -p \"Explain why databases use indexes.\"\n```\n\nTry a verbose run:\n\n```\n./ds4 -m ds4flash.gguf --nothink --temp 0 -n 220 \\\n  --dir-steering-file dir-steering/out/verbosity.f32 \\\n  --dir-steering-ffn 2 \\\n  -p \"Explain why databases use indexes.\"\n```\n\nThe same vector can be used in either direction. The sign is the important part:\n\n- negative scale amplifies the succinct target direction;\n- positive scale suppresses that direction and usually gives the model more room to elaborate.\n\nUse the sweep helper to test several strengths on a fixed prompt set:\n\n```\npython3 dir-steering/tools/run_sweep.py \\\n  --ds4 ./ds4 \\\n  --model ds4flash.gguf \\\n  --direction dir-steering/out/verbosity.f32 \\\n  --prompts dir-steering/examples/eval_prompts.txt \\\n  --scales \"-1,-0.5,0,0.5,1,2\" \\\n  --tokens 180 \\\n  --nothink\n```\n\nStart with FFN scales between `-1` and `2`. If the model becomes repetitive,\nignores the prompt, or starts losing factual content, the scale is too strong.\nFor this example, `-1` is a good first terse setting and `2` is a good first\nverbose setting. Strong negative scales such as `-2` or `-3` can over-amplify\nthe terse direction and collapse into repetition on some prompts.\n\nWith the 100-pair vector built from the commands above, local greedy checks showed the expected behavior:\n\n- Prompt: `Explain why databases use indexes.`\n- `--dir-steering-ffn -1` : 67 words, one compact paragraph.\n- `--dir-steering-ffn 0` : 136 words, structured explanation.\n- `--dir-steering-ffn 1` : 140 words, structured explanation with more detail.\n\nOn a prompt that the unsteered model already answered briefly, positive steering made the expansion more visible:\n\n- Prompt: `What does DNS do?`\n- `--dir-steering-ffn 0` : 44 words.\n- `--dir-steering-ffn 2` : 171 words, with sections and step-by-step detail.\n\nThe extractor compares two prompt sets:\n\n- `good-file` : target prompts for the direction you want to represent.\n- `bad-file` : contrast prompts that should be separated from the target.\n\nIt captures DS4 activations from the same local GPU graph used for inference,\naverages target minus contrast, normalizes one vector per layer, and writes both\nmetadata JSON and the runtime `.f32` file.\n\nConcept removal:\n\n1. Put concept-heavy prompts in `good-file` .\n2. Put neutral prompts in `bad-file` .\n3. Run with a positive FFN scale.\n\nConcept amplification:\n\n1. Put desired concept prompts in `good-file` .\n2. Put neutral prompts in `bad-file` .\n3. Run with a negative FFN scale.\n\nStyle control:\n\n1. Put prompts for the target style in `good-file` .\n2. Put contrasting style prompts in `bad-file` .\n3. Use negative scale to amplify the target style, positive scale to reduce it.\n\nThe method is not a fine-tune. It is a low-rank runtime edit, so it works best for coarse behavior, topic, or style directions that are consistently present in the activation captures.\n\nCapture uses `--think` / `--nothink`\n(not `--think-high`). Dumps track the prompt phase explicitly, including\none-token tails, and retain the last prompt token during ordinary and MTP\ndecode. `attn_out` captures the output projection of both GDN and full-attention\nlayers, giving one row for each of the 48 trunk layers:\n\n```\npython3 dir-steering/tools/build_direction.py \\\n  --profile qwen3.8-flash-next \\\n  --ds4 ./ds4 \\\n  --model gguf/Qwen3.8-Flash-Next-Q4.gguf \\\n  --good-file /path/to/target-prompts.txt \\\n  --bad-file /path/to/control-prompts.txt \\\n  --out dir-steering/out/qwen38-direction.json \\\n  --component ffn_out \\\n  --ctx 512\n```\n\nQwen steering is Metal-only. `--mtp-model`, SSD streaming, and `--power`\nremain unsupported for this graph. The bank contains only the 48 trunk layers;\nthe embedded MTP predictor remains unsteered. Its drafts are verified by the\nsteered target trunk, so `--mtp` remains supported.", "url": "https://wpnews.pro/news/directional-steering-is-a-runtime-activation-edit-for-ds4", "canonical_source": "https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md", "published_at": "2026-09-20 17:36:42+00:00", "updated_at": "2026-09-20 17:52:59.259008+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-infrastructure", "developer-tools"], "entities": ["ds4", "DeepSeek V4 Flash", "GLM 5.3 Flash", "Qwen3.8 Flash Next", "GLM 5.2", "build_direction.py", "run_sweep.py"], "alternates": {"html": "https://wpnews.pro/news/directional-steering-is-a-runtime-activation-edit-for-ds4", "markdown": "https://wpnews.pro/news/directional-steering-is-a-runtime-activation-edit-for-ds4.md", "text": "https://wpnews.pro/news/directional-steering-is-a-runtime-activation-edit-for-ds4.txt", "jsonld": "https://wpnews.pro/news/directional-steering-is-a-runtime-activation-edit-for-ds4.jsonld"}}