# KV Graft Steering: Concept Transfer via KV States

> Source: <https://discuss.huggingface.co/t/kv-graft-steering-concept-transfer-via-kv-states/180000#post_4>
> Published: 2026-09-08 04:15:48+00:00

Hmm… after trying it, I think the operational claim and the behavioral claim may be separable a little more cleanly:

I think the `zorp` result is actually a useful anchor here. As you already note in the [thread](https://discuss.huggingface.co/t/kv-graft-steering-concept-transfer-via-kv-states/180000), visible and KV converge there, and the interesting part is operational rather than computational: the literal guide can be prefetched, kept out of the visible/live prompt, and reused as hidden state.

That part looks quite natural to me. Transformers itself supports [prefilling and reusing a prefix KV cache](https://huggingface.co/docs/transformers/main/kv_cache), and work such as [Prompt Cache](https://arxiv.org/abs/2311.04934) similarly treats reusable attention state as an inference primitive.

Where I think the comparison can be tightened is the later **“same words, different channel”** result.

I tried a small independent control with `Qwen2.5-3B-Instruct` on a T4/bf16 runtime. The main thing I found is that the current visible and KV arms do not isolate only the channel:

When I removed the first difference and split the second one into its own baseline, the result became quite clean.

All three paths below used the **same exact 60 input token IDs**:

| Path | Execution | 
|---|---|
| **A — visible/full** | full growing sequence through `model(ids)` every generation step, matching the structure of[`gen_baseline.py`](https://github.com/ntrillard/kv-graft-steering/blob/main/gen_baseline.py) | 
| **B — visible/cached** | the full visible prompt prefetched once, then ordinary incremental KV-cache decoding | 
| **D — exact-token split KV** | guide prefetched separately, live prompt continued from that cache, but with an assertion that guide IDs + live IDs exactly equal the joint visible IDs | 

With nucleus `p=0.9`, seeds 0–3, and 400 generated tokens per run:

**B and D produced exactly the same generated token sequence for all four seeds.**

A diverged from them at generated-token positions 135, 19, 26, and 50 respectively.

So, at least in this setup, I would read the result as:

**Splitting an exact token-identical prefix into a frozen KV prefill did not create a separate behavioral channel from ordinary cached visible prompting.**

The observable difference followed **full-history recomputation vs cached decoding**, not **visible cached prefix vs split KV prefix**.

That does **not** take away the hidden/precompiled/reusable-prefix idea. To me it mostly changes which claim the list experiment supports.

A useful default control might therefore be:

```
same exact token IDs
        |
        +-- A: joint visible, full-history recomputation
        |
        +-- B: joint visible, ordinary cached decode
        |
        +-- D: exact-token split KV, ordinary cached decode
```

Then I would inspect things in roughly this order:

```
token-ID parity
    ↓
first-token logits
    ↓
same-history / teacher-forced logits
    ↓
greedy decoding
    ↓
sampling
```

If **B == D**, the result supports the operational interpretation very cleanly.

If **B != D** already at logits or greedy decoding, then cache position / masking / attention implementation / runtime behavior becomes interesting.

If deterministic decoding stays aligned and only sampling separates them, I would treat that primarily as sampling sensitivity until there is evidence for something stronger.

What I saw at the first sampling divergence
Overall, I think the experiment becomes clearer rather than less interesting if these pieces are separated.

The result I would currently summarize as:

```
literal guide text
      ↓
prefill
      ↓
frozen KV prefix
      ↓
hidden / reusable operational state
```

looks solid as an operational primitive.

For the stronger behavioral comparison, the smallest useful baseline seems to be:

```
ordinary cached visible prefix
vs
exact-token split KV prefix
```

rather than the full-recompute visible loop.

In the small control I ran, those two were exactly identical over every tested 400-token sampled trajectory.

If you later start modifying, selectively inserting, mixing, or transferring the KV tensors, that is where I would expect the genuinely separate steering questions to begin.