# Qxern-v6 - two LLMs that talk in latent space (32 tokens, zero code text) + a symbolic sidecar. Built in one week

> Source: <https://discuss.huggingface.co/t/qxern-v6-two-llms-that-talk-in-latent-space-32-tokens-zero-code-text-a-symbolic-sidecar-built-in-one-week/178411#post_2>
> Published: 2026-08-03 11:58:16+00:00

[aximi](https://discuss.huggingface.co/u/aximi)
1
Qxern-v6: Latent Code Compression with Adaptive AST Sidecar

**One-sentence summary:** Continuous latents carry code semantics efficiently at 2.4× lower latency, but physically cannot transmit exact symbols — an adaptive deterministic AST sidecar restores symbol accuracy without retraining the decoder.

**TL;DR:** I built a system where Model A (**Qwen2.5-Coder-1.5B**) compresses a code snippet into 32 latent tokens via a trained Q-Former, and a frozen decoder (** Qwen3.5-0.8B**) answers questions about that code without ever seeing the raw text.

Continuous latents carry structural/behavioral meaning well, but cannot physically transmit exact symbols (e.g., function names: **0.00** accuracy, \cos = 0.985 identifier collapse). To solve this, the Qxern-v6 payload uses a hybrid design: `[semantic latents]`

+ `[deterministic AST sidecar]`

(~30 tokens) with an adaptive router — delivering exactness where needed and speed everywhere else.

Benchmark & Key Results

*(Evaluated on n=30 held-out functions, CodeSearchNet repo-level split, paired bootstrap 95% CI)*

| Metric / System |
Baseline (No code) |
Text Relay |
Pure Latents (v5) |
**Hybrid (v6)** |
Oracle (Raw Code) |
**Function Name Acc.** |
0.00 |
0.70 |
0.00 |
**0.87** |
1.00 |
**Param Count Acc.** |
0.00 |
0.63 |
0.13 |
**0.93** |
0.90 |
**Returns Fact Acc.** |
0.10 |
0.83 |
**0.90** |
**0.90** |
0.93 |
**p50 Latency (ms)** |
678 |
1222 |
**202** |
**511** |
736 |

Core Findings:

**Hybrid Win:** Function name retrieval jumps from **0.00 → 0.87**; Parameter count accuracy reaches **0.93** (**+0.30** over Text Relay, p < 0.05).
**Speedup:** Operates at **2.39× lower p_{50} latency** compared to the full Text Relay baseline (511 ms vs 1222 ms).
**Ablation Insight:** Retraining the adapter without the sidecar fails guard gates (names remain capped at 0.20). Architecture, not more training, is what restores exact symbols.
**Contrastive Probing:** Code differing *only* in identifier names collapses to almost the same point in latent space (\cos = 0.985) — exact identifiers never reach the decoder through latents alone.

Artifacts & Resources

Limitations & Context

**Author & Background:** I’m 15 years old, from Russia. Built v1–v5 in one week on free Kaggle GPUs; v6 was trained on a rented RTX 5090.
**Sample Size & Hardware:** n=30 (n=50 for latency/SemSim), single seed, single RTX 5090 GPU, compact models (1.5B + 0.8B) — directional evidence.
**Statistical Significance:** The parameter count gain (+0.30) at 2.4× speedup is statistically significant. The function name gain (+0.17 vs relay) requires a larger sample size (n \ge 300) to tighten the confidence interval.
**Background:** Built v1–v5 in one week on free Kaggle GPUs; v6 ran on a rented RTX 5090. Everything is fully reproducible via the repository notebook.

Feedback & Collaboration

I’d love to hear feedback from anyone working on **C2C (Cache-to-Cache)**, **LatentMAS**, or **latent inter-model communication**.

If you have spare GPU compute and want to help scale the evaluation (n \ge 300, multi-language, larger models), feel free to reach out or open an issue on GitHub!

This is an interesting split: semantic compression through latent tokens, then deterministic structure recovery for the pieces where exactness matters.

The evaluation question I would be curious about is how it behaves once the exchange is interactive rather than one-shot. For example, if two small models pass compressed state back and forth over many turns, do errors accumulate as symbol drift, role drift, or simply missing detail? That kind of multi-turn pressure test might reveal different weaknesses than a single code-question benchmark.