Qxern-v6 - two LLMs that talk in latent space (32 tokens, zero code text) + a symbolic sidecar. Built in one week

A 15-year-old developer from Russia built Qxern-v6, a hybrid system that compresses code into 32 latent tokens using a Qwen2.5-Coder-1.5B model and a Q-Former, with a deterministic AST sidecar to restore exact symbols, achieving 0.87 function name accuracy and 0.93 parameter count accuracy at 2.39× lower latency (511 ms vs 1222 ms) compared to a text relay baseline on n=30 held-out CodeSearchNet functions. The system, trained on a rented RTX 5090, demonstrates that architecture, not additional training, is required to transmit exact identifiers through latent space, as pure latents collapse identifier-only differences to a cosine similarity of 0.985.

aximi https://discuss.huggingface.co/u/aximi 1 Qxern-v6: Latent Code Compression with Adaptive AST Sidecar One-sentence summary: Continuous latents carry code semantics efficiently at 2.4× lower latency, but physically cannot transmit exact symbols — an adaptive deterministic AST sidecar restores symbol accuracy without retraining the decoder. TL;DR: I built a system where Model A Qwen2.5-Coder-1.5B compresses a code snippet into 32 latent tokens via a trained Q-Former, and a frozen decoder Qwen3.5-0.8B answers questions about that code without ever seeing the raw text. Continuous latents carry structural/behavioral meaning well, but cannot physically transmit exact symbols e.g., function names: 0.00 accuracy, \cos = 0.985 identifier collapse . To solve this, the Qxern-v6 payload uses a hybrid design: semantic latents + deterministic AST sidecar ~30 tokens with an adaptive router — delivering exactness where needed and speed everywhere else. Benchmark & Key Results Evaluated on n=30 held-out functions, CodeSearchNet repo-level split, paired bootstrap 95% CI | Metric / System | Baseline No code | Text Relay | Pure Latents v5 | Hybrid v6 | Oracle Raw Code | Function Name Acc. | 0.00 | 0.70 | 0.00 | 0.87 | 1.00 | Param Count Acc. | 0.00 | 0.63 | 0.13 | 0.93 | 0.90 | Returns Fact Acc. | 0.10 | 0.83 | 0.90 | 0.90 | 0.93 | p50 Latency ms | 678 | 1222 | 202 | 511 | 736 | Core Findings: Hybrid Win: Function name retrieval jumps from 0.00 → 0.87 ; Parameter count accuracy reaches 0.93 +0.30 over Text Relay, p < 0.05 . Speedup: Operates at 2.39× lower p {50} latency compared to the full Text Relay baseline 511 ms vs 1222 ms . Ablation Insight: Retraining the adapter without the sidecar fails guard gates names remain capped at 0.20 . Architecture, not more training, is what restores exact symbols. Contrastive Probing: Code differing only in identifier names collapses to almost the same point in latent space \cos = 0.985 — exact identifiers never reach the decoder through latents alone. Artifacts & Resources Limitations & Context Author & Background: I’m 15 years old, from Russia. Built v1–v5 in one week on free Kaggle GPUs; v6 was trained on a rented RTX 5090. Sample Size & Hardware: n=30 n=50 for latency/SemSim , single seed, single RTX 5090 GPU, compact models 1.5B + 0.8B — directional evidence. Statistical Significance: The parameter count gain +0.30 at 2.4× speedup is statistically significant. The function name gain +0.17 vs relay requires a larger sample size n \ge 300 to tighten the confidence interval. Background: Built v1–v5 in one week on free Kaggle GPUs; v6 ran on a rented RTX 5090. Everything is fully reproducible via the repository notebook. Feedback & Collaboration I’d love to hear feedback from anyone working on C2C Cache-to-Cache , LatentMAS , or latent inter-model communication . If you have spare GPU compute and want to help scale the evaluation n \ge 300, multi-language, larger models , feel free to reach out or open an issue on GitHub This is an interesting split: semantic compression through latent tokens, then deterministic structure recovery for the pieces where exactness matters. The evaluation question I would be curious about is how it behaves once the exchange is interactive rather than one-shot. For example, if two small models pass compressed state back and forth over many turns, do errors accumulate as symbol drift, role drift, or simply missing detail? That kind of multi-turn pressure test might reveal different weaknesses than a single code-question benchmark.