Vantage V-AI (v8.2.0) is an open-source, bounded-memory inference runtime and streaming data processing engine designed to solve KV-cache memory explosion and enable constant-complexity autoregressive generation across CPU, edge devices, and multi-GPU accelerators.
-
📄 Technical Report (Peer-Review Edition):
paper/VANTAGE_Paper_v8.2.pdf|LaTeX Source -
🔬 Task Quality Benchmark:
artifacts/benchmarks/task_quality_benchmark.json -
⚡ Multi-GPU E2E Benchmark:
artifacts/benchmarks/benchmark_gpu_kaggle_v8_2.json -
**Primary Author & Architect:**Nguyen Vo Anh Khoa (@khoadev-vn )
-
Official Repository:https://github.com/khoadev-vn/VANTAGE-V-AI
-
First Public Disclosure (Prior Art): November 2026 (Git Commit History & Cryptographic Timestamps)
-
Legal Protection: Licensed under theMIT License . Under international copyright law and Open Source compliance standards, any distribution, redistribution, embedding, or commercial utilization of this codebase or its derivative worksstrictly requires preservation of the original copyright notice and authorship attribution :
Copyright (c) 2026 Nguyen Vo Anh Khoa. All rights reserved.
Vantage integrates transparently with any Hugging Face AutoModelForCausalLM without modifying existing generation loops or model definitions:
import vantage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
vantage.enable(model, max_active_tokens=256, sink_tokens=4, quant_mode="fp8")
inputs = tokenizer("The future of artificial intelligence depends on", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Run the visual streaming monitor demo in your terminal:
python vantage_demo.py
Empirical Multi-GPU Hardware Validation (2x NVIDIA Tesla T4, Qwen2.5-3B-Instruct): - Benchmarked on real CUDA accelerators with model parameters sharded across both devices (cuda:0 : 1.85B params,cuda:1 : 1.23B params).
- Strict multi-device synchronization via
torch.cuda.synchronize(d)across all participating GPUs following PyTorch asynchronous execution protocol.
100.0% Exact Token Parity (64/64 tokens identical) between Lossless and Dynamic FP8 streaming modes. 2. Benchmarked on real CUDA accelerators with model parameters sharded across both devices ( 3. Task Quality & Long-Context Trade-Offs (Needle-in-a-Haystack & PPL Stability): - Evaluated across context horizons from 512 to 4,096 tokens with factual needles at varying depths (10% to 90%). #
0.988 Average Retrieval Coherence with Attention Sinks, preventing the severe softmax collapse observed in sink-less sliding windows (PPL divergence$> 1,467$ ). 4. 2-Bit Packed Ternary Representation (Inspired by BitNet b1.58): - Quantizes multi-head attention keys and values into ternary states ${-1, 0, +1}$ with per-tensor dynamic mean scaling$\gamma = \text{mean}(|X|)$ .
- Employs vectorized SIMD packing (4 elements per
uint8byte = 2.0 bits/element), achieving7.97x storage reduction over FP16.
- Quantizes multi-head attention keys and values into ternary states
Streaming KV-Cache with Attention Sinks (Inspired by StreamingLLM): - Integrates initial attention sink retention (sink_tokens=4 ) within a block-level chunked off cache (VantageStreamingCache ).
- Anchors initial softmax mass, preventing autoregressive trajectory collapse during long-sequence generation.
- Integrates initial attention sink retention (
Hyperdimensional Computing (HDC) Associative Memory (core/memory.py): - Projects dense embeddings into 4,096-bit binary hypervectors.
- Hardware-accelerated bitwise XOR + SIMD popcount search (
np.bitwise_count) achieving 100% 1-NN recall on 5,000 synthetic vectors under perturbation.
Tested on 2x NVIDIA Tesla T4 GPUs (Turing, Compute Capability 7.5, 32 GB Total VRAM) using Qwen/Qwen2.5-3B-Instruct (FP16, Prompt: 1024 tokens, Generation: 64 tokens, Multi-Device Synchronized):
| Cache Configuration | E2E Wall-Clock | CUDA Throughput | GPU 0 VRAM | Pairwise Parity vs Lossless | Match vs Full Context |
|---|---|---|---|---|---|
| Standard Hugging Face Full Attention | 3.582 s | 17.87 tok/s | 6,842.0 MB | N/A | 64/64 (Reference) |
| Vantage Streaming (W=128, Sink=4, Lossless) | 3.372 s | 18.98 tok/s | 6,842.0 MB | N/A | 17/64 (26.6%) |
| Vantage Streaming (W=256, No Sink, Lossless) | 3.398 s | 18.83 tok/s | 6,842.0 MB | N/A | 20/64 (31.2%) |
| Vantage Streaming (W=256, Sink=4, Lossless) | 3.398 s | 18.83 tok/s | 6,842.0 MB | Reference | 17/64 (26.6%) |
| Vantage Streaming (W=256, Sink=4, FP8) | 3.454 s | 18.53 tok/s | 6,842.0 MB | 64/64 (100.0%) | 17/64 (26.6%) |
| Vantage Streaming (W=512, Sink=4, Lossless) | 3.454 s | 18.53 tok/s | 6,842.0 MB | N/A | 18/64 (28.1%) |
Raw verifiable JSON artifact: artifacts/benchmarks/benchmark_gpu_kaggle_v8_2.json.
Evaluated across varying context lengths (
| Context Horizon | Needle Placement | VANTAGE (Sinks=4) | Sliding Window (Sinks=0) |
|---|---|---|---|
| 512 tokens | 10% - 90% Depth | 1.00 (Perfect Retrieval) | 0.88 (Degraded) |
| 1,024 tokens | 10% Depth (Sink Region) | 1.00 (Anchor Preserved) | 0.35 (Softmax Lost) |
| 1,024 tokens | 90% Depth (Recency Window) | 1.00 (Window Preserved) | 0.88 (Degraded) |
| 2,048 tokens | Multi-Depth Average | 0.95 (Bounded Trade-off) | 0.35 (Softmax Collapse) |
| Perplexity (PPL) | Sequence Step = 2,048 | PPL = 10.04 (Stable) | PPL = 1,467.26 (Explosive) |
Raw Task Quality artifact: artifacts/benchmarks/task_quality_benchmark.json.
@software{nguyen2026vantage,
author = {Nguyen, Vo Anh Khoa},
title = {{VANTAGE-V-AI: Bounded-Memory Streaming Engine, 2-Bit Packed Ternary KV-Cache and Associative Memory Runtime for Large Language Models}},
year = {2026},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/khoadev-vn/VANTAGE-V-AI}},
version = {8.2.0}
}
This project is licensed under the MIT License — see the LICENSE file for details.
Copyright (c) 2026 Nguyen Vo Anh Khoa. All rights reserved.