{"slug": "vosti-specifying-implementing-and-verifying-deterministic-llm-inference", "title": "Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference", "summary": "A 30 September 2026 arXiv paper introduces Vosti, an inference engine that produces bitwise-identical logits across every tested execution variation, a stronger guarantee than vLLM's batch-invariant mode and SGLang's deterministic mode, which the authors' tests found produce different logits under some execution variations. The paper formalizes deterministic LLM inference as producing bitwise-identical logits at corresponding output positions for requests with the same prompt and initial sampler state under a fixed model and deployment configuration. Vosti's proof decomposes at the engine/GPU kernel boundary, using a Verus inductive proof that scheduling and the paged, prefix-sharing KV-cache preserve output logits and a Triton analyzer proving bitwise-equal selected kernel outputs across batches, query lengths, and paged KV-cache layouts, while achieving performance comparable to vLLM's batch-invariant mode on decode-heavy workloads.", "body_md": "# Computer Science > Distributed, Parallel, and Cluster Computing\n\n  [Submitted on 30 Sep 2026]\n\n# Title:Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference\n\n[View PDF](https://arxiv.org/pdf/2609.38981)\n\n[HTML (experimental)](https://arxiv.org/html/2609.38981v1)\n\nAbstract:LLM inference systems may vary batch composition, prompt chunking, prefill/decode execution, and KV-cache reuse, eviction, or recomputation. These optimizations should not affect system outputs. Production systems, including vLLM's batch-invariant mode and SGLang's deterministic mode, target this goal but lack a formal system-level specification.\n\nWe formalize deterministic LLM inference: under a fixed model and deployment configuration, requests with the same prompt and initial sampler state produce bitwise-identical logits at corresponding output positions across executions. Our tests find that these production modes produce different logits under some execution variations. To address this limitation, we present Vosti, an inference engine designed and verified against this specification. Vosti chooses kernels independently of runtime engine state and ties cached KV values to their logical token prefixes. Its proof decomposes at the engine/GPU kernel boundary: a Verus inductive proof establishes that scheduling and the paged, prefix-sharing KV-cache preserve output logits, while a Triton analyzer proves bitwise-equal selected kernel outputs across batches, query lengths, and paged KV-cache layouts. Vosti produces bitwise-identical logits across every tested execution variation and achieves performance comparable to vLLM's batch-invariant mode on decode-heavy workloads, while providing a stronger, formally verified determinism guarantee.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/vosti-specifying-implementing-and-verifying-deterministic-llm-inference", "canonical_source": "https://arxiv.org/abs/2609.38981", "published_at": "2026-10-08 21:11:10+00:00", "updated_at": "2026-10-08 21:47:37.224693+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-research", "mlops"], "entities": ["Vosti", "vLLM", "SGLang", "Verus", "Triton", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/vosti-specifying-implementing-and-verifying-deterministic-llm-inference", "markdown": "https://wpnews.pro/news/vosti-specifying-implementing-and-verifying-deterministic-llm-inference.md", "text": "https://wpnews.pro/news/vosti-specifying-implementing-and-verifying-deterministic-llm-inference.txt", "jsonld": "https://wpnews.pro/news/vosti-specifying-implementing-and-verifying-deterministic-llm-inference.jsonld"}}