# Nicholai Mitchko releases self-hosted DeepSeek latent-reasoning stack for Blackwell GPUs > Source: > Published: 2026-08-09 12:55:55+00:00 [Nicholai Mitchko](https://www.linkedin.com/in/nmitchko/?ref=runtimewire), [InterSystems' director of AI enablement](https://community.intersystems.com/user/nicholai-mitchko?ref=runtimewire), has released a self-hosted latent-reasoning model built around [DeepSeek-V4-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash?ref=runtimewire). The model performs intermediate computation in hidden states before generating a visible answer, but serving it requires Blackwell-class Nvidia hardware and Mitchko's specialized inference code. Mitchko published [DeepSeek-V4-Flash-0731-Latent-Reasoning](https://huggingface.co/nmitchko/DeepSeek-V4-Flash-0731-Latent-Reasoning?ref=runtimewire) around August 7, 2026. He describes the project as personal research rather than an InterSystems product. Public materials do not identify a separate company, outside funding, customers or production deployments tied to the release. The project applies Compressed Latent Reasoning, or CoLaR, to a 284 billion-parameter mixture-of-experts model with 13 billion active parameters. CoLaR was introduced by Wenhui Tan and five co-authors in the 2025 paper ["Think Silently, Think Fast"](https://arxiv.org/abs/2505.16552?ref=runtimewire). The method represents intermediate reasoning as continuous states rather than a sequence of generated reasoning tokens. ### A specialized serving path Mitchko's earlier [CoLaR release](https://huggingface.co/nmitchko/deepseek-v4-Flash-CoLaR?ref=runtimewire) distributed the reasoning component as an adapter that users had to attach to a separate DeepSeek-V4-Flash checkpoint. Its earlier setup required users to assemble the base model, reasoning head, stopping mechanism and custom runtime separately. The new release places the weights needed for Mitchko's configuration in one model repository, while its serving path remains divided between the [ds4-reasoning-addon](https://github.com/nickmitchko/ds4-reasoning-addon?ref=runtimewire) and a [separate vLLM fork](https://github.com/nickmitchko/vllm-ds4-sm120?ref=runtimewire). That division sets a clear boundary around the release. An inference team can download the model and supporting code, but cannot move it unchanged onto an ordinary vLLM deployment. Operators must run Mitchko's addon and maintain his fork of the inference engine. The latent-reasoning component has 35.7 million parameters. It reads the backbone's layer-35 hidden state, projects that state into a 1,024-dimensional representation and writes the decoded result into the residual stream at layer 42. The checkpoint records a compression factor of six, and a learned stop head determines when the latent loop ends. Visible token generation begins after that internal loop. The package also includes a three-layer DSpark draft block for speculative decoding. Its precision settings vary by component: the routed mixture-of-experts weights use NVFP4 with group size 16, while attention modules, shared experts, the language-model head and draft block remain at higher precision. Mitchko estimates that the checkpoint occupies about 158 to 164 GiB across two GPUs. ### One benchmark, two aggregate figures Mitchko reports a BIG-Bench Hard evaluation spanning 27 subtasks, with 50 examples per task and 1,350 examples overall. The [model card](https://huggingface.co/nmitchko/DeepSeek-V4-Flash-0731-Latent-Reasoning?ref=runtimewire) reports an aggregate score of 0.880 +/- 0.008 using the flexible exact-match extractor in lm-evaluation-harness 0.4.12. His accompanying [release post](https://blog.n.ichol.ai/packaging-latent-reasoning-as-a-real-model?ref=runtimewire) also shows 0.94 +/- 0.008 near the start of the benchmark section, then identifies 0.880 as the reliable aggregate. The conflicting numbers prevent a clean comparison with other systems. The lower figure matches the model card and the post's later qualification, but both remain self-reported. The per-task results vary widely. Mitchko reports perfect scores on shuffled-object tracking, Boolean expressions, formal fallacies and the benchmark's penguins-in-a-table task. The score drops to 0.58 on disambiguation questions and 0.26 on Dyck languages, a bracket-matching test. He characterizes the Dyck result as a genuine weakness on mechanical, syntax-heavy reasoning. Answer extraction materially affects the evaluation. BIG-Bench Hard's [strict matcher](https://huggingface.co/nmitchko/DeepSeek-V4-Flash-0731-Latent-Reasoning?ref=runtimewire) looks for the literal phrase "The answer is X." Mitchko's model does not use that output format, so he reports results from the benchmark's flexible extractor instead. That method may be appropriate for the model's output, but it must remain fixed in any attempted comparison or replication. The evaluation covers only 50 examples per subtask, and Mitchko estimates uncertainty of roughly +/- 0.05 to 0.07 for individual tasks. The release provides no independently replicated results. The model card also does not publish evaluations for coding, long-context work, tool use or production workloads. The available evidence shows that the serving configuration runs and can be scored on one reasoning benchmark; it does not establish a general advantage over systems that generate explicit reasoning tokens. ### Hardware limits the first users Mitchko's implementation targets Blackwell-class Nvidia hardware. His recommended configuration has at least 192 GiB of total VRAM, and the setup he verified uses two 96 GiB RTX PRO 6000 Blackwell GPUs. That requirement narrows the likely audience to inference teams with recent multi-GPU servers. Public weights reduce the barrier to inspecting and testing the project, while the memory footprint and forked serving engine make it unsuitable for most individual developers or standard hosted deployments. Latent reasoning also changes what operators can inspect. Generated reasoning text can be incomplete or misleading, but it gives developers an artifact to review during debugging and evaluation. Mitchko's system instead carries intermediate computation through a 1,024-dimensional state. Operators can inspect the final answer and benchmark outcomes, but the reasoning path is less accessible to conventional logging and text-based analysis. The learned stop head introduces another operational dependency because it controls how long the model remains in its internal loop. Failures during that phase may surface only through latency, resource use or a poor final answer. Testing the stop behavior across prompts and workloads will therefore matter as much as reproducing the headline benchmark. Mitchko has supplied the weights and serving code needed for experienced teams to test that proposition on their own hardware. Independent runs will determine whether the approach holds up beyond his Blackwell configuration and BIG-Bench Hard evaluation.