DeepSeek open-sources DeepInfer, its engine for 155T tokens a day

Production inference stacks have mostly stayed closed at Fireworks, Baseten and Modal. DeepInfer ships under Apache 2.0 with batch-invariant inference and deterministic sampling, though the KV cache and disaggregation layers behind DeepSeek's service stay private. Read: Production inference stacks have mostly stayed closed at Fireworks, Baseten and Modal. DeepInfer ships under Apache 2.0 with batch-invariant inference and deterministic sampling, though the KV cache and disaggregation layers behind DeepSeek's service stay private. Read: In OpenSSH 10.4, about 4,500 lines of ML-DSA-44 code were reachable by unauthenticated clients even when the algorithm was disabled, roughly 16% of default pre-auth code. Gating that code lowers the severity of bugs nobody has found yet, without fixing each one. Read: The same open-weight model scored 44.9% to 53.9% in three other open-source harnesses, two of them on high reasoning while Cline ran with thinking off. Harness choice moved the score by up to 17 points, though Cline has not isolated which feature caused the gap. Read: One model replaces the speech recognition, turn-taking, LLM and TTS pipeline teams usually tune by hand. The API follows OpenAI's Realtime spec, so an existing integration switches by changing the endpoint and model name, with no rewrite. Read: On an M3 Ultra, Qwen3.6-35B-A3B went from 9.4 to 61.3 tokens/s, a gain from software alone on hardware people already own. FP8 storage also halves weight memory against BF16, so larger models fit in the same unified memory. Read: Anthropic rated catastrophic misalignment risk low as of July 15. METR agrees for current models but doubts the report's methods would catch a riskier one, and says redacted incident examples showed Anthropic's monitoring was weaker than public evidence suggested.