{"slug": "foundational-ternary-model-inference-and-training-cuda-cpu-bitnet-tq", "title": "Foundational ternary-model inference and training – CUDA, CPU, BitNet/TQ", "summary": "Quitetall released Tritium 1.1.0-rc.0, an Apache-2.0 infrastructure for converting dense neural networks into compact additive-ternary models with weights in {-1, 0, +1}, supporting training, inference, and auditable artifact lifecycles. The release candidate is not yet a public v1.1, with gates open for package, browser, flagship-model, deployment, and second-machine validation; the compatibility matrix is authoritative, and no public registry artifact is claimed before activation. Tritium supports additive PTQ with up to three ternary planes, differentiable PyTorch QAT, packed inference kernels for CPU, CUDA, Metal, ROCm, wgpu, WASI/WASM, browser WebGPU, C, Candle, Burn, and ONNX, plus OpenAI-compatible serving and receipt-backed release tooling.", "body_md": "Tritium is Apache-2.0 infrastructure for converting dense neural networks into compact additive-ternary models, refining or training them with PyTorch, and executing their packed weights across native and portable runtimes.\n\nWeights use `{-1, 0, +1}`\n\nplanes. One plane is the compact base model; additional\nresidual planes trade physical bytes for quality. Tritium treats the recipe,\ncalibration evidence, ancestry, packed bytes, runtime memory, and measured model\nquality as one auditable artifact lifecycle—not as an informal `bits=2`\n\nlabel.\n\nRelease status:the repository uses the`1.1.0-rc.0`\n\ncandidate version. It is neither`LOCAL_RC_READY`\n\nnor a public v1.1 release. Package, browser, flagship-model, deployment, and second-machine gates remain open. The generated[compatibility matrix]is authoritative:`pending`\n\nis not support, and no public registry artifact is claimed before activation.\n\n| Tier | Role | Frozen model | Admission rule |\n|---|---|---|---|\n`accessible` |\n`tutorial` |\n`HuggingFaceTB/SmolLM2-135M-Instruct` |\ncandidate artifact + admitted receipt ancestry |\n`accessible` |\n`recipe` |\n`HuggingFaceTB/SmolLM2-1.7B` |\ncandidate artifact + admitted receipt ancestry |\n`native-reference` |\n`native` |\n`microsoft/bitnet-b1.58-2B-4T` |\ncandidate artifact + admitted receipt ancestry |\n`flagship` |\n`language+mtp` |\n`Qwen/Qwen3.6-27B` |\ncandidate artifact + admitted receipt ancestry |\n\nThis table is an admission inventory, not a support or SOTA claim. A row becomes releasable only when its exact candidate artifact, immutable source and tokenizer identity, model card, and required evidence ancestry validate.\n\n- Additive PTQ with one to three ternary planes, calibration receipts, strict physical-byte accounting, packed artifacts, scale-only refinement, and hard-PV refinement.\n- Differentiable PyTorch QAT with STE estimators, tied-weight preservation, hard conversion, optimizer-compatible latent masters, strict checkpointing, Hugging Face integration, and ternary diagnostics.\n- Packed inference kernels and runtime adapters for CPU, CUDA, Metal, ROCm, wgpu, WASI/WASM, browser WebGPU, C, Candle, Burn, and ONNX at different qualification levels.\n- Native SALT formats plus compact B3 packing, sparse/additive layouts, seek-backed payloads, canonical manifests, digests, and bounded readers.\n- OpenAI-compatible serving with bounded requests, rotating bearer authentication for exposed binds, per-principal admission, readiness, metrics, hardened OCI/Kubernetes assets, and evidence-gated deployment.\n- Receipt-backed release tooling for wheels, crates, npm archives, OCI images, SBOMs, provenance, compatibility, model evidence, and local-RC sign-off.\n\nImplementation is not the same as release qualification. See\nthe release ADR and execution plan in the\n[research repository](https://github.com/Quitetall/tritium-research) for the gates (see [docs/RESEARCH.md](/Quitetall/tritium/blob/main/docs/RESEARCH.md)).\nThe [backend guide](/Quitetall/tritium/blob/main/docs/book/src/backends.md) describes source capabilities;\nan implementation without a generated compatibility cell and admitted receipt\nremains unqualified.\n\nThe public API exposes preparation, calibration, conversion, refinement, export, load, and inspection as distinct typed phases:\n\n``` python\nfrom pathlib import Path\n\nimport torch\nfrom tritium.torch import (\n    TernaryConfig,\n    calibrate,\n    convert,\n    inspect,\n    load_quantized_module,\n    prepare,\n)\n\ndense = torch.nn.Linear(128, 64).eval()\nrecipe = TernaryConfig.ptq(\n    profile=\"compact-v1\",\n    target_modules=(\"Linear\",),\n)\nprepared = prepare(dense, recipe, inplace=False)\ncalibration = calibrate(\n    prepared,\n    [torch.randn(4, 128) for _ in range(4)],\n    evidence_dir=Path(\"work/calibration\"),\n)\nresult = convert(prepared, calibration, work_dir=Path(\"work/conversion\"))\nternary = load_quantized_module(dense, result, inplace=False).eval()\n\nprint(result.artifact_id)\nprint(inspect(prepared))\nprint(ternary(torch.randn(2, 128)).shape)\n```\n\nFor QAT, Hugging Face, refinement, ONNX, and browser workflows, use the\n[book](/Quitetall/tritium/blob/main/docs/book/src/SUMMARY.md). The pinned SmolLM2 tutorial is generated and\nheadless-testable, but its clean candidate-wheel/Colab receipt is still a release\ngate rather than a published quick-start claim.\n\nThe default workspace build is CPU-only and uses Cargo—no CMake:\n\n```\ncargo build --locked\ncargo test --locked -p tritium-core -p tritium-format -p tritium-cpu\ncargo run --locked -p tritium-cli -- list-backends\n```\n\nCUDA and other accelerator adapters are feature-gated because their toolchains\nand physical device receipts are target-specific. Portable targets use their\nown target-build and package gates. The source-level CLI and server tutorial is\nin the [quickstart](/Quitetall/tritium/blob/main/docs/book/src/quickstart.md). Release users should install\nonly exact artifacts admitted by the candidate manifest once those artifacts\nare authorized and published.\n\nDependencies point inward:\n\n```\nfoundation  (core · format · spec · testkit)\n   ↑\nbackends    (cpu · cuda · metal · rocm · wgpu · wasm · mcu)\n   ↑\nruntime     (runtime · quantize · salt · nn · train)\n   ↑\nfrontends   (ffi · python · onnx · candle · burn · cli · serve · web)\n```\n\nA frontend does not own a concrete backend, and one backend does not depend on another. Shared schemas and conformance vectors define semantic portability; physical device claims additionally require target receipts.\n\nThe v1.0 codebase has real BitNet b1.58 2B4T CPU/CUDA evidence recorded in the\n[changelog](/Quitetall/tritium/blob/main/CHANGELOG.md). The v1.1 flagship target is a separately labeled\nQwen3.6-27B language-plus-MTP additive-PTQ/refinement campaign. It is not a\ncompleted SOTA claim until its frozen quality, retention, runtime, memory, and\nreproduction gates pass.\n\nThe `scripts/release-status`\n\nvalidator consumes an explicit candidate manifest\nand evidence registry and refuses incomplete gates. This working tree does not\nyet contain a sign-off candidate/registry. Current work-order blockers include:\n\n- whole-model Qwen/MTP quality and performance evidence is not yet admitted;\n- public wheel/crate/npm/container artifacts are not activated;\n- browser support awaits physical Chrome, Firefox, and Safari receipts;\n- whole-model Qwen/MTP inference and clean-wheel ONNX Runtime generation receipts remain open;\n- trainable whole-model ONNX is intentionally deferred to v1.3;\n- vision/multimodal tensors are identity-bound but deferred from v1.1;\n- public links and community channels remain inactive while the repository is private, and an independent conduct adjudicator must be designated first.\n\nTritium is licensed under [Apache-2.0](/Quitetall/tritium/blob/main/LICENSE). Vendored upstream work is\nattributed in [NOTICE](/Quitetall/tritium/blob/main/NOTICE).", "url": "https://wpnews.pro/news/foundational-ternary-model-inference-and-training-cuda-cpu-bitnet-tq", "canonical_source": "https://github.com/Quitetall/tritium", "published_at": "2026-08-02 03:40:39+00:00", "updated_at": "2026-08-02 03:52:17.225930+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "neural-networks", "ai-infrastructure", "developer-tools"], "entities": ["Quitetall", "Tritium", "PyTorch", "HuggingFaceTB/SmolLM2-135M-Instruct", "HuggingFaceTB/SmolLM2-1.7B", "microsoft/bitnet-b1.58-2B-4T", "Qwen/Qwen3.6-27B", "Apache-2.0"], "alternates": {"html": "https://wpnews.pro/news/foundational-ternary-model-inference-and-training-cuda-cpu-bitnet-tq", "markdown": "https://wpnews.pro/news/foundational-ternary-model-inference-and-training-cuda-cpu-bitnet-tq.md", "text": "https://wpnews.pro/news/foundational-ternary-model-inference-and-training-cuda-cpu-bitnet-tq.txt", "jsonld": "https://wpnews.pro/news/foundational-ternary-model-inference-and-training-cuda-cpu-bitnet-tq.jsonld"}}