CMF: new, memory-mappable format for quantized LLMs

A developer releasing under the name Cortiq published CMF (Cortiq Model Format), a single-file, memory-mappable container for quantized LLMs with a Rust runtime, and 53 converted models tagged `cortiq` are hosted on the Hugging Face Hub under the account infosave (Oleg Kirichenko). The format selects quantization per tensor — f16, q8 and q8_2f for sensitive tensors, q4tp and q2tp for large matrices — and ships with a `cortiq verify` command that checks every section's bounds and every tensor hash, plus `cortiq convert`, `cortiq import-gguf` for existing GGUF files, and `cortiq serve` for an OpenAI-compatible API. The runtime runs on CPU and on GPU through Vulkan/DX12 (wgpu) or Metal, and the author is soliciting feedback on the header and tensor directory from people who have written safetensors or GGUF loaders.

CMF: a single-file, memory-mappable format for quantized LLMs Hi all, I’ve been building CMF Cortiq Model Format single-file container for quantized models with a Rust runtime. Most of the converted models live on the Hub, so I wanted to share it here. What one .cmf file holds Quantization is chosen per tensor: f16 / q8 / q8 2f for sensitive tensors, q4tp / q2tp for the large matrices. cortiq verify checks every section’s bounds and every tensor hash, and there is a small independent Try it cargo install cortiq-cli cortiq convert --model Qwen/Qwen3-0.6B --quant q8 --output qwen.cmf cortiq run qwen.cmf --prompt "What is the capital of France?" --greedy --no-think Already have a GGUF: cortiq import-gguf model.gguf --output model.cmf . cortiq serve exposes an OpenAI-compatible API. It runs on CPU, and on GPU through Vulkan / DX12 wgpu or Metal. On the Hub 53 converted models tagged cortiq : infosave Oleg Kirichenko https://huggingface.co/infosave I’d welcome feedback on the format itself, especially from people who have written safetensors or GGUF loaders: what would you want in the header or tensor directory that isn’t there?