{"slug": "cmf-new-memory-mappable-format-for-quantized-llms", "title": "CMF: new, memory-mappable format for quantized LLMs", "summary": "A developer releasing under the name Cortiq published CMF (Cortiq Model Format), a single-file, memory-mappable container for quantized LLMs with a Rust runtime, and 53 converted models tagged `cortiq` are hosted on the Hugging Face Hub under the account infosave (Oleg Kirichenko). The format selects quantization per tensor — f16, q8 and q8_2f for sensitive tensors, q4tp and q2tp for large matrices — and ships with a `cortiq verify` command that checks every section's bounds and every tensor hash, plus `cortiq convert`, `cortiq import-gguf` for existing GGUF files, and `cortiq serve` for an OpenAI-compatible API. The runtime runs on CPU and on GPU through Vulkan/DX12 (wgpu) or Metal, and the author is soliciting feedback on the header and tensor directory from people who have written safetensors or GGUF loaders.", "body_md": "**CMF: a single-file, memory-mappable format for quantized LLMs**\n\nHi all,\n\nI’ve been building **CMF (Cortiq Model Format)** single-file container for quantized models with a Rust runtime. Most of the converted models live on the Hub, so I wanted to share it here.\n\n**What one `.cmf` file holds**\n\nQuantization is chosen per tensor: f16 / q8 / q8_2f for sensitive tensors, q4tp / q2tp for the large matrices. `cortiq verify` checks every section’s bounds and every tensor hash, and there is a small independent\n\n**Try it**\n\n```\ncargo install cortiq-cli\ncortiq convert --model Qwen/Qwen3-0.6B --quant q8 --output qwen.cmf\ncortiq run qwen.cmf --prompt \"What is the capital of France?\" --greedy --no-think\n```\n\nAlready have a GGUF: `cortiq import-gguf model.gguf --output model.cmf`. `cortiq serve` exposes an OpenAI-compatible API. It runs on CPU, and on GPU through Vulkan / DX12 (wgpu) or Metal.\n\n**On the Hub**\n\n53 converted models tagged `cortiq`: [infosave (Oleg Kirichenko)](https://huggingface.co/infosave)\n\nI’d welcome feedback on the format itself, especially from people who have written safetensors or GGUF loaders: what would you want in the header or tensor directory that isn’t there?", "url": "https://wpnews.pro/news/cmf-new-memory-mappable-format-for-quantized-llms", "canonical_source": "https://discuss.huggingface.co/t/cmf-new-memory-mappable-format-for-quantized-llms/191095#post_1", "published_at": "2026-10-27 17:46:08+00:00", "updated_at": "2026-10-27 18:14:46.614339+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "developer-tools", "ai-tools"], "entities": ["Cortiq", "CMF (Cortiq Model Format)", "cortiq-cli", "Hugging Face Hub", "Oleg Kirichenko", "infosave", "Qwen3-0.6B", "GGUF"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/cmf-new-memory-mappable-format-for-quantized-llms", "markdown": "https://wpnews.pro/news/cmf-new-memory-mappable-format-for-quantized-llms.md", "text": "https://wpnews.pro/news/cmf-new-memory-mappable-format-for-quantized-llms.txt", "jsonld": "https://wpnews.pro/news/cmf-new-memory-mappable-format-for-quantized-llms.jsonld"}}