{"slug": "glm-5-3-flash-on-apple-silicon", "title": "GLM-5.3-Flash on Apple Silicon", "summary": "WARP, an embeddable inference engine written in C, now runs the full 2.78-trillion-parameter Kimi K3 model on a 64 GB MacBook Pro at about 0.6 tokens per second, and the 313-billion-parameter GLM-5.3-Flash at about 3.9 tokens per second, according to the project's documentation. The engine keeps the model trunk in memory, streams selected experts from disk, and uses a bounded expert cache, achieving these speeds without distillation or pruning. The project aims to run huge frontier models on consumer hardware, with the ultimate goal of executing Kimi K3 locally to improve itself.", "body_md": "WARP is an embeddable inference engine written in C, with no third-party runtime dependencies. It keeps the model trunk in memory, streams selected experts directly from disk, and uses the remaining RAM as a bounded expert cache.\n\nThe project is driven by humans: the ideas, hypotheses, priorities, tests, and decisions are human. The code is written by LLMs. At this scale, that is the only way to iterate on new algorithms and test hypotheses fast enough.\n\nThe goal is to run huge frontier models such as Kimi K3 on consumer hardware. Today, the complete 2.78-trillion-parameter Kimi K3 runs on a 64 GB MacBook Pro at about **0.6 tokens per second**, and the 313-billion-parameter GLM-5.3-Flash — text and images — at about **3.9**.\n\n**Ultimately we want WARP to execute Kimi K3 locally to improve itself** (we are currently using Opus 5 with extra thinking).\n\nWARP is intentionally narrow, and it exists to find out how far local inference can be pushed when model weights live mostly on fast storage instead of RAM.\n\n``` bash\n$ waste run ~/models/k3.waste 'What is the capital of Italy?'\nwaste: no --budget, using 46.39 GB of 64.00 GB (expert cache 17.56 GB)\nThe capital of Italy is **Rome**.\n[16 tokens, 26.87 s, 0.60 tok/s | experts 9000 hit / 14552 miss = 38%]\n```\n\n**This is the full model, not a distilled or pruned version.** Its published weights occupy 1.42 TB; the converted WARP container is 982 GB.\n\nKimi K3 is a mixture-of-experts model. It has 2.78 trillion parameters, but only about 4% of them are active for each token. WARP keeps the shared part of the model in RAM and reads only the selected experts from disk.\n\nThe container is arranged so that one expert requires one aligned read. Those reads overlap with computation, while unused RAM becomes a bounded expert cache. A lookahead router predicts the experts needed by the next layer and starts reading them early; the real router still makes the decision, so this changes timing, not the result. Experts use 3-bit residual vector quantization, while the more sensitive shared weights remain at 4 or 8 bits.\n\nK3's linear attention and compressed latent KV cache also matter: at 4K context, the KV cache is about 0.21 GB instead of 11.25 GB. The result is an engine that needs 29.19 GB to open K3 and uses the rest of the available memory to avoid repeated disk reads.\n\nFor the full design and measurements, see [docs/ENGINE.md](/sqliteai/warp/blob/main/docs/ENGINE.md) and [docs/EFFICIENCY.md](/sqliteai/warp/blob/main/docs/EFFICIENCY.md). The on-disk layout is documented in [docs/FORMAT.md](/sqliteai/warp/blob/main/docs/FORMAT.md), while [docs/KDA.md](/sqliteai/warp/blob/main/docs/KDA.md) describes Kimi Delta Attention.\n\nMeasured on a 64 GB MacBook Pro with an M5 Pro and the model container on the internal SSD:\n\n| Model | Container | Minimum RAM | 64 tokens | 200 tokens |\n|---|---|---|---|---|\n| Kimi K3 2.78T | 982 GB | 29.19 GB | 0.45–0.62 tok/s | — |\n| GLM-5.3-Flash 313B | 112 GB | 5.14 GB | 3.32 tok/s | 3.86 tok/s |\n| Kimi-Linear 48B | 19 GB | 1.32 GB | 14.29 tok/s | 17.22 tok/s |\n\nThe longer run is faster because the expert cache is still filling during the first few dozen tokens; both columns are what the same command prints, not a steady state extrapolated from it. K3 has no 200-token column here because one run of it takes ten minutes and reads 4.6 TB.\n\nFor K3, 64 GB is the practical minimum. A 32 GB machine can open the model but will page heavily. The default memory budget on the test machine is 46.39 GB, including a 17.56 GB expert cache.\n\nKimi-Linear's figure is the one that moved: the automatic budget used to stop three working sets short of the machine, so a 19 GB container got a 1.65 GB cache on a 64 GB laptop. It now climbs to the container's whole expert set when the machine has the room — 18.48 GB resolved, every expert resident — and that is worth 11.13 → 12.60 tok/s over 64 tokens, with the bytes read falling from 66.3 GB to 17.7. On top of it the thread pool stopped waking its efficiency cores for jobs too small to hide the ~54 µs that costs, which is another 14.41 → 16.74 over 150 tokens. K3 is unchanged by both: its 962.83 GB of experts do not fit on any machine here, and at 465 GB read per 20 tokens neither residency nor dispatch is where its time goes. [docs/LEARNED.md](/sqliteai/warp/blob/main/docs/LEARNED.md) §66, §67.\n\nMost of that requirement is the 27.28 GB resident trunk rather than the cache. Shrinking the expert cache from 17.32 GB to 3.32 GB costs about 10% of throughput; enlarging it past the default costs everything. Measured across four cache sizes in one process:\n\n| expert cache | hit rate | decode |\n|---|---|---|\n| 3.32 GB | 29.1% | 0.56–0.58 tok/s |\n| 17.32 GB | 36.2% | 0.63 tok/s |\n| 23.32 GB | 38.4% | 0.07–0.09 tok/s |\n| 29.32 GB | 41.3% | 0.07–0.08 tok/s |\n\nThe last two rows are the failure mode worth knowing about: the hit rate keeps climbing and the bytes read keep falling while throughput drops eightfold. The engine is inside its budget and the machine is not, so a cache hit becomes a page fault. Giving the process more memory is not always faster.\n\nDecoding with fewer experts per token is a knob rather than a rebuild:\n`num_experts_per_token`\n\nin the container manifest. K3 ships at 16. Measured\non this machine, one load with the arms interleaved:\n\n| experts/token | decode | KL from top-16 | working set |\n|---|---|---|---|\n| 16 | 0.59 tok/s | — | 17.01 GiB |\n| 12 | 0.70 tok/s | 0.007 | 12.76 GiB |\n8 |\n0.89 tok/s |\n0.037 |\n8.50 GiB |\n| 4 | 1.06 tok/s | 0.118 | 4.25 GiB |\n\nTop-8 is 1.49x for a divergence twice that of a quantization this project rejects elsewhere, and it reproduces top-16's greedy continuation on the prompts tested. Top-4 does not: its next-token distribution still looks close, and it stops following the prompt within a few tokens — which is why the gate here is a continuation and not a KL. This is a quality trade and the default stays 16.\n\nStorage is the main constraint. A cold K3 token reads about 17 GB of experts. The internal SSD sustains 12.78 GB/s; a tested USB enclosure managed 0.94 GB/s. Put the converted container on internal NVMe storage.\n\nIf you have more than one drive, since 0.7.2 the expert banks can be spread\nacross them: `WASTE_BANK_SHARDS=/mnt/a,/mnt/b`\n\nreads expert `e`\n\nfrom shard\n`e % N`\n\n, so the k experts a single token routes to land on different\ndevices instead of queueing behind one. `tools/split_banks.py`\n\nwrites and\nbyte-verifies the shard sets, and the logits are identical either way.\n**No speedup is claimed here** — that needs two drives of comparable speed\nand a real container. Striping across the internal SSD and the USB\nenclosure above would measure the enclosure, not the striping. The\nmechanism ships; the measurement does not.\n\nAll layers are checked against a PyTorch reference. Final logits agree within 3.6e-06, and the vision tower agrees with its oracle within 2.3e-06.\n\nAdditional measurements, profiling data, router-lookahead results, and quantization experiments are collected in [docs/TECHNICAL.md](/sqliteai/warp/blob/main/docs/TECHNICAL.md).\n\nKimi K3 and GLM-5.3-Flash are both multimodal, and WARP can use one or more images together with text. Pass `--image`\n\nonce per image:\n\n```\n./waste run ~/models/k3.waste \"Describe this image\" --image photo.jpg\n./waste run ~/models/k3.waste \"Compare these images\" \\\n    --image before.png --image after.png\n```\n\nIn interactive mode, `/image FILE`\n\nattaches an image to the next message. An image is expanded into many prompt positions: an 896×896 image uses 256 positions at the default patch budget. The vision tower takes about 15.7 seconds for 1024 patches on the test machine, but most of the cost comes afterward because every image position passes through the language model like a text position. In the current K3 measurements, that is about 2.8 seconds per image position.\n\nGLM's tower is a different one and cheaper to feed: the 200×140 picture in\nthe [GLM section](#glm-53-flash) costs 40 prompt positions, and generation\nafter it runs at the same speed as without it. Its tower is 282 MB against\nK3's 434 MB, and both are loaded only when images are asked for.\n\nSee [docs/K3.md](/sqliteai/warp/blob/main/docs/K3.md) and [docs/GLM.md](/sqliteai/warp/blob/main/docs/GLM.md) for the two vision architectures and their measurements, and [examples/README.md](/sqliteai/warp/blob/main/examples/README.md) for CLI, C, and HTTP multimodal examples.\n\nK3 is the target and the best-tested model, and Kimi-Linear is the small one to\nstart with. Since 0.6.8 the converter and the engine also handle the\n**DeepSeek-V3 family — V3, R1 and Kimi K2**, which needed two changes rather\nthan one.\n\n`convert.py`\n\nnow reads fp8 block-scaled weights, applying the per-tile scales\nthese checkpoints ship in a companion tensor, and normalises DeepSeek's MoE\ntensor and config names to the single spelling the engine reads. And MLA now\napplies rotary to its rope dims. The engine had implemented none: the Kimi\nmodels set `mla_use_nope`\n\nand pass those dims through unrotated, which is\ncorrect for them and wrong for everything in the V3 family, where in MLA those\ndims are the only positional signal there is. A container built before this was\nnot degraded, it was unordered — it could not tell which turn of a conversation\ncame first.\n\nNo throughput figures here, because nobody on this project has a K2 container.\nWhat has been measured, by [@fab2s](https://github.com/fab2s) who contributed\nboth changes: a `Kimi-K2-Instruct`\n\nconversion — 61 layers, 384 experts top-8,\nVQ3R, a 354 GB expert set and a 6.9 GB trunk — opens and reports 1.03 T\nparameters total, 31.69 B active per token; and the rotary arithmetic agrees to\n0.000023% relative L2 with an oracle whose YaRN helpers are taken verbatim from\nthe DeepSeek release's own `modeling_deepseek.py`\n\n.\n\nKimi K3 and Kimi-Linear are unaffected: their forward pass is byte-identical to 0.6.7, by construction rather than by a runtime branch.\n\n`zai-org/GLM-5.3-Flash`\n\n— 313 B parameters, 328 GB of fp8 as published — is\nconverted and running, text and images.\n\n``` bash\n$ waste run ~/models/glm53.waste \"What is the capital of Italy? Answer in one sentence.\"\nwaste: no --budget, using 46.37 GB of 64.00 GB (expert cache 41.36 GB)\nThe user is asking a simple factual question: What is the capital of Italy?\nThey want the answer in one sentence.\n\nThe capital of Italy is Rome. This is a well-established fact. I should\nanswer in one sentence as requested.</think>The capital of Italy is Rome.\n[56 tokens, 12.79 s, 4.38 tok/s | experts 17327 hit / 1489 miss = 92%]\n```\n\nEverything before `</think>`\n\nis the model's reasoning. GLM's generation\nprompt always opens that channel and the model closes it itself; the CLI\nprints both, and over HTTP they come back as `reasoning_content`\n\nand\n`content`\n\nseparately.\n\n| parameters | 313.89 B total, 17.31 B active per token |\n| container | 112 GB — 5301 MB trunk, 42 expert banks of 2598 MB |\n| minimum RAM | 5.14 GB, plus 805 MB when images are enabled |\n| default budget here | 46.37 GB, of which 41.36 GB expert cache |\n| decode | 3.32 tok/s over 64 tokens, 3.86 over 200 |\n\nAgainst a PyTorch oracle built from the same container: relative L2\n**2.41e-5**, argmax and top-10 identical.\n\n**It is the model that fits this class of machine.** K3 needs 29.19 GB\nbefore it caches a single expert and then gets a token's working set and a\nhalf; GLM's floor is 5.14 GB, so a 64 GB laptop caches 36% of its entire\nexpert set — and a much smaller machine still runs it. Measured over 64\ntokens, varying only `--budget`\n\n:\n\n| budget | expert cache | hit rate | read | decode |\n|---|---|---|---|---|\n| 9 GB | 4.0 GB | 66.0% | 228 GB | 2.82 tok/s |\n| 12 GB — what a 16 GB machine resolves | 7.0 GB | 70.2% | 190 GB | 2.99 tok/s |\n| 16 GB | 11.0 GB | 74.0% | 160 GB | 3.06 tok/s |\n| 24 GB | 19.0 GB | 79.4% | 121 GB | 3.14 tok/s |\n| 46 GB — the default here | 41.4 GB | 87.7% | 73 GB | 3.32 tok/s |\n\nThe curve is shallow because the reads overlap the arithmetic: six times the\ncache cuts the disk traffic by two thirds and buys 18% of throughput. What\nthat means in practice is that **a 16 GB machine runs a 313 B model at 90%\nof the speed a 64 GB one does** — the engine leaves a quarter of RAM to the\nOS, so it resolves about 12 GB there — and more RAM mostly buys quiet disks.\nUntil the disk is slow: on the 0.94 GB/s enclosure this project has\nmeasured, 228 GB against 73 is four minutes of reading against one.\n\nIt turned out to be mostly this engine already: the same KDA recurrence, the\nsame MLA with `kv_b_proj`\n\nabsorbed, the same router, the same fp8 reader and\nthe same nested config layout as K3. Three things are new and each is behind\na config key that is absent everywhere else: **mHC**, which carries four\nparallel residual streams instead of one and mixes them through a\nSinkhorn-projected matrix at every sublayer; a **clamped SwiGLU**; and\n**DeepSeek Sparse Attention** in its k-pool form, where a full-attention\nlayer scores pools of four cached tokens and attends over the best 512 of\nthem plus the tail.\n\nThe re-encoded tokenizer agrees with the release's own on 21 of 21 strings, and VQ3R lands at the same 0.195 relative error on GLM's experts as on K3's.\n\nIts vision tower is a second one — 24 blocks with 2D rope, per-head q/k norms, a gated merger — and matches its own PyTorch oracle to rel L2 3.3e-5:\n\n``` bash\n$ waste run ~/models/glm53.waste \"What does this image look like? One sentence.\" \\\n      --image x.png -n 200\n[x.png: 40 image tokens]\nThe image is a colorful, abstract pattern. It consists of diagonal stripes\nof various colors (green, blue, purple, pink, yellow, red) with some\nvertical lines within them.</think>This image displays a vibrant, abstract\npattern of diagonal stripes in various colors like green, blue, purple, and\npink, overlaid with fine vertical lines.\n[104 tokens, 24.98 s, 4.16 tok/s | experts 32067 hit / 2877 miss = 92%]\n```\n\n`x.png`\n\nis a 200×140 test pattern generated from\n`(x*7+y*3, x*x+y, x+y*11) mod 256`\n\n, which really is diagonal colour bands\nwith vertical structure — the description is of the file, not of a\nplausible-sounding picture.\n\nAn image costs the context what text of the same length costs: 40 merged tokens for that 200×140 picture, and the generation that follows runs at the same speed as any other. The tower itself is 282 MB and is loaded only when images are asked for.\n\n[docs/GLM.md](/sqliteai/warp/blob/main/docs/GLM.md) has the architecture, the three places the\nrelease states something differently, and what is still left out.\n\nTo build and test WARP:\n\n- a C11 compiler and\n`make`\n\n; - macOS, Linux (arm64 and x86_64), or Windows via MinGW-w64 — CI builds and runs the suite on all four, plus an ASan/UBSan job;\n- no BLAS, Python, CUDA, or other external dependency for the current CPU inference path.\n\nTo run GLM-5.3-Flash, which is the one most machines can hold:\n\n**16 GB of RAM is enough**; 5.14 GB is the hard floor at 4K context, and 64 GB is what the measurements above were taken on;** 112 GB of internal NVMe storage**for the converted container;- another\n**306 GiB of temporary storage** if converting the published weights yourself. This staging storage may be external and can be freed afterward, or reclaimed as the conversion proceeds.\n\nTo run Kimi K3:\n\n**64 GB of RAM recommended**; 29.19 GB is the hard floor at 4K context;** about 1 TB of internal NVMe storage**for the converted model;- another\n**1.42 TB of temporary storage** if converting the published weights yourself. This staging storage may be external and can be freed afterward.\n\nIf you only want to try the engine, start with Kimi-Linear. Its container is 19 GB, it needs 1.32 GB of RAM, and it runs at about 14.4 tok/s on the same machine.\n\nPython, PyTorch, and safetensors are needed only for model conversion and validation, never for inference.\n\nBuild the engine and run the model-free test suite:\n\n```\ngit clone https://github.com/sqliteai/warp\ncd waste\nmake\nmake check\n```\n\n`make`\n\nbuilds the `waste`\n\nCLI and `libwaste.a`\n\n. `make check`\n\ncreates a small synthetic model, so it does not download weights.\n\nThe shortest path to a working 313 B model on a laptop. Every figure below was measured on the machine at the top of this file; the download and the conversion are both resumable and both safe to kill.\n\n**Before you start**, you need room for two things at once: **306 GiB** of\npublished weights on the staging disk, and **112 GB** for the container. The\ncontainer belongs on internal NVMe — a container on an external disk is\ncorrect and slow, and the difference is 12.78 GB/s against 0.94 on a tested\nenclosure. The staging weights can live anywhere.\n\n```\n# 1. Build. Takes under a minute; no weights involved.\ngit clone https://github.com/sqliteai/warp\ncd waste\nmake\n\n# 2. Check the download before starting it: shard count, size, free space.\ntools/fetch_weights.sh --repo zai-org/GLM-5.3-Flash \\\n    --dest /Volumes/staging/glm53 --dry-run\n\n# 3. Download. 62 shards, 306 GiB. About 2 hours here, at a rate that\n#    varied between 36 and 97 MB/s. Re-run it if it stops; nothing\n#    already fetched is fetched twice.\ntools/fetch_weights.sh --repo zai-org/GLM-5.3-Flash \\\n    --dest /Volumes/staging/glm53\n\n# 4. Convert. About 45 minutes with three workers: 42 expert layers at\n#    ~160 s each, then the trunk. Put the output on the internal SSD.\nuv run --with torch python tools/convert.py \\\n    --src /Volumes/staging/glm53 \\\n    --out ~/models/glm53.waste \\\n    --jobs 3\n\n# 5. Run it. Leave room for the reasoning channel: GLM thinks before it\n#    answers, and -n counts both.\n./waste run  ~/models/glm53.waste \"What is the capital of Italy?\" -n 200\n./waste chat ~/models/glm53.waste\n```\n\nThat is all of it. There are no GLM-specific flags: the converter recognises the architecture, writes the chat format and the vision config, and re-encodes the tokenizer, and the engine picks its own memory budget and says what it picked.\n\n```\n$ ./waste chat ~/models/glm53.waste\nwaste: no --budget, using 46.37 GB of 64.00 GB (expert cache 41.36 GB)\nchat format from ~/models/glm53.waste/chat.json\n\n> What is the capital of Italy? Answer in one sentence.\nThe user is asking a simple factual question: What is the capital of Italy?\nThey want the answer in one sentence.\n\nThe capital of Italy is Rome. This is a well-established fact. I should\nanswer in one sentence as requested.</think>The capital of Italy is Rome.\n```\n\nThe text before `</think>`\n\nis the model's reasoning channel, which GLM\nalways opens and closes itself. Over HTTP it comes back as\n`reasoning_content`\n\n, separate from the answer. `-n`\n\ncounts both, so a\nquestion that needs thinking needs a larger budget than the answer alone\nsuggests.\n\n**What to expect on the way.** The first few dozen tokens are slower than\nthe rest — the expert cache is still filling — so a short reply runs at\naround 3.3 tok/s and a long one settles near 3.9. Prefill runs at the same\nspeed as decode, so a 2000-token prompt takes minutes before the first\noutput token; that is a property of the engine and not of this model.\n\n**If you have less RAM**, nothing changes about the commands: the engine\nresolves a smaller budget on its own and says so. The table in the\n[GLM-5.3-Flash](#glm-53-flash) section above measures what each budget\nbuys — 16 GB is enough, at 3.06 tok/s against 3.32.\n\n**If you are short of disk**, `--reclaim on`\n\ndeletes each source shard as\nthe converter finishes with it, so the peak is the container plus the\nshards still owed instead of both in full. It is not reversible and\n`tools/verify_container.py`\n\ncan no longer check the result against its\nsource, so prove the recipe with `--reclaim dry`\n\nfirst — [docs/K3.md](/sqliteai/warp/blob/main/docs/K3.md)\nhas the refusals and the ledger discipline.\n\nThe fastest route is to download the already converted container over BitTorrent. It skips the 1.42 TB source download, the 4.7-hour conversion, and the temporary staging storage entirely — only the 982 GB container lands on disk. Any BitTorrent client will do; the torrent's own piece hashes verify the container as it arrives.\n\n```\nmagnet:?xt=urn:btih:54db69b0df8baf5e617744dda5d46c90a2d0f632&dn=k3.waste&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=https%3A%2F%2Ftracker.tamersunion.org%3A443%2Fannounce\n```\n\nWith [aria2](https://aria2.github.io/), which resumes and needs no GUI:\n\n```\naria2c --dir ~/models --seed-time=60 \\\n  'magnet:?xt=urn:btih:54db69b0df8baf5e617744dda5d46c90a2d0f632&dn=k3.waste&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=https%3A%2F%2Ftracker.tamersunion.org%3A443%2Fannounce'\n```\n\nThis writes `~/models/k3.waste`\n\n, the directory the commands below expect. Point\n`--dir`\n\nat internal NVMe storage: a container downloaded onto an external disk\nhas to be copied before it is usable at full speed. Raise `--seed-time`\n\nif you\ncan afford to share it back.\n\nConvert it yourself if you would rather not trust a third-party copy, or if you already hold the original weights. The download and conversion are resumable:\n\n```\n# Check required download space.\ntools/fetch_weights.sh --dest /Volumes/staging/k3 --dry-run\n\n# Download the original weights.\ntools/fetch_weights.sh --dest /Volumes/staging/k3\n\n# Convert them. Put the output on the internal SSD.\nuv run --with torch --with safetensors python tools/convert.py \\\n    --src /Volumes/staging/k3 \\\n    --out ~/models/k3.waste \\\n    --jobs 3\n```\n\nConversion takes about 4.7 hours with three workers on the test machine. See [docs/K3.md](/sqliteai/warp/blob/main/docs/K3.md) for validation, recovery, and storage details.\n\n```\n./waste plan ~/models/k3.waste\n./waste run  ~/models/k3.waste \"The capital of France is\" -n 32\n./waste chat ~/models/k3.waste\n```\n\nDo not set `--budget`\n\nunless you have a reason to. By default WARP chooses a\nsafe memory budget, reports it, and refuses to start below the model's floor.\nInside a container it sizes against the cgroup limit rather than the host's RAM.\nUse `./waste --help`\n\nfor the complete command list.\n\nMore CLI examples, including evaluation, tokenization, saved sessions, and multimodal prompts, are in [examples/README.md](/sqliteai/warp/blob/main/examples/README.md).\n\nThe optional server implements the OpenAI chat-completions API:\n\n```\nmake libwaste.dylib                 # use libwaste.so on Linux\npython3 -m serve ~/models/k3.waste --port 8000\n\ncurl localhost:8000/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"model\":\"k3\",\"messages\":[{\"role\":\"user\",\"content\":\"Why is the sky blue?\"}]}'\n```\n\nIt supports streaming, tools, structured output, thinking controls, and images. See [docs/SERVE.md](/sqliteai/warp/blob/main/docs/SERVE.md) for the protocol and [examples/README.md](/sqliteai/warp/blob/main/examples/README.md) for complete requests.\n\nA GLM container is served the same way, from its own `chat.json`\n\n: plain\nconversation and images, with the reasoning channel returned as\n`reasoning_content`\n\nbeside `content`\n\n. Tools are refused by name rather than\nhalf-rendered — four strings cannot express a tool declaration, and GLM's\ntokenizer carries no protocol that could.\n\nKimi-Linear's does. Since 0.7.2 a container whose tokenizer holds all five\nof Kimi's native tool-call markers gets tool calling over HTTP even though\nits `chat.json`\n\ndescribes only the ordinary turns — the format lives in\n`serve/kimitools.py`\n\n, and the server says which of the three capabilities a\ncontainer has when it starts. All five or none: half of that rendering\nencodes as ordinary text, so a partial set is a different protocol rather\nthan a smaller one.\n\n```\npython3 -m serve ~/models/glm53.waste --port 8000\n```\n\nWARP is also an embeddable C library. The CLI and server both use the public API in [src/waste.h](/sqliteai/warp/blob/main/src/waste.h). The inference path depends only on libc and pthreads. Text generation, memory planning, session persistence, and multimodal C examples are available in [examples/README.md](/sqliteai/warp/blob/main/examples/README.md).\n\nEvery token answered by a cloud service is paid for twice: once on the invoice, and once in the electricity of a datacenter running a model that would fit — barely, awkwardly, but genuinely — on hardware already sitting on a desk. WARP means to be the first concrete step toward ending that waste of tokens — which the project's first name said outright, and which the rename did not change.\n\nThe format and API are not frozen. K3 is the main target and the best-tested model. The CPU path is currently the fastest measured implementation for this workload, but it is not assumed to be the final answer. CUDA, Metal, and other hardware-specific optimizations remain to be explored and may provide significant gains. Current backend results are documented in [docs/BACKENDS.md](/sqliteai/warp/blob/main/docs/BACKENDS.md), while open directions are tracked in [docs/RESEARCH.md](/sqliteai/warp/blob/main/docs/RESEARCH.md). Read [docs/LEARNED.md](/sqliteai/warp/blob/main/docs/LEARNED.md) before proposing an optimization: failed ideas and negative results are kept there deliberately.\n\nContributors are more than welcome. New experiments, support for additional hardware, and open discussion about how to improve performance are all encouraged—even when an idea produces a negative result.\n\nThe software is currently changing very quickly. Before each release, a large QA run is executed; however, instabilities are definitely possible.\n\nMeasurements are treated as experimental results rather than marketing numbers. Each result is tied to the hardware, container, configuration, and commit on which it was obtained; unstable measurements are reported as ranges, and results later found to be wrong remain recorded as such. The detailed snapshots are in [docs/TECHNICAL.md](/sqliteai/warp/blob/main/docs/TECHNICAL.md) and the full history, including negative results, is in [docs/LEARNED.md](/sqliteai/warp/blob/main/docs/LEARNED.md).\n\nValidation covers more than successful generation. The model-free suite builds a synthetic container; real-model checks compare individual layers and final logits against PyTorch, verify conversion round trips, test vision against its oracle, and exercise the server prompt renderer segment by segment against K3's reference encoder. The validation criteria and current evidence are documented in [docs/GATES.md](/sqliteai/warp/blob/main/docs/GATES.md), with server-specific differential tests in [docs/SERVE.md](/sqliteai/warp/blob/main/docs/SERVE.md).\n\nOn a mixture-of-experts model, a distance between logits is not by itself a verdict. A top-K router turns an arbitrarily small arithmetic difference into a discrete one, and past the first flipped expert the two paths are running different weights — the distance then measures how much the model cares which of two indistinguishable experts it used, not how far the arithmetic moved. Since 0.7.1 the suite compares the logits and, where they part, asks which decision moved and whether anything could have resolved it: [tests/route_diff.py](/sqliteai/warp/blob/main/tests/route_diff.py) over the expert ranking and [tests/dsa_diff.py](/sqliteai/warp/blob/main/tests/dsa_diff.py) over the sparse-attention pool ranking, each answering *identical*, *tie*, or *diverged*. Three checks were red against a 1e-3 threshold and none was an engine defect: on K3 the paths first disagree on the closest call in the entire forward pass, a relative margin of 7.3e-07 where the median decision is 7.4e-03, and on GLM they disagree on an exact tie between pools scoring zero. The result is a stricter suite rather than a looser one — a flip on a margin the reference could resolve now fails while naming the token and the layer, and a difference with the routing *unchanged* is its own verdict instead of being pooled with the tie. [docs/LEARNED.md](/sqliteai/warp/blob/main/docs/LEARNED.md) §71–§72 has the distributions behind the thresholds.\n\nThe general form of that, and the reason 0.7.2 is mostly test code: a check that compares a thing to itself is not a weak oracle, it is not an oracle. Kimi's tool rendering shipped with 438 self-consistent tests, every one of them passing on a defect one of them had pinned; it is now diffed against the release's own chat template ([tests/serve/test_chatfmt_upstream.py](/sqliteai/warp/blob/main/tests/serve/test_chatfmt_upstream.py)), the way K3's encoder has always been diffed against `encoding_k3.py`\n\n. The synthetic `index_bits 6`\n\ncontainer the VQ4P checks run on is packed by a second implementation of the converter's own layout, so the two are now run against each other ([tests/test_vq4p_packing.py](/sqliteai/warp/blob/main/tests/test_vq4p_packing.py)) — the arm itself cannot notice, since both backends would decode a wrongly packed container the same wrong way. Neither defect was visible from inside the code, and both were found by comparing against something outside it.\n\nUseful references:\n\n[docs/FORMAT.md](/sqliteai/warp/blob/main/docs/FORMAT.md): container format;[docs/BACKENDS.md](/sqliteai/warp/blob/main/docs/BACKENDS.md): CPU, SIMD, and Metal backends;[docs/KDA.md](/sqliteai/warp/blob/main/docs/KDA.md): Kimi Delta Attention;[docs/GATES.md](/sqliteai/warp/blob/main/docs/GATES.md): correctness and performance gates;[docs/RESEARCH.md](/sqliteai/warp/blob/main/docs/RESEARCH.md): current research directions.[docs/TECHNICAL.md](/sqliteai/warp/blob/main/docs/TECHNICAL.md): detailed measurements and technical experiments.\n\nWARP is distributed under the permissive Apache 2.0 license, and **the project will always remain open source under a permissive license**. See [LICENSE](/sqliteai/warp/blob/main/LICENSE).\n\nCopyright 2026 SQLite Cloud, Inc.", "url": "https://wpnews.pro/news/glm-5-3-flash-on-apple-silicon", "canonical_source": "https://github.com/sqliteai/warp", "published_at": "2026-08-28 15:33:17+00:00", "updated_at": "2026-08-28 15:48:22.307228+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-infrastructure", "ai-research"], "entities": ["WARP", "Kimi K3", "GLM-5.3-Flash", "MacBook Pro", "M5 Pro", "Kimi-Linear", "Opus 5"], "alternates": {"html": "https://wpnews.pro/news/glm-5-3-flash-on-apple-silicon", "markdown": "https://wpnews.pro/news/glm-5-3-flash-on-apple-silicon.md", "text": "https://wpnews.pro/news/glm-5-3-flash-on-apple-silicon.txt", "jsonld": "https://wpnews.pro/news/glm-5-3-flash-on-apple-silicon.jsonld"}}