{"slug": "llambda-lisp-an-llm-engine-in-pure-common-lisp", "title": "Llambda.lisp an LLM Engine in Pure Common Lisp", "summary": "A developer released llambda.lisp, a pure Common Lisp inference engine for running quantized large language models from .gguf files without external C or C++ libraries. The engine uses AVX2/FMA acceleration and multi-threading to achieve competitive performance, challenging the assumption that C++ is required for bare-metal AI inference.", "body_md": "**A Bare-Metal, Multi-Threaded, AVX2-Accelerated LLM Inference Engine in Pure\nCommon Lisp.**\n\n`llambda.lisp`\n\nis an independent, zero-dependency (beyond `sb-simd`\n\nand\n`lparallel`\n\n) inference engine for running quantized Large Language Models\ndirectly from `.gguf`\n\nfiles.\n\nIt does not wrap `llama.cpp`\n\n. It does not call out to external C or C++\nlibraries. It reads the raw weights, unpacks the 4-bit/6-bit nibbles, constructs\nthe transformer architecture, and executes the forward pass natively within\nSBCL.\n\nBecause the industry has succumbed to the dogma that C++ is the only path to\nbare-metal AI inference. `llambda.lisp`\n\nexists to prove that properly\narchitected, aggressively typed, and hardware-aware Common Lisp can achieve\nC-level throughput without sacrificing the interactive, REPL-driven elegance of\nLisp.\n\n**Native GGUF Parsing:** Directly ingests and parses`Q4_K_M`\n\nand`Q6_K`\n\nquantized tensors from disk.**AVX2 / FMA Acceleration:** The core GEMV (Matrix-Vector Multiplication) bottleneck is pulverized using SBCL's`sb-simd`\n\n. Unrolled`f32.8`\n\nvectors, unaligned loads (`VMOVUPS`\n\n), and packed Fused Multiply-Add (`VFMADD213PS`\n\n) instructions are emitted natively by the Lisp compiler.**Multi-Threaded Execution:** The outer loop of the GEMV processing is fully parallelized via`lparallel`\n\n, saturating modern multi-core memory buses (e.g., 24-core Ryzen processors) with isolated, lock-free writes.**Zero-Drift KV Cache:** Safe, shared-KV reuse and perfectly aligned RoPE scaling. Exact-logit replay tests against fresh un-cached generations yield a`max_diff`\n\nof`0.0`\n\n.**Advanced Sampling:** Built-in Top-K, Top-P (Nucleus), and repetition penalties executing in-place with zero heap allocation in the hot path.**Gemma4 Support:** Full support for Gemma4 architectures, including BPE tokenization, proper instruction-tuning chat templates (`<bos><|turn>user...`\n\n), and explicit tool-calling channel overrides.\n\n**SBCL:** You must run a modern SBCL compiled with SIMD support.**Hardware:** An x86_64 CPU with AVX2 instruction set support. Multi-core processors heavily recommended to prevent memory-bus starvation.**Dependencies:**`sb-simd`\n\n,`lparallel`\n\n.\n\n```\n(ql:quickload :llambda)\n\n;; Load a model and run an end-to-end inference pass\n(llambda:test-gguf-file-response \n  \"D:/path/to/your/model/gemma-4-E4B-it-Q4_K_M.gguf\" \n  \"Write a haiku about a hacker drinking coffee.\"\n  :top-k 40 \n  :top-p 0.90 \n  :repetition-penalty 1.15)\n```\n\nIf you are modifying the core dot-product macros (`expand-q4-k-body`\n\n), heed\nthis warning: **Do not allocate in the inner loop.** The hot paths rely on\nstrict `(declare (optimize (speed 3) (safety 0) (debug 0) (space 0)))`\n\npolicies\nand zero-consing execution. If the compiler begins boxing floats or allocating\nvectors on the heap, performance will catastrophically collapse.\n\n- Gemma4 Base & Instruct (Verified)\n- Top-K / Top-P / Rep-Pen Sampler\n- AVX2/FMA\n`Q4_K_M`\n\nand`Q6_K`\n\npaths - Qwen3 / MoE Routing (In Progress)\n- LLaMA 3.1 Architecture (Planned)\n\nMIT License. See [LICENSE](/jrm-code-project/llambda/blob/main/LICENSE) for details.\n\nContributions are welcome! Please submit pull requests or open issues for bug reports and feature requests.", "url": "https://wpnews.pro/news/llambda-lisp-an-llm-engine-in-pure-common-lisp", "canonical_source": "https://github.com/jrm-code-project/llambda", "published_at": "2026-07-13 07:05:03+00:00", "updated_at": "2026-07-13 07:35:43.447118+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-infrastructure", "developer-tools"], "entities": ["llambda.lisp", "SBCL", "sb-simd", "lparallel", "AVX2", "FMA", "Gemma4"], "alternates": {"html": "https://wpnews.pro/news/llambda-lisp-an-llm-engine-in-pure-common-lisp", "markdown": "https://wpnews.pro/news/llambda-lisp-an-llm-engine-in-pure-common-lisp.md", "text": "https://wpnews.pro/news/llambda-lisp-an-llm-engine-in-pure-common-lisp.txt", "jsonld": "https://wpnews.pro/news/llambda-lisp-an-llm-engine-in-pure-common-lisp.jsonld"}}