Llambda.lisp an LLM Engine in Pure Common Lisp

A developer released llambda.lisp, a pure Common Lisp inference engine for running quantized large language models from .gguf files without external C or C++ libraries. The engine uses AVX2/FMA acceleration and multi-threading to achieve competitive performance, challenging the assumption that C++ is required for bare-metal AI inference.

A Bare-Metal, Multi-Threaded, AVX2-Accelerated LLM Inference Engine in Pure Common Lisp. llambda.lisp is an independent, zero-dependency beyond sb-simd and lparallel inference engine for running quantized Large Language Models directly from .gguf files. It does not wrap llama.cpp . It does not call out to external C or C++ libraries. It reads the raw weights, unpacks the 4-bit/6-bit nibbles, constructs the transformer architecture, and executes the forward pass natively within SBCL. Because the industry has succumbed to the dogma that C++ is the only path to bare-metal AI inference. llambda.lisp exists to prove that properly architected, aggressively typed, and hardware-aware Common Lisp can achieve C-level throughput without sacrificing the interactive, REPL-driven elegance of Lisp. Native GGUF Parsing: Directly ingests and parses Q4 K M and Q6 K quantized tensors from disk. AVX2 / FMA Acceleration: The core GEMV Matrix-Vector Multiplication bottleneck is pulverized using SBCL's sb-simd . Unrolled f32.8 vectors, unaligned loads VMOVUPS , and packed Fused Multiply-Add VFMADD213PS instructions are emitted natively by the Lisp compiler. Multi-Threaded Execution: The outer loop of the GEMV processing is fully parallelized via lparallel , saturating modern multi-core memory buses e.g., 24-core Ryzen processors with isolated, lock-free writes. Zero-Drift KV Cache: Safe, shared-KV reuse and perfectly aligned RoPE scaling. Exact-logit replay tests against fresh un-cached generations yield a max diff of 0.0 . Advanced Sampling: Built-in Top-K, Top-P Nucleus , and repetition penalties executing in-place with zero heap allocation in the hot path. Gemma4 Support: Full support for Gemma4 architectures, including BPE tokenization, proper instruction-tuning chat templates <bos <|turn user... , and explicit tool-calling channel overrides. SBCL: You must run a modern SBCL compiled with SIMD support. Hardware: An x86 64 CPU with AVX2 instruction set support. Multi-core processors heavily recommended to prevent memory-bus starvation. Dependencies: sb-simd , lparallel . ql:quickload :llambda ;; Load a model and run an end-to-end inference pass llambda:test-gguf-file-response "D:/path/to/your/model/gemma-4-E4B-it-Q4 K M.gguf" "Write a haiku about a hacker drinking coffee." :top-k 40 :top-p 0.90 :repetition-penalty 1.15 If you are modifying the core dot-product macros expand-q4-k-body , heed this warning: Do not allocate in the inner loop. The hot paths rely on strict declare optimize speed 3 safety 0 debug 0 space 0 policies and zero-consing execution. If the compiler begins boxing floats or allocating vectors on the heap, performance will catastrophically collapse. - Gemma4 Base & Instruct Verified - Top-K / Top-P / Rep-Pen Sampler - AVX2/FMA Q4 K M and Q6 K paths - Qwen3 / MoE Routing In Progress - LLaMA 3.1 Architecture Planned MIT License. See LICENSE /jrm-code-project/llambda/blob/main/LICENSE for details. Contributions are welcome Please submit pull requests or open issues for bug reports and feature requests.