{"slug": "building-an-llm-system-from-scratch-in-pure-python-numpy-architecture-invariants", "title": "Building an LLM System from Scratch in Pure Python & NumPy: Architecture, Invariants, and Clean Code", "summary": "A developer built Draco AI, a full-stack, hardware-agnostic LLM system from scratch in pure Python and NumPy, eliminating heavy framework dependencies like PyTorch or HuggingFace. The system is 100% open-source across its entire execution path, including the Inference Engine and Thinking Engine, and enforces strict execution invariants to prevent silent corruptions. The project emphasizes code quality, mathematical correctness, and clean architecture.", "body_md": "Hi everyone,\n\nI wanted to share the design invariants and architecture of **Draco AI**, a full-stack, hardware-agnostic LLM system built completely from scratch in pure Python and NumPy. The core philosophy behind this project is strict code quality, mathematical correctness, and clean architecture—completely eliminating heavy framework dependencies like PyTorch or HuggingFace.\n\n⚠️\n\n100% Fully Open-Source:To clarify, this isnota half-baked open-source release where only the inference wrapper is public. Draco AI is 100% open-source across its entire execution path, including BOTH deep decoupled layers: the execution mechanics of theInference Engine(`modeling/`\n\n) and the cognitive/reasoning paths of theThinking Engine(`thinking_engine/`\n\n).\n\nThe baseline includes a robust pytest validation suite covering GQA, MLA, hybrid attention, Medusa heads, and speculative tree decoding execution paths.\n\n`Y = (X @ pos_mask.T - X @ neg_mask.T) * scale`\n\n) without floating-point multiplications on the core execution path.`HealthMonitor`\n\ntracking NaN/Inf propagation, saturation, and adversarial expert collapse. It signals a `DynamicPrecisionManager`\n\nfor advisory dtype escalation/de-escalation based on overflow EMA while strictly respecting a hard VRAM budget.To prevent silent corruptions common in speculative systems, Draco AI enforces strict execution invariants:\n\n`runtime → layers → ops → kernels`\n\n. No upward execution calls are permitted. `device.py`\n\nacts as the single source of truth for hardware capabilities.`rope_offset`\n\nis captured exactly `SnapshotStack`\n\n. It restores both the `KVCache`\n\nslab and the `EngramCache`\n\npointers to their exact states `add_noise=False`\n\non replay to eliminate stochastic routing drift.I’d highly appreciate your insights regarding the mathematical fusion choices for uncertainty assessment, the decoupled architecture, or the pure-NumPy tensor layout. Let's discuss in the comments below!", "url": "https://wpnews.pro/news/building-an-llm-system-from-scratch-in-pure-python-numpy-architecture-invariants", "canonical_source": "https://dev.to/ducnguyen-creator/building-an-llm-system-from-scratch-in-pure-python-numpy-architecture-invariants-and-clean-code-5a8c", "published_at": "2026-07-11 00:56:51+00:00", "updated_at": "2026-07-11 02:11:23.806313+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "developer-tools", "ai-infrastructure"], "entities": ["Draco AI", "NumPy", "Python", "PyTorch", "HuggingFace"], "alternates": {"html": "https://wpnews.pro/news/building-an-llm-system-from-scratch-in-pure-python-numpy-architecture-invariants", "markdown": "https://wpnews.pro/news/building-an-llm-system-from-scratch-in-pure-python-numpy-architecture-invariants.md", "text": "https://wpnews.pro/news/building-an-llm-system-from-scratch-in-pure-python-numpy-architecture-invariants.txt", "jsonld": "https://wpnews.pro/news/building-an-llm-system-from-scratch-in-pure-python-numpy-architecture-invariants.jsonld"}}