Hi everyone,
I wanted to share the design invariants and architecture of Draco AI, a full-stack, hardware-agnostic LLM system built completely from scratch in pure Python and NumPy. The core philosophy behind this project is strict code quality, mathematical correctness, and clean architecture—completely eliminating heavy framework dependencies like PyTorch or HuggingFace.
⚠️
100% Fully Open-Source:To clarify, this isnota half-baked open-source release where only the inference wrapper is public. Draco AI is 100% open-source across its entire execution path, including BOTH deep decoupled layers: the execution mechanics of theInference Engine(modeling/
) and the cognitive/reasoning paths of theThinking Engine(thinking_engine/
).
The baseline includes a robust pytest validation suite covering GQA, MLA, hybrid attention, Medusa heads, and speculative tree decoding execution paths.
Y = (X @ pos_mask.T - X @ neg_mask.T) * scale
) without floating-point multiplications on the core execution path.HealthMonitor
tracking NaN/Inf propagation, saturation, and adversarial expert collapse. It signals a DynamicPrecisionManager
for advisory dtype escalation/de-escalation based on overflow EMA while strictly respecting a hard VRAM budget.To prevent silent corruptions common in speculative systems, Draco AI enforces strict execution invariants:
runtime → layers → ops → kernels
. No upward execution calls are permitted. device.py
acts as the single source of truth for hardware capabilities.rope_offset
is captured exactly SnapshotStack
. It restores both the KVCache
slab and the EngramCache
pointers to their exact states add_noise=False
on replay to eliminate stochastic routing drift.I’d highly appreciate your insights regarding the mathematical fusion choices for uncertainty assessment, the decoupled architecture, or the pure-NumPy tensor layout. Let's discuss in the comments below!