{"slug": "leviathan-a-local-persistent-ai-runtime-for-windows-nvidia-no-server-no-cloud-no", "title": "Leviathan — a local, persistent AI runtime for Windows + NVIDIA (no server, no cloud, no account). Feedback welcome", "summary": "Solo developer OmegaVR (GitHub handle CuppaTea1983) released Leviathan, a local Windows AI runtime that loads GGUF models directly into NVIDIA GPU VRAM in-process with no server, Docker, Python, CUDA toolkit, or API key required. Leviathan uses hand-written CUDA kernels validated byte-exact against llama.cpp on Llama/Mistral, Qwen2.5, and DeepSeek-V2 (MLA + MoE), converting each .gguf once to a native .lev file that is mmapped to VRAM, and adds persistent .fqm memory, ANI knowledge routing, a layered security membrane, and an OpenAI-compatible local bridge. The developer states Leviathan is NVIDIA-only (GTX 16 / RTX 20 or newer), is not faster than llama.cpp on raw throughput, and is set to be released soon under CC BY-NC 4.0, with a Vulkan backend for AMD/Intel planned but not shipped.", "body_md": "Hi all — I’ve been building this solo for a while and it’s finally at a state worth sharing. It’s part of a project I call Sovereign; Leviathan is the public Windows app.\n\nWhat it is: you open the app, pick a GGUF convert it, and then talk to it — the model loads straight into your GPU’s VRAM and runs in-process. No local server, no Docker, no Python or CUDA toolkit to install, no API key. Just a recent NVIDIA driver. It’s its own CUDA inference engine (hand-written kernels), and I’ve validated it byte-exact against llama.cpp on the models it runs — Llama/Mistral, Qwen2.5, and DeepSeek-V2 (MLA + MoE). It converts a .gguf to a native .lev once, then mmaps it to VRAM.\n\nWhat makes it different is everything around the model (the host/guest idea — Leviathan is the host, the model is the guest):\n\nPersistent memory (.fqm) — the model remembers across sessions; close the app and reopen, it’s still there. Per-tab scoped so a coding workspace and a chat don’t bleed into each other.\n\nKnowledge routing (ANI) — teach it things without retraining. Knowledge is kept as text on a shared embedding space and retrieved on relevance — drained from a model’s own confident answers, folded from your chat logs / documents / audio / screen / voice. (I tested weight-level transfer thoroughly and it doesn’t survive across architectures — text + retrieval is the honest carrier, so that’s what it uses.)\n\nA security membrane — a layered input guard (prompt-injection, malicious-code, an always-on floor, a model-file integrity scan before a model reaches VRAM, and knowledge-bank poisoning checks). The sovereignty cuts both ways: nothing leaves your machine, and nothing hostile gets in.\n\nA local bridge — your own programs (a game engine, a script) can use the model Leviathan is hosting, over an OpenAI-compatible endpoint.\n\nHonest boundaries, because I’d rather you trust the page than catch me overclaiming:\n\nNVIDIA-only right now (GTX 16 / RTX 20 or newer). A Vulkan backend for AMD/Intel is planned, not shipped.\n\nIt is not faster than llama.cpp on raw throughput — llama.cpp is the most optimised engine out there. Leviathan’s edge is architectural (in-process, no middleman, + the memory/knowledge/security layers), not tokens/sec.\n\nWeb-free by default; if you connect a hosted model (e.g. Grok) that’s clearly separate and obviously leaves your machine — everything on your own GPU stays local.\n\nLeviathan is set to be released soon, just ironing out UI mostly.\n\nSpace Includes: How it works (full guide) [Leviathan - a Hugging Face Space by Omega-Dev](https://huggingface.co/spaces/Omega-Dev/Leviathan)\n\nGitHub: [CuppaTea1983/Sovereign: Sovereign — A Geometric Substrate for Persistent Intelligence A unified cognitive substrate architecture integrating memory, compression, geometry, and identity.](https://github.com/CuppaTea1983/Sovereign)\n\nNon-commercial (CC BY-NC 4.0). I’d genuinely love eyes on the knowledge-routing / persistence design and the security layering — tell me where it’s weak. — OmegaVR / @CuppaTea1983", "url": "https://wpnews.pro/news/leviathan-a-local-persistent-ai-runtime-for-windows-nvidia-no-server-no-cloud-no", "canonical_source": "https://discuss.huggingface.co/t/leviathan-a-local-persistent-ai-runtime-for-windows-nvidia-no-server-no-cloud-no-account-feedback-welcome/182544#post_1", "published_at": "2026-10-01 03:21:15+00:00", "updated_at": "2026-10-01 03:49:11.404482+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-tools", "ai-safety"], "entities": ["Leviathan", "Sovereign", "OmegaVR", "CuppaTea1983", "NVIDIA", "llama.cpp", "DeepSeek-V2", "Qwen2.5"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/leviathan-a-local-persistent-ai-runtime-for-windows-nvidia-no-server-no-cloud-no", "markdown": "https://wpnews.pro/news/leviathan-a-local-persistent-ai-runtime-for-windows-nvidia-no-server-no-cloud-no.md", "text": "https://wpnews.pro/news/leviathan-a-local-persistent-ai-runtime-for-windows-nvidia-no-server-no-cloud-no.txt", "jsonld": "https://wpnews.pro/news/leviathan-a-local-persistent-ai-runtime-for-windows-nvidia-no-server-no-cloud-no.jsonld"}}