AI Stack as Code: Define It Once, Run It Everywhere Flox has released two reference patterns for implementing declarative AI stacks as code, enabling reproducible, cross-platform AI environments. The first pattern composes four modular environments to create a portable RAG stack for development or production, while the second uses llama.cpp to power a RAG-based knowledge assistant for Flox documentation. By declaring dependencies in a TOML manifest rather than packaging platform-specific containers, Flox's graph-backed resolver materializes hardware-optimized dependencies on any machine without global installation. AI Stack as Code: Define It Once, Run It Everywhere “Do you have some AI stack-as-code examples?” It’s one of the most common questions we get from customers. What they want are reference patterns for implementing declarative stacks for things like model-training https://flox.dev/blog/training-nanogpt-on-slurm-with-a-nix-pinned-environment/ , retrieval-augmented generation RAG https://flox.dev/popular-packages/get-a-portable-turn-key-rag-stack-with-verba-and-flox/ , embeddings, and other AI use cases. So we went ahead and built some . One pattern composes https://flox.dev/docs/concepts/composition composing-environments four modular AI environments to create a reproducible, cross-platform, completely declarative AI RAG stack. It can be used either for development or as the foundation for a production RAG pattern. A second pattern, a llamacpp https://github.com/ggml-org/llama.cpp -powered model serving environment, uses Flox’s documentation and articles from our blog to provide a RAG-powered knowledge assistant for asking questions about Flox. It demonstrates a production pattern for deploying AI stacks-as-code. Let’s dig into what a stack-as-code is… and why it’s the best way to create, share, and manage AI stacks. AI Stacks That Run on Every Machine If you’ve ever experienced the mounting frustration, the mix of fear and loathing, of attempting to run someone’s AI stack on your local system, the genius of AI-stacks-as-code should resonate with you. Say you’ve built something on your MacBook using Apple’s MPS APIs https://developer.apple.com/documentation/metalperformanceshaders and you go to share it with the rest of your team… but not everybody can use it because they're on Windows or Linux machines with NVIDIA CUDA GPUs. Or imagine the reverse, where ML and AI Stacks someone built for CUDA won't run accelerated on your MacBook using MPS or Apple’s MLX framework https://opensource.apple.com/projects/mlx/ . If It runs at all, it’s in a container, with no access to a GPU. The ML- or AI-stack-as-code makes these problems go away. Instead of packaging and materializing an AI or ML stack’s runtime dependencies for a specific machine platform , you declare them, like so: The TOML above is an example of one kind of stack-as-code implementation: a Flox environment. It declares the basic dependencies for a reproducible, cross-platform AI RAG stack. Paste this into a Flox manifest and you get a runtime environment with exactly these declared dependencies. Note : Flox isn’t the only declarative, stack-as-code technology out there. Others include Nix and Guix https://flox.dev/blog/standardized-development-environments-explained/ . If you need GPU-accelerated packages for MPS/MLX Macs and NVIDIA CUDA to coexist in the same environment , no worries: just declare platform-specific versions of PyTorch and other performance-sensitive packages: You’re not building or re building anything, least of all a multi-gigabyte container image. You’re delegating to a graph-backed resolver https://www.tweag.io/blog/2025-09-04-introduction-to-dependency-graph/ the work of pulling in just the right platform- and hardware-optimized dependencies. The resolver uses the stack’s declarative spec to identify compatible versions of runtime dependencies. These get materialized into an immutable, cryptographically hashed store that lives on each machine. In other words: You declare what the stack needs; the tool resolves the dependency graph and figures out how to materialize what you’ve declared locally. Nothing gets installed globally, so your machine won’t break. You can even install conflicting versions of packages side-by-side… on the same system. AI Stacks That Run at Any Time This pattern also obviates one of the most frustrating parts of working with AI stacks: you’ve got the right dependencies in the right places, but the stack still won’t run because… because what now ? Because the runtime environment itself isn’t standardized: the Jupyter notebook starts with the wrong Python; or the dynamic linker finds the wrong versions of native libraries; or the model cache lives in the cloud and needs an Oauth token; or the GPU runtime only works when certain shell variables get exported. An AI stack-as-code runs and behaves the same way everywhere because it doesn’t just declare that stack’s runtime software packages… it declares the environment in which the stack itself is to run . The Bash shell logic you see above is all that’s required to exactly reproduce the same environment across macOS, Linux, and Windows with WSL2, on x86 or ARM. If a machine has a CUDA or MPS-/MLX-compatible GPU, the runtime detects and uses it. Under the hood, Flox, which is built on Nix, wires up the environment, setting declared packages on PATH so they take priority over local versions. This means commands like , python , and uv always resolve to their declared versions. jupyter There’s one more thing. An AI stack-as-code travels across space and time. Five years from now, it will resolve to exactly the same declared dependencies and reproduce exactly the same runtime environment on each supported target platform. Assuming its platform, hardware, and accelerator APIs remain supported by upstream vendors The upshot is that an AI stack-as-code created two or more years ago and pinned at that time to now-historical dependencies still runs and behaves exactly the same today. The Power of Declarative Environments The way this works is simple. Let’s use Flox as an example: You declaratively define software packages, environment variables, setup logic, services, even build recipes as part of the Flox manifest. This makes recreating an AI stack’s runtime environment as simple as copying the .flox directory that’s co-located alongside the AI project’s Git repo or even more convenient activating it on-demand https://flox.dev/docs/concepts/floxhub-environments activating-a-floxhub-environment from FloxHub https://flox.dev/docs/concepts/floxhub what-is-floxhub . Each Flox environment generates a lockfile that feezes software in place until you run to update one or more packages. flox upgrade Note : you can also pin packages to specific versions to keep them from being upgraded. A Flox AI stack-as-code standardizes the complex moving parts of an AI stack, declaring Python versions and dependencies, native C/C++ libraries, document parsers, embedding models, vector databases, notebooks, and other software … even runtime services. It runs and behaves reproducibly, pulling in the same versions of the same packages. It lets you replace READMEs or runbooks with one command: Defining AI stacks as code gives teams declarative artifacts they can version, diff, review, and promote or roll back atomically. The same Flox AI stack-as-code runs everywhere: on macOS, Linux, and Windows with WSL2 ; on x86 and ARM; on Nvidia CUDA or Apple Metal/MLX. It’ll even run slowly on bare CPUs, if GPU acceleration isn’t available. An AI stack-as-code can also travel from local dev for model R&D/training, RAG prototyping, document-ingestion workflows, etc. to eval, CI, and production. Finally, the declarative stack-as-code simplifies CVE remediation: not only can you catalog exactly which environments run which dependencies, but identifying vulnerabilities is as simple as a database query, while promotion or rollback takes the form of an atomic edit to a text file: You might be thinking: “I can do basically the same thing with containers.” But, respectfully, you basically cannot. A Dockerfile is neither purely declarative nor perfectly reproducible https://flox.dev/blog/standardized-development-environments-explained/ . Teams must rebuild OCI images to patch CVEs. And with AI workloads, especially, multi- or double-digit-gigabyte OCI images are costly to build, store, move, and run. Declarative stacks-as-code eliminate image rebuild churn; tools like Flox and Nix can even be used to generate distroless containers from declarative stack-as-code specs. Flox AI Stack-as-Code Patterns Flox’s RAG pattern https://hub.flox.dev/flox-labs/ai-rag-stack-pattern defines: - A Python 3.13 base; - A document-parsing layer; - An embeddings layer consisting of , torch ; sentence-transformers ChromaDB https://github.com/chroma-core/chroma vector store ; - A notebook layer consisting of JupyterLab, configured to run as a service; and - A working, extractive-RAG pipeline with an evaluation harness. This stack runs locally, without phoning home to external services. You can pull and run it just by doing flox activate -s flox-labs/ask-flox-llamacpp ; alternatively, you need only clone this GitHub repo https://github.com/flox/ai-stacks-as-code.git to get the same stack locally. Helper functions , ai-doctor , ingest , index , etc. run the RAG-building workflow. brief Flox’s llama.cpp production model server https://hub.flox.dev/flox-labs/llamacpp environment can either layer on top of this RAG stack for building and testing or run on its own in production. It ships a built-in llama.cpp model server for Linux and macOS, along with Anthropic’s Claude Code, OpenAI’s Codex, OpenCode, and a slew of other agentic harnesses. The RAG pattern augments users’ prompts with the context required to answer them. It’s a neat demonstration of how you can use Flox to build, package, and publish anything: including in this case the index and contents of a ChromaDB vector database Run it by doing flox activate -s -r flox-labs/ask-flox-llamacpp . A pre-defined helper function, , runs Claude Code with the optional local model of your choice. If you don’t specify a local model, it runs against the latest version of Claude Opus. However you run it, Claude Code uses the ask-flox