{"slug": "millfolio-my-take-on-local-hybrid-ai", "title": "Millfolio – my take on local/hybrid AI", "summary": "Developer Millfolio launched a local-first AI tool that runs entirely on Mac, inverting the typical data-to-model flow by sending a program to the user's data instead. The system uses a local model for indexing and a frontier model (Claude) for writing query programs, with actual data never leaving the device. The project is built on Mojo, a compiled Python-like language, and aims to provide privacy-preserving AI on personal data.", "body_md": "# Send the program to your data, not your data to the model\n\nI wanted to use AI on my own data — spending now, health and daily notes later — without handing that data to a provider. millfolio is my attempt, and it runs entirely on any Mac.\n\n## Don’t send your data to the model — send a program to your data\n\nThe usual way to put an LLM on your files is to stuff the files into the prompt. millfolio inverts that. Instead of sending your data to the model, it sends a *program* to your data. When you ask a question:\n\n- A\n**local model** has already read and indexed your files on-device — chunking them, embedding them into a local vault index. This is the only part that touches your actual data. - A\n**frontier model**(Claude) writes a small program to answer the question — but it only ever sees an*aliased schema*of your vault: column names, tag names, types, shapes. Never a real value. It’s writing code against a de-identified manifest. - That program runs in an\n**on-device sandbox** over your real files, and the answer is assembled locally.\n\nSo the frontier model does the planning and the coding; your actual data never leaves the machine. What crosses the network is a de-identified manifest plus a query program — not your transactions.\n\nThe claim worth poking at is exactly that boundary: *the model sees the schema, never the values.* I think it holds, and the [privacy design](https://millfolio.app/privacy-box) lays out where the line is — but it’s the thing I most want scrutiny on.\n\n## The data stack: almost entirely Mojo\n\nThe other experiment here is the stack. millfolio’s backend and data pipeline is written in [Mojo](https://www.modular.com/mojo) — including a from-scratch inference engine with custom-written GPU/Metal kernels. The local model reads, indexes, and answers targeted questions on-device. The engine doing the unglamorous work that makes a small model usable on a Mac mini: int4 GEMV, tensor-core attention, a persistent Metal pipeline cache, prompt-lookup speculative decode. There is some additional work to run the local model in batch mode, with different priorities, to keep your laptop responsive should you use it for unrelated work in the mean time. More will come in a future blog post.\n\nI’ve spent years in Python and JavaScript and wanted to see how far a compiled, Python-ish, GPU-capable language gets you with far less ecosystem to pull in and audit. That trade feels newly relevant now that an AI writes most of the code anyway — a smaller surface to review might be a feature, not a cost.\n\n## Honest caveats\n\n- It’s a personal project, Apple-Silicon-only, and mostly AI-written with heavy design and testing from me.\n- It needs an API key for the code-writing step. The local model reads and answers, but it isn’t strong enough to reliably\n*write*the query programs — that’s the frontier model’s job. So it’s local-first, not air-gapped. (If someone thinks a local model can write these programs well, I’d love to be proven wrong — that removes the last remote dependency.) - No warranty, no liability. Privacy is protected by design, but no software is perfectly secure. Use it with data you’re comfortable putting at some risk.\n\n## Try it\n\nThe fastest way to get the idea is the live demo — a synthetic vault, one click, no install and no key:\n\n**Demo:**[demo.millfolio.app](https://demo.millfolio.app)** How it works:**[millfolio.app](https://millfolio.app)** Code:**[github.com/millfolio](https://github.com/millfolio)\n\nI’d love feedback — especially on whether the privacy model holds up to scrutiny, and on betting a real inference stack on Mojo.", "url": "https://wpnews.pro/news/millfolio-my-take-on-local-hybrid-ai", "canonical_source": "https://millfolio.app/blog/send-the-program-to-your-data/", "published_at": "2026-07-07 01:46:31+00:00", "updated_at": "2026-07-07 02:30:51.368721+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "developer-tools"], "entities": ["Millfolio", "Claude", "Mojo", "Apple", "Mac"], "alternates": {"html": "https://wpnews.pro/news/millfolio-my-take-on-local-hybrid-ai", "markdown": "https://wpnews.pro/news/millfolio-my-take-on-local-hybrid-ai.md", "text": "https://wpnews.pro/news/millfolio-my-take-on-local-hybrid-ai.txt", "jsonld": "https://wpnews.pro/news/millfolio-my-take-on-local-hybrid-ai.jsonld"}}