I wanted to use AI on my own data — spending now, health and daily notes later — without handing that data to a provider. millfolio is my attempt, and it runs entirely on any Mac.
Don’t send your data to the model — send a program to your data #
The usual way to put an LLM on your files is to stuff the files into the prompt. millfolio inverts that. Instead of sending your data to the model, it sends a program to your data. When you ask a question:
- A local model has already read and indexed your files on-device — chunking them, embedding them into a local vault index. This is the only part that touches your actual data. - A frontier model(Claude) writes a small program to answer the question — but it only ever sees analiased schemaof your vault: column names, tag names, types, shapes. Never a real value. It’s writing code against a de-identified manifest. - That program runs in an on-device sandbox over your real files, and the answer is assembled locally.
So the frontier model does the planning and the coding; your actual data never leaves the machine. What crosses the network is a de-identified manifest plus a query program — not your transactions.
The claim worth poking at is exactly that boundary: the model sees the schema, never the values. I think it holds, and the privacy design lays out where the line is — but it’s the thing I most want scrutiny on.
The data stack: almost entirely Mojo #
The other experiment here is the stack. millfolio’s backend and data pipeline is written in Mojo — including a from-scratch inference engine with custom-written GPU/Metal kernels. The local model reads, indexes, and answers targeted questions on-device. The engine doing the unglamorous work that makes a small model usable on a Mac mini: int4 GEMV, tensor-core attention, a persistent Metal pipeline cache, prompt-lookup speculative decode. There is some additional work to run the local model in batch mode, with different priorities, to keep your laptop responsive should you use it for unrelated work in the mean time. More will come in a future blog post.
I’ve spent years in Python and JavaScript and wanted to see how far a compiled, Python-ish, GPU-capable language gets you with far less ecosystem to pull in and audit. That trade feels newly relevant now that an AI writes most of the code anyway — a smaller surface to review might be a feature, not a cost.
Honest caveats #
- It’s a personal project, Apple-Silicon-only, and mostly AI-written with heavy design and testing from me.
- It needs an API key for the code-writing step. The local model reads and answers, but it isn’t strong enough to reliably writethe query programs — that’s the frontier model’s job. So it’s local-first, not air-gapped. (If someone thinks a local model can write these programs well, I’d love to be proven wrong — that removes the last remote dependency.) - No warranty, no liability. Privacy is protected by design, but no software is perfectly secure. Use it with data you’re comfortable putting at some risk.
Try it #
The fastest way to get the idea is the live demo — a synthetic vault, one click, no install and no key:
Demo:demo.millfolio.app** How it works:millfolio.app Code:**github.com/millfolio I’d love feedback — especially on whether the privacy model holds up to scrutiny, and on betting a real inference stack on Mojo.