I want to see how much useful work I can get from a small language model running on my own laptop. I am choosing a Surface Laptop, 8th Edition, partly because battery life and portability matter to me, but also because I am curious about what I can build on a reasonably priced device.
There are machines better suited to some of these experiments. I could look at the NVIDIA-powered Surface Laptop Ultra, an M5 MacBook Pro, or a dedicated desktop AI machine such as NVIDIA's DGX Spark. There are different tradeoffs in each, but I am simply unwilling to spend extra money chasing local AI capability or make that workload the deciding factor in the laptop I buy.
I have written about bringing Visual Studio natively to Arm64 and building out the workloads around it. The interest then was in making Arm a useful place to do development, rather than relying on emulation. Now I want to explore what a small model could add to that development experience on the same machine.
The configuration I am choosing has a Snapdragon X2 Elite with 12 CPU cores, 32GB of memory, and a Qualcomm Hexagon neural processing unit (NPU) rated at up to 80 TOPS. I am choosing a 1TB SSD to give me room for model files, repositories, and indexes. The storage is separate from the memory available to run a model.
TOPS means trillions of operations per second. The NPU is designed to run supported AI workloads efficiently, but that number does not tell me how quickly a model will respond. I still need to establish whether the model and runtime I choose can use it.
I want to try models that can work with code and use tools while leaving enough of the machine available for development. Phi-4-mini-instruct is one candidate, with 3.8 billion parameters and support for function calling. Its size gives me a concrete starting point for thinking about what the laptop would need to run it.
The first estimate is straightforward:
Weight memory (GB) = parameters_in_billions * bits_per_parameter / 8
For Phi-mini at 4 bits:
3.8 * 4 / 8 = 1.9GB
The model also needs working buffers and a cache for the context it retains, including instructions, repository code, and test results. For Phi-4-mini, one example cache configuration would need about 1.1GB at 8,192 tokens, or 4.3GB at 32,768 tokens, separate from the weights. Windows and my development tools need their share too, so 1.9GB of weights does not mean a 1.9GB application.
That calculation gives me a starting point, but I do not want to repeat a detailed memory exercise for every model I consider. I also have to consider that using 16 bits to store each weight, rather than 4, allows more numerical precision but increases the weight memory to about 7.6GB.
A model runtime should handle much of that setup. Foundry Local is one Microsoft option, with a curated catalog from which I can choose a model and let the software select a hardware-appropriate variant. That is different from a guarantee that it will fit comfortably alongside Visual Studio at the context length I want to use.
I currently have access to what are, for me, essentially free tokens. That is something of a cheat code, but the allowance could be reduced or removed. It makes experimenting easy today, without telling me much about what I could sustain independently. Working on DasBlog Core with a local model would let me explore that, while still using the hosted models where they help.
DasBlog gives me an existing application to work with, and one whose behavior I know well enough to evaluate the result. I could start by asking a local agent to trace how an old permalink resolves to a post, using the relevant code to explain its answer. Given a reproducible problem, the next step could be proposing a regression test, then attempting a small fix and running the tests. Preserving existing links would be part of the requirement, not something to check afterward. I am interested in whether a small model can help with a specific maintenance task, not whether it can take responsibility for the whole application.
Timnit Gebru makes a related point in Frugal AI, her conversation with Alix Dunn published by AI Now Institute:
You want to create a specific tool for a specific context.
It would be trivial for me to keep using the most capable model I can access without asking whether the work needs it. DasBlog gives me a way to explore that question with problems I already care about.
Gebru talks about Te Hiku Media in New Zealand, which gathers data to help revitalize the Maori language. They have refused to license that data to an American company because their work must serve the Maori people first. She suggests that organizations working on different languages could share tools and resources, rather than each having to build a model for everyone.
That argument is broader than my local experiment. It concerns who defines the need, who controls the data, and who benefits from the result. Choosing a smaller model does not settle any of those questions. As Gebru puts it:
We need to both build other things and do it with other resources.
Part of what bothers me about this AI moment is how often the solution arrives before anyone has clearly articulated the problem. If we had invented the car before the bicycle, we would still have reasons to want two-wheeled transportation. Having something more powerful does not make every other approach redundant.
I have written before about following the customer problem rather than identifying with a particular solution. The attention given to frontier models can make other possibilities easier to overlook. Some are new opportunities created by more capable local hardware, others are established approaches, like code search and static analysis, that have not stopped being useful. I want to explore these possibilities, rather than assume that every problem needs a larger model running in a data center.