{"slug": "not-your-compute-not-your-model", "title": "Not Your Compute, Not Your Model", "summary": "Open-weight AI models like DeepSeek V3.2 (685 billion parameters, ~685 GB) and Kimi K3 (2.8 trillion parameters, ~3 TB) are increasingly difficult to run locally due to hardware constraints, prompting the New York Compute Club to pool member hardware for distributed inference. The club's software, including nycc-engine and nycc-grid, is open source on GitHub, and applications are open at newyorkcomputeclub.com/about.", "body_md": "# Not Your Compute, Not Your Model\n\nOn why open weights are so hard to run even when you're licensed to run them, and on a club in New York that pools the hardware to do it.\n\nCrypto produced exactly one saying that stayed true through every collapse, which is not your keys, not your coins. Hold your coins on an exchange and what you actually hold is an IOU from whoever runs the exchange. I keep coming back to that saying with the nouns swapped, because the open-model era has reproduced the same structure. Not your compute, not your model.\n\nThe strange part is that open weights have never been better or easier to get. DeepSeek releases weights close to the frontier under an MIT license, and all of it is sitting on Hugging Face right now for anyone to download. If this were ordinary open source software, this would be the point where control over AI spreads out to anyone with a machine and a compiler. That isn't what has happened. Most people who download those weights, and I include myself, have nowhere to run them. This post is about the gap between being licensed to run a model and being able to run it. I'll start with what a few of us in New York are building to close it, and then go through why the gap exists, and why it is wider than it looks and not only a matter of GPUs.\n\n## The Club\n\nThe [New York Compute Club](https://newyorkcomputeclub.com) is the\nthing I've been building. The cluster today is a few nodes and an\nextension cord. The shape is simple. Members rack their own hardware\nsomewhere in the five boroughs, and the club schedules jobs across all\nof it. What makes it more than a shared GPU spreadsheet is the trust\ndesign. Jobs travel as libsodium sealed boxes, so the coordinator that\nroutes them sees only ciphertext, a node ID, and a self-reported wattage.\nIt cannot read what you run, and that is a property of the construction\nrather than a policy. I wrote the threat model down, and it says what\nthe design does not protect against, which is how you can tell it's a\nreal threat model and not marketing.\n\nAll of the software is on GitHub. [nycc-engine](https://github.com/jamesbaker1/nycc-engine) is a\nsmall inference engine, with a paged KV cache, continuous batching,\npriority scheduling, and a NumPy reference model you can read in an\nafternoon. [nycc-grid](https://github.com/jamesbaker1/nycc-grid) is the\nmesh, with sealed jobs, signed node traffic, and an untrusted coordinator. [newyorkcomputeclub](https://github.com/jamesbaker1/newyorkcomputeclub)\nis the site, which renders Manhattan as a load histogram. Pull requests\nmove your application up the pile.\n\nThe applications exist to change that node count. If the\narithmetic below describes your relationship with open models, weights\nyou're licensed to run and no way to run them, they're open at\n[newyorkcomputeclub.com/about](https://newyorkcomputeclub.com/about).\nI read every one.\n\n## Doing the Numbers\n\nDeepSeek V3.2 has 685 billion parameters and is FP8 native, so the weights alone occupy about 685 GB before the KV cache. Kimi K3, the largest open model right now, is 2.8 trillion parameters, closer to three terabytes. The largest consumer GPU Nvidia sells holds about 32 GB.\n\nQuantization narrows the gap less than you'd hope. Push DeepSeek down to 4 bits and it still wants roughly 340 GB, which is eleven consumer cards' worth of VRAM. Sparsity doesn't rescue you either. These are mixture-of-experts models, so only a few experts fire for any given token, but you don't know which ones the next token will route to, and every expert has to be held in memory. The memory bill is for the whole model, always.\n\nSo could you wire eleven consumer cards together? Not usefully. Splitting a model across GPUs means shipping activations between them at every layer. The bandwidth that makes that viable, NVLink and the switching fabric around it, is exactly what Nvidia reserves for its datacenter parts. Consumer cards talk to each other over PCIe, and recent generations dropped NVLink entirely.\n\nWhich leaves the 8x H100 node as the practical unit of open-model inference. Those start around $250,000 racked, or you can rent one by the hour from the same few clouds the open weights were supposed to make optional.\n\n## Weights Are the Binary\n\nThe next wall is source code. The source of a model, meaning everything you'd need to reproduce it or meaningfully fork it, is the training data, the filtering pipeline, the training code, the schedule, the post-training recipe, and the evals that decided which checkpoint shipped. A typical open release includes none of those. You can fine-tune the artifact, the way you can patch bytes in a binary, but you can't rebuild it, audit what went into it, or fork the process that produced it. What you hold is the artifact and a license to run it, on hardware you don't have.\n\n## The Rest of the Stack\n\nThe last wall is that even with weights and hardware in hand, inference is its own software problem. You need an engine, with a paged KV cache, continuous batching, and quantized kernels. You need a scheduler that decides whose job runs when the VRAM runs out. You need a way to move jobs between machines without the machines being able to read them. None of that arrives with the weights either. Laid out end to end, the stack under a running model has six layers, and weights are only one of them.\n\n- Training data\n- Training code and recipe\n- Weights\n- Inference engine\n- Scheduling and routing\n- Power and cooling\n\nThe fight over weights is the loud one, because it's the only layer with a license attached. The other five get decided by default, and the default at every one of them is renting. Renting is fine right up until the landlord deprecates the model you built on, or rate-limits you in the middle of an experiment, or logs what you send. None of that requires malice. It's just what happens when the layer you depend on is operated for someone else's purposes, and yours only coincide with theirs on average. I've been on the wrong end of each of those at least once.\n\nSo here is the problem as I see it. The open-model era has the licenses of a commons and the physical structure of a landlord economy. The fix is not everyone buying their own 8x H100 node, since the arithmetic above rules that out for almost everyone, me included. The fix is the one communities have always reached for when the capital is too lumpy for any one person, which is to pool it. That is the club.\n\nAnd there's an obvious next chapter, because compute isn't the bottom of the stack either. Not your power, not your model. An H100 turns electricity into gradients at 700 watts a card, and whoever meters that electricity holds a veto over everything above it. One member has a basement in Red Hook and strong opinions about generators. But that's a later post.\n\n## Artifacts\n\n[newyorkcomputeclub.com](https://newyorkcomputeclub.com), the site[The application](https://newyorkcomputeclub.com/about)[nycc-engine](https://github.com/jamesbaker1/nycc-engine), the engine[nycc-grid](https://github.com/jamesbaker1/nycc-grid), the mesh[The site's source](https://github.com/jamesbaker1/newyorkcomputeclub)\n\n[James Baker](https://www.linkedin.com/in/jamesfrancisbaker/), August 2026 · The club answers slowly · [[email protected]](/cdn-cgi/l/email-protection#2842684c4d5b43474e424145064b4745)", "url": "https://wpnews.pro/news/not-your-compute-not-your-model", "canonical_source": "https://deskofjim.com/blog/compute/", "published_at": "2026-09-03 16:01:26+00:00", "updated_at": "2026-09-03 16:23:27.517873+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-infrastructure", "ai-tools", "ai-research"], "entities": ["New York Compute Club", "DeepSeek", "Kimi K3", "Nvidia", "Hugging Face", "nycc-engine", "nycc-grid", "newyorkcomputeclub"], "alternates": {"html": "https://wpnews.pro/news/not-your-compute-not-your-model", "markdown": "https://wpnews.pro/news/not-your-compute-not-your-model.md", "text": "https://wpnews.pro/news/not-your-compute-not-your-model.txt", "jsonld": "https://wpnews.pro/news/not-your-compute-not-your-model.jsonld"}}