![Best Local LLM for Coding: $0 Weights vs $2 API [2026]](https://shattered.io/wp-content/uploads/2026/11/best-local-llm-for-coding-free-weights-vs-2-api-2026-gen.webp)
The best local LLM for coding in mid-November 2026 depends less on a leaderboard score than on three constraints: how much memory your machine has, what license the weights carry, and whether you can verify the files you download. Google’s Gemma 4 family, released under Apache 2.0 with commercial use allowed, OpenAI’s gpt-oss-120b and gpt-oss-20b, and Moonshot AI’s Kimi K3.1 Pro all compete for the same developer attention, yet they sit at very different points on the hardware ladder.
Choosing among them is harder than the headlines suggest. Parameter counts run from small on-device variants to a model that Moonshot describes at the 3-trillion level, and “open” means something different in each license file. Readers who want the licensing side in isolation can start with our open-weight license comparison. The narrower question is which of these models should actually sit behind your editor, terminal agent or CI job.
The short answer, with the evidence below. For a single developer with a 16 GB-class machine, gpt-oss-20b is the safest starting point because OpenAI ships it under Apache 2.0 and documents a small memory footprint. For a team with an 80 GB-class workstation, gpt-oss-120b is the next step up. Gemma 4 is the pick when you want the cleanest license and a size for every machine. Kimi K3.1 Pro is a server-class model, not a laptop model. DeepSeek, Qwen and Llama stay on the watch list because the evidence we could verify today does not support ranking them for local use.
Don't miss new tech stories on Google
Add Shattered once in the Google app and our stories show up in your news suggestions.
Why Run a Coding Model on Your Own Hardware in November 2026 #
Three reasons keep pulling developers toward local inference: code privacy, cost predictability and independence from someone else’s deprecation calendar. The third reason got a live demonstration this week. On November 10, 2026, DigitalOcean published a release note that pulled several open models from its serverless inference product. The notice names Llama 4 Maverick 17B 128E Instruct, and a research summary of the same notice reports that Gemma 4, Qwen 3.5 397B A17B and DeepSeek V4 Flash left serverless inference too, with DigitalOcean pointing users toward DeepSeek V4.1 Flash. Gemma 4 stayed available on dedicated inference. Nothing was discontinued by its maker. A hosting decision simply removed a cheap endpoint, and every team that had wired a pipeline to it had to migrate.
Weights on your own disk do not get deprecated that way. If you pin a model file and a runtime version, the behavior of your coding assistant stays the same next month, which matters when your test suite or your review bot depends on consistent output. That stability is the strongest practical argument for local, stronger than raw price.
What the industry data says about open models
Adoption data points the same direction, though unevenly. IDC’s 2025 FutureScape predictions for Asia-Pacific (excluding Japan) state: “By 2026, 60% of Asia-Pacific organizations are predicted to build applications using open-source AI foundation models.” The source is IDC’s developer and DevOps predictions document, and the figure is a forecast, not a measured result.
Measured usage tells a more cautious story. The 2025 Stack Overflow Developer Survey reports: “OpenAI GPT models were used by 82% of developers for development work during the past year.” That line comes from the technology section of the survey. Those are cloud-served models, so the number suggests that local setups remain the exception for day-to-day coding in 2025, even as the open-weight option gets better. The gap between the 60% prediction and the 82% usage figure is the opening for local models: most teams still call hosted APIs first and consider local weights when privacy, cost or availability bites.
The Contenders at a Glance: Specs Table #
The table below holds only facts that two independent sources support, or that the makers state on their own pages. Where a fact is missing, the cell says so rather than guessing. A blank is useful information here, because several of these models ship without the hardware guidance a buyer would want.
| Attribute | gpt-oss-20b | gpt-oss-120b | Gemma 4 | Kimi K3.1 Pro |
|---|---|---|---|---|
| Maker | OpenAI | OpenAI | Moonshot AI | |
| Weights license | Apache 2.0 plus gpt-oss usage policy | Apache 2.0 plus gpt-oss usage policy | Apache 2.0 | Reported custom terms (see below) |
| Commercial use | Yes, subject to usage policy | Yes, subject to usage policy | Yes | Yes, but a separate agreement is reported for large hosting businesses |
| Modality | Text only | Text only | Multilingual, trained on 140+ languages | Aimed at coding, 3D creation, finance and agent tasks |
| Intended role | Lower latency, constrained environments | Heavier production workloads | General open model, several sizes | Long-horizon coding and agents |
| Context window | Not confirmed here | Not confirmed here | Not confirmed here | 1 million tokens |
| Official VRAM table | No complete table published | No complete table published | Per-variant model cards | None found on official pages |
| Served in ChatGPT or OpenAI API | No | No | Not applicable | Not applicable |
| Hosted API price per million tokens | No OpenAI API price | No OpenAI API price | $0.14 input, $0.40 output (31B, one tracker) | $2 input, $10 output |
| Predecessor | None named | None named | Earlier Gemma generations | Kimi K3 (July 16, 2026 launch entry) |
| Official download confirmed | Yes | Yes | Yes | Not on the official pages we could retrieve |
| Realistic single-machine fit | Laptop or single GPU | Workstation class | Depends on variant | Multi-accelerator server |
| Status on November 12, 2026 | Active | Active | Active | Active |
Two rows deserve a closer look. The “realistic single-machine fit” row is our judgment, built from the stated model scale and OpenAI’s memory guidance, not a vendor claim. And the Kimi download row reflects a real discrepancy: third-party sites say the weights are public, while Moonshot’s own pages in our fetch showed no checkpoint, repository or model-card license. Until Moonshot’s page or its Hugging Face organization confirms the files, treat the open-weight status as reported rather than established. Our runtime comparison of Ollama, LM Studio, llama.cpp and vLLM covers the software side of serving whichever file you settle on.
gpt-oss-20b and gpt-oss-120b: The Safe Defaults #
OpenAI released gpt-oss-120b and gpt-oss-20b on August 5, 2025, as text-only open-weight reasoning models meant for self-hosting or deployment through hosting providers. Some aggregator pages now show November 2026 dates next to the family. OpenAI’s own announcement page carries an update dated November 5, 2026, which explains the confusion: the documentation moved, the models did not. We found no confirmed successor, rename or retirement as of today, so the pair remains OpenAI’s current open-weight line.
The license is the first selling point. Both models ship under Apache 2.0, which permits commercial use, modification and redistribution, with the added condition that use follows OpenAI’s gpt-oss usage policy. For a company legal team that is a short review. The second selling point is availability: OpenAI’s help center states that the models are downloadable weights for self-hosted or third-party-hosted inference, and that neither is served through ChatGPT or the OpenAI API. If you want OpenAI-lineage reasoning behaviour on a machine that never touches the internet, this family is the only door.
Memory and runtime fit
OpenAI does not publish a complete VRAM table on the pages we retrieved, but its documentation reportedly gives two anchor points. The natively MXFP4-quantized 120b model fits within 80 GB of memory, and the 20b model needs about 16 GB. Treat both as single-source figures and verify them against the model card before you buy hardware. If they hold, the 20b model reaches a 16 GB GPU or a laptop with 16 GB or more of unified memory, while the 120b model targets a single 80 GB accelerator or a large unified-memory workstation. The same documentation lists vLLM, Ollama, llama.cpp and Transformers as supported ways to run the weights, which covers nearly every serving stack a developer already uses. A step-by-step setup for running LLMs on an RTX 5090 shows how that looks on a 32 GB consumer card.
Where gpt-oss falls short for coding
Two cautions apply. First, OpenAI frames these as reasoning models, and reasoning models spend tokens thinking before they answer. On a local machine that means slower time to the first useful line of code than a non-reasoning model of the same size. Second, OpenAI also lists gpt-oss-safeguard-120b and gpt-oss-safeguard-20b, open-weight safety-reasoning models. Those are classifiers for policy enforcement. Do not pull them into a coding setup by mistake because the names look similar.
For context on what the closed-model alternative costs, OpenAI’s own release page lists GPT-6.1 Terra, introduced on October 26, 2026, at $1 per million input tokens and $5 per million output tokens through the API. That is the benchmark every local setup has to beat on convenience, since free weights still cost hardware, power and your time.
Gemma 4: Apache 2.0 and a Size for Every Machine #
Google’s announcement post, titled “Gemma 4: Our most capable open models to date” and dated April 2, 2026, positions the family as its strongest open release so far. Three independent pages agree on the license: Apache 2.0, commercial use included. That puts Gemma 4 level with gpt-oss on legal simplicity and ahead of any model with a custom agreement. Google’s documentation also states that the models were natively trained on more than 140 languages, which matters for teams whose codebases carry comments, commit messages and tickets in several languages.
The variant lineup is where Gemma 4 earns its place in a tiered guide. Third-party listings report a 31B dense model, a 26B A4B mixture-of-experts model and two small edge variants called E2B and E4B. Single-source listings are the only evidence for those names, so confirm them on Google’s model cards. If they hold, the structure is attractive: a small variant for a thin laptop, a mid-size variant for a 24 GB to 32 GB card, and a larger dense model for a workstation, all under one license and one prompt format.
Hosted pricing and the DigitalOcean change
One tracker, llmbenchmark.ai, lists Gemma 4 31B at $0.14 per million input tokens and $0.40 per million output tokens for hosted access. DigitalOcean removed Gemma 4 from serverless inference on November 10 while keeping it on dedicated inference. The pattern is the argument for local weights: the model is stable, the hosting around it is not.
Google’s Gemma page also lists FunctionGemma, a function-calling variant dated January 2026, which is relevant if you plan to build tool-using agents on top of a small model. We could not confirm anything newer in that sub-line during the last three months, so we do not rank it separately.
Kimi K3.1 Pro: Open Weights You Probably Cannot Run #
Kimi K3.1 Pro is the headline model of the week and the one most likely to mislead a buyer. Moonshot’s site lists it as the post-trained successor to Kimi K3, which launched with a July 16, 2026 entry and a stated scale of roughly 2.8 trillion parameters. Moonshot describes K3.1 Pro at the 3-trillion level with a 1 million-token context window, aimed at long-horizon coding, 3D creation, financial research and complex agent tasks. Our coverage of the Kimi K3.1 Pro open-weight release tracks the launch itself.
Whether the weights are truly downloadable is unsettled in the sources we can verify. Moonshot’s official pages in our fetch did not show a checkpoint, repository, model-card license or hardware guidance. Some third-party reports say the weights have been released, but we could not confirm that. Moonshot describes the model at the 3 trillion-level, which is far beyond a consumer GPU. Running it would take a multi-accelerator server, which makes it a team or cloud decision.
The license clause that matters
The reported license lets you use and redistribute the model but requires a separate commercial agreement for a model-as-a-service business with more than $20 million in revenue in any 12-month period, according to the Kimi K3.1 Pro page on LMSpedia. A developer running it internally is not affected. A company selling inference to others at that scale is. That is a narrower restriction than the usage bans in some older community licenses, but it breaks the clean Apache 2.0 story and needs a lawyer’s eye before you build a product on top.
Hosted access runs at $2 per million input tokens, $10 per million output tokens and $0.20 per million cached input tokens, with a faster Highspeed tier at $4 and $20 reported on November 4 to 6. Reports also say Highspeed reached Amazon Bedrock on November 6, which would give AWS customers a managed path without any hardware purchase. If the goal is simply to test what Kimi can do on your repository, rent it by the token first. Do not order a GPU server on a headline.
DeepSeek, Qwen and Llama: Where the Verifiable Evidence Runs Out #
Three names missing from the specs table are the three most common in forum threads. Leaving them out is deliberate. Each has real news this month, but the news does not answer the local-coding question with facts we can stand behind.
DeepSeek V4.1 Flash
DeepSeek’s newest documented model is V4.1 Flash, with a later update named V4.1-Flash-1017 on its web and app products. DeepSeek’s change log shows the older V4 Flash identifiers were temporarily routed to the new model for compatibility, and a research summary reports that V4 Flash and the V4 Flash Vision Exp variant retired when V4.1 Flash launched. Reportedly, the 1017 update focused on software-project development with lower cost and faster execution, which fits a coding buyer’s needs.
What the verified sources do not show is a model card, license and file size for local download. Without those, a hardware recommendation would be a guess. DeepSeek stays on the watch list until its repository answers the question. For a wider head-to-head, see our DeepSeek, Qwen and Mistral comparison.
Qwen and Qwen Code
Alibaba’s strongest verified activity is on the tooling side. Qwen Code, described as a terminal-based agentic coding tool, shipped versions 0.26.4 through 0.26.7 in its most recent weekly update. An October 9 release added Qwen Live, a voice mode with screen sharing, and a /batch-api command for repetitive background jobs priced at half the real-time rate after user approval. That cadence tells you the harness moves fast, so pin a version in any team setup.
On the model side, the sources we could verify did not establish which Qwen weights are current, what they weigh or how they are licensed. DigitalOcean’s November 10 notice points users toward Qwen3.8-Max, but that is a hosted migration target, not a confirmed local release. Qwen Code release notes mention a qwen4 thinking-depth setting, but that does not amount to a confirmed model release. A research summary also lists Qwen 3.5 397B A17B among the models DigitalOcean pulled from serverless on November 10. Treat Qwen’s local status as unverified for now. The Qwen Code documentation blog is the primary source for the tooling facts above.
Meta Llama
We found no current Llama announcement we could verify. The one dated item is negative: DigitalOcean’s November 10 notice names Llama 4 Maverick 17B 128E Instruct among the models leaving serverless inference. That does not mean Meta retired the model. It means a hosting provider stopped serving it cheaply. Llama keeps a huge installed base of fine-tunes and tooling, but with no verified new release, it does not earn a spot in a ranking that insists on current facts.
Benchmarks: What Public Scores Can and Cannot Tell You #
Public coding benchmarks come from three kinds of source, and each has a blind spot. Vendor model cards publish the flattering numbers, run on the vendor’s preferred harness. Aggregator sites such as llmbenchmark.ai, which lists Gemma 4 31B benchmarks, pricing and specs for November 2026, collect those numbers and add price data, but they rarely re-run anything. Community leaderboards and forum threads test on real hardware, yet they use mixed quantizations and prompts, so scores do not line up.
DeepSeek’s own page makes the harness problem visible. A footnote there says Terminal-Bench 4.0 was tested with the Claude Code framework and DeepSWE with the Mini-SWE-V2 framework, and that one multimodal test used Pass@5. Swap the framework or the sampling rule and the score moves. A model that tops a leaderboard through an agent scaffold can look ordinary in a plain chat window, and the reverse is just as common.
For this comparison, we found no dated, independent benchmark that runs gpt-oss, Gemma 4 and Kimi K3.1 Pro on the same harness, the same quantization and the same hardware. We decline to print scores that would imply otherwise. The honest workflow is to shortlist by license and memory, then score two or three candidates on your own code. That takes under an hour once the harness exists.
A short harness for your own repository
The script below talks to any OpenAI-compatible local endpoint, which Ollama, llama.cpp’s server and vLLM all provide. It sends a small set of coding tasks, extracts the code from each reply, runs it against a test and records pass rate and tokens per second. Replace the model tag with whatever your runtime lists, and swap the sample task for functions drawn from your own codebase. Run generated code inside a throwaway container or a locked-down VM, never directly on your workstation, because a model can emit anything.
import re, subprocess, tempfile, time
import requests
BASE = "http://localhost:11434/v1"
MODELS = ["gpt-oss:20b"]
TASKS = [
{
"name": "slugify",
"prompt": "Write a Python function slugify(text) that lowercases, "
"trims, replaces runs of non-alphanumerics with one hyphen "
"and strips edge hyphens. Return only code.",
"test": "assert slugify(' Hello, World!! ') == 'hello-world'\n"
"assert slugify('A--B') == 'a-b'\nprint('ok')",
},
]
def ask(model, prompt):
start = time.time()
body = {"model": model, "temperature": 0,
"messages": [{"role": "user", "content": prompt}]}
data = requests.post(BASE + "/chat/completions", json=body, timeout=900).json()
text = data["choices"][0]["message"]["content"]
tokens = data.get("usage", {}).get("completion_tokens", 0)
return text, tokens, time.time() - start
def extract(text):
match = re.search(r"```(?:python)?\n(.*?)```", text, re.S)
return match.group(1) if match else text
def passes(code, test):
with tempfile.NamedTemporaryFile("w", suffix=".py", delete=False) as f:
f.write(code + "\n" + test)
try:
run = subprocess.run(["python3", f.name], capture_output=True, timeout=30)
return run.returncode == 0
except subprocess.TimeoutExpired:
return False
for model in MODELS:
wins, tokens, seconds = 0, 0, 0.0
for task in TASKS:
text, n, secs = ask(model, task["prompt"])
wins += passes(extract(text), task["test"])
tokens, seconds = tokens + n, seconds + secs
print(model, f"{wins}/{len(TASKS)} passed", f"{tokens / max(seconds, 1):.1f} tok/s")
Fifteen to twenty tasks from your real tickets beat any public leaderboard for your purposes. Include at least two that need a long context, such as refactoring across several files, because a model that passes short puzzles can still lose the thread on a large diff.
Hardware Tiers: Matching Memory to Model #
Two rules explain most local-LLM buying mistakes. First, memory for the weights scales with total parameters. At 4-bit precision each billion parameters costs roughly half a gigabyte, before you add the cache and runtime overhead. Second, speed scales with active parameters in a mixture-of-experts model. A model that activates only a small slice of its weights per token can run far faster than its file size implies, but you still need room to hold every expert.
Context length adds a third cost. The key-value cache grows with every token of history, and coding agents that read many files can burn gigabytes on cache alone. If a model fits your VRAM with no spare room, it will fall over the moment you feed it a real repository. Leave headroom of at least a few gigabytes and test with your longest realistic prompt.
Quantization trade-offs for code
Quantization shrinks a model by storing weights at lower precision, and it hits code harder than it hits chat. A slightly degraded model can still write fluent prose, but one wrong token in a function signature breaks a build. In practice, 4-bit files are the standard compromise for consumer hardware, 8-bit files give a visible accuracy margin at roughly double the memory, and anything below 4-bit deserves a direct test before you trust it with refactors.
Some weights ship already quantized by the maker. OpenAI’s documentation reportedly describes gpt-oss-120b as natively MXFP4-quantized, which means the 80 GB figure reflects the format the model was released in, not a community squeeze. That is a quiet advantage. When the vendor publishes the low-precision format itself, you avoid the quality lottery of third-party conversions. If you do use a community file, pick one from an organization you can identify, compare its hash with the listed value and run your harness on it before it touches real work.
| Hardware tier | Best-fit pick | Main caveat |
|---|---|---|
| Thin laptop or 8 to 12 GB GPU | Smallest Gemma 4 variants | Lower accuracy on multi-file edits |
| 16 GB GPU or 16 GB+ unified memory | gpt-oss-20b | Reasoning tokens slow the first answer |
| 24 to 32 GB GPU | Mid-size Gemma 4 variant or gpt-oss-20b with long context | Check cache room at long contexts |
| 80 GB accelerator or large unified-memory workstation | gpt-oss-120b | Price of the hardware |
| Multi-GPU server | Kimi K3.1 Pro, if the weights are confirmed | Custom license terms and file size |
| No suitable hardware | Hosted API from any vendor | Data leaves your network |
The tier table is guidance, not a measured result. The 16 GB and 80 GB anchors come from OpenAI’s documentation as reported above, and the Gemma 4 mapping assumes the variant sizes listed by third parties. Re-check both against the official model cards. If you need to see where your memory actually goes during a run, a per-process VRAM monitor such as the one in our GPU Taskman write-up shows the cache growing in real time.
Pricing: Free Weights Versus Hosted APIs #
Local weights cost nothing to download and a great deal to run well. A hosted API flips that: no hardware, but a meter that never stops. The table lists what each route costs on the pages we could verify, with the closed-model baseline at the bottom so the comparison stays honest.
| Route | Input per million tokens | Output per million tokens | Notes |
|---|---|---|---|
| gpt-oss-20b or 120b, local | $0 for weights | $0 for weights | Apache 2.0 plus usage policy, no OpenAI API price |
| Gemma 4, local | $0 for weights | $0 for weights | Apache 2.0, commercial use allowed |
| Gemma 4 31B, hosted | $0.14 | $0.40 | One tracker, provider prices move |
| Kimi K3.1 Pro, standard API | $2 | $10 | Cached input $0.20 |
| Kimi K3.1 Pro Highspeed | $4 | $20 | Faster serving tier, reported November 4 to 6 |
| DeepSeek V4 family, API | Peak and off-peak | Peak and off-peak | Not listed |
| GPT-6.1 Terra, closed API | $1 | $5 | Cached input $0.05, OpenAI pricing page |
Two readings stand out. The closed GPT-6.1 Terra undercuts Kimi K3.1 Pro’s standard API on both input and output, so a local or open route cannot win on token price alone against a frontier vendor at current rates. The savings case only holds when volume is high, when hardware is already owned, or when the data cannot leave the building. And the Gemma 4 hosted price shows how cheap small open models have become on someone else’s GPUs, which is the real competitor to your own box.
A break-even sketch without invented numbers
Take your own monthly token volume from your provider dashboard and multiply it by the per-million prices above. Then add the monthly cost of the hardware you would buy, spread over a two to three year life, plus the electricity it draws. If the API bill is lower, stay on the API. If the API bill is higher and the model quality is acceptable on your test harness, local wins on money. We deliberately skip a worked example with invented volumes. Your numbers decide this one.
Market data suggests the pricing pressure will continue. Gartner’s 4Q25 forecast for generative AI models, listed in the Gartner research document, projects the generative AI models market growing from $26 billion in 2026 to $76 billion by 2029. A market growing that fast attracts more hosts, and more hosts push per-token prices of open models down. That trend favors renting open-weight models first and buying hardware only when usage is steady.
Six Real-World Examples From the Last Few Weeks #
Abstract advice only goes so far. These six cases, all from the past ten weeks, show how the choice plays out.
- A hosting provider pulls models overnight. DigitalOcean’s November 10 note removed Gemma 4 from serverless inference and, per a summary of the notice, did the same for Qwen 3.5 397B A17B, Llama 4 Maverick and DeepSeek V4 Flash. Teams on those endpoints had a migration to run. Teams with local weights had nothing to do.
- An API alias changes under you. When DeepSeek launched V4.1 Flash, its change log shows the old deepseek-v4-flash identifiers were temporarily routed to the new model. Output changed without a code change on the customer side. Pinning a local file prevents that class of surprise.
- A hosted price moves in a week. Trackers reported price increases on Gemma 4 26B A4B at some providers in early November. A budget built on the September rate would have been wrong by November. Local cost moves only when your electricity bill does.
- A harness ships four releases in seven days. Qwen Code went from 0.26.4 to 0.26.7 in the week of November 5. A team that auto-updates its coding agent may find a changed default on Monday morning. Pin the version, test, then upgrade on purpose.
- A license clause appears on a trillion-parameter model. The reported Kimi K3.1 Pro terms require a separate agreement for a model-as-a-service business above $20 million in revenue over 12 months. A startup reselling inference should read that before it builds, not after.
- An offline requirement meets an OpenAI-lineage model. A team in a restricted network wants reasoning behavior similar to what it uses in the cloud. OpenAI states that gpt-oss is not served through ChatGPT or its API, so the downloadable weights are the only route. That is exactly the gap gpt-oss fills.
Pair any of these models with a local agent and the result is a private coding assistant. The Goose setup with Ollama and the Aider workflow with Ollama and Git both work with an OpenAI-compatible local endpoint, so the model choice here carries straight over.
Use-Case Recommendations: Which Model for Which Developer #
The right pick changes with the job. These six profiles cover most readers.
Solo developer on a 16 GB laptop or GPU
Start with gpt-oss-20b. OpenAI’s reported 16 GB footprint, Apache 2.0 license and broad runtime support make it the lowest-friction option. Expect slower first tokens than a non-reasoning model. If latency annoys you, try a mid-size Gemma 4 variant and compare on your harness.
Small team sharing one workstation
Run gpt-oss-120b on an 80 GB-class machine behind vLLM or llama.cpp’s server, and let the team point editors and agents at one endpoint. A shared box makes the hardware cost easier to justify, and it lets you rate-limit and log usage centrally.
Regulated or air-gapped environment
Choose by license and provenance first. Apache 2.0 weights from gpt-oss or Gemma 4 carry the simplest compliance review. Download from the official repositories, record file hashes, and keep the runtime version in your change log. Skip any model whose terms you cannot read in full.
Startup building a product on open weights
Prefer plain Apache 2.0. The gpt-oss usage policy adds a condition worth reading, and Kimi’s reported revenue threshold adds a contract to the roadmap. Gemma 4 is the cleanest of the group for a commercial product, subject to the same checks.
Team with GPU cluster access and long-horizon tasks
Test Kimi K3.1 Pro through a hosted endpoint first. Moonshot positions it for long-horizon coding and agent work with a 1 million-token context window. If it wins on your tasks and the weights are confirmed, plan a multi-accelerator deployment and get legal sign-off on the license.
Student or hobbyist on modest hardware
Start with the smallest Gemma 4 variants, accept lower accuracy, and use the model for explanations and small functions rather than multi-file refactors. When you outgrow it, a hosted Gemma 4 endpoint costs cents per million tokens.
Migration Guide: Moving From a Hosted API to a Local Model #
Most teams do not start from zero. They already call a hosted model from an editor plugin, a script or a CI step, and they want to swap the backend without rewriting everything. Because Ollama, llama.cpp’s server and vLLM all speak the OpenAI chat format, the swap is mostly configuration. Follow these seven steps in order.
Step 1 to 3: Inventory, shortlist, install
First, list every place your code calls a hosted model, with the model name, average prompt size and monthly token volume. Second, shortlist two models from the tier table above based on your hardware. Third, install a runtime. Ollama is the quickest start. For a lower-level setup, our llama.cpp walkthrough for running an LLM locally covers building the server and choosing quantizations.
ollama pull gpt-oss:20b
ollama run gpt-oss:20b
curl http://localhost:11434/v1/models
Step 4 to 5: Point your tools at the local endpoint, then verify files
Most OpenAI-compatible clients read a base URL and an API key from environment variables. Set the base URL to your local server and use any placeholder key. Then check what you downloaded. Pull weights only from the maker’s official organization, record the file hash, and prefer formats that cannot run code on load. A poisoned model file is a supply-chain risk like any other dependency.
export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=local
sha256sum path/to/model-file
Step 6 to 7: Score, then cut over gradually
Run the harness from the benchmark section against both the hosted model you use today and the local candidate. Route low-risk work first, such as docstrings, test scaffolding and commit messages, and keep the hosted model for hard refactors until the numbers justify the switch. A hybrid setup is a legitimate end state. Many teams keep a local model for private repositories and a hosted frontier model for everything else. If you want a wider field of candidates before you commit, our broader open-source LLM ranking lists more options.
Pros and Cons of Going Local for Coding #
Pros
- Source code never leaves your machine or network, which simplifies privacy and compliance reviews.
- The model behaves the same tomorrow as today because you control the file and the runtime.
- Running locally means no hosting provider can deprecate your endpoint, unlike the hosted models DigitalOcean deprecated from serverless inference on November 10.
- Apache 2.0 weights from OpenAI and Google carry clear commercial terms.
- Marginal cost per token is close to zero once the hardware exists.
- Offline work is possible, which helps on planes, in secure labs and during provider outages.
Cons
- Hardware costs money up front, and memory is the scarce resource.
- Frontier closed models such as GPT-6.1 Terra remain a stronger bet for the hardest tasks, at a price of $1 and $5 per million tokens.
- Reasoning models can feel slow on consumer hardware.
- You own patching, runtime upgrades and file verification.
- The largest open models, Kimi K3.1 Pro among them, need server-class infrastructure and carry extra license terms.
- Public benchmarks do not transfer cleanly, so you must test on your own code.
Verdict: Which Local Coding Model to Pick Today #
Pick by hardware first. For 16 GB-class machines, gpt-oss-20b is the best-documented choice we can verify, with an Apache 2.0 license and official guidance on a small footprint. For an 80 GB-class workstation, gpt-oss-120b is the next rung. Gemma 4 takes the crown for license cleanliness and for covering the most hardware sizes, and it is the right default for products and for thin laptops. Kimi K3.1 Pro is a trillion-parameter-class model for clusters, and its open-weight status still needs confirmation from Moonshot’s own pages.
The data behind that call is modest but solid. Two Apache 2.0 families have official pages. Hosted Gemma 4 31B costs $0.14 and $0.40 per million tokens on one tracker, Kimi’s API costs $2 and $10, and the closed GPT-6.1 Terra costs $1 and $5. No independent benchmark compares them on identical hardware, so the final selection should come from your own harness run. Watch three things over the coming weeks: confirmation of the Kimi weights and license, a verified DeepSeek model card for V4.1 Flash, and a clear Qwen model release. Any of them could reshuffle this list. For the broader category, the AI and machine learning section tracks each release as it lands.
Frequently Asked Questions #
What is the best local LLM for coding right now?
For most developers, gpt-oss-20b on a 16 GB-class machine or gpt-oss-120b on an 80 GB-class workstation, based on OpenAI’s reported memory figures and Apache 2.0 licensing. Gemma 4 is the alternative when license simplicity or a small variant matters most. Confirm the choice with your own test tasks.
Can I run Kimi K3.1 Pro on my own computer?
Almost certainly not on a consumer machine. Moonshot describes the model at the 3 trillion-level, so it needs multi-accelerator server hardware, and Moonshot’s official pages in our fetch did not yet confirm a public checkpoint.
Is gpt-oss available in ChatGPT or the OpenAI API?
No. OpenAI’s help center describes gpt-oss-120b and gpt-oss-20b as downloadable weights for self-hosted or third-party-hosted inference, and states they are not served through ChatGPT or its API.
Is Gemma 4 free for commercial use?
Google’s announcement and two other sources report an Apache 2.0 license that includes commercial use. Read the license file shipped with the specific model variant you download before you build a product on it.
How much memory do I need to run a coding model locally?
A rough rule is half a gigabyte per billion parameters at 4-bit precision, plus cache for your context length and some runtime overhead. OpenAI reportedly cites about 16 GB for gpt-oss-20b and 80 GB for gpt-oss-120b. Always leave headroom for long prompts.
Why are DeepSeek, Qwen and Llama not ranked?
The sources we could verify today did not establish current model cards, licenses or file sizes for local use. DeepSeek V4.1 Flash is well documented as an API product, Qwen Code is a verified tool, and Llama has no verified new release. Each will join the ranking once its weights and terms are confirmed.
Is a local model cheaper than an API?
Only at high volume, on hardware you already own, or when privacy rules out the cloud. Closed GPT-6.1 Terra lists $1 per million input tokens and $5 per million output tokens, and hosted Gemma 4 31B is cheaper still on one tracker. Compare your own monthly token count against hardware and power costs.