cd /news/large-language-models/running-claude-code-with-a-local-llm · home topics large-language-models article
[ARTICLE · art-10151] src=gist.github.com ↗ pub= topic=large-language-models verified=true sentiment=· neutral

Running Claude Code with a local LLM

The article provides instructions for running Claude Code using a local large language model (LLM) instead of Anthropic's cloud-based models. It recommends downloading specific quantized Qwen3.6 models from the MLX community, such as the 35B or 27B parameter versions, with RAM requirements ranging from 36GB to 64GB+. The guide details configuration steps including enabling TurboQuant KV Cache, adjusting context and token limits, and adding a specific environment variable to disable attribution headers in the settings file.

read1 min views24 publishedApr 21, 2026

https://github.com/jundot/omlx/releases Go to model down Multiple options, depending on your RAM 35B parameters with 3 billion active:

unsloth/Qwen3.6-35B-A3B-UD-MLX-3bit
- 17.4 GB (36GB+ RAM ideal)unsloth/Qwen3.6-35B-A3B-UD-MLX-4bit
- 21.6 GB (48GB+ RAM ideal)unsloth/Qwen3.6-35B-A3B-MLX-8bit
- 37.7GB GB (64GB+ RAM ideal)

27B billion parameters

unsloth/Qwen3.6-27B-UD-MLX-4bit
- 26.2GB (48GB+ RAM ideal)unsloth/Qwen3.6-27B-UD-MLX-6bit
- 30.5GB (64GB+ RAM ideal)unsloth/Qwen3.6-27B-UD-MLX-8bit
- 34.7GB (64GB+ RAM ideal)
  • Go to model settings
  • Pin and default model to the downloaded one
  • Open the model's settings
  • Enable TurboQuant KV Cache in3.5-bit
  • Go to global settings
  • Turn on Fallback to Default Model
  • Set Hot Cache Limit (In-Memory Cache) to 10% - Set Cold Cache Limit (SSD Cache) to 10% - Increase Max Context Window to256000
  • Increase Max Tokens to64000
  • Save

Add "CLAUDE_CODE_ATTRIBUTION_HEADER": "0" inenv key inside~/.claude/settings.json

(Ref)Example:
{ "env": { "CLAUDE_CODE_ATTRIBUTION_HEADER": "0" } }
── more in #large-language-models 4 stories · sorted by recency
── more on @claude code 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/running-claude-code-…] indexed:0 read:1min 2026-04-21 ·