How to Run Carbon-X Locally for DNA Analysis on One GPU Hugging Face Bio's Carbon-X, a 14.8-billion-parameter open-weight DNA language model, can be self-hosted on a single 80GB GPU in bfloat16, keeping unpublished genomic data on local infrastructure. The model averages 66.7% across five DNA benchmarks with thinking disabled, nearly matching GPT-6.1 Sol's 67.5% at roughly 1/1,800th the cost per problem, and requires Transformers v5 or later with trust_remote_code=True for its custom hybrid tokenizer that pairs BPE for English text with non-overlapping 6-mers for DNA. vLLM is the recommended serving path, exposing an OpenAI-compatible endpoint, and the guide advises keeping enable_thinking=False since that is how the model was post-trained and benchmarked. How to Run Carbon-X Locally for DNA Analysis on One GPU A setup guide for self-hosting Carbon-X, the 14.8B DNA language model, on a single 80GB GPU using vLLM or Transformers for local inference. What is Carbon-X and why run it locally? Carbon-X is a 14.8-billion-parameter open-weight language model from Hugging Face Bio that reads and writes DNA sequences inside normal chat conversations. It predicts whether mutations are benign or pathogenic, flags regulatory regions, classifies sequences by organism, and predicts upcoming bases, all while explaining its answers in plain English. Because the full model fits in bfloat16 on a single 80GB GPU, labs and researchers can run it entirely on their own hardware instead of sending unpublished genomic data to a third-party API. TL;DR - Carbon-X runs on one 80GB GPU in bfloat16, which means sensitive or unpublished DNA sequences never have to leave your own infrastructure. - The model averages 66.7% across five DNA benchmarks with thinking disabled, nearly matching GPT-6.1 Sol 67.5% while costing roughly 1/1,800th as much per problem. - It uses a hybrid tokenizer : standard BPE for English text and non-overlapping 6-mers for DNA, unified in a single token stream with no separate DNA encoder. - You need Transformers v5 or later and trust remote code=True when loading the tokenizer, since the DNA tokenization logic is custom code. - vLLM is the recommended serving path for production use, exposing an OpenAI-compatible endpoint with one command. - Thinking should stay disabled enable thinking=False , since that’s how the model was post-trained and benchmarked, and it keeps output token counts and cost low. - DNA tagging is automatic , but only A/C/G/T form full-resolution 6-mers; ambiguity codes like N can degrade token efficiency if not filtered first. Seven tools to build an app. Or just Remy. Editor, preview, AI agents, deploy — all in one tab. Nothing to install. What hardware do you actually need? Carbon-X’s weights fit on a single 80GB GPU an H100 or H200-class card in bfloat16 precision. That’s the headline spec in the model card, and it’s what makes local deployment realistic for a university lab or a biotech startup rather than something that requires a multi-node cluster. The model card’s own cost modeling uses an H200 at $5/hour as its reference point for throughput, so that’s a reasonable baseline to plan around if you’re estimating compute budgets. There’s no official quantized release mentioned in the documentation, so if you’re working with a smaller GPU 40GB or less , you’d be quantizing it yourself and should expect some accuracy loss on the benchmarks below. For most self-hosting scenarios aimed at keeping genomic data private, the single 80GB card in full bfloat16 is the tested, supported path. How do you set up Carbon-X with Transformers? The basic Python setup mirrors any modern Hugging Face causal language model, with two specifics worth calling out: you need Transformers v5 or later , and the tokenizer requires trust remote code=True because Carbon-X ships custom tokenizer code to handle the DNA 6-mer logic. pip install -U transformers python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model id = "HuggingFaceBio/Carbon-X" tokenizer = AutoTokenizer.from pretrained model id, trust remote code=True model = AutoModelForCausalLM.from pretrained model id, dtype=torch.bfloat16, device map="cuda" From there, you build a chat message containing both the reference and variant sequences or just a raw sequence and a question , apply the chat template with enable thinking=False , and call generate . The model’s generation config.json sets sampling defaults of temperature 0.6, top-p 0.95, and top-k 20, which are also the exact settings used to produce every benchmark number in the model card. If you change them, expect results to drift from published performance. One operational detail to plan for: on first load, the tokenizer needs Hub access to download the pinned BPE vocabulary it’s built on. If you’re running in a fully air-gapped environment, you’ll need to cache that vocabulary locally ahead of time. How do you serve Carbon-X with vLLM? For anything beyond single-shot experimentation, vLLM is the documented path for production serving. It turns Carbon-X into an OpenAI-compatible API endpoint running locally: pip install -U vllm vllm serve HuggingFaceBio/Carbon-X --trust-remote-code Once the server is up, you query it like any OpenAI-style chat endpoint: python from openai import OpenAI client = OpenAI base url="http://localhost:8000/v1", api key="EMPTY" response = client.chat.completions.create model="HuggingFaceBio/Carbon-X", messages= {"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"} , extra body={"chat template kwargs": {"enable thinking": False}}, print response.choices 0 .message.content vLLM applies the same sampling defaults from generation config.json automatically, unless you override them in the request body. This setup is also what you’d wrap in a function if you want a general-purpose coding or research agent to call Carbon-X specifically for DNA-related sub-questions, passing it named sequences and a question rather than routing genomic data through a general-purpose model. What’s going on with the tokenizer, and where do people get tripped up? - ✕a coding agent - ✕no-code - ✕vibe coding - ✕a faster Cursor The one that tells the coding agents what to build. Carbon-X doesn’t use a separate DNA encoder. Instead, it extends a standard BPE vocabulary inherited from its base model, Qwen3-14B with 5,515 additional DNA-specific tokens: 4,096 non-overlapping 6-mers, 1,364 “tail” tokens for sequences whose length isn’t a clean multiple of six, and three special markers