Tencent Releases Hy3: An Open 295B Mixture-of-Experts (MoE) Model with 21B Active Parameters and 256K Context

Tencent released Hy3, an open-source 295B-parameter Mixture-of-Experts model with 21B active parameters and 256K context length, under the Apache License 2.0. The model achieves state-of-the-art results on coding and reasoning benchmarks, including 78.0 on SWE-Bench Verified and 90.4 on GPQA Diamond, and is designed for agentic workflows and long-context tasks.

Tencent’s Hy team released Hy3. https://hy.tencent.com/research/hy3 Hy3 is a 295B-parameter Mixture-of-Experts MoE model. It activates only 21B parameters per token. The weights ship under the Apache License 2.0 . Hy3 is aimed at reasoning, agentic workflows, and long-context tasks. What is Hy3? Hy3’s architecture contains a sparse MoE with 192 experts and top-8 routing. Only 8 experts fire per token, so compute stays low. The model also uses a Multi-Token Prediction MTP layer. MTP predicts several tokens at once for faster decoding. Both vLLM and SGLang enable it through speculative decoding. | Property | Value | |---|---| | Architecture | Mixture-of-Experts MoE | | Total parameters | 295B | | Activated parameters | 21B | | MTP layer parameters | 3.8B | | Layers excluding MTP | 80 | | MTP layers | 1 | | Attention heads | 64 GQA, 8 KV heads, head dim 128 | | Hidden size | 4096 | | Intermediate size | 13312 | | Context length | 256K | | Vocabulary size | 120832 | | Experts | 192 experts, top-8 activated | | Supported precisions | BF16 | A separate Hy3-FP8 checkpoint is also released. FP8 lowers the memory footprint for cheaper serving. Benchmark and Performance The research team published scores across coding, agents, and STEM. On coding, Hy3 reports 78.0 on SWE-Bench Verified. It also reports 57.9 on SWE-Bench Pro and 75.8 on SWE-Bench Multilingual. Terminal-Bench 2.1 lands at 71.7, and DeepSWE at 28.0. On STEM and reasoning, the numbers climb higher. Hy3 reports 90.4 on GPQA Diamond and 72.0 on USAMO 2026. IMOAnswerBench reaches 90.0, and HLE with tools reaches 53.2. The research team ran a blind test with 270 experts. That test collected 312 valid comparisons on real workflows. Hy3 scored 2.67 out of 4, ahead of GLM-5.1 at 2.51. The edge was clearest in frontend development, CI/CD, and data and storage. Reliability and Production Behavior The research team focused much of this release on production reliability. Three failure modes got direct attention, backed by internal numbers. Tool calling and output formatting : The team fixed baseline stability issues that broke agents. Invalid calls that trigger infinite loops dropped. Hy3 also generalizes across agent scaffoldings. On SWE-Bench Verified, accuracy variance across CodeBuddy, Cline, and KiloCode stays within 4%. World knowledge and anti-hallucination : The target behavior is simple: answer when grounded, flag when evidence is missing. In internal evaluations, the hallucination rate fell from 12.5% to 5.4%. Commonsense error rates fell from 25.4% to 12.7%. Multi-turn intent tracking : Joint SFT and RL improved coreference and constraint tracking. The internal issue rate dropped from 17.4% to 7.9%. On the MRCR long-dialogue benchmark, scores rose from 42.9% to 75.1%. How to Call Hy3 Hy3 exposes an OpenAI-compatible API. You deploy it with vLLM or SGLang, then call the endpoint. One flag, reasoning effort , controls how much the model thinks. python from openai import OpenAI client = OpenAI base url="http://127.0.0.1:8000/v1", api key="EMPTY" response = client.chat.completions.create model="hy3", messages= {"role": "user", "content": "Refactor this function and explain the change."}, , temperature=0.9, top p=1.0, reasoning effort: "no think" default , "low", "high" deep chain-of-thought extra body={"chat template kwargs": {"reasoning effort": "high"}}, print response.choices 0 .message.content Use no think for direct answers, and high for math, coding, or multi-step tasks. Tencent research team recommends temperature=0.9 and top p=1.0 . You can also try Hy3 without local hardware. OpenRouter lists a tencent/hy3:free route at $0 per token. That free tier is scheduled to end on July 21, 2026. Where Hy3 Fits: Use Cases Hy3 is built around agent-style, long-context work. A few concrete examples: Coding agents : Feed a full repository into the 256K window. Ask Hy3 to fix a failing test with reasoning effort="high" . Stable tool calls help it run edits across many files. Document processing : Pass a long contract or filing as context. The anti-hallucination training reduces fabricated clauses and misquotes. Financial analysis : Combine tables and prose in one prompt. Ask for a grounded summary that flags missing data rather than guessing. Frontend and game development : Generate a React component or a small game loop. The blind test showed a frontend advantage over GLM-5.1. Hy3 vs GLM-5.2 Tencent’s research team benchmarked Hy3 against GLM-5.2 in its own appendix. GLM-5.2 is roughly a 744B MoE with about 40B active parameters. Hy3 is less than half that total size, with 21B active. On coding, GLM-5.2 leads across the suite. | Benchmark | Hy3 21B active | GLM-5.2 ~40B active | |---|---|---| | SWE-Bench Verified | 78.0 | 84.2 | | SWE-Bench Multilingual | 75.8 | 83.0 | | Terminal-Bench 2.1 | 71.7 | 81.0 | | DeepSWE | 28.0 | 46.2 | | Total / active params | 295B / 21B | ~744B / ~40B | | License | Apache 2.0 | Open weights | The focus here is about size, not just score. Hy3 trades some coding accuracy for a far smaller active footprint. That footprint matters when you self-host and pay for GPUs. Deployment Notes Hy3 has 295B total parameters, so serving needs real memory. Tencent’s research team recommends 8 GPUs, such as the H20-3e or cards with larger memory. vLLM and SGLang both ship recipes with MTP enabled. A minimal vLLM launch looks like this: vllm serve tencent/Hy3 \ --tensor-parallel-size 8 \ --speculative-config.method mtp \ --speculative-config.num speculative tokens 2 \ --tool-call-parser hy v3 \ --reasoning-parser hy v3 \ --enable-auto-tool-choice \ --port 8000 \ --served-model-name hy3 For compression, The research team points to its AngelSlim toolkit. AngelSlim covers quantization, low-bit methods, and speculative sampling. Tencent also provides a complete finetuning pipeline for Hy3. Try It: Interactive Explorer The demo below is an interactive explorer for Hy3. It visualizes MoE routing, reasoning modes, benchmarks, and sparse efficiency. Check out the Technical details, Model weight on HF and . GitHub Repo https://github.com/Tencent-Hunyuan/Hy3 Also, feel free to follow us on and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://www.aidevsignals.com/ now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58