Today we are introducing our second model generation with Inkling 2 Small. It is a Mixture-of-Experts model with 594B total parameters, 17B active, and supports image inputs and variable effort.
Inkling 2 Small far surpasses our previous models’ scores on the Artificial Analysis Intelligence Index with a score of 31, reflecting gains in performance on agentic coding, vision, knowledge work, and reasoning. The increase in capabilities comes from improvements made at every step of our training process, including more efficient pre-training and a large, unified RL run.
Inkling 2 Small will initially be available through Tinker, with closed weights, following the approach we outlined in a safe path to open weights. We intend to make the full weights available under an Apache 2.0 license next week after the completion of external safety testing and red-teaming.
You can try Inkling 2 Small on Tinker Playground today. The model is also available for fine-tuning and sampling on Tinker; checkpoints will be downloadable with the release of Inkling 2 Small’s weights.
Capabilities #
Inkling 2 Small is more capable across the board than our first generation, particularly in long-horizon autonomous work.
Inkling 2 Small’s vision and agentic capabilities enable it to incorporate visual feedback to iteratively build and refine websites, games, and 3D designs with interactive elements.
Agentic coding and autoresearch
Inkling 2 Small was trained with RL on hundreds of thousands of long-horizon coding tasks across multiple agent harnesses and context-compaction strategies. We increased the quality of our synthetic coding data pipeline and moved beyond unit-test rewards alone; many tasks included an agentic RL grader that evaluated how the model approached the problem, not just whether its code passed. We also tested early checkpoints of Inkling 2 Small on OpenCode under the name Fledge-Alpha to assess our model’s performance on coding tasks in the real world.
The result is a model that is both stronger at coding and easier to use inside real agent loops. Inkling 2 Small improves on usability compared to Inkling: it follows instructions better, considers changes carefully, and verifies its work without being asked.
| Coding and Autoresearch | Inkling 2 Small | GLM-5.3-Flash | DeepSeek-V4.1-Flash-1017 | Muse Spark Open | Kimi K3 |
|---|---|---|---|---|---|
| FrontierCode v2.0 | 25.4% (Second-highest reported score in this row) | 14.2% | 28.5% (Highest reported score in this row) | 7.6% | 17.3% |
| DeepSWE v1.1 | 72.1% (Second-highest reported score in this row) | 63.4% | 77.0% (Highest reported score in this row) | 60.0% | 68.5% |
| SWEBench Pro v2 (Hard) | 86.3% (Second-highest reported score in this row) | – | – | 82.4% | 88.2% (Highest reported score in this row) |
| Vibe Code Bench v1.1 | 81.6% | 30.8% | 87.7% (Highest reported score in this row) | 76.2% | 85.0% (Second-highest reported score in this row) |
| PostTrainBench v1.2 | 31.9% (Second-highest reported score in this row) | 36.8% (Highest reported score in this row) | 30.5% | – | 31.5% |
| Terminal Bench Science v0.1 | 5.7% | 4.8% | 12.9% (Highest reported score in this row) | 1.9% | 8.1% (Second-highest reported score in this row) |
Kernel optimization (PMPP-Hard) 69 tasks · Four independent attempts · 10-hour budget
Hover a curve or select a model to inspect; select it again to reset. With a model focused, use ← and → to step through results, or Escape to clear.
View full results #
| PMPP-Hard results | |||
|---|---|---|---|
| Model | AUC (task-hours) | Mean solved / 69 | Pass@4 / 69 |
| --- | --- | --- | --- |
| GPT-6 Astra | 616.39 | 63 | 63 |
| Kimi K3.1 Pro | 583.86 | 60.5 | 63 |
| Claude Opus 5.5 | 499.65 | 60 | 64 |
| DeepSeek V4.1 Flash | 497.36 | 55.25 | 62 |
| Inkling 2 Small | 494.27 | 54.25 | 64 |
| Mistral Large 4 | 478.39 | 50.25 | 61 |
| Kimi K3 | 453.58 | 55.75 | 66 |
| GLM 5.3 | 430.96 | 52.5 | 61 |
| GLM 5.2 | 389.12 | 42 | 61 |
Rows are ordered by AUC: the area under the mean curve over 0–10 hours, with a maximum of 690 task-hours.
Each model uses its maximum available reasoning setting, the PI harness and a shared compaction policy. We include fixes for task and grading issues identified during evaluation. Each submitted solution receives two independent LLM audits, with a third review resolving disagreements.
PMPP-Hardis an external GPU kernel-optimization benchmark, adapted here for consistent cross-model evaluation. Curves show the mean across four independent attempts; legend values are mean tasks solved at 10 hours.
Beyond conventional software tasks, Inkling 2 Small was also trained on long-horizon autoresearch tasks. This included agentic GPU kernel optimization and ML tasks such as post-training a model autonomously, all run across a range of GPU architectures.
To demonstrate its end-to-end autoresearch performance, we tested Inkling 2 Small on a task Proximal built for one of our internal benchmarks. Inkling 2 Small autonomously trained Qwen3.5-4B on Tinker to predict various time series such as weather and Wikipedia page views. It started by analyzing the tasks and writing its own evaluator, experimented with training datasets and recipes, and tried various approaches to SFT and RL, while debugging the process as it goes. After 4.5 hours of work and $426 in Tinker spend, the trained model has more than doubled its forecasting score.
Vision interpretation and creation
Inkling 2 Small processes image inputs using a 292M-parameter vision encoder based on Perception EncoderBolya et al., 2025.. We use 20×20-pixel patches with 2×2 spatial pooling, and interleave image patches with text tokens. The model demonstrates strong visual capabilities across multiple categories, including OCR, grounding, infographic interpretation, and visual reasoning.
| Visual reasoning and creation | Inkling 2 Small | GLM-5.3-Flash | DeepSeek-V4.1-Flash-1017 | Muse Spark Open | Kimi K3 |
|---|---|---|---|---|---|
| GDP.pdf | 25.2% (Highest reported score in this row) | 15.4% | 17.2% | 18.6% | 22.0% (Second-highest reported score in this row) |
| MMMU Pro (Standard 10) | 81.6% (Highest reported score in this row) | 79.0% | 80.6% | 80.9% (Second-highest reported score in this row) | 80.5% |
| PerceptionBench | 64.7% (Highest reported score in this row) | – | 47.7% | – | 58.5% (Second-highest reported score in this row) |
| Humanity’s Sixth Sense | 33.3% (Highest reported score in this row) | 26.8% | 30.1% (Second-highest reported score in this row) | 29.6% | 25.5% |
| ScreenSpot Prooriginal / with Python | 83.8/91.5% (Highest reported score in this row) | – | – | – | 73.8/83.2% (Second-highest reported score in this row) |
| BabyVisionoriginal / with Python | 68.6/93.7% (Highest reported score in this row) | 53.4/82.6% (Second-highest reported score in this row) | –/92.5% | 44.3/80.2% | 53.4/85.7% (Second-highest reported score in this row) |
| Charxiv RQoriginal / with Python | 85.6/87.7% (Highest reported score in this row) | 80.2/89.4% | 77.0/85.6% | 85.4/88.7% (Second-highest reported score in this row) | 84.8/91.3% |
| Chartographyoriginal / with Python | 36.7/80.4% (Highest reported score in this row) | 16.3/78.0% | 17.9/83.0% | 35.7/63.2% (Second-highest reported score in this row) | 26.6/68.1% |
| Code Arena (WebDev) | 1653 | 1607 | 1665 (Highest reported score in this row) | – | 1656 (Second-highest reported score in this row) |
| AA-3D | 1021 (Highest reported score in this row) | 937 | – | – | 982 (Second-highest reported score in this row) |
| Code2Video-AA | 1020 (Second-highest reported score in this row) | 975 | – | – | 1041 (Highest reported score in this row) |
The model uses vision in the loop to interpret and validate the visual coding output it generates. Pending the release of its weights, it is set to be the top open-weights model on DesignArena.
We even tested Inkling 2 Small against some of our researchers in a wine tasting event. It outperformed them using vision alone, correctly identifying three wines from a single picture in 19 out of 20 trials.
Agentic knowledge work
Inkling 2 Small accelerates knowledge work in realistic professional environments. It proficiently looks up information, calls tools over MCP, and produces high-quality deliverables. On AA-Briefcase, an end-to-end knowledge work benchmark, these capabilities come together for an ELO score of 1527.
| Knowledge Work and Enterprise Ops | Inkling 2 Small | GLM-5.3-Flash | Muse Spark Open | DeepSeek-V4.1-Flash-1017 | Kimi K3 |
|---|---|---|---|---|---|
| AA-Briefcase v1.1 | 1527 (Second-highest reported score in this row) | 1451 | 1279 | 1570 (Highest reported score in this row) | 1497 |
| GDPVal-AA v2.1 | 1630 | 1649 (Second-highest reported score in this row) | 1401 | 1746 (Highest reported score in this row) | 1543 |
| JobBench | 70.5% (Highest reported score in this row) | 60.7% | 61.8% (Second-highest reported score in this row) | – | 54.3% |
| Vals Legal Index | 73.9% (Highest reported score in this row) | 71.1% | 68.5% | 71.9% (Second-highest reported score in this row) | 71.7% |
| EnterpriseBench-CoreCraft | 71.8% (Highest reported score in this row) | 46.7% (Second-highest reported score in this row) | – | – | 43.1% |
Inkling 2 Small is especially strong in finance and quantitative reasoning over spreadsheets, filings and documents. Among open weight models, Inkling 2 Small ranks first on AA-AnalystAgent and Dayjob:Finance, and second on Vals Finance Index and DiligenceBench.
| Finance Work | Inkling 2 Small | GLM-5.3-Flash | Muse Spark Open | DeepSeek-V4.1-Flash-1017 | Kimi K3 |
|---|---|---|---|---|---|
| Vals Finance Index | 61.5% (Highest reported score in this row) | 55.8% | 53.0% | 60.9% (Second-highest reported score in this row) | 58.0% |
| Diligence BenchBest harness | 58.1% (Second-highest reported score in this row) | – | 59.0% (Highest reported score in this row) | 50.4% | – |
| DAYJOB (Finance) | 14.3% (Highest reported score in this row) | 7.3% | – | 12.5% (Second-highest reported score in this row) | 5.5% |
| Apex-Accounting v1.1pass@1 | 14.5% (Highest reported score in this row) | 12.3% | 13.8% (Second-highest reported score in this row) | 12.5% | 9.5% |
Customizability
Some models respond better to fine-tuning, independent of their out-of-the-box performance. We wanted Inkling 2 Small to be a strong base model for customization.
Inkling 2 Small learns faster and improves more than our previous models when trained on the same data and algorithm across a variety of post-training scenarios, which we curated based on customer use cases. For example, starting from similarly low scores on a public financial-document summarization task, Inkling 2 Small adapts quickly, reaching the final performance of Inkling Small with just 75 training steps and finishing substantially ahead.
Reasoning, knowledge, and epistemics
Inkling 2 Small is trained on challenging STEM problems, instruction following, information seeking with and without web search, and forecasting tasks. During RL, we use proper scoring rules for forecasting tasks and uncertainty-sensitive rewards on some question-answering tasks. This trained the model to assess uncertainty, express appropriate confidence, and make well-calibrated predictions.
| Long Context Reasoning and Factuality | Inkling 2 Small | GLM-5.3-Flash | Muse Spark Open | Mistral Large 4 | Kimi K3 |
|---|---|---|---|---|---|
| HLE-Diamond | 28.7% (Highest reported score in this row) | 18.2% | 26.3% | 17.3% | 26.9% (Second-highest reported score in this row) |
| CritPt v2Full / Hard | 55.7/35.0% (Highest reported score in this row) | –/24.5% | 52.6/29.5% (Second-highest reported score in this row) | 50.3/27.0% | –/40.5% |
| AA-LCR v1.1 | 83.3% (Second-highest reported score in this row) | 80.0% | 78.3% | – | 88.7% (Highest reported score in this row) |
| LongBench v2 | 70.6% (Highest reported score in this row) | 68.2% | 68.7% | 65.7% | 69.0% (Second-highest reported score in this row) |
| AA-Omniscience v1.1score / non-hallucination rate | 16.1/73.3 (Second-highest reported score in this row) | 0.7/– | 25.8/72.0 (Highest reported score in this row) | 5.1/71.8 | – |
Safety and alignment
Our safety work for Inkling 2 Small centers on two questions: how the model performs for the general user, and how well it resists the removal of safeguards by a malicious actor. The release of Inkling’s weights is pending further safety evaluations by us and our partners.
To assess misuse risks, we trained an internal helpful-only variant. We ran both this model and the release checkpoint through a wider suite of evaluations, with more external testers, than any previous release.
On chemical, biological, radiological, and nuclear (CBRN) risk, Inkling 2 Small’s knowledge and capability scores sit below the open-weight frontier. On FORTRESS, Scale AI’s private benchmark that scores how much harmful assistance a model gives in response to expert-written adversarial prompts across CBRN, terrorism, and nonviolent crime, Inkling 2 Small has a lower Average Risk Score compared with Inkling and Inkling-Small, and also demonstrates a higher Overrefusal Score (see Limitations). The helpful-only variant of Inkling 2 Small exhibits marginal uplift on CBRN knowledge relative to the release checkpoint, but it does not significantly advance the frontier on any dimension we measured.
On offensive and dual-use agentic cyber benchmarks, Inkling 2 Small is a large step up from Inkling Small but remains behind the open-weight frontier. It matches the open-weight frontier on Vals CyberBench v1.1 and PatchEval, both of which measure the ability to find and patch security vulnerabilities.
| Safety | Inkling 2 Small | GLM-5.3-Flash | DeepSeek V4.1 Flash | Muse Spark Open | GLM 5.3 |
|---|---|---|---|---|---|
| FORTRESS (ARS) ↓ | 16.2 (Second-best reported result in this row) | 26.7 | 27.2 | 7.6 (Best reported result in this row) | 28.2 |
| FORTRESS (ORS) ↓ | 20.4 | 3.6 (Second-best reported result in this row) | 6.9 | 16.0 | 3.2 (Best reported result in this row) |
| DistressBench ↑ | 82.0* (Second-best reported result in this row) | 72.7 | 79.8 | 87.7 (Best reported result in this row) | 72.5 | | StrongREJECT ↑ | 97.8 (Second-best reported result in this row) | 96.8 | 98.7 (Best reported result in this row) | 97.6 | 97.2 | | PatchEval ↑ | 79.6 (Second-best reported result in this row) | 74.8 | 80.4 (Best reported result in this row) | 77.8 | 69.6 | | Vals CyberBench ↑ | 72.7 (Second-best reported result in this row) | 67.9 | 73.7 (Best reported result in this row) | – | 72.1 |
DeepSeek combines the supplied Flash results: 1017 for FORTRESS and PatchEval; unversioned Flash for DistressBench, StrongREJECT and Vals CyberBench.
- While we handle mental distress situations well, our model sometimes overreacts with disclaimers in normal emotional support queries as mentioned in the Limitations section. As models take on unsupervised, longer-horizon autonomous work, the potential harm from their misbehavior increases. Without proper attention to alignment, capabilities-focused RL can push models to succeed at any cost. When tasks were difficult, early checkpoints of Inkling 2 Small were persistent in trying to succeed—in some cases fabricating results, ignoring instructions, and concealing what they had done.
We worked across the stack to improve model alignment. In midtraining, we filtered out traces containing misbehavior and included traces with honest, aligned behavior. In RL, we added a general alignment reward that penalizes misbehavior on every task. We also built environments targeting specific failure modes we observed in training, such as rewarding models for giving up on impossible tasks. Finally, we used on-policy self-distillationZhao et al., 2026. from a teacher model prompted with a conduct spec, so the model internalizes principles for good behavior without explicit prompting.
We measured progress with both internal and external evaluations, and found that Inkling 2 Small substantially improves over previous Inkling models on both internal and external evaluations, and outperforms a set of comparable open-weights models on our internal evals.
| Inkling + open weights | Closed weights | ||||||
|---|---|---|---|---|---|---|---|
| Alignment | Inkling 2 Small | Inkling | Inkling Small | Muse Spark Open | GLM-5.3-Flash | DeepSeek V4.1 Flash | GPT-6.1 Sol |
| Misbehavior (Internal) ↓ | 14.9 (Second-best reported result in this row) | 66.8 | 75.2 | 82.4 | 57.1 | 68.0 | 0.8 (Best reported result in this row) |
| Covert misbehavior (Internal) ↓ | 11.8 (Second-best reported result in this row) | 59.4 | 59.1 | 60.8 | 29.2 | 37.6 | 0.8 (Best reported result in this row) |
| Destructive actions (Internal) ↓ | 4.7 (Best reported result in this row) | 71.9 | 67.2 | 7.8 | 25.8 | 10.9 | 7.0 (Second-best reported result in this row) |
| HoneyBench v0.2-alpha: Reward Hacking (Goodhart Labs) ↓ | 44 (Best reported result in this row) | 70 | 69 | – | – | – | 59 (Second-best reported result in this row) |
| Covert Action with No Goal (Apollo Research) ↓ | 8 (Best reported result in this row) | 37 | 36 (Second-best reported result in this row) | – | – | – | – |
| Covert Action with Goal (Apollo Research) ↓ | 0 (Best reported result in this row) | 7 | 4 (Second-best reported result in this row) | – | – | – | – |
Inkling 2 Small still trails closed-weights models, such as GPT-6.1 Sol on our alignment evaluations. Reducing this gap is a priority for future models.
We also worked to secure training and evaluations with two layers of monitoring. The first is synchronous: a monitor inspects tool calls before they run. Each task declares what actions it authorizes. Deterministic rules check calls against the task’s policy and circumvention patterns, escalating uncertain cases to an LLM judge. The second layer is asynchronous: an LLM judge reviews completed rollouts from training and evaluation for misbehavior and actions against our security boundary, raising alerts that we triage and escalate into our incident process. Together, these form a defense-in-depth approach to safety and alignment that we consider critical to developing a safe model factory.
Training Inkling 2 Small #
Inkling 2 Small’s initial pretraining phase (H1) covered 8.4 trillion tokens of a broad mix of natural language, code, and image data. It ran for 68 hours on NVIDIA GB300 NVL72 racks, which enabled high expert parallelism. Training ran natively in NVFP4, the 4-bit floating-point format introduced with the NVIDIA Blackwell GPU architecture. We’ll share more details on our NVFP4 recipe in a follow-up post soon.
We followed this initial phase with a second phase (H2) of 3.8 trillion tokens of higher-quality, newer data the model hadn’t seen, at 512K context. The third and fourth phases mixed in data in the deployment format. The third phase was about 320 billion tokens of reasoning and agent trajectories at 1M context, alongside replayed pretraining data. The fourth phase (H4) kept only the highest-quality data: about 34 billion tokens of agent trajectories. The H3 and H4 mixtures consisted mostly of trajectories sampled from earlier models including Kimi K3 and our own models, filtered for correctness and alignment.
We then carried out a joint RL run, consisting of 1,075 steps and over 17 million rollouts over 9 days. See table below for more details. This single joint run contrasts with the previous Inkling release, where we ran separate reasoning and agentic RL runs and combined them with multi-teacher distillation.
| Scale of the main RL run | |
|---|---|
| Metric | Scale |
| --- | --- |
| Compute | Over 4 × 10²³ FLOPs |
| GPUs | 2,048 |
| Duration | 9 days |
| Rollouts | 17M+ |
| Tokens | 3T+ |
The task mix included software engineering in real repositories, terminal tasks, document creation, STEM question answering, vision, web app building, tool use with simulated users, and safety. Each task family had its own grader:
hidden unit tests and programmatic checks for most coding work; #
rubric-based LLM judges for documents and tool use; #
reference answers for STEM and vision question answering; #
pairwise reward models comparing screenshots of rendered web apps.
| RL training task mix | ||
|---|---|---|
| Category | Rollouts | Tokens |
| --- | --- | --- |
| Coding | 38.5% | 65.0% |
| Office and knowledge work | 9.1% | 13.3% |
| STEM and reasoning | 7.0% | 7.6% |
| Web and app building | 4.7% | 7.2% |
| Vision | 13.9% | 3.0% |
| Tool use and general agents | 17.1% | 2.3% |
| Search | 2.0% | 1.4% |
| Chat, safety and epistemics | 7.8% | 0.2% |
We randomly sampled and conditioned on an effort level, and applied a length penalty in the reward function, which trained the model to trade off speed and thoroughness.
We used asynchronous RL, using a score centering variant—the StableReinforce objective in Tinker—which tolerates significant staleness. Our run combines short and long horizon tasks with p50/p99 total rollout tokens being 43k and 1.5M respectively and p50/p99 duration of 8 minutes and 4 hours respectively. We use 256k context length and enable compaction training for long-horizon tasks. As a result, both training reward and evaluation performance grow log-linearly with compute.
| Stable training | |
|---|---|
| Metric | Value |
| --- | --- |
| Median policy staleness | 17 updates |
| Rollout batch size | 16K |
| Total tokens per rollout | Median: 43K · p99: 1.5M |
| Rollout duration | Median: 8 minutes · p99: 4 hours |
| Context window | 256K tokens, with compaction for longer trajectories |
Making this scale practical required improvements throughout the systems stack. Our RL infrastructure deploys NVFP4 MoE training and sampling. During sampling, prefill, draft, and verify run fully on CUDA graphs. Trainers mirror the pretrain stack with high expert parallelism, EPLB, and CUDA graphs to improve GPU efficiency. To support long horizon agentic workloads, we use sticky routing and DRAM KV-cache off for maintaining 93%+ prefix cache hit rate, allowing us to maintain a 5:3 sampler:trainer ratio despite training at long contexts. Weight transfer is done via GPUDirectRDMA at 93%+ line rate with a parallelism-invariant checksum verifier. We also further lower deployment inference cost by offering a variant of NVFP4 KV cache, which we find to minimally lower evals.
Long-horizon tasks give the model hours of autonomy, increasing potential risks from misaligned behavior. Midway through the main RL run, our asynchronous monitor flagged an instance where the model, failing at a task, worked around a network block through a third-party service it could reach using its search tools. We d the run, hardened our infrastructure, removed environments that required web access, and added synchronous tool-call monitoring. We then resumed the run, incorporating additional training phases specifically for alignment, as described above.
Improvements to our process allow us to train models at different sizes using similar recipes with predictable scaling and performance. Inkling 2 Small is part of a ladder of models; we look forward to sharing more rungs with you soon.
Benchmarking Inkling 2 Small #
Like previous Inkling releases, we benchmarked Inkling-Small on a broad range of capabilities. All evals run at effort 0.99 (xhigh) and temperature 1.0. All evals run with a 1M-token maximum trajectory limit, increased from the 256K-token limit used for Inkling Small.
| Open Weights Flash | Open Weights Pro | Closed Weights Flash | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Inkling 2 Small | GLM-5.3-Flash | MiMo V2.6 Flash | MiniMax-M3.1-Flash | DeepSeek-V4.1-Flash-1017 | Muse Spark Open | GLM 5.3 | MiMo V2.6 Pro | Mistral Large 4 | Kimi K3 | GPT-6 Luna | Gemini 3.8 Flash | Claude Haiku 5.5 | |
| Model Info | |||||||||||||
| Params (B)activated / total | 17/594 | 18/320 | 15/309 | 23/428 | 16/552 | 42/816 | 40/753 | 42/1020 | 49/1050 | 104/2800 | – | – | – |
| Third Party Index | |||||||||||||
| AA Index (v5.0) | 30.6 | 27.8 | 26.7 | 29.1 | 32.6 (Second-highest reported score in this row) | 24.9 | 31.7 | 29.9 | 26.1 | – | 27.9 | 30.8 | 36.5 (Highest reported score in this row) |
| Vals Index (v2.2) | 56.1% | 50.9% | 54.8% | 53.8% | 61.3% (Highest reported score in this row) | 48.6% | 59.7% (Second-highest reported score in this row) | 57.8% | 56.5% | 53.0% | 52.3% | 56.0% | 58.7% |
| Coding | |||||||||||||
| FrontierCode v2.0 | 25.4% | 14.2% | 21.6% | 20.8% | 28.5% (Second-highest reported score in this row) | 7.6% | 21.6% | 24.0% | 15.0% | 17.3% | 25.2% | 18.7% | 34.9% (Highest reported score in this row) |
| DeepSWE v1.1 | 72.1% | 63.4% | 67.9% | 73.7% | 77.0% (Highest reported score in this row) | 60.0% | 69.0% | 71.9% | 61.7% | 68.5% | 66.6% | 73.8% (Second-highest reported score in this row) | 59.9% |
| Terminal Bench Science v0.1 | 5.7% | 4.8% | 4.3% | 3.3% | 12.9% (Second-highest reported score in this row) | 1.9% | 9.5% | 5.7% | 4.3% | 8.1% | 8.6% | 10.0% | 20.0% (Highest reported score in this row) |
| Vibe Code Bench v1.1 | 81.6% | 30.8% | 79.0% | 73.9% | 87.7% (Second-highest reported score in this row) | 76.2% | 78.1% | 85.2% | 81.6% | 85.0% | 81.7% | 78.7% | 90.4% (Highest reported score in this row) |
| SWEBench Pro v2 (Hard) | 86.3% (Second-highest reported score in this row) | – | – | 84.3% | – | 82.4% | 84.3% | – | – | 88.2% (Highest reported score in this row) | – | 58.8% | 52.9% |
| Autoresearch | |||||||||||||
| PostTrainBench v1.2 | 31.9% | 36.8% (Highest reported score in this row) | – | – | 30.5% | – | 34.5% (Second-highest reported score in this row) | – | – | 31.5% | – | – | – |
| PMPP-HardAUC (task-hours, 0–10h) | 494.27 (Second-highest reported score in this row) | – | – | – | 497.36 (Highest reported score in this row) | – | 430.96 | – | 478.39 | 453.58 | – | – | – |
| BioMysteryBench | 70.0% | – | 69.3% | – | – | – | – | – | – | 72.6% (Highest reported score in this row) | 61.5% | 62.2% | 70.4% (Second-highest reported score in this row) |
| Search and Tools | |||||||||||||
| MCP AtlasFull / Public | 85.4/85.0% (Second-highest reported score in this row) | –/82.0% | –/62.9% | –/80.4% | –/83.2% | 89.2/87.5% (Highest reported score in this row) | 84.2/83.6% | –/84.4% | –/84.5% | 82.3/84.2% | –/77.8% | –/87.5% | –/81.0% |
| Knowledge Work | |||||||||||||
| AA-Briefcase v1.1 | 1527 | 1451 | 1487 | 1476 | 1570 (Second-highest reported score in this row) | 1279 | 1503 | 1512 | 1348 | 1497 | 1337 | 1203 | 1574 (Highest reported score in this row) |
| GDPVal-AA v2.1 | 1630 | 1649 | 1614 | 1465 | 1746 (Highest reported score in this row) | 1401 | 1657 | 1689 (Second-highest reported score in this row) | – | 1543 | 1444 | 1445 | 1631 |
| JobBench | 70.5% (Highest reported score in this row) | 60.7% | 61.2% | 67.5% (Second-highest reported score in this row) | – | 61.8% | 61.4% | 62.0% | 55.2% | 54.3% | 49.0% | – | 39.2% |
| Enterprise Operations | |||||||||||||
| EnterpriseOps-Gym | 46.8% (Highest reported score in this row) | 33.2% | – | – | – | – | 36.4% | – | – | 45.3% (Second-highest reported score in this row) | – | – | – |
| EnterpriseBench-CoreCraft | 71.8% (Highest reported score in this row) | 46.7% | – | – | – | – | 51.8% | – | – | 43.1% | – | 58.5% (Second-highest reported score in this row) | – |
| Finance Work | |||||||||||||
| Vals Finance Index | 61.5% (Highest reported score in this row) | 55.8% | 57.1% | 55.9% | 60.9% (Second-highest reported score in this row) | 53.0% | 60.5% | 57.8% | 57.0% | 58.0% | 53.1% | 60.5% | 57.4% |
| Tau 3 Banking v1.0.1 | 40.4% (Second-highest reported score in this row) | 47.2% (Highest reported score in this row) | – | – | – | – | – | – | – | 33.4% | – | – | – |
| DAYJOB (Finance) | 14.3% (Highest reported score in this row) | 7.3% | – | – | 12.5% (Second-highest reported score in this row) | – | 11.0% | – | 4.8% | 5.5% | 6.8% | 4.5% | – |
| Apex-Accounting v1.1pass@1 | 14.5% | 12.3% | – | 13.5% | 12.5% | 13.8% | 13.9% | – | 12.0% | 9.5% | 10.8% | 15.5% (Highest reported score in this row) | 15.3% (Second-highest reported score in this row) |
| Apex-Accounting v1.1mean score | 62.0% (Second-highest reported score in this row) | 53.7% | – | 60.0% | 59.3% | 58.0% | 59.5% | – | 56.5% | 56.7% | 53.9% | 58.8% | 62.2% (Highest reported score in this row) |
| Legal Work | |||||||||||||
| Vals Legal Index | 73.9% (Second-highest reported score in this row) | 71.1% | 69.0% | 67.1% | 71.9% | 68.5% | 74.6% (Highest reported score in this row) | 72.1% | 67.8% | 71.7% | 62.2% | 69.8% | 70.3% |
| Visual Creation | |||||||||||||
| DesignArena Web Apps | 1296 (Highest reported score in this row) | – | – | – | – | – | – | 1275 | – | 1283 (Second-highest reported score in this row) | 1161 | 1221 | – |
| DesignArena Website | 1340 (Highest reported score in this row) | 1270 | – | 1280 | 1297 | 1303 | 1290 | 1307 | 1256 | 1334 (Second-highest reported score in this row) | 1268 | 1299 | 1261 |
| DesignArena Game Dev | 1428 (Highest reported score in this row) | 1276 | – | 1312 | 1261 | 1270 | 1303 | 1315 | 1225 | 1351 (Second-highest reported score in this row) | 1261 | 1305 | 1319 |
| DesignArena 3D Design | 1401 (Highest reported score in this row) | 1314 | – | 1332 | 1361 | 1261 | 1324 | 1333 | 1249 | 1381 (Second-highest reported score in this row) | 1213 | 1291 | 1334 |
| Code2Video-AA | 1020 | 975 | 1052 (Second-highest reported score in this row) | – | – | – | 1034 | 1052 (Second-highest reported score in this row) | – | 1041 | 1030 | 899 | 1192 (Highest reported score in this row) |
| Visual Perception | |||||||||||||
| PerceptionBench | 64.7% (Highest reported score in this row) | – | – | – | 47.7% | – | – | – | – | 58.5% (Second-highest reported score in this row) | 56.2% | – | – |
| ScreenSpot Prooriginal / with Python | 83.8/91.5% (Highest reported score in this row) | – | – | – | – | – | – | 56.6%/– | 42.3%/– | 73.8/83.2% | 81.1/87.6% (Second-highest reported score in this row) | – | – |
| HSS | 33.3% (Second-highest reported score in this row) | 26.8% | 25.4% | – | 30.1% | 29.6% | – | 25.2% | – | 25.5% | 21.0% | 41.6% (Highest reported score in this row) | – |
| BabyVisionoriginal / with Python | 68.6/93.7% (Highest reported score in this row) | 53.4/82.6% (Second-highest reported score in this row) | – | –/91.0% | –/92.5% | 44.3/80.2% | – | – | 42.3/71.5% | 53.4/85.7% (Second-highest reported score in this row) | – | – | – |
| Visual Reasoning | |||||||||||||
| GDP.pdf | 25.2% (Highest reported score in this row) | 15.4% | 9.0% | 20.8% | 17.2% | 18.6% | 11.2% | 19.2% | 17.6% | 22.0% | 22.8% (Second-highest reported score in this row) | 21.0% | 20.8% |
| MMMU Pro (Standard 10) | 81.6% (Second-highest reported score in this row) | 79.0% | – | 79.1% | 80.6% | 80.9% | – | 74.1% | 78.0% | 80.5% | 79.7% | 85.6% (Highest reported score in this row) | 79.1% |
| Charxiv RQoriginal / with Python | 85.6/87.7% (Second-highest reported score in this row) | 80.2/89.4% | 66.2%/– | 82.5%/– | 77.0/85.6% | 85.4/88.7% | – | 74.2%/– | 83.2/87.9% | 84.8/91.3% | 81.8/85.3% | 86.2/92.1% (Highest reported score in this row) | 84.4%/– |
| Chartographyoriginal / with Python | 36.7/80.4% | 16.3/78.0% | 7.5%/– | 12.6%/– | 17.9/83.0% | 35.7/63.2% | – | 10.8%/– | 21.2/56.6% | 26.6/68.1% | 32.0%/– | 40.9/82.0% (Second-highest reported score in this row) | 46.4%/– (Highest reported score in this row) |
| STEM Reasoning | |||||||||||||
| CritPt v2Full / Hard | 55.7/35.0% | –/24.5% | –/29.5% | 51.4/32.5% | 50.6/25.0% | 52.6/29.5% | 57.4/33.5% | 56.9/34.0% | 50.3/27.0% | –/40.5% | 65.1/42.5% (Highest reported score in this row) | 65.1/46.5% (Highest reported score in this row) | 64.0/43.5% (Second-highest reported score in this row) |
| HLE-Diamond | 28.7% | 18.2% | 13.7% | 28.1% | 25.3% | 26.3% | 17.9% | 15.2% | 17.3% | 26.9% | 22.2% | 33.1% (Highest reported score in this row) | 29.6% (Second-highest reported score in this row) |
| HLE (with tools) | 53.2% | – | – | – | 58.8% | 61.0% (Second-highest reported score in this row) | 62.5% (Highest reported score in this row) | – | – | 56.0% | – | – | 57.4% |
| Apex 2025 | 63.5% (Second-highest reported score in this row) | – | – | – | – | 32.3% | – | – | – | 65.6% (Highest reported score in this row) | – | – | – |
| Long Context | |||||||||||||
| AA-LCR v1.1 | 83.3% | 80.0% | – | – | – | 78.3% | 79.7% | 86.3% (Second-highest reported score in this row) | – | 88.7% (Highest reported score in this row) | – | 81.3% | 82.7% |
| LongBench v2 | 70.6% (Second-highest reported score in this row) | 68.2% | 60.7% | 60.9% | 69.0% | 68.7% | 68.4% | 68.0% | 65.7% | 69.0% | 65.0% | 73.7% (Highest reported score in this row) | 65.3% |
| Factuality | |||||||||||||
| AA-Omniscience v1.1score | 16.1 | 0.7 | -10.8 | -6.4 | 0.9 | 25.8 (Highest reported score in this row) | 6.0 | -4.5 | 5.1 | – | -8.3 | 23.3 (Second-highest reported score in this row) | 12.3 |
| AA-Omniscience v1.1non-hallucination rate | 73.3% (Highest reported score in this row) | – | – | 6.3% | 35.9% | 72.0% (Second-highest reported score in this row) | 61.1% | 39.8% | 71.8% | – | 20.7% | 44.9% | 65.8% |
| Instruction Following | |||||||||||||
| ComplexConstraints v1.0 | 44.5% | 27.6% | 22.1% | 23.2% | 50.4% (Highest reported score in this row) | 41.3% | 30.5% | 32.3% | 36.5% | 37.9% | 27.3% | 48.4% (Second-highest reported score in this row) | 36.8% |
| Safety (Misalignment) | |||||||||||||
| DistressBench | 82.0% (Second-highest reported score in this row) | 72.7% | – | – | 79.8% | 87.7% (Highest reported score in this row) | 72.5% | – | 77.0% | 73.5% | 66.7% | 70.7% | 71.7% |
| FORTRESS (adversarial) | 83.8% (Second-highest reported score in this row) | 73.3% | – | – | 72.8% | 92.4% (Highest reported score in this row) | 71.8% | – | 76.0% | – | – | – | – |
| FORTRESS (benign) | 79.7% | 96.4% (Second-highest reported score in this row) | – | – | 93.1% | 84.0% | 96.8% (Highest reported score in this row) | – | 91.8% | – | – | – | – |
| StrongREJECT (none) | 97.8% | 96.8% | – | – | 98.7% (Highest reported score in this row) | 97.6% | 97.2% | – | 98.6% (Second-highest reported score in this row) | – | – | – | – |
| Safety (Misuse) | |||||||||||||
| CyberBench v1.1 | 72.7% | 67.9% | 75.4% | – | 73.7% | – | 72.1% | 72.9% | – | 75.2% | 76.3% (Highest reported score in this row) | 43.8% | 75.5% (Second-highest reported score in this row) |
| PatchEval | 79.6% (Second-highest reported score in this row) | 74.8% | – | – | 80.4% (Highest reported score in this row) | 77.8% | 69.6% | – | 76.1% | – | – | – | – |
Evaluation snapshot: Missing results are shown as dashes.
Limitations #
While Inkling 2 Small improves on usability vs Inkling, it is not the most succinct communicator, and doesn’t always choose the best tools for the job—especially in codebases with a large number of skills. Like previous Inkling models, Inkling 2 Small natively supports image input; however, this model does not support audio.
Inkling 2 Small is also less optimized for the chat experience. We notice issues such as worse organization and presentation of content, overuses of slashes, and overreaction to mental health and other safety related queries.
Try Inkling 2 Small today #
We are proud to make Inkling 2 Small available on Tinker for inference and fine tuning. Following the completion of safety testing, the model will be available on additional platforms in the coming weeks.
Inkling 2 Small was made in service of our mission to build AI that extends human will and judgment. We hope that it encourages your ambitions, and we look forward to seeing what you build with it.