# GPT 5.6 sol leads AutoCAD-Bench with 46% one-shot success rate > Source: > Published: 2026-07-23 18:18:10+00:00 On **July 23rd 2026**, [Dev (@DevvMandal)](https://x.com/DevvMandal) announced the release of **AutoCAD-Bench**, a new benchmark designed to assess whether artificial‑intelligence models can complete precise AutoCAD tasks using computer‑use alone. The announcement appeared in a three‑part thread on X, which included a link to the full benchmark report hosted on Markov Studios' research site and a calendar link for follow‑up conversations. The benchmark focuses on “computer‑use” – the ability of an AI system to control a desktop environment, launch applications, and interact with user interfaces without any internal plug‑ins or API hooks. This mode of operation has become a focal point for the community after OpenAI, Anthropic, and DeepMind released agents that can navigate web browsers, spreadsheets, and design tools through simulated mouse‑clicks and keystrokes. AutoCAD, a long‑standing computer‑aided design (CAD) platform used by architects, engineers, and product designers, represents a particularly demanding test case because it requires high‑precision geometry input, layer management, and adherence to drafting standards. In the accompanying report, **GPT 5.6 sol** – a proprietary version of OpenAI’s GPT‑5 series – achieved the highest success rate, solving **46 %** of the tasks on a one‑shot basis. The term “one‑shot” refers to the model receiving a single instruction prompt for each task, without any iterative feedback or fine‑tuning specific to AutoCAD. According to the report, GPT 5.6 sol handled a range of basic to intermediate operations, such as drawing lines, creating circles, applying dimensions, and inserting block references. The remaining 54 % of tasks were either partially completed or failed, indicating that while large language models can navigate the AutoCAD UI, they still fall short of the reliability required for production‑grade design work. The benchmark contains **30 distinct tasks** split across three difficulty tiers. Tier 1 covers elementary commands (e.g., line, rectangle), Tier 2 introduces layer manipulation and dimensioning, and Tier 3 demands multi‑step workflows like creating a parametric part and exporting a DWG file. Each task is evaluated on two metrics: **completion rate** (whether the final drawing matches a ground‑truth reference) and **precision score** (the geometric error measured in millimeters). The report notes that GPT 5.6 sol’s 46 % figure reflects the composite success across all tiers, with a higher hit‑rate on Tier 1 and a steep drop‑off on the more complex Tier 3 challenges. AutoCAD‑Bench joins a growing ecosystem of benchmarks that move beyond pure language understanding toward embodied AI. Prior efforts such as **Big‑Bench**, **MMLU**, and **HumanEval** tested reasoning and coding abilities in a purely textual context. More recent initiatives like **RobotBench** and **AgentBench** evaluate agents that operate physical robots or graphical user interfaces. By targeting a professional engineering tool, AutoCAD‑Bench fills a niche that has been largely ignored: the ability of LLM‑driven agents to perform domain‑specific, precision‑oriented tasks that have direct commercial relevance. The release also serves a strategic purpose for the emerging AI‑agent market. As venture capital continues to fund companies that build “computer‑use” agents, a standardized, reproducible benchmark offers a way for investors, developers, and enterprise buyers to compare disparate solutions. The inclusion of a public leaderboard, as described in the report, allows any research team to submit results, fostering transparency and competition. In the thread, Dev invited interested parties to discuss “computer‑use data and environments” and provided a calendar link for a 30‑minute meeting ([schedule here](https://cal.com/dev-markov/30min)). The invitation was addressed to [Markov (@markov__ai)](https://x.com/markov__ai), a partner credited with co‑authoring the benchmark report. While the benchmark’s initial results highlight the promise of large language models in CAD workflows, they also underscore the current limitations. The 46 % success rate suggests that substantial work remains before AI agents can replace human designers on day‑to‑day projects. Future iterations of AutoCAD‑Bench are expected to expand the task set, incorporate more complex 3D modeling operations, and evaluate multi‑modal inputs such as voice commands and sketch‑based prompts. AutoCAD‑Bench’s release marks a concrete step toward quantifying AI progress in professional design software, providing a reference point for both academic research and commercial product development. **Source**: The announcement and link to the full benchmark report were posted by Dev on X on July 23 2026 ([original tweet](https://x.com/DevvMandal/status/2080325376463736986)).