Microsoft Mage-VL-Cowork: A 4B Open Model That Clicks and Codes, 66.0% on OSWorld-Verified

Microsoft released Mage-VL-Cowork, a 4B-parameter open-weight computer-use agent, on Hugging Face under Apache-2.0, with the company's model card reporting 66.0% on OSWorld-Verified under a 50-step budget across 361 tasks, which Microsoft describes as the best result among 4B models. The model, a fine-tune of the Mage-VL 4B checkpoint pairing a Mage-ViT vision encoder with a Qwen3-4B-Instruct-2507 backbone, learns turn by turn to either issue one of 11 GUI primitives in normalized [0, 999] coordinates or run bash commands in a persistent shell and Python environment, with no external router. Microsoft reports that pruning its teacher harness raised a 4B student trained on the cleaner trajectories to 37.6%, against 25.1% for one trained on the prior data, and the card notes the benchmark numbers are Microsoft's own rather than independent.

Microsoft has put a new open-weight computer-use agent on Hugging Face: Mage-VL-Cowork https://huggingface.co/microsoft/Mage-VL-Cowork?ref=explainx , a 4B-parameter model that decides turn by turn whether to click through a screenshot or to run code in a terminal. Microsoft's model card reports 66.0% on OSWorld-Verified under a 50-step budget, which it describes as the best result among 4B models, and the weights are released under Apache-2.0 . The idea is simple and practical: most desktop agents work only through pixels, so every screenshot burns thousands of tokens and a spreadsheet edit turns into dozens of mouse clicks. Mage-VL-Cowork keeps the GUI for visual work and drops to a shell for structured edits. This post walks through what Microsoft published, the numbers and where they come from, how to try it safely, and what the release does not prove. TL;DR: the questions people ask first | Question | Answer | |---|---| | What is it? | A 4B vision-language agent that uses a computer use tool 11 GUI primitives and a bash tool in one policy | | Who made it? | Microsoft's Mage team, built on the Mage-VL 4B backbone Mage-ViT plus Qwen3-4B-Instruct-2507 | | License? | Apache-2.0 | | Headline score? | 66.0% OSWorld-Verified, 50 steps, two-run average, 361 tasks | | Size on disk? | About 9.48 GB in bfloat16, two safetensors shards | | How do I run it? | SGLang Docker image or Transformers with trust remote code=True | | Is the benchmark independent? | No. The numbers are Microsoft's own, from the model card | | Best use? | Research, local agent prototypes, cheap spreadsheet and document automation in a sandbox | What Mage-VL-Cowork actually is The model card calls it "a full training recipe to learn hybrid computer use from evolved harnesses." Underneath, it is a fine-tune of the intermediate Mage-VL 4B checkpoint, which pairs a from-scratch vision encoder Mage-ViT, a 16 by 16 patch grid with 3D RoPE with a Qwen3-4B language backbone. Microsoft lists a 32,768-token active context window, with the architecture supporting up to 262,144. Each turn the model emits a three-part structured output: a <think rationale, a one-line Action: summary, and a <tool call that is either a GUI action in normalized 0, 999 coordinates or a bash command in a persistent shell and Python environment. There is no external router or second model deciding which channel to use. That decision is learned. To keep long tasks affordable, the card describes an asymmetric working memory. The three most recent screenshots and four full responses stay in context, while everything older is folded into single-line Action: summaries placed after the system prompt. If you have used coding agents that compact history, this is the same idea applied to screenshots. How was it trained? Microsoft's recipe has three parts, and the second one is arguably the most reusable. Microsoft Mage-VL-Cowork 4B computer use model scoring 66.0% on OSWorld-Verified against larger models Image: Microsoft https://huggingface.co/microsoft/Mage-VL-Cowork?ref=explainx 1. Evolve the teacher harness before collecting data A strong teacher does not automatically produce clean training data. Microsoft used an authoring agent to refine the scaffold around the teacher model: the tool interfaces, system prompt and pre-installed terminal environment. It removed overlapping tools such as xdotool and verbose accessibility-tree dumps and pre-installed Python libraries such as python-pptx , openpyxl , pandas , pypdf and pillow . The card reports a controlled comparison with the GPT-5.4 teacher fixed, on 672 held-out multi-application tasks. The pruned GUI-plus-coding harness gave the teacher 92.0% success in 25.0 mean steps, against 89.6% in 35.33 steps with accessibility data added. The bigger effect showed up in the student: a 4B model trained on the cleaner trajectories reached 37.6%, against 25.1% for the one trained on the cluttered harness, a 12.5 point gap. The lesson, in Microsoft's words, is that "more tools do not automatically make a better agent", especially for small students. 2. Mage-CUA-Forge: workflow-first task synthesis Instead of running an open-ended agent loop to invent each training task, Mage-CUA-Forge writes a reusable generation workflow once per atomic skill. Per the card, three student-feedback rounds produced 293,806 tasks across 6,915 atomic skills 38.24% multi-application, 278 file categories and kept 318,837 verified hybrid trajectories, 68.19% GUI tool calls and 31.81% terminal calls. On cost, Microsoft says that with the same generator model the workflow-first approach cut tokens per task from 1,276K to 85K down 93.3% and recurring spend from $0.906 to $0.158 per task down 82.6% . That is a claim about Microsoft's own pipeline, but it is a useful data point for anyone building synthetic agent environments. Mage-CUA-Forge training pipeline that evolves the teacher harness for a hybrid GUI and terminal agent Image: Microsoft https://huggingface.co/microsoft/Mage-VL-Cowork?ref=explainx 3. A five-stage specialize-then-unify curriculum Rather than learning shell access as a workaround for weak visual grounding, the model first builds two strong standalone skills and then merges them: | Stage | Focus | Scale per the card | |---|---|---| | 1. GUI midtraining | Grounding, reasoning, tool-call format | 24.6B tokens per epoch | | 2. GUI agent SFT | Multi-step desktop and web workflows | 156.7K trajectories | | 3. Coding and terminal SFT | Repo engineering, programmatic file edits | 563.7K trajectories | | 4. Hybrid CUA SFT | Choosing between computer use and bash | 318.8K trajectories | | 5. Online hybrid RL | Asynchronous GRPO with verifiable rewards | 1,988 verifiable tasks | Total nominal compute was 7.79 ZFLOPs on 32 NVIDIA B200 GPUs plus 1,120 CPU sandbox slots for the RL stage. Progress on OSWorld-Verified went from 45.56% after GUI SFT, to 55.84% after hybrid SFT, to 66.04% after online RL. The RL stage also made the agent leaner: model calls per task fell from 21.57 to 18.77 and output tokens from 7,374 to 5,499. The numbers, and where they come from All of the following are from Microsoft's model card, not from an independent evaluation. The 66.0% is the average of two full passes 65.33% and 66.75% on 361 OSWorld-Verified https://arxiv.org/abs/2404.07972?ref=explainx tasks, excluding Google Drive tasks, using Microsoft's own MAGEHybridAgent harness at 1920 by 1080 resolution. | Model | Size | OSWorld-Verified | Source note | |---|---|---|---| | GPT-5.5 | not stated | 78.7 | listed in the card's table | | Claude Opus 4.7 | not stated | 78.0 | listed in the card's table | | GPT-5.4 | not stated | 75.0 | listed in the card's table | | Kimi-K2.6 | not stated | 73.1 | listed in the card's table | | Human reference | n/a | 72.4 | listed in the card's table | | GPT-5.4-mini | not stated | 69.6 | one run by Microsoft, same 361 tasks | | ScaleCUA-Qwen3.5-9B | 9B | 68.7 | listed in the card's table | | Mage-VL-Cowork | 4B | 66.0 | two-run average | | Muse Glimmer-30B | 30B | 65.9 | listed in the card's table | | Qwen3.6-27B | 27B | 63.9 | listed in the card's table | | Fara1.5-4B | 4B | 13.9 | listed in the card's table | Two things stand out. First, the "best at 4B" claim is true within the table, but a 9B model ScaleCUA-Qwen3.5-9B at 68.7 scores higher, and frontier proprietary models sit 9 to 13 points above. Second, 66.0% is still below the 72.4% human reference in the same table, so read the whole table, not the headline. Its strongest categories are OS tasks 87.5% and multi-application workflows 62.7% , where the card says shell access turns long click sequences into short, verifiable ones. On the browser side, the card reports 58.6% on WebVoyager-Static, a deterministic offline version of the benchmark with 643 tasks across 15 sites, graded on both navigation traces and backend database state. That score is for the SFT checkpoint before RL, and it trails the browser-only specialist Fara1.5-4B at 68.0%. Microsoft's point is that the generalist covers both full desktops and web in one small model. The efficiency story Against GPT-5.4-mini, Mage-VL-Cowork is less accurate 66.0% against 69.6% but cheaper to run per task: similar interaction turns 18.77 against 18.24 calls with about 55.6% fewer output tokens 5,499 against 12,397 . If you are running thousands of automation tasks on your own hardware, that ratio may matter more than three points of accuracy. The card also shows two side-by-side demos against Qwen3.6-27B in GUI-only mode. In LibreOffice Calc, merging ranges, labeling, adding a chart and saving took Mage-VL-Cowork 39 seconds in 6 steps by manipulating the workbook with Python, against 9 minutes 56 seconds in 39 steps for the 27B model. In GIMP, the 4B model finished in 1:58, while the 27B model stopped at 7:02 with a file that failed verification. These are two hand-picked demos, so treat them as illustrations of the mechanism, not as averages. How to try it Microsoft's quick start uses SGLang, and there is a pinned container docker.io/xiaoyizhang/mage-vl-cowork:0.1.0 with SGLang 0.5.2 and Transformers 5.7.0 . A minimal Transformers setup, per the card, is: pip install "transformers =4.57.3" accelerate pillow torch torchvision qwen-vl-utils Then load microsoft/Mage-VL-Cowork with trust remote code=True . The model uses a custom llava onevision2 architecture with bundled modeling files and a Qwen2.5-VL-style processor, so do not assume it drops into standard Qwen3-VL classes. The repo also contains an osworld eval/ folder with scripts to reproduce the OSWorld-Verified run, which is the first thing worth doing if you plan to rely on the number. Practical advice from the way the card is written: 1. Reproduce before you trust. Run the provided OSWorld harness on a small slice and compare with the reported 65 to 67% band. 2. Sandbox everything. The agent has a persistent shell. Give it a disposable VM or container, no saved logins, no network access you do not need. 3. Give it the libraries it was trained with. The training environment pre-installed openpyxl , python-pptx and friends, so the shell channel works best when those exist. 4. Expect English-only behavior. The Hugging Face metadata lists English as the language. If you are building agents rather than benchmarking them, our guides on loop engineering for coding agents https://explainx.ai/blog/loop-engineering-coding-agents-claude-code-guide-2026 and agent harnesses https://explainx.ai/blog/terminal-era-over-agent-is-the-new-primitive-claude-code-codex-desktop-2026 cover the surrounding scaffolding that Microsoft's own result suggests matters as much as the model. What the release does not tell you - No independent verification yet. The model went up on October 30, 2026 with almost no downloads, and the numbers are the authors' own. OSWorld has already had scoring disputes across the field, so wait for third-party reproductions. - A 50-step budget and a custom harness. Scores depend heavily on the harness. Microsoft's MAGEHybridAgent settings three screenshots, three history turns are part of the result. - Safety is mostly your job. A model that can run bash on your desktop can delete files or exfiltrate data if prompted by hostile page content. The card includes a Responsible AI section, but a benchmark pass rate is not a security guarantee. For agents that act on real machines, add a policy layer such as AgentBeam https://agentbeam.com/?ref=explainx , the agent security platform from the explainx.ai team that blocks risky agent actions before they run. - Headline tables mix sources. Some rows are reproduced, some "reported scale", some one-run evaluations. Compare like with like. Where it fits among open computer-use agents October has been busy for open GUI agents. Alibaba's OpenGCUA stack https://explainx.ai/blog/alibaba-opengcua-open-stack-generalist-computer-use-agents-osworld-v2-2026 and the MAI-Gym 9B GUI agent RL platform https://explainx.ai/blog/mai-gym-9b-gui-agent-rl-platform-tongyi-2026 focus on generalist training and reinforcement-learning environments, OpenMobile 2 https://explainx.ai/blog/openmobile-2-open-mobile-agent-stack-mobilegym-androidworld-79-9-2026 targets phones, and Work2Skill https://explainx.ai/blog/work2skill-human-gui-demonstrations-reusable-agent-skills-paper-2026 turns human demonstrations into reusable skills. On the product side, Cursor now drives desktop apps and the iOS simulator https://explainx.ai/blog/cursor-computer-use-desktop-apps-ios-simulator-local-agents-2026 and Microsoft itself is pushing local and cloud hybrid agents in Windows https://explainx.ai/blog/microsoft-windows-hybrid-intelligence-local-cloud-agents-mxc-october-2026 . What Mage-VL-Cowork adds is a concrete, reproducible argument that a small model can match much larger GUI-only agents if it is allowed to use the terminal, and that cleaning the teacher's toolset can matter more than adding tools. If you are comparing small models more broadly, see our small models comparison https://explainx.ai/blog/small-models-comparison-haiku-5-5-gpt-6-luna-deepseek-v4-1-flash-gemini-3-7-flash-glm-5-3-flash-2026 . What this means for builders For people building agents, three takeaways stand out. First, channel choice is a trainable skill: if your agent has both a GUI and a shell, measure how often it picks the right one rather than assuming a frontier model will. Second, token cost per task is a first-class metric next to success rate, and RL on verifiable rewards visibly improved it here. Third, an Apache-2.0 4B model that fits on one GPU makes it realistic to run private desktop automation without sending screenshots to a cloud API, which matters for regulated workplaces. For everyone else, the story is that computer-use agents are moving from giant proprietary models toward small open ones that run on a workstation. Whether this particular checkpoint holds up outside Microsoft's harness is the open question, and the next few weeks of community tests will answer it. Related reading Details reflect the Hugging Face model card as of October 30, 2026. Benchmarks are self-reported by Microsoft and may change as the card is updated.