Ultramarine 1 Flash: Scaling Non-Verifiable RL Ultramarine 1 Flash, an agentic model built by applying large-scale reinforcement learning to the open base model GLM-5.3 Flash, improves on that base across all six knowledge-work benchmarks the company evaluated, by 4.1 points on average, while generating 6.5% fewer tokens. On Ultramarine's internal knowledge-work suite the model costs $0.13 per task and outscores Claude Fable 5 ($8.19) and GPT-5.6 Sol ($2.09), coming within three points of Claude Opus 5.5 ($6.96) and GPT-6 Sol ($1.26), all four competitors set to max reasoning. The release targets non-verifiable professional work such as diligence memos and litigation chronologies, where reward is shaped per task through rubric criteria rather than regex parsers or unit tests. Ultramarine 1 Flash: Scaling Non-Verifiable RL Introduction We present Ultramarine 1 Flash, an agentic model for professional knowledge work. Ultramarine applies large-scale reinforcement learning to a strong open base model, GLM-5.3 Flash, and improves on it across all six knowledge-work benchmarks we evaluate, by 4.1 points on average, while generating 6.5% fewer tokens. On our internal knowledge-work suite, Ultramarine costs $0.13 per task. It outscores Claude Fable 5 $8.19 and GPT-5.6 Sol $2.09 and comes within three points of Claude Opus 5.5 $6.96 and GPT-6 Sol $1.26 , all four set to max reasoning. Recent work has explored scaling agent post-training through synthetic tasks and environments \ 1\ ref-1 \ 2\ ref-2 \ 3\ ref-3 \ 4\ ref-4 \ 5\ ref-5 \ 6\ ref-6 \ 7\ ref-7 \ 8\ ref-8 \ 9\ ref-9 , though this research has so far largely concentrated on verifiable tasks, where coding has consistently held the largest share of the RL mix. Most professional work is not verifiable: models are asked to produce artifacts ranging from diligence memos to litigation chronologies and therefore cannot be checked by regex parsers or unit tests. The quality of the answer often turns on judgment and taste, and, as such, a defensible reward for open-ended work has to be shaped per task. In this work, we scale non-verifiable task synthesis across agentic knowledge work. We formalize each RL task the way recent work formalizes a verifiable task: an environment, a model instruction, and a verification system \ 10\ ref-10 . Each environment and task is synthesized with a custom synthesis pipeline per domain, and then used in a standard RL post-training setup. Non-Verifiable Task Synthesis Scaling Non-Verifiable Verification Scaling post-training for professional knowledge work requires tasks that are realistic and open-ended, yet specified well enough to distinguish better work from a different interpretation of the assignment. The reward must be shaped to recognize correct work without requiring a particular solution \ 11\ ref-11 . Task and rubric co-design is the central problem of authoring high quality RL training data. Per our rubric-grading work \ 12\ ref-12 , each criterion defines a boundary over possible answers. A criterion copied too closely from one golden reference rejects valid alternatives, while a vague criterion such as “provides an appropriate recommendation” can reward plausible sounding errors. The criterion must instead state the substantive condition that earns credit, together with the facts, thresholds, and context needed to check it against the deliverable. Naming the relevant entities, dates, and assumptions keeps the standard explicit, reducing judge variance and increasing alignment with expert human judgment. A criterion therefore tests one coherent proposition. Conjunctions may be used to express conditions that hold together: a note for the managing director may need to state what was checked "and" the decision implication "and" what remains open, so the reader can distinguish verified findings from unresolved decisions. Similarly, "or" s admit genuinely equivalent ways to meet a requirement. Independent accomplishments remain separate criteria so that success on one can receive credit even when another is missed. Each proposition is judged met or unmet, and its wording defines the acceptable variants to support generalization. These binary judgments combine into the raw reward, and the full rubric’s coverage and weights should reflect professional importance. Reliable grading thus depends on the boundaries of individual criteria and how they compose, rather than simply choosing a stronger or agentic judge. There are three sources of data we considered to curate the Ultramarine curriculum: 1 production traces from real users driving agents, 2 expert-authored tasks, and 3 synthetically generated data. Production traces offer realism, but users often leave assumptions unstated and rely on shared context \ 12\ ref-12 . Additionally, production state is non-stationary and converting traces into group-relative RL tasks requires reconstructing an environment with perfect fidelity to production in order to replay rollouts. The latter issue is solvable with careful state snapshotting, but the former is harder: a reward built from one user’s query bakes in that user’s unstated preferences and penalizes other correct answers. As a toy example, a user asking for the margin profiles of the top 10 medical technology companies may have meant the top 10 by revenue, and so a successful task shaped from that trajectory would embed that implicit assumption in the reward, penalizing perfectly valid responses choosing market cap, enterprise value, or profit to rank them. Solving for this would require materially reworking the query to make it constrained and evaluable. We chose not to pursue this path given the aforementioned complexity, though we believe it remains a rich area of research. Expert-authored tasks resolve that ambiguity, as annotators are given explicit instruction to construct tasks that are constrained, well-defined, and evaluable, but maintaining breadth and grading quality by hand is costly, and narrow tasks and rubrics still exclude defensible answers. As model intelligence grows at a clip, meeting all of these requirements per-task is difficult to scale, highlighted by the growing cost of human data and errors in human authored public benchmarks \ 13\ ref-13 \ 14\ ref-14 . As such, we continued with a purely synthetic recipe to scale RL task quality and grading fidelity, reliably, as a function of compute, across finance, law, consulting, and operations. Unlike production data, it allows us full control over data distribution and lets us choose seeds from scratch, and unlike expert authored data, it enables us to create low-cost, high-throughput tasks and filter for quality and difficulty at scale. Domains, Taxonomy, and Preventing Mode Collapse The training curriculum spans four domains finance, law, consulting, and operations , 19 professional practice areas, and 88 distinct environment types Table 1 . Each practice area and environment type is expressed as a standalone generation pipeline that samples over a taxonomy of axes. For each instance of generation, the taxonomy is sampled across its axes: deliverable format, sources of complexity, stakeholders, source conflicts, hints, the stage of the transaction or matter, numerous domain-specific dimensions, and so on. Sampling across these axes gives each practice area a nominal configuration space on the order of 10⁷–10¹⁰ combinations. Each environment instantiates further into many concrete instances with different companies, documents, and figures. This combinatorial breadth helps prevent mode collapse in LLM-generated data \ 1\ ref-1 \ 15\ ref-15 . Anatomy of a Task Tasks follow the Harbor specification \ 10\ ref-10 . Each task is packaged as a self-contained, runnable instance with three parts: the instruction, handed to the agent as a professional would receive it; the environment, the workspace and any live services, with a Dockerfile and service definition; and the grader, the task's rubric and the harness that scores the delivered work. Appendix A appendix-a-worked-task-examples presents a complete example, including the task instructions, workspace files and connected services, and full weighted rubric. Environments The curriculum is built on environments. An environment is a mounted workspace an agent inhabits the way a professional would: - A filesystem of realistic documents, including models, memos, filings, trackers, decks, correspondence, and so on. - A live system of record it reads and mutates as part of the work, including CRM and ERP/general-ledger systems, warehouse and transport management WMS/TMS , manufacturing and process control MES , GRC and ticketing, and property and work-order platforms. - Synthesized resource interfaces spanning MCP, HTTP, and CLI To build environments, we retrieve real documents and synthesize materials that have no suitable public source. The workspace combines: - Regulatory filings: SEC EDGAR filings, including 10-Ks, 10-Qs, 8-Ks, proxy statements, and S-1s, plus exhibits such as credit agreements and indentures. - Public web material: company sites, press releases, investor presentations, and regulatory notices. - Research material: analyst reports, market-data snapshots, and sector studies, provided as point-in-time snapshots labeled with their source and date. - Synthetic documents: realistic contracts, patents, internal reports, internal emails, and so on based on seed data. Each environment also has a private ground-truth record to provide privileged information to task proposer agents. Instructions and Graders We construct filesystems, document corpora, and MCP servers through bespoke pipelines for each of the 88 environment types. Within each environment, task instructions and rubrics are adversarially generated along the sampling axes described above, spanning 10⁷–10¹⁰ possible combinations. Generating frontier tasks is compute intensive: Producing a single task involves several dozen independent agent loops and multiple rounds of review, checking grounding, professional realism, difficulty, and consistency between the assignment and its grading criteria. Additional filters check for answer leakage and reward hacking, using classifiers iteratively added or refined in response to failures observed during training. A final curriculum filter screens tasks based on the base model's performance. Grading Logic Deliverables are practical work products, including Excel models, PowerPoint decks, and Word or PDF documents that the agent authors or edits in place. This result is then parsed and rendered as text to a naive LLM rubric grader. We use our open-source rubric package \ 16\ ref-16 , with PerCriterionOneShotGrader and a custom grading prompt tuned for agreement with human experts following the evaluation process described in Notes on Choosing a Rubric Judge \ 12\ ref-12 . Feather: The Agent Harness Recent work uses multi-harness training to improve generalization \ 9\ ref-9 \ 17\ ref-17 . However, training on several harnesses at once was not feasible on our compute budget. Instead, we surveyed the tool contracts and execution structures of the coding harnesses in wide use Claude Code \ 18\ ref-18 , Codex \ 19\ ref-19 , Cursor \ 20\ ref-20 , Kimi Code \ 21\ ref-21 , OpenAI Agents SDK \ 22\ ref-22 , OpenCode \ 23\ ref-23 , OpenHands \ 24\ ref-24 and its Software Agent SDK \ 25\ ref-25 , Qwen Code \ 26\ ref-26 , Mini SWE Agent \ 27\ ref-27 , Terminus \ 28\ ref-28 , and SWE-agent \ 29\ ref-29 , and created a new Python harness built on common shared conventions, aiming to train a policy with the most transfer across harnesses. Training and evaluation share Feather's agent loop, tools, and compaction. Training routes model requests through Verifiers’ inference interception layer \ 30\ ref-30 ; the asynchronous training loop refreshes inference weights mid-rollout \ 31\ ref-31 . Tool Interface We compared the file-editing and process-control interfaces of existing harnesses and used paired evaluations and trajectory inspection to determine Feather's tool configuration. File Editing Tools | Editing approach | How changes are described | Representative implementations | |---|---|---| | Patch editing | Changes described through context lines and additions/deletions | Codex, OpenCode¹ | | String replacement and file creation | Changes specified as old/new text; new files supplied as complete content | Claude Code, Qwen Code, OpenCode¹, OpenHands | | Terminal file operations | File operations through terminal commands | Mini SWE Agent, Terminus | ¹ OpenCode uses apply patch for selected GPT models and separate edit and write tools for other models. Existing harnesses abstract file editing in two main ways: a patch interface describes changes through context lines and additions/deletions, while a string-replacement interface describes changes through old and new text. To change timeout: 30 to timeout: 60 in config.yaml , Codex's apply patch uses textual context to locate the patch and does not require the model to supply line numbers: Begin Patch Update File: config.yaml @@ - timeout: 30 + timeout: 60 End Patch String replacement requires only the old and new text, with the tool locating the match in the file: edit file path="config.yaml", old string="timeout: 30", new string="timeout: 60" A second design choice is whether to expose file operations through a single combined tool or through dedicated per-operation tools. | Interface | Tool organization | Representative harnesses | |---|---|---| | Combined tool | file editor command="view", ... , file editor command="str replace", ... | OpenHands | | Dedicated tools | read ... , edit ... , write ... | Claude Code, Qwen Code, Kimi Code, OpenCode edit/write configuration ; Feather | We ran both configurations on the same set of tasks and inspected the generated calls. The trajectory analysis did not show a clear advantage for split tools. Feather adopted dedicated tools for their simpler per-operation schemas and consistency with several widely used harnesses. Beyond the tool interface, execution order also matters for correct file edits. A model turn may contain multiple tool calls, but Feather executes them sequentially in model-produced order to preserve dependencies between reads, shell commands, and file modifications. Even with sequential execution, intervening shell commands or background processes may change a file after it has been read. Feather therefore records a content fingerprint a hash of the complete file contents on each read and checks it before editing or overwriting the file. If the contents have changed, the modification is rejected and the model is asked to reread the file. Feather's tool set comprises read, edit, write, grep, and glob: - read returns text with line numbers and accepts offset and limit parameters to bound the requested range. A successful read records a content fingerprint of the file. - edit and overwrites through write check the content fingerprint before applying changes, rejecting the modification when the current contents do not match the record. edit requires the old text to match uniquely in the file by default, and supports explicit multi-occurrence replacement through replace all. - grep and glob provide structured search, avoiding the need for models to construct Bash commands and parse their output. grep returns matching files and line numbers, glob returns paths by filename pattern. Both enforce a result limit and include a truncation indicator when results exceed the limit, so the model knows to narrow its query. Bash Tool Across existing harnesses, shell interfaces differ primarily in what happens to a process after a call returns and how the model regains access to it. | Behavior | How shell execution is managed | Representative harnesses | |---|---|---| | Session handle | Returns a handle after the wait for output | Codex | | Persistent shell | Retains shell state across commands | OpenHands, SWE-agent | | Explicit background parameters | Selects background execution on the call | Qwen Code | Feather uses explicit background tasks. Each call starts a fresh shell. Changes to the working directory and environment variables do not carry into later calls. The model starts background processes with run in background , reads their output with task output , stops them with task stop , and lists them with task list . Todo Tool To let the model maintain a work plan over long-horizon tasks and preserve progress tracking across multiple rounds of context compaction, Feather provides a todo tool. The model can create or update the list at any point. The list persists across compaction within a single agent run and is reattached in its current state to the compressed context. Final Tool Set | Category | Tools | Behavior | |---|---|---| | File operations | read , edit , write , grep , glob | Dedicated entry points with bounded output. Edits and overwrites require a prior read and reject stale contents. | | Process control | bash , task list , task output , task stop | Foreground by default, explicit background with run in background . No persistent shell state. | | Planning | todo write | Direct list replacement. Persists across compaction. | Optional web search , web fetch , and MCP servers are configured per task. Paper Office Package Agents use Python packages, such as python-docx , python-pptx , and openpyxl , to edit Word, Excel, and PowerPoint files. Often, they hit API gaps, drop into raw OOXML, and create silent corruptions and regressions. We train Ultramarine using our open source forks of these libraries, paper-docx , paper-pptx , and paper-xlsx \ 32\ ref-32 . Each package keeps the familiar import name and expands the API for understanding and safely editing Office files. The libraries expose file structure as data, validate supported changes, and refuse operations they cannot handle safely. System Prompts and Exploration "Training on your production harness" is an emerging strategy in agent post-training, and it is especially helpful to familiarize the model with the tools, arguments, and interfaces it will use in production. However, this approach can restrict exploration when applied to prompts, tool descriptions, and other model instruction. These show up as prescriptive context that increases performance during test time, but constrains model exploration and conditions behavior to a specific prior during training. We only introduced production model instruction in final stages of training. For the bulk of training, we rewrite all model-facing prose to be general, lightweight, and unopinionated. It is up to the policy to decide how to make do with the options and information presented to it, and there is no one prescribed workflow it should adhere to. Our training system prompt is therefore three sentences, about 50 tokens, where the production prompts we surveyed in real harnesses run from roughly 1,800 to 4,400 Table 6 . Similarly, our tool descriptions state what a tool does and what it guarantees, rather than when or how to use it, and we omit the planning and verification workflows that production harnesses often prescribe. You are an agent working in a Linux sandbox with access to the provided tools and workspace. Do not expose, log, transmit, or write secrets or credentials. When relevant, put requested file deliverables at the paths specified by the user or environment. Feather's tool descriptions, also shown to the model, similarly opt for exposition rather than prescription. Read a text file with line numbers. offset and limit bound the returned range. A successful read records the file revision required to edit or overwrite its current contents. | Harness | System prompt | Read tool | Example instructions | |---|---|---|---| | Feather training | 50 | 36 | See full prompt above | | OpenCode 1.18.4 | 1,771 | 295 | “You MUST answer concisely with fewer than 4 lines of text … IMPORTANT: DO NOT ADD ANY COMMENTS unless asked …NEVER commit changes unless the user explicitly asks you to.” | | OpenHands SDK 1.37 | 2,802 | 663 | “NEVER create multiple versions of the same file with different suffixes … Do not push to the remote branch and/or start a pull request unless explicitly asked to do so. …You must always test real code paths in tests, NOT mocks.” | | OpenAI Agents SDK 0.18 | 3,495 | Shell | “You MUST adhere to the following criteria when solving queries … Do not waste tokens by re-reading files after calling apply patch on them. …NEVER add copyright or license headers unless specifically requested.” | | Codex CLI | 3,576 | Shell | “Use apply patch for local file edits. Do not create or edit files with cat or other shell write tricks. … Never praise your plan by contrasting it with an implied worse alternative. …you must use that skill for that turn.” | | Kimi Code 0.1.1 | 3,916 | 669 | “you MUST use the appropriate tools available to you to make actual changes … Never give the user more than what they want. …ALWAYS, keep it stupidly simple.” | | Qwen Code 0.20.1 | 4,403 | 483 | “NEVER assume a library/framework is available or appropriate. … Do not revert changes to the codebase unless asked to do so by the user. …Never give up after a single tool failure.” | Vision We did not train with vision. To nudge the model against repeated image-read attempts, without imposing additional harness constraints, read returns a text observation for supported image extensions: read {"file path": "chart.png"} → Vision is not available in this environment. Tool Failure Handling A failure should shape the policy only when the policy caused it. An error that follows from the model's own actions, such as a malformed call or a failing command, is part of its exploration, so we return the error and let the model correct course within the same trajectory. When the environment fails beneath it, such as a lost sandbox, the outcome carries no information about the policy, and therefore scoring it would only add noise to the reward. We discard that attempt, excluding it from training rather than assigning zero reward, and restart the task in a fresh environment. | Class | Examples | Handling | |---|---|---| | Model-correctable error | Invalid arguments, unknown tool, missing file, failed string replacement, command error or timeout | Return the error to the model and let it continue within the same trajectory. | | Unrecoverable runtime failure | Lost sandbox, unrecoverable tool transport failure, unexpected tool exception | End the attempt. The task can be restarted in a fresh environment. | Sandboxing Training agents on professional work requires environments that support both document editing and interactions with live services. Each rollout uses an isolated E2B sandbox with separate Docker Compose containers for agent execution, MCP services, and grading. We built our sandbox architecture around four choices: restore initialized environments from snapshots, keep the agent loop outside the execution environment, isolate task execution from services and grading, and treat remote communication as both a latency cost and a source of recoverable failures. Initialized Snapshots Environment-level hot snapshots preserve filesystem and memory state, including running services \ 33\ ref-33 . Tasks and rollout replicas restore independent copies, then load task-specific overlays and the pinned Feather runtime. Task-input and Feather updates do not require rebuilding every world snapshot. Agent Loop Outside the Sandbox The agent loop runs on the orchestrator, while commands and file operations execute inside the sandbox \ 34\ ref-34 . Model inference credentials and agent-loop code remain outside the environment the model can modify, so orchestration updates can ship without rebuilding environment snapshots. More importantly, nothing the model does in the sandbox can alter its own harness, credentials, or grader. Execution, Services, and Grading The agent accesses application services through MCP, without direct access to service processes or their storage. Deliverables are writable, and grading artifacts and verifier state are outside the agent's execution boundary. Reusable verifier dependencies and world-level grading infrastructure are baked into snapshots in advance. Finally, task-specific grading material is supplied after the rollout for isolation. Remote Calls and Recovery Operations that look cheap locally can become expensive when each requires a remote request. Our initial file-loading path created directories and naively wrote files individually. On an upload test of about 50 files, packing them into a single tarball cut the time from 52 seconds to 2.2 seconds, a 96% reduction. Each tool call made several roundtrips, and fusing those roundtrips into fewer requests removed the same kind of delay. Training Training Objective We adapt the RL setup used for Kos-1 Lite \ 35\ ref-35 and Kos-1 Experimental \ 36\ ref-36 to full agent trajectories. For a group of rollouts on the same task, our modified CISPO objective is \ 31\ ref-31 \ 37\ ref-37 : marks model-generated training tokens, and counts those tokens in the batch. In addition to CISPO we incorporate Rollout Routing Replay R3 \ 38\ ref-38 , mean-centered advantages \ 39\ ref-39 , and zero-advantage filtering \ 40\ ref-40 . Async Updates and Mini-Steps Our earlier synchronous setup used three policies current, pre-update, and rollout \ 41\ ref-41 : CISPO compared the current and pre-update policies, while token-level truncated importance sampling corrected the pre-update/rollout mismatch \ 42\ ref-42 . That implementation used an extra forward pass for pre-update log probabilities and multiple minibatch updates per rollout batch. Here, completed task groups enter a buffer continuously, and a full batch triggers one optimizer step, with no mini-steps \ 31\ ref-31 \ 43\ ref-43 . Off-policyness therefore depends on trainer throughput relative to rollout throughput. The update uses the importance sampling ratio of the current policy to the rollout policy directly, which removes the separate pre-update policy and its forward pass. Length Penalty We penalize generated output and excessive turns, alongside malformed tool calls and calls to nonexistent tools: counts generated tokens across the trajectory, counts agent turns, and counts invalid tool calls. Each invalid call incurs a penalty , capped in total at . The turn factor fades positive reward from to zero at the turn cap . enables output penalties when the group mean reward after this fade is positive. The output penalty grows at rate beyond allowance , up to . A separate extreme-length penalty starts at , rises over tokens, and caps at . Training Details | Setting | Value | |---|---| | Base model | GLM-5.3-Flash \ 44\ ref-44 | | Loss | CISPO \ 37\ ref-37 , with modified two-policy objective | | Optimizer | AdamW | | Batch size | 64 × 8 | | LoRA rank \ 45\ ref-45 | 64 | | Learning rate | 5 × 10⁻⁵ | | Sampling config | temp = 1 , top p = 1 , top k = -1 | | Advantage | Group-centered mean without standard deviation normalization \ 39\ ref-39 | | Loss normalization | Normalized by the total count of valid generation tokens in the training batch | | Context window | 131,072 tokens | | Max agent turns | 250 | | Compaction trigger | At 85% of the context window or when 30,000 tokens remain, whichever occurs first | Results We evaluate Ultramarine 1 Flash on six knowledge-work benchmarks, JobBench \ 46\ ref-46 , PI Knowledge Work, FrontierFinance \ 47\ ref-47 , AutomationBench \ 48\ ref-48 , DiligenceBench \ 49\ ref-49 and APEX-Agents 1.1 \ 50\ ref-50 , and report two coding benchmarks for reference, Terminal-Bench 2.1 \ 51\ ref-51 and DeepSWE \ 52\ ref-52 Table 8 . All models run at the highest reasoning setting they support. On PI Knowledge Work we additionally sweep each model's reasoning settings for the cost–quality frontier. The notes to Table 8 give each benchmark's setup and the source of each comparison score. | Benchmark | Ultramarine 1 Flash | GLM‑5.3 Flash | Claude Fable 5 | GPT-5.6 Sol | Gemini 3.8 Flash | DeepSeek V4.1 Flash | Kimi K3 | |---|---|---|---|---|---|---|---| | Knowledge Work | | | | | | | | | JobBench