Introduction #
dfs-large1
is now in preview within the depthfirst platform. It is our most capable model to date, designed for vulnerability discovery and validation across large enterprise repositories. Built on GLM 5.2, it was post-trained inside depthfirst’s security agent harness using our RL stack, in partnership with Fireworks AI to scale training and inference.
dfs-large1
has shown consistent gains throughout our ongoing RL climb on long-horizon tasks that combine repository-level context, tool use, and executable verification. Performance continues to improve, with no clear sign of plateauing. Across our evaluations, dfs-large1
achieves best-in-class performance and establishes a new Pareto frontier for our production workload.
To measure progress, we constructed depthfirst-bench
to reflect the complexity of real-world security audits. Tasks are sourced from vulnerabilities in large repositories, where the evidence needed to establish a finding may be distributed across multiple components or repositories. Individual tasks can also contain several vulnerabilities, requiring the model to continue beyond the first plausible issue. depthfirst-bench
captures the full workflow of a security audit, requiring findings to hold up under broader analysis and, where possible, executable verification. We will describe the construction and methodology of depthfirst-bench
in greater detail in a future post.
In this report, we describe the multi-task RL recipe used to train dfs-large1
, the audit behaviors that have emerged during training, and zero-day vulnerabilities discovered by the model.
Security Harness #
Large enterprise monorepos are difficult for single-agent architectures to audit effectively. As repositories grow, audits become more expensive while code coverage and performance degrade. Our production security harness divides each repository into targeted audit scopes, giving every agent a bounded investigation surface while preserving the broader context required for vulnerability analysis. Each agent receives a bounded list of files and the corresponding scoped threat model, including attack surfaces, trust boundaries, and relevant data flows. Its shell remains rooted at the full repository, so it can inspect shared libraries, callers, dependencies, and other out-of-scope files when tracing reachability or exploitability. In multi-repository audits, related repositories are also available as supporting context. The harness executes these scopes concurrently at scale, allowing us to maximize code coverage while shortening the time required to assess the repository as a whole. dfs-large1
is optimized to operate within this framework across even the largest repositories.
Repository context is essential because vulnerability analysis is rarely local. An input that appears attacker-controlled may have been validated upstream, while a dangerous-looking sink may be unreachable from any untrusted path. Code that looks safe in isolation may also become exploitable through behavior elsewhere in the repository. Consequently, agents that reason only from local snippets can miss real vulnerabilities or report findings that do not survive broader analysis.
dfs-large1
is trained jointly on three tasks: detecting vulnerabilities in application code, detecting vulnerabilities in low-level systems code, and validating candidate findings. We include both detection and validation as learning objectives because they provide complementary learning signals. Low-level repositories often expose fuzz targets and reproducers that make defects directly testable, with a reproducible address sanitizer (ASan) report providing strong evidence of reachable memory corruption. Web application findings are harder to validate automatically because they often depend on application state and external services. Training across both combines direct verification signals from systems code with the diversity of application security tasks, while relying on the same core reasoning about attacker control and reachability.
The training recipe below targets the reasoning required for broad vulnerability discovery, rigorous validation, and end-to-end data flow analysis. It also optimizes for efficient token and tool use.
Training Recipe #
dfs-large1
is post-trained with a multi-task RL recipe designed for long-horizon tasks.
Reward Design
Each completed trajectory receives a scalar task reward, which we shape before computing advantages. The adjusted reward incorporates two behavioral priors motivated by our production workload: an effort penalty that encourages efficient investigation and a report budget that discourages speculative findings.
Effort Penalty
We measure trajectory effort using a linear combination of action tokens, agent rounds, and tokens returned by tools. We then apply the following penalty as a function of effort:
Ceffort(x)=λlog(1+α)log(1+αx)Here, x represents aggregate trajectory effort, λ controls the overall penalty strength, and α controls its curvature. We choose an increasing and concave penalty function because we believe the marginal penalty for additional work should decrease as effort grows. In this manner, extra action tokens, turns, and tool output tokens for short rollouts are penalized more heavily than long investigations. This is inspired by Composer 2’s rationale, where they encourage efficiency on simpler tasks without suppressing the sustained exploitation required for more difficult tasks [1]. We use a simpler logarithmic form with one curvature parameter to make tuning simpler.
We observed that including this additive effort cost in the shaped reward increased parallel tool use while reducing or holding constant the number of agent rounds and policy action tokens. In ablations without the penalty, rounds and action tokens continued to grow indefinitely with additional RL compute.
Finding Budget
Within an agent audit scope, true vulnerabilities are typically sparse, so we apply a soft budget penalty to the number of findings returned by our detection agent:
q(n;Bt)=max(1,n−Bt+1)1Here, n is the number of reported vulnerabilities and Bt is the full-credit budget at training stage t. Reports within the budget retain full credit, while additional vulnerabilities reported progressively scale it down. Our desire is to discourage speculative findings proposed by an agent without imposing a hard cap for audit scopes genuinely containing multiple vulnerabilities.
We begin training with a large budget and gradually lower it in stages, increasing pressure against over-reporting only after the policy has established baseline recall. In controlled toy runs, we found this penalty was essential to preserve data-flow reasoning; without it, the policy would devolve into reporting locally plausible vulnerabilities without establishing any reachability or exploitability evidence. As discussed below, this penalty also encouraged greater use of project harnesses to test reachability and exploitability.
Policy Optimization
We optimized dfs-large1
with a masked importance sampling objective chosen for stability under the policy mismatch introduced by our training system. Our asynchronous actor-learner pipeline overlaps rollout generation with optimization, which is essential for tasks that span roughly 100 agent turns. Synchronizing each update on the slowest trajectories would otherwise leave GPUs idle [2], [3]. The tradeoff for this speedup is policy lag, since the learner may advance several steps before consuming trajectories sampled by our actors.
A separate source of mismatch exists between rollout and training engines. Even under identical weights, differences in numerical precision, kernel implementations, etc. can produce different token probabilities [4]. In MoE models, these small numerical differences can also change which experts are selected, further amplifying this discrepancy. We mitigate this by recording the expert routes selected per generated token and replaying them in the training forward pass [5].
We correct for both sources of mismatch using a masked importance-sampling REINFORCE update presented by Periodic Labs [6]. Following this formulation, the update is:
Empirically, masking tokens outside our trust region from the surrogate objective produced more stable asynchronous training than clipped alternatives. CISPO clips extreme importance weights while retaining every token’s gradient contribution [7], but this was less effective for our long-horizon tasks. We suspect that a substantial fraction of these p99 logprob differences reflect trainer-sampler mismatch from numerical and systems differences rather than useful policy movement. We therefore treat ratios outside the trusted interval as unreliable training signals, and we remove them from the loss calculation.
Advantage Estimation
We use Dr. GRPO to compute advantages, centering rewards within each rollout group while removing group standard deviation and response-length normalization [8]. These terms can overweight low-variance prompts and weaken negative updates on long trajectories.
In multi-task training, however, different task families can produce advantages at substantially different scales. We’ve observed that this can lead high-variance tasks to dominate the shared policy update for reasons unrelated to the quality of their learning signals. Following AgentRL [9], we apply an additional task-level normalization:
Context Compaction
For long-horizon rollouts, dfs-large1
reuses the summarization-based context compaction recipe introduced with dfs-mini1
[10]. As the context window fills up, the policy compresses the trajectory into a concise state summary and continues from that representation rather than the full transcript. Because the same policy generates these summaries and receives the final trajectory reward, context management becomes part of the learned agent behavior.
Results: Learned Audit Strategy #
During training, we observed that dfs-large1
’s audit strategy shifted from broad enumeration toward testing specific vulnerability hypotheses. In judged samples, later checkpoints spent a larger share of turns tracing attacker-controlled inputs and used executable harnesses more often, while repository reads and searches declined.
These two condensed trajectories show the shift: dfs-large1
tests critical assumptions and carries hypotheses through to executable evidence.
We suspect the finding budget, effort penalty, and joint training on vulnerability validation all contributed to this shift. The finding budget creates pressure against unsupported reports, the effort penalty rewards targeted investigation over repeated browsing, and validation training rewards turning a hypothesis into executable evidence.
Real-World Vulnerability Discovery #
Beyond our evaluations, we are testing dfs-large1
across production code-scanning workflows. Separately, a previous checkpoint supported parts of our Ruby ecosystem campaign. That broader campaign validated 105 vulnerabilities across 34 projects, roughly 61% of them memory-safety issues [11]. We plan to apply dfs-large1
to more open-source security research.
Continued Training #
Ultimately, our goal is to deliver the best possible agentic product experience for customers using the depthfirst security harness. That means shorter time to actionable findings, lower cost per audit, and fewer false positives.
We have collected thousands of executable security environments. We have not yet saturated these environments, leaving substantial headroom to keep scaling the training recipe, annealing our finding penalty to further reduce false positives, and improving token and turn efficiency.
Evaluation Methodology #
Unless otherwise specified, all models are evaluated at high thinking difficulty and executed within depthfirst’s security harness. To preserve evaluation integrity, our sandboxes block internet access and expose only a shallow clone of each repository with depth 1 and no commit history. We also use LLM behavioral monitors to flag reward hacking, superficial pattern matching, or attempts to obtain information unavailable in the evaluation by searching the internet or inspecting future commits containing vulnerability patch information.
Finally, we reviewed every passing trajectory to verify that it reflected authentic vulnerability discovery, and we confirmed that no failures resulted from cybersecurity-safety refusals.
References #
[1] Cursor Research, “Composer 2 Technical Report,” arXiv:2603.24477, 2026. [2] W. Fu et al., “AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning,” arXiv:2505.24298, 2025.
[3] A. Piché et al., “PipelineRL: Faster On-policy Reinforcement Learning for Long Sequence Generation,” arXiv:2509.19128, 2025. [4] H. He and Thinking Machines Lab, “Defeating Nondeterminism in LLM Inference,” Thinking Machines Lab, 2025.
[5] W. Ma et al., “Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers,” arXiv:2510.11370, 2025. [6] R. Agarwal, “The Hitchhiker’s Guide to Frontier Reinforcement Learning,” ICLR 2026 Workshop on Scaling Post-Training for LLMs, 2026.
[[7]](https://arxiv.org/abs/2506.13585) MiniMax, “[MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention](https://arxiv.org/abs/2506.13585),” arXiv:2506.13585, 2025.
[[8]](https://arxiv.org/abs/2503.20783) Z. Liu et al., “[Understanding R1-Zero-Like Training: A Critical Perspective](https://arxiv.org/abs/2503.20783),” arXiv:2503.20783, 2025.
[9] H. Zhang et al., “AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework,” arXiv:2510.04206, 2025.
[10] A. Kommula and A. Michi, “Training State of the Art Vulnerability Discovery Agents through Reinforcement Learning,” depthfirst, 2026. [11] Z. Yu et al., “Behind the GitLab RCE: A depthfirst Journey into the Ruby Ecosystem,” depthfirst, 2026.