Microsoft FORGE: Three Lessons for Scaling Frontier AI Vulnerability Research

#

1. Basic Information

#

2. Quick Summary

Microsoft reported that to turn AI-driven vulnerability discovery into defensive outcomes, system design covering verification, patching, and regression testing—rather than raw candidate generation volume—along with reasoning cost allocation and a continuous evidence loop are critical.

#

3. Defensive Challenges

• While AI can generate a large volume of vulnerability candidates, the results do not translate into defensive outcomes unless they progress through build reproduction, attacker control, deduplication, impact analysis, patching, and regression testing.

#

4. Proposed Methodology and Workflow

• Combine candidate generation, automated verification, human review, patching, regression testing, and release into a continuous loop backed by evidence.
• Allocate reasoning budgets based on uncertainty, impact, and verifiability rather than increasing them uniformly.

1. Broad analysis extracts suspicious code paths.
2. Component-specific provers and harnesses verify reachability and execution results.
3. Human reviewers assess security impact and invariant violations.
4. Component teams evaluate patches and compatibility risks to connect to release validation.
5. Failures, rejections, and regression results are also fed back into the next scan as structured evidence.

#

5. Inputs and Outputs

Inputs

• Source code, binaries, symbols, build artifacts, component-specific harnesses, configurations, runtime conditions, past validation results, and maintainer feedback.

Outputs

• Reproducible triggers/PoCs, causal paths, security impact, uncertainty, patch suggestions, regression tests, and rejection/duplication reasons.

#

6. Evaluation Design and Results

• Windows: Reported 140 CVEs addressed in Windows security releases from May to September 2026, with 52 included in the September release. Monthly figures count CVEs grouped by announcement and security-update release, not by discovery date or scan throughput.
• Open-source: Over 3 months across 23 projects, 155 internally verified reports were generated (including 39 by 50 hackathon participants). 93 achieved maintainer acknowledgment or acceptance.
• Linux kernel: Supporting evidence was generated for 627 findings. Using GPT-5.5 for verification, generating PoCs for 182 confirmed crashes took an average model cost of $3.61 and 21.5 minutes, while automated exploit generation (AEG) to examine local privilege escalation potential for 6 selected cases averaged $8.56 and 25.4 minutes.
• Averages include all trials associated with reported successful cases, but exclude initial triage, failed or excluded cases, human investigation, and patch preparation. They do not represent the average cost or time for the entire research process, nor do they mean that privilege escalation succeeded in all six cases.
• Outputs belong to different stages, and report counts cannot be equated with public CVEs or shipped fixes.

#

7. Practical Implications

• Measure success by validated triggers, decision-changing evidence, patch rework time, regression evidence, and the time from defect to approved fix or release, rather than raw candidate volume.
• Storing reasons for failed validation (such as unreachable paths, missing preconditions, wrong builds, insufficient attacker control, or duplicates) turns them into learning assets.

#

8. Success Conditions and Limitations

Success Conditions

• Ability to provide binaries, symbols, configurations, and reproduction harnesses for the target version.
• Clear handoffs and responsibilities among models, provers, human reviewers, component owners, and release teams.
• Ability to track submitted, accepted, fixed, and released states separately.

Limitations

• As Microsoft's own research program, the same costs and yields cannot be generalized to other organizations.
• The 155 items are internally verified reports, and 93 are acknowledgments or acceptances, which does not mean all of them became CVEs, were patched, or were publicly disclosed.
• The aggregate data does not provide a complete breakdown of false positives, duplicates, severity, or fix completion rates.

#

9. Implementation Guidance and Required Evidence

Implementation Guidance

• Establish component-specific build/test harnesses and owners in a small pilot, and make status from findings to release machine-readable.
• Focus reasoning and human review on high-impact candidates, and return failed validation reasons to the dataset.
• Avoid automatically applying patch suggestions; human owners must approve invariants, compatibility, regression, and servicing.

Required Evidence

• Preserve target commits/builds, compilers/configurations, triggers/PoCs, crash/sanitizer traces, attacker-controlled inputs, reachability, human decisions, patch diffs, regression tests, and release status.
• Cost comparisons should separate model usage, compute, elapsed time, human review, setup costs, duplicate rates, and sample sizes per project.

#

10. Facts, Inferences, and Hypotheses

Facts

• Microsoft reported 140 CVEs addressed in Windows security releases from May to September 2026, with 52 included in the September release. Monthly figures count CVEs grouped by announcement and security-update release, not by discovery date or scan throughput.
• In open-source projects, 155 internally verified reports (including 39 by 50 hackathon participants) were submitted to 23 projects over 3 months, with 93 achieving maintainer acknowledgment or acceptance across 14 projects/families.
• In the Linux kernel, the validation agent generated supporting evidence for 627 findings. Using GPT-5.5 for verification, generating PoCs for 182 confirmed crashes took an average model cost of $3.61 and 21.5 minutes, while AEG examining local privilege escalation potential for 6 selected cases averaged $8.56 and 25.4 minutes.
• Averages include all trials associated with reported successful cases, but exclude initial triage, failed or excluded cases, human investigation, and patch preparation. They do not represent average costs or times for the entire research process, nor do they mean privilege escalation succeeded in all six cases.
• Reports span different stages such as submitted, acknowledged, accepted, publicly disclosed, and fixed; not all 155 reports mean CVE creation and fixing.

Inferences

• Defensive KPIs should focus on reproducible triggers, evidence that component owners can act on, approved fixes, regression tests, and time to release, rather than raw finding counts.
• Because low model costs can still lead to bottlenecks from inadequate build, harness, or human review preparation, investments in component-specific validation environments are necessary.

Hypotheses

No additional hypotheses. Unverified items are listed in section 11.

#

11. Unresolved Questions and Further Investigation

• Candidate counts per target Windows component, duplicate rates, false positive rates, and time from verification to patch delivery.
• Acceptance criteria for the 155 reports, handling of duplicates, and the proportion that ultimately reached a fix.
• Costs for execution infrastructure, environment setup, human investigation, and patch preparation separate from published model costs, along with overall verification efficiency including failed cases and the scope for generalization to other projects.

#

12. Implications for Defenders

Organizations introducing AI into vulnerability research should avoid treating candidate counts as outcomes, and instead track triggers reproducible in the target build, confirmation of attacker control, component owner judgment, and patches through regression testing within the same workflow. Similarly, SOCs can benefit from a design that feeds not only automated triage success rates but also false positive reasons, non-reproducibility conditions, and post-fix detection/regression evidence back into subsequent model and rule improvements.

#

13. Summary by Target Audience

For SOCs : Track reproduction conditions, attacker control, owner decisions, and fix/regression evidence rather than AI-generated finding counts. #

For Administrators : Establish clear divisions of responsibility for CI/CD-integrated validation harnesses, symbols/binaries, component owners, and patch reviews. #

For Users : No action is required for general users. Follow administrator instructions when product updates are released.

source & further reading

dev.to — original article My Eval Said RAG Made Things Up. My Eval Was Wrong. Asynchronous Parallel Validation & Diff Report Generation Tool for Multiple AI Platforms I built a free n8n workflow that turns messy meeting notes into action items