# Testing LLMs against seL4: Can AI break formally proven code?

> Source: <https://promptcube3.com/en/news/4190/>
> Published: 2026-07-29 03:57:14+00:00

# Testing LLMs against seL4: Can AI break formally proven code?

In a perfect world, the AI should find absolutely nothing because the proof covers the implementation. However, there is a massive gap between a mathematical proof and the actual execution of code on physical hardware. If you're planning a deep dive into this, here are the specific angles where an LLM agent might actually find something useful:

## The Specification Gap

Formal verification doesn't prove a system is "perfect"; it proves the code matches the specification. If the specification itself has a logical flaw or misses a real-world hardware quirk, the proof is valid, but the system is still vulnerable. An AI trained on thousands of CVEs might recognize a pattern—like a specific race condition or a side-channel leak—that the original formal specification simply didn't account for.

## Compiler and Hardware Mismatches

The seL4 proof typically targets a specific model of the CPU. But we aren't running code on a mathematical model; we're running it on silicon. There are three layers where things break:

1. The C code is verified.

2. The compiler transforms it into binary.

3. The binary runs on a CPU with branch predictors and speculative execution.

AI is particularly good at identifying "weird" hardware behaviors. If you feed an LLM the assembly output of a verified kernel and ask it to look for speculative execution vulnerabilities (like Spectre-style leaks), it might spot a sequence that the formal proof ignored because the proof assumed the CPU executes instructions linearly.

## Practical Testing Workflow

If anyone wants to try this as a real-world experiment, don't just paste a file into a chat window. To get actual results, you need a structured AI workflow:

1. **Context Loading:** Feed the AI the formal specification documents first so it understands the "rules" the kernel is supposed to follow.

2. **Differential Analysis:** Provide the verified C source and the compiled binary. Ask the AI to find discrepancies where the compiler might have introduced an optimization that violates the intended security property.

3. **Targeted Prompting:** Instead of asking "find a bug," use specific prompt engineering to target known microkernel weaknesses:

```
Analyze the following memory management routine for potential TOCTOU (Time-of-Check to Time-of-Use) vulnerabilities, specifically focusing on how the hardware cache might interact with the verified state transitions.
```

Ultimately, using AI on proven software isn't about proving the mathematicians wrong; it's about finding the "seams" where the math ends and the physical hardware begins. That's where the real security flaws usually hide.

[Generative AI Training Data: The Pirated Book Controversy 55m ago](/en/news/4186/)

[Hacker News Workflow: Stop Switching Tabs for Comments 1h ago](/en/news/4182/)

[LearnVector: Scaling Personalized Education with AI 1h ago](/en/news/4180/)

[Manim WebGPU: Running Math Animations in the Browser 2h ago](/en/news/4176/)

[Web Scraping Rights: Why Google and Reddit Don't Own the Web 2h ago](/en/news/4173/)

[TSMC Arizona Expansion: The AI Hardware Bottleneck 3h ago](/en/news/4169/)

[Next Generative AI Training Data: The Pirated Book Controversy →](/en/news/4186/)