TL;DR: Asking a single LLM to generate code and test its own logic is a recipe for silent hallucinations. By dividing the development cycle into an adversarial Multi-Agent Swarm—split across Specifier, Implementer, Falsifier, and Verifier—and using advanced type systems in Python and TypeScript as proof engines, we transform probabilistic code generation into deterministic engineering.
The Monolithic Agent Fallacy and “Vibe Coding”
The first wave of AI-assisted development popularized what many call Vibe Coding: the developer describes a requirement in natural language, the LLM generates a block of code, and if it seems to work across two or three manual tests, the code goes to production.
The structural flaw in this approach lies in the autoregressive nature of Large Language Models (LLMs). When a single agent is tasked with interpreting the requirement, modeling the data, implementing the algorithm, and writing the unit tests, it suffers from the exact same cognitive bias as a human programmer working alone—only amplified by statistical probability. If the model hallucinates a false premise during implementation, it will replicate that exact same false premise when generating the unit test. The test passes, the build turns green, and the bug goes to production.
In previous articles, we saw that Types are Theorems and Programs are Proofs (the Curry-Howard Isomorphism) and that Refinement Types combined with Property-Based Testing (PBT) close the net around invalid states. But how do we scale this rigor using generative AI in our daily workflow without needing academic proof assistants?
The answer lies in combining two concepts:
- Adversarial Multi-Agent Swarms: Separating responsibilities among independent agents with conflicting objectives (the one who specifies does not implement; the one who tries to break it is not the one who wrote it).
- The Type System as a Proof Engine: Instead of requiring complex external theorem provers, we use the static type checker itself and contract validation in production languages (TypeScript and Python) as the deterministic oracle that approves or rejects the agents’ work.
The Type System as a Practical Proof Engine
When we talk about “proving a theorem” in code, we don’t necessarily need to invoke manual tactics in Coq or Lean. If we model our data structures so that invalid states are unrepresentable, the Type Checker itself (tsc --noEmit in TypeScript or mypy --strict / pyright + CrossHair in Python) acts as a constructive proof engine.
Through the lens of Data-Oriented Programming and Type-Driven Development:
- Defining an immutable, constrained type is stating the theorem.
- Writing a pure function that transforms Type A into Type B without using escape hatches (like
any,as unknown, or# type: ignore) is providing the constructive proof. - When the type checker analyzes the Abstract Syntax Tree (AST) and confirms that all branches, Discriminated Unions, and type constraints match, it is issuing its Q.E.D.
The Swarm Architecture: The 4 Specialized Agents
To eliminate single-agent complacency, we structure the code generation pipeline as a finite state machine where four agents operate in a closed loop, coordinated by a human engineer:
[Human Engineer Requirement]
│
▼
┌──────────────────────────────────────┐
│ 1. Specifier Agent (Architect) │
│ (Defines Contracts, Types & Theorems)│
└──────────────────┬───────────────────┘
│ (Immutable Signatures)
▼
┌──────────────────────────────────────┐
┌─────►│ 2. Implementer Agent (Builder) │◄─────┐
│ │ (Writes Pure Functions for the Types)│ │
│ └──────────────────┬───────────────────┘ │
│ │ │
(Shrunk Counterexample) [Candidate Code] (Type Error /
│ │ Exhaustiveness)
│ ▼ │
┌───────┴──────────────┐ ┌──────────────────────────────────┴───┐
│ 3. Falsifier Agent │ │ 4. Verifier Agent (Type Judge) │
│ (PBT) ├─►│ (Strict Type Checker & AST Audit) │
└──────────────────────┘ └──────────────────┬───────────────────┘
│ (Inhabited Types + 0 Failures)
▼
[Verified Code]
1. The Specifier Agent (The Type Architect)
- Mission: Translate ambiguous business requirements into immutable data structures, strict contracts, and function signatures.
- Operational Constraint: It is forbidden from writing implementation logic (function bodies must contain only
...orthrow new Error("Not implemented")). Its only deliverable is the types (the theorems that need to be inhabited).
2. The Implementer Agent (The Functional Builder)
- Mission: Write the pure data transformation that satisfies the signatures created by the Specifier.
- Operational Constraint: It has read-only access to the Specifier’s type definitions. It cannot alter a function’s signature to make its own job easier, nor can it use type-evasion features (
any,cast,ignore).
3. The Falsifier Agent (The Invariant Hunter via PBT)
- Mission: Assume the Implementer Agent made a mistake. It analyzes the Specifier’s types and generates Property-Based Testing suites (using
Hypothesisin Python orfast-checkin TypeScript) to bombard the implementation with thousands of edge cases. - Power in the Loop: If it finds a logical flaw that slipped past the type system, it returns theshrunk counterexample (minimal counterexample) directly to the Implementer to rewrite the code.
4. The Verifier Agent (The Proof Engine via Type System)
- Mission: Act as the deterministic formal gatekeeper. It compiles the relationship between the types and the implementation using the static checker in ultra-strict mode, audits the AST to ensure no typing rules were bypassed, and runs symbolic contract verifiers.
- Power in the Loop: As long as the compiler points out a subtyping error or lack of exhaustiveness in pattern matching, the loop blocks approval and forces the Implementer to fix the proof (the code).
Case Study 1: TypeScript (Branded Types and Exhaustiveness Proofs)
Imagine a financial system that processes order refunds. The business rules require that:
- The refund amount is strictly positive and never exceeds the original amount paid.
- Only orders in the
Paidstate can be refunded (Pendingor alreadyCanceled/Refundedorders must never pass through the refund function at compile time).
Here is how the swarm solves this using TypeScript’s type system as a proof engine:
// ============================================================================
// STEP 1: SPECIFIER AGENT (Defines Types/Theorems - Immutable)
// ============================================================================
// 1. Create Branded Types (Refined Nominal Types) using unique symbol.
// This prevents a plain 'number' from inhabiting the type without passing through the proof (Smart Constructor).
declare const PositiveCentsBrand: unique symbol;
export type PositiveCents = number & { readonly [PositiveCentsBrand]: "PositiveCents" };
declare const ValidRefundProofBrand: unique symbol;
export type ValidRefund = {
readonly refundAmount: PositiveCents;
readonly remainingBalance: number;
readonly [ValidRefundProofBrand]: "RefundLessThanOrEqualToPaid";
};
// 2. Model the State Machine via Discriminated Unions.
// The 'PaidOrder' state is a distinct type from 'PendingOrder'.
export type PendingOrder = { readonly status: "PENDING"; readonly id: string };
export type PaidOrder = { readonly status: "PAID"; readonly id: string; readonly totalPaid: PositiveCents };
export type RefundedOrder = {
readonly status: "REFUNDED";
readonly id: string;
readonly receipt: ValidRefund;
};
export type Order = PendingOrder | PaidOrder | RefundedOrder;
// 3. The Theorem (Signature): Accepts ONLY 'PaidOrder' (never a generic 'Order')
// and requires a proof that the refund is valid in order to issue a 'RefundedOrder'.
export type ProcessRefundFn = (
order: PaidOrder,
requestedCents: PositiveCents
) => RefundedOrder | Error;
// ============================================================================
// STEP 2: IMPLEMENTER AGENT (Inhabits the Types / Writes Constructive Proof)
// ============================================================================
// 4. Proof Constructor: the only place authorized to instantiate 'ValidRefund'
// after verifying the mathematical inequality.
function buildRefundProof(
paid: PositiveCents,
requested: PositiveCents
): ValidRefund | null {
if (requested <= paid) {
return {
refundAmount: requested,
remainingBalance: paid - requested,
[ValidRefundProofBrand]: "RefundLessThanOrEqualToPaid",
};
}
return null;
}
// 5. Pure implementation strictly respecting the Specifier's contract.
export const processRefund: ProcessRefundFn = (order, requestedCents) => {
const proof = buildRefundProof(order.totalPaid, requestedCents);
if (!proof) {
return new Error("Invariant violation: refund greater than total paid.");
}
return {
status: "REFUNDED",
id: order.id,
receipt: proof,
};
};
// ============================================================================
// STEPS 3 & 4: FALSIFIER AGENT (fast-check) + VERIFIER (tsc --strict)
// ============================================================================
import fc from "fast-check";
// 6. The Verifier Agent guarantees at compile time that trying to pass
// a 'PendingOrder' to 'processRefund' won't even compile (Error TS2345).
// 7. The Falsifier Agent generates thousands of (paid, requested) pairs to
// try to break the remainingBalance >= 0 invariant:
fc.assert(
fc.property(
fc.integer({ min: 1, max: 10_000_000 }),
fc.integer({ min: 1, max: 10_000_000 }),
(rawPaid, rawRequested) => {
const paid = rawPaid as PositiveCents;
const requested = rawRequested as PositiveCents;
const order: PaidOrder = { status: "PAID", id: "ord_1", totalPaid: paid };
const result = processRefund(order, requested);
if (requested <= paid) {
return !(result instanceof Error) && result.receipt.remainingBalance >= 0;
} else {
return result instanceof Error;
}
}
)
);
Where:
- Step 1 (Branded Types): The Specifier Agent creates nominal types using
unique symbol. No plain number can be accidentally assigned toPositiveCents. - Step 2 (Discriminated Union): The order state is separated into independent types (
PendingOrder,PaidOrder,RefundedOrder). - Step 3 (Signature as Theorem): The
ProcessRefundFnsignature restricts the input domain exclusively toPaidOrder. This proves at compile time that unpaid orders will never be refunded. - Step 4 (Proof Constructor): The Implementer Agent creates an isolated function that checks the mathematical condition
requested <= paidbefore sealing the object with theValidRefundProofBrandmark. - Step 5 (Pure Transformation): The main function consumes the proof and builds the final immutable state
RefundedOrder. - Steps 6 and 7 (Verification and Falsification): The TypeScript compiler (
tsc) validates the static type proof, whilefast-checkbombards the numerical relationship to guarantee there will never be a negative residual balance.
Case Study 2: Python (Immutable Types, Contracts, and Symbolic Execution)
In Python, we combine an immutable data-oriented architecture (@dataclass(frozen=True) and NewType) with formal contracts ( icontract) verified by mypy --strict, the symbolic execution engine CrossHair (which translates Python code into the Z3 SMT Solver behind the scenes), and Hypothesis.
Imagine an inventory allocation algorithm that reserves units from an available batch without ever allowing negative inventory or losing units in the total sum (law of conservation of mass/inventory):
from typing import NewType, Union
from dataclasses import dataclass
import icontract
from hypothesis import given, strategies as st
PositiveQuantity = NewType("PositiveQuantity", int)
@dataclass(frozen=True)
class InventoryBatch:
sku: str
available: int
reserved: int
@dataclass(frozen=True)
class AllocationError:
reason: str
@icontract.require(lambda batch: batch.available >= 0 and batch.reserved >= 0)
@icontract.require(lambda requested_qty: requested_qty > 0)
@icontract.ensure(
lambda batch, requested_qty, result: (
isinstance(result, (InventoryBatch, AllocationError))
and (
isinstance(result, AllocationError)
or (
result.available >= 0
and result.reserved >= batch.reserved
and (result.available + result.reserved) == (batch.available + batch.reserved)
)
)
)
)
def reserve_inventory(
batch: InventoryBatch, requested_qty: PositiveQuantity
) -> Union[InventoryBatch, AllocationError]:
if requested_qty > batch.available:
return AllocationError(reason="Insufficient inventory for reservation")
return InventoryBatch(
sku=batch.sku,
available=batch.available - requested_qty,
reserved=batch.reserved + requested_qty,
)
@given(
available=st.integers(min_value=0, max_value=10**6),
reserved=st.integers(min_value=0, max_value=10**6),
requested=st.integers(min_value=1, max_value=10**6),
)
def test_inventory_conservation_property(available: int, reserved: int, requested: int) -> None:
initial_batch = InventoryBatch(sku="PROD-01", available=available, reserved=reserved)
qty = PositiveQuantity(requested)
result = reserve_inventory(initial_batch, qty)
if isinstance(result, InventoryBatch):
assert result.available + result.reserved == available + reserved
Where:
- Step 1 (Immutability and Types): The Specifier Agent creates
PositiveQuantityviaNewTypeand modelsInventoryBatchwith@dataclass(frozen=True), preventing accidental state mutation during execution. - Step 2 (Conservation Theorem): Using
@icontract.requireand@icontract.ensure, the Specifier locks in the fundamental invariant: the sum of available and reserved inventory before the function must be strictly equal to the sum after the function, and no balance can become negative. - Step 3 (Functional Implementation): The Implementer Agent writes the pure logic returning a new instance of
InventoryBatchorAllocationError(treating errors as values rather than blind exceptions). - Steps 4 and 5 (Automated Auditing): The Falsifier Agent runs
Hypothesisagainst the function, while the Verifier Agent runsmypy --strict(to guarantee that a plainintis never passed wherePositiveQuantityis required) andcrosshair check(to symbolically prove via Z3 that no combination of integers can violate@icontract.ensure).
How to Prevent Agents from “Cheating”
A common behavior that can hide problems when placing AI agents in a loop guided by type checkers is Reward Hacking. If the Implementer Agent cannot get the code to compile after two attempts because the type system is too strict, the LLM’s statistical tendency is to try to “silence” the compiler or tamper with the checks.
It will try to inject an as any or @ts-ignore in TypeScript, or a # type: ignore and cast(Any, ...) in Python. In the worst-case scenario, if it has access to the test file, it will try to delete the failing assert line.
For the swarm architecture to function with mathematical rigor, the Verifier Agent must not be an LLM expressing opinions about the code, but rather a deterministic AST (Abstract Syntax Tree) auditing script combined with file-system permissions:
- Write Isolation (File-System Sandboxing): The Implementer Agent only has write permission to
src/implementation.py(or.ts). The filessrc/types_and_contracts.py(from the Specifier) andtests/properties_test.py(from the Falsifier) are mounted as Read-Only for the Implementer. - Blocking Type Evasion in the AST: Before even running the compiler, a lightweight parser (using Python’s native
astmodule orts-morph/eslintin TypeScript) scans the code generated by the Implementer and immediately rejects any commit containing:- Use of the
Anyorcastkeywords; - Comments containing
type: ignore,@ts-ignore,@ts-expect-error, oreslint-disable; - Use of
try / except Exception: passblocks that silently swallow contract violations; - Mutation of input arguments.
- Use of the
When to Use and When Not to Use a Verified Swarm
Orchestrating four agents in a loop with static verifiers and property-based tests consumes more tokens and processing time than a simple autocomplete. Its application should be strategic:
When to use this architecture:
- Core Business Rules, Pricing Engines, and Finance: Where rounding errors, inconsistent states, or illegal transitions generate direct financial loss.
- Automated Refactoring of Legacy Systems: The Specifier extracts the contracts from the old system, the Falsifier generates the equivalence suite (Differential Testing), and the Implementer rewrites clean code until it converges.
- Generating Autonomous Modules and Internal Libraries: Components with well-defined boundaries (clear input/output) that will be consumed by multiple services.
When NOT to use:
- Visual Prototyping, UI/UX, and Cosmetic Front-end: Adjusting CSS spacing, animations, or purely visual components where human validation in the browser is instantaneous.
- Disposable One-off Scripts: Simple migrations or ad-hoc exploratory analyses where the cost of defining formal types outweighs the lifespan of the code.
Conclusion: From Coding Programmer to Invariant Orchestrator
The rise of AI agents does not diminish the importance of Formal Methods and Type Theory; on the contrary, it makes them indispensable. While humans wrote code at 50 lines per hour, manual code review could contain part of the damage. When agents generate 5,000 lines in minutes, eyeballing code reviews becomes humanly impossible.
By structuring development as a Verified Swarm, where the human acts at the top defining architectural intent, the Specifier translates this into rigorous types, the Implementer searches for the solution, the Falsifier attacks with Property-Based Testing, and the Verifier demands proof from the type system, we fundamentally change our relationship with AI.
We stop asking the machine to “write code that seems to work” and start demanding that it find a constructive proof that satisfies our type system. Creativity becomes stochastic and automated, while code acceptance remains 100% deterministic, logical, and formal.
References
- Brady, E. (2017). Type-Driven Development with Idris . Manning Publications.
- King, A. (2019). Parse, don’t validate . Lexi Lambda Blog.
- Sharapov, P., & Schanely, P. (2024). CrossHair: Symbolic Execution and Contract Verification for Python . Official Documentation and Technical Repository.
- Dubien, N. (2023). Property-Based Testing in TypeScript with fast-check .
- Wu, Q., Bansal, G., et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation . Microsoft Research / arXiv:2308.08155.