Engineering Verifiable Systems: From Zero-Knowledge Proofs to AI Agent Trust

Originally published on tamiz.pro.

The fundamental unit of trust in software has historically been the authority of the entity that built the system. We trust sha256 because NIST says it’s secure; we trust our payment processor because it has a brand; we trust our AI model because the vendor claims it aligns with safety guidelines. This "Trusted Execution Environment" (TEE) model, whether physical or logical, is breaking. As we move into an era of autonomous AI agents, decentralized financial protocols, and distributed edge computing, we can no longer rely on a central party to attest to the integrity of an operation. We must shift from asking "Do I trust this system?" to "Can I verify this system?"

This article explores the engineering of Verifiable Systems. It dissects how Zero-Knowledge (ZK) proofs, cryptographic receipts, and attestation frameworks allow us to prove the correctness of computation without revealing the data, or without exposing the internals of the process. We will bridge the gap between high-level blockchain anti-cheat mechanisms and the emerging challenge of verifying the behavior of Large Language Model (LLM) agents.

To understand why verifiable systems are critical now, we must look at the two primary domains where trust is failing:

The solution lies in replacing "trust" with "proof." A verifiable system generates a cryptographic artifact that allows a third party to verify a claim (e.g., "computation X was performed correctly") in milliseconds, without needing to re-execute the entire computation or inspect the private inputs.

Before we can engineer a verifiable system, we must understand the three pillars that support it.

ZKPs allow a prover to convince a verifier that a statement is true without conveying any information aside from the truth of the statement. In software engineering terms, this is the difference between saying "I know the password" (verifying by showing the password) and saying "Here is a hash of the password that matches the known hash" (verifying without revealing it).

There are two main families of ZKPs relevant to production systems:

TEEs are hardware-isolated enclaves within a CPU (like Intel SGX or ARM TrustZone). They allow code to run in a protected memory region that is isolated from the Operating System and other processes. The hardware generates an "attestation"—a cryptographic signature that proves the code running inside the enclave has a specific hash. While TEEs provide strong security, they rely on the CPU vendor's integrity. ZKPs, by contrast, are mathematical and verifiable by anyone.

While not "zero-knowledge" in themselves, Merkle trees are essential for batching proofs. In a game or ledger, every state change is a leaf in a tree. To prove a specific user's balance, you don't need to publish the whole database; you just publish the Merkle path (a logarithmic number of hashes). This optimizes the data size of ZKP inputs.

A modern verifiable system usually follows a Prove-Verify pattern. Let's visualize this in a generic pipeline, independent of the specific domain (gaming or AI).

[ Data / Logic ] -> [ ZK Circuit ] -> [ Witness Generation ] -> [ ZK Proof ] -> [ On-Chain / Client Verification ]

IF action == "purchase" AND balance >= cost AND item_stock > 0 THEN output = "success". In traditional online games, anti-cheat is a client-side process. Cheaters can easily bypass it by modifying the game client to report false coordinates or health values. The server cannot distinguish between a legitimate move and a spoofed input without re-simulating the entire game state, which is too expensive.

The ZK Solution:

Instead of sending raw inputs, the client runs the game logic inside a local, tamper-resistant environment (often a TEE) or compiles the movement logic into a ZK circuit.

new_position is valid given old_position, input_vector, and physics_constants, without revealing the specific This shifts the burden of integrity from the network to the mathematics.

This is the cutting edge. LLMs are non-deterministic, probabilistic, and opaque. An agent that controls a browser or a financial account introduces a "Principal-Agent" problem in reverse: The human (principal) cannot easily monitor the AI (agent).

How do we verify that an AI agent did what it claimed?

We can engineer a system where every significant action taken by an AI agent is accompanied by a cryptographic receipt. This receipt is not just a log entry; it is a ZK proof that validates the agent's intent and execution constraints.

The Workflow:

action_type == "purchase" amount < 500 domain == "https://secure-coffee.com" timestamp < expiry_date ZK proofs are deterministic. LLMs are not. How do we prove a probabilistic output?

We don't prove the output directly; we prove the bounds of the output.

This is similar to how we trust a random number generator: we don't check every bit; we check that the sequence passes statistical tests. For AI, we check that the behavior passes constraint tests.

Building these systems is not just about plugging in a library. It requires deep consideration of performance and security.

Writing ZK circuits is painful. High-level languages like Circom (for Solidity/Blockchain) or Ligature (for Rust) help, but translating general-purpose logic (like a Python AI agent's decision tree) into arithmetic constraints is non-trivial.

For zk-SNARKs, the trusted setup is a single point of failure. If the

certifying party colludes with malicious participants, the entire proof system collapses. The generated toxic parameters can be used to generate fake proofs for false statements.

Mitigation Strategies:

A zero-knowledge proof verifies the computation was performed correctly according to the circuit. It does not verify that the input data was truthful or that the AI model’s weights were not maliciously altered to produce "poisoned" outputs.

Solution: Compositional Trust

You cannot ZK-proof the entire AI reasoning process. Instead, you must decompose the trust model:

This section provides a practical, runnable example of a lightweight verification layer. While full ZK implementations are complex, we can simulate the cryptographic integrity checks and logic verification using Python and the eth-hash library to represent on-chain anchors.

We are verifying that an AI Agent has adhered to a "No-PII" policy. The circuit takes the output text and checks for regex patterns associated with emails and phone numbers.

import hashlib
import json
import re

class PolicyVerifier:
    def __init__(self, policy_hash: str):
        """
        policy_hash: The Merkle root or hash of the policy definition.
        In a ZK context, this is part of the circuit constraints.
        """
        self.policy_hash = policy_hash
        self.regex_pii = [
            r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b', # Email
            r'\b\d{3}-\d{3}-\d{4}\b' # Phone
        ]

    def generate_zk_input_commitment(self, output_text: str) -> str:
        """
        Simulates the commitment of the output to the ZK circuit.
        """
        return hashlib.sha256(output_text.encode('utf-8')).hexdigest()

    def verify_policy_compliance(self, output_text: str) -> dict:
        """
        Simulates the ZK proof verification.
        In production, this would call a ZKVM (like RISC Zero or Lepton) 
        or a smart contract verifier.
        """
        violation_found = False
        violations = []

        for pattern in self.regex_pii:
            matches = re.findall(pattern, output_text)
            if matches:
                violation_found = True
                violations.extend(matches)

        proof_valid = not violation_found

        return {
            "proof_valid": proof_valid,
            "commitment": self.generate_zk_input_commitment(output_text),
            "violations_detected": violations,
            "policy_ref": self.policy_hash
        }

Here is how an AI agent interacts with a "Trust Layer" before submitting results to a verifier.

class TrustAwareAgent:
    def __init__(self, verifier: PolicyVerifier):
        self.verifier = verifier
        self.audit_log = []

    def execute_task(self, prompt: str) -> dict:
        """
        Simulates the LLM generation and subsequent verification.
        """
        raw_output = self._generate_response(prompt)

        verification_result = self.verifier.verify_policy_compliance(raw_output)

        self.audit_log.append({
            "prompt_hash": hashlib.sha256(prompt.encode()).hexdigest(),
            "output_hash": verification_result["commitment"],
            "proof_status": "PASSED" if verification_result["proof_valid"] else "FAILED",
            "timestamp": __import__('time').time()
        })

        if not verification_result["proof_valid"]:
            return {
                "status": "REJECTED",
                "reason": "Policy violation detected",
                "violations": verification_result["violations_detected"]
            }

        return {
            "status": "ACCEPTED",
            "output": raw_output,
            "proof_commitment": verification_result["commitment"]
        }

    def _generate_response(self, prompt: str) -> str:
        """
        Mock LLM response generator for demonstration.
        """
        if "email" in prompt:
            return "Your email is john.doe@example.com. Let us know if that's correct."
        else:
            return "The server is running optimally. No issues detected."

if __name__ == "__main__":
    policy_obj = {"rules": ["no_pii", "max_length_1000"]}
    policy_hash = hashlib.sha256(json.dumps(policy_obj, sort_keys=True).encode()).hexdigest()

    verifier = PolicyVerifier(policy_hash)
    agent = TrustAwareAgent(verifier)

    print("--- Test 1: Safe Query ---")
    result1 = agent.execute_task("Check server health")
    print(json.dumps(result1, indent=2))

    print("\n--- Test 2: PII Leak ---")
    result2 = agent.execute_task("Who is the admin?")
    print(json.dumps(result2, indent=2))

    print("\n--- Audit Log ---")
    for entry in agent.audit_log:
        print(entry)

As systems evolve from single agents to multi-agent swarms, trust becomes a graph problem. Agent A might trust Agent B, but not Agent C. ZK proofs allow for transitive trust.

Challenge: Latency.

ZK proof generation for complex agent logic can take seconds. For real-time interaction, you must use recursive proofs or proof aggregation.

We are moving from an era of implicit trust (we assume the API works as documented) to cryptographic verifiability (we prove the API acted as intended).

For AI engineers, this means:

The future of AI agents is not one where we blindly hope the model is aligned. It is one where we know it is aligned, because we hold the cryptographic key to that certainty. Build for verification. Build for trust.