Repligraphs: A Primitive for AI Provenance

A new artifact called a "repligraph" can prove that a piece of software was produced by a specific AI model, according to a post introducing the primitive, which claims its creators have "created and verified simple repligraphs that produce real software, using open-weights models and our deterministic harness." The approach relies on re-executing a deterministically run LLM with the same input to confirm identical output, and the authors built a bespoke deterministic WASM sandbox, described as a "console for agents," because nondeterminism from batching, tensor parallelism and cache layout makes standard harnesses like Claude Code unsuitable. The authors argue that as AI outclasses humans, proof that an artifact is "human-free" will become increasingly valuable, and they expect deterministic inference to become a standard offering of frontier labs and inference providers.

A repligraph is an artifact that was provably produced by a specific mind. Humans are not capable of producing repligraphs: There is no way for me to prove that I wrote this text. Sure, it’s hosted on my website, but that proves nothing about its provenance. Nor would a cryptographic signature help: Maybe I hired a ghostwriter. Maybe someone hacked me. There is a way, however, to prove that an artifact is AI-generated. An LLM is just a function. If that function is executed deterministically, then anyone can re-execute it with the same input and verify that it produces the same output. Ready to record and verify your first repligraph? Point your agent of choice at the GitHub repo and ask it to build something cool. In a world where AI increasingly outclasses humans, proof that an artifact is “human-free” is increasingly valuable. In particular, it will soon be considered grossly negligent to depend on a cryptography library implemented by humans. Humans are attack surface. We — and the models themselves — will demand libraries that are provably untainted by human hands. Yes, LLMs can write backdoors too; but constraining model behavior is, at its core, an engineering problem. Constraining human behavior is not. No matter who or what is writing the code, the security calculus is unchanged: If you spend less effort on defense than your attackers spend on offense, you are in danger. This suggests that even very powerful AIs will not “roll their own crypto.” Why waste the compute, when you could just import a battle-tested library that has withstood thousands of audits? In this way, repligraphs allow mutually distrusting parties to pool their compute, creating an artifact more secure than any one of them could create individually. It is a common misconception that LLMs are “inherently random.” This is unsurprising, given that the world’s most popular chatbots are served with nondeterministic inference. People put in the same prompt twice, get two different answers, and assume that this reflects a fundamental property of the technology. It does not. What is true is that inference is very sensitive. Batching, tensor parallelism, cache layout — all of these can influence the result. While these parameters can, in principle, be controlled, doing so is difficult and costly . And because there is presently little demand for determinism, no one bothers. And yet. It can be done. We have created and verified simple repligraphs that produce real software, using open-weights models and our deterministic harness. Once the value of deterministic inference is widely appreciated, we expect it to become a standard offering of frontier labs and inference providers. When people today talk about “AI,” they are usually referring not to an LLM, but to an agent: An LLM running inside a harness. A harness multiplies the capability of a model enormously, enabling it to search the web, understand sprawling codebases, delegate tasks to subagents, formulate and execute sophisticated plans, and produce artifacts that are far more impactful than a single chatbot reply. Repligraphs, too, are much more powerful with a harness. But crucially, the harness — like the inference driving it — must be deterministic. This is a surprisingly big ask. You cannot simply run Claude Code in a Docker image; there are far too many places where nondeterminism can sneak in. So we took a radically different approach, and built something bespoke: A batteries-included WASM sandbox, with deterministic versions of all the tools an agent needs. We call this a console for agents, and we expect it to suck for a while. Agents are RL’d to use bash in a Unix environment, not to directly invoke a compile tool. But in the long run, everyone prefers a fixed target to a moving one. Just as console game developers can optimize their hardware-specific code to within an inch of its life while PC game developers are at the mercy of the lowest common denominator , we expect that the guarantees of a fixed harness will unlock significant gains in performance, security, and functionality. At a high level, a repligraph has the following components: A set of profiles, specifying a model, an inference engine, and a harness environment, A context, typically containing a system prompt and task prompt, A filesystem tree, containing files that the model can access, And a transcript of all the inferred tokens i.e. the model’s thinking traces, tool calls, and final response . manifest.json { "version": 0, "model": "m-qwen2.5-coder-7b", "inference": "i-vllm-0.29.0-h100", "environment": "e-repligraph-c-v1", "initial context": "Review src/main.c", "seed": 0, "temperature": 0.7, } transcript <|im start| assistant <tool call {"name":"read file", "arguments":{"path":"src/main.c"}}<|im end| <|im start| user <tool response <result ok int main void { return 0; } </result </tool response <|im end| … Verifying a repligraph is straightforward, and straightforwardly costly: Recompute each inference step, re-execute each tool call, and check that the resulting transcript and artifact exactly match the original claim. There is, however, a key asymmetry here: Verification is not autoregressive. We have everything up front, so the full token stream can be recomputed in massive parallel. This has a few consequences. First, it means GPUs can run at close to 100% utilization when verifying. This doesn’t reduce the total FLOPs required, but it does make verification faster and cheaper than generation. Second, verification can be crowdsourced. Even a compute-starved participant can publish “I verified three turns of this transcript.” In aggregate, these contributions can achieve full coverage. Third, verifying a mismatch is cheap. If someone claims “Turn 37 is invalid,” anyone can confirm this themselves with a single inference pass. An audit is a repligraph that outputs a judgment or a ruling . We anticipate that audits will be one of the most common forms of repligraph, so it is worth outlining them more concretely here. For developers, the most familiar form of audit is a code review. Here, the prompt might look something like: “Audit the code in src/ for security vulnerabilities. Output one finding per line.” The model is then run inside a harness where it can inspect the code, write and run tests, etc. The resulting repligraph consists of the source code being audited, the model specification, its thinking traces and tool calls, and the final verdict. Audits can themselves be audited, whether by humans or models. Since the audit repligraph includes the model’s thinking trace and tool calls, claims such as “This key piece of evidence wasn’t considered” can have objective grounding. This is especially important in an adversarial context: For example, reviewing the transcript could reveal the presence of a prompt injection attack that caused the model to ignore a backdoor. Due to steering concerns, we expect audit prompts to be standardized, with a preference for concise and direct instructions. Linus’s law: “Given enough eyeballs, all bugs are shallow.” A single human is unlikely to catch all bugs. But because humans are diverse, in the aggregate we can approximate a single perfect auditor. In practice, however, bugs remain depressingly common, even in high-profile, high-stakes software such as OpenSSL. Three things are going on here. First, humans are not independent variables: Our blind spots, while not perfectly correlated, have substantial overlap. Second, the bystander effect: Everyone assumes that everyone else looked at the code. Third, rubber-stamping: “LGTM” can mean anything from “I scoured every line” to “Whatever, CI is green, ship it.” Repligraph audits turn eyeballs into an objective metric of review: “A frontier intelligence looked at this, and it didn’t find anything.” And they let us precisely measure diversity: Re-running the same repligraph produces no additional information, but varying the model, the prompt, the seed, etc. does. And since we can objectively measure the effectiveness of these variations by testing them against code with known flaws , we can compute an artifact’s “eyeball score” by aggregating and weighting its audits. Hover to pausePaused Many applications of repligraphs rely on a public log. Such logs serve to establish two things: Identity binding “A is signed by key K” and a shared chronology “A was added to the log before B” . The log can be implemented as a CT-style distributed ledger, a blockchain, or any other suitable substrate. Public commitments don’t make defection impossible, but they can make it costly: If a website commits to a particular moderation policy, they cannot later deviate from that policy without breaking their commitment and thereby damaging their reputation. If a frontier lab silently quantizes their flagship model, they can’t gaslight you about it; all repligraphs that predate the change will fail verification. If a researcher preregisters their analysis code in the form of a human-reviewable prompt , they cannot later p-hack their way to significance by tweaking their code. Perhaps surprisingly, repligraphs do not require open weights. There is a spectrum of trust with regard to deterministic execution of a mind-shaped function, and repligraphs can operate throughout that spectrum. At one end of the spectrum is an open-weights model, run within a widely-available inference environment. These repligraphs require minimal trust, at the cost of sub-frontier performance. On the other end of the spectrum is “The lab that trained the model is also the sole inference provider, and they only they let you see the final artifacts which are at least empirically deterministic .” While this arrangement is better than nothing at least you can cry foul if the claimed determinism is violated , it is not really worthy of being called a repligraph. We hope to nudge frontier labs towards the middle of this spectrum. They are unlikely to release open-weights models, or even reveal their models’ chain of thought, but they can still offer deterministic inference. The resulting repligraph would contain a partially encrypted transcript, and inference could only be verified by the set of providers entrusted with the model weights and decryption keys. Still, this compromise allows frontier models to participate in the repligraph ecosystem. Your younger brother comes to you, whining that he just got kicked from his favorite game server. You check the moderation log and see a repligraph attesting that he was banned for repeatedly making inflammatory comments in chat. “It’s not fair,” he protests. “They just hate me because I’m better at the game ” You download the repligraph: “If the following message is in violation of policy.txt, output BAN.” You click a button and verify it: Yup, the model does indeed judge your brother’s trolling as ban-worthy. Which, honestly, you agree with. “They’re not ganging up on you, dude. You agreed to this when you signed up.” A content moderation policy is a program. A lease agreement is a program. The Constitution is a program. We have been programming for a long time. The difference is that these “programs” are written informally and executed subjectively. Repligraphs let us keep the informal part while eliminating subjectivity. No more power-tripping mods. No more self-serving prediction market resolutions. No more packing the Supreme Court. Of course, solving adjudication is just one piece of the puzzle. Rulings still need to be enforced. Policies still need to be amended. And most perniciously, there is the question of what evidence is admissible. But is it crazy to expect that, in the coming years, people will form communes where all disputes are mediated by an AI? Formal verification is like type-checking on steroids. Proof-assistant languages like Rocq and Lean allow you to constrain program behavior at a very high level, making entire classes of errors impossible by construction and closing off wide swathes of attack surface. The rub is that you have to write a bunch of mathematical theorems. More precisely, you need to translate your intent about how a program should behave into a formal specification. This turns out to be a rather challenging task. As in, you need to spend 10x more time verifying your code than writing it. So no one bothers. The good news is that frontier models are increasingly proficient at this task. As a result, we expect that formal verification will soon be considered table stakes, at least for security-critical libraries. The bad news is that the “last-mile problem” remains: You can prove that the code matches the spec, but how do you prove that the spec e.g. the ML-KEM standard, expressed in Lean satisfies the original intent e.g. “Implement a post-quantum key exchange algorithm” ? Our answer is simple: Use a repligraph to prove that the spec was written by a capable model, prompted with your intent. This shrinks the attack surface considerably. Humans no longer review code; they review prompts, augmented by audit results that flag potential mismatches between the spec and the presumed intent. Predicting the future is a lucrative skill. Even a slight edge can translate to enormous profits. Accordingly, companies will pay big bucks for a forecaster they can trust. The problem is that trusting a human forecaster is itself a bet: A bet that past performance is indicative of future results. When people trusted Nate Silver’s 2016 election prediction, it was because of how prescient he looked in 2008, having accurately predicted 49 out of 50 states. But 2008 Nate Silver and 2016 Nate Silver are not the same forecaster. You see where this is going: Deterministic agents are the same forecaster, forever. Which means you can objectively evaluate and rank them. And unlike a human superforecaster, they are not a scarce resource: Their hourly rate is just the cost of inference. Trust is scarce in a zero-sum game like a prediction market, but none is needed here: Repligraphs allow anyone to independently confirm the accuracy of a given model. Each prediction can be committed to a public log for independent verification. Later, when you’re deciding which model to seek advice from, you can just sort by Brier score. LLM forecasters have another feature that humans don’t: seeds. It’s one thing to ask a human forecaster for their confidence interval; it’s another thing to resample a model 1000 times to derive its true probability distribution. Hover to pausePaused An eval is a press release. The lab posts some charts with their model’s score tastefully highlighted, but that’s for the birds; they probably used some internal model with a custom harness and an infinite compute budget. You really think that’s what you’re getting access to when you pay 20 bucks a month? Nah. The real test of a model is what your friends say about it on Twitter, and also how well it can draw a pelican riding a bicycle. This is a cynical view, but not an entirely unfounded one. We shouldn’t have to take labs at their word. When Anthropic claims their model scored 84.2% on VibeBench, independent verifiers should be able to reproduce the exact same score. When tinfoilers claim that GPT’s recent breakthroughs were actually the laundered results of human mathematicians, OpenAI should be able to publish the full transcript as a repligraph. Who is Claude, really? What are Claude’s values and preferences? How well can Claude introspect? We can investigate these questions today, but it is a very inexact science. As with evals, when someone publishes the results of their experiments, we are expected to shrug and take them at their word. You could try the same prompts they used, and see if you get similar results. But if you don’t — what then? Maybe you just got unlucky. Deterministic inference gives us a straightforward procedure for all such experiments: Use the same prompt 100 times, varying the seed value, and report the distribution of answers. With a deterministic harness, we can even run agentic experiments, such as those requiring access to a dataset larger than the context window e.g. the model’s own weights . In the 1980s, Douglas Hofstadter introduced the concept of superrationality: If you assume that all perfectly rational agents analyzing the same problem will arrive at the same answer, and you assume that both you and your counterparties are perfectly rational agents, then you can defeat many standard decision theory traps, such as the Prisoner’s Dilemma. “Either we will both cooperate, or both defect; cooperation yields a better outcome, so I will cooperate and so will they .” While elegant, this approach is actually a special case of a far stronger coordination assumption: What if you could read your counterparty’s source code? What if you could predict, with perfect accuracy, how they would behave in a given situation? Like many of the applications presented here, this leans heavily on precommitment to make defection reputationally costly. Agents commit to a claim of the form, “My decision procedure for a given input consists of running model M with prompt P.” Subsequent actions taken by the agent can then be checked against this claim. This doesn’t grant protection from adversaries who misrepresent themselves. It grants honest but mutually-distrusting parties the ability to coordinate. A repligraph proves that a model generated some artifact. It does not prove that the model did a good job, or that the artifact is safe. Even today’s best models have quirks. Minor variations can result in completely different responses. Prompt injection attacks can be hidden in source code being audited, or a web page being fetched, or a comment being reviewed. More worryingly, the model itself could be trained to introduce a backdoor under specific circumstances. There are two main defenses against these steering attacks: Peer review: Have another model audit the repligraph, searching for vulnerabilities in the artifact or suspicious behavior in the transcript. Standardization: Lock down as many parameters as possible. For example, an audit should carry more weight if it uses a standard prompt and seed. Interestingly, determinism cuts both ways here. On one hand, the attacker just needs to find one prompt that produces the outcome they want. On the other hand, it can be objectively shown that this one prompt was truly a needle in the haystack, i.e. that it was statistically unlikely to have been an accident. Today, you can hire some patsy to plant a backdoor for you. In the world to come, you have to train your model to write it, and if the backdoor is ever discovered, you have zero plausible deniability and your whole company goes under.