Claude’s Protein Design Hit Rate Was 26.8%. One Target Returned 0 for 90.

Anthropic reported that its AI agent Claude achieved a 26.8% hit rate in autonomous protein design, with 354 binders out of 1,320 designs across 16 targets, validated independently by Adaptyv Bio. However, one target, maltose-binding protein, produced 90 designs with zero binders, and a statistical analysis by the article's author suggests the portfolio average masks high variance, with a 38% chance a new target falls below the 10% industry floor.

Ninety designs went into the wet lab against maltose-binding protein. Ninety came back with nothing. That is the part of the story that stopped us. On August 18, 2026, Anthropic published the results of an autonomous protein design campaign, and the number everyone repeated was the Claude protein design hit rate: 354 binders out of 1,320 designs, 26.8%, against an industry norm of 10–15%. It is a real result, independently validated in two contract labs. But sitting inside that 26.8% is a target where the agent produced ninety designs, every one of which was synthesized, expressed, and measured — and not one of them bound. Both facts came out of the same campaign, the same prompt, the same model. So we spent August 25 doing the arithmetic nobody in the coverage did: 26.8% is a portfolio average across sixteen targets, not the probability that your target works. Run the published per-target counts through a binomial test and a single shared hit rate is rejected at p ≈ 1.8 × 10⁻⁶³. Fit a model that allows targets to differ and the picture inverts: for a new target, the chance of landing below the 10% industry floor is about 38%. Key takeaways TL;DR: An autonomous agent orchestrated a dozen open-source protein design tools and beat expert human hit rates on most targets. That happened, and it matters. But the headline number is a portfolio statistic, and almost nobody deploys a portfolio — they deploy against one target, one ticket, one customer. When we modeled the published spread instead of the mean, the expected experience of a single new target looked dramatically worse and dramatically noisier than 26.8% implies. And the pipeline’s own confidence scores could not tell in advance which regime it was in. That combination — high average, enormous variance, uncalibrated self-assessment — is the shape of nearly every agentic AI benchmark result you will read this year. Anthropic gave Claude Mythos Preview and Opus 4.8 a set of 16 protein targets and asked for 30 minibinders per target per design arm — small proteins engineered to latch onto a target, the mechanism behind a large share of modern biologic drugs. With three arms running, most targets ended up with 90 designs in the lab. Fifteen targets produced usable measurements; one, mature GDF-8, was dropped because the target aggregated in the assay. The agent did not invent a protein model. It installed and ran existing open-source tools: backbones came from PXDesign 358 designs , RFdiffusion3 267 , Genie 3 185 , FreeBindCraft 135 , BoltzGen 134 , RFdiffusion 118 and Proteina-Complexa 100 , among others, with sequences mostly from SolubleMPNN and filtering through ESMFold2 and Protenix v2. What was new was the layer above: choosing the epitope, installing the software, combining it across 24 distinct workflows, and ranking the output with no human touching a design decision. Validation was independent. Adaptyv Bio’s wet-lab case study https://www.adaptyvbio.com/blog/anthropic-1 , published 2026–08–19, reports that the designs arrived anonymized — the lab did not know which model produced which sequence — and were run on surface plasmon resonance at five target concentrations in duplicate. 95% of designs expressed. 354 of 1,320 bound. That is a genuinely strong result and we are not going to shave it down. Against RBX1, an open design competition had produced 9 binders from 245 entries 3.7% ; Claude produced 28 from 90. Anthropic had the competition’s winning design rebuilt and measured on the same assay plate: it bound at 45 nM, while Claude’s best bound at 3.9 nM, roughly ten times tighter. Because the targets are not interchangeable, and the published per-target counts make that impossible to ignore. Here are the eight per-target results disclosed across Anthropic’s post, Adaptyv Bio’s case study and The Decoder’s technical breakdown https://the-decoder.com/anthropic-says-any-lab-can-now-let-a-language-model-agent-run-the-whole-protein-design-stack/ of the report: Binders per target, from the published counts: Now ask the question the coverage skipped: if every design really had a 26.8% chance of binding, how surprised should we be by the bottom row? The answer is that we should be about as surprised as it is possible to be. The probability of drawing zero successes in 90 independent trials at p = 0.268 is 0.732 raised to the 90th power, or 6.3 × 10⁻¹³ — one in roughly 1.6 trillion. TREM2 is equally impossible from the other direction: getting 72 or more hits in 90 trials at that rate has probability 1.1 × 10⁻²⁵. Neither of those is a fluke you explain away. They are the model being wrong. We ran a formal likelihood-ratio test comparing one shared rate against a model that lets each target have its own rate, and the shared-rate model loses by a likelihood ratio of 282.9 on one degree of freedom, p ≈ 1.8 × 10⁻⁶³. The pooled 26.8% is not a property of the method. It is a property of the target list . This is the part we computed ourselves, so here is the method before the number. On 2026–08–25 we took the eight published per-target counts above and fit a beta-binomial model by maximum likelihood — the standard way to handle a binomial process whose success probability varies between groups. Each target draws a hit rate from a Beta α, β distribution; designs within a target are then binomial at that rate. Two parameters, eight data points, Nelder-Mead from nine starting points, Python 3.11 with SciPy 1.15.3, log-likelihood including the binomial coefficient so it is directly comparable to a plain binomial fit. Every input count is published and cited in the table above. Nothing here is a measurement we made — these are derived statistics computed from Anthropic’s and Adaptyv Bio’s published numbers, and anyone with those eight counts can reproduce them in about twenty lines. The fit came out at α = 0.459, β = 1.169. That implies a mean of 28.2%, close to the pooled figure, which is the reassuring part. Everything else is not reassuring: What a new, unseen target looks like — one shared rate vs. the fitted model: Read the last two rows again. Under the headline number, thirty designs is overkill — you would need ten to be 95% confident of at least one binder. Under the fitted model you would need 638, a 64× difference, and even then a fifth of the probability mass sits on targets where more designs barely help. That last point has a shape worth naming. Because the fitted α is below 1, the probability of a total washout decays like N^ −α rather than exponentially. We checked the asymptote: at one million designs, the model still assigns a 0.17% chance of zero binders. There is no batch size that buys you certainty, because some targets are not low-probability — they are, as far as the current toolchain is concerned, closed. Maltose-binding protein is that. It is a large, flexible bacterial protein with a smooth, water-loving surface. There is nothing for a binder to grab. This is the finding that should worry anyone shipping an agent, and it comes straight from Anthropic’s own report: the folding-model confidence scores warned of neither failure. Designs against MBP and BBF-14 received scores about equal to designs against targets that worked. So the loop looked like this. The agent generated candidates, scored them with co-folding models, ranked them, and shipped its top 30 with normal-looking confidence. Ninety of those came back dead, and nothing upstream of the wet lab knew. The scorer was discriminative within a target — Anthropic reports that designs the agent ranked first bound 49% of the time versus 26.8% overall — and blind across targets. We have written before about how the evaluator, not the model, is the real bottleneck in agentic discovery https://medium.com/p/86b8e93f1ce2 and how agent failures that return success are the ones no dashboard catches https://medium.com/p/b690d82b00bd . This is both at once, in a domain where the ground truth costs weeks and real money to obtain. Worth pinning down, because “35.1%” circulated widely as the real number. In multi-target mode, all targets handled inside one 48-hour session, Mythos Preview hit 26.7% and Opus 4.8 hit 22.6%. Running one target per session over 24 hours, Mythos Preview reached 35.1%. That looks like a clean argument for narrow scope — and it partly is, in the same family as the context-dilution problem we measured in long-context models https://medium.com/p/10ee7bae1c1b . But the single-target runs also got 2.8× the compute budget per target: 2,500 NVIDIA H100-hours each and a $10,000 budget per target, versus 12,500 H100-hours and $50,000 for an entire multi-target campaign. Anthropic says so plainly in the report — focus and budget cannot be separated here. Anyone quoting 35.1% as evidence that scoping an agent narrowly is free is quoting half a finding. Yes, and this is why it belongs in your reading even if you will never touch a pipette. Every agentic benchmark you read this year is reported the same way: a pooled pass rate over a task set. SWE-bench, ARC-AGI, τ-bench, internal eval suites — one number averaged across items that differ enormously in difficulty. Then a team reads “62%” and plans as if their ticket has a 62% chance. It does not. It has whatever probability its own item-level difficulty implies, and the pooled score tells you the mean of a distribution whose spread was never published. The protein campaign is unusually instructive precisely because the per-target counts were published, so we can see how wide that distribution actually is: 80% to 0%, on the same run. The practical version, which costs nothing to adopt: Nothing above says the campaign was oversold. Take the opposing case seriously: A pooled hit rate is exactly the right metric if you are asking “is autonomous agentic design competitive with expert humans across a representative benchmark suite?” — which is the question Anthropic asked. Human campaigns have per-target variance too; the RBX1 competition’s 3.7% and BenchBB’s spread are evidence of that. Comparing a mean to a mean is fair. Anthropic also published the per-target counts, the prompts, the design sequences and both measurement datasets on Hugging Face https://huggingface.co/datasets/Anthropic/claude-protein-binder-design , which is precisely what made this analysis possible; a lab hiding the variance would not have handed us the table. And the report names its own limits without prompting: no parallel human control arm, no structural resolution of any design, each model-format-target combination run exactly once, and four of six competition targets present in the agent’s reading list. The honest summary is that 26.8% is a true and impressive statement about a benchmark suite, and a misleading statement about your next project. Both. Our first run of this analysis reported a likelihood ratio of −254, which is not a thing that can happen: the binomial is a limiting case of the beta-binomial, so the richer model cannot fit worse. We had dropped the binomial coefficient from the beta-binomial log-likelihood, which is harmless for the parameter fit and fatal for any comparison against a plain binomial. Corrected, the ratio is +282.9. The broken version would have supported the exact opposite conclusion, and it looked perfectly plausible in the output. Two further limits we will not bury. First, eight targets is a small sample for a two-parameter fit, and the eight that were disclosed are the ones coverage found interesting — disproportionately the extremes. We can quantify that bias: those eight targets ran 219 hits from 720 designs 30.4% , while the undisclosed remainder ran 135 from 600 22.5% . The published slice is skewed high and skewed wide. Refitting with TREM2 and MBP removed gives α = 0.806, β = 2.253, and drops the probability of a sub-10% new target from 37.8% to 29.5% and the zero-in-30 probability from 20.1% to 11.4%. The direction of the conclusion survives; the magnitude roughly halves. Take the conservative pair as the defensible ones. Second, “hit rate” here means measurable binding, nothing more. Not structure, not selectivity, not function, not developability. Adaptyv Bio’s own write-up puts binder design at step two of a six-step ladder to a therapeutic. A binder is a long way from a drug. Across Anthropic’s campaign published 2026–08–18, Claude models produced 354 binders from 1,320 designs — a pooled hit rate of 26.8%, against a typical industry range of 10–15%. In multi-target mode the rates were 26.7% Mythos Preview and 22.6% Opus 4.8 ; a single-target mode with 2.8× the compute per target reached 35.1%. Independent wet-lab validation was performed by Adaptyv Bio and Twist Bioscience. No. It is an average across sixteen targets whose individual rates ranged from 80% TREM2, 72/90 to 0% maltose-binding protein, 0/90 . A single-rate binomial model is rejected against the published per-target counts at p ≈ 1.8 × 10⁻⁶³. Our beta-binomial fit, run 2026–08–25, gives a 90% predictive interval of 0.1% to 84.7% for a new target and a median of 18.6%. If you assume a flat 26.8% success rate, ten designs give you 95% odds of at least one binder. Modeling the published between-target variance instead, the figure is 638 designs — a 64× difference — and a 30-design batch has roughly a 20% chance of returning nothing at all. On the more conservative refit that excludes the two extreme targets, the zero-binder probability is 11.4%. No. Claude orchestrated existing open-source tools — PXDesign, RFdiffusion3, Genie 3, FreeBindCraft, BoltzGen and Proteina-Complexa for backbones, SolubleMPNN for sequences, ESMFold2 and Protenix v2 for filtering — across 24 distinct workflows. AlphaFold-3 weights, Rosetta and ESM3 were excluded for licensing reasons. The novel contribution was autonomous orchestration, epitope selection and ranking, not a new design model. Here is the question we actually want answered, and we suspect several of you have the data: in your own eval suite, what is the gap between your pooled pass rate and the 10th-percentile per-item pass rate? If you have ever computed both numbers, we would like to know how far apart they came out. Written by Decoding AI. We take the week’s biggest AI claim, re-derive the number ourselves, and publish the method so you can check our work. Previously: why the evaluator is the real bottleneck in agentic discovery , what a 4% win rate does to your frontier model bill , and the agent failures no dashboard catches . Claude’s Protein Design Hit Rate Was 26.8%. One Target Returned 0 for 90. https://pub.towardsai.net/claudes-protein-design-hit-rate-was-26-8-one-target-returned-0-for-90-e0d75451431d was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.