Agent reliability calculator: how many steps will your agent finish?

An agent that succeeds on 95% of its steps finishes a 48-step job in a single uninterrupted run only 8.5% of the time and needs about 11.7 runs on average when every failure restarts the job, according to a reliability calculator published by Construct. Construct, which sells plans from $9 a month for up to 50 steps per task on Lite, $59 for 150 steps on Starter and $299 for up to 1,000 steps on Pro, says saving finished steps to workspace files lets a run that stops at step 30 keep steps 1 through 29 so the next run only completes what is missing. Cutting the same 48-step job into eight independent six-step pieces raises the share finished on the first run to about 73.5%, cuts the average to about 2.5 runs, and makes completion within three runs 86.1% instead of 23.5%.

Agent reliability calculator: how many steps will your agent finish? AI agent reliability falls exponentially with task length: an agent that gets 95% of its steps right finishes a 48-step job in one run only 8.5% of the time, and needs about 11.7 runs on average if every failure starts the job over. Construct, from $9 a month, saves every finished step to workspace files as it is produced, so a run that stops at step 30 keeps steps 1 to 29 and the next run only does what is missing. Cut the same job into eight independent six-step pieces that save their work, and the first run finishes about 73.5% of it, the whole job takes about 2.5 runs, and it is done within three runs 86.1% of the time instead of 23.5%. Same agent, same per-step reliability, same total work. Construct is built for that second shape: finished work stays in Files, Lite gives each task up to 50 steps, and the Calendar turns retries into a schedule. The calculator below lets you put in your own numbers and see where the difference comes from. Why run long jobs in Construct? - Finished steps are kept. Reports, spreadsheets, and scripts land in workspace files as the agent produces them. A failure at step 30 costs the step that failed, not the 29 before it. - Reruns resume instead of repeating. One sentence such as "for each client without a report in reports/2026-10/ , write one" makes the next run fill the gaps, which is the "skip and carry on" row of the table below. - Step limits sized to the job. Up to 50 steps per task on Lite at $9, 150 on Starter at $59, and up to 1,000 on Pro at $299. - Retries on a schedule. Calendar schedules run on Construct's servers with your laptop closed, so three nightly runs before a Monday deadline happen without anyone starting them. Lite includes 3 scheduled tasks. - A record of what broke. Activity lists each action an agent took with a short reason, so you can find the step that failed and fix its cause. The AI agent reliability calculator Enter your agent's per-step success rate, the number of steps in the job, how many resumable pieces it is cut into, and how many runs you can afford. It starts on the 48-step example from your agent has a half-life https://construct.computer/blog/agent-task-half-life/ . Agent reliability calculator Enter how often each step succeeds and how long the job is. The math assumes every step succeeds or fails independently, which real agents do not quite do, so read the results as a planning floor. Start from an example How often one step goes right, from 50 to 100. Count every tool call or action the job needs. 1 means no checkpoints: any failure starts the job over. Finished pieces are saved and never redone. For example, three nightly runs before a deadline. At 95% per step, a 48-step job finishes in one uninterrupted run 8.5% of the time and needs about 11.7 runs on average if every failure starts over. It is done within 3 runs 23.5% of the time. Add resumable pieces to see what checkpoints change. - Finishes in one run - 8.5% - Runs needed - 11.7 - Half-life of this agent - 13.5 steps | Agent reliability with and without resumable checkpoints | | | |---|---|---| | Measure | Start over on failure | Resumable add pieces | |---|---|---| | Finishes in one run | 8.5% | 8.5% | | Work done after the first run, on average | 8.5% | 8.5% | | Average runs to finish | 11.7 | 11.7 | | Average steps executed, redone work included | 215 | 215 | | Done within 3 runs | 23.5% | 23.5% | Why do AI agents fail on long tasks? Mostly because small per-step error rates multiply. If each step succeeds independently with probability p, a job of n steps succeeds in one go with probability p to the power n, and that number falls faster than intuition expects. At 95% per step, ten steps finish about 60% of the time and 48 steps finish 8.5% of the time. Nothing about the long job is harder. There is just more exposure. Toby Ord made the same point with real agent benchmarks. Looking at how agent success falls as tasks get longer, he found it is explained by "a constant rate of failing during each minute a human would take to do the task," which "implies an exponentially declining success rate with the length of the task" Ord, arXiv:2505.05115 https://arxiv.org/abs/2505.05115 . We worked through what that means for business jobs in your agent has a half-life https://construct.computer/blog/agent-task-half-life/ . This page turns that argument into a tool. How reliable are AI agents today? More reliable every year, but still measured at coin-flip odds on long work. METR reported in March 2025 that the length of tasks frontier agents can complete with 50% reliability "has been doubling approximately every 7 months for the last 6 years" METR https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ , and its later measurements are on its time horizons page https://metr.org/time-horizons/ , which METR says is no longer actively updated. That is real progress, but the headline measure is 50% reliability, a coin flip, and the exponent in the formula below does not care which model you use. A better model nudges p. Restructuring the job changes n for each run, which is usually the bigger lever. What the calculator inputs mean The calculator takes four numbers and one choice: how often a step succeeds, how many steps the job has, how many saved pieces it is cut into, how many runs you can afford, and what a failed piece does to the rest of the run. Per-step success rate. How often one step goes right. A step here is one action the agent takes: a tool call, a page load, a file write, a decision. If you have never measured it, start at 95% and treat the answer as optimistic for judgment-heavy work and pessimistic for simple, mechanical work. Steps in the job. Count actions, not instructions. "Send the monthly report to each of eight clients" is one instruction and dozens of steps. Resumable pieces. How many parts the job is cut into, where a finished part is saved and never redone. One piece means no checkpoints: any failure sends the whole job back to the start. The calculator splits steps evenly, so 48 steps in eight pieces is six steps each. When a piece fails. Two shapes of job behave differently: - Skip it and carry on. The pieces are independent, like one report per client. A failure on client three does not stop client four, and the next run retries only what is missing. - Stop the run. Each piece needs the one before it, like extract, then clean, then load. A failure ends the run, and the next run resumes at the failed piece. Runs you can afford. Scheduled jobs come with a natural budget: three nightly runs before a Monday deadline, say. The calculator reports the chance the job is done within that many runs. The math behind the calculator Everything comes from one assumption: each step succeeds independently with the same probability p. For a job of n steps cut into k pieces of s steps each: % One uninterrupted run finishes the whole job P \text{one run} = p^{n} % Runs needed when every failure starts over geometric distribution E \text{runs} = \frac{1}{p^{n}} % Steps executed in total, counting the step that failed each time E \text{steps} = \frac{1 - p^{n}}{ 1 - p \, p^{n}} % The agent's half-life in steps: where one-run success hits 50% n {1/2} = \frac{\ln 2}{-\ln p} % Resumable pieces: each piece of s steps is its own geometric trial q = p^{s}, \qquad E \text{steps} = k \cdot \frac{1 - q}{ 1 - p \, q} % "Stop the run": one run, plus one more for every failure E \text{runs} = 1 + k \left \frac{1}{q} - 1 \right % "Skip and carry on": done within R runs if every piece succeeds once in R tries P \text{done within } R = \left 1 - 1 - q ^{R} \right ^{k} Two results are worth reading slowly. First, checkpoints do not change the chance that one run finishes everything. That is still p to the power n, 8.5% in the example. What they change is what a failure costs. Without checkpoints a failure at step 40 throws away 39 good steps. With six-step pieces it throws away at most five. Second, the half-life. At 95% per step, one-run success drops to 50% at about 13.5 steps. At 99% it is about 69 steps, and at 90% it is about 6.6. If a single uninterrupted piece of your job is much longer than your agent's half-life, most runs of that piece will fail, and the fix is a shorter piece rather than a longer prompt. The "stop the run" mode has no tidy closed form for the chance of being done within R runs, so the calculator adds up the distribution of failures across pieces directly. The "skip" mode's average run count is the expected maximum of k independent geometric trials, which the calculator sums numerically. What checkpoints change: the 48-step example Checkpoints cut the work an agent redoes on a 48-step job from about 215 steps to about 58, and if the pieces are independent, the job is done within three runs 86.1% of the time instead of 23.5%. Here is the illustrative job from the half-life post: monthly reporting for eight clients, six steps per client, an agent that gets 95% of steps right, and three runs to get it done. | A 48-step job at 95% per step, with and without resumable pieces | | | | | | |---|---|---|---|---|---| | Setup | Finishes in one run | Work done after first run | Average runs | Steps executed, redone work included | Done within 3 runs | |---|---|---|---|---|---| | One piece, start over | 8.5% | 8.5% | 11.7 | 215 | 23.5% | | 8 pieces, stop on failure | 8.5% | 31.7% | 3.9 | 57.7 | 48.1% | | 8 pieces, skip and carry on | 8.5% | 73.5% | 2.5 | 57.7 | 86.1% | Three things stand out. The one-run column never moves. The steps executed fall from 215 to under 58 the moment finished work is saved, whichever failure mode you pick, because a retry only redoes the piece that failed. And the shape of the job matters as much as the checkpoints: if a failure on one client does not have to stop the others, the job converges in about two and a half runs instead of four. Try 99% in the calculator and the picture softens. A 99% agent finishes 61.7% of 48-step runs and averages 1.6 runs even without checkpoints. Try 90% and it collapses: 0.6% in one run and about 157 runs on average to start over until it works, against about 4.1 runs with eight independent pieces. Run a resumable job on Construct https://app.construct.computer Why do real AI agents fail differently from the math? Real agents fail in correlated, uneven, and sometimes silent ways, so the calculator is a planning floor rather than a prediction. The independence assumption is what makes the math clean, and it is also the part real agents break most. Read the calculator as a planning floor, and adjust for these. Steps are not independent. If a step fails because the login expired or a page layout changed, the retry fails the same way. A deterministic failure is not a 5% chance repeated; it is a 100% chance until someone fixes the cause. In that case expected runs are infinite, and no number of checkpoints helps. Retries pay off for random failures, such as a timeout or a misread page, not systematic ones. Steps are not equally hard. A step that copies a value is far more reliable than a step that decides which clients count as at risk. One weak step dominates the product. If you can find it, put it in its own piece so a bad call costs that piece and not the run. Agents recover from some errors. A tool call that returns an error is often retried or worked around inside the same run. The calculator counts a failure as fatal to the piece, so for agents that self-correct well it overstates the damage. The per-step rate you plug in should be the rate of failures the agent does not recover from. Some failures are silent. The calculator assumes you know a step failed. An agent that confidently writes the wrong number into a report has not failed by this math, and it is the more expensive case. We wrote about why that check is the real bottleneck in nobody merges an email https://construct.computer/blog/agent-verification-gap/ . Checkpoints are not free. Each piece has to read its state back in: the client list, the template, what is already done. That adds steps, and each added step can fail. If your pieces become so small that the reading back is most of the work, you have overdone it. Retries can repeat side effects. Skipping and retrying is safe only if a retried piece can tell it already ran. A report file that exists is easy to check. An email that was sent, a payment, or a public post needs the job to check before acting, or a retry does the thing twice. For email, the safest pattern is drafts the agent saves and a person sends, as in how to let an AI agent use your Gmail safely https://construct.computer/blog/ai-agent-gmail-access-safely/ . How do you measure your AI agent's success rate? Run the job you care about five to ten times while you watch, then divide the unrecovered failures by the total steps. You do not need a benchmark, only a few honest runs. 1. Run the job on demand five to ten times while you watch, before scheduling it. 2. For each run, count the steps the agent took and the steps that went wrong and were not recovered. 3. Divide. If ten runs took 400 steps between them and 16 failed unrecoverably, your per-step rate is about 96%. 4. Note which steps failed. If the same step fails every time, you have a systematic failure to fix, not a rate to plug in. The rate you get is specific to that job, that set of tools, and that week's websites. It will drift. Re-measure when a job that used to finish starts needing extra runs. How Construct turns this math into finished jobs Construct keeps every finished piece by default, so you get the cheap-retry rows of the table without building checkpoints yourself. The table above has one lesson: save finished work and retry only what is missing. Construct does the saving for you. Every agent works in a persistent cloud workspace, so finished work lands in files as it is produced, and a run that dies at step 30 keeps steps 1 to 29. Your instruction decides how the next run uses them. Write "for each client in clients.md without a report in reports/2026-10/ , write one" and a rerun becomes a retry of the missing clients, not a duplicate of the finished ones. That is the "skip and carry on" row of the table, built from a file listing and one sentence. Pick a plan by the length of your pieces Each plan caps how far a single task can go: up to 50 steps per task on Lite, up to 150 on Starter, and up to 1,000 on Pro pricing https://construct.computer/pricing/ . That cap is a ceiling on one task's length, not a success rate, and the calculator shows why you should not aim for it with one uninterrupted piece. At 95% per step, a task that really uses all 50 of Lite's steps finishes in one go about 7.7% of the time. Cut the same work into shorter tasks that each leave a file behind and a failure costs one piece, not the run. Move to Starter or Pro when a single piece genuinely needs more steps, or when you need more agents and more scheduled tasks. Let the Calendar be your retry budget Recurring jobs can be saved as workflows https://construct.computer/use-cases/workflows/ and scheduled on the Calendar https://construct.computer/use-cases/scheduling/ , and the schedule becomes your retry cadence. Calendar schedules run on Construct's servers with your laptop closed. Set the monthly reports to run nightly for the last three nights of the month, and the calculator's "done within 3 runs" number becomes your real odds of a finished folder on the first. The workflow guide https://construct.computer/blog/ai-workflow-automation/ shows how steps and schedules fit together, and AI agent memory https://construct.computer/blog/ai-agent-memory/ explains what context carries from one run to the next and how to correct it. Before you start - Order-dependent pieces run in sequence. Construct's workflows are linear, one step after another, with no branching, delays, approvals, or fan-out yet, so pieces are worked through inside a run rather than fanned out in parallel. For extract, clean, then load jobs, that is the "stop the run" row: have each stage write its output to a file so the next run starts at the stage that failed. - Make side effects safe to retry. For anything that sends, posts, or pays, ask the agent to save a draft and check for it before acting, then send it yourself once you have read it. A retried piece then cannot do the thing twice. - Use Activity to find the broken step. Activity records each action with a short reason for the last 30 days, and chat keeps the tool records of each run. Together they show where a run stopped. They are operating records, not a complete audit log, so keep a run log file in Files for jobs you need to account for later. Read next Your agent has a half-life https://construct.computer/blog/agent-task-half-life/ How do you design an AI agent job that finishes? Measure the per-step rate, keep each uninterrupted piece shorter than the agent's half-life, and make every piece leave a file a later run can check for. In order: 1. Measure p on the real job , as above. Do not guess from a benchmark. 2. Find the half-life. If one uninterrupted stretch of the job is longer than the half-life the calculator shows, cut it. 3. Cut at the loop first. "For each client" or "for each invoice" pieces are usually independent, which puts you in the "skip" mode where retries are cheapest. 4. Make every piece leave an artifact a later run can check for, so a retry knows what is done. 5. Set the run budget from the deadline. If the calculator says 86% within three runs, decide whether a 14% chance of chasing it on Monday morning is acceptable, and if not, cut further or raise p. What should you do next? Choose Construct for long agent jobs. The math does not change with the product; what changes is whether a failure costs one piece or the whole job. Construct keeps every finished piece in workspace files by default, gives each task up to 50 steps from $9 a month, and runs your retries from the Calendar with nobody online. Give Construct your monthly client reports as its first job: put the client list in clients.md , ask for one report per client in reports/2026-10/ , skip any client that already has one, and run it on demand three times while you count the failed steps. Then put it on the Calendar for the last three nights of the month. Start on Lite at $9 https://construct.computer/pricing/ , or try Pro free for seven days. Sources - Toby Ord, "Is there a half-life for the success rates of AI agents?" https://arxiv.org/abs/2505.05115 , arXiv:2505.05115, May 8, 2025. - METR, "Measuring AI Ability to Complete Long Software Tasks" https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ , March 19, 2025, and METR's time horizons page https://metr.org/time-horizons/ , last updated September 8, 2026. - Construct plan limits from the pricing page https://construct.computer/pricing/ , checked on October 11, 2026. Related resources Frequently asked questions - How reliable are AI agents on long tasks? - Less reliable than intuition suggests, because per-step errors multiply. An agent that gets 95% of its steps right finishes a 48-step job in one uninterrupted run only 8.5% of the time, since 0.95 to the power 48 is about 0.085. If every failure starts the job over, it needs about 11.7 runs on average. METR reported in 2025 that the task length agents complete at 50% reliability was doubling about every 7 months. - Why do AI agents fail on long tasks? - Mostly because small per-step error rates compound. If each step succeeds with probability p, a job of n steps succeeds in one go with probability p to the power n. At 95% per step, ten steps finish about 60% of the time and 48 steps 8.5%. Real agents also fail in correlated ways, such as an expired login, and silently, such as a wrong number written confidently into a report. - Do checkpoints make an AI agent more reliable? - They do not change the chance that one run finishes everything, which stays at p to the power n. They change what a failure costs. Cutting a 48-step job at 95% per step into eight saved six-step pieces cuts the steps executed from about 215 to about 58, and if the pieces are independent the job takes about 2.5 runs and is done within three runs 86.1% of the time instead of 23.5%. - What is an AI agent's half-life in steps? - It is the number of steps at which the chance of finishing in one run falls to 50%, calculated as ln 2 divided by minus ln p, where p is the per-step success rate. That is about 13.5 steps at 95% per step, about 69 steps at 99%, and about 6.6 steps at 90%. If one uninterrupted piece of a job is much longer than the half-life, cut it shorter. - How do I measure my AI agent's success rate? - Run the job on demand five to ten times while you watch, count the total steps and the steps that failed without the agent recovering, and divide. Ten runs with 400 steps and 16 unrecovered failures gives about 96% per step. If the same step fails every time, that is a systematic failure to fix, not a rate to plug into the calculator. Re-measure when a job starts needing extra runs. - How does Construct handle a long agent task that fails partway? - Construct saves finished work to workspace files as it is produced, so a run that stops at step 30 keeps steps 1 to 29. An instruction scoped to the missing work, such as writing a report only for clients without one, makes the next run a retry of the gaps rather than a duplicate, and a Calendar schedule on Construct's servers turns those retries into a cadence with your laptop closed. - How many steps can a Construct task take? - Up to 50 steps per task on Lite at $9 a month, 150 on Starter at $59, and up to 1,000 on Pro at $299. The cap is a ceiling on length, not a success rate: at 95% per step, a task that uses all 50 steps finishes in one go about 7.7% of the time, so shorter tasks that each leave a file behind are the more reliable design. Keep reading All our Agents get computers, we pay for almost noneConstruct is an AI employee with a real Linux computer per user. Don't get excited - we are not paying for all of them. How we run that on Cloudflare Durable Objects, Sandboxes, and R2. - engineering https://construct.computer/blog/tag/engineering/ - cloudflare - durable-objects - infrastructure Your agent has a half-lifeWhy AI agents keep failing on long multi-step jobs: a 95% reliable agent finishes 48 steps 8.5% of the time. The fix is a resumable run, not a better model. Clef beat Jev on Jev's home turfWe ran Cloudflare's Clef and Clef-flash against Jev on five production AI agent decisions: 1,053 calls, accuracy, determinism, drift, cost, and what we are switching. - ai-agent https://construct.computer/blog/tag/ai-agent/ - clef - decision-models https://construct.computer/blog/tag/decision-models/ - reliability https://construct.computer/blog/tag/reliability/ - benchmark We put Jev inside our AI employeeWe shipped TypeSafe's Jev into our AI employee's memory five days after launch: measured latency, how we set a 0.7 threshold, where Jev fails, and what it costs.