For most of my career I had backups I had never restored. The backup job ran every night. The green checkmark appeared every morning. The storage bucket filled up with files that had the right names and roughly the right sizes. I looked at all of that and told myself the data was safe. What I never did was take one of those files, restore it into a clean database, and confirm that the result was a working copy of production. I trusted the checkmark, and the checkmark was measuring the wrong thing. The checkmark was measuring whether the backup job ran, not whether the backup could be restored. Those are completely different questions, and the gap between them is where disaster lives. A backup job can run perfectly every night for a year and produce a year of files that cannot be restored, because of a subtle format change, a missing table, a permissions issue in the restore path, or an encryption key that no longer exists. The only way to know a backup works is to restore it, and almost nobody restores their backups until the day they have no choice.
Claude Code let me close that gap by making restore verification a routine, automated part of the backup workflow rather than a heroic effort that happens once a year during a disaster recovery drill, if it happens at all. The backups are still taken by the usual tools. What changed is that every backup is now proven restorable shortly after it is taken, and the proof is captured as evidence rather than assumed. Here is the workflow.
The green checkmark on a backup job is one of the most dangerous signals in operations, because it produces confidence without producing safety. It tells you the job finished. It tells you nothing about whether the output is usable. And the difference between those two things is invisible until the worst possible moment.
The reason the checkmark lies is that a backup is only half of a system. The other half is the restore, and the restore is the half that almost never gets exercised. Teams build elaborate backup pipelines with retention policies and offsite replication and encryption, and then never once run the restore path end to end. The restore path is full of assumptions that were true when it was written and may not be true anymore, and none of those assumptions get tested until a real restore is needed.
There is a grim pattern in postmortems where a company loses data despite having backups. The backups existed. They just could not be restored, for a reason that would have been obvious the first time anyone tried. The cost of never testing the restore is zero on every ordinary day and catastrophic on the one day it matters.
A backup you have not restored is not a backup. It is a file that resembles a backup. The only backup that counts is one you have restored into a working system and verified, and that verification has to happen on a schedule, not once during an audit.
The workflow below turns restore verification from a rare heroic drill into a boring scheduled routine. The Claude Code skills handle the mechanical work of restoring and verifying, which is exactly the work that is too tedious to do by hand often enough to matter.
If you care about production safety more broadly, the mindset here is the same one behind the Claude Code for Chaos Engineering workflow, which replaces hope with measurement in a different corner of operations. The first skill answers a question most teams cannot answer precisely: what data do we have, and what is the backup situation for each piece of it. The skill builds and maintains an inventory of every data store and its backup posture.
For each data store, the inventory records what it is, how critical it is, how it is backed up, how often, where the backups live, how long they are retained, and when a restore was last verified. That last field is the one that usually reveals the problem. Most stores have never had a verified restore, and the inventory makes that absence visible instead of letting it hide. The inventory also surfaces the stores that have no backup at all. Every system accumulates data stores that were added quickly and never wired into the backup pipeline. A cache that quietly became a source of truth. A new service with its own database that nobody added to the nightly job. The inventory finds these gaps by reconciling the list of live data stores against the list of stores with backup coverage.
The output is a single honest picture of the backup situation across the whole system. It is uncomfortable to look at the first time, because it usually shows more gaps than the team expected. That discomfort is the point. You cannot fix a coverage gap you cannot see.
The second skill is the heart of the workflow. On a schedule, it takes a recent backup, restores it into an isolated environment, and confirms that the restore succeeds.
The restore happens in a clean, throwaway environment that has no connection to production. The skill provisions the environment, runs the actual restore procedure against the actual backup file, and tears the environment down when it is done. This is the exact path that would run during a real disaster, exercised on a routine schedule against real backups.
Running the real restore path is what makes this valuable. It catches the failures that a checkmark never could. A backup that decompresses wrong. A restore that needs a tool version that is no longer installed. An encryption key that was rotated without updating the restore path. A schema that the restore cannot reconstruct. Every one of these is invisible until a restore is attempted, and the skill attempts one on a schedule so the invisible failures become visible while they are still cheap to fix.
The skill times the restore as well. Knowing that a backup can be restored is necessary but not sufficient. You also need to know how long the restore takes, because the restore time is the floor on your recovery time during a real incident. A restore that works but takes eleven hours is important information to have before the disaster, not during it.
If the restore fails, the skill produces a detailed report of exactly where and why it failed. That report is the early warning that a static backup pipeline can never give you. The failure gets fixed as ordinary work instead of discovered during a catastrophe. Restoring without error is not the same as restoring correct data. The third skill checks that the restored data is actually a faithful copy of what it should be, not just a database that started successfully.
The skill runs a set of integrity checks against the restored data. It confirms that the expected tables and collections exist, that the row counts are within a sane range of the source, that referential integrity holds, that critical records are present and well formed, and that the most recent data in the backup is as recent as it should be given the backup schedule. A restore that produces an empty database, or a database missing its most important table, will pass a naive "did it start" check and fail these.
The verification is tuned per data store, because what counts as correct differs by store. For a store where losing the last hour of writes is catastrophic, the skill checks the recency of the newest records tightly. For a store where approximate counts are fine, it checks more loosely. The inventory drives these expectations, so the verification is as strict as each store's importance demands.
The verification output is a per store verdict that says not just "restored" but "restored and verified correct against these specific criteria." That verdict is the thing I actually wanted all those years I was staring at the green checkmark. It answers the real question instead of a proxy for it.
The approach here mirrors the continuous verification idea in Claude Code for Log Analysis, where the value comes from turning raw signals into a confident verdict rather than a pile of data to interpret under pressure.
The fourth skill turns all of this into a posture the team can see at a glance. It aggregates the inventory, the restore results, and the verification verdicts into a single ongoing report of the backup health across the whole system.
The report answers the questions that matter. Which data stores have a verified restore within their required window. Which stores have failed their last restore. Which stores have no backup coverage at all. What the current restore times are, and whether any of them exceed the recovery targets. The report is the dashboard I wish I had before I learned the hard way that a backup job succeeding is not the same as a backup working.
The report also tracks trends. A restore time that is creeping up over weeks is a warning that the recovery window is quietly closing, and the trend surfaces it before it becomes a surprise. A store whose verification criteria are starting to slip is flagged before the slip becomes a gap.
Critically, the report distinguishes between the two failure modes that a checkmark collapses into one. It separates "the backup did not run" from "the backup ran but cannot be restored." Those need completely different responses, and a system that treats them as the same thing will fix the wrong one.
The inventory skill establishes the picture first. It reconciles the live data stores against the backup coverage and produces the honest map, including the gaps. The team closes the coverage gaps as a first pass, so that every store that matters is at least being backed up.
From there, the restore skill runs on a schedule per store. It pulls a recent backup, restores it into a clean environment, times the restore, and tears the environment down. The verification skill checks that the restored data is correct against the store's criteria. The results flow into the reporting skill. When a restore fails or a verification comes back wrong, it lands as ordinary maintenance work with a detailed report attached. The team fixes the restore path while the fix is cheap, instead of discovering the broken path during a data loss event. The failures that used to hide for a year now show up within a scheduling cycle.
Over time, the report becomes a standing source of truth about recoverability. When someone asks whether the data is safe, the answer is no longer a hopeful "the backups are running." It is a specific "every critical store has a verified restore within its window, and here are the current restore times." That is a different kind of confidence, and it is grounded in evidence.
The first change was that I found problems I did not know I had. The very first scheduled restore run surfaced a store whose backups had been technically succeeding for months while producing files that could not be fully restored. That is exactly the failure that ruins companies, caught on an ordinary afternoon instead of during a disaster.
The second change was in how I think about recovery time. Before the workflow, my recovery time was a guess. After the workflow, it is a measured number per store, updated on every restore run. Planning around a measured number is a fundamentally different activity than planning around a guess, especially when someone above you asks how long a recovery would take and expects a real answer.
The third change was that backup coverage stopped drifting. New data stores get caught by the inventory reconciliation instead of quietly living without backups until they are lost. The gap between "we added a database" and "the database is backed up and verified" shrank from months to a scheduling cycle.
The fourth change was subtler and more valuable. I stopped trusting the green checkmark, and I taught the team to stop trusting it too. The checkmark now means what it should have always meant, which is "the job ran," and the real safety signal comes from the verified restore. Replacing false confidence with real confidence is worth more than any single problem the workflow caught.
For the full picture of how I run production systems with Claude Code, the complete series on DEV.to covers every workflow I depend on, from chaos engineering to log analysis to backup verification. Is it safe to restore into an environment near production?
The restore should happen in an isolated environment with no path back to production, provisioned fresh and torn down after. The whole point is that the restore is a rehearsal, and a rehearsal that can touch the live system is not a rehearsal, it is a risk. Keep the restore environment sealed off, and the workflow adds safety instead of subtracting it.
How often should I run the restore verification?
The right cadence depends on how much data loss the store can tolerate and how often its backup format or restore path changes. A critical store with an active schema deserves frequent verification. A stable, low criticality store can be verified less often. The inventory's criticality rating is what drives the cadence, so the most important stores get the most attention.
What about very large databases where a full restore is expensive?
For very large stores, the full restore can run less frequently, with cheaper partial checks in between that verify the backup's structure and recency without a complete restore. The goal is to balance cost against confidence, and a mix of occasional full restores and frequent partial checks often hits that balance well. What you must not do is skip the full restore entirely, because the partial checks cannot catch every failure the full path can. Does this replace my existing backup tooling?
No. The backups are still taken by whatever tools you already use. This workflow sits on top of the existing pipeline and verifies that its output is actually recoverable. You are not replacing your backup system, you are finally testing it.
The move from trusting the checkmark to verifying the restore is one of the highest leverage changes I have made in how I run production systems. The cost was a few weeks of building the skills. The payoff was discovering, on a calm afternoon, the unrecoverable backup that would have otherwise revealed itself during a data loss event, and every quiet night since when I have known, rather than hoped, that the data was safe. If I started a new system tomorrow, restore verification would be part of it from the first week, because the alternative is trusting a signal that measures the wrong thing.