Restore & Recovery Playbook for Defensible Appraisal Archives: Schedules, Acceptance Criteria, and Test‑Report Templates

Most appraisal firms treat their archive like a smoke detector they installed and never tested. Files go in, the retention clock runs, everyone assumes the restore button works. Then an AMC audit request, a state board inquiry, or a legal hold lands — and someone tries to pull a 3‑year‑old file. The fixity check fails. Or the restore completes but the exhibits are corrupt. Or nobody can prove the file hasn't been touched since ingest.

That's the gap this playbook closes. If you've already built an end-to-end evidence lifecycle and archival process, this is the piece that makes it defensible: repeatable restore testing with real frequencies, pass/fail thresholds, a machine‑readable report format, and a quarantine workflow for when things break. We're not rehashing how you capture files or how long you keep them. One question only: can you prove, on demand, that what you stored comes back intact?

The failure nobody catches until it's expensive

Here's the specific scenario that burns firms. A review appraiser requests the full workfile for an assignment completed 28 months ago — comps, MLS pulls, the signed report PDF, inspection photos, the data manifest. The archive system returns the files. Everything looks fine. Nobody runs a hash. Three weeks later, during a deeper audit, someone compares the restored report PDF to a copy an underwriter had saved, and the byte counts don't match. Now you can't demonstrate the archived version is the version you delivered.

This usually happens for one of three reasons:

1. 1
   Silent bit rot on aging storage — cheap cold storage or an old NAS — that no periodic fixity check ever caught.
2. 2
   A migration or backup job that re‑encoded or truncated files months ago, with nobody noticing because the restore path was never exercised.
3. 3
   A broken dependency — the file restored fine, but the embedded exhibits or MISMO attachments were stored separately and the pointer is dead.

Common thread: the archive was never tested as a restore system. Backups running green in a dashboard tell you the write succeeded. They tell you nothing about whether a read succeeds years later, or whether the bytes match what went in.

Backups running green in a dashboard tell you the write succeeded. They tell you nothing about whether a read succeeds years later, or whether the bytes match what went in.

What "tested" actually means here

A restore test has to check four distinct things, and firms almost always skip at least two:

1. 1
   Retrievability — the object can be fetched from the archive within an acceptable time window.
2. 2
   Fixity — the SHA‑256 of the restored object matches the hash recorded at ingest.
3. 3
   Completeness — all related objects (report, exhibits, manifest, photos) come back together, not just the headline PDF.
4. 4
   Usability — the restored file actually opens and renders correctly.

Fixity is what carries the legal weight. If your ingest process didn't record a hash per object, stop and fix that first — there's nothing to test against. Everything below assumes you capture a SHA‑256 at ingest and store it in a manifest separate from the object itself.

Fixity is what carries the legal weight. If your ingest process didn't record a hash per object, stop and fix that first — there's nothing to test against. Everything below assumes you capture a SHA‑256 at ingest and store it in a manifest separate from the object itself.

Recommended test frequencies

You don't restore everything constantly — that's expensive and pointless. You sample often and go deep occasionally. This cadence works for firms from solo shops to multi‑office operations:

The post‑change restore is the one most firms don't have — and it's the one that would have caught the "migration re‑encoded the PDFs" scenario. Any time you touch the storage layer, you test.

Sampling methodology that holds up in an audit

1. 1
   Age bands — e.g., 0–12 months, 13–36 months, 37+ months. Older files on colder storage fail more often; weight the sample toward them.
2. 2
   Storage tier — hot, warm, cold/archive. A hybrid shop might have hot in cloud object storage and cold on tape or deep‑archive; both need coverage.
3. 3
   File type — report PDF, photo bundle, data manifest, MISMO/XML attachment. A restore that pulls PDFs fine but drops photo bundles is a real and common partial failure.

“

Pro-tip: Weight the quarterly sample toward the oldest age band and cold tiers — they tend to fail more often.

For the quarterly sample, a practical size for most firms is the larger of 30 objects or 1% of the archive, distributed across the strata with over‑weighting on the oldest band. A firm with 8,000 archived assignments would pull roughly 80 objects, maybe half from the 37+ month band. The point isn't statistical perfection — it's a consistent, written rule you follow every quarter so the sample isn't cherry‑picked.

Pass/fail acceptance criteria and alert thresholds

Vague criteria are how borderline failures get waved through. Make them explicit and binary wherever possible.

Per‑object pass criteria (all must be true):

1. 1
   Object retrieved within the SLA for its storage tier (hot

   ≤ 5 min; warm: ≤ 1 hr; cold: ≤ 12 hr — adjust to your tier's published retrieval times).

2. 2
   Restored SHA‑256 exactly matches the ingest manifest hash.
3. 3
   All linked objects in the manifest restored (completeness count matches expected count).
4. 4
   File opens/renders without error on a current standard viewer.

Any single object failing fixity = hard fail. A hash either matches or it doesn't.

Batch‑level alert thresholds:

1. 1
   Green

   100% of sampled objects pass all criteria.

2. 2
   Yellow (investigate)

   1 retrievability or usability failure, fixity 100% clean. Log it, open a ticket, no quarantine needed yet.

3. 3
   Red (quarantine + escalate)

   Any fixity failure, OR ≥ 2 retrievability failures in a single sweep, OR any failure in the annual full restore.

The asymmetry is deliberate. A slow retrieval is an operational annoyance. A fixity mismatch is a potential integrity problem that could invalidate the file's evidentiary value, so it trips the serious alarm immediately.

Automating the triggers

Running this by hand every quarter means it quietly stops happening by Q3. The schedule should fire itself. Most archive and backup platforms expose an API or support webhooks — wire the tests to a scheduler and route results somewhere a human actually reviews.

A minimal trigger pattern looks like this — a cron‑style scheduler calls your restore‑test endpoint, which pulls the sample, runs the checks, and posts a report:

The diagram shows the end‑to‑end flow from scheduler to alerting; it's the minimal plumbing you need so tests don't rely on memory.

{ "testtype": "quarterlysample", "samplerule": "stratifiedrandom", "strata": { "agebands": ["0-12", "13-36", "37+"], "weights": [0.25, 0.25, 0.50], "tiers": ["hot", "warm", "cold"], "filetypes":["reportpdf", "photobundle", "manifest", "mismoxml"] }, "samplesize": 80, "notify_webhook": "https://ops.internal/hooks/restore-report" }

{ "event": "restoretest.failed", "severity": "red", "objectid": "APR-2023-04817-report", "reason": "fixitymismatch", "expectedsha256": "a3f9...c21", "actual_sha256": "a3f9...e07" }

If your platform supports scheduled jobs and outbound webhooks natively, you may not need custom code — just configuration. Where firms use a workflow or operations platform to coordinate assignments, the same event‑driven plumbing that handles handoffs can also schedule these tests and route failures to the right person, so restore testing stops depending on someone remembering to do it.

A JSON test‑report schema (and a one‑row CSV)

Auditors and your own future self both want a consistent, machine‑readable record of every test run. Here's a schema that captures what matters without bloat:

{ "reportid": "RT-2026Q1-0001", "testtype": "quarterlysample", "runstarted": "2026-03-31T14:02:11Z", "runcompleted": "2026-03-31T14:49:53Z", "operator": "automated-scheduler", "approvedby": null, "sample": { "method": "stratifiedrandom", "size": 80, "stratadistribution": { "0-12": 20, "13-36": 20, "37+": 40 } }, "results": { "objectstested": 80, "passed": 79, "retrievabilityfailures": 0, "fixityfailures": 1, "completenessfailures": 0, "usabilityfailures": 0 }, "severity": "red", "failedobjects": [ { "objectid": "APR-2023-04817-report", "storagetier": "cold", "agemonths": 33, "failuretype": "fixitymismatch", "expectedsha256": "a3f9...c21", "actualsha256": "a3f9...e07", "quarantined": true, "quarantineref": "Q-2026-0007" } ], "remediation_ticket": "REM-2026-0041" }

reportid,testtype,runcompleted,objectstested,passed,fixityfailures,retrievabilityfailures,severity,remediationticket RT-2026Q1-0001,quarterlysample,2026-03-31T14:49:53Z,80,79,1,0,red,REM-2026-0041

Keep every report. A clean quarterly history is itself evidence — it shows a reviewer or regulator that integrity testing is a standing control, not something you scrambled together the week before the audit.

The quarantine + evidence‑capture workflow for failures

When a fixity failure trips, resist the urge to immediately "fix" it by overwriting from a backup. That destroys the evidence of what failed. Follow a capture‑first sequence:

1. 1
   Freeze the object in place. Mark it quarantined in the archive so no automated job overwrites or deletes it.
2. 2
   Capture the failed state. Record the restored object's actual hash, byte count, timestamps, storage tier, and the retrieval path used. Snapshot the manifest entry showing the expected hash. This is your proof of what went wrong and when it was detected.
3. 3
   Preserve chain of custody. Log who detected it (the automated run), who was notified, and the quarantine reference ID. Do not modify the failed object.
4. 4
   Locate a known‑good source. Check secondary backups, the delivery copy sent to the client/AMC, or an offline replica. Hash any candidate against the original ingest manifest before trusting it.
5. 5
   Restore and re‑verify. Replace the corrupted object only from a source that passes fixity against the original manifest hash. Run the full acceptance criteria on the replacement.
6. 6
   Document root cause. Bit rot, bad migration, hardware fault, deleted dependency — note it on the remediation ticket so the pattern is visible if it recurs.
7. 7
   Close with evidence attached. Link the original failure report, the quarantine record, the known‑good source, and the re‑verification result.

The step teams most consistently skip is the freeze. Someone notices a bad object and overwrites it within minutes, and then has nothing to show an auditor except "it's fine now" — which is exactly the kind of unverifiable claim the whole program exists to prevent. Document first, fix second.

Remediation SLAs and the role/approval matrix

Failures need owners and deadlines, or they linger. Tie remediation urgency to severity:

A simple approval matrix keeps the right eyes on the right events:

1. 1
   Automated scheduler — runs tests, generates reports, fires alerts. No approval authority.
2. 2
   Operations lead — reviews all reports, triages yellow events, owns retrievability remediation.
3. 3
   Compliance lead — owns every red fixity event, signs off on quarantine and chain‑of‑custody records.
4. 4
   Firm principal / owner — co‑approves closure of any fixity failure and reviews the annual full‑restore result.
5. 5
   IT / storage vendor contact — executes technical recovery; does not approve its own work.

Separating who executes recovery from who approves closure is the control that keeps the record honest. The person who restored the file shouldn't be the sole person certifying it came back clean.

On‑prem vs. cloud vs. hybrid: what actually changes

The checklist is the same everywhere; the knobs differ.

On‑prem — You own the fixity sweeps entirely, so run them deliberately (quarterly minimum). Cheap disk and aging NAS arrays are where silent corruption shows up most. Watch for RAID that reports healthy but is quietly returning stale blocks.

Cloud object storage — Many platforms offer built‑in integrity checks and even automatic repair, but don't assume they validate against your ingest hash. Their checksum is theirs. You still verify against the SHA‑256 you recorded at ingest. Cold and deep‑archive tiers have long retrieval windows — factor those into your tier SLAs so a slow pull isn't miscounted as a retrieval failure.

Hybrid — The dangerous seam is the handoff between tiers and the pointer/manifest linking an object in one place to exhibits stored in another. Your completeness check matters most here. The annual full restore should deliberately pull one assignment whose objects span both environments.

When this level of rigor makes sense — and when it's overkill

For a solo appraiser with a few hundred files and a single cloud backup, monthly automated fixity plus a quarterly 30‑object sample is plenty. The annual full‑path restore drill still matters at that size, because the one time you need it is the one time it has to work.

Where this falls apart is doing it manually at scale. A firm with tens of thousands of objects trying to run stratified sample restores by hand every quarter will burn hours on it and skip it the moment things get busy. If you can't automate the triggers and the reporting, scale the frequency down to something you'll actually sustain rather than pretending you'll do a heroic manual run every three months.

A short real scenario

A two‑office residential appraisal firm with around 11,000 archived assignments moved from an aging on‑prem NAS to cloud object storage over a weekend. Backups stayed green the whole time. Nobody ran a post‑change restore test — it wasn't part of their process yet.

Four months later they built a restore testing checklist and ran the first quarterly stratified sample of roughly 110 objects. Three failed fixity, all from the 24–36 month band, all photo bundles. The migration had silently dropped a subfolder structure on a batch of older bundles — objects restored but came back incomplete. Because the pre‑migration NAS was still sitting offline, they recovered clean copies, re‑verified against the original manifests, and closed remediation inside the 5‑day SLA.

The meaningful outcome wasn't the three recovered files. It was that at the next AMC audit, they handed over a year of clean quarterly reports plus a documented failure that was caught and remediated under a written SLA. That paper trail did more for their credibility than a flawless‑but‑untested archive ever could have.

Bringing it together

A restore testing checklist isn't paperwork for its own sake — it's the difference between believing your archive works and proving it. On‑ingest fixity, periodic sweeps, quarterly stratified sample restores, an annual full‑path drill, hard binary acceptance criteria, and a capture‑first quarantine workflow give you a control you can actually defend.

Set frequencies you can sustain, automate the triggers so they fire without anyone remembering, keep every report, and separate who recovers from who approves. Do that consistently and the scary scenario — the file that won't come back, or comes back altered, right when it matters most — stops being a risk you're exposed to and becomes one you've already tested against.

Set frequencies you can sustain, automate the triggers so they fire without anyone remembering, keep every report, and separate who recovers from who approves. Do that consistently and the scary scenario — the file that won't come back, or comes back altered, right when it matters most — stops being a risk you're exposed to and becomes one you've already tested against.