An operational readiness review is a decision process that asks whether a team can operate a workload through normal demand, change, degradation, incident, and recovery. It is not a final meeting where reviewers discover missing monitoring three days before launch. The review begins during design, scales with risk, collects verifiable evidence as delivery proceeds, and records residual risk and ownership when the organization decides to launch.
A strong ORR program converts actual operational learning into reusable requirements. Repeated capacity exhaustion becomes a tested load-and-shedding control. A failed certificate rotation becomes inventory, alerting, and rehearsal evidence. The checklist remains short enough to guide decisions because each question names the risk it controls, expected evidence, applicability, and exception path. Generic compliance theater is removed.
Define review scope, triggers, and proportional depth
Trigger review for a new production service, critical feature, material architecture or dependency change, data migration, new region, changed availability target, ownership transfer, major traffic increase, or recurring operational failure. Classify workload risk by customer criticality, data sensitivity, blast radius, reversibility, novelty, regulatory obligation, external dependency, and operator experience. The risk tier determines required evidence, reviewer specialties, rehearsal depth, and approval authority.
AWS describes Operational Readiness Reviews as a data-driven mechanism focused on eliminating known common causes of workload impact in its official ORR guidance. Define what the ORR does not replace: security threat assessment, privacy review, architecture review, legal approval, and change authorization may provide inputs or separate decisions. Integrate their evidence rather than creating duplicate questionnaires.
| Tier | Typical change | Minimum process | Decision authority |
|---|---|---|---|
| Low | Reversible internal service with established pattern | Self-service evidence and peer approval | Service owner |
| Moderate | Customer feature or meaningful dependency change | Cross-functional review and staged launch plan | Engineering and product owners |
| High | Critical journey, sensitive data, new architecture, hard migration | Independent specialists, failure rehearsal, executive risk acceptance | Accountable business and technology leaders |
| Recurring | Existing workload after repeated incidents or ownership transfer | Focused review against changed risk and prior actions | Service owner plus reliability governance |
Build checklist questions that request evidence and a decision
Each question should contain risk statement, applicability criteria, required evidence, acceptance condition, owner, source incident or standard, and review interval. Ask Can the service shed optional work before saturation, and where is the test result? rather than Is capacity handled? Evidence can be an architecture decision, test report, dashboard, alert evaluation, runbook exercise, restore result, ownership record, supplier agreement, or approved exception. Links need immutable revision or captured result, not an editable homepage.
Cover ownership and on-call, service objectives, architecture and dependencies, capacity, deployment and rollback, configuration, observability, incident response, data integrity, backup and restore, security and privacy operations, continuity, customer support, cost controls, vendor failure, and end-of-life. AWS's consistent readiness review practice groups architectural recommendations, operational process, event management, and release quality and says post-incident analysis should evolve the review.
| Domain | Decision question | Strong evidence | Weak substitute |
|---|---|---|---|
| Ownership | Can an authorized responder act at all supported times? | Roster, access test, escalation exercise | Team name in a wiki |
| Reliability | Do SLOs and failure modes reflect user journeys? | SLI query tests, failure-mode review, budget policy | Infrastructure uptime target |
| Capacity | Can the system meet forecast and degrade safely? | Representative load test and shedding result | CPU screenshot from staging |
| Change safety | Can a bad release be detected and reversed? | Canary gates and timed rollback rehearsal | Statement that rollback exists |
| Data recovery | Can required data and service be restored within objectives? | Successful restore with integrity checks | Backup job success only |
| Incident response | Can people diagnose, communicate, and hand off? | Scenario exercise and updated runbook | Unopened incident template |
Collect readiness evidence throughout delivery
At design, identify risk tier, applicable questions, owners, dependencies, SLO intent, and hard-to-reverse decisions. During build, create dashboards, runbooks, alerts, capacity tests, deployment controls, and recovery paths alongside functionality. Before launch, validate evidence in the target-like environment and rehearse critical failures. After launch, verify production telemetry, ownership, cost, paging, and customer support during progressive exposure. Readiness is a lifecycle, not a document handoff.
AWS's ORR adoption guidance explicitly places the process across the software development lifecycle, beginning during design. Automate collection where semantics are reliable: repository ownership, deployment strategy, alert linkage, recent restore test, dependency catalog, and open exceptions. Automation can flag missing or stale evidence but should not decide that a complex failure mode is acceptable.
Run the review as risk discussion, not checklist recitation
Provide the evidence pack before the meeting. Reviewers focus on high-risk, nonapplicable, failed, stale, or exceptional items. The service owner demonstrates customer journey, architecture, critical dependencies, SLOs, failure behavior, launch sequence, rollback or containment, and incident command path. Ask how evidence was produced and what changed because of it. Do not spend meeting time reading green answers aloud.
Record one of four outcomes: approved; approved with bounded follow-up; delayed pending mandatory evidence; or rejected because risk is outside tolerance. Every gap has severity, customer consequence, owner, due date, launch relation, and verification. The accountable decision maker owns residual business risk; reviewers own the quality of their advice, not the service. Psychological safety matters because hidden uncertainty is more dangerous than an honest red item.
Govern exceptions with compensating controls and expiry
An exception names requirement, reason, risk, affected customers and duration, compensating controls, monitoring, rollback trigger, owner, approver, expiry, and remediation. A lower launch cohort, manual watch, restricted functionality, or shorter review interval can reduce exposure while a durable control is completed. We will monitor closely is not a control unless the signal, threshold, responder, action, and coverage are explicit.
Block launch when an unmitigated gap threatens safety, data integrity, legal obligation, basic recovery, or a critical customer promise beyond authorized tolerance. Avoid treating every low-risk documentation gap equally. Track exception age and recurrence by requirement and team. Repeated waivers indicate an impractical standard, missing platform capability, chronic underinvestment, or weak decision authority. Resolve the systemic cause instead of normalizing overdue risk.
Convert incident and exercise learning into better gates
Review postmortems and exercises for reusable conditions, not only service-specific fixes. Ask whether the failure could recur elsewhere, which readiness question would have exposed it, what evidence would be sufficient, and which workloads need reassessment. Create a candidate requirement with source, owner, affected population, implementation cost, and expected risk reduction. Pilot it before adding it globally; some incident lessons are too local for a universal gate.
Google's Postmortem Culture chapter emphasizes concrete, owned, prioritized, measurable action items and system improvement rather than blame. Use the same standard for ORR changes. Retire duplicate or low-value questions, update platform templates, and notify affected owners. A checklist that only grows eventually receives ceremonial yes answers. Track which questions find material risk and which controls reduce incident frequency or impact.
Measure adoption, decision quality, and operating outcomes
Measure review lead time, evidence freshness, late risk discovery, exception age, repeated gaps, launch rollback, incident classes covered by questions, restoration exercise success, and reviewer load. Do not optimize pass rate; a high pass rate can mean mature teams or a harmless checklist. Sample approved services to verify evidence in production. Compare incidents with review claims to find false confidence and blind spots.
Give the program an owner and editorial process. Domain experts maintain question quality; platform teams provide paved controls; service teams own evidence; product and technology leaders accept residual risk. Microsoft's February 2026 Operational Excellence checklist includes observability, structured incident management, automation, and safe deployment, providing a current cross-cloud comparison. Use external frameworks as inputs, while internal incident evidence determines local priorities.
Operational readiness review takeaways
- Trigger reviews by material operational risk and scale depth by criticality, novelty, blast radius, and reversibility.
- Ask decision-quality questions that name risk, applicability, acceptance, and verifiable evidence.
- Begin during design and collect proof through build, rehearsal, progressive launch, and production verification.
- Focus review meetings on uncertainty, failed evidence, exceptions, and residual risk ownership.
- Give exceptions compensating controls, approvers, expiry, and a verified remediation path.
- Turn recurring incident lessons into tested questions, and retire checklist items that no longer reduce risk.
Operational readiness review FAQ
Is an ORR the same as an architecture review?
No. Architecture review examines design quality and tradeoffs. ORR asks whether the organization can safely deploy, observe, support, degrade, respond, and recover the workload. Architecture evidence is an important input, and the reviews can share a forum, but their decisions differ.
When should an ORR start?
At design, when teams can still change ownership, dependencies, objectives, and recovery architecture economically. Final verification occurs before launch and during progressive production exposure. Starting in the final week turns foreseeable design work into waivers.
Can operational readiness be fully automated?
Evidence collection and stale-control detection can be highly automated. Risk interpretation, exception acceptance, novel failure analysis, and business tradeoffs still need accountable people. Automate factual checks while preserving explicit judgment and decision records.
Conclusion
An operational readiness review is valuable when it changes design and launch behavior before customers discover the gaps. Risk-tiered scope, concrete evidence, rehearsed controls, owned exceptions, and incident-fed standards make readiness an operating mechanism rather than paperwork. The program stays credible by measuring outcomes and continually removing questions that do not help teams run services more safely.