An error budget policy turns a customer reliability objective into predetermined action. Without policy, a service-level objective is only a chart: product leaders continue launches, operators request caution, and executives arbitrate each incident from scratch. A usable policy states what consumes budget, which release and investment decisions change at different burn conditions, who can approve exceptions, and how service owners return to normal operation.
Portfolio use does not mean pooling every service into one percentage. Each service needs user-centered indicators and an economically justified objective. Leadership can then use common decision categories, exposure, and trend evidence to allocate reliability work across services. The aim is a repeatable negotiation between innovation and reliability, not a punishment for an outage or a target of zero budget consumption.
Anchor every budget in a user-centered SLO
Define service, users, valid events, good event, exclusions, target, compliance window, measurement source, data quality, and owner. Examples include successful eligible checkout attempts, search responses under a latency threshold, or jobs completed before a customer deadline. Use request-based measures when volume should weight impact and window-based measures when time availability is the promise. Separate internal SLOs from contractual SLAs and set them with enough margin to trigger action before a legal or commercial breach.
An error budget is the allowed bad fraction implied by the objective. Google's current service monitoring concepts express request budget as (1 - SLO goal) x eligible events. For 10 million eligible requests and a 99.9% objective, 10,000 bad requests are allowed in that period. That arithmetic is simple; deciding which requests and failures represent the user promise is the difficult governance work.
| Field | Question | Evidence | Review trigger |
|---|---|---|---|
| User journey | Which customer outcome is protected? | Journey map and product owner approval | Material product or audience change |
| SLI | How are good and valid events counted? | Query, instrumentation, exclusions, tests | Telemetry or architecture change |
| Objective and window | How reliable and over what period? | Risk and cost decision | Demand, contract, or tolerance change |
| Budget owner | Who changes release and investment posture? | Named product and engineering owners | Ownership reorganization |
| Dependencies | Which external failures count and who acts? | Dependency map and contracts | Provider or topology change |
| Data quality | Can the decision measure be trusted? | Coverage, delay, correction process | Missing or biased telemetry |
Define policy bands from healthy to exhausted
Use remaining budget and burn rate together. Remaining budget shows period position; burn rate shows trajectory. A service can have substantial budget left but be consuming it fast enough to exhaust within hours. Conversely, a recently exhausted rolling window may be recovering as bad events age out. Define multi-window alerts for urgent fast burn and slower chronic burn, then map them to actions that are proportional to customer risk.
Google's published example error budget policy halts nonessential changes after budget exhaustion and directs reliability work when the service caused the miss. Treat that as a pattern, not a universal threshold. A low-risk internal service may slow release; a critical payment service may freeze broad changes earlier while allowing tested mitigations, security fixes, and capacity work. State how emergency changes are authorized and observed.
| Band | Evidence | Release posture | Investment response |
|---|---|---|---|
| Healthy | Consumption near planned pace; no fast burn | Normal progressive delivery | Fund planned reliability backlog |
| Watch | Projected exhaustion or repeated localized impact | Reduce batch size and raise canary scrutiny | Assign owner to dominant budget spend |
| Constrained | Material burn or little budget remains | Pause high-risk changes; permit low-risk fixes | Move capacity to prevention and mitigation |
| Exhausted | SLO missed for policy window | Freeze nonessential risk with documented exceptions | Execute reliability recovery plan |
| Recovering | Burn controlled and safeguards verified | Resume by staged approval | Track recurrence controls to completion |
Connect budget state to release control without blocking recovery
Attach service budget state and policy action to deployment risk assessment. In constrained state, require smaller batches, stronger test evidence, narrower canaries, longer observation, and a proven rollback or disable path. Avoid an indiscriminate pipeline lock that blocks incident mitigation. Classify changes by risk and intent: customer feature, routine maintenance, reliability improvement, security response, emergency mitigation, and data repair.
Record every exception with change, customer rationale, risk controls, approver, expiry, and outcome. Exceptions must remain exceptional; repeated approval indicates the SLO, release process, or business priority needs formal revision. Automate evidence and routing, not the final business judgment for consequential launches. After resumption, expand progressively and verify that the budget burn and user indicators remain stable.
Allocate reliability investment from budget evidence
Attribute budget spend to incident and failure classes: deployment, capacity, dependency, data, operational procedure, abuse, client, region, or unknown. Attribution supports investment but should not remove user-visible failures from the SLI simply because another team caused them. A customer experiences the product boundary. Use dependency evidence to negotiate supplier SLOs, isolation, fallbacks, or product promises, while keeping internal accountability separate from external reporting.
At portfolio planning, compare customer harm, budget trend, dominant failure mode, recurrence, concentration, contractual exposure, change demand, and costed options. Fund the intervention with the best expected risk reduction per constrained resource, not automatically the service with the reddest dashboard. Options include test coverage, decoupling, capacity, graceful degradation, operability, migration, or changing an unjustifiably strict objective. Google's Embracing Risk chapter explains that extreme reliability has resource and opportunity costs and that objectives should align with business tolerance.
Govern portfolio views, dependency risk, and exceptions
A portfolio view should show service criticality, objective, current burn, remaining fraction, recent large spends, trend, policy band, open reliability commitments, exceptions, and data quality. Do not add raw budget percentages across services: different SLIs, event volumes, windows, and user harms make that arithmetic meaningless. Group by customer journey or business capability and show the weakest critical dependency alongside end-to-end outcomes.
Establish a reliability council or existing portfolio forum to approve SLO creation and material changes, resolve disputed counting, review prolonged constrained states, and allocate cross-team work. Product and engineering jointly own objectives. Finance and risk participate where contractual or material exposure exists. Keep a decision log. A service should not quietly loosen its objective after an incident to make the chart green; changes take effect prospectively with rationale.
Alert on burn and review whether policy improved outcomes
Page only when responders can take urgent action. Use fast- and slow-burn windows to balance detection and noise. Budget dashboards show impact; diagnostic dashboards explain causes. Google Cloud's SLO alert policy guidance uses burn-rate selectors for alerting. Validate queries against synthetic good and bad events, monitor late data, and define how corrections affect decisions.
Review whether the policy reduced repeat impact, shortened high-burn periods, improved release decisions, and closed reliability work. Also inspect costs: blocked valuable changes, excessive approvals, noisy pages, distorted SLIs, or teams routing releases around controls. Update actions after postmortems and exercises. Preserve policy versions and evaluate changes prospectively rather than continually moving thresholds until incidents disappear from reports.
Apply the policy to a portfolio decision
Consider checkout, catalog search, and internal merchandising services entering quarterly planning. Checkout has little budget left because one database failure class recurred; search is healthy but has an untested regional failover; merchandising missed its internal objective during a low-volume batch delay. The portfolio should not rank them by remaining percentage. It may fund checkout isolation immediately, schedule a search failover exercise before a traffic peak, and accept merchandising risk with a smaller fix because customer harm and contractual exposure differ.
Record the alternatives not chosen and expected reduction in bad events. At the next review, compare actual incident class, budget trajectory, delivery delay, and cost against that forecast. This closes the investment loop and reveals whether a reliability project reduced risk or merely produced infrastructure. When evidence changes, reallocate work; a budget policy should make tradeoffs reproducible, not freeze an annual plan against new customer impact.
Error budget policy takeaways
- Start with a user-centered SLI and economically justified objective; the remaining fraction is the budget.
- Use budget remaining and multi-window burn rate to distinguish position from trajectory.
- Map policy bands to proportional release controls and reliability work before incidents occur.
- Allow controlled mitigation and security exceptions so a freeze does not block recovery.
- Allocate portfolio investment from customer harm and dominant failure evidence, not pooled percentages.
- Version objectives and policies, log exceptions, and review unintended incentives.
Error budget policy FAQ
Is the goal to preserve the entire error budget?
No. The budget represents tolerated unreliability under the chosen objective. Persistently spending none may mean the service is overengineered or the SLI misses harm. The goal is to operate within an intentional risk envelope while learning from significant spend.
Should dependency failures consume the product's budget?
If they make the protected user event bad, they normally count in the product SLI. Internal attribution can assign action to the dependency owner or supplier. Excluding them from the user measure hides the actual promise and weakens incentive to build resilience.
Does budget exhaustion require a total change freeze?
Not universally. A policy may pause nonessential risk while allowing security fixes, mitigations, and well-controlled reliability work. Define risk classes, approvals, and verification in advance. An undifferentiated freeze can prolong impact.
Conclusion
Error budgets become portfolio controls only when reliability evidence changes an actual decision. A defensible SLO, proportional bands, risk-aware release posture, costed remediation, and transparent exceptions create that connection. Leadership can then balance feature investment and reliability using shared rules while each service remains accountable to the customer outcome its budget protects.