SRE toil measurement should reveal work that scales with service growth and crowds out engineering, not create a timesheet contest. Toil is operational work with several recurring characteristics: it is manual, repetitive, automatable, tactical, carries little enduring value, and grows as the service grows. A task need not satisfy every characteristic, and unpleasant work is not automatically toil.
The purpose of measuring is to choose interventions and verify that capacity returns. Teams need enough evidence to compare recurring release repair, access requests, alert handling, certificate rotation, quota adjustments, and data correction without forcing exact minute-by-minute accounting. The best result may be automation, but it can also be deleting a process, simplifying the service, changing a policy, improving a product interface, or accepting low-volume manual work.
Classify work before counting hours
Create a shared rubric with examples from the team. Separate toil from engineering project work, incident response, learning, and organizational overhead. Writing an automation that permanently removes recurring manual action is engineering. Executing the action for the fiftieth time is likely toil. A novel incident investigation may create enduring knowledge; repeatedly dismissing the same unactionable alert does not.
Classify the activity, not the person or job title. The same database failover can be valuable practice during a planned exercise and toil when a known defect requires weekly intervention. Record the source system and trigger so the backlog points toward a root cause. Allow operators to challenge the category, since a management-only taxonomy tends to miss hidden work and procedural nuance.
| Work example | Likely class | Reason | Potential response |
|---|---|---|---|
| Repeat a documented deployment repair | Toil | Manual, recurring, automatable, no enduring change | Fix pipeline or remove fragile step |
| Investigate a novel failure mode | Engineering or incident work | Creates new understanding and controls | Document and prevent recurrence |
| Approve routine access from complete evidence | Often toil | Rule-based demand scales with users | Policy automation or self-service |
| Review a risky one-off exception | Judgment work | Context and accountability matter | Improve evidence, retain human decision |
| Attend team planning | Overhead | Necessary coordination, not service operation | Improve meeting design, do not label toil |
Sample operational demand with low friction
Use several inputs: ticket categories, pages, chat or support requests, deployment interventions, access workflows, recurring calendar work, and short periodic operator surveys. For two to four representative weeks, ask people to tag activities with service, trigger, duration band, interruption level, and toil confidence. Sampling is usually sufficient to identify the largest sources without creating permanent tracking toil.
Account for interruption and waiting. Ten five-minute actions distributed across a day may cost more focus than one fifty-minute block. Distinguish active handling from elapsed process time, but include delay imposed on customers or releases. Capture after-hours burden and concentration: an activity that only one specialist can perform creates continuity and burnout risk even when total hours are modest.
Turn observations into a normalized toil inventory
Aggregate by recurring work item rather than by operator. A useful record contains demand count, active minutes per occurrence, interruption cost, growth driver, error or security risk, customer delay, current owner, automation readiness, and confidence in the estimate. Normalize to a monthly range. Preserve uncertainty; “12 to 18 hours” is more honest than a precise number derived from incomplete tickets.
Estimate scaling behavior. Some work grows with deployments, tenants, certificates, incidents, data volume, or regions; some is fixed. A small task with steep growth can outrank today’s largest item. Also look for correlated toil: a poor service interface may generate access requests, support tickets, manual data fixes, and on-call pages that appear as separate backlog entries but share one cause.
| Prioritization factor | How to estimate | Why it matters | Caution |
|---|---|---|---|
| Recurring labor | Frequency times active effort range | Shows reclaimable capacity | Do not equate salary with total value |
| Interruption | Pages, context switches, after-hours events | Captures cognitive and retention cost | Use team-level data, not rankings |
| Risk exposure | Error, security, compliance, and recovery impact | Manual steps can fail at the worst time | Avoid invented monetary loss |
| Growth rate | Demand driver and forecast range | Finds work that will become dominant | Revisit forecast after product change |
| Removal confidence | Evidence the intervention prevents demand | Reduces speculative automation | Pilot before funding a large platform |
Prioritize root-cause removal over task automation
For each item, ask whether the work must exist. Delete obsolete reports, remove unused environments, simplify approval rules, improve defaults, or make the service own its lifecycle before automating clicks. Automation can freeze a bad process and add software that itself needs operation. Prefer changing the demand-generating system when one change eliminates several manual workflows.
When automation is appropriate, define supported inputs, safe outputs, permissions, audit evidence, failure handling, rollback, and an owner. Start with the common low-risk path and make exceptions visible. Do not conceal a human judgment step inside brittle rules. Self-service should provide validation and understandable errors so the work is eliminated rather than shifted from SRE to confused application teams.
Build an investment case with ranges and cost of delay
Compare intervention effort and ongoing maintenance with expected monthly demand removed, risk reduced, customer delay improved, and engineering options unlocked. Calculate a break-even range, not a promise. Include migration, documentation, support, and decommissioning. A three-week automation that saves two hours a year may be irrational unless it also removes severe error or compliance exposure.
Use cost of delay to protect preventive work from an endless feature queue. Show how toil grows if no action is taken and which roadmap work will be displaced. Reserve explicit engineering capacity or set a team-level operational-work threshold with leadership. A threshold is a trigger for reviewing demand and staffing, not a quota that encourages relabeling work or rejecting necessary operations.
Fund a balanced automation portfolio
Mix quick removals, root-cause projects, self-service improvements, and strategic redesign. Assign a product owner for the backlog and require service teams that generate toil to participate. Sequence dependencies: standardize an interface before building workflow automation, and improve telemetry before automating remediation. Limit work in progress so half-built tools do not add another manual path.
Set stop criteria. Cancel or reshape work when demand disappears, a pilot does not remove handling time, exception rates remain high, or maintenance exceeds the benefit. For sensitive actions, keep human confirmation until evidence supports broader autonomy. Automation success is not lines of code, tasks executed, or a launch date; it is safe recurring demand that no longer needs human attention.
Verify that time and reliability were actually returned
After launch, compare occurrence rate, active handling, interruption, completion time, error rate, escalation, and operator experience with the baseline. Watch for work displaced to another team, new exception handling, or manual monitoring of the automation. Decommission the old path and update runbooks. Reinvest released capacity in reliability engineering and product improvement rather than allowing a new stream of unmeasured operational demand to fill it.
Report at team and service level. Do not publish individual toil league tables or use the data as a performance score. People who surface hidden work are improving the system; penalizing them corrupts the measure. Share the top sources, funded responses, verified reduction, residual risk, and next review. Qualitative feedback explains whether fewer hours also produced a healthier on-call experience.
Calibrate estimates after completion. Compare forecast demand, build effort, exception rate, maintenance, and time returned with the original range. Use the difference to improve future investment cases rather than to punish optimistic teams. Some benefits appear as fewer errors or faster customer completion instead of saved hours; retain those outcomes separately. When no measurable benefit appears, retire the automation or change the process instead of sustaining it to defend the sunk cost.
Key takeaways
- Use a shared toil rubric and classify activities rather than people.
- Sample multiple demand sources and account for interruption, expertise concentration, and growth.
- Aggregate a range-based inventory by recurring work item and root cause.
- Delete or simplify work before automating it, and fund based on delay, risk, and verified removal.
- Measure demand after launch and prevent individual productivity scoring.
Frequently asked questions
Should every team use a 50% toil limit?
Google describes limiting operational work for SRE teams, but a local threshold must reflect team mission and service stage. Use it as a governance signal that protects engineering capacity, not as a universal benchmark or a reason to hide required work.
Should all toil be automated?
No. Rare low-risk work can cost less to perform manually than to automate safely. Remove unnecessary demand first, then automate where recurrence, growth, risk, delay, or interruption justifies build and maintenance cost.
Does toil measurement require detailed time tracking?
Usually not. Short samples, duration bands, workflow events, and ticket data can identify the major sources. Use detailed observation for ambiguous high-value candidates, and stop collecting fields that do not change prioritization.
Conclusion
Toil becomes manageable when teams can name its source, estimate its trajectory, and choose the smallest durable intervention. Measure lightly, prioritize system changes, fund a portfolio with explicit tradeoffs, and verify that human attention was truly returned. The result is not automation for its own sake; it is more capacity for engineering that improves the service over time.