AI incident reporting is the controlled path from an unusual model outcome to decisions about containment, affected people, customers, authorities, and learning. It cannot simply inherit the cyber process. An AI event may arise without an intrusion: a ranking model can systematically disadvantage a group, a medical assistant can generate unsafe guidance, or a general-purpose model can enable misuse at scale. Conversely, prompt injection and model theft may require one coordinated AI and security response rather than two competing command structures.
Build the process before production, while teams can still agree definitions and evidence. Connect it to Edilec's web incident-response operating guide for command discipline, the model-monitoring guide for detection, and AI workflow escalation rules for durable human decisions.
Define the AI incident boundary and reporting objective
Define an AI incident as an event or pattern involving an AI system that causes, contributes to, or creates a credible risk of material harm, rights impact, security compromise, prohibited behavior, or loss of control. Include near misses when a safeguard prevents impact but reveals a reusable failure mode. Do not make the definition depend on proven model causality; early triage works with credible hypotheses. Separate the operational incident record from the narrower legal conclusion that a statute calls an event reportable.
The EU AI Act creates role- and system-specific obligations, while the Commission's GPAI guidance notes serious-incident duties for providers of GPAI models with systemic risk. Other regimes may apply to privacy, products, employment, finance, health, or cybersecurity. Maintain a jurisdiction-and-role matrix, but let counsel make reportability determinations from preserved facts rather than asking operators to memorize law during an event.
| Event family | Example signal | Immediate lead | First containment question |
|---|---|---|---|
| Safety or physical harm | Unsafe instruction linked to real-world action | Safety owner and incident commander | Can the affected function be disabled? |
| Rights or discrimination | Outcome disparity or credible complaint | Responsible AI and legal | Are similar decisions still occurring? |
| Security or privacy | Injection, exfiltration, model theft | Security incident commander | Which access path and credentials remain live? |
| Integrity or misinformation | High-impact false output pattern | Product and domain owner | Can users identify and correct affected records? |
| Systemic model risk | Capability or misuse event at scale | Model risk executive | Which deployments and downstream parties are exposed? |
Use consequence, reach, reversibility, and control loss for severity
Severity should not depend only on current confirmed loss. Score plausible consequence, number and vulnerability of affected people, geographic and product reach, duration, reversibility, ongoing exposure, and whether controls remain effective. Add confidence as a separate field; a low-confidence catastrophic signal deserves urgent investigation, not a low severity. Define four levels with concrete examples and authority: monitor locally, activate a cross-functional team, appoint an executive incident commander, or invoke crisis governance.
Triage should identify system and model versions, use case, affected population, first and last observed time, reporter, evidence location, linked cyber or privacy case, and current business effect. Assign one incident ID across teams. The NIST GenAI Profile treats generative-AI risks as lifecycle concerns spanning governance, content, information integrity, security, and human interaction; that breadth is a useful check against routing every report to model engineers.
Preserve model, context, configuration, and decision evidence
Capture the exact user input where lawful, system and developer instructions, retrieved passages, tool schemas and results, model and provider version, sampling settings, safety-policy version, identity and permissions, timestamps, and rendered output. Preserve linked application records and the human decision that followed. Hash or otherwise protect high-value artifacts, control access, and record collection gaps. A screenshot alone cannot establish what hidden context or tool result shaped the response.
Plan for privacy and privilege. Collect the minimum necessary data into a restricted case workspace, mark legal requests, and avoid copying sensitive prompts into broad chat channels. Store raw evidence separately from the working chronology so redaction does not destroy originals. The public AI Incident Database illustrates why structured descriptions and taxonomies support learning across events, but internal reports also need confidential deployment identifiers, customer impact, control status, and decision records.
Coordinate containment across AI, cyber, product, and legal teams
Choose containment at the narrowest effective layer: block a tool, revoke a credential, remove a poisoned source, quarantine a tenant, roll back a prompt, pin a model version, increase review, or suspend the feature. Record who authorized the action and which risks it introduces. A blanket shutdown can create safety or service harm; a prompt-only patch can leave the underlying authority exposed. Test that containment changed the failure path and continue monitoring for variants.
Use one commander with named leads for model analysis, security, product operations, communications, legal and affected-domain expertise. Establish update times and decision logs. Legal should assess notification duties; communications should prepare accurate audience-specific language; engineering should preserve evidence before destructive remediation. Vendors need explicit cooperation channels and contract clocks. If a provider controls model telemetry or downstream contacts, escalation instructions must exist outside the affected service.
Run reporting clocks from facts, not from a finished root cause
Maintain separate clocks for internal escalation, customer commitments, contractual notices, regulators, insurers, and affected individuals. Start each from its defined trigger and show deadline, decision owner, counsel, status, and evidence. The first report can be preliminary: known facts, uncertainty, affected versions, containment, and next update. Never delay a time-bound notice merely to produce a polished causal narrative. Also avoid speculative attribution that later becomes difficult to correct.
| Reporting artifact | Decision owner | Required contents | Quality gate |
|---|---|---|---|
| Initial alert | On-call governance lead | Signal, system, plausible impact, preservation action | Enough detail to assign severity |
| Situation report | Incident commander | Timeline, scope, hypotheses, controls and decisions | Facts separated from inference |
| Reportability memo | Legal counsel | Applicable role, trigger, deadline and rationale | Jurisdiction and privilege confirmed |
| External notice | Authorized legal or communications lead | Required facts, mitigation, contact and updates | Consistent with current evidence |
| Closure report | Risk owner | Cause, impact, corrective actions and residual risk | Independent challenge for serious events |
Close only after corrective actions change the system
Root-cause analysis should examine the sociotechnical chain: data, model, prompt, retrieval, tools, interface, staffing, incentives, monitoring, and governance. Distinguish initiating condition, failed barriers, impact multipliers, and detection gaps. Translate findings into owned corrective actions with deadlines and verification evidence. Add representative cases to evaluations and tabletop scenarios, update severity examples, and share sanitized lessons with dependent teams. Closure means residual risk is accepted by the right owner, not merely that production resumed.
Exercise the process twice a year and after material architecture change. Use scenarios that force cross-domain choices: a hidden instruction causes data access; a model update changes decisions without a security alert; a harmful output spreads through an integration; or a vendor withholds needed logs. Measure time to triage, evidence completeness, decision latency, containment effectiveness, notice readiness, and corrective-action aging. These are readiness measures, not targets to game by prematurely downgrading incidents.
Make incident intake accessible without creating a data leak
Provide several intake routes: an authenticated employee form, product escalation, security operations, customer support, vendor notification, and a protected channel for affected people or whistleblowers. Ask reporters for observed behavior, system, time, potential impact, and where evidence lives; do not require them to classify law or prove causality. Train triage staff to preserve records and avoid pasting sensitive model context into ordinary tickets. Acknowledge reports, protect reporters from retaliation, and communicate status where appropriate. Measure abandoned forms and misrouted cases, because a theoretically complete process fails when users cannot recognize an AI event or fear that reporting a near miss will be treated as personal fault.
Publish a compact responder card with severity examples, commander contacts, evidence fields, legal escalation, and vendor emergency routes. Keep an offline copy in case the AI platform, identity provider, or collaboration system is unavailable. Local teams should know which immediate containment actions they may take without waiting and which require executive authority. Translate user-facing reporting instructions where the deployment population needs it. During exercises, include a credible report that turns out not to be an AI failure; the process should close it respectfully, preserve the rationale, and route any underlying product or conduct issue rather than discouraging future reports.
Maintain a decision directory by deployment and jurisdiction so responders can locate the business owner, model provider, privacy lead, safety specialist, counsel, communications approver, insurer contact, and regulator portal without improvisation. Verify the directory quarterly and whenever ownership changes. The record should state roles and secure channels, not embed reusable passwords or tokens. During a live event, log unsuccessful contact attempts and invoke a named alternate; a reporting clock cannot depend on one unavailable expert.
Key takeaways for AI incident reporting
- Define a broad operational incident boundary, then make legal reportability a separate evidence-based decision.
- Severity combines consequence, reach, reversibility, ongoing exposure, and control loss; confidence is not severity.
- Preserve the complete inference and action context, not only the visible output.
- Use one incident identity and commander across AI, cyber, legal, product, and communications.
- Track every reporting clock independently and verify corrective actions before closure.
FAQ about AI incident reporting
Is every hallucination an incident? No. Treat routine low-impact errors through quality management, but escalate patterns, high-consequence outputs, control failures, or credible harm. Should the AI team command a prompt-injection event? Not automatically; use the established security commander with AI expertise when compromise dominates. Can a vendor's incident report replace the customer's investigation? No. It is evidence, while the customer must assess its own use, affected people, contracts, and legal role.
Conclusion: make fast reporting compatible with careful truth
A mature reporting process does two things at once: it moves quickly enough to stop continuing harm and remains disciplined enough to avoid turning uncertainty into fact. Clear definitions, consequence-based severity, complete evidence, one command structure, and explicit clocks give teams that balance. Design those mechanisms before the serious event, test them against uncomfortable scenarios, and keep the final report connected to evaluations, contracts, monitoring, and governance so the same failure does not return under a new model name.