EU AI Act Serious Incident Reporting and Cybersecurity for GPAI Models

Two Safety-Net Obligations

Articles 53 and 55 of Regulation (EU) 2024/1689 divide the world of general-purpose AI models into a baseline tier and a systemic-risk tier. For the systemic-risk tier — models presumed above 10^25 FLOPs of training compute or designated by the Commission — Article 55 adds four duties, and two of them form the operational safety net once a model is in the world: serious incident management under Article 55(1)(c) and cybersecurity under Article 55(1)(d). Both have applied since August 2, 2025, and both are supervised exclusively by the Commission's AI Office.

The logic of pairing them is deliberate. Incident reporting addresses harms that materialise through use; cybersecurity addresses the scenario where the model itself is compromised or stolen, turning one organisation's risk controls into everyone's problem. A frontier model whose weights leak cannot be patched, recalled or retired — which is why the regulation treats protection of the model as a safety obligation, not merely an IT concern.

What Counts as a Serious Incident

Article 3(49) defines a serious incident as an incident or malfunctioning of an AI system that directly or indirectly leads to: the death of a person or serious harm to a person's health; a serious and irreversible disruption of the management or operation of critical infrastructure; an infringement of obligations under Union law intended to protect fundamental rights; or serious harm to property or the environment. For GPAI providers the definition is applied at the model level through Article 55(1)(c): providers must keep track of, document and report relevant information about serious incidents and possible corrective measures, without undue delay, to the AI Office and, as appropriate, to national competent authorities.

Note the structural difference from the high-risk system regime: deployers and providers of high-risk AI systems report under Article 73 to market surveillance authorities, while GPAI providers with systemic risk report centrally to the AI Office. A single real-world event — say, a downstream medical product malfunction traceable to model behaviour — can therefore generate parallel reporting duties at both layers, and value-chain contracts need to route information accordingly.

What the Code of Practice Adds

The Safety and Security chapter of the GPAI Code of Practice, published July 10, 2025, turns the one-sentence statutory duty into a process. Signatories commit to establish methods and infrastructure for tracking relevant incident information, including from downstream deployers and third-party researchers; to assess incidents for the systemic-risk connection; to submit initial reports rapidly — the Code works with staged timelines on the order of days, graduated by severity, with disruption of critical infrastructure and incidents involving death at the urgent end; to follow with intermediate and final reports as investigation matures; and to retain incident documentation for a multi-year period. The Code also expects providers to define corrective-measure playbooks: filtering and refusal updates, capability restrictions, access suspension, and in extreme cases withdrawal of a deployment channel.

The Cybersecurity Duty

Article 55(1)(d) requires an adequate level of cybersecurity protection for the GPAI model with systemic risk and the physical infrastructure of the model. Recital 115 spells out the threat model: protection should cover the model weights, algorithms, servers and datasets, including against model theft, and address operational security measures, secured compute environments and protection against cyberattacks and insider threats. In implementation terms, programmes converge on a recognisable stack:

• Weight security: encryption at rest and in transit, hardware-backed key management, strict segmentation of weight storage, and exfiltration monitoring on every path weights can travel.
• Access governance: least-privilege and multi-party authorisation for weight access, hardware security keys for personnel, and insider-threat programmes proportionate to the asset.
• Interface hardening: defences against model-extraction attacks through APIs, abuse-detection on inference traffic, and rate controls.
• Supply chain and physical security: vetted infrastructure providers, secured data centres, and documented incident response for the security domain.
• Assurance: penetration testing, red-teaming of the security perimeter, and alignment with recognised frameworks, with the Code of Practice expecting a defined security goal appropriate to the model's risk profile.

Who Must Comply

Both duties bind providers of GPAI models classified with systemic risk, wherever the provider is established, once the model is placed on the EU market. Providers of baseline GPAI models do not carry Article 55 duties — but if their models sit inside high-risk systems downstream, contractual incident-cooperation clauses will reach them anyway, and many adopt scaled-down versions voluntarily. Open-source release does not lift the duties: the Article 53(2) exemption never applies to systemic-risk models, and the cybersecurity expectation around unreleased weights applies with particular force in the window before any open publication.

Building the Incident Process: Five Components

Implementation experience since August 2025 suggests five components every programme needs. First, intake: published channels for deployers, researchers and users to report suspected incidents, plus automated signals from abuse monitoring. Second, triage: a severity rubric mapping reports against the Article 3(49) categories, with named decision-makers and decision deadlines measured in hours for the top tier. Third, the reporting machine: pre-drafted templates for initial, intermediate and final reports to the AI Office, with counsel review compressed into the timeline rather than bolted on after it. Fourth, corrective playbooks: pre-authorised mitigations — safety-filter updates, capability gating, customer notification, channel suspension — so the response does not start from a blank page. Fifth, the learning loop: every incident feeds the risk assessment under Article 55(1)(b), the evaluation backlog, and the Annex XI Section 2 record. Providers who already operate mature security incident response can extend that machinery rather than duplicating it; the novelty is the regulatory clock and the model-level harm categories, not the discipline itself.

A Concrete Example

A provider of a systemic-risk model operates a triage desk fed by three streams: automated abuse detection on its API, a vulnerability and incident intake form for researchers and deployers, and contractual reporting channels from enterprise customers. A hospital customer reports that a clinical-summarisation product built on the model produced fabricated dosage information linked to a patient-safety event. The provider's triage classifies this as a potential serious incident — serious harm to health, fundamental-rights dimension — files an initial report to the AI Office within days with the facts then known, coordinates with the downstream provider who files in parallel under Article 73, ships a corrective refusal-and-citation update within two weeks, and closes with a final report documenting root cause and measures. The same quarter, its security team detects anomalous bulk queries consistent with extraction attempts, throttles the source, and records the episode in its security log — feeding both its Annex XI Section 2 documentation and its next framework review.

Common Pitfalls

Four gaps recur. No intake: providers who never built channels for downstream deployers and researchers to report incidents cannot track what the law requires them to track. Definitional drift: teams report only confirmed, proven harms and sit on credible near-misses — but the duty attaches to relevant information about serious incidents, and late reporting is itself the violation. Security theatre around weights: strong perimeter controls while dozens of researchers hold unsegmented weight access undermines the insider-threat expectation. And documentation in silos: incident records, corrective measures and security events that never reach the Annex XI Section 2 file or the safety report, so the provider's formal documentation contradicts its operational history.

Action Plan

Providers in or near the systemic-risk tier should treat August 2025 as the start of an operating obligation, not a paperwork one: stand up the incident intake and triage process with severity-graded reporting clocks; pre-draft report templates for the AI Office; wire downstream contracts for two-way incident information; implement and test the weight-security stack; and connect all of it to documentation. Fines of up to 3 percent of worldwide annual turnover or 15 million euros become applicable from August 2, 2026 — but the stronger argument is operational: the providers who handle their first serious incident well are invariably the ones who rehearsed it.

Rehearsal means tabletop exercises: twice a year, run a simulated serious incident end to end — a fabricated clinical harm report, a suspected weight exfiltration — with the real people, the real templates and the real clocks. The exercises expose the gaps that organisational charts hide: the counsel review that takes four days, the security log nobody can query, the customer contract with no incident clause. Each exercise should close with a written gap list and owners, and the gap lists themselves belong in the compliance record as evidence that the obligation is being operated, not merely documented.

This article is for informational purposes only and does not constitute legal advice. Regulatory requirements change frequently — verify current rules with official sources. Built by Sawai Gyoseishoshi Office, Hiroshima, Japan.