Systemic risk in AI arises when widespread adoption of similar AI models creates correlated failure modes, herding behavior in financial markets, or cascading disruptions across interconnected critical infrastructure, addressed by the EU AI Act's GPAI systemic risk provisions (Articles 51-55) and financial stability frameworks.
Systemic Risk in AI Systems: Cascading Failures and Financial Stability
What Makes AI a Source of Systemic Risk
Systemic risk in AI differs from individual system failures. It arises from the interaction effects of widespread AI adoption: when many institutions use similar models, train on overlapping data, or depend on common infrastructure, individual risks become correlated. A flaw in a widely-used model, a data quality issue in a common training dataset, or a vulnerability in shared infrastructure can simultaneously affect multiple sectors, creating cascading failures that no single institution can control.
Systemic Risk Channels
| Channel | Mechanism | Example |
|---|---|---|
| Model herding | Similar models produce correlated decisions, amplifying market movements | Multiple AI trading systems selling simultaneously, triggering a flash crash |
| Data monoculture | Models trained on similar data develop similar blind spots | Credit scoring models all failing to predict the same type of default |
| Infrastructure dependency | Common cloud/model provider creates single point of failure | Foundation model API outage disabling thousands of downstream services |
| Feedback loops | AI outputs become inputs to other AI systems, amplifying errors | Algorithmic news generation triggering algorithmic trading triggering more news |
| Opacity cascades | Complex AI-to-AI interactions become impossible to audit | Supply chain AI systems producing unexplainable allocation decisions |
EU AI Act GPAI Systemic Risk Provisions
The EU AI Act creates a specific regime for GPAI models posing systemic risk (Articles 51-55). A GPAI model is presumed to have systemic risk if trained using total computing power measured in FLOPs greater than 10^25, or if designated by the Commission based on criteria including the number of users, degree of market integration, or its capacity for autonomous action.
Providers of GPAI models with systemic risk must conduct model evaluations including adversarial testing, assess and mitigate systemic risks, track and report serious incidents to the AI Office, and ensure adequate cybersecurity protection. These obligations are enforced by the European AI Office, which has direct enforcement powers for GPAI provisions.
Financial Stability Concerns
The Financial Stability Board (FSB) identified AI in finance as a potential source of systemic risk in its 2023 report. Key concerns include procyclicality (AI systems amplifying market cycles), herding (correlated trading strategies), and third-party dependency (concentration in AI model and data providers). The European Systemic Risk Board (ESRB) is monitoring AI-related systemic risks through its macroprudential lens.
DORA (Regulation 2022/2554) addresses systemic risk from ICT third-party providers. AI model providers may be designated as critical ICT third-party service providers under Article 31, enabling direct oversight by the European Supervisory Authorities. This mechanism could be used to impose systemic risk controls on AI providers serving the financial sector.
Critical Infrastructure Cascades
AI systems managing energy grids, water supply, transport networks, and telecommunications infrastructure (classified as high-risk under Annex III, point 2) can create cascading failures if they share common failure modes. An AI-driven optimization algorithm that performs well under normal conditions may behave unpredictably during extreme events, precisely when reliable infrastructure is most critical.
The NIS2 Directive requires essential entities in critical infrastructure sectors to assess and manage cybersecurity risks, including risks from AI components. Combining NIS2 obligations with EU AI Act high-risk requirements provides a dual framework for managing systemic risk in critical infrastructure AI.
Mitigation Approaches
- Diversify AI model sources to reduce correlation in decision-making
- Implement circuit breakers that automatically limit AI system actions during detected anomalies
- Conduct cross-institutional stress tests simulating common AI failure scenarios
- Maintain independent validation datasets to detect shared blind spots across AI models
- Participate in information-sharing initiatives on AI incidents and near-misses
- Design AI systems with graceful degradation rather than abrupt failure modes
Macroprudential AI Governance
Managing systemic AI risk requires coordination beyond individual organisations. Regulators are developing macroprudential tools for AI, including AI incident reporting aggregation (EU AI Act Article 62 feeds into the AI Office's monitoring capabilities), designation powers for systemically important AI providers, cross-sector stress testing frameworks, and international coordination through the FSB, IOSCO, and BCBS. Organisations should engage constructively with these developing frameworks while implementing internal systemic risk controls.
Check your AI compliance readiness — free.
Take the Readiness Check 3 minutes · 10 questions · no signup requiredThis article is for informational purposes only and does not constitute legal advice. Regulatory requirements change frequently — verify current rules with official sources. Built by Sawai Gyoseishoshi Office, Hiroshima, Japan.