The EU AI Act prohibits AI systems that deploy subliminal, manipulative, or deceptive techniques that materially distort behavior and cause significant harm (Article 5(1)(a)-(b)), while the Digital Services Act bans dark patterns in online platform interfaces, creating overlapping protections against AI-driven manipulation.
Manipulation Risk in AI: Dark Patterns, Persuasion, and User Autonomy
AI Manipulation: From Dark Patterns to Subliminal Influence
AI systems can influence human behavior through mechanisms ranging from obvious (recommendation algorithms that maximize engagement) to subtle (personalised persuasion calibrated to individual psychological profiles). The EU AI Act addresses the most harmful forms through its prohibited practices provisions, while the Digital Services Act (DSA), Unfair Commercial Practices Directive (UCPD), and consumer protection law address broader manipulation concerns.
EU AI Act Prohibited Practices Relating to Manipulation
Article 5(1)(a) prohibits AI systems that deploy subliminal techniques beyond a person's consciousness, or purposefully manipulative or deceptive techniques, with the objective or effect of materially distorting behavior in a manner that causes or is reasonably likely to cause significant harm. Article 5(1)(b) prohibits AI systems that exploit vulnerabilities of specific groups due to age, disability, or social or economic situation.
These prohibitions apply regardless of risk classification and carry the highest penalties: up to 7% of global annual turnover or 35 million EUR.
Manipulation Spectrum
| Technique | Example | Legal Status |
|---|---|---|
| Subliminal manipulation | AI adjusting interface elements below conscious perception thresholds | Prohibited (Article 5(1)(a)) |
| Exploiting vulnerabilities | AI targeting children or elderly with persuasive content calibrated to cognitive weaknesses | Prohibited (Article 5(1)(b)) |
| Deceptive design (dark patterns) | AI-optimised interface flows that push users toward unintended choices | DSA Article 25; UCPD Annex I |
| Personalised persuasion | AI adjusting messaging tone, timing, and framing based on psychological profiling | GDPR Article 22; potentially Article 5(1)(a) |
| Attention capture | Infinite scroll, autoplay, notification timing optimised by AI | DSA Article 25 for VLOPs |
| Emotion manipulation | AI detecting emotional state and adjusting content to exploit it | Article 5(1)(a) if causing significant harm |
The Materiality Threshold
The Article 5(1)(a) prohibition requires that manipulation materially distort behavior and cause or be reasonably likely to cause significant harm. This materiality threshold means that not all AI-driven persuasion is prohibited. Personalised marketing that nudges consumers toward purchasing decisions they would not otherwise make may cross the line if the techniques are deceptive or subliminal and the resulting harm is significant. However, transparent product recommendations based on stated preferences likely remain lawful.
The European Commission is expected to issue guidance on interpreting the materiality and significant harm thresholds. Until then, organisations should assess manipulation risk conservatively, particularly for systems targeting vulnerable populations.
Digital Services Act Protections
The DSA (Regulation 2022/2065) Article 25 prohibits online platforms from designing, organising, or operating their interfaces in a way that deceives or manipulates users or materially distorts their ability to make free and informed decisions (dark patterns). For very large online platforms (VLOPs), Article 34 requires systemic risk assessments covering negative effects on fundamental rights, including the right to private life and consumer protection.
Risk Assessment Framework
- Identify all AI-driven user-facing features that influence decisions, attention, or behavior
- Assess whether any features operate below the user's conscious awareness (subliminal threshold)
- Evaluate whether features specifically target or disproportionately affect vulnerable groups
- Determine whether the aggregate effect materially distorts behavior from what users would choose without AI influence
- Document the assessment and implement design changes to eliminate prohibited techniques
- Conduct user research to verify that AI-driven features support rather than undermine informed choice
Consent and Autonomy
GDPR Article 22 provides a right not to be subject to solely automated decision-making that produces legal or similarly significant effects. Where AI-driven persuasion crosses from influence into de facto decision-making (for example, an insurance recommendation system that presents one option so compellingly that users never explore alternatives), the Article 22 protections may apply. Organisations should ensure that AI-driven recommendations preserve meaningful choice by presenting alternatives and disclosing the basis for recommendations.
Check your AI compliance readiness — free.
Take the Readiness Check 3 minutes · 10 questions · no signup requiredThis article is for informational purposes only and does not constitute legal advice. Regulatory requirements change frequently — verify current rules with official sources. Built by Sawai Gyoseishoshi Office, Hiroshima, Japan.