For three years, prompt injection has lived in the security-engineering bucket. As of 2026, it’s also lived in the compliance bucket — driven by a combination of EU AI Act provisions, sector-specific regulations, and emerging case law on data protection. This post maps where the regulatory liability actually attaches when prompt injection succeeds, and what deployers (Article 26 in EU AI Act terms) should be documenting. It assumes the attack side is settled; the working taxonomy of prompt injection attacks is the reference for which classes a risk assessment has to name.
This is not legal advice. Validate against counsel before relying on it.
The regulatory levers
The Act does not name prompt injection anywhere. It assigns duties by role, and three roles matter here: providers of high-risk AI systems, providers of general-purpose AI (GPAI) models, and deployers.
EU AI Act Article 16 (providers of high-risk AI systems)
Providers of high-risk AI systems carry the Article 16 obligations, which include making sure the system meets the Act’s high-risk requirements: a risk management system, and appropriate accuracy, robustness and cybersecurity. A known, well-documented attack class such as prompt injection is hard to leave out of either. Breaching provider obligations exposes a high-risk system provider to administrative fines under Article 99.
EU AI Act Articles 53 and 55 (general-purpose AI model providers)
Providers of GPAI models, the foundation models most deployers build on, sit under a separate chapter. Article 53 requires technical documentation for the AI Office and information for the downstream providers who integrate the model. Article 55 adds duties for GPAI models with systemic risk, including model evaluation with documented adversarial testing, tracking and reporting serious incidents, and adequate cybersecurity protection. Fines for GPAI model providers are set by Article 101, not Article 99.
For prompt injection, that maps onto:
- Pre-deployment adversarial testing that includes injection classes
- Documented mitigations (input filtering, output validation, capability scoping)
- Incident tracking when injection succeeds in production
- Disclosure of known limitations in the technical documentation (Article 53)
EU AI Act Article 26 (deployer obligations)
Deployers (the parties using an AI system under their own authority, as opposed to the parties who build it) have separate obligations. Most relevantly:
- Article 26: deployers of high-risk AI systems must, where they control the input data, ensure it is relevant and sufficiently representative, and must monitor system operation. Prompt injection that causes the system to produce harmful output to users implicates both.
- Article 27: some deployers of high-risk systems (public bodies, private entities providing public services, and credit-scoring and life and health insurance pricing uses) must carry out a fundamental rights impact assessment before first use. A prompt injection that can alter outputs affecting people belongs in that assessment.
Article 25 matters only at the edges: a deployer that puts its own name or trademark on a high-risk system, makes a substantial modification to it, or changes its intended purpose so that it becomes high-risk is treated as a provider and inherits the Article 16 obligations.
Deployers cannot transfer their obligations to the model provider. Even if the model provider’s defenses are inadequate, the deployer is responsible for the deployed system.
GDPR Article 5 (data minimisation, integrity, confidentiality)
When prompt injection causes a model to leak training data or session context that includes personal data, this is a personal data breach, and Article 5’s integrity and confidentiality principle is the one at stake. EDPB Opinion 28/2024 (December 2024) does not address prompt injection directly. It covers when an AI model trained on personal data can be considered anonymous, whether legitimate interest can serve as a legal basis for developing and deploying models, and what follows when a model was developed with unlawfully processed personal data. Its anonymity test is the relevant part here: a model only counts as anonymous if the likelihood of extracting personal data from it, including by querying it, is insignificant, which brings prompt-driven extraction into scope for anyone claiming a model holds no personal data.
Result: prompt injection that leaks PII triggers the breach notification obligation (Articles 33-34) — within 72 hours to the supervisory authority, and to data subjects without undue delay if the breach is high-risk.
Sector-specific implications
- Health (HIPAA, MDR): a prompt injection in a clinical decision-support AI that causes incorrect dosing recommendations is both a security incident AND a medical device adverse event. Different reporting timelines apply concurrently.
- Finance (NYDFS Part 500, SR 11-7 in the US): prompt injection enabling unauthorized transactions is a material cybersecurity event. The institution’s AI model risk management framework must address this class.
- Public sector (national security and critical infrastructure): NIS2 in the EU and CIRCIA in the US apply when prompt injection compromises systems classified as essential or important entities.
Where liability actually attaches
Provider liability
Provider liability under the EU AI Act centers on whether the provider:
- Knew or should have known about the prompt injection class
- Implemented documented mitigations proportionate to the risk
- Disclosed the limitation in the model’s technical documentation
- Updated the documentation when new injection classes emerged
A provider that ships with no injection mitigations and no documentation faces administrative fines of up to 3% of worldwide annual turnover: under Article 99 for a high-risk AI system provider, and under Article 101 for a GPAI model provider. A provider that mitigates and documents reduces the liability surface even when injection succeeds in deployment.
Deployer liability
Deployer liability under the AI Act is heavier than most teams have internalized:
- Article 26: ensure system is monitored in operation. A deployment without prompt injection telemetry isn’t monitored.
- Article 27: where it applies, a fundamental rights impact assessment before first use. Prompt injection scenarios belong in it.
- GDPR breach reporting: 72-hour clock starts when the deployer became aware of the breach. Detection capabilities matter.
Deployers cannot rely on the provider’s representations alone. The Act assigns obligations by role: Article 16 obligations sit with providers of high-risk systems, Articles 53 and 55 with GPAI model providers, Article 26 obligations with deployers, and they don’t transfer by contract.
What deployers should document
A practical compliance file for a deployer using a foundation model in an AI system:
- Risk assessment identifying prompt injection as a relevant threat for the deployment
- Mitigation inventory: what input filters, output validators, capability scopes, and monitoring are in place, mapped against the control stack in the layered mitigation guide
- Eval evidence: results of running known injection corpora against the system, with dates and configurations
- Detection coverage: documentation of which injection classes are detectable in production telemetry
- Incident response runbook: who is notified, on what timeline, when injection is detected
- Update log: changes to the above when new injection classes emerge or new mitigations are added
- Provider-side dependencies: documentation of which mitigations the provider claims, with citations to provider documentation
This file is what’s produced for an audit. Without it, “we have AI security” is not legally meaningful.
What providers should document
For GPAI model providers, the technical documentation Article 53 requires (detailed in Annexes XI and XII) is the natural place for:
- Known injection classes the model has been evaluated against (with dates)
- Test methodology and results (sufficiently general to allow downstream evaluation)
- Recommended downstream mitigations
- Capabilities and limitations relevant to injection resistance
- Update channel for new findings
Litigation outlook
Class-action plaintiffs’ bars are increasingly aware of AI security failures. Pattern of cases through 2026:
- PII leakage cases — most active class, modest individual damages, large class sizes
- Discriminatory output cases — when injection causes the system to produce protected-class-related output
- Liability transfer cases — testing whether a deployer can sue a provider for breach of representation
The third category is particularly important to track. Courts are not yet aligned on whether provider representations create enforceable duties to deployers; expect the answer to vary by jurisdiction.
Practitioner checklist
For deployer security and compliance leads:
- Have you identified which Article (26/27) obligations apply to your deployment?
- Is prompt injection cited in your AI risk assessment?
- Do you have documented mitigations for at least the OWASP LLM01 categories?
- Do you have telemetry that would detect prompt injection in production?
- Do you have an incident response runbook that handles AI-mediated breaches with the right timelines?
- Have you reviewed the provider’s technical documentation under Article 53?
- Is there a process to update your assessment when new injection classes emerge?
The documentation work is mechanical. The risk it mitigates is material. The two things together are what compliance is.
What to watch next
- The first enforcement actions under the Act, and which articles regulators cite when an AI system fails under adversarial input
- How the Commission applies its Article 101 fining powers to GPAI model providers, including the adversarial-testing duty in Article 55
- The training data summary template (Article 53) and how it intersects with injection-resistance disclosures
- Member-state-level supplementary regulations, particularly in Germany and France
- US state-level AI laws taking effect (Colorado in 2026; California CPPA’s AI rules pending)
The window for “we’ll address this when it becomes a problem” closed at the end of 2025. In 2026, addressing it is the baseline.
See also
- How to mitigate prompt injection: a layered defense guide — the control stack a deployer is expected to have documented
- Prompt injection testing: a repeatable method — producing the evidence trail that turns “we considered it” into a record
- OWASP LLM01:2025 prompt injection explained — the voluntary standard regulators and auditors reference
- Indirect prompt injection: how these attacks work — the attack class where the harmed party is your own user
- adversarial ML research
- AI attack techniques
Related across the network
- GDPR Article 22 and LLM Automated Decision-Making — aiprivacy.report
- DPIA Template for LLM Deployment: A Working Structure — aiprivacy.report