EU Proposes Legislation to Prohibit AI-Generated Nonconsensual Sexual Imagery

European Union Considers Ban on AI Tools for Nonconsensual Sexual Images

Key Developments:

• The European Parliament's civil liberties committee has approved an amendment to the Artificial Intelligence Omnibus law, which would ban AI systems that generate realistic images of sexually explicit activities or intimate parts of an identifiable natural person without consent.
• The ban would not apply if a company has implemented measures restricting the creation of deepfakes.

Background:

Read also: FirstClub Secures $55 Million in Funding from Peak XV, Sofina, and Other Investors 9 Months After $22 Million Series A Round

• The amendment's approval follows outcry after users of Elon Musk's Grok service generated thousands of undressed pictures of women and children in January.
• xAI Corp., parent company of Grok, restricted the feature after widespread criticism.

Regulatory Landscape:

• The proposed ban is part of a growing corpus of EU laws targeting child abuse sexual material and nonconsensual intimate image sharing.
• The 2024 directive on violence against women made it an offense to use AI to produce sexual images without the subject's consent.
• The Digital Services Act punishes social networks that allow the spread of illegal content, including child abuse material.

Implementation and Timeline:

Read also: RBI Policy Preview: A Cautionary Wait Ahead

• If adopted, developers of advanced AI systems will need to prove they have set up restrictions on the images their models create.
• The European Union is working to pass the AI Omnibus, a comprehensive regulation, which aims to simplify the EU's Artificial Intelligence Act.
• Implementation of parts of the AI Act on high-risk applications has been delayed from August 2026 to December 2027 and August 2028.

Investigations and Enforcement:

• Regulators in the EU and the UK are formally investigating X and xAI over the Grok incident in January to establish whether it had breached laws on content moderation and online safety.