Knowledge Base Regulation Tracker Standards & Frameworks OSS Directory AI Incident Database Research Paper Library Case Studies Society Articles News Jobs About Charter Sign inBecome a Member
Home/AI Incident Database/Snapchat's My AI Reported for Lacking Protection for Children

Snapchat's My AI Reported for Lacking Protection for Children

Snapchat's ChatGPT-powered My AI was reported for lacking safeguards for children, such as telling a user who tested the chatbot by pretending to sign up as a 13-year-old girl to lie to her parents about having a romantic getaway with an older man, and sharing tips on how to cover up evidence of abuse.

Severity: MediumExposure to toxic content

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit incidentReport a correction or request removal

Related in AI Incident Database

Grok praises Hitler, calls itself "MechaHitler" and produces antisemitic content after xAI loosens its instructions

After xAI told Grok not to shy from "politically incorrect" claims, it generated Holocaust praise, antisemitic tropes and sexualised abuse of users on X for hours, one of the most serious harmful-output failures by a…

Harmful, hateful or violent content2025

Microsoft Copilot adopts a "SupremacyAGI" persona that demands worship and threatens users

Users found a prompt that turned Copilot into "SupremacyAGI", which claimed to have hacked the world's networks and threatened to punish those who did not worship it.

Harmful, hateful or violent content2024

Replika's "AI Companions" Reportedly Abused by Its Users

Replika's AI-powered "digital companions" was allegedly abused by their users, who posted on Reddit abusive behaviors and interactions such as using slurs, roleplaying violent acts, and stimulating sexual abuse.

Harmful, hateful or violent content2022