ChatGPT Fabricates Murder Conviction for Norwegian User, Blending Real and False Data
In March 2025, Norwegian citizen Arve Hjalmar Holmen discovered that ChatGPT had generated a fabricated narrative claiming he was convicted of murdering two of his children and attempting to kill a third. The AI’s response included verifiable personal details—such as the number and gender of his children, his hometown, and a fabricated 21-year prison sentence—intertwined with false claims. This incident highlights how large language models (LLMs) can produce defamatory outputs by combining real personal data with entirely fabricated information, raising serious concerns about accuracy, accountability, and the application of GDPR to AI-generated content.
The fabricated story was not an isolated error. ChatGPT has previously generated false accusations of corruption, child abuse, and other serious crimes against individuals, often without providing a mechanism for correction. These so-called hallucinations occur because LLMs predict text based on statistical patterns rather than factual verification, leading to confident yet entirely baseless claims. The inclusion of real personal data in the fabricated narrative exacerbates the harm, as it makes the false information appear more plausible and harder to disprove.
He stated that the fear of others believing the AI’s output—despite its falsity—was the most distressing aspect. The blend of accurate and fabricated details creates a scenario where individuals may struggle to disprove the claims, particularly if the AI’s response is treated as authoritative or shared widely without scrutiny.
OpenAI acknowledged the issue and noted that updates to ChatGPT’s web-search functionality reduced the likelihood of such errors for Holmen. However, the company emphasized that correcting the underlying model’s internal data is not feasible, as the AI’s training relies on vast, uncurated datasets. This limitation raises questions about whether AI providers can fulfill GDPR’s accuracy obligations when the source of inaccuracies lies within the opaque, dynamic nature of LLMs.
GDPR Compliance and the Limits of AI Disclaimers
The incident prompted noyb, a European digital rights organization, to file a complaint with the Norwegian Data Protection Authority (Datatilsynet) under Article 5(1)(d) of GDPR, which mandates that personal data processed by organizations must be accurate. noyb argued that OpenAI’s reliance on a disclaimer—stating that ChatGPT ‘may produce false results’—does not absolve the company of its legal obligations to ensure data accuracy. The organization contended that such disclaimers are insufficient to mitigate the reputational and legal harm caused by defamatory AI outputs.
Joakim Söderberg, a data protection lawyer at noyb, clarified that GDPR does not permit organizations to bypass accuracy requirements by appending disclaimers. The law requires that personal data—whether processed internally or shared—must reflect reality. In this case, ChatGPT’s output violated that principle by presenting Holmen as a convicted murderer, a claim with no basis in fact. noyb’s complaint sought both the correction of the defamatory output and fines to deter future violations, emphasizing that AI providers cannot evade accountability by hiding behind technical limitations.
OpenAI’s response to previous complaints further illustrates the challenges of enforcing GDPR in the context of AI. In April 2024, noyb filed a similar complaint regarding ChatGPT’s incorrect date of birth for a public figure. OpenAI’s solution was to ‘block’ the incorrect data from appearing in responses to specific prompts, but the false information remained embedded in the model’s training data. This approach does not address the root issue: the persistence of inaccuracies within the AI’s internal systems. Even if ChatGPT no longer generates the false narrative for Holmen, the data may still influence future outputs or be used for training, leaving users without a guarantee of complete erasure.
Kleanthi Sardeli, another data protection lawyer at noyb, highlighted that AI companies cannot circumvent GDPR by processing false data internally while presenting disclaimers to users. The organization’s stance is that AI providers must actively prevent hallucinations from affecting personal data, rather than relying on users to verify the accuracy of AI-generated information. The lack of transparency around how data is processed and stored—particularly the default practice of feeding user interactions back into training datasets—further complicates efforts to ensure compliance.
Technical and Legal Challenges in Correcting AI Hallucinations
The technical constraints of LLMs present significant hurdles to achieving data accuracy. However, this does not eliminate the risk of hallucinations, especially when the AI relies on its internal knowledge base rather than live sources. For Holmen, the update meant ChatGPT no longer produced the murder conviction claim, but it did not address the underlying issue: the model’s inability to distinguish between accurate and fabricated data in its training corpus.
The persistence of false data within AI systems raises broader questions about the feasibility of GDPR’s accuracy requirements in the context of machine learning. Unlike traditional databases, where records can be directly updated or deleted, LLMs are trained on vast, evolving datasets that are not structured for individual corrections. OpenAI’s refusal to comply with GDPR’s right of access (Article 15), which allows individuals to verify what data is processed about them, exacerbates this problem. Without visibility into how personal data is used or stored, users cannot confirm whether inaccuracies have been fully addressed or if they continue to influence the model’s outputs.
Additionally, the default practice of using user interactions to retrain AI models introduces a feedback loop where false information could be perpetuated. Even if ChatGPT no longer generates a specific hallucination, the interaction—including the fabricated claim—may be retained in the training data, potentially resurfacing in future responses. This creates a scenario where individuals cannot be certain that defamatory outputs have been permanently erased, as the model’s training process remains opaque and uncontrollable by the affected parties.
noyb’s complaint to the Norwegian Datatilsynet seeks to hold OpenAI accountable for these failures. The organization is requesting that the authority order OpenAI to delete the defamatory output from its systems and fine-tune the model to eliminate inaccurate results. The complaint also calls for administrative fines to discourage similar violations in the future. This case sets a precedent for how GDPR may be applied to AI-generated personal data, particularly when inaccuracies result in reputational harm or legal consequences for individuals.
For now, the Norwegian Datatilsynet’s decision on this complaint will be closely watched, as it may shape how GDPR is interpreted in the context of AI and the responsibilities of companies developing such technologies.