Incident Overview
On June 3, 2025, the UK Crown Prosecution Service (CPS) submitted extradition appeal documents in the cases of Andreea-Maria Tobosaru and Marian Tofan. These filings included citations to two nonexistent legal authorities, which were later identified as likely generated by a large language model (LLM). The erroneous citations appeared in the grounds of opposition and were repeated in a subsequent submission before the High Court of Justice (England and Wales). The court ruled in August 2026 that the false citations did not affect the appeals' outcomes but highlighted the risks of AI-generated legal documents without adequate human review.
The CPS acknowledged the error in a letter to the court, attributing it to insufficient verification by a reviewing lawyer. The incident marks the first known instance in the UK where prosecutors submitted AI-generated legal citations that were later proven fabricated, raising concerns about judicial integrity and epistemic reliability in court proceedings.
The High Court’s judgment, published on July 8, 2026, referenced the incident in footnote 95 of its 19-page ruling, confirming the false citations were detected before the February 2026 hearing. The court’s acceptance of the CPS’s apology did not alter the legal proceedings but underscored the need for rigorous oversight of AI-assisted legal drafting.
The appeals involved two individuals, Andreea-Maria Tobosaru and Marian Tofan, who were subject to extradition requests. The CPS’s use of AI in drafting legal submissions introduced fabricated legal precedents, which, though not decisive in the appeals, exposed vulnerabilities in the adoption of AI tools in high-stakes legal environments.
Possible Root Cause
The primary cause of the incident was the CPS’s reliance on a large language model to generate legal citations without implementing robust verification protocols. The reviewing lawyer failed to detect the fabricated authorities, suggesting a systemic gap in human oversight when AI tools are deployed in legal drafting.
Large language models are known to produce hallucinations—plausible but false outputs—that can mislead users if not cross-checked against authoritative sources. In this case, the AI system likely generated citations that mimicked real legal precedents but did not correspond to any existing case law.
The CPS’s internal review following the incident revealed that the reviewing lawyer did not conduct sufficient due diligence to validate the citations. This oversight indicates a failure in the workflow design, where AI-assisted drafting was integrated without adequate safeguards to prevent the propagation of errors into official court filings.
The use of AI in legal research and document preparation is increasingly common, but this incident demonstrates the critical need for structured validation processes. The CPS’s apology highlighted the lack of verification as the root cause, emphasizing that even advanced AI tools require human scrutiny to maintain legal and judicial standards.
Takeaways
The incident serves as a cautionary example for public authorities and legal professionals adopting AI tools in high-stakes environments. The submission of fabricated legal citations, even unintentionally, erodes trust in judicial processes and undermines the integrity of legal arguments presented in court.
For organizations deploying AI systems in legal or regulatory contexts, the incident reinforces the necessity of implementing multi-layered verification processes.
The High Court’s response, while acknowledging the error did not impact the appeals, sent a clear message about the potential consequences of AI misuse in legal proceedings. Judicial bodies may increasingly scrutinize the reliability of AI-generated submissions, particularly in cases where epistemic integrity is paramount.
The CPS’s apology and subsequent review process reflect a broader industry challenge: balancing efficiency gains from AI tools with the risks of inaccuracies and misinformation. This incident may prompt regulatory bodies to issue guidelines or standards for AI use in legal drafting, ensuring that technological advancements do not compromise the foundational principles of justice and accuracy.
For AI developers, the case highlights the importance of designing systems with built-in safeguards against hallucinations and misinformation. Legal and regulatory applications of AI require transparency in how outputs are generated, along with mechanisms to flag potential errors before they enter official proceedings.
It includes reputational damage to the CPS, potential regulatory penalties for negligence, and broader implications for public trust in AI-driven decision-making processes. The episode underscores the need for proactive risk management in AI adoption across public sector organizations.
Sources
Incident summary from the AI Incident Database, used under CC BY-SA 4.0.