Incident Overview
On May 6, 2026, a legal filing in litigation involving Trump administration layoffs contained fabricated legal quotations attributed to Claude, Anthropic’s large language model. The filing was part of a motion to quash a subpoena in American Federation of Government Employees, AFL-CIO v. Trump, a case tied to layoffs under the Trump administration. The quotations did not exist in the cited legal sources, marking another instance of AI-generated hallucinations in judicial proceedings.
The attorney later acknowledged the error to U.S. District Judge Susan Illston in San Francisco, apologizing for the inclusion of these phantom quotations. The firm, Binnall Law Group, confirmed the use of AI in drafting the filing and stated it would implement additional safeguards and training to prevent future occurrences.
This case follows a similar incident in 2025, where Anthropic’s own systems were implicated in generating inaccurate legal citations. The repeated occurrence underscores ongoing challenges in ensuring the reliability of AI-generated legal content in high-stakes environments such as court filings.
The litigation itself involves a dispute over layoffs conducted during the Trump administration, with the American Federation of Government Employees (AFL-CIO) challenging the legality of the dismissals.
The court’s response to the incident remains documented but not publicly detailed beyond the acknowledgment of the error. The filing’s apology, however, indicates that the judge was made aware of the AI-generated inaccuracies, reinforcing the need for transparency in AI-assisted legal work.
Possible Root Cause
The primary root cause of this incident appears to be the inherent limitations of large language models (LLMs) in generating factually accurate legal citations. Claude, like other advanced AI systems, is prone to producing hallucinations—outputs that appear plausible but are entirely fabricated. In this case, the model generated quotations that mimicked legal authority but had no basis in actual case law or judicial precedent.
While AI tools can assist in legal research and drafting, their outputs must be rigorously verified by human experts, particularly in contexts where precision is critical, such as court filings.
Anthropic’s own 2025 incident involving citation errors suggests that this is not an isolated issue but a recurring challenge for AI systems deployed in legal settings.
The attorney’s reliance on the AI-generated text without sufficient human review highlights a systemic risk: the assumption that AI outputs are accurate without verification.
The incident also reflects broader concerns about the integration of AI in professional fields where trust and accuracy are paramount. Legal professionals must balance efficiency with the need for meticulous fact-checking, especially when AI tools are involved in high-stakes processes like litigation.
Takeaways
- Judicial Integrity at Risk: The inclusion of fabricated quotations in a court filing directly undermines the integrity of judicial proceedings. Courts rely on accurate legal citations to make informed decisions, and AI-generated errors can introduce misleading information into the record, potentially influencing outcomes.
- While AI can streamline drafting and research, legal professionals must verify all AI-generated content against primary sources to ensure accuracy.
- Institutional Accountability: Binnall Law Group’s response—apologizing to the judge and committing to additional safeguards—demonstrates the importance of accountability in AI adoption. Law firms and organizations using AI tools must establish protocols for detecting and mitigating errors, including training for staff on the limitations of AI systems.
- Broader Implications for AI in Legal Settings: The repeated occurrence of such incidents raises questions about the readiness of AI tools for widespread use in legal practice.
- Technical and Ethical Challenges: The incident highlights the ethical dilemmas surrounding AI in high-stakes fields. Legal professionals, in turn, must stay informed about the capabilities and limitations of AI systems they deploy.
- Precedent for Future Cases: This case may serve as a precedent for how courts handle AI-generated errors in legal filings.
Sources
Incident summary from the AI Incident Database, used under CC BY-SA 4.0.