Incident Overview
On March 2, 2026, a federal prosecutor in the United States Department of Justice (DOJ) filed a legal brief containing fabricated quotations and misstated holdings from cited cases. Attorney Rudy Renfer submitted a response brief that the court later identified as containing inaccuracies. This marks the highest-profile instance of AI-generated hallucinations within a government legal filing, raising concerns about judicial integrity and epistemic reliability in legal proceedings.
The court demanded an explanation from the DOJ after discovering the fabricated content. Renfer reportedly admitted to using a large language model (LLM) to assist in drafting the brief, following the loss of an earlier draft. The court also found similar fabricated quotations in other filings signed by Renfer, indicating a pattern of AI-induced inaccuracies in legal submissions. The incident prompted a review of the DOJ’s AI-use policy and led to a show-cause hearing for senior leaders in the U.S. Attorney’s Office for the Eastern District of North Carolina.
The misstatements included fabricated quotations and false representations of case holdings, which directly undermined the credibility of the legal arguments presented. The court’s response highlighted the severity of the issue, as such inaccuracies can mislead judges, opposing counsel, and the public about established legal precedents. This incident aligns with broader concerns about AI-generated misinformation in professional and high-stakes environments.
The harm was categorized as impacting judicial integrity, epistemic integrity, and the specific case of Derence V. The incident underscores the risks associated with AI systems in legal workflows, particularly when deployed without adequate oversight or verification mechanisms.
Renfer’s reliance on an LLM to recover lost work led to the inclusion of fabricated content, suggesting that the tool’s outputs were not critically reviewed or cross-verified against original sources.
The DOJ’s response to the incident included a mandatory appearance by senior leaders at a show-cause hearing, where they were required to explain why fabricated quotes were included in court filings.
The incident has broader implications for AI adoption in legal and regulatory environments. Legal professionals must treat AI-generated content as provisional and subject to thorough review, rather than as authoritative or error-free.
The court’s identification of fabricated quotations extended beyond the initial filing, revealing that Renfer had previously submitted other documents containing similar inaccuracies. This pattern suggests that the issue may not be isolated to a single incident but reflects systemic challenges in AI-assisted legal work.
Possible Root Cause
The primary root cause of this incident was the unchecked use of a large language model to draft legal filings without sufficient human oversight or verification. Renfer reportedly turned to AI after losing an earlier draft, which created an environment where accuracy was compromised in favor of expedience. The reliance on AI as a primary or unvetted source for legal citations and quotations introduced systemic risks, as LLMs are known to generate plausible but incorrect information—a phenomenon commonly referred to as ‘hallucination.’
Legal drafting typically requires meticulous attention to detail, particularly when citing case law or statutory holdings. In this instance, the AI-generated content was not subjected to the same level of scrutiny as manually researched material, leading to the inclusion of fabricated quotations and misstated legal precedents. This oversight suggests that the DOJ’s guidelines on AI use may have lacked clear protocols for validating AI outputs in high-stakes legal contexts.
Renfer’s admission that he used AI after losing an earlier draft implies that time constraints may have influenced the decision to prioritize speed over accuracy. However, the stakes in judicial proceedings demand that such shortcuts do not come at the expense of factual integrity. The incident serves as a reminder that AI tools, while efficient, should not replace fundamental legal diligence.
The absence of mandatory review processes for AI-generated legal content allowed the inaccuracies to persist undetected until the court intervened.
Large language models are trained on vast datasets but lack inherent understanding of legal nuance, precedent, or the specific context of a case. When prompted to generate legal citations or quotations, these models may produce outputs that appear authoritative but are factually incorrect. Without human intervention to cross-reference sources or verify claims, the risk of misinformation increases significantly. This incident illustrates how AI’s probabilistic nature can lead to errors that have real-world consequences in legal settings.
The incident suggests that the DOJ’s AI-use policy, while existing, may not have been effectively integrated into daily practices or reinforced through training.
Takeaways
This incident serves as a critical case study in the risks of AI integration in legal and regulatory workflows. The primary takeaway is that AI tools, while powerful, must be used with stringent oversight to prevent the dissemination of inaccurate or fabricated information. Legal professionals must adopt a skeptical approach to AI-generated content, treating it as a draft or suggestion rather than a final product. This principle applies not only to citations and quotations but to all aspects of legal research and drafting where precision is paramount.
Organizations adopting AI in high-stakes environments must implement robust validation protocols to mitigate the risk of hallucinations or misinformation. The incident demonstrates that no amount of efficiency should outweigh the need for factual integrity in legal proceedings.
The case also highlights the importance of clear AI-use policies that are not only documented but actively enforced. Training programs must educate legal professionals on the limitations of AI, such as its tendency to hallucinate or misrepresent information, and emphasize the necessity of human judgment in critical tasks. The DOJ’s response to this incident—including the show-cause hearing—signals a commitment to accountability, which should be a model for other institutions.
Institutions should encourage a culture of openness about AI use to preemptively address potential issues.
The incident also raises questions about the broader implications of AI in legal systems. As AI tools become more prevalent in legal research, drafting, and argumentation, there is a risk that their use could erode public confidence in the judiciary if inaccuracies go unchecked. Courts and legal bodies must remain vigilant in detecting and addressing AI-related errors, while also fostering an environment where practitioners feel empowered to question or verify AI outputs. The DOJ’s internal review and policy adjustments may help mitigate these risks, but the legal community at large must remain proactive in adapting to AI’s role.
For AI developers, this incident underscores the responsibility to design tools that minimize the risk of misinformation in professional settings. The DOJ case serves as a reminder that AI systems must be built with safeguards tailored to the domains in which they are deployed.
Similarly, policymakers must stay ahead of technological advancements to ensure that regulations keep pace with the risks and opportunities AI presents in legal practice.
Sources
Incident summary from the AI Incident Database, used under CC BY-SA 4.0.