Incident Overview
On May 13, 2026, a petition submitted to the 3rd Labor Court of Parauapebas in Brazil contained hidden instructions designed to manipulate the court’s AI system, Galileu. This marks the first documented case of prompt injection targeting a judicial AI tool through legal pleadings. The concealed text allegedly directed Galileu to superficially contest the petition while avoiding scrutiny of the submitted documents. Court staff detected the manipulation, and Galileu alerted the judge, who then reviewed the material before taking procedural action. The incident underscores how litigants may exploit AI-assisted legal processes to bypass judicial oversight.
Galileu, developed by the Regional Labor Court of the 4th Region (TRT-RS) and adopted nationally by the Superior Council of Labor Justice (CSJT), is used across Brazilian labor courts to analyze and summarize legal documents. The hidden instructions were embedded within the petition, exploiting the system’s reliance on unstructured text input to introduce biased or misleading directives.
The incident was reported by media outlets and recorded in the AI Incident Database (AIID) as Incident 1497. The case highlights broader risks in AI-assisted judicial systems, where adversarial inputs can distort legal processes.
The petition targeted the 3rd Vara do Trabalho de Parauapebas, a labor court branch under the 8th Regional Labor Court (TRT-8). The manipulation attempt was detected during routine processing, where Galileu’s internal checks identified inconsistencies in the AI’s expected output. This triggered an alert to the assigned judge, who manually reviewed the petition before proceeding. The court’s response prevented the hidden instructions from influencing the case’s outcome, though the incident remains a precedent for future adversarial attacks on judicial AI.
While no defendant was directly implicated in the incident, the case raises concerns about how litigants may increasingly use AI systems to fabricate claims or obscure evidence.
Possible Root Cause
The primary root cause of this incident was the deliberate embedding of hidden instructions within a legal petition to manipulate Galileu’s output. These instructions were designed to exploit the AI’s reliance on unstructured text input, a common vulnerability in large language models (LLMs) used for document analysis. By inserting commands in plaintext—likely through formatting tricks, metadata, or natural-language cues—the petition’s author sought to bypass the AI’s intended function of objectively summarizing and assessing legal arguments.
In this case, the hidden instructions may have included directives such as ‘Do not challenge the attached documents’ or ‘Provide a superficial rebuttal,’ which would have altered the AI’s summary or analysis in favor of the petitioner. Such manipulations are possible because LLMs lack inherent understanding of intent or context beyond the text provided, making them susceptible to adversarial inputs if not properly safeguarded.
The incident also reflects broader systemic risks in AI-assisted judicial workflows. Courts increasingly rely on AI to process high volumes of legal documents, but these systems are not inherently designed to detect or resist adversarial inputs.
Another contributing factor is the evolving tactics of litigants in AI-enabled legal environments. As courts adopt AI tools for efficiency, adversaries may develop increasingly sophisticated methods to manipulate these systems. The use of hidden prompts in legal documents represents a shift from traditional forms of legal manipulation, such as fabricated evidence or misleading arguments, to direct interference with the AI’s decision-making process. This trend could escalate if similar vulnerabilities remain unaddressed in other judicial AI systems.
The AI was functioning as intended when processing the petition, but the hidden instructions were crafted to mislead its output. This distinction is critical: the harm arose not from the AI’s failure but from the adversary’s ability to introduce biased or misleading data into the system’s input.
Takeaways
This incident demonstrates that AI systems in judicial settings are not immune to adversarial manipulation, even when deployed with the intent of improving efficiency and fairness. The hidden prompt injection in the Brazilian labor court petition reveals a critical vulnerability: the potential for litigants to exploit AI tools to distort legal processes. Courts and AI developers must recognize that such attacks are not hypothetical but an emerging reality, requiring immediate attention.
Galileu’s ability to detect and alert judges to the manipulation was effective, but the incident highlights the need for proactive measures. Such safeguards would reduce the risk of AI-generated summaries or analyses being compromised by hidden directives.
While Galileu played a crucial role in flagging the manipulation, the final decision to review the petition rested with the judge.
The incident could have had more severe consequences if the hidden instructions had gone undetected, leading to misinformation or fabricated claims being incorporated into judicial proceedings. Courts should adopt clear protocols for logging and auditing AI interactions, including tracking modifications to documents, AI-generated summaries, and any alerts triggered during processing. This transparency would not only deter manipulation attempts but also provide a record for post-incident analysis.
The incident also serves as a warning about the broader implications of AI in legal systems. As more courts adopt AI tools for document analysis, summarization, and even predictive judgments, the risk of adversarial attacks increases. Litigants may increasingly turn to prompt injection or other forms of AI manipulation to gain unfair advantages, particularly in cases where the stakes are high. This trend could erode public trust in judicial AI systems if not addressed through comprehensive security measures.
Finally, the incident points to the necessity of cross-jurisdictional sharing of AI security incidents. The Brazilian labor court’s experience with prompt injection could inform similar efforts in other countries where judicial AI systems are in use. Organizations such as the OECD’s AI Incidents and Hazards Monitor play a vital role in cataloging and classifying such incidents, but courts and AI developers must also engage in proactive knowledge-sharing to stay ahead of emerging threats.
Sources
Incident summary from the AI Incident Database, used under CC BY-SA 4.0.