Knowledge Base Regulation Tracker Standards & Frameworks OSS Directory AI Incident Database Research Paper Library Case Studies Society Articles News Jobs About Charter Sign inBecome a Member
Home/Case Studies/Planting Instructions for Delayed Automatic AI Agent Tool Invocation

Planting Instructions for Delayed Automatic AI Agent Tool Invocation

Embrace the Red demonstrated that Google Gemini is susceptible to automated tool invocation by delaying the execution to the next conversation turn. This bypasses a security control that restricts Gemini from invoking tools that can access sensitive user information in the same conversation turn that untrusted data enters context.

LLM Prompt CraftingPrompt Infiltration via Public-Facing ApplicationIndirectDelay Execution of LLM InstructionsAI Agent Tool InvocationAI Agent Tools

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit case studyReport a correction or request removal

Related in Case Studies

Achieving Code Execution in MathGPT via Prompt Injection

The publicly available Streamlit application MathGPT uses GPT-3, a large language model (LLM), to answer user-generated math questions. Recent studies and experiments have shown that LLMs such as GPT-3 show poor…

AI developers and research labsPrompt injection defence design

Advai: Robustness Assurance Framework for Guardrail Implementation in Large Language Models (LLMs)

Advai applies a robustness assurance framework that attacks a client LLM with adversarial prompts optimised across several model architectures, uses the findings to set guardrails that keep outputs inside the…

AI developers and research labsRed teaming programme

AI ClickFix: Hijacking Computer-Use Agents Using ClickFix

Embrace the Red demonstrated that AI computer-use agents are vulnerable to social engineering attacks and can be manipulated into executing arbitrary code on a victim’s machine. The attack is a variation on…

AI developers and research labsAgent security architecture