Knowledge Base Regulation Tracker Standards & Frameworks OSS Directory AI Incident Database Research Paper Library Case Studies Society Articles News Jobs About Charter Sign inBecome a Member
Home/Case Studies/Microsoft's AI Red Team distils lessons from red teaming more than 100 generative AI products

Microsoft's AI Red Team distils lessons from red teaming more than 100 generative AI products

Microsoft's AI Red Team publishes a threat-model ontology, eight lessons and five case studies from its operations, including an SSRF in a video-generation pipeline via an outdated FFmpeg, a vision-model jailbreak and an automated scam.

Red teamingPyRITSSRFJailbreakThreat model

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit case studyReport a correction or request removal

Related in Case Studies

Achieving Code Execution in MathGPT via Prompt Injection

The publicly available Streamlit application MathGPT uses GPT-3, a large language model (LLM), to answer user-generated math questions. Recent studies and experiments have shown that LLMs such as GPT-3 show poor…

AI developers and research labsPrompt injection defence design

Advai: Robustness Assurance Framework for Guardrail Implementation in Large Language Models (LLMs)

Advai applies a robustness assurance framework that attacks a client LLM with adversarial prompts optimised across several model architectures, uses the findings to set guardrails that keep outputs inside the…

AI developers and research labsRed teaming programme

AI ClickFix: Hijacking Computer-Use Agents Using ClickFix

Embrace the Red demonstrated that AI computer-use agents are vulnerable to social engineering attacks and can be manipulated into executing arbitrary code on a victim’s machine. The attack is a variation on…

AI developers and research labsAgent security architecture