Get my drift? Catching LLM Task Drift with Activation Deltas

Research question

Can injection be detected from changes in the model's internal activations?

Key findings

Comparing activations before and after processing external data detects task drift with over 0.99 ROC AUC - Generalises to unseen attacks without model modification

Sources

arxiv.orghttps://arxiv.org/abs/2406.00799

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

StyleAT: Defending Face Recognition Against Semantic Attacks

With face-recognition models now embedded in everyday authentication and surveillance, recent works have pinpointed a critical weakness: these models remain acutely vulnerable to adversarial semantic edits. I.e.,…

Guardrails and classifier defences2026

Robot Visions: Breaking reCAPTCHA at Zero Cost and Zero Shot

Google reCAPTCHA is the most widely deployed visual CAPTCHA service, protecting hundreds of thousands of websites from automated bots. It serves as a critical line of defense against automated attacks, including…

Guardrails and classifier defences2026

The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphs

We have developed the MAL Simulator, a cyber operation simulator based on the Meta Attack Language (MAL). The MAL Simulator is intended for decision-driven cyber attack and defense simulations, for system analysis…

Guardrails and classifier defences2026