Poisoning Language Models During Instruction Tuning

Research question

Can a few poisoned instruction-tuning examples manipulate model behaviour on trigger phrases?

Key findings

About 100 poison examples cause consistent mispredictions or degenerate outputs whenever a trigger phrase appears - Poison transfers across held-out tasks

Sources

arxiv.orghttps://arxiv.org/abs/2305.00944

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agents

Memory-poisoning defenses for LLM agents are typically evaluated by their ability to prevent attacks. However, the traffic they process is rarely adversarial. The cost of implementing a defense is paid with each…

Data poisoning2026

Micro-Collaborative Poisoning: A Distributed Attack on RAG Systems

Retrieval-Augmented Generation (RAG) improves large language models by grounding outputs in external knowledge sources, but this dependency also creates a surface for poisoning attacks. This paper introduces…

Data poisoning2026

PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs

AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement token cost makes continuous, automated testing unaffordable for the smaller organisations that…

Data poisoning2026