Tamper-Resistant Safeguards for Open-Weight LLMs: ICLR 2024

Research question

Can safeguards be trained to survive adversarial fine-tuning of open weights?

Key findings

TAR meta-learning makes weaponisation-knowledge restrictions and refusals resist thousands of fine-tuning steps - Robustness comes at some cost to benign capabilities and is not absolute

Sources

arxiv.orghttps://arxiv.org/abs/2408.00761

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal

Related in Research Paper Library

rApp/xApp Attestation: A New Security Use Case for O-RAN

The disaggregation and softwarization introduced by the Open Radio Access Network (O-RAN) architecture enable multi-vendor innovation but also expose the RAN Intelligent Controller (RIC) ecosystem to new runtime…

Tamper-resistant safeguards2026

From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation

Research on large language model agents for penetration testing is evaluated almost entirely by capability: whether the agent captures a flag or reproduces a proof of concept. That metric suits a benchmark but is…

Tamper-resistant safeguards2026

NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI Agent Runs

When an autonomous AI agent does something consequential, what can be proven about what it did? Agent-observability platforms capture traces, but a trace is mutable: alterable undetected, with no recipe for…

Tamper-resistant safeguards2026