Become a Member

Regulation TrackerLaws, orders and treaties read for their security duties and clocks Standards & FrameworksAttack taxonomies, control catalogues and frontier frameworks AI Incident DatabaseThe security incident record, with mechanism and lesson Research Paper LibraryThe papers that define the field, summarised OSS DirectoryRed teaming, guardrail, supply chain and evaluation tools Case StudiesProgrammes built, with measured outcomes AI Vulnerability TrackerCVEs and advisories for the components AI systems are built from

Latest in the tracker

New York Stop Addictive Feeds Exploitation (SAFE) for Kids Act and implementing rulesNew York's law restricting algorithmic feeds for minors takes effect on 25 January 2027 under final rules published in July 2026, with certified age assurance and $5,000-per-violation penalties.

Knowledge Base Regulation Tracker Standards & Frameworks OSS Directory AI Incident Database Research Paper Library Case Studies Society Articles News Jobs About Charter Become a Member

Home/Research Paper Library/Purple Llama CyberSecEval

Purple Llama CyberSecEval

Meta's CyberSecEval suite, the most widely run cyber-safety benchmark, now in its fourth version.

cybersecevalsecure codingmeta

Research question

How often do LLMs generate insecure code or comply with cyberattack requests?

Key findings

Models suggested insecure code 30% of the time and complied with 52% of cyberattack-assistance requests - Later versions added prompt injection, vulnerability exploitation and autonomous attack tests

Sources

arxiv.orghttps://arxiv.org/abs/2312.04724

Contribute

Spot an error, or know a record we should add? Members can propose additions and submit corrections.

Submit research Report a correction or request removal