Prompt Injection Benchmarks: AgentDojo, BIPIA, CyberSecEval
Seven prompt injection benchmarks are in active use and their headline attack success rates run from 24% to 84.30%: AgentDojo (629 security cases, ETH Zurich, NeurIPS 2024), BIPIA (Microsoft, KDD 2025), InjecAgent (1,054 cases, ACL Findings 2024), Open-Prompt-Injection (USENIX Security 24), CyberSecEval 2 (Meta, 26-41% on every model), Agent Security Bench (ICLR 2025, 84.30% peak), and MCPTox (45 real MCP servers). Every one of those is a static score. When researchers from OpenAI, Anthropic, and Google DeepMind ran adaptive attacks against twelve published defenses, all twelve fell, most above 90%. This piece gives the comparison table, what each benchmark measures, and the design conclusion the numbers force.
Read post →