The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections
Oct 2025 paper systematically breaking 12 published defenses using gradient descent, RL, random search, and human-guided exploration. Most defenses originally claimed near-zero attack success rates; adaptive attacks exceeded 90% against all of them.
- from
- Prompt Injection
- added
- 2026-10-10
- likes
- 0
similar
-
New Prompt Injection Papers: Agents Rule of Two and The Attacker Moves Second simonwillison.net
Simon Willison's Nov 2025 commentary on both landmark papers, including the finding that 12 published defenses were bypassed at >90% success rate using gradient descent and RL-based adaptive attacks.
-
Securing AI Agents Against Prompt Injection Attacks arxiv.org
Nov 2025 benchmark of 847 adversarial test cases across 5 attack categories against 7 LLMs. The combined defense framework reduces attack success from 73.2% to 8.7% while retaining 94.3% of baseline task performance.
-
Continuously Hardening ChatGPT Atlas Against Prompt Injection Attacks openai.com
OpenAI's Dec 2025 disclosure of a real attack chain (malicious email → agent sends resignation letter) and the RL-trained automated attacker they built to find new injection classes before external adversaries do. OpenAI explicitly states deterministic guarantees are not…
-
Attention Tracker: Detecting Prompt Injection Attacks in LLMs aclanthology.org
NAACL 2025 Findings paper detecting prompt injection by tracking attention distribution shifts, with no modification to the underlying model required, making it deployable as a wrapper on any LLM.
-
Data driven exploratory attacks on black box classifiers in adversarial domains sciencedirect.com
(Neurocomputing) Reverse engineers remote classifier models (e.g., for evading a CAPTCHA test).
-
Prompt Injection 2.0: Hybrid AI Threats arxiv.org
Jul 2025 paper showing how prompt injections now combine with XSS, CSRF, AI worm propagation, and multi-agent infections to evade traditional WAFs entirely. Evaluates Preamble's classifier, data-tagging, and RL-based defenses against these hybrid scenarios.
Prompt Injection › Research Papers: “Oct 2025 paper systematically breaking 12 published defenses using gradient descent, RL, random search, and human-guided exploration. Most defenses originally claimed near-zero attack success rates; adaptive attacks exceeded 90% against all of them.”