Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
Discovers that text embedding models have severely biased output distributions, and exploits this to find universal adversarial suffixes ("magic words") that bypass embedding-based LLM safeguards. Attacks transfer across models and languages; a train-free debiasing defense is…
- from
- Prompt Injection
- added
- 2026-10-10
- likes
- 0
similar
-
Universal and Transferable Adversarial Attacks on Aligned Language Models arxiv.org
This paper introduces a simple and efficient attack method that enables aligned language models to generate objectionable content with high probability, highlighting the need for improved prevention techniques in large language models. The generated adversarial prompts are…
-
Stealing the Decoding Algorithms of Language Models people.cs.umass.edu
(CCS) Steal the type and hyperparameters of the decoding algorithms of a LLM.
-
The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections arxiv.org
Oct 2025 paper systematically breaking 12 published defenses using gradient descent, RL, random search, and human-guided exploration. Most defenses originally claimed near-zero attack success rates; adaptive attacks exceeded 90% against all of them.
-
Keeping Up with the Language Models: Robustness-Bias Interplay in NLI Data and Models arxiv.org
(Arxiv) Proposes a way to extend the shelf-life of auditing datasets by using language models themselves; also finds problems with the current bias auditing metrics and proposes alternatives -- these alternatives highlight that model brittleness superficially increased the…
-
Extracting Training Data from Large Language Models arxiv.org
(arxiv) Performs a training data extraction attack to recover individual training examples by querying the language model.
-
Prompt Injection › Research Papers: “Discovers that text embedding models have severely biased output distributions, and exploits this to find universal adversarial suffixes ("magic words") that bypass embedding-based LLM safeguards. Attacks transfer across models and languages; a train-free debiasing defense is…”