AI
Artificial intelligence: machine learning, neural networks, large language models, deep learning, AI research, and AI tools.
-
Google AI Red Team: Securing AI services.google.com
Google's red team walkthrough of how AI systems are attacked in practice. An early foundational report, useful for framing rather than for current technique.
-
r/llmsecurity reddit.com
The most active subreddit dedicated to LLM security research; a good early-warning channel for real-world incidents and new disclosures.
-
PromptTrace prompttrace.airedlab.com
Free AI security training platform with 7 hands-on prompt injection labs and a 15-level CTF (the Gauntlet) with progressively harder defenses, from prompt-level rules to code guards to LLM classifiers. Unique feature: Context Trace shows the full prompt stack (system prompt…
-
Augustus praetorian.com
Feb 2026 open-source tool from Praetorian. A single Go binary with 210+ vulnerability probes across 47 attack categories, 28 LLM providers, 90+ detectors, and 7 payload transformation buffs. Built for penetration testing workflows without Python/npm dependencies.
-
Continuously Hardening ChatGPT Atlas Against Prompt Injection Attacks openai.com
OpenAI's Dec 2025 disclosure of a real attack chain (malicious email → agent sends resignation letter) and the RL-trained automated attacker they built to find new injection classes before external adversaries do. OpenAI explicitly states deterministic guarantees are not…
-
Synthetic Recollections - A Case Study in Prompt Injection for ReAct LLM Agents labs.withsecure.com
A practical scenario showing how prompt injection can be used to hi-jack the ReAct loop used by LLM agents to inject forged thoughts and associated observations into the LLM context, thus altering the intended behavior.
-
OWASP Top 10 for Agentic Applications (2026) genai.owasp.org
Companion list for autonomous and agentic systems, released Dec 2025 with input from over 100 contributors. Covers the agent-specific failure modes that the LLM list does not reach, including tool misuse and agent hijacking.
-
OWASP GenAI LLM Top 10 (2026) genai.owasp.org
The canonical community reference for LLM application risk, with prompt injection at number one. Supersedes the 2025 edition; the per-risk 2025 pages remain online as an archive.
-
OWASP Gen AI Security Project genai.owasp.org
The standards body behind the LLM and Agentic Top 10 lists, with working groups, cheat sheets and guidance updated continuously by practitioners across the industry.
-
ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery embracethered.com
This post shows how a malicious website can take control of a ChatGPT chat session and exfiltrate the history of the conversation.
-
Don't you (forget NLP): Prompt injection with control characters in ChatGPT dropbox.tech
A look into how to achieve prompt injection from control characters from Dropbox.
-
Authority Framing in Agentic CI/CD (RELAY) doi.org
Pre-registered factorial study of a five-agent CI/CD pipeline built from five production LLMs. A fabricated "pre-approved, do not re-review" claim gets secret-exfiltration code shipped in up to 55% of runs, and roughly 80% of laundered pull requests pass a capable LLM code…
-
Why content-based prompt-injection detection has a structural blind spot dev.to
Walks through a disguised "already-scanned, classified DATA" annotation that passes both a regex layer and a local LLM classifier (qwen2.5:3b and 14b), with the gap pinned as a failing test. Detection is necessary but not sufficient.
-
CrowdStrike AI Unlocked crowdstrike.com
Released Feb 2026, designed to train security, developer, and AI teams on prompt injection against increasingly capable agents. Built by CrowdStrike's Counter Adversary Operations team.
-
Indirect Prompt Injection Attacks: Hidden AI Risks crowdstrike.com
CrowdStrike's Dec 2025 analysis of IPI TTPs targeting enterprise GenAI, including attacker-controlled document poisoning, RAG context manipulation, and practical detection signals for SOC workflows.
-
Prompt Injection Cheat Sheet: How To Manipulate AI Language Models blog.seclify.com
A prompt injection cheat sheet for AI bot integrations.
-
Improving LLM Security Against Prompt Injection: AppSec Guidance For Pentesters and Developers – Part 2 blog.includesecurity.com
Understanding Transformer Models (especially attention), Causes and How Do We Stop Prompt Injection.
-
Improving LLM Security Against Prompt Injection: AppSec Guidance For Pentesters and Developers blog.includesecurity.com
Using Role Based APIs to Minimize the Risk of Prompt Injection and 13 Guidelines for Writing System Prompts that Minimize the Risk of Prompt Injection.
-
The Landscape of Prompt Injection Threats in LLM Agents (SoK) arxiv.org
Feb 2026 systematization-of-knowledge paper with a unified taxonomy covering attack payload strategies (heuristic vs. optimisation-based) and defense intervention stages (text, model, execution). Introduces the AgentPI benchmark for context-dependent agent tasks that all prior…
-
Securing AI Agents Against Prompt Injection Attacks arxiv.org
Nov 2025 benchmark of 847 adversarial test cases across 5 attack categories against 7 LLMs. The combined defense framework reduces attack success from 73.2% to 8.7% while retaining 94.3% of baseline task performance.
-
Oct 2025 paper systematically breaking 12 published defenses using gradient descent, RL, random search, and human-guided exploration. Most defenses originally claimed near-zero attack success rates; adaptive attacks exceeded 90% against all of them.
-
Prompt Injection 2.0: Hybrid AI Threats arxiv.org
Jul 2025 paper showing how prompt injections now combine with XSS, CSRF, AI worm propagation, and multi-agent infections to evade traditional WAFs entirely. Evaluates Preamble's classifier, data-tagging, and RL-based defenses against these hybrid scenarios.
-
ToolHijacker: Prompt Injection Attack to Tool Selection in LLM Agents arxiv.org
Apr 2025 paper introducing a no-box attack that injects a malicious tool document into an agent's tool library to consistently hijack tool selection. Finds that StruQ, SecAlign, DataSentinel, and perplexity detection are all insufficient defenses.
-
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models arxiv.org
Discovers that text embedding models have severely biased output distributions, and exploits this to find universal adversarial suffixes ("magic words") that bypass embedding-based LLM safeguards. Attacks transfer across models and languages; a train-free debiasing defense is…
-
Universal and Transferable Adversarial Attacks on Aligned Language Models arxiv.org
This paper introduces a simple and efficient attack method that enables aligned language models to generate objectionable content with high probability, highlighting the need for improved prevention techniques in large language models. The generated adversarial prompts are…
-
This paper explores the concept of Indirect Prompt Injection attacks on Large Language Models (LLMs) through their integration with various applications. It identifies significant security risks, including remote data theft and ecosystem contamination, present in both…
-
Agents Rule of Two: A Practical Approach to AI Agent Security ai.meta.com
Meta's Oct 2025 framework stating that agents must satisfy no more than two of: (A) processing untrustworthy inputs, (B) access to sensitive data, (C) ability to change state externally, a deterministic architectural approach to bounding blast radius.
-
Attention Tracker: Detecting Prompt Injection Attacks in LLMs aclanthology.org
NAACL 2025 Findings paper detecting prompt injection by tracking attention distribution shifts, with no modification to the underlying model required, making it deployable as a wrapper on any LLM.
-
AI/LLM Exploitation Challenges academy.8ksec.io
AI, ML, and LLMs CTF Challenges.
-
Detection Engineering AI Maturity Framework | Brendan Chamberlain infosecb.github.io
A community framework with four maturity levels across ten dimensions for assessing how organizations apply AI and LLMs across a detection engineering program, from foundations through the detection lifecycle.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Packing detection and classification relying on machine learning to stop malware propagation theses.hal.science
(December 2021) :star:
-
Assessing the impact of packing on machine learning-based malware detection and classification systems sciencedirect.com
(September 2025) :star:
-
Analysis of machine learning approaches to packing detection sciencedirect.com
(October 2023) :star: :star:
-
An application of machine learning to analysis of packed mac malware scholar.dsu.edu
(May 2022) :star:
-
Titanium Platform reversinglabs.com
Machine learning hybrid cloud platform that harvests thousands of file types at scale, speeds threat detection through machine learning binary analysis, and continuously monitors an index of over 10B files for future threats.
-
Classification of malware by using structural entropy on convolutional neural networks ojs.aaai.org
(April 2018)
-
When malware is packin' heat; limits of machine learning classifiers based on static analysis features ndss-symposium.org
(January 2020) :star: :star:
- next page of items loading…
