CALM: Curiosity-Driven Auditing for Large Language Models
(AAAI) Auditing as a black-box optimization problem where the goal is to automatically uncover input-output pairs of the target LLMs that exhibit illegal, immoral, or unsafe behaviors.
- from
- Audit Algorithms
- added
- 2026-10-10
- likes
- 0
similar
-
Keeping Up with the Language Models: Robustness-Bias Interplay in NLI Data and Models arxiv.org
(Arxiv) Proposes a way to extend the shelf-life of auditing datasets by using language models themselves; also finds problems with the current bias auditing metrics and proposes alternatives -- these alternatives highlight that model brittleness superficially increased the…
-
Active Fairness Auditing proceedings.mlr.press
(ICML) Studies of query-based auditing algorithms that can estimate the demographic parity of ML models in a query-efficient manner.
-
Auditing Pay-Per-Token in Large Language Models arxiv.org
(arXiv) Develops an auditing framework based on martingale theory that enables a trusted third-party auditor who sequentially queries a provider to detect token misreporting.
-
Under manipulations, are some AI models harder to audit? grodino.github.io
(SATML) Relates the difficulty of black-box audits to the capacity of the targeted models, using the Rademacher complexity.
-
Robust ML Auditing using Prior Knowledge arxiv.org
(ICML) Formally establishes the conditions under which an auditor can prevent audit manipulations using prior knowledge about the ground truth.
-
The Fair Game: Auditing & debiasing AI algorithms over time cambridge.org
Jouarnal of Cambridge Forum on AI: Law and Governance
Audit Algorithms › Papers > 2025: “(AAAI) Auditing as a black-box optimization problem where the goal is to automatically uncover input-output pairs of the target LLMs that exhibit illegal, immoral, or unsafe behaviors.”