Keeping Up with the Language Models: Robustness-Bias Interplay in NLI Data and Models
(Arxiv) Proposes a way to extend the shelf-life of auditing datasets by using language models themselves; also finds problems with the current bias auditing metrics and proposes alternatives -- these alternatives highlight that model brittleness superficially increased the…
- from
- Audit Algorithms
- added
- 2026-10-10
- likes
- 0
similar
-
CALM: Curiosity-Driven Auditing for Large Language Models arxiv.org
(AAAI) Auditing as a black-box optimization problem where the goal is to automatically uncover input-output pairs of the target LLMs that exhibit illegal, immoral, or unsafe behaviors.
-
Auditing Pay-Per-Token in Large Language Models arxiv.org
(arXiv) Develops an auditing framework based on martingale theory that enables a trusted third-party auditor who sequentially queries a provider to detect token misreporting.
-
Active Fairness Auditing proceedings.mlr.press
(ICML) Studies of query-based auditing algorithms that can estimate the demographic parity of ML models in a query-efficient manner.
-
Universal and Transferable Adversarial Attacks on Aligned Language Models arxiv.org
This paper introduces a simple and efficient attack method that enables aligned language models to generate objectionable content with high probability, highlighting the need for improved prevention techniques in large language models. The generated adversarial prompts are…
-
Improved Membership Inference Attacks Against Language Classification Models arxiv.org
(ICLR) Presents a framework for running membership inference attacks against classifier, in audit mode.
-
Mapping the Field of Algorithm Auditing: A Systematic Literature Review Identifying Research Trends, Linguistic and… arxiv.org
(Arxiv) Systematic review of algorithm auditing studies and identification of trends in their methodological approaches.
Audit Algorithms › Papers > 2023: “(Arxiv) Proposes a way to extend the shelf-life of auditing datasets by using language models themselves; also finds problems with the current bias auditing metrics and proposes alternatives -- these alternatives highlight that model brittleness superficially increased the…”