Auditing Black-Box Prediction Models for Data Minimization Compliance cs.bu.edu

(NeurIPS) Measures the level of data minimization satisfied by the prediction model using a limited number of queries.

Auditing: Active Learning with Outcome-Dependent Query Costs cs.bgu.ac.il

(NIPS) Learns from a binary classifier paying only for negative labels.

The Fair Game: Auditing & debiasing AI algorithms overtime cambridge.org

(Cambridge Forum on AI: Law and Governance) Aims to simulate the evolution of ethical and legal frameworks in the society by creating an auditor which sends feedback to a debiasing algorithm deployed around an ML system.

The Fair Game: Auditing & debiasing AI algorithms over time cambridge.org

Jouarnal of Cambridge Forum on AI: Law and Governance

Auditing Pay-Per-Token in Large Language Models arxiv.org

(arXiv) Develops an auditing framework based on martingale theory that enables a trusted third-party auditor who sequentially queries a provider to detect token misreporting.

Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks arxiv.org

(arXiv)

Robust ML Auditing using Prior Knowledge arxiv.org

(ICML) Formally establishes the conditions under which an auditor can prevent audit manipulations using prior knowledge about the ground truth.

Hardware and software platform inference arxiv.org

(arXiv) A method for identifying the underlying GPU architecture and software stack of a black-box machine learning model solely based on its input-output behavior.

FairProof: Confidential and Certifiable Fairness for Neural Networks arxiv.org

(Arxiv) Proposes an alternative paradigm to traditional auditing using crytographic tools like Zero-Knowledge Proofs; gives a system called FairProof for verifying fairness of small neural networks.

Fairness Auditing with Multi-Agent Collaboration arxiv.org

(ECAI) Considers multiple agents working together, each auditing the same platform for different tasks.

Mapping the Field of Algorithm Auditing: A Systematic Literature Review Identifying Research Trends, Linguistic and… arxiv.org

(Arxiv) Systematic review of algorithm auditing studies and identification of trends in their methodological approaches.

Improved Membership Inference Attacks Against Language Classification Models arxiv.org

(ICLR) Presents a framework for running membership inference attacks against classifier, in audit mode.

Auditing Yelp’s Business Ranking and Review Recommendation Through the Lens of Fairness arxiv.org

(Arxiv) Audits the fairness of Yelp’s business ranking and review recommendation systems, with demographic parity, exposure, and statistical tests such as quantile linear and logistic regression.

Auditing Fairness by Betting arxiv.org

(Neurips) [Code] Sequential methods that allows for the continuous monitoring of incoming data from a black-box classifier or regressor.

Keeping Up with the Language Models: Robustness-Bias Interplay in NLI Data and Models arxiv.org

(Arxiv) Proposes a way to extend the shelf-life of auditing datasets by using language models themselves; also finds problems with the current bias auditing metrics and proposes alternatives -- these alternatives highlight that model brittleness superficially increased the…

SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistency arxiv.org

(ICLR) Considers backdoor detection under the black-box setting in machine learning as a service (MLaaS) applications.

XAudit : A Theoretical Look at Auditing with Explanations arxiv.org

(Arxiv) Formalizes the role of explanations in auditing and investigates if and how model explanations can help audits.

Your Echos are Heard: Tracking, Profiling, and Ad Targeting in the Amazon Smart Speaker Ecosystem arxiv.org

(arxiv) Infers a link between the Amazon Echo system and the ad targeting algorithm.

Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors arxiv.org

(CHI) Makes the case for "everyday algorithmic auditing" by users.

When the Umpire is also a Player: Bias in Private Label Product Recommendations on E-commerce Marketplaces arxiv.org

(FAccT) Do Amazon private label products get an unfair share of recommendations and are therefore advantaged compared to 3rd party products?

Extracting Training Data from Large Language Models arxiv.org

(USENIX Security) Extract verbatim text sequences from the GPT-2 model’s training data.

Extracting Training Data from Large Language Models arxiv.org

(arxiv) Performs a training data extraction attack to recover individual training examples by querying the language model.

Making targeted black-box evasion attacks effective andefficient arxiv.org

(arXiv) Investigates how an adversary can optimally use its query budget for targeted evasion attacks against deep neural networks.

Neural Network Model Extraction Attacks in Edge Devicesby Hearing Architectural Hints arxiv.org

(arxiv) Through the acquisition of memory access events from bus snooping, layer sequence identification bythe LSTM-CTC model, layer topology connection according to the memory access pattern, and layer dimension estimation under data volume constraints, it demonstrates one can…

Stealing Neural Networks via Timing Side Channels arxiv.org

(arXiv) Stealing/approximating a model through timing attacks usin queries.

xGEMs: Generating Examplars to Explain Black-Box Models arxiv.org

(arXiv) Searches bias in the black box model by training an unsupervised implicit generative model. Thensummarizes the black-box model behavior quantitatively by perturbing data samples along the data manifold.

Learning Networks from Random Walk-Based Node Similarities arxiv.org

(NIPS) Reversing graphs by observing some random walk commute times.

“Why Should I Trust You?”Explaining the Predictions of Any Classifier arxiv.org

(arXiv) (Code) Explains a blackbox classifier model by sampling around data instances.

P2NIA: Privacy-Preserving Non-Iterative Auditing arxiv.org

(ECAI) Proposes a mutually beneficial collaboration for both the auditor and the platform: a privacy-preserving and non-iterative audit scheme that enhances fairness assessments using synthetic or local data, avoiding the challenges associated with traditional API-based audits.

CALM: Curiosity-Driven Auditing for Large Language Models arxiv.org

(AAAI) Auditing as a black-box optimization problem where the goal is to automatically uncover input-output pairs of the target LLMs that exhibit illegal, immoral, or unsafe behaviors.

Queries, Representation & Detection: The Next 100 Model Fingerprinting Schemes arxiv.org

(AAAI) Divides model fingerprinting into three core components, to identify ∼100 previously unexplored combinations of these and gain insights into their performance.

LLMs hallucinate graphs too: a structural perspective arxiv.org

(complex networks) Queries LLMs for known graphs and studies topological hallucinations. Proposes a structural hallucination rank.

Auditing Local Explanations is Hard arxiv.org

(NeurIPS) Gives the (prohibitive) query complexity of auditing explanations.

Setting the Record Straighter on Shadow Banning arxiv.org

(INFOCOM) (Code) Considers the possibility of shadow banning in Twitter (ie, the moderation black-box algorithm), and measures the probability of several hypothesis.

Adversarial Model Extraction on Graph Neural Networks arxiv.org

(AAAI Workshop on Deep Learning on Graphs: Methodologies and Applications) Introduces GNN model extraction and presents a preliminary approach for this.

TamperNN: Efficient Tampering Detection of Deployed Neural Nets arxiv.org

(ISSRE) Algorithms to craft inputs that can detect the tampering with a remotely executed classifier model.

Knockoff Nets: Stealing Functionality of Black-Box Models arxiv.org

(CVPR) Ask to what extent can an adversary steal functionality of such "victim" models based solely on blackbox interactions: image in, predictions out.

Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data arxiv.org

(IJCNN) (Code) Stealing black-box models (CNNs) knowledge by querying them with random natural images (ImageNet and Microsoft-COCO).

Adversarial Frontier Stitching for Remote Neural Network Watermarking arxiv.org

(Neural Computing and Applications) (Alternative implementation) Check if a remote machine learning model is a "leaked" one: through standard API requests to a remote model, extract (or not) a zero-bit watermark, that was inserted to watermark valuable models (eg, large deep…

Towards Reverse-Engineering Black-Box Neural Networks arxiv.org

(ICLR) (Code) Infer inner hyperparameters (eg number of layers, non-linear activation type) of a remote neural network model by analysing its response patterns to certain inputs.

Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR arxiv.org

(Harvard Journal of Law & Technology) To explain a decision on x, find a conterfactual: the closest point to x that changes the decision.

Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation arxiv.org

(AIES) Treats black box models as teachers, training transparent student models to mimic the risk scores assigned by black-box models.

The topological face of recommendation: models and application to bias detection arxiv.org

(Complex Networks) Proposes a bias detection framework for items recommended to users.

Auditing Black-Box Models for Indirect Influence arxiv.org

(ICDM) Evaluate the influence of a variable on a black-box model by "cleverly" removing it from the dataset and looking at the accuracy gap

Certifying and Removing Disparate Impact arxiv.org

(SIGKDD) Proposes SVM-based methods to certify absence of bias and methods to remove biases from a dataset.

Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems andrew.cmu.edu

(IEEE S&P) Evaluate the individual, joint and marginal influence of features on a model using shapley values.

Stanford Complexity Group youtube.com

YouTube channel with videos from the group's events.

Boccara, N. (2014). Modeling complex systems. Springer-Verlag New York. worldcat.org

An overview of important mathematical models of complex systems.

Newman, M. E. J. (2010). Networks: an introduction. Oxford; New York: Oxford University Press. worldcat.org

A comprehensive introduction to network science.

Gros, C. (2015). Complex and adaptive dynamical systems: a primer. Cham: Springer. worldcat.org

Technical and accessible introduction to a number of important concepts in complex systems science.