Do we really need to learn from our mistakes? Couldn’t we just… make optimal decisions like computers 🤖 and not make mistakes?. Well, our new work accepted in Computational Brain & Behavior journal indicates that it might be possible!. 🧵👇1/9

We’ll be presenting @FAccTConference on 06.24 at 10:45 AM during the Evaluating Explainable AI session!. Come chat with us. We would love to discuss implications for AI policy, better auditing methods, and next steps for algorithmic fairness research. #AIFairness #xAI.

But if they are indeed used to dispute discrimination claims, we can expect multiple failed cases due to insufficient evidence and many undetected discriminatory decisions. Current explanation-based auditing is, therefore, fundamentally flawed, and we need additional safeguards.

Despite their unreliability, explanations are suggested as anti-discrimination measures by a number of regulations. GDPR ✓ Digital Services Act ✓ Algorithmic Accountability Act ✓ GDPD (Brazil) ✓

So why do explanations fail?. 1️⃣ They target individuals, while discrimination operates on groups.2️⃣ Users' causal models are flawed.3️⃣ Users overestimate proxy strength and treat its presence in the explanation as discrimination.4️⃣ Feature-outcome relationships bias user claims.

BADLY. When participants flag discrimination, they are correct ~50% of the time, miss 55% of the discriminatory predictions, and keep a 30% FPR. Additional knowledge (protected attributes, proxy strength) improves the detection to roughly 60% without affecting other measures.

Our setup lets us assign each robot a ground-truth discrimination outcome, which lets us evaluate how well each participant could do under different information regimes. So, how did they do?.

We recruited participants, anchored their beliefs on discrimination, trained them to use explanations, and tested to make sure they got it right. We then saw how well they could flag unfair predictions based on counterfactual explanations and feature attribution scores.

Participants audit a model to predict if robots sent to Mars will break down. Some are built by “Company X.” Others by “Company S.”. Our model predicts failure based on robot body parts. It can discriminate against Company X by predicting that robots without an antenna fail.

We cannot tell if explanations work or not due to these reasons. To tackle this challenge, we introduce a synthetic task where we:.- Teach users how to use explanations.- Control their beliefs.- Adapt the world to fit their beliefs.- Control the explanation content.

Users may fail to detect discrimination through explanations due to:. - Proxies not being revealed by explanations.- Issues with interpreting explanations.- Wrong assumptions about proxy strength.- Unknown protected class.- Incorrect causal beliefs.

Imagine a model that predicts loan approval. Would a rejected female applicant get approved if she somehow applied as a man?. If yes, her prediction was discriminatory. Fairness requires predictions to stay the same regardless of the protected class (sex, age, etc.)

Right to explanation laws assume explanations help people detect algorithmic discrimination. But is there any evidence for that?. In our latest work w/ David Danks @berkustun, we show explanations fail to help people, even under optimal conditions. PDF

RT @GoogleAI: Today we introduce an AI co-scientist system, designed to go beyond deep research tools to aid scientists in generating novel….

RT @KexinHuang5: 🧪 Introducing POPPER: an AI agent that automates hypothesis validation by sequentially designing and executing falsificati….

RT @HaileyJoren: ✅Easy: train a model to automate a routine task.❌Hard: ensure the model is accurate with minimal human oversight. In our l….

RT @berkustun: 📢 Please RT 📢. I am recruiting PhD students to join my group at UCSD!. We develop methods for responsible machine learning….

RT @RtnlAltruismLab: @FalkLieder gave a talk on Improving #Decision_making @PennMindCORE. Here’s the recording:

RT @cogconfluence: 🚨 NEW PAPER 🚨. Understanding increasingly large and complex neural networks will almost certainly require other AI model….

RT @FalkLieder: I'm thrilled to announce the launch of the Rational Altruism Lab at UCLA. We strengthen the scientific foundations for moti….

RT @berkustun: Machine learning models often use group attributes like sex, age, and race for personalization. In our latest work, we show….