I have officially graduated with a Ph.D. from @EPFL! It was an honor to defend my thesis before an exceptional jury: Francis Bach @BachFrancis, Samy Bengio, Gautam Kamath @thegautamkamath, Adam Smith, Rachid Guerraoui (advisor), and Emre Telatar (chair).

Stanford Trustworthy AI Research (@stai_research) has exceptional researchers on the faculty market this year! 🧵 🔹 Anka Reuel - Technical AI governance 🔹 Dr. Olawale Salaudeen - AI measurement & robustness 🔹 Dr. Andreas Haupt - AI, Economics, and Policy

My thesis on the "cost of trust in machine learning" is available online (link below). I could not have done this research without my collaborators, mentors, friends, and family. I am deeply grateful to all of them. Thesis link:

@sanmikoyejo @Ana_koloskova @sanmi 𝗣𝗼𝘀𝘁𝗲𝗿: 𝗧𝗿𝘂𝘀𝘁𝘄𝗼𝗿𝘁𝗵𝘆 𝗙𝗲𝗱𝗲𝗿𝗮𝘁𝗲𝗱 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗨𝗻𝘁𝗿𝘂𝘀𝘁𝗲𝗱 𝗣𝗮𝗿𝘁𝗶𝗰𝗶𝗽𝗮𝗻𝘁𝘀 (𝗧𝗵𝘂𝗿𝘀𝗱𝗮𝘆, 𝗝𝘂𝗹𝘆 𝟭𝟳). A culmination of several works on privacy vs. robustness trade-offs. Paper: https://t.co/JUhpY5wehp w/ John, Rachid

@sanmikoyejo 𝗣𝗼𝘀𝘁𝗲𝗿: 𝗖𝗲𝗿𝘁𝗶𝗳𝗶𝗲𝗱 𝗨𝗻𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝗡𝗲𝘂𝗿𝗮𝗹 𝗡𝗲𝘁𝘄𝗼𝗿𝗸𝘀 (𝗧𝘂𝗲𝘀𝗱𝗮𝘆, 𝗝𝘂𝗹𝘆 𝟭𝟱). An exciting and important step towards making certified unlearning practical. Paper: https://t.co/F7og8rjDtE (w/ @Ana_koloskova, Animesh, Rachid, @sanmi )

𝗢𝗿𝗮𝗹 𝗮𝘁 𝘁𝗵𝗲 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗨𝗻𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗔𝗜 𝘄𝗼𝗿𝗸𝘀𝗵𝗼𝗽 (𝗙𝗿𝗶𝗱𝗮𝘆, 𝗝𝘂𝗹𝘆 𝟭𝟴). On our latest research introducing a new distributional framework for unlearning. Paper: https://t.co/PWbnmu7Y41 (w/ @sanmikoyejo, Rachid)

I will be attending #icml2025 in Vancouver next week! This is a special one for me, as it may be my last as a PhD student. I'll be presenting works on unlearning, privacy, and robustness. I'm happy to connect and exchange ideas! You can find me at these presentations:

Joint work with @Ana_koloskova (co-first author), Animesh Jha, Rachid Guerraoui, and @sanmikoyejo. @stai_research @StanfordAILab @ICepfl

Excited our paper "Certified Unlearning for Neural Networks" is accepted at ICML 2025! We introduce a method for provable machine unlearning-- truly "forgetting" data without restrictive assumptions like convexity. Paper: https://t.co/F7og8rjDtE Code: https://t.co/LLQD8HvcbY

I am presenting the poster for our unlearning paper (below) at @iclr_conf in Hall 2, now! Come by to have a chat.

This is joint work with @JoshuaK92829, R. Guerraoui, @sanmikoyejo. Paper: https://t.co/VJv6P2nq33 n/n

Solution: Use robust mean estimation during pre-training. Trimmed mean strategies make OOD unlearning faster & more reliable. See controlled experiment below comparing vanilla (Alg. 1) vs. robust fine-tuning (Alg. 2) for various number f (out of n) of forget samples. 10/n

Finding 3. For Out-of-Distribution Forget Data, Robust Pre-Training is Crucial If the forget data is very different from the retained data, fine-tuning alone can fail—it might take more time to unlearn a single sample, than to fully retrain the model. 😨 9/n

The latter capacity decreases with the model dimension for DP (Sekhari et al., 2021, Huang & Canonne, 2023). We show that the same quantity is independent of model dimension for noisy fine-tuning. See experiment on linear regression below (Alg. 1 = Noisy fine-tuning) 8/n

Finding 2. Differential Privacy (DP) implies Unlearning—But is an Overkill DP implies unlearning, but it severely limits model performance. We prove a tight separation between DP and noisy fine-tuning, in terms of how many samples can be deleted at a fixed test loss. 7/n

For our analysis, we require access to an approximate global risk minimizer. For convex tasks, this can simply be SGD, with near-linear time complexity (Neel et al., 2021). Such a minimizer also exists for structured non-convex tasks, e.g., PCA, matrix completion. 6/n

Finding 1: Noisy Fine-Tuning Works for In-Distribution Forget Data If the forget data comes from the same distribution as the retained data, noisy fine-tuning (see below) is a highly effective and practical unlearning method. 5/n

Problem: What Does It Mean to "Unlearn"? We define certified unlearning as making the post-unlearning model statistically near-indistinguishable from a model that was trained without the forget data. This is inspired from differential privacy (Dwork & Roth, 2014). 4/n

Our main insight: it depends on the nature of the forget data. If it's in-distribution → Noisy fine-tuning works great. Differential privacy may be an overkill. If it's out-of-distribution → (Noisy) Fine-tuning struggles. A robust pre-training strategy is needed. 3/n

Key Question: Can we truly remove data from a trained AI model—without hurting performance or requiring a full retrain? 2/n