Huge congrats to @ganevgv for receiving the Distinguished Paper Award at #ieeesp25 @IEEESSP for his work "The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against “Truly Anonymous” Synthetic Datasets." https://t.co/CGVbVGmCok

Happy to announce that Dr. Emiliano De Cristofaro was recently recognized as a Distinguished Member of the ACM https://t.co/Ih7B8jKCHE

.@sundarmsa's upcoming WWW paper on browser fingerprinting is now available on arXiv https://t.co/nk7DsxtvbJ

Today at #NeurIPS2024 @sundarmsa will present his paper "Nearly Tight Black-Box Auditing of Differentially Private Machine Learning." Starting 4:30pm in West Ballroom A-D, poster #6208. Paper:

I'll be attending #NeurIPS2024 next week presenting our work on Black-Box Auditing of DP-SGD ( https://t.co/boNvQTg3AE). Looking forward to having lots of chats on privacy and memorization in ML models! Also I am on the other app too!🦋msundarmsa

Congrats to @sundarmsa -- the paper "Nearly Tight Black-Box Auditing of Differentially Private Machine Learning" was accepted to #NeurIPS2024 ! Pre-print:

🚨 Our paper featured in @newscientist 😎 Our research, published at @ACM_IMC_2024 (w/@vekariayash,@giaanselmi,@Homiefe,@pcallejop,@zubair_shafiq), reveals that Smart TVs from major brands can capture snapshots of what you're watching several times per second📺 Find out more!

On the other hand, non-convex loss functions used in practice might satisfy (currently unknown) constraints that could still hold potential for hidden state privacy amplification. Eitherways our work shows that analyzing the loss function is crucial to hidden state analysis. 10/n

So what now? The loss function we have constructed is non-convex. But hidden state privacy amplification theorems do not even cover convex functions in general (only strongly convex or linear). Constraining our loss function might reveal the (im-)possibility of that result. 9/n

We emphasize that this is not a realistic loss function by any means, but it acts as a counter-example / impossibility result and shows the theoretical limits of any privacy analysis of DP-SGD in the hidden state setting. 8/n

What does this mean? Given our loss function, an adversary can distinguish between the final model of DP-SGD just as easily as they can distinguish between all intermediate models combined, which matches the current state of the art theoretical privacy analysis of DP-SGD. 7/n

...until now. In this paper, we show that this is impossible to do in general. What do we do? We construct a worst-case loss function that encodes all of the information from the intermediate models into the final model. 6/n

Naturally, the question is: Can we hope to one day extend these hidden state privacy amplification theorems to cover non-convex loss functions as well? The answer so far has been “this is extremely difficult to do” but no definitive proof or impossibility result exists... 5/n

Furthermore, all of the ways to analyze hidden state DP-SGD (aka hidden state privacy amplification) place restrictions on the loss function (e.g., strongly convex, smooth, linear) that are not satisfied in practice, which typically relies on non-convex loss functions. 4/n

Privacy analyses of DP-SGD typically assume that all intermediate models are released, even though in practice only the final model is released (aka hidden state). This is not ideal, but there are limited known ways to theoretically analyze the hidden state privacy leakage. 3/n

TLDR: Can we eventually extend hidden state privacy amplification theorems for DP-SGD to cover all (possibly non-convex) loss functions in general? No we cannot. 2/n

I’m pleased to announce that our paper “It’s Our Loss: No Privacy Amplification for Hidden State DP-SGD With Non-Convex Loss” has been accepted to AISec ‘24 (co-located with CCS). Paper: https://t.co/kfCC2N5iql. 1/n

Today we publish our new pre-print experimenting with tight auditing of differentially private synthetic data generation (DP-SDG) algorithms. Work by @sundarmsa and @ganevgv, to appear at Usenix Security'24. 1/ https://t.co/b6uxAsidBw

I am honoured to have presented 2 papers at USENIX Security 2024 (one with @AndreaGadotti + @cynddl and the other with @ganevgv + @EmilianoDeCristofaro). Met lots of great people, had lots of great conversations, and made a lot of great memories! ✌🏽 📸 pc: @matthieu_meeus

Tomorrow at #usesec24 @sundarmsa will present two papers: 1) A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Data 2) "What do you want from theory alone?" Experimenting with Tight Auditing of Differentially Private Synthetic Data Generation