We discovered how to fix diffusion model's diversity issues using interpretability! It's all in the first time-step!⏱️ Turns out the concepts to be diverse are present in the model - it simply doesn't use them! Checkout our @wacv_official work - we added theoretical evidence👇

We tested several unlearning methods and found none of them really erase knowledge from the model - they simply hide it! 🧐 What does this mean? We must tread carefully with unlearning research within diffusion models🚨 Here is what we learned 🧵👇(led by @kevinlu4588)

work done with @NickyDCFP, @rohitgandikota, @mnphamx1, @davidbau, @chegday and @CohNiv @Northeastern @nyuniversity Code: https://t.co/LGmltXapLB Paper:

Probes include: 1. Inpainting & diffusion completion 2. Diffusion trajectory expansion via custom noise scheduler 3. Latent classifier guidance (on par with Textual Inversion) 4. Dynamic concept tracing Erasing methods can fail with the original prompt + light conditioning!

Excited to share our paper “When Are Concepts Erased from Diffusion Models?” at @NeurIPSConf! We introduce two conceptual models for erasure mechanisms in diffusion models, and a suite of probes to recover supposedly forgotten concepts. Project website:

How can a language model find the veggies in a menu? New pre-print where we investigate the internal mechanisms of LLMs when filtering on a list of options. Spoiler: turns out LLMs use strategies surprisingly similar to functional programming (think "filter" from python)! 🧵

[📄] Are LLMs mindless token-shifters, or do they build meaningful representations of language? We study how LLMs copy text in-context, and physically separate out two types of induction heads: token heads, which copy literal tokens, and concept heads, which copy word meanings.

Curious what your model really knows? Try probing it 🕵️‍♀️ Code: https://t.co/bV8fJBKkA0 Project: https://t.co/udVQIqYXhZ Paper: https://t.co/MEYtdQdQqu work done with Nicky Kriplani, @rohitgandikota, @mnphamx1, @davidbau, @chegday and @CohNiv @Northeastern @nyuniversity

We also find a deep trade-off: Robust methods (destruction-based🧨) tend to distort unrelated generations. Understanding this helps researchers choose or design erasure methods that fit their needs.

Surprisingly even “strong” erasure methods that beat adversarial prompts… .... can still regenerate the erased concept via inpainting or noise-based probes 😳 Our findings show: No single probe is enough. True erasure requires a holistic evaluation.

Do these robust unlearning methods still somehow leave some knowledge traces? We propose a suite of probing methods to test this: 🎯 Adversarial attacks 🖼️ Inpainting 🌫️ Diffusion trajectory expansion 📉 Dynamic concept tracing Each reveals different residual traces 👀

Similarly building on this finding that erased knowledge can be resurfaced through optimization, @koushik_srivats et al. proposed STEREO, which exhaustively searches and removes knowledge traces inside an erased diffusion model. https://t.co/mNxc3jZn4Q

@mnphamx1 et al. found that erased concepts can be resurfaced by text inversion ( @RinonGal et al.), by optimizing the text input using a few images of the erased concept. They proposed task vectors as a solution for robust unlearning. https://t.co/rS3JHr7ajE

Our work evaluates several concept erasure methods. ESD is a self-guided erasure method that uses the model's own knowledge of a concept to ablate it. By design, ESD directs the model to steer away from the concept’s representations. https://t.co/imCf3x7GRU

When we "erase" a concept from a diffusion model, is that knowledge truly gone? 🤔 We investigated, and the answer is often 'no'! Using simple probing techniques, the knowledge traces of the erased concept can be easily resurfaced 🔍 Here is what we learned 🧵👇

Our work evaluates several concept erasure methods. ESD is a self-guided erasure method that uses the model's own knowledge of a concept to ablate it. By design, ESD directs the model to steer away from the concept’s representations. https://t.co/imCf3x7GRU