Hey this is me!.Our paper: Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs. Blog post:

Our oral presentation and penal discussion will take place at 09:00-10:30, Wednesday, July 30. Session 11, Panel 4: Interpretability and model analysis. Huge thanks to my mentors (i.e., coauthors) at Microsoft Research! It was a wonderful intern experience. Bis bald in Wien!.

The idea is simple:.🦙Contextual Entrainment: LLMs give higher probability to any tokens that previously appeared in context.😈Counterfactual prompts can exacerbate distraction.🤖We can identify "entrainment heads" that corresponds to the entrainment phenomenon with DiffMask.

📢 Next week, I will be presenting our paper "Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs" at ACL 2025!. Paper: Blog Post: Talk:

RT @harish: Is In-Context Learning (ICL) in LLMs Memorisation? Emergence? Some Algorithmic Capability? 🤔. 📢New work exploring ICL in LLMs:….

Blog post: Arxiv: Code:

See our #ICLR2024 Spotlight paper: What does the Knowledge Neuron Thesis have to do with Knowledge this afternoon 4:30-6:30 CEST 🎉. We find that the KN Thesis is an oversimplification. MLP weights store complex patterns but do not constitute “knowledge.”.

I will be at #ICLR2024! Feel free to come by and say hi!.I will be presenting our work: What does the Knowledge Neuron Thesis Have to do with Knowledge?.

Me and the boys chilling under a train at #ACL2023