First-time NeurIPS attendee here! Super excited to talk about our paper with @yveitsman, @mhahn29 and to discover the amazing work by everyone else :D https://t.co/rt0peAA0SR

Had great fun at @dagstuhl this week! 😍 Thanks for the invite and interesting discussions! @JiUngLee1 @WolfStammer @SoyoungOh5 @SwetaMahajan1 Aleksandra Bakalova I talked about parameter-grounded units for mech interp, covering our recent work on MAPS and SNMF:

We’re hiring PhD students and postdocs on LLM theory and interpretability! Topics: 1️⃣ abilities & limitations of transformers and other architectures; 2️⃣ LLM interpretability; 3️⃣ foundations of LLM reasoning; 4️⃣ foundations of AI safety.

Is there such a thing as too many agents in multi-agent systems? It depends! 🧵 Our work reveals 3 distinct regimes where communication patterns differ dramatically. More on our findings below 👇 (1/7)

Got awarded Top Reviewer at NeurIPS 2025 @NeurIPSConf. Really gratifying to get this at my first attempt at Reviewing 😃

Read the cookbook: https://t.co/ymBPgfwGxa Join us for weekly seminars on formal language theory, ML, NLP, and more:

Announcing the first workshop on Foundations of Language Model Reasoning (FoRLM) at NeurIPS 2025! 📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies. 📆 Deadline: Sept 3, 2025

Do LLMs store information in interpretable subspaces -- similar to variables in a program? In our new paper, we decompose representation space into smaller, interpretable, non-basis-aligned subspaces with unsupervised learning.

New on arXiv: Knee-Deep in C-RASP, by @pentagonalize, Michael Cadilhac and me. The solid stepped line is our theoretical prediction based on what problems C-RASP can solve, and the numbers/colors are what transformers (no position embedding) can learn.

How do architectural limitations of Transformers manifest after pretraining?

Can language models learn implicit reasoning without chain-of-thought? Our new paper shows: Yes, LMs can learn k-hop reasoning; however, it comes at the cost of an exponential increase in training data and linear growth in model depth as k increases. https://t.co/1CGTRJGnGQ

1/11 Solving tasks with Chain-of-Thought reasoning is incredibly effective, but generating long CoTs is pricey. So we want them to be as short as possible to solve the task. But wait – how short is that? In our new ICML paper, we aim to answer that question.

Fantastic thread by my colleague & dear friend @abakalova13175 diving into how LLMs perform in-context learning, based on her new preprint By tracing information flow, she reveals how LLMs dynamically handle examples via a "Contextualize, then Aggregate" process. Worth the read!

How do LLMs perform in-context learning? LLMs can infer tasks from just a few examples in prompts - but how? Our new preprint proposes a two-step mechanism: first contextualize, then aggregate. 🧵

Today we are launching a server dedicated to Tokenization research! Come join us! https://t.co/Jor3G6VFcC

LLMs can now track states, finally matching this cat! And we prove it. But how? 🧵👇 1/ Paper: https://t.co/aKvrqYtkWh with @julien_siems @jkhfranke @ZelaArber  @FrankRHutter   @MPontil

When do transformers length-generalize? Generalizing to sequences longer than seen during training is a key challenge for transformers. Some tasks see success, others fail — but *why*? We introduce a theoretical framework to understand and predict length generalization.

Excited and honored to receive a Best Paper Award for our work on the inductive bias of the transformer architecture with @broccolitwit 🌟🎉 #ACL2024NLP

Discover more such exciting projects at our lab, LaCoCo: https://t.co/3utwc5fDh5.

For a given finite-state problem, one can decidable check whether it falls into this class. Such decidable characterizations may make it easier to theoretically predict abilities and anticipate failures of LLMs.