We've made progress in our quest to understand how Claude and models like it think!. The paper has many fun and surprising case studies, that anyone who is interested in LLMs would enjoy. Check out the video below for an example.

Join millions who have switched to Grok.

Friday we published our no-jargon explainer of how LLMs "think". It's been great to see the response - already at 80k views!. We covered why models hallucinate, why they flatter users, and if they are just glorified autocomplete!. If you are curious about LLMs, check it out! 👇

If you're curious about the broader research landscape in the field of interpretability, and would like to dive into the weeds, experts from across the field came together to write this (more technical) overview.

In it, we discuss common mental models and where they fall short. For example, thinking of LLMs as next token predictors or as search engines over the training data isn't quite right!. Watch until the end to hear the answer to the real question: do models think like humans?.

Think you know how LLMs work? Think again!. Today, most people have the wrong mental models about LLMs. But we do know *some* things about how they work!. So we made a video discussing what we know in a conversational format, for anyone who is curious.

RT @ch402: Our interpretability team is planning to mentor more fellows this cycle!. Applications are due Aug 17.

RT @ludwigABAP: The "Circuit Analysis Research Landscape" for August 2025 is out and is an interesting read on "the landscape of interpreta….

Researchers from Goodfire, Google DeepMind, Decode, Eleuther, and Anthropic wrote a post about tracing circuits in language models!. We cover how to train replacement models and compute graphs of model internals, and even filmed a 2-hour walkthrough of interpreting some examples!.

RT @GoodfireAI: New research with coauthors at @Anthropic, @GoogleDeepMind, @AiEleuther, and @decode_research! We expand on and open-source….

In which the gang (@RunjinChen, @andyarditi, @Jack_W_Lindsey ):. - identifies vectors for bad personas (evil, sycophancy, hallucinations, etc).- shows that if you inject the bad vectors in training, the model learns to not do the bad thing!!. aka vaccines but for LLMs.

It turns out these heads are used in a variety of domains, like simple arithmetic prompts. There are more examples and details in the writeup, including investigating multiple choice circuits and induction.

This lets us do pretty targeted interventions. We can swap in the yellow feature for a red feature and see the model change its prediction!. Importantly, we only need to intervene on the input to the heads (not the residual stream), and only for the heads we identified)

One set of heads attends when the attribute matches the object, the other when it doesn't! . When a given head attends, it writes to the corresponding feature, and the model uses that feature to decide its output

So how does the model know that one statement is plausible, and the other is discordant?. It turns out that there are "concordance heads" and "discordance heads" which attend from attributes on the query side (yellow) to object on the key side (banana)

The model has incorrect/correct answer which activate over the respective sentences. Where do these features come from?. It turns out they originate from these slightly more nuanced features about things that are either plausible, or discordant ("the bathroom is for eating")

Discordance heads!. How does the model decide if something is true?. If you take a simple example like the one below, where you ask if a banana is yellow or red, some interesting features show up

A key component of transformers is attention, which directs the flow of information from one token to another, and connects features. In this work, we explain attention patterns by decomposing them into a list of feature/feature interactions. We find neat things, for example

Earlier this year, we showed a method to interpret the intermediate steps a model takes to produce an answer. But we were missing a key bit of information: explaining why the model attends to specific concepts. Today, we do just that 🧵

RT @claudeai: You're absolutely right.

Lots of good research came from the first iteration of the program, including an open source mech interp library to trace circuits (. Recommend applying if you're interested!.