RT @nickhjiang: What makes LLMs like Grok-4 unique?. We use sparse autoencoders (SAEs) to tackle queries like these and apply them to four….

RT @HCasademunt: Problem: Train LLM on insecure code → it becomes broadly misaligned.Solution: Add safety data? What if you can't?. Use int….

Join millions who have switched to Grok.

RT @_jake_ward: Do reasoning models like DeepSeek R1 learn their behavior from scratch? No! In our new paper, we extract steering vectors f….

RT @balesni: A simple AGI safety technique: AI’s thoughts are in plain English, just read them. We know it works, with OK (not perfect) tra….

RT @emmons_scott: Is CoT monitoring a lost cause due to unfaithfulness? 🤔. We say no. The key is the complexity of the bad behavior. When w….

Code & Datasets: Demo Notebook: Paper: Work done with my awesome collaborators @IvanArcus @ArthurConmy @NeelNanda5 @philiptorr as part of the @MATSprogram.

But wait, there's more! Our follow-up work (coming soon) uses SAEs + clustering to build a principled and complete taxonomy of reasoning behaviors and provides new insights into the difference between reasoning- and base models. Stay tuned!.

Where do these behaviors live?. We use attribution patching to find in which layer of the model steering vectors are most effective, and find that there are clear peaks in the middle layers.

How do we find these vectors?. First, annotate reasoning chains with GPT-4o, then average the internal activations for each behavior across its annotated text positions. Lastly, subtract the overall mean; and that’s your steering vector. Simple and surprisingly effective.

We tested this across 500 tasks in 10 categories using three DeepSeek-R1-Distill models (Qwen-14B, Qwen-1.5B & Llama-8B). The steering effects are consistent and robust; you can literally dial up or down specific reasoning behaviors at inference time 🚀.

The key insight: reasoning behaviors like backtracking and knowledge addition live in linear directions in the model's activation space. Want more backtracking? Add the vector. Less? Subtract it. Simple, interpretable, and it works!

Can we actually control reasoning behaviors in thinking LLMs?. Our @iclr_conf workshop paper is out! 🎉. We show how to steer DeepSeek-R1-Distill’s reasoning: make it backtrack, add knowledge, test examples. Just by adding steering vectors to its activations!.Details in 🧵👇

Huge thanks to collaborators @NeelNanda5 @soniajoseph_ @philiptorr @ashk__on for making this happen!. Paper: Code & weights:

💡 Summary: Visual representations align with language features in middle-to-late layers, not early ones. This raises key questions: How important is LLM feature "initialization" for VLM performance? Are current adapter architectures optimal for cross-modal learning?. (6/6).

📊 Key finding: Reconstruction error drops, sparsity decreases, AND semantic alignment between SAE features and visual content sharply increases around layer 18/26! .Visual-linguistic alignment happens in middle-to-late layers, not early ones. (5/6)

🔍 Using pre-trained sparse autoencoders as analytical probes, we tracked how visual representations flow through the language model layers. We measured SAE reconstruction error and sparsity patterns to detect when visual features align with language representations. (4/6)

✅ Despite this constraint, our VLM (CLIP-Gemma-2-2b-it) performs on par with LLaVA-v1, showing that the LLM's feature space already contains good representations for visual instruction tuning - the adapter just needs to align them effectively. (3/6)

🔬 Our approach: We deliberately froze both the vision transformer (CLIP ViT) AND the language model (Gemma-2-2B-it), training only a linear adapter between them. This forced the adapter to map visual features directly into the LLM's existing feature space. (2/6)

🔍 New paper: How do vision-language models actually align visual- and language representations?. We used sparse autoencoders to peek inside VLMs and found something surprising about when and where cross-modal alignment happens!. Presented at XAI4CV Workshop @ CVPR. 🧵 (1/6)

RT @MarroSamuele: LLMs are continuous models, but language is discrete. What happens when a continuous model approximates a discrete sequen….