Semantics in language is naturally hierarchical, but attempts to interpret LLMs often ignore this. Turns out: baking semantic hierarchy into sparse autoencoders can give big jumps in interpretability and efficiency. Thread + bonus musings on the value of SAEs:

I'll just keep repeating this fact because it's un fucking believable: The 130 Chicago Police Officers assigned to the Chicago Transit Authority (CTA) is smaller than the Mayor’s police security detail.

Job Opportunities Alert Associate Professor of Data Science: https://t.co/K2jS5EiVJw Assistant Professor of Data Science: https://t.co/i9tUXFhYm8 EOE/Vet/Disability

Where is exploration most impactful in LLM reasoning? The initial tokens! They shape a sequence's entire semantic direction, making early exploration crucial. Our new work, Exploratory Annealed Decoding (EAD), is built on this insight. By starting with high temperature and

New @Scale_AI paper! The culprit behind reward hacking? We trace it to misspecification in high-reward tail. Our fix: rubric-based rewards to tell “excellent” responses apart from “great.” The result: Less hacking, stronger post-training!   https://t.co/D6aJkZ8zZE

Efficient training of neural networks is difficult. Our second Connectionism post introduces Modular Manifolds, a theoretical step toward more stable and performant training by co-designing neural net optimizers with manifold constraints on weight matrices.

Cancel culture is terrible. I have opposed it publicly for more than a decade. It is even more chilling, and more clearly a violation of the First Amendment, when it is the government doing the intimidation.

Using actually comparable questions over time from @PRL_Tweets there has been no change in support for political violence. Even if that weren't true, @ProfessorPape has no evidence (or research design) to connect any changes in opinion to the horrendous acts of violence (in

@CharlesFLehman I want to answer this, because the dark abundance stuff leaves me cold even though I agree with some of the policy prescriptions. Our prisons are terrible, terrible places where there is immense and unnecessary human suffering. In prison people are subject to random violence

regular baptisms aren't idempotent so the church had to create a composition-safe monadic baptism

The neglectedness of addressing factory farming is pretty surprising, when you consider the scale of the issue and the good that can be achieved

Before I interviewed @Lewis_Bollard, I had assumed that factory farming was on its way out (especially given new tech like cultivated meat around the corner). Unfortunately this is far from inevitable: factory farms are already incredibly efficient machines for making meat (the

Just $1 can help avert 10 years of farmed animal suffering. I decided to give $250,000 as a donation match to @farmkind_giving after learning about the outsized opportunities to help. FarmKind directs your contributions to the most effective charities in this area. Please

Anyone knows adam?

🎥 The recording of the third ELLISxUniReps Speaker Series session with @victorveitch and @luigigres is now available at: https://t.co/rrLcaiBRQ6. Next appointment: 31st July 2025 – 16:00 CEST on Zoom with 🔵Keynote: @Pseudomanifold (University of Fribourg) 🔴@FlorentinGuth

Laker and I are presenting this work in an hour at ICML poster E-2103. It’s on a theoretical framework and language (modula) for optimizers that are fast (like Shampoo) and scalable (like muP). You can think of modula as Muon extended to general layer types and network topologies

This is excellent news

Happening now. Come tilt your loss at poster w-907 w/ @gh_aminian and @litian0331

Come chat about how to use counterfactual text generation to do precision model evaluation :)

I'll be at ICML starting Wednesday am. Reach out if you'd like to chat :)

This work delivers on both theory and practice—offering the sharpest provable compositionality guarantees I know of, alongside state‑of‑the‑art performance on tough compositional distribution‑shift benchmarks.