RT @janleike: If you don't train your CoTs to look nice, you could get some safety from monitoring them. This seems good to do!. But I'm s….

RT @BarackObama: At a time when people are understandably focused on the daily chaos in Washington, these articles describe the rapidly acc….

RT @AnthropicAI: Our interpretability team recently released research that traced the thoughts of a large language model. Now we’re open-s….

RT @michaelwhanna: @mntssys and I are excited to announce circuit-tracer, a library that makes circuit-finding simple!. Just type in a sent….

The stakes are high and time is short.

Of course, there are many brilliant people in AI safety. But at least for myself, there are clearly many people in math and the sciences who are much smarter than I, and I assume would do a much better job than me.

I often feel like, in some sense, humanity is failing to bring its intellectual weight to bear on AI safety, and this is a grave failure.

I admire Daniel's courage in his convictions. I'm excited to see what he (and Timaeus) do.

RT @sleepinyourhat: 🧵✨🙏 With the new Claude Opus 4, we conducted what I think is by far the most thorough pre-launch alignment assessment t….

RT @michael_nielsen: My three-sentence summary of Lakatos's "Proofs and Refutations", with apologies to Don Knuth: . "Premature definitio….

I should also mention that I wrote a blog post listing a bunch of specific analogies between deep learning and biology several years back. (It's probably of much narrower interest!).

Of course, I'd be remiss to not mention that many others have made analogies between work in machine learning and biology -- most notable for us is the "bertology" work, which framed it self as studying the biology of the BERT models.

But we also think it's important for such "biology" results (which are more foreign in style to machine learning) to be treated as worthy of publication independent of methods work (which looks more similar to normal machine learning).

This was partly a convenient way to handle the length (jointly, the two papers are ~150 pages!).

But why did the language come up in our paper title? There was actually a further reason, which is that we wanted to separate our "methods" work and what we called our "biology" work (i.e. the empirical research we did using our method).

Finally, you need to believe that a worthy mode of investigation is empirical (rather than theoretical), and a style of empirical research that's more open to the qualitative than purely quantitative. This evokes biology more than physics.

One further needs to believe that individual neural networks, and in fact sub-components of those networks, warrant investigation. That's more idiosyncratic!.

At a basic level, one needs to believe deep learning warrants scientific investigation. This doesn't seem very controversial these days, but note that it's already kind of radical. See eg. Herbert Simon's The Sciences of the Artificial.

I've written multiple papers characterizing (small sets of) individual neurons. Historically, this hasn't seemed like a worthy topic of a paper in ML – I've had to justify it!.

One way in which this is important is that the *types of questions* we're interested in are quite bizarre from a traditional machine learning perspective, but natural under the biological frame.