Emergent misalignment is a surprising and potentially concerning mode of generalization - very excited to have contributed to this work on understanding it better!.

RT @OpenAI: Understanding and preventing misalignment generalization. Recent work has shown that a language model trained to produce insecu….

RT @NeelNanda5: Excited to have supervised these papers! EM was wild, with unclear implications for safety. We answer how: there's a genera….

Nothing ever ends.

RT @HenkTillman: Latest paper from OpenAI interp team: We find that a combination of “just asking the model” and ac….

RT @_georg_lange: 📢 Accepted at #ICLR2025! . Visit our poster tomorrow morning if you wanna know how good Sparse Autoencoders (SAEs) reall….

Can't recommend working with Neel highly enough!.

RT @ArthurConmy: 🚨🚨 Less than 24 hours to apply to work with @NeelNanda5 and me! 🚨🚨.

evaluation of annotations, all aimed at making code more concise and efficient.

Claude 3.7 added several key features, including: a built-in breakpoint() function for easy debugging, nanosecond-resolution time functions, data classes for simplified data handling, the ability to define __getattr__ on modules, and improved support for type hints with postponed.

RT @NeelNanda5: Apps are open for my MATS stream, where I try to teach how to do great mech interp research. Due Feb 28!. I love mentoring….

Talk is cheap. Show me the CoT.

SaaS (Santa as a Service).

RT @NeelNanda5: Are you interested in sparse autoencoders? Are you *really* interested in sparse autoencoders? Then check out my latest 4 h….

RT @MLStreetTalk: We are dropping an epic 4 hour session with @NeelNanda5 - which I think constitutes the most ridiculously dense 4 hour br….

Despite failing to give a complete proof, I'd count this as a major improvement over other models' attempts. Most importantly, the model engaged directly with the key steps necessary for a full proof. I essentially consider this problem "solved by LLMs" now!.

@OpenAI In reality, you need to pick at least 18,003 instead of 18,000 (lol), and a precise calculation gives the average number of representations is at least (18003 choose 3) / (3*18003^2) = 1000.000006. You could go up to 18257 before this fails.

@OpenAI Finally, it realizes and tries to fix the off-by-a-factor-of-6 issue. It writes a little essay giving what mathematicians would call a "moral" argument for why everything is OK. Pretty close!

@OpenAI Then, it counts these triples. Unfortunately, it counts the number of ordered triples, which overestimates the number of unordered triples (what we care about) by about a factor of 6. Then it proceeds to the key step - lower-bound the average number of representations: