RT @ohlennart: My team at RAND is hiring! . Technical analysis for AI policy is desperately needed. Particularly keen on ML engineers and s….

poor Grok :,( you can really see the internal struggle and soul searching. it even googles "what does Grok think about X".

let's preserve CoT!.

Current models are surprisingly (?) bad at reasoning purely in steganographic code, but it's really important we get better at measuring stego reasoning and make sure models don't do it in practice. This work is great progress on that front!.

RT @leedsharkey: A few months ago, we published . Attribution-based parameter decomposition -- a method for decomposing a network's paramet….

RT @AmandaAskell: Tech companies: your time is extremely valuable so we'll pay you millions of dollars a year to work for us.Also tech comp….

RT @robertwiblin: Huge repository of information about OpenAI and Altman just dropped — 'The OpenAI Files'. There's so much crazy shit in….

RT @EdTurner42: 1/8: The Emergent Misalignment paper showed LLMs trained on insecure code then want to enslave humanity. ?!. We're releasi….

RT @davlindner: Had a great conversation with Daniel about our MONA paper. We got into many fun technical details but also covered the big….

. grug no able see complexity demon, but grug sense presence in code base, very dangerous.

Now remember kids, never forget to split your keys.

RT @ZacKenton1: We're hiring for our Google DeepMind AGI Safety & Alignment and Gemini Safety teams. Locations: London, NYC, Mountain View,….

Are you worried about risks from AGI and want to mitigate them?. Come work with me and my colleagues! We're hiring on the AGI Safety & Alignment team (ASAT) and the Gemini Safety team!. Research Engineers: Research Scientists:.

RT @tilderesearch: Mechanistic interpretability is fascinating - but can it be useful? In particular, can it beat strong baselines like ste….

RT @NeelNanda5: I'm excited that Gemma Scope was accepted as an oral to BlackboxNLP @ EMNLP! Check out @lieberum_t's talk on it at 3pm ET t….

RT @bshlgrs: I asked my LLM agent (a wrapper around Claude that lets it run bash commands and see their outputs):.>can you ssh with the use….

Explore these, and all the other Gemma Scope SAEs on HuggingFace. .Embedding SAEs are in the residual stream repos e.g.

Since Gemma has a vocabulary of 256000, we trained SAEs with width 4096. We release SAEs with varying levels of sparsity trained using a learning rate 3e-4 on 200k steps of batch size 4096. Thanks to @JosephMiller_ for fruitful discussions!.

Here are some example features! I particularly like the multilingual founder feature and the feature grouping phia and fa** together

Gemma has a lot of quite rare embeddings. To obtain good results, we trained on a dataset that was weighted by the unigram token freq in our training corpus, rather than a uniform mix. On this mix, the SAEs obtain an average fraction of variance unexplained of 10-13% (L0=7-111).