1\ Hi, can I get an unsupervised sparse autoencoder for steering, please? I only have unlabeled data varying across multiple unknown concepts. Oh, and make sure it learns the same features each time! Yes! A freshly brewed Sparse Shift Autoencoder (SSAE) coming right up. 🧶

[1/9] While pretraining data might be hitting a wall, novel methods for modeling it are just getting started! We introduce future summary prediction (FSP), where the model predicts future sequence embeddings to reduce teacher forcing & shortcut learning. 📌Predict a learned

I’m at CoLM this week! Come check out our work on evaluating RMs for agent trajectories! These days, I’m thinking about forecasting generalization, scaling laws, and safety/adversarial attacks. Ping me if you wanna chat about research!

^on Saturday, 19th July.

I will be at the Actionable Interpretability Workshop (@ActInterp, #ICML) presenting *SSAEs* in the East Ballroom A from 1-2pm. Drop by (or send a DM) to chat about (actionable) interpretability, (actionable) identifiability, and everything in between!

🚨 New Paper! 🚨 Guard models slow, language-specific, and modality-limited? Meet OmniGuard that detects harmful prompts across multiple languages & modalities all using one approach with SOTA performance in all 3 modalities!! while being 120X faster 🚀 https://t.co/r6DGPDfwle

⚡⚡ Llama-Nemotron-Ultra-253B just dropped: our most advanced open reasoning model 🧵👇

𝐓𝐡𝐨𝐮𝐠𝐡𝐭𝐨𝐥𝐨𝐠𝐲 paper is out! 🔥🐋 We study the reasoning chains of DeepSeek-R1 across a variety of tasks and settings and find several surprising and interesting phenomena! Incredible effort by the entire team! 🌐: https://t.co/CDlFHD28xQ

Presenting ✨ 𝐂𝐇𝐀𝐒𝐄: 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐧𝐠 𝐜𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐢𝐧𝐠 𝐬𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 𝐝𝐚𝐭𝐚 𝐟𝐨𝐫 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 ✨ Work w/ fantastic advisors @DBahdanau and @sivareddyg Thread 🧵:

Curious to find out more? Check out our pre-print at: https://t.co/iKlCxwNhgC. Work done with an amazing set of researchers: @andrea_dittadi, @seblachap, and @dhanya_sridhar!

5\ So, does it actually work? We show that SSAE accurately steers embeddings on both semi-synthetic and real-world datasets (like TruthfulQA) using Llama-3.1-8B, handling in- and out-of-distribution data with ease.

4\ What does this mean for steering? You get access to steering vectors for individual concepts, such that each vector consistently steers only a single concept, and can be scaled according to the context.

3\ With sufficiently diverse data (such as in the real world), SSAEs remain identifiable up to permutation and scaling— repeated runs yield consistent representations, differing only by trivial indeterminacies.

2\ The SSAE is designed to map the difference between text embeddings (varying across multiple unknown concepts) to a sparse representation. Unlike standard SAEs, which impose sparsity on the concept representations themselves, we focus on sparsity of the shifts to the concepts.

📣 📣 📣 Our new paper investigates the question of how many images 🖼️ of a concept are required by a diffusion model 🤖 to imitate it. This question is critical for understanding and mitigating the copyright and privacy infringements of these models! https://t.co/bvdVU1M0Hh

🚨NEW PAPER OUT 🚨 Excited to share our latest research initiative on in-context learning and meta-learning through the lens of Information theory !🧠 🔗 https://t.co/Tj5cYudDwy Check out our insights and empirical experiments! 🔍

I am thrilled to announce that I will be joining the Gatsby Computational Neuroscience Unit at UCL as a Lecturer (Assistant Professor) in Feb 2025! Looking forward to working with the exceptional talent at @GatsbyUCL on cutting-edge problems in deep learning and causality.

Presenting tomorrow at #NAACL2024: 𝐶𝑎𝑛 𝐿𝐿𝑀𝑠 𝑖𝑛-𝑐𝑜𝑛𝑡𝑒𝑥𝑡 𝑙𝑒𝑎𝑟𝑛 𝑡𝑜 𝑢𝑠𝑒 𝑛𝑒𝑤 𝑝𝑟𝑜𝑔𝑟𝑎𝑚𝑚𝑖𝑛𝑔 𝑙𝑖𝑏𝑟𝑎𝑟𝑖𝑒𝑠 𝑎𝑛𝑑 𝑙𝑎𝑛𝑔𝑢𝑎𝑔𝑒𝑠? 𝑌𝑒𝑠. 𝐾𝑖𝑛𝑑 𝑜𝑓. Internship @allen_ai work with @pdasigi and my advisors @DBahdanau and @sivareddyg.

Adversarial Triggers For LLMs Are 𝗡𝗢𝗧 𝗨𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹!😲 It is believed that adversarial triggers that jailbreak a model transfer universally to other models. But we show triggers don't reliably transfer, especially to RLHF/DPO models. Paper: https://t.co/nRdw2h1rgS

📢 Exciting new work on AI safety! Do adversarial triggers transfer universally across models (as has been claimed)? 𝗡𝗼. Are models aligned by supervised fine-tuning safe against adversarial triggers? 𝗡𝗼. RLHF and DPO are far better!