New paper alert 🚨 What if I told you there is an architecture that provides a _knob_ to control quality-efficiency trade-offs directly at test-time? Introducing Compress & Attend Transformers (CATs) that provide you exactly this! 🧵(1/n) 👇

1/3 🥳Excited to share our new paper ‘Simplicial Embeddings Improve Sample Efficiency in Actor–Critic Agents’! Project your features onto a product of simplices → sparse, stable reps, stronger grads, faster learning. 🧵For more details, check out Pablo’s thread 👇

Introducing linear scaling of reasoning: 𝐓𝐡𝐞 𝐌𝐚𝐫𝐤𝐨𝐯𝐢𝐚𝐧 𝐓𝐡𝐢𝐧𝐤𝐞𝐫 Reformulate RL so thinking scales 𝐎(𝐧) 𝐜𝐨𝐦𝐩𝐮𝐭𝐞, not O(n^2), with O(1) 𝐦𝐞𝐦𝐨𝐫𝐲, architecture-agnostic. Train R1-1.5B into a markovian thinker with 96K thought budget, ~2X accuracy 🧵

Work done with my amazing collaborator @bicycleman15 . Excited to hear your thoughts! (4/4)

Which easy examples you add matters. Trivial ones don’t help much. But you don’t need to hunt for “perfect difficulty.” Mixing all the easier instances you have usually works fine. We’re releasing our hackable implementations at https://t.co/Mwge7elqnC. Check it out 🙂 (3/n)

We test this on the graph-search task from Bachmann et al. (2024). Dense rewards, diversity incentives, and improved credit assignment all underperform in our setting when the base model fails to sample correct answers. Mixing in easier instances helps unlock RL training. (2/n)

Zero rewards after tons of RL training? 😞 Before using dense rewards or incentivizing exploration, try changing the data. Adding easier instances of the task can unlock RL training. 🔓📈To know more checkout our blog post here: https://t.co/BPErVcLmP8. Keep reading 🧵(1/n)

Thrilled to share our new work EARL 🚀 1⃣ An AR + RL image editing model that outperforms diffusion baselines w/ 5x less data. 2⃣ First systematic SFT vs RL study in image editing → RL post-training shines on complex edits where paired data is scarce. See thread for details👇

Introducing a framework for end-to-end discovery of data structures—no predefined algorithms or hand-tuning needed. Work led by Omar Salemohamed. More details below. https://t.co/lFb2kn2NpE