🥳 Excited to announce that our work “Language Guided Exploration for RL Agents in Text Environments” was accepted into #NAACL2024 Findings! 🚀 .Paper: 🧵(1/n).

RT @HannaHajishirzi: Excited to release our newest, largest, and best Tulu yet. Our RLVR recipe works at scale, outperforming Deepseek V3.….

RT @natolambert: The DeepSeek R1 recipe, what questions we need to answer to train an o1 replication ourselves at home, and what it means f….

RT @SonglinYang4: I've created slides for those curious about the recent rapid progress in linear attention: from linear attention to Light….

RT @cgarciae88: Google Cloud just recently released "The PyTorch developer's guide to JAX fundamentals". Contains a side-by-side implement….

RT @nrehiew_: How to train a 670B parameter model. Let's talk about the DeepSeek v3 report + some comparisons with what Meta did with Lla….

RT @cocoweixu: We wrapped up CS 8803 "Large Language Model" class at @GeorgiaTech for Fall 2024. Here is the reading list:. • learning fr….

RT @natolambert: I've spent the last two years scouring all available resources on RLHF specifically and post training broadly. Today, with….

RT @simon_jegou: 🚀 Excited to announce KVPress — our open-source library for efficient #LLM KV cache compression!.👉 Check it out (and drop….

RT @currying: Around ten years ago, I started studying inverse problems in Topological Data Analysis (TDA). For decades people in computati….

RT @jxmnop: Top-rated papers from ICLR 2025. Scaling In-the-Wild Training for Diffusion-based Illumination Harmonization and Editing by Imp….

RT @abeirami: RLHF provably can't teach models any new knowledge. If you need to teach new skills, you need to look at pre-training and SFT….

RT @jaseweston: 🚨 Self-Consistency Preference Optimization (ScPO)🚨.- New self-training method without human labels - learn to make the mode….

RT @kayembruno: Diffusion models are so ubiquitous, but it's difficult to find an introduction that is concise, simple and comprehensive.….

RT @wellingmax: “We are on the brink of an irreversible climate disaster. This is a global emergency beyond any doubt. Much of the very fab….

RT @srush_nlp: If you know Torch, I think you can code for GPU now with OpenAI's Triton language. We made some puzzles to help you rewire….

The work was done during my Masters degree with super amazing co-authors - @sahil_yerawar, @_dhruveshp from IESL Lab @UMass_NLP at UMass Amherst and Soham Dan, @keerthi166 from @IBMResearch! Congratulations to all! 🎉 🧵(6/n).

The Guide is then used to prune ✂️ the action space while training the Explorer 🔭. Its invocation is stochastic 🎲(controlled using ε), and can be constant or decaying (in curriculum learning fashion). We used DRRN, but you can plug in your favorite RL/LLM Explorer. 🧵(5/n).

The Guide 🔎 is first trained to learn an embedding space where relevant actions are close to task instructions. This is done using supervised contrastive learning with task instructions as anchors ⚓️, gold actions as positives ➕and available actions as hard negatives. 🧵(4/n).

We outperform all RL based baselines including DRRN, TDT, CALM, Behavior Cloning in the challenging ScienceWorld benchmarks 🏆, which have 30 interactive environments to test scientific reasoning abilities with thousands of variations divided into train/dev/test sets. 🧵(3/n)

Text-based games are riddled with sparse rewards 🥕 and large action spaces 🕹️. We impart common sense 🧠to the agent (called Explorer 🔭) via a contrastively trained Language Prior (called Guide 🔎) which uses task instructions to significantly prune ✂️ the action space. 🧵(2/n)