RL with verifiable rewards? Works great ✨ Realistic or non-verifiable tasks? Still a mess 📉 Reward models and AI judges? Fragile and inconsistent 💔 Our proposal? RL from Checklist Feedback 📜 https://t.co/fRYqiiDBQP 👇

Can LLMs accurately aggregate information over long, information-dense texts? Not yet… We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!

This shouldn't be controversial. Science requires sharing with the public. If you're never sharing your research, you're not a research scientist. I don't think you have to share it via peer review, but vague musings on Twitter or on a podcast definitely don't count as science

We just built and released the largest dataset for supervised fine-tuning of agentic LMs, 1.27M trajectories (~36B tokens)! Up until now, large-scale SFT for agents is rare - not for lack of data, but because of fragmentation across heterogeneous formats, tools, and interfaces.

Agents are joining us at work -- coding, writing, design. But how do they actually work, especially compared to humans? Their workflows tell a different story: They code everything, slow down human flows, and deliver low-quality work fast. Yet when teamed with humans, they shine

Introducing 𝐃𝐮𝐚𝐥-𝐖𝐞𝐢𝐠𝐡𝐭𝐞𝐝 𝐑𝐞𝐢𝐧𝐟𝐨𝐫𝐜𝐞𝐦𝐞𝐧𝐭 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 (𝐃𝐖𝐑𝐋) — a new framework for 𝐂𝐨𝐓 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐨𝐧 𝐩𝐫𝐞𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐝𝐚𝐭𝐚, working @AIatMeta Key ideas: ❌ Transform preference modeling into RLVR and use GRPO ✅ Integrate the

Tinker looks awesome! And since Andrej nicely highlighted the value of finetuning a smaller LLM specifically for your narrow task, this is a great time for me to mention Prompt2Model - the library we released back in 2023 to support the same goal! https://t.co/bdohmjqxCt

Prove me wrong: anything we didn’t publish and secretly did is 10x more advanced than all the open source repos or publications.

If this statement is about algorithms, it misses the fact that finding an RL abstraction that is natural and ergonomic enough for *problem specification* is very difficult for most problems of interest. You can’t climb a hill you don’t even know how to define.

📜 Paper on new pretraining paradigm: Synthetic Bootstrapped Pretraining SBP goes beyond next-token supervision in a single document by leveraging inter-document correlations to synthesize new data for training — no teacher needed. Validation: 1T data + 3B model from scratch.🧵

(see our original twitter thread for more! https://t.co/f4X0OjNdBj)

In https://t.co/N5sHpY7qBc we showed that this exact approach can make already-strong models even better at instruction following.

Our software lets you generate rubrics, scoring responses via rubric-grounded LM judges and rubric-grounded code verifiers, and training models (currently using DPO) on these rewards: https://t.co/uMX4dDZlNT. You'll need a node with 4-8 GPUs to use this

Closed-source models like Kimi K2 are post-trained using rubrics, but these aren't available open-source for researchers. To change this, we're releasing the WildChecklists dataset ( https://t.co/wTdPKBwtRl) and code ( https://t.co/SwgmhzPVdq) from our paper on checklist-based RL!

On my way to VLDB! 🇬🇧 I am on the job market this year, seeking tenure-track CS faculty positions. I will be giving a talk on DocETL and on a panel titled “Where Does Academic Database Research Go From Here?” I would love to meet folks; please reach out if you’re also attending!

OpenAI hasn’t open-sourced a base model since GPT-2 in 2019. they recently released GPT-OSS, which is reasoning-only... or is it? turns out that underneath the surface, there is still a strong base model. so we extracted it. introducing gpt-oss-20b-base 🧵

@Yuchenj_UW They didn't evaluate on 23 of the 500 instances though, so the actual score is: 74.9 * (500 - 23) / 500 = 71.4%, which is a few points below Claude Sonnet 4.

If you're at ACL, join us for our tutorial on Synthetic Data in the Era of LLMs with @vijaytarian @xiangyue96 @yizhongwyz @gneubig!! 🕑 2pm - 5:30pm 📍 Hall B

Bring questions, stick around, and chat with us after! Details + sneak peek at our slides and content: https://t.co/6DC6VIg28K Hope to see you there!

We’ll introduce the core abstractions and algorithms of synthetic data - what are the building blocks, how to combine them, and how they show up in real LLM pipelines. @yizhongwyz has also prepared a segment on open questions in synthetic data that I can't wait to hear

We’ve prepared a tutorial for ACL in Vienna this year to give you some answers. Come join @xiangyue96, @alisawuffles, @yizhongwyz, @gneubig, and me for “Synthetic Data in the Era of LLMs.” 📍 Sunday 2–3:30pm Hall B #ACL2025