AI agents thrive on context. But with too much context, they go off the rails. This is why we’re going to see subagents for particular tasks or roles in a workflow. And this also means there’s a ton of opportunity for building these deep, domain specific agents.

In era of pretraining, what mattered was internet text. You'd primarily want a large, diverse, high quality collection of internet documents to learn from. In era of supervised finetuning, it was conversations. Contract workers are hired to create answers for questions, a bit

*very* excited to share a new *efficient* method for learning *marginally stable* and NONLINEAR dynamical systems, w. brilliant students Evan Dogariu and Anand Brahmbhatt @AnandBrahm15501: https://t.co/gwVAaPVVst more info in thread

Are frontier AI models really capable of “PhD-level” reasoning? To answer this question, we introduce FormulaOne, a new reasoning benchmark of expert-level Dynamic Programming problems. We have curated a benchmark consisting of three tiers, in increasing complexity, which we call

It seems GPT‑OSS is very prone to hallucinations … check out our RLCR paper to see how we trained reasoning models to know what they don't know. Website 🌐 and code 💻 out today! https://t.co/YqLu92enIy 🚀

AI Research Agents for ML Achieves state-of-the-art on MLE-bench lite! Using AI to automate the training of ML models is one of the most exciting and promising areas of research today. Lots of cool ideas in this paper:

@seohong_park Yeah! We wondered the same in casual inference / offline RL. We found earliest disagreement times, a type of adaptive time scale, are a useful concept for continuous or finely discretized times. Could benefit from some large scale experiments, lots to do! https://t.co/uWC2wwG8Sx

Key to research success: ambition in vision, but pragmatism in execution. You must be guided by a long-term, ambitious goal that addresses a fundamental problem, rather than chasing incremental gains on established benchmarks. Yet, your progress should be grounded by tractable

It was a dream come true to teach the course I wish existed at the start of my PhD. We built up the algorithmic foundations of modern-day RL, imitation learning, and RLHF, going deeper than the usual "grab bag of tricks". All 25 lectures + 150 pages of notes are now public! 🧵

I now have a draft of my introduction to CTDE in (cooperative) MARL. It is meant to introduce new graduate students (that already know a bit about RL) to the area. Check it out and let me know your thoughts! https://t.co/6PyE9eMXlL

Modern scaling law research often feels like this: 1. Train a few models 2. Plot metrics on a log-log scale 3. Fit a line 4. Call it a new law Maybe it’s time to ask: are we uncovering principles, or just describing artifacts?🤔

The most rewarding aspect of scientific adventure is why we do it ? How we get there and what we achieve through understanding the governing laws of the world is less important.

Tomorrow it is time again for a great seminar! Join us to hear out one of Jeongyeol's latest findings.

@chrodan @YishayMansour @MehryarMohri tl;dr: a fun project that required us to rethink of a new framework w/ Ben Kretzu, @danielrjiang, @bhandari_jalaj, @ZheqingZhu and @karen_ullrich

we actually started by asking this question in the multi-armed / tabular RL, and after spending some time on it realized it has been explored already by @chrodan, @YishayMansour, @MehryarMohri:

accepted to #ICML25🍁 we asked 🤔 how can we improve gradient-descent in the presence of multiple aligned or similar objectives?🤔 this becomes increasingly important when having access to multiple reward functions / datasets / tasks

Meet us (but not me sadly) at the poster session: https://t.co/sJdUpeNleu #ICLR2025 (Also, much more interesting things to explore in MARL and offline MARL imo)

@eastskykang Nine days after, it walks! E pur si muove 🤖

Hiring researchers and engineers for a stealth, applied research company with a focus on RL x foundation models. Folks on the team already are leading RL / learning researchers. If you think you'd be good at the research needed to get things working in practice, email me

There are multiple postdoc positions available as part of an exciting new AI-agent initiative at Columbia that tackles challenges at the frontier of agentic systems and sequential decision-making. I am not very active here so please help me spread the word!