Happy to share our new preprint for my MIT-IBM internship project! https://t.co/EBDWBaDOIb In a controlled synthetic pretraining setup, we uncover a surprising twist in scaling laws: Bigger models can hurt reasoning.

We're looking for 2 interns for Summer 2026 at the MIT-IBM Watson AI Lab Foundation Models Team. Work on RL environments, enterprise benchmarks, model architecture, efficient training and finetuning, and more! Apply here:

This fall, we're welcoming 8 new postdocs! From reinforcement learning to human-AI collaboration, their work will power forward our initiatives. Meet them and learn more about their research: https://t.co/tfZwpgj72i

✈️Currently attending ACL #ACL2025 in Vienna, Austria. Will present at In-Person at Hall 4/5 (July 30, 10:30 - 12:00): 🚩Gödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement Come and say hi!

Thrilled Gödel Agent got noticed by Sakana AI & excited for their expansion! Accepted at ACL 2025, our 1st fully self-referential agent can read & modify its entire logic (even that logic). Done via recursion. Paper:

Disentangling Memory and Reasoning in LLMs (ACL 2025 Main) We propose a new inference paradigm that separates memory from reasoning in LLMs using two simple tokens: ⟨memory⟩ and ⟨reason⟩. ✅ Improves accuracy ✅ Enhances interpretability 📄 Read: https://t.co/v3wBFw1ZEA #LLM

I'm presenting our mem v.s. gen paper at ICLR on Saturday morning at #244. Come and check it out if you're interested!

I’m attending #ICLR in Singapore! Also excited to share that I’m joining the Princeton Language and Intelligence Lab as a postdoc in July. In Fall 2026, I’ll be starting as an Assistant Professor at the University at Buffalo. I’ll be recruiting—feel free to reach out and chat!

A contrarian result I like a lot: smaller language models perform better on knowledge graphs than larger ones, as "overparameterization can impair reasoning due to excessive memorization".

Do Larger Language Models Imply Better Reasoning? A Pretraining Scaling Law for Reasoning LLMs trained on synthetic multihop graphs show a U-shaped curve in reasoning: too small underfit, too large overfit. Overparameterization hurts edge completion due to memorization. A linear

Do Larger Language Models Imply Better Reasoning? A Pretraining Scaling Law for Reasoning On a synthetic multihop reasoning environment designed to closely replicate the structure and distribution of real-world large-scale knowledge graphs, the authors observe that

This project has taken a bit too long due to my job search. Many thanks to the support of my amazing collaborators @tanshawn @fnruji316625 @WilliamWangNLP @rpanda89 @Yikang_Shen !

We systematically vary graph size, structure, and training steps. Key findings: 1. Each graph has an optimal model size. 2. This size is stable across (large) training steps. 3. Optimal model size grows linearly with graph search entropy.

Our task: complete missing edges of the knowledge graph that the language model is pretrained on—a proxy for (implicit) multi-hop reasoning natually emerge from connected knowledge. We find that the reasoning loss follows a U-shaped curve as model size increases.

We pretrain LLMs from scratch on triples of synthetic knowledge graphs designed to mimic the real-world knowledge distribution and the multi-hop reasoning structure. This gives us a clean testbed to study reasoning.

🚀Thrilled to introduce ☕️MoCha: Towards Movie-Grade Talking Character Synthesis Please unmute to hear the demo audio. ✨We defined a novel task: Talking Characters, which aims to generate character animations directly from Natural Language and Speech input. ✨We propose

🚨Hiring🚨 We are looking for research scientists and engineers to join IBM Research (Cambridge, Bangalore). We train large language models and do fundamental research on directions related to LLMs. Please DM me your CV and a brief introduction of yourself if you are interested!

🙌We are calling for submissions and recruiting reviewers for the Open Science for Foundation Models (SCI-FM) workshop at ICLR 2025! Submit your paper: https://t.co/mqFWsfLVKU (deadline: Feb 13) Register as a reviewer: https://t.co/QkuR3v8uf1 (review submission deadline: Feb 28)

🎉 Announcing the first Open Science for Foundation Models (SCI-FM) Workshop at #ICLR2025! Join us in advancing transparency and reproducibility in AI through open foundation models. 🤝 Looking to contribute? Join our Program Committee: https://t.co/U9eIGY0Qai 🔍 Learn more at:

We have few intern positions open in our ML team @ MSR Montreal, come work with @Cote_Marc @kim__minseon @LucasPCaccia @mathe_per @ericxyuan on reasoning, interactive envs/coding and LLM modularization.. 🤯 @mathe_per and I will also be at #NeurIPS2024 so we can chat about this