Seungwook Han
@seungwookh
Followers
404
Following
638
Media
21
Statuses
135
phd-ing @MIT_CSAIL, prev @MITIBMLab @columbia
Joined June 2017
🧙‍♂️Excited to share our new whitepaper “General Reasoning Requires Learning to Reason from the Get-Go.” . We argue that simply making models bigger and feeding them more data is NOT enough for robust, adaptable reasoning. (1/n).
1
12
79
Blazing-fast image creation – using just your voice. Try Grok Imagine.
283
567
3K
to clarify, not saying we’re there atm. i dont have a formal definition of what it means to be human, but agency and the ability to continually learn seem to be important.
0
0
0
this tendency to anthropomorphize is too real and shuns me from reading work with such titles. on the other hand, a part of me also asks: how do we know when a thing is conscious and start to analyze as if it is another human-like organism. we automatically assume all humans are.
Resist the tendency to anthropomorphize that which is not human.
1
0
3
uncertainty-aware reasoning, akin to how humans leverage our confidence.
🚨New Paper!🚨.We trained reasoning LLMs to reason about what they don't know. o1-style reasoning training improves accuracy but produces overconfident models that hallucinate more. Meet RLCR: a simple RL method that trains LLMs to reason and reflect on their uncertainty --
0
1
3
was actually wondering with @hyundongleee the fundamental differences between diffusion and autoregressive modeling other than the structure imposed in the modeling of the sequential conditional distribution and how they manifest. a poignant paper that addresses this thought.
🚨 The era of infinite internet data is ending, So we ask:. 👉 What’s the right generative modelling objective when data—not compute—is the bottleneck?. TL;DR:. ▶️Compute-constrained? Train Autoregressive models. ▶️Data-constrained? Train Diffusion models. Get ready for 🤿 1/n
0
1
13
omw to trying this out đź‘€.
Some news: We're building the next big thing — the first-ever AI-only social video app, built on a highly expressive human video model. Over the past few weeks, we’ve been testing it in private beta. Now, we’re opening early access: download the iOS app to join the waitlist, or.
0
0
0
how particles can act differently under different scales and conditions and how we can equip it as part of design is cool.
8. Jeonghyun Yoon: Precisely Loose: Unraveling the Potential of Particles. A big thank you goes out to the entire architecture community, including advisors, readers, staff, family and peers who helped bring these projects to light. Image credit: Chenyue “xdd” Dai . 2/2
0
0
4
RT @LakerNewhouse: [1/9] We created a performant Lipschitz transformer by spectrally regulating the weights—without using activation stabil….
0
78
0
But actually this is the og way of doing it and should stop by E-2103 to see @jxbz and Laker Newhouse whiteboard the whole paper.
Laker and I are presenting this work in an hour at ICML poster E-2103. It’s on a theoretical framework and language (modula) for optimizers that are fast (like Shampoo) and scalable (like muP). You can think of modula as Muon extended to general layer types and network topologies
1
6
74
RT @jyo_pari: If you are interested in questioning how we should pretrain models and create new architectures for general reasoning . - the….
0
6
0
@qx_dong proposes a version of how we can scalably enable RPT ( and would love to discuss how to enable it with synthetic data.
arxiv.org
In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a...
0
0
1
Presenting our ICML spotlight poster at today 11am @ E-606 w/ @jyo_pari!. We need to fundamentally change how we train to achieve true reasoning. Reward-based Pretraining (RPT) > Supervised Pretraining.
🧙‍♂️Excited to share our new whitepaper “General Reasoning Requires Learning to Reason from the Get-Go.” . We argue that simply making models bigger and feeding them more data is NOT enough for robust, adaptable reasoning. (1/n).
1
2
14
How do task vectors emerge during pretraining—and can they predict ICL performance?. Come see our ICML spotlight poster "Emergence and Effectiveness of Task Vectors in ICL" at 11am @ East Hall A-B (#E-2312) with @jinyeop_song!. 🔗
1
5
13
At #ICML 🇨🇦 this week. I'm convinced that the core computations are shared across modalities (vision, text, audio, etc). The real question is the (synthetic) generative process that ties them. Reach out if you have thoughts or want to chat!.
0
3
16
wholeheartedly agree with this direction that games can be a good playground for learning reasoning. makes us think what other synthetic environments we can design and grow over complexity.
We've always been excited about self-play unlocking continuously improving agents. Our insight: RL selects generalizable CoT patterns from pretrained LLMs. Games provide perfect testing grounds with cheap, verifiable rewards. Self-play automatically discovers and reinforces
0
1
7
robot arms becoming more human-like. now with a wrist 🦾.
What’s keeping robot arms from working like human arms?. They're big, slow, have the wrong joints, and can't conform to their environment. DexWrist solves all of these issues and simplifies learning constrained, dynamic manipulation👉Â
0
0
3
RT @phillip_isola: Our computer vision textbook is now available for free online here:. We are working on adding so….
visionbook.mit.edu
0
620
0
a step towards self-learning models — self-synthesizing data to train on and evolving.
What if an LLM could update its own weights?. Meet SEALđź¦: a framework where LLMs generate their own training data (self-edits) to update their weights in response to new inputs. Self-editing is learned via RL, using the updated model’s downstream performance as reward.
1
1
8