Keyon Vafa
@keyonV
Followers
5K
Following
2K
Media
174
Statuses
1K
Postdoctoral fellow at @Harvard_Data | Former computer science PhD with @Blei_Lab at @Columbia University | Researching AI + world models
Joined August 2011
Can an AI model predict perfectly and still have a terrible world model?. What would that even mean?. Our new ICML paper formalizes these questions. One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵
211
1K
7K
RT @benscharfstein: One of the most fascinating research agendas I’ve seen. Colloquially people using LLMs refer to them having world mod….
0
1
0
Here's a video I made that goes over methods we've worked on for evaluating world models. Thank you @srush_nlp for the opportunity!.
How can we evaluate whether LLMs and other generative models understand the world? . New guest video from Keyon Vafa (@keyonV) on methods for evaluating world models.
0
4
37
RT @srush_nlp: How can we evaluate whether LLMs and other generative models understand the world? . New guest video from Keyon Vafa (@keyo….
0
15
0
Great @QuantaMagazine article about world models that covers some of our recent research.
The wide-ranging abilities of large language models like ChatGPT can give users the (mistaken) impression that AI understands our world. A scaled-down world model is a long-sought and still unrealized goal. @johnpavlus explains:
0
0
6
RT @MITLIDS: Can #LLMs grasp the real world? MIT & Harvard researchers (@m_sendhil, @asheshrambachan, @petergchang, @keyonV) propose a new….
0
5
0
RT @JiayiiGeng: 📢 We're thrilled to announce the CMU AI for Science Workshop on Sept 12 at CUC-MPW! . Featuring an amazing lineup of speak….
0
13
0
Work with Emma!.
🚨 New postdoc position in our lab @Berkeley_EECS! 🚨 (please retweet + share with relevant candidates). We seek applicants with experience in language modeling who are excited about high-impact applications in the health and social sciences!. More info in thread. 1/3
0
0
5
RT @alexolegimas: Key question for incorporating AI into firms: can AI recover signal that human managers miss?. @brian_jabarian’s (w @Henk….
0
7
0
RT @rajivmovva: 📢NEW POSITION PAPER: Use Sparse Autoencoders to Discover Unknown Concepts, Not to Act on Known Concepts. Despite recent res….
0
63
0
RT @katie_m_collins: How do people reason so flexibly about new problems, bringing to bear globally-relevant knowledge while staying locall….
0
20
0
RT @OwainEvans_UK: New paper & surprising result. LLMs transmit traits to other models via hidden signals in data. Datasets consisting only….
0
1K
0
Excited to be one of the organizers for this workshop tomorrow. Stop by if you're interested in evaluating world models!.
Join us for the Workshop on Assessing World Models at ICML tomorrow!. When: Friday July 17, 8:45am-5:15pm.Where: West Ballroom B (same floor as registration)
0
0
9
RT @Alber_RomGar: Researchers from Harvard, Keyon Vafa (@keyonV) and MIT, Peter Chang (@petergchang), Ashesh Rambachan (@asheshrambachan),….
0
27
0
RT @rajivmovva: 1. We will present HypotheSAEs at #ICML2025, Wednesday 11am (West Hall B2-B3 #W-421). 2. Let me know if you'd like to chat….
0
9
0
Paper: Co-authors: Peter Chang (@petergchang), Ashesh Rambachan (@asheshrambachan), Sendhil Mullainathan (@m_sendhil).
arxiv.org
Foundation models are premised on the idea that sequence prediction can uncover deeper domain understanding, much like how Kepler's predictions of planetary motion later led to the discovery of...
21
14
278
This is one way to evaluate world models. But there are many other interesting approaches. Plug: If you're interested in more, check out the Workshop on Assessing World Models I'm co-organizing next Friday at ICML.
worldmodelworkshop.org
Date: Friday, July 18 2025 Time: 8:45am - 5:15pm (Pacific Time) Location: West Ballroom B at ICML 2025 in Vancouver, Canada (Same Floor as Registration)
6
5
206
Last year we proposed different tests that studied single tasks. We now think that studying behavior on new tasks better captures what we want from foundation models: tools for new problems. It's what separates Newton's laws from Kepler's predictions.
New paper: How can you tell if a transformer has the right world model?. We trained a transformer to predict directions for NYC taxi rides. The model was good. It could find shortest paths between new points. But had it built a map of NYC? We reconstructed its map and found this:
3
4
179
Summary:.1. We propose inductive bias probes: a model's inductive bias reveals its world model. 2. Foundation models can have great predictions with poor world models. 3. One reason world models are poor: models group together distinct states that have similar allowed next-tokens.
7
11
280
Inductive bias probes can test this hypothesis more generally. Models are much likelier to conflate two separate states when they share the same legal next-tokens.
1
2
171