Andy Zeng
@andyzengineer
Followers
8K
Following
1K
Media
61
Statuses
395
Building robot foundation models @GeneralistAI. Prev @GoogleDeepMind, PhD @Princeton. One experiment away from magic.
Joined September 2017
This is one-shot assembly: you show examples of what to build, and the robot just does it. (see original post: https://t.co/zrm9FA8Xz1) To share more on how this works, the robot is controlled in real time by a neural network that takes in video pixels and outputs 100Hz actions.
17
32
278
Glad you caught this 🙂 Never thought we’d go this deep into networking. Move enough robot data globally 🌎 and your NAT gateways might get silently banned mid-Atlantic... 🤦 The world is not ready yet for massive-scale robots—but its coming, and we’ll lay the groundwork for it.
ok actually i think this is probably the most underappreciated part. these guys are serious about scaling. it’s not just talk.
3
10
97
ok actually i think this is probably the most underappreciated part. these guys are serious about scaling. it’s not just talk.
Introducing GEN-0, our latest 10B+ foundation model for robots ⏱️ built on Harmonic Reasoning, new architecture that can think & act seamlessly 📈 strong scaling laws: more pretraining & model size = better 🌍 unprecedented corpus of 270,000+ hrs of dexterous data Read more 👇
1
6
91
At Generalist, robotics is no longer limited by data. Breaking through the data wall has enabled us to become a true foundation model company, building and scaling models from the ground up for embodied intelligence. In today’s blog post we’re excited to share details about
generalistai.com
Embodied Foundation Models That Scale with Physical Interaction
Introducing GEN-0, our latest 10B+ foundation model for robots ⏱️ built on Harmonic Reasoning, new architecture that can think & act seamlessly 📈 strong scaling laws: more pretraining & model size = better 🌍 unprecedented corpus of 270,000+ hrs of dexterous data Read more 👇
17
35
266
General dexterity involves "physical commonsense" —learning long tail of cases like: 🤏nudging objects to make space for fingers to grasp 🫴placing down slipping objects to get better grip etc... This👇shares more on how we think robots🦾can get there with models & lots of data
Introducing GEN-0, our latest 10B+ foundation model for robots ⏱️ built on Harmonic Reasoning, new architecture that can think & act seamlessly 📈 strong scaling laws: more pretraining & model size = better 🌍 unprecedented corpus of 270,000+ hrs of dexterous data Read more 👇
1
3
89
"Train a model to predict how many timesteps left until task success" - a simple, yet powerful way to get rewards from episodic BC data Lots of nuggets in the paper (including steps-to-go fn is distributionally multimodal) Kamyar's post 👇 on how it drives RL self-improvement
Super excited to finally share our work on “Self-Improving Embodied Foundation Models”!! (Also accepted at NeurIPS 2025) • Online on-robot Self-Improvement • Self-predicted rewards and success detection • Orders of magnitude sample-efficiency gains compared to SFT alone •
2
10
91
One shot imitation learning! Brings back good memories from eons ago (aka 2016). This is probably my favorite demo of the year (so far). The smoothness and agility of these systems speak to the quality of the full stack system. Sometimes I tell my students that a good number of
This is one-shot assembly: you show examples of what to build, and the robot just does it. (see original post: https://t.co/zrm9FA8Xz1) To share more on how this works, the robot is controlled in real time by a neural network that takes in video pixels and outputs 100Hz actions.
2
7
78
it just worked 🤷♂️
At Generalist, we’re working towards a future where robots can do anything. To that end, the robots build now, too. We’ve trained a robot to do one-shot assembly, constructing Legos end-to-end: no custom engineering, just pixels in → Lego copies out.
0
1
21
Imagine having a "copy-paste" button 📋 for the physical world – a pretty good litmus test for embodied AGI
At Generalist, we’re working towards a future where robots can do anything. To that end, the robots build now, too. We’ve trained a robot to do one-shot assembly, constructing Legos end-to-end: no custom engineering, just pixels in → Lego copies out.
4
6
105
🎉Advanced Robotics Best Survey Paper Award has been awarded to our survey paper "Real-World Robot Applications of Foundation Models: A Review"! We are truly grateful to everyone who contributed! Thank you @__tmats__, Andrew, @jiaxianguo07, @chris_j_paxton, and @andyzeng_ !
How can existing robot systems be replaced with foundation models? Check out our new survey paper on the real-world robot applications of foundation models: https://t.co/PabTnuyzD1 Thread👇
4
18
116
@JonathanTo64772 @GeneralistAI_ Thank you 🙏🙏 We ran the policies with “--butter” flag for the videos 😄
1
1
7
Excited to be out of stealth! Since graduating from MIT, I’ve been building something incredibly FUN. Every day at work is full of laughter and “wows.” Super proud of the fantastic team making it all happen. Let’s go 🚀
Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early
9
14
192
There’s something satisfying to see the robot slotting in the box flaps so nicely in the end ... 😌
Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early
1
9
139
Check out our robots! 🤖
Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early
1
2
25
To see emergent behaviors from low-level policies was a first for many of us on the team. They don't happen often enough yet, but it certainly feels like we're headed in the right direction. Reach out if you're interested in working together.
Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early
8
4
118
I’m excited to announce @GeneralistAI_ We believe the path to general-purpose robots starts with precise, fast, and resilient manipulation. What you see here are end-to-end AI models, trained from scratch, doing some very hard tasks. This is pixels in, actions out. While I’m
Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early
16
14
181
Last Spring I took off from Google DeepMind, and I've been heads-down building since with an amazing team. Excited to share more today -- introducing Generalist. It's felt to me for a couple years, since we started bringing multimodal LLMs into robotics, that a subset of the
Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early
27
36
415
We've been heads-down building. The robots have gotten pretty good. We'll be sharing a brief update soon.
1
8
136
Position control can only go so far. For contact-rich tasks, robots must master both position and force – that’s where compliance comes in! But what’s the right compliance? 🤔Hint: being always compliant in all directions won’t cut it. Check out @YifanHou2’s solution 😉⤵️
Can robots learn to manipulate with both care and precision? Introducing Adaptive Compliance Policy, a framework to dynamically adjust robot compliance both spatially and temporally for given manipulation tasks from human demonstrations. Full detail at https://t.co/fVsG4jq0T6
1
10
92
UMI data doubles as a 3D vision and robotics dataset. I'm curious to see what people can do with it! Consider prototyping with and contributing to the community dataset 👇
We recently launched https://t.co/mshIJSnIYu as a community-driven effort to pool UMI-related data together. 🦾 If you are using a UMI-like system, please consider adding your data here. 🤩🤝 No dataset is too small; small data WILL add up!📈
0
1
28