Michael Equi
@michael_equi
Followers
1K
Following
900
Media
9
Statuses
92
building robot brains @physical_int | ex Optimus @Tesla_AI | ex @1x_tech | EECS @ucberkeley | @ZFellows_ | past VP @berkeleyML | @berkeley_ai
California
Joined February 2020
RT @KarlPertsch: We’re releasing the RoboArena today!🤖🦾. Fair & scalable evaluation is a major bottleneck for research on generalist polici….
0
82
0
RT @physical_int: Our models need to run in real time on real robots, but inference with big VLAs takes a long time. We developed Real-Time….
0
78
0
RT @physical_int: We figured out how to train VLAs with diffusion outputs much faster (7.5x faster), inheriting better language following f….
0
102
0
We cover these improvements along with multiple others in our paper and blog post. The blog also provides many more examples demonstrating π-0.5 doing a variety of tasks, all in unseen environments!. blog: https://www.π.com/blog/pi05.paper:
https://www.π.com/download/pi05.pdf.
0
0
5
One surprising insight is that co-training on the HL objective significantly improves performance even without the hierarchical inference scheme. We call this implicit HL
1
0
7
Hierarchical inference also lets us condition on different levels of abstraction. We can directly talk to the LL with commands like "drive base left" and "pick up cup" or we can tell the HL to "clean the bedroom" and "place the dishes in the sink".
1
0
1
Another key element is hierarchical inference. π-0.5 is a single policy that can produce language commands and robot actions. During inference we use a two-stage process where the policy first produces a command like "pick up spoon" and then conditions on it for multiple steps.
1
0
1
The main principle behind π-0.5 is co-training. We built a large robot dataset that includes mobile, static, cross-embodiment, and web data. Each additional datasource provides a clear improvement to performance
1
0
0
RT @physical_int: We got a robot to clean up homes that were never seen in its training data! Our new model, π-0.5, aims to tackle open-wor….
0
260
0
RT @physical_int: We are excited to share new experiments with AgiBot @AgiBot_zhiyuan on multi-task, multi-embodiment VLAs! With one model….
0
64
0
Today we released Hi Robot 👋 a method for adding system 2 thinking to robot policies in a way that improves interactivity and performance!.
Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly?. We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).
1
7
85
Today we open sourced π₀!.
Many of you asked for code & weights for π₀, we are happy to announce that we are releasing π₀ and pre-trained checkpoints in our new openpi repository! We tested the model on a few public robots, and we include code for you to fine-tune it yourself.
0
0
14
At π we developed a JPEG inspired tokenizer that speeds up VLA training by 5x! An amazing achievement with some very clever insights.
There are great tokenizers for text and images, but existing action tokenizers don’t work well for dexterous, high-frequency control. We’re excited to release (and open-source) FAST, an efficient tokenizer for robot actions. With FAST, we can train dexterous generalist policies
2
3
46
Excited to be in Vancouver for #NeurIPS2024! Reach out if you want to chat anything robot learning 🤖🧠 or are curious about what it takes to bake a π @physical_int.
2
1
19
RT @physical_int: At Physical Intelligence (π) our mission is to bring general-purpose AI into the physical world. We're excited to show….
0
327
0
Everything is compiled into a blog post and a paper that you can check out at
314159.com
Our first generalist policy, π0, a prototype model that combines large-scale multi-task and multi-robot data collection with a new network architecture to enable the most capable and dexterous...
0
0
6
Excited to share what we've been up in the past 8 months @physical_int! We trained a 3B vision-action-language flow matching generalist and fine-tuned on complex tasks. Take a look at the results!
10
24
181