Catherine Glossop
@CatGlossop
Followers
241
Following
143
Media
5
Statuses
20
I had an amazing time working on this release, and in general my internship at Pi has been a blast :) Bonus - take a look at some of our 1x footage on YouTube! ☕️ https://t.co/6hu0K1au5b
0
0
6
π*0.6 has just been released! Not only can our policy do complex, long horizon tasks, but it can also keep on doing them for hours on end with the power of RL and coaching 🦾 Blog: https://t.co/R1dqcC8FJQ Paper:
pi.website
A method for training our generalist policies with RL to improve success rate and throughput on real-world tasks.
Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.
1
1
56
Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.
51
198
1K
Language following is a tough problem for VLAs: while these models can follow complex language, in practice getting datasets that enable language following is hard. We developed a method to counterfactually and automatically label data to improve language following! 🧵👇
10
69
420
To learn more about CAST, please visit our website https://t.co/h0RoooaBwk! This work was in collaboration with @verityw_ @shahdhruv_ arjun bhokar and @svlevine and was lots of fun to work on :)
0
0
2
We augment a navigation dataset with CAST, train a VLA, and compare it to a standard VLA and SOTA methods on a set of visual navigation tasks. We find that our VLA exhibits overall better language following, without a massive VLM or additional sensors.
0
0
3
These atomic commands can then be mapped to more complex language instructions by querying a VLM, creating counterfactual endings that can be spliced into existing trajectories.
0
0
3
With CAST, we use counterfactual trajectories to force the policy to attend to language by broadening the distribution of actions and language at each state. Our key observation is it is easy to learn a policy that can follow atomic commands like “turn right” or “go forward”.
0
0
4
Imagine a policy trained to navigate through an indoor environment. If the data only contains navigating down the center of a hallway, or moving left of obstacles, the words “keep to the side of the hallway” or “move to the right of the obstacle” become meaningless to the policy.
0
0
7
Inherent biases and imbalances in robot data can make training steerable VLA policies challenging. We introduce CAST, a method to augment datasets with counterfactuals to induce better language following https://t.co/h0RoooaBwk ← paper, code, data, and more available here! 🧵
7
10
60
@NoriakiHirose @shahdhruv_ @KyleStachowicz @svlevine @frodobots While large-scale data collection can lead to mixed quality data, we find that by reannotating with MBRA, we can still leverage the visual diversity present in the data. We demonstrate that our policy can navigate long distances in 6 cities across 3 continents!
0
0
2
Leveraging large-scale data sources can enable extremely general and robust policies. See our recent work MBRA! https://t.co/8D4h7IIvDL Led by @NoriakiHirose @shahdhruv_ @KyleStachowicz Lydia Ignatova, @svlevine and thanks @frodobots for making large-scale data for nav possible!
2
4
31
We trained a robotic foundation model that can drive mobile robots in six different countries, and navigate Sproul Plaza in midday on the UC Berkeley campus! Some cool new work w/ @NoriakiHirose, Lydia Ignatova, @KyleStachowicz, @CatGlossop, @shahdhruv_
https://t.co/tkl6IogDCL
6
52
325
I'm excited to announce that we'll be hosting a Workshop on Learned Robot Representations (RoboReps) at #RSS2025! This will be a full day workshop on June 25, 2025, at USC. Submissions open at https://t.co/8OFCKjqV3W - Due May 28 AOE Website: https://t.co/dga9T8voWb (1/🧵)
1
10
23
Excited to share our recent research, LeLaN for learning language-condtitioned navigation policy from in-the-wild video in UC Berkeley and Toyota Motor North America. We present the LeLaN on CoRL 2024. @CatGlossop
@ajaysridhar0
@shahdhruv_
@oier_mees and @svlevine
3
17
93
Hearty congratulations to BAIR students, faculty and alumni for their many awards at #ICRA2024 this week in Japan BAIR alumni @pulkitology @LerrelPinto @RCalandra won Early Career awards; students from @svlevine @Ken_Goldberg @JitendraMalikCV labs won both Best Paper awards!
2
10
60
Cross-embodied robot policies hold the promise of one policy to control all robots. But how far does transfer go? In new work, we study positive transfer between *manipulation* & *navigation* and show that nav data helps manipulation, and vice versa! https://t.co/XyqJ0vMwz6 🧵 👇
1
44
176
Visual Nav Transformer 🤝 Diffusion Policy Works really well and ready for deployment on your robot today! We will also be demoing this @corl_conf 🤖 Videos, code and checkpoints: https://t.co/cqiPMPqewZ Work led by @ajaysridhar0 in collaboration with @CatGlossop @svlevine
ViNT (Visual Nav Transformer) now has a diffusion decoder, which enables some cool new capabilities! We call it NoMaD, and it can explore new environments, control different robots, and seek out goals. If you want an off-the-shelf navigation foundation model, check it out! A 🧵👇
3
21
133
ViNT (Visual Nav Transformer) now has a diffusion decoder, which enables some cool new capabilities! We call it NoMaD, and it can explore new environments, control different robots, and seek out goals. If you want an off-the-shelf navigation foundation model, check it out! A 🧵👇
1
57
386