Adam Wei
@adamwei_
Followers
117
Following
42
Media
4
Statuses
11
Learning from both sim+real data could scale robot imitation learning. But what are the scaling laws & principles of sim+real cotraining?. We study this in the first focused analysis of sim+real cotraining spanning 250+ policies & 40k+ evals (1/6)
6
13
101
RT @giannis_daras: Announcing Ambient Diffusion Omni — a framework that uses synthetic, low-quality, and out-of-distribution data to improv….
0
61
0
Really cool work on environment generation + inference time search with diffusion!.
Want to scale robot data with simulation, but don’t know how to get large numbers of realistic, diverse, and task-relevant scenes?. Our solution:.➊ Pretrain on broad procedural scene data.➋ Steer generation toward downstream objectives. 🌐 🧵1/8
0
0
2
RT @MichaelAPosa: We're really honored to be named alongside Ajay and Justin for this award, from the hands-down best technical committee i….
0
10
0
Amazing concurrent work showing the effectiveness of sim+real cotraining on a wide range of tasks! The similarity of our findings suggests that sim data could unlock massive improvements in robotics. I'm excited to see what comes next in this space.
How to use simulation data for real-world robot manipulation? We present sim-and-real co-training, a simple recipe for manipulation. We demonstrate that sim data can significantly enhance real-world performance, even with notable differences between the sim and the real. (1/n)
0
1
6
For more information, please check out our paper A huge thank you to my amazing collaborators @_abhinav_ag @BoyuanChen0 , Rohan Bosworth, @NicholasEPfaff , and @RussTedrake ! More experiments and project website are on the way soon 👀(6/6).
0
0
2
Our work presents a clean & thorough study on a canonical task. Concurrent work from @SteveTod1998 shows that cotraining is also effective in the multi-task regime. The alignment of our findings provides strong signal that our analysis is informative for multi-task as well (5/6).
0
0
3
Our analyses show that high-performing policies discern sim vs real & learn distinct behaviors for each domain since their dynamics require different actions. If policies learn distinct behaviors for sim & real, how does cotraining help? We study this carefully in our paper (4/6)
0
0
4
To study the qualities that matter in sim data, we sweep 6 sim2real gaps. Cotraining still improves performance, but large gaps reduce this effect. Physics, task + camera shifts are most impactful. Surprisingly some visual gap is desirable! We analyze this critical subtlety (3/6)
0
0
1
We find that.1️⃣ Cotraining can improve performance by 2-7x but is sensitive to mixing ratio.2️⃣Scaling sim reduces this sensitivity and improves performance up until a performance ceiling.3️⃣Real data is still needed to increase the performance ceiling and cross sim2real gap .(2/6)
0
0
5