RT @qiayuanliao: Want to achieve extreme performance in motion tracking—and go beyond it? Our preprint tech report is now online, with open….

RT @letian_fu: @qiyang_li will present OTTER tomorrow at #ICML2025!.A lightweight, instruction-following VLA! See OG post below!.👉Code alre….

Join millions who have switched to Grok.

RT @qiyang_li: Everyone knows action chunking is great for imitation learning. It turns out that we can extend its success to RL to better….

Join us to explore the frontier of humanoid agents at CVPR👇.

RT @GuanyaShi: ✈️to #CVPR2025 to give three workshop/tutorial talks about learning humanoid whole-body control and loco-manipulation:. - We….

RT @younggyoseo: Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-sourc….

Ppl are collecting large-scale teleoperation datasets, which are often just kinematics-level trajectories. Real2Render2Real is a new framework that can generate these data w.o. teleoperation or tricky sim+rl. High data quality for BC + nice scaling effect, plz dive in for more!.

RT @smithlaura1028: My goal throughout my PhD has been to take robots out of the lab and into the real world. It was so special to be a par….

RT @Agentica_: Introducing DeepCoder-14B-Preview - our fully open-sourced reasoning model reaching o1 and o3-mini level on coding and math.….

RT @walterzhu8: Join us at the 1st Workshop on Humanoid Agents @CVPR! #CVPR2025. Speakers in CV, CG, Robotics & CogSci will share insights….

RT @philippswu: New VLA work from @fangchenliu_ @RavenHuang4 @letian_fu and its all open source! Cool insights on how to better leverage pr….

RT @letian_fu: We had all the ingredients years ago—CLIP has been around since 2021! OTTER shows that combining these existing tools in the….

11/N It’s been an exciting collaboration between @berkeley_ai and @Meta! I had a fantastic time working with co-leaders Raven (@RavenHuang4) and Max (@letian_fu). Grateful for the invaluable insights from Tingfan Wu, @mukadammh, @JitendraMalikCV, @Ken_Goldberg, and @pabbeel!

10/N We open-sourced the code, models, and datasets! Please refer to the paper for more technical details. More pre-trained models are also coming, please stay tuned!.Jax Code: Pytorch Code: Paper:

9/N For more manipulation primitives beyond pick-and-place (covering our full set of tasks), Otter demonstrates superior performance. It outperforms fine-tuned Octo, OpenVLA, and even Pi0, showcasing its strong generalization ability and robustness across diverse robotic tasks.

8/N Otter performs well on physical robots for pick-and-place tasks (10 out of 19 tasks). It successfully generalizes to both training tasks with unseen configurations and distractors and held-out tasks. Notably, Otter outperforms Octo, OpenVLA, and the recently introduced Pi0.

7/N (Cont) To further assess Otter’s zero-shot generalization capabilities for unseen tasks, we introduced new objects and out-of-distribution language instructions that were not present in the training dataset.

7/N We collected demonstrations using the DROID setup under challenging generalization settings. For every task, we randomized object locations and added 2-3 distractors. During evaluation, we use different object locations and distractors.

6/N OTTER combines ClearCLIP’s fine-grained visual features together with (1) robot proprioception for precise control and (2) language instructions for flexible handling of the same object—whether picking, pushing, or poking.

5/N Building on these observations, we propose a simple yet effective approach: we measure the similarity between each language token and visual patch. Specifically, we use these similarity scores to “retrieve” the image patches that closely match the task instruction.