I love robot videos, and there are some quite interesting ones to check out on our project. Excited to share the results of the team’s work, and glad to have been a core contributor — go check out the paper because there are a lot of interesting details!

Probably my favorite plot from the paper, which sums it all up, is this one. The plot compares performance using different amounts of pretraining data used before training a new task: 0% (aka single task), 25, 50, or 100% of TRI’s data, then 100% of TRI’s data + all of the

🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts. At TRI, we’ve been researching how LBMs can

have been waiting for this release! Robotics needs rigorous and careful evaluation now more than ever 🦾

At @ToyotaResearch, we've been studying how LBMs can help robots learn faster and better. We built a rigorous evaluation pipeline to benchmark LBM performance with statistical confidence. Results suggest that pre-training on hundreds of tasks yields 80% data savings on new tasks.

Awesome paper on robot foundation models with super rigorous evaluation. Definitely a must-read!

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: https://t.co/1PWPdVTFAk ArXiv: https://t.co/oSIBKtUTbk

**Steerability** remains one of the key issues for current vision-language-action models (VLAs). Natural language is often ambiguous and vague: "Hang a mug on a branch" vs "Hang the left mug on the right branch." Many works claim to handle language input, yet the tasks are often

How can we achieve both common sense understanding that can deal with varying levels of ambiguity in language and dextrous manipulation? Check out CodeDiffuser, a really neat work that bridges Code Gen with a 3D Diffusion Policy! This was a fun project with cool experiments! 🤖

We recently launched https://t.co/mshIJSnIYu as a community-driven effort to pool UMI-related data together. 🦾 If you are using a UMI-like system, please consider adding your data here. 🤩🤝 No dataset is too small; small data WILL add up!📈

🚀

With grad school admissions being open now, I'd like to re-share our list with all the awesome faculty at MIT that work in #AI and #ClimateChange https://t.co/QW0JNpjKhk @ClimateChangeAI @MIT @eapsMIT @MITCSConsortium @priyald17 @sarameghanbeery

Check out our #ECCV2024 paper on "Zero-Shot Multi-Object Scene Completion"! Drop by poster 313 this morning if you're interested! Website: https://t.co/iMYwLdqVpi Code&Dataset: https://t.co/d6hjNrYYIL

Read this thread! Then go read the paper 😀

🥁 Speakers Announced! 🥁 The #WiCV workshop at @eccvconf is thrilled to announce our amazing speakers: @dimadamen, @ftm_guney, Stella Yu, Hedvig Kjellström, and dinner speaker @HildeKuehne! 🎤 Join us on Monday 2pm-6pm 🇮🇹#ECCV2024! Full program here: https://t.co/AWTk18jXqf

We are grateful to be awarded an oral presentation -- please come by Wed 10/2 at 1:30pm (I believe we are the first talk in the oral session) as well as the poster session afterward (number 156) at 4:30pm! #ECCV2024 🎉

My group will be seeking new PhD students in the coming cycle! The best way to reach us is to apply to the @MITEECS PhD program. I look for independent, creative, interactive, supportive, passionate, bright students who want to work on fundamental problems with a geometry flavor.

What is a really really hard problem to work on in #AI? My own answer is Spatial Intelligence - a technology that could empower and enable countless possible use cases in creation, design, learning, AR/VR, robotics, and beyond. It’s a real honor that my cofounders @jcjohnss

Do you actually know how well your policy works? Excited to have folks like @imp_aa and co. pushing on this front.

👇Introducing DPPO, Diffusion Policy Policy Optimization DPPO optimizes pre-trained Diffusion Policy using policy gradient from RL, showing 𝘀𝘂𝗿𝗽𝗿𝗶𝘀𝗶𝗻𝗴 𝗶𝗺𝗽𝗿𝗼𝘃𝗲𝗺𝗲𝗻𝘁𝘀 over a variety of baselines across benchmarks and sim2real transfer https://t.co/uOkkcnVFCf