RT @TheHumanoidHub: Boston Dynamics and Toyota Research Institute are advancing humanoid robotics with Large Behavior Models (LBMs) for Atl….

This was a massive effort by the entire team, with a number of individuals really pouring their hearts into this paper. The paper is packed full of (too many?) details. Your comments and feedback would be very welcome.

Probably my favorite plot from the paper, which sums it all up, is this one. The plot compares performance using different amounts of pretraining data used before training a new task: 0% (aka single task), 25, 50, or 100% of TRI’s data, then 100% of TRI’s data + all of the

The short version is: LBMs work! We see consistent and statistically significant improvements as we increase the amount of pretraining data. But doing the science is still hard; as a field we have more work to do to improve the statistical power of our experiments.

TRI's latest Large Behavior Model (LBM) paper landed on arxiv last night! Check out our project website: One of our main goals for this paper was to put out a very careful and thorough study on the topic to help people understand the state of the.

Very proud of Nicholas, who recently shared (for physics-quality assets from a small amount of interaction with a robot) and is now following up with his work on scene-level generation.

In my mind, it's a bit like a biology paper that is focused on a particular animal model. I hope we'll learn more quickly from each other if we can make precise, substantiated claims about particular setups, so that as a field we can assemble those claims into a coherent picture.

Side note: I'm proud of the title of this paper, which we intentionally made pretty narrow/specific. I think that some of the most important work that we have to do as a field right now is careful empirical work to interrogate the properties of these models that we're creating.

One of the most interesting take-aways for me is that "high-performing policies need to know whether they are executing in sim or in real." A number of implications flow from that, including that sim+real cotraining can decrease performance if the visual gap is too small.

This work really sharpened my thinking about sim+real cotraining.

RT @NicholasEPfaff: New Paper: "Scalable Real2Sim: Physics-Aware Asset Generation via Robotic Pick-and-Place Setups"! 🤖. We introduce a ful….

RT @BoyuanChen0: Announcing Diffusion Forcing Transformer (DFoT), our new video diffusion algorithm that generates ultra-long videos of 800….

I'm super excited to start a great new collaboration with the fantastic team at Boston Dynamics. Scott Kuindersma and I chatted with Evan Ackerman about it earlier today.

RT @LerrelPinto: This #RSS2024 on July 19, we are organizing a tutorial on supervised policy learning for real world robots!. Talks by @not….

RT @BoyuanChen0: Introducing Diffusion Forcing, which unifies next-token prediction (eg LLMs) and full-seq. diffusion (eg SORA)! It offers….

RT @chris_j_paxton: And if you’re at all interested in humanoids you need to check out Punyo, from TRI, a soft-body humanoid capable of who….

RT @KennethCassel: a bit surprised TRIs new robot has gotten little love on X . current humanoid approaches seem very hand centric but we u….

RT @SongShuran: Check out @chichengcc's step-by-step tutorial on building the UMI gripper. We really hope to see more UMIs running in the w….

RT @chichengcc: Can we collect robot data without any robots?. Introducing Universal Manipulation Interface (UMI). An open-source $400 syst….