[2/2] .📄 arXiv preprint: 🌐 Project website: 💻 Code on GitHub: 🎥 Demo video:

[1/2] Excited to announce GRaD-Nav! We propose a new framework that integrates 3DGS and Differentiable RL to train vision-based drone navigation policies. Our method achieves efficient end2end training, zero-shot sim2real transfer, and strong in-task adaptability.

[5/5] We show in hardware experiments that LatentToM solves tasks with two decentralized arms as well as a fully centralized bi-manual policy. Paper: Project:

[4/5] LatentToM is comms flexible. Without comms, the robots rely completely on Theory of Mind for coordination. With comms, they use a single communication round to align their consensus embeddings at each policy inference.

[3/5] The consensus embedding is supervised with a sheaf Laplacian loss to ensure alignment between robots, and a Theory of Mind loss to ensure that it can predict the other agents' ego embeddings.

[2/5] In LatentToM, each robot encodes its sensor stream into two types of embeddings: (i) an "ego embedding" to condition its own policy, and (ii) a "consensus embedding" to predict the other agents' ego embeddings.

[1/5] Humans collaborate with each other by simulating the state of mind of their teammates, a concept called Theory of Mind (ToM). We propose LatentToM, a method to endow robots with a theory of mind in latent space for cooperative manipulation.

[5/5] We embrace these findings by proposing an Action Lookup Table (ALT) policy, which equals the diffusion policy's reactivity and dexterity with a fraction of the memory footprint and inference time. And no diffusion denoising steps!.

[4/5] A visual hash function indexing a memorized action lookup table gives closed-loop visual reactivity without the need for action generalization, which seems to be a powerful recipe for imitation learning with few demonstrations.

[3/5] They are not capable of action generalization, neither interpolation nor extrapolation. Even when conditioned on OOD images of a cat, they just recall one of the training action sequences. How can these policies perform so well?.

[2/5] In our paper "Demystifying Diffusion Policies: Action Memorization and Lookup Table Alternatives," we find that robot diffusion policies trained with typical data volumes (30-120 task demonstrations) simply memorize and recall training action sequences.

[1/5] What happens when you prompt a robot diffusion policy with an image of a cat? .Website: Paper:

[3/3] This learned policy is designed to adapt at runtime to variations in drone dynamics. It outputs thrust and body rate commands and runs at 20hz on a commodity drone with only onboard compute and perception.

[2/3] To do this, we train a Gaussian Splatting model of a scene and virtually "fly" the drone within it with a massive volume of motion and dynamics perturbations. This produces 100k+ image-action pairs, which then supervise the training of a our learned policy.

[1/3] We're excited to announce SOUS VIDE, a method to train visuomotor navigation policies for autonomous drones without data from real-world flights, using only Gaussian Splat reconstructions of the scene. 📄 Paper: .🌐 Project:

Excited to announce Splat-MOVER for multi-stage, open-vocabulary manipulation, with:.- Semantic and affordance scene understanding.- Scene-editing.- Robotic grasp generation!. Find out more at and join us Friday, 11/08/24, at the #CoRL2024 poster session.

How much should you trust your robot? Check out our latest paper, where we place sample-efficient confidence bounds on the performance of robots in OOD environments. Project page: ArXiv:

RT @simonlc_: We're excited to present a differentiable physics engine for NeRF-represented objects! We augment object-centric NeRFs with d….

Have a deep-learned control system and want to find exact forward and backward reachable sets, control invariant sets, and regions of attraction? Want to know if your neural network has a continuous inverse?. All of this and more in our new paper:.

RT @leto__jean: Come to our NeRF-Shop today at #ICRA22!. We may have some live demos 😊.