Now that PHC's code is out... Introducing PULSE: Physics-based Universal motion Latent SpacE: 📜: 🌐: All downstream tasks here use the same pretrained latent space. (1/6)

The key features of PULSE: 1. Once the latent space is learned, randomly sampled latents create stable and human-like behavior (instead of random jitters) -- better downstream exploration. Here we visualize training for a "reach" and "move forward with speed" tasks. (2/6)

2. The latent space should have good **coverage**. In theory, it contains the latents for the motor skills required to perform all AMASS, so you are not crippled by using this latent space. Here we use PULSE for free-from motion tracking (tracking VR controllers) (3/6)

3. The motor skills from the latent space should be able to extrapolate to unseen scenarios. Here we show policy trained using PULSE can handle complex terrain traversal using human-like behavior, using only trajectory following reward (no additional adversarial reward) (4/6)

PULSE is distilled from a imitator (PHC+) that has achieved 100% success rate on AMASS. It uses a variational information bottleneck (similar to a VAE) to form the latent space, jointly learning a prior that increases the expressiveness of the space. (5/6)

Work done at @RealityLabs Pittsburgh. Much thanks to the team! @jinkuncao , Josh Merel , @awinkler_ , @kkitani , @xuweipeng000 (6/6)

@zhengyiluo So we can do a lot of downstream tasks with PULSE's powerful generator...