Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with

Learning Transient Lidar Sensing! Transient imaging has been around a while, but we finally made it work for lidar sensing (#ICCV2025 highlight)! We find that a transformer-based DSP allows us to learn directly from the spatio-temporal histograms of a SPAD array! The method can

3D Object Tracking without Training Data? In our @Nature Machine Intelligence paper ( https://t.co/UczJ9NxnjZ), we recast 3D tracking as an inverse neural rendering task where we fit a scene graph to an image that best explains this image. The method generalizes to completely

Flat Nanophotonic Cameras! Treating optics like neural network layers, we were able to optimize a collaborative metasurface with 100 million nanopillars for the first time. The resulting camera sits directly on the sensor cover glass, a few mm away from the sensor. If you are at

Thanks for coming to our Embodied Intelligence for Autonomous Systems workshop @CVPR We had such a dense & fun program. Videos will be up soon. Kudos to the team: @smch_1127 @AnaMari77147030 @CSakaridis @francislee2020 @PhilionJonah @fbartoc @wongfaikit K. Chitta #cvpr2025

When at @CVPR a major challenge is how to split yourself among super amazing workshops. I'm afraid to announce you that with our workshop on "Embodied Intelligence for Autonomous Systems on the Horizon" we will make this choice even harder: https://t.co/r6LmN0bIm7 #cvpr2025

Broadband Imaging with a 1cm Ultra-Flat Metalens! We overcome intrinsic spectral bandwidth limitations in new work in Nature Communications. See dynamic outdoor captures "in-the-wild" below. We find that computational design and probabilistic diffusion methods make it possible

Scenario Dreamer has been accepted at #CVPR2025! Website: https://t.co/p4X77xCgHy We train a vectorized latent diffusion model to synthesize high-fidelity driving simulation environments (agents+map). Scenario Dreamer enables fully data-driven closed-loop generative simulation!

Vectorized Diffusion without Rasterized Encodings! Scenario Dreamer (CVPR 2025) directly operates on vectorized scene elements to generate novel unseen scenes - making fully data-driven closed-loop generative driving simulation possible. We trained a novel vectorized latent

.@_FelixHeide_ uses algorithms to make sense of the world. He has built a new kind of camera that uses optical elements as a neural network that identifies images at the speed of light. Learn more: https://t.co/9MUqwmzKAL

Using algorithms to sense and understand the world is the focus of @_FelixHeide_'s work. Now, he has created a new kind of camera — one that has optical elements that function as a neural network and can identify images at the speed of light. 📸 https://t.co/CO3Aqf9VmO

Our work on neural étendue expansion for holographic AR/VR displays was highlighted in Optics & Photonics News "Optics in 2024"! Read the summary at https://t.co/RFToQ4PRSp See the original Nature Communications paper here: https://t.co/qTlKkCUrNE @OpticaWorldwide

Evaluating Neural Networks at the Speed of Light (with Light!). See live optical inference in the video below. Excited to share recent academic work on optical neural networks as a collection of computing elements embedded in the camera lens! These elements perform computation

Check out CtRL-Sim later this week at #CoRL2024 in Munich! How can we generate interesting edge cases to test autonomous vehicles in simulation? We propose CtRL-Sim for closed-loop behavior simulation that enables fine-grained control over agent behaviors. CtRL-Sim leverages

Multimodal BEV Fusion over Long Ranges and in Adverse Weather! Check out our work tomorrow at #ECCV2024. We fuse multimodal sensor data from lidar, radar, and stereo cameras through attentive, depth-based blending schemes, with learned refinement on the Bird’s Eye View (BEV)

Implicit Neural Light Spheres lets you turn panoramic captures into dynamic wide FOV renders (with real-time rendering!). Instead of generating panoramas with image stitching, we use neural light spheres to jointly estimate the camera path and a high-resolution scene