This work is done with amazing collaborators @ethantqiu , Karim El-Refai, Hiya Shah, @ChungMinKim , @justkerrding , @Ken_Goldberg , @UCBerkley , @berkeley_ai

We can also find geometric defects like small deformations or extra orange tip on the wire plug. (8/8)

We evaluate Omni-Scan for defect inspection. By comparing reconstructed 3DGS models, the system can localize visual defects such as scratches and patch tape. (7/8)

We show some object reconstructions here. (6/8)

Omni-Scan then trains two Gaussian Splat models, one each for the left and right scans. To align the two scans into a single frame, we extract colored point clouds from the models and initialize Iterative Closest Point (ICP) algorithm using the relative transform between the left

After collecting the images, Omni-Scan processes them to get object masks. It first extracts the foreground and uses SAM to generate candidate object masks. Then it evaluates these masks using two criteria: (1) Non-Robot score, which helps us to reject robot arm masks, and (2)

With Omni-Scan, a robot first grasps the object from the tabletop, then scans it by turning the object in front of a static camera to capture multiple viewpoints. It then performs a handover, passing the object from one gripper to the other to scan it again, which exposes

High-quality 3D object models are important for many tasks. In e-commerce, they let customers explore products interactively. In manufacturing, they serve as digital twins for automated defect inspection and people use them in sim2real robot learning tasks. (2/8)

Can a robot inspect all views of an object? Today @IROS, we present Omni-Scan from @berkeley_ai, a novel method for bimanual robo 360° object scanning & reconstruction using 3D Gaussian Splats. (1/8) 🔗 https://t.co/8emyJfUNk4

AI can generate text and images, can it generate and construct physical 3D structures? Blox-Net from @UCBerkeley @AUTOLab_Cal takes as input a set of available blocks and a word like "giraffe" and uses ChatGPT and custom software to guide a real 🤖robot to build it. 1/n

Robot See, Robot Do allows you to teach a robot articulated manipulation with just your hands and a phone! RSRD imitates from 1) an object scan and 2) a human demonstration video, reconstructing 3D motion to plan a robot trajectory. https://t.co/jH1kZqbtqC #CoRL2024 (Oral)