Jiageng Mao
@PointsCoder
Followers
424
Following
202
Media
7
Statuses
52
Can Vision-Language Models (VLMs) truly understand the physical world? 🌍🔬. Introducing PhysBench – the first benchmark to evaluate VLMs’ understanding of physics! PhysBench is accepted to #ICLR2025 as an Oral presentation (only 1.8% out of 11k submissions)!. 🌐 Project:
5
74
413
This project is co-led by our incredible intern Wei Chow and me, and I am especially grateful to my advisor, @yuewang314 , for his invaluable guidance and support throughout this work. 🙏 We also deeply appreciate the contributions and insights of @Boyiliee, @DanielSeita, and.
0
0
9
How do we fix this?.Introducing PhysAgent 🚀 – a new framework that enhances VLMs by integrating:.🔹 Vision foundation models (Depth, SAM, GroundingDINO).🔹 A physics knowledge memory for improved reasoning.🔹 Chain-of-thought inference for self-verification.PhysAgent boosts
1
1
14
What did we find?.🧐 We evaluated 75 top VLMs, including GPT-4o, Gemini, and open-source models, and found:.✅ Strong commonsense reasoning but poor physical reasoning.✅ Closed-source models outperform open-source ones, but still struggle.✅ Scaling data and model size does not
1
2
13
What is PhysBench?.PhysBench is a comprehensive benchmark with 10,002 video-image-text entries that assess VLMs across four major domains:. 1️⃣ Physical object properties (number, mass, stiffness, elasticity, etc.).2️⃣ Physical object relationships (distances, depths, velocities,
1
0
11
Great work! Can't wait to try on our robot!.
🚀 Can we make a humanoid move like Cristiano Ronaldo, LeBron James and Kobe Byrant?. YES!. 🤖 Introducing ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills. Website: Code:
0
1
13
Robotic apple🍎 peeling: not quite ready to open a fruit stand, but A for effort 🤖
0
0
7
Thanks @drmapavone for sharing my internship work! It was a fantastic experience collaborating with you and your team at @nvidia. DreamDrive is our preliminary exploration of driving everywhere leveraging the Internet street view images. Stay tuned for more updates!.
Introducing DreamDrive, which combines the complementary strengths of generative AI (video diffusion) and neural reconstruction (Gaussian splatting) to transform any street-view image into a dynamic 4D driving scene!. Web: Paper:
0
4
31
Check out our first paper on humanoid robots!!!.
🎬Can internet videos enhance the scalability of humanoid learning?. 🤖Introducing Humanoid-X, a comprehensive dataset comprising over 20 million humanoid robot poses paired with text-based motion descriptions, on which we develop Universal Humanoid-1 (UH-1), a large model for
0
0
14
RT @yuewang314: [Hiring!] I am hiring multiple PhDs @CSatUSC @USCViterbi for this cycle. If you're interested in scene representations, neu….
0
49
0
RT @haoyue_bai: AHA: Human-Assisted Out-of-Distribution Generalization and Detection #NeurIPS 2024. AHA strategically labels examples withi….
0
2
0
RT @yuewang314: Agent-Driver is accepted to @COLM_conf with top 1% reviews (7, 7, 8, 9) among all submissions. We sincerely thank reviewers….
0
18
0
RT @yuewang314: Amazing Friday! Our USC-Stanford (@USCViterbi @StanfordEng) joint team, led by students @PointsCoder @JunjieYe9 and co-advi….
0
4
0
RT @UnitreeRobotics: Unitree H1 The World's First Full-size Motor Drive Humanoid Robot Flips on Ground. Unitree H1 Deep Reinforcement Learn….
0
421
0
I like this humanoid.
🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time?. 🤖 Introducing H2O (Human2HumanOid):.- 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework.- 💃 Scalable retargeting and training using large
0
0
0