✨✨Proud of my research group’s NeurIPS 2025 Spotlight paper on improving zero-shot performance of embodied AI agents using neurosymbolic representations! Come see us at the conference in San Diego, play with our live demo / model / dataset on HuggingFace, and check out our

We’d love to see you at NeurIPS 2025 @ San Diego — Exhibit Hall C/D/E, Booth #4908 Wed, Dec 3 · 11:00 a.m.–2:00 p.m. PST Thanks to all my collaborators: Amish Sethi, Matthew Kuo, Mayank Keoliya, Neelay Velingker, JungHo Jung, Prof. Ser-Nam Lim, Prof. Ziyang Li, Prof. Mayur Naik

Check out our foundation model demo, code, and full paper for more details! ESCA paper: https://t.co/0miwUYAnCc (NeurIPS 2025 spotlight) LASER demo: https://t.co/n4Mtgrj161 LASER paper: https://t.co/24lOYxlIqQ (ICLR 2025)

At the heart of ESCA is LASER, a 454M-parameter model for scene-graph generation. Trained on 87K+ open-domain videos with a neurosymbolic caption scene-graph alignment pipeline, it learns fine-grained video semantics with zero human labels, saving over 800,000 hours of annotation

Embodied agents operate in a think–act–feedback loop. ESCA processes each instruction and visual update through perception → reasoning → planning to produce the next action. By injecting scene-graph context into the prompt, ESCA boosts any MLLM’s perception—no finetuning needed

ESCA delivers large gains for embodied agents. Across four environments, it boosts tested MLLMs by an average of 34% and up to 209%, pushing both open-source and commercial models to SOTA performance. InternVL-2.5 with ESCA even surpasses base GPT-4o on navigation tasks.

ESCA enables any MLLM to avoid perception errors merely by prompting. The task planner now sees both text and annotated visual cues about all key objects, their attributes, and how they relate to each other, enabling more reliable reasoning and action decisions.

MLLMs like Gemini-3.0, GPT-4o (shown here), Qwen3-VL, and InternVL-2.5 frequently make basic perception errors when acting as embodied agents.

Announcing our ✨ NeurIPS’25 Spotlight ✨ paper: ESCA: Contextualizing Embodied Agents via Scene-Graph Generation TLDR: We introduce a framework that grounds multimodal embodied agents in scene graphs, leading to more reliable perception, stronger reasoning, and better actions.

I am delighted to introduce Vieira, a declarative language and framework for neuro-symbolic programming with foundation models. (1/9)

How can we seamlessly combine Language Models and Symbolic Reasoners to get the best of both worlds? Introducing our work appearing at #ACL2023 this week: Paper: https://t.co/BJhojTyaI9 Code: https://t.co/4AiaYNk10F 🧵👇 (1/6)

Registration is free! We will cover some recent advances in reasoning over graphs, as well as hands-on learning sessions. Look forward to seeing you during this online tutorial!

I will speak at Google NYC tomorrow (Thu May 19) at 11:30 ET. My talk will survey exciting results in AI for code, and discuss future challenges and opportunities. Based on joint work with @Pardis and @Aaditya, and @Hanjun and Petros Maniatis from @GoogleAI. (1/n)

Models dont understand negation remains my fav example.

I am excited to share TOGA, a method for neural generation of test oracles. Joint work with @gabe_ryan95 and @MSFTResearch @RiSE_MSR to appear at ICSE 2022. Writing good tests is time-consuming and challenging. Can AI help automate this? ⬇ (1/10)

Neurosymbolic combines neural models and logical reasoning programs. Enter Scallop: Datalog based(in Rust) Differentiable logical reasoning engine that can be integrated with deep neural models like CNN & transformers for perception and reasoning tasks. https://t.co/AeSU2ENwWB

I am excited to share a preview of Scallop: a new programming language and toolchain for neurosymbolic AI. Website: https://t.co/M5Tx1Otaua (1/18)