🚨 Checkout our new method ZEAL -> zero-shot action localization in long-form video using LLMs + VLMs, accepted to the VidLLMs workshop at #CVPR2025! .📍 No labels. No fine-tuning. Strong zero-shot results. 📝💫 Shoutout to my amazing collaborator.

RT @nickhjiang: Vision transformers have high-norm outliers that hurt performance and distort attention. While prior work removed them by r….

RT @lschmidt3: Very excited to finally release our paper for OpenThoughts!. After DataComp and DCLM, this is the third large open dataset m….

RT @Zhang_Yu_hui: 📢 The First Workshop on Multimodal Foundation Models for Biomedicine (MMFM-BIOMED) at #CVPR2025 is still accepting submis….

RT @jmhb0: I'm at #ICLR2025 presenting "Video Action Differencing". Keen to chat with anyone interested in MLLMs - both for general data….

We dive more in our paper: 📄

(6) ⭐Shoutout to our lead-author Anita Rau and the amazing co-authors who made this work possible: @mark_endo1, Jaewoo (Jeff) Heo, @KhaledSaab11, Alberto Paderno, Jeffrey Jopling, Chris Holsinger, and @yeung_levy!.

*Select models. Comprehensive results can be found in the paper.

(5) 📌General-purpose models like GPT-4o excel on lower-expertise tasks (e.g., tool recognition) by drawing on their broad understanding of common concepts, but struggle with tasks requiring domain expertise. .

(4) 📌They demonstrate strong in-context learning: with as few as five examples, leading VLMs can triple their performance—highlighting major potential for low-resource and rare clinical use cases. .

(3) 📌VLMs can generalize across tasks and domains, sometimes even outperforming state-of-the-art supervised models when tested outside their training distribution. .

(2) We also explore how these tasks map onto real, clinically impactful applications, and offer an educated guess at the most viable near-term use of VLMs in surgery. Our key findings were. .

(1) We asked.🥇 Which models show the most promise?.⚖️ How do general-purpose models stack up against domain-specific ones?.🔓 What surgical tasks can AI realistically tackle today?.🎯 Can zero-shot VLMs rival supervised models trained on thousands of expert-annotated examples?.

There’s growing excitement around VLMs and their potential to transform surgery🏥—but where exactly are we on the path to AI-assisted surgical procedures?. In our latest work, we systematically evaluated leading VLMs across major surgical tasks where AI is gaining traction. 🧵.

RT @orr_zohar: 🤗The SmolVLM report is out, with all the experiments, findings, and insights that led to high performance at tiny sizes🤏. 📱….

RT @Ale9806_: Earlier this year, we released the BIOMEDICA dataset, featuring 24 million unique image caption pairs and 30 million image re….

Tackling huge challenges at the intersection of #cv #nlp #healthcare. Submissions are still open!.