I’m thrilled to start my new journey as a PhD student at @MIT_CSAIL! 🎉 I’ll be working with the amazing @MarzyehGhassemi with a fellowship from @AIHealthMIT. I’m excited for this new chapter and look forward to meeting people working on ML, CV, or healthcare.

Huge thanks to my incredible collaborators @Sa_9810 , @YonglongT, @guohao_li, @philiptorr, Yoon Kim, and @MarzyehGhassemi ! This synthetic data approach is a simple but effective step, and we're excited to see other works already building on NegBench to push negation.

Why does this happen? VLMs are typically trained on image-caption pairs where captions describe what is in an image, almost never what is not. To tackle this, we propose generating massive synthetic datasets, where we explicitly inject negation into captions. Finetuning CLIP

Our findings are striking: even recent large VLMs (like AimV2) and CLIP-based models specifically designed to handle negation (like ConCLIP) still largely fail to understand it. Their performance on NegBench is often close to a random guess!. 5/n

To address this flaw, we built #NegBench: a benchmark designed to expose VLMs' struggles with negation. It covers a wide range of tasks, from natural image and video understanding to crucial medical X-ray image analysis, including naturally expressed negation. 4/n

This limitation has serious implications in critical domains. Imagine a doctor searching a database for 'evidence of lung swelling with no sign of enlarged heart' for a diagnostics task. If the AI misunderstands 'no,' it could retrieve misleading images, impacting patient care.

When is understanding negation crucial? In any task where visual concepts often appear together! For instance, if you're looking for a clear image of your apartment bedroom but have many photos of your cat in the apartment, a simple 'bedroom' query might return cat-filled images.

We've seen hilarious fails from generative models struggling with "NO" (e.g., asking for "a clear sky with no planes", and getting many planes!). But did you know this limitation extends to AI understanding tasks? Excited to share our @CVPR 2025 paper that studies this in detail!

RT @AIHealthMIT: Does an AI model understand "no(t)"? 🚫@KumailAlhamoud, Shaden Alshammari, @YonglongT, @guohao_li, Philip Torr, Yoon Kim, a….

RT @Sa_9810: Excited to share our ICLR 2025 paper, I-Con, a unifying framework that ties together 23 methods across representation learning….

RT @KrithikTweets: 🧬 Meet Lyra, a new paradigm for accessible, powerful modeling of biological sequences. Lyra is a lightweight SSM achievi….

RT @ybkim95_ai: 🤷‍♂️ When and why do Foundation Models hallucinate or confabulate in healthcare, and what's the real-world impact on medica….

RT @csail_alliances: Our top Spotlight of 2024 #2:. @MIT_CSAIL PhD student @KumailAlhamoud's research focus is on the generalizability of #….

RT @lateinteraction: 🔗 Thoughts on Research Impact in AI. Grad students often ask: how do I do research that makes a difference in the cur….

RT @KaustVision: 📢 Open Call: Fully funded #PhD student positions starting in 2025 at IVUL @KAUSTVision Lab with Prof. @BernardSGhanem. If….

I am very thankful for my amazing collaborators @YasirGhunaim and @MotassimF and for my mentors/supervisors @tom_hartvigsen @philiptorr @BernardSGhanem @Adel_Bibi @MarzyehGhassemi .I feel lucky to work with them and excited for what’s to come 🤗.

4/ 📂 Open Science: All code, dataset processing details, example bash scripts, and plotting guides are available on github. We're excited about this project and welcome any feedback or collaboration inquiries! .

3/ 🌐 Real-World Impact: We simulate realistic pandemic conditions (with COVID-19 data) making our work relevant for those working on either continual learning or AI for healthcare applications.

2/ 🏆 Key Findings:.•Simple batch-balancing techniques outperformed advanced methods. 😅.•Highlights the need for more realistic benchmarks simulating real-world healthcare settings.

🎉 Happy to share my first paper as a PhD student with @MarzyehGhassemi! It's accepted to MICCAI 2024! Here’s a thread:. 1/ 🌟 Introducing FedMedICL: A framework for evaluating federated continual learning in medical imaging, capturing label, demographic, and temporal shifts.

RT @yang_yuzhe: Life update: Absolutely thrilled to share that I'll be joining @UCLA as an Assistant Professor in summer 2025, with joint a….