🎉 Excited to share: "𝐌𝐢𝐱𝐭𝐮𝐫𝐞-𝐨𝐟-𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬 (𝐌𝐨𝐓)" has been officially accepted to TMLR (March 2025) and the code is now open-sourced! 📌 GitHub repo: https://t.co/KiDbxpDWt0 📄 Paper: https://t.co/KQoZ3cunEf How can we reduce pretraining costs for

Even the smartest LLMs can fail at basic multiturn communication Ask for grocery help → without asking where you live 🤦‍♀️ Ask to write articles → assumes your preferences 🤷🏻‍♀️ ⭐️CollabLLM (top 1%; oral @icmlconf) transforms LLMs from passive responders into active collaborators.

Thank you, @VictoriaLinML , for the write-up.

🚀 Excited to share the most inspiring work I’ve been part of this year: "Learning to Reason without External Rewards" TL;DR: We show that LLMs can learn complex reasoning without access to ground-truth answers, simply by optimizing their own internal sense of confidence. 1/n

🌐 On United Nations (UN) adoption: Even the world's most prominent international bodies are embracing LLMs! UN press releases showed a rapid initial surge (3.1% to 10.1%) in early 2023, then steadily climbing to 13.7% by Q3 2024.

Work done in collaboration w/@yaohuiz3, @m_codreanu, Jiayu Wang, @CaoHancheng, @james_y_zou

🔍 Key findings: - Lower education areas showed higher LLM adoption in consumer complaints - Urban areas have higher LLM usage (18.2% vs 10.9%) - Science & tech companies lead in corporate adoption - Younger firms (post-2015) use LLMs 3x more than older ones (pre-1980)

🚨 New research: We analyzed 1.5M+ documents to track LLM-assisted writing adoption across society from 2022-2024. The results? 📊By late 2024, LLMs assist in writing: - 18% of financial consumer complaints - 24% of corporate press releases - Up to 15% of job postings (esp. in

Thanks @liang_weixin We all enjoyed reading the paper! And we appreciate your paper for helping the community gain a deeper understanding of the modality gap 🥰

We are excited to announce that Voyage AI is officially joining @MongoDB ! Joining @MongoDB enables us to bring our cutting-edge AI retrieval technology to a broader audience and seamlessly integrate it into mission-critical applications. Learn more: https://t.co/V8PTq3v5ZM

Update: check out https://t.co/U94zIRGSoj for our code, data, and model!

We introduce Mixture-of-Mamba, a multi-modal SSM that leverages modality-aware sparsity for efficient multi-modal pretraining! At the core of Mixture-of-Mamba: 🔹Modality-aware sparsity to optimize efficiency 🔹Mixture-of-SSMs to enable cross-modal interactions 🔹Scales

🚀 Want 2x faster pretraining for your multi-modal LLM? 🧵 Following up on Mixture-of-Transformers (MoT), we're excited to share Mixture-of-Mamba (MoM)! https://t.co/OTTpAlB4Vq 🔥 Why it matters: MoM applies modality-aware sparsity across image, text, and speech—making

📢 Can LLMs program themselves to run faster? 🏃⏱️ LLM self-taught to code for next-gen AI hardware! https://t.co/wiwgiPEpeH 1/ Programming AI accelerators is a major bottleneck in ML. Our self-improving LLM agent learns to write optimized code for new hardware, achieving 3.9x

🔍 ML library development is crucial but requires expertise in ML algorithms & architecture-specific programming languages (ASPLs). 🤖 LLM agents can enable better automation. We propose an adaptive self-improvement agentic system for generating ML libraries in STeP—a

🔍 Vision language models are getting better - but how do we evaluate them reliably? Introducing AutoConverter: transforming open-ended VQA into challenging multiple-choice questions! Key findings: 1️⃣ Current open-ended VQA eval methods are flawed: rule-based metrics correlate

Introducing 𝐋𝐥𝐚𝐦𝐚𝐅𝐮𝐬𝐢𝐨𝐧: empowering Llama 🦙 with diffusion 🎨 to understand and generate text and images in arbitrary sequences. ✨ Building upon Transfusion, our recipe fully preserves Llama’s language performance while unlocking its multimodal understanding and

🤔 Why are VLMs (even GPT-4V) worse at image classification than CLIP, despite using CLIP as their vision encoder? Presenting VLMClassifier at #NeurIPS2024: ⏰ Dec 11 (Wed), 11:00-14:00 📍 East Hall #3710 Key findings: 1️⃣ VLMs dramatically underperform CLIP (>20% gap) 2️⃣ After

I’m open to academia & industry in 2025. My work in #XR 🥽 + #HCI 👩‍💻 enables low-friction XR experience thru #EmbodiedInteraction, unlocking potential for all -- tech-savvy or not 🌍 Design+Science+Engineering. Let's shape the future of spatial computing ✨ RT appreciated! (1/8)

Honored that @Nature has highlighted our work again in their latest piece examining #ChatGPT's transformative impact on scientific research and academia over the past two years. h/t @Nature https://t.co/wK4ayZYH9w