How well can LLMs & deep research systems synthesize long-form answers to *thousands of research queries across diverse domains*? Excited to announce 🎓📖 ResearchQA: a large-scale benchmark to evaluate long-form scholarly question answering at scale across 75 fields, using

🤖✨ What if models that take action in the physical world could think through your instructions? Meet MolmoAct, our new fully open Action Reasoning Model (ARM) that does just that. 🧵

🤖💬 Herding instincts… in AIs? Yes, even LLMs can follow the crowd! • 📉 Conformity ↑ when agents lack confidence but trust peers • 🧠 Presentation format shapes peer influence • 🎯 Controlled herding can boost collaboration outcomes 👉 Read more: https://t.co/Ym0rtKyVzH

Successfully defended my PhD thesis and got hooded this week! Thanks to all the friends who supported me throughout this incredible journey! Excited to join PRIOR at @allen_ai next and continue exploring open vision-language research!

🎉CoSyn is accepted by ACL2025!

#NAACL2025 How to compare cultural differences with social media data in scale? Our work uses lexica to annotate X 🇺🇸 & Weibo 🇨🇳 posts with valence (😄☹️) & arousal (🔥❄️) scores, revealing cross-cultural differences in emotional expression. https://t.co/2tNFceO9GD

#ICLR2025 Oral LLMs often struggle with reliable and consistent decisions under uncertainty 😵‍💫 — largely because they can't reliably estimate the probability of each choice. We propose BIRD 🐦, a framework that significantly enhances LLM decision making under uncertainty. BIRD

Exciting news! 🎉 Our paper “ViUniT: Visual Unit Tests for More Robust Visual Programming” got accepted at #CVPR2025

✨ Introducing MutaGReP (Mutation-guided Grounded Repository Plan Search) - an approach that uses LLM-guided tree search to find realizable plans that are grounded in a target codebase without executing any code! Ever wanted to provide an entire repo containing 100s of 1000s of

This work is done during my great summer internship at @allen_ai with my awesome collaborators: Ajay Patel, @mattdeitke, @tanmay2099, @LucaWeihs, @drewmikehead, @yatskar, Chris Callison-Burch, @RanjayKrishna, @anikembhavi, Christopher Clark.

We also show we can create synthetic pointing data to improve the click accuracy of VLMs in GUI agent tasks. On the ScreenSpot click prediction benchmark, our model trained on synthetic pointing data can outperform existing methods with much less training data.

We notice open VLMs struggle with novel out-of-domain tasks like interpreting nutrition labels. However, CoSyn’s controllable data generation can create targeted synthetic data for task-specific fine-tuning, achieving strong zero-shot performance with significantly less data.

On 7 text-rich benchmarks (e.g., ChartQA, DocVQA), our model trained on synthetic data outperforms competitive open and proprietary VLMs. Our zero-shot model, trained without benchmark examples, beats most baselines, proving the generalizability of training on synthetic data.

CoSyn uses code as the intermediate representation to build synthetic multimodal datasets. We prompt a text-only LLM to generate code that renders images, and then we use code as context to create instruction-tuning data, such as QA pairs, for fine-tuning VLMs.

Our CoSyn framework integrates 11 rendering tools for 20 robust generation pipelines, which support the creation of diverse text-rich images, including charts, documents, diagrams, tables, and even music sheets 🎼, and many more!

We share Code-Guided Synthetic Data Generation: using LLM-generated code to create multimodal datasets for text-rich images, such as charts📊, documents📄, etc., to enhance Vision-Language Models. Website: https://t.co/9IQ4CgeKMF Dataset: https://t.co/yiERrZup8X Paper:

Articulate Anything has just been accepted to @iclr_conf #ICLR2025 ! Looking forward to seeing everyone in Singapore 🇸🇬 🙀❤️!

📢Applications are open for summer'25 internships at the PRIOR (computer vision) team @allen_ai: Come join us in building large-scale models for: 📸 Open-source Vision-Language Models 💻 Multimodal Web Agents 🤖 Embodied AI + Robotics 🌎 Planet Monitoring Apply by December

Excited to share ✨ Contextualized Evaluations ✨! Benchmarks like Chatbot Arena contain underspecified queries, which can lead to arbitrary eval judgments. What happens if we provide evaluators with context (e.g who's the user, what's their intent) when judging LM outputs? 🧵↓

🤔What model explanation method should you use? How to ensure it reflects the model’s true reasoning? 🌟 In our CL survey, Towards Faithful Model Explanation in NLP, we review 110+ explainability methods through the lens of faithfulness. Check out my presentation at #EMNLP2024!