🚀 Excited to announce our paper "No Preference Left Behind: Group Distributional Preference Optimization (GDPO)" has been accepted to #ICLR2025! 🎉 We investigate: How can a probabilistic LLM align with distributional preferences within a group? 🧵[1/n]

RT @wregss: Training text-to-image models?. Want your models to represent cultures across the globe but don't know how to systematically ev….

RT @lucy3_li: I'm joining @WisconsinCS @uwcdis as an assistant professor in fall 2026!! There, I'll continue working on language models, co….

🧐Check it out at:

Sadly, I won’t be at #ICLR2025 this year 😭— but my amazing labmate @RUppaal will be there presenting our GDPO paper “No Preference Left Behind: Group Distributional Preference Optimization” 😁 .Catch our poster at Session 3 (Hall 3 + Hall 2B), #533, on Friday🗓️from 10am–12:30pm

RT @RUppaal: Stop by our ICLR poster tomorrow! .3-5:30PM local time, at Hall3 - poster #528. We have an exciting bit of work drawing princi….

[n/n] 📄 Read the paper: 👥 Joint work with amazing folks! @Zefan_Cai, @SeanChuang4, @shanglinbadger, @SeleenaJiang, @Diyi_Yang and @JunjieHu12.🖥️ Code & data coming soon—stay tuned!.

[6/n] 🔬 Experiments tell the story!.On synthetic opinion data & real-world movie reviews, we show that:.❌ DPO struggles to align with belief distributions. ✅ GDPO consistently improves pluralistic alignment during training.

[5/n] How does GDPO help? Unlike instance-based methods, GDPO statistically estimates belief distributions and optimizes learning across all preferences—majority & minority. ✅ GDPO boosts reward margins for both groups, effectively reducing the alignment gap.

[4/n] Why not DPO? 🤔DPO & similar methods skew toward dominant preferences, ignoring minority opinions. We show that training on conflicting preference data causes LLMs to gradually ignore minority preferences, worsening the alignment gap.

[3/n] GDPO, incorporating an epistemological concept of human beliefs that shape individual preferences, calibrates LMs using statistical estimation of the group’s belief distribution and aligns the model with belief-conditioned preferences.

[2/n] We are motivated by a key observation: Human preferences aren't singular—they follow a distribution. Instead of aligning LLMs to one preference at a time, GDPO aligns models with the full spectrum of group preferences using a belief-conditioned alignment objective.

RT @RUppaal: Excited to share our latest research on improving the safety of LLMs! We've developed DeTox, a tuning-free and noise robust al….

To bridge this gap, we present CAMT with CSI annotations in six language pairs, offering insights into culturally-aware machine translation. Our results highlight LLMs’ superior ability to handle CSIs through external cultural knowledge.

Translating culture-specific items (CSIs) is essential for effective cross-cultural communication, but these items often lack direct translations. This limitation challenges the ability of machine translation (MT) systems—both neural MT and large language models (LLMs).

Check out our paper here: Join us for a chat at our poster session on Nov. 12, 4:00-5:30 PM. #EMNLP2024 #MachineTranslation #CulturalAwareness.

🚀 Excited to present our paper "Benchmarking Machine Translation with Cultural Awareness" at #EMNLP2024! . We build CAMT, a novel parallel corpus enriched with culture-specific item annotations, and evaluate how well NMT and LLM-MT systems handle cultural entities.

RT @SeanChuang4: Excited to present my paper on role-playing LLM agents at #EMNLP2024! 🎉. Paper Title: “Beyond Demographics: Aligning Role-….

RT @SeanChuang4: (1/9) LLM agents are increasingly used to simulate human-like societies. But how human-like are their “social interactions….