RT @YooYeonSung1: 🏆ADVSCORE won an Outstanding Paper Award at #NAACL2025 @naaclmeeting!!. If you want to learn how to make your benchmark *….

RT @slatornews: 👉 Recent studies from @AlibabaGroup, @Cohere_Labs, and @Google highlight major gaps in #multilingua….

RT @mziizm: 🚨 Excited to share our latest paper! .Multilingual LLMs are getting really good. But the way we evaluate them? Not the best som….

RT @bryanlics: Externally retrieving knowledge empowers LLMs for domain-adapted MT ⚖️🩺. But how is knowledge best represented, and how viab….

RT @iseeaswell: 😼SMOL DATA ALERT! 😼Anouncing SMOL, a professionally-translated dataset for 115 very low-resource languages! Paper: https://….

RT @_danieldeutsch: 🚨New machine translation dataset alert! 🚨We expanded the language coverage of WMT24 from 9 to 55 en->xx language pairs….

Contaminated data can inflate #LLM performance—which factors matter the most? . Our controlled, multilingual pre-training study examines how contamination timing ⏰, data format 📄, model size ⚖️, and language representation 🌍 impact performance overestimation.

RT @_danieldeutsch: New application link! I am at EMNLP/WMT this week. Please come find me if you want to learn mo….

RT @CohereForAI: Introducing ✨Aya Expanse ✨ – an open-weights state-of-art family of models to help close the language gap with AI. Aya Ex….

RT @naaclmeeting: 📢 NAACL needs Reviewers & Area Chairs! 📝. If you haven't received an invite for ARR Oct 2024 & want to contribute, sign….

RT @_danieldeutsch: Interested in doing research on Google Translate and Gemini? Good news! I’m hiring for full-time roles on the Google Tr….

RT @slatornews: Researchers from @Google reveal that verbose #LLMs, 🤖 which offer multiple translations 🔄 or refuse to translate, 🚫 pose si….

RT @naaclmeeting: 📢 Call for demos is out!!. #NAACL2025 #NLProc. Check the website for submission guidelines and a chance to win the Best D….

RT @davlanade: Join my lab! I’m currently recruiting new students (MSc & PhD) for admission in the fall of 2025 at.@Mila_Quebec. https://t.c….

Joint work with Zhongtao Liu, Colin Cherry, and Markus Freitag at @Google @GoogleAI . #NLProc #NLP.

[5/5] ⚖️ How do such verbose translations impact LLM rankings? . Our findings show that current evaluation methods, both automatic and human, over-penalize more verbose LLMs, leading to potentially misleading rankings.

[4/5] 💬 Beyond refusal to translate, LLMs might contextualize or explain their translations, usually to compensate for insufficient input context. Of the LLMs studied, #Gemini provides translation commentary more frequently.

[3/5] 🚫 The most prevalent form of verbosity is their refusal to translate. Our analysis uncovers three main triggers: safety concerns, detection of copyrighted content, or encountering source text in non-natural language--with varying priorities across different models.

[2/5] We found that verbosity is a common trait among LLMs, though its prevalence differs significantly across models and languages. #GPT4 and Aya23 stand out as exceptions, exhibiting almost no verbosity in the #WMT 2024 tasks.

[1/5] Are verbose #LLM translations skewing evaluation results? . TLDR: Yes!. Our recent work dives into the prevalence and impact of LLM verbosity in automatic and human evaluations. 📎 Paper: