RT @robinson_n8: This was my star-struck 🤩 moment at @aclmeeting, getting to have lunch with @guzmanhe, @costajussamarta, and of course the….

RT @davlanade: @seb_ruder giving the first keynote talk on Llama 4

RT @s_scardapane: *The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs*.by @p_nawrot @PontiEdoardo @cheeesio @seb_ruder. T….

I'll be at ACL 2025 in Vienna next week. Say hi if you want to chat about multilinguality, LLM evaluation or doing research in industry. You can also check out our ACL papers:.- M-RewardBench (Main): - Global MMLU (Main): - Arabic.

RT @cheeesio: The Multilingual Team at @cohere is hiring!. If this sounds like you, please apply:.- strong coding skills and a keen eye for….

RT @hugobowne: I had lunch with @seb_ruder in Berlin a few days ago. Had delicious food and a wonderful, generative conversation about how….

RT @p_nawrot: We built sparse-frontier — a clean abstraction that lets you focus on your custom sparse attention implementation while autom….

@p_nawrot @cheeesio @PontiEdoardo There is a huge amount of variety in this research area spanning.- when sparse attention is used (prefilling vs decoding).- which units are sparsified (blocks or vertical slashes).- what type of patterns are used (fixed or content-aware).- how the computational budget is

Check out the paper for more insights and details: . This was a fun collaboration with @p_nawrot, @cheeesio, @PontiEdoardo, Robert Li, and Renjie Huang!.

Our findings:.1) For short seqs, increasing density or size provides gains. For long seqs, high sparsity performs best. 2) Higher sparsity is possible for decoding and larger models. However, most configs deteriorate performance significantly for at least one task. 3) There is no

The Sparse Frontier. Efficient sparse attention methods are key to scale LLMs to long contexts. We conduct the largest-scale empirical analysis that answers:.1. 🤏🔍 Are small dense models or large sparse models better?.2. ♾️What is the maximum permissible sparsity per task?.3.

RT @_akhaliq: The Sparse Frontier. Sparse Attention Trade-offs in Transformer LLMs

RT @p_nawrot: Sparse attention is one of the most promising strategies to unlock long-context processing and long generation reasoning in L….

I'm super excited about these new models and what's still to come, in English and many more languages! 🌍.

RT @robertarail: Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬. We introduce MLGym and MLGym-Bench,….

@AIatMeta I've had a great time at @cohere where we made a lot of progress building multilingual LLMs. I wish my previous colleagues, in particular the Multilingual team all the best. cc @cheeesio @weiyinko_ml @KocmiTom @SCahyawijaya Alex Bérard, Théo Dehaze, Nithya Govindarajan.

A new year, a new challenge. I recently joined @AIatMeta to improve evaluation and benchmarking of LLMs. I'm excited to push on making LLMs more useful and accessible, via open-sourcing data/models and real-world applications. I'll continue to be based in Berlin.

RT @yanaiela: On that note, someone organizing a workshop at @aclmeeting (ACL 2025) wants to switch with our NAACL 2025 slot?.(I guess it's….

Reward models are crucial for aligning models to human preferences but so far their evaluation has been limited to English. I was fortunate to be involved with this @CohereForAI project, which introduces a new multilingual RM benchmark and many insightful analyses.

RT @sGx_tweets: ✨ New Evaluation Benchmark for Reward Models - We Go Multilingual! ✨. Introducing M-RewardBench: A massively multilingual R….