Olga Golovneva
@OlgaNLP
Followers
1K
Following
1K
Media
13
Statuses
134
We have been cooking! 👨🍳.🧵(1/6).
🚨Multi-Token Attention🚨.📝: Attention is critical for LLMs, but its weights are computed by single query & key vectors, limiting capability. MTA combines query, key & head operations over multiple tokens, improving performance in terms of PPL, std
8
42
375
RT @jaseweston: 🪜Introducing: StepWiser🦉.📝: - Reframes stepwise reward modeling as a reasoning task: outputs CoT +….
0
97
0
RT @jaseweston: . is today a good day for new paper posts? .🤖Learning to Reason for Factuality 🤖.📝: - New reward f….
0
49
0
RT @julien_c: BREAKING:. we've partnered with @metaai and @paperswithcode to build a successor to Papers with Code (which was sunsetted yes….
0
221
0
After almost 10 years in AI, my most cited work is still about *underground* hidden layers
0
0
10
RT @adinamwilliams: Our team is hiring a postdoc in (mech) interpretability! The ideal candidate will have research experience in interpret….
0
10
0
✨MTA was accepted at #COLM2025 ✨ Since our first announcement, we have updated the paper with scaling laws, new baselines, and more evaluations! Code is now available in our repo: @COLM_conf.
We have been cooking! 👨🍳.🧵(1/6).
0
5
19
RT @jaseweston: Reasoning, Attention & Memory Workshop @ COLM.Submission Deadline: June 23, 2025 -- Today!.
0
14
0
Don't lose it, reuse it! Instead of filtering out bad data samples, we propose to rewrite low-quality samples for better quality.
Web data, the “fossil fuel of AI”, is being exhausted. What’s next?🤔.We propose Recycling the Web to break the data wall of pretraining via grounded synthetic data. It is more effective than standard data filtering methods, even with multi-epoch repeats!.
0
0
13
🍿🍿🥤.
A follow-up study on Apple's "Illusion of Thinking" Paper is published now. Shows the same models succeed once the format lets them give compressed answers, proving the earlier collapse was a measurement artifact. Token limits, not logic, froze the models. Collapse vanished
0
0
2
Our presentations are now available online!.
The videos of the workshop are now online!.
0
1
4
Just look at the list of the invited speakers! Submit your papers by June 23rd, we are waiting for you 🫵🏿.
🚨Announcing RAM 2 workshop @ COLM25 - call for papers🚨 .- 10 years on, we present the sequel to the classic RAM🐏 (Reasoning, Attention, Memory) workshop that took place in 2015 at the cusp of major change in the area. Now in 2025 we reflect on what's happened and discuss the
0
1
7
RT @polkirichenko: We are hiring a PhD research intern at FAIR w/ @marksibrahim @kamalikac to start this summer or Fall!.Potential topics:….
0
36
0
RT @ArchikiPrasad: 🎉 Excited to share that my internship work, ScPO, on self-training LLMs to improve reasoning without human labels, has b….
0
35
0
Thanks to all the organizers, it was a pleasure to attend and learn from great speakers!.
2
3
36
There are still some spots in the room, but they are running low fast!.
0
3
12
RT @Ar_Douillard: the workshop is tomorrow!. and it will be livestreamed at for those not in Singapore.
0
6
0
RT @jaseweston: Google friends & ex-colleagues -- Google scholar seems pretty broken😔. Our most cited paper from last year "Self-Rewarding….
0
10
0