Maximilian Beck
@maxmbeck
Followers
889
Following
640
Media
32
Statuses
223
ELLIS PhD Student @ JKU Linz Institute for Machine Learning & PhD Researcher @nx_ai_com, Research Scientist Intern @Meta FAIR
Linz, Österreich
Joined June 2021
Yesterday, we shared the details on our xLSTM 7B architecture. Now, let's go one level deeper🧑🔧. We introduce. ⚡️Tiled Flash Linear Attention (TFLA), ⚡️. A new kernel algorithm for the mLSTM and other Linear Attention variants with Gating. We find TFLA is really fast!. 🧵(1/11)
3
61
346
RT @HochreiterSepp: NXAI has successfully demonstrated that their groundbreaking xLSTM (Long Short Term Memory) architecture achieves excep….
0
21
0
RT @KorbiPoeppel: Ever wondered how 'Composition over Inheritance' can be used efficiently in ML Experiment Configuration (and beyond)?.Che….
0
2
0
MesaNet is beautiful! A great paper with extensive benchmark of recent RNNs (including xLSTM) on synthetic tasks and language modeling.
Super happy and proud to share our novel scalable RNN model - the MesaNet! . This work builds upon beautiful ideas of 𝗹𝗼𝗰𝗮𝗹𝗹𝘆 𝗼𝗽𝘁𝗶𝗺𝗮𝗹 𝘁𝗲𝘀𝘁-𝘁𝗶𝗺𝗲 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 (TTT), and combines ideas of in-context learning, test-time training and mesa-optimization.
1
1
18
RT @KorbiPoeppel: Ever wondered how linear RNNs like #mLSTM (#xLSTM) or #Mamba can be extended to multiple dimensions?.Check out "pLSTM: p….
0
38
0
RT @HochreiterSepp: Mein Buch “Was kann Künstliche Intelligenz?“ ist erschienen. Eine leicht zugängliche Einführung in das Thema Künstliche….
0
4
0
RT @SonglinYang4: 📢 (1/16) Introducing PaTH 🛣️ — a RoPE-free contextualized position encoding scheme, built for stronger state tracking, be….
0
88
0
Excited to share that 2 of our papers on efficient inference with #xLSTM are accepted at #ICML25. A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks (. and . xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference:.
📢🔔I am excited to share the details on our optimized xLSTM architecture for our xLSTM 7B model!🚨. We optimized the architecture with two goals in mind:. - Efficiency (in Training and Inference).and .- Stability. 🧵(1/7)
2
10
71
RT @HochreiterSepp: xLSTM for Multi-label ECG Classification: "This approach significantly improves ECG classifica….
0
10
0
RT @KorbiPoeppel: Hope to see you around at #ICLR2025 in #Singapore!.I'm happy to present our work on xLSTM kernels, applications and scali….
0
1
0
I will also present our recent papers: . xLSTM-7B .( .and .Tiled Flash Linear Attention .( . on .Sunday in the 2nd Workshop on Foundation Models in the Wild Workshop.and on .Monday in the Open Science for Foundation Models Workshop.
0
1
2
⚡️Fast sLSTM kernels via the FlashRNN framework.Thursday 10-12.30, Hall 3 & 2B. 📷xLSTM for vision via the Vision-LSTM.Friday 15-17.30, Hall 3 & 2B.
1
0
2
I will talk about our xLSTM 7B, today! Tune in 💫.
🚀 Join us for an exclusive discussion on xLSTM 7B!. To the future of fast and efficient LLMs w/ Maximillian Beck, PhD researcher at Johannes Kepler University & protégé of Mr. LSTM himself, Sepp Hochreiter. Hosted by @ceciletamura of @ploutosai .
0
1
23
Does SSMax in Llama4 avoid attention sinks?.
@maxmbeck @PetarV_93 Great question! I imagine that temperature scaling should actually make sinks stronger (as it should help sharpen attention patterns over long context) -- although we have not checked yet. Worth noting that we proposed something similar to SSMax here
0
0
1
RT @julien_siems: 1/9 There is a fundamental tradeoff between parallelizability and expressivity of Large Language Models. We propose a new….
0
34
0
1
0
10