1/7 Still using Adam?. If anyone wants to try a distributed PyTorch implementation of SOAP/eigenvalue-corrected Shampoo with support for low precision data types instead, here you go.

RT @ThomasTCKZhang: I’ll be presenting our paper “On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning” a….

RT @kayembruno: You don't need bespoke tools for causal inference. Probabilistic modelling is enough. I'll be making this case (and dodgin….

RT @tmpethick: When comparing optimization methods, we often change *multiple things at once*—geometry, normalization, etc.—possibly withou….

RT @akristiadi7: 📢 [Openings] I'm now an Assistant Prof @WesternU CS dept. Funded PhD & MSc positions available! Topics: large probabilisti….

RT @joost_v_amersf: Never will be.

RT @MarkSchmidtUBC: My former PhD student Fred Kunstner has been awarded the @c_a_i_a_c Best Doctoral Dissertation Award:..

RT @aaron_defazio: Why do gradients increase near the end of training? .Read the paper to find out!.We also propose a simple fix to AdamW t….

RT @Ar_Douillard: duality of humanity

RT @orvieto_antonio: Adam is similar to many algorithms, but cannot be effectively replaced by any simpler variant in LMs. The community is….

RT @_katieeverett: 1. We often observe power laws between loss and compute: loss = a * flops ^ b + c.2. Models are rapidly becoming more ef….

RT @aaron_defazio: Write the paper you would want to read.

RT @roydanroy: This is a huge development. I want to highlight the theoreticians behind the scene, because this paper represents the reali….

RT @kayembruno: Great to be back from Singapore from #ICLR2025, and super excited to have given my first oral presentation on influence fun….

RT @JonathanWenger5: We have a fantastic lineup of speakers who have made deep contributions to open-source in ML, e.g. @sarahookr, @ChrisR….

RT @zhiyuanli_: Why does Adam outperform SGD in LLMs training? Adaptive step sizes alone don't fully explain this, as Adam also surpasses a….

RT @GuilleAngeris: @typedfemale literally is all you need.

RT @CambridgeMLG: All the MLG papers at #ICLR2025 main conference!

RT @wormaniec: Ever wondered how the loss landscape of Transformers differs from that of other architectures? Or which Transformer componen….

RT @frankstefansch1: Tired of your open-source ML work not getting the academic recognition it deserves? 🤔 Submit to the first-ever CodeML….

RT @JonathanWenger5: Built a new ML library? Maintain a crucial project? Improved OSS practices? Your work deserves recognition! Submit you….