RT @unireps: 📢 We're thrilled to announce that the UniReps workshop will return to @NeurIPSConf 25 for its 3rd edition!. 🔵Check our new Cal….

With amazing collaborators @nimatabari, @RobinSFWalters, and @yuqirose. 📄 “Understanding Mode Connectivity via Parameter Space Symmetry”. 🔗 (6/6).

We also derive closed-form expressions for the connecting paths. Loss remains low on curves induced by approximate symmetries. (5/6)

We use symmetry to characterize a large portion of the minima, and give examples of when mode connectivity holds or fails. (4/6)

This lets us count how many disconnected groups of solutions exist, and show how architectural features, like skip connections in ResNets, can make solutions more connected. (3/6)

Mathematical structure of symmetry groups is well understood. The structure of the loss landscape? Much less so. By linking the two, we use parameter space symmetries to uncover the structure of the loss landscape. (2/6).

When and why are neural network solutions connected by low-loss paths?. In our #ICML2025 paper, we show that mode connectivity often arises from symmetries—transformations of parameters that leave the network’s output unchanged. Paper: (1/6)

RT @HaggaiMaron: 🔬 Excited to announce our ICLR 2025 Workshop on *Weight Space Learning*! Applying deep learning to model weights could rev….

What can we learn from neural network model weights?. Join us for the Weight Space Learning Workshop at #ICLR2025! @iclr_conf . 📄Accepting extended abstracts & full papers .🗓️Submission Deadline: Feb 4, 2025 . 🔗

@unireps @scifordl Links to abstracts: . Parameter symmetry: Emotion representation in LLMs:

I'll be at #NeurIPS2024 next week! Happy to chat about symmetry, LLMs, and everything. We will present 2 workshop posters:. - Finding Symmetries in Neural Network Parameter Spaces @unireps . - Emergence of Hierarchical Emotion Representations in LLMs @scifordl

RT @WiMLworkshop: 📢 ⏰ Deadline Alert! ⏰ .Only 10 days left to submit your abstracts for the 19th Women in Machine Learning Workshop at #Neu….

Excited to share our latest paper on symmetries in neural network parameters! #ICLR2024. Oral: Wed 10:00am, Halle A 2.Poster: Wed 10:45am-12:45pm, Halle B #206. Disappointed that I can’t join in person due to visa issues, but thrilled that @RobinSFWalters will represent our work!

@neur_reps @gowerrobert @RobinSFWalters @yuqirose We prove a stronger convergence guarantee for SGD with teleportation. We additionally show that teleportation can be used to search for minimizers with better generalization ability. We also design a meta-learning algorithm that learns where to teleport.

@neur_reps @gowerrobert @RobinSFWalters @yuqirose Our paper builds on symmetry teleportation (, which exploits parameter space symmetry to explore loss level sets.

Looking forward to presenting our extended abstract “Improving Convergence and Generalization Using Parameter Symmetries” @neur_reps workshop, with @gowerrobert, @RobinSFWalters and @yuqirose. #NeurIPS2023. Paper:

RT @yuqirose: How to design Diffusion Models for spatiotemporal forecasting? Check out our #NeurIPS2023 Paper on ``DYffusion: A Dynamics-in….

RT @yuqirose: Many local minima in Deep Networks are connected through low-loss valleys. Curious about the theoretical origin of such valle….

RT @yuqirose: How can #symmetry help optimization in #DeepLearning? Check out our #NeurIPS2022 paper on ``Symmetry Teleportation for Accele….