RT @maxzimmerberlin: Come visit our #ICLR2025 poster „On the Byzantine-Resilience of Distillation-Based Federated Learning“ in Poster Sessi….

RT @maxzimmerberlin: Now with a blogpost (and the first one on my website!):

RT @BerkantTuran_: Join our poster session *today* at #ICML2024 in the TF2M Workshop. Looking forward to the inspiring discussions.

RT @maxzimmerberlin: A good time to share our #ICLR2023 paper:.How I Learned to Stop Worrying and Love Retraining. We explore sparsity-adap….

For details, see or come talk to us at the poster session on Tuesday 3pm in Hall C 4-9. 8/8.

Removing the assumption of bounded iterates uncovers a complex landscape of tradeoffs between oracle complexity, bounds on D, efficient computability of updates, and whether prior knowledge of the initial distance to the optimizer is needed. 7/8

Next, we show that the prox is quasi-nonexp. and that the Moreau envelope is smooth, prove convergence of RIPPA for general manifolds (prev. results only applied to Hadamard manifolds) showing D=O(R) and provide efficient implementations for smooth min and min-max opt. 6/8.

For L-smooth and g-convex functions, the seemingly different convergence rates of O(ζ_D LR²/ɛ) [ZS16] and O(LD²/ɛ) [MP23] which simply assume D is bounded both coincide to O(ζ_R² LR²/ɛ). Further we show that setting the step size to 1/(L ζ_R), RGD becomes quasi-nonexpansive. 5/8.

We show for RGD with 1/L step size that the distance D between the iterates and an optimizer is of order O(R ζ_R), where R is the initial distance to the optimizer and ζ_R= ϴ(R+1) is a geometric quantity. Applying this bound on D to prior results has surprising effects: 4/8.

We revisit two foundational algorithms: Riemannian GD (RGD) and Riemannian inexact PPA (RIPPA), showing that the iterates stay in a ball of radius D around an optimizer, removing the unreasonable assumption. 3/8.

1) the convergence rates of the algorithms often depend on geometric quantities scaling with the radius D of this set and 2) in some manifolds the condition number of a function (κ=L/μ) scales with the size of the optimization domain. This leads to unfinished analyses. 2/8.

In our paper "Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point" at #ICML2024, we examine a blind spot in the Riemannian opt literature: Most works simply 𝘢𝘴𝘴𝘶𝘮𝘦 that the iterates stay in a bounded set. This is a problem because 1/8

RT @maxzimmerberlin: 🌟 Join our Team in Berlin 🌟. We are seeking highly motivated PhD students to work on (efficient) Deep Learning, prefer….

RT @maxzimmerberlin: On my way to Vienna for #ICLR2024 with our paper "Sparse Model Soups: A Recipe for Improved Pruning via Model Averagin….

Excited to be at #NeurIPS2023! We'll be presenting our work on Accelerated Riemannian Min-Max Optimization at the OPT+ML workshop during poster session 2 from 3:00 - 4:00 pm.