Come see our analysis of a Gaussian multi-index model #AISTATS2025 on Sunday at Hall A—E 183. My favorite result is when the dot product between the ideal vectors exceeds a threshold, gradient flow fails to separate them under correlation loss! 😎.

Very exciting research direction ! 🙌🏼.

cross-posted @bsimsek.bsky.social .(I'm slowly migrating there).

This was done in collaboration with Amire Bendjeddou & Daniel Hsu. 🙌 Paper link:

Another result is a tight characterization of the time complexity of the gradient flow early in dynamics, a generalization of the known result for single-index to multi-index models. This generalization applies to arbitrary geometries as well. 🙃.

Below this threshold, a mild overparameterization (log k factor) for k index vectors is sufficient to match the neurons to the ideal vectors due to a coupon collector argument. 🤠.

When the ideal vectors form an equiangular frame, all learned weights converge to their average (no matter how much overparameterization is used), which has turned from a saddle to a local minimum after a certain threshold of the dot product. 😲.

Searching for an exact inverse map from learned weights to ideal (concept) vectors is an intricate geometry question, even for "simple" idealized models. 🧐.

I now have a new account on Bluesky..follow me !.

Here is a link to my talk on distillation for neural networks. at Les Houches together with many other talks on algorithmic theories of learning 🙌 Thanks to organizers @_brloureiro and Vittorio.

Nice talk by Jarod Alper at JMM’25 🙌🏼

I don’t mind if o1 does not think clearly like humans. It’s great for computing formulas like integrals, even better in combination with Wolfram alpha 🙌🏼.

RT @jacobandreas: Ekin Akyürek (@akyurekekin) builds tools for understanding & controlling algorithms that underlie reasoning in language m….

6/n My application package is available at feel free to reach out! (n=6)

5/n I'm enthusiastic about championing Gaussian multi-index models as mathematically analyzable and insightful models for MLPs. I continue developing new results for this model, it is fun!.

4/n Here is a blog post: You may enjoy this exposition if you like Toy Models of Superposition.

3/n It is unclear whether the LLMs in the wild can be robustly interpreted by ad-hoc methods. My approach is to analyze toy models that give insight into the non-linear feature compression of the MLPs. Fascinating math challenges are to be expected in this journey!.

2/n Studying the loss landscape is essential for understanding deep learning optimization and generalization. I developed a combinatorial complexity framework that quantifies the non-convexity of the deep learning loss landscapes.

📢 I'm on the faculty job market this year! . My research explores the foundations of deep learning and analyzes learning and feature geometry for Gaussian inputs. I detail my major contributions👇Retweet if you find it interesting and help me spread the word! DM is open. 1/n.

RT @deepcohen: The Center for Computational Mathematics at Flatiron Institute is hiring research fellows (postdocs) to start next year -- a….