Nadav Cohen
@nadavcohen
Followers
735
Following
224
Media
12
Statuses
107
Do NNs need GD to generalize? Check out our new paper 👇.
Do neural nets really need gradient descent to generalize?🚨. We dive into matrix factorization and find a sharp split: wide nets rely on GD, while deep nets can thrive with any low-training-error weights!. 🧵
0
1
11
RT @loreloc_: Live from the CoLoRAI workshop at AAAI. @nadavcohen is now giving his talk on "What Makes Data Suitab….
0
1
0
RT @YuvalRanMilo: Thrilled to share that our paper is being presented today at #NeurIPS2024! Catch it at 16:00, Poster #4932, in Exhibit Ha….
0
1
0
Thrilled to share that I've just been promoted to Associate Prof (with tenure). This wouldn't have been possible w/o the support of my dept, my collaborators, and of course my amazing students (that did all the actual work). Looking forward to the next chapter of our journey!.
8
2
68
Check out our lecture notes. Ideal for teaching our studying theory of deep learning.
@nadavcohen and I uploaded lecture notes on the theory (and surprising practical applications) of linear neural networks. Hope that it can be useful, especially to those entering the field as it highlights distinctions between DL and "classical" ML theory.
0
0
14
RT @tetraduzione: Submit your papers on #lowrank #factorizations & #representations in #AI to our workshop at @RealAAAI in Philadelphia!. w….
0
9
0
SSMs are becoming popular in LLMs and in general. Like all deep learning models, they're fueled by implicit bias that leads to generalization. We show special training examples *with clean labels* can distort their implicit bias, ruining generalization.
How can the implicit bias of SSMs fail, even with clean labels? 🚨. Like many deep learning models, SSMs trained with GD are often implicitly biased to fit data in ways that generalize well. But we show this bias can be poisoned by cleanly labeled data!.🧵
0
3
15
Honored to have been awarded an #ERC! The theory of neural networks progressed a lot in simple settings like supervised learning, but much less in control, a.k.a. reinforcement learning. Our project ("A Theory of Neural Networks for Control") aims to bridge this gap.
📣 The latest ERC Starting Grant competition results are out! 📣. 494 bright minds awarded €780 million to fund research ideas at the frontiers of science. Find out who, where & why 👉 🇪🇺 #EUfunded #FrontierResearch #ERCStG @HorizonEU @EUScienceInnov
1
0
18
Check out our poster today at #ICML2024! Theory for implicit bias of policy gradient.
At #ICML2024? In Wednesday's afternoon poster session, Yotam Alexander will present our work on how policy gradient extrapolates to unseen initial states in control problems. You're welcome to stop by poster #1500 to chat/ask questions!
5
1
8
Proud advisor moment.
Excited to share that I am joining Princeton Language and Intelligence @PrincetonPLI as a postdoctoral fellow this fall! Looking forward to working on the fundamentals of language models. It was a pleasure to do my PhD under @nadavcohen, a bit surreal to wrap it up :).
0
0
20
RT @noamrazin: Accepted to #ICML2024. Excited about new possibilities regarding implicit bias in control/reinforcement learning!.
0
3
0
The implicit bias of gradient descent does wonders in supervised learning, but what about optimal control (reinforcement learning)?. Check out new paper w/ @noamrazin Yotam Alexander Edo Cohen-Karlik @RGiryes @amirgloberson .
arxiv.org
In modern machine learning, models can often fit training data in numerous ways, some of which perform well on unseen (test) data, while others do not. Remarkably, in such cases gradient descent...
1
4
18
RT @noamrazin: In control problems, when does policy gradient extrapolate to unseen initial states?. 🚨📰 We show that extrapolation depends….
arxiv.org
In modern machine learning, models can often fit training data in numerous ways, some of which perform well on unseen (test) data, while others do not. Remarkably, in such cases gradient descent...
0
4
0
@noamrazin Another #NeurIPS2023 poster presented by @noamrazin today! We developed a theory for expressiveness of Graph Neural Nets, and derived a STATE OF THE ART edge sparsification algorithm (very rare for a purely theoretically-driven algorithm to give SOTA results in deep learning).
0
1
8
Today @ #NeurIPS2023, Yotam Alexander & @noamrazin will present a spotlight proving that data is suitable for deep learning IF AND ONLY IF it has low quantum entanglement. @noamrazin did amazing work in his PhD, and is now on post-doc job market. Come catch him in the poster!
1
1
14
Arriving at #NeurIPS2023 today. Looking forward to meeting many friends. DM me if you're around.
0
0
5
RT @noamrazin: I'll be presenting our work on the ability of GNNs to model interactions in the @TAGinDS workshop at #ICML2023 on Friday (4p….
0
1
0
Suppose:.(2,5,7) maps to (2,7,14).(1,2,6) maps to (1,3,9). What does (6,4,1,2) map to? Many would say (6,10,11,13), but what would ML models do?. We give theory + experiments showing RNNs behave like humans on such tasks. Check out our paper! #ICLR2023.
Excited to present our work on the implicit bias of gradient descent applied to RNNs in @ICLR2023!. If you're interested in why RNNs perform well on long OOD sequences, let's meet in person, Tue 2 May 16:30-18:30, Poster #160.@ItamarMenuhin @rajagiryes @nadavcohen @amirgloberson.
0
3
7
RT @noamrazin: Excited to speak at the LoG2 reading group today!. Feel free to join the session if you're interested on hearing more about….
0
2
0
Nice thread by @noamrazin summarizing our new paper. Not often do theories of deep learning produce SotA algorithms. This is the case here, and we are really excited about it!.
📢New preprint on expressivity of Graph Neural Networks w/ Tom Verbin & @nadavcohen!. Quantifies the ability of GNNs to model interactions between vertices. Our theory leads to a simple edge sparsification alg that outperforms alternative methods ✂️. 🧵1/4.
0
0
9