Mitchell Ostrow
@neurostrow
Followers
1K
Following
1K
Media
13
Statuses
130
PhD Candidate in Computational Neuroscience + ML at @mitbrainandcog, prev @yaleneuro, @yaledatascience, @meta. NSF GRFP Fellow
Joined September 2022
Accepted to @NeurIPSConf!! Looking forward to presenting in New Orleans this December 🧠💻.
3
14
109
RT @catherineliangq: I will be presenting our results at #NeurIPS2024 on Friday 11-2 at East Exhibit Hall A-C #1608. Drop by and say hi😊.
0
4
0
Very appreciative of all the interest surrounding this post! I’ll do my best to get through them all but for now I have to ask y’all to hold off contacting me for a bit.
1
0
12
Dms are unlocked if you want to contact me there, but please be patient.
2
0
23
Wow, this blew up! I'll try to get to everyone's requests but please be patient.
Applying to grad school is hard. The expectations for essays are murky, and not everybody comes from a wealthy institution with resources needed to optimize an application. If you're applying to grad school now, please reach out and I'll be happy to help you edit your essays.
3
1
36
Applying to grad school is hard. The expectations for essays are murky, and not everybody comes from a wealthy institution with resources needed to optimize an application. If you're applying to grad school now, please reach out and I'll be happy to help you edit your essays.
70
226
2K
RT @RylanSchaeffer: Shout out to my wonderful collaborators @ZahediNika @KhonaMikail @dhruv31415 @sangttruong @du_yilun @neurostrow othe….
0
1
0
Really excited about these results led by @catherineliangq !!.
How do diffusion models factorize and generalize? Extending our previous work (arXiv:2402.03305), @ZimingLiu11 @neurostrow @FieteGroup and I further peaked into the model’s internal representation in relation to its ability to compose and generalize
0
0
9
Had a lot of fun writing about @courellis 's and @UeliRutishauser 's amazing research!.
It's accompanied by an excellent @NatureNV by @neurostrow and Ila Fiete
1
1
13
RT @UeliRutishauser: Also check out the the excellent News & Views by Mitchelle Ostrow & Ila Fiete that accompanies the paper! https://t.co….
0
4
0
RT @adamjeisen: Our paper is out today in Neuron! Huge thanks to my co-first author @Leokoz8, co-advisors @MillerLabMIT and @FieteGroup, an….
0
8
0
addendum: I made minimal implementations of the LRU, 1-layer GPTs, and RNNs available here!.
0
0
4
If you made it to the end, thanks for reading, and stay tuned for more! (n/n).
1
0
4
Our work provides a new theoretical lens through which to analyze neural sequence models. It also helps explain why transformers have been previously observed to be worse architectures for dynamics prediction tasks! (12/n).
3
0
4
As expected, we find that a number of embedding quality metrics are significantly related to prediction performance in each architecture, and SSMs outperform Transformers on each metric. (11/n)
1
0
2
This results in SSMs being less noise robust, as quantified by the relative decrease in performance as the noise is increased. MASE = Mean Absolute Standardized Error. (10/n)
1
0
1
SSMs also learn more aesthetic + qualitatively accurate embeddings, but are actually quite folded up and lower dimensional relative to the Transformers and the true state space. (9/n)
1
0
1
Both systems can learn effective delay embeddings, as quantified by linear and nonlinear decoding performance of unobserved Lorenz states! SSMs do well at initialization though, unlike transformers. (8/n)
1
0
1
To assess this capacity, we trained small SSMs (Linear Recurrent Units) and Transformers on a noisy 1-d projection on the chaotic Lorenz attractor, for next-step prediction. Both systems do well, but the SSMs perform better across all hyperparameters we searched. (7/n)
1
0
3