1/x Our new method, the Inter-Animal Transform Class (IATC), is a principled way to compare neural network models to the brain. It's the first to ensure both accurate brain activity predictions and specific identification of neural mechanisms. Preprint: https://t.co/hPqo5PrZoc

@AllenInstitute @jvrsgsty @aran_nayebi @_jacobprince_ @luosha @dyamins 11/X Check out the full paper here: https://t.co/hPqo5PrZoc

@AllenInstitute 10/X Overall, our work provides a principled framework for evaluating brain models, improving on previous approaches and contextualizing prior findings. A huge thanks to my incredible co-authors on this work! @jvrsgsty @aran_nayebi @_jacobprince_ @luosha @dyamins

@AllenInstitute @jvrsgsty @aran_nayebi @_jacobprince @luosha @dyamins 11/X Check out the full paper here:

9/X There’s a lot more to this in the paper, including estimating the IATC on real neural data: a mouse dataset from @AllenInstitute and a human fMRI dataset.

8/X In fact, while linear regression has been thought to be overly powerful, our work suggests that a *non-linear* mapping is needed to capture the actual relationships between brains in a population.

7/X One of the striking takeaways of this work is that stricter mapping methods aren’t necessarily better at mechanism identification, and in fact often perform worse, because they aren’t able to align responses across subjects (as required for the IATC).

6/X Our IATC estimate achieves both high accuracy in predicting neural activity and high specificity in mechanism identification. This shows there is no tradeoff between the engineering goal of predicting brain activity and the scientific goal of identifying neural mechanisms.

5/X So what is the IATC for the brain? In a simulated setting, we found that the neuronal activation function causes subjects’ activation patterns to diverge at each layer before re-converging. Correctly accounting for this “Zippering Effect” leads to a better IATC estimate.

4/X We propose the Inter-Animal Transform Class (IATC)—the strictest set of functions needed to map neural responses accurately between any two actual brains. We can use the IATC to align models to brains, effectively asking if a model can masquerade as a typical subject.

3/X On the other hand, more flexible methods like linear regression, while decent for prediction, seem like they may be too flexible to identify the actual neural mechanism. So what's the right "sweet spot" between strict and flexible?

2/X A key challenge is that individual brains are all somewhat different. So strict methods that match individual neurons struggle to make accurate predictions when mapping a single model to typical brains.

1/ A good world model should be promptable like an LLM, offering flexible control and zero-shot answers to many questions. Language models have benefited greatly from this fact, but it's been slow to come to vision. We introduce PSI: a path to truly interactive visual world

Sunrise over Prague

Devoured these pastries from the oldest bakery in Copenhagen today, Skt. Peders Bageri.

AI models segment scenes based on how things appear, but babies segment based on what moves together. We utilize a visual world model that our lab has been developing, to capture this concept — and what's cool is that it beats SOTA models on zero-shot segmentation and physical

It's always surprised me that ketchup flavored chips, which are so widespread in Canada (and delicious!) are not a thing in the US.

What is the probability of an image? What do the highest and lowest probability images look like? Do natural images lie on a low-dimensional manifold? In a new preprint with @ZKadkhodaie @EeroSimoncelli, we develop a novel energy-based model in order to answer these questions: 🧵

#NeurIPS2024 parties be like this;