Question: can a model trained on a *plausible* amount of *raw* speech explain both intelligent behavior and its brain bases? Here, we train wav2vec 2.0 w/ 600h of audio and map its activations onto the brains of 417 volunteers recorded with fMRI while listening to audio books.

Result 1: self-supervised learning suffices to make this algorithm learn brain-like representations (i.e. most brain areas significantly correlate with its activations in response to the same speech input). 2/n

Result 2: The hierarchy learnt by the algorithm maps onto the brain's: The auditory cortex is best aligned with the first layer of the transformer (blue), whereas the prefrontal cortex is best aligned with its deepest layers (red). 3/n

Result 3: With an additional 386 subjects, we show that wav2vec 2.0 learns both the speech-specific and the language-specific representations of the prefrontal and temporal cortices, respectively. 4/n

Conclusion: Modeling human-level intelligence is a far-off goal. Still, the emergence of brain-like functions in self-supervised algorithms suggests that we may be on the right path. 5/n

This is a joint work with our great team 🤩🤩 Juliette Millet, @PierreOrhan, Y Boubenec, @agramfort, E Dunbar, @chrplr and @JeanRemiKing, at @MetaAI, @ENS_ULM, @Inria & @Neurospin 6/n

🙏 Thanks to @samnastase, @HassonUri, John Hale, @nilearn, @pyvista and the open-source and open-science communities for making this possible! 7/7

@c_caucheteux @samnastase @HassonUri @nilearn @pyvista Is there data for time stamping audio events? Similar to "Our Brains “Time-Stamp” Sounds to Process the Words We Hear" https://t.co/l4Zmu6iLMW.

@c_caucheteux @samnastase @HassonUri @nilearn @pyvista "You wouldn't download a brain"😅 Seriously is it the 1st steps of it, and injecting some data into the brain to learn new languages just with an upload ?