Jerry Tang
@jerryptang
Followers
1K
Following
9
Media
7
Statuses
32
phd student @HuthLab interested in language and brain-computer interfaces
Austin, TX
Joined May 2021
RT @cathychen23: How does the human brain represent semantic information from different languages?. Our new preprint suggests that bilingua….
0
86
0
RT @cathychen23: Do brain representations of language depend on whether the inputs are pixels or sounds?. Our @CommsBio paper studies this….
0
39
0
I’ll be presenting this work at the #NeurIPS2023 5pm poster session on Wednesday — come by to chat about the relationship between language and vision in the brain!.
Multimodal transformers make it possible to transfer fMRI encoding models between language and vision! (though mostly from L->V and not V->L 🤔) New paper from @jerryptang @_du_meng @vvobot @vasudev_lal
2
8
66
RT @AmandaLeBel3: This paper and dataset is now officially published in Scientific Data!! And not only that but we have nearly doubled the….
0
9
0
RT @WillettNeuro: Our new study is out today in Nature! We demonstrate a brain-computer interface that turns speech-related neural activity….
0
223
0
RT @KayloLittlejohn: Our paper on a high-performance neuroprosthesis for speech decoding and avatar control is out in Nature! Thankful to w….
0
29
0
RT @shaileeejain: 🌟NEW PREPRINT🌟.🤖-based🧠models are all the rage!. But,.- How do we interpret 🧠if 🤖are SO un-interpretable?.- How do we _te….
0
13
0
And here's a short explainer of how our decoder works and what it can do!
2
2
26
If you want to learn more about brain decoding, I wrote a quick introduction to how brain decoding technology works and how we think it could impact society
2
7
22
We hope that language decoding can help restore communication to people who have lost the ability to speak due to injury or disease. However, it is crucial to ensure that this technology is used appropriately in a way that respects mental privacy.
3
1
14
Our language decoding paper (@AmandaLeBel3 @shaileeejain @alex_ander) is out! We found that it is possible to use functional MRI scans to predict the words that a user was hearing or imagining when the scans were collected
15
79
218
this is generating a lot of discussion and we wanted to clear a few things up. first, we take privacy very seriously and reject the use of brain decoders for surveillance, interrogation, and other unethical applications.
2
2
21
we hope to eventually offer a non-invasive alternative, but our decoder has a long way to go. other possible uses include memory and attention prostheses, but our results are just a step towards those goals as well (9/9).
3
1
10
finally, some have asked about the positive uses of brain decoding. our initial goal was to restore speech to people with locked-in syndrome. since then, intracranial decoders have gotten very good at speech restoration, and we didn't want to draw attention away from that (8/9).
2
0
16
to this end, we wanted to be very clear about what our decoder can and cannot do (see our supplementary tables for full transcripts), and we believe that transparency and awareness provide the best way forward (7/9).
1
0
6
our biggest concern is that malevolent entities can intentionally misinterpret decoder results - this doesn't even require an accurate decoder. we believe that it is critical to enact policies that protect mental privacy in the face of such violations (6/9).
2
0
12
decoding performance fell to unintelligble levels when subjects performed simple mental tasks while staying still. motion should further lower performance. this could also change, so we are staying vigilant and working to test other resistance tasks (5/9)
finally, we tested whether our decoder respects user privacy, and found that subject cooperation is currently required to train and to run the decoder. for instance, we developed a set of covert resistance strategies that substantially lowered decoding performance (6/7)
2
0
9
we were relieved to find that there doesn't seem to be a way around that - it's unlikely that we could train our decoder without someone's full cooperation. of course this can change as technology improves, which motivated us to look at resistance strategies (4/9)
3
0
13
fMRI records brain activity using a large magnet, and the signals are very weak since they are recorded from outside of the skull. training our decoder for a subject requires many hours of data while that specific subject stays perfectly still and pays full attention (3/9)
1
0
10
@AmandaLeBel3 posted a great thread about the privacy sections in our paper, and i'd like to further unpack some of those ideas (2/9)
I am sure most of you have seen @jerryptang 's thread on our new paper on non-invasive brain decoding. It's clear that this a big development and could be useful for many people. But I wanted to highlight what my favorite thing in this paper is!.
1
0
10