Tyler Chang
@tylerachang
Followers
169
Following
85
Media
3
Statuses
20
Research scientist @GoogleDeepMind. He/him/his.
Joined June 2022
We're organizing a shared task to develop a multilingual physical commonsense reasoning evaluation dataset! Details on how to submit are at:
As part of the workshop, we are also organizing a shared task to develop a collaborative physical commonsense reasoning evaluation dataset. See the shared task page for more information:
0
3
15
RT @davlanade: Excited to announce the call for papers for the Multilingual Representation Learning workshop #EMNLP2025 .
0
12
0
Presenting our work on training data attribution for pretraining this morning: -- come stop by in Hall 2/3 #526 if you're here at ICLR!.
We scaled training data attribution (TDA) methods ~1000x to find influential pretraining examples for thousands of queries in an 8B-parameter LLM over the entire 160B-token C4 corpus!.
0
5
20
RT @camrobjones: One of the major pieces of feedback that we got on the last Turing test is that it was "too easy" because it used a 2-play….
0
8
0
And we hope you enjoy our paper: This work wouldn't have been at all possible without @dheerajgopal @tolgab0 @iislucas and @iftenney !.
0
0
13
Play with it yourself: see influential pretraining examples from our method for facts, factual errors, commonsense reasoning, arithmetic, and open-ended generation:
1
0
8
As models increase in size and pretraining tokens, "influence" more closely resembles "attribution". I.e. "better" models do seem to rely more on entailing examples.
1
0
5
Many influential examples do not entail a fact, but instead appear to reflect priors on common entities for certain relation types, or guesses based on first or last names.
1
0
6
In a fact tracing task, we find that classical retrieval methods (e.g. BM25) are still much better for retrieving examples that *entail* factual predictions (factual "attribution"), but TDA methods retrieve examples that have greater *influence* on model predictions.
1
0
7
Our method, TrackStar, refines existing gradient-based approaches to scale to much larger settings: over 100x more queries and a 30x larger retrieval corpus than previous work at this model size.
1
0
9
We scaled training data attribution (TDA) methods ~1000x to find influential pretraining examples for thousands of queries in an 8B-parameter LLM over the entire 160B-token C4 corpus!.
1
20
128
RT @linguist_cat: ✨New pre-print!✨Successful language technologies should work for a wide variety of languages. But some languages have sys….
0
13
0
RT @linguist_cat: @tylerachang and my paper “When is Multilinguality a Curse?” was awarded outstanding paper! Thank you @emnlpmeeting ❤️.
0
4
0
RT @linguist_cat: Our paper “When is Multilinguality a Curse?” will be presented at #EMNLP2024! We found that multilingual data hurts high-….
0
9
0
RT @linguist_cat: Super excited to finally release the Goldfish models, joint work with @tylerachang. These are small, comparable models fo….
0
18
0
RT @linguist_cat: New preprint with @tylerachang and Benjamin Bergen! We find that some languages need up to five times as much storage in….
0
3
0