RT @jesstyping: I LOVE this talk by @ShriramKMurthi on how to design a curriculum to teach programming. It's tem….

"I would love somebody to take away from this paper that datasets are situated. It's not just the perspectives of the creators but also the socio-technical processes like search engines and the time+place particulars that filter through in the act of creation.".

"There's a history of making these data sets. Well, what are the things that people bring to the table when they do that? If we can understand that, then we can see where the deficiencies are that could lead to things going forward that are just better approaches.".

"The vast majority of dataset publications don't foreground the dataset as a core contribution. So even though datasets are really fundamental to machine learning, we don't value the construction of datasets like we value algorithmic and modeling contributions.".

"Some interesting patterns which are not not too surprising but are a little disheartening is basically zero papers talking about IRB approval. The only papers that discuss IRB approval processes are review papers. I think only one paper discussed ethical considerations.".

"The first Q is trying to understand how dataset developers motivate the decisions that go into the dataset creation. The idea was to read [the dataset artifacts] as texts and understand the values, motivations, and assumptions based on what is said and unsaid within the texts.".

". the concerns with datasets go much far beyond the statistical properties of who is represented, and that's what we're really trying to do with this paper. The examination of ImageNet both from the categorical and the distributional sides is what sparked our research . ".

The paper ( is an interrogation of how datasets in ML are made and their influence. It motivates having genealogical methods for datasets to trace their history and ensure that users are aware of the biases they introduce into downstream applications.

Today, we have @cephaloponderer and @cinjoncin talking about Emily's paper from July 2020 --> Bringing The People Back In: Contesting Benchmark Machine Learning Datasets. Check it out here:

". we invested a lot in monitoring and this was a pretty large scale effort. We're monitoring on the order of 10000 pieces of information a second. This was critical to our success because there's so many moving pieces and so many places where the optimization can go wrong.".

"One difficulty in this work is just getting a better sense of what's going. People often monitor certain attributes in their models. But now there's not one model being trained. There's huge numbers of models being trained in very different settings. So throughout this work . ".

"Scale matters. We needed a large number of tasks to train the optimizer on, on the order of 6000, and we needed a lot of compute over a long period of time. About 30,000 CPU cores for a month, four times longer than what most people do.".

"In this work, we're creating these general purpose optimizers. We want them to work on a wide variety of models without tuning the settings. What's new is that instead of actually designing it by hand, like fiddling around with equations, we're going to learn it with data.".

"I think there's some analogy to using learning how humans learn because this is exactly what evolution has created in us. Evolution itself is an optimization procedure that optimized us, learning machines, to become who we are.".

"A motivating factor for me, for my research for the last couple of years, has been trying to use machine learning techniques to improve machine learning. And I believe that by doing this, we'll start to construct new types of algorithm that enable more things that we can do.".

This week, we have @Luke_Metz and @cinjoncin talking about Luke's 3 (now 4) year journey on learning optimizers. Find the audio and transcript at Some choice quotes in the thread 👇.

". just want the model to be compact. We may want it to be interpretable or robust or fair. The message of this paper is that by optimizing for one property, we may be compromising others.".

"I grew up in Africa and am interested in making AI accessible in resource constrained environments similar to what I grew up in, but it's also important to understand that we are often making tradeoffs when we talk about properties that we want a model to have. We might not . ".

"When we deliver a model, we want it to work the same in Latin America as it does in the US. What we mean by disparate is if a model performs disproportionately worse for protected attributes, like nationality or age or race.".

"If I want to deploy a model to your phone, and most of you have some type of model on our phones, we have to do something which we call compression. We use a series of techniques that make these large models smaller. Popular ways are pruning, removing weights, or quantizing.".