Zach Mueller
@TheZachMueller
Followers
9,608
Following
391
Media
1,723
Statuses
14,722
🤗 Technical Lead for the Accelerate Project | Passionate about Open Source | Nerd who enjoys touching the grass | #ADHD | He/Him
Joined April 2016
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Davido
• 553206 Tweets
Wizkid
• 548984 Tweets
#TheKingdomsConcert
• 216517 Tweets
#WWERaw
• 124516 Tweets
Patricia
• 58597 Tweets
Smackdown
• 31784 Tweets
Tatum
• 28868 Tweets
Derrick White
• 23783 Tweets
Saint Ram Rahim
• 21886 Tweets
Braun
• 20351 Tweets
Cause of Conspiracy
• 19623 Tweets
Luciano
• 19459 Tweets
Ghar Vapasi
• 18752 Tweets
Dario
• 17414 Tweets
Aleska
• 16270 Tweets
#DesafioXX
• 15387 Tweets
Rony
• 14611 Tweets
All To Myself D-2
• 14068 Tweets
Endrick
• 12930 Tweets
Olimpia
• 12635 Tweets
Gunther
• 12408 Tweets
Porzingis
• 11203 Tweets
Freya
• 10627 Tweets
Pinned Tweet
Hi everyone, I'm Zach 👋
- 👨💻 I work at
@huggingface
, working on Accelerate and distributed
@pytorch
code to make your lives easier
- 👨🏫 Occasionally I write courses on
#python
and
@fastdotai
- ⚕️Have
#ADHD
- 🏋♂️ Into lifting and comics!
Who are you, and what are you into?
19
4
173
Happy to announce that starting Monday, I will be beginning my role as a Machine Learning Engineer at
@huggingface
!
While there, I'll be helping write user-centric API's for the HuggingFace ecosystem, with the aim of empowering folks to use the libraries as best they can
65
31
808
New article on
#python
decorators is out! Specifically this shows you how decorators are written, what they do, and the power you can do with them. I even show an example of when you'd use the strange "nonlocal" 1/3
14
115
768
Today I officially begin my journey as a Machine Learning Engineer 🎉🎉🎉 It’s been a crazy road these last two years, and I’m thankful to all the mentors and friends I’ve made along the way. I’m extremely excited for the road ahead 😊
48
25
733
You may know that
@huggingface
Accelerate has big-model inference capabilities, but how does that work?
With the help of
#manim
, let's dig in!
Step 1:
Load an empty model into memory using
@PyTorch
's `meta` device, so it uses a *super* tiny amount of RAM
14
146
632
Excited to announce a new
@huggingface
space to help with one of machine learning's biggest questions:
How much space does {X} model take in vRAM? And most importantly: when using `device_map="auto"`
18
132
563
🎉🎉New Blog Alert!🎉🎉
Come learn how to integrate your
@PyTorch
into
@fastdotai
with the most minimal fastai possible!
What does this entail?
- Pytorch Datasets
- Pytorch DataLoaders
- Pytorch Models
But, is it really so easy as advertized?
11
87
383
I've created a small little knowledge repository on
@huggingface
transformers here:
Essentially these contain all the `task` notebooks converted as scripts, showcasing end-to-end usage in under 150 lines of code (but still readable!)
6
55
365
Ecstatic to announce something the team have been hard at work on over the last two months:
@huggingface
Transformers is officially using Accelerate for its PyTorch backend! In this thread I'll discuss why that matters, what you as a user might see, and more 🧵
9
53
356
Just hit 500 stars on my Practical Deep Learning for Coders 2.0 (Walk with
@fastdotai
course) repo!!! Thank you so much guys, and I'm glad folks are finding it useful as time goes by😁
3
46
346
There’s a new optimizer out, and they tried out one of the longest combinations yet: AdaBelief + RAdam + Lookahead + Gradient Centralization, combined into one optimizer. And it’s in Pytorch!
12
69
316
What is
@Docker
? What is it good for? And why as a Data Scientist should you worship the almighty whale?
Come join me in discovering and understanding how Docker containers can ease your life, as well as tips to setup CUDA without breaking everything! 1/
6
47
294
Hey Twitter data science folks! Interested about fastai and want to learn fastaiv2 along with some techniques not shown in the original lectures? Come learn with me! I’ll be running a course and study groups next semester! More info:
4
65
280
Did you know that you might be slowing down your training if you're using Gradient Accumulation in Distributed Training with
@PyTorch
and not even realize it? A small thread on a new feature in
@huggingface
Accelerate to help you out
1/6
7
43
262
After some deliberation in the discord, I'm happy to say that Walk with
@fastdotai
: Revisited is now free and open to the public!
Come learn what I believe are its best practices and ship a model with production-ready code. 1/5
15
49
252
Did you know:
You can use Python in bash (!) cells from your Jupyter Notebook? Just wrap a curly brace around it!
5
24
237
Now announcing fastdebug, a helpful little library for debugging your
@PyTorch
and
@fastdotai
code, including providing clearer errors and sanity checking:
Install with `pip install fastdebug`
But how does it all work, and what can it do?
1/
5
57
228
Did you hear?
@kaggle
let's you have two T4's to run in a notebook session!
But wait, *how* do you even train on two GPU's in a notebook with
@PyTorch
?
My new tutorial shows how to do so with
@huggingface
Accelerate, find out now!
2
53
224
Best resources for learning about RAGs? (Assume no prior knowledge about it).
cc
@HamelHusain
@abacaj
21
14
222
Another useful bookmark I just discovered today:
First and formost finally I can get rid of all my tons of "untitled.ipynb". Who knew colab had a scratch notebook? Or that we can run flask apps in colab?!
1
50
201
WOW! The Practical Deep Learning for Coders 2.0 (A walk with fastai2) repository just hit 100 stars! This is an exciting first for me 😊 I’m glad so many people are finding it useful
3
27
196
This is AWESOME!
@fastdotai
being used by the UN Operational Satellite Application Program to help improve rapid response to natural disasters! *And* a notebook with how they did it!
1
55
198
Can't wait for "It's Time Deep Learning Learned from Software Engineering," a talk by two of my favorite people
@jeremyphoward
and
@HamelHusain
. I would highly recommend signing up before it's too late!
5
28
192
Ever wanted to use
#SHAP
with
#fastdotai
tabular models? Well now you can! Here is an example implementation to help boost model interpretability (also works in Colab!):
8
43
186
Announcing !
This project is an effort to collect interesting techniques dotted throughout the
@fastdotai
forums, courses, and more into a centralized location!
Sign up to the email list now for when new chapters are added: 1/
11
33
183
With that the entirety of my course, A Walk with
@fastdotai
is now live on ! It took a bit to finish up due to adding some parts here and there and adjusting some code to have it work with the latest versions of most libraries
5
29
184
So. Post graduation plans:
-
@jeremyphoward
's Matrix Calculus for DL
-
@math_rachel
's Computational Linear Algebra
- W&B's Math for ML
(In this order)
New video series out this week (and into next!) on the
@weights_biases
YouTube channel.
They're Socratic livecoding sessions where
@_ScottCondron
and I work through the exercise notebooks for the Math4ML class.
Details in 🧵⤵️
3
58
292
3
28
181
TIL of light-the-torch, a package designed to automatically install the proper
@PyTorch
versions based on what GPU you're using. You no longer have to go digging for the proper torch wheels! Super easy to use as well, just two lines of code
4
29
177
What day is it?
@huggingface
Accelerate release day!
v0.20.0 is now out, and there is a HUGE list of amazing features to get through with Big Model Inference, QLoRA, distributed inference, new device support, and more! Let's dig in🧵
2
29
172
I can't believe that
@huggingface
made it to Time's 100 most influential companies 🤩
The dream job continues, love this place and so proud of all we do!
6
21
172
Given there's so many new folks exploring the
@fastdotai
library with the new course, thought I'd push a tutorial I wrote for using
@PyTorch
in the framework from scratch, step by step. Hope it helps someone! 🤗
2
30
166
With the new release of
@huggingface
Accelerate performing gradient accumulation in GPU, multi-GPU, and TPUs now comes with a consistent easy-to-use API that handles the complicated bits for you!
Check out the full documentation tutorial to get started:
2
23
167
Some advice a coworker told me Friday: when applying for a job, downplay your skills in non-relevant areas.
Applying for ML, but know Docker, Kubeflow, Kubernetes, and GH Actions along the way? Don't highlight that on your resume. It may be sent to the tech team instead.
15
12
159
What are some good
#python
resources (or resources in general) for those who are coming from research and wanting to dip their toes into raw Software Engineering? (And learn the bits that were ignored/passed as a researcher)
*Not for me
22
20
162
@serious_mehta
Just use Docker. That's what it's meant for.
I have 5-6 different images I jump between based on what I work on, never need to configure anything since it's all just there.
And if I want something custom, I just build the base image off the Jupyter one once and never again
7
1
162
We were able to achieve SOTA on the poker hand dataset with 99.44% using a standard
#fastdotai
model with Adam + One Cycle in ~300 epochs (translates to 1.57 minutes). Previous best was 99.3% with TabNet and 65.1% with a Neural Decision Tree. Notebook:
2
24
151
Today I’ve come across some wonderful
@fastdotai
YouTube channels with some excellent content! Below is a thread of my findings for folks to check out (I’ve subscribed to them all!) 1/
2
24
148
One of my goals for the year was to start blogging again. With the new
@fastdotai
course this gives me a great motivator to do so! So meet the first one: fastai and the new DataBlock API:
@bhutanisanyam1
tried to beat me to it, I had to
prove him wrong.
9
34
149
Announcing preorders for Walk with
@fastdotai
, the missing pieces for success!
If you want to take your fastai skills to the next level, this course is a must have. Come learn why:
13
29
149
Today is a new achievement for
@huggingface
Accelerate, we've passed 50 million downloads on pypi! In honor of this I wanted to highlight some history about the framework, where we've gone to, and how the community is relying on it more and more:
1/6
1
12
150
Small update, considering most of what I do is building ML/DL software libraries (such as adaptnlp and lib2nbdev), I'm now officially a Deep Learning Software Engineer 😁
6
5
147
Okay now for the big reveal at 8k followers as promised!
As many of you know, I'm writing a new course! This one is focused on how to write user-facing APIs (like fastai, accelerate, keras, transformers, etc) but from a software engineering perspective 1/6
7
13
147
TIL
#python
has a concept of conditional inheritance. A direct use-case is exposing a class that needs a particular import to be available, but having it inherit an abstract class if it's not.
This removes hiding the class definition under an import and having more abstraction.
6
21
145
1
29
142
Thread on the recent drama:
At the end of the day, it's not about who copied who. It's about taking and using the ideas of some other library (which also is published paper), without giving the proper attribution in your GitHub or documentation, which gets the most traffic. 1/7
3
17
141
So I guess
@GoogleColab
added
@fastdotai
to their list of auto-updated libraries, so you always have the latest version! 😍
Did I mention how much I love Colab 🤗
3
15
141
This is an extremely hot take of mine, but I do believe every Data Scientist needs to take at least 2-3 computer science courses at least.
Not in the beginning per-se (though ideally before), but eventually.
Especially if they keep asking how xyz programming concept works. 1/
14
9
140
Deep Learning from the Foundations (
@fastdotai
)'s lesson 1 always gives me a chuckle.
We have Practical Deep Learning for Coders, this is *Impractical* Deep Learning for Coders
3
9
140
Today is an extra-special release of
@huggingface
Accelerate!
Among other features, this latest version (with collaboration from
@PyTorch
) integrates a PyTorch-native pipeline-parallel inference framework!
(So no need for DeepSpeed or Megatron in deployment!) 1/6
3
27
139
We've merged scheduler-free optimizer support into
@huggingface
accelerate, give it a whirl with our new example:
(Note: requires installing from git)
0
18
138
My paper on using a
@fastdotai
tabular model to identify movements from Apple Watch data is finally published 🎉
Combining feature engineering, entity embeddings, and a "time-step" proved to be the secret to success with the data
5
21
136
.
@jeremyphoward
@GuggerSylvain
the magic has finally arrived 🤗 Your course and all your work has helped my dad and I out so much through our journies (not to mention making them FREE!) This is the least we can do to repay your kindness over the years and hard work. Thank you!
1
18
135
I was made aware of a new paper in the Lancet, and not only do they reference
@fastdotai
, they also used my SHAP integrator and feature importance code (and referenced me!)
7
12
136
Deep Learning for Tabular Data: An Exploratory Study by Jan Andre Marais. A 144 long thesis discussing
@fastdotai
tabular neural networks and exploring many aspects of it including MixUp, Attention, Self-Norm, and more! . 1/
1
28
133
To co-inside with the official
@fastdotai
release, I am publishing "Zero to Hero" articles to help YOU up to speed! 1/
Today is ’s biggest day in our four year history. We are releasing our new course, fastai v2, fastcore, fastscript, fastgpu, and the official launch of the book. 1/
93
992
4K
3
27
134
*Reads through the
@huggingface
Accelerate tutorials*
*Sees one on CV*
Huh. This seems to be like what fastai does for their training... but in raw PyTorch + timm. It's really good! Who wrote this again?
Oh yeah. Just
@GuggerSylvain
🤦♂️
2
13
131
It was a tough semester, but totally worth it as I made the Dean's List!
14
1
128
This looks like a great resource for those wanting to use
#fastdotai
in deployment: (I’ve certainly bookmarked it)
3
24
128
What is distributed training? How do you do it in
@PyTorch
? And how does
@huggingface
Accelerate and Transformers relate?
Come learn how to get started with distributed PyTorch training now in our new article 🤗
1
28
126
My Walk with fastai2 repo just hit 350 stars, almost 500 subscribers on my
@fastdotai
YouTube channel, and almost 1,000 Twitter followers! I’m glad you guys are enjoying the content and I would have never imagined it turning into this. Thanks everyone 😊
2
21
127
Apparently since the secret is fully out thanks to
@mervenoyann
the other day, I'll announce it formally.
I'm currently writing a book aimed at Intermediate Python, discussing fastcore's magic, GitHub Actions, and nbdev.
Why? 1/3
9
7
126
Deploying with
@fastdotai
isn't always learn = load_learner(), learn.predict. There are numerous scenarios when you might only want some, part, or none of both the API and the library as a whole. In this thread we will be exploring your options, how they work, and what to do: 1/n
4
24
125
One of the biggest errors with distributed
@PyTorch
code comes in the form of Gradient Accumulation and doing it efficiently.
In this blog, you learn how to do it well, how not to, and how to avoid a 2x slowdown, featuring
@huggingface
Accelerate:
2
19
123
(Link was broken the first time)
Here's my new article on speeding up the
@fastdotai
tabular pipeline! By utilizing NumPy I was able to get a 40% speedup during training, while still using TabularPandas pre-processing, Read more here:
3
26
117
Last week
@huggingface
Accelerate v0.21.0 was released, and a few important updates along with it!
2
24
118
2
8
115
New article alert: Fine-Tuning Language Models from
@huggingface
on the GLUE benchmark in
@fastdotai
, by
@arampacha
!
Come learn how to utilize fastai's Callback system to have it work nicely with transformer models, and how to write the proper DataBlock!
1
18
115
fasterai, a
@fastdotai
library aimed at network distillation with sparse neural training (and knowledge distillation)
4
21
113
CUDA OOM? Not anymore with 🤗 Accelerate!
This new example shows how you can combine two existent features in
@huggingface
Accelerate to *never again* reach CUDA OOM!
Let's talk about how 🧵
1/7
2
9
110
TIL if you want to quickly take a list of lists in python and turn it into a single list, this is all you need:
(there's probably some fastcore op that does this too)
8
5
110
Would people be interested in a video tutorial on building custom dataloaders in the new version of
@fastdotai
? It'd specifically mimic the NumPy one I posted about a few days ago, as personally I feel it's a good "Hello World" project. Let me know!
18
7
108
Our verbose
@PyTorch
in
@fastdotai
tutorial is now live! This mocks my "PyTorch to fastai, briding the gap" atticle by showing the full training, as well as inference!
Thanks everyone who joined the stream! We got two fastai PR's up! 🥳
Also apparently we streamed til 9pm? **OOPS**
Once they're merged I'll let everyone know! 🤗
0
0
13
2
17
106
Want to make sure others can replicate your code, without having to tangle through `pip freeze`?
Quickly use `dependency_checker` and throw it's results into a requirements.txt file in 5 lines of code!
4
12
107
Did you know
@fastdotai
let's you utilize multiple GPU's? -- Probably
Did you know it utilizes
@huggingface
Accelerate to get there? -- Probably not!
With it, you can train distributed code directly from your 💫Jupyter Notebook💫via the notebook_launcher
1
23
104
To say I'm excited about this blog post is an understatement: Gaussian Heatmaps done in
@fastdotai
by
@ElteHupkes
!
The main benefit? The architecture can be trained for any number of keypoints possible. Well worth the read
5
27
105
TIL einops can be faster than raw PyTorch 🤯
On stream we're reimplementing
@fastdotai
's AdaptiveConcatPool2d, which just calls Average and Max pooling, and concatentates. We found that doing the same with einops can be a *little* faster!
4
25
104
Probably the most underrated (and unknown)
@fastdotai
library, fastcore. It's an awesome Python utils library adding tons of features (most of the coolest bits unrelated to fastai are trapped inside of the utils file!)
3
10
103
We hit 5k, you know what that means!
Hi everyone, I'm a Machine Learning Engineer with
@huggingface
and a community leader + major contributor for the
@fastdotai
libraries. AMA!
#AMA
#dataScience
#MachineLearning
#PyTorch
22
16
102
Walk with
@fastdotai
is officially released! We have four new chapters: two introductions to vision and tabular, an example walking you through the Pets dataset, as well as a guide on how to integrate
@wightmanr
's timm library: 1/
2
31
100
Successfully rebuilt ✅
Should I post the parts list? 🤔
17
4
103
New
@fastdotai
blog alert! Looking in-depth at the test_dl. How do you know what transforms it will use? How can it be adapted to a change in how our inference data comes in? All of this and more is explored!
0
23
97
Great way to end the week. For the first time ever, the entire
@huggingface
Accelerate team existed in one place!
1
3
100
How can you learn to use the
@fastdotai
framework to its fullest extent? A thread on what I believe is the most important lesson you can teach yourself: 👇
1/
4
18
100
Full stop: in love with CUDA + Docker image setup for workflow development. This stuff is seriously so cool.
~10 minutes to configure CUDA drivers, 4 to install CUDA Docker.
Pull
@fastdotai
docker image and bam!
5
6
99
Why do I love the
@fastdotai
framework? I found a paper on coral species identification and wanted to try to bring it in and compare results. Right away (as this is just part 1) I was able to increase the accuracy from 83% to 89%. Read more here!
3
16
95
Very, very good straightforward tutorial for setting up SLURM yourself. No notes, phenomenal.
1
18
97
Twitter in the last few days:
- Don't do kaggle, it's not real data science and doesn't prepare you for the real world! Stats is better
- Don't do stats, it's worthless. Do Kaggle as it'll prepare you to be a data scientist.
Why do I stay on this app. 🙃
13
7
97
As promised, here is my
@fastdotai
tutorial integrating
@numpy
into their DataLoaders for more efficiency, as well as customizing it for our data! If folks would like a TfmdDL walkthrough, let me know!
2
18
95
With school over, I'll reveal one of my secrets. I'm making a new course!
Running with
@fastdotai
! Times and dates are TBD, but I'm shooting for this Fall to hold the course. This will be a continuation on Walk with fastai, building on what we learned there and applying it 1/
3
13
96
Happy to announce
#fastinference
! The bad news: fastshap and ClassConfusion are gone. The good news: they moved to a new home! In this module we have all of the above plus some speed-up and QOL integrations into
@fastdotai
's inference methods, see here: 1/
5
17
93
And powered by 🤗
@huggingface
Accelerate!
Imagen - Pytorch: Implementation of Imagen, Google's Text-to-Image Neural Network that beats DALL-E2, in Pytorch /by lucidrains
👉
#kornia
used for some image processing
#computervision
#opensource
#ai
#pytorch
1
54
265
1
7
93
Currently loving the
#fastdot
library out of
@fastdotai
. Great tool for trying to visualize ideas on your Neural Networks!
1
18
95
Now announcing the
@fastdotai
community docment sprint!
What: As a community, we're going to try and improve the functional documentation on all aspects of the fastai library
When: Two weeks, from February 14th through the 26th
How? Read below 🧵
5
25
95
A resource I keep coming back to often is the
@huggingface
Trainer example zoo I made awhile back. Want a quick way to just "run the trainer" and verify your setup works? Or just see minimal working examples that can be substituted for any model? Check it out…
5
10
94
Who are some of the best folks that teach beginner to intermediate
#python
skills that you all love?
Specifically looking at written resources, not lectures.
TIA 🙏
22
11
88
Here's an excellent example of using
@fastdotai
trained embeddings, loading it into a Random Forest, and showing improved accuracy as a result!
3
13
92