Yannic Kilcher 🇸🇨
@ykilcher
Followers
67,302
Following
867
Media
723
Statuses
3,710
I make videos. Skill > Destiny. vi / vim
Joined July 2010
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
BECKY X MAYBELLINE LIVE
• 180622 Tweets
Northern Lights
• 178829 Tweets
#REBECCAPINKLOVELIVE
• 177629 Tweets
オーロラ
• 151126 Tweets
#Auroraborealis
• 74710 Tweets
#ヒロミのおせっ買い
• 29318 Tweets
BUMP
• 29030 Tweets
京王杯SC
• 22330 Tweets
Brush Teeth With Patrick🪥
• 20950 Tweets
UsutTuntasRBT UsutRBS
• 19977 Tweets
バチコン
• 14542 Tweets
Kアリーナ
• 12164 Tweets
ソウルイーター
• 11912 Tweets
太陽フレアのせい
• 11120 Tweets
Antero
• 10314 Tweets
Pinned Tweet
🥳Special Video🥳This has been in the works for a while. I used CLIP + BigGAN to make a music video for a song with lyrics made from ImageNet class labels🤠"Be my weasel", performed by me on a looper🎸Code & references available, make your own! Enjoy🤟
38
114
684
GPT-4 paper literally is just saying "we trained a model on data and it's better". Spread over 98 pages.
85
263
3K
🔥EVERYONE🔥We’re excited to announce the release of OpenAssistant.
The future of AI development depends heavily on high quality datasets and models being made publicly available, and that’s exactly what this project does.
Watch the annoucement video:
52
508
2K
Asking for more regulation is a classic move of market leaders to suppress all competition. How petty of OpenAI to sink to this level.
Sam Altman, CEO of the start-up behind the AI chatbot ChatGPT, agreed with members of the Senate on Tuesday on the need to regulate increasingly powerful AI technology.
71
218
648
78
177
1K
"neural networks focus too much on texture"
The prior in your brain is wrong. This isn’t fried chicken
65
345
2K
25
168
1K
Ok hear me out, each person who leaves OpenAI just has to memorize a billion weights then we get GPT-4
32
45
1K
There are 4 big Machine Learning conferences now: NeurIPS, ICML, ICLR, and Google I/O
9
117
1K
We must urgently stop all further development on this new "keyboard" technology. In the near future, anyone will just be able to type anything!!! The world will be flooded with fake news and civilization will fall😱
56
100
888
🔥New Video🔥Convolutions are DEAD as Transformers continue to ruin absolutely everything 😱 New SotA on ImageNet, VTAB, etc using only Transformers + massive data 👑 Also Peer Review is broken. Watch Now!👀
@GoogleAI
@giffmana
@__kolesnikov__
@XiaohuaZhai
18
189
865
NEW & BREAKING: A Sharpie engineer has spent months testing its "pen" product. He's disturbed by the violent/sexual content it can create & Walmart's decision not to take it off the shelves to investigate.
NEW & BREAKING: An Adobe engineer has spent months testing its image-generator software, Photoshop. He's disturbed by the violent/sexual content it can create & Adobe's decision not to take it offline to investigate.
10
11
248
25
74
761
New Video 🔥 Deep Learning is very good at fitting functions numerically, but what about deriving symbolic expressions? How Graph Networks can learn Newtonian Physics and Dark Matter!
@MilesCranmer
@PeterWBattaglia
@KyleCranmer
@DavidSpergel
@cosmo_shirley
12
160
709
🔥New Video🔥
"Isn't Mamba just a fancy LSTM?" - turns out, there are some key differences! This video is a close look at selective state spaces:
9
80
674
"Grokking" is weird: Neural Networks trained to fill in binary operation tables will quickly overfit to the training data, but after many, many steps suddenly "get it" and achieve 100% validation accuracy.
19
121
660
Stable Diffusion had a good run. It was the cool new kid on the block. Sadly, it's now in TensorFlow. Have fun with it boomers...
Stable Diffusion implemented using
@Tensorflow
and
#Keras
.
- Converted pre-trained models
- Easy to understand code
- Minimal code footprint
Code :
Google Colab with
@Gradio
demo :
27
338
2K
24
37
653
This model learns, unsupervised, to translate code from Python to C++, including standard library calls and type inference! 👀 Watch this video to find out how!
@MaLachaux
@b_roziere
@LowikChanussot
@GuillaumeLample
@facebookai
17
148
612
Another sad day for open source. I personally wrote the first version of token-streaming for this.
Today is a huge milestone for one of our latest libraries: Text Generation Inference - we released v1.0 and under a new license: HFOIL 1.0
This 🧵 explains what this new license means, and why the change!
17
26
123
18
65
606
Men who use LARGE language models, is it possible that you're compensating for something?
48
34
604
This is the worst AI ever! I trained a language model on 4chan's /pol/ board and the result is.... more truthful than GPT-3?! See how my bot anonymously posted over 30k posts on 4chan and try it yourself. Watch here (warning: may be offensive):
35
84
581
Apparently, Stanford is putting together a strongly worded letter against me.
I'm not kidding. A strongly worded letter.
53
9
586
The AI ethics community is dead. It has no more power. This is good because it was never about ethics.
Next are the effective altruists, most of which are neither effective nor altruistic.
Sanity will win
35
47
591
JavaScript be like
"==" the same
"===" really the same
"====" really, actually the same
"=====" you won't even believe how the same those things are
12
47
555
🔥New Video🔥
Flow matching (not classic diffusion) is the basis for state-of-the-art text to image models, like Stable Diffusion 3.
Here is how it works:
4
88
555
GPT-3 is out and it is HUGE 🤯 Turns out that a pure Language Model can zero-shot almost any NLP Task! Here's my video summary of this 175 BILLION parameter beast!
@nottombrown
@8enmann
@AlecRad
@Dario_Amodei
@arvind_io
@girishsastry
@AmandaAskell
@ilyasut
14
144
541
NeurIPS introduces a track dedicated to advancing kids of rich parents even more than they already are
NeurIPS 2024 will have a track for papers from high schoolers.
79
92
595
20
36
545
I have a secret for you...
#manufacturedoutrage
Will releasing the weights of large language models grant widespread access to pandemic agents?
Turns out, yes, probably.
1/5
61
108
461
18
28
539
According to the current AI landscape, Microsoft Word is Open Source, because I can use it for free as a student.
16
31
509
🔥New Video🔥
RWKV takes the best of both worlds: Transformers and RNNs and combines them into a scalable architecture that is refreshingly different. This video dives deep into how it works and where its tradeoffs are:
12
81
496
To all "it's merit based" responders: if you reward skills before they are introduced in the public school system, the vast majority of rewardees will come from extremely privileged backgrounds that support and incentivize them to acquire those skills privately.
NeurIPS introduces a track dedicated to advancing kids of rich parents even more than they already are
20
36
545
20
31
495
Shocking: A trained model beats an untrained model. It's 2023 everyone 😁
Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks
introduce Goat, a fine-tuned LLaMA model that significantly outperforms GPT-4 on a range of arithmetic tasks. Fine-tuned on a synthetically generated dataset, Goat achieves state-of-the-art performance on BIG-bench…
20
183
739
15
17
490
It's surprisingly fun to collect data for OpenAssistant - Our open-source alternative to ChatGPT!
Check out the video:
#openassistant
#chatgpt
23
94
474
Who is behind
#StableDiffusion
? Check out this interview with
@EMostaque
, Founder of Stability AI. We chat about open sourcing models, building a giant compute cluster from scratch, and how he envisions a true democratization of AI.
@StableDiffusion
8
89
467
🔥Short Video🔥MLP-Mixer by
@GoogleAI
already has about 20 GitHub implementations in less than a day. An only-MLP network reaching competitive ImageNet- and Transfer-Performance due to smart weight sharing! Check it!
@neilhoulsby
@giffmana
@__kolesnikov__
5
74
452
Oh yes, "neural networks", that one algorithm 😁
There are thousands of machine learning algorithms out there, but you'll rarely need more than a handful.
A good start:
• Linear/Logistic Regression
• Decision Trees
• Neural Networks
• XGBoost
• Naive Bayes
• PCA
• KNN
• Random Forests
• K-Means
23
271
2K
16
21
433
New Video 🥳 This paper uses lots of compute to learn a single, unified LSTM-based optimizer on over 6000(!) different tasks, then uses that optimizer to TRAIN ITSELF! We're in full meta-land 😱
@GoogleAI
@Luke_Metz
@niru_m
@bucketofkets
@poolio
@jaschasd
2
93
430
A strategy that never fails: Reel them in with the hype, then, when they least expect it, educate them!
11
33
436
🔥New Video🔥Decision Transformers gets remarkably good performance on Offline RL by just ditching everything RL and using sequence modeling🤯Check it
@lchen915
@_kevinlu
@aravindr93
@kimin_le2
@adityagrover_
@MishaLaskin
@pabbeel
@AravSrinivas
@IMordatch
11
86
422
@lexfridman
Continued by GPT-3:
"2. Those who cannot
In the first case, the person is a scientist. In the second case, the person is a journalist."
9
9
417
New Video 🔥 How I Read A Machine Learning Paper
Here's my process of reading and understanding the DETR object detection paper in an efficient manner.
9
82
414
🔥Special Video🔥I built a Neural Network in Minecraft
🎲No Mods, No Command Blocks
📶Analog, not Digital
⛏️Backpropagation & Weight Updates
⚙️Fully Automatic
🧑💻Open Source
This video details what it does, how it works, and how it's built. Don't miss it 😉
17
97
413
How do Machine Learners diet?
They turn on weight decay.
16
42
414
I am
[x] pro vaccine
[x] anti excessive government pressure
yet when I protest the latter, I'm immediately lumped in with the antivax crowd. My opinion is not registered anywhere because people like me just don't speak up, and I feel I'm not the only one. Who else feels this way?
60
26
406
I asked this person twice already for an actual, concrete instance of "harm" caused by gpt-4chan, or even a likely one that couldn't be done by e.g. gpt-2 or gpt-j (or a regex for that matter), but I'm being elegantly ignored 🙃
This week an
#AI
model was released on
@huggingface
that produces harmful + discriminatory text and has already posted over 30k vile comments online (says it's author).
This experiment would never pass a human research
#ethics
board. Here are my recommendations.
1/7
51
102
332
37
19
376
Join me for this video👯We take a look at
@facebookai
's DINO architecture, pushing Self-Supervised Learning for Vision Transformers to truly impressive levels🔥🔥🔥
Check it out!
@julienmairal
@armandjoulin
3
76
413
Google has a weird definition of "shared".
1/ In 2021, we shared next-gen language + conversation capabilities powered by our Language Model for Dialogue Applications (LaMDA). Coming soon: Bard, a new experimental conversational
#GoogleAI
service powered by LaMDA.
742
3K
15K
9
19
405
New👏Video👏NFNets achieve new ImageNet SotA by DROPPING batchnorm😱They train 9 times faster than EfficientNet and excel at transfer learning🔥Code is available, too💪Watch now & don't miss some spicy comments from me😄
@ajmooch
@sohamde_
@SamuelMLSmith
13
69
403
New Video 🥳 Modern Hopfield Networks can store & retrieve exponentially many patterns and have a surprising and intricate connection to Transformer Attention Mechanism! 🔥
@HRamses2
@MichaelWidrich
@milenapavl
@SandveGeir
@victorgreiff
@jbrandi6
@LITAILab
7
92
396
@percyliang
Could you please at least link the video in the letter so people can make up their own mind?
11
6
375
It's 2025. MLP-Supermixer-200T outperforms every human at every task.
... "bUt DoEs It ReAlLy UnDeRsTaNd AnYtHiNg?"
18
24
373
Yesterday I released a video going over V-JEPA, how it works, and why it matters (including a recap of the original JEPA).
Watch here:
6
40
374
🔥New Video🔥GLOM is
@geoffreyhinton
's new Computer Vision idea🥳The model represents part-whole hierarchies into implicit parse trees via a multi-step attention-based consensus algorithm👀Excited? Me too! Watch the video to find out more!👇
@GoogleAI
9
58
370
🔥New Video🔥
@DeepMind
AlphaFold 2 delivers major AI breakthrough in Protein Folding🧬Beats all competition by HUGE margins🤯Watch to learn how AlphaFold 1 works and what we can guess about AlphaFold 2💪 (Hint: Transformers 😉)
@demishassabis
#AlphaFold2
7
57
366
Turns out loading models from the hub (or any other place) is ⚠️ NOT SAFE ⚠️ and opens you up to arbitrary code execution by an attacker🤯
Learn how to do it yourself (and how to protect against it) in this video:
10
59
360
🔥New Video🔥How to backpropagate through an algorithm? Seems crazy, but this paper shows it's actually possible for a large class of algorithms, such as k-subset, ILP, and many graph algorithms. Watch my (amateur 🙃) attempt at an explanation here:
3
68
361
Give us the models? 🤷♀️
If you are a developer using the
@OpenAI
API, DALL-E, ChatGPT, etc. what can we do to make the developer experience better? 🧵👇
290
147
1K
8
21
360
New Video 🔥 No more O(N^2) complexity in Transformers: Kernels to the rescue! 🥳 This paper makes Attention linear AND shows an intriguing connection between Transformers and RNNs 💪
@angeloskath
@apoorv2904
@nik0spapp
@francoisfleuret
@EPFL_en
@Idiap_ch
4
74
354
🔥New Video🔥This almost seems like magic🪄DeepMind's AlphaTensor finds new algorithms for doing matrix multiplication that use less multiplication operations(!) than any algorithm humans have discovered so far. Watch here to see how they do it 👇
4
65
356
How to make your CPU as fast as a GPU? 🔥 Nir Shavit explains how clever algorithms can make use of sparsity in neural networks to deliver unprecedented inference speed, without any need for specialized hardware!
Watch here:
6
56
345
One month from now: SotA on ImageNet by really large logistic regression on patches.
11
17
345
For the common good, download and backup this model.
15
48
338
🎉ML News: Generative MEGA-Models🎉
- Google PaLM: Amazing 540B Transformer
- OpenAI DALL-E 2: Text-to-Image breakthrough
- Open CLIP, open VQGAN diffusion, open datasets
- Salesforce CodeGen
- ...and the surprises one finds in Zurich 😉
5
52
340
Meta, you did almost everything right.
Now grow a pair and keep that demo up.
15
10
333
Computer Vision just got an Upgrade 🔥 SpineNet is a smaller, better and faster replacement to ResNet by
@GoogleAI
obtained using Neural Architecture Search 💪 Watch the Video 👀
@Phyyysalis
@tanmingxing
@YinCui1
@quocleix
Thumbnail Art by Lucas Ferreira!
3
73
339
New Video on the recently released Mixtral of Experts paper. We look into sparse mixture of experts routing, and note the distinct absence of any mention whatsoever where the training data came from.
Watch here:
6
38
333
People who advocate for "safe" LLMs sometimes don't consider what this word means to other people
Just checking in on alignment of LLMs in China, it's going about how you'd expect.
87
208
2K
16
23
333
🥳Special Video🥳You've just started PhD have no clue what to do? Welcome to the club🙂A Survival Guide for PhDs in Machine Learning🧑🔬How to do topic selction, conferences, paper writing & what I learned from many mistakes👍Watch, Like, Share🔥Thank You
9
62
328
👉Paper Explained Video👈Today:
@DeepMind
's new Perceiver model solves Transformers' quadratic bottleneck by using cross-attention into a self-attentive RNN backbone🦴Can attend to 50k pixels at once!👀Watch Now!
@drew_jaegle
@OriolVinyalsML
@joaocarreira
9
53
319
🔥Here we go🔥 The first OpenAssistant models are out! We have collected the most amazing human dataset ever and it shows: This model is really cool!
Watch the video to see it in action and come give it a try:
16
62
319
Oh no! Now we will be flooded with fictional plays that are COMPLETELY MADE UP!!1!
Introducing Dramatron, a new tool for writers to co-write theatre and film scripts with a language model. 🎭
Dramatron can interactively co-create new stories complete with title, characters, location descriptions and dialogue.
Try it yourself now:
178
959
4K
14
20
318
A joke. It's called a joke.
Oh the things people can get offended by 🤦🏽♀️
This is one reason why people are afraid of contributing to the community
-Divam did a great job! spent their time creating something super cool and shared with everyone
-Just to have someone come and shi* on their head for no reason!
This is very sad!
Don't be like that!
17
10
132
35
5
311
🎉 New Video 🎉 Knowledge Graphs are very expensive to make, they need human experts. Or do they? 🧐 What if we replaced them with BERT or GPT-2? 🤯 Turns out, works really well, all without training! 🥳
@ChenguangWang
@dawnsongtweets
@ShawLiu12
#AI
#NLP
4
79
311
🚀New Video🚀
ReST bootstraps its own extended dataset and trains on ever higher-quality subsets of it. Re-using generated data multiple times means an efficiency advantage with respect to Online RL techniques like PPO.
Watch here:
5
58
314
🔥New Video🔥 EVERYBODY is talking about
@OpenAI
's new DALL·E model 👀 It takes any piece of text and turns it into an image, absolutely crazy 😱 Watch the video to learn more💪
#DALLE
@ilyasut
@_jongwook_kim
@MikhailPavlov5
@gabeeegoooh
@scottgray76
7
59
306
To make things even better, we are making this entire dataset free and accessible to all who wish to use it. Check it out today at ! 🎉
5
48
309
🔥New Video🔥 FFT Magic🪄Fourier Neural Operators speed up PDE solvers by orders(!) of magnitude 🤯 Trained once, solve entire PDE families for any discretization!🎉Watch to find out more⏭️
@ZongyiLiCaltech
@kazizzad
@AnimaAnandkumar
@Caltech
#ai
#science
7
38
305
🔥New Video🔥 Linear Attention! Unbiased Estimator! Random Features! Orthogonal Features! Low Variance! Tight Bounds! Kernels! Backw. Compatible! The PERFORMER has it all🤯 Watch!💪
@XingyouSong
@kchorolab
@andreea_gane
@lukaszkaiser
@dmdohan
@CambridgeMLG
4
44
295
No son of a construction worker is just going to randomly start doing ML research if they never hear of it and don't get told that it could be important for their future career, no matter how intelligent the kid is
16
8
300
How to make money with NFTs:
1. Buy an NFT
2. Use it as a reminder for the rest of your life to not make shitty decisions.
3
28
298
🎉New Video🎉TransGAN is the first successful attempt at building GANs with NO convolutions🔥Generator and Discriminator are Transformers (of course)👀Watch now to find out what 3 tricks make it all work!🧙(
#3
will surprise you ;))
@CodeTerminator
5
44
291
Ok I get it, I'm not not the favorite child 😁
11
1
288
🌏New Video🌎
Scaling Transformers to 1 MILLION tokens and beyond. We'll take a look at what lies behind the Recurrent Memory Transformer and see whether it lives up to the hype.
Watch here:
3
39
295
my_opinions != your_opinions
my_opinions = !your_opinions
important difference
11
28
290
👉Video Out Now👈Self-Supervised Learning: The Dark Matter of Intelligence by
@ylecun
,
@imisra_
,
@facebookai
: "We believe that SSL is one of the most promising ways to [...] approximate a form of common sense in AI systems."🔥Watch to learn more!
6
58
291
🔥New Video🔥Do Transformers learn universal computation primitives? GPT-2 pre-trained on language can transfer to vision while COMPLETELY FREEZING all attention weights🤯Only .1% of parameters tuned👀
@_kevinlu
@adityagrover_
@pabbeel
@IMordatch
4
55
285
New Video 🥳 Transformers are coming for Images 😱 Axial-DeepLab combine learned Positional Embeddings w/ Axial Attention and get SotA on Segmentation with a fully Attentional model! No Convolutions 🐐
@YuilleAlan
@imadamtm
@JohnsHopkins
@GoogleAI
2
74
287
Here's that Stanford letter condemning me. Of course, there's no link to my video, otherwise people could make up their own mind. We don't want that! Just shut up and sign!
There are legitimate and scientifically valuable reasons to train a language model on toxic text, but the deployment of GPT-4chan lacks them. AI researchers: please look at this statement and see what you think:
73
138
504
37
19
276
YouTube's format just doesn't lend itself to educational long-form content anymore. I will henceforth do my paper reviews on TikTok.
14
3
285
Were you always suspicious of VAEs? 👀 Too blurry? Unstable? 😱 Well stop right there, because
@NVIDIAAI
has built NVAE, a hierarchical multi-scale VAE that can output crispy clear samples at high resolution 💃🔥 Watch the Video!
@ArashVahdat
@jankautz
7
45
277
🎞️Saturday Video Time🎞️Involution is a drop-in replacement for Convolution in a CNN. Clever weight-sharing and a hint of self-attention make this a very performant and efficient layer for image analysis💪Their RedNet outperforms ResNet by quite some margin
5
40
280
⛱️Paper Video⛱️FNet completely REMOVES Attention from BERT and replaces it with a Fourier Transform. No parameters at all🤯works almost as well as full Transformers and trains an order of magnitude faster💪
@ilyaeck
@santiontanon
5
50
277
Neural Architecture Search is usually done using RNNs or Genetic Algorithms, but both are painfully slow 🐌 What if there was a way to predict the performance of a new architecture *at initialization* 😱 ? Watch the Video to find out how it's done 😎
4
46
275
Open Data is king👑LAION-5B is a dataset of over 5 BILLION image-text-pairs, available to download. In this video I speak to three of its creators about operating at scale on a budget, grassroots research, and the challenges of building such huge datasets.
7
56
275