Dr. Karen Ullrich
@karen_ullrich
Followers
4,420
Following
564
Media
19
Statuses
216
Research scientist at FAIR NY + collab w/ Vector Institute. ❤️ Machine Learning + Information Theory. Previously, PhD at UoAmsterdam, intern at DeepMind + MSRC.
she / her
Joined December 2013
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
México
• 1598655 Tweets
México
• 1598655 Tweets
Morena
• 846177 Tweets
Xóchitl
• 489474 Tweets
PREP
• 299600 Tweets
増山さん
• 88910 Tweets
増山江威子さん
• 84344 Tweets
峰不二子
• 70253 Tweets
ルパン三世
• 69761 Tweets
#FraudeElectoral
• 34433 Tweets
Kaveh
• 29180 Tweets
土砂降り
• 23840 Tweets
肺炎のため死去
• 22614 Tweets
ゲリラ豪雨
• 21741 Tweets
SmangatPRABOWO UntukNKRI
• 21498 Tweets
KitaKOMPAK KitaKOKOH
• 20648 Tweets
カーヴェ
• 15639 Tweets
Happy New Week
• 15291 Tweets
キューティーハニー
• 13898 Tweets
不二子ちゃん
• 13035 Tweets
#圧倒的ザクザク感
• 12980 Tweets
#MondayMotivation
• 12894 Tweets
ベンチマーク
• 12582 Tweets
#وزاره_التربيه
• 10074 Tweets
Pinned Tweet
I uploaded our
#Neurips2022
Tutorial on Data Compression with Machine Learning to Youtube.
collab. w/
@YiboYang
+
@s_mandt
❤️🔥
4
47
239
I finally uploaded my PhD thesis
“A coding perspective on deep latent variable models”
() on Gscholar 🙃
It’s my ❤️ letter to the minimum description length principle for machine learning (+ pastel gradients 😋).
22
112
1K
🚨 Internship opportunity 🚨
You are interested in working in information theory+neural compression (+representation learning+continual learning+mechanistic interpretability more broadly) + work at FAIR NY + flexible starting date in 2024. DM me + send a CV/link to your website
13
90
537
During my
@deepmind
internship supervised by
@deepspiker
, I have been working on improving the quality of skype/hangout/zoom calls with generative models.
Our paper (w
@FabioViola
) 👇
"Neural Communication Systems with Bandwidth-Limited Channel" () [1/3]
6
58
485
Dear neural compression enthusiasts,
there is a new
@PyTorch
-repo in town
Includes;
🔥neural image and video compression
🔥bits-back coders
🔥GPU entropy coders
We already work on extensions, feel invited to contribute ❤️❤️❤️
3
72
365
🚨Internship application season is open y’all 🚨
If you are interested in information theory, neural compression, and continual learning; let’s learn and explore together @ FAIR New York.
I got one internship spot for 2022 with a flexible starting date. DM me.
14
50
304
Proud to show our work on differentiable graphical models in the Fourier domain for protein reconstruction and other projection methods:
Thanks to my amazing collaborators David Fleet,
@marcusabrubaker
,
@vdbergrianne
and
@wellingmax
. ❤️❤️❤️
1
44
238
2015: I was the only girl in
@wellingmax
's AMLAB and pretty lonely at Neurips
2019: 👇Look at us now 😍
Thanks
#WiML2019
1
4
120
🔥🔥🔥NEW PAPER ON ARXIV 🔥🔥🔥👇🏻
Most data is processed by algorithms, but compressors (eg JPEG) are for human eyes.
🤓Our fix: formalize lossy compression that ensures perfect downstream predictions
🔥1000x gains vs JPEG on ImageNet🔥
w. Ben Bloem-Reddy
@karen_ullrich
@cjmaddison
1/9
9
140
820
1
3
78
📢 Neural Compression Enthusiasts @
#Neurips2022
; Tue Nov 29th, 3.30 pm , Room 282 inside the Convention Center.
Let’s meet, chat, and get inspired!
Hope to see ya there ❤️
3
10
77
@y0b1byte
@BahareFatemi
I am sorry to hear about your experience. I would like to add that we should start to understand the depression of grad students as systematic, not just individual problems. Consequently, we should also seek systematic change to how grad school works.
1
1
70
New paper accepted as long talk at ICML! We improve the compression capabilities of latent variable models.
TLDR; 👇🏻
w/ the fantastic
@YangjunR
@_dsevero
@_j_towns
@AliMakhzani
Arnaud Doucet, Ashish Khisti and all held together by
@cjmaddison
You want to compress data with a latent variable model, but bits-back achieves a suboptimal code length (neg. ELBO). We show how to break this barrier with asympt. optimal coders: Monte Carlo Bits-Back (McBits, ) 1st auths
@YangjunR
@karen_ullrich
@_dsevero
2
30
157
0
9
63
2.5y ago I started work on cryo electron microscopy data. My first paper, w David Fleet and
@WellingMax
, introduced fast variational approx. to unknown proteins.
@a_punjani
, also student of David, leads a company that is vital in visualizing
#COVID19
right now.
🤩👇
1
4
60
Eliminating architecture as an HP. Compressing models by a factor up to 700x.
Code for our
#nips2017
paper
0
19
59
🤩 Tomorrow the
#ICML2023
workshops shall begin.🎙️ Join me at 9 AM I will discuss absolutely all there is to know about
#BitsBackCoding
@ the Structured Probabilistic Inference & Generative Modeling workshop.
#AI
#ML
#DataCompression
1
7
58
And you thought we couldn't make another bits back paper 😈
This time, we use bits back to strictly remove information🤯Specifically, we turn a sequence into a set by removing permutations
👉🏻 Compress a dataset when all you got is a sequence codec, e.g. an arithmetic coder.
We've found a way to save bits during compression by forgetting the order between examples in a dataset.
No machine learning required!
Authors:
@_j_towns
(eq contr) A. Khisti
@AliMakhzani
@karen_ullrich
1/6
4
21
114
1
9
53
Check out
@wellecks
' podcast the
@thesisreview
. He entertains conversations around the development of research ideas through researchers PhD thesis.
Great resource for undergrads and early PhD students.
Episode 28 of The Thesis Review:
Karen Ullrich (
@karen_ullrich
), "A Coding Perspective on Deep Latent Variable Models"
We discuss information theory & minimum description length, covering her PhD research on compression and communication.
1
4
21
2
10
53
I couldn't find a simple implementation of a binary VAE on github, so I made one: .
❤️
#tensorflow_probability
2
9
53
Another one of
@3blue1brown
’s educational master pieces. Great intro to what we understand as information. Will steal for presentations and lectures for sure ❤️😍💗
1
4
52
My slides on sparse Bayesian learning from yesterday's talk at the MILA Deep Learning Summer School.
1
22
47
Yeah!
@ChrLouizos
and I got accepted to
#NIPS2017
.
Getting rid of up to 99.4% of all parameters the Bayesian way.
0
10
36
Data/Model Compression enthusiasts, what do we think about a mini-meetup at
#Neurips2022
on Monday Nov 28th ?
@_dsevero
@lucastheis
@mentzer_f
@YiboYang
@s_mandt
6
0
34
Jakub explains lossy neural compression 🔥🔥🔥
After a long time, the 8th blog post is out! This time I discuss neural compression with deep generative modeling.
Post 🤓:
Code 💻:
4
64
310
0
1
30
I will be at the affinity groups poster session (!!) today. Really excited to meet and discuss with everyone there🤩 If you are interested in information theory and coding look for me👇 I can’t wait to meet and discuss your work, potential collaborations and/or a FAIR internship.
1
0
29
🔥New implementation 🔥 of our set-compressor available.
Use your favorite codec, e.g. PNG, JPEG or flow+ANS and get significant gains 👇🏻
Best part: this method doesn't compete with your favorite codec, it only improves it!
🔥 7.6% reduction in the number of bits needed to compress BMNIST using the "Bits-back with ANS" neural codec, with only a 10% increase in compute time
Craystack code:
0
2
27
1
3
28
🌍, I am on a 🇪🇺 tour: I will be giving talks in Leipzig on Sep 7th, Vienna Sep 9th and Amsterdam Sep 16th + 20th.
Come by if you are interested in chatting about machine learning and information theory IRL.
more infos 👇🏻
2
0
27
👇🏻We showed not just THAT but HOW known equivalence relationships reduce compression rates by orders of magnitude.
🔥 And got a spotlight at
#NeurIPS
🔥
Grateful to have worked w.
@yanndubs
, Ben Bloem-Reddy +
@cjmaddison
on this project ❤️
We released the code for our paper (now spotlight at
#NeurIPS2021
🥳):
Pretrained compressors are also on torch hub, use the following few lines to compress your image datasets (1000x gains vs JPEG on ImageNet):
Colab:
3
14
104
0
1
26
AI for the Planet mini conference starting now!
Spend 4h to hear about waste management, climate change and biodiversity.
#FaceTheClimateEmergency
👉
0
6
25
@emidup
@sedielem
@adam_golinski
@notmilad
@yeewhye
@ArnaudDoucet1
😍 love the execution of the realisation that in compression we may want to overfit !!
3
0
23
Join us in discussing the frontiers of communication at our ICML “Neural Compression workshop”. And stay tuned: call for papers coming soon.
🎉Exciting news! Our "Neural Compression" workshop proposal has been accepted at
#ICML
2023! Join us to explore the latest research developments, including perceptual losses and more compute-efficient models!
@BerivanISIK
,
@YiboYang
,
@_dsevero
,
@karen_ullrich
,
@robamler
4
24
99
0
2
23
Also, I hid an unrequested but fun little puzzle in the cover page.
3
0
22
.
@hen_drik
and I will be speaking tomorrow at
#34c3
12:45 in
#SaalAdams
on
#ArtificialIntelligence
and how it influences our public life
0
6
20
If you missed it yesterday, I heard the poster will make a reappearance at the Neural Compression workshop today in Room 317A 😉
0
2
20
2. We propose a design to model missing information instead of ignoring it.
3. By introducing auxiliary latent variables in the decoder, we can sample more realistic messages. [3/3]
2
0
17
We just put our slides for "Beeinflussung durch Künstliche Intelligenz" our talk at
#34c3
online.
0
5
13
After sitting on both sites of the reviewing process, here is a proposal to make this more enjoyable for everyone. [1/4]
2
0
16
My student
@eelcovdw
worked on convolutional seq2seq models for Music.
Our paper was just accepted to
@ISMIR2017
.
0
2
16
Using a generative model to analyse a generative model, so clever :)
Happy share our work: Shaping Belief States with Generative Environment Models for RL
Thanks Karol Gregor, Frederic Besse, Yan Wu, Hamza Merzic and
@avdnoord
!
#RL
#SelfSupervised
#GenerativeWorldModels
#BeliefStates
2
57
217
0
2
14
"Hard to believe" is an understatement for large language model atm.
2
1
14
@maosbot
TBH I think this statement is tone deaf. It ignores that we are
👉🏻already massively contributing to providing tools that help marginalization at scale
👉🏻often do not understand the consequences of our tools
👉🏻traditionally do not give space to underrep. groups in this community
2
0
13
Huge service to the community by
@julberner
: 1st open-source implementation of bits-back compression for diffusion models (= SOTA for lossless compression)❤️🔥🔥
📢📢New feature in
#NeuralCompression
repo: Bits-Back compression for diffusion models!
Compress image data 🖼️ using diffusion models at an effective rate close to the (negative) ELBO.
See:
Some context ⏩[1/4]
1
6
35
0
2
13
Excited to be at the
#Bayesian
#DeepLearning
Workshop all day. Also presenting a poster, come by and have a chat
#AMLab
#NIPS2017
0
3
13
Did any of u try ELIMINATING ALL BAD LOCAL MINIMA (by
@jaschasd
+ Kenji Kawaguchi) for real? what application? what is ur experience?
theoretical argument or actually useful? how did u tune optimization for a and b?
1
4
12
Find me as on mastodon. I will be cross-posting for now.
1
1
12
@egrefen
@UniofOxford
@facebookai
Some additional spread;
@WiMLworkshop
@QueerinAI
@AiDisability
@black_in_ai
@Khipu_AI
@DeepIndaba
@_LXAI
@women_in_ai
1
0
12
@danijarh
@emidup
@sedielem
@adam_golinski
@notmilad
@yeewhye
@ArnaudDoucet1
Yeah :) From a minimum description length point of view, neural architectures act as universal computer language and the weight vector as code! Very OG Solomonoff interpretation of MDL :) Love it.
0
0
11
My invited talk at the CIFAR DL Summer School is now online :)
I am on TV, mom!
0
0
9
We found 3 modelling choices relevant when the bandwidth of the noisy channel, aka weak wifi, varies.
1. Instead of separating the sub-tasks of compression (source coding) and error correction (channel coding), we propose to model both jointly. [2/3]
1
0
10
I will be at the
#WiMLworkshop
mentorship tables now! Come by and talk about neural compression, internships and whatever is on your mind.
0
1
10
I made another tool for bit enthusiasts: Turn your float tensor to binary (and back) according to IEEE-754 standard (or any custom format)
It's differentiable 🤩 but not very efficient 😬 . ❤️
#PyTorch
0
0
8
All my amazing colleagues Putri van der Linden,
@ElisevanderPol
,
@vdbergrianne
,
@YugeTen
,
@LindaPetrini
and
@sindy_loewe
0
0
9
0
0
8
Note that you must currently be in a relevant PhD program.
3
0
7
Amsterdam Sep 16th, 13.oo @ Vrije Universiteit Amsterdam
Hosted by Prof. Jakub Tomczak
2
1
6
This is no corporate event: no fancy food, no preparations, it’s just us in a room. Bring your own coffee (BYOC). Maybe I can hook up the stereo with my playlist, that's top fancy to be expected. Also no registration required.
1
0
6
0
0
6
Amsterdam Sep 20th 16.oo @ University of Amsterdam, Science Park LAB42
Hosted by Prof. Jan-Willem van de Meent
(might change, this one is still under construction)
0
0
6
automatic hyperparameter optimization for deep learning
0
3
5
Leipzig September 7th, 15.oo @ AI Institute / Leipzig room A.03.07
Hosted by Prof. Sayan Mukherjee
1
0
4
0
0
4
@egrefen
@UniofOxford
@facebookai
@WiMLworkshop
@QueerinAI
@AiDisability
@black_in_ai
@Khipu_AI
@DeepIndaba
@_LXAI
@women_in_ai
I saw
@NandoDF
advocating for it, thought it's a good idea :)
1
0
4
@senya_ashuha
@liyzhen2
@dpkingma
@latentjasper
Pleasure and privilege to have been invited! Congratulations 🔥🎉❤️❤️
1
0
4
1.Don’t say “This is bad”, ask “Is this bad?”. 2. Let's not grade a paper before the author rebuttals.[2/4]
1
0
4
@george_toderici
@mattmucklm
@_dsevero
It does open the door 🤩 and with that we may well blur the boundaries between probabilistic modelling and arithmetic coding. Very exciting, I think.
0
0
4
Vienna September 9th 10.oo @ Institute of Science and Technology Austria
Hosted by Prof Christoph Lampert
1
0
4
Great (technical) read about reality after AI-generated-fake-content takes over
After many exciting discussions with
@SimoneLini
@mortendahlcs
and Hamish Ivey-Law, we wrote on the implications of AI and digital forgery on modern society and discuss plausible technical solutions:
And we did human vs. machine exp, e.g. which is real?
3
19
34
0
0
4
@AIandMLonly
Having a PhD is actually not a requirement. But being in a PhD program is. Hope I did not sound rude, just wanted to save myself from replying to many emails having to clarify that <3
1
0
4
Guilty 🙈
First blog post in 5 years! Don't cite the "No Free Lunch" theorem! cc
@betatim
@mrocklin
@hug_nicolas
This was probably the longest of the blog-posts I was thinking about writing, maybe the next one won't take 5 years.
6
66
262
0
0
4
Proposal for AI conscienceless test: Instead of just reward hacking, show me your AI demonstrated malicious compliance. How can we test that?
1
0
3
@mmbronstein
@adjiboussodieng
@_joaogui1
Doesn’t that very example show that the issue is not 0 and 1? There are different levels of community (family, municipality, state ect) with different levels of solidarity (shared income, taxation, infrastructure ect). We can define the rules of this continuum however we see fit.
1
0
3
How to ... Ethical NLP by Katharine Jarmul (
@kjam
)
Wanna watch my (first ever) keynote? On Ethical Machine Learning? You can! 🎉 Thanks
@pydataamsterdam
2
16
36
0
0
3
@victor_p91
Thank you if fixed my settings. There will likely be more open spots for FAIR in 2024.
0
0
3
0
0
2
@_dsevero
@lucastheis
@mentzer_f
@YiboYang
@s_mandt
@mentzer_f
@jefrankle
@_dsevero
@Alii_Ganjj
Love that there is so much interest 😍
I sent a room request to the
#Neurips2022
organizers, and will send an invite link 📅 to the event on Monday.
In case twitter is down by than follow me on karen_ullrich
@sigmoid
.social
0
0
2
@F_Kaltheuner
@karen_ullrich
(Postdoc probabilistic AI)
@MaxIlse
(PhD cand. causal AI)
@SarahEskens
(Ass. prof. for digital rights + AI)
0
0
2
Plus if you happen to be wrong as a reviewer you don’t actually have to admit to it ;). After all you was only asking a question.[4/4]
1
0
2
1
0
2
0
0
2
@Alii_saays
Preferably yes, but we can also arrange for a remote option if visa is an issue (not for all remote locations though). Pref. will not influence hiring process.
1
0
2
@cjmaddison
Feels like a very big publicity stunt to me. Also MSF drawing up a CEO contract for Altman over the course of a weekend... don't know if I believe that.
0
0
2
@maosbot
I know, otherwise I wouldn’t have responded :)
I think it would be more helpful to discuss here instead of deleting anything.
1
0
2
@YoSiJo
@hen_drik
Zb Wenn ich meinen Algorithmus darauf trainiere, Katzen von Hunden zu unterscheiden, und ihm dann einen Teller Spagetti zeige, wird er mir sagen das ist 99%ig eine Katze und 1%ig ein Hund. Ein Problem mit dem sich meine Forschungsgruppe in Amsterdam intensiv auseinandersetzt. 2/2
0
0
1
@mmbronstein
@adjiboussodieng
@_joaogui1
The EU being a good example of how national borders got dissolved slightly. And IMO it is about time for more global solidarity.
1
0
1
@maosbot
I work in data compression and I worry a lot about biases! I think we could be better at developing standardized tests for AI instead of presenting a novel method yet again.
What do you think can be done by us (concretely)?
1
0
1