Karan Desai (KD)
@kdexd
Followers
2,026
Following
433
Media
31
Statuses
474
I fight the devil in the details 🧐 Current: Cool startup, Prev: CS PhD @ University of Michigan
Ann Arbor, MI
Joined March 2017
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#LoveBullyรักให้ร้ายEP3
• 1019429 Tweets
#FreeBilly
• 339906 Tweets
BLOOMING MOOD PHOTO
• 267236 Tweets
MILE PHAKPHUM IN DIOR
• 190740 Tweets
lorde
• 138450 Tweets
Hawk Tuah
• 73397 Tweets
Austria
• 63107 Tweets
Poland
• 48817 Tweets
No Riley
• 43654 Tweets
Jacob Juma
• 36523 Tweets
Clarence Thomas
• 32841 Tweets
Indigenous
• 31766 Tweets
#POLAUT
• 30461 Tweets
STRAY KIDS ATE THE TRACKLIST
• 27476 Tweets
Polônia
• 27312 Tweets
$PIKA
• 24282 Tweets
Vargas
• 23684 Tweets
Mercer
• 20486 Tweets
Doña Segunda
• 10266 Tweets
$RGOAT
• 10139 Tweets
Lewandowski
• 10076 Tweets
Introducing "VirTex": a pretraining approach to learn visual features via language using fewer images.
Pretrain: CNN+Transformer from scratch on COCO Captions.
Transfer CNN: Results on 6 vision tasks match/exceed ImageNet pretraining (10x size wrt COCO)!
14
151
685
The ImageNet dataset is now 12+ years old. So it contains pre-2009 images. Hmm, so hundreds of thousands of dogs in ImageNet are now... dead?! 😭
17
45
679
Once in every 3 years, the Indian lunar calendar gets an extra month. In that year, CVPR deadline wastes my Navratri, Vijayadashmi, and Diwali, and ECCV/ICCV deadline wastes Holi.
Sometimes it is hard, staying alone thousands of miles away from family, knowing what I am missing.
6
7
298
📢New dataset!📢 RedCaps: 12M image-text pairs from Reddit for vision and vision-and-language applications.
Website:
Paper:
Check out captions from a RedCaps-trained model!⬇️
Try more here:
What's new?🧵1/8
8
73
282
@ylecun
@soumithchintala
I'm thinking of getting my son registered for NIPS 2040. Once I get a confirmation I will start searching for a wife.
3
13
230
I noticed a footnote in this paper. Why the rush?
It is strange to arXiv a paper with a "known* bug, but the bug is mentioned in a tiny footnote. This could be fixed by re-training in a few days. We as a community, need to slow down.
VMamba: Visual State Space Model
paper page:
Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) stand as the two most popular foundation models for visual representation learning. While CNNs exhibit remarkable scalability with linear
2
49
250
8
22
205
I defended my PhD dissertation last week!! 🎉🍾
Thanks to
@jcjohnss
for being a wonderful advisor, to my committee (
@jasonbaldridge
,
@andrewhowens
, Stella Yu) for their feedback, and to all my collaborators and friends for being a part of this journey!
22
3
179
VirTex is accepted to
#CVPR2021
! 🎉🎉
More details on paper and code (updated)⬇️
Introducing "VirTex": a pretraining approach to learn visual features via language using fewer images.
Pretrain: CNN+Transformer from scratch on COCO Captions.
Transfer CNN: Results on 6 vision tasks match/exceed ImageNet pretraining (10x size wrt COCO)!
14
151
685
7
23
168
This! I got all PhD rejects (2018 after undergrad). Took a gap year, did intern at
@gtcomputing
, got admit from
@UMichCSE
(2019)! Apart from LOR, I gained experience, calibrated expectations, felt better prepared to start a PhD. + a network of awesome friends and collaborators!
This is a bit non-standard but if you missed opportunities to do research as an undergrad, you could take a gap year and join a lab as a full time paid (usually not a lot) research assistant/intern. One full year of research experience could open up more doors for grad school.
4
10
221
3
6
152
Today marks the end of an awesome year-long internship at the
@GeorgiaTech
CVMLP lab of
@DhruvBatraDB
&
@deviparikh
. Thanks for this solid opportunity! Next position: PhD at
@UMich
, advised by
@jcjohnss
-- starting Fall 2019!
15
2
143
ICCV reviewing done!
CVPR 2021 prep done!!
PhD quals/prelims done!!!
And finally NeurIPS done!!!!
FAIR internship in momentum now!
So I survived May! I can enjoy a day off after nearly a month, explore NYC, and play some board games, I have missed in a long time! 🎉
2
0
135
Accepted to
#ICML2019
! This one is memorable for me, my first publication :-)
Our paper "Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering" is now on ArXiv ( )! w/
@vrama91
,
@stefmlee
, Marcus Rohrbach,
@DhruvBatraDB
,
@deviparikh
. We propose a class of probabilistic models for symbolic reasoning ...(1/3)
1
13
75
9
6
111
This week at
#ICML
I will present “Hyperbolic Image-Text Representations” — we bring hyperbolic geometry to CLIP and learn interpretable models that also match/outperform CLIP on standard vision tasks.
📜
📅 Jul 27, 10:30. Say hi!
With
@MetaAI
@michigan_AI
0
21
86
Our paper "Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering" is now on ArXiv ( )! w/
@vrama91
,
@stefmlee
, Marcus Rohrbach,
@DhruvBatraDB
,
@deviparikh
. We propose a class of probabilistic models for symbolic reasoning ...(1/3)
1
13
75
I am super excited to be interning at
@facebookai
this summer! 🎉
Today we’re welcoming our first group of AI Summer Research Interns to
@facebookai
! We have 198 interns joining this summer globally (and remote!), with over 250 interns signed for 2021 representing nearly 100 different universities in 15 countries.
3
10
315
2
0
76
US Visa approved, bags packed, tickets ready! I will moving to Atlanta, GA next Sunday for joining as a Visiting Research Scholar at
@gtcomputing
under supervision of
@DhruvBatraDB
,
@deviparikh
! New life is pretty exciting! End of the undergrad chapter at IIT Roorkee.
14
0
74
Sad and frustrated to see Indian students being denied a Canadian visa (with very vague reasons!) for a mere week-long visit to attend a top conference like
@CVPR
.
I urge the organizing committee to kindly help if possible (cc:
@drfeifei
@greg_mori
).
Also
@CanadainIndia
Indian PhD students from
@iiscbangalore
, who have first-authored papers at prestigious conferences like
@CVPR
, are facing unjust denials of Canadian visas. With Shocking reasons "limited employment possibilities in India" and "purpose of visit not consistent with a temp. stay."
69
419
2K
1
10
67
I had fun visiting Hawai’i for
#ICML
2023 and presenting our work on “Hyperbolic Image-Text Representations”! 🌋🏖️Code and pre-trained models are now released!
Paper:
Code:
@MetaAI
@michigan_AI
3
15
65
📢📢 New paper (w/
@ramprs21
@jcjohnss
@nikhil_ai
) identifies two bottlenecks when scaling MoCo-like contrastive SSL methods to complex scene images, and proposes a general training method to fix them!
Paper:
Blog:
More Details 👇👇
📢New Preprint📢 Introducing CAST--a generic training recipe to fix visual grounding of contrastive SSL models (eg MoCo) and make them work on complex scene images!
Paper:
Blog:
(w/
@kdexd
@jcjohnss
@nikhil_ai
)
@SFResearch
(1/3)
1
13
56
0
11
57
I am so excited to return for a summer internship at
@MetaAI
once again, this time in-person! 🎉
0
0
53
Code for our
@icmlconf
2019 paper "Probabilistic Neural-Symbolic Models for Interpretable Visual Question Answering" is now available!
Get it on Github:
Accompanying docs:
Want data-efficient “reasoning” models that expose their beliefs in a coherent manner?
Checkout our paper just accepted to
#icml2019
! W/
@kdexd
@stefmlee
Marcus
@DhruvBatraDB
@deviparikh
.
Code coming soon!
0
7
34
0
9
48
Very nice and succinct 30-min overview to our work! Thanks
@ykilcher
for making our work more accessible! 🎉
0
8
50
I recently open-sourced the implementation of EMNLP 2017 Best Short Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using
@PyTorch
and
@fb_research
/ParlAI. Find it on Github! ⭐️
1
12
50
New version of the Visual Dialog Challenge starter code: now with docker + multi-GPU support, pre-extracted Faster-RCNN image features, a stronger baseline and much more! Come participate, $10k prize up for grabs!
Github:
Challenge:
0
11
46
📢📢 An owl is an ‘owl’ no matter how it appears in countless different images. New CVPR paper with
@_mbanani
and
@jcjohnss
, we use language models to find images with similar objects and train strong self-supervised visual models! Details ⬇️
Language models keep getting better, can we use them for better visual learning? In our CVPR paper, we use language models to sample conceptually similar and visually dissimilar image pairs! We use those pairs for stronger contrastive learning.
🧵⬇️
3
41
212
1
13
45
nocaps has been accepted to
#iccv2019
! 🎉🎉
Presenting "nocaps", the first large-scale benchmark for novel object captioning, and a step towards captioning in the wild!
Paper:
Evaluation Server & leaderboard:
Download/explore the dataset:
1
9
38
1
3
45
@hardmaru
Two reviewers cite "ResNets strike back" paper in their review and use it partly to justify their rejection rating — ConvMixer submitted to ICLR in Sept and ResNet++ paper showed up on arxiv in Oct, not peer reviewed. This is not cool, possibly violates reviewer guidelines.
3
1
45
Decisions for
#CVPR2024
are out. One of my submissions was rejected with SA,WA,WR. WR reviewer (and AC) show signs of gatekeeping: "X was popular in the past. Does the community need a better version of X (= our work)?"
How can one reviewer decide what our huge community needs?
6
0
45
CAST is accepted to
#CVPR2021
! 🎉🎉
🧵⬇️
📢New Preprint📢 Introducing CAST--a generic training recipe to fix visual grounding of contrastive SSL models (eg MoCo) and make them work on complex scene images!
Paper:
Blog:
(w/
@kdexd
@jcjohnss
@nikhil_ai
)
@SFResearch
(1/3)
1
13
56
1
3
42
*le ECCV Reviews
R1: Recent method [X] outperforms your proposed approach on the dataset [Y].
ECCV deadline: 5 March
[X] came out on arxiv: 9 March
Sure R1, let me hop on tardis.
(╯°□°）╯︵ ┻━┻
3
2
39
Presenting "nocaps", the first large-scale benchmark for novel object captioning, and a step towards captioning in the wild!
Paper:
Evaluation Server & leaderboard:
Download/explore the dataset:
1
9
38
Build your models for the Visual Dialog Challenge on top of our starter code in PyTorch!
Get it from GitHub:
Details:
(with
@abhshkdz
@ayshrv
@SatwikKottur
@deshrajdry
Stefan
@deviparikh
@DhruvBatraDB
)
1
13
38
CVPR rebuttal: When authors say they can't run my suggested evaluation due to "computational constraints", but they train a huge model *from scratch* as a response to another reviewer *in the same rebuttal*!
???
6
0
36
I am taking a break to recharge for the second half of my Ph.D! I will be on a vacation till January 3, 2022. Logging off Twitter — please expect a delay in my response to all emails and social media.
Goodbye and happy new year (in advance)!
1
0
36
Happy to be on this list! 😄
3
0
33
Attending
#CVPR2021
? Come chat with me about VirTex and more on learning vision directly from natural language supervision. :)
When: June 23 [7 pm PT | 10 pm ET].
5-min video:
Introducing "VirTex": a pretraining approach to learn visual features via language using fewer images.
Pretrain: CNN+Transformer from scratch on COCO Captions.
Transfer CNN: Results on 6 vision tasks match/exceed ImageNet pretraining (10x size wrt COCO)!
14
151
685
2
8
30
Thing I'm not carrying forward to 2019 after many years of having them:
— Prescription glasses
— Hydrophobia
— Insomnia
— Sweet tooth
— Python2.7
2
0
29
Thanks
@Starbucks
for all the free wifi for my research project development, I spent many afternoons sipping cold brews and matcha green tea !
Finally, a well deserved shout-out to
@Starbucks
in the acknowledgements. Hope to visit soon someday 😬
@jcjohnss
2
1
36
0
0
26
New year resolution: no code on Saturdays.
It roughly translates to "take Saturdays off", but n(lines of code) is a concrete measure. This metric doesn't account for time spent reading/reviewing papers etc. but I would be happy if I at least take this much time off.
2
0
21
@math_rachel
"Large Image Datasets: A Pyrrhic Win for Computer Vision?"
by
@vinayprabhu
@Abebab
Unexpected and shocking (as a junior researcher), and highly educational!
1
6
20
Wow, the volume of ML papers has increased so much that two very unrelated works posted on arxiv 3 days apart have an acronym collision!
SEER () and SEER ()!! 😆
Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings
SEER saves significant computation and memory across DeepMind Control environments and Atari games without degrading the performance.
2
2
29
0
0
21
During the pandemic, WFH and social isolation led me to working seven days a week — it blew up to exhaustion + burnout. At the start of 2022, I decided to have "no code Saturdays". Today
@DandanShan_
saw my Github and reminded me of it. 2022 is 66% over, progress so far >> (1/4)
New year resolution: no code on Saturdays.
It roughly translates to "take Saturdays off", but n(lines of code) is a concrete measure. This metric doesn't account for time spent reading/reviewing papers etc. but I would be happy if I at least take this much time off.
2
0
21
2
0
20
I recently noticed that our recent dataset RedCaps () has an updated dataset page on Hugging Face (). It also has a nice dataset explorer!
All credits to
@MarioSasko
, thanks for making our work more accessible
@huggingface
! 🤗
0
4
20
@hardmaru
@edersantana
Probably people haven't started using it because they are still reading the appendix.
0
2
17
.
@Abebab
et al. have done the much needed grunt work to uncover problematic trends in the data distribution of LAION (e.g. ) and ImageNet (e.g. ). I am shocked to see that (1) Stanford study missed these citations, and (2) contd. >>
me & my collaborators have done the most extensive research of the LAION datasets (3 academic papers & the first to investigate dataset in 2021 showing misogyny, pornography, & malignant stereotypes)
yet, the Stanford study has not cited us once. this is academic misconduct
18
457
1K
1
3
18
Playing with — predicting captions for images of landmarks by prompting "r/absoluteunits" (a subreddit with photos of large objects, often pets and fruits/veggies — )
1
3
17
*chuckles in natural language*
At long last, the truth is out. I’ve always said “language is all you need” — it sounds even better when admitted by Computer Vision researchers 🙃 (very cool work, btw)
0
12
87
0
0
17
... I recently found a better explanation for how to say my name — it sounds like "car run" (🚗🏃♂️). It is much more accurate than "kæ - ren". It's funny how I got this from a
@Starbucks
name typo! I felt like shouting "Eureka" 😆
2
0
16
Thank you
@CShorten30
for featuring our latest work CAST, about making contrastive self-supervised models work on complex scene images!
Paper:
Blog:
New AI Weekly Update - December 14th, 2020 (
#24
)!
- Francois Chollet
@fchollet
on "Abstraction & Reasoning" at NeurIPS 2020
- Neurosymbolic AI: The 3rd Wave
- CASTing your Model
- Distilling Reader to Retriever
- and more!
#100DaysOfMLCode
1
22
82
1
3
16
Hilarious conversation with a cute little chatbot from my lab :)
More at
1
3
16
@kamalgupta09
The second rule of Computer Vision club - if it works on ImageNet, it doesn't mean it would work on everything else.
#computervision
#phdlife
😄
1
0
15
It's always great to see friends' papers land on arXiv and then in AK's tweets! This one looks solid
@julberner
! 🎉
0
1
15
I am super excited that
@OpenAI
is exploring the multimodal direction and discover how far we push the ability to learn vision from language supervision in massive data+compute scenarios! Very excited by CLIP — thanks for the shoutout to (my recent work) VirTex! >>
We’ve developed two neural networks which have learned by associating text and images. CLIP maps images into categories described in text, and DALL-E creates new images, like this, from text.
A step toward systems with deeper understanding of the world.
99
951
3K
1
0
15
@jcjohnss
@jasonbaldridge
@andrewhowens
@michigan_AI
@UMichCSE
I have a lot of humans to thank, here are some of my defense slides (see more in next tweet) — thank you, everyone! ❤️
1
0
15
@mmitchell_ai
@timnitGebru
@emilymbender
@mcmillan_majora
Happy birthday to 🦜 and Shmargaret Shmitchell!
1
0
13
>> Good for them. I didn't have a great time and they all feel nauseating to me. I moved 🇺🇲➡️🇮🇳 to stay with family and things are better for me now. If they are not for you (reader), remember that nobody but you, decide how *you* deal with a pandemic. All things must pass. 2/2
1
0
13
Nowadays, slightly smiling face is almost never used for what it was meant to be. 🙂
Upside down face is better than slightly smiling face for what slightly smiling face is actually used nowadays. 🙃
1
0
13
(2) Authors of (
@Abebab
,
@vinayprabhu
) — this paper appeared on arxiv ~1 month after I started working on this project. The paper uncovers problematic trends in large image datasets, which I would not have accounted for, had I not read this paper. >> 10/11
1
3
13
Thanks to all the collaborators at
@GeorgiaTech
: (
@stefmlee
,
@panderson_me
,
@harsh_092
,
@ayshrv
,
@rishabhjain2018
,
@abhshkdz
),
@facebookai
: (
@rama_vedantam
,
@endernewton
,
@marcus_rohrbach
) and
@Macquarie_Uni
: (
@GaryyufeiWang
, Mark Johnson), and all the members of CVMLP!
2
0
12
I have observed that it is often hard for Americans to pronounce my name "Karan", and it ends up sounding more like "Karen" (first 'a' gets an æ sound). I generally go by my initials "KD" because it's convenient and hard to mess up. While I don't mind any mispronunciations, (1/2)
4
0
12
I am a middle aged white lady part-time.
1
0
11
@jcjohnss
@jasonbaldridge
@andrewhowens
@michigan_AI
@UMichCSE
(... continued) more wonderful humans in my journey. Thank you, everyone!
0
0
11
Contd: *le ECCV Post-rebuttal comments
R1: "I'd urge the authors to compare more with methods that do <what we do>"
There are exactly ZERO more methods so far published/on arxiv *TILL DATE* which do <what we do>.
So novelty is a weakness? 🤔
Unfortunately drowned in the noise.
*le ECCV Reviews
R1: Recent method [X] outperforms your proposed approach on the dataset [Y].
ECCV deadline: 5 March
[X] came out on arxiv: 9 March
Sure R1, let me hop on tardis.
(╯°□°）╯︵ ┻━┻
3
2
39
0
0
11
Been a while since I have been using
@ai2_allennlp
to its full utilization in my source code. It's a beautiful piece of work with neat API and docs. They do "take masking and padding seriously" — there's not a single `pack_padded_sequence` in my code now! Shoutout to the team!
1
0
11
Successfully completed GSoC 2017 with
@tinydnn
@opencvlibrary
! Overwhelmed, it was a joyous ride!
#gsoc
0
0
11
Wow,
@TensorFlow
is a mentoring org for GSoC 2019!
Want to make
@TensorFlow
better and get mentored by experts and paid a stipend to do so?
We now have TensorFlow set up as a mentoring organization for Google Summer of Code (
@gsoc
). A great opportunity to learn, get paid, while making TensorFlow better in myriad ways!
6
189
566
0
0
10
It may be an unpopular opinion, but food for thought: if Github Co-pilot becomes popular, it will make an entire generation of developers heavily dependent on it, and bring down the average competence of the next gen of developers. (1/2)
4
0
8
My first contrib in
#tensorflow
, adding 3D deconvolution layer! Make a 3D deconvnet for spatiotemporal data easily!
1
0
9
@SameerDharur
+ appearing for PhD quals
+ preparing for poster/video for a different conference altogether
+ doing an internship
My goal for this month is to only remain sane 😭😭
1
0
9
(Last of my retweeting spree)
I primarily use Twitter for following AI research, but today I am off-limits. This IS a global crisis. Mask up, keep your sanitizer handy, and get vaccinated ASAP!
Thank you
@sibinmohan
for putting all of this together. 🙏
In the end, I would say: at least raise awareness of the issue! This is not another “not my problem” situations. COVID raging this badly in one of the world’s most populous nations will find its way to affect *everyone*, Americans included!
Thanks for your help!
16/17
7
127
895
1
1
9
Conversational flavor of data: RedCaps data is created with a specific intent of human interaction on social media. Reddit users have an incentive (upvotes) to upload high-quality data — sometimes witty or emotional, unlike HTML alt-text. 2/8
1
1
9
messed up with my Ubuntu last night. Never give display control to NVidia while you are tinkering with tensorflow source.
1
0
9
I like this idea, may we come across other logs and sone logs make us go "we could do ours better!"
Also my first
#CVPR2021
as a (not-just-emergency) reviewer!
Disclaimer: Will round-off counts to avoid any bizarre break of double blind reviews. (1/n) 🧵
1
1
9
TIL: In `if [ cond ]; then` of bash, `[` is a command and `]` is one of its args. Running `which [` gave me `/usr/bin/[`
WTF?!
@LucyDavinhart
Always threw me for ages before I realised that '[' is just a program (technically usually a built-in, but whatever) that requires its last argument to be ']' 😆
14
26
148
1
0
9
I have four "code" Saturdays to spend in case of emergency (e.g. conf deadlines). If I use all days and miss a deadline, I will face the consequences. Will share my Github commit activity at the end of 2022!
Saying this 📢 on Twitter now I am committed! (pun intended)
1
0
7
*Finally* a stable release. Thanks for acknowledging my contribution
@tensorflow
@GoogleBrain
Time to make a new contrib.
TensorFlow 1.2.0 has been released! Please see the full release notes for details on added features and changes:
7
247
372
0
0
8
Dataset available at !
Pre-trained models coming soon.
(w/
@gauravkaul7
,
@zubinaysola
,
@jcjohnss
)
To appear at NeurIPS 2021 datasets and benchmarks.
8/8 Fin.
1
0
8
This thread resonates so much with me (Mar-Nov 2020) and a lot of conversations I have had recently with many friends/colleagues
#AcademicTwitter
. More to add: I see countless *quarantine productivity, positivity* blogs and tweets by random folks. -- 1/2 >>
So, my take on wfh vs wfo: I've been pretty much stuck at home in the past 10 months. During that time I took no vacation (because covid) not counting two months of paternity leave (hardly a vacation). I spent 14-16 hours a day in front of a computer except for meals.
2
0
18
2
0
7
Dataset size: One of the largest public image-text datasets in 2021! RedCaps contains data from the past 13 years (2008–2020). However, it is not static by design! It will continue to grow in the future as more data gets uploaded to Reddit. 4/8
1
0
7
@CSProfKGD
"Barf" means ice in Hindi! If I were an author, I would have put an ice cube and confused a lot of people😆
1
0
7
I wonder why releasing and sharing Dockerfiles is not so prevalent yet to accelerate reproducibility in Deep Learning. People spend a lot of time to get the right libraries and drivers along with other dependencies of the code to reproduce a paper implementation.
2
1
7
(1,2) have affected our design choices for the better. We avoided subreddits having lot of people images, filtered data using NSFW/face detectors, and added a form on for anyone to request image removal from RedCaps. Not perfect, but a positive step. 11/11
0
0
7