Delip Rao e/σ
@deliprao
Followers
46,493
Following
4,915
Media
4,629
Statuses
50,736
Busy inventing the shipwreck. @Penn . Past: @johnshopkins , @UCSC , @Amazon , @Twitter ||Art: #NLProc , Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
NYC, 🇺🇸🇮🇳🇹🇼🏳️🌈
Joined October 2008
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
オーロラ
• 411486 Tweets
Joost
• 306479 Tweets
CHAWARIN ON STAGE
• 225230 Tweets
FOURTH SINGING TIME
• 139169 Tweets
#ขวัญฤทัยตอนจบ
• 128335 Tweets
#不破湊3Dライブ
• 85092 Tweets
Fulham
• 71741 Tweets
自分これ
• 59703 Tweets
Man City
• 55635 Tweets
Gvardiol
• 42174 Tweets
Manchester City
• 36446 Tweets
最高のライブ
• 35754 Tweets
#العين_يوكوهاما
• 35470 Tweets
太陽フレアのせい
• 33419 Tweets
カクレンジャー
• 32690 Tweets
HeavenlyVoice WithMrC
• 31920 Tweets
#東京タワー
• 30280 Tweets
マリノス
• 27156 Tweets
Dremo
• 24672 Tweets
アルバムとライブ
• 20999 Tweets
#FULMCI
• 20446 Tweets
自分の魚
• 15953 Tweets
わたしの誕生魚
• 15074 Tweets
Sark
• 12973 Tweets
GET WELL SOON JISUNG
• 10136 Tweets
Pinned Tweet
I guess we’re doing this:
1
3
24
At 2100+ (not a typo) pages 🤯, this is almost all the Math you need for Machine Learning without dumbing it down! PDF link:
169
2K
11K
NEWS: Apple just entered the AI open source arena by quietly releasing their new DL framework called MLX! It runs code natively on Apple Silicon with a single pip install and no other dependencies.
Sharing what I discovered from this initial release:
105
993
7K
Label the green and purple ovals.
Can you predict IQ based on grades? One meta-analysis suggests the answer is yes. It found that the correlation between IQ and grades was r = 0.54. That means a scatterplot of the data would look like this:
234
111
1K
987
248
5K
How much of math is “scary” because people are primed to think that way? I taught some really advanced math concepts to my pre-teen nephew like we were learning the next letter of the English alphabet and it went swimmingly.
116
465
4K
When Linus tried GPT-4 for coding for the first and only time:
50
207
3K
I have been testing mistral-medium & GPT-4’s code generation abilities for non-trivial problems. These are problems even experience engineers will take time to work it out. I am summarizing some examples and overall impression in this thread: 🧶
97
313
3K
We finally have a word for people who are experts in AI, immunology, and Afghanistan all at once.
102
810
2K
People who believe in this graph have not actually spent significant time coding (with and without copilot).
the most important graph in software rn. perhaps the most important graph for human productivity in decades.
127
314
2K
121
125
2K
All coding projects have two parts:
1. The fun part: where you get to "create"
2. The pain part: where you have to debug
Code LLMs are "automating" the fun parts while introducing bugs and not helping much with debugging. As a developer, you’re left with more pain to deal with.
147
249
2K
China using AI models of 3D spatial data to give feedback to their athletes. Not sure if this is the reason for their victories, but when you are competing at those levels, every bit makes a huge difference!
48
529
2K
Would’ve been cool if the dx was not under the radical. Maybe the password is “nonsense”?
77
104
2K
This is huge! Now watch the LLM API costs dropping even further.
[.cn PDF link]
45
229
2K
Crazy AF. Paper studies
@_akhaliq
and
@arankomatsuzaki
paper tweets and finds those papers get 2-3x higher citation counts than control.
They are now influencers 😄 Whether you like it or not, the TikTokification of academia is here!
65
284
2K
India's lunar mission cost approximately $90 million*, considerably less than what some AI startups have raised as "seed" rounds. Let that sink in.
52
115
2K
the success of ChatGPT has lead to a cottage industry of thoughtbois making vacuous hype threads
45
63
1K
Drop everything you are doing!!
Alex Graves pushed a paper on arXiv, so nothing could be more important than reading it. First thing I did was go look for any comments in the TeX file. Unfortunately, it’s all been scrubbed.
24
162
1K
Interesting story from the interwebs. 26yo Stephen Wolfram was unhappy at being treated badly at the IAS. He wanted to start his own institute to continue his research and asked his ex-colleague, Feynman, for advice. Feynman’s reply:
21
218
1K
One way to disclose you never studied physics seriously is to recommend “Feynman’s lectures” as a good physics book. The real deal know it is and it will always be Landau-Lifshitz.
115
78
1K
OAI will discontinue support for Codex models starting March 26. And just like that, all papers and ideas built atop codex (> 200 on ArXiv) will not be replicable or usable as is. Do we still think openness doesn’t matter?
48
222
1K
🚨 BREAKING: Introducing a new language model with ZERO hallucinations!
Best of all, it’s easy to implement, easy to deploy, safe, and low resource!
78
68
1K
@LongFormMath
I think it implies you’ve cultivated a good level of safety with the students for them to be comfortable sharing this with you.
2
4
1K
What current students think of AI literature published before 2018.
37
107
1K
What an absurd line of reasoning here. I think he left out Nvidia for providing the GPUs, the power companies for supplying power, and the construction company for building the data center.
103
48
1K
When you understand the technique/algorithm/modeling, but have no clue about the domain.
24
140
1K
imagine replying “ship something better” to the person who leads Pixel’s computational photography ("AI" if you prefer) that's raising the bar of what to expect from a camera.
@docmilanfar
Sam is crushing you guys at AI. Ship something better and then you can make fun of him.
59
130
5K
37
46
996
LaTeX is amazing and fun, but it hit me today why I love it. It’s productive procrastination. As you work on your document, you feel like you are making progress but it’s not the progress you want and you’re okay with that as you have no idea about the other kind of progress.
24
79
970
I have long maintained LLMs make the poor performers mediocre, the average slightly above average, but do not change, and maybe hinder, the performance of top performers.
Here’s a result from a university-level physics coding task.
41
157
934
History class: This 2015 paper is the mother of all LM based pre-training approaches, including the GPT, but few are aware of it. GPT (Radford et al 2018) was a direct application of Transformers (Vasvani et al 2017) to the result in this paper (w/ lot of work & insight ofc).
13
125
931
one small patching from Intel, and all your scikit-learn operations will run anywhere between 10-100x faster. Just make sure to import and run before any scikit imports. Details:
8
136
913
In generative deep learning, if your model works it’s a “product”. If it doesn’t, it’s “art”.
9
81
891
First time realizing Good Will Hunting whiteboard stuff was graph theory 101 homework problems.
9
25
861
Startups are hiring influencers to peddle bullshit. This is a new kind of misinformation to peddle snake oil AI products with no backing. When you dig in, you see the growth rate has been hovering around 8.5%, with leading and lagging indicators for March showing a -16% drop.
I just heard that Perplexity is growing at 40% per month.
If they keep it up they’re going to start making a real dent by the end of the year.
23
11
146
57
60
860
Regulatory capture if you see one. Yuck.
Also, great way for America to shoot itself in its foot and stifle innovation.
79
85
837
Reminder: this was (part of) the team that thought GPT-2 was too dangerous to release, and now they are making models stronger than GPT-4 available on AWS for anyone with an Amazon account to use.
This is why I have little trust in “AI safety” claims by Anthropic/OpenAI. It all…
Congrats to Dario and the
@AnthropicAI
team on their new Claude 3 family of models. Very impressive benchmarks, and excited to have all of them coming to Amazon Bedrock (w/ Sonnet avail today). Many AWS customers are already building with Anthropic’s foundation models, and…
20
77
697
27
82
827
“Affordable RLHF for all” ❤️
It’s almost like an openly rebellious group at MSR have decided to subvert Microsoft’s investments in ClosedAI.
14
129
792
PSA: Please don’t do this in your code for _any_ keys, especially if you use 3rdparty libraries.
I just discovered some 3rd party code exporting all environment vars for telemetry and debugging. The increasing adoption of DevOps tools/libs can lead to key leaks with this code.
33
75
781
I cannot understand why folks are selling $GOOG because of the Bard “demo mistake”. ChatGPT and other equivalent models make similar mistakes a million times a day. A quick day trade here would be buy & hold GOOG until it bounces back in a couple days once this hysteria is over.
60
26
742
Grok code is now open source, but no weights.
39
75
745
So let me get this straight. It’s dangerous to have powerful open source models but okay to make the similar or more powerful models available via AWS?
48
48
726
Meanwhile, Sergey is out talking about debugging Gemini looking like someone who actually spent the night debugging. Google rumors are overstated.
Zuck’s personal stylist and PR team are doing the lord’s work. Give these people a raise.
25
28
675
19
39
704
The Untold History of Deep Learning is yet to be written.
After reading this Mikolov post, I will never again teach the Seq2Seq paper the same way.
Also, doubt people and organizations when they make proclamations about working for the benefit of humanity.…
19
101
702
If this were a science paper, you would expect a country that picks its science workforce at random as a “weak baseline” and a leading nation like the US to actively experiment towards state-of-the-art, or at least beat the baseline.
Not providing a guaranteed path for…
H1B lottery ❌
It was less than a 1 in 3 chance, but sucks anyway!
116
47
1K
34
123
686
OpenAI released their ChatGPT API today. Here’s a deep dive:
1. It’s not only a new model, but a new endpoint. Notice the model name says, “gpt-3.5-turbo”.
Turbo model is something the paid ChatGPT users (“PLUS”) got a preview a week or so ago.
16
101
672
that urge to drop everything and go do a math PhD on an esoteric topic in a quiet and beautiful countryside campus.
10
41
665
ChatGPT pricing between $0 and $42, there exists a power law of customers willing to pay that. That’s a big gap this pricing creates. The bigger this gap, the more incentive for competitors (they exist) to accelerate their efforts to fill it, and OpenAI will be forced to reprice.
65
67
645
This is another one of those ill-thought, fear-mongering scientific disinformation about LLMs, and I will explain why in this long thread. 🧶
I flip-flop on how bad releasing model weights is, but what is clear to me is that we're in a honeymoon period before something bad happens like mass social manipulation and surely Meta is gonna regret making "we let anyone use our great models for anything" a selling point.
60
12
169
6
158
649
arXiv: "The World as a Neural Network -- We discuss a possibility that the entire universe on its most fundamental level is a neural network."
Me: Yeah, and 2020 is the year of NaNs
26
129
625
I absolutely love this idea of putting negative results in gray. This is a gift you can give to readers for the low effort of throwing in a few annotations while writing. Also, the gray with black is not jarring at all.
17
68
588
From
@chris_j_paxton
. Apparently OpenAI is hitting content farms hard.
This is why being open about what is going into your models is so important.
17
61
618
Where my Indian friends at!? I have missed all the sweet nepo deals! Also where do I go to collect my minority perks?? Feeling livid I missed out on whatever memos my Indian friends are sending themselves while I’ve been slugging it out for the past 15+ years and paying fat taxes
Indians leave their country of 1.38 billion, come to America/the west and claim minority status to get all the perks. Then they practice nepotism and get ahead on the shoulders of Europeans while we get attacked. Isn't 'diversity' great?
2K
660
4K
23
36
605
If you see people flaunting ⏸️ or ⏹️ emojis in their profile, keep in mind that this is the level of their understanding of AI, Biology, and the interaction of the two.
79
45
608
at this point we have moved away from science to a d*ck measuring contest.
42
36
602
I’ve tested many codegen models and GPT-4 was a clear winner until now. Congratulations to
@MistralAI
for bringing in mistral-medium as a strong competitor for code generation tasks.
4
20
585
Closed science companies like OpenAI and Anthropic parasitically extract value from open science and open source without giving credit to people or organizations building them. Open science with citations would’ve addressed that, but alas that’s too much to ask.
Pace of progress in AI is lightning!
@OpenAI
released MRL style text embeddings, mirroring our NeurIPS '22 paper (w/ awesome folks from UW and Harvard).
However, as an advocate of open science, I am a bit disappointed with rebranding to "shortening embs" without ref to MRL 1/n
11
100
660
25
64
563
Finally, a tree all computer science folks can relate to.
Upside-down fig tree in Bacoli, Italy. "No one is quite sure how the tree ended up there or how it survived, but year after year it continues to grow downwards and bear figs."
#archaeohistories
319
6K
38K
4
78
558
Mistral-Medium: “Write cuda-optimized code for generating a PyTorch Dataset of Fibonacci primes”
Verdict: No-nonsense, full code, ✅
13
30
559
This is like insisting on baby-proofing every power tool.
If anything, this is an endorsement of how versatile and unlobotomized the mistral model is as a *base model*.
If you are capable of making an inference on an LLM, it is your responsibility to use it safely.
After spending just 20 minutes with the
@MistralAI
model, I am shocked by how unsafe it is. It is very rare these days to see a new model so readily reply to even the most malicious instructions. I am super excited about open-source LLMs, but this can't be it!
Examples below 🧵
217
107
768
22
42
542
I love my partner to death, but dear god, I made the mistake of peeking at some simulation source code he was writing. Never look at source code written by Physicists.
19
19
542
Graduate school hack: become somebody’s first PhD student. Not joking.
7
20
541
I cannot wait to share my deepest secrets with my toaster.
Also, Apple ML research team is putting out bangers after another. 😽
14
52
537
LLM-based company is using Upwork to hire Ph.D.-level Physicists to RLHF-train their models.
23
48
538
I have a very niche use of ChatGPT -- knock out some code, stick it in ChatGPT, and ask it to generate Python docstrings. I want to write docstrings, but I'm too lazy. ChatGPT is very good at understanding code and summarizing it, and docstring generation is a subset of that.
22
24
534
Turns out you cannot replace decades of painstaking work optimizing around every kind of web content with an LLM and retrieval augmentation.
Who could’ve guessed. 🙃
Perplexity only beats Google at answering questions. for general search (e.g. looking up restaurants, movies, celebrities etc) Google is still unparalleled
i also much prefer Google’s instant results for quick searches
16
12
481
30
31
524
True story. Startups pay high salaries to hire competent employees, and disempower them by not taking their suggestions. Then they invite consultants like me, who sometimes end up giving similar suggestions and get paid for it. Founders, start trusting the people you hired!
10
54
515
Looking at TIME 100 for AI, it appears like list of AI “influencers” with Hinton and a few other actual AI research folks randomly thrown in to give it legitimacy?
37
29
514
Don’t join a friend’s startup. It will almost always ruin friendships.
Instead, join a startup you like and make a new friend. Never fails.
14
17
502
Ha! This job title doesn’t mince words on what machine learning research has become.
15
34
486
Price for 26 minutes transcription:
Humans: $39 to $130 (*)
Rev AI: $6.5 (closed model)
Whisper on Mac: pennies
Open Source AI will upset so many company business models, but it will enable more company business models and, most importantly the individuals, leaning on it to…
WhisperKit-v0.6.0 dropped yesterday!
In the demo, 200 audio files (~26 minutes) are transcribed in ~13 seconds on an M2 Ultra Mac using whisper-base. WhisperKit harnesses all available compute, roughly 60 TFlops on Mac (GPU + 2xANE). This release is our first step towards…
13
75
516
9
39
483
What’s the difference between CBOW and Skip-gram, in terms of performance? Turns out nothing if you fix bugs in popular implementations. I love this genre of papers.
8
58
477
I am absolutely in love with this mouse-over stuff
@huggingface
API documentations have. It clearly shows they understand their users' pain -- ML APIs have too many arguments. If you are wondering why 🤗 is so popular, little things like these go a long way in reducing friction.
1
40
478
Anyone underestimating the power of autoregressive models has not fully figured out autoregressive models. Even people actively building autoregressive regressive models have not figured them out fully either.
Hat-tip
@natfriedman
16
37
474