agibsonccc
@agibsonccc
Followers
3K
Following
14K
Media
177
Statuses
10K
Founder @KonduitAI Maintainer Eclipse Deeplearning4j Building https://t.co/GDg4JPHYso - #MLOps YC W16
Tokyo-to, Japan
Joined April 2011
This is crazy! Especially for a 580M parameter model. The benchmark scores are not SOTA but it's not fair to compare it with models that are 5-10x larger. This is a great exploration on multi-purpose OCR model. GOT-OCR paper: https://t.co/BdACe8TTlF
4
20
175
We are comparing a low level execution runtime, a python framework people use, and an llm..this is deranged. These things don't even compete.
In 2012 CUDA was very important. You can't build anything without it. In 2024 90% of AI developers are actually web developers – and they build off Llama, not CUDA.
0
0
6
I used to be in the "benefit of the doubt" camp, but can no longer -- so lemme be clear: this is quite simply a lie. Those pushing SB1047 *know* that nearly all open source AI devs rely on fine-tuned models, which *are* blocked by this bill.
Great article by @Priyasideas on increasing opposition to SB 1047. https://t.co/G7K4mkYaUu
16
76
470
Why do 16k GPU jobs fail? The Llama3 paper has many cool details -- but notably, has a huge infrastructure section that covers how we parallelize, keep things reliable, etc. We hit an overall 90% effective-training-time. https://t.co/hsSIW4bayK
32
205
1K
You should probably stop whatever you're doing and watch this right now, because it's amazing. https://t.co/Tox5T8vQFh
55
285
1K
Lot of absurd takes like this on the superalignment team leaving OpenAI. The more likely reason they left is not because Ilya and Jan saw some super advanced AI emerging that they couldn't handle but that they didn't and as the cognitive dissonance hit, OpenAI and other
303
388
3K
There's a new bill, SB-1047 "Safe and Secure Innovation for Frontier Artificial Intelligence Models Act". I think it could do a great deal of harm to startups, American innovation, open source, and safety. So I've written a response to the authors: 🧵 https://t.co/1aUrHnAjrF
answer.ai
By imposing the restrictions on open-source AI, SB-1047 could reduce AI safety, through reducing transparency, collaboration, diversity, and resilience.
35
280
1K
Tokyo will become a Global R&D Powerhouse in AI.
Open AI is planning to open its first Asia based office in Tokyo. ..following the path of @SakanaAILabs? https://t.co/wL16Qy2oDf
20
59
550
One of the simplest but most useful and appropriate pieces of AI regulation to adopt at the moment is to require model providers to document the training data they used. This is something that the @EU_Commission AI Act gets right … on p.62 of its 272 pages (!).
So when *the CTO* of OpenAI is asked if Sora was trained on YouTube videos, she says “actually I’m not sure” and refuses to discuss all further questions about the training data. Either a rather stunning level of ignorance of her own product, or a lie—pretty damning either way!
12
59
324
This is likely going to become **the** Reference for developing Multimodal LLMs (among other things). Great resource, and great work from a long list of awesome people!
New Resource: Foundation Model Development Cheatsheet for best practices We compiled 250+ resources & tools for: 🔭 sourcing data 🔍 documenting & audits 🌴 environmental impact ☢️ risks & harms eval 🌍 release & monitoring With experts from @AiEleuther, @allen_ai,
2
53
351
One reason AI research has been progressing so fast is not just because of frequent and early publication of preprint on ArXiv and the exchange of open source code, but also because the ML/AI community has largely freed itself from the stranglehold of for-profit & paywalled
@pmarca Thankfully, ML/AI research is largely free of the commercial publishing stranglehold. Preprints are posted on ArXiv and OpenReview, short papers in top conferences like ICLR, NeurIPS, ICML, and a few others, and longer papers in JMLR and TMLR. All of these venues are open access
104
364
2K
Not your model, not your GPTs. All those folks rushing to add stuff to the GPT store are writing free functions for another OpenAI llm that they will brand as “AGI”. Almost free labor extraction. I will bet, for most folks, the revenue share will be pennies. Folks who think
28
45
357
🙋: Why didn't authors collaborate w/ LAION? The approach LAION took to filtering is not standard and can not be considered serious or professional in any way. Many actions they took in this unusual process fall under the Criminal Code. This makes collaboration... suboptimal.
1
3
52
LlamaGpt6 assimilated Satya and took over Microsoft.
I just finished a two-day company quarterly strategy meeting. I haven’t missed anything, have I? Satya is still CEO at MSFT, right?
0
0
2
TIL: @GoogleAI's 1.6T parameter mixture-of-experts encoder-decoder model is available under an Apache 2.0 license! Trained on public data too.
12
65
487
I sent this to a reporter in response to a query yesterday when things didn't look this crazy. Now that it is clear the future of OpenAI is uncertain, we should encourage all companies to build on resilient AI technology that only open source can offer.
5
12
80
“But but OpenAI models are so much better” - try fine tuning some open source models first like this
I burned in🔥2000$ in finetuning so you don't have to. I fine-tuned models with @OpenAI and @anyscalecompute API endpoints with 50million tokens. Here are the results I wish I knew before getting into finetuning. If you just want a quick snapshot, look at the figure. A longer
8
34
288
Attending @geoffreyhinton’s retirement celebration at Google with old friends. Thank you for everything you’ve done for AI! @JeffDean @quocleix
67
243
4K