After ~4 years building SOTA models & datasets, we're sharing everything we learned in ⚡The Smol Training Playbook We cover the full LLM cycle: designing ablations, choosing an architecture, curating data, post-training, and building solid infrastructure. We'll help you

timm's got a new vision transformer (NaFlexVit), and it's flexible! I've been plugging away at this for a bit, integrating ideas from FlexiViT, NaViT, and NaFlex and finally ready to merge for initial exploration. The model supports: * variable aspect/size images of NaFlex (see

With great help by @gpuccetti92, @tommiekerssies and @rom1504 . More to come.

When all of the sudden puzzle pieces fall into right places and predicting the unknown starts to work, those are rare beautiful moments I am grateful for in science. Made by rare minds of @marnezhurina @tomerporian @mehdidc https://t.co/DiuR6g8pil https://t.co/snYXA7AcRg

Very excited to finally release our paper for OpenThoughts! After DataComp and DCLM, this is the third large open dataset my group has been building in collaboration with the DataComp community. This time, the focus is on post-training, specifically reasoning data.

Introducing π-0.5! The model works out of the box in completely new environments. Here the robot cleans new kitchens & bedrooms. 🤖 Detailed paper + videos in more than 10 unseen rooms: https://t.co/64ze7ToQup A short thread 🧵

🚀New Paper! https://t.co/cZZGbeVrgR Everyone’s celebrating rapid progress in math reasoning with RL/SFT. But how real is this progress? We re-evaluated recently released popular reasoning models—and found reported gains often vanish under rigorous testing!! 👀 🧵👇

After 6+ months in the making and burning over a year of GPU compute time, we're super excited to finally release the "Ultra-Scale Playbook" Check it out here: https://t.co/mnC0UzZYsJ A free, open-source, book to learn everything about 5D parallelism, ZeRO, fast CUDA kernels,

Finally took time to go over Dario's essay on DeepSeek and export control and to be honest it was quite painful to read. And I say this as a great admirer of Anthropic and big user of Claude* The first half of the essay reads like a lengthy attempt to justify that closed-source

My thoughts on DeepSeek sent to a reporter asking for comments

Directed by David Lynch.

Director-writer David Lynch, who radicalized American film with with a dark, surrealistic artistic vision in films like “Blue Velvet” and “Mulholland Drive” and network television with “Twin Peaks,” has died. He was 78. https://t.co/T2GOao28ux

The data science revolution is getting closer. TabPFN v2 is published in Nature: https://t.co/Ybb15pnZ5P On tabular classification with up to 10k data points & 500 features, in 2.8s TabPFN on average outperforms all other methods, even when tuning them for up to 4 hours🧵1/19

Introducing PaperQA2, the first AI agent that conducts entire scientific literature reviews on its own. PaperQA2 is also the first agent to beat PhD and Postdoc-level biology researchers on multiple literature research tasks, as measured both by accuracy on objective benchmarks

Ever feel frustrated when you vaguely know what paper you want to cite but can't find it on Google? Can LM-based agents automatically find paper citations for you? Our new paper presents a tough new benchmark for this task along with an LM-based agent for finding citations.

🧵1/8 We resolve the discrepancy between the compute optimal scaling laws of Kaplan (exponent 0.88, Figure 14, left) et al. and Hoffmann et al. (“Chinchilla”, exponent 0.5). Paper: https://t.co/QKFbNl4J9t Data + Code: https://t.co/N3p0Xg0THH

Do you want to improve the performance of your text-to-image model without any training? That too by just looking for a better initialization noise? Sounds too good to be true? 🧵👇 https://t.co/b8jbASFiXE

"fly through tour of a museum with many paintings and sculptures and beautiful works of art in all styles" Video generated by #Sora

Beluga whales love to play, scare, joke and generally interact with humans. This compilation is a good example. [📹 aquariumadvicesa] https://t.co/P2ifmVPKqm

For those of you who were wondering what I’ve been doing since I joined @Meta Reality Labs late 2022. Here is the first detailed scientific communication about our work. You can read the paper at: