We've just published the Smol Training Playbook: a distillation of hard earned knowledge to share exactly what it takes to train SOTA LLMs ⚡️ Featuring our protagonist SmolLM3, we cover: 🧭 Strategy on whether to train your own LLM and burn all your VC money 🪨 Pretraining,

We’re releasing pre-anneal checkpoints for our Nano/Mini base models. Still plenty of math + code exposure, but easier to CPT and customize than our post-anneal checkpoints. Have fun exploring.

Big vision? Big startup? Big domain. 💥 The right name: Sparks excitement Draws customers in Powers your growth Find the domain that fuels your idea.

Rnj-1-Instruct is now the #1 trending text generation model on HF!

Sharing the slides from a talk I gave this week on bridging the gap between research experiments and building production-ready models, based on our recent Smol Training Playbook. https://t.co/RmG53PytMv

HUGE notebook for doing reinforcement learning on agents with TRL 0.26.0 and GRPO. In this notebook: - set up an internal knowledge base agent with postgres for managing hotel bookings, and define python function tools. - I defined a simple dataset for use prompts like "book a

Much like Woj — the man who brought him into the game — @ShamsCharania is the guy you look to verify the story when you see it breaking on social media. If it doesn’t come directly from him, you rush to his accounts to see if what you just read is real or fake news. Every

I'm looking to hire someone very strong in post-training with solid OSS experience. If you think you’d be a good fit, DM me.

We are now the #1 trending text-gen <256B size model on HuggingFace!!

My new blog post discusses the physical reality of computation and why this means we will not see AGI or any meaningful superintelligence:

We just released TRL v0.26.0! It comes packed with updates: > Agent training with tools in GRPO > New CISPO & SAPO losses > Reasoning rewards > vLLM quantization in colocate mode > Dataset shuffling in SFT > Lots of NEW examples > Tons of fixes and documentation improvements

if the agents are so good why are you mfs in the office til 9 still

Today we open source Nomos 1. At just 30B parameters, it scores 87/120 on this year’s Putnam, one of the world’s most prestigious math competitions. This score would rank #2/3988 in 2024 and marks our first step with @hillclimbai towards creating a SOTA AI mathematician.

1/5 🚀Apriel-1.6-15B-Thinker: a 15B multimodal reasoner scoring 57 on the Artificial Analysis Intelligence Index - approaching the performance of ~200B-scale frontier models while remaining an order of magnitude smaller. 🧠Model weights: https://t.co/GE22SOIBfT 📄Blog:

OpenAI leadership (@gdb, @markchen90) are promoting a paper in Physics Letters B where GPT-5 proposed the main idea — possibly the first peer-reviewed paper where an LLM generated the core contribution. One small problem: GPT-5's idea tests the wrong thing. 1/

Glass half empty or glass half full?

We used Claude Code to train open LLMs. Check out the tutorial. basically, we plugged HF skills into claude code and it was able to train LLMs end-to-end. Best thing, this works on all major coding agents: Codex, Cursor, and Gemini CLI. - You tell the agent to fine-tune a model

The HuggingFace team just got Claude Code to fully train an open LLM. You just say something like: “Fine-tune Qwen3-0.6B on open-r1/codeforces-cots.” Claude handles the rest. ▸ Picks the best cloud GPU based on model size ▸ Loads dataset (or searches if not specified) ▸

Putnam, the world's hardest college-level math test, ended yesterday 4p PT. Noon today, AxiomProver solved 9/12 problems in Lean autonomously (3:58p PT yesterday, it was 8/12). Our score would've been #1 of ~4000 participants last year and Putnam Fellow (top 5) in recent years

The obvious answer is open source AI 🤗

Also a nice reminder to start your post-training experiments with SFT before reaching for RL :) https://t.co/Sa1kkYqLlj

Nice easter egg in the Rnj-1 release: post-training was just SFT! This makes the SWE-Bench scores even more impressive, since that's a benchmark where recovering from errors is crucial and typically hard to instil with SFT alone

Today, we’re excited to introduce Rnj-1, @essential_ai's first open model; a world-class 8B base + instruct pair, built with scientific rigor, intentional design, and a belief that the advancement and equitable distribution of AI depend on building in the open. We bring