We will soon get to a point, as AI model progress continues, that almost any time something doesn’t work with an AI agent in a reasonably sized task, you will be able to point to a lack of the right information that the agent had access to. This is why context engineering is

the best the future could muster a power outage in SF. sad.

another reasonable explanation (we also do multi-turn, tool-use heavy RL:

We improve both pass@1 AND pass@n during training. The issue is that lots of claimants: 1) train on domains with heavy mid/posttraining in the base models (math) 2) don't train for very long In many of these small-scale experiments, gains come from re-learning the format

Label noise really matters in RL SID-1's task requires reporting the documents most likely to contain the answer to a question. When the ground truth data contains errors, the model will start overreporting in hopes of catching spurious targets. For one public dataset where

Most RL frameworks are fundamentally unstable. We wasted more H100 hours on debugging this than any other issue fornour multi-turn, multi-env RL run (below). When using OpenAI-style messages for env interactions, parsing and retokenizing leads to subtly different tokens. This

Good RL environments are much richer than you think. We evaluate training for 100 epochs and see eval reward increase steadily. Partly, this is because our RL setting allows obfuscating the answer between epochs, largely mitigating memorization (when inspecting train rollouts).

we just released our first model: SID-1 it's designed to be extremely good at only one task: retrieval. it has 1.8x better recall than embedding search alone (even with reranking) and beats "agentic" retrieval implemented using all frontier LLMs, including the really large and

We believe retrieval is the ideal playground for self-play RL on LLMs. SID-1 was trained with "pseudo self-play:" New questions were generated to cover gaps in model behavior as training progressed. We think we're not far away from closing that loop: Generating hard, verifiable

This is such a sad showing from Zoom. A note from someone who has *trained* a SOTA LLM. Let me explain: @Zoom strung together API calls to Gemini, GPT, Claude et al. and slightly improved on a benchmark that delivers no value for their customers. They then claim SOTA. The

it's fascinating how small hparam changes will lead to vastly different behaviors, strategies, and personalities in models we have no idea why

OpenAI-style messages will make your RL run collapse! This explains behavior seen in Search-R1 and also during training of SID-1. Fix: Handle environment responses directly in tokens. This is NOT the case in veRL. From the report: >Parsing a token list to a message list is

SID-1 knows what it needs to know at all times.

first nvidia-smi in space

1e-6 is the best lr for grpo

ETH Zurich has 10,000 H100s

it was a privilege to work on this model! it's also great to finally be able to share some of the details in our tech report! we've been working on retrieval for a while. this is the way it should always have worked. the results speak for themselves.

length bias removal in common GRPO settings (Dr. GRPO, Mistral) is unstable and leads to collapse over time. we first observe this during training SID-1 and then prove mathematically that this bound to happen. this might explain why DeepSeek did not remove the "length bias"!

git blame for google docs would light the world on fire

"AI written" is the ultimate insult for a piece of writing. if the author admits an llm wrote it, they seem lazy. if the author insists they wrote it, they seem dull. lazy seems more palatable.