Since everyone is talking about RL Environments and GRPO now but no one knows how it works we thought it would be cool to make an explainer video + code you can run:. This is an example of using GRPO to train Qwen 2.5 to play 2048 (code in thread) 🧵:

IMO the only reason CUA agents aren't more prevalent is because of 1) speed, 2) cost and 3) inability to do long horizon tasks reliably (in that order). The models are already quite good for many important tasks. Scaling compute will fix 1) and 2) in ~1 year.

Join millions who have switched to Grok.

There's been a lot of cloudflare hate recently but tbh they're justified in wanting to restrict ai agents. Until very recently, the de facto business model of the internet was ads. Ads require a human being to view your website and interact with it. If most of your site traffic

Getting a lot of people telling me LLMs can accurately simulate human behavior. This isn't true. If LLMs could truly model human preferences, markets would be obsolete. Prices are just a clumsy way to guess what people want. Perfect simulation would make price signals.

This is downstream of the fact that we’re able to model basically everything in the physical world except ourselves.

This is my new favorite hub.

You can't make an RL env out of is things that require human behavior inside of the env. Ex: you can't have an accurate RL environment that simulates a Twitch streamer interacting with their fans, because that requires accurate simulation of the human utility function.

Ofc a large obstacle to this is ChatGPT not giving you all the traces, but it may be possible with other models.

This may be the blueprint for AI apps moving forward:. 1) Make ChatGPT/Claude wrapper that users love. 2) Collect production traces and create evals . 3) SFT an OSS model on the traces and RL on the evals to get parity with ChatGPT/Claude. Similar quality and lower costs.

Code to run this 2048 example on Qwen 2.5 is here:.

Training using an env can be broken into a 3 step process:. Step 1: Generate trajectories from the environment: The model (agent) takes actions like “move left/right/up/down” and generates a trajectory of moves until the board reaches 2048. Step 2: Assign rewards: Each

Since RL environments are becoming a lot more mainstream it probably makes sense to explain them for people who see it vague posted incessantly on the TL. An RL environment is the "world" or "problem space" in which a reinforcement learning (RL) agent operates in order to learn.

The biggest bottleneck to RL environments scaling may unironically be docker build times.

If you want to learn the dark art of how to make a kino (not slop) env though you should join us at @hud_evals.

I usually don’t talk about RL envs on the tl out of respect for our customers but this take won’t age well. Making problems that provide signal for models to get better is pretty hard, and is only going to get harder every year as models improve. The notion that you can vibe.

Kind of crazy how gpt-oss mogs everything from China. If they ever release r2 it’ll have to multimodal to be relevant at all.

> gpt-oss has no multimodal support. It’s so over.

The impact of gpt-oss might be much bigger than gpt-5. It almost feels wrong to have something this powerful be oss. Can think of so many use cases.