Today we’re announcing Mechanize, a startup focused on developing virtual work environments, benchmarks, and training data that will enable the full automation of the economy. We will achieve this by creating simulated environments and evaluations that capture the full scope of.

The bitter lesson of the past decades is that the best algorithms are discovered by applying massive compute for search & learning. Progress in architectures has been incremental, while data-driven innovations have been highly impactful. We expect this trend to continue.

Read the blog post here:.

Current RL methods like verifiable rewards can teach models to solve neat puzzles or prove theorems. But real-world tasks aren’t neatly packaged. To build genuinely capable AIs, we need richer RL environments, ones that capture the messiness of reality and reward good judgment.

Despite being trained on more compute than GPT-3, AlphaGo Zero could only play Go, while GPT-3 could write essays, code, translate languages, and assist with countless other tasks. That gap shows that what you train on matters. Rich RL environments are now the bottleneck.

xAI is probably the first to spend as much compute on RL as on pretraining. The easy gains from shifting compute to RL are now gone. With this arbitrage closed RL scaling will slow. Progress will now come from the quality and realism of RL environments rather than mere scaling.

Read the blog post here:

The future is building software, not curating static datasets. Today’s AI systems learn best by interacting with digital environments, attempting tasks, and learning from outcomes. This demands dedicated, full-time specialists with strong expertise, not outsourced contractors.

Previously, AI progress relied heavily on monotonous, low-skill labeling from third-party contractors producing basic text, visual, and audio data at scale. But models have outgrown simple tasks, demanding richer context and deeper expertise. The era of sweatshop data is over.

Read the blog post here:

We call this approach replication training: tasking AI models with exactly replicating existing software using clear specs and references. This trains models to read precisely, execute reliably, and demonstrate resilience in complex, long-horizon tasks.

Imagine if pretraining a language model meant manually creating the entire training corpus. Clearly, this would be impractical. Instead, we leverage vast content available online. We expect RL will follow a similar path, replicating abundant existing software as RL tasks.

Beyond this point, we expect models trained via RL to acquire powerful, task-agnostic, few-shot capabilities on tasks that currently require painstaking, task-specific training, just as GPT-3 unlocked few-shot capabilities for language.

How much bigger must RL get to have a GPT-3 moment?. We expect this will soon require roughly 10,000 years of cumulative human-equivalent task time, comparable to GTA V or major operating systems.

Read the blog post here:.

Replication tasks target critical skills current models lack: accurately interpreting instructions, reliably recovering from mistakes, and sustaining precise execution on tasks that humans take months to complete, moving us closer to reliable, capable AI agents.

Replication training naturally extends existing AI trends by building tasks directly from abundant human-generated data already available online.

We are proposing a new AI paradigm called replication training: tasking AIs to precisely replicate existing software. Abundant internet text unlocked powerful language models. Similarly, abundant software available today will enable massive-scale, efficient RL training.

Read the blog post here:

Before GPT-3, achieving good performance required specialized fine-tuning for each task. Today's RL is similar: models need to be carefully trained to handle tasks like deep research, web search, or coding. But we think RL will soon have its GPT-3 moment.

RT @tamaybes: Most AI labs talk about merely “augmenting” humans at work. They say this because AI currently falls short, not out of some d….