Excited to announce our new work! 🧬 Some highlights are: - sequences likelihoods predict zero-shot fitness capabilities - a new method to calculate pLM likelihood in O(1) instead of O(L) forward passes - providing a causal between training data and outputs - suggesting a new

Patient work, careful hands. See what grew.

Introducing the next generation: Claude Opus 4 and Claude Sonnet 4. Claude Opus 4 is our most powerful model yet, and the world’s best coding model. Claude Sonnet 4 is a significant upgrade from its predecessor, delivering superior coding and reasoning.

Excited to share that I'll be joining @Anthropic to work on pretraining science! I've chosen to defer my Stanford PhD, where I'm honored to be supported by the Hertz Fellowship. There's something special about the science, this place, and these people. Looking forward to joining

Many agents (Claude Code, Codex CLI) interact with the terminal to do valuable tasks, but do they currently work well enough to deploy en masse? We’re excited to introduce Terminal-Bench: An evaluation environment and benchmark for AI agents on real-world terminal tasks. Tl;dr

👏 Meet the 2025 Hertz Fellows—19 rising leaders in science and tech advancing breakthroughs in robotics, energy, medicine & more. 🔗Learn more: https://t.co/RH9zCoCzoR

🎓🤖 We’re thrilled to welcome @CadeGordonML to the 2025 class of Hertz Fellows! Cade’s AI research is advancing biomedical discovery. A future PhD student at @Stanford, he joins a growing community shaping the future of #science and #tech! 🔗 https://t.co/6hKzo1t4Dd

Documenting and sharing research in real-time is underrated in discussions about open science. @jainhiya_ and I think software can help change problem selection, collaboration, and funding. We write about how and why we should create real-time, open lab notebooks.

Chinese policy on clinical trial approvals liberalized massively in the mid 2010s. A decade later the effects of this move are perceptible in where our drugs come from.

Arrived in Singapore for ICLR—excited to see old & new friends! I’ll also be at the: - Thursday 3:30-5pm main conference poster session, presenting work led by @CadeGordonML on the subtleties of using protein LM likelihoods for fitness prediction (see 🔗👇) - GEM workshop

A simple idea to build the @UCBerkeley startup alumni network has grown beyond my wildest dreams into #AccelScholars, a tight-knit community of the most ambitious, talented, kind-hearted people, whose individual stories we’ve been fortunate to support for the past eight years

the IGI wrote a bit about our (in progress) work on building statistical tools for genome mining and discovery! check it out below ⬇️ 🔍

🧪 Introducing POPPER: an AI agent that automates hypothesis validation by sequentially designing and executing falsification experiments with statistical rigor. 🔥POPPER matched PhD-level scientists on complex bio hypothesis validation - while reducing time by 10-fold! 🧵👇

Spent a good few hours and $50 wrangling with a few different implementations not yet finding success. Initial issues in compilation and now OOMs that would need me to reduce batch size or attempt some other tricks. I hope someone can give my code a whirl and have better luck!

We have a similar trick available for writing out the log of harmax, which motivates a way that we can rewrite the equation into logits.

Recall that we can compute the log of softmax with better precision by representing it as the logit minus the logsumexp of all logits. Jax has a great implementation of this https://t.co/GRhwsebL0c

1. We'll keep everything in the training framework fixed except for the LM head. 2. As we're still using CE loss of some probability distribution at the end of the day, we can compare to the normal NanoGPT loss curves. 3. At the end we can try to eek out more performance in the

Now we can start writing up an implementation. I'll list out my plans as this is where engineering choices might start leading to differences in performance. I invite the community to build from my fork and let me know any mistakes!

Getting our hands a bit more dirty, we have the loss calculation. I think this is where the MLEs in the audience might have their eyes light up! We're generating our probabilities, taking the log of them, and sending the result off to the cross_entropy loss. Chaining together

The authors share their original code for the experiment here: https://t.co/ILcMptkH9n Let's start with the computation of the logits. The logits can be written as a modified version of a linear layer with no bias. In particular the original implementation uses the fact that the

The original paper shows some strong early results that for a GPT2 small based off of NanoGPT they get an improved training curve--albeit using a larger learning rate.