For too long, users have lived under the software lottery tyranny of fused attention implementations. No longer. Introducing FlexAttention, a new PyTorch API allowing for many attention variants to enjoy fused kernels in a few lines of PyTorch. https://t.co/IXeUS6AkrY 1/10

Thanks to everyone who helped me with the figures and design(@alhyunsoo), helped me with experiments (@jacobmenick), and also to cut down my exclamation points by a factor of 3. :)

Claude Code & OpenAI Codex are coming to Bolt. Build enterprise-grade products visually right in your browser.  No setup. No CLI tools. No 💔 error loops. Which agent are you most excited for?

Apologies that I haven't written anything since joining Thinking Machines but I hope this blog post on a topic very near and dear to my heart (reproducible floating point numerics in LLM inference) will make up for it!

Suno 4.5 is quite impressive. Previously AI music was only ever interesting for the novelty. Now, I wouldn't blink if I heard one of these songs on a playlist. First generation I tried: Prompt: "Pop song about optimizing CUDA kernels for LLM training" https://t.co/p2ehQlpacr

https://t.co/WsFghCe7jJ

$SNES: The Tiny Company Tackling a Billion-Dollar Problem Introduction @SenesTech ($SNES) is tackling one of the world’s most persistent problems: rodent infestations. From major cities to farms across rural America, rats cause billions in annual losses by damaging crops,

When it comes to hardware that's meant for training or inference, most think about in hardware specs like memory bandwidth even though dev velocity is often a more important factor. One implication is that RL training and prod. inference are meaningfully different workloads.

This is super cool! With FlexAttention, you can now build a super minimal "throughput-oriented" inference system without needing custom kernels! One super cool part about using FlexAttention for this is that PagedAttention just ended up being a special case of the abstraction!

Not a great look that after presenting GPT5's reduced hallucinations, their first example repeats a common error of how plane wings generate lift ("equal transit theory").

You're no match for OpenAI's marketing team.

Don't settle: do it all.

Other than OpenAI, how many other AI efforts do you think will have gotten a gold medal at the IMO? Several other AI labs are vagueposting about their IMO results, but seem to abiding by IMO's request for a week's delay.

It's been an exciting 3 months at Thinky and so much has happened already! Imo we're building some of the best research infra around. Research infra is about jointly optimizing researcher *and* GPU efficiency, and it's been a joy to work on this with the other great folk here!

I'll be at MLSys today! DM me if you want to chat about Pytorch, ML systems, or life at Thinking Machines!

The fundamental question here (computing MFU) is a very reasonable question to ask in an interview (and if I'd recommend learning it if you don't know how). However, the real interview question I would like to ask is this: "I see 3 assumptions in this question that range from

When this word started popping up I initially smugly thought that people were misspelling "syncophant" only to realize that I'd entangled "sycophant" with "syncopation" in my head.

This is pretty neat. They insert into torch.compile and insert some profile-guided optimizations as well as a bunch of other specific optimizations like offloading. Since torch.compile is all in Python all their compiler passes are fairly accessible too! https://t.co/gxpcGQlILf

I'll be here and talking about ML systems! There'll be some of the best GPU folk I know here, so come and learn more together about Blackwell GPUs!

If you're interested in working together at @thinkymachines, please DM me on Twitter. (also free to DM me if you want to work on PyTorch). I've been grateful to work at PyTorch, and I hope Thinking Machines will be just as fulfilling. 6/6

However, @thinkymachines ended up being an extremely compelling opportunity. The opportunity to be part of an extremely strong (and nice!) founding team, being able to to continue to contribute to open systems, and an approach to "making AI go good" that resonated with me. 5/6

The actual day-to-day on PyTorch has also been amazing - working on a project that undergirds the industry and values OSS impact provides a shelter from big-tech politics and amazing opportunities for career. https://t.co/SDeBA8U0BQ 4/6