Applications closing today! 👻 If you need an extension to finish up, reach out to us directly, and we can give you a few days.

Today we're very happy to announce that we’re launching the Tilde Fellowship Program to support research in a mechanistic understanding of pre-training science (arch, optimizers, learning dynamics, etc.). Much of modern ML progress has come from scaling models and empirically

Modern optimizers can struggle with unstable training. Building off of Manifold Muon, we explore more lenient mechanisms for constraining the geometry of a neural network's weights directly through their Gram matrix 🧠 A 🧵… ~1/6~

Vignette #2 is here! Join @AlecDewulf to: Learn about circuit complexity theory Derive theoretical capabilities and limitations of transformers Discuss the future of theoretical computer science in architecture design A thread 🧵

We'll be at the Berkeley EECS Career Fair this Tuesday & Wednesday with cool custom puzzles (and prizes). @berkeley_ai @UCBerkeley

Today we’re launching our weekly vignette series! 📚 Vignette #1: Attention through a regression lens 📈 • derive attention from regression • analyze the geometry of kernel smoothers • introduce Epanechnikov attention (feature map + recurrent form) A thread 🧵

Our puzzle series is back! 🧩 This one’s about how linear attention emerges from a simple quadratic objective - and how to extend it! Think you got a shot? Try it out here → A thread 🧵

At Tilde, we believe mechanistic understanding of models is key to enabling entirely new architectures and capabilities. We’ve put together a position piece on what interpretability means to us. A thread 🧵

Mixture‑of‑Experts (MoE) powers many frontier models like R1, K2, & Qwen3 ⚡️ To make frontier-scale MoE models accessible to train, we open-source MoMoE, a hyper-performant MoE implementation built for training and inference, outpacing the fastest existing ones by up to: - 70%

We’re excited to announce that Tilde completed an $8M seed round earlier this year, led by Khosla Ventures. Understanding model intelligence is the most important problem in the world, and the key to actualizing the promise that ASI can offer. 🧵 A thread on our approach:

Sparse attention (MoBA/NSA) trains faster & beats full attention in key tasks. But we’ve had no idea how they truly work…until now. 🔍 We reverse-engineered them to uncover: - Novel attention patterns - Hidden "attention sinks" - Better performance - And more A 🧵… ~1/8~

Today, we open-source Activault, a simple, high-throughput, and cost-effective solution to activation data management for accelerating interpretability research on frontier models. A 🧵… ~1/6~ https://t.co/ZMTbtHRUJD

Over the past few weeks, we've been using this graph theory problem in interviews and figured we'd open it up to everyone here! https://t.co/GQCg2rk1Q4 If you solve it, we’ll move you directly to the last rounds of our process!

I'm excited about our case study! SAEs can be practical for preventing behavior, especially rare behavior, as we only intervene when needed. Other tools, like finetuning or system prompts, have side effects as they modify every forward pass. Some thoughts below.

Mechanistic interpretability is fascinating - but can it be useful? In particular, can it beat strong baselines like steering and prompting on downstream tasks that people care about? The answer is, resoundingly, yes. Our new blog post with @a_karvonen, Sieve, dives into the

We're thrilled to be launching Tilde. We're applying interpretability to unlock deep reasoning and control of models, enabling the next generation of human-AI interaction. By understanding a model's inner mechanisms, we can enhance both its reliability and performance—going