Radical Numerics
@RadicalNumerics
Followers
1K
Following
15
Media
5
Statuses
8
San Francisco & Tokyo
Joined May 2025
Excited to share Phalanx, our new layer for sequence modeling! Each block communicates with its neighbor, like the shield cover of a neighboring hoplite. Phalanx can replace sliding window attention and trains faster than optimized baselines while maintaining quality.
Sliding window attention (SWA) is powering frontier hybrid models for efficiency. Is there something better? Introducing Phalanx, a faster and better quality drop-in replacement for sliding window attention (SWA). Phalanx is a new family of hardware and numerics-aware windowed
2
9
50
More on Phalanx and our research kernel library: Blog: https://t.co/NzyyFuSIKr Code: https://t.co/kHosQhyHSD Report: https://t.co/iVLETD2AYx
1
2
13
Sliding window attention (SWA) is powering frontier hybrid models for efficiency. Is there something better? Introducing Phalanx, a faster and better quality drop-in replacement for sliding window attention (SWA). Phalanx is a new family of hardware and numerics-aware windowed
12
52
205
We just released the largest open-source diffusion language model (RND1). RND1 is important to me on a personal level: it symbolizes our commitment to open-source exploration of radically different designs for AI at scale — training objectives, architectures, domains. There is
Introducing RND1, the most powerful base diffusion language model (DLM) to date. RND1 (Radical Numerics Diffusion) is an experimental DLM with 30B params (3B active) with a sparse MoE architecture. We are making it open source, releasing weights, training details, and code to
9
40
330
5
0
70
We’re also hiring aggressively. Reach out if you’re interested in building automated research environments and agents. (AI researchers and SWEs, pre/mid/post training, architecture design, kernels, lots of backend system design, and automation) Our team is behind the tech for
16
5
112
More on RND1 models: Blog: https://t.co/VGHEu7J98P Code: https://t.co/rqUmMDsC2Q Report: https://t.co/JlnejayKV2 Weights: https://t.co/3pc1NngnmF
1
5
86
Introducing RND1, the most powerful base diffusion language model (DLM) to date. RND1 (Radical Numerics Diffusion) is an experimental DLM with 30B params (3B active) with a sparse MoE architecture. We are making it open source, releasing weights, training details, and code to
103
256
1K