LLMs trained with just 8-bit numbers? That led me down a rabbit hole. How does something so complex still work with such coarse formats?. Wrote about mixed-precision training, loss scaling, and what it might say about intelligence:.📚

Coverage by Scientific American 🥳.

RT @nblqbl: @pguptacs is at #ICML2025 this week presenting our work on using ML to find dualities in statistical physics. so far we can red….

Excited to present 2 papers at #ICML!. 🧠 Dualities via ML: We rediscover the Kramers–Wannier duality using neural nets.📚 🌍 RICE-N: Agents learn to negotiate climate deals.📚

📚 Blog: 💻 Code: I wrote this to understand the nuts and bolts of LLM infra —.If you're on the same path, this might help. #PyTorch #LLMEngineering #DistributedTraining #MLInfra.

What’s inside 🛠️. ✅ Communication primitives (broadcast, all_reduce, etc.).✅ Data Parallelism — naive, hooks, bucketed.✅ Tensor Parallelism — MLP & attention, custom autograd ops.✅ Pipeline Parallelism — AFAB, 1F1B, Gantt charts.

So I built a series of toy implementations —.not for scale, but to explore how data, gradients, and layers move across devices. This is a playground for understanding:.- No clusters.- No DeepSpeed or Megatron.- Just PyTorch + Docker.

🧵 I just published:.Distributed Training for Dummies — a 4-part blog series. I was curious how LLMs are trained across 1000s of GPUs. Reading HuggingFace’s UltraScale Playbook helped —.but I wanted to see:.👉 What does this actually look like in code?.📚

I learnt a lot by “delving” into various topics around how word usage evolves with time — and learning about the fast-growing world of podcasting was fun. There is quite a lot of research being conducted to understand how podcast topics evolve with time.

I loved reading the empirical studies with societal impact, but this time I contributed to one. Our paper on the impact of ChatGPT on spoken communication ( just got featured in The Verge (.

Exciting to see our work come to life! This was such a fun collaboration. Looking forward to more insights!.

RT @nblqbl: Something a little bit different from my usual: with Andrea Ferrari and @pguptacs, we investigated whether we can use machine l….

I got 100 points in spooking a machine! Can you beat me? #SpookTheMachine @MachinesSpooked

RT @hiromu1996: 📢 New Preprint!.Now it's known ChatGPT overuses words like 'delve' and 'adept.' This raises the possibility that through th….

Grateful for the opportunity to guide engineering minds at G-Research through the fascinating world of Attention, Transformers, and LLMs. The blend of academia and industry sparks great innovation! Let's keep the AI momentum going! #AI #LLMs

Can't wait for the AI4GCC workshop! Over 2 years in the making, this competition bridges the gap between researchers and policymakers to tackle the lack of cooperation among nations. It's just the beginning of something big! #AI4GCC #bridgingthegap #researchers #policymakers.

RT @AI4ClimateCoop: 📅 Save the Date!.Join us for the #AI4GCC2023 Workshop on April 26 at 10 a.m. EST. Delighted to have @alicelepissier &….

RT @AI4ClimateCoop: We’re thrilled to announce that @DeepMind is now an official partner of AI4GCC, providing jury members and logistical s….

RT @DeepMind: Today in @Nature: #AlphaTensor, an AI system for discovering novel, efficient, and exact algorithms for matrix multiplication….

RT @AI4ABM: We're excited to support the "AI for Global Climate Cooperation", a competition and collaborative research effort for everyone….