Introducing DeepConf: Deep Think with Confidence 🚀 First method to achieve 99.9% on AIME 2025 with open-source models! Using GPT-OSS-120B even without tools, we reached this almost-perfect accuracy while saving up to 85% generated tokens. It also delivers many strong

🚨 New Work! 🤔 Is RL black-box weight tinkering? 😉 No. We provably show RLVR follows a 🧭 — always updating the same off-principal regions while preserving the model's core spectra. ⚠️ Different optimization regime than SFT — SFT-era PEFT tricks can misfire(like PiSSA, the

We’ve always assumed stale and off-policy data hurts RL a lot — but our latest work shows the opposite. 🧠 M2PO (Second-Moment Trust Policy Optimization) reveals that even data stale by 256 model updates can train LLMs as effectively as on-policy RL, unlocking scalable and

Transfusion combines autoregressive with diffusion to train a single transformer, but what if we combine Flow with Flow? 🤔 🌊OneFlow🌊 the first non-autoregressive model to generate text and images concurrently using a single transformer—unifying Edit Flow (text) with Flow

Truly congrats to my friend @jxbz! He was my first mentor, guiding me into the field of optimization back in undergrad, and I’ve seen him stay true to his vision of uncovering the laws of neural networks through all the challenges and breakthroughs. Rethinking how we approach

I’ll be giving a guest lecture at Princeton tomorrow (Thursday, 9/25), sharing our recent works on LLM Reasoning and Efficiency. Time and location below: 2:55–4:15pm at CS Building 402 Thanks to @liuzhuang1234 for organizing this!

Thanks @_akhaliq for sharing our work! MobileLLM-R1 marks a paradigm shift. Conventional wisdom suggests that reasoning only emerges after training on massive amounts of data, but we prove otherwise. With just 4.2T pre-training tokens and a small amount of post-training,

We need more reviewers for the 1s Workshop on Efficient Reasoning(ER) at @NeurIPSConf, if you are interested, please fill out the nomination form

Want to try DeepConf NOW? While our full repo is coming, we just dropped a ready-to-run example in our vLLM (@vllm_project ) PR: https://t.co/TtlxtKlGfK DeepConf + DeepSeek-R1-8B + BRUMO25 = • 93.3% accuracy (+2.5% boost) • 52.9% fewer tokens generated • 31% faster

Thanks @vllm_project folks for pushing this. Please give it a try and let us know!

⏰ Submission deadline coming up fast! (Sep 1) Working on efficient reasoning? Don’t miss the chance to share it at NeurIPS 2025!

We released DeepConf that can achieve 99.9% on AIME'25 with open source models with only 15% of the compute, compared to majority voting@512. The secret? Simple. Just to pruning the rollouts if they show a consecutive stream of low-confidence😀. Can be applied to any models

Thank you for sharing it! @_akhaliq https://t.co/D3PQmpfGDI

🛠️ Super easy to implement! Just ~50 lines of code in vLLM. Plug & play - works with ANY existing model. Zero training or hyperparameter tuning needed. Code example: https://t.co/cqn5Anf88P Live vLLM PR: https://t.co/eW9qCTGSm2 💻 Open-source code coming soon!

📈 Online results: 33-85% token savings across all benchmarks! AIME 2025: 97.9% accuracy with 85% fewer tokens (GPT-OSS-120B). Works across 8B to 120B models - real-time efficiency without sacrificing quality. Chart shows AIME 25 results.

🏆 Offline results: 99.9% accuracy on AIME 2025 (vs 97% baseline)! Universal gains across 5 models × 5 datasets. Consistent ~10% accuracy boost across all settings. Check Table 1 in the paper for more details.

DeepConf works in 2 modes: Offline: Filter completed reasoning traces by confidence, then weight votes by quality Online: Stop generating when confidence drops below threshold in real-time

💡 The secret? LLMs already know when they're uncertain - we just weren't listening! Previous methods use confidence/entropy AFTER full generation for test-time and RL. We're different - we capture reasoning errors DURING generation. DeepConf monitors "local confidence" in

Excited to see Logarithmic format (LNS, UE8M0 FP8) used in production by @deepseek_ai! LNS enables efficient multi (just addition between exponents) + great dynamic range. Our LNS-Madam optimizer, built for LNS, was proposed years ago before LLM-era - hope it shines again!