@danielhanchen, glad you liked the post! You're spot on to suspect lower-level implementation issues. That's exactly what we found in the original blog. The disable_cascade_attn finding (Sec 4.2.4) was the symptom, but the root cause was that silent FlashAttention-2 kernel bug

Had fun contributing a bit to this project! I especially liked this - masked diffusion (any-order generation) can be better than fixed-order AR on problems without a canonical ordering

How to organize your talk? I used to present like this, thinking that I was being "academic", "organized", and "professional". BUT, from the audience's viewpoints, this sucks. 😱 Look how far they need to hold a long-term context to just make sense of what you're saying!

Thrilled to release new paper: “Scaling Latent Reasoning via Looped Language Models.” TLDR: We scale up loop language models to 2.6 billion parameters, and pretrained on > 7 trillion tokens. The resulting model is on par with SOTA language models of 2 to 3x size.

Might also be interested in checking our TARFlow series! TARFlow: https://t.co/Gb7NETqEw2 ICML2025 Oral STARFlow: https://t.co/bpkY7SYx4z NeurIPS2025 Spotlight TARFlow-LM: https://t.co/BLHoXt9m5Q NeurIPS 2025 … and more maybe soon🤖

Please steal my AI research ideas. This is a list of research questions and concrete experiments I would love to see done, but don't have bandwidth to get to. If you are looking to break into AI research (e.g. as an undergraduate, or a software engineer in industry), these are

@wenhaocha1 Thanks, Wenhao! Really appreciated your recognition, and really lucky to meet you back in the early days when we were all starting developing multimodal models - so many new models, datasets, and discussions, bringing many new insights to everyone. From lmms-eval to

Back in 2024, LMMs-Eval built a complete evaluation ecosystem for the MLLM/LMM community, with countless researchers contributing their models and benchmarks to raise the whole edifice. I was fortunate to be one of them: our series of video-LMM works (MovieChat, AuroraCap, VDC)

world model

Stanford NLP 25th Anniversary🤩🤩🤩

Throughout my journey in developing multimodal models, I’ve always wanted a framework that lets me plug & play modality encoders/decoders on top of an auto-regressive LLM. I want to prototype fast, try new architectures, and have my demo files scale effortlessly — with full

@josh1yan I joined @OpenAI as a resident. First, get the fundamentals down. If there's one subject you need to know inside and out, it's linear algebra. Read and understand a classic textbook like Bishop's Pattern Recognition and Machine Learning. Then, take on an ambitious project. I

end-to-end training just makes latent diffusion transformers better! with repa-e, we showed the power of end-to-end training on imagenet. today we are extending it to text-to-image (T2I) generation. #ICCV2025 🌴 🚨 Introducing "REPA-E for T2I: family of end-to-end tuned VAEs for

Congrats to FlowEdit for winning #ICCV2025 Best Student Paper. “Inversion-free” is a very cool idea. We proposed the first inversion-free, optimization-free, and model-agnostic framework (for latent diffusion and consistency models) back at CVPR 2024 ( https://t.co/zMrIfyVFpq).

I’ve always wanted to write an open-notebook research blog to (i) show the chain of thought behind how we formed hypotheses, designed experiments, and articulated findings, and (ii) lay out all the intermediate results that did not make it into the final paper, including negative

Our paper Video-MMLU has been awarded Outstanding Paper at the ICCV Workshop! I happened to receive this wonderful news while soaking in the water couldn’t be happier! Huge thanks to the Knowledge-Intensive Multimodal Reasoning Workshop Committee for the honor.

Paper: https://t.co/wvMxdVTvHl Blog: https://t.co/2TPDgzrygX (you can also see updated LiveCodeBench Pro leaderboard) How to start evaluation: https://t.co/zi441pmL9z Amazing work led by @ZhouShang for AutoCode and @ZihanZheng71803 for Eval toolkit.

LiveCodeBench Pro remains one of the most challenging code benchmarks, but its evaluation and verification process is still a black box. We introduce AutoCode, which democratizes evaluation allowing anyone to locally run verification and perform RL training! For the first time,

So excited to be part of the team bringing the 1st Multimodal Spatial Intelligence (MUSI) workshop to @ICCVConference, with a huge shout-out to @songyoupeng for leading the effort! We've put together an incredible program. If you'll be at ICCV, you should definitely stop by! 🗓️

The work opened my eyes. Since my PhD, I've been studying visual representations for understanding and generation. I long thought pretrained vision encoders (CLIP, DINO, etc.) produced features too semantic for generation/reconstruction, but that's not true! These features