Low-bit LLM quantization doesn’t have to mean painful accuracy trade-offs or massive tuning runs. Intel's AutoRound PTQ algorithm is now integrated into LLM Compressor, producing W4A16 compressed-tensor checkpoints you can serve directly with vLLM across Intel Xeon, Gaudi, Arc

Congrats to the @MistralAI team on the launch of Devstral 2! 🚀 vLLM now delivers Day-0 support for the Devstral 2 Instruct models — optimized for agentic coding, deep codebase exploration, and multi-file editing at scale. Feel free to reach out 👇

In this webinar, you’ll learn how model distillation can cut inference costs by up to 70% while maintaining enterprise-level performance.

🎉Congrats to the @Zai_org team on the launch of GLM-4.6V and GLM-4.6V-Flash — with day-0 serving support in vLLM Recipes for teams who want to run them on their own GPUs. GLM-4.6V focuses on high-quality multimodal reasoning with long context and native tool/function calling,

Big news for AI builders. Ministral 3, DeepSeek-V3.2, and vLLM v0.12.0 are now available on Docker Model Runner! Run frontier-class, open-weights models with one command Read the announcement blog here:

Beyond the engine, v0.12.0 ships EAGLE speculative decoding improvements, new model families, NVFP4 / W4A8 / AWQ quantization options, and tuned kernels across NVIDIA, AMD ROCm, and CPU. We recommend building new images with PyTorch 2.9.0 + CUDA 12.9, validating on staging

Trade with Leverage? Trade with T-REX!

Two engine paths are now available for early adopters: GPU Model Runner V2 (refactored GPU execution with GPU-persistent block tables + a Triton-native sampler) and prefill context parallel (PCP) groundwork for long-context prefill. Both are experimental, disabled by default, and

📢vLLM v0.12.0 is now available. For inference teams running vLLM at the center of their stack, this release refreshes the engine, extends long-context and speculative decoding capabilities, and moves us to a PyTorch 2.9.0 / CUDA 12.9 baseline for future work.

🚀 vLLM now offers an optimized inference recipe for DeepSeek-V3.2. ⚙️ Startup details Run vLLM with DeepSeek-specific components: --tokenizer-mode deepseek_v32 \ --tool-call-parser deepseek_v32 🧰 Usage tips Enable thinking mode in vLLM: –

We’re taking CUDA debugging to the next level. 🚀 Building on our previous work with CUDA Core Dumps, we are releasing a new guide on tracing hanging and complicated kernels down to the source code. As kernels get more complex (deep inlining, async memory access), standard

The latest Microsoft Research Forum episode is now available on-demand. Explore purposeful research and its real-world impact.

🤝 Proud to share the first production-ready vLLM plugin for Gaudi, developed in close collaboration with the Intel team and fully aligned with upstream vLLM. 🔧 This release is validated and ready for deployment, with support for the latest vLLM version coming soon. 📘 The

LLM agents are powerful but can be slow at scale. @Snowflake's model-free SuffixDecoding from Arctic Inference now runs natively in vLLM, beating tuned N-gram speculation across concurrency levels while keeping CPU and memory overhead in check. Quick Start in vLLM:

🎉 Congratulations to the Mistral team on launching the Mistral 3 family! We’re proud to share that @MistralAI, @NVIDIAAIDev, @RedHat_AI, and vLLM worked closely together to deliver full Day-0 support for the entire Mistral 3 lineup. This collaboration enabled: • NVFP4

You can also use the Gradio example provided in the Omni repository. https://t.co/kXsRBXGAHf

Most Wanted Thermal Gloves! Say goodbye to freezing hands. Water and windproof. Great for use with touch screen devices

Currently only supports Qwen-Omni and Qwen-Image, and this is just the beginning. More models are coming. https://t.co/gKKY0uhtNv https://t.co/cmkFjvCN3y

More inference workloads now mix autoregressive and diffusion models in a single pipeline to process and generate multiple modalities - text, image, audio, and video. Today we’re releasing vLLM-Omni: an open-source framework that extends vLLM’s easy, fast, and cost-efficient

Transformers v5's first release candidate is out 🔥 The biggest release of my life. It's been five years since the last major (v4). From 20 architectures to 400, 20k daily downloads to 3 million. The release is huge, w/ tokenization (no slow tokenizers!), modeling & processing.

Love this: a community contributor built vLLM Playground to make inferencing visible, interactive, and experiment-friendly. From visual config toggles to automatic command generation, from GPU/M-chip support to GuideLLM benchmarking + LLMCompressor integration — it brings the

Expand your outlets instantly with this rotating wall plug extender. Keep chargers, appliances, and devices organized while freeing space in tight corners. A simple way to upgrade any room. Discover it today. 👉 Ready to buy? Just click the video or the link.

vLLM is proud to support @PrimeIntellect 's post-training of the INTELLECT-3 model🥰

Interested in how NVIDIA Nemotron-H is being optimized for high performance inference in @vllm_project? Join @RedHat and @NVIDIAAI next week as we cover the Nemotron-H architecture, vLLM support, optimized MoE kernels, async scheduling, and new nsys profiles. Join links below 👇