OpenAI just released GPT-OSS: An Open Source Language Model on Hugging Face Open source meaning: 💸 Free 🔒 Private 🔧 Customizable

OMG, the U.S. just downloaded more than 5PB of DeepSeek-R1 on @huggingface in the last few days! Feeling late FOMO in Silicon Valley? 🤔🚀

🧵(2/2) With inference-benchmarker you can: 🧪 Simulate real workloads (chat, code-gen...) 📊 Measure throughput, time-to-first-token, inter-token latency ⚙️ Compare performance across backends & infra 👉 Check it out:

🧠 LLM inference isn’t just about latency — it’s about consistency under load. Different workloads, configs, and hardware = very different real-world performances. At Hugging Face 🤗 we built inference-benchmarker — a simple tool to stress-test LLM inference servers. 🧵 (1/2)

@huggingface GPU-fryer helps us detect silent throttling failures: one GPU slows down and every other unit ends up waiting, creating a bottleneck 🚦. Check it out:

At @huggingface we rely on GPU-fryer 🍳 to load-test our 768 H100 GPU cluster. It runs matrix multiplications and monitors TFLOPs outliers to catch any software or hardware throttling — often a sign of cooling issues that need a hardware fix ❄️🔧. 🧵 1/2

This first step will very soon be followed by the integration of new backends (TRT-LLM, llama.cpp, vLLM, Neuron and TPU). We are polishing the TensorRT-LLM backend which achieves impressive performances on NVIDIA GPUs, stay tuned 🤗 ! https://t.co/eGpEvqVM8L

We are introducing multi-backend support in @huggingface Text Generation Inference! With new TGI architecture we are now able to plug new modeling backends to get best performances according to selected model and available hardware.

Just 10 days after o1's public debut, we’re thrilled to unveil the open-source version of the groundbreaking technique behind its success: scaling test-time compute 🧠💡 By giving models more "time to think," LLaMA 1B outperforms LLaMA 8B in math—beating a model 8x its size.

We're turning @huggingface Hub's files into content-defined chunks to speed up your workflows!⚡️ This means: - 🧠We store your file as deduplicated chunks - ⏩ You only upload changed chunks when iterating! - 🚀 Pulling changes? Only download changed chunks!

An easy way to understand Pipeline Parallelism with a self contained implementation. Check it out!

New feature on the Hub! ☁️ Carbon emissions emitted during training now show up on the model card! (requires model authors to fill that info first) Hopes it will prompt more people to show the carbon emissions of their model training! 🌍 Thanks a lot to the team who pushed

We passed 5 million users. 🥳That's 5 million of you who have signed up on the Hub 🚀 thank you for contributing to the ecosystem and making open Machine Learning happen! We're just getting started 🤗

Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet. Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context

I am mind blown by this new technology! AI is now embodied. And we are open-sourcing it all. Listen to @HaixuanT casually discussing with its cute robot at the @linuxfoundation: 🙂 What's your name? > I am Reachy, a robot from @pollenrobotics, I have two arms. 😀 What do you

sota RAG in 2024

Llama 3 released! 🚨🔔@AIatMeta just released their best open LLM! 👑🚀 Llama 3 is the next iteration of Llama with a ~10% relative improvement to its predecessor! 🤯 Llama 3 comes in 2 different sizes 8B and 70B with a new extended tokenizer and commercially permissive license!

Introducing: Zephyr 141B-A35B 🥁 🔥Mixtral-8x22B fine-tune 🤯 Using DORPO: new alignment algorithm (no SFT, open ) 🚀 With 7k instances of (open) data Very strong IFEval, BBH, AGIEval... Enjoy! 🤗 https://t.co/MVxTJorIGc

this 30-min-read blog post on how to craft and generate a 25B+ tokens synthetic text dataset distills more information and alphas than a typical NeurIPS best paper

Huge spatial images dataset released by @ESA_EO and @huggingface 🛰️ so much to build on it!