Harry Mellor
@hmellor_
Followers
167
Following
147
Media
8
Statuses
44
ML Engineer @huggingface maintaining @vllm_project, prev @graphcoreai, @uniofoxford
Joined September 2022
RT @ClementDelangue: And just like that, @OpenAI gpt-oss is now the number one trending model on @huggingface, out of almost 2M open models….
0
97
0
RT @dylan_ebert_: OpenAI just released GPT-OSS: An Open Source Language Model on Hugging Face. Open source meaning:.💸 Free.🔒 Private.🔧 Cust….
0
39
0
RT @ClementDelangue: When @sama told me at the AI summit in Paris that they were serious about releasing open-source models & asked what wo….
0
259
0
RT @ariG23498: Did you know you can now run your own AI Job on the Hugging Face infrastructure?. Introducing `hf jobs`, the latest addition….
0
4
0
RT @LysandreJik: The new transformers release comes w/ a surprise: kernels support ⚡️. It integrates deeply with precompiled kernels on the….
0
17
0
RT @mervenoyann: We have recently merged fast processors for many models, the speed-up in Qwen-VL series is 🔥. you get speed-up up to 3x on….
0
25
0
RT @vllm_project: The @huggingface Transformers ↔️ @vllm_project integration just leveled up: Vision-Language Models are now supported out….
0
47
0
RT @casper_hansen_: vLLM is finally addressing a long-standing problem: startup times. 35s -> 2s for CUDA graph capture is a great reductio….
0
42
0
RT @ErikKaum: We just released native support for @sgl_project and @vllm_project in Inference Endpoints 🔥. Inference Endpoints is becoming….
0
7
0
RT @LysandreJik: BOOOM! transformers now has a baked-in http server w/ OpenAI spec compatible API. Launch it with `transformers serve` and….
0
30
0
📖 has had a makeover!. Over the past week I wrote a script to port all of @vllm_project's docs from Sphinx ➡️ MkDocs. The DX and UX are so much nicer! 😌
3
3
58
As of the 🤗 Transformers backend for @vllm_project supports interleaved sliding window attention architectures like Gemma 2 and Cohere 2! 🚀.
github.com
Tested with Gemma 2 by asking it to summarise >4k tokens of the Wikipedia page on frogs. vLLM reference: Generated text: '\n\nFrogs are a diverse group of amphibians with a wide rang...
1
0
7
Looks like the crawlers have found ChatGPT is about to become a lot more knowledgeable about @vllm_project 😅
0
0
3
RT @OpenAIDevs: Announcing the first Codex open source fund grant recipients:. ⬩vLLM - inference serving engine @vllm_project.⬩OWASP Nettac….
0
147
0
RT @vanstriendaniel: Need blazing-fast classifier inference with minimal code?. ModernBERT now runs on @vllm_project — fast enough to proce….
danielvanstrien.xyz
Modern Inference for modern classifier models. Using vLLM to scale inference for classifiers to clean and curate datasets
0
60
0