Marc Sun
@_marcsun
Followers
2K
Following
3K
Media
23
Statuses
572
Machine Learning Engineer @huggingface Open Source team
New york
Joined February 2023
RT @art_zucker: Holy. `transformers` reached 1B downloads 😠thanks everyone for making this possible, what an amazing community https://t….
0
26
0
RT @RisingSayak: Had the honor to present diffusion transformers at CS25, Stanford. The place is truly magical. Slides: .
0
86
0
RT @realHongyu_Wang: We just released the fine-tuning code and fine-tuned models of BitVLA in Huggingface🔥🔥.Enjoy these hyper-efficient 1-b….
0
11
0
RT @LysandreJik: BOOOM! transformers now has a baked-in http server w/ OpenAI spec compatible API. Launch it with `transformers serve` and….
0
28
0
RT @RisingSayak: Boy, we shipped, and we shipped hard 🧨. From new SoTA open models to improved support for torch.compile to features, inspi….
0
14
0
0
0
1
🤗 Transformers is becoming the source of truth for model definitions, regardless of backend or runner.
The Transformers library is undergoing it's largest pivot to date 🙌. It now cements its role as the central model definition, irrespective of the backend and runner. One ground truth to bring more reliability across the ecosystem. Why is this important?
1
0
1
In practice, this means you can now use any transformers-compatible model with SGLang. To explicitly use the Transformers backend, just set `impl="transformers"`.
1
0
2
🚀 SGLang now supports Hugging Face Transformers as a backend!. Run any transformers-compatible model with fast, production-grade inference — no native support needed. Just plug and play 🥳. Blogpost:
3
24
109
RT @TheZachMueller: This is starting to feel more like a conference, less like a course every day. We're now having the amazing @wanchao_ a….
0
6
0
We added support for regional compilation with the DeepSpeed engine. DeepSpeed’s .compile() modifies models in-place using torch.nn.Module.compile(. ), rather than the out-of-place torch.compile(. ), so we had to account for that.
0
0
1
We updated the CCL_WORKER_COUNT variable and added KMP parameters for Intel CPU users. This significantly improves distributed training performance, with up to a 40% speed-up on Intel 4th Gen Xeon when training transformer TP models.
1
0
0
We've simplified how to prepare FSDPv2 models, as there were too many ways to compose FSDP2 with other features. Although the setup is now more restrictive, it leads to fewer errors and a more performant user experience. We’ve also added support for FP8 !.
1
0
0
🚀 Accelerate v1.8.0 is here! . Highlights of this release: .- 📷FSDPv2 + FP8 support by @m_sirovatka .- Faster distributed training on Intel CPU by jiqing-feng.- Regional compilation for deepspeed by @IlysMoutawwakil. Release notes:
1
5
26
RT @mgoin_: Exciting first day talking about @vllm_project in Singapore! I had an great time discussing in depth with @EmbeddedLLM on how w….
0
11
0
RT @_derek_liu_: Now you can make Flux.1 your own within just 10GBs of VRAM. In our new blog post we walk you through the process step by s….
0
9
0
RT @reach_vb: Let's goooo! @kyutai_labs just dropped SoTA Speech to Text transcriptions model - CC-BY-4.0 Licensed 🔥. > kyutai/stt-1b-en_fr….
0
56
0
RT @RisingSayak: Overlap compute with communication while offloading to disk instead of CPU 🔥. This (group offloading) provides a very nice….
0
5
0
RT @PyTorch: Interested in pushing the limits of optimizing LLMs on GPUs? Join engineers from @AMD , PyTorch, @GPU_MODE, @huggingface & mo….
0
6
0