Wei (Will) Feng
@weifengpy
Followers
103
Following
2K
Media
1
Statuses
37
PyTorch Distributed, FSDP, float8
United States
Joined March 2011
verl is embracing @PyTorch fsdp2! Better throughput, memory usage, and composability with torch.compile! Please try it out and give us feedbacks: https://t.co/Ppa4VxBULk
0
6
18
We have been working on PyTorch native float8 and FSDP2 for distributed training. Check out TorchTitan and TorchAO/float8 https://t.co/fG295IMBO6 with Andrew Gu, @wanchao_ , @drisspg , @vkuzo , @brian_hirsh
1
6
15
@mikiobraun I appreciate your jBlas. I found the pre-compiled version still has DoubleMatrix.get(Range cr, Range rs) bug. Would you update?
0
0
1
How to rank vectors based on their features?: http://t.co/COVrBYSs
0
2
1
On @Quora: What is the best open source implementation of R-tree? Answer: http://qr.ae/7CABN
0
0
1