Explore tweets tagged as #torchrun
Introducing RL2: Ray-Less Reinforcement Learning for LLMs 🚀 . Want to run RL experiments but tired of complicated abstractions? We've got you covered with <1K lines PPO/REINFORCE implementation: .🎯 Ray Less = Launch RL exps with torchrun just like SFT .⚡ Long-context
11
36
293
En we feesten nog even door. De Torchrun van NL heeft met ons meegedacht voor speciaal vervoer voor het galadiner van Dion. Met @POL_Breda met een busje naar het bureau en door naar school. Wat een fijne mensen!! 💞💞 Dank jullie wel!! #KleinGeluk . 💛🖤.#GehandicaptKind.#ZieOns
19
1
65
Implemented Gemma 3 (text only) from scratch in PyTorch. Oretrained (90 M parameters) model on 100M tokens from the TinyStories dataset from scratch using DDP (torchrun) in PyTorch. Used P100 from Kaggle. Link -
8
36
281
H100で GRPOの強化学習に52時間かかってたけど、8個のH100で16時間に短縮成功!. torchrun使って、複数GPUの割り振りサクッとできた。
大体計算すると3EPOCをGRPOで強化学習するのにH100 で60時間ぐらいかかるっぽい。1時間500円だとすると、30000円ぐらいで、ベースモデルにGRPOで強化学習してアハ現象とか入れられるかもしれないのか.
1
15
138
#specialolympics #freezinforareason #polarplunge #torchrun #schp #lawenforcement #statetroopers . 2024 Polar Plunge at Lake Greenwood was a blast. There were hot beverages, costume contest and of course really cold lake water. We were freezing for a reason!
1
8
13
Yesterday, #TeamUPD participated in the annual 2024 Special Olympics Law enforcement TorchRun 🔥.#UPD_ProudToServe #LETR4SOSC #SoCalTorchRun
0
2
12
genuine question, why bother with argparse when you can use torchrun with click?.Seriously, every distributed torch codebase uses argparse, do people just not know click exists or is argparse more robust for some reason?.Sorry if I sounded arrogant AF, i'm genuinely questioning
63
11
273
Here is a recipe that untangles interleaved python tracebacks in multi-gpu pytorch training. 1. add `--role \$(hostname -s): --tee 3` to your `torchrun` command and ` | tee -a log.txt` to the end of it so the log is saved (you may or may not need \ before $ depending on whether
2
2
37
We will continue our TorchRun towards Social justice 🖤❤️. #என்_உயிரினும்_மேலான .#கலைஞர்100 . @dmk_youthwing @mkstalin @Udhaystalin.@Kalaignarnews
0
18
53
Introducing SmolLlama! An effort to make a mini-ChatGPT from scratch!. Its based on Llama (123 M) structure I coded and pretrained on 10B tokens (10k steps) from the FineWeb dataset from scratch using DDP (torchrun) in PyTorch. Used 2xH100 (SXM) 80GB VRAM from Runpod.
17
17
274
As requested I added annotated launchers one can easily adapt to use in their SLURM environment to launch training with @PyTorch torchrun, @huggingface accelerate and @pytorchlightnin . Enjoy
3
20
104
Deputy Guilford & Superintendent Churney have joined our members, volunteers, @RCMPONT, EMDC and Special Olympics Athletes on today's #TorchRun. Thank you to the @londonmajors for inviting our contingent to join you at the game! Shout out to Derek Spence, Dave Payette & Michelle
1
4
34
Our Special Olympians, community members and law enforcement teams finished up today's #TorchRun with a #BBQ in Owen Sound. Best of luck to our Canadian Olympians competing at the world's games in Germany starting on June 17th. #GoCanadaGo #CanadianAthletes #CommunitySpirit
0
2
22
Happy to support @BCLETR at Surrey Headquarters, today is about coming together to support these special athletes. #community #torchrun #SPS #copwhocares
1
4
23
0
8
25
🔥 South Texas Torch Run 🔥. Yesterday, our @TxDPS Troopers and Recruiters helped kick off the Special Olympics Torch Run!. A powerful event that brings pride, unity, and inspiration. If they can do it, so can we. #LETR #SpecialOlympics #TXDPS #InclusionMatters #TorchRun
0
2
6
Day 2! We’re up early supporting the @OklahomaLETR & @sooklahoma. Next leg is from Edmond to Stillwater. The torch called the Flame of Hope, marks the beginning of the 2024 Summer Games! We’re proud to help carry the torch the entire way. #moorestrong #specialolympics #torchrun
2
3
13
