Marco Ciccone @ ICML 🇨🇦
@mciccone_AI
Followers
881
Following
19K
Media
53
Statuses
1K
Postdoctoral Fellow @VectorInst - Collaborative, Decentralized, Modular ML - Competition chair @NeurIPSConf 2021, 2022, 2023 - PhD @polimi ex @NVIDIA @NNAISENSE
Toronto, Canada
Joined April 2015
🚨 Life update 🚨 I moved to Toronto 🇨🇦and joined @VectorInst as a Postdoctoral Fellow to work with @colinraffel and his lab on collaborative, decentralized, and modular machine learning to democratize ML model development. Exciting times ahead! 🪿
12
4
104
Nice parallel!.
ML labs seem to have quite the parallels with F1 teams. Over the top of my mind:.
0
0
1
Dan’s talk was a masterclass — Go watch the recording. Super clear, packed with results, and genuinely one of the most well-delivered talks I’ve seen in a while.
Happening in 30 minutes in West Ballroom A - looking forward to sharing our work on Distillation Scaling Laws!
1
0
6
RT @ffabffrasca: 6/. @JoshSouthern13 and I be at #ICML2025, poster session Tuesday — stop by and chat if you're around!. I would also b….
0
1
0
RT @Ar_Douillard: I'll discuss distributed learning on Saturday, July 12. First, I'll cover current methods needing high bandwidth, then n….
0
12
0
Even crazier is that the rmsprop algorithm was introduced by @geoffreyhinton in his neural networks course, and we cite his slides - so it's fine!
@samsja19 @kellerjordan0 even crazy part is there's no muon paper.
1
0
2
Fantastic work from @allen_ai - asynchronous training of MoEs on private datasets and a domain-aware router. Akin to cross-silo FL, but no synchronization and no fear of heterogeneity anymore on LLMs. Nice, clean, and of course modular!.
The bottleneck in AI isn't just compute - it's access to diverse, high-quality data, much of which is locked away due to privacy, legal, or competitive concerns. What if there was a way to train better models collaboratively, without actually sharing your data? . Introducing
0
2
18
RT @roydanroy: The Department of Statistical Sciences in the Faculty of Arts and Science at the University of Toronto invites applications….
0
11
0
Check the videos of our workshop here 👀
The videos of the workshop are now online!.
0
0
1
🚨 Missed our #ICLR2025 workshop on modularity & collaborative learning? If you are interested in these topics, get in touch! .
1
1
2
Heading to #ICML2025 in Vancouver 🇨🇦 July 13–19!. 🚀Ping me if you want to discuss research and collabs on how to build better LLMs via:. • Model merging & routing.• Continual & modular learning.• Distributed and Collaborative Learning
1
5
14
This is probably the most compelling result of the paper. Clear improvements at scale — 1K, 10K, and 40K clients, beyond synthetic benchmarks like CIFAR. You know what that means.
1
2
6
Check the full thread 🧵here
Do you feel FL research is stuck with methods that do not work well in realistic scenarios? 🤔. 🫵We got you!.Introducing 🚀Generalized Heavy-Ball Momentum (GHBM)🚀, accepted at TMLR:.the FL algorithm with both SOTA theoretical guarantees and much better empirical results. 🧵1/9
0
1
2
The timeline was (much) longer than we'd hoped, but the technical depth we gained along the way was definitely worth it. Honestly, I'm not sure I would have had Riccardo's persistence to see this through. Nice work!.
1
0
0
We didn't know enough about optimization for FL at that point! We discovered tons of related works and spent so much time trying to position our contribution and trying to explain why it works so well.
1
0
0
Early in his Phd, @RickZack96 showed me some experiments of a preliminary version of GHBM applied to a different problem, and I remember pushing him to be ambitious and turn it into an actual FL algorithm.
What began as a hunch became a path I’m proud of—this journey through FL & optimization has been rich with growth and purpose. Deep thanks to my PhD advisor @masone_carlo, and to Sai Praneeth Karimireddy & @mciccone_AI—your guidance lit the way. The best is yet to come.
1
0
0
Faster local optimization by recycling past outer gradients! . Incredibly proud of this work and collaboration. We have a bunch of ideas in the pipeline, looking forward to the next one with @RickZack96!.
Do you feel FL research is stuck with methods that do not work well in realistic scenarios? 🤔. 🫵We got you!.Introducing 🚀Generalized Heavy-Ball Momentum (GHBM)🚀, accepted at TMLR:.the FL algorithm with both SOTA theoretical guarantees and much better empirical results. 🧵1/9
0
2
3
Despite the impressive quality of video generation, what I don't understand about #Veo3 is how it is possible to create such models without infringing on copyright. Did they make a deal with the movie industry?.
0
0
2
Sadly, I won't be able to travel and attend our Modularity workshop :( I'm sure I'll miss many amazing discussions and talks. Luckily @Ar_Douillard and @derylucio will be there to hold down the fort! Proud to be part of the organizing team - can't wait to meet you all next time!.
the workshop is tomorrow!. and it will be livestreamed at for those not in Singapore.
0
0
6