Rowan Zellers
@rown
Followers
14K
Following
1K
Media
36
Statuses
565
multimodal @thinkymachines. I also like to climb rocks and throw pottery. https://t.co/5Er4j39K71 (he/him)
San Francisco, CA
Joined November 2008
Excited to introduce GPT-4o. Language, vision, and sound -- all together and all in real time. This thing has been so much fun to work on. It's been even more fun to play with -- with moments of magic where things feel totally fluid and I forget I'm video chatting with an AI.
22
39
370
One interesting "fundamental" reason for Tinker today is the rise of MoE. Whereas hackers used to deploy llama3-70B efficiently on one node, modern deployments of MoE models require large multinode deployments for efficiency. The underlying reason? Arithmetic intensity. (1/5)
Introducing Tinker: a flexible API for fine-tuning language models. Write training loops in Python on your laptop; we'll run them on distributed GPUs. Private beta starts today. We can't wait to see what researchers and developers build with cutting-edge open models!
15
68
830
Tinker provides an abstraction layer that is the right one for post-training R&D -- it's the infrastructure I've always wanted. I'm excited to see what people build with it. "Civilization advances by extending the number of important operations which we can perform without
Introducing Tinker: a flexible API for fine-tuning language models. Write training loops in Python on your laptop; we'll run them on distributed GPUs. Private beta starts today. We can't wait to see what researchers and developers build with cutting-edge open models!
49
116
1K
Introducing Tinker: a flexible API for fine-tuning language models. Write training loops in Python on your laptop; we'll run them on distributed GPUs. Private beta starts today. We can't wait to see what researchers and developers build with cutting-edge open models!
220
761
6K
In many finetuning settings, LoRA can match finetuning all parameters
LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.
3
2
136
LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.
81
549
3K
I wrote this blog post that tries to go further toward design principles for neural nets and optimizers The post presents a visual intro to optimization on normed manifolds and a Muon variant for the manifold of matrices with unit condition number https://t.co/EhhKN2Jylx
Efficient training of neural networks is difficult. Our second Connectionism post introduces Modular Manifolds, a theoretical step toward more stable and performant training by co-designing neural net optimizers with manifold constraints on weight matrices.
23
52
465
Looking through those little hidden gem stories in the footnote, you will find it so inspiring that researchers with interests on the same topic are able to work together to advance a field despite their roles and locations. This is the power of open science and community.
Efficient training of neural networks is difficult. Our second Connectionism post introduces Modular Manifolds, a theoretical step toward more stable and performant training by co-designing neural net optimizers with manifold constraints on weight matrices.
11
40
762
Efficient training of neural networks is difficult. Our second Connectionism post introduces Modular Manifolds, a theoretical step toward more stable and performant training by co-designing neural net optimizers with manifold constraints on weight matrices.
118
463
3K
Apologies that I haven't written anything since joining Thinking Machines but I hope this blog post on a topic very near and dear to my heart (reproducible floating point numerics in LLM inference) will make up for it!
Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference” We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to
73
211
3K
Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference” We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to
240
1K
8K
Halo and Noa: Price check The little reminders help. The ones that stop me from sleeping on the couch are better❤️
1
11
46
Always amusing to overhear machine learning discussion in the mission cliffs sauna
1
0
41
this link seems to have the context https://t.co/LQ6si4D0mh but the links don't work anymore, instead I stumbled upon it by entering an address and clicking backwards on the year
maps.googleblog.com
Editors Note: Today’s guest author is David Rumsey, President of Cartography Associates and founder of the David Rumsey Map Collection ...
0
0
6
TIL - you can view San Francisco as it was in 1938, on Google Earth - Highway 101 hadn't been built yet, there are still houses shown - Valencia St still has streetcars - A baseball stadium is where the strip mall was at 16th and Bryant - Much of the sunset is still dunes
4
3
70
From Megawatts to Market Momentum: Terawulf’s Playbook, Biotech, Real Estate and Small Caps
4
7
45
Excited to share that I joined @thinkymachines recently! It’s been an incredible experience so far working alongside many talented folks here. We are building multimodal AI that are collaborative with human, as well as a great research infra to accelerate AI and science!
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're
22
8
324
We are moving incredibly fast. Come light up GPUs with us.
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're
12
12
347
Yes - 🥳 Thinky starts hiring again:
thinkingmachines.paperform.co
For people who are interested in opportunities at Thinking Machines Lab.
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're
19
23
577
We have been working hard for the past 6 months on what I believe is the most ambitious multimodal AI program in the world. It is fantastic to see how pieces of a system that previously seemed intractable just fall into place. Feeling so lucky to create the future with this
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're
14
19
342
Silence is g̶o̶l̶d̶e̶n̶ electric. A look back at Midnight's highly-anticipated flights at the Salinas Airshow this weekend showcasing smooth, quiet and thrilling electric flight. Despite reaching speeds upwards of 125 MPH, the lack of noise coming from Midnight was the
35
142
866
It’s really fun to work with a talented yet small team. Our mission is ambitious - multimodal AI for collaborating with humans, so the best is yet to come! Join us— or fill out the application below if interested!
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're
8
7
138
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're
653
703
8K