templar
@tplr_ai
Followers
2K
Following
445
Media
20
Statuses
344
incenτivised inτerneτ-wide τraining
subnet 3
Joined March 2025
2 days into #CCloco , and miners are already converging 7.5 times faster!. "We are yet to see a task well incentivised miners could not achieve". We plan to push them much harder. #Accelerate
10
19
113
RT @const_reborn: I mean, you need to come up with an abstract incentive layer and reinvent all of torch distributed to work on block time….
0
2
0
Anyone in the world is free is join now, no queues, no waitlists, only incentives. Join us:
0
0
0
The Templar Protocol v1.0.0 represents the convergence of our work on incentives, permissionless systems, and communication efficiency.
0
0
1
This isn't just research – CCLoco powers TEMPLAR II right now, proving that decentralized AI training is both economically viable and technically competitive.
2
0
2
The results are transformative. Nodes can now process 15x more data between synchronizations. Contributors with standard internet connections can participate effectively.
1
0
0
Second, we chunk weight tensors into 64x64 blocks before compression, improving efficiency and reducing scaling issues. Third, we discovered that DCT transforms actually harm performance in multi-iteration settings - a counterintuitive finding that unlocked major improvements.
1
0
0
Three key innovations made this possible. First, we removed DiLoCo's outer momentum, which conflicted with compression error feedback.
0
1
10
By combining local multi-iteration training with advanced gradient compression, we've achieved a 50x reduction in communication overhead. CCLoco needs just 2.8 GB while maintaining model quality.
2
2
11
Our previous approach with DeMo compression took too long between synchronizations, limiting scalability. Other methods like DiLoCo still sent 114.3 GB per training run - impractical for global collaboration. CCLoco (Chunk Compressed DiLoCo) is our answer.
2
1
9
The challenge was clear: training LLMs across the internet required exchanging massive gradient updates between nodes. Even with our permissionless infrastructure, communication costs were the bottleneck limiting true decentralization.
1
0
8
Now, with CCLoco, we've conquered the final frontier: communication efficiency. CCLoco is already powering TEMPLAR II, our latest production training run .
1
0
7