ZILIANG Peng 🏴☠️
@cyberpengk
Followers
3K
Following
50K
Media
3K
Statuses
40K
AI means love. Robotics. Graphics. Economics. Physics. and random topics.
Promised Land
Joined July 2009
It is super early days for GPU hacking. And it's not just GPUs.
The latest MLX has a CUDA back-end! To get started:. pip install "mlx[cuda]". With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses
0
0
1
and caught up with the sglang cache-aware routing design. this primarily routes based on perfix overlap, and each worker process the full range of text. i still think there should be a routing algorithm that squashes a sub range into each worker to maximize prefix overlap in a.
0
0
0
Spent a whole day reading transformer impl on Tinygrad and distributed LLM inference hmm.
0
0
0
looking to implement a minimal / educational llm inference engine on top of tinygrad, but pounders the question: . what optimization should be implemented by hand, and which part will be supposed to be optimized by @__tinygrad__ itself?.
0
1
18
Doing things produce information, just keep doing things.
0
0
0
Can it not be called "AskGPT"?. What else do people say beside asking questions?.
0
0
0
Went into Tesla service center for a leaky tire replacement, tells me Tesla lowers prices for tires as "needs to be competitive".
1
0
1
yacine posts get infinitely better since unemployment.
Just because someone has never done it it doesn't mean it is too hard to do. It just means that no one thought they could do it. No one tried. Once you try you'll surprise yourself. If it's possible in principle it is possible.
0
0
0