My matrix multiplication (written in plain zig) achieves performance on par with Intel's MKL for my CPU! . And I will release a blog on how I got here soon :)

repost cause Brandon got autocorrected to bryan.

if you're into writing I highly recommend watching Brandon Sanderson's videos on yt, especially his lecture series

@Aryvyo it as in a language model.

cc @Aryvyo I should make it play HSR. .

also in picture, the fact it says "no hard feelings" is crazy.

It would be really interesting to see how unavoidable is strong survival instinct without adversely affecting model performance, given resource optimization is integral to many coding related tasks. also I really wanna have a minecraft pvp environment for models.

quick iteration is best iteration.

RT @tensorqt: attention sinks may be a bias in causal transformers. as some of you know, i've been writing a long blogpost on attention a….

how can they go so hard.

after some painful rewrites for the trainer I had to go back to being correct first.

cooking cooking cooking

the chance of an unlucky chunk crashing your hnet run is low but not zero.

also, could have a context annealing effect, temperature goes up with context length but keeps attn entropy stable ; more decisive model with more seen ctx.

log scaling again, i presume probably for gradient stability and better interaction with the softcapping.

scale scaling probably for long context attn? along with the attn logits being capped at 30.0

RT @YinpeiD: Wow 🤯 DINO v3 features aren’t just good at multi-view consistency and objectness… they even capture shadows and cloth folds! h….

most cited data is probably bad data . never ever trust a single source of data.

RT @tenderizzation:

okay CuTe DSL is actually kinda goated.

everyone complains about CPU compilers.