cider
@jeffreycider
Followers
3K
Following
10K
Media
116
Statuses
763
my purpose in life is to forget linear algebra 2x a year
San Francisco, CA
Joined September 2019
linear transformations stretch euclidean space ReLU folds euclidean space neural networks are just repeated origami on high-dimensional laffy taffy
8
36
469
i love how jianlin su blogs really convey the feeling of working in optimization theory
Why does linear attention need Short Conv? https://t.co/luUybG3RXj
0
0
11
optimization theorem: "assume a lipschitz constant L..." the lipschitz constant:
[1/9] We created a performant Lipschitz transformer by spectrally regulating the weights—without using activation stability tricks: no layer norm, QK norm, or logit softcapping. We think this may address a “root cause” of unstable training.
10
32
520
i'm at a ~serious~ lab people explain things to me in chinese, and i have to tell them in english that i'm a banana
5
0
98
> i love math so much, i hope my new job involves a lot of it monkey's paw curls
7
5
274
first chess tournament in 17 years i was so scared of getting demolished by a 2nd grader but joke's on me my elo was literally not high enough to get paired with any children
7
11
658
my friends don't want fuck you money per se. bc they don't want to say fuck you to people what they really want is plot armor money
0
0
32
one week ago: idk fp8 quantization seems pretty straightforward 7 days later: do you know what continuity is? it's the thousand blades of dedekind cuts, a story we agree to tell each other over and over, until we forget that it's a lie.
3
3
70
adjusting my MoE width so my attention heads harmonize better with the humming of my MLPs
1
4
104
fated to utter true prophecies, never to be believed
bc they forgot to read chinchilla and are starved of tokens. fast takeoff is just a sperm whale with an audible subscription
2
1
34
sleep evolutionarily predates brains sleep might be more fundamental than brains
2
0
32
modern homeopathy is discovering that refilling a hint water bottle with tap water creates more hint water
3
0
75
spinor geometry is so traumatizing that math youtube has to schedule therapy breaks into their videos
5
1
52
help she's too good at abstract algebra does there even exist meaningful consent across a mathematical maturity gap
7
0
129
the math suggests that hippogriffs are possible
1
2
55
nominative determinism runs everything around me apparently the heaviside step function is named after some dude named oliver heaviside, and not just because it has ... a heavy side ...?
8
0
57
bc they forgot to read chinchilla and are starved of tokens. fast takeoff is just a sperm whale with an audible subscription
if scale is all you need then why are sperm whales (20 lb brain) dumber than humans (3 lb brain)?
2
0
71