though i think optimising for what’s interesting is good, even if it doesn’t an out (immediately) for a career. i also think about melbourne and nyc a bit too often.

it’s both fortunate and unfortunate that there are some ideas/people/places i can’t get out of my head.

if everyone just had cats, i’d be able to go to sleep right now.

> new neighbor.> dog.> barks

why can't the idea of long-range dynamics/video be learnt *AFTER* pretraining a static observation encoder. this could work as long as your representation of static observations still convey *how* the static scene can change.

why shouldn't we have two things for representing video:.1) static observation encoder.2) dynamics model for describing how things change. instead of:.1) one large ViT that eats everything.

do you actually *need* to shove the entire video through a big model to get good representations for a video?. how much of it is just pure static content, and how much else are things that can change? perhaps we only need to process multiple timesteps for learning the latter?.

i’m not watching MI8 in theatres out of protest bc rebecca ferguson is no longer in the franchise.

just have kids ig?. you don't even need to go full elon with it, you can just produce a few (<15) of your own little cultists (children).

my curiosity got the better of me and tonight i'll be finding out what the words "garlic epic stuffed crust" entail.

goomba lab will be the best lab name until @menhguin starts goon lab.

people found companies because it’s too hard to found a religion or country.

i don't personally advocate for *exact* architecture implementations, but this provides a very different perspective on architecture design. you should go read a lot of yilun's stuff.

the public messaging of this field is sometimes irresponsible. it's overly-optimistic in the wrong places, and insufficiently nuanced in others. likely:.asi will not be here in five years to cure your rare disease. it will however cause economic shifts you aren't prepared for.

RT @MLStreetTalk: AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their….

whenever there’s a fly that’s obviously a bit slow and not in an annoying place (eg my house) i don’t kill it. i hope that it can spread its slow genes to other flies, so i can kill the flies in my house more easily.

seems to be something inherently different in how complexity emerges in finite (ViT encoder) vs infinite (LLM) “working space” designs, and thus how simplicity bias seems to benefit the latter more. we don’t understand how to scale the former purely. the largest ViT << LLM sizes.

the idea of embodied data in robotics is weird. human actions are embodied, but i don’t think a human’s (or an optimal) world model has to be. data should be for learning an (implicit) world model, i don’t know why there’s a tendency to do it from an embodied POV.