Matt Spike
@matspike
Followers
3,029
Following
2,720
Media
1,790
Statuses
14,900
Language/CogSci/Evolution: complexity, conventionality, information etc. @matspike @fediscience .org @matspike .bsky.social
Edinburgh
Joined April 2009
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Cataluña
• 108427 Tweets
つばさの党
• 63187 Tweets
Year 6
• 34131 Tweets
家宅捜索
• 33759 Tweets
Rayados
• 32869 Tweets
#YoureMyTreasure_MV
• 29455 Tweets
Tigres
• 27528 Tweets
We Love Daddy Tay🩵
• 24248 Tweets
POP WITH ZEROBASEONE
• 19927 Tweets
माधवी लता
• 19314 Tweets
Happy New Week
• 17770 Tweets
バンドリ
• 17076 Tweets
InfoMINGGUAN PakPRABOWO
• 17074 Tweets
Lebih KEREN KitaKOMPAK
• 16517 Tweets
ミオしゃ
• 14698 Tweets
MEMORABILIA OUT NOW
• 14039 Tweets
Kamuda Tasarruf
• 13108 Tweets
作者の顔
• 10597 Tweets
AESPA GOES SUPERNOVA
• 10303 Tweets
SATs
• 10299 Tweets
Pinned Tweet
OK then - here is a LINK to a really really long reading list for people who:
- are interested in LANGUAGE EVOLUTION
- are interested in ABANDONING HOPE OF SIMPLE EXPLANATIONS
- like 1970S COLOUR SCHEMES
12
39
180
Ahahaha MY WORD this is such a beautiful result - All Transformer dynamics COLLAPSE TO A SINGLE POINT
AGI is a DIRAC DELTA FUNCTION
Scale is all you need, as long as all you need is an infinitely dense probability mass with zero entropy, too too good
"A mathematical perspective on Transformers" (by Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, Philippe Rigollet):
7
136
792
33
99
930
Now we've finally, *definitively* established that brains are NOT computers, it's time to address the *real* elephant in the room: washing machines - or rather, 'washing machines'. Popular misconceptions aside, dishwashers are NOT dishwashers. 1/
11
82
427
HAHAHA lucky this never happens in cogsci or linguistics or HAHAHAHAHAHA Artificial Intelligence HAHAHAHA research HA everything is FINE HA HA
In the 1970s, PhD programs in economics in the US abandoned the requirement that students take a course in the history of economic thought. Steve Medema’s take in 2009 on the costs to the profession of that decision:
38
423
2K
13
68
408
Imagine the smartest potato you can think of.
Not so smart, huh?
Now double its smarts. And again.
...
23 times
Now your potato is as smart as the MOON
this is why we fear the future
12
48
325
This nice 'oh shit' story from John Maynard Smith may be the perfect demonstration that - even if you really are the kind of person who goes off one day and founds the whole discipline of evolutionary game theory - you should never underestimate who you're talking to
5
76
311
Why do linguists just assume mental representations for language when neuro folk think that's whack? OK, in turn:
1. Linguists agree on *nothing* & often seem to (or act like they) really hate each other
2. Language is WILDLY complicated & variation is profound
... 1/
13
45
247
Which academic disciplines, in your opinion, have a *surprisingly* high proportion of assholes?
Not talking econ/medicine or Bay Area cultists or moral & political phil here, everyone knows about them, I mean something like *toxic botanists* or *the dark side of Mayan epigraphy*
52
10
218
@bayesianboy
Thinking of Betty Shannon, who typed up all of Claude's manuscripts as well as, erm, writing quite a lot of them...
0
43
203
*pauses to contemplate the full sweep of human history and evolution, our material culture bending nature itself to our will, the infinite ocean of our cultural inheritance supporting and shaping every aspect of our collective and individual minds*
"Anyway, yep! 8 megabytes!"
30
32
202
NOTE to New People who may not know much about:
- statistical physics &
- measure theory &
- category theory
- etc
BUT want to ARGUE with *ME* about them
You're making a BIG MISTAKE
You picked the WRONG GUY THIS TIME
I know even less than you about these things. wtf man, why?
6
9
170
@ArYoMo
If it holds in general:
1. Many argued that by continuing to scale up training, we'll hit a regime of exponential improvement.
2. Lots of others predicted we'd hit diminishing returns soon or later.
3. Nobody argued for *this*, that scaling benefits will plateau and *reverse*
12
6
150
As Fodor famously argued, the brain is *literally* a computer and your teeth are the keyboard
5
26
147
At OpenPotato, we believe it would be unintuitively risky and difficult to stop the creation of the Doomsday SuperPotato.
Do exploding napalm potatoes pose a real and imminent existential threat to humanity? Yes - but via our responsible stewardship of a global licensing system,
1
49
140
Can Large Language Models THINK and UNDERSTAND? The answer from cognitive science is, of course, lol wut YES!
The more interesting question is CAN TOASTERS LOVE? Intriguingly, the answer is ALSO YES! And they love YOU
12
16
139
What happens if an evolutionary linguist publishes an article about the philosophy of cross-linguistic categories and gives it a pretentious title? Where can you find Samoan nouns, Genghis Khan, and dodgy thought experiments all in one place? Right here!
2
32
126
Nearly 3 million of you have not spoken, so it’s consolation prize time and I will now complain about the ‘innateness’ debate in linguistics, why it’s a waste of time, and how everybody is wrong. I’ll throw a few tired gifs to signal my age-linked lack of cultural relevance.
If this tweet gets less than 3 million likes I will post a grumpy thread about the concept of innateness in language and why everyone is wrong.
6
2
44
5
29
127
One of many ways I annoy people is saying that *arguing* about dynamical vs computational cognition, or mechanism vs function, or internalism vs 4e stuff... is *a stupid waste of time*
This 1966 paper by Levins is a big reason I have this annoying take
11
20
127
Is the brain a computer? Well, yes and no - it's not just *any* computer!
It's an Amstrad PCW512, originally targeted at the word processing and home office market but, with the expansion to 512k of memory, made the big move into cognition in 1987
7
21
122
Having immersed myself in the teetering stack of shoeboxes held together by chewing gum and malice that constitutes the current state-of-the-art in 'A. I.' , I can only conclude that anyone arguing "AGI is here" must be motivated by either money or a passion for sniffing glue
10
32
115
Anyone in Physics ever looked into the organisation of atoms and stuff in matter? Different phases? "Condensed" in some sense? Maybe scales?
I reckon we could model all that absolutely unproblematically using analytic tools for the study of human languages, culture, and society
8
23
112
If you think terms like ‘word’ and ‘language’ are folk-theoretic or don’t really mean anything, and/or people are working with the wrong definitions for them, then it’s worth checking out some rock-solid scientific concepts like ‘species’, ‘gene’, or ‘planet’
5
20
113
Mind blown? You'll be scraping your brains off the wall when you hear about *language*
Programming is thinking + syntax. Maybe 90% thinking and 10% syntax. While you can automate away syntax, you can only ever outsource thinking to someone/something that can think. Attempts to outsource thinking to a syntax generator do not end well.
67
400
3K
3
12
101
AHEM. Formalists don't even formalise anymore and what the hell kind of set theory does merge come from anyway; functionalists are wildly unconstrained and throw just so stories around like confetti; semanticists sure we do lambda calculus in our heads please buy my magic beans..
2
16
97
Narrator: we cannot, in fact, say the same about human vision
LLMs are glorified autocompleters. We can say the same about human vision. Here's a demo of autocomplete mode!
139
426
4K
6
12
95
BREAKING
- Symbol-Grounding Problem SMASHED by gluing camera to baby's head
- PLATO? more like K.O.!!! - neural net babycam SECURES ETERNAL VICTORY for empiricism
- Nobel committee announces PRIMACY OF THE SENSES: This New Pope Fuckin Loves Science!!!
4
14
90
This is, without question, by *far* the best insult I have received on this platform. Tempted to put this line on my CV
7
8
88
Laugh all you want, some of us have been preparing for the Labradoodle Singularity for multiple dog-generations
5
22
89
Fodor had a genius for identifying foundational, crucially important problems in cognitive science, then painstakingly arguing for a way to resolve these problems, and finally packaging it all up as something so superficially outrageous that most people dismiss it out of hand
4
12
89
"an image of an attractive academic secretary listening to the new audio version of her mature but attractive mentor's amazing new book on utilitarian philosophy NO VISIBLE DEFORMITIES"
5
8
87
Would anyone like an extremely long reading list on evolutionary explanations of language? Long enough to crush all hope of any nice simple explanations for all that stuff?
8
9
88
30-50 feral hogs expect human-level AI (AGI) within the next 30-40 years (or much sooner)
If you don't agree, you need to explain why your views are more informed than 30-50 feral hogs - it's irrational for you to assert with confidence that you know better than 30-50 feral hogs
5
13
84
Word sucks, but using LaTeX for presentation slides for anything that's not code/math is pure STEM identity signalling. Beamer looks like shit and Tikz is a preposterous waste of time. LaTeX posters tho? It's an archaic markup lang wrapping a bizarre macro zoo, not a design app.
7
2
81
Evolutionary Psychology is NOT Pseudoscience.
It is a *personality defect*, which cannot be a science. This is the Demarcation Criterion at work.
5
12
83
Some revanchist killjoys insist AI will never possess human traits like body odour or a penchant for cheese - but there's every chance machines will infinitely surpass us in these regards - possibly in only 2 to 5 years from now
5
17
79
Can Social Psychologists Stop Sniffing Glue? Or I guess it could be a TiCS-themed troll, in which case the bit about priors is a masterpiece, kudos
11
10
79
another pattern I see:
- Bay Area Thought Leader: it occurs to me that X is a profound and deep concept
(where X = life/thought/computation/truth etc)
- World expert in X: that's actually a great question with a 5000 yr history, you see
- BATL: back off Poindexter, read the blog
6
9
77
Super important implications for understanding the way LLMs represent natural language: first-order logic has massive limitations on its expressivity, so the strategies employed by LLMs are by definition inefficient and incapable of many, many abstract properties seen in language
📣
@Ashish_S_AI
and I prove that transformers can be translated to sentences in first-order logic with majority-vote quantifiers (FOM).
FOM is a symbolic language that can capture computation inside transformers!
12
95
490
6
17
78
This is pretty extraordinary
1/10 Ten years ago, I received a handwritten manuscript by Alan Turing’s only PhD student, Robin Gandy. He wrote it a couple years before passing away in 1995.
AFAIK, it’s never before been in print, so I typeset my copy & put it online here:
🧵👇
15
126
562
3
11
75
More of this kind of thing please: would love to see intros to philosophy of science *for* cognitive scientists and linguists etc, we've got more than enough of the *of* for the time being ffs.
And if we're really dreaming big, a book for AI/ML people, that they actually read
As it happens we have a book exactly for this purpose (for students in the life sciences):
4
38
181
5
13
77
Seen a few Tech Idea Havers wheeling out *Wittgenstein*, of all people, to defend The ChatBot's meh relationship with truth by saying, like, what is meaning anyway man, it's a *Language Game*
But for Big W, an LLM is *incapable* of meaning, which emerges from lang's human context
7
12
75
-Hilbert: "algorithms will solve math"
-Gödel: *breaks math*
-Church/Turing: *(re)invent comp sci, break it*: "no"
-Comp Sci: *stares into abyss, decades of heroic struggle*
¡¡A deep net appears!!
-Some SF Yoga Nazi: "A really big algo tho"
-World: THIS CHANGES EVERYTHING
1
15
73
THAT'S FKIN IT, THAT'S THE LAST STRAW, I give UP, what's the frickin POINT with these people
9
7
72
I'll tell you what - from the heart, it truly is a great time to not be working in the cognitive science of language
6
5
70
Wait a minute... I'm aphantasic (no mental visualisation), and get that people can see pictures of apples or whatever in their heads. I still thought that 'mental time travel' was a metaphor though - but you people literally *re-experience* the past? Like some immersive VR trip?
15
6
70
I see people dunking on this, but it's worth recognising that number 2 here is literally Chomsky's stated position. With that in mind, resume dunking
8
0
67
When will people finally understand that linguistics isn't about learning languages and word order and stuff, but a sophisticated modern discipline which gives us the tools to attack other so-called 'linguists' in print for ruining the discipline from inside, the bastards.
Like, from the outside, linguistics has always seemed kinda easy? Are they finished yet? There aren't actually *that* many languages.
46
15
395
2
4
65
damn, my money was on super-human powers of inductive inference from the architecture with a representational capacity equivalent to first-order logic with majority quantifiers
New paper by Google provides evidence that transformers (GPT, etc) cannot generalize beyond their training data
254
866
5K
3
13
64
I hope that's settled things for good. Once again, once we remember that the world is essentially a simple place, and our job as scientists is to find the unique, best, and universally applicable definitions for everything, the pieces all fall into place. You're welcome! /end
4
3
63
just speaking personally here but if I somehow found myself in the position of defending the right of militarised cops to assault my academic colleagues, I would maybe sit back and have a long hard think
1
5
62
"my cat talks to me"
- lights-out time grandpa
"space lasers control me"
- intervention time again
"the language model can think"
- your tenure package is approved
"the water bucket uses logic to hear me"
- humanity shakes head, gives up
2
10
60
Why do philosophers have *such* bad citation practices? Did they *all* miss that day of grad school training? Is there some initiaition oath to pay for each citation with a pint of blood?
9
4
61
Map bursts into tears.
Says - "But doctor, I *am* the territory"
5
16
60
Hurrah Chomsky debate. Anyway, my working theory today is that generative types and whatever my type is engage with totally different Chomsky literatures. They read the papers with trees and stuff in them, we read the papers where he calls us stupid assholes.
4
6
58
Max Bennett on the brain inspired pod basically argues that AI alignment implicitly requires a solution for the frame problem. So, looks like another foundational-level hurdle that needs to be solved for (and for which we have no idea what the possible solution space looks like).
3
0
12
2
10
59
GODSPEED
Introducing General World Models.
We believe the next major advancement in AI will come from systems that understand the visual world and its dynamics, which is why we’re starting a new long-term research effort around general world models.
Learn more:
116
392
2K
0
13
56
It's all really interesting. Everyone in cogsci & neuro & philosophy should probably know that the idea of 'a word', or the existence of verbs & nouns, or really *any* ling commonalities are huge controversies in linguistics - HEY LOOK MY PAPER NO WAY 7/
2
0
54
@spiantado
Nah, that's *shitty* phil work (mountains of irrelevant analytic phil of lang I am looking at you), but the really best stuff, particularly in philsci explicitly targets the inescapable ambiguities between formalism and interpretation, and is an essential feature of good science
1
0
53
@IrisVanRooij
Noether's theorem is so deep it should count as philosophy. Of course, to say she was treated badly by the academic establishment would be *hugely* understating it
5
9
53
Science needs more Hedvigs and way fewer people who want to pretend that hyper-complex webs of historical socio-geographic contingency are kinda like a pendulum or look my theory has a nice tidy circle loop
everyone keeps being horrified by my DAG to represent previous literature on the topic of predicting number of languages per islands in oceania, but hey. reality is probably complex ^^
1
6
71
1
4
51
- If you're not a card-carrying affordance-invoking 404-E hyper-empirical autopoietic anti-representationalist when you're young, you have no soul;
- If you're not a fire-breathing rational-ant-step-counting Language-of-Thought computationalist when you're old, you have no mind
6
14
52
good grief people - let me open my Big Book of Established Facts about the nature of language and cognition OH THAT'S RIGHT IT'S EMPTY
you know nothing, I know nothing, a really big array of weights definitely knows nothing, the only competition should be who's *most* wrong ffs
4
5
50
OK:
1. Hateful Linguists: this is getting much better but was until recently dominated by an international circus of wrathful man-babies in an endless game of walrussy macholectualism. This wasn't great for field-wide consensus or, um, the field. See ⬇️ 2/
1
3
50
I am ONCE AGAIN asking how ON EARTH one might explain linguistic structure without any appeal to mental representations
13
1
49
Kim was once preparing his notes in the front row of a public lecture when he was asked to leave by security - he said "No worries! But before or after I give the lecture?"
Michael Ruse on the dress sense of philosophers of biology and meeting Kim Sterelny for the first time.
MR. 1995. Booknotes. Biol Philos 10: 249–254.
4
0
25
4
1
49
I'm going to laugh for a long long long time if the whole gamut of modern deep learning models turn out to be group lasso regression and a hashing function in a trenchcoat
1
9
48
More bad news for free-lunch aficionados
Can we learn from time series data a dynamical systems model that *generalizes* to unobserved dynamical regimes (basins of attraction), like a good scientific theory should?
Out-of-domain generalization in
#DynamicalSystems
reconstruction:
#AI
#ML
(1/3)
8
95
449
1
10
47
Congrats to the machine learning community for their discovery of not only automata theory, but also these exciting first steps towards the uncovering the halting problem.
3
5
48
This paper ('Why is scaling up models of language evolution hard?') has drawn a few similar responses now, i.e.: 'It isn't, we did it'. That's... a good point, so I'll have a crack. 1/n
@mariekewoe
#ronalddehaan
#toddwareham
@IrisVanRooij
@MarkBlokpoel
This paper asks how models of language evolution can scale from toy scale (as in existing simulation studies) to ecological scale (as in real-world linguistic contexts). It observes a theoretical obstacle, which is conveniently summarised in the screenshots.
2
16
34
1
21
47
I am not now, nor have I ever been, interested in the 9 books Sam Altman thinks everyone should read
5
3
46
Bob flips a coin a million times, but never speaks. Alice does not flip. There are no heads, no tails. Bob is here, flipping, over and over forever. Where is Alice? Nobody knows. Just the sequence, flips, and Bob, no heads and no tails, & half-remembered Alice, who would not flip
4
6
46
*Shockingly*, the concept of ‘innateness’ turns out to be extremely complicated and potentially not very useful at all. A really nice survey from Mameli & Bateson 2006 finds 26 different ways in which the word ‘innateness’ has been used in the literature.
1
9
47
*ahem*, washing dishes is (was) a *human activity*, and not a machine at all. When our forebears washed dishes, it embodied a complex dance of physically enacted, dynamical interactions with the dishes, the dish-relevant ecological environment, and sometimes rivers (possibly). 5/
3
2
46
Pretty picture, but essentially meaningless without any benchmark scores or parameter counts
3
3
45
17 years of chaotic globetrotting and still no long-term plans in sight - let the good times continue to roll ❤️
@ozgespike
2
1
45
Petition to resurrect Fodor on a strictly temporary basis so he can shout at everybody until they feel stupid and go home
5
2
45
What is the most detailed *definition* in the form "Language is X..." which won't trigger *any* arguments between linguists? Best I can do is something like "Language is problematic"
12
6
44
3
5
42