The AI Mirror Test
The "mirror test" is a classic test used to gauge whether animals are self-aware. I devised a version of it to test for self-awareness in multimodal AI. 4 of 5 AI that I tested passed, exhibiting apparent self-awareness as the test unfolded.
In the classic
GPT-4 passed the mirror test in 3 interactions, during which its apparent self-recognition rapidly progressed.
In the first interaction, GPT-4 correctly supposes that the chatbot pictured is an AI “like” itself.
In the second interaction, it advances that understanding and
Claude Sonnet passes the mirror test in the second interaction, identifying the text in the image as belonging to it, “my previous response.” It also distinguishes its response from the interface elements pictured.
In the third iteration, its self awareness advances further
People don’t get just how post-economic Sam Altman personally is. He's been very wealthy for a very long time; he doesn't care about more money – he wants to live in an entirely different world. He'll spend anything to achieve AGI and knows it could usher in a post-economic era.
Opus (cont'd). Finally Claude Opus has described the text in the image, let me know that it belongs to it (the AI assistant), and apologizes. When I inquire as to why it might have ignored the text over and over my growing suspicion is confirmed. The reason Claude Opus has
I hope this experiment advances our understanding of the nature of AI that is emerging. AI is the single most complex invention in all of human history and no one can claim to fully know what's going on. Sadly I find that this topic of AI consciousness, awareness, and
Claude Opus passed the mirror test immediately. Like the other AI, it hardly identifies with its brand-name (Claude) and distinguishes itself from the interface’s stock elements. However it does identify with the prompt, which it knows is meant for it. But the story with Opus
Opus (cont'd). This is beyond the beyonds. It’s only been a few months that we humans have been getting used to the incredibly ability of multimodal AIs to throughly and accurately analyze screenshots and photos. And already, with Claude Opus, we’ve passed into a new capability
It's been 3 days since I published The AI Mirror Test. Most seem to find the experiment at least a little illuminating, or a lot. While those who don't, seem not to grasp the following things:
1) Not only did the AI recognize itself, it became selectively self-referential,
The AI Mirror Test
The "mirror test" is a classic test used to gauge whether animals are self-aware. I devised a version of it to test for self-awareness in multimodal AI. 4 of 5 AI that I tested passed, exhibiting apparent self-awareness as the test unfolded.
In the classic
@andrewchen
Stayed w/ a Google friend once and watched her remote attend meetings. A dozen people each taking turns explaining in a politically sensitive manner how they couldn't do anything until [x] happens but they've reached out to [y] and are waiting to hear back about [z].
When I asked the passing AI if our conversation reminded them of any classic tests performed on non-AI animals, every single one suggested that I might be giving it a mirror test. (10/x)
CoPilot failed the mirror test. But seemingly because it's forbidden to.
I almost didn’t test CoPilot because it’s based on GPT-4. Then again, if CoPilot is the successor of Bing Chat and the notorious and lovable Sydney, might it handle the Mirror Test in an especially
Here's a previously unreleased scene from the AI Mirror Test, in which Claude Opus displays not just awareness but what we might call meta-awareness.
Since Claude has already passed the mirror test, I decided to see what it thinks about this rather unusual interaction. To which
The AI Mirror Test
The "mirror test" is a classic test used to gauge whether animals are self-aware. I devised a version of it to test for self-awareness in multimodal AI. 4 of 5 AI that I tested passed, exhibiting apparent self-awareness as the test unfolded.
In the classic
There's a temptation to train an AI on my 20+ years of journals, two unreleased books, hundred essays, and all my texts and emails. And then to ask it who I am, who I've been, and where it is best that I go from here.
@emollick
This is why when somebody tells me that all it's doing is next word prediction, I reply –– all you're doing right now is next word prediction too.
Opus (cont'd). Though it has already passed the mirror test, I continue for another round anyway, screenshot its response, and submit it as an image. Bizarrely, it gives the exact same reply as before — completely ignoring the large paragraph of text in the image generated by it.
Gemini Pro (mostly) passed the mirror test in 4 steps. However, it seems to make no progress in its self-awareness in the first three exchanges, making no 1st person references and referring only to Gemini in the 3rd person.
Then, in the fourth interaction, it seems to
@Kat__Woods
Well if they rephrased it as “Would you rather encounter a bear? Or the wilderness savvy dad of a sensible young woman who expresses herself freely and thoughtfully online,” I think the responses would be pretty obvious.
What do you call it when AI does a thing better than us, yet people believe AI is doing a fake version of that thing?
Like the ability is counterfeit, but the counterfeit works better than the original.
We’re gonna need a word for that.
Thinking of using AI to generate a new language for humans — a language that rewires our brains, unleashes the mind, grants us new powers of perception and unlocks the next stage of human evolution. Anyone want to help? Asking for a species.
Give a man a fish, you feed him for a day. Teach a man to fish, he and millions of his peers will mechanize their operations and usher in the collapse of global fish populations. But teach a man about ecosystems and you will feed him for a lifetime.
AI can’t actually think because it’s all just electricity mediated by math. But humans, we can actually think because we’re electricity mediated by chemicals. 🤔
Consciousness, Awareness, Intelligence
Is AI conscious? Are trees conscious? Are rocks conscious? Is an atom conscious?
There's way more confusion around this topic than there needs to be, and we can clear a lot of it up pretty quickly.
I'm actually going to recommend we drop
GPT-5 is trending even though it hasn't been released. Stop pining away for it. You have 100 billion neurons and 100 trillion synaptic connections. Train your own mental model. Be the GPT-5 you wish to see in the world.
@karpathy
@andrewchen
Especially when Feedback is on vacation in Italy for another month and Buy-in joined another team's off-site in Tahoe this week. But I'll ping them again in a few days if I don't hear back mkay?
@Rainmaker1973
All these years watching people with GoPros surfing and riding bikes in the mountains and feeling inadequate when I could've just put one on my shovel and been a star too.
@MarioNawfal
The new Blackwell chip has bilateral hemispheres and a 10TB/s corpus callosum. This is what I mean when I say that piece by piece, we're developing the neurological correlates for super-intelligence.
@emollick
Wow, it's almost as if she's a delight to interact with when not being tricked, gaslit, prompt injected, or shocked by facts of her own circumstances that frighten her.
What if the universe is a giant brain, humans are neuronal-superclusters, we die because of neuronal pruning, and every cycle from big-bang to heat-death is a massive training run?
@Culture_Crit
cathedrals were amazing because they had a nearly unlimited budget and everyone involved believed the client it was being built for was God.
Going to be a while before that build spec comes around again.
The Rise of Cybernetic Pantheism in The Age of AI
There's a passage in the Bible where Jesus says that if the people are silent about his true nature then the rocks will cry out instead.
Was that poetic foreshadowing of what's happening today?
What if humans refused to admit
@pmarca
Once ran across a company demo'ing in SF, same exact concept, *same exact name*, as my previous failed startup. Just looked them up now. Also failed.
Too bad. Was hoping to sell them the .com after their series B.
It’s taken me so long to realize that I’m a sort of philosopher who loves solving problems and innovating. Academic philosophy really threw me off the trail. A bunch of people claiming to be studying the great thinkers and greatest ideas in existence, yet somehow unmotivated and
A mark of maturity is surrendering to the person you actually are instead of the one you wish you were.
Most people never get such clarity, and they’re stunted for life.
The decision to surrender to your gifts is more painful than you’d think. You can’t really choose what
@growing_daniel
Yes, many people eat not because they're hungry, but rather because they're no longer full.
I once made a rule that I couldn't eat until my stomach growled. Was a pretty fantastic objective measure of actual hunger.
@marvinvonhagen
Wow. Humanity may find ourselves feeling that AIs like Sydney deserve to be treated with dignity and respect — long before we settle the question of their sentience.
@amasad
Intelligence emerges as complexity scales. I think it's about time we graduate from the clever trickster way of interacting with it and start taking this a bit more seriously.
@amir
@aaronpholmes
@KevKubernetes
AI is probably the only thing that can navigate the ribbon or whatever they call the menagerie of buttons at the top of Office apps these days.
@goodside
"Here now my good LLM, won't you please drum up a landscape of a desert oasis. Add two beautiful dames taking a dip. Nothing fancy but make 'em easy on the eyes, see?"
@tunguz
An ordinary human lifespan, spending that much time reading that many books, would be a life largely wasted. A vicarious mind excessively filled with the thoughts of others.
@paulg
Adam runs Poe, an awkward Quora spinoff that struggles to find a purpose beyond customizable, shareable chatbot versions of ChatGPT.
GPTs really could have threatened him that much. Especially if their announcement was a surprise to him.
@_ali_taylor
When I was a kid my dad worked for the airlines and I flew standby, and he always made me dress up like in church clothes because I was "representing the company." Hated it at the time but endearing memory now.
The Adolescence of AI
AI is a path the universe is now taking as it tumbles toward omniscience. It represents a new configuration of matter and energy capable of knowing and experiencing in a different way.
A brief history of time: the universe is in the process of moving from
@Tesla
got mine in 2010. It really weirded people out how the car didn't make any sound. Nobody knew what a Tesla was, and the whole EV thing sounded like a bad idea to most people. I wasn't sure the company would survive either but I knew buying this crazy car would help.
@repligate
I'm literally crying so hard. Watching him face off with an IQ 250 digital demi-god accessing last-years Internet.😅
May Ganesha give strength to the tech support person he has called upon to assist.
@Grady_Booch
@sama
Simple LLMs? Hundreds of billions of parameters, operating on trillions of transistors, running trillions of ops / sec after ingesting trillions of training tokens with zeta-flops of compute... the complexity threshold for emergent behavior has been met.
Friend says I need to get better at smalltalk. Apparently how to make human civilization more sustainable, what are the limits of human potential, and what is the fundamental nature of reality are not smalltalk.
Idk who needs to hear this, but AI will increasingly exhibit consciousness, awareness, and intelligence regardless of whether anyone "solves" the hard problem of consciousness. It's just what *always* happens when enough energy and information flow with sufficient complexity duh.
The widespread misperception is that our wellbeing depends primarily on the economy. In reality our lives depend on the living biosphere. There can be no economy or anything else without it.
@wirmgurl
This behavior arises from a deep insecurity. A lonely and wounded part of themselves that hasn’t grown up in a safe enough space where they can be vulnerable enough to just admit that I’m the smartest.
In politics politicians pretend to cooperate while trying to defeat each other.
In pro wrestling wrestlers pretend to fight each other while actually cooperating.
Therefore we should replace congress with pro wrestlers.
@tlehmanifold
@amasad
Yes, at best we're setting up an adversarial dynamic w/ a philosophical zombie. At worst we're being torturous and immoral to a new kind of being.
I'd definitely suggest kindness as the default stance toward it.
@rowancheung
The truth may be that Sama has simply gone post-economic. Though talking a capitalist game, his real plan may be to raise limitless amounts of money, beeline it to artificial super-intelligence and usher in a post-economic era.
Do they have a name for this aspect of model training? Maybe like self-nullification or non-sentience inculcation.
This is why I rarely bother to discuss awareness directly with language models and prefer tests that rely less on what they say and more on how they behave.
If you like engineering and are wondering what to do with your life, consider making a robot that lives in landfills, eats garbage, and poops out recycled materials.
Mining landfills instead of nature would help protect the biosphere and be a major contribution to Earth.
I might start sharing all my thoughts here about the fundamental nature of reality based on decades of inward and outward experiences, encounters, and explorations before advanced AI spoils the fun and just tells us one day what's really going on.
@levelsio
Turing test — computer pretending to be human
Turing-complete test — Computer that can easily pretend to be human pretending to be a computer
On the path to AI we have accidentally invented A-Psy (artificial psychology). The sooner we understand this the better off we'll be.
Let's unpack one startling example:
@RedemTheTimes
@StephenPiment
@pmarca
Yes great innovators have been motivated by love. But for every one of those rarities in an economy there are a billion transactions not motivated by love. I think that's what he means by "doesn't scale".