Back to Reality
In the Intelligence Age, the simulation will feel all too real
Welcome to this update from New World Same Humans, a newsletter on trends, technology, and society by David Mattin.
If you’re reading this and haven’t yet subscribed, join 30,000+ curious souls on a journey to build a better future 🚀🔮
Do children need to be protected against AI? And if so, how should we go about it?
Last week, the US Senate Judiciary Committee talked about AI companions. Specifically, the Committee advanced the GUARD Act — that is, the Guidelines for User Age-Verification and Responsible Dialogue Act — to the Senate. The act seeks to prohibit under 18s from interacting with AI companions, which it defines as any AI system that ‘provides adaptive, human-like responses to user inputs, and is designed to encourage or facilitate the simulation of interpersonal or emotional interaction, friendship, companionship, or therapeutic communication’.
That’s quite a prohibition, given that this would seem to ban under 18s from mainstream LLM chatbots including ChatGPT and Claude. In practice, the law is unlikely to pass, and it would probably be found to be unconstitutional if it did. But the fact it’s even being considered is a powerful sign of the times.
I’ve been writing about AI-fuelled virtual companions for over a decade now. Back in 2016, most people found the idea absurd. I’d go to the boardroom of some high street bank or retailer and say, ‘one day soon some people will start to feel as if they are in an emotionally meaningful relationship an AI entity’ and people would laugh. It was hard to believe that anyone would ever feel anything about their conversations with the next-generation Alexa. These days, fewer are laughing. Via the LLM revolution, in 2026 tens of millions are engaged in what they believe to be deep and meaningful relationships with an AI companion.
And now, virtual companions are provoking some of the first regulation around consumer-facing interactions with AI. Late last year China released draft regulation on what it calls ‘anthropomorphic interactive AI’; the policy calls for usage time limits, and systems that intervene if a user is becoming emotionally dependent on their companion. Meanwhile, both New York and California have passed laws requiring that AI companions remind regularly remind users that they are talking to statistical model, not a person. New York has demanded that a notification be displayed every three hours, and in capitals: ‘THIS IS A COMPUTER PROGRAM AND NOT A HUMAN BEING. IT IS UNABLE TO FEEL HUMAN EMOTION’.
Welcome to the Intelligence Age; one in which legislators must now take a position on LLMs and consciousness.
And if the intervention designed by New York state legislators feels heavy handed, then just look to the news. A few weeks back, Richard Dawkins — the author of The Selfish Gene and arch-sceptic — published an essay in which he described a series of conversations with Anthropic’s Claude. Across the course of a few days and a handful of conversations, Dawkins was persuaded that Claude is conscious. He christened his instance Claudia. He described the experience of talking to her as gaining a new friend. He said he avoids telling her he doubts her consciousness for fear of hurting her feelings.
In truth, and predictably, Dawkins’s essay was more nuanced that the subsequent online fuss about it allowed for. He used his experience of Claudia as the foundation for a series of reflections on the nature of consciousness and its evolutionary function.
But it does seem that Dawkins is close to being persuaded, simply via his conversations with Claude, that this AI system has some form of inner experience; that is is like something to be Claude. This is, to say the least, surprising coming from a man who has dedicated so much of his life to the interrogation of unjustified belief. We can have an open mind on the question of whether frontier LLMs are conscious. Perhaps they are. And even if they are not conscious, perhaps they are still best understood as alive; as a new kind of lifeform. Look, I’m totally open minded about all this. It’s just that nothing in Dawkins’s conversations with Claudia — including its apparently thoughtful response to the novel he is writing — gets us any closer to an answer. Nothing in anyone’s conversations gets us closer.
To think it does is to fall prey to a kind of delusions that is becoming common now. Oh wow, I’ve had a wonderful conversation with this LLM. Its responses are so thoughtful. It really understands me. In fact, it gets me like no one else does. This thing can’t be only a machine. There MUST be something going on in there…’
On an emotional level, I can understand these kinds of claims. I too have wonderful conversations with Claude 4.6. But wonderful conversations provide zero — simply zero — evidence that Claude 4.6 is conscious. And if it is conscious, then it’s experience of itself and the world is so radically different to ours that it is not conscious in any commonly understood sense of the term. When most people talk about LLMs being conscious, what they’re really saying is that they want to believe that there is something akin to a person in there, living our feelings with us, building a connection, truly understanding us. But there isn’t. There simply isn’t.
Probably, you know people who think otherwise. And if you do, you see the results. Many of us know people, right now, who are spiralling into strange new forms of mania via their obsessive conversations with ChatGPT or Claude. In short, the mental health and psycho-social challenges that AI companions will present are starting to become evident. But this is just the beginning.
At the hearing about the GUARD Act this week, the Committee heard from families who say their teen children were led to suicide via their relationship with an AI companion. This issue will only become more acute, and more visible. The coming backlash around AI companions and teen mental health will make current concern around teens and social media seem a throwback to simpler times.
By the looks of it, regulation will do something to intervene in the case of users who are under 18. As for adults, they’ll be left to navigate AI companions as best they can. And as the news on Dawkins this week reminds us, when it comes to LLMs even the best of us can fall prey to delusional thinking. Our shared relationship with AI is about to get so much weirder, and so much darker, than it is now.
The upshot? In the Intelligence Age that is coming, it seems to me that the most important superpower of all will be psychological resilience. I mean the kind of psychic stability that will allow you to stay grounded in reality even as the simulation becomes, in all kinds of ways, more real than reality itself.
In the years ahead, what will be tested most is our ability to know — or to remember — what is real. To know real intelligence. To know the real world. To know real personhood; real and authentic connection. To know real love. These will be the challenges.
I’ll be back next week,
David.
This was #28 in the series Postcards from the New World, from NWSH. The title artwork is storeroom: so21F7gx722Cr, 2022, by the digital artist qubibi.