Welcome to this update from New World Same Humans, a newsletter on trends, technology, and society by David Mattin.
If you’re reading this and haven’t yet subscribed, join 30,000+ curious souls on a journey to build a better future 🚀🔮
Life has been doing its thing recently, and in consequence I haven’t sent you a postcard for a while. I hope this helps to pay the debt, and I’ll endeavour to ensure that I’m back in your inbox next week with another.
A telling sign of the times emerged out of China this month, and few noticed. The CCP enacted a ban on AI companions.
The new rules came into force on July 15. They mean that virtual partners and family members are now outright banned for children, and highly restricted for adults. ByteDance’s Doubao is China’s most popular chatbot app, with 350 million monthly active users; it switched off its companion features overnight. Alibaba’s Qwen went further and deleted historic chats, leading to an outcry from those who had grown attached to their AI friend. On the evening before the new rules came into effect, many users reportedly stayed awake all night, immersed in one last, and presumably somewhat tragic, tête-à-tête with their companion.
I’ve been talking about the rise of AI companions since long before the explosion of LLMs into the culture. Back in 2015, the idea that people would one day form close emotional attachments to AI-fuelled conversational entities seemed ludicrous to most people. And now, here we are.
So what’s really behind China’s ban? And will it have implications beyond Chinese borders?
The CCP’s official line is that the new rules are aimed at protecting the mental integrity of China’s young people. I’ve written before on the strict, and face recognition-enabled, rules that China has imposed on young people when it comes to video gaming: they are limited to under three hours per week. In that piece, I said many western parents would look on in envy at China’s ability to end child video game addiction. Now, those same parents may well feel the same about the new law on AI companions. There’s no way around this weird truth: China’s strange new techno-authoritarian regime can take instant, summary action against socially damaging technologies, in a way western democratic governments can’t. Over time, that difference is going to become ever-more apparent, and more challenging for the liberal democracies of the west.
Underneath the new rules on AI companions, though, sits a deeper obsession. The CCP is rightly worried about demographics. China’s birthrate has fallen to just 5.63 births per 1,000 people, a record low. If that doesn’t change, the country is facing demographic catastrophe. The Party, then, wants young people dating, marrying, and having babies. The last thing it needs is to have romantic impulses rerouted towards AI.
Why does this matter for those of us outside China? The answer is stark: because historically low birth rates are not just a China problem, they are a problem across the industrialised world. For a nation to have a stable population, the Total Fertility Rate (TFR) — which counts the number of children born per woman — needs to be at least 2.1. In 1950 only Luxembourg, uniquely among the nations of Earth, had a TFR below two. Today half of the world’s nations do, including all the affluent ones. If current trend continues, South Korea — which today has a population of around 52 million — will be peopled by just 11.5 million people by 2125. No country can experience this and remain functional. The industrialised world is on an acute demographic slide trend towards socio-economic collapse.
It’s impossible to ignore the apparent connection between all this and digital technologies. South Korea, one of the most wired countries on Earth, has a birth rate of just five per 1,000 people: the lowest in the world. In Silicon Valley, meanwhile, the equivalent number is 9.7, significantly lower than the average across the US.
Immersion in new digital realities, phones, social media, and attendant atomisation: it’s not hard to see how all this results in fewer babies. Any parent today knows the dark truth here: once children have phones, they just hang out with each other less.
And now, into that deeply depressing set of circumstances, arrives the AI companion. A facsimile of a real person — girlfriend, boyfriend, soulmate, whatever — often dressed in a sexualised avatar, available 24/7, endlessly supportive and enthusiastic, an entity without its own emotional needs or insecurities, without bad habits, or contrary opinions, or a difficult childhood, a being designed only to serve. It is so telling, and such an affirmation of the statement new world, same humans, that the cleverest among us found a way to compress all human text output, the agglomerated cultural production of hundreds of years and billions of people, into these little black boxes that we call LLMs, which are surely the most complex objects we humans have ever created, and that having managed this monumental achievement, we now watch as millions use those LLMs to create virtual people who simulate for them, in a manner that would have seemed absurd or incredible just a few years ago, the experience of being loved.
If phones were bad for birth rates, it’s hard to see how virtual companions won’t be worse. China has acted. Can the liberal democracies of the west find a response of their own? What should it look like?
In the end, all this taps deep into questions I’m now immersed in via the writing of my book, Creature vs Machine. Is our future one of ongoing abstraction away from physical reality, and into a world in which we rarely meet in person, or speak to one another, or, in the end, have babies? Or can we find a way to lean back into the embodied experience of being human? The question is one with deep philosophical underpinnings. But the question of birth rates makes it concrete. And existential.
One final thought.
We look out at a universe of two trillion galaxies and see no sign of life; and certainly not of another advanced civilisation. The physicist Enrico Fermi captured the mystery in three words: where is everybody? Decades after he posed that question, it remains an abiding puzzle. But perhaps the answer is now becoming clear.
Perhaps the truth is that every advanced civilisation inevitably, at some stage during its development, builds technologies of representation so compelling that its people drift away from reality altogether and, in the end, stop reproducing. Maybe that is the answer. Not war, pandemics, or AI doom, but a fatal abstraction away from the given world; from the shared and actually existing reality we inhabit, the only reality in which biological reproduction can occur.
We coaxed the machines into a strange new kind of life. We made them capable of talking to us. Are we now, as a consequence, to stop talking to one another? If so, it will be the end we deserve.
But I can’t see it playing out that way. My conviction is that a resistance movement is coming; a movement to conserve the human. Amid the rise of thinking machines, we’re about to lean back, hard, into what is special about one another.
Have a fruitful week; I’ll be back soon.
David.
I’ve been meaning to send a David Hockney as a postcard for a while now, to pay homage to the great man in the aftermath of his death last month. The title artwork this week is Garden, painted by Hockney in 2015.
Thanks for these insights, David!
This is all a huge self-own. We’ve built more and more complex versions of the cocaine-filled lab rat feeder, and we do so while deliberately looking away so as to not witness our own demise.