What happens the day after humanity creates artificial general intelligence?
By Louis Rosenberg | Published: 2025-06-09 18:58:00 | Source: The Future – Big Think
Sign up for Big Think on Substack
The most surprising and impactful new stories delivered to your inbox every week for free.
Over the past few weeks, I’ve been asking people in my circle of AI professionals whether they are mentally prepared for super-AI. They tend to mock and express various concerns: potential or threatened impacts on the labor market Disinformation powered by artificial intelligence. They also mention potential positives, such as the potential for superintelligence to help us cure diseases, revolutionize clean energy, uncover the secrets of the universe, and perhaps even achieve world peace.
In other words, they never thought about what life would be like the next day Artificial general intelligence (AGI) is widely available and exceeds our cognitive capabilities. From knowledge and experience to planning, thinking, creativity and problem solving, we may soon find ourselves completely overachievers. This is a real possibility in the very near future, and no one I know honestly faces the profound (and potentially depressing) impact it could have on who we are as human beings.
I apply this criticism to myself as well. I’ve been writing about the risks of superintelligence for over a decade, and I’ve also focused on specific tactical risks e.g The problem of artificial intelligence manipulation and The paradox of access to reason. At the same time, I failed to face honestly what life is will I really feel like When we humans collectively realize that we have lost Cognitive superiority On Earth and likely never to return.
No, that last statement is not personal enough.
What I failed to confront is what my life I would really feel when I’m standing alone in the elevator – just me and my phone – and the smartest person moving between floors is the phone. When you allow yourself to think about this, you realize that the biggest impact on humanity will not be the looming disruptions in the labor market or the dangerous new AI technologies we will undoubtedly use to manipulate each other.
No, the biggest impact on humanity will be an identity crisis that hits us like a robotic punch in the face, jolting us into a new reality where artificial intelligence in our pockets (and soon, built into our glasses, earbuds, or pendants) can solve any problem we face in our daily lives, and do so faster, smarter, and more creatively than we can do it ourselves.
In this new reality, we will automatically ask AI for advice before we even bother using our brains. In many cases, we won’t even need to ask, but the guidance will flow into our eyes and ears. This is because the AI in your wearables will have access to onboard cameras and microphones, enabling it to see and hear everything you see and hear and thus track the full context of your life in real time. They will also compile a historical record of your actions and reactions, enabling them to record them accurately Predict your behaviors and emotions In almost every situation, anticipate your wants and needs before they even appear in your mind.
In other words, these Context-aware AI assistants He will give you advice without you needing to say a word. While you’re walking down the street, an AI might see a jewelry store in front of you and remind you that your 20th wedding anniversary is approaching and you need a gift for your wife. It will then help you choose something you’ll like, as the AI will know its taste better than you do. This will change who we are, how we live, and how we relate to others.
The first time I wrote about the profound societal impact of context-aware AI agents was in my 2012 graphic novel He promotes. I called these agents “spokespeople” because they were what they were Artificial Intelligence spokespersons Who help and guide us, while also manipulating us on behalf of our paying sponsors. What I fail to realize is that the greatest danger to context-aware AI agents is not the manipulative influence of deliberative influence.
The truth is that even if these super-intelligent assistants are banned Overt manipulationThe fact that they are able to outthink us can undermine our sense of power and destroy our sense of self.
Imagine having a super-intelligent assistant sitting on your shoulder, observing your life and whispering advice in your ears. This will seem like a superpower at first, but a great digital reckoning is sure to follow as we all gradually realize that the voice in our ears is smarter than our inner monologue. What happens when we trust the sound in our earphones more than the sound in our heads? I fear we will become puppets of our AI assistants, not because of an evil plan to control us, but simply because they will feed us a torrent of advice beyond anything we could come up with on our own.
I know many readers will balk at this, insisting that they will never prefer the voice of an AI in their ears to their inner thoughts. I’d like to believe that too, but I suspect I’m not the only idiot who followed my GPS to the end of a dark alley even though I knew things didn’t look right. And GPS is far from super intelligent. It’s just a navigation tool that replaces your need to look at a map. But relying on a superintelligence that can out-logic you, out-plan you, out-negotiate with you, and do it all more creatively than ever before — that will certainly strike at the heart of what it means to be human. How could he not feel frustrated?
We are racing toward a new era in which we outsource the cognitive abilities that are fundamental to our identity as thinking beings. Some experts believe this will make us feel smarter and more capable, and view it as a boost, but that’s not the only way things can go. This age of the “boost mentality” can easily make us feel smaller, less confident, and less important. It can even undermine our personal relationships. For example, when your husband gives you a gift, you will wonder if his AI chose it for you. You’ll also wonder if the words they say while handing you the gift are their own feelings or if they’re being trained by an ever-present AI (see the short film, “Privacy lost“, for fun examples).
I raise these concerns as someone who has spent his entire career creating technologies that can achieve this Increasing human capabilitiesstarting from my early work Augmented reality development For the US Air Force from decades ago, to my recent work in artificial intelligence Collective intelligence. I believe technology can greatly enhance our identity, greatly expanding our physical and mental capabilities while also keeping us human.
Unfortunately, there is an increasingly fine line in between Increase ourselves and Undermining ourselvesor even Replace ourselves. Unless we think carefully about how we deploy context-aware AGI assistants, I fear we will cross that line. As this threat now accelerates towards us, I ask AI researchers and entrepreneurs to think beyond that fateful day when we collectively create AGI to what it will feel like to be human. the next day.
Sign up for Big Think on Substack
The most surprising and impactful new stories delivered to your inbox every week for free.
ــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــ
