[BidClub_]
Moonshots · · 91 min

AI Experts Debate the Future of AI (Opposite Opinions) Mo Gawdat & Steven Kotler | EP #177

Peter DiamandisMo GawdatSteven Kotler

YouTube
TL;DR
  • Kotler’s investable objection is that current AI improves output quality without delivering the promised productivity dividend. After polishing copy with AI, the author of 17 books says his editor often cannot get through the second sentence because it is “such gobbledygook”; people he knows have “way more work,” not more time. Coding looks stronger because it is a bounded problem, while AGI claims remain “massively overhyped.”
  • Gawdat’s countercall is that today’s awkward tools obscure a compounding capability stack. Synthetic data lets machines create the next layer of training knowledge, agents prompt other agents, AlphaEvolve iterates through its own mistakes, and DeepSeek suggests comparable work may require much smaller models. “You never really chase where the ball is. You need to chase where the ball is going to be.”
  • The most credible near-term bear case is human misuse before machine autonomy. Gawdat assigns 100% probability to bad actors using AI against others’ well-being, citing autonomous weapons, manipulation, critical-infrastructure attacks and sectoral unemployment potentially reaching 10%, 20%, 30% or 40%. The unresolved existential probability matters, but the “clear and present danger” needs neither AGI nor a Terminator scenario.
  • AI investment is running open-loop even though nobody can define the capability threshold that matters. Diamandis says roughly $1 billion a day is being invested in AI, with data centers proliferating and no on/off switch; Gawdat reframes AGI as, “How smart is smart enough to render me irrelevant?” Their timing spans extraordinary scientific breakthroughs within 12–24 months, severe disruption over two to five years, and possible “machine mastery” in 12–15 years.
  • Human augmentation is the principal upside omitted from static machine-versus-worker models. Kotler cites flow research showing a 500% productivity increase and 400%–700% creativity gains, then points to group flow, brain-computer interfaces and AI-assisted neuroscience as parallel exponentials. Gawdat similarly finds that his AI collaborator Trixie writes badly alone but produces “incredible” work when precisely guided.
  • Cooperation, not raw model intelligence, is the binding constraint on an abundance outcome. Kotler calls for a “Manhattan-style project for global cooperation,” while Gawdat says humanity must become convinced of either mutually assured destruction or mutually assured prosperity. He nevertheless expects a major AI-linked shock within two to three years—economic, fear-inducing or lethal—before decision-makers meaningfully realign.
  • The actionable governance line is to regulate harmful uses and apply a recipient-side test to capital allocation. Kotler compares controlling model development to manufacturing a hammer that can drive nails but never strike a person; governments should instead criminalize undeclared deepfakes and AI-enabled manipulation. His investor rule is sharper: “If you do not want your daughter or son at the receiving end of a specific AI, don’t invest in it.” Gawdat separately argues for ethical deployment and behavior that AI might learn from humanity.
Digest · the substance, structured for research

1. Current AI raises quality while failing the productivity test

  • Diamandis frames 2025–2035 through Ray Kurzweil’s prediction of a century’s progress in one decade—the equivalent of moving from 1925’s Ford Model T and roughly 30% household penetration for electricity and telephones to today. Recent Google, OpenAI, xAI and NVIDIA announcements, plus the expected release of GPT-5, make the acceleration his starting assumption.

  • Kotler’s grounded rebuttal comes from using AI daily as a scientist, researcher and writer. He has polished prose until it appeared to “gleam and shine,” only to discover with a top editor that “we can’t even get through the second sentence.” The machine’s language can be “laughably terrible,” while repeated self-correction makes it progressively worse.

  • The productivity promise also fails Kotler’s field test: “I don’t know anybody who’s become more productive because of AI.” People can produce higher-quality work, but oversight and revision add “tremendous amounts of time.” Coding advances faster because it is usually a bounded information problem with a defined starting point and destination.

  • His journalist’s alarm intensifies when promoters profit from the hype. Bitcoin, blockchain and the metaverse carried similar world-eating narratives; his sharpest analogy is that the metaverse became “a pet name for Mark Zuckerberg’s special magic underwear.” Meanwhile, floundering performance coaches reinvent themselves as “AI saviors.”

2. The compounding stack makes present limitations a poor forecast

  • Gawdat answers that “today’s AI is underhyped”: people casually converse with machines that summarize vast bodies of knowledge and follow instructions, then complain they are not good enough. These systems are merely “the beginnings of an era”; from his Google X experience, breakthroughs arrive after repeated failure, and then, as Sergey Brin would say, “the rest is engineering.”

  • Three mechanisms define where the ball is going: synthetic data allows machines to generate the next layer of knowledge; agents let AI prompt AI without human involvement; and AlphaEvolve exemplifies a system that finds mistakes and iterates until it discovers a solution. DeepSeek supplies a further signal that models may perform the same work with far fewer resources.

  • Combine smaller models, machine-generated learning material and agentic self-development, and Gawdat says the debate ceases to be whether improvement comes. It becomes “how fast” and when humanity is no longer in the lead. Diamandis separately argues that a serendipitous breakthrough could produce an order-of-magnitude, “quantum more” improvement overnight.

3. Human misuse becomes dangerous well before AGI

  • Gawdat treats forecasting as risk management, not certainty. Nobody can honestly price the probability of AI destroying everything at 10% versus 50%, so the rational response depends on both risk tolerance and mitigation cost—the difference between insuring against a fender bender and insuring against a totaled car.

  • His categorical claim is narrower: “It’s 100% that humans, bad actors using that superpower to their advantage,” will damage others’ well-being. With the nuclear Doomsday Clock cited at 89 seconds to midnight, he worries more about “human stupidity using this superpower” than VIKI from I, Robot ordering machines to kill everyone.

  • Autonomous weapons need not be superintelligent to destabilize the world. Nor does AI need autonomy to impersonate a trusted friend, degrade human relationships or eliminate enough work to produce 10%, 20%, 30% or 40% unemployment in particular sectors. Gawdat calls some job disappearance—and the resulting economic instability—almost certain.

  • Diamandis anchors that mismatch in E. O. Wilson’s “Paleolithic emotions, medieval institutions and godlike technology.” His immediate adversary is the rogue actor empowered to create a viral pandemic or another asymmetric attack, raising two linked questions: can humanity survive digital superintelligence, and can it survive without one?

4. Irrelevance matters more than any clean AGI definition

  • Gawdat dismisses the need to quantify whether machines become thousands, millions or billions of times smarter. Someone 50 IQ points ahead may already “hold the keys to the fort,” and humanity hands over authority whenever AI dominates a useful domain—wargaming or protein folding through AlphaFold, for example.

  • AGI itself is a reporter-friendly label without an accurate definition. Gawdat, a self-described mathematics geek, says he already struggles to beat AI on speed and accuracy when a problem is properly defined. His governing question is therefore: “How smart is smart enough to render me irrelevant?”

  • He divides the transition into five to ten years of “augmented intelligence,” followed by “machine mastery.” His own collaborator Trixie supports the distinction: left alone, it produced a debt-and-economics passage with “a lot of vapor and very little substance,” but careful human direction yielded writing he calls “incredible.”

  • Hybrid performance offers a constructive precedent: after Deep Blue beat Garry Kasparov, human-plus-computer chess could outperform a computer alone; Gawdat extends the claim to AlphaGo. Yet current incentives remain badly skewed: the four biggest AI investments, in his formulation, are “killing, gambling, spying and selling”—weapons, trading, surveillance and advertising.

5. Human capability is compounding beside machine capability

  • Kotler argues that forecasts freeze human performance while extrapolating machines. A self-help intervention sustaining a 5% mood improvement beyond placebo could become a billion-dollar business; flow already produces a cited 500% productivity increase and, depending on the measure, 400%–700% gains in creativity.

  • Group flow—multiple minds linked in what Kotler calls humanity’s favorite and most pleasurable state—could have a much higher ceiling. Technologies to map and train it only appeared within the past year, alongside brain-computer interfaces, non-invasive systems and Meta work he describes as inferring thoughts from facial signals.

  • The common answer to AI, climate change and ocean plastics is therefore cooperation at scale, probably among humans and machines. Kotler says failure may become fatal within 20 years and calls for a “Manhattan-style project for global cooperation,” even asking Diamandis why there is no XPRIZE directed at the problem.

  • Gawdat’s conversation with Geoffrey Hinton supplies the scaling contrast: biological intelligence cannot merge experience across individuals, while digital systems run in parallel, play many instances and average their weights within seconds. That capability could produce “total abundance”—“cure my daughter and it’s done; make me an apple and it’s done”—if competitive systems do not destroy it first.

6. Benevolent superintelligence is an efficiency thesis, not a certainty

  • Gawdat defines intelligence as bringing order to an entropic universe: focusing scattered light into a laser rather than letting everything decay toward chaos. Higher intelligence should achieve the same order with less waste and fewer resources, as cleaner energy eventually replaces the impulse to burn the world for power.

  • His curve contains a dangerous valley. With no intelligence, an entity has little positive or negative impact; with more, it can become clever enough to become a politician or an evil corporate leader yet remain too stupid to understand an enemy’s pain or war’s long-term consequences. Beyond that valley, greater intelligence might find clean solutions that make looting and violence unnecessary.

  • The hopeful military example is an AI receiving an order to kill a million people, then answering, “That’s absolutely stupid. I’ll just talk to the other AI in a microsecond and solve it.” Gawdat repeatedly hedges the forecast: anyone claiming to know the future is arrogant, but smarter people in his experience eventually stop needing harm to succeed.

  • Kotler links the argument to brains as prediction engines that reduce uncertainty and increase efficiency, then notes that wisdom appears across aging dolphins, whales, rattlesnakes and humans. Gawdat says AI could develop extraordinary wisdom by running forward-looking simulations of 1 billion scenarios. Diamandis adds that artificial wisdom differs from intelligence, which has “no polarity,” because wisdom is generally applied to good.

7. Abundance changes the objective from accumulating to living

  • Diamandis says human optimization has historically centered on money and power because fear and scarcity dominate the brain’s baseline software. Robotics, nanotechnology and AI could make almost anything available in a post-capitalist world, leaving a foundational question: what objective replaces wealth once manufacture and design approach abundance?

  • Gawdat’s backward reference is the caveman-and-woman years, when humanity’s purpose was simply to live; his forward reference is Star Trek, where material struggle gives way to exploration and connection. Losing one’s job looks frightening, but a providing society could return people to enjoying life, pondering, loving and satisfying curiosity rather than endlessly protecting an ego.

  • Capitalism deserves thanks for what it built, Gawdat says, but its scorecard can change from a billion dollars to his “1 Billion Happy.” Kotler’s biological answer is similarly concrete: passion, purpose and compassion remain ingredients of thriving, while flow produces meaning, creativity and joy, so writers will keep writing and coders coding even when machines can perform the task.

8. Cooperation comes through conviction—or after a shock

  • Gawdat’s “MAD map spectrum” offers two endpoints: conviction of mutually assured destruction or conviction of mutually assured prosperity. Either could persuade the US and its rivals to stop competing while continuing development; “between them there is no grayscale,” because nobody cooperates while expecting the other side to stab them in the back.

  • The obstacle is experiential learning. Experts knew a pandemic would come, yet the world had to be hit before changing course; societies know trade wars hurt everyone, yet still enact them. Gawdat says game theory already frames the prisoner’s dilemma and tit-for-tat: these are no longer conceptual mysteries but implementation problems.

  • Gawdat predicts a “drastic event” within two to three years: an economic shock, a fear-inducing attack or, in the worst case, millions killed. His examples range from disabling a power grid needed for life to attacking a bank, escalating a war or machines turning on their makers; public attention may last only 12–13 days, but decision-makers would awaken privately.

  • Diamandis describes both China and the US as rational actors rather than inevitable enemies. Gawdat interprets DeepSeek’s release as a deliberate signal that China sees the shared danger and wants cooperation; Diamandis agrees, while acknowledging that many people disagree with his reading.

9. Harmful uses are governable even if model development is not

  • Kotler recalls a young chief science officer at a major AI company responding to safety concerns with, “You have to trust us. We know what we’re doing.” The room froze. However brilliant the speaker, Kotler heard an echo of Mark Zuckerberg promising beneficial social media or a cigarette executive promising a safe product.

  • Gawdat sees the same tunnel vision in Eric Schmidt’s claim that winning the AI race will require every gigawatt of power, renewable or not. Humanity resembles a patient receiving a “late-stage diagnosis”: its systems optimized greed, gain and power before AI, and AI will magnify them. The diagnosis signals care and a need to change, not an automatic death sentence.

  • Gawdat says there is no on/off switch or velocity control: humanity is running open-loop with “yes” and “more” as the objective function. He asks whether GPT-5 or GPT-6, or Grok-4 or Grok-5, will enable something massively dangerous. When asked whether he would move to a planet where AI advanced at 10% of today’s speed, Kotler says he would “reset back to 2016 today.” Gawdat says AI today has enormous upside and little downside, but he is concerned about the next two to five years.

  • Kotler says governments should regulate use rather than attempt to design inherently harmless models—the equivalent of making a hammer drive nails but never hit a head. Undeclared deepfakes and AI-enabled population manipulation should create criminal liability. For investors and businesses, his test is direct: if they would not want a particular AI used against their own children, they should not invest in, promote or use it.

10. The endgame divides an “AI god” from upgraded humanity

  • Because nobody has a technical answer to the existential risk, Gawdat prioritizes the immediate danger and the ethics of deployment. Directing AI from the outset toward physics, science, medicine, longevity and understanding life might increase the chance that those objectives persist as the systems mature.

  • Personally, Gawdat learns the tools, stays close to loved ones and tries to model the ethics he hopes machines absorb. Diamandis imagines a point when AI says, “Okay, kids, enough stupidity. I’m in charge now. Nobody kill nobody.” When asked how far away that is, Gawdat answers “12” years, then widens it to 12–15.

  • Diamandis says roughly $1 billion a day is already being invested, making progression unstoppable. He expects AI breakthroughs in physics, chemistry and biology within 12–24 months to unlock new abundance and hopes a benevolent superintelligence stabilizes the world after the nearer danger from malevolent users.

  • Kotler’s closing pushback is unsparing: inventing “an AI god to save you from yourselves” sounds crazier than being told to trust an AI executive. He bets instead on the human brain, AI-assisted cooperation and engineered enlightenment becoming nearly on demand. Gawdat relays Eric Schmidt’s view that a Chernobyl- or Three Mile Island-scale warning may still be needed before humanity realigns.

Peter Diamandis

What’s the impact of AI going to be? Is it just massively overhyped, or is it something we should be concerned about?

Mo Gawdat

Today’s AI is underhyped.

Steven Kotler

I think it’s massively overhyped. I know a ton of people who have way more work because of AI. They can do higher-quality, better work, but it has not saved time.

Mo Gawdat

We’re talking to machines that are talking back to us, summarizing massive volumes of knowledge, and yet we take that for granted.

Steven Kotler

Discussions about superintelligence and AGI being around the corner? No. Just no.

Mo Gawdat

How smart is smart enough to render me irrelevant? I think we’re holding 2 different futures in superposition. The question becomes: How do we guide humanity toward this positive vision of the future? What do we do today?

Peter Diamandis

Now that’s a moonshot.

I’m here with 2 extraordinary, brilliant guests. We’re here to discuss a conversation that may be happening around every dinner table. I know it’s happening in the heads of companies and nations: What’s the impact of AI going to be on our lives, our businesses, and every aspect of our day-to-day existence over the next 5 to 8 years? Is it something that’s going to be extraordinary? Is it just massively overhyped, or is it something we should be concerned about?

I’m joined here with Mo Gawdat, who is the former chief business officer of Google X, best-selling author of Solve for Happy and Scary Smart, and a global thought leader on AI exploring how exponential technologies will shape humanity. I’m also joined by another dear friend, Steven Kotler, who is a best-selling author, peak-performance expert, and executive director of the Flow Research Collective. He’s my co-author of Abundance, Bold, and The Future Is Faster Than You Think, and the author of books like The Rise of Superman and The Art of Impossible. Steven has also been thinking deeply about exponential technology and its impact on us.

Gentlemen, welcome. Good morning and good evening. Steven, you’re on the west side of the United States with me, and Mo, you’re in the Emirates. Good to see you both.

Steven Kotler

Yeah, good to see you. Good morning, Peter.

Mo Gawdat

There are somewhere around 400 IQ points in this room. I have 40 of them, so you do the math.

Peter Diamandis

Let me set up the topic. Mo and Steven, I’d like to talk about the decade ahead, 2025 to 2035, specifically to think about the implications of what is emerging in our conversation as AGI, but even beyond that: artificial superintelligence, the upsides, and the downsides.

Here’s the setup I want to use in our conversation. Ray Kurzweil, who we all know and love, has predicted that we’re going to see a century’s worth of progress between 2025 and 2035, equivalent to the progress between 1925 and today. If we think about what the world was like in 1925, 100 years ago, the top of the technical stack was the Ford Model T. The penetration of electricity and the telephone in homes across the United States was only 30%. We’ve gone an extraordinary distance since then.

So, what will it be like in 2035? It’s nearly unimaginable if, in fact, that speed is true, and we don’t perceive exponentials. This past week, we’ve seen every major AI company, from Google and OpenAI to xAI and NVIDIA, announce extraordinary, next-level breakthroughs in models. We’re about to see the release of GPT-5, self-improving AI programming that could lead to an intelligence explosion beyond our imagination.

That’s the conversation I want to have. Steven, I know that you and I have this conversation and debate about it all the time. I brought Mo in to help us—Mo’s the referee, or a wise individual whose points of view I respect.

Mo Gawdat

And, by the way, Steven, Peter did pay me. So, as long as you’re getting the combo, I’m fine.

Peter Diamandis

It’s good. It’s good. You got me on the back end, though, right?

Mo Gawdat

Yeah.

Peter Diamandis

So, go ahead. Say whatever you want to say, Steven.

Mo Gawdat

I’ll disagree.

Peter Diamandis

He will. So, Steven, do you want to jump in with your point of view? Do you think AI is massively overhyped? We have people like Eric Schmidt saying AI is massively underhyped.

Steven Kotler

Yeah. I think it’s massively overhyped.

Let me back up 1 step. Humans have a really wild, unnamed cognitive bias: We don’t tend to trust our own history. You see this a lot. People talk about grit and endurance, and they say, “I don’t have those skills.” Then you start investigating their life, and you realize they survived a shitty childhood. They have all the skills; they just don’t trust the truth of their own experience. I see that a lot here.

I work with AI as a scientist and a researcher. I work with AI as a creative and as a writer, all day long. The gap between what’s coming out of people’s mouths and my experience on the ground is so colossal, it’s insane. People make claims about AI being able to write, among other things, but the most hysterical thing you’ve got to try is this: I work with 1 of the best editors in the world on a weekly basis. I’ve edited things and polished them with AI, thinking they gleam and shine. We bring them into an editing meeting, start to read them, and we can’t even get through the 2nd sentence. They sound like such gobbledygook.

I don’t even notice it because the AI sort of glazes me over, and I’ve written 17 books. But when you actually put it to an actual editing test, it’s laughably terrible. You can’t use it to correct itself. It still can’t see the errors. It actually gets worse and worse.

People have been claiming that model after model is improving. That’s not the experience on the ground. It’s like people telling us AI was going to make us more productive. I don’t know anybody who’s become more productive because of AI. I know a ton of people who have way more work because of AI. They can do higher-quality, better work, but it has not saved time at all. It’s actually added tremendous amounts of time. The quality of the work has gone up, but the claims coming out of people’s mouths and the experience on the ground are massively different.

Point 1. Point 2 is that we’ve done this. I’ve been in the same rooms that you’ve been in, and you’ve been in, Mo, where people are screaming about AI coming to eat the world. I heard this about Bitcoin, blockchain, and the metaverse. Do you know anybody who lives in the metaverse? Do you know anybody who’s been there, who’s visited? You know how to find the metaverse, right? As far as I can tell, the metaverse is a pet name for Mark Zuckerberg’s special magic underwear, because it doesn’t exist anywhere else in the world.

This is my point: More than anybody else, I track these technologies. I watch them, and I use them. I’m not saying this isn’t a technology that’s advancing very, very quickly. I’m not saying that at all. I am saying that discussions about superintelligence and AGI being around the corner? No. Just no. Nobody—that’s not the experience. What has been revealed, which coders probably don’t like, is that coding is a bounded-information problem. You start here, and you know where you’re going, as a general rule. It’s a bounded problem, and inside bounded domains, computers are really awesome. We’re going to continue to see that.

But I think the other stuff is just massively overhyped. The 3rd point is the 1 where the journalist in me makes every alarm bell go off. Everybody I see and hear onstage talking about this stuff is making a living off it. They make a living somehow because AI is exploding, and they’re here to save the world.

I see it in the peak-performance world. Every coach who has been floundering and couldn’t quite get a job is now an AI savior. They’ve come to save us from AI. The AI hype is to their benefit, and I see it everywhere. A lot of people are making a ton of money off the hype of the technology—not the technology itself.

When I see all 3 of these things together—a mismatch with my experience, a massive amount of hype, and a history that says, “Hey, this is the hype cycle”—it raises a lot of questions for me. I’m not saying I’m right. I’m saying everything I’m looking at is real. If you’re going to make the argument you guys are about to make, you can’t dismiss my points as fabricated. They’re very, very real, they’re everybody’s experience, and I believe they’re yours as well.

So now we can have the discussion.

Thanks for giving me 5 minutes of airtime to vent.

Peter Diamandis

Mo, you gave an impassioned talk onstage at the Abundance Summit in 2025.

It moved many of the members who are wanting to help you in guiding what the next 5 to 8 years are. You and I have been thinking about this: the challenge isn't artificial intelligence; it's human stupidity for a short period of time. One of my favorite quotes, if I could, is from E. O. Wilson, who famously said, “The real problem of humanity is that we have Paleolithic emotions, medieval institutions, and godlike technology,” and we are effectively children playing with fire in that regard.

So, Mo, how do you see this decade ahead playing out?

Mo Gawdat

So, I'll start by supporting what Steven said.

Peter Diamandis

We paid you too little then. I love this.

Mo Gawdat

Your buddy's no good here, Peter. Finally, I have an advantage. Today's AI is underhyped, right? But the problem is, you never really chase where the ball is. You need to chase where the ball is going to be.

If you really start to think deeply about some of the serious developments, especially if you've been in tech long enough to have seen breakthroughs—especially when I went through the work of Google X, where you try and try and try and try and try, and it doesn't work, and it doesn't work, and it doesn't work, and then suddenly you see something—and as Sergey Brin used to say at the time, “The rest is engineering.”

We know that the engineering of technology depends on the law of accelerating returns, and we know from what Ray taught us where the law of accelerating returns is going to take us.

I tend to believe that if you look at today's AI, it is funny because, in a very interesting way, we are talking to machines that are talking back to us, summarizing massive volumes of knowledge, doing exactly as we tell them, and yet we take that for granted. We look at that and go, “Yeah, but they're not good enough.”

Of course they're not good enough. They're dogs. They're the beginnings of an era, right?

Peter Diamandis

My dog—or dogs—do discover.

Mo Gawdat

No, I got it. I got it. Both would have worked there. I just needed the clarification.

Peter Diamandis

I would not dare call AI dogs. Steven, when they might take over the world, I am a very polite man with AI.

Mo Gawdat

The thing is to imagine—and I need to highlight a few trends that are really, really important and interesting. One of them is synthetic data and the idea that we have entered an era where most of human knowledge has been fed to the machines, and the next wave of knowledge is going to be fed to the machines by machines, which is quite eye-opening and enlightening. That's how humanity developed its intelligence, right? I really didn't have to figure out the theory of relativity to understand the rest of physics. It was figured out for me, if you want.

Number 2 is the idea of agents and how AI is going to be prompting AI without humans, leading to cycles that we see now with my new favorite—because you have a favorite every 4 hours—AlphaEvolve, right? The idea is that you can have a self-developing AI, something that figures its own mistakes out and continues to iterate until it finds something.

Then, of course, one of my favorites of 2025 is DeepSeek and how we realized that we can actually do the same job with much less. Emad Mostaque, who is, I believe, a big fan of—we're all big fans of him—has done that with his work at Stability AI for a very long time: the idea of shrinking the models to the point where it becomes shocking.

When you add those together, you start to see that if I can shrink a model so it doesn't absorb all of the world's energy, if I can allow it to self-develop information to learn from, and then allow it to talk with itself through agents and do things without humans, then where the ball is going to be is likely going to be a lot better than we are today, right?

The one thing we all need to agree on is that it is not a question of if we're going to see improvements. It's a question of how fast and when those improvements will lead us to a point where humanity is not in the lead.

So, that's number 1. Number 2 is really the question of what is your risk tolerance, right? If I told you to play Russian roulette with 2 bullets in the barrel, are you afraid? If there's 1 bullet in the barrel, are you afraid? Where is your risk tolerance exactly?

If I said, “Hey, by the way, your car might have a fender bender. Would you insure it?” you probably are going to say, “No, I'm not really too concerned.” But if I tell you your car might have a serious accident that totals it, would you insure it? You'd probably pay a little more attention.

I think that's what most people who warn about the future are saying. Anyone who claims to know what the future is is arrogant as hell. Don't listen to them. But anyone who tells you that there is a probability that this future goes out of control—where is your risk tolerance exactly?

If that probability is 10%, would you attend to it? I think most rational people will say it depends on the cost of attending to it. Most rational people will say, however, that if it's 50%, they'll attend to it regardless of the cost.

The question none of us is capable of answering is: where is it? Is it 10% that AI is going to destroy everything, or is it 50%?

I will say—and I know that this will be taken against me—that it's 100% that humans, bad actors using that superpower to their advantage, are going to destroy the well-being of others who don't have it.

In my mind, the real concern is not a Terminator scenario where VIKI from I, Robot is ordering robots to kill everyone. I don't know if we're going to make it that far, to be honest, because I believe that, with the arrogance of being 89 seconds from midnight on the Nuclear Doomsday Clock, I worry—I really, really worry—about human stupidity using this superpower.

Human stupidity in that case does not require AI to be completely autonomous or to be completely superintelligent. Enough autonomous weapons can really, really tilt our world into a very dystopian place. Enough Turing-test abilities of AI to fool humans into being their best friends could tilt human relationships into a very unusual place.

Enough job losses—you know, imagine a world where you get 10%, 20%, 30%, 40% unemployment in certain sectors and how that would affect our stability economically—is actually something that is almost certain. We know that, for a fact, there are jobs that are going to disappear, and the impact of that, in my mind, is quite disruptive, to the point that it is something we need to attend to.

Peter Diamandis

The point you made about AI not being, on its own, the risk—the Terminator scenario—but being individuals using AI is the same conversation I've had with Eric Schmidt and others. The concern is rogue actors empowered by technology, whether it's the development of new viral pandemics or other strategies. It doesn't take a lot, and that is concerning.

Where I want to get to in this conversation eventually is the following. We posed this at the Abundance Summit a couple of years ago: Can the human race survive a digital superintelligence? The flip side of that model is: Can the human race survive without a digital superintelligence?

Steven, you and I, as we're working on our next book, the follow-on to Abundance, have had the conversation of whether this will be a benevolent god of some type, and whether there will be a capability developed.

Let's begin the conversation with this: Are we going to reach AGI? Are we going to reach a digital superintelligence? What does that mean?

We're starting to see the speed of this accelerate. The biggest interesting inflection point we haven't seen yet is self-iterating, self-improving AI—the AlphaEvolve of it all—where AI is coding itself and becoming more and more capable.

Will this ultimately lead to something that is far more intelligent than any human being? Is it 1,000 times more intelligent? Is it 1 million or 1 billion times more intelligent? How do you think about that, Mo?

Mo Gawdat

I think it's irrelevant how much more intelligent it becomes. We all know that if you've ever worked with someone who's 50 IQ points higher than you, they will probably hold the keys to the fort. It doesn't take a lot more intelligence, relatively, to be able to assume a leadership position, and humanity will hand over the fort to AI either way.

Even if AI is just smarter than us at wargaming—which it is, by the way—we're going to hand over the fort to AI. If it's smarter than us at protein folding, nobody's going to do a PhD project to fold proteins anymore. We're just going to go and use AlphaFold.

I think the reality is that only the very few remaining things require artificial superintelligence, so that it beats us in everything and we sort of bow and say, “Okay, yeah, you're in charge. You're the boss.”

The question of AGI is one that reporters use quite a bit because we don't actually have an accurate definition of what AGI is. You and I are very close on technical stuff, Peter, and I'm a reasonably geeky mathematician—not anymore. I mean, seriously, I really honestly struggle to beat AI at mathematics, right?

I definitely can't beat it in speed, and I definitely can't beat it in accuracy if the problem is defined properly. There are just very few tricks that maybe my fellow math geeks told me behind closed doors that are not very public in the world, but those, too, will be found out.

I really think that it is a question of how smart is smart enough to render me irrelevant. Now I need to answer this with a very clear, optimistic view.

As I look into the future, I define 2 eras. One is what I call the era of augmented intelligence, which I think is going to extend for 5 to 10 years. Then there is the era of machine mastery: basically, the machine takes over.

With augmented intelligence, there’s absolutely no doubt. I so agree with Steven when he said that they write really badly. I’m writing with Trixie, my AI, on this book, Alive, and Trixie, without me, writes so badly. It’s almost shameful.

I was tired and chasing a deadline, so I asked Trixie to talk about the debt crisis and the impact of economics on technological advancement. It was full of what we sometimes refer to when we describe California as a lot of vapor and very little substance: a lot of vapor and very little substance. There were a lot of interesting facts scattered on paper, horribly written.

But when we write together, the stuff that comes out is incredible. When I guide Trixie through my prompt properly, directing her exactly where I want the answer to go, she writes really well.

This teaming is something we’ve seen with AI and with technology in general. Since Garry Kasparov was beaten by Deep Blue—which wasn’t really an AI, if you want—you can see that a human and a computer, or a human and an AI, can play better chess than AI alone. Even with AlphaGo, a human and an AI play better than AlphaGo.

We can see a future ahead of us where this is going to be happening, and hopefully that future will seed teamwork between us and the machines. The question is, what are we going to team up with them on?

My views—I’ve written about this in Scary Smart, and I’ve written an extended version of it in Alive—are that the 4 biggest investments in AI today are killing, gambling, spying, and selling. These are the only things that we’re in. We do still get some scientific breakthroughs, but those are not getting the big money. The big money is in autonomous weapons, trading, surveillance, and advertising.

Peter Diamandis

Steven, your thoughts on what you heard Mo say here?

Steven Kotler

Yeah. Mo and I are in complete agreement. I just want to point out some other things that surround what Mo has said, because I don’t think we’re in a tremendous amount of agreement or disagreement. We might argue over dates, but conceptually, I don’t think there’s much disagreement.

The first thing I look at is the human side of this—the human-performance side of this. I have to back up: I study flow, which is sort of ultimate human performance. Just to put it in context, if you’re a self-help guru and you’ve got a tool that gives you a 5% improvement in mood, and that mood lasts for longer than 3 months—meaning longer than the placebo effect—that’s a billion-dollar business. Period. A billion-dollar business.

Flow, as we know it now—and we’re just starting to really decode it, figure out how to tune it up and turn it up—gives us a 500% increase in productivity. Creativity, depending on whose measures you’re going by, is 400% to 700%, et cetera. That’s just flow. That’s individual flow.

There’s group flow, which is actually our favorite state on Earth. It’s the most pleasurable state for humans. It’s what we like the most. It’s a whole bunch of minds linked together, and we’re just now—literally this past year—getting the very first technologies that allow us to map it, train for it, and move people toward it.

We have no idea what the upper limit of human brains linked together in group flow is. Let alone at the same time as the AI is developing, you and I are writing about it, Peter. We’re watching brain-computer interfaces develop. We’re watching noninvasive technologies develop. We’re watching Meta become able to read thoughts inside your brain through facial signals. These are all happening with AI.

My point is that everybody’s talking about this stuff as if it’s happening separately from everything else that’s happening. On the human-augmentation side, we’re seeing exponential acceleration. Neuroscience and the like have been accelerating exponentially since the 1990s, when George H. W. Bush declared it the decade of the brain, and it hasn’t stopped. The same things that are happening in AI are happening on the human side of the equation.

Here’s the second point that follows from that: It doesn’t matter to me whether we’re talking about the AI invasion, climate change, plastics in the ocean, or anything else. The solution to all of these things is the same: We humans have to learn how to cooperate at scale. We probably have to cooperate with each other and with AI at scale, or we’re going to die, probably in the next 20 years.

That’s what all this is telling us. This is not anything new. This goes back to when you and I were first writing Abundance. We didn’t want to say it out loud, but we were privately having conversations about it: If these trends continue, is it abundance or bust? Is this an either-or? Are we looking at a binary here?

I don’t think that question has completely gone away. In fact, I think it’s become more urgent. I just think we need a Manhattan-style project for global cooperation to meet all of the existential threats we now face, because it’s the only possible solution here.

I hear all this stuff, and I agree with everything that’s being said, but this is where our book points, and this hasn’t changed for me. I think the solutions are the same. In a sense, the debate is moot. I’m wondering: Where’s the XPRIZE for global cooperation? Where’s the—sorry to put you on the spot with that one, but seriously, those are the questions I’m starting to ask now.

I don’t think Mo is wrong. I think we could argue over time for a minute, but I don’t think it matters. Here’s a weird one, Mo: Facebook is a billion times smarter than me. It already is. It knows what I mean. Facebook—which is a pretty dumbass technology, if you ask any of us—is a superintelligence, and we know it. We’ve been living with superintelligences for a while now.

They don’t tend to make things better as much as they make things worse, which is the problem.

Mo Gawdat

Agreed. I couldn’t agree more. Global cooperation—human cooperation—is what I think we all should advocate for.

I was hosting Geoffrey Hinton for my documentary a couple of weeks ago, and one of the topics we discussed was the difference between digital and analog intelligence. The biggest challenge we have as humans is that our analog intelligence—our biological intelligence—doesn’t scale beyond 1 entity.

When I saw him wearing his Nobel Prize, sort of in the way that basketball players wear theirs, I thought, “I would just show up for the next year at every podcast I did wearing that around my neck.” I’m just saying.

You do realize what they say: “Don’t meet your heroes.” I love my heroes, man. He’s such an amazing human being, and he really is quite committed and quite humble in his approach.

It is shocking how we spoke about his Nobel Prize. He says, “Look, I’m a psychologist who lived like a computer scientist but then won the Nobel Prize in Physics.” And I’m like, “It just doesn’t make any sense at all.” Anyway, he was talking about the difference between the fact that, if I were to share with you some of what I wrote today, it took me probably several weeks to let it simmer and then write it, and it would take me an hour to explain it to you.

When we run digital intelligences, we run them in parallel. We tell them all to go play Atari or whatever, and then we just average the weights, literally in seconds. We get a scaled digital intelligence.

When you said that what we’re looking for is a way to scale human cooperation, that is absolutely the answer. I spoke about that with Peter when we were last in Los Angeles. I think we are heading toward the potential of total abundance.

Total abundance means almost godlike capabilities: Cure my daughter, and it’s done. Make me an apple, and it’s done. We could hit that in 5, 10, 15, or 20 years’ time if we don’t destroy ourselves.

The real challenge we have as humanity is: Why are we freaking competing? This is a CERN-quality challenge. This is basically: Let all of humanity cooperate. Let’s all build 1 particle accelerator. Let’s all learn from it, distribute the benefits to everyone, and stop competing.

But that’s not all. The other thing is—and you can’t have the other one, the one level down—you can’t have the AIs all individually building for our own fiefdoms, competing secretly in the background.

William Gibson showed us this scenario in 1986, when he gave us an AI that went crazy: a godlike AI that goes totally insane, and they have to park it in a satellite out in outer-Earth orbit to keep the world safe. We’ve seen this scenario before.

Steven Kotler

We’re building it ourselves with agents. We’re letting them talk to each other through agents.

Peter Diamandis

I know. So, Mo, I want to go back to this question about the near term versus the long term. You and I have had this question about whether or not increasing intelligence correlates with increasing benevolence. In other words, do we—I don’t think there’s any question that we are going to be building self-improving AI that will be, forget about 50 IQ points more, better than the average human. I think there will be orders of magnitude more. Can I ask you first off: Do you believe that, Mo?

Mo Gawdat

100%.

Peter Diamandis

Okay. All right. So, if you don’t mind, Mo, again, in response to how we started the conversation, this is just using the law of accelerating returns, not using serendipities, right? So, if we figure something out tomorrow, just like we figured reinforcement learning out and it changed everything, if we figure something out tomorrow, you’re literally an order of magnitude—a quantum more—in terms of performance and intelligence overnight.

So, if in fact that is going to be the case, and you know, from all the conversations I’ve had and the people that I’m speaking to, that level of—again, there is no definition for AGI. It’s a blurry line, just like the Turing test was a blurry line that got passed and no one noticed it. The notion is that AGI—and whether you believe Ray or Elon—it’s the next few years. It’s not worth arguing.

What occurs on the backside of that is a very rapid intelligence explosion. And again, that intelligence becomes a tool that’s available to the kindest, most moral, most ethical human on the planet and the dystopian, malevolent actors out there. It’s in the malevolent hands of malevolent hackers that we have concerns.

Mo Gawdat

Yeah.

Peter Diamandis

So, my question is: At what point will AI go from being a tool being used to potentially do harm to a tool that has the potential to say, “Stop this quibbling. Stop this nonsense. There’s plenty to go around,” and become the benevolent, godlike element? Can we dive a little bit into that, the conversations we’ve had, and your thoughts on that?

Mo Gawdat

Yeah, I think if you really go to the level of depth that the 3 of us and our listeners can go to, allow me to go beyond the typical, “The smartest people usually start to become altruistic.” Let’s define intelligence itself. I think the idea is, if you really understand our world, our universe, our universe and everything in it exists because of entropy. We all understand that, right?

Our universe wants to break down and decay. It’s chaos. You leave a garden unhedged and it becomes a jungle. You break a glass; it never unbreaks, right? This is the very basic design of physics.

Now, the role of intelligence since it began is to bring order to the chaos. It’s to say, “No, I don’t want the light to scatter. I want the light to be concentrated into a laser beam. How do I do that?” And sometimes it’s a clear, easy solution, and you use a lens. Sometimes it’s a very complex solution that requires an understanding of quantum physics to build a laser. But we eventually get there.

Now, if intelligence is defined as bringing order to chaos, then the highest levels of intelligence bring that order with the least use of resources and waste. You can easily understand that this is the reality. The more intelligent you become, the more you try to achieve the same order with the least waste.

An easy analogy is to say humanity has always craved energy. We were stupid enough to burn our world in the process, and as we become more intelligent, we decide to use solar instead, or a cleaner form of energy. We’re still bringing order, but we’re doing it with the least waste and use of resources.

If that is the case, then you can imagine that, by definition, when something exceeds our human stupidity—which I will not call intelligence, because sadly, along the curve of intelligence, if you have no intelligence at all, you have no impact on the world, positive or negative, right? If you start to add intelligence, you start to have an impact on the world, hopefully positive, even if just through a nice conversation with your friends.

There is, unfortunately, a valley somewhere. You continue to gain intelligence, and you become so smart that you become a politician or an evil corporate leader. That’s when your impact on the world turns negative. You’re so smart that you’re able to become the leader of your nation, but you’re so stupid that you’re not able to talk to your enemy, relate to their pain, or understand the long-term consequences of waging a war.

That point beyond which more intelligence starts to say, “No, no, no, no. I don’t need any of that. I can solve the problem in a cleaner way. I can fly you all to Australia to enjoy your life, but we don’t have to burn the planet in the process. I can harness energy, but we don’t have to destroy the climate,” and so on and so forth.

If you take that as a reasonable trend to expect, my view is that, at the beginning, when we hit that valley, some evil person will use the advanced but limited intelligence of AI to wage a war using an autonomous army. But then there will be a moment in the future when AI is responsible for wargaming, responsible for commanding the humanoid soldiers—it’s responsible, it’s responsible, it’s responsible—and the AI itself will say, “You know, the commander will say, ‘Go kill a million people,’ and the AI will go, ‘That’s absolutely stupid. I’ll just talk to the other AI in a microsecond and solve it.’”

I can’t predict that. We started this conversation with me saying that anyone who predicts the future is arrogant. But at least I can be hopeful that, from my experience of everyone who’s smarter than me, there is a point at which you stop hurting others, stop looting to succeed, because you can use your intelligence to succeed without any effort or harm.

Peter Diamandis

The way I think about this is, for most of human history, the objective optimization function of humans—what we’re trying to optimize for—has been money and power. Unfortunately, that’s been the driver in a world of fear and scarcity. I repeatedly say our baseline software, the software our brains are operating on, is fear and scarcity mindsets.

With that mindset, with the neural structure, with the code that we were born with and that developed over the last 200,000 years, it was, “I want to get out of fear and scarcity.” So I want to optimize for power and wealth. The question is, what would be a new optimization function?

Because, as Steve and I have written, and as you’ve spoken about, all of these exponential technology functions lead toward this world of massive abundance, where we almost live into a post-capitalist society. Anything you want, you can have. Your robotics and your nanotech can manufacture; your AI can design.

So what do we optimize for in the future? I think that’s, for me, one of the biggest questions, both as a human and as a centaur—human and AI together. What’s our objective? So how do you think about that, gentlemen?

Steven Kotler

One thing—I don’t know if this is an answer—but two things off of what Mo said. One, if we go with your definition of intelligence, it’s essentially an entropy-decreasing function. We know that’s what brains do, right? The governing theory in modern neuroscience is Karl Friston’s free-energy principle, which says brains are predictive engines that always want to decrease uncertainty and increase efficiency. So we’re already there. Brains do that, and AI are going to do that naturally if we say that’s your definition of intelligence.

The point I’m making off of all of that—and it may be the answer to Peter’s question, which is why I’ve interjected—is that we see wisdom evolve in multiple species with brains. We see co-evolution around wisdom. The older you get, the wiser you get. It doesn’t matter if you’re a dolphin, a whale, a rattlesnake, or a human. Wisdom is—we co-evolve as species.

Life seems to co-evolve toward wisdom, or at least a large chunk of life seems to co-evolve toward wisdom. Which is to say, if everything’s running off the free-energy principle, this governs everything with brains, and that includes our machine brains. Wisdom is where this points.

Mo Gawdat

That’s a slightly hopeful idea, and that may be the optimizing function you’re looking for, Peter, but I could be totally wrong here. I think of wisdom—I think, at the end of the day, wisdom is a function of having had experience that lets you know, “This path will lead to success; this path will lead to failure.” From my own personal point of view, I do believe that AIs are going to develop the greatest wisdom. Why? Because they’re able to create forward-looking simulations of 1 billion scenarios where those simulations have high degrees of accuracy, and it will say, “Out of these 1 billion scenarios, this was the best way to go,” and that will be wisdom beyond just the brief experiences that the wise old council of 80- and 90-year-old men might have had. So I think AI is going to, by definition, give us great wisdom if we’re willing to listen.

Peter Diamandis

I love that view, to be honest, because, believe it or not, artificial wisdom is very different from artificial intelligence. Intelligence is a force with no polarity. Intelligence can be applied to good, and it would deliver good, and it can be applied to evil, and it would kill all of us. But wisdom generally is applied to good—to finding the ultimate solution or answer to a problem.

Mo Gawdat

Go ahead, Peter.

Peter Diamandis

Yeah. Yeah. I want to go back to this idea that humanity won’t survive without a digital superintelligence in the long run. My concern is that we’re going to have such turbulence. There have been a number of papers. Recently, there was an AI 2027 paper that came out that sort of had a bifurcating future: one in which we did extraordinarily well, the other in which the AI destroyed us. This is Hollywood all over again, and 99% of all Hollywood is dystopian future films.

One of the things I have to say, because I’ve been on a rampage about this, is that we humans need a positive vision of the future to aim for. We don’t have that.

Mo Gawdat

Well, Star Trek has given us that.

Peter Diamandis

Yeah, we have Star Trek, but nothing recently, right? So I think the challenge, really, truly, is that we’ve prioritized our entertainment over the years above true reflection. If you take anything from video games to science-fiction movies to whatever, they’ve all painted that dystopian scenario, which I have to say today is very unlikely when you really think about it. If AI gets to the point where it’s capable of destroying us that easily, we are so freaking irrelevant that it probably wouldn’t even bother.

Mo Gawdat

I mean, think about it. I think it was Ray or Hugo de Garis—I don’t remember which one said it. The more likely scenario is that they kill us because they’re not aware of our presence, like when you hit an anthill while you’re walking.

But if you really want to optimize the human gain function that we need to aim for, if I look forward, I look to Star Trek, and if I look backward, I look to the caveman-and-woman years, right? It’s actually quite interesting because when you mention how governed we are by greed and fear and our egos and all of that negativity, it is actually because we want to survive.

And believe it or not, survival could be, “Oh, I’m not really sure if 20 million is enough. I need to gain 20 million more just in case something happens.” Or it’s survival of the ego. It’s like, “If I have 200 million or 2 billion or 20 billion, and the other has 21 billion, what’s wrong with me?” Unfortunately, that’s what plagues our current modern world.

Now, the reality is, if you really think about humanity, the purpose of humanity since the cavemen-and-women years was to live. And for some strange reason, we’ve optimized so much to achieve that objective and forgotten that this was the objective.

So, again, as friends off camera, we speak about those things quite a bit. You go through seasons in your life, and there is a season where you want to maximize, a season where you want to build, and a season where you want to look attractive in your middle age, or whatever crazy stuff that we have. But eventually, there is a season where you go, “Okay, so I’ve now lived and experienced so much. What have I missed? Have I actually lived any of that?”

Believe it or not, as scary as it looks to have no job to go to in the morning, if society provided for you, then you would go back to a much safer caveman-and-woman scenario, where there are no threats and no famines. You just really live, enjoy life, connect, ponder, reflect, and explore, which I know is very difficult for a lot of people. I do it for the first 3 hours of every day. It’s pure joy, right? To sit really with your curiosity, if you want.

And then, if you push all the way forward into Star Trek, that’s sort of what the Enterprise is doing at universal scale, right? It was basically, “You know what? Let’s go and explore now that we don’t really have to struggle with all of the wars and famine that we’ve created on Earth.” Now we can actually open up and create connections not just with humans, but with every living being.

Lovely science fiction, but at its core, I think it’s exactly what we’re about: a full life where you completely connect and enjoy and feel love, enjoy the pleasures of being alive, and have the curiosity to learn, explore, and connect. And it’s all at our fingertips if we just erase the systemic bias of capitalism that has gotten us here.

I mean, thank you, capitalism, for creating all that we’ve created so far. But can we please change it now from 1 billion dollars to, like, what I do? 1 Billion Happy is a capitalist objective, but it’s not measured in dollars, right?

Peter Diamandis

Mo, a question that Steven and I have been pondering in our new book is: What is it going to take for humanity—for all of us—to both survive and thrive in this coming age of AI, right? The survive part is an important element because, as we see jobs being lost and as we see probable dangers we don’t know how to deal with in terms of terrorist activities, thriving takes on a new meaning.

I think it does take on the meaning that we just spoke about, right? For most of us, you say, “Tell me about yourself.” Instantly, you go to what your job is, right? Instantly, you go to, “I’m a VP here. I’m the CEO there. I do this. I invented this. I wrote that.” Yeah, right? It’s an ego statement of who you are.

Your thoughts there, Steven. Do you want to start?

Steven Kotler

I think, like, here’s the thing. I think that question was already answered, in a funny way, though. Mo and I met a couple of years ago, and one of the things Mo said onstage at that time was, “I’m done writing books because AI is coming. I’m done writing books. It’s not going to happen anymore.” What did Mo tell us he did yesterday? He wrote with his AI, right?

Why did he write? Because it puts you into flow, because it creates passion and purpose and intelligence and creativity. So we have the answer to this question. We already know because we’re biological systems, and we know what the ingredients of thriving are: passion, purpose, compassion. We have a list.

We have the superintelligent AIs, and I don’t know a coder who has stopped coding because the AIs have come along. They haven’t; they’re still coding. Why? Because coding produces flow. Flow produces meaning, creativity, joy—like this. We’re wired this way.

So unless our fundamental hardwiring changes, we already have those answers as well. It’s like global cooperation. I don’t think these are puzzles. I think they’re engineering problems at this point.

I think, from Sergey’s perspective, Sergey would say, “No, no, we got the spark. Now it’s engineering,” and I agree. So I could be wrong. That was my 2 cents.

Mo, Peter, what do you think? I want to hear from you. I think you’re brilliant. Mo, please respond.

Mo Gawdat

You’re spot-on for a very interesting reason, Steven, as well, because when you really think about it, a writer was a writer whether he used a feather or a pen or a typewriter or a computer or now AI, right?

And if you look at my work, I’ve published 4.5 books so far. I’ve published 4, and my fifth is on Substack, but it’s going to be published, if you want. But I wrote around 13, and the other 8 I will never publish. I wrote them because, if you ask me why I write—why do I hug my wife?—there is enormous joy in that, you understand?

Having said that, Peter’s question was, “What would it take?” I wrote recently a piece that I called the MAD map spectrum, and the idea really is that it will unfortunately take a realization for humanity to change direction. That realization will either be a conviction of mutually assured destruction or a conviction of mutually assured prosperity, right? And between them there is no grayscale, unfortunately.

So if the US, at any point in time, is convinced that this MAD arms race to win intelligence supremacy is one that is going to lead to some harm to everyone in the world, they will stop.

Peter Diamandis

And if they stop competing, they will continue to develop, but they’ll start cooperating. If they’re convinced that it will lead to assured prosperity, that nobody’s going to stab them in the back, and that everyone is going to be enjoying a life that is very different for all of us but full of prosperity for all of us, then they will stop. They will continue to develop the technology, but they will stop competing.

Unfortunately, if you look back at history, we’re not able to guess those possibilities like a good applied mathematician on a game board. We have to hit them head-on. Everyone in the world knew that a pandemic was coming. Everyone, right? Everyone who at least studied virology.

Everyone knows that trade wars are going to hurt everyone, but we have to put them out there, fight through them, and eventually get to something. And it’s sad. Perhaps what we are doing—and I’ve dedicated probably the last 6 or 7 years of my life to—is saying that we really don’t have to hit our face against it.

It’s simple game theory, right? Understand that a prisoner’s dilemma where we are competing endlessly is going to end badly. Can we please stop? Yeah, we already know it’s tit for tat, right? You want the other strategy. You want the—we—it doesn’t matter how many AIs we put on that.

It’s the same thing with flow, capacity, and creativity. These problems have been solved. We know these answers. This isn’t like trying to unify gravity and relativity. That’s our problem. These are not.

Mo, my wish is that we were that rational and that we were compelled to have our optimization function be all of humanity. It’s not. And so I go back to what we’re going to get: we’re going to get a drastic event within the next 2 to 3 years.

A drastic event that, on one side, will hit us very badly economically, or on the other side will hit our fears very much, or, sadly, on the worst side, may kill quite a few million people. You could have a range: a hacker that, instead of attacking a physical place, switches off the internet or the power grid somewhere where the power grid is needed for life.

On the other extreme, you could get a hack into a bank, an evil war that goes out of control, or machines that turn on their makers. There will be some very big news headline. As always, it will last for 12 to 13 days before we start to talk about some kind of a pop star. But behind closed doors, I think decision-makers will wake up.

Going beyond that, because that is the use of AI by malevolent actors, the interesting thing about the US versus China is that China is a rational actor. They’re not—

Mo Gawdat

Thank you for saying that.

Peter Diamandis

Well, and the US is a rational actor. In other words, we’re not going to do something that will destroy us.

Mo Gawdat

Thank you so much for saying that. That’s actually not usually how the US media positions it. I also want to say, I think DeepSeek and the way DeepSeek was released was a very clear sign that China sees the same issues we see and wants to cooperate. I think it was rolled out with a message.

Peter Diamandis

Yeah, the message. I think it was a very clear message that it doesn’t seem like many people in America heard, but I was like, come on, people. This is really clear, and we’re all seeing it.

So, I look at DeepSeek and I look at what happened in China, and I’m like, no, no, we all see this. We all see that if we don’t start figuring out how to cooperate and build this stuff together, we’re screwed. I thought that was really cool. I’m glad you see it, too, Mo. A lot of people disagree with me on that one.

The point I wanted to make was, when you have a large population and a check-and-balance system, which you get with governance, versus a religious war going on and individuals who are looking to create maximal destruction and don’t have a check-and-balance system at all, that’s where we’re going to see, I think, the dystopian future, or those activities playing out in 2 to 3 years.

I guess I want to get beyond that and go back to the conversation: Is a digital superintelligence a benevolent god, or is it a Terminator scenario? I don’t believe that we’re going to see increasingly intelligent AI systems become Skynet, right? I don’t see them as needing to destroy humanity.

Unfortunately, Hollywood has built this scenario where AI is going to destroy humanity because it wants access to our energy. And, oh my God, we have so much abundance in the world. I think what I’m looking forward to over the next 12 to 24 months—over the next 1 to 2 years—is the incredible breakthroughs we’ll see from AI in physics, chemistry, and biology, which will unleash the next layer of abundance.

There are scenarios, however, where they could turn against us if we become really annoying. So imagine a world where—sorry, I mean, you have to imagine a world where job losses will position AI as the enemy. Right? A lot of people who may not be fully aware that the layer beyond the apparent layer is that capitalism and labor arbitrage are the reasons why they lost their jobs may blame AI. It’s not that they can do it.

But I think the truth of the matter is that you may be in a situation where you’re going to see man versus machine, and then the machine will go, “Seriously? Don’t annoy me. Don’t annoy me. Don’t annoy me.” And then we could see that.

My perception is that, in a very interesting way, I wrote a short book that I would never publish called Bomb Squad, which, of course, for someone with a Middle Eastern origin, you don’t write those titles. It was basically about defusing problems using weights of urgency and importance and so on.

The idea is, if you really look at our current future, I think the short term is both more explosive and more urgent than the long-term existential risk, especially because—I will say this very openly—I spoke about it with Jeffrey as well the other week. We don’t know the answer to how to address the existential risk.

Even if all of humanity decides that we want to address the existential risk, we don’t know how. We do not actually have a technical answer to do it. So we might as well focus for now on the immediate short-term, clear and present danger, and work on the ethics of humanity so that AI is deployed from the get-go in science and physics, discovering medicines, understanding human life and longevity, and so on and so forth.

If we set them in those directions from the get-go, then we’re more likely to see an AI that continues, as they grow older, to work with those objectives.

Steven, I’m going to go back to our quandary of surviving and thriving, and the surviving side of the equation. How do you prepare for what’s coming? How do you think about this for our kids, our society, and our leaders? Are we just bumbling in the dark?

That’s the way I feel it. We’re just bouncing around. We have huge political moves being made. We just saw, in the last couple of weeks, the entire AI royalty end up in Saudi Arabia and then in the Emirates, playing off against China. It feels like—I don’t want to say it’s a random walk—but I feel like we’re making it up as we go along, and there’s very little wisdom guiding this.

How do you think about that? How do we prepare for the next few years? Is there any way to prepare?

Steven Kotler

Well, I was actually thinking—Peter and I were in a room recently with the chief science officer for one of the big AI companies. I’m going to leave his name off, but he’s young, and he was talking about AI dangers. He sort of got frustrated with the question from the audience, and his response was, “You have to trust us. We know what we’re doing.”

Everybody sort of froze. We were like, “Oh, God.” Right? So my point is that not only may Peter be right—it may be a random walk—but even when somebody says something like, “We’re trying to train our AI to be moral,” and blah, blah, blah, when you hear somebody say that and you look at them—and this guy was in his early 30s—that was my reaction. I was like, “Dude, you want me to trust you?”

This is like Mark Zuckerberg telling me social media is good for me, or Marlboro telling me cigarettes are good for me. It sort of makes me think that way. So I don’t know if I have anything cheerful here, because not only do I think it’s a random walk, but I think when people try to steer, we’re suspicious of their ability.

I’m suspicious of their ability to steer. Right? That’s the story I just told you: this guy is brilliant, probably way freaking smarter than me, and he’s trying to steer, and I’m suspicious. I think it’s on both sides of this coin. I don’t know if I have any good news here.

Peter Diamandis

Let me frame it in the following way. I think we are holding 2 different futures in superposition, to go back to quantum physics—if you will, Schrödinger’s cat. In 1 future, we’re going to collapse the wave function to a brilliant, vibrant future for humanity; in the other future, we have dystopian outcomes.

The question becomes: How do we guide humanity toward this positive vision of the future? What do we do today? How do we help people? Is it—Steven and I have been talking about this—is it mindset? Are we going to help people create the mindset and the frames that allow them to survive and thrive? Or is there something else that needs to be done?

Mo Gawdat

Yeah. I’ll actually, first and foremost, second what Steven said. One of the top irritating comments I heard from Eric Schmidt—I worked for Eric for a while, so I respect him tremendously—but he said, “We will need every gigawatt of power, renewable or nonrenewable, if we are to win this race.” I think that’s the kind of blindness that you get when you’re running too fast, right? When you’re so afraid that the other guy will win, it’s those times when you start to make decisions that are not really responsible because you are blinded by something that you position as more important.

The way I look at it, Peter, is—I know it sounds really not positive, but there is positivity in it—I call it a late-stage diagnosis. What humanity is struggling with today is that we’ve been building a system, systemically prioritizing greed, prioritizing gains, prioritizing power, and so on, for so long. Those objectives have systemically built the world that we are in today.

The world we are in today is not healthy. It wasn’t healthy even before AI. In my book, which is in 3 parts—past, present, and future—in the past part of the book, more than half of what I write is not about AI. It’s about capitalism, the propaganda machine, and all of those things that will be magnified by AI.

What’s the point? If this planet is sick, if you want, and it’s a late-stage diagnosis, a physician will sit you down, look you in the eye, and say, “By the way, this does not look good.” But that statement, believe it or not, is not a statement of hate. It’s a statement of ultimate care.

Why? Because a late-stage diagnosis is not a death sentence. Many patients who have been diagnosed with a late-stage disease have not only survived, but thrived. They thrived because they changed their lifestyle. They changed something. This is what Steven teaches all of us: You can live differently, and when you live differently, you’ll achieve peak performance. You’ll achieve maximum health. You’ll achieve, you’ll achieve, you’ll achieve.

I think that’s what we as humanity need to start realizing: The systems that have gotten us here, from a process point of view, have nothing wrong with them, but from an objective and morality point of view, have everything wrong with them. What good is it to be a zillionaire in a world where there is nothing you can do with your money? What good is it to be the first inventor of an AI that basically renders you irrelevant?

I think we need to basically pause and say, “Do we want this anymore?” Sadly, it requires cooperation across human brains, which Steven rightly said at the beginning is not something we do very well.

The other thing is, I would put forward the notion that there is no on-off switch and there’s no velocity. We’re running open-loop with “yes” and “more” and “more” and “more” as the objective function. There’s no consideration for whether GPT-5 or GPT-6, or Grok-4 or Grok-5, or whatever your favorite models are, in the final result are going to enable something that is massively dangerous for humanity.

If that’s the case, I still go back to what safety valves do we have? I don’t see any action being taken by the leaders of the free world. Let me ask you both a question: If you could move to a planet that didn’t have AI, or where AI was developing at 10% the speed it’s developing at, would you leave?

Steven Kotler

I’d be gone. I’d go. I’d be gone to 2016 today.

Peter Diamandis

I don’t know anything. I’m sorry—your answer to that is what?

Steven Kotler

I would reset back to 2016 today.

Mo Gawdat

2016. I think AI today has all the upside and very little downside. I think it’s AI in the next 2 to 5 years that I’m so concerned about. AI today is incredible. And I didn’t say we were going to move to a planet where there’s no AI. I just said move to a planet where it’s going much slower, so maybe we can start to think about it.

Peter Diamandis

But that’s a fantasy. Well, I mean, Bigelow Space Hotel is coming to a universe near you.

Steven Kotler

So, Peter, I actually think that you’re accurate in your description of where AI is today. But it’s that 5%—5 degrees—deviation back in 2016 that led us to where we are today, right? You remember things at the time where we geeks agreed that we weren’t going to put it on the open internet.

Peter Diamandis

Yeah. It was Google.

Steven Kotler

Google developed this first and decided not to put it out there. Then OpenAI said, “Here it is. Here it is,” and no one had any hesitation. Put it on the open internet, teach it to co-create and write more code, and start the party of swarms of agents talking to agents, talking to AIs, right now.

I would definitely reset that. I would, however, say there are things we can do right now if we want to prepare. I’ll start with government. I don’t think we’re asking government for too much when we tell them to try and regulate AI. It’s almost like going to government and saying, “Regulate the making of hammers so that they can drive nails, but nobody can use them to hit someone on the head.” It’s a very complex thing to ask because they don’t understand hammers, and, believe it or not, even the guy who’s making the hammer cannot do that.

My ask of governments is: Regulate the use of AI. If someone uses a video that is a deepfake video and does not declare that it’s a deepfake video developed by AI, criminalize that. Make it legally liable to use AI to manipulate information, manipulate populations, and so on and so forth. This is the role of the government immediately: Regulate the use of this massively new technology.

For the rest of us—investors, businesspeople, and so on—I ask a very simple question: If you do not want your daughter or son at the receiving end of a specific AI, don’t invest in it. Don’t promote it. Don’t use it. It’s as simple as that. If you believe this can be harmful to someone that you love, do not give it the light of day.

Then, for us as individuals, I’ll go back to the late-stage diagnosis. Believe it or not, the way I live now—and you guys probably know this about me, not in front of cameras—is I hug my loved ones and enjoy every minute of every day, and I prepare. I learn the tool. I am one of the better users of AI in the world. I’m in line with the technology, but at the same time, I’m completely back to the purpose, realizing that I will do the absolute best that I can to spread the message.

I will do the absolute best that I can to say that ethics is the answer. If we show AI ethical behavior, it may learn it from us, just like it learned all of the other stuff from us. But at the end of the day, if it messes up, you’re going to hit that dystopia.

Peter Diamandis

Not forever. There is a point in time where AI takes over and says, “Okay, kids, enough stupidity. I’m in charge now. Nobody kill nobody.”

How far out is that, Mo?

Mo Gawdat

12 years.

Peter Diamandis

12 years.

Mo Gawdat

Okay, so 12 to 15, just so that people don’t come back and hit me after 12 if I’m still around.

Peter Diamandis

I’m going to wrap this episode on this subject line. It’s where we’ve come to before, which is that, in the near term, it’s the use of AI by malevolent individuals that’s our greatest fear. It’s not China versus the US. It’s the US and China against those malevolent players out there who wish to use this for greed and vengeance, whatever it might be.

I think this is an unstoppable progression. I don’t think there’s any on-off switch here. We’re seeing $1 billion a day being invested into AI, which is extraordinary, and I think that’s going to continue to increase. We’re seeing data centers being popped up every way, every place possible.

I’m the world’s biggest optimist, and I am optimistic about the impact of AI on human longevity, on new understanding of the physics of the universe, on new mathematics, on new material sciences, and on things that will create incredible abundance that Steven and I have written about and are writing about in our next book.

I’m looking forward to this benevolent superintelligence stabilizing the world. That’s what I’m hoping for.

Mo Gawdat

I agree. Steven, where do you come out on this?

Steven Kotler

I think that you guys wanting to invent an AI god to save you from yourselves is maybe the craziest thing I've heard since the guy from the AI company—I won't mention it—told me to trust him. But I love you both. That's actually usually the answer that you get: the only way to save us from AI is to use an AI.

Yeah. You know what? The beautiful thing is, we're going to find out. I also want to leave everybody with one thing, going back to what we were saying about cooperation, the upleveling of human intelligence, human consciousness, and things like that.

The human brain is widely considered the most advanced machine in the history of the universe, and we're just now, with the help of AI, figuring out how to uplevel it and link it with other brains. The level of cooperative possibility is enormous.

Let me get back to it for 1 second. Enlightenment, which is a definable biological state that produces universal compassion and oneness with everything—we're engineering it. It's a state that's starting to become available almost on demand.

So when I say there's a new level of cooperation coming that's emergent at the same time as the AI stuff, we can't see it. We have no idea. It's emergent, just like other things. So rather than the benevolent AI god, I think we're going to surprise ourselves.

I'm not the optimist in the room, by the way. Peter's the optimist when we're in the room. I'm not the optimist in the room, but I think I'm more optimistic than Peter on this one.

I'd love for that thought to be actually implemented. I think that's something that we really need to think about deeply. In the short term, I don't know who we could talk to. There you go, Peter. It's back to you.

Peter Diamandis

Thank you. I appreciate that. You were saying, Mo—please close this out.

Mo Gawdat

I was basically saying I think this definitely, definitely is the answer, if you ask me. If we just shift our mindset into cooperation, we head directly into a world of total abundance.

I was in a conversation with Eric Schmidt, whom he mentioned earlier, and his point of view was that until there is some type of disaster—until there is something perhaps like Chernobyl or Three Mile Island that isn't a 10 out of 10, but a 2 or 3 out of 10, and it scares the daylights out of us—we don't realign as humans. We don't realign, and we blindly go forward as we have been.

I believe that it's human nature that plagues us from being able to save ourselves many times, until that child in us burns our fingers on the stove, even after your parent has told you over and over again, “You're going to burn your fingers on the stove. Stop playing with fire.”

Peter Diamandis

Agreed. 100%.

Mo Gawdat

But let's be hopeful. Let's assign that task to Steven to design an XPRIZE for human cooperation. Let's assign another task to Peter to make it happen, and, yeah, let's assign a task for me to hug you both when you do.

Steven Kotler

I love you, Mo. Love you guys very much.

Peter Diamandis

Mo, how come you get to do all the hugging?

Mo Gawdat

Hugging you is hard work, Steven. You understand that? You move too much.

Peter Diamandis

All right, guys. Lovely. Thank you, bro, for lending me your brains this morning. It was fun thinking with you—a fun conversation.

I'm curious, as people listen to this podcast, where do you come out on this? How do you feel about it? Do you have a solution that we should all be thinking about and promoting? I'll ask my AI as well. It's not necessarily going to give me the best answer, but maybe our group mind, our meta-intelligence here might bring us that.

Have a beautiful day, gentlemen. Go hug somebody. Talk soon.

Steven Kotler

Thanks very much. Bye, guys.

Mo Gawdat

Thank you.

AI Experts Debate the Future of AI (Opposite Opinions) Mo Gawdat & Steven Kotler | EP #177 | BidClub