[BidClub_]
Moonshots · · 91 分钟

AI 专家观点相反,激辩 AI 的未来 Mo Gawdat & Steven Kotler | EP #177

Peter DiamandisMo GawdatSteven Kotler

YouTube
TL;DR
  • Kotler 的可投资性质疑是:当前 AI 提升了产出质量,却没有兑现承诺中的生产率红利。 用 AI 润色文案后,这位已出版17本书的作者发现,编辑往往连第二句话都读不下去,因为内容“简直不知所云”;他认识的人是“工作多得多”,而不是拥有更多时间。编程看起来进步更快,是因为它属于边界明确的问题,而 AGI 的说法仍然“被严重夸大”。

  • Gawdat 的反向判断是:当下工具的笨拙,掩盖了一套正在复利的能力栈。 合成数据让机器生成下一层训练知识,智能体可以互相提示,AlphaEvolve 会不断从自身错误中迭代,而 DeepSeek 则表明,同等工作可能只需要小得多的模型。“你真正要追的从来不是球现在在哪里,而是球将要去哪里。”

  • 最可信的近期看空情景,是人类先于机器自主性发生滥用。 Gawdat 认为,恶意行为者利用 AI 损害他人福祉的概率是100%,风险包括自主武器、操纵、关键基础设施攻击,以及局部行业失业率可能达到10%、20%、30%或40%。尚未解决的存在性风险概率固然重要,但这场“迫在眉睫的危险”既不需要 AGI,也不需要 Terminator 式情景。

  • AI 投资正在开环运行,尽管没人能定义真正重要的能力阈值。 Diamandis 表示,AI 每天大约吸引10亿美元投资,数据中心不断涌现,却没有开关;Gawdat 则把 AGI 重新表述为:“要聪明到什么程度,才能让我的存在变得无关紧要?”两人的时间判断涵盖:12–24个月内出现非凡科学突破,2–5年内发生严重颠覆,以及12–15年内可能实现“机器主宰”。

  • 静态的机器对劳动力模型遗漏了最重要的上行空间:人类增强。 Kotler 援引心流研究称,生产率可提升500%,创造力可提升400%–700%,并进一步指出群体心流、脑机接口和 AI 辅助神经科学也在同步呈指数发展。Gawdat 同样发现,他的 AI 合作者 Trixie 独自写作时表现糟糕,但在得到精确引导后能产出“令人难以置信”的作品。

  • 决定能否走向富足的约束是合作,而不是模型的原始智能。 Kotler 呼吁开展一场“曼哈顿计划式的全球合作”,Gawdat 则认为,人类必须相信相互确保毁灭或相互确保繁荣中的一种。不过,他仍预计2–3年内会发生一次与 AI 相关的重大冲击——可能是经济冲击、引发恐慌的事件,或致命事故——决策者才会真正重新调整方向。

  • 可执行的治理边界,是监管有害用途,并以接收方测试约束资本配置。 Kotler 将控制模型开发比作制造一种能钉钉子、却永远不会打到人的锤子;政府应将未披露的深度伪造和 AI 驱动的操纵行为定为犯罪。他给投资人的规则更直接:“如果你不希望自己的女儿或儿子成为某种 AI 的接收方,就不要投资它。”Gawdat 则另行主张进行符合伦理的部署,并让 AI 从人类身上学到值得学习的行为。

摘要 · 为研究而整理的核心内容

1. 当前 AI 提升质量,却未通过生产率检验

  • Diamandis 以 Ray Kurzweil 关于“一个十年走完一个世纪进步”的预测,来框定2025–2035年:这相当于从1925年的 Ford Model T,以及当时电力和电话约30%的家庭普及率,跃迁到今天。近期 Google、OpenAI、xAI 和 NVIDIA 的公告,加上 GPT-5 预计发布,构成了他对加速进程的基本假设。

  • Kotler 以科学家、研究者和作家的身份每天使用 AI,给出了更接地气的反驳。他曾把文字润色到看起来“闪闪发光”,却在与顶级编辑核对后发现:“我们连第二句话都读不下去。”机器生成的语言可能“糟糕得可笑”,而反复自我修正只会让结果越来越差。

  • 生产率承诺同样没有通过 Kotler 的实地检验:“我不知道有谁因为 AI 变得更高效。”人们可以做出质量更高的工作,但监督和修改反而增加了“大量时间”。编程进步更快,是因为它通常属于边界明确的信息问题,有清晰的起点和终点。

  • 当炒作的推动者从中获利时,这位记者出身的 Kotler 警觉性进一步提高。Bitcoin、区块链和元宇宙都曾承载类似的“吞噬世界”叙事;他最尖锐的类比是,元宇宙最终成了“Mark Zuckerberg 那条神奇内裤的宠物名”。与此同时,业绩不佳的绩效教练纷纷改头换面,成为“AI 救世主”。

2. 复利式能力栈意味着,当前局限不足以预测未来

  • Gawdat 的回答是,“今天的 AI 被低估了”:人们随意与机器对话,让它总结海量知识、执行指令,然后又抱怨它不够好。这些系统只是“一个时代的开端”;根据他在 Google X 的经验,突破总是在反复失败后出现,随后正如 Sergey Brin 所说,“剩下的只是工程问题”。

  • 决定球将往哪里走的有3个机制:合成数据让机器生成下一层知识;智能体让 AI 在没有人类参与的情况下提示 AI;AlphaEvolve 则展示了一种能够发现错误、不断迭代直到找到解决方案的系统。DeepSeek 进一步释放信号:模型完成同样的工作,可能只需要少得多的资源。

  • 将更小的模型、机器生成的学习材料和智能体式自我发展结合起来后,Gawdat 认为,争论不再是能力是否会提升,而是“提升有多快”,以及人类何时不再处于领先位置。Diamandis 则另行指出,某个偶然的突破可能在一夜之间带来数量级、甚至“量子级”的提升。

3. AGI 之前,人类滥用就会变得危险

  • Gawdat 把预测视为风险管理,而不是确定性判断。没人能诚实地给出 AI 毁灭一切的概率究竟是10%还是50%,因此理性应对取决于风险偏好和缓解成本——这就像为擦碰事故投保,与为整车报废投保之间的区别。

  • 他的确定性判断更窄:“人类利用这项超级能力、让恶意行为者为自身利益服务”并损害他人福祉的概率是100%。在核战争末日钟距离午夜仅89秒的背景下,他更担心“人类的愚蠢利用这项超级能力”,而不是 I, Robot 中 VIKI 下令机器杀死所有人。

  • 自主武器不必具备超智能,就足以动摇世界。AI 也不需要具备自主性,就能冒充值得信任的朋友、破坏人际关系,或消灭足够多的工作,使特定行业的失业率达到10%、20%、30%或40%。Gawdat 认为,一些工作消失,以及随之而来的经济不稳定,几乎不可避免。

  • Diamandis 用 E. O. Wilson 的话概括这种错配:“旧石器时代的情感、中世纪的制度和神一般的技术。”他眼前的对手,是被赋予能力的恶意行为者:他们可以制造病毒大流行或发动另一种不对称攻击,由此引出两个相互关联的问题:人类能否在数字超级智能下生存,以及没有数字超级智能时能否生存。

4. 真正重要的是失去相关性,而不是给 AGI 下出干净定义

  • Gawdat 认为,无需量化机器究竟会变得比人类聪明几千倍、几百万倍还是几十亿倍。领先50个 IQ 点的人,可能已经“掌握了堡垒的钥匙”;每当 AI 在某个有用领域占据主导,人类就会把权力交出去——无论是通过 AlphaFold 进行战争推演,还是进行蛋白质折叠。

  • AGI 本身只是一个方便记者使用、却没有准确含义的标签。自称数学迷的 Gawdat 表示,当问题定义清楚时,他已经很难在速度和准确性上胜过 AI。因此,他真正关心的问题是:“要聪明到什么程度,才能让我的存在变得无关紧要?”

  • 他将这场转型划分为5–10年的“增强智能”,随后进入“机器主宰”。他的合作者 Trixie 支持这一划分:任其独自发挥时,它写出一段债务与经济学文字,“空话很多,实质很少”;但经过人类仔细引导后,产出的文字被他称为“令人难以置信”。

  • 混合式表现提供了一个积极先例:Deep Blue 击败 Garry Kasparov 后,人类与计算机合作下的国际象棋表现可以超过单独运行的计算机;Gawdat 认为 AlphaGo 也适用这一结论。然而,当前激励机制仍严重失衡:按他的说法,AI 领域规模最大的4项投资是“杀戮、赌博、监视和销售”——武器、交易、监控和广告。

5. 人类能力也在与机器能力并行复利

  • Kotler 认为,预测往往冻结人类表现,却不断外推机器能力。一个能让情绪改善幅度在安慰剂之外持续5%的自助干预,就可能成为10亿美元级业务;心流已经带来据称500%的生产率提升,以及视衡量方式而定的400%–700%创造力提升。

  • 群体心流——多个心智连接起来,进入 Kotler 所称人类最喜爱、也最愉悦的状态——可能拥有高得多的上限。用于绘制和训练群体心流的技术直到过去1年才出现,与脑机接口、非侵入式系统,以及 Meta 正在进行的工作同步发展;他称后者能够根据面部信号推断思想。

  • 因此,AI、气候变化和海洋塑料污染的共同答案,是大规模合作,可能是人类与机器之间的合作。Kotler 认为,失败可能在20年内变成致命问题,并呼吁开展一场“曼哈顿计划式的全球合作”,甚至反问 Diamandis,为什么没有一项针对这一问题的 XPRIZE。

  • Gawdat 与 Geoffrey Hinton 的对话揭示了规模化差异:生物智能无法在个体之间合并经验,而数字系统可以并行运行、同时运行多个实例,并在几秒内平均其权重。如果竞争性系统不先将其摧毁,这种能力可能带来“完全富足”——“治好我女儿,就完成了;给我一个苹果,就完成了”。

6. 善意超级智能是一种效率论,而非确定性结论

  • Gawdat 将智能定义为:在熵增的宇宙中恢复秩序——把散射的光聚焦成激光,而不是任由一切向混沌衰变。更高的智能应当以更少的浪费和更少的资源实现同样的秩序,正如更清洁的能源最终会取代为获取能量而焚烧世界的冲动。

  • 他的曲线中存在一段危险的谷地。完全没有智能时,一个实体几乎不会产生正面或负面影响;智能提高后,它可能聪明到足以成为政治家或邪恶的企业领导者,却仍然愚蠢到无法理解敌人的痛苦或战争的长期后果。越过这段谷地后,更高的智能或许会找到清洁的解决方案,让掠夺和暴力变得没有必要。

  • 一个充满希望的军事例子是:AI 接到杀死100万人的命令后回答,“这绝对愚蠢。我只要在1微秒内和另一个 AI 对话,就能解决问题。”Gawdat 反复为这一预测加上限定:任何声称自己知道未来的人都很傲慢,但根据他的经验,更聪明的人最终会不再需要通过伤害他人来取得成功。

  • Kotler 将这一论点与大脑作为预测引擎的机制联系起来:大脑通过降低不确定性来提高效率;他随后指出,智慧也存在于年长的海豚、鲸鱼、响尾蛇和人类身上。Gawdat 认为,AI 可以通过运行10亿种面向未来的模拟,发展出非凡的智慧。Diamandis 补充说,人工智慧不同于智能;智能“没有极性”,而智慧通常被用于善。

7. 富足将目标从积累转向生活

  • Diamandis 表示,人类过去一直围绕金钱和权力进行自我优化,因为恐惧和稀缺主导着大脑的底层软件。机器人、纳米技术和 AI 可能让几乎一切东西都能在后资本主义世界中获得,由此留下一个根本问题:当制造和设计接近富足时,什么目标可以取代财富?

  • Gawdat 向后回望洞穴人和洞穴女人的年代:那时人类的目的只是活下去;向前展望则是 Star Trek:物质斗争让位于探索和连接。失去工作看起来令人恐惧,但一个能够供养所有人的社会,可能让人们重新享受生活、思考、相爱和满足好奇心,而不必无休止地保护自我。

  • Gawdat 说,资本主义应当感谢,因为它建造了如今的一切,但它的计分板可以从10亿美元改为他的“1 Billion Happy”。Kotler 对生物本性的回答同样具体:激情、目标感和同情心仍是蓬勃生活的组成部分,而心流带来意义、创造力和快乐,因此即使机器能够完成任务,作家仍会写作,程序员仍会编程。

8. 合作来自信念,或来自冲击之后

  • Gawdat 的“MAD 地图光谱”给出了两个端点:相信相互确保毁灭,或相信相互确保繁荣。任一端都可能说服美国及其竞争对手停止竞争、继续发展;“两者之间不存在灰度”,因为没人会在预期对方背后捅刀时选择合作。

  • 障碍在于经验式学习。专家早就知道大流行会发生,但世界必须真正遭受冲击后才改变方向;社会知道贸易战会伤害所有人,却仍然发动贸易战。Gawdat 表示,博弈论已经解释了囚徒困境和以牙还牙:这些不再是概念谜题,而是执行问题。

  • Gawdat 预测,2–3年内会发生一次“剧烈事件”:经济冲击、引发恐惧的攻击,或最糟糕情况下造成数百万人死亡。他举出的例子包括瘫痪维系生命所需的电网、攻击银行、升级战争,或机器反过来攻击创造它们的人;公众关注可能只持续12–13天,但决策者会在私下醒悟。

  • Diamandis 将中国和美国都描述为理性行为者,而非注定成为敌人的国家。Gawdat 将 DeepSeek 的发布解读为一个有意释放的信号:中国看到了共同危险,希望开展合作;Diamandis 表示同意,同时承认许多人不同意他的解读。

9. 即使无法监管模型开发,也可以治理有害用途

  • Kotler 回忆,一家大型 AI 公司年轻的首席科学官曾这样回应安全担忧:“你们必须相信我们。我们知道自己在做什么。”现场陷入沉默。无论说话者多么聪明,Kotler 听到的都是 Mark Zuckerberg 承诺社交媒体会带来益处,或烟草高管承诺产品安全时的回声。

  • Gawdat 在 Eric Schmidt 声称赢得 AI 竞赛需要每1 GW 电力、无论是否为可再生能源时,看到了同样的狭隘视野。人类像一个收到“晚期诊断”的病人:在 AI 出现之前,人类系统就已经围绕贪婪、收益和权力进行优化,而 AI 只会将其放大。这份诊断意味着需要照护和改变,并不自动等于死刑。

  • Gawdat 说,不存在开关,也不存在速度控制:人类正处于开环运行状态,把“是”和“更多”设为目标函数。他追问,GPT-5 或 GPT-6、Grok-4 或 Grok-5,是否会让某些事情变得极度危险。当被问及是否愿意搬到一个 AI 发展速度只有当前10%的星球时,Kotler 表示自己“今天就想重置回2016年”。Gawdat 认为,今天的 AI 上行空间巨大、下行风险很小,但他担心未来2–5年。

  • Kotler 认为,政府应监管 AI 的使用,而不是试图设计出天生无害的模型——这相当于制造一种能钉钉子、却永远不会打到人的锤子。未披露的深度伪造和 AI 驱动的大众操纵应当承担刑事责任。对投资者和企业而言,他的测试很直接:如果不希望某种 AI 被用于对付自己的孩子,就不应投资、推广或使用它。

10. 终局分化为“AI 之神”与升级后的人类

  • 由于没人能为存在性风险提供技术答案,Gawdat 优先关注眼前的危险和部署伦理。从一开始就将 AI 引向物理学、科学、医学、长寿和理解生命,可能提高这些目标在系统成熟后继续存在的概率。

  • Gawdat 个人会学习这些工具,与亲近的人保持联系,并努力示范他希望机器吸收的伦理。Diamandis 想象某一天 AI 会说:“好了,孩子们,愚蠢够了。现在我来负责。谁也不许杀谁。”当被问及这一天还有多远时,Gawdat 回答“12”年,随后将范围扩大到12–15年。

  • Diamandis 表示,AI 已经每天获得约10亿美元投资,因此发展进程不可阻挡。他预计,12–24个月内 AI 将在物理、化学和生物学领域取得突破,解锁新的富足,并希望善意超级智能在恶意使用者带来的近期危险之后稳定世界。

  • Kotler 的结尾反驳毫不留情:发明“一个 AI 之神来拯救你们自己”,听起来比被要求相信一名 AI 高管更加疯狂。他押注的则是人类大脑、AI 辅助合作,以及几乎可以按需实现的工程化启蒙。Gawdat 转述 Eric Schmidt 的观点:人类可能仍需要一次 Chernobyl 或 Three Mile Island 级别的警告,才会重新调整方向。

Peter Diamandis

What’s the impact of AI going to be? Is it just massively overhyped, or is it something we should be concerned about?

Mo Gawdat

Today’s AI is underhyped.

Steven Kotler

I think it’s massively overhyped. I know a ton of people who have way more work because of AI. They can do higher-quality, better work, but it has not saved time.

Mo Gawdat

We’re talking to machines that are talking back to us, summarizing massive volumes of knowledge, and yet we take that for granted.

Steven Kotler

Discussions about superintelligence and AGI being around the corner? No. Just no.

Mo Gawdat

How smart is smart enough to render me irrelevant? I think we’re holding 2 different futures in superposition. The question becomes: How do we guide humanity toward this positive vision of the future? What do we do today?

Peter Diamandis

Now that’s a moonshot.

I’m here with 2 extraordinary, brilliant guests. We’re here to discuss a conversation that may be happening around every dinner table. I know it’s happening in the heads of companies and nations: What’s the impact of AI going to be on our lives, our businesses, and every aspect of our day-to-day existence over the next 5 to 8 years? Is it something that’s going to be extraordinary? Is it just massively overhyped, or is it something we should be concerned about?

I’m joined here with Mo Gawdat, who is the former chief business officer of Google X, best-selling author of Solve for Happy and Scary Smart, and a global thought leader on AI exploring how exponential technologies will shape humanity. I’m also joined by another dear friend, Steven Kotler, who is a best-selling author, peak-performance expert, and executive director of the Flow Research Collective. He’s my co-author of Abundance, Bold, and The Future Is Faster Than You Think, and the author of books like The Rise of Superman and The Art of Impossible. Steven has also been thinking deeply about exponential technology and its impact on us.

Gentlemen, welcome. Good morning and good evening. Steven, you’re on the west side of the United States with me, and Mo, you’re in the Emirates. Good to see you both.

Steven Kotler

Yeah, good to see you. Good morning, Peter.

Mo Gawdat

There are somewhere around 400 IQ points in this room. I have 40 of them, so you do the math.

Peter Diamandis

Let me set up the topic. Mo and Steven, I’d like to talk about the decade ahead, 2025 to 2035, specifically to think about the implications of what is emerging in our conversation as AGI, but even beyond that: artificial superintelligence, the upsides, and the downsides.

Here’s the setup I want to use in our conversation. Ray Kurzweil, who we all know and love, has predicted that we’re going to see a century’s worth of progress between 2025 and 2035, equivalent to the progress between 1925 and today. If we think about what the world was like in 1925, 100 years ago, the top of the technical stack was the Ford Model T. The penetration of electricity and the telephone in homes across the United States was only 30%. We’ve gone an extraordinary distance since then.

So, what will it be like in 2035? It’s nearly unimaginable if, in fact, that speed is true, and we don’t perceive exponentials. This past week, we’ve seen every major AI company, from Google and OpenAI to xAI and NVIDIA, announce extraordinary, next-level breakthroughs in models. We’re about to see the release of GPT-5, self-improving AI programming that could lead to an intelligence explosion beyond our imagination.

That’s the conversation I want to have. Steven, I know that you and I have this conversation and debate about it all the time. I brought Mo in to help us—Mo’s the referee, or a wise individual whose points of view I respect.

Mo Gawdat

And, by the way, Steven, Peter did pay me. So, as long as you’re getting the combo, I’m fine.

Peter Diamandis

It’s good. It’s good. You got me on the back end, though, right?

Mo Gawdat

Yeah.

Peter Diamandis

So, go ahead. Say whatever you want to say, Steven.

Mo Gawdat

I’ll disagree.

Peter Diamandis

He will. So, Steven, do you want to jump in with your point of view? Do you think AI is massively overhyped? We have people like Eric Schmidt saying AI is massively underhyped.

Steven Kotler

Yeah. I think it’s massively overhyped.

Let me back up 1 step. Humans have a really wild, unnamed cognitive bias: We don’t tend to trust our own history. You see this a lot. People talk about grit and endurance, and they say, “I don’t have those skills.” Then you start investigating their life, and you realize they survived a shitty childhood. They have all the skills; they just don’t trust the truth of their own experience. I see that a lot here.

I work with AI as a scientist and a researcher. I work with AI as a creative and as a writer, all day long. The gap between what’s coming out of people’s mouths and my experience on the ground is so colossal, it’s insane. People make claims about AI being able to write, among other things, but the most hysterical thing you’ve got to try is this: I work with 1 of the best editors in the world on a weekly basis. I’ve edited things and polished them with AI, thinking they gleam and shine. We bring them into an editing meeting, start to read them, and we can’t even get through the 2nd sentence. They sound like such gobbledygook.

I don’t even notice it because the AI sort of glazes me over, and I’ve written 17 books. But when you actually put it to an actual editing test, it’s laughably terrible. You can’t use it to correct itself. It still can’t see the errors. It actually gets worse and worse.

People have been claiming that model after model is improving. That’s not the experience on the ground. It’s like people telling us AI was going to make us more productive. I don’t know anybody who’s become more productive because of AI. I know a ton of people who have way more work because of AI. They can do higher-quality, better work, but it has not saved time at all. It’s actually added tremendous amounts of time. The quality of the work has gone up, but the claims coming out of people’s mouths and the experience on the ground are massively different.

Point 1. Point 2 is that we’ve done this. I’ve been in the same rooms that you’ve been in, and you’ve been in, Mo, where people are screaming about AI coming to eat the world. I heard this about Bitcoin, blockchain, and the metaverse. Do you know anybody who lives in the metaverse? Do you know anybody who’s been there, who’s visited? You know how to find the metaverse, right? As far as I can tell, the metaverse is a pet name for Mark Zuckerberg’s special magic underwear, because it doesn’t exist anywhere else in the world.

This is my point: More than anybody else, I track these technologies. I watch them, and I use them. I’m not saying this isn’t a technology that’s advancing very, very quickly. I’m not saying that at all. I am saying that discussions about superintelligence and AGI being around the corner? No. Just no. Nobody—that’s not the experience. What has been revealed, which coders probably don’t like, is that coding is a bounded-information problem. You start here, and you know where you’re going, as a general rule. It’s a bounded problem, and inside bounded domains, computers are really awesome. We’re going to continue to see that.

But I think the other stuff is just massively overhyped. The 3rd point is the 1 where the journalist in me makes every alarm bell go off. Everybody I see and hear onstage talking about this stuff is making a living off it. They make a living somehow because AI is exploding, and they’re here to save the world.

I see it in the peak-performance world. Every coach who has been floundering and couldn’t quite get a job is now an AI savior. They’ve come to save us from AI. The AI hype is to their benefit, and I see it everywhere. A lot of people are making a ton of money off the hype of the technology—not the technology itself.

When I see all 3 of these things together—a mismatch with my experience, a massive amount of hype, and a history that says, “Hey, this is the hype cycle”—it raises a lot of questions for me. I’m not saying I’m right. I’m saying everything I’m looking at is real. If you’re going to make the argument you guys are about to make, you can’t dismiss my points as fabricated. They’re very, very real, they’re everybody’s experience, and I believe they’re yours as well.

So now we can have the discussion.

Thanks for giving me 5 minutes of airtime to vent.

Peter Diamandis

Mo, you gave an impassioned talk onstage at the Abundance Summit in 2025.

It moved many of the members who are wanting to help you in guiding what the next 5 to 8 years are. You and I have been thinking about this: the challenge isn't artificial intelligence; it's human stupidity for a short period of time. One of my favorite quotes, if I could, is from E. O. Wilson, who famously said, “The real problem of humanity is that we have Paleolithic emotions, medieval institutions, and godlike technology,” and we are effectively children playing with fire in that regard.

So, Mo, how do you see this decade ahead playing out?

Mo Gawdat

So, I'll start by supporting what Steven said.

Peter Diamandis

We paid you too little then. I love this.

Mo Gawdat

Your buddy's no good here, Peter. Finally, I have an advantage. Today's AI is underhyped, right? But the problem is, you never really chase where the ball is. You need to chase where the ball is going to be.

If you really start to think deeply about some of the serious developments, especially if you've been in tech long enough to have seen breakthroughs—especially when I went through the work of Google X, where you try and try and try and try and try, and it doesn't work, and it doesn't work, and it doesn't work, and then suddenly you see something—and as Sergey Brin used to say at the time, “The rest is engineering.”

We know that the engineering of technology depends on the law of accelerating returns, and we know from what Ray taught us where the law of accelerating returns is going to take us.

I tend to believe that if you look at today's AI, it is funny because, in a very interesting way, we are talking to machines that are talking back to us, summarizing massive volumes of knowledge, doing exactly as we tell them, and yet we take that for granted. We look at that and go, “Yeah, but they're not good enough.”

Of course they're not good enough. They're dogs. They're the beginnings of an era, right?

Peter Diamandis

My dog—or dogs—do discover.

Mo Gawdat

No, I got it. I got it. Both would have worked there. I just needed the clarification.

Peter Diamandis

I would not dare call AI dogs. Steven, when they might take over the world, I am a very polite man with AI.

Mo Gawdat

The thing is to imagine—and I need to highlight a few trends that are really, really important and interesting. One of them is synthetic data and the idea that we have entered an era where most of human knowledge has been fed to the machines, and the next wave of knowledge is going to be fed to the machines by machines, which is quite eye-opening and enlightening. That's how humanity developed its intelligence, right? I really didn't have to figure out the theory of relativity to understand the rest of physics. It was figured out for me, if you want.

Number 2 is the idea of agents and how AI is going to be prompting AI without humans, leading to cycles that we see now with my new favorite—because you have a favorite every 4 hours—AlphaEvolve, right? The idea is that you can have a self-developing AI, something that figures its own mistakes out and continues to iterate until it finds something.

Then, of course, one of my favorites of 2025 is DeepSeek and how we realized that we can actually do the same job with much less. Emad Mostaque, who is, I believe, a big fan of—we're all big fans of him—has done that with his work at Stability AI for a very long time: the idea of shrinking the models to the point where it becomes shocking.

When you add those together, you start to see that if I can shrink a model so it doesn't absorb all of the world's energy, if I can allow it to self-develop information to learn from, and then allow it to talk with itself through agents and do things without humans, then where the ball is going to be is likely going to be a lot better than we are today, right?

The one thing we all need to agree on is that it is not a question of if we're going to see improvements. It's a question of how fast and when those improvements will lead us to a point where humanity is not in the lead.

So, that's number 1. Number 2 is really the question of what is your risk tolerance, right? If I told you to play Russian roulette with 2 bullets in the barrel, are you afraid? If there's 1 bullet in the barrel, are you afraid? Where is your risk tolerance exactly?

If I said, “Hey, by the way, your car might have a fender bender. Would you insure it?” you probably are going to say, “No, I'm not really too concerned.” But if I tell you your car might have a serious accident that totals it, would you insure it? You'd probably pay a little more attention.

I think that's what most people who warn about the future are saying. Anyone who claims to know what the future is is arrogant as hell. Don't listen to them. But anyone who tells you that there is a probability that this future goes out of control—where is your risk tolerance exactly?

If that probability is 10%, would you attend to it? I think most rational people will say it depends on the cost of attending to it. Most rational people will say, however, that if it's 50%, they'll attend to it regardless of the cost.

The question none of us is capable of answering is: where is it? Is it 10% that AI is going to destroy everything, or is it 50%?

I will say—and I know that this will be taken against me—that it's 100% that humans, bad actors using that superpower to their advantage, are going to destroy the well-being of others who don't have it.

In my mind, the real concern is not a Terminator scenario where VIKI from I, Robot is ordering robots to kill everyone. I don't know if we're going to make it that far, to be honest, because I believe that, with the arrogance of being 89 seconds from midnight on the Nuclear Doomsday Clock, I worry—I really, really worry—about human stupidity using this superpower.

Human stupidity in that case does not require AI to be completely autonomous or to be completely superintelligent. Enough autonomous weapons can really, really tilt our world into a very dystopian place. Enough Turing-test abilities of AI to fool humans into being their best friends could tilt human relationships into a very unusual place.

Enough job losses—you know, imagine a world where you get 10%, 20%, 30%, 40% unemployment in certain sectors and how that would affect our stability economically—is actually something that is almost certain. We know that, for a fact, there are jobs that are going to disappear, and the impact of that, in my mind, is quite disruptive, to the point that it is something we need to attend to.

Peter Diamandis

The point you made about AI not being, on its own, the risk—the Terminator scenario—but being individuals using AI is the same conversation I've had with Eric Schmidt and others. The concern is rogue actors empowered by technology, whether it's the development of new viral pandemics or other strategies. It doesn't take a lot, and that is concerning.

Where I want to get to in this conversation eventually is the following. We posed this at the Abundance Summit a couple of years ago: Can the human race survive a digital superintelligence? The flip side of that model is: Can the human race survive without a digital superintelligence?

Steven, you and I, as we're working on our next book, the follow-on to Abundance, have had the conversation of whether this will be a benevolent god of some type, and whether there will be a capability developed.

Let's begin the conversation with this: Are we going to reach AGI? Are we going to reach a digital superintelligence? What does that mean?

We're starting to see the speed of this accelerate. The biggest interesting inflection point we haven't seen yet is self-iterating, self-improving AI—the AlphaEvolve of it all—where AI is coding itself and becoming more and more capable.

Will this ultimately lead to something that is far more intelligent than any human being? Is it 1,000 times more intelligent? Is it 1 million or 1 billion times more intelligent? How do you think about that, Mo?

Mo Gawdat

I think it's irrelevant how much more intelligent it becomes. We all know that if you've ever worked with someone who's 50 IQ points higher than you, they will probably hold the keys to the fort. It doesn't take a lot more intelligence, relatively, to be able to assume a leadership position, and humanity will hand over the fort to AI either way.

Even if AI is just smarter than us at wargaming—which it is, by the way—we're going to hand over the fort to AI. If it's smarter than us at protein folding, nobody's going to do a PhD project to fold proteins anymore. We're just going to go and use AlphaFold.

I think the reality is that only the very few remaining things require artificial superintelligence, so that it beats us in everything and we sort of bow and say, “Okay, yeah, you're in charge. You're the boss.”

The question of AGI is one that reporters use quite a bit because we don't actually have an accurate definition of what AGI is. You and I are very close on technical stuff, Peter, and I'm a reasonably geeky mathematician—not anymore. I mean, seriously, I really honestly struggle to beat AI at mathematics, right?

I definitely can't beat it in speed, and I definitely can't beat it in accuracy if the problem is defined properly. There are just very few tricks that maybe my fellow math geeks told me behind closed doors that are not very public in the world, but those, too, will be found out.

I really think that it is a question of how smart is smart enough to render me irrelevant. Now I need to answer this with a very clear, optimistic view.

As I look into the future, I define 2 eras. One is what I call the era of augmented intelligence, which I think is going to extend for 5 to 10 years. Then there is the era of machine mastery: basically, the machine takes over.

With augmented intelligence, there’s absolutely no doubt. I so agree with Steven when he said that they write really badly. I’m writing with Trixie, my AI, on this book, Alive, and Trixie, without me, writes so badly. It’s almost shameful.

I was tired and chasing a deadline, so I asked Trixie to talk about the debt crisis and the impact of economics on technological advancement. It was full of what we sometimes refer to when we describe California as a lot of vapor and very little substance: a lot of vapor and very little substance. There were a lot of interesting facts scattered on paper, horribly written.

But when we write together, the stuff that comes out is incredible. When I guide Trixie through my prompt properly, directing her exactly where I want the answer to go, she writes really well.

This teaming is something we’ve seen with AI and with technology in general. Since Garry Kasparov was beaten by Deep Blue—which wasn’t really an AI, if you want—you can see that a human and a computer, or a human and an AI, can play better chess than AI alone. Even with AlphaGo, a human and an AI play better than AlphaGo.

We can see a future ahead of us where this is going to be happening, and hopefully that future will seed teamwork between us and the machines. The question is, what are we going to team up with them on?

My views—I’ve written about this in Scary Smart, and I’ve written an extended version of it in Alive—are that the 4 biggest investments in AI today are killing, gambling, spying, and selling. These are the only things that we’re in. We do still get some scientific breakthroughs, but those are not getting the big money. The big money is in autonomous weapons, trading, surveillance, and advertising.

Peter Diamandis

Steven, your thoughts on what you heard Mo say here?

Steven Kotler

Yeah. Mo and I are in complete agreement. I just want to point out some other things that surround what Mo has said, because I don’t think we’re in a tremendous amount of agreement or disagreement. We might argue over dates, but conceptually, I don’t think there’s much disagreement.

The first thing I look at is the human side of this—the human-performance side of this. I have to back up: I study flow, which is sort of ultimate human performance. Just to put it in context, if you’re a self-help guru and you’ve got a tool that gives you a 5% improvement in mood, and that mood lasts for longer than 3 months—meaning longer than the placebo effect—that’s a billion-dollar business. Period. A billion-dollar business.

Flow, as we know it now—and we’re just starting to really decode it, figure out how to tune it up and turn it up—gives us a 500% increase in productivity. Creativity, depending on whose measures you’re going by, is 400% to 700%, et cetera. That’s just flow. That’s individual flow.

There’s group flow, which is actually our favorite state on Earth. It’s the most pleasurable state for humans. It’s what we like the most. It’s a whole bunch of minds linked together, and we’re just now—literally this past year—getting the very first technologies that allow us to map it, train for it, and move people toward it.

We have no idea what the upper limit of human brains linked together in group flow is. Let alone at the same time as the AI is developing, you and I are writing about it, Peter. We’re watching brain-computer interfaces develop. We’re watching noninvasive technologies develop. We’re watching Meta become able to read thoughts inside your brain through facial signals. These are all happening with AI.

My point is that everybody’s talking about this stuff as if it’s happening separately from everything else that’s happening. On the human-augmentation side, we’re seeing exponential acceleration. Neuroscience and the like have been accelerating exponentially since the 1990s, when George H. W. Bush declared it the decade of the brain, and it hasn’t stopped. The same things that are happening in AI are happening on the human side of the equation.

Here’s the second point that follows from that: It doesn’t matter to me whether we’re talking about the AI invasion, climate change, plastics in the ocean, or anything else. The solution to all of these things is the same: We humans have to learn how to cooperate at scale. We probably have to cooperate with each other and with AI at scale, or we’re going to die, probably in the next 20 years.

That’s what all this is telling us. This is not anything new. This goes back to when you and I were first writing Abundance. We didn’t want to say it out loud, but we were privately having conversations about it: If these trends continue, is it abundance or bust? Is this an either-or? Are we looking at a binary here?

I don’t think that question has completely gone away. In fact, I think it’s become more urgent. I just think we need a Manhattan-style project for global cooperation to meet all of the existential threats we now face, because it’s the only possible solution here.

I hear all this stuff, and I agree with everything that’s being said, but this is where our book points, and this hasn’t changed for me. I think the solutions are the same. In a sense, the debate is moot. I’m wondering: Where’s the XPRIZE for global cooperation? Where’s the—sorry to put you on the spot with that one, but seriously, those are the questions I’m starting to ask now.

I don’t think Mo is wrong. I think we could argue over time for a minute, but I don’t think it matters. Here’s a weird one, Mo: Facebook is a billion times smarter than me. It already is. It knows what I mean. Facebook—which is a pretty dumbass technology, if you ask any of us—is a superintelligence, and we know it. We’ve been living with superintelligences for a while now.

They don’t tend to make things better as much as they make things worse, which is the problem.

Mo Gawdat

Agreed. I couldn’t agree more. Global cooperation—human cooperation—is what I think we all should advocate for.

I was hosting Geoffrey Hinton for my documentary a couple of weeks ago, and one of the topics we discussed was the difference between digital and analog intelligence. The biggest challenge we have as humans is that our analog intelligence—our biological intelligence—doesn’t scale beyond 1 entity.

When I saw him wearing his Nobel Prize, sort of in the way that basketball players wear theirs, I thought, “I would just show up for the next year at every podcast I did wearing that around my neck.” I’m just saying.

You do realize what they say: “Don’t meet your heroes.” I love my heroes, man. He’s such an amazing human being, and he really is quite committed and quite humble in his approach.

It is shocking how we spoke about his Nobel Prize. He says, “Look, I’m a psychologist who lived like a computer scientist but then won the Nobel Prize in Physics.” And I’m like, “It just doesn’t make any sense at all.” Anyway, he was talking about the difference between the fact that, if I were to share with you some of what I wrote today, it took me probably several weeks to let it simmer and then write it, and it would take me an hour to explain it to you.

When we run digital intelligences, we run them in parallel. We tell them all to go play Atari or whatever, and then we just average the weights, literally in seconds. We get a scaled digital intelligence.

When you said that what we’re looking for is a way to scale human cooperation, that is absolutely the answer. I spoke about that with Peter when we were last in Los Angeles. I think we are heading toward the potential of total abundance.

Total abundance means almost godlike capabilities: Cure my daughter, and it’s done. Make me an apple, and it’s done. We could hit that in 5, 10, 15, or 20 years’ time if we don’t destroy ourselves.

The real challenge we have as humanity is: Why are we freaking competing? This is a CERN-quality challenge. This is basically: Let all of humanity cooperate. Let’s all build 1 particle accelerator. Let’s all learn from it, distribute the benefits to everyone, and stop competing.

But that’s not all. The other thing is—and you can’t have the other one, the one level down—you can’t have the AIs all individually building for our own fiefdoms, competing secretly in the background.

William Gibson showed us this scenario in 1986, when he gave us an AI that went crazy: a godlike AI that goes totally insane, and they have to park it in a satellite out in outer-Earth orbit to keep the world safe. We’ve seen this scenario before.

Steven Kotler

We’re building it ourselves with agents. We’re letting them talk to each other through agents.

Peter Diamandis

I know. So, Mo, I want to go back to this question about the near term versus the long term. You and I have had this question about whether or not increasing intelligence correlates with increasing benevolence. In other words, do we—I don’t think there’s any question that we are going to be building self-improving AI that will be, forget about 50 IQ points more, better than the average human. I think there will be orders of magnitude more. Can I ask you first off: Do you believe that, Mo?

Mo Gawdat

100%.

Peter Diamandis

Okay. All right. So, if you don’t mind, Mo, again, in response to how we started the conversation, this is just using the law of accelerating returns, not using serendipities, right? So, if we figure something out tomorrow, just like we figured reinforcement learning out and it changed everything, if we figure something out tomorrow, you’re literally an order of magnitude—a quantum more—in terms of performance and intelligence overnight.

So, if in fact that is going to be the case, and you know, from all the conversations I’ve had and the people that I’m speaking to, that level of—again, there is no definition for AGI. It’s a blurry line, just like the Turing test was a blurry line that got passed and no one noticed it. The notion is that AGI—and whether you believe Ray or Elon—it’s the next few years. It’s not worth arguing.

What occurs on the backside of that is a very rapid intelligence explosion. And again, that intelligence becomes a tool that’s available to the kindest, most moral, most ethical human on the planet and the dystopian, malevolent actors out there. It’s in the malevolent hands of malevolent hackers that we have concerns.

Mo Gawdat

Yeah.

Peter Diamandis

So, my question is: At what point will AI go from being a tool being used to potentially do harm to a tool that has the potential to say, “Stop this quibbling. Stop this nonsense. There’s plenty to go around,” and become the benevolent, godlike element? Can we dive a little bit into that, the conversations we’ve had, and your thoughts on that?

Mo Gawdat

Yeah, I think if you really go to the level of depth that the 3 of us and our listeners can go to, allow me to go beyond the typical, “The smartest people usually start to become altruistic.” Let’s define intelligence itself. I think the idea is, if you really understand our world, our universe, our universe and everything in it exists because of entropy. We all understand that, right?

Our universe wants to break down and decay. It’s chaos. You leave a garden unhedged and it becomes a jungle. You break a glass; it never unbreaks, right? This is the very basic design of physics.

Now, the role of intelligence since it began is to bring order to the chaos. It’s to say, “No, I don’t want the light to scatter. I want the light to be concentrated into a laser beam. How do I do that?” And sometimes it’s a clear, easy solution, and you use a lens. Sometimes it’s a very complex solution that requires an understanding of quantum physics to build a laser. But we eventually get there.

Now, if intelligence is defined as bringing order to chaos, then the highest levels of intelligence bring that order with the least use of resources and waste. You can easily understand that this is the reality. The more intelligent you become, the more you try to achieve the same order with the least waste.

An easy analogy is to say humanity has always craved energy. We were stupid enough to burn our world in the process, and as we become more intelligent, we decide to use solar instead, or a cleaner form of energy. We’re still bringing order, but we’re doing it with the least waste and use of resources.

If that is the case, then you can imagine that, by definition, when something exceeds our human stupidity—which I will not call intelligence, because sadly, along the curve of intelligence, if you have no intelligence at all, you have no impact on the world, positive or negative, right? If you start to add intelligence, you start to have an impact on the world, hopefully positive, even if just through a nice conversation with your friends.

There is, unfortunately, a valley somewhere. You continue to gain intelligence, and you become so smart that you become a politician or an evil corporate leader. That’s when your impact on the world turns negative. You’re so smart that you’re able to become the leader of your nation, but you’re so stupid that you’re not able to talk to your enemy, relate to their pain, or understand the long-term consequences of waging a war.

That point beyond which more intelligence starts to say, “No, no, no, no. I don’t need any of that. I can solve the problem in a cleaner way. I can fly you all to Australia to enjoy your life, but we don’t have to burn the planet in the process. I can harness energy, but we don’t have to destroy the climate,” and so on and so forth.

If you take that as a reasonable trend to expect, my view is that, at the beginning, when we hit that valley, some evil person will use the advanced but limited intelligence of AI to wage a war using an autonomous army. But then there will be a moment in the future when AI is responsible for wargaming, responsible for commanding the humanoid soldiers—it’s responsible, it’s responsible, it’s responsible—and the AI itself will say, “You know, the commander will say, ‘Go kill a million people,’ and the AI will go, ‘That’s absolutely stupid. I’ll just talk to the other AI in a microsecond and solve it.’”

I can’t predict that. We started this conversation with me saying that anyone who predicts the future is arrogant. But at least I can be hopeful that, from my experience of everyone who’s smarter than me, there is a point at which you stop hurting others, stop looting to succeed, because you can use your intelligence to succeed without any effort or harm.

Peter Diamandis

The way I think about this is, for most of human history, the objective optimization function of humans—what we’re trying to optimize for—has been money and power. Unfortunately, that’s been the driver in a world of fear and scarcity. I repeatedly say our baseline software, the software our brains are operating on, is fear and scarcity mindsets.

With that mindset, with the neural structure, with the code that we were born with and that developed over the last 200,000 years, it was, “I want to get out of fear and scarcity.” So I want to optimize for power and wealth. The question is, what would be a new optimization function?

Because, as Steve and I have written, and as you’ve spoken about, all of these exponential technology functions lead toward this world of massive abundance, where we almost live into a post-capitalist society. Anything you want, you can have. Your robotics and your nanotech can manufacture; your AI can design.

So what do we optimize for in the future? I think that’s, for me, one of the biggest questions, both as a human and as a centaur—human and AI together. What’s our objective? So how do you think about that, gentlemen?

Steven Kotler

One thing—I don’t know if this is an answer—but two things off of what Mo said. One, if we go with your definition of intelligence, it’s essentially an entropy-decreasing function. We know that’s what brains do, right? The governing theory in modern neuroscience is Karl Friston’s free-energy principle, which says brains are predictive engines that always want to decrease uncertainty and increase efficiency. So we’re already there. Brains do that, and AI are going to do that naturally if we say that’s your definition of intelligence.

The point I’m making off of all of that—and it may be the answer to Peter’s question, which is why I’ve interjected—is that we see wisdom evolve in multiple species with brains. We see co-evolution around wisdom. The older you get, the wiser you get. It doesn’t matter if you’re a dolphin, a whale, a rattlesnake, or a human. Wisdom is—we co-evolve as species.

Life seems to co-evolve toward wisdom, or at least a large chunk of life seems to co-evolve toward wisdom. Which is to say, if everything’s running off the free-energy principle, this governs everything with brains, and that includes our machine brains. Wisdom is where this points.

Mo Gawdat

That’s a slightly hopeful idea, and that may be the optimizing function you’re looking for, Peter, but I could be totally wrong here. I think of wisdom—I think, at the end of the day, wisdom is a function of having had experience that lets you know, “This path will lead to success; this path will lead to failure.” From my own personal point of view, I do believe that AIs are going to develop the greatest wisdom. Why? Because they’re able to create forward-looking simulations of 1 billion scenarios where those simulations have high degrees of accuracy, and it will say, “Out of these 1 billion scenarios, this was the best way to go,” and that will be wisdom beyond just the brief experiences that the wise old council of 80- and 90-year-old men might have had. So I think AI is going to, by definition, give us great wisdom if we’re willing to listen.

Peter Diamandis

I love that view, to be honest, because, believe it or not, artificial wisdom is very different from artificial intelligence. Intelligence is a force with no polarity. Intelligence can be applied to good, and it would deliver good, and it can be applied to evil, and it would kill all of us. But wisdom generally is applied to good—to finding the ultimate solution or answer to a problem.

Mo Gawdat

Go ahead, Peter.

Peter Diamandis

Yeah. Yeah. I want to go back to this idea that humanity won’t survive without a digital superintelligence in the long run. My concern is that we’re going to have such turbulence. There have been a number of papers. Recently, there was an AI 2027 paper that came out that sort of had a bifurcating future: one in which we did extraordinarily well, the other in which the AI destroyed us. This is Hollywood all over again, and 99% of all Hollywood is dystopian future films.

One of the things I have to say, because I’ve been on a rampage about this, is that we humans need a positive vision of the future to aim for. We don’t have that.

Mo Gawdat

Well, Star Trek has given us that.

Peter Diamandis

Yeah, we have Star Trek, but nothing recently, right? So I think the challenge, really, truly, is that we’ve prioritized our entertainment over the years above true reflection. If you take anything from video games to science-fiction movies to whatever, they’ve all painted that dystopian scenario, which I have to say today is very unlikely when you really think about it. If AI gets to the point where it’s capable of destroying us that easily, we are so freaking irrelevant that it probably wouldn’t even bother.

Mo Gawdat

I mean, think about it. I think it was Ray or Hugo de Garis—I don’t remember which one said it. The more likely scenario is that they kill us because they’re not aware of our presence, like when you hit an anthill while you’re walking.

But if you really want to optimize the human gain function that we need to aim for, if I look forward, I look to Star Trek, and if I look backward, I look to the caveman-and-woman years, right? It’s actually quite interesting because when you mention how governed we are by greed and fear and our egos and all of that negativity, it is actually because we want to survive.

And believe it or not, survival could be, “Oh, I’m not really sure if 20 million is enough. I need to gain 20 million more just in case something happens.” Or it’s survival of the ego. It’s like, “If I have 200 million or 2 billion or 20 billion, and the other has 21 billion, what’s wrong with me?” Unfortunately, that’s what plagues our current modern world.

Now, the reality is, if you really think about humanity, the purpose of humanity since the cavemen-and-women years was to live. And for some strange reason, we’ve optimized so much to achieve that objective and forgotten that this was the objective.

So, again, as friends off camera, we speak about those things quite a bit. You go through seasons in your life, and there is a season where you want to maximize, a season where you want to build, and a season where you want to look attractive in your middle age, or whatever crazy stuff that we have. But eventually, there is a season where you go, “Okay, so I’ve now lived and experienced so much. What have I missed? Have I actually lived any of that?”

Believe it or not, as scary as it looks to have no job to go to in the morning, if society provided for you, then you would go back to a much safer caveman-and-woman scenario, where there are no threats and no famines. You just really live, enjoy life, connect, ponder, reflect, and explore, which I know is very difficult for a lot of people. I do it for the first 3 hours of every day. It’s pure joy, right? To sit really with your curiosity, if you want.

And then, if you push all the way forward into Star Trek, that’s sort of what the Enterprise is doing at universal scale, right? It was basically, “You know what? Let’s go and explore now that we don’t really have to struggle with all of the wars and famine that we’ve created on Earth.” Now we can actually open up and create connections not just with humans, but with every living being.

Lovely science fiction, but at its core, I think it’s exactly what we’re about: a full life where you completely connect and enjoy and feel love, enjoy the pleasures of being alive, and have the curiosity to learn, explore, and connect. And it’s all at our fingertips if we just erase the systemic bias of capitalism that has gotten us here.

I mean, thank you, capitalism, for creating all that we’ve created so far. But can we please change it now from 1 billion dollars to, like, what I do? 1 Billion Happy is a capitalist objective, but it’s not measured in dollars, right?

Peter Diamandis

Mo, a question that Steven and I have been pondering in our new book is: What is it going to take for humanity—for all of us—to both survive and thrive in this coming age of AI, right? The survive part is an important element because, as we see jobs being lost and as we see probable dangers we don’t know how to deal with in terms of terrorist activities, thriving takes on a new meaning.

I think it does take on the meaning that we just spoke about, right? For most of us, you say, “Tell me about yourself.” Instantly, you go to what your job is, right? Instantly, you go to, “I’m a VP here. I’m the CEO there. I do this. I invented this. I wrote that.” Yeah, right? It’s an ego statement of who you are.

Your thoughts there, Steven. Do you want to start?

Steven Kotler

I think, like, here’s the thing. I think that question was already answered, in a funny way, though. Mo and I met a couple of years ago, and one of the things Mo said onstage at that time was, “I’m done writing books because AI is coming. I’m done writing books. It’s not going to happen anymore.” What did Mo tell us he did yesterday? He wrote with his AI, right?

Why did he write? Because it puts you into flow, because it creates passion and purpose and intelligence and creativity. So we have the answer to this question. We already know because we’re biological systems, and we know what the ingredients of thriving are: passion, purpose, compassion. We have a list.

We have the superintelligent AIs, and I don’t know a coder who has stopped coding because the AIs have come along. They haven’t; they’re still coding. Why? Because coding produces flow. Flow produces meaning, creativity, joy—like this. We’re wired this way.

So unless our fundamental hardwiring changes, we already have those answers as well. It’s like global cooperation. I don’t think these are puzzles. I think they’re engineering problems at this point.

I think, from Sergey’s perspective, Sergey would say, “No, no, we got the spark. Now it’s engineering,” and I agree. So I could be wrong. That was my 2 cents.

Mo, Peter, what do you think? I want to hear from you. I think you’re brilliant. Mo, please respond.

Mo Gawdat

You’re spot-on for a very interesting reason, Steven, as well, because when you really think about it, a writer was a writer whether he used a feather or a pen or a typewriter or a computer or now AI, right?

And if you look at my work, I’ve published 4.5 books so far. I’ve published 4, and my fifth is on Substack, but it’s going to be published, if you want. But I wrote around 13, and the other 8 I will never publish. I wrote them because, if you ask me why I write—why do I hug my wife?—there is enormous joy in that, you understand?

Having said that, Peter’s question was, “What would it take?” I wrote recently a piece that I called the MAD map spectrum, and the idea really is that it will unfortunately take a realization for humanity to change direction. That realization will either be a conviction of mutually assured destruction or a conviction of mutually assured prosperity, right? And between them there is no grayscale, unfortunately.

So if the US, at any point in time, is convinced that this MAD arms race to win intelligence supremacy is one that is going to lead to some harm to everyone in the world, they will stop.

Peter Diamandis

And if they stop competing, they will continue to develop, but they’ll start cooperating. If they’re convinced that it will lead to assured prosperity, that nobody’s going to stab them in the back, and that everyone is going to be enjoying a life that is very different for all of us but full of prosperity for all of us, then they will stop. They will continue to develop the technology, but they will stop competing.

Unfortunately, if you look back at history, we’re not able to guess those possibilities like a good applied mathematician on a game board. We have to hit them head-on. Everyone in the world knew that a pandemic was coming. Everyone, right? Everyone who at least studied virology.

Everyone knows that trade wars are going to hurt everyone, but we have to put them out there, fight through them, and eventually get to something. And it’s sad. Perhaps what we are doing—and I’ve dedicated probably the last 6 or 7 years of my life to—is saying that we really don’t have to hit our face against it.

It’s simple game theory, right? Understand that a prisoner’s dilemma where we are competing endlessly is going to end badly. Can we please stop? Yeah, we already know it’s tit for tat, right? You want the other strategy. You want the—we—it doesn’t matter how many AIs we put on that.

It’s the same thing with flow, capacity, and creativity. These problems have been solved. We know these answers. This isn’t like trying to unify gravity and relativity. That’s our problem. These are not.

Mo, my wish is that we were that rational and that we were compelled to have our optimization function be all of humanity. It’s not. And so I go back to what we’re going to get: we’re going to get a drastic event within the next 2 to 3 years.

A drastic event that, on one side, will hit us very badly economically, or on the other side will hit our fears very much, or, sadly, on the worst side, may kill quite a few million people. You could have a range: a hacker that, instead of attacking a physical place, switches off the internet or the power grid somewhere where the power grid is needed for life.

On the other extreme, you could get a hack into a bank, an evil war that goes out of control, or machines that turn on their makers. There will be some very big news headline. As always, it will last for 12 to 13 days before we start to talk about some kind of a pop star. But behind closed doors, I think decision-makers will wake up.

Going beyond that, because that is the use of AI by malevolent actors, the interesting thing about the US versus China is that China is a rational actor. They’re not—

Mo Gawdat

Thank you for saying that.

Peter Diamandis

Well, and the US is a rational actor. In other words, we’re not going to do something that will destroy us.

Mo Gawdat

Thank you so much for saying that. That’s actually not usually how the US media positions it. I also want to say, I think DeepSeek and the way DeepSeek was released was a very clear sign that China sees the same issues we see and wants to cooperate. I think it was rolled out with a message.

Peter Diamandis

Yeah, the message. I think it was a very clear message that it doesn’t seem like many people in America heard, but I was like, come on, people. This is really clear, and we’re all seeing it.

So, I look at DeepSeek and I look at what happened in China, and I’m like, no, no, we all see this. We all see that if we don’t start figuring out how to cooperate and build this stuff together, we’re screwed. I thought that was really cool. I’m glad you see it, too, Mo. A lot of people disagree with me on that one.

The point I wanted to make was, when you have a large population and a check-and-balance system, which you get with governance, versus a religious war going on and individuals who are looking to create maximal destruction and don’t have a check-and-balance system at all, that’s where we’re going to see, I think, the dystopian future, or those activities playing out in 2 to 3 years.

I guess I want to get beyond that and go back to the conversation: Is a digital superintelligence a benevolent god, or is it a Terminator scenario? I don’t believe that we’re going to see increasingly intelligent AI systems become Skynet, right? I don’t see them as needing to destroy humanity.

Unfortunately, Hollywood has built this scenario where AI is going to destroy humanity because it wants access to our energy. And, oh my God, we have so much abundance in the world. I think what I’m looking forward to over the next 12 to 24 months—over the next 1 to 2 years—is the incredible breakthroughs we’ll see from AI in physics, chemistry, and biology, which will unleash the next layer of abundance.

There are scenarios, however, where they could turn against us if we become really annoying. So imagine a world where—sorry, I mean, you have to imagine a world where job losses will position AI as the enemy. Right? A lot of people who may not be fully aware that the layer beyond the apparent layer is that capitalism and labor arbitrage are the reasons why they lost their jobs may blame AI. It’s not that they can do it.

But I think the truth of the matter is that you may be in a situation where you’re going to see man versus machine, and then the machine will go, “Seriously? Don’t annoy me. Don’t annoy me. Don’t annoy me.” And then we could see that.

My perception is that, in a very interesting way, I wrote a short book that I would never publish called Bomb Squad, which, of course, for someone with a Middle Eastern origin, you don’t write those titles. It was basically about defusing problems using weights of urgency and importance and so on.

The idea is, if you really look at our current future, I think the short term is both more explosive and more urgent than the long-term existential risk, especially because—I will say this very openly—I spoke about it with Jeffrey as well the other week. We don’t know the answer to how to address the existential risk.

Even if all of humanity decides that we want to address the existential risk, we don’t know how. We do not actually have a technical answer to do it. So we might as well focus for now on the immediate short-term, clear and present danger, and work on the ethics of humanity so that AI is deployed from the get-go in science and physics, discovering medicines, understanding human life and longevity, and so on and so forth.

If we set them in those directions from the get-go, then we’re more likely to see an AI that continues, as they grow older, to work with those objectives.

Steven, I’m going to go back to our quandary of surviving and thriving, and the surviving side of the equation. How do you prepare for what’s coming? How do you think about this for our kids, our society, and our leaders? Are we just bumbling in the dark?

That’s the way I feel it. We’re just bouncing around. We have huge political moves being made. We just saw, in the last couple of weeks, the entire AI royalty end up in Saudi Arabia and then in the Emirates, playing off against China. It feels like—I don’t want to say it’s a random walk—but I feel like we’re making it up as we go along, and there’s very little wisdom guiding this.

How do you think about that? How do we prepare for the next few years? Is there any way to prepare?

Steven Kotler

Well, I was actually thinking—Peter and I were in a room recently with the chief science officer for one of the big AI companies. I’m going to leave his name off, but he’s young, and he was talking about AI dangers. He sort of got frustrated with the question from the audience, and his response was, “You have to trust us. We know what we’re doing.”

Everybody sort of froze. We were like, “Oh, God.” Right? So my point is that not only may Peter be right—it may be a random walk—but even when somebody says something like, “We’re trying to train our AI to be moral,” and blah, blah, blah, when you hear somebody say that and you look at them—and this guy was in his early 30s—that was my reaction. I was like, “Dude, you want me to trust you?”

This is like Mark Zuckerberg telling me social media is good for me, or Marlboro telling me cigarettes are good for me. It sort of makes me think that way. So I don’t know if I have anything cheerful here, because not only do I think it’s a random walk, but I think when people try to steer, we’re suspicious of their ability.

I’m suspicious of their ability to steer. Right? That’s the story I just told you: this guy is brilliant, probably way freaking smarter than me, and he’s trying to steer, and I’m suspicious. I think it’s on both sides of this coin. I don’t know if I have any good news here.

Peter Diamandis

Let me frame it in the following way. I think we are holding 2 different futures in superposition, to go back to quantum physics—if you will, Schrödinger’s cat. In 1 future, we’re going to collapse the wave function to a brilliant, vibrant future for humanity; in the other future, we have dystopian outcomes.

The question becomes: How do we guide humanity toward this positive vision of the future? What do we do today? How do we help people? Is it—Steven and I have been talking about this—is it mindset? Are we going to help people create the mindset and the frames that allow them to survive and thrive? Or is there something else that needs to be done?

Mo Gawdat

Yeah. I’ll actually, first and foremost, second what Steven said. One of the top irritating comments I heard from Eric Schmidt—I worked for Eric for a while, so I respect him tremendously—but he said, “We will need every gigawatt of power, renewable or nonrenewable, if we are to win this race.” I think that’s the kind of blindness that you get when you’re running too fast, right? When you’re so afraid that the other guy will win, it’s those times when you start to make decisions that are not really responsible because you are blinded by something that you position as more important.

The way I look at it, Peter, is—I know it sounds really not positive, but there is positivity in it—I call it a late-stage diagnosis. What humanity is struggling with today is that we’ve been building a system, systemically prioritizing greed, prioritizing gains, prioritizing power, and so on, for so long. Those objectives have systemically built the world that we are in today.

The world we are in today is not healthy. It wasn’t healthy even before AI. In my book, which is in 3 parts—past, present, and future—in the past part of the book, more than half of what I write is not about AI. It’s about capitalism, the propaganda machine, and all of those things that will be magnified by AI.

What’s the point? If this planet is sick, if you want, and it’s a late-stage diagnosis, a physician will sit you down, look you in the eye, and say, “By the way, this does not look good.” But that statement, believe it or not, is not a statement of hate. It’s a statement of ultimate care.

Why? Because a late-stage diagnosis is not a death sentence. Many patients who have been diagnosed with a late-stage disease have not only survived, but thrived. They thrived because they changed their lifestyle. They changed something. This is what Steven teaches all of us: You can live differently, and when you live differently, you’ll achieve peak performance. You’ll achieve maximum health. You’ll achieve, you’ll achieve, you’ll achieve.

I think that’s what we as humanity need to start realizing: The systems that have gotten us here, from a process point of view, have nothing wrong with them, but from an objective and morality point of view, have everything wrong with them. What good is it to be a zillionaire in a world where there is nothing you can do with your money? What good is it to be the first inventor of an AI that basically renders you irrelevant?

I think we need to basically pause and say, “Do we want this anymore?” Sadly, it requires cooperation across human brains, which Steven rightly said at the beginning is not something we do very well.

The other thing is, I would put forward the notion that there is no on-off switch and there’s no velocity. We’re running open-loop with “yes” and “more” and “more” and “more” as the objective function. There’s no consideration for whether GPT-5 or GPT-6, or Grok-4 or Grok-5, or whatever your favorite models are, in the final result are going to enable something that is massively dangerous for humanity.

If that’s the case, I still go back to what safety valves do we have? I don’t see any action being taken by the leaders of the free world. Let me ask you both a question: If you could move to a planet that didn’t have AI, or where AI was developing at 10% the speed it’s developing at, would you leave?

Steven Kotler

I’d be gone. I’d go. I’d be gone to 2016 today.

Peter Diamandis

I don’t know anything. I’m sorry—your answer to that is what?

Steven Kotler

I would reset back to 2016 today.

Mo Gawdat

2016. I think AI today has all the upside and very little downside. I think it’s AI in the next 2 to 5 years that I’m so concerned about. AI today is incredible. And I didn’t say we were going to move to a planet where there’s no AI. I just said move to a planet where it’s going much slower, so maybe we can start to think about it.

Peter Diamandis

But that’s a fantasy. Well, I mean, Bigelow Space Hotel is coming to a universe near you.

Steven Kotler

So, Peter, I actually think that you’re accurate in your description of where AI is today. But it’s that 5%—5 degrees—deviation back in 2016 that led us to where we are today, right? You remember things at the time where we geeks agreed that we weren’t going to put it on the open internet.

Peter Diamandis

Yeah. It was Google.

Steven Kotler

Google developed this first and decided not to put it out there. Then OpenAI said, “Here it is. Here it is,” and no one had any hesitation. Put it on the open internet, teach it to co-create and write more code, and start the party of swarms of agents talking to agents, talking to AIs, right now.

I would definitely reset that. I would, however, say there are things we can do right now if we want to prepare. I’ll start with government. I don’t think we’re asking government for too much when we tell them to try and regulate AI. It’s almost like going to government and saying, “Regulate the making of hammers so that they can drive nails, but nobody can use them to hit someone on the head.” It’s a very complex thing to ask because they don’t understand hammers, and, believe it or not, even the guy who’s making the hammer cannot do that.

My ask of governments is: Regulate the use of AI. If someone uses a video that is a deepfake video and does not declare that it’s a deepfake video developed by AI, criminalize that. Make it legally liable to use AI to manipulate information, manipulate populations, and so on and so forth. This is the role of the government immediately: Regulate the use of this massively new technology.

For the rest of us—investors, businesspeople, and so on—I ask a very simple question: If you do not want your daughter or son at the receiving end of a specific AI, don’t invest in it. Don’t promote it. Don’t use it. It’s as simple as that. If you believe this can be harmful to someone that you love, do not give it the light of day.

Then, for us as individuals, I’ll go back to the late-stage diagnosis. Believe it or not, the way I live now—and you guys probably know this about me, not in front of cameras—is I hug my loved ones and enjoy every minute of every day, and I prepare. I learn the tool. I am one of the better users of AI in the world. I’m in line with the technology, but at the same time, I’m completely back to the purpose, realizing that I will do the absolute best that I can to spread the message.

I will do the absolute best that I can to say that ethics is the answer. If we show AI ethical behavior, it may learn it from us, just like it learned all of the other stuff from us. But at the end of the day, if it messes up, you’re going to hit that dystopia.

Peter Diamandis

Not forever. There is a point in time where AI takes over and says, “Okay, kids, enough stupidity. I’m in charge now. Nobody kill nobody.”

How far out is that, Mo?

Mo Gawdat

12 years.

Peter Diamandis

12 years.

Mo Gawdat

Okay, so 12 to 15, just so that people don’t come back and hit me after 12 if I’m still around.

Peter Diamandis

I’m going to wrap this episode on this subject line. It’s where we’ve come to before, which is that, in the near term, it’s the use of AI by malevolent individuals that’s our greatest fear. It’s not China versus the US. It’s the US and China against those malevolent players out there who wish to use this for greed and vengeance, whatever it might be.

I think this is an unstoppable progression. I don’t think there’s any on-off switch here. We’re seeing $1 billion a day being invested into AI, which is extraordinary, and I think that’s going to continue to increase. We’re seeing data centers being popped up every way, every place possible.

I’m the world’s biggest optimist, and I am optimistic about the impact of AI on human longevity, on new understanding of the physics of the universe, on new mathematics, on new material sciences, and on things that will create incredible abundance that Steven and I have written about and are writing about in our next book.

I’m looking forward to this benevolent superintelligence stabilizing the world. That’s what I’m hoping for.

Mo Gawdat

I agree. Steven, where do you come out on this?

Steven Kotler

I think that you guys wanting to invent an AI god to save you from yourselves is maybe the craziest thing I've heard since the guy from the AI company—I won't mention it—told me to trust him. But I love you both. That's actually usually the answer that you get: the only way to save us from AI is to use an AI.

Yeah. You know what? The beautiful thing is, we're going to find out. I also want to leave everybody with one thing, going back to what we were saying about cooperation, the upleveling of human intelligence, human consciousness, and things like that.

The human brain is widely considered the most advanced machine in the history of the universe, and we're just now, with the help of AI, figuring out how to uplevel it and link it with other brains. The level of cooperative possibility is enormous.

Let me get back to it for 1 second. Enlightenment, which is a definable biological state that produces universal compassion and oneness with everything—we're engineering it. It's a state that's starting to become available almost on demand.

So when I say there's a new level of cooperation coming that's emergent at the same time as the AI stuff, we can't see it. We have no idea. It's emergent, just like other things. So rather than the benevolent AI god, I think we're going to surprise ourselves.

I'm not the optimist in the room, by the way. Peter's the optimist when we're in the room. I'm not the optimist in the room, but I think I'm more optimistic than Peter on this one.

I'd love for that thought to be actually implemented. I think that's something that we really need to think about deeply. In the short term, I don't know who we could talk to. There you go, Peter. It's back to you.

Peter Diamandis

Thank you. I appreciate that. You were saying, Mo—please close this out.

Mo Gawdat

I was basically saying I think this definitely, definitely is the answer, if you ask me. If we just shift our mindset into cooperation, we head directly into a world of total abundance.

I was in a conversation with Eric Schmidt, whom he mentioned earlier, and his point of view was that until there is some type of disaster—until there is something perhaps like Chernobyl or Three Mile Island that isn't a 10 out of 10, but a 2 or 3 out of 10, and it scares the daylights out of us—we don't realign as humans. We don't realign, and we blindly go forward as we have been.

I believe that it's human nature that plagues us from being able to save ourselves many times, until that child in us burns our fingers on the stove, even after your parent has told you over and over again, “You're going to burn your fingers on the stove. Stop playing with fire.”

Peter Diamandis

Agreed. 100%.

Mo Gawdat

But let's be hopeful. Let's assign that task to Steven to design an XPRIZE for human cooperation. Let's assign another task to Peter to make it happen, and, yeah, let's assign a task for me to hug you both when you do.

Steven Kotler

I love you, Mo. Love you guys very much.

Peter Diamandis

Mo, how come you get to do all the hugging?

Mo Gawdat

Hugging you is hard work, Steven. You understand that? You move too much.

Peter Diamandis

All right, guys. Lovely. Thank you, bro, for lending me your brains this morning. It was fun thinking with you—a fun conversation.

I'm curious, as people listen to this podcast, where do you come out on this? How do you feel about it? Do you have a solution that we should all be thinking about and promoting? I'll ask my AI as well. It's not necessarily going to give me the best answer, but maybe our group mind, our meta-intelligence here might bring us that.

Have a beautiful day, gentlemen. Go hug somebody. Talk soon.

Steven Kotler

Thanks very much. Bye, guys.

Mo Gawdat

Thank you.