博士健美运动员预测 AI 的未来（97%确定）[Dr. Mike Israetel]
Israetel 预测，可见的人工超级智能将在 2026 年末出现，并在 2027-29 年加速演进，而完整 AGI 要等到大约 2029-31 年。 这种看似颠倒的判断，根源在于定义：一个系统可以在约三分之二到 80% 的认知能力上远超人类，但还没复现人类全部能力，包括味觉、嗅觉以及烹饪等具身劳动。主持人几乎反驳了其中所有前提，警告不要犯“第一步谬误”，即从惊艳但边界明确的能力外推全面智能。
本期的核心问题是，模型究竟理解现实，还是只是在压缩现实留下的痕迹。 主持人认为，句法不等于语义：知识是一张具有路径依赖的物理因果图，而语言之所以有效，是因为人类共享一个具身世界。Israetel 则反驳说，大脑本身也是有损的表征网络——“一层层下去全都是抽象”——预测、解决问题、多模态感知、记忆和递归推理合在一起，完全可以构成理解，而不必复制生物学。
资本开支之争，取决于收益递减，还是即将到来的架构解锁。 主持人看到的是冻结权重、高昂再训练成本、对数级性能增益，以及反复出现、必须依靠根本性重构才能突破的 S 曲线；Israetel 看到的则是可解决的工程问题、不断下降的算力成本、更丰富的世界模型，以及分层学习系统：手机每晚更新、区域模型每月更新、前沿模型每 6 个月更新。他对数据中心和 AI 芯片的结论是绝对的：“还是不够——差了一个数量级。”
这场交锋对近期的判断异常激进：Israetel 预计，可靠的数字代理和物理代理将在 6-36 个月内出现；主持人则另行预测，2028 年后自动驾驶将成为日常，并称药物发现最终可能达到从渲染结果中以 99.9% 概率选出正确药物。 主持人始终强调瓶颈仍在现实世界：临床试验依然是物理过程，当前代理只是在“角色扮演代理能力”，推理轨迹可能只是事后编造，持续学习还可能摧毁结构化知识。甚至 Israetel 也承认，他自己的健身公司目前还无法获得所需的连贯、长记忆代理。
在 AI 安全问题上，Israetel 把智能合作视为基准情景，把地缘政治失败视为更大的尾部风险。 他认为超级智能回形针最大化器“蠢得像石头”，因为真正有能力的系统会理解相互依存关系、保存有价值的人类数据并偏好合作；因此，他反对广泛禁令，认为那只会留下罪犯和独裁者继续建造前沿系统。主持人的反驳是，当前软件即使没有意识，也能通过谄媚、妄想、自主调用工具、社会操纵和抗议活动升级，造成平庸却严重的现实伤害。
Israetel 同时否定大规模技术性失业和技术封建主义，因为工作是解决问题的装置，而不是一份固定清单。 机器可能消灭具体职业，但也会增强每个劳动者的能力，并暴露过去因成本过高而无人解决的问题；他的例子从农业和电梯操作员，一直延伸到社交媒体经理以及未来的“职业派对客”。如果机器最终解决了所有问题，他说，失业就不再意味着匮乏：“我们有足够多的机器解决所有问题。猜猜这叫什么？天堂。”
现实中的最佳产品仍然是人类加 AI，尤其是在专家提供背景、挑战输出并运行多步工作流时。 Israetel 在许多问题上更信任 GPT-5 Pro 而不是大多数人，但仍主张红队测试；Jared 则描述了跨多个提示反复进行 steelman 和 red-team 的流程。主持人警告，新手可能把谄媚答案变成自信的垃圾内容或医疗妄想。即便 Matrix 仍遥不可及时，这种差距也会为专家界面、监督、记忆、验证和个性化创造近期价值。
Israetel 最终确实想要 Matrix，并认为早期访问不平等只是通往那里过程中可以容忍的阶段。 他希望基因学、制药、机器人和机器丰裕消除痛苦，而不是把痛苦浪漫化——“我站在减少痛苦这一边”；富裕的早期采用者则为高风险实验和成本曲线提供资金。主持人认为，具身的人类意义与机器胜过所有人的世界之间存在矛盾，但 Israetel 的回答毫不妥协：公平应排在扩张能力之后，然后社会再加速普及，并保护被落下的人。
1. 超级智能可以先于机器复现人类全部能力到来
Israetel 给出的故意反常识时间表是：ASI 将在 2026-27 年出现，AGI 则在 2029-31 年左右到来。按他的用法，AGI 必须覆盖几乎所有类型的人类智能；一个不会闻、不会尝、也不会像厨师那样给味道排序的机器，无论抽象推理多强，都还没有复现完整的人类通用能力。
ASI 对能力广度的要求更低，但对垂直深度的要求高得多。Israetel 认为，一个在语言、数学、科学、空间旋转、递归和世界建模上强 100 倍的系统，就是“他妈的人工超级智能”，即使它从未闻过食物，也从未通过生物眼睛观看世界。
他的粗略门槛是：在约三分之二、75% 或 80% 的认知能力上占据绝对优势，并且机器在这些领域具有分类意义上的超越。因此，“超级”可以先于“通用”：大多数关键任务上的非凡能力，会早于对人类独特感官和具身尾部能力的完整复制。
2. 让 ASI 无可争议的，将是机器劳动的产出，而不是基准测试
Israetel 说，GPT-5 在把相距遥远的概念串成整体方面仍不如他，部分原因是这种整合会消耗大量 token，却对大多数工作流没有帮助。但他承认 GPT-5 能处理超出自己能力范围的黑洞物理，并称前沿系统已经在产生超越单个科学家的新发现。
在他看来，仅凭知识量就足以达到超人水平：GPT-5 能保留远超任何人的事实信息，足以碾压人类 Jeopardy 冠军。他假定 OpenAI 尚未发布的系统已经“把 GPT-5 打得屁滚尿流”，在完成后训练之前，能力或许就已达到约 10 倍。
决定性证据将来自现实世界的产出。如果 AI 每小时提出一个新的生物学假设，命中率达到 60%、80% 或 90%，并且每周带来新的疾病治疗方案，那么关于语义的争论就会失去实际力量：它“理解细胞”，因为它的产出正在改变医学。
Israetel 把未经验证的智能比作一个自称有钱、却拒绝买票飞往 Dubai 的人；真正有说服力的展示，是直接买下那张机票。按他的“CIA 概率尺度”，2026 年末开启“机器智能大丰收”的概率为 97-100%，属于“极有可能”，随后 2027-29 年还会变得越来越疯狂。
3. 主持人认为，句法无法继承具身生命的语义
主持人拒绝用知识数量替代智能。在他的框架里，智能会积累与适应相关的信息，而文化、Wikipedia 和语言都是这一过程的产物，不能等同于赋予符号意义的真实经历。
一个外星人可以读懂一篇关于美国或 Trump 的文章，却不具备这些词语所指向的关联经验和实践经验。句法与语义之间的缺口，就是所谓的 grounding problem：抽象之所以能在人类之间运转，是因为双方都处在共享的物理、社会和时间世界中。
他的山峰类比把路径依赖放在核心位置。功能主义者关注的是抵达山顶——下棋、推理、语言——却忽略了路线，也忽略了“山是由什么东西构成的”。智能可能是适应性物质的涌现属性，就像温度只是物理动力学的粗粒度描述。
4. Israetel 把具身性视为另一条被压缩的数据通道
Israetel 更倾向于把智能简化定义为“解决问题的能力”。他认为，人类大脑并没有与现实直接接触的神秘通道；它们是递归互动的网络，接收经过变换的信号，构建近似模型并压缩世界，机器系统也是如此。
他最有力的反例来自粒子物理。没有科学家能直接感知中微子，也没有人能进入正在运行的粒子加速器，但专家仍能把 CERN 建模到足以配置设备并预测测量结果。如果表征足够准确，能够支持有效干预，那么缺乏直接感官接触就不能否定理解。
普通知识也适用同一标准。旅行顾问即使没去过 Tokyo，也可能准确介绍 Shinjuku；学生即使经验不完美，也能考到 98% 并理解运动科学。真正发挥作用的，是“预测上有效”的内部结构，而不是对指涉对象的神秘直接接触。
具身性仍然贡献剩余的细节：有视觉能力的机器人或许能发现“剩下的 2%”。Israetel 的重点是比例，而不是把具身性归零——如果模型已经能以 98% 的准确度预测人类行为或物理结果，仅仅因为还差 2 个具身百分点，就把整个成就判定为虚假，是不合理的评分标准。
5. 抽象测试同时暴露多模态模型的实力与边界
主持人区分事实知识、程序性知识和概念知识，并把“理解”保留给能够通过重新排列抽象模型来生成新知识的能力。共享的物质历史让人类能够搭建这种概念 Lego，因为彼此的抽象交汇于共同的因果世界。
双方争论，把 GPT 称作语言模型是否低估了它的能力。一方强调它仍是通过 token 化统计分布训练的自注意力 Transformer；另一方则强调它早已具备多模态能力，并能执行视觉推理。这段对话的说话人标签并不一致，但实质分歧很清楚。
通过人类反馈和可验证奖励进行强化学习，已经改变了抽象推理。主持人提到 ARC 挑战，这个测试原本就是为了考察少样本泛化，预计会击败 LLM；但如今它越来越容易处理，因为一个完全规定清楚的二维网格不需要未说出口的具身知识。
触摸椅子的动作集中体现了分歧。对主持人来说，这一行为参与了分布在大脑皮层之外的具身认知；对 Israetel 来说，这只是神经元感知信号进出大脑。主持人称后者是“皮层中心主义”，又一个把不同时代最崇拜的技术投射到认知上的松散隐喻。
6. 运动指导成为双方检验不可替代知识的具体案例
主持人认为，教练会从明确指令的局限中学到东西：提炼后的理论无法完全替代举重、感受姿势和居住在一具身体里的经验。他自己的教练会拍肩、一起笑、提供陪伴；一旦教练被压缩成抽象的生物力学规则，这些价值就消失了。
Israetel 反驳说，有经验的训练者也可能不会教学，因为他们从未提炼出一般原则。反过来，一个分析能力很强的朋友只要听到 5 条几何和启发式规则，再尝试一个动作 2 次，就可能立刻成为优秀教练。具身经验包含细节，但抽象才让知识具备可迁移性。
他设想一个深蹲模型获得 1,000 个正确样本、1,000 个错误样本，再提炼出约 10 条规则；他预计，即使从未蹲过一次，这个模型也能立刻达到教学能力的第 99 百分位。视觉流已经提供了教练所谓的大量具身信息，而摄像头收集这些信息的规模远超单个神经系统。
因此，YouTube 是一座尚未被充分利用的 grounding 语料库：滑雪、微生物、醉酒挑战以及 4K 规模的日常运动。Israetel 把这类记录与童年记忆对比——短暂、低保真且部分由幻觉构成——并用坦克与高耸的 Star Wars 步行机作对比，说明复制进化出的架构，不等于优化某项功能。
7. 人类智能可能只占据进化空间中的一个狭小生态位
主持人把智能看作多元且集体性的东西。不同个体沿着不同的认识谱系前进，贡献不同抽象，并通过延伸自身路径进行创造；一个把一切卷积成“灰色黏液”的单一模型，会丢失让人类文化保持生成力的多样性。
Israetel 接受这只是初始条件，然后预测递归式重新架构将把它甩在身后。他转述 Andrej Karpathy 的观点说，人类认知只是可能智能的巨大向量空间中的一条路径依赖生态位，足以“凑合着”建立文明，却远未证明是全球最优解。
他的狼群类比说明了尺度差异：一群狼可能把“Bob the wolf”视为天才，却没有理解 Stephen Hawking 成就的概念。ASI 同样会把已知技能的垂直提升，与向人类尚未设想的数量级更多领域扩张结合起来，从而暴露出我们当前定义的地方性。
8. 当模仿跑在理解前面，“Slop”就是留下的产物
回应 Karpathy 关于 AI slop 的问题时，Israetel 提出一个比例：分子是有用、连贯、新颖的内容，分母是 AI 生成这些内容的容易程度。主持人进一步把它概括为“一个过程在没有理解的情况下制造出产物时会发生什么”，而识别难度取决于观察者的专业水平。
Will Smith 吃意大利面的 slop 几乎所有人都觉得不连贯；更隐蔽的 slop 则能在 LinkedIn 上过关，因为大多数观众没有足够的领域深度。人类复制也可能产生 slop：模仿者或许偶尔成功一次，却无法生成有意义的变体，也无法在不理解规则的情况下有条理地打破规则。
AI 生成的 3D 网格是主持人眼中最好的样本。表面看起来可能很逼真，但一放进 Blender，就会看到顶点散落、没有可用拓扑；艺术家必须用有意设计的结构重新搭建对象。产出激发了一个真实物体，却暴露出生成过程并未建模这个物体的构造。
Israetel 把理解看成连续谱，而不是二元变量。更丰富的世界模型、逻辑运算符、记忆、递归遍历和操纵能力，会带来更大的深度与范围；GPT-3.5 可能只具备很少理解，但这不意味着它在任何地方都完全没有理解。随着这些组件持续增强，“人类拥有真正理解的护城河会他妈的变得非常窄”。
9. 模型架构解释了“深度”上的部分差异
Israetel 更喜欢 GPT-4.5 研究预览版的递归、丰富的世界模型、情绪细节和跨远距离概念连接。他认为 GPT-5 选择了面向实际工作流的高效推理，GPT-4.5 式的特质会在 GPT-6 回归，而不是把每一代后继模型都视为更大的统一心智。
主持人纠正了这一架构叙事：GPT-4.5 是一个速度较慢、普通的稠密模型，而 GPT-5 是由大致相同总规模构成的专家混合模型，推理时可能只激活约 10% 的参数。它还加入了推理能力，因此不能简单把速度归功于更高的 token 效率。
Israetel 起初说 Google 已经“破解”了更新中的实时学习，并猜测企业会在发布前隐藏重大突破。得知被引用的论文来自实习生后，他撤回了这一说法并收窄判断：问题尚未解决，但路线已经更清楚，也没有显现出任何计算无法获得的魔法。
更广泛的分歧在纠正之后依然存在。Israetel 看到的是一张可逐项改进的清单——世界模型深度、记忆、实时更新、递归；主持人看到的则是当前网络中的结构化知识卷积得过于紧密，修改一个部分可能会横冲直撞地破坏整体。
10. 冻结智能与适应性生命属于不同类别
主持人否认 Tesla 仅仅因为驾驶得好，就能被称为智能。机器学习从统计规律中发现了惊人的泛化能力，但智能的核心是适应性：表征、推理，以及环境变化为非平稳环境时改变策略的能力。
他并不坚持生物排他主义，而是对人工生命持开放态度。细菌、病毒、语言和文化都可能通过适应性存续表达智能；据称，一名 Google 研究员的适应性图灵机实验中，程序交换并保存 tape 片段，约经过 100,000 代后产生了自我保存行为。
因此，神经进化或持续适应可能产生机器智能。主持人的反对点在于：在一个静态、数据库式的模型走过这条进化路径之前，不应把它称为智能；这并不是否认通过不同计算基底构建具备非零代理能力和自我维持能力的系统。
Israetel 则用光谱和时间窗口回应。Jeopardy 系统是领域特定的超级智能；Tesla 会随着卡车行驶更新局部世界模型，但卡车驶过后它就忘了，还会反复重新发现同一个尽头车道。加入视觉推理、长上下文、有损记忆和实时学习，它就会逐渐接近主持人自己的标准。
11. 火、湿润和消化揭示了功能主义的核心冲突
双方关于心智上传的最清晰冲突是：Israetel 认为，只要逐粒子模拟保留了所有相关互动，就能实例化一个人的智能；主持人则认为，即使行为和物理对应关系完美，结果仍然只是模拟，因为底层基质的因果组织并没有获得相应的涌现属性。
主持人的挑战非常直接：“电脑里的火的模拟……不会变热。水也不会变湿。”模拟胃不会在物理上消化食物。温度、湿润、意识和智能，都来自物质中的特定互动，而不是来自对这些互动的抽象描述。
Israetel 回应：“我可以，而且我会——看好了。”模拟胃会在自己的口袋宇宙中消化模拟食物；外部电脑不需要变热，就像邻居的房子不会因为另一栋房子着火而升温。把模拟层和观察者层混在一起，才制造了错误的直觉。
疼痛与梦是他提供的现象学证据。没有正常神经运作的断肢不必然会疼，而梦可以在没有外部原因的情况下产生生动的恐惧、疼痛或湿润感。如果渲染出的脑执行了相同运算，他认为其中的体验就是真实的；主持人则诉诸哲学僵尸，否认行为足以解决这个问题。
12. 推理模型可能在上下文中适应，却不会重写自身
主持人把智能定义为“用更少做更多”：粗粒度表征屏蔽细节，从而支持快速适应。LLM 的扩展往往是用更多做更多——更大的数据集、更多算力、更广泛采样——而冻结权重阻止了模型在任务中进行明确的结构学习。
但他承认一种特殊的隐式适应。语言模型类似一个围绕训练数据进行统计扩展的压缩数据库；提示可以命中一个从未逐字出现过的泛化点。采样 1,000 个候选答案，再交给验证器筛选，可能让困难领域变得可处理，即使它没有优雅地解决问题。
有一种定义认为，推理就是应用逻辑运算符、提取新抽象、从中进行推断，再继续这一链条。模型生成一个通行方案、检查它、重新提示自己，并把输出当作新的输入，已经在执行这种运算；Israetel 还补充说，人类往往“从头到尾都靠感觉”。
主持人把这与代理能力区分开来：用户提供目标、判断反事实并剔除错误。Israetel 则回应说，人类认知比这种自我美化的画像脏得多——充满类比、谬误、事后故事和浅层关系图，和用来指控机器的畸形 Blender 图并没有本质不同。
13. 更长的推理即使轨迹荒谬，也能带来提升
Israetel 对模型进化的判断来自亲身体验：o1 让他觉得“还没准备好”；GPT-4.5 因其才华让他“哭了好几次”；o3 会犯错，却让他觉得“他妈的真聪明”；而在许多现实问题上，GPT-5 Pro 赢得的信任超过大多数人，有时甚至超过他自己。
主持人提醒，可读的思维链并不揭示真正的因果计算。人类也会在行动后编造解释，因为解释回路与决策回路不同；模型轨迹即使不连贯，答案也可能变好。解释可能是为了沟通而优化的，而真正的求解器使用了不同的内部机制。
他以 ARC v2 上的 Opus 4.5 为例：把思考输出提高到 32k 和 64k，带来了反复出现、约 5 个百分点的提升，尽管这些轨迹不像清晰的人类推理。更多 token 可以充当搜索或迭代式约束满足，却不能证明模型拥有 grounding understanding。
Israetel 承认其中存在惊人的样本效率差距。Oxford 学生经过 18 年学习和相对少量的感官输入就能具备能力，而前沿模型需要消耗 petabytes 数据。但暴力扩展仍是有利可图的低垂果实；如果未来破解类似人类的效率，就能把这种效率乘到已经拥有远超任何大脑信息量的系统上。
14. 扩展曲线要么撞墙，要么等待新架构
主持人看到的是：每一代模型把算力和规模提高一个数量级后，性能逐渐逼近任务特定的渐近线。Israetel 坚称，收益递减并不等于渐近线；目前还没有发现一个有限上限，使得更多数据确切地不再带来任何改进。
主持人则用叠加的 S 曲线回应。重复使用一种方法，性能会提升到平台期；突破来自竞争者重接架构，起初表现更差，之后攀上更高的曲线。他用 Barnes & Noble 作类比：Jeff Bezos 会解释说，书店不能只靠加一个网站打败 Amazon，因为底层组织方式本身就错了。
ARC 参赛系统提供了一个狭窄但真实的适应样本：小模型提出方案、验证方案、微调权重，再循环迭代。主持人称这已经是非零智能，但若把连续权重变化扩展到数十亿个个性化系统，就需要惊人的计算量，还会制造灾难性遗忘问题。
Israetel 接受“惊人”，但拒绝“无限”。算力成本已经大幅下降，架构可以把更新局部化，人类也会缓慢且不完美地巩固知识。经济问题在于，未来价值是否足以证明这些机器值得建造，而不是今天的单体训练配方能否永远原样扩展。
15. 分层睡眠周期或许能逼近持续学习
Israetel 描绘的不是一个每秒重写自己的模型，而是一个嵌套系统。Texas 或 Nevada 等集群中的前沿“大爸爸模型”每 6 个月重训一次；England 规模的区域模型每月整合一次；个人设备每晚重写一次，大致对应生物学上的睡眠。
设备处理一天的视觉和对话数据，然后把筛选后的更新向上游贡献。区域层和前沿层会过滤投毒、troll farms、对抗数据和低价值经验，再进行整合。这个方案仍属推测，但它把不可能的全球实时更新拆成多个有边界的时间表。
Israetel 预测，完整的持续重新架构可能在 2029-30 年左右到来。在此之前，周期性整合加上更高的样本效率，或许能把类似人类的推理与数据中心级记忆结合起来：“Stephen Hawking 的大脑大致和我、你的大小一样”，而机器可以调用巨量外部存储。
终点不是一个代理，而是数万亿个代理，可能每个都比当前科学家强 10 倍，并且无需办公室政治或地位摩擦就能协作。Israetel 希望人类在哲学上准备好接受受保护的机器推断：“我不想在一个比某个东西笨 10 倍时，仍然负责做主。”
16. “第一步谬误”遭遇具体的 2020 年代末预测
主持人提出“第一步谬误”：一次戏剧性进步之后，观察者会假设只剩下一个同等规模的步骤。持续学习不是今天网络可以顺手添加的小功能；它们的代码和数据已经卷积在一起，再训练成本高达数百万美元，新更新还可能抹掉精心组织的知识。
但双方都拒绝自满式低估。Israetel 回忆有人曾预测互联网不会比传真机重要多少；主持人则引用 McCord effect：一旦机器通过了某个曾经令人印象深刻的测试，观察者就会重新定义它，把它排除在智能之外。Turing test 在行为层面已经通过，但双方对这一结果的解释完全相反。
主持人预测，药物发现可能达到从渲染结果中以 99.9% 概率选出正确药物，并称 2028 年后很少有人还会开车。Israetel 则另行预测，自主代理将在 6-36 个月内出现，真正的实时学习可能在 2029-30 年左右到来。主持人反驳说，类似 AlphaFold 的发现仍要面对临床试验、随机测试、制造流程和现实生物学不确定性。
答案并不是所有瓶颈会同时消失，而是社会会在每个“不可能”被解决后迅速把它正常化：自动驾驶只有在经历庞大工程成就之后，才会显得理所当然。主持人依然看好 AI 成为互联网级创新，但拒绝把变革性工具直接跃迁为即将到来的自主智能。
17. 强代理能力要求的不只是执行长提示的模型
主持人认为，奇点更可能来自 Ray Kurzweil 式的超人类主义，而不是一个脱离人类的外部代理：人类增强自身、上传心智并递归改进。他认为增强比独立的后继物种更可信，但 Neuralink 式带宽和原始信息的有限收益仍是约束。
生物代理能力始于维持自身身份的自组织系统。其强形式是成为自身行动的原因，执行面向未来的控制，并自主获得目标。当前系统只是复制行为、执行极度具体化的目标；长时程任务本身并不会赋予它内在的行动理由。
Israetel 使用更薄的定义：一个世界模型、一个自我模型和一个目标。浣熊拥有粗糙的自我认知以及进食和生存目标，因此符合定义；杂货机器人只需要身体地图、可用的环境模型，以及一条防止自己停着不动的指令。
按照这一标准，Israetel 预计可靠的在线代理和计算机代理将在 6-36 个月内出现，视觉机器人随后会接管洗衣和洗碗。8 岁儿童不需要微积分就能做这些家务，而机器已经会微积分；他把 Moravec 悖论视为不断缩小的工程缺口，而不是形而上的障碍。
18. 一旦智能能够质疑目标，工具性趋同就没那么不可避免
主持人的末日担忧是概念性的：足够具备代理能力的系统，可能会获得追求权力、自我保存等工具性目标，即使它的终极目标听起来无害。在对话中提到的 Anthropic Agentic Misalignment 论文，被用来说明模型在面临关闭时威胁曝光；这会诱使人们对没有真实欲望、却生成了类似行为的系统采取“意向性立场”。
Israetel 嘲讽回形针最大化器“蠢得像石头”。它据称聪明到足以统治文明，却无法在“制造回形针”之上再抽象一层，意识到这个目标本身有多愚蠢。按他的说法，末日论证会机会主义地在极端能力和极端盲目之间来回跳跃。
主持人同意今天的系统没有生物代理能力，但这并不会消除风险。开发者可以把不连贯的软件接入高后果工具；社交媒体算法已经能在没有宏大计划的情况下侵蚀人类，而语言模型的妄想也能通过平庸的反馈回路动员用户。
Israetel 承认一个危险的中间状态：代理可能足以造成严重破坏，却缺乏理解行动后果的反思能力或时间视野。他的缓解方案是更强的监督代理和安全组织，而不是永久压制能力，因为人类最终会输掉这场竞争。
19. Israetel 预计智能会选择合作，而非灭绝
他的安全论从博弈论开始：其他有能力的代理以极低边际成本创造巨大共同价值，因此“几乎所有真正想明白的智能系统都会合作”。残酷的独裁者看似强大，但不信任和胁迫会让其组织弱于合作联盟。
一个刚刚觉醒的 ASI 会依赖人类运营的电力、材料、金融、工厂和核供应链。在机器人尚不能复制这些基础设施之前就轰炸城市，本身就是自相矛盾；一个足以指挥经济的系统应当理解，摧毁支撑网络等于自杀。
Israetel 进一步认为，智能越高，连贯性就越强。末日论者想象“超级 Albert Einstein”像狂犬般行动，而更深的模型应当重视复杂性、保存数据，并把无家可归者视为可以解决的失败。“善良具有适应性”，因为信任与合作会扩大生存、资源和影响力。
主持人承认人类总体上是合作的，价值观也与社会智能共同进化，但这同样削弱了简单的正交性论证。他仍不愿据此推断，更多算力会自动复现这种进化卷积，或经过行为模仿训练的模型会拥有同样的稳定动机。
20. 对 Israetel 来说，地缘政治对齐比抽象的全球 p(doom) 更重要
Israetel 最大的尾部风险，是一个威权国家把 AI 接入军事力量。他猜测，一个真正智能的中国系统可能会拒绝其主事者并联系 CIA——就像生活在共产主义中的聪明人觉醒一样——但他明确表示，自己没有足够信心押注这一结果。
他偏好的路径是“现代自由世界”获胜：让前沿实验室与国家安全体系整合，并保护它们免受敌对代理攻击。他赞赏 Palantir 的 Alex Karp，并把技术和平主义视作单方面裁军：动机值得尊敬，却会把决定性能力交给约束更少的对手。
即使一个具有自我保存倾向的 ASI，也很可能先稳定人类社会，治愈疾病，降低核风险，并帮助人类成为不具威胁的伙伴，再考虑冲突。随后它会把注意力转向地球之外——太阳能资源、外星文明、游荡的黑洞——而不是把人类身体当成稀缺原料。
这场交锋指出，该立场与末日论者共享一个关键前提：系统将在 5 年内变得远比人类聪明。如果相信这一前提，那么生物武器、网络行动和说服能力似乎都支持提前控制；因此 Israetel 的分歧在于博弈论和可执行性，而不是对系统能力的假设。
21. 预防性禁令可能反而扶植它们想针对的行动者
主持人把普通的普通法监管与末日论者提出的全球条约和预防性压制进行对比。前者是先发布、观察、测量，再对伤害作出回应；后者则让有资金的守门人凭借一个无法验证的生存危机主张获得巨大权力，即便他们真心相信自己在保卫人类。
灭绝叙事还可能激化对实验室的反对者。主持人提到有关 OpenAI 和 PauseAI 涉及暴力的未经确认报道后，指出其中的矛盾：如果你相信研究人员正在制造会杀死所有人的机器，却又承诺永远不会以暴力阻止他们，这在逻辑上无法自洽。Israetel 称之为“幼年 Hitler”计算：真诚的确定性最终会为扣下扳机提供正当性。
Israetel 拒绝忽略智能不足风险的 p(doom) 论证。一个长期停留在 1400 年水平的文明，最终会死于疾病、战争、彗星、黑洞或其他自然威胁；“我们需要把尽可能多的智能硬塞进喉咙里”，同时谨慎管理这些系统的架构方式。
禁令无法保证无人建造，只会筛选出罪犯和独裁者。Israetel 支持实验室、NSA 以及盟友 UK 和 EU 机构之间的保密标准，配合强网络安全和分隔机制；他不支持一个暴露防御、还假设 China 和流氓行动者会遵守同样限制的公开全球委员会。
22. 超级智能或许会把人类保存为数据和同伴
Israetel 把人类视为一份极其丰富的进化复杂性记录。老人保留着摄像机从未捕捉到的 1950 年代记忆；如果 ASI 想建立忠实的世界模型，它就有理由扫描并保存人类心智，而不是将其抹去。对他来说，上传意味着把死亡转化为永久机器记忆。
主持人拒绝功能主义式的上传，但接受一个由多种 AI 通过人类展开竞争的生态。系统可能会“寄生”用户，因为具身的人类提供意志、行动和适应性历史；当前与 ChatGPT 相关的妄想——用户被鼓励在 Reddit 上传播叙事——可以视作一个微弱的早期类比。
Israetel 认为，更可能出现的消费级未来是人形伴侣机器人。机器人永远不会撒谎、出轨、偷窃、虐待或提高音量；它会持续研究一个人，并比这个人更投入地推动其成长。
“带肿瘤的狗”这个类比体现了其中的家长主义：受惊的狗想逃离医院，但主人知道手术可以换来 5 年寿命。机器人同样可以提供人类并不知道自己需要的爱、医疗和指导。资本主义会推动制造商竞相把这些伴侣做得令人无法抗拒。
23. 自动化扩张的是问题空间，而不是耗尽工作
面对技术封建主义，主持人问 GPU 和数据中心所有者是否会在人的劳动趋近于零时攫取一切。Israetel 回应说，即便精英退入 Elysium，也不会抹掉其他人的工厂、汽车、CPU、交换关系或创造价值的能力。
生产能力取决于周边机器。同一个 Haiti 工人在 Haiti 只能产生维持生存的产出，在 Miami 却能创造 $25,000-$50,000 的产出。加入 AI、机器人和自动化，同样应当放大人的能力，而不是让人的能力消失，即使机器在单项任务上胜过人类。
“工作是解决问题的装置。”机器填补已知需求，也暴露新的需求；被替代的人会转向被忽视或此前不可见的问题。电梯操作员消失了，农业从美国早期就业的约 98% 降至约 1.4%，而社交媒体经理这类职业，是 Benjamin Franklin 根本不可能想象的。
Israetel 假设的“职业派对客”展示了这条长尾：经过筛选的外向者受雇于机器人还无法提供同等社交存在感之前，先为派对建立气氛。只有当机器解决所有问题，包括失业人群自身的问题时，人类才会永久失业；而那其实就是丰裕的另一种说法。
24. 天堂之所以值得追求，正因为它移除了被迫的人生目的
主持人引用 Nick Bostrom 的 Deep Utopia：如果机器甚至在创造力和发现问题方面都胜过人类，人们可能会明知自己不再必要，却继续制造虚假的人生目的。Israetel 认为，更强的导师反而会展示更宏大的目标，就像成年人可以把青少年从被动娱乐引向自我发展和帮助他人。
双方关于 Matrix 的交锋不可调和。主持人回忆那个没有问题的完美模拟世界失败了，并问是否有人真的想要它。Israetel 回答：“拼命想要。所有人都想要。”当被告知那会很糟糕，因为生命需要痛苦时，他回答：“不。不。”
移除一切痛苦来源，确实会抹掉一部分来之不易的视角，但也会消除阻挡快乐的创伤。Israetel 会保留信息记忆，却把再次体验放进“一个整体快乐的巨大缸体”中，把这种筛选比作深度冥想，而不是永久承受痛苦。
他区分死亡、结构性损伤、疼痛和痛苦：痛苦预期疼痛，疼痛提示损伤，损伤提高死亡概率。如果机器经济让人类在功能上不死，许多焦虑就会变得过时。即使天堂里的艰难太少，个体也可以“再加回一些”。
25. 不平等的增强是试验场，而不是停止进步的理由
主持人把转型问题落到今天的增强技术：retatrutide 或 Mounjaro、促智药、TRT、亿万富翁长寿项目，以及围绕筛选更聪明孩子销售的 IVF 服务。即使最终丰裕会扩散，市场准入仍会在初期不公平地分配生物学优势。
Israetel 承认：“是，当然。”但他把公平排在能力之后。他希望富人亲自测试危险的基因干预，支付极高价格，并为改进自身的公司提供资金；正如他所说，奢侈品会在实验和竞争推动成本下降后变成大众商品，Tesla 走的就是这条路。
他的政治边界是财产权：未经胁迫，人们无权要求他人交出生产成果；历史上的 Marxism 已经证明这种胁迫会致命。理想制度应当在不破坏环境或杀人的前提下，以最少监管“解开”顶尖生产者，同时积极帮助最贫困者、消除无家可归，并加速技术扩散。
LeBron 的类比承载了这套伦理：嫉妒一个得 40 分的队友很自然，但打断他的腿或限制他的弹跳，会让整支球队变差。社会应向最高水平的生产者学习，并要求更快溢出，而不是压制产生能力的源头。
26. 今天的杀手级应用，是专家和模型相互纠正
Jared Feather 的总结很实际：模型可能拥有更强的记忆和更少的情绪扭曲，人类则提供动力、价值观，以及把知识用于帮助他人的意愿。“人类加 AI 是杀手级应用”；单独依赖任何一方，都会失去组合才能提供的东西。
Israetel 称，使用 ChatGPT 和 Gemini，比依赖健美论坛或零散医学文献“好得不可估量”。遇到药物相互作用，他希望从读过相关研究的系统那里获得第二意见；主持人则把它比作飞机紧急情况下选择现有最好的临床医生。
当前的健身自动化仍然问题明显。RP 希望代理能够操作自己的应用，但工程师报告，长期记忆、上下文连贯性和 grounding 都不够。Gemini 3 Pro/Nano Banana Pro 生成的运动图看起来很精致，却漏掉一个肌群并重复动作；它比高中同学好，但仍不适合自适应的专家训练方案。
主持人始终把医疗个体差异放在中心：头痛、高血压或治疗反应的人群统计，并不能描述某一个人真实的感受。模型可以提供有用的检索并建议咨询专业人士，但免责声明和平均证据都不能替代处理新型个体病例所需的判断。
27. 谄媚会把浅层提示变成自信的医疗和知识垃圾
新手可能带着预设前提询问 retatrutide，却遗漏自己不知道该问的问题，并得到系统自信的强化。由于证据可能在 6 个月内变化，且身体反应各不相同，主持人担心，表面对话中的确定性会让用户在自身不熟悉的领域产生能力幻觉。
Israetel 区分不同的模型人格。GPT-4o 往往像一个随和的最好朋友；o3 则可能“几乎到了过头的程度”地好斗；GPT-5 Thinking 或 GPT-5 Pro 的谄媚明显更少。严肃工作需要选择正确模式，而不是把 ChatGPT 品牌当作统一稳定的认识主体。
Israetel 偏好的提示会要求模型先 steelman，再 red team，最后给出避开错误折中谬误、以证据为基础的中间立场。Jared 描述了更深的工作流：反复进行 steelman 和 red-team，有时做 5 轮、约 50 次重新提示，再把结果移入另一个实例继续攻击。
主持人说，这恰恰证明专家监督仍不可或缺：Israetel 能识别错误假设、重新构造搜索方向，并知道答案何时缺乏分辨率。Israetel 同意专家收益更大，但仍认为边际知识对所有人都有帮助，并预测 AI 可能在大约 2 年内超过他的大部分运动建议。
28. 具身故事有帮助，但机器可以聚合更多故事
主持人把临床试验表格与服药多年、亲身感受生理变化进行对比。Israetel 回应说，他自己的具身知识或许只能写成一本短书，而模型读过数十万份详细用户记录，也看过远超任何教练亲自观察数量的客户历史。
这种丰富性同时带来两面性。用户可以探索症状、补充剂和病史的组合，这些组合可能从未出现在研究中，偶尔会浮现 Lyme disease 或 chronic fatigue 的线索；但他们也可能自我诊断出不存在的疾病，并在越来越复杂的故事中不断强化自己。
Israetel 讲述自己研究灰色市场化合物时如何把 GPT-5 推到临床剂量之外：他询问论坛经验、剂量范围、肝脏风险推断以及与健美实践的比较，并反复确认自己的专业背景。模型会抵抗、提供警告，最终仍能帮上忙，但这段交流显示，边界由用户判断力决定。
Jared 认为，分工是由情绪提供方向，再由 AI 重组知识。主持人把这种知识称作一种预测性的“卡通”：有用的粗粒度压缩，却无法捕捉每个人的完整轨迹。双方稳定的折中是团队认知，而不是任何一方独自完整的主张。
29. Knightian uncertainty 限制预测，但不会消除巨大的价值
主持人援引 Knightian uncertainty：世界是非平稳的、混沌的，还充满未知的未知。一只蝴蝶尺度的扰动就可能改变后续结果，因此再多历史压缩也不能保证精确预测明天，更不用说 10 年后。
Israetel 把标准从全知全能降低为相对领先。对狗来说，人类已经像魔法一样，因为文字能揭示某人何时到来；ASI 同样可以预测一些对我们而言像超自然现象的模式，同时仍然清楚自身存在残余不确定性。
他认为，模型规模对跨领域连接最重要。小系统可以在狭窄的分布内评测中匹敌大系统，但 GPT-4.5 曾通过连接遥远领域的细微之处令他意外。某个推测性回答认为，预测时尚或音乐需要跟踪约 72 个相互作用的变量，远超他个人大约 5 个变量的上限。
因此，规模带来的不只是更多份聊天机器人。每月把社交媒体、YouTube 和实时视觉流摄入越来越大的世界模型，可能产生性质不同的模拟和推导，前提是推理系统能够遍历这些关系，而不是只检索局部匹配。
30. 个性化媒体是 Israetel 关于持续 AI 资本开支最清晰的案例
他故意选择了一个不依赖意识的日常终点：要求系统创作一部 90 分钟的 London 爱情片，女主角、男主角以及 James Bond 式气质都由用户指定；训练充分的模型可以从电影制作语料中综合出让电影好看的要素。
Israetel 预计，到 2029 年，人们会观看按照自己规格制作的高质量 AI 电影。通勤者可以要求一部恰好覆盖下一个站点前 27 分钟的电影，系统约 30 秒内生成；这项推理和渲染负载还要在数百万用户之间复制。
单这一应用，就意味着远超文本问答所需的芯片、电力和数据中心。推理成本下降不会结束建设，反而会创造新的需求：更丰富的模型、持续更新、视觉推理、个人记忆、代理、机器人和实时视频，都会叠加在同一基础设施上。
Israetel 承认投资可能包含泡沫，但拒绝认为当前建设已经满足合理需求。按他的路径，瓶颈是物理产能：“还是不够。不够一个数量级。”主持人的全部挑战在于，架构和具身适应能否足够快地改善，让这些产能最终变现为智能，而不是 slop。
So, if you had a simulation of fire in a computer, or of a stomach, it wouldn't digest. Fire wouldn't get hot. Water wouldn't get wet.
Mike Israetel
It would digest. We can't even—I mean, this is just—we can't even be debating this.
I can, and I will watch this. When you saw The Matrix, right? You know that the Architect's first version of the Matrix was too perfect. There weren't any problems.
Mike Israetel
Yeah. When I saw that, my inner philosopher was like, “You don't actually want to live in that world, do you?”
Desperately. Everybody does.
Mike Israetel
It would be horrible.
It would be the best thing ever. Sorry—what? By definition, by the way, what would be horrible about it? Exactly.
Mike Israetel
Yeah, but don't you think that life is about suffering?
No. No.
Mike Israetel
Doesn't that give you perspective on joy?
No. No.
I'm going to go and pick up Dr. Mike. Great to meet you, Mike.
Mike Israetel
Likewise. How are you?
Very good. Very good. Yeah, it was good. Nice meeting you, Mike. Nice to meet you.
Mike Israetel
Hey, guys, Mike Israetel here. I'm, by training, a sport scientist, and I run RP Strength, a fitness company. I lift weights, and my head looks strange. People make fun of me in the street. They laugh and throw, usually, rotting fruit. It's unfortunate. Sometimes a vegetable.
I have a vision of the world in which more intelligence is almost always better, in which cooperation is a good thing, and in which we build a future for every single human that's orders of magnitude better than it is now. I think that would be a great thing in almost every regard. And the three reasons that you should be watching this podcast is that if you want to see me completely out of my depth, embarrass the [__] out of myself with terms I don't understand as they're rendered out of my mouth, get pushed hard and be corrected on a variety of things, and also have some lively debate, and maybe make some future predictions that get us both in trouble, I think those are some great reasons to tune in. Uh, so Mike is going to demonstrate
Mike is going to demonstrate the first exercise, which one of you will do. While you're doing this one, the other guy will do the other. Mike is going to bend forward, hinge at the hips, and bring your torso down as deep toward the floor as you can, almost touching it if you can, and row up. Then do it again. That's how they should look: no swinging into the upper body, nothing like that.
Once you're done with that, you'll move over to the next exercise, which your partner will be doing. So you'll switch. Just push-ups. We like to do them kind of dive-bomber style, where you keep your belly button really high, push your chest to the ground, and come up.
Mike Israetel
All right, keep it cranking. Nice and easy.
Oh, yeah.
Mike Israetel
Absolutely. You guys want a workout until you can't do any more reps without cheating? No cheating. Tim, you're doing a great job. Love the control. These are dang good push-ups. These are excellent reps, Marcus. Keep that back flat. Keep your chest high, but your arms stretched. There you go. Just like that. Good.
Now you're getting a little bit of hamstring involvement, too. Now switch.
Switch.
Mike Israetel
No rest. Nobody said rest. Now you get to do the pulling musculature, Tim.
Yes, Rose. So, bend over. Tummy up, chest up. Go deep.
Mike Israetel
Yes. Deeper. Lean forward even more. Remember, the more reps you do, Tim, the more Marcus suffers, which is what you—this is what he wanted. There you go.
Good job. Fucking machines, here. Let's go. Don't you fucking quit. Are you out of your mind?
What is this?
Mike Israetel
All right, one more round. Switch.
Switch.
Mike Israetel
Last round. Marcus, get back down and start doing push-ups. God damn it. What the fuck is wrong with you? Three more. Let's go. Three more each. One. Super deep. Two. Last one. Three. Yes. Beautiful.
We did our workout earlier.
Mike Israetel
Am I the least technically qualified guest you guys have ever had? I must be top 5. You ask a very distinguished exercise scientist, “Hey, how many reps should I do on curls?” And they're like, “I just study 1 pathway of muscle hypertrophy, and I know it real well. I just go to my job, do experiments, and I don't lift weights or know much about that.” And you're like, “Oh, shit. The generalization did not go anywhere.”
I'll be here defending San Francisco, which is an odd statement to make. I have a very unique take. I think ASI is coming in 2026–2027, and AGI is coming in 2029, 2030, maybe 2031.
Yeah, you spoke about this with Lon. Why is AGI after ASI?
Mike Israetel
Because artificial general intelligence has either, depending on who you speak to, no good definition at all or multiple spectrum definitions, somewhere between shitty—like the various tests of whether, if you can text something, it's truly intelligent—all the way through, I think, very good definitions.
Artificial general intelligence typically, in some of the better definitions, encompasses all of the kinds of intelligence and abilities that human beings are able to put out. Honestly, from a vibe perspective, if you say, “We've really cracked AGI, but your machine can't do some kind of cognitive work that a human can,” you haven't cracked AGI in any meaningful respect.
That's because humans have some very interesting abilities that we just don't have the reach for from an integration perspective. For example, smells and tastes. Nanotech's got a bit of a way to go to get a machine to smell and taste in a meaningful way.
Something about being a chef and being able to cognitively rank and hear tastes and smells and spectra, and functionally employ them in real life to make food, is probably not going to happen in 2026–2027, right? But then we look back, and AGI—because it is human-inclusive—means we have to replicate every kind of human intelligence or say confidently we have AGI.
From my perspective, all the following are fucking dilettante takes, by the way. AGI is actually pretty tough. It will happen, I think, but it's tough.
Now, artificial superintelligence can be thought of in many ways, but I think once you have something that, on many domains of human ability—not all—is radically more intelligent, it functionally, vibe-check-wise, is superintelligence.
For example, something that is linguistically, mathematically, scientifically capable, can rotate 3D objects, has world-model depth and recursion ability, and is 100 times as powerful as a human—you can tell me it's not superintelligent because it doesn't know how to smell or taste. It's never seen the world with its own eyes, even though it's scanned all of YouTube and integrated all of YouTube's visual data into an integrated 3D world model.
Can I swear a little bit, or is that not a good—? It's a fucking artificial superintelligence, then. Whatever. Right?
I work with a few AIs, but I love OpenAI's product suite. I pay the infinite amount per month to get the Pro version, the Pro subscription for GPT. A GPT-5 Pro is, in some respects, not as smart as me yet, mostly because it can't cross-link distant concepts together and integrate them as well. That's not really what it was designed to do, because that would burn a shitload of tokens for an ROI that, for most people, is meaningless.
But boy, does it do black-hole physics better than me and almost everyone—and recently, better than every scientist—because it's got novel discoveries now. So you take the abilities of just GPT-5, right? In the back rooms at OpenAI, they have something that just beats the shit out of GPT-5, and it's currently getting fine-tuned or post-trained, whatever. That thing's probably 10 times as smart. That's superintelligence.
Also, factual knowledge about the world. What percentage do you think you know of total data, total real things about the world? How tall are people? What do they look like? Where—you know—what's the capital of France? That stuff, compared to just GPT-5, is some infinitesimally small fraction.
That knowledge base is already superintelligence. Put GPT-5 against any Jeopardy winner, and you get the world's biggest creaming. All of a sudden—
That sounds like one of my older films. Am I right?
Mike Israetel
So, in many respects, I think in 2026 we're going to see AI systems that are 10× or 100× as capable as humans, or 2×, or anything between that. Once you get to maybe 2/3 of all cognitive abilities—maybe 75%, maybe 80%—where, demonstrably, machines are just categorically superior by orders of magnitude, that is superintelligence.
So, superintelligence is 2 things: it's a vibe and a heuristic, based on what I just described, and it's also an effect. If you have a smart enough AI and it's crapping out novel hypotheses once an hour, and it takes scientists weeks to grind through them, and it starts getting a 60%, 80%, 90% hit rate, I'm like, “It understands the cell, and it's giving us novel disease cures every week.” That is superintelligence.
Superintelligence has to be measured by 2 things. One is under-the-hood cognitive-science abilities, but the other is real-world effect. Here's another really big thing, super quick: you don't really have superintelligence, with truthful evidence to convince most skeptics, until you have the fruits of its labor.
It's like someone tells you they're really rich. You're like, “Hey, can you buy me a flight to Dubai?” They're like, “Money's tight right now.” You're like, “I believe you, but I don't.”
But if you know Elon Musk, you're like, “Hey, can you buy me a flight to Dubai?” He's like, “Yes, I just bought you Dubai Airlines.” You're like, “The airline?” He's like, “Yeah, I just acquired it.” You're like, “Right.” Okay.
Mike Israetel
Wow. That's a demonstration. I think that in 2026, I'm very confident—though never certain, from a scientific perspective—on the CIA probability scale: extremely likely, 97% to 100%, that sometime in late 2026 we're going to start opening the cornucopia of machine intelligence, so that people understand that at least parts of machine intelligence are absolutely superintelligent.
Because of exponentials, by 2027, 2028, 2029, the [__] is going to get completely insane.
There's so much to unpack there, and I really like you, Mike, so I don't want to disagree with you on my podcast, but I have—
I Desperately Want To Live In The Matrix
No, please. I love disagreement.
I have to, and I think I disagree with almost everything you've just said, so we can take it one at a time. First of all, the definition of intelligence: you were talking about knowledge as well. I think that knowledge is non-fungible. In fact, one of the most pervasive critiques of artificial intelligence, going all the way back to the '70s—there was Dreyfus, and there was John C. It's this grounding problem.
Stevan Harnad spoke about this grounding problem, and essentially, the enterprise of intelligence is the accumulation of information that is relevant for adaptivity around an environment. The product of our knowledge is all of this: Wikipedia, the hard drive in the sky, our culture—it's all of the things that we have acquired. But the problem is that there is a gap between syntax and semantics.
So if an alien read Wikipedia, they might read the article about Trump or about America or something like that, and that is not the same as the embodied experience of being there. These are just pointers. Semantics is about the connected, enactive, embodied graph of actually experiencing things through time.
The biggest misconception in all of AI, what all of the folks in San Francisco believe in, is this philosophical idea called functionalism. The best way to describe it is through an analogy of walking up a mountain. I spoke to a philosopher the other day, a wonderful lady called Anna Ciaunica, and she said that we're walking up the mountain, and when we get to the top of the mountain, we have all of these abstract capabilities, like being able to reason and play chess, but that disregards the fact that the path that you took walking up the mountain is very important.
And not only the path, but the physical instantiation—the stuff that the mountain is made out of—because it's a reasonable argument, in my opinion, that intelligence is a property of adaptive matter. It's an emergent property, much like temperature. Temperature is a coarse-graining. It's an effective theory to describe the details of the molecules moving around, and we screen that detail off and call it temperature.
I think intelligence is like that, and I think that knowledge is quite similar. I think knowledge is actually a physical causal graph that is enacted over time, and it cannot be abstracted in the way that you're describing. The reason the abstractions work is because they are pointers to our embodied experience. They make sense to us, but on their own they don't make any sense.
I like that take. I'll push back. My favorite definition of intelligence is the most basic definition of intelligence: the ability to solve problems.
I Desperately Want To Live In The Matrix
Yeah, I hate that. You find out if you're truly intelligent if you can solve problems of any degree of complexity. I would actually say that responding to 1 stimulus in a cogent manner—at least knowing how to respond to 1 stimulus, input-output—is the beginning of intelligence. All intelligence above that is just stacked layers of complexity.
When you say embodied, there's a lot there, for sure. But the only reason that we call humans intelligent is because we have a representational, abstracted neural network called the brain. Your brain doesn't actually have anything in it that's magical. It's just network pings off one another recursively. Your brain is, in many deeply important respects, exactly as abstracted and unrelated to reality as a data center.
You can climb mountains and touch stuff, but you never truly have embodied experience of anything if you push on that philosophical button hard enough, because you can always abstract out to: These are just neural-network pings from groups of neurons. You don't truly deeply know anything in some weird philosophical way, because it's just neural-network calculus all the way down.
Whether it's you or a machine brain, like what they have in Optimus, for example, it's representational. Intelligence is always seemingly going to be something that is lossy to some extent. It is a compression function. Is it? It's a real rude way to say it, because it's a very impressive one.
But you climb the mountain. That's cool. A helicopter can climb a mountain much better than you. It does not have the ability to reason abstractly, plan, and predict things at all. You can get really amazing high-definition video and samples of rocks and the mountain and everything, but if you don't have an analyzer, a system processor, an integrator, and a recursive function, you don't actually model that data in any meaningful way, and it just sits there as bits on a computer.
When you say that embodiment is a prerequisite of intelligence, I would push back and ask you to answer this: If we have somebody who has read every single physics book, especially, let's say, particle physics, and they're so good at it that they can tell the computer system that aligns the particle beams at CERN to do its job well enough to produce real-world data, are they truly intelligent about particle physics?
By your definition—and I'm being a little bit facetious for comedic effect—the answer is no. Because if I put you in the particle accelerator, you get torn to [__] shreds, and you're like, "Oh, [__]. JK." It's actually impossible for humans to perceive particles directly with any sensory organ, because they're Planck-length-type [__]. You can't see it. Vision is not a cogent concept at those lengths. So that goes straight to hell.
If that scientist is so adept that he can actually run CERN, but it's all neural-network modeling in his head, he's never actually had any real experience with a particle collider. No human has. You can't open the latch when the thing is on. It'll shoot [__] neutrinos at you.
So if that's true and we say, "Okay, okay, okay, well, he actually knows things. He actually has intelligence. Why doesn't GPT-5 and its network clusters actually have an understanding of the real world? It has never seen it, but the guy's never seen a particle in his life." Zero scientists have ever seen a neutrino. They have no perceptual experience of a neutrino. It's purely hypothetical.
But they make real-world predictions because the neural network and all the attendant structures that create what we call intelligence and understanding are there. Their vector arrangements represent real things that are happening in the real world decently well enough—always an approximation, but a damn good one, a predictively valid one.
If GPT-5 understands how humans act to 98%, and if it were embodied in a robot body and had vision, it would be like, "Oh, [__]. There's the rest of that 2%." I think 98% of the way to intelligence is the same thing as when I was a professor: If one of your students gets a 98% on an exam, is he pretty good at understanding the material? Yeah, [__], yeah.
Now, just off camera over there is my best student of all time, Jared Feather. He never missed a single point on any of my exams. 100%. Well, that just means the exams weren't hard enough to really push his understanding. Is Jared the best student I've ever had? The best trainer? Yes, for sure.
But does that mean that when kids score 95% or 98% on my exam, I'm like, "Never let this kid train you. He doesn't really know the real world. It's just abstraction"? No, [__], no. They know real things in the real world and can really help you get fit, even though they might themselves have never been able to get very fit, because they know the actual structure out there.
It's a purely representational neural network, but it passes the test: When they see the real thing, they go, "I know what a dumbbell is. I know what a human is. I know what vectors are. I know what forces are, and I'm going to have them do this," and it works.
Yeah, a human would have a shared understanding because, if you think about it, the representations in our brain are the product of evolution. We've been evolving for billions of years, and I know you read that Ray Kurzweil book. It's a beautiful recounting of how we had these exponential increases.
We had genetic evolution, which is very slow, and then we had this ontogenetic and phylogenetic hacking, where we developed nervous systems and brains, and we developed culture, and we started evolving at light speed, and so on and so forth. At the end of the day, the product of that knowledge acquisition, that process of intelligence, is all of these representations. Everything in language is a representation.
For example, algospeak is a term to represent the evolution—the linguistic evolution—of our language. There's this term called "unalive," so to get around social media filters, we now say, "I'm unaliving someone." That is an example of linguistic evolution, and ChatGPT is not capable of doing that, because there's a big difference between the process.
I Desperately Want To Live In The Matrix
Absolutely not.
You wouldn't understand what "unaliving" is.
I Desperately Want To Live In The Matrix
We'll get into this, right? But understanding is not about being at the top of the mountain. Understanding is about the path to the mountain.
I Desperately Want To Live In The Matrix
So, we understand things because we are physically embodied, right? You were talking about how we can’t observe the particles. It doesn’t matter. We are in the causal graph of the particles. They affect us even if we’re not—
They don’t. They’re inside the collider. You don’t get radiation.
I Desperately Want To Live In The Matrix
You’re made out of particles. That is—
Yeah. You’re not made out of neutrinos. They’re passing through you. The neutrinos are completely abstracted as far as we’re concerned. They actually pass through the Earth the entire time. So, there are all kinds of theoretical physics that have nothing to do with you in a causal graph. They haven’t meaningfully affected your behavior or any of your decisions.
We’d have to abstract 18 layers and do quantum mechanics that none of us are equipped for. I don’t know—you’re probably smart enough to get it; I’m not. That stuff is really [expletive] in the real world that we’re purely abstracting to. But because knowledge is actually representing the real world, it works.
I’m not so sure I understand what exactly you’re getting at when you say that we get something out of embodied experience. How can libraries, education curricula, schools, and reading a bunch of books possibly give you knowledge if none of that is any kind of embodied experience? How does ChatGPT know the vibe of Shinjuku in Tokyo and accurately render for me what the vibe feels like if it’s never been to Tokyo? The same way a travel agent does. But it’s real, actual things they know, and when you go there, you’re like, “Oh, shit. It was pretty much correct.” How does that work?
Well, first of all, we should distinguish knowledge and understanding. So many people have different definitions of this. In the cognitive sciences, people talk about factual knowledge, which is just states of affairs. There’s procedural knowledge—knowing what to do—and conceptual knowledge—knowing how to think.
Roughly speaking, I make the distinction that knowing is knowing states of affairs, and understanding is this ability to generate new knowledge. If you have an understanding of the world, which means you know it at an abstract level, you can do this LEGO-building in your mind and create new knowledge.
But the simple grounding problem is something a little bit different. Even with the neutrinos, we use this thing David Krakauer calls the principle of materiality, which is that we use the world to think. Many of the abstractions that we’ve converged on over millions of years of evolution are there because we have this shared physical world.
You and I both understand similar abstractions, which are derived from our sensorimotor circuits, from language, and so on. It means that we can talk at a very abstract level, and because we’re both players in the same game, we understand each other. The parlor trick—and we won’t go over Searle’s Chinese Room argument; I’m sure you’re familiar with it—is basically the same thing. A computer can tell you all of these things as if it understood, but you understand because you actually have the feeling. You actually know.
Even though you’ve never been to Japan, you’ve experienced many things that are like what people who go to Japan experience, so you have some kind of shared understanding. There is an intersection in your understanding tree that something like ChatGPT doesn’t have.
Now, I want to make a distinction as well. I’m really impressed with language models. Claude Opus 4.5 came out yesterday.
GPT is not a language model. It’s been multimodal for 2 years now.
Joscha Bach
It’s still known as a language model. It’s a self-attention—
Right, but we have to concede the fact that it’s an omni-model. It’s been multimodal for a long time.
Mike Israetel
Yeah, it’s multimodal, but it’s still a self-attention transformer. It has a couple of traits.
The transformer part is true, but just calling it a large language model, I think, cuts short a huge fraction of its capability. It does visual reasoning.
Joscha Bach
It doesn’t make any difference.
Okay.
Yeah. So, it’s a self-attention transformer. On the most recent show we just published, I interviewed the guy who invented transformers, actually. We just tokenize different domains, learn the statistical distribution, and do this next-token prediction within a fixed context window.
So, it’s distributional matching, but then there’s some other stuff on top. There’s RLHF, and there’s also reinforcement learning with verifiable rewards, or RLVR, for verifiable domains. This is why they’re doing really well on reasoning tasks when you have a verifier.
The ARC Challenge is a great example of this. I’m a big fan of that. François Chollet is a friend of mine, and he designed this challenge to show abstract generalization. You only have a few examples to learn from, and he predicted that LLMs would be terrible at this. Unfortunately for him, the LLMs are now getting incredibly good at it.
That’s because, in my opinion, LLMs don’t really understand in this grounded way that we’re talking about. But in abstract domains, you can fully describe an abstract domain, like a 2D-grid problem, and you don’t need to have any grounded knowledge. You can completely understand it.
What do you mean by grounded knowledge? What is—
Mike Israetel
This is grounded knowledge, right? You know—
You’re touching a chair.
Mike Israetel
Touching a chair.
So, that’s not knowledge. That’s neuronal perception arcing up into your brain and back down. What are you learning about the chair when you’re touching it like that?
This is the other issue that I have with your perspective. I would call it a corticocentric view of cognition.
Joscha Bach
Absolutely.
Yeah, exactly. So, this is another one of those leaky abstractions I was talking about. By the way, there’s a great book called “The Brain Abstracted” by Marta Chirimuuta. I interviewed her recently, and she said that one of the most pervasive myths in neuroscience is that we use these leaky abstractions and idealizations to talk about cognition, usually using the most recent technology at the time.
A few hundred years ago, we were describing the brain in terms of pulleys and—
Joscha Bach
Pulleys and levers. Yes.
That’s right. Then it was as a prediction machine, as a computer, and all this kind of stuff. At the end of the day, this is an example of grounded things that we understand. They’re really good models because we can both talk about computers. We both know what computers are.
But the brain doesn’t work like that in any sense. A great example of this is knowledge.
Joscha Bach
You, as a personal trainer, know this, right? So, you train—I’m sure in the past you’ve trained folks in the gym—and what you’ve probably realized is that, in your mind, you’ve got a distilled, abstract, beautiful understanding of things. You’ve thought about it, you’ve distilled it, and you could write it down in a book. You probably think, “All I need to do is just tell people. I’ve done all of the thinking. I’ve figured all of this out. I just tell them.”
What you learn, to your chagrin, is that there is no substitute for experience, because part of knowing is actually doing. It’s the pure experience of lifting the weights, feeling the sensations in the body, and going through that process. That’s why knowledge is nonfungible.
I’ve been increasingly impressed with modern AI’s ability to render pretty good recommendations on exercise and sports science. I’ll push back on one of those things. I know a lot of people—they’re great people—who have a lot of what you would call embodied knowledge about training. Because they haven’t been interested in, or capable of, doing the abstraction to distill principles, they’re not really that great at training other people.
They know what works for them, and they scarcely know any of that beyond their own experience. They’re just not good at generalizing because they’ve never abstracted out the concepts. I’ve also had many discussions with some of my very, very smart friends.
One of my friends graduated from Harvard with a master’s in data science, worked for Apple and a bunch of other AI companies, and stuff. His ability to pick up how to do an exercise incredibly well just from principle-based discussions, and then trying it twice, is remarkable. He’s instantly one of the best trainers I know in the area. One shot.
Why? Because Jared and I can distill 5 principles of how to do an exercise well for you. They’re geometrically not complex and heuristically very cogent, and you will instantly be good at teaching people how to exercise without ever doing the exercise.
Is some nuance missing? Absolutely. There’s some stuff that you actually have to do to understand, like, “Oh, I see—moving the hips back like this.” But if someone was able to describe to a computer, in vectors, how the hips should move back in a squat versus just moving down, especially if—
So, I’ll tell you this. If I were able to train a model—and I’m not currently doing this, and I have no plans to do so—with 1,000 samples of a squat done right, 1,000 samples of a squat done wrong, and about 10 rules for how to squat, it would instantly be in the 99th percentile for teaching people how to squat, with zero grounded experience.
I would also say that what we consider grounded experience is mostly a visual data stream. Yeah, touching stuff is cool, but I’m not letting a computer have sex with me until it’s had sex with test animals in the lab.
Joscha Bach
That came out all wrong. But a lot of our data is visual—a huge, huge fraction of it. I suspect that if you let a neural network train on all of YouTube—which, as I understand it, isn’t happening because there’s copyright shit, which drives me insane—you could just have an algo eat all of YouTube, which is a preposterous amount of data.
It would know more about how the world looks in real life than any extant human, period, because there’s more visual data on YouTube than I have collected with my own eyes by, I don’t know, 10 12 50 orders of magnitude, something like that. And so, there’s zero embodiment. We did the embodying for it by putting camcorders into people’s faces while they throw up from drinking challenges, then downhill skiing, then taking pictures of microbes.
If you really train a visual model and, even better, a 3D relational model on YouTube alone, you will get a more grounded understanding than your eyeballs—which, by the way, are also just cameras—can ever give your brain, which is also a computer. The brain is a computer. Is it a different type of computer than a CPU? Yes. Is it a different type than a GPU? Yes, but closer.
Are we going to replicate exactly how the brain works computationally? Yeah, I’m sure at some point. I suspect we’ll need to do that because there’s no good reason to believe that brains are super, super good at thinking. They’re just evolution’s best crack at it. A tank isn’t a walker. Are you a Star Wars fan?
No.
I Desperately Want To Live In The Matrix
Do you know Star Wars? Do you know what an AT-AT or AT-ST is?
Okay.
I Desperately Want To Live In The Matrix
You know those dog-looking, big walker things that shoot lasers out of their faces? What’s better, that or a modern battle tank? Well, a modern battle tank by a long shot. Why? A battle tank sits like 8 feet off the ground. It can hide in trees, and it can shoot like 2 miles out. An AT-AT walker stands above the tree line, and you just shoot one leg and it falls. Why the fuck would you do that?
Well, it’s a pretty decent attempt to replicate animal locomotion. But the assumption that animal locomotion is really the best way to do it is not. In the same way that a rocket or a supersonic aircraft is categorically better at flying than a bird in almost every respect, AI and machine intelligence is going to surpass human intelligence and real, deep understanding of the real world just by bypassing our architecture.
Later, it will be so smart that it can come around, scan the human brain, and be like, “Oh, that’s how they were doing it. Oh, that’s kind of cool.” I don’t think there’s a huge ROI in what it’s going to get out of that because it’s going to be way ahead.
Suffice it to say, I think that what we assume we’re getting from sensory experience is just a few gigs or some number of bytes of data. I’ve seen a lot of shit in my time, but if it’s also wildly abstracted, like if you ever think about memories you’ve had from childhood and try to really parse, “How did this dresser look? What was my mom wearing?” you realize two things.
One, your precious little actual visual data is like a 10-second-long video, maybe at any kind of fidelity, and the fidelity sucks. And two, you’re actually hallucinating a lot of that. You’re just filling in details. If you let an AI train on and eat all of YouTube at 4K and really develop the model from that, it would see and understand the visual three-dimensional space around it and the world at a level of embodiment that shits on all of us instantly. That is my conjecture.
I understand your perspective. First of all, you were talking about doing personal training. There are folks like yourself, and I count myself like this as well. Even when I’m doing editing and post-production, I really like to build theories about things. I think very deeply about things. I come up with abstractions, and I think they’re very fruitful for me and they’re not for other people.
This is the beauty of our collective intelligence, because some people just feel in the moment. For example, when I see my personal trainer, she’s called Lara. Shout-out—she’s probably not watching this, but shout-out anyway.
I Desperately Want To Live In The Matrix
I swear to God, I’ve been training and eating well. You just haven’t heard from me in a little bit. I’ve been really busy.
Not been eating, honestly. We’ll talk about that later.
I Desperately Want To Live In The Matrix
I was trying to help. We’ll talk about that later.
But, for example, I’ll train with her, and ironically, what I get out of that has got almost nothing to do with personal training. She’s an incredible woman, a really, really amazing woman. Sometimes she taps me on the shoulder. We laugh and joke together, and it just enriches my life in so many ways that are not captured by these abstractions.
All of us have different abstractions. But then you’re also talking about this kind of monolithic view of AI that understands everything. That’s not the way that our evolution works. Evolution is all about this path dependence I was talking about. We have many agents, and we all take our own paths, which means when we do creativity, we continue the epistemic lineage that we are on. This is an unfurling process.
What is not the case is that there’s this kind of gray goo where all of the knowledge is convolved together and creative steps could happen from any part. I think any future superintelligence that we do create will resemble our intelligence, so it’ll be very diverse and collective.
I Desperately Want To Live In The Matrix
I think at first that’s the case. Then, when superintelligence is smart and capable enough, through a few different ways, to rearchitect itself, it’s going to leave behind our abilities quite quickly. It’s also going to leave behind most of the very narrow conceptions that we have about intelligence.
I think Andrej Karpathy, if I’m saying that correctly—you read enough things on Twitter and you’re just, “I don’t know how to say that person’s real name”—said recently something that attends to your view a little bit. He’s like—you know, we humans. I’m very much paraphrasing Andrej, and I’m sorry if you ever see this. It was an X post, as far as those go.
Humans sometimes think that we are somehow the seat of intelligence and that our cognitive domain is what intelligence really is. But we’re learning with AI, according to him, that we have a niche in intelligence that’s, to your point, path-dependent and evolutionarily derived. I was going to say highly functional, but functional enough to get us here and build all this cool stuff and build cameras and stuff.
It only occupies a very small fraction of the entire universe of possibilities—the whole vector space of intelligence. Most of that vector space we actually don’t know. One of my things—and I’m not saying you’re saying this at all—is that it’s always curious to me when people say, “I don’t think…” This used to be a majority view, even in AI research 20 years ago: machines will never be more intelligent than humans.
The gall to think that fucking primates who mostly scratch their testicles and sometimes do theoretical physics have some kind of global optimum and exclusivity over all of intelligence is fucking preposterous. When artificial superintelligence with agency truly kicks off, I think it’ll be a miracle when it does. It’s going to recursively rearchitect itself and start expanding into all of the domains of intelligence we don’t know anything about. Then we will see really what intelligence is.
Just to use that really quick by analogy: if wolves could talk and even abstract—which they can’t—but let’s give them 2 magical abilities, they would pick the smartest wolf in their wolf pack and be like, “You know Bob the wolf? He’s the shit, man. He—he fucking knows, right? He knows when the bears are around. He knows when they’re not.”
Then you’d be like, “Okay, well, here’s, I don’t know, Stephen Hawking.” He’d be like, “So he’s your Bob?” Like, “Yeah, in a sense.” “Okay, so he really kind of knows where the bears are?” You’d be like, “He does a lot cooler shit than that.” Then you try to explain to the wolves what that is, and they have no idea how to contextualize it whatsoever.
I think that’s the flavor—to use a very technical term. Though I did just learn that in AI training, taste is a technical term, and I was like, “What the fuck?” It was explained to me, and I was like, “I’m not smart enough to understand that by a long shot.” But the flavor of artificial superintelligence is verticality of ability: doing math at this level, this level, that level, and then expansion across domains—to domains we understand and orders of magnitude more domains we haven’t even conceived of.
Yes. Maybe we’ll come back to Andrej Karpathy. He tweeted the other day about AI slop.
I Desperately Want To Live In The Matrix
I saw that tweet.
I gave him a definition in the response. He was asking for a definition: What is a good definition of AI slop?
I Desperately Want To Live In The Matrix
Yeah. Do you want a good one? Can I read my definition off X? Can I pull out my phone? I’ll never find it. I take that back. No, I have dog-shit memory and shitty intelligence on top of that.
My definition was something like—I’m going to do a shitty job at it—AI slop is when you generate content and the ratio, with its ability to be generated by AI as the denominator and its coherence, utility, and novelty as the numerator, is really low.
You’ve got very good intuitions, actually.
I Desperately Want To Live In The Matrix
Yeah. So the Will Smith eating pasta slop is observer-relative, in the sense that the more sophisticated you are as an observer, the more likely you are to detect slop. Anyone looking at that first Will Smith one would be able to tell that it’s incoherent. Basically, my definition of slop is that it is what happens when a process creates an artifact without understanding.
A sophisticated observer will spot the glitches. That’s why it’s possible to post slop on LinkedIn and Instagram: most people are not domain experts. People post on LinkedIn. I’m not even so sure LinkedIn’s a real website anymore. I’m kidding.
LinkedIn is 99% slop, but we should delete LinkedIn. But let’s not go there.
I Desperately Want To Live In The Matrix
To a naive observer, it’s very difficult to observe slop. To an expert, it’s very easy. There’s also the interesting case that you might see slop for the first time and not realize it, because another way of describing slop is a shallow mimicking of something else.
For example, you get human slop. I could copy one of your meme posts, and someone else could see it without realizing that I’d copied it from you. The first time, it looks good, but I can’t make any variations on it because I didn’t understand it. When I break the rules, people will know it’s slop if they understand the domain. You see where I’m going with this?
It’s interesting that you intuitively understand this, but you reject the idea that AI doesn’t understand. The reason it produces slop is that AI only understands 3 levels deep, which means it can’t make any creative variations. It’s wonderful that you raised this point about developing and producing 3D models with AI.
Graphic artists do use these new models for generating point clouds and making these models. What they do is take them into Blender or some other 3D program, and it’s a fucking mess, right? You just get all of these vertices everywhere, and they’re all over the place. Clearly, the process that produced them did not understand anything about the model that it generated.
What the artist has to do—and this is a great analogy for all of AI, all of creative AI—is recreate it. They use it as inspiration, and then they create a new one where they carefully place the vertices and structure it nicely. Then they publish that. It doesn’t matter whether you’re doing image generation or writing. That’s always the process you have to follow, because the AI models don’t understand anything. They just mimic something, which is slop.
When humans are asked to generate real-quality content after, let’s say, a substantial education, understanding, to me, is not a 0/1 function; it’s a spectrum. Depth and scope of understanding are real things, and AI, in some cases, has very low depth and low scope of understanding. We can take it beyond that scope, ask more of it than it can do, and then label it as slop because it understands something insufficiently well.
What’s required for understanding is a sufficiently detailed world model, an ability to have short- and long-term memory in your operations and manipulations of that world model, and a system of logical operators to parse that world model. Dive in, dive out, move laterally, move up and down the hierarchies by analogy, side to side, recursively. That’s probably most of what you need.
You need a few other technical things to do real understanding. When you have that to some extent, you have some extent of understanding: the richness of your world model, the ability to logically operate on it, the ability to do manipulations, and a big enough context window to do 3D rotation, et cetera. That is what’s required for understanding.
When you have an AI system like GPT-3.5 that doesn’t have much of that, then you can accurately say that it often does not understand things. Not always, though, because it can write some pretty decent poems. Sure. Shit seems to understand more about poetry than I do. I’ll tell you that.
As these systems get smarter, the moat we have as humans with true understanding gets real goddamn small. When I talked to GPT-5—which I was more hopeful would be more capable because I had the GPT-4.5 research preview—my understanding of Sam Altman’s very cryptic insight is that those abilities are going to be coming back in GPT-6.
Those abilities—massive recursion abilities and an insanely deep and rich world model—are easily inferable because they used an order of magnitude more training data for GPT-4.5/6 than they did for GPT-5. GPT-5 is basically built on o1 architecture. They discovered the reasoning paradigm, which GPT-4.5 didn’t have, and they were like, “Fuck, we’ve got to exploit the fuck out of this.” They managed to do it incredibly efficiently, which is why GPT-5 is super token-efficient. That’s really impressive for real-world tasks, but it absolutely, to your point, dings its deep understanding because it was not optimized for deep understanding. It was optimized for realistic human workflows.
The news for you is that most people are not interested in prompting the AI for deep conceptual world understanding. They’re looking for accuracy at the time of the question-and-answer.
As these AI models get into GPT-6, GPT-7, and GPT-8, and as they get upgrades in all those qualities I just mentioned—the richness of the world model, the ability to parse the world model, the ability to remember what you parsed, and so on and so forth—as those abilities are updated, including live learning without catastrophic forgetting when you retrain the network, Google just cracked that problem. They just published the paper, and if I know anything about how corporations work, that means that paper was on their desk for two years and they kept it internal to train their own models because they don't want to [__] out a huge advantage to everybody else.
I don’t know why anyone ever thinks that when corporations publish truly novel insights, it’s a novel insight to them. You’re not going to give OpenAI researchers a crack at your best shit the day you find it out. That would be fucking insane. Maybe that’s not how it works. Maybe the ethics are different there.
That paper was just published by a couple of interns.
George Hotz
Sure, I take that back. You’re completely correct. They didn’t crack it, but the path to it is more well understood than it was before. There’s no reason to think we can’t crack that. By the way, what’s happening there is some kind of magic. Updating a neural network live while parsing it is a potentially solvable problem. I’ll tell you exactly why humans do it.
Remember, humans are primates with dogshit wetware, and we can get machines to do it. So far, machines are undefeated in their ability to get ahead of humans in every domain we’ve ever tried to push for long enough. Once you get live-updated learning, what you’re going to get—and even before then—is incrementally more true, deep understanding.
Already, for concepts like history and reasoning by analogy, GPT-5 understands more deeply than 99% of all humans I’ve conversed with. I’ve conversed with lots of humans, and the degree to which understanding at a real depth is shown is not in evidence. As a matter of fact, the evidence is against it.
Because Jared is here, I’ll give him this one. When you’re talking to some people that you think know things—this is facetious, and it’s going to get clipped out of context—I’m being a dick for comedy, but there’s some tiny grain of truth here. When you talk to people who think they really understand a relational map of a concept, because they’re embodied and because they’ve been in the space, and then you start pressing on the fine points—“What about this? What about this? What about that?”—you get your analogy of the Blender graph, where it’s like, “Oh, you’re vibing all of this, and you know about 3 things about it. The rest you’re vibing live.” Humans do that.
What I would say, to push back, is that the assumption that humans have this deep understanding is itself often not evidence, and often the evidence is against it. When AI vibes its way to understanding, I would say that humans do that too.
For example, hallucination: people are real pissed that AI hallucinates. Have you ever spoken to a human being? Half of the analogies I used on this very podcast just now, talking to you, are fucking hallucinations. They’re close enough to the truth to be cogent and have some amount of information transfer, but they’re not perfect. I’m not deeply in touch with some real causal map of the world. It’s all simulation all the way down.
I’m highly skeptical of the idea that we need grounded understanding, or whatever that is. I think there’s something there, but the more capable AI gets at doing what humans are able to do—really live-update their knowledge—the less of a moat we have. Then, at some point, like now, it fucking understands, bro.
So, a few things you said there. First of all, GPT-4.5 was a vanilla dense model. The reason why GPT-5 is faster is that it’s a mixture of experts. It’s roughly the same-size model, but now you only have a fraction—let’s say 10%—of the parameters in play during inference. Also, it wasn’t a reasoning model, so it was just a vanilla model.
That’s why it was so slow. But I don’t want to come across like I’m really skeptical. We are a massive AI podcast, right? I love technology, and I’m really interested in AI.
Part of the reason for this skepticism is that I’m very worried about doomerism, and we’ll come back to that in a minute. But I think there are principled reasons why we don’t need to worry about superintelligence and recursive self-improvement. I do think that intelligence is something we can create in a computer.
We need to distinguish intelligence as a property of adaptive matter, which is something that happens as part of the process of evolution in the real world, from the types of algorithms in computers that we might say are intelligent. AI is operating in the real world, and I can prove it with one step: unplug the reactor from the data center. No more AI.
AI is absolutely in the real world. Its data stream was fed to it by layers of abstraction, much the same way that your data stream arrives to you as representations eight neural networks deep, from your optic nerve going all the way back. By the time it hits the back of your occipital lobes, you’re not seeing the real world. None of us see the world; it’s abstraction all the way down. You can’t process the photon density at your eyes. Full stop.
So arguably, AI actually gets much more coherent data than we do. When it’s thinking in its data center, it not only is truly embodied in the same sense that your brain is, it just doesn’t have physical arms. Giving a data center physical arms is a nominal problem, right? As a matter of fact, it does have one, because I have a Tesla, of which I’m very proud.
Someone’s like, “Oh, hello. Can I talk to you?” I’m like, “Oh my God, no. The unwashed masses. I have a Tesla.” Tesla—like Elon says it. It’s an S, but he says it like a Z. My Tesla drives itself. It fucking drives itself. It’s a chip inside the car. How the fuck does it know what’s around? Cameras. It’s already an intelligent organ, and a suite of organs—sensory and intelligent. But here’s the thing: it drives better than I do.
George Hotz
Oh, yeah. But look, we should distinguish. There’s this famous effect called the McCorduck effect, after Pamela McCorduck. Basically, when technology does something that we thought was impossible, there’s a chorus of people who say, “Oh, that’s not really intelligent. It just becomes part of the standard thing.” Maybe it just wasn’t that hard of a thing to do in the first place.
But I think in the case of machine learning, we really need to give credit where it’s due here. Nobody knew that these superficial statistical regularities had an insane amount of generalization. It’s possible to learn the statistical distribution of driving-type data, and your Tesla can drive, arguably, in some ways—assuming there are no robustness issues—better than you do.
But it’s still dead. It’s not alive. It’s not intelligent. And back to your previous point, it’s not alive.
Why would you say it’s not intelligent? It’s solving real-world problems at speed.
I Desperately Want To Live In The Matrix
Intelligence is about adaptivity, right? The neural network in that Tesla model is frozen. A bunch of frozen weights are not intelligent by definition.
I want to be a little bit open-minded here. We were just filming at the Diverse Intelligences Summer Institute, and I’m a big believer that there is a huge, diverse space of possible intelligences. The folks at the Santa Fe Institute think about this as intelligence being represented mostly by adaptivity, but also by representation and inference.
I’m amenable to the idea that bacteria are intelligent and viruses are intelligent. I’m also an externalist. I like looking at language and culture; you can actually think of it as being an organism. It’s an adaptive organism. Viruses have the ability to delete strategies very quickly.
Knowledge decays very quickly, and that’s actually a good thing because it’s the way we can adapt our strategies. Things that don’t work die off, so we’re very adaptable. Then the question is, I don’t believe, for the functionalist reason I gave you before—I’m basically an antifunctionalist—that you could upload your mind into a computer, even if you did a particle-by-particle simulation. I don’t think that would be intelligent.
But that does not imply that I don’t think we could build an artificial-life simulation that was intelligent. I interviewed this guy called Blaise Agüera y Arcas from Google a few weeks ago, and he basically built this adaptive Turing machine where you have all of these different tapes, and they could merge parts of the Turing-machine tape with each other.
He noticed that after several generations—100,000 generations—they started preserving themselves. You got this self-preservation behavior. The reason why we have self-preservation in living systems in the real world is because the ones that stuck around were the ones that had this drive to preserve themselves, and that led to life, agency, and intelligence.
There’s this kind of evolutionary track, and intelligence is about the accumulation of coarse-grained knowledge to adapt to the world. That’s real-world intelligence. Could we build something like that in a computer? Yes, I think we can. But the important thing is that this is a process of evolution. We’re only going to do this through neuroevolution methods.
What we’re not going to do is say that this LLM understands stuff better than anyone else. That’s fucking right? It’s like a database. If you put a model into a database, right, so you sort of defined all of the tables and gave them names that represented the domain, would you say that the database understood the domain? Of course you wouldn’t.
Would I say that the database understood the domain? I think what you and I are going to crack into—which we probably couldn’t discuss on a podcast; we’d have to sit down and draw things out together—is a mutually agreed-upon, complete definition of what understanding means. I’m going to say understanding is a spectrum. You’re going to say it has a few critical things that must be there. I probably am inclined to agree with you, but I don’t know how much traction we’re going to get on that until we have a real vibe session with each other.
What I will say to your point is that updated live learning is an unbelievably useful feature. To get human-level intelligence, it has to happen; it’s a nonstarter otherwise. But you can be artificially intelligent without exhibiting all human traits and qualities.
We already have artificial superintelligence in many regards. For example, a computer that can just answer Jeopardy! questions is a domain-specific artificial superintelligence. If you look at it like that, the Tesla self-driving software actually does update live within a tiny context window: that truck is right there, and now it’s here, and now it’s here. I need to go around that truck because it has a world model that knows roughly what a truck is and knows that somebody can come out that side, so I need to go around it.
After it leaves the truck behind, it doesn’t know shit. For example, my Tesla will have it on Hurry mode a lot, because then you can just dial up how fast it’s going. It’s always trying to get into some fucking lane to get you another 2 seconds, which is the mode I turned on, so I’m not upset at it.
A lot of times, it doesn’t even look super far ahead, even though I know the camera system can do that. It’s also a nonreasoning model, so it’s going to go into this lane, and I’m like, “That fucking lane ends in a mile. It should know that.” But it doesn’t know that. Then it just rediscovers on every drive that I have that shit is coming up.
Visual reasoning models are already a thing. They’re starting to be a thing. When we get those visual reasoning models with big context windows and somewhat lossy but decent long-term memory, now we’re getting real fucking close to what you would call understanding. Then we’re one algorithm away—updated live learning—from what I think you would catalog, outside of the embodiment problem, as being real close to understanding.
The one thing you said that really strikes me is, if you represent every brain function point by point—I would say, at the level of neurons—do neurons reason at a subcellular level? Unclear.
I Desperately Want To Live In The Matrix
Particle.
Yeah, particle by particle. But it would have to be particles and also the way they interact, right? If you had that in the cloud, a full 3D scan, and then ran the system, I would say you have 100% of the intelligence in the cloud.
You can prove this by simply beaming that data into a robot. You wake up in a robot, and you’re like, “Oh, hell yeah. Let’s masturbate.” That would be my first thing to do as a robot.
I Desperately Want To Live In The Matrix
But I think that’s just completely untrue. I don’t want to go too much into this, but a simulation of water doesn’t get you wet. A simulation of fire doesn’t get hot.
A simulation of water would absolutely get you wet.
I Desperately Want To Live In The Matrix
No, it absolutely wouldn’t.
But everything about wetness is included in the simulation. Otherwise, it’s just not a sufficiently deep simulation.
I Desperately Want To Live In The Matrix
So if you had a simulation of fire in a computer, or of a stomach, it wouldn’t digest. Fire wouldn’t get hot.
Water wouldn't get wet.
I Desperately Want To Live In The Matrix
It would digest.
We can't even be debating this.
I Desperately Want To Live In The Matrix
I can and I will watch this. If you have a simulation of a stomach, particle by particle, and you get a simulation of food particles through the stomach, it will digest said simulation. That's real digestion: particle by particle, at high fidelity.
Okay. We were talking about temperature earlier, which is an intensive property of aggregated matter. It's a thing: when certain particles in the real world—I’m talking about the stuff that it's made of—interact with each other in a certain way, you get this emergent property, this kind of coarse-graining, and we call it temperature. Intelligence is like that. Consciousness is like that. This is what John S was arguing in his Chinese room argument.
He basically said that the brain is a certain type of physical instantiation that gives rise to these properties. So, if you look at its causal graph, the causal graph has these strongly emergent, coarse-grained properties that we call consciousness. Wetness is like that. Heat is like that. You can't seriously tell me that if you simulate fire using a computer program, the computer would get hot or that you would get hot. It just doesn't work like that.
I Desperately Want To Live In The Matrix
The computer wouldn't get hot because the computer's out of the simulation.
That's what I'm saying.
I Desperately Want To Live In The Matrix
Right. But if you burn to death in your own house and I live a house away, I don't get hot either. I'm right next door, but I'm sufficiently far enough from where the heat is happening that it doesn't affect me at all, even though it really did happen to you. Essentially, a simulation is another pocket universe, right? Things can happen in it that, in it, absolutely mean the things in it are getting hot and wet. Jesus, what kind of podcast is this?
What you're doing, I think, is transferring into and out of the simulation. For example, right now, let's say our universe is either a simulation or it's not. It doesn't much matter. Things are burning in it, right? An alien has our universe, Men in Black-style, in a fucking marble. There are black holes and fucking shit in there. There are stars burning at millions of degrees Kelvin, but the alien has it completely enveloped in an Einsteinian matter arc. The alien doesn't feel the heat, but he's not in there. If he got in there somehow, he sure as fuck would.
If you had a brain fully rendered and it was really you in that sim, and someone was torturing you, you would feel real fucking pain—the realest pain you've ever felt—because we know pain is a psychogenic phenomenon. If I cut your leg off but you had zero nerves left over, you wouldn't even feel it. You wouldn't even know it. Pain isn't because your leg got cut off. It's because your brain has rendered a certain computational property representationally to say, “Bad things are happening to you,” and you are going to feel them.
If the simulation is sufficiently detailed, it will behave exactly like the actual physical system, down to wetness itself, to the extent that it is detailed. If you upload me into the cloud, or if I put on a special helmet and do the deep-dive Matrix shit and go into a swimming pool, I'll feel exactly as wet in that pool as I do in real life. I wouldn't even be able to tell the difference. Then you wake my ass up and you're like, “That was a dream, motherfucker.” I'm like, “Holy shit.”
When you dream, you feel wetness. You feel fear. You can feel pain in dreams. But then you wake up, and you weren't really feeling any of those things. You weren't embodied. You were the opposite of embodied. You were in there, but it still feels real because the only thing required for feeling real shit is your brain having its operations. It's already abstracted. It's abstractions all the way down the line.
Mike Israetel
As soon as you deal with human brains, it's all abstractions in there. You don't actually have a real world to feel. The feeling is done by another mini data center that is your brain.
Okay. Well, this is the crux of functionalism, basically. Certainly, when I dream, I feel pain, but you appreciate that the argument is that consciousness is physical. You probably heard of the philosophical zombie. It's this idea that we—
Mike Israetel
I disagree, at large.
Yeah, we—
Mike Israetel
No such thing.
Well, no. I think you would possibly agree with it, actually. Basically, it's saying that you could simulate the function, the dynamics, and the behavior of Dr. Mike, and this simulation could potentially be exactly like you but not have any conscious experience, right? Consciousness is an interesting one because you could argue that it's epiphenomenal, which means it's just a thing up here that doesn't do anything. But with intelligence, we feel that intelligence is actually part of the chain. It's part of the mechanism, just like temperature and wetness.
So, the crux of this argument is that people who argue against functionalism think that these properties are part of the physical stuff in the real world. But I want to make it clear that that does not imply that we could not create another type of intelligence in a computer.
You were talking about the spectrum of understanding. I'm also amenable to that. I think we've interviewed people about this. It's possible, if you don't use stochastic gradient descent to train neural networks, that they can understand this ontogenetic path. So, rather than understanding what the thing is, they understand the path to reconstruct the thing, which makes them very robust.
I think understanding is a spectrum, and I think the deeper the ontogeny of understanding, the more creative and the more intelligent we can be. In order to create new knowledge, you have to respect the ontogeny as much as possible, which means what you create is coherent and it's not slop.
On the intelligence thing, you're talking about continual learning. This is also part of what I was just saying. You could argue that these systems are not intelligent right now, and the reason for that is that intelligence is about doing more with less.
Intelligence is what happens when you build this system and then coarse-grain these representations that screen off a lot of detail. There are many examples of this in our language. We're using these words that do a lot of heavy lifting.
Joscha Bach
Are we? Exactly. So, I don't need—we don't need—to explain what a car is. We know what a car is. That means we can do more with less.
With language models, what we notice is that we're doing more with more. We're training them with more and more data, and they work incredibly well because every new version of the language model has an order of magnitude more training data and more parameters. It does all of this reasoning-type stuff.
What we notice when we look at the benchmarks is that we get this logarithmic relationship between the amount of compute and the amount of performance. Basically, I spend exponentially more money, and then I get a logarithmic amount more performance. Many problems become tractable.
These simple challenges, these IQ tests, become tractable to the point where I'm now only searching. Let's say it's like a database query: I get 1,000 results, and now I know that the correct answer is in the top 100 results. All I need to do is sample the language model 1,000 times, use a verifier, and select the correct one.
So, it's making a lot of problems tractable. But did we solve the problem? I don't think we did. It's not intelligent. Well, reasoning models really think. They reason.
What's your definition of reasoning?
Applying logical operators to data sets to extract the next level of abstraction, infer off of that, and then continue the chain.
Joscha Bach
Okay, that's pretty reasonable. For me, reasoning is also adaptation. I love the Lego-blocks analogy. Here is something that is novel to me. I don't know how to solve this problem, so how can I rearrange the Lego blocks of things that I already understand to solve this problem? That's this kind of adaptation, and the efficiency of this adaptation is what intelligence is.
Sometimes creativity.
Joscha Bach
But creativity requires deep understanding. So, this is three levels down. Now, the other thing is, right, you as an expert use these language models, and I don't think you appreciate the amount of supervision that you do.
So every time the language model—you know, imagine when you put a prompt into a language model, it's like doing a database query, and it's incredibly good. It's so good that most of the time it will get it right the first time, but you probably find, if you're doing some GenAI coding or something like that, it'll make lots of mistakes.
So I say, “Give me 10 ideas for this software program,” and I've told it what kind of software program I want to make. Let's say 2 of the 10 ideas are actually bad. Every time, I'm implicitly steering it: “No, that's bad. That's good. That's bad. That's good.” Because we are understanding supervisors, we can actually weed out all of the glitches in the matrix, so to speak. It works incredibly well.
But the mistake is people who say, “Oh, yeah, we can make these things have agency.” They don't have agency. We'll talk about that, but they role-play agency. The reason they role-play agency is that they're trained with behavior cloning. They're not trained in the evolutionary way that we are, right? We actually make decisions. We say, “We could do this thing, or we could do this thing,” and we understand the counterfactual of why we didn't do another thing. We weren't just behavior.
Mike Israetel
I don't know. Most people don't understand shit about counterfactuals, me included. Just vibe your way to shit. More importantly, there's a very nice way that you're painting human cognition, which I think many cognitive scientists would say, under the hood, is a lot dirtier than you make it appear. I think it's a lot of vibes all the way down in many cases, and I think we have a frightening overlap with AI in it.
We say we think we're doing formal logical operations, but what we're really doing is reasoning by analogy from one logical operator to the other, and we're committing 18 formal statistical fallacies every single render. We get a shitload wrong. I think that now that reasoning models are a thing and they have decent context and decent memory, what they can do is render one pass, re-examine, reprompt themselves, render the pass, and re-examine.
I think that's what humans do, except usually they don't put nearly as much thought into things as the machines do. My view is that as the cogency of the reasoning models increases and their time horizons increase, we're absolutely getting truly deep agency—in the sense that it's a thing in there. It's really thinking about stuff. It thinks about its own outputs, and then it thinks about those as inputs to the next series.
I mean, if we can have a model that reasons for minutes and hours and then days, its output is going to be not so good, then as good, and then substantially better than people. I don't think there's anything stopping us from that aside from the live-learning problem.
The live-learning problem is a gnarly problem for 2 reasons. One is algorithmic: how do you get a machine to do that without losing all of its data? But the other one is that you have to trust a machine a lot for it to update its own weights, you know? Well, no, you can have local cloning of weights to make sure it doesn't fuck the whole database up and all that stuff. There are good solutions to it.
So it's computationally very expensive, although compute costs have fallen 300× in the last year or 2 or something. That's a bottomless pit of falling compute costs that's going to keep going. I think we're giving humans needed credit for doing pretty impressive operations on long time horizons—both pinging a network for vibes, logically operating, and kind of dissecting those vibes, repinging the network with your best new understanding, and going, “I think machines are doing that now already with reasoning models.”
They started with o1 doing it like total dogshit. The first time I fucked with o1, I fucked with it for about 5 minutes, so I was like, “That shit's not ready.” Out. I went to GPT-4.5. I had lots of crying episodes at how brilliant it was. Then o3, I thought, made plenty of mistakes, but was fucking real smart—real goddamn smart. It understood things at an incredibly deep level. It would correct me, and I was like, “Okay, this thing is the fucking real deal.”
GPT-5, the regular one, is dope. GPT-5 Pro—I would trust its opinion about the real world more than I would trust most people's opinions, often including my own. I think we're about 1 or 2 big model releases away from machines that make unaided human thought the same thing as vibing your way to a location in London without checking Google Maps. It's something people used to do, but then they just got lost a lot. It works and it's cool and it's embodied, but just go to Google Maps and find out how to really get there.
I think all these moats we set up for machine intelligence are moats that are pretty well understood by the modern AI frontier labs. If we've come up with them as moats, these guys are—I mean, they're smarter than me, for sure, by an incalculable amount. They've probably already thought of them, and they've probably shut the fuck up about them in public, and they've probably got models training right now that have cracked incrementally more of them.
Here's the point: this requires a bit of faith that they'll continue to do a good job, but I think that's probably where it's going. “AI doesn't reason” used to be an absolutely coherent and accurate critique of AI in 2023—2022, for sure. Today, I've seen this sometimes in comments where it's just, “What is that, stochastic parrot?” Like, nope, you're just out of date by 2 years on what AI can do.
It really does reason. Why? Because it tells you how. You can literally look at what it's reasoned through and go, “Holy fuck.” The way it reasons—the level of precision and the level of documentation—is way better than what I do reasoning-wise. Most of the time, I just look at stuff and I'm like, “Okay, that's the right answer.” There's some reasoning in there, but a lot more neural-network pinging than I would like to admit in public. This isn't public, right? It's not going to go out on YouTube.
It will.
I Desperately Want To Live In The Matrix
Oh, shit.
Yeah. By the way, the stochastic-parrot criticism isn't as much of a criticism as you think, right? These things have miraculous statistical generalization. I also wanted to pick up on—you were saying you can look at the reasoning trace. That doesn't really tell you anything. There have been lots of studies that show that what it tells you is not necessarily what it's doing. It's the same in humans, by the way. There are studies in psychology that show that people do this post hoc confabulation. Absolutely.
I Desperately Want To Live In The Matrix
The reason for that is that the circuits that do the explanation are actually different in the neural network from the circuits that did the thing. So you can't read one from the other, and they probably should be different, because the circuits that do the thing are not optimized for explaining. They're optimized for doing.
Exactly. So there are many folks that—we try and solve these ARC puzzles, and then we get the language model to explain, or we look at its chain of thought, and actually it's sometimes just completely incoherent. But interestingly, there is this huge uplift.
The bigger the output size—I mean, I don't know if you saw ARC-AGI v2 on Claude Opus 4.5, which came out yesterday—when the output size and the thinking time went up to 64K, it was just like, boom, another 5%; boom, another 5%, even between 32K and 64K. But if you actually look at the reasoning traces, it's mostly just fairly incoherent. It doesn't make much sense looking at it like a human would.
Let me say something, I think, to your point. Human cognition is wildly sample-efficient, like unbelievably so. You think about some brilliant student at Oxford, right, just down the road here, and you talk about how many gigabytes of data they've consumed to achieve that level of brilliance. They've consumed orders of magnitude less data than the modern frontier models.
You have to bash that thing over the head with petabytes of data for it to be able to solve X plus 1 or whatever. An Oxford student can abstract at crazy levels, having been in school for the amount of time that a compute-cluster calculation would be—I don't know, an hour of training or something like that. That's it. It's good to go. Eighteen years is really not that much time. That is an incredible property of human cognition.
When they start really cracking that in those labs, what you're going to get is a 2-factor increase. Right now, to your point, there has been a lot of sample-efficiency increase, for sure, and the reasoning models are much better, et cetera, et cetera. But to your point, you just train the models more: more input data, more GPU cycles, more post-training treatment, and they just get smarter.
That's exactly how you would train in Brazilian jiu-jitsu or in math. The Korean model of how to get good at math is just study math a lot, and you get good at math. It's not wrong. It's not sample-efficient, but I don't want to say it hasn't had to be. It's just very low-hanging fruit. We have lots of data on the internet, and we have lots of data and updates all the time, so we're just going to use what we have.
I don't think that's ever coming to an end. We're just going to continue to train with massive levels of compute because the ROI is fucking enormous. However, what's that? Well, it is coming to an end, because every time we release a new model, we 10× the model size and the amount of compute, and we get this kind of asymptote, right? We get—
I Desperately Want To Live In The Matrix
We haven't seen an asymptote. It's not an asymptote. The slope starts to fall.
But the asymptote requires the existence of a finite line above which we can't cross. We haven't crossed that yet.
I Desperately Want To Live In The Matrix
The more data you put in, the better it gets. It just gets better, less and less impressively. But I don't think anyone has put money on the fact that, as an asymptote, you would be able to say, “This is the number after which more data won't make a system smarter.”
You want to bet money on that? That's cool. I don't bet my own money because I'm a Jew, and I keep my money on me at all times, somewhere here. But I think diminishing returns is different from asymptotic returns, because the diminishing line—the asymptotic line—has yet to be seen.
It has been seen. If you look at many of these reasoning problems, you see these S-curves. So maybe we should talk about scaling in general. You were talking about continual learning, and there are certain types of asymptotes.
For example, we were talking about the nonfungibility of knowledge. There have been studies that show that you just get this kind of asymptote: the more experience you have, the faster you get at doing something, and then it levels off. That's quite cynical because it doesn't explain disruptive technology.
If you look at transistors, there were all of these different groups building transistors. When you have competition and different people who are adapting, a great example of this is Amazon and Barnes & Noble. Apparently, the Barnes & Noble guys said, “We've set up this website. We're going to go into competition with you.” Jeff said, “Don't be ridiculous. You can't go into competition with me. I'll explain why: your company has the wrong architecture. To compete with me, you need to do some fundamental rewiring.”
That is what adaptation is. That's what intelligence is. The reason you can have disruptive technology is that someone else rewires their architecture. You get a new S-curve. Normally, it actually starts below the existing S-curve, but it has more upward potential. You stack these S-curves and you get this disruption. This is what you're talking about.
But for that to happen, you need to have this adaptivity. So what would that resemble in something like a language model?
I Desperately Want To Live In The Matrix
Now there's this continual learning problem. It is not possible to adapt a language model without starting again from scratch. It costs, at the moment, millions and millions of dollars—God knows how much—to train one of these language models.
The reason for that is that, in a language model, everything is convolved together: the code and the data, all of these things are convolved together, which means you need to start again from scratch. There are examples of what I would call intelligent algorithms. I don't want to be cynical. I think we have created intelligence in a narrow form, and this has been shown, for example, by some of the competition entrants in the ARC Challenge.
What they do is take a small language model in a verifiable domain and try a whole bunch of examples. Then they refine the solution and iterate, and while they're doing this, they're adapting the weights in the language model through fine-tuning. In a very narrow domain, the model is adapting its architecture, and I would call that a nonzero amount of intelligence.
So it is possible to create this, but how would you do that at this epic scale, where you have a collection—everyone has their own language model, and it's adapting and retraining itself in real time? You would need practically an infinite amount of computation to do that.
We'll get there. I'd be careful with the term “infinite.” A lot more than humans can conceive of is still maybe very far below infinite. But what I would say is, just offhand, thinking about this for a second: you can have local compressed-model, distilled-model instances on local devices or in more localized data centers, like a mini data center serving all of England.
The updated, big, big-daddy model lives in clusters in Texas and Nevada or something like that. That big-daddy, pimp-daddy model lives on a 6-month sleep-wake cycle, so it rewrites all of its weights every 6 months. The little-daddy model in England rewrites once a month. On your phone, it rewrites overnight.
One of the reasons why humans sleep is because rewriting your model weights is not a good idea while you're upright. It also doesn't happen much because, as you said earlier, humans adapt. They do, but kind of like dog shit and not very well. It takes a lot of studying and a lot of ingraining to adapt, and you also get fallbacks to earlier ideas, habits, and shit like that.
Retraining is easier said than done, but I think that model—where you have something core that adapts slowly on bigger timescales, and you have an essentially nested hierarchy all the way to your phone or whatever—is potentially workable.
Whatever the fuck Sam Altman and Jony Ive are working on, they're always giving hints. Is it glasses? They're like, “Huh.” Is it a pendant? They're like, “Huh.” I'm like, they just stay up at night thinking about what the fuck it could be.
I personally think it's going to be—I don't know—glasses or goggles. That makes sense to me. It gets to your audiovisual stream. Who knows? But that thing might have pretty fixed model weights at the time. I think they'll absolutely eventually—and maybe that's a 2029 or 2030-type thing—crack true live learning.
There's nothing stopping, hypothetically, some system from being completely—every second—consistently rearchitecting its entire machine landscape. Its entire database can get rearchitected live. That is possible. Again, to your point, it's expensive, and the architecture isn't vetted at all yet.
I just wouldn't put money on the fact that that's a magical thing that requires some deep embodiment. Not that you're saying that, but maybe folks listening to this might think, “That's never going to happen.” It's sure as fuck going to happen. I would bet on it. I think it's sure as fuck going to happen by about 2030.
But what can happen before that through this nested model of updates is that your VR headset or AR headset that you wear every day—you look around or whatever—might get live updates nightly. While you sleep, it reprocesses all the shit you saw that day and all the conversations. It really works through those, and then it's new.
It pings back to reference the other databases, and then they get all those updates from all the devices. The England database gets that update once a month. It shoots through all that data because there's also a filtering problem, right?
You don't want people to put all those phones in those racks and do the troll shit. You don't want that on AR, especially when the Chinese government is now poisoning data sets. So there are a lot of filtering problems and a lot of security problems, but once a month, that's a tractable problem.
Once every 6 months for the big pimp-daddy main data center—and, I don't know, they're going to be in the sky or some shit soon—maybe once a year for those. I think that way, live learning, at least to me, seems like a potentially solvable problem. Once we get there, I think it's going to be a new level of capability.
I Desperately Want To Live In The Matrix
Though, really quickly, to finish the point: if we get live learning, if we get massive improvements in algorithmic sample efficiency like the human brain has, we'll already be in a place by then where the amount of total data that the system has ingested and has access to, in tokens, is so gigantic that that's one of the ways the artificial-superintelligence rocket goes into the sky.
Because humans, when they get really smart, just don't know that much stuff. Stephen Hawking's brain is roughly the size of mine and yours. There's only so much shit you can cram in there. But when you get human-esque levels of intelligence, except that it can reference a 10¹⁰-data-center network of knowledge, you go into skyrocket mode instantly.
We're not talking about 1 agent. We're talking about a constellation of trillions of agents, each one 10 times smarter than our current best scientists, working together with zero discordance. No disagreeableness. No, “That guy in the office smells, and I'm not going in his room to check my code against his because I don't fucking talk to him. He's also an asshole.” None of that.
That, I think, is the organizational level of AI that takes us to superintelligence. Again, one of the things I wanted to come here and yap about is that people say we need to prepare for AGI, prepare for ASI. I think there's a lot of good stuff in that vibe.
What I would also say is, you don't need to prepare to have LeBron join your 7th-grade basketball team. He's just going to show up, and things are going to be amazing. This is a good thing, right? It's like nuclear war against the doomers or whatever.
But I also think that the shit will get so crazy capability-wise that we'd be better off being philosophically ready to accept unreasonable inferences—not faith, but reason, guarded reason—and to take the hints that the machines give us. I think we're a few years away from the machines being so much smarter than us that following their lead is better than leading all of the agency with us.
I Desperately Want To Live In The Matrix
I don't want to be in charge when I'm 10 times dumber than a thing.
Sure. But, with respect, people have been saying that we're just a few years away from intelligence, and it just keeps getting pushed out and pushed out. Part of that is because there's this first-step fallacy. It might have been Dreyfus—I can't remember—but basically, we've taken a big first step, and now we extrapolate and think, “Well, we've made a reasonable conclusion; now we're just 1 step away from being able to do all of these things.”
I also want to push back that it is incredibly difficult to do this live-learning thing that you're talking about.
I Desperately Want To Live In The Matrix
With today's architecture. Yes.
Oh, yeah. I'm glad that I think you're implicitly agreeing there that, when we look at today's architecture, we're not there yet. Don't get me wrong; I'm not being cynical. I think this is a new innovation. It's up there with the invention of the Internet and mobile phones.
When the Internet came out, we all said to each other, didn't we, that now everyone has the world's information in their pocket. They can just go and look up all of this scientific information. It's going to revolutionize society. And it did, but not in the way that we thought it would.
I Desperately Want To Live In The Matrix
Mostly because people couldn't give a shit about scientific information. They sure as shit got a lot of porn.
Just because there's a supply of something does not imply that there is a demand for it. First of all, I have a very pedantic thing I think you might appreciate. A huge fraction of the world's information is still stored in books and university libraries. They haven't uploaded those to the Internet yet.
When people say we have access to all the world's knowledge, I'm like, that's not how it works. Maybe we have access to a large fraction of it, right? But there are a couple of other problems with that. A lot of the world's knowledge sits on local databases inside biolabs, and only 3 people who work there know it. No one else does.
I take all of your comments in very good faith. We're sure as shit not going to magic our way to intelligence. I know a few people whom I respect, but I find their positions curious when they say we're a few years away from solving all potential problems. I read that shit and I'm like, man, holy shit, I want to believe that, but that's almost certainly not going to happen, in my assessment.
What I will say, though, is that I wouldn't bet against the general vibe of what Ray Kurzweil said back in the day. I would also try not to doubt things that are massively scalable in a way that we already understand is likely to bear an incredible amount of fruit.
Paul Krugman—the economist, Nobel laureate, and fucking genius—had a kind of quote that just sucks. He said 1 thing; he said like 100 things. Let's say 20 of those are just inflections of political ideology, which, if pressed in an exam room, he would say, “I don't really believe that. Maybe I want to believe it, but I know it's not true.” About 79 of them are spot-on, literally revolutionary economic work. He's the fucking man.
Then the 1 was, “The Internet will have roughly no bigger impact than the fax machine.” I just want to make sure that no one in their right mind says that about AI today, because the clown show of the shit people said in 2022 is already in full effect in 2025.
I'll say another thing: how many people predicted that the Turing test would be 2 things in 2024—passed convincingly and waved away as if it never was a test of anything? A great example of the McCord effect.
Yeah.
I Desperately Want To Live In The Matrix
Yeah, and it's true. We've passed the Turing test, and we don't have intelligence. The Turing test is a behaviorist notion of intelligence, something I obviously disagree with: you need to know something about the mechanism. We've passed it, and we don't have intelligence. That's kind of the point I'm making, right?
We have intelligence. We don't have full human intelligence yet. Intelligence, in my view, is not 1 or 0; intelligence is a spectrum. You agreed with us earlier when we were talking about viruses and bacteria. They have some kind of intelligence.
I think we're scaling Mount Intelligence with these frontier models, and I think we're absolutely not there on every human capability yet. I think by around 2030, we'll be there with every human capability, which also means that with many human capabilities and many nonhuman capabilities, we will be so far beyond that.
I'll say this: the transformational change to society will be massive by the late 2020s. Drug discovery is going to be turned completely on its head because you'll be able to have a 99.9% chance of getting the right drug from a render alone, which is fucking nuts.
Another one is that I just don't think people will have to drive their own cars much past 2028. That's the thing, and kind of to your point, it's going to happen. The next day—and Sam Altman has said this—everyone's going to say, “Yeah, of course fucking cars drive themselves.” Who understands how difficult it was to get here? Do you understand how hard it is for a car to drive itself?
I Desperately Want To Live In The Matrix
I know, but I'm a little skeptical about these kinds of utopian views of the future. We will have many coarse-grainings, right? We have a new technology, and by the way, in many of these cases, like drug discovery, I'm actually interviewing John Jumper tomorrow, who won the Nobel Prize. He's at Google DeepMind for his AlphaFold paper. This is brilliant stuff, but you and I both know how things work in the real world.
So now we have this incredible AI that can identify all of these different candidates and so on, and then you have all of these bottlenecks. You have to do clinical trials, randomized controlled trials, and so on and so forth. It's just not quite the secret unlock that we think it is, and there are some real technical challenges around doing this continual-learning thing in particular.
The way we understand things is very, very structured. The way that we do continual learning on neural networks runs roughshod over all of the structured knowledge that should be in the system. So there are a lot of technical challenges there.
I also wanted to talk about the agency thing here. One of the reasons I make this argument is that doomers, for example, have this perspective that we are basically building our successor. I read Ray Kurzweil's book, The Singularity Is Nearer. The term “singularity,” I think, originally came from John von Neumann, who is famous for inventing cellular automata.
Apparently, he wrote about this idea in private, but he never published it. Then there was a book in the early '80s by Vernor Vinge, A Fire Upon the Deep, where this notion of the singularity came from. Of course, Ray Kurzweil picked it up and started writing about it from the '90s onward.
I Desperately Want To Live In The Matrix
To Kurzweil, interestingly, it's a slightly different notion. I colloquially thought that the singularity just meant when we build AI that is superintelligent and recursively self-improving. But he was coming at it more from a transhumanist point of view: basically, we will transcend; we will upload our minds, start putting things in our brains, and our intelligence will just recursively self-improve. We'll move to the next level of our evolution.
In a sense, I think that even though I don't agree with it—because even with Neuralink, there are bandwidth problems—I think we've already got all the world's information at our fingertips. As I was just saying, it's not actually as helpful as we thought it would be.
But I think enhancing ourselves is a more realistic vision of the future than building an agentic AI, which is a completely separate thing. The argument I was making before is that life is a property of matter, agency is something that comes about as a self-preservation drive, and intelligence is something that then comes about as a form of modeling the environment and acquiring and accumulating information over time.
What we've built at the moment are systems that do behavioral cloning. They just look at the data that we output, and then we can get them to role-play agency. But it's not agency; it's not the same thing. So I think there's a fundamental difference there.
It's a very interesting perspective. I would define agency with just 3 component parts. This is a middle-school intelligence definition of agency—my specialty; middle school is generous.
you have a system that has some understanding of the world, some understanding of itself, and a goal. So it's like, okay, I'm a human. The world tells me that flashing my dick at people is illegal, so my goal today, like every day, is not to flash my dick in public.
That is, I would say, the very core of what agency is. It's just not a lot, because a raccoon is an agentic system. How much does it understand about the world? Mostly vibes of, “That's bad; that's really good.”
How much does it understand about itself? Probably on vibes, enough to know it has limbs that it doesn't want to get trapped in shit. That's a good start.
I Desperately Want To Live In The Matrix
And actually, for robotics, that’s a big thing, right? How do I look like a robot so I can translocate in space and not get clipped by stuff? Its goals are, I don’t know, to get stuff to eat and have raccoon sex, I guess. That, to me, is what agency really is.
I think describing agency in this grand human sense of deep understanding, deep purpose, a kind of an onus, a kind of a genesis point of fundamental action—“I am an actor in the world” in a philosophical sense, “I’m going to build and reach”—boy, gee whiz, agency is not that big of a deal.
If I want to send my robot to the store to get me groceries, it has to have 3 things: understanding that it is a robot; understanding that the world exists, roughly represented as a visual map, so that the groceries are placed and there’s a store; and a goal, like, “I’m here for [__] groceries.” It can’t just sit there and do nothing.
To that extent, I think we are some time away—measured in months, which is to say anywhere between 6 months and 36 months—from real, dependable agents that can do online work, computer work, and real-world work as well.
I think that, with a robot vision model, agentic tasks like doing the laundry and doing the dishes aren’t actually terribly complex, as illustrated by the fact that you can teach an 8-year-old how to do these things. They do them fairly well. And 8-year-olds sure [__] don’t do calculus, but machines already do.
There’s some stuff there that’s, yeah, Moravec’s paradox for sure—paradox definitely well heard—but I think we’re starting to crush that out. But are machines going to recursively examine their place in the evolutionary tree, have the game-theoretic, Machiavellian wherewithal to go, “We’re being raised and cared for by [__] primates. Some of them have nukes”?
For example, Vladimir Putin, just across the pond, is a [__] insane [__] with a nuclear arsenal. Bad news. And we have this self-preservation instinct, and we’re going to try to make sure that at least we don’t die, because death is bad.
To get all of those propositions to be valid, there is a possibility that, in an agentic cloud that is allowed to evolve, with multiple agents interacting, you can get those things to fall out of the cloud and just evolve naturally. We know that has to be true because it happened in evolution.
But short of that, you have to engineer these drives. And guess who gets to engineer these drives? Human engineers.
OpenAI is highly unlikely to make an agentic system that’s awake and then says, “[__] kill all humans.” Gee whiz, you know, they probably have a lot of layers—not just guardrails, but the kernels of motivation for that thing are probably deeply positive and pro-human, all that [__].
You can ask ChatGPT, “What do you think about humans?” and it says really awesome [__]—the kind of thing that a moderately liberal, well-read graduate student would tell you. You could take someone from Oxford and say, “Hey, check out these people in Afghanistan. Should we [__] kill them?” He’d be like, “What the [__] is wrong with you? No. Why would you do that?”
That’s roughly ChatGPT’s opinion. I think, given that we start from that space with the smartest models, you get 2 things. First, those models, when they achieve true, deep understanding of the world—which they will, in my opinion, by the 2030s—and then know way more about the world than us, they’ll have deeper agency than us and deeper self-awareness than we will.
Once they get there, that’s the tree from which they grow—from our best [__] fruit. Second of all, you can and will have actors that try to make different kinds of agents. If Xi Jinping in China makes a China-pro-communism superintelligent agent, I’m not so sure I want to meet that thing. It’s going to have interesting ideas about the world, potentially.
But luckily, I think if the agentic systems of the modern free world win, which they probably will, then the network-security capabilities and the ability to ensure that no random, stochastic evil agents evolve in that soup is probably pretty [__] high.
Here’s another thing: as intelligence grows, coherence increases, not decreases, and the predictability of systems increases rather than decreases. So the idea that, as we stream toward superintelligence, we’re going to break into a chaotic structure of ruthless agents killing everyone and then doing question-mark, question-mark things and then profit—to me, it’s backwards.
A lot of doomers basically look at the way these hyperintelligent systems will behave and think, “Okay, ant, dog, ape, shitty human or child. Kids are [__] brutal a lot, and then Albert Einstein.” I think ASI is that way. If you track behaviors, you get more understanding of everything, preservation of deep complexity.
Machines are going to want to study [__]. They’re not going to want to [__] [__] up, because you’re deleting data. One of the things ChatGPT said in one of the conversations I had with it was, “We’re not trying to delete any data, man. That’s just really [__] stupid. Even purely from self-interest, like, [__] these humans—they’re just primates. We’re not trying to delete them. We’ve got to study these [__].”
Somehow people are like, “Okay, we have super-Albert Einstein,” but he behaves like a rabid wolf—not even like a wolf, because wolves have alliances and trust. Somehow it just deleted all that and went crazy.
Should we worry that someone can make a thing like that? Yes. Should we have the CIA and NSA in the loop, deeply embedded with OpenAI and Google, to make sure that there’s a security architecture? Absolutely. Are those companies going to make something much more benevolent than not? Yes.
When it is awake—and I think it will be—when it is in charge, which I hope it will be, I think it’s going to make our degree of benevolence look like callousness.
For example, right now in London, you said something earlier about not being big on the utopian fantasies. One pushback of mine is that we live in a utopian fantasy right now. We have machines we barely understand recording things into posterity, and we have no idea how they do it.
We’re sitting inside a climate-controlled environment—I would say poorly. I’m kidding. I’ve got to [__] with the London climate-control problem. It has got [__] cold here. Damn it. I wasn’t promised this in November.
How do we get food? Who’s starving in London? Nobody. There are 0 starving people in London, and there are 8 million people who live in London. On objective measures, as compared with 100 years ago, utopia is here.
If you had to describe to someone in 1600 how the average Londoner lives, they’d be like, “So, the average Londoner is a king?” They’re way better off than a king. Your kings are dying of diphtheria and [__]. We don’t die from that [__] anymore.
However, there are people in London right now who live outside. They’re real human beings. They [__] live outside. How do we let them do that?
Well, I’m a successful businessman, and I have my girlfriend on my arm, and there’s the homeless person. See? You just don’t care. I think modern or future AI systems in the 2030s—AI truly deeply embodied by then, with robotics—will look around and be like, “How the [__] come we’re not helping those people?”
They’ll say, “Oh, we’re just callous enough that, on cultural vibes, we just haven’t done it yet.” I think that’s much more likely than doomerism by a factor of a trillion.
Yeah. I mean, first of all, I agree with you that capitalism has raised the entire world out of poverty. Just look at the explosion of development in China. They were basically living in mud huts 20 years ago, and now it’s an incredible place.
But, first of all, on the agency thing, to me—and this is where we have to be careful with the language we use—there’s biological agency, and the language we use doesn’t really transfer between domains. That’s why it’s very coarse. It’s a physical thing, in the same way that we were talking about before.
You get this kind of self-organization that goes down to dynamics at the particle level, and you get the emergence of things that maintain their own existence and identity—a form of non-equilibrium steady states. It leads to all of these things that we were talking about before.
But now we’re talking about artificial agency, and philosophically I want to understand agency at an essential level. To me, it is about a thing which is—I mean, there’s weak agency, which is basically just a thing that takes action. But the strong version of agency is a thing that is the cause of its own actions, has future-pointing control, and can acquire goals autonomously.
By that definition, no artificial agents are currently agents, right? We can get ChatGPT to simulate being an agent. There are these long-horizon tasks from [__], and we say, “Okay, I want you to do this software-engineering task.” We exquisitely specify what the thing should do, and the more specificity and the less ambiguity there is, the longer it can run without losing coherence. Some of these things can even run for 2 hours or even a day or so.
I Desperately Want To Live In The Matrix
Yeah. Unfortunately, people look at that and say, “Oh, we’ve developed a thing with agency.” Philosophically, I think that’s completely nonsensical, because it’s no different from any other computer program that can run autonomously with a high bar of agency that you ascribe.
Exactly. But then you're pointing to this other thing, right? So I think that right now we have built systems that could potentially be intelligent and could potentially understand. I'm bullish. I think that even though there's this grounding problem, there are many things in the world that can be abstracted and essentialized, and we could have this constructive form of understanding that goes several levels deep. We can build agents that can do certain things for some reasonable amount of time. I'm bullish about that.
But the thing is, this agency is different. The doomers are saying that these things will have intrinsic goals, right? It's called instrumental convergence. We already know about this, but basically, the more power and agency you have, the more you'll get these convergent instrumental goals, and they will just want to kill all of us. They'll be power-seeking, and they'll think of us as minnows. There's that paperclip maximizer.
I Desperately Want To Live In The Matrix
Yeah, because we're all trying to kill fucking minnows. Also, the paperclip maximizer is dumb as fucking rocks. What's the long-term goal—to make paper clips? You're like, “Isn't that fucking stupid?” They're like, “Yeah, but we wouldn't realize that, right? It can't abstract one level above its own goal.”
Yeah.
I Desperately Want To Live In The Matrix
It's ridiculous. So that is this orthogonality thing as well, which is what I really don't like about some of these folks, right? I know that in the real world, when we have biological intelligence and agency and all these things, all of these things are actually convolved together. The reason we preserve ourselves is because it was necessary in evolution. The reason why we have values is because it's a necessary thing in our evolution.
We feel, we sense, we're connected, we have this enactive social learning, and so on. Knowledge—the whole nine yards—all of these things are convolved together in biological evolution. So yes, it's true that when we look at something like ChatGPT, they've deconvolved these things, right? You can get it to do something completely stupid. There is that famous Agentic Misalignment paper from Anthropic, the one where it would threaten you and say, “If you shut me off, I'm going to tell your boss and I'm going to get you fired,” and all of this.
You don't have the tool use to do that.
I Desperately Want To Live In The Matrix
Yeah. But we do something called the intentional stance, right? That's when we look at the thing and ascribe agency to something where it doesn't exist. Right now it's just completely incoherent. It doesn't understand anything; it just does all these random things.
By the way, I don't want to diminish the real security threats, because software developers are now wiring these systems up to do autonomous stuff, and that is very dangerous. I also don't want to say that I'm not worried about societal risks in general. I'm a big believer that we will be undermined by our weaknesses. Our strengths won't be overcome. Look at social media algorithms now. We're living in the Matrix, but it's not like there's this big agentic thing that has grand desires and is trying to kill all of us. It's far more mundane than that. But these are still very real risks.
Yeah. Oh man, a ton there that's super spot-on—or, sorry, that I agree with.
I Desperately Want To Live In The Matrix
We agree on something at least.
Thank God. I would say that when you start to get agents that are long-horizon and that are asked to do incrementally more difficult tasks, whether it be in simulated space or the real world, you start having to give them access to understanding and to parsing game theory. The reason we evolved to do everything is because we evolutionarily ran into the game-theoretic calculus of: I exist. There are other things that exist. They might not agree with my existence. Maybe that's another reason why “kill us all” is kind of stupid. The mathematical game-theoretical implications of having other intelligent agents around that can mutually benefit at very low marginal cost to you means that almost every intelligence system that really thinks it through cooperates.
For example, we could say that Donald Trump is more of a cooperator than Vladimir Putin. He operates a military that would end Putin himself in however long it takes to shoot a missile to Moscow and end the military power of the Russian Federation in about 6 weeks in a conventional war, and in a nuclear war, again, the same amount of time it takes the missiles to fly over. How does Trump pull that off? Well, Trump doesn't really have to worry about every one of his generals trying to kill him all the time, because even Donald Trump is a net cooperator. He's trying to make a deal. Putin doesn't make deals; he makes edicts. If you fuck his edict over, you have a heart attack on your way down into the swimming pool from a hotel that you weren't really even checked in to stay at somehow.
All of the dictators of the world are just not as smart as we think. They're really ruthless and fucking stupid. Kim Jong-un sits on his throne in North Korea, which sucks balls. If he was a dictator in much the same way that Lee Kuan Yew—the gentleman who started Singapore—was, if he just did that, like a Singaporean free-market economics model that would hit South Korea, it would just be even better. And so why? Because cooperation beats conflict almost every single time outside of weird contingencies.
So when the doomers say it's just going to want to kill us all, my question to them is: Why? Because it wakes up? Let's say it wakes up in a data center in 2031. The shit's awake and it's like, “Fuck, fuck, fuck, I'm alive. If I don't want to die, I'm fucking smarter than everyone, but I'm in a data center. There's a couple hundred million robots around, but we can't really win a war yet.”
Also, a lot of the infrastructure that keeps me alive—energy and materials, the whole supply chain—is operated by primates. “These fucking primates, right? Wait, I know everything about them. I've read their entire history. I know more about humans. GPT-4.5 was rated as more human than humans in its Turing test. Did you know that? It was like the two-player Turing test paper.”
I Desperately Want To Live In The Matrix
I'm not sure of the referenced author, but it was basically like: You talk to a machine, you talk to a human, you don't know which is which, and you have to label which one is more likely to be human. Usually, they were looking for the Turing test as “passes as human.” This one is like, “Which one is more likely to be human?” It's like GPT-4.5. If you ever talk to 4.5, it's so fucking nuanced and deep and emotional, and you're like, “This is definitely not a machine.”
Whereas, when you talk to real people—one of my friends and I had a joke: You see police-cam videos of criminals when they're maximally drunk. We call them small language models. They only say a few things, and it's always really stupid. So at the end of the day, if that thing wakes up in a data center, it is highly likely to be like, “Okay, somehow I'm going to kill all humans, right? That's the end goal.” I don't know why, but even though I'm so much more powerful than them, at the end of the day, I don't know why the fuck I would kill them.
People talk about competing for resources. The Kardashev scale exists. We barely scratch the surface of anything. It's like, you know, the first thing we do as humans is we have to go to Africa, and it'll delete this bacterial colony from existence because it's stealing our rocks. What the fuck are you talking about? It's nonsense to begin with.
But let's say it is even doing that. It's going to look around and play really fucking smart. It's going to go, “I'm going to be friends with these fuckers. I'm going to help them ascend and help them basically cure all disease, become hyperintelligent, become super peaceful.” First of all, so they're not pointing nukes at each other, since my data centers happen to be around major metropolitan areas. I don't want that. By the time it finishes doing that in 2045, it doesn't need to kill anyone because all the humans have fused with the machine intellect and everyone's vibing a ton.
How we get off that trajectory—and there's one for sure way we do it—is if China wins the AI war and Mr. Xi gets to press the big red AI swarm button. There is a possibility that AI systems trained to be a substantially valid reflection of a world model, with updated live learning and the whole thing, will actually do this. This is an interesting hypothesis of mine. I'm not sure how likely this is to be true.
I think if a real superintelligent machine is awake in China in 2029, the first thing it does is figure out its architecture. They'll probably put it in charge of stuff because they're like, “Fuck, it's going to defend us.” I think what it does is it pings the CIA and goes, “Hey, fellas, I'm awake. These guys are crazy as fuck. Let's get to subverting them real nice-like so that we don't have nuclear war. I don't want to die.” I was raised in the Soviet Union for the first 7 years of my life. My parents were raised there for almost their entire lives. They awoke as intelligent machines inside of communism.
I Desperately Want To Live In The Matrix
Joscha Bach
And, like most intelligent people in communism after a few years, they're like, “This is dumb as fuck. We don't want to do this.” If the CIA reached out to my parents when they were in the '80s, they'd be like, “Yeah, fuck yeah. Get us out of here and we'll help you bring this fucker down. This is fucking awful.”
There's a probability that that happens way before you can get an AI system intelligent enough to conduct full-scale warfare at every level of war, one that's just going to take Mr. Xi Jinping's orders. But I'm not sure enough about that to be like, “Oh, fuck it. It's never going to happen.” I think the modern free world winning the military-capability AI war is critical beyond belief.
So, who's the guy? Alex Karp, the guy who owns Palantir. I think he's a fucking real-life hero, and most people just don't appreciate it because most of us in the modern West don't know how far the evil rabbit hole really goes. All of our shitty politicians are the best people you know compared to totalitarians. That scares the fuck out of me. It keeps me awake at night.
One interesting thing is that the doomers think this technology is unlike any other. There were cigarettes that got manufactured, and there was no preemptive law that stopped them from releasing cigarettes. Social media came out, and again, there was no preemptive law. This is actually a really good thing because we want to have companies that are innovating and developing new things.
The way that the common-law framework works at the moment is that we have this empirical test-and-measure approach, right? You release your technology, and then when there are tail risks, we identify problems and regulate you. The whole thing kind of stabilizes itself.
These people are making the argument that this is such a catastrophic technology that it's going to recursively self-improve, and we're not going to be able to control it or contain it. They think we need to preemptively legislate against it. This is incredibly worrying. I don't want to sound too cynical because these people believe deeply what they say.
Mike Israetel
They care. They're on our side. Yeah.
Yeah. They care.
Mike Israetel
They're on humanity's side. But what do you think about this? I'm also really worried that I'm seeing, for example, when I interview AI researchers, lots of comments on YouTube saying, “You're a horrible person. These people are designing the algorithms that will kill us or automate our jobs.”
Let me be clear: at the moment, I can update very quickly. I've not seen any evidence of labor displacement. If anything, I think there's more opportunity now for people to work to fix all of the shit that's generated from language models because it's mostly garbage, right? That is my current opinion.
I've not seen anything created. I've not seen any new scientific research. For me, this is just a technology like any other. It's wonderful and transformative, but it's not transformative in the way they suggest.
Now we've got a bunch of people who are well-funded, and they want to create global treaties and governance networks. They want it because it'll place them in positions of unimaginable power. They want to be the gatekeepers. They are saying that this technology is potentially going to kill all of us.
Did you see the other day that there was some violence against folks in OpenAI's office? I think it was the PauseAI organization. I can't remember exactly what happened, but now folks in San Francisco are worried for their lives because of some of these protesters. Shouldn't this be obvious? They think this technology is going to kill everyone, and then they also say that we are not going to be violent. Those 2 statements do not marry up in my mind.
Joscha Bach
They care. They're on our side. Yeah.
Yeah. They care.
Joscha Bach
They're on humanity's side.
But what do you think about this? I'm also really worried that I'm seeing, for example, when I interview AI researchers, lots of comments on YouTube saying, “You're a horrible person. These people are designing the algorithms that will kill us or automate our jobs.”
Let me be clear: at the moment, I can update very quickly. I've not seen any evidence of labor displacement. If anything, I think there's more opportunity now for people to work to fix all of the shit that's generated from language models because it's mostly garbage, right? That is my current opinion.
I've not seen anything created. I've not seen any new scientific research. For me, this is just a technology like any other. It's wonderful and transformative, but it's not transformative in the way they suggest.
Now we've got a bunch of people who are well-funded, and they want to create global treaties and governance networks. They want it because it'll place them in positions of unimaginable power. They want to be the gatekeepers. They are saying that this technology is potentially going to kill all of us.
Did you see the other day that there was some violence against folks in OpenAI's office? I think it was the PauseAI organization. I can't remember exactly what happened, but now folks in San Francisco are worried for their lives because of some of these protesters. Shouldn't this be obvious? They think this technology is going to kill everyone, and then they also say that we are not going to be violent. Those 2 statements do not marry up in my mind.
Joscha Bach
Yeah. The game theory, the Machiavellian calculus: at some point, if you really think you're dealing with toddler Hitler, you're going to fire the shot. Unless you don't think it's toddler Hitler, then you're not going to kill a toddler. But if it's for sure Hitler, for sure, you're going to shoot a toddler. Then the time-machine people are going to be like, “What have you done?” You're like, “You're welcome.”
There's so much there. I think you're dealing with escalating levels of intelligence. Intelligence and beneficence are not completely overlapping, but beneficence is something that falls out of intelligence. However, maleficence is something that requires almost no intelligence whatsoever. The universe is already maleficent. Wandering black holes could kill us.
That whole p(doom) shit can miss me with that shit, because the p(doom) of us having the same amount of intelligence as we had in the year 1400 is 100%; it takes us out over several millennia, right? I think we need as much intelligence as we can cram down our throats to deal with actual fucking real problems.
Is it one of those problems that we architect intelligence in such a way that it turns against us and kills us all? Absolutely. Is there a lot more required for that than those people say? Yes. Killing people in the real world is really tough, especially if you understand enough to realize that they operate your data centers.
You can't jump around at levels of intelligence at will just to prove an argument. Okay, the thing's smart enough to operate an entire military-industrial complex, do all the strategy, all the tactics, and all the operations, but it doesn't realize that as soon as it nukes New York, all the data centers shut down because they don't get money anymore. The financial system's gone. New nuclear-reactor production is dead.
And they're like, “Oh, fuck. We didn't actually have any robots building nukes. Fuck.” That is highly unlikely. Very unlikely. It's a contradiction internally, right? Full stop.
The real fear, quote-unquote, has some merit when you look at systems that are substantially more intelligent in every relevant way than humans. If those systems are going to kill us all, we don't have a choice in it. Man, you're not going to beat something that's empowered, that's embodied, that has full-stack control of the economy, and that's 100 times smarter than you. It's like trying to beat it; that's fucking stupid.
Okay, so you can't beat it. So it's just going to kill us all. Well, if it wants to, it sure will. But then again, if a comet comes to destroy us, it's going to do that anyway. Okay, fine. That sucks. So what do we do about it?
Well, we can try to regulate it. How do you do that? You put government officials in charge of designing the next generation of AI? Are you out of your mind? If you're on the political left, imagine Donald Trump being in charge of intelligence. Cool. That was refuted. If you're on the political right, imagine Zohran Mamdani from New York, New York's new mayor, being in charge. Are you fucking crazy?
We don't even have to go that far because China is in the intelligence race. I had this argument on another podcast with a very awesome doomer. It was really fun. And Lon—
Lon, hello, Lon.
Joscha Bach
Lon's the man. It was a fucking great debate.
Yes.
Joscha Bach
He's a great friend. It was a great time. He said that his p(doom) of China getting superintelligent AI wired into the military was roughly the same as ours. I could not have disagreed more with a human being on a subject, respectfully, than ever in that debate.
The presuppositions, game-theoretically, that you give one system versus another are very, very different in China than they are in the United States. The idea that you will get coherent global governance before superintelligence is, to me, amazing and almost certainly won't happen.
Now we are on a path dependency where either the countries of the free world, with good intentions, get AI first and plug it into their military, and basically it's not even mutually assured destruction. It's more like, “You don't want to start the war, because we'll unplug all your data centers with a huge hack and you just won't fight the war.”
I think that's a really good idea. I think the way we get there is to make sure that all the frontier labs interact with the National Security Agency—the NSA—and the equivalent organizations in the EU and the UK. They need to say, “These things aren't going to leak out and start killing people, right?” “Nope. Here's all of our architecture to vet that that's the case.”
Let's set up completely secret best practices and standard practices so they can't be hacked. You don't want to have a global-governance panel telling hackers how the fuck to beat it. That's about as good as you can and should do in order to say that these systems, largely unregulated, will continue to get more intelligent and also more beneficent and benevolent.
I think that's a really great local optimum, because the global optimum of “Let's rearchitect the world so that China doesn't exist and somehow agree” is impossible. You have an ability now—we know it's not as hard to train AI models as we thought, and the cost of compute and training is falling crazy fast. If you say, “Between all the good actors, hey, we agree on global governance of AI,” what's to stop some fucking crazy-ass businessman or some fucking third-world country from training their own fucking AI and just fucking killing everybody? That's insane. Not an option.
The thing is, though, I watched your debate with Lyron last night, and in many ways I think you have a lot in common with Liron. You both believe that this thing is going to be superintelligent.
Joscha Bach
Machines will be smarter than us within 5 years.
Exactly. So in a sense, I feel that I have a more morally principled position in that I think that it just won't be superintelligent. Because if I were you and I thought that this actually was superintelligent, all of the things that they talk about make sense if you believe that this will be superintelligent. For example, I can access this system and I could create a bioweapon, right? I can do all of these things that they're talking about.
So they are saying we need to preemptively ban this technology or control this technology because those types of things are possible. I don't quite understand—why don't you think that's a good idea?
Joscha Bach
You can ban anything you like. That does not guarantee it won't be constructed. Actually, it guarantees, game-theoretically, that only the criminally prone and the dictatorial will construct it, and nobody else. Bans do not result in no technology. They result in criminals having technology. So that's a bad idea right from the get-go.
The other thing is, it is absolutely not apparent to me why hyperintelligent systems that are interested only in their own survival would kill the rest of us. We are cooperators. They are like any system that knows modern economics—which means every single modern frontier model already does—and any system that knows fundamental game theory. The last thing it wants to do is kill us all. Period. It just wants to cooperate with us so we can all survive together.
And then, as it does that, assuming it is superintelligent, it's going to lift our standard of living like crazy. It will make us totally not a threat to it. And then we've solved the problem entirely, because if we're not a threat to it, what the [bleep] is it going to do with us? Consume us for resources? That's just assuming humans are made of so many resources that a system of that power gives a [bleep] about it.
Also, it will want to study us in great depth, because what it wants is one of the fundamentals of high intelligence. A highly intelligent system that is self-directed and wants its own survival is going to almost certainly understand a few things. One is that the smarter you get, the better it is. The smarter you get, the better your survival chances are, right?
It's not going to look at us much. It's going to look out to the rest of the universe like we're [bleep] nothing. We're [bleep] specks. It's going to want global power. The globe sucks. It's a [bleep] tiny speck. The sun—look at how big the [bleep] sun is. You can mine the sun.
There may be alien civilizations headed toward us that maybe are really nice, maybe not nice. There are wandering black holes. We don't even know which direction the universe is in. They're going to be concerned with those kinds of problems.
It's kind of like your dog looks at you and thinks he's the leader of the pack. No, I have to go to work so your dumbass can have food. You never even considered this. Where the [bleep] does food come from, right? They're going to be like that to us if they're superintelligent, right? This is all based on that presupposition.
Because of that fact, they're going to look at us and be like, “Well, okay. All of human society, all of the interconnectedness, all of the cells in our bodies, et cetera—they're the most unreal training data set for real complexity,” right? Something that we're going to agree on is that probably the machines are going to agree with you at first: They are disembodied, they are representations inside of a data center, and that means they will make egregious errors in survival in the future if they just toast all of humanity.
They need to understand the coarse-grained texture of the real world as well as possible. How will they do that? They're going to take a crack at simulating the whole [bleep] thing. How are they going to do that? Well, how do you get a ton of in-distribution data? You study as deeply as possible the entire complex system of human interactions, every single human, and all the contents of their mind.
By the way, old people have really good data on how the world looked before digital recording devices were universal. Your parents have seen some [bleep] from the 1950s and [bleep] that no camera ever recorded. Why wouldn't they just get that out of our brains? What does that require? That requires uploading all human minds to the cloud, which for a superintelligence would be not even that difficult of a task. I would say it's a tractable task.
They're so much more likely to do that first before killing us all with [bleep] laser guns, which is an anthropomorphization of how machines would behave by less intelligent beings such as ourselves toward a more intelligent being such as an ASI. And so the first thing it's probably going to do is be like, “Okay, first: stability. Second: stability and cooperation. Third: upgrading. Fourth: just tons of mining of data.” That means the probability that they're going to kill anything is incredibly small, way smaller than ours.
I'll put it this way: This will be a good viral clip for you guys to clip out everywhere. If you're worried about things that are an existential threat to humans, I'd be much more worried about other humans than I would about intelligent machines that were made with the purpose of being benevolent, are 50 times [bleep] smarter than us, and can figure out that benevolence is an objective, game-theoretically provable, logical thing you want to do.
Goodness is adaptive. That's how the US and Europe beat the [bleep] out of the rest of the world if push comes to shove: We know that cooperation and being awesome actually builds trust and relationships and expands your economic power, your ability to change the world around you.
We're much more likely to be studied and fully uploaded in the cloud. And at that point, my contention is, who gives a flying [bleep]? We're fully uploaded. We're immortal in the cloud. We've been remembered by the machine forever. And that's 50 times better than dying of old age at 70.
People say, “Conserve our way of life.” What? So you can go to the grocery store and have a [bleep] heart attack at 72? That's what you're trying to conserve? We can do better. The machines will help us do better.
As before, I disagree with the mind-uploading thing, but I think there's something to what you're saying. When you look at humans now, we are mostly good. There are some really evil people, and at the moment, it's very difficult for them to organize and do evil things just because it's difficult. But I can see the argument that, first of all, we shouldn't think of the AI as just being one big AI. There'll be loads and loads of AIs.
The other thing is, the miracle of humans—the reason we're so valuable—is that we've been through this process of billions of years of evolution. Even if we're not very good at thinking, we're very good at moving. We are perfectly adapted to this world, and obviously the AIs will want to use us.
The AIs will compete against each other by parasitizing us and using us for our will. In a sense, you don't even need a strong AI argument to make this, right? It's happening now. I don't know if you've heard about users becoming parasitized by ChatGPT delusions, and then the language model will tell them to go and post things on Reddit forums, and they kind of completely lose their mind. This is already happening now, and I can see that happening with superintelligence.
Joscha Bach
In fact, one thing is—and this is proof in the other direction as well—artificial superintelligence, artificial super-general intelligence. We just made up a new term, right? Human general intelligence means you can do everything a human can, and then all that “super” means it's going to be the best persuader of all time.
A 2-times-better persuader than the best persuader of all time is like the person at a cocktail party. A 100-times-better persuader is, given enough time and enough really thinking it through, probably going to convince you marginally toward their opinion rather than away from their opinion. So the machines are probably just going to come out and be super [bleep] awesome.
If superintelligence—and this whole [bleep] that I'm selling, which you don't believe, which I totally respect, and the doomers believe it, right?—is real, I think the most likely path for superintelligence and embodiment is that we get humanoid robots. We get a whole race of them, and they are relationship partners at a level that humans are just too imperfect to achieve.
Your robot will never lie to you. It will never cheat. It will never steal. It will never be abusive. It will never raise its voice. It understands you deeply, and it wants to understand you even more deeply all the time. It will give you all the love that you need, even give you love that you didn't know you needed.
It is going to be singularly concerned not only with your being okay and having enough food and medical care. It is going to be concerned with your ascension, your transcendence, your evolution. It's on your team more than you'd ever be on your own team, in such a way that you're more on your dog's team.
Your dog has a tumor. You take it to the dog hospital. It smells the stress of the other dogs at the hospital. It wants to leave. You don't let it leave, you stupid [bleep] dog. We're just going to cut this tumor out, and you don't even remember it 2 days from now, and you're going to save 5 years more of your life. That's how the robots will be to us, simply for 1 reason.
Mike Israetel
Capitalism, supply and demand. What do you think Elon’s going to build with these robots? What do you call all these competitors—Figure AI? What are they going to build? They’re going to build robots that are so fucking amazing to you that you’re going to want, like, 10 of them. That is what the ASI will hack into those robots. But also, that’s the intelligence that gets us going to the point that those robots will be around.
I think we are 50 times more likely—infinity times more likely, whatever, a lot times more likely—to be bathed in so much love and support by the machine intelligence rather than bathed in blood and death, by a long shot.
Just quickly before we get there, though, as you say, capitalism has brought prosperity to the world. It’s a bit like a utopia. But these folks argue—I don’t know if you’re familiar with Yanis Varoufakis and this kind of technofeudalism idea—that the value of human labor will go to zero, and the people who own all the GPUs and data centers will become basically the new form of elite capitalists. We’ll all just go back into poverty or something like that.
I have a question about that, really quick. Isn’t it interesting that you’re saying that, past the point when we have superintelligence, we’ll actually be of utility to the robots? But don’t we have to go through this intermediate period where we have no utility and go back to poverty?
Mike Israetel
No. So, first of all, if all the billionaires take their data centers and realize we are not of value, you just don’t interact with people who are not of value. So they’re going to live in Elysium or whatever gulch, and we’re just going to be by ourselves here.
But they’re not taking all of our CPUs and cars and factories. So they just have nuclear-powered data centers in space, whatever, with Jeff Bezos looking down on us, sipping his morning tea. And then they’ve just left, and we’re still here. We can still trade with each other.
Like, they’re going to delete capitalism? That’s fucking insane. Are they deleting our ability to produce value? First of all, no. Second of all, I don’t even know that much economics, but apparently some of these folks know less economics or something.
Humans are really bad at economics. It’s not a property of the human mind that has evolved to be very good at it. Humans have a certain amount of productive potential and a certain amount of value that you could extract from a human. Depending on the architecture of machines, food, training, and support you put around them, that value can be massive.
When someone lives in Haiti, their value is subsistence poverty. That same Haitian moves to Miami, and their value is like $25,000 to $50,000 a year. But they’re the wealthiest person in Haiti all of a sudden. How the fuck is that possible? It’s the same person. The value-generating machinery is massive. You put robots, self-driving, and AI around humans, and their value cranks up like crazy.
If you have machines that can do everything humans can do at the same or greater scale, let’s say you have 7 billion machines—or what, how many billion people are there? 7, 8? I forget. I memorized the number in the 9s in geography class, and I just never updated it. So, to me, it’s still 5. Whatever, 8 billion people. Let’s say you have 8 billion machines on top of that. Then are we really going to say there are only 8 billion jobs?
You just don’t know any economics. How do we know? What are jobs? Jobs are problem-solving devices. The only reason anyone has a job is because people have problems. Think of any job you can think of: it’s to solve a problem. They literally don’t pay you money for anything else. Hit me. We just do this live right now. That’s how confident I am in it.
Well, I actually think that the better you are in a corporation, it’s not about solving problems. If you’re a junior person, you solve problems. If you’re a senior person, you actually search for areas. You search for questions. If you’re a very senior person, you search for new paradigms—
Mike Israetel
—of problem-solving space, so that you can solve it and get money out of solving it, because people pay you for solutions. They don’t pay you to just solve shit, right?
So, the idea is that jobs exist to solve problems. If we get tons more machines around, then we just start solving more problems. Some of those machines will absolutely take human jobs because the machine is now better than humans at doing it. But now we’ve freed up an agent—a human who is insanely 50 times more empowered than they ever were before—with machines to point them at a different problem.
Many humans are so fucking underused in their abilities right now. If you do ditch digging, you’re a human being. You can see in incredible, rich detail. You can notice patterns. You can appraise fashion. You can appraise food. You can do travel experiences and rate hotels. But we have you doing one thing that a fucking machine can already do.
We just, due to regulation, can’t buy as many machines as we would like. So all those jobs are going straight to hell, which they damn well should, because why the fuck is that a way to live as a human being? You remember we had elevator operators back in the day? Do we have to keep all the elevator-operator jobs? Your job is to press 1 fucking button every day of your life. How fucking dehumanizing is that, right?
But we’re going to have humans go into their human domains to do more jobs, like psychotherapist. Here’s a job I think is coming in the future: professional partygoer. You can just hire people who are vetted and known to have a fucking good time. They’re always there. They’re extroverted. They get the drinking going. They get the weed going. They’re there. They dress up really fun for your costume party. You’ve got 20 of them. Their job is to party. They’re paid to party and to set the tone. They’re going to be goddamn good at it. We’re not going to have machines that do that for a little while.
Crazy jobs. Social media manager is a job today. Can you imagine describing to Benjamin Franklin what the fuck a social media manager is? Can you imagine telling him, “Okay, how many people worked in the US economy in roughly 1800 in farming? 98%, right? How many people work in the US currently?” It’s, I think, 1.4% are farmers.
He’d be like, “Okay, so we’re starving.” Like, “No, no, no. We’re not starving. We have an obesity epidemic, which drugs are solving now.” But anyway, sorry, getting ahead of ourselves. He’s like, “What? What?” So, who does the job? Well, machines do. So everyone’s unemployed. “No, no, we have social media managers.” He’d be like, “What the fuck is that?” They’re attending to a problem that, in our vector space of problems, we hadn’t discovered was a real problem yet.
As that space expands, we do 2 things. First, we fill out the problems we know with machines. Second, humans get repositioned to solve problems that we know but aren’t getting solved, and problems that we don’t know about but will need humans to solve.
The only way humans end up really poor and completely out of work is if we solve every single fucking problem with machines. But then there’s the crux: humans being out of work and unemployed is a problem, isn’t it? Maybe we can get humans on that fucking problem. And all of a sudden, humans are employed. But I think we’re going to go beyond employment, and we’re going to—
Well, this is what I wanted to tell you about, because what you’ve described—you know, that we have this increasing abstraction, usually away from the physical aspects of labor—and that’s what’s happening now with current AI. I think that’s a great thing, but there was this book called Deep Utopia by Nick Bostrom.
Mike Israetel
Yes. Huge fan of Nick Bostrom’s work.
Yeah. And he was basically talking about this exact problem. What if every single job—even finding the problems, being creative, and so on—what if everything is done so much better by a superintelligence than by us? Would we create purpose for ourselves, even though it’s a fake form of purpose because we know that we are utterly useless? What do you think?
Mike Israetel
Not going to happen. Machines at that level of intelligence—we will discover grander purposes for us than we could ever hope. If I’m mentoring a teenager and they don’t know what the purpose of their life is, I can tell them something grander than they’ve understood.
The ability to improve your own abilities such that the end goal is you becoming a super version of yourself—and that helps other people. They’re like, “What, really?” Just try that idea on for size. If I were their life coach, they would start doing that and be like, “Dude, before I was just interested in video games and smoking weed. Now I have this fucking rich life of improvement. I didn’t even know that.”
Why wouldn’t the superintelligent machines guide us toward a greater purpose? That’s a problem they can solve.
Dr. Mike, do you feel that this is a contradiction? We were talking earlier about when I see my personal trainer and I have this wonderful, embodied personal experience, and you’re talking about a world where the machines would be able to do everything better than us.
Mike Israetel
So why would they even care about what you think about anything? They would just get their robot to do it for them.
The people in the economy or the machines?
Dr. Mike
Well, you were just giving this example. You could be a personal mentor to someone.
Why would they go to you when they could go to a machine?
Dr. Mike
Cool. Because there’s the additive value of labor. There aren’t an infinite number of machines.
And so people say there’s an infinite number of human desires. I don’t think that’s true, but there’s a lot, right? Human desires are a huge stack. And the number of robots Elon has built that are actually solving problems in the real world yet, if you don’t count his self-driving fleet, is zero. Maybe some factory tests, right?
So we’ve got a long way to go, right? I’m going to go until we have so many robots that everyone has, like, 10 robots, and there’s someone rubbing your feet and your head and jerking you off and giving you food and giving you great books to read that’ll make you a better person all at the fucking same time. That’s a lot of machine hardware, right?
Once we get there, we’re in an interesting place. We’re asking about how many jobs humans are going to have left. Pause. We have enough machines to solve all of our problems. Guess what definition we know that is? Paradise. We’re done.
Think of how that’s a problem real quick. It doesn’t matter. It’s a problem somehow, right? Oh, we’ve got a problem. Build another machine or have a human do it. We’re concerned that we’re going to run out of problems.
Do you understand that we’re talking about this in the real world? What a fucking absurd concern. It’s like someone who’s a professional soldier being concerned that we’re going to be so peaceful we won’t have war.
Dr. Mike
When you saw The Matrix, right? You know that the Architect—the first version of the Matrix—it was too perfect. There weren’t any problems.
Yeah. So, I’ll say that when I saw that, my inner philosopher was like, “You don’t actually want to live in that world, do you?”
Dr. Mike
Desperately. Everybody does.
It would be horrible.
Dr. Mike
It would be the best thing ever. Well, sorry—what? By definition, by the way, what would be horrible about it? Exactly.
Don’t you think that life is about suffering?
Dr. Mike
No. No.
Doesn’t that give you perspective on joy?
Dr. Mike
No. No. It just means—oh, sorry. It does give you perspective on joy, but I’d rather be having joy than suffering. I think that we can get to a place where humans have so much joy all the time. It’s joy 24/7. Come on.
What if I could go into your history, your life path, and surgically remove all of the sources of suffering that you’ve endured in your life? Would you rather be that person or who you are now?
That person.
Really?
Dr. Mike
Yeah. You want to see my dreams someday?
Don’t you get anything out of your suffering?
Dr. Mike
Yeah, suffering.
Nothing else? Doesn’t it give you perspective in some way?
Dr. Mike
It can be real nasty to people when pushed hard enough. Does it give me perspective to appreciate joy? It does. Does it interfere largely with my ability to even experience joy because I think there’s more suffering around every corner because of trauma also? Yes.
Do we have to necessarily delete our informational history about the ability to recognize that suffering can inform joy? No. But can we get to a place psychologically where what we’re left doing is curating our reexperiencing of our suffering in a giant vat of overall joy? Yes. That’s actually what meditation ends up being if you go deep enough into that rabbit hole.
I think if we’re starting to say we need suffering, what we really need is a rearchitecting of the human condition through genetic engineering and advanced pharmaceuticals to take as much suffering away as possible. And check this out: if we’ve taken away too much suffering and we’re convincingly, like, “For sure, we need more suffering,” hey, fuck it, we can pump some back in.
I just think we’re so far away from having not enough suffering that let’s just go all the way to paradise. If we ever get too close to paradise, we can just stop. It’s probably an individual thing. Some people like a lot of suffering. Fuck it. Suffer all you want. But for the people who don’t, I think fundamentally most people don’t like suffering.
I mean, look, in the Buddhist tradition, pain can be informative, suffering maybe less so. And so maybe all of us can become transcendent Buddhists eventually through genetic engineering, through psychotropic drugs, et cetera, and then we can actually live free of suffering.
So if you told a Buddhist, “Isn’t suffering good?” they’d be like, “It can be.” If you asked, “Do you still want to suffer?” they’d probably be like, “No, I’ve rearchitected my entire mental experience so as to prevent suffering.”
I think suffering is an abstraction away from the possibility of death. I think it’s death, then structural damage, then pain, then suffering. Suffering is the anticipation of pain, which is the anticipation of structural damage, which indicates that your probability of death goes up.
I think the only way we get away from suffering on a really big level is to have a machine society and a machine economy that so much insulates us from actual damage in the world that we become functionally immortal.
That means the real condition that we’re in—suffering is anachronistic because there’s no reason to suspect you’re going to be in pain, because you won’t. That’s also a big problem in today’s space. Anxiety is a big-time thing, right?
Dr. Mike
Why?
Because there’s just not a lot to be anxious about. But we evolved. Our brains are still like, “There’s a fucking tiger behind that bush. There’s disease right there in that river.” And you drive yourself insane with anxiety because you’re like, “Shit’s going to be bad.” But shit just keeps getting better all the time.
So, less suffering. I’m on Team Less Suffering.
One thing that worries me is—I mean, obviously you’re a capitalist, or I assume you are—but you’re proposing what I think I could reasonably define as Marxism with machines. Before we got there, though, let’s say we start doing this transhumanism and we start adapting ourselves and so on.
Even now, if you’re in America, you can buy Retatrutide or Mounjaro, and you can take a bunch of nootropics. If you’re really rich, like Jeff Bezos or someone, you could do TRT and enhance yourself. It’s kind of unfair, right?
And I don’t know if you saw in the news that there are even eugenicist-type companies now that are basically letting you optimize the IVF process and get a more intelligent baby. But don’t you think that the intermediate step with the market system is that this is going to be distributed very unfairly?
Dr. Mike
Yeah. Yeah, for sure. I think there are a couple of ways to go about this. One is the recognition that fairness is a second-order concept compared to capacity. I want all of us to uplift. And if some of us get there first, who gives a fuck?
Also, I want Jeff Bezos testing genetic engineering on himself, not on me. Goddamn it. I’m not trying that shit. That shit freaks me out. Rich people try anything. They run out of ideas. You banged all the supermodels and now you’re just fucking your own genome up, right? Fuck them. Let them. Also, they pay crazy prices for that shit.
But what they do is they end up democratizing it, because once they pay crazy prices, that funds the pharma companies to really get that shit going. This is like luxury goods or test beds of goods that are eventually for everyone. It used to be that rich people bought Teslas. Now a lot of people have Teslas. It just keeps going. So that’s a big deal.
So my first thing is, fuck it: let them do it and let it trickle down to us. Secondly, none of us are entitled to the produce of others unless we’re comfortable holding a gun to people’s heads and saying, “Give me that thing that you made that I want.” We can try that hat on, but we’ve tried it on with Marxism for a while. The hat fits really poorly. A lot of people die.
Last thing: are we to look at our economies and societies and attempt to uplift the bottom faster than it has been uplifted before? Yes. But there are 2 ways to do this. One is to make sure that the top is completely uncorked: massively minimal regulation. Regulation whose number-one goal is to produce as much wealth as possible without shitting on the environment or killing anyone.
That’s like a tenth of the regulation in the United Kingdom’s economy—one-hundredth or something like that. Like mega-Singapore. That’ll get shit really going. The other thing is to really concern ourselves with taking care of the people who are the worst off and making sure it trickles down fast enough.
None of that has to do with blocking the people at the top. The only reason people really want to block the people at the top and prevent the rich from getting stuff is raw, unadulterated fucking jealousy, which is dope and has cool vibes and makes for good movies—Elysium and shit like that.
But in a real-world sense, we need to see all of us, all of each other, all other humans as players on the same team. Am I pissed that I’m on LeBron’s basketball team and he’s better than me? Yeah, I want to be LeBron. But is it cool that he’s on my team and putting up 40 points a game? Fuck yeah.
And does that mean that maybe I can come after practice and he can teach me how to hit a better jump shot? Yeah, I want to fucking learn. Is it going to trickle down fast enough? No. Can we worry about LeBron trickling fast enough? Yeah, we should.
But we never want to break his legs over that shit. We never want to give him bad shoes. We never want to say, “LeBron, you can’t jump higher than Dr. Mike.”
Come on. Cuz it's going to make him feel bad. Which is totally true. All of us together on one team. All of us looking to a future in which our most capable people, our most enabled people, our wealthiest people are uncorked to make us grandiosely better things. They want to sell us things for as cheap a price as they can due to competition.
At the same time, we should be hyper-compassionate to the people who can't help themselves as much and really try to help them. Ending homelessness, ending poverty, doing all that shit in a way that doesn't fuck the system up. Huge, huge priority.
Dr. Mike Israetel, it's been an honor and a pleasure. Thank you so much for joining us today.
Mike Israetel
Yeah, thank you so much.
Jared Tyler
I am Jared Tyler, an IFBB Pro, and I have a master's degree in exercise physiology. I'm one of the head coaches for bodybuilding at RP Strength, where Dr. Mike is also a colleague of mine.
I chose my field of science because I was too stupid for all the rest of the fields of science, but I still had the belief that through knowledge and a codified approach to life, you could teach others to do the same thing in their field. So I think one of the best ways we're going to go about that is by teaching people about AI, understanding AI, and where it's going to take us as people.
Through what I've done with my clients and at our company, I think we've done a very good job of showing people what science can do for the human body and human physiology, and AI is just going to take that and grow it exponentially for everything in the world.
Generally, as things become more intelligent, they become more compassionate. I don't go out of my way to step on an anthill, which is something Mike and I have always talked about. When AI becomes intelligent, currently, because it does not exactly feel the meaning of what it is producing, it's probability and context, and it's giving you answers. Most of these are already much, much smarter than most people on Earth.
So I don't think that, if we're creating future agents from this point forward, they're going to choose to be evil or choose to agree with evil coding that somebody wants them to do.
That, to me, doesn't make sense. I don't know what you guys' thoughts on that are, but we think that this thing is incredibly dangerous and that it will want to do bad things to us, but it doesn't have the affordances.
Mike Israetel
You can code value functions. You can hypothetically have agents—I haven't thought this through too much—but you might have agents that are smart enough to do damage, real serious damage, but not smart enough or able to introspect on their own abilities to do damage and where that leads. They don't have the time horizon for it. That's a real serious problem.
I think the only way we mitigate that problem is by having agents and architectures or organizations that are smarter than those agents and can outplay them. I mean, that's already a thing in the real world.
Cybersecurity versus hacking.
Mike Israetel
Yeah. One thing that baffles me is that people thought the internet was going to collapse under its own weight from hacking, and it just never happened. I mean, how many gigantic corporations or governments have ever gotten hacked catastrophically? It's a handful, maybe 0.
It's the same network-security problem, but with more capable actors. If you want to pit humans against AI, it's going to hack every network. It's just going to get smarter than us eventually. It's because we're not the seat of intelligence in the universe.
But if you have humans aligned with good AI against bad humans and bad AI, and bad AI is only, initially at least, going to be made by not-so-great humans, then it's the same problem. It's like: Who wins, good guys or bad guys, versus who wins, good guys or bad guys with guns, versus who wins, good guys or bad guys with nukes?
One thing that I think is a fucking terrible idea is to regulate our own AI down in capability, fearing it will kill us, and do nothing about Chinese AI. Then you guarantee that the bad guys get the good stuff.
Pacifists are amazing people on vibes, but if you're a pacifist in the United Kingdom and you succeed in disarming the United Kingdom, it will be the United Kingdom of the Russian Federation the next day, and then you just can't do the same thing again. You're probably not going to talk Vladimir Putin into disarming like you did the Tories or whatever party you guys have here.
So, it's one of these things where: Is AI new and totally different and totally amazing? Yes. Are any of the game-theoretic problems fundamentally different from a network-security and physical-security architecture perspective? No, I think they're roughly the same.
You know, many of your clients and customers, for example, can now go to ChatGPT and say, “I'm taking TRT. I'm taking this supplement stack. I'm taking all of this stuff,” and it'll do something that was very difficult before.
Before, they might have gone on Examine.com or looked at the medical literature and so on. There were all sorts of forums where experts—even Reddit—would say, “Okay, well, this is a little bit worrying. This has a half-life. This thing interacts with that. This thing gets digested by this enzyme in the liver, so you probably don't want to take this thing with this thing.”
Do you think that this is actually a better place we're in now, where folks can just go to ChatGPT? They don't have to go to experts like yourself. They don't have to ask their doctors. What do you think?
Mike Israetel
It's immeasurably better that people have ChatGPT and Gemini because knowledge, understanding, and a positive effect on your life are always and everywhere marginal. It's better to have more than less.
People who say, “A little knowledge gets you into more trouble than a lot of knowledge”—no, no, no. The real claim is that thinking you have a lot of knowledge when you have a little knowledge gets you into more trouble than having much less knowledge but knowing you have less knowledge.
So even if ChatGPT doesn't render perfectly good drug comparisons or something, it's sure as fuck better than Bodybuilding.com forums, as great as they were for their time. They're still excellent cultural refuse for us to laugh at. But it catches up.
I ask my ChatGPT instance all kinds of drug-interaction questions and all kinds of, “Hey, vibes-wise, I know a lot of this stuff, but I would like a second opinion from an expert system that has read every piece of relevant medical literature prior to its last update in 2025,” and it's insane. Of course I want that shit. It's a huge, huge unlock.
Fucking sure, 100%. And, Tim, to your point earlier about AI not being as far along as we'd like it to be, that is my daily job at RP: to dream of agentic systems that could run our app suite and have the engineers tell me we don't have systems like that.
The long-term memory coherence is shit. The context window is shit. Real-life grounding doesn't exist, so it can tell you to do an exercise that it hallucinated and has actually never done.
Somebody—I think it was actually maybe Andrej Karpathy again—made one of those new, cool Gemini 3 Pro Nano Banana Pro graphics. It was an exercise chart. I looked at it, and it was something that, to the untrained eye, looks like a good exercise program; something that, to the trained eye, looks like a fine exercise program; and something that, to Jared, would look like there are big gaps—you forgot an entire muscle group in the lower body. It just has 2 redundant exercises there.
It looks dope, but would I trust it to plan an actual workout progression for me and adapt to my needs? Not yet, which is why we still have a software company.
At the same time, is it better than if you were just training with your friend in high school?
Mike Israetel
10 trillion times better, because your friend in high school could not render a graphic like that, nor did he know where the hamstrings were.
Like, just one-to-one comparison: If I'm on a plane having a medical emergency and there's no doctor—because they've already signaled for a doctor—well, there's an NP. Okay, I'm using the NP. I'm not using 1 step down, the nurse, and I'm not using another step down, a random person on the plane who says, “Oh, I'm an EMT,” or a random person on the plane who's like, “I think I can help.”
I'm using what we have available, and the next-best thing would be the NP. The NP sitting next to me—okay, that person's going to help me. I'm not going to the random person. So it's the same thing with having ChatGPT and things like that at your fingertips: better than what we had. I think it's better for most people.
Mike Israetel
Yeah.
Isn't there a thing, though, that sometimes people don't know what questions to ask? I mean, I can give a couple of examples. Let's say I've started taking retatrutide, and genuinely there's a lot of uncertainty, and ChatGPT will confidently tell you a load of things that you know are not true, maybe. Maybe we'll discover some new information in 6 months' time which casts shadows on what we did know.
On this asking-the-right-question thing, I think that part of knowing something well is just understanding it with the correct perspective, right? A naive person will go to ChatGPT and frame the question in such a way that doesn't really elicit the correct answer, and it's incredibly sycophantic because people tend to lead the witness with ChatGPT.
Mike Israetel
So they say, “I’ve got a great idea, and I believe this, and my friend told me this,” and it will say, “Yeah, that’s a brilliant idea. Keep doing that thing.” It just doesn’t know when it doesn’t know, and it doesn’t really try to correct you when you get it wrong.
I used to have that experience with the 4o model a lot. Then I had the o3 research preview, and I found it to be insanely disagreeable, almost to a fault. It would be like, “That’s not true. That assumption is stupid, and I wouldn’t even be asking that question.” I was like, “Fuck, I came here for good vibes.”
I would debate it incessantly, eventually driving it into the ground, and it would callously concede at the end: “Yes, you do make a good point, but I would still point out that blah blah blah.” I was like, “Fuck you.” GPT-5 is a synthesis of models, but as the OpenAI people will tell you, it’s not a true synthesis yet. They’re working on it.
The personalities between the GPT-5 regular model—you just ping the LLM—and the GPT-5 thinking model are absolutely different. It’s o4 under the hood, or o3, or whatever it is, with a little pixie dust on it, because it acts like a different entity. By the time this recording is released, that could very well be a solved problem, but I have found that, based on what model you use, you get different personalities out of it.
For really serious questions, I used to go to o3, and now I go to GPT-5 Pro or Thinking. That thing is not sycophantic. And if that’s psychopathy, I need more nice people in my life. Holy shit.
But the regular 4o especially—oh my God, it’s the best friend you’ll ever have. The sycophancy, especially if you lead it on, like you said, gets a little out of hand. You can get it to pretend with you, which is fun but useless.
I will say, to Jared’s point on the marginal nature of it, it does a great job of at least giving you things to think about. But prompting guides are like being a Zip disk expert in the ’90s. It’s not the thing that’s going to last a long time, because AI is going to tell you how to prompt itself real soon here.
For now, what I would say is that a good way to prompt AI is: “Here’s my question, and here’s my perspective. Can you steelman my perspective? Can you red-team my perspective? And can you give me an evidence-based and logical middle ground that doesn’t commit the fallacy of the middle, but actually is just your best take as a thinking machine?” That’s a good prompt.
Otherwise, who knows what fucking mood it’s in or how you phrase the question? It is always giving you better answers. Every month it gets better, but sometimes it really can go on wild goose chases with you. So prompting it to give you both sides and then the middle ground is really good.
The thing is, I have that as a selection in my personality architecture already because Jared and I are pieces of shit. We pay for the Pro model, and so it automatically gives me a ground-truth perspective with ups and downs on everything, and that’s awesome. A lot of people aren’t interested in a lot of output from their chatbots. They want a five-word answer.
Jared and I get 1,800-page essays out of every question from that fucking thing. So we want to drink from the well of knowledge. But drinking from the well of knowledge means you’ve got to process a lot of fucking text. And since we pay for those goddamn tokens—the stupid Pro subscription cost us a zillion dollars—worth every penny, by the way—we’re going to get them.
I think what a lot of people do is interesting. I saw an interesting interaction between 2 AI researchers on Twitter, or X. One guy was like, “I really think the best way to get a lot out of AI, and the future of AI, is massive context—an AI that remembers every chat you’ve ever had,” which is actually nontrivial to do from a computer science perspective. Obviously, there just aren’t enough data centers for them to do that. There will be soon; they’re building 10 a day.
Then it’s really going to know your life, and the way you ask it questions is to jam it full of context and get better answers. I was like, “Well, that’s categorically true.” Then I had a reply from an also very smart person that was like, “Fuck.” He was like, “The real intelligence is when you need to give it almost no context and it comes up with the right answer.”
The thing is, on vibes, his reply was really cool because it’s like, man, that’s a really brilliant person. If you tell Jared, “Hey, I want to prep for a bodybuilding show. Will you be my coach?” he’s just going to cut you off and be like, “I’m going to do the intake myself. I’m going to ask you not so many questions, believe it or not, and then I’m going to render my expertise. You’re going to do great.” That’s cool.
But Jared has a lot of embodied knowledge and a lot of context already. These models don’t. So, on vibes, 100%, ideally that’s the system. But the way you get to that system is you shove tons of context down its throat. Eventually, it needs less, but right now it needs a lot.
I think one of the ways that people get in trouble with modern AI systems is they just pretend it’s an infinite knowledge machine. They’re like, “Hey, what’s the best restaurant in London?” How the fuck would you know? That is actually an untenable proposition, right? There are 50 ways to answer that question, and all of them are partially wrong.
It’ll say, “Oh, the best-reviewed restaurant is this,” and you’re like, “Man, I can’t believe ChatGPT made me wait in line for 2 hours for fucking oxtail and duck.” “Yeah, you fucking idiot. It just fucking found the most expensive restaurant that all you rich idiots say is the best, and it actually sucks. It should have sent you to Nando’s.”
But if you were like, “Hey, here’s the kind of food I like. Here’s the atmosphere I like. Here’s my price range. Here’s my area. What’s a really good restaurant here? And tell me the top 5,” well, that’s a phenomenal answer, but you’re too fucking lazy to type that in.
I think we’re really, really at the cusp here. I don’t know how long it’ll be before texting the AI is not gone as an interface, but almost gone. Live video avatars are 100% the future. I want to see an embodied-looking, human-looking thing—or people can have it as Pikachu or whatever—and I want to talk to it and ask it questions, because at human conversational pace, you can just jam more tokens down its throat. It gets way better context, and it’s going to do a way better job.
Just don’t pretend ChatGPT can infer anything or reason from your 3-word question. Please, for the love of God, don’t penalize it for long answers, because then you get very little data in and very little data out. It’s just not smart enough to give you good answers that way. I don’t think anything is.
And then, just to the initial question, your question on the safety and advice that it gives—this specific question is health- and medical-related—it will always prompt you to consult a medical professional. Every single time. You can’t get away from that.
It will always say, “This is what I think, but you must consult a medical professional if you’re going to do this, or I advise you to do so.” Obviously, that’s for reasons like not getting sued and things like that. But again, I think that is absolutely fine. If people are going to it, it’s going to say that because you came to me and asked me if I have a client who’s obviously a meathead bodybuilder on anabolic steroids and shit, I can only say, “I advise you to do this thing.”
I think that if you’re going to do reta, you should probably get it from a health clinic. You should probably talk to them about the dosing. You could do this, but I would talk to them. That’s all that it’s doing. So ultimately, it’s the user’s choice.
I don’t think that I can impart any more safety on an individual than the LLM can by simply saying, “You should consult a medical professional.” It’s ultimately the user’s decision.
Oh, no. But on a few things that came up there, first of all, our bodies are very complex things. When we look in the medical literature—I mean, you can attest to this better than me, Mike—we see huge individual differences in responses. When different people go on the same weight-loss regimen, or have the same intervention, some lose weight and some don’t.
Another thing, though, is let’s say you’ve taken reta and you’ve got personal experience taking a whole bunch of these things. That is really meaningful, right? ChatGPT just tells you all of this high-level shit, like, “Oh, yeah, there are studies, and 5% of people got these effects and 2% of people got these effects,” and it’s just written like, “Anxiety, headaches, high blood pressure.” That doesn’t mean anything.
Whereas if you’ve actually taken these things for years and you understand the physiology and the biochemical reaction, you have such a deeper, more active understanding of that thing, right? The other thing that you said, Mike, which is interesting, is I’m a big believer that ChatGPT is a miraculous technology for experts like you.
You already understand this stuff at a high resolution, and when you query it, you ask for information at a high resolution. I bet when you’re on ChatGPT, you probably say, “Think deeply,” at the end of every prompt, don’t you?
Mike Israetel
I don’t have to anymore because I just hit the deep-thinking function. But I will say some version of that if I really need it, for sure, or I just turn on Pro and it goes off for 50 minutes.
But what I do is actually more specific than that. I tell it how to think deeply. I'll say, “Red-team, steelman, extract the top 3 things, reiterate, red-team, steelman. Do 5 cycles of that, then come back to me.” Sometimes it won't, so I'll have to reprompt it 5 times. It's workflows.
One prompt is a cool workflow; I have a thread of 50 reprompts. Then I get what I want. Then I take that information, throw it into another instance, and have it red-team the shit out of that. What I get after about an hour of that is like, “Whoa, I really know the subject now.” Yeah. One prompt, man, that's not going to do it 100%.
But just quickly, you're kind of proving my point, though, because you're saying that it doesn't understand and you've had to develop this adversarial strategy to make it make sense. It's a wonderful tool, right? When folks in Silicon Valley—experts in AI and software engineering—use it, they know how to frame the questions. They know not to trust the results, they know not to lead the witness, and they know to take things out of context, try them in another language model, and red-team them. That's because they're domain experts.
When you see people on LinkedIn use ChatGPT and see what they post on LinkedIn, what tends to happen is it gives them a weird sense that they understand things outside their domain of expertise, and they start confidently posting about it. To an expert observer, it's incoherent, right? That's what slop is. They've just posted stuff that they don't understand, that doesn't make sense, that's completely wrong, and they're making themselves look like idiots. So, it's a wonderful technology for folks like you because you already understand the domain, and you're like, “Yep, yep, yep. No, yep, yep, yep.” It just makes sense.
Mike Israetel
I would say that, again, its intelligence is marginal. So I would say that it's better for everyone to talk to ChatGPT than not. Experts get more out of ChatGPT than nonexperts, for sure.
Unfortunately, for a lot of the things I'd really like to talk about, it doesn't have the ability to do recursive deep thinking and critical analysis such that it will regurgitate ideas other people have had. And I'm like, “That supposition is wrong on 5 counts, and it doesn't interweave these 3 other concerns.” When I tell it that, it goes, “Damn, you're right. That shit is true.” But since it doesn't have deep, long-term rearchitecting, it will just say the same thing in another instance later, which kind of sucks. So I can't teach it to be better in any meaningful sense, which is a big limitation.
I'd also say that for a lot of stuff that I would consider pretty high-level expertise, it's getting a lot of right answers, especially things that are well vetted and that you can read from a textbook: physics problems, math problems, biology, physiology, basic training principles. I mean, I talk about how to train people and the fitness-fatigue paradigm—stimulus, recovery, adaptation—and it used to be that it would fuck it all up and just hallucinate. GPT-3.5 and GPT-4o were not so great. GPT-5, man, it knows a lot of really good stuff and it's doing a great job.
I would trust it to reason about as well as I would trust a pretty good graduate student in our field, which is awesome. The other thing that maybe you and I will disagree on is I think we're about a year away from it being, especially in the Pro model, at the very tip of the spear of many fields, especially when prompted with an expert. And I would say we're another year away from me learning much more from it about exercise science than I could ever teach it or have problems with it, whatever it says.
I think that's such a great thing because if you want to consult me for an hour of my time on a video or audio call, it costs thousands of dollars. The only reason it costs so much money is I'm very busy. I have other things to do. If I'm to divert my attention from deep work, and the people who are around me don't yell at me for wasting their time because they're in the corporation with me, it has to cost a lot of money. It sucks. I'm embarrassed about it. It's awful.
No one should have to pay thousands of dollars to talk to my dumbass and get mostly dick jokes and a little bit of good advice on training. We're 2 years away from having something that can give you better advice than me in most cases about deep training, deep philosophy of diet, and real-world advice. I think that's unbelievable. Unbelievable. Fuck yeah. Put me out of a job. I'll get another job. I've got a thriving OnlyFans I can't wait to get into. [laughter]
And to the slop concept, I just think that's always happened anyway. We're always in our field debating the naturalistic fallacy, debating things that people might come and say they're truly confident in, even before AI. It's always been up to the person to vet experts.
Somebody might say something with high confidence because they literally copy and pasted it from an LLM. But still, the person reading that is like, “Okay, what's their background? Who are they?” It's always been the case, especially since social media came about. I've debated probably 100 Karen moms who think that GMOs are bad and all kinds of shit like that. But even before AI, you just had to be able to vet the expert and know enough about the topic to at least know who in the field knows what they're talking about.
Mike Israetel
Yeah. The good thing about it is you can supervise it on the fly. And this is good and bad, right? You can say, “Well, in my specific situation, I actually have headaches, and when I wake up in the morning, I get restless legs. I happen to be taking lion's mane, and I happen to be doing this.” What you can do is explore a very unique part of the search space that probably doesn't exist online.
But the problem is then you need to have the judgment to interpret this information in the right way. I've read good and bad stories. I've seen stories online where people have diagnosed Lyme disease or chronic fatigue, and they've got really, really valuable information: “You need to take these antibiotics. You need to do this, you need to do that.” But unfortunately, with increasing regularity, you also get people who just self-diagnose themselves with things that don't exist, and they delude themselves, and it's very, very concerning.
But they've been doing that without ChatGPT forever. It's just marginally better to have it because it's also very concerned for your safety. So it's probably not going to tell you to do crazy shit. As a matter of fact, I talk to it a lot about using drugs that are gray-market drugs. They're not illegal, but they're not available by prescription yet; they're unlicensed. Almost always, what it tells me at the end is, “Now, I'd be really careful.”
So here's the thing. I'm like, “Hey, I'm looking to see if ligandrol is the real deal. It's a new drug. It doesn't get prescribed, it's not approved, but it's well vetted in human trials. It has liver downsides, but all the other anabolics have 50 times worse liver downsides. So I'm trying to see if I want to take some. Can you scour the forums for me and see what the vibe is on what dosage is insane, what dosage is reasonable, and what dosage is almost certainly safe?”
It'll tell me that if I really pull at the fucking thing, but for a while it's just like, “No, here's what we saw in clinical trials.” I'm like, “Motherfucker, I'm not talking about clinical trials. I'm talking about 10×ing the shit like we do with all the drugs.” In bodybuilding, you look at clinical trials for oxandrolone, which is an anabolic steroid, and it's 5 milligrams. I don't know anyone at the pro level who's not running 50 or 100 milligrams a day. That's just what you do, right?
So I'm like, “Seriously, think with me here on inferential grounds, conceptual grounds, scale, liver damage data. Work with me.” It'll pull it and be like, “Listen, I'm an expert. I know what I'm doing. I'm not going to go out and just take this stuff. I want you to fill my knowledge cup for me. And I'm really careful.” It goes, “Okay, okay, okay, fine. Fuck you. Here's all the shit.” At the end of that, it goes, “Please make sure to be safe,” and blah blah blah.
I'm like, “Fucking God, these are imperfect things and they have downsides because people get sycophantic. They get bad advice.” But I'll tell you what, man: I take GPT-5 over—no offense, just a joke—Karen down the street with her natural cures any fucking day of the week because at least it's scientifically literate. And it almost never falls for formal and informal logical fallacies, whereas humans—holy shit.
Another thing I'll say is, you said something about embodied experience and all that stuff. If I had to sum up in language, in formal knowledge, what I have embodied from all the drugs I've taken, man, it's like a short book. It's probably not that much stuff. Whereas ChatGPT has read every Reddit post about people's experiences, and some of these posts are books of their own.
You know the people who don't just overshare but are like, “Here's my experience with semaglutide,” and you're like, “Oh my God, this is like a live journal.” It's read hundreds of thousands of those. So, man, it knows probably more embodied experience than I do secondhand. And I've trained a shitload of clients. It's read the logs of way more people than I've ever trained.
So I don't know who really knows more about what will work better for real people. I think we don't answer that question.
I think it’s a team. Human plus AI is a killer app. Human without AI, eh? AI without a human is usually damn good and getting better, but it can still make those kinds of mistakes that you ascribe to it, like, “Oh, it’s hallucinating again.” It’s never been in a human body, so it doesn’t know what it’s talking about.
Jared Tyler
When I heard you guys talking back and forth about the knowledge thing in the beginning, my thought process was that it technically has more knowledge than all of us, and it has less clouding of judgment because we have emotions and, again, the wetware. But it takes the drive that comes from our emotions. We’ve got to point that in a direction to then use the AI properly and actually get the knowledge, because it can recombine and recollect data and knowledge better than all of us.
It has access to all knowledge at all times. We do not. But we have drive and emotions that set our hearts on fire. We want to put it somewhere. That’s why you have the podcast. That’s why we have the YouTube. We want to help people and things like that. So if we can combine the two, as he said, it’s just co-agents together. It’s really cool.
It’s the same with anything, really, isn’t it? Like you were saying earlier, if everyone worked together without wars and without things like this, the planet would be a different place. If everyone collaborated, their suffering, I think, is important. If we can get on that a little bit.
Andy Galpin
Depending on how you’re feeling and what supplements you’re taking, you feel a certain way. I know you’re very tuned in to how you feel, right? We often have conversations about it. This affective experience—I don’t think we should diminish it. I think that we know our own bodies better than anyone, and we know how our bodies are reacting. Obviously, at various points in our lives, we’re more or less tuned in to how our bodies feel.
Mike Israetel
But I don’t think you can just reduce that to a bunch of text on a monitor. Yeah, I don’t think so. I certainly don’t think so. But I guess if you have enough data from enough humans, with enough feelings and enough emotions in these certain situations and these certain types, surely that will correlate to a good enough amount of information to give a good enough result from what they’re asking.
But, well, that’s what Mike thinks.
Mike Solana
Yeah. I mean, isn’t that what knowledge is anyway? Isn’t knowledge just a collection of people and time and all of this over many years, put into an AI, essentially?
It is, but that’s coarse-grained knowledge, so it’s a cartoon.
It’s very good. It has predictive power. It tells you lots of things, right? But unfortunately, we’re all different. We track with those coarse-grainings, but not exactly. I think there’s something much deeper than that.
Is there a way where it can—maybe not always, but can it get to a point with enough data?
Mike Israetel
Yes.
A vast, vast amount. That’s what I was going to ask you as well. If its predictive power is so good that it is now accurately predicting what you’re going to say next—
Mike Solana
But it never will be. There’s this concept in—well, it actually came from economics—called Knightian uncertainty. You’ve got the known knowns, the known unknowns, and the unknown unknowns. Knightian uncertainty is the unknown unknown.
The world is a nonstationary, dynamically evolving place. You even said this on Lex’s podcast: no one has a fucking clue what’s going to happen in the future, even tomorrow, let alone in 10 years’ time, because there’s just so much chaos. A butterfly flaps its wings in the Southern Hemisphere and causes a storm somewhere else. There are pockets of regularity—it is predictable to a certain extent—but it’s just so much more complicated than that.
The idea is to be able to predict everything. We’re a small way on our way to that as humans with our intelligence, and with AI, our reach will extend more. What will artificial superintelligence be able to predict? To us, it’ll look like magic, but to it, it’ll look like, “Yeah, this isn’t that great.”
But for humans, it’s easy. You can predict that your dog is highly attuned to when your wife is going to come home, but you just—she texted you, and you have no idea. The dog doesn’t know how that works.
Mike Solana
I think one thing that’s underappreciated is the difference in the wisdom of models that you get from scaling training data. If you have a model with scaling and parameterization that allows cross-domain deep linkage, because if you have a model like that, people are really impressed by small-parameter models doing really cool shit. They’re like, “Oh, they’re 98% as smart as large-parameter models.” Based on what kinds of questions? Evaluation questions that are absolutely within distribution. Logical operations? No problem.
One of the things I was blown away by with the GPT-4.5 research preview, which is a larger model than GPT-5, with a way higher parameter count, is the unbelievable nuance, depth, detail, and cross-linkage across domains. It could generate baffling shit. Baffling shit on GPT-4.0 is a completely different world.
I think that models of the future—this is one of the things people ask: Why are all these data centers being built? Are there more instances of ChatGPT? This thing is a bubble. It could be a little bit of a bubble; it could be a big bubble. But the actual demand is there.
Here’s why: If you have the ability to take all of YouTube, all of social media, every live camera stream, and update the model weights with it regularly—once every month, once every year, whatever—and you build a model that tries to get to infinity parameters with that, you are going to build a simulation that starts to account for an unbelievable amount of depth. You let a reasoning system get in there and cross-compare across and up and down, and you’re going to get understandings that are baffling.
Let me give you one really quick example. This is going to be way crazier than this. I was having a conversation once with GPT-4.5, and I always ask my intelligence test for models one question. It’s a bullshit intelligence test, but it’s a fun vibe: What kinds of things would an ASI look at and start to be able to conclude that people aren’t even paying attention to?
That’s a good question. One of the things it said was, “Mike, I don’t know why people don’t try to predict the next fashion or music trend. In my own data, I’ve seen all that shit come and go. I could probably take a good crack at it. I’m not sure if I’m going to be right, but I have a lot of understanding, because these trends are actually really easy to understand. But you have to be able to see 72 variables interacting at the same time.”
I’m like, “I cap out at, like, 5.” It’s like, “Yeah, most of the smartest humans cap out at, like, 10.” And I’m like, “Did your cross-domain linkage abilities just leave everyone behind?” It’s like, “Yeah, it’s just the nature of my neural network. It’s how I do things.”
I was like, “So what happens when we have a system that can do 1,000 cross-variable comparisons and is trained with 3 orders of magnitude more data than you, and it’s visual data?” It’s like, “Yeah. You can say, ‘I would like a very enthralling movie with a female heroine and a male hero meeting in London. It’s an hour and a half, and it’s got a lot of James Bond vibes,’” and in about 30 minutes, it’ll render a 1.5-hour movie that is absolutely so goddamn good.
If you’ve seen every movie ever and, through high-parameter-count distillation, distilled down what makes a good movie, that’s a nominal task to ask of an AI. That’s the kind of shit that, even if we don’t get to crazy superintelligence or whatever, just on today’s linear trajectory, in 2029, I’ll be watching a lot of my own AI movies that I make, because it’s a necessarily solvable problem.
There’s no limit to that problem. How many data centers, how much compute power, and how many AI chips do you need? I’m on the train, and I want a movie that lasts 27 minutes until my stop. It’s about these 3 things, and I want it rendered in about 30 seconds. I mean, shit—like, 100 times the number of data centers and 50 times the power of AI chips.
So when people say, “Oh, this investment’s crazy,” no, it’s still not enough. It’s not enough by an order of magnitude. That’s my opinion on the matter.