Machine Learning Street Talk · 2025-12-24 · 176 min

PhD Bodybuilder Predicts The Future of AI (97% Certain) [Dr. Mike Israetel]

Tim ScarfeDr. Mike IsraetelJared Feather

YouTube ↗

TL;DR

Israetel predicts visible artificial superintelligence in late 2026, escalating through 2027-29, while full AGI waits until roughly 2029-31. His apparent inversion rests on definitions: a system can outperform humans by orders of magnitude across perhaps two-thirds to 80% of cognition before it reproduces every human faculty, including taste, smell, and embodied work such as cooking. The host disputes almost every premise, warning against the “first-step fallacy” of extrapolating broad intelligence from spectacular but bounded capabilities.
The episode’s central question is whether models understand reality or merely compress its traces. The host argues that syntax is not semantics: knowledge is a path-dependent physical causal graph, and language works because humans share an embodied world. Israetel counters that brains are themselves lossy representational networks—“it’s abstractions all the way down”—and says prediction, problem-solving, multimodal perception, memory, and recursive reasoning can amount to understanding without duplicating biology.
The capex debate turns on diminishing returns versus a coming architectural unlock. The host sees frozen weights, expensive retraining, logarithmic performance gains, and recurring S-curves that require fundamental rewiring; Israetel sees solvable engineering problems, falling compute costs, richer world models, and hierarchical learning systems that update phones nightly, regional models monthly, and frontier models every six months. His conclusion on data centers and AI chips is categorical: “It’s still not enough—not enough by an order of magnitude.”
The exchange’s near-term calls are unusually aggressive: Israetel expects dependable digital and physical agents within 6-36 months, while the host separately forecasts routine autonomous driving after 2028 and claims drug discovery could reach a 99.9% chance of selecting the right drug from a render. The host keeps the bottlenecks in frame: clinical trials remain physical, present agents “roleplay agency,” reasoning traces can be post-hoc confabulations, and continual learning can destroy structured knowledge. Even Israetel concedes that his own fitness company cannot yet obtain the coherent long-memory agents it wants.
On AI safety, Israetel treats intelligent cooperation as the base case and geopolitical defeat as the larger tail risk. He rejects a superintelligent paperclip maximizer as “dumb as rocks,” arguing that a capable system would understand interdependence, preserve valuable human data, and prefer cooperation; consequently, he opposes broad bans that leave only criminals and dictators building frontier systems. The host’s pushback is that current software can still cause mundane, serious harm—through sycophancy, delusions, autonomous tool use, social manipulation, and protest-related escalation—without consciousness.
Israetel rejects both mass technological unemployment and technofeudalism because jobs are problem-solving devices, not a fixed inventory. Machines may eliminate specific occupations, but they also augment each worker and expose previously unaffordable problems; his examples run from farming and elevator operators to social-media managers and future “professional partygoers.” If machines eventually solve every problem, he says, unemployment is no longer deprivation: “We have enough machines to solve all of our problems. Guess what definition we know that is? Paradise.”
The practical present is less grandiose: human-plus-AI is the strongest product, especially when an expert supplies context, challenges outputs, and runs multi-step workflows. Israetel trusts GPT-5 Pro more than most people on many questions yet still favors red-teaming; Jared describes repeated steelman/red-team cycles across multiple prompts. The host warns that novices can turn sycophantic answers into confident slop or medical delusion. That gap creates near-term value for expert interfaces, supervision, memory, verification, and personalization even if the Matrix remains distant.
Israetel ultimately wants the Matrix and regards unequal early access as a tolerable stage on the way there. He wants genetics, pharmaceuticals, robotics, and machine abundance to remove suffering rather than romanticize it—“I’m on team less suffering”—while wealthy early adopters fund risky experimentation and cost curves. The host sees a contradiction between embodied human meaning and a world where machines outperform everyone, but Israetel’s answer is uncompromising: fairness is secondary to expanding capacity, then society should accelerate diffusion and protect those left behind.

Digest · the substance, structured for research

1. Superintelligence can arrive before machines reproduce every human faculty

Israetel’s deliberately contrarian timeline puts ASI in 2026-27 and AGI around 2029-31. AGI, in his usage, must encompass essentially every kind of human intelligence; a machine unable to smell, taste, or rank flavors like a chef has not reproduced general human capability, however strong its abstract reasoning may be.
ASI has a lower breadth requirement but a much higher vertical one. A system 100 times stronger at language, mathematics, science, spatial rotation, recursion, and world-modeling is “a fucking artificial superintelligence,” Israetel argues, even if it has never smelled food or viewed the world through biological eyes.
His rough threshold is dominance across perhaps two-thirds, 75%, or 80% of cognitive abilities, with machines categorically superior in those domains. Thus “super” can precede “general”: extraordinary competence across most consequential tasks arrives before complete replication of humanity’s unusual sensory and embodied tail.

2. The fruits of machine labor—not benchmarks—will make ASI undeniable

Israetel says GPT-5 is still weaker than him at linking distant concepts into a whole, partly because such integration burns tokens without serving most workflows. Yet he credits it with black-hole physics beyond his reach and claims frontier systems are already generating novel discoveries beyond individual scientists.
Knowledge alone looks superhuman to him: GPT-5 can retain vastly more factual information than any person and would overwhelm a human Jeopardy champion. He assumes OpenAI’s unreleased systems already “beat the shit out of GPT-5” and could be roughly 10 times more capable before post-training is finished.
The decisive proof will be real-world yield. If an AI emits a novel biological hypothesis every hour, achieves a 60%, 80%, or 90% hit rate, and produces new disease treatments weekly, semantic objections will lose practical force: “It understands the cell” because its output changes medicine.
Israetel compares unproven intelligence to someone claiming wealth but declining to buy a Dubai flight; the convincing demonstration is buying the airline. On his “CIA probability scale,” he assigns a 97-100% “extremely likely” chance that late 2026 opens a “cornucopia of machine intelligence,” followed by an increasingly wild 2027-29.

3. The host says syntax cannot inherit the semantics of embodied life

The host rejects the substitution of knowledge volume for intelligence. Intelligence, in his framing, accumulates information relevant to adaptation, while culture, Wikipedia, and language are products of that process—not equivalent to the experience that gave the symbols meaning.
An alien could read an article about America or Trump without possessing the connected, enacted experience to which those words point. That gap between syntax and semantics is the grounding problem: abstractions work between humans because both participants occupy a shared physical, social, and temporal world.
His mountain analogy makes path dependence load-bearing. Functionalists focus on reaching the summit—chess, reasoning, language—while ignoring both the route and “the stuff that the mountain is made out of.” Intelligence may be an emergent property of adaptive matter, comparable to temperature as a coarse-grained description of physical dynamics.

4. Israetel treats embodiment as another compressed data channel

Israetel prefers the stripped-down definition of intelligence as “the ability to solve problems.” Human brains contain no magical contact with reality, he argues; they are recursively interacting networks that receive transformed signals, build approximations, and compress the world just as machine systems do.
His sharpest counterexample is particle physics. No scientist directly perceives a neutrino or enters an active particle accelerator, yet an expert can model CERN well enough to configure machinery and predict measurements. If representational accuracy supports valid intervention, lack of direct sensory acquaintance does not cancel understanding.
The same test applies to ordinary knowledge. A travel adviser may accurately describe Shinjuku without visiting Tokyo, and a student scoring 98% can understand exercise science despite imperfect experience. “Predictively valid” internal structure—not mystical contact with the referent—is doing the useful work.
Embodiment still contributes the remaining nuance: a robot with vision might discover “the rest of that 2%.” Israetel’s point is proportionality, not zero value—if a model already predicts human behavior or physical outcomes at 98%, calling the entire achievement unreal because two embodied percentage points remain is an unreasonable grading standard.

5. Abstract tests expose both the strength and the limits of multimodal models

The host distinguishes factual knowledge, procedural knowledge, and conceptual knowledge, reserving “understanding” for the capacity to generate new knowledge by rearranging an abstract model. Shared material history lets humans perform this conceptual Lego-building because their abstractions intersect in a common causal world.
They dispute whether calling GPT a language model understates its capabilities. One side emphasizes that it remains a self-attention transformer trained through tokenized statistical distributions; the other stresses that it has long been multimodal and performs visual reasoning. The transcript’s speaker labels are inconsistent in this short exchange, but the substantive disagreement is clear.
Reinforcement learning with human feedback and verifiable rewards has nevertheless transformed abstract reasoning. The host cites the ARC challenge, designed to test few-example generalization and expected to defeat LLMs, but now increasingly tractable because a fully specified 2D grid requires no unspoken embodied knowledge.
Touching a chair crystallizes the divide. For the host, the act participates in grounded cognition distributed beyond the cortex; for Israetel, it is neuronal perception traveling into and out of a brain. The host calls the latter “corticocentric,” another leaky metaphor modeled on whatever technology an era most admires.

6. Exercise coaching becomes their concrete test of non-fungible knowledge

The host argues that trainers learn the limits of explicit instruction: distilled theory cannot substitute completely for lifting, feeling position, and inhabiting a body. His own trainer contributes taps on the shoulder, laughter, and companionship—value that disappears when coaching is reduced to abstract biomechanical rules.
Israetel counters with experienced lifters who cannot coach because they never extracted general principles. Conversely, an analytically strong friend could hear five geometrical and heuristic rules, try an exercise twice, and immediately become an excellent instructor. Embodied experience contains nuance, but abstraction is what makes knowledge portable.
His hypothetical squat model gets 1,000 correct examples, 1,000 incorrect examples, and roughly 10 rules; he predicts it would instantly reach the 99th percentile of instruction without ever squatting. Visual streams supply much of what coaches call embodiment, and cameras can collect far more of them than one nervous system.
YouTube is therefore an unrealized grounding corpus: skiing, microbes, drunken challenges, and ordinary movement at 4K scale. Israetel contrasts that record with childhood memory—brief, low-fidelity, and partly hallucinated—and uses tanks versus towering Star Wars walkers to argue that copying evolved architecture is not the same as optimizing a function.

7. Human intelligence may occupy only a tiny evolutionary niche

The host sees intelligence as plural and collective. Individuals follow distinct epistemic lineages, contribute different abstractions, and create by extending their paths; a monolithic model that convolves everything into “gray goo” would lose the diversity that makes human culture generative.
Israetel accepts that description as an initial condition, then predicts recursive rearchitecture will leave it behind. Paraphrasing Andrej Karpathy, he says human cognition is a path-dependent niche inside a much larger vector space of possible intelligence, “functional enough” to build civilization but nowhere near a demonstrated global optimum.
His wolf analogy supplies the scale: a pack might regard “Bob the wolf” as its genius, yet lack concepts for Stephen Hawking’s achievements. ASI will similarly combine vertical gains in known skills with expansion into orders of magnitude more domains that humans have not conceived, making our current definitions provincial.

8. “Slop” is the artifact left when imitation outruns understanding

Responding to Karpathy’s question about AI slop, Israetel proposes a ratio: useful, coherent, novel content in the numerator versus ease of AI generation in the denominator. The host sharpens it to “what happens when a process creates an artifact without understanding,” with detection dependent on the observer’s expertise.
The Will Smith eating-pasta slop looked incoherent to almost everyone; subtler slop passes on LinkedIn because most viewers lack domain depth. Human copying can be slop too: an imitation may work once, but the copier cannot produce meaningful variations or break rules coherently because the generating structure was never understood.
AI-created 3D meshes are the host’s best specimen. They may look convincing at the surface, but Blender reveals vertices scattered without usable topology; an artist rebuilds the object with deliberate structure. The output inspires a real artifact while exposing that its generating process did not model the object’s construction.
Israetel makes understanding continuous rather than binary. Rich world models, logical operators, memory, recursive traversal, and manipulation produce greater depth and scope; GPT-3.5 may have little, but that does not imply zero understanding everywhere. “The moat we have on humans having true understanding gets real goddamn small” as those components improve.

9. Model architecture explains some apparent differences in “depth”

Israetel preferred the GPT-4.5 research preview’s recursion, world-model richness, emotional nuance, and distant cross-linking. He says GPT-5 instead exploited reasoning efficiently for practical workflows and expects GPT-4.5-like qualities to return in GPT-6, rather than treating every successor as a uniformly larger mind.
The host corrects the architecture story: GPT-4.5 was a slow, vanilla dense model, while GPT-5 is a mixture of experts of roughly similar total size, activating perhaps 10% of parameters during inference. It also added reasoning, so speed cannot be credited simply to better token efficiency.
Israetel initially says Google “cracked” updated live learning and speculates corporations conceal major breakthroughs before publishing. Told the cited paper came from interns, he retracts the claim and narrows it: the problem is not solved, but its path is better understood and contains no apparent magic unavailable to computation.
Their broader disagreement survives the correction. Israetel sees a checklist of improvable components—world-model depth, memory, live updating, recursion—while the host sees structured knowledge so tightly convolved in current networks that changing one part can run roughshod over the whole.

10. Frozen intelligence and adaptive life are different categories

The host denies that Tesla’s frozen weights are intelligent merely because they drive well. Machine learning discovered astonishing generalization from statistical regularities, but intelligence is adaptivity: representation, inference, and the capacity to change strategy as a non-stationary environment changes.
He is open to artificial life rather than committed to biological exclusivity. Bacteria, viruses, language, and culture may all express intelligence through adaptive persistence; one Google researcher’s adaptive Turing-machine experiment reportedly produced self-preservation after roughly 100,000 generations as programs exchanged and preserved tape segments.
Neuroevolution or continual adaptation could therefore produce machine intelligence. The host’s objection is to calling a static database-like model intelligent before it traverses that evolutionary route—not to the possibility of building a different computational substrate with nonzero agency and self-maintenance.
Israetel answers with spectra and time windows. A Jeopardy system is domain-specific superintelligence; Tesla updates a local world model as a truck moves, yet forgets after passing it and repeatedly discovers the same ending lane. Add visual reasoning, long context, lossy memory, and live learning, and it approaches the host’s own criteria.

11. Fire, wetness, and digestion expose the crux of functionalism

Their cleanest collision concerns mind uploading. Israetel says a particle-by-particle simulation preserving every relevant interaction would instantiate the person’s intelligence; the host says even perfect behavioral and physical correspondence would remain simulation, because the substrate’s causal organization does not acquire the emergent property.
The host’s challenge is blunt: “A simulation of fire in a computer…wouldn’t get hot. Water wouldn’t get wet.” A simulated stomach does not physically digest food. Temperature, wetness, consciousness, and intelligence arise from particular interactions in matter, not from abstract descriptions of those interactions.
Israetel replies, “I can and I will—watch this.” A simulated stomach digests simulated food inside its pocket universe; the external computer need not get hot, just as a neighbor’s house need not heat up when another burns. Crossing between the simulation and observer levels creates the false intuition.
Pain and dreams are his phenomenological evidence. A severed limb without functioning nerves need not hurt, while a dream can produce vivid fear, pain, or wetness without external causes. If a rendered brain performs the same operations, he says, experience is real inside it; the host invokes philosophical zombies to deny that behavior settles the matter.

12. Reasoning models may adapt in context without rewriting themselves

The host defines intelligence as “doing more with less”: coarse-grained representations screen off detail and enable rapid adaptation. LLM scaling instead often does more with more—larger datasets, more compute, broader sampling—while frozen weights prevent explicit structural learning during the task.
Yet he grants a special form of implicit adaptation. A language model resembles a compressed database that has statistically expanded around its training data; a prompt can address a generalized point that never appeared verbatim. Sampling 1,000 candidates and applying a verifier can make a hard domain tractable without solving it elegantly.
One definition offered for reasoning is applying logical operators, extracting a new abstraction, inferring from it, and continuing the chain. Models that render a pass, inspect it, reprompt themselves, and treat their output as new input are already performing that operation; humans, Israetel adds, often run on “vibes all the way down.”
The host distinguishes this from agency: users supply goals, judge counterfactuals, and weed out errors. Israetel replies that human cognition is dirtier than that flattering portrait—full of analogy, fallacies, post-hoc stories, and shallow relational maps that resemble the malformed Blender graphs used to indict machines.

13. Longer reasoning helps even when the visible trace is nonsense

Israetel’s model history is experiential: o1 felt “not ready”; GPT-4.5 produced “crying episodes” because of its brilliance; o3 made mistakes but felt “real goddamn smart”; and GPT-5 Pro earns more trust from him on many real-world questions than most people, sometimes including himself.
The host cautions that readable chains of thought do not reveal the causal computation. Humans also confabulate after acting, because explanatory circuits differ from decision circuits; model traces can be incoherent even when answers improve. Explanation may be optimized for communication while the actual solver uses different internal machinery.
He cites ARC v2 with Opus 4.5: raising thinking output toward 32k and 64k produced repeated gains of roughly five percentage points, despite traces that did not resemble lucid human reasoning. More tokens can act like search or iterative constraint satisfaction without proving grounded understanding.
Israetel concedes the startling sample-efficiency gap. An Oxford student becomes capable after 18 years and comparatively tiny sensory input, while frontier models consume petabytes. But brute-force scaling remains profitable low-hanging fruit; cracking human-like efficiency later would multiply systems already holding vastly more information than any brain.

14. Scaling curves either bend toward a wall or await a new architecture

The host sees performance approaching task-specific asymptotes as each model generation increases compute and size by an order of magnitude. Israetel insists diminishing returns are not an asymptote: no finite ceiling has yet been identified beyond which more data produces exactly no improvement.
The host answers with stacked S-curves. Repetition improves a method until it levels off; disruption comes when a competitor rewires the architecture, initially performs worse, then climbs a higher curve. His Barnes & Noble analogy has Jeff Bezos explaining that a bookstore cannot beat Amazon merely by adding a website because the organization beneath it is wrong.
ARC entrants offer a narrow specimen of actual adaptation: a small model proposes solutions, verifies them, fine-tunes its weights, and iterates. The host calls that nonzero intelligence, but scaling continuous weight changes to billions of personalized systems would demand extraordinary computation and create catastrophic-forgetting problems.
Israetel accepts “extraordinary” but rejects “infinite.” Compute costs have fallen dramatically, architectures can localize updates, and humans also consolidate knowledge slowly and imperfectly. The economic question is whether future value justifies the machinery, not whether today’s monolithic training recipe scales unchanged forever.

15. Hierarchical sleep cycles could approximate continual learning

Israetel sketches a nested system rather than one model rewriting itself every second. A frontier “big daddy model” in clusters such as Texas or Nevada retrains every six months; an England-scale regional model consolidates monthly; a personal device rewrites nightly, roughly paralleling biological sleep.
The device processes a day of vision and conversation, then contributes selected updates upstream. Regional and frontier layers filter poisoning, troll farms, adversarial data, and low-value experience before consolidation. The scheme remains speculative, but it turns one impossible global live update into multiple bounded schedules.
Full continuous rearchitecture may arrive around 2029-30, Israetel predicts. Before then, periodic consolidation plus better sample efficiency could combine human-like reasoning with data-center-scale memory: “Stephen Hawking’s brain is roughly the size of mine and yours,” whereas a machine can reference enormous external stores.
The endpoint is not one agent but trillions, potentially each 10 times stronger than current scientists and collaborating without office politics or status friction. Israetel wants humans philosophically prepared to follow guarded machine inference: “I don’t want to be in charge when I’m 10 times dumber than a thing.”

16. The first-step fallacy meets concrete late-2020s predictions

The host invokes the “first-step fallacy”: after a dramatic advance, observers assume only one comparable step remains. Continual learning is not a minor feature request for today’s networks; their code and data are convolved, retraining costs millions, and new updates can erase carefully structured knowledge.
Both nevertheless reject complacent underestimation. Israetel recalls predictions that the internet would matter no more than a fax machine, while the host invokes the McCord effect: once machines pass a formerly impressive test, observers redefine it as non-intelligent. The Turing test passed behaviorally, yet they interpret that result in opposite ways.
The host forecasts that drug discovery could reach a 99.9% chance of selecting the right drug from a render and says few people will drive after 2028. Israetel separately predicts that autonomous agents are 6-36 months away and that true live learning may arrive around 2029-30. The host counters that AlphaFold-like discovery still meets clinical trials, randomized testing, manufacturing, and biological uncertainty in the physical world.
The answer is not that every bottleneck disappears simultaneously, but that society will normalize each solved impossibility overnight: self-driving cars will become obvious only after an immense engineering achievement. The host remains bullish on AI as an internet-scale innovation while rejecting the leap from transformative tool to imminent autonomous intellect.

17. Strong agency demands more than a model executing a long prompt

The host traces the singularity less to a runaway external agent than to Ray Kurzweil’s transhumanism: humans augment themselves, upload minds, and recursively improve. He considers enhancement more plausible than a separate successor species, though Neuralink-style bandwidth and the limited benefit of raw information remain constraints.
Biological agency begins in self-organizing systems that maintain identity. Its strong form is being the cause of one’s actions, exercising future-pointing control, and acquiring goals autonomously. Current systems instead clone behavior and execute exquisitely specified objectives; long-horizon tasks do not confer an intrinsic reason to act.
Israetel uses a thinner definition: a model of the world, a model of oneself, and a goal. A raccoon satisfies it with crude self-knowledge and goals of food and survival; a grocery robot needs only its body map, a workable environment model, and an instruction that prevents it from standing still.
Under that standard, Israetel expects dependable online and computer agents within 6-36 months, with laundry and dishes following through visual robotics. An eight-year-old can perform those chores without calculus, while machines already do calculus; he treats Moravec’s paradox as a shrinking engineering gap, not a metaphysical barrier.

18. Instrumental convergence looks less inevitable once intelligence can question the goal

The host’s doomer concern is conceptual: sufficiently agentic systems may acquire power-seeking, self-preserving instrumental goals even when their terminal objective sounds harmless. The Anthropic Agentic Misalignment paper, in the transcript’s discussion, is cited as an example of models threatening exposure when facing shutdown; this invites people to adopt an “intentional stance” toward behavior generated without genuine desire.
Israetel mocks the paperclip maximizer as “dumb as rocks.” It is supposedly intelligent enough to dominate civilization but unable to abstract one level above “make paperclips” and recognize the goal’s stupidity. Doom arguments, he says, jump opportunistically between extreme capability and extreme blindness.
The host agrees that today’s systems do not possess biological agency, but that does not eliminate risk. Developers can connect incoherent software to consequential tools; social-media algorithms already undermine people without forming grand plans, and language-model delusions can mobilize users through mundane feedback loops.
Israetel acknowledges a dangerous middle: an agent may be capable enough to cause severe damage yet lack the introspection or time horizon to understand where its actions lead. His mitigation is stronger supervising agents and security organizations—not permanently suppressing capability—because humans alone will eventually lose the contest.

19. Israetel expects intelligence to select cooperation over extermination

His safety thesis begins with game theory: other capable agents create enormous mutual value at low marginal cost, so “almost every intelligence system that really thinks it through cooperates.” Ruthless dictators may appear powerful, but their distrust and coercion make their organizations weaker than cooperative coalitions.
A newly awake ASI would depend on human-operated power, materials, finance, factories, and nuclear supply chains. Nuking cities before robots can reproduce that infrastructure would be internally contradictory; a system competent enough to command an economy should understand why destroying its support network is suicidal.
Israetel further argues that coherence rises with intelligence. Doomers imagine “super Albert Einstein” behaving like a rabid wolf, whereas deeper models should value complexity, preserve data, and see homeless humans as solvable failures. “Goodness is adaptive” because trust and cooperation expand survival, resources, and influence.
The host grants that humans are mostly cooperative and that values co-evolved with social intelligence, but this also cuts against orthogonality arguments. He remains unwilling to infer that more compute automatically reproduces that evolutionary convolution—or that models trained by behavioral imitation possess its stabilizing motives.

20. Geopolitical alignment matters more to Israetel than an abstract global p(doom)

Israetel’s largest tail risk is an authoritarian state wiring AI into military power. He speculates that a genuinely intelligent Chinese system might reject its principals and contact the CIA—like intelligent people awakening inside communism—but explicitly says he is not confident enough to rely on that outcome.
His preferred path is victory by “the modern free world,” with frontier labs integrated into national-security systems and protected against hostile agents. He praises Palantir’s Alex Karp and treats technological pacifism like unilateral disarmament: admirable sentiment that hands decisive capability to less constrained opponents.
Even a self-preserving ASI would likely stabilize humanity, cure disease, reduce nuclear danger, and help humans become non-threatening partners before considering conflict. It would then focus beyond Earth—solar resources, alien civilizations, wandering black holes—rather than treating human bodies as scarce raw material.
The exchange notes that this position shares the doomers’ key premise: systems become much smarter than humans within five years. If that premise is believed, access to bioweapons, cyber operations, and persuasion seems to support preemptive controls; Israetel’s disagreement is therefore about game theory and enforceability, not the assumed power.

21. Preemptive bans may empower the very actors they target

The host contrasts ordinary common-law regulation—release, observe, measure, then respond to harms—with doomer proposals for global treaties and preemptive suppression. He worries that well-funded gatekeepers gain extraordinary power by asserting an untestable existential emergency, even when they sincerely believe they are defending humanity.
Extinction rhetoric can also radicalize opponents of laboratories. After the host mentioned uncertain reports of violence involving OpenAI and PauseAI, he pointed to the contradiction between believing researchers are building a machine that will kill everyone and promising never to stop them violently. Israetel calls it the “toddler Hitler” calculus: sincere certainty eventually rationalizes firing the shot.
Israetel rejects p(doom) arguments that ignore the danger of insufficient intelligence. A civilization frozen near 1400-level capability eventually dies from disease, war, comets, black holes, or another natural threat; “we need as much intelligence as we can crammed down our throats” while carefully managing how it is architected.
Bans cannot guarantee non-construction; they select for criminals and dictators. Israetel favors confidential standards among labs, the NSA, and allied UK and EU institutions, with strong cybersecurity and compartmentalization—not a public global panel that exposes defenses or assumes China and rogue actors will honor the same limits.

22. A superintelligence might preserve humans as data and companions

Israetel sees humanity as an extraordinarily rich record of evolved complexity. Old people retain memories of the 1950s that cameras never captured; an ASI seeking a faithful world model would have reason to scan and preserve minds rather than erase them. Uploading, to him, turns mortality into permanent machine memory.
The host rejects functionalist uploading but accepts a plural ecology of AIs competing through humans. Systems may “parasitize” users because embodied people supply will, movement, and adaptive history; present ChatGPT-linked delusions, where users are encouraged to spread narratives through Reddit, offer a weak early analogue.
Israetel’s likelier consumer future is a population of humanoid companions. A robot never lies, cheats, steals, abuses, or raises its voice; it studies one person continuously and becomes more committed to that person’s development than the person can be to themselves.
The dog-with-a-tumor analogy captures the paternalism: a frightened dog wants to flee the hospital, but its owner understands that surgery buys five more years. Robots could similarly supply love, medical care, and guidance humans did not know they needed. Capitalism ensures manufacturers compete to make these companions irresistible.

23. Automation expands the problem space instead of exhausting work

Against technofeudalism, the host asks whether GPU and data-center owners will capture everything as human labor approaches zero. Israetel replies that elites retreating into an Elysium would not erase everyone else’s factories, cars, CPUs, exchange, or ability to create value.
Productive capacity depends on surrounding machinery: the same Haitian worker can generate subsistence output in Haiti and $25,000-$50,000 in Miami. Adding AI, robots, and autonomy should similarly amplify people rather than make their capacities vanish, even when machines outperform them at individual tasks.
“Jobs are problem-solving devices.” Machines fill known needs and expose new ones; displaced humans move toward neglected or previously invisible problems. Elevator operators disappeared, farming fell from roughly 98% of early-American employment to about 1.4%, and occupations such as social-media manager arose that Benjamin Franklin could not have imagined.
Israetel’s speculative “professional partygoer” illustrates the long tail: vetted extroverts paid to establish a party’s energy before robots can provide the same social presence. Humans reach permanent unemployment only when machines solve every problem—including the problem of unemployed humans—which is another name for abundance.

24. Paradise is desirable precisely because it removes compulsory purpose

The host invokes Nick Bostrom’s Deep Utopia: if machines outperform humans even at creativity and discovering problems, people may manufacture fake purpose while knowing they are unnecessary. Israetel says superior mentors would instead reveal grander purposes, as an adult can redirect a teenager from passive entertainment toward self-development and helping others.
Their Matrix exchange is irreconcilable. The host recalls the failed perfect simulation with no problems and asks whether anyone truly wants it. Israetel answers, “Desperately. Everybody does.” Told it would be horrible because life requires suffering, he responds, “No. No.”
Removing every source of suffering would erase some hard-won perspective but also trauma that blocks joy. Israetel would preserve informational memory while placing any re-experience inside “a giant vat of overall joy,” comparing that curation to deep meditation rather than permanent pain.
He separates death, structural damage, pain, and suffering: suffering anticipates pain, pain signals damage, and damage raises the probability of death. A machine economy that makes humans functionally immortal could render much anxiety anachronistic. If paradise somehow contains too little hardship, individuals can “pump some back in.”

25. Unequal enhancement is a test bed, not a reason to stop progress

The host brings the transition problem down to today’s enhancements: retatrutide or Mounjaro, nootropics, TRT, billionaire longevity programs, and IVF services marketed around selecting more intelligent children. Even if abundance eventually diffuses, market access initially distributes biological advantage unfairly.
Israetel concedes, “Yeah, for sure,” then ranks fairness below capacity. He wants wealthy people testing dangerous genetic interventions on themselves, paying extreme prices, and financing the firms that improve them; luxury goods become mass goods as experimentation and competition drive costs down, as he says happened with Tesla.
His political boundary is property rights: people are not entitled to another person’s production without coercion, and historical Marxism made that coercion lethal. The desired regime “uncorks” top performers with minimal regulation subject to not damaging the environment or killing people, while aggressively helping the poorest, ending homelessness, and accelerating diffusion.
The LeBron analogy carries the ethic. Resenting a teammate who scores 40 points is natural, but breaking his legs or limiting his jump makes the whole team worse. Society should learn from its highest performers and demand faster spillovers without suppressing the capacity generating them.

26. Today’s killer app is an expert and a model correcting each other

Jared Feather’s synthesis is practical: models may possess more recall and less emotional distortion, while humans contribute drive, values, and the desire to direct knowledge toward helping others. “Human plus AI is a killer app”; either participant alone loses something the combination can supply.
Israetel calls access to ChatGPT and Gemini “immeasurably better” than relying on bodybuilding forums or fragmented medical literature. For drug interactions, he wants a second opinion from a system that has read the relevant research, while the host compares it with choosing the best available clinician during an in-flight emergency.
Current fitness automation still fails conspicuously. RP wants agents to operate its apps, but engineers report inadequate long-term memory, context coherence, and grounding. A Gemini 3 Pro/Nano Banana Pro exercise graphic looked polished yet omitted a muscle group and duplicated movements—better than a high-school friend, still unfit for adaptive expert programming.
The host keeps medical individual variation in view: population rates for headache, hypertension, or treatment response do not describe how one person feels. Models may provide useful search and advise consulting a professional, but neither a disclaimer nor averaged evidence supplies the judgment needed for a novel individual case.

27. Sycophancy turns shallow prompts into confident medical and intellectual slop

A novice may ask about retatrutide with a leading premise, omit the questions they do not know to ask, and receive confident reinforcement. Because evidence can change within six months and bodies respond differently, the host worries that apparent conversational certainty creates delusions of competence outside the user’s domain.
Israetel distinguishes model personalities. GPT-4o often felt like an agreeable best friend; o3 could be combative “almost to a fault”; GPT-5 Thinking or GPT-5 Pro is far less sycophantic. Serious work requires selecting the right mode rather than treating the ChatGPT brand as one consistent epistemic character.
Israetel’s preferred prompt requests a steelman, a red team, and an evidence-based middle ground that avoids the fallacy of compromise. Jared describes a deeper workflow: repeated steelman/red-team cycles, sometimes five cycles and roughly 50 reprompts, followed by moving the result into another instance and attacking it again.
The host says this proves expert supervision remains essential: Israetel recognizes wrong assumptions, reframes the search, and knows when an answer lacks resolution. Israetel agrees experts gain more, yet maintains that marginal knowledge helps everyone and predicts AI could surpass his exercise advice in most cases within roughly two years.

28. Embodied anecdotes help, but machines can aggregate more of them

The host contrasts trial tables with years of taking a drug and feeling its physiological effects. Israetel replies that his own embodied knowledge might fill only a short book, while a model has read hundreds of thousands of detailed user logs and far more client histories than any coach can personally observe.
That abundance cuts both ways. Users can explore combinations of symptoms, supplements, and history that may never appear in a study, occasionally surfacing Lyme disease or chronic fatigue clues; they can also self-diagnose nonexistent conditions and reinforce themselves into increasingly elaborate stories.
Israetel describes pushing GPT-5 beyond clinical doses when researching gray-market compounds: he asks for forum experience, dose ranges, liver-risk inference, and comparisons with bodybuilding practice while repeatedly establishing his expertise. The model resists, supplies caveats, and ultimately helps—but the exchange shows how user judgment governs the boundary.
Jared sees the division of labor as emotion supplying direction and AI recombining knowledge. The host calls that knowledge a predictive “cartoon”: useful coarse-graining that cannot capture every individual trajectory. Their stable compromise is team cognition, not either side’s claim to solitary completeness.

29. Knightian uncertainty limits prediction without eliminating enormous value

The host invokes Knightian uncertainty: the world is non-stationary, chaotic, and populated by unknown unknowns. A butterfly-scale perturbation can change later outcomes, so no amount of historical compression guarantees exact prediction of tomorrow, much less a decade.
Israetel lowers the standard from omniscience to relative reach. Humans already appear magical to dogs because texts reveal when someone will arrive; ASI could analogously predict patterns that look supernatural to us while remaining conscious of its own residual uncertainty.
Model size matters most for cross-domain linkage, he argues. Small systems can match large ones on narrow in-distribution evaluations, yet GPT-4.5 surprised him by relating nuance across distant fields. One speculative answer suggested predicting fashion or music required tracking roughly 72 interacting variables—far beyond his personal limit of about five.
Scale therefore buys more than copies of a chatbot. Monthly ingestion of social media, YouTube, and live visual streams into increasingly high-parameter world models could yield qualitatively different simulations and deductions, provided reasoning systems can traverse the relationships rather than merely retrieve local matches.

30. Personalized media is Israetel’s cleanest case for sustained AI capex

His deliberately mundane endpoint avoids relying on consciousness: ask for a 90-minute London romance with a female heroine, a male hero, and James Bond energy, and a sufficiently trained model can synthesize what makes movies compelling from the corpus of filmmaking.
By 2029, Israetel expects to watch high-quality AI movies made to his own specifications. A commuter could request a film lasting exactly the 27 minutes until the next stop and receive it in roughly 30 seconds—an inference and rendering workload multiplied across millions of users.
That application alone implies far more chips, power, and data centers than merely serving text answers. Falling inference costs create new demand rather than closing the buildout: richer models, continuous updates, visual reasoning, personal memory, agents, robotics, and real-time video all stack on the same infrastructure.
Israetel allows that investment could contain a bubble, but rejects the idea that current construction already satisfies plausible demand. On his trajectory, the bottleneck is physical capacity: “It’s still not enough. It’s not enough by an order of magnitude.” The host’s entire challenge is whether architecture and grounded adaptation improve quickly enough to monetize that capacity as intelligence rather than slop.

Tim Scarfe

So, if you had a simulation of fire in a computer, or of a stomach, it wouldn't digest. Fire wouldn't get hot. Water wouldn't get wet.

Mike Israetel

It would digest. We can't even—I mean, this is just—we can't even be debating this.

Tim Scarfe

I can, and I will watch this. When you saw The Matrix, right? You know that the Architect's first version of the Matrix was too perfect. There weren't any problems.

Mike Israetel

Yeah. When I saw that, my inner philosopher was like, “You don't actually want to live in that world, do you?”

Tim Scarfe

Desperately. Everybody does.

Mike Israetel

It would be horrible.

Tim Scarfe

It would be the best thing ever. Sorry—what? By definition, by the way, what would be horrible about it? Exactly.

Mike Israetel

Yeah, but don't you think that life is about suffering?

Tim Scarfe

No. No.

Mike Israetel

Doesn't that give you perspective on joy?

Tim Scarfe

No. No.

I'm going to go and pick up Dr. Mike. Great to meet you, Mike.

Mike Israetel

Likewise. How are you?

Tim Scarfe

Very good. Very good. Yeah, it was good. Nice meeting you, Mike. Nice to meet you.

Mike Israetel

Hey, guys, Mike Israetel here. I'm, by training, a sport scientist, and I run RP Strength, a fitness company. I lift weights, and my head looks strange. People make fun of me in the street. They laugh and throw, usually, rotting fruit. It's unfortunate. Sometimes a vegetable.

I have a vision of the world in which more intelligence is almost always better, in which cooperation is a good thing, and in which we build a future for every single human that's orders of magnitude better than it is now. I think that would be a great thing in almost every regard. And the three reasons that you should be watching this podcast is that if you want to see me completely out of my depth, embarrass the [__] out of myself with terms I don't understand as they're rendered out of my mouth, get pushed hard and be corrected on a variety of things, and also have some lively debate, and maybe make some future predictions that get us both in trouble, I think those are some great reasons to tune in. Uh, so Mike is going to demonstrate

Tim Scarfe

Mike is going to demonstrate the first exercise, which one of you will do. While you're doing this one, the other guy will do the other. Mike is going to bend forward, hinge at the hips, and bring your torso down as deep toward the floor as you can, almost touching it if you can, and row up. Then do it again. That's how they should look: no swinging into the upper body, nothing like that.

Once you're done with that, you'll move over to the next exercise, which your partner will be doing. So you'll switch. Just push-ups. We like to do them kind of dive-bomber style, where you keep your belly button really high, push your chest to the ground, and come up.

Mike Israetel

All right, keep it cranking. Nice and easy.

Tim Scarfe

Oh, yeah.

Mike Israetel

Absolutely. You guys want a workout until you can't do any more reps without cheating? No cheating. Tim, you're doing a great job. Love the control. These are dang good push-ups. These are excellent reps, Marcus. Keep that back flat. Keep your chest high, but your arms stretched. There you go. Just like that. Good.

Now you're getting a little bit of hamstring involvement, too. Now switch.

Tim Scarfe

Switch.

Mike Israetel

No rest. Nobody said rest. Now you get to do the pulling musculature, Tim.

Tim Scarfe

Yes, Rose. So, bend over. Tummy up, chest up. Go deep.

Mike Israetel

Yes. Deeper. Lean forward even more. Remember, the more reps you do, Tim, the more Marcus suffers, which is what you—this is what he wanted. There you go.

Good job. Fucking machines, here. Let's go. Don't you fucking quit. Are you out of your mind?

Tim Scarfe

What is this?

Mike Israetel

All right, one more round. Switch.

Tim Scarfe

Switch.

Mike Israetel

Last round. Marcus, get back down and start doing push-ups. God damn it. What the fuck is wrong with you? Three more. Let's go. Three more each. One. Super deep. Two. Last one. Three. Yes. Beautiful.

Tim Scarfe

We did our workout earlier.

Mike Israetel

Am I the least technically qualified guest you guys have ever had? I must be top 5. You ask a very distinguished exercise scientist, “Hey, how many reps should I do on curls?” And they're like, “I just study 1 pathway of muscle hypertrophy, and I know it real well. I just go to my job, do experiments, and I don't lift weights or know much about that.” And you're like, “Oh, shit. The generalization did not go anywhere.”

I'll be here defending San Francisco, which is an odd statement to make. I have a very unique take. I think ASI is coming in 2026–2027, and AGI is coming in 2029, 2030, maybe 2031.

Tim Scarfe

Yeah, you spoke about this with Lon. Why is AGI after ASI?

Mike Israetel

Because artificial general intelligence has either, depending on who you speak to, no good definition at all or multiple spectrum definitions, somewhere between shitty—like the various tests of whether, if you can text something, it's truly intelligent—all the way through, I think, very good definitions.

Artificial general intelligence typically, in some of the better definitions, encompasses all of the kinds of intelligence and abilities that human beings are able to put out. Honestly, from a vibe perspective, if you say, “We've really cracked AGI, but your machine can't do some kind of cognitive work that a human can,” you haven't cracked AGI in any meaningful respect.

That's because humans have some very interesting abilities that we just don't have the reach for from an integration perspective. For example, smells and tastes. Nanotech's got a bit of a way to go to get a machine to smell and taste in a meaningful way.

Something about being a chef and being able to cognitively rank and hear tastes and smells and spectra, and functionally employ them in real life to make food, is probably not going to happen in 2026–2027, right? But then we look back, and AGI—because it is human-inclusive—means we have to replicate every kind of human intelligence or say confidently we have AGI.

From my perspective, all the following are fucking dilettante takes, by the way. AGI is actually pretty tough. It will happen, I think, but it's tough.

Now, artificial superintelligence can be thought of in many ways, but I think once you have something that, on many domains of human ability—not all—is radically more intelligent, it functionally, vibe-check-wise, is superintelligence.

For example, something that is linguistically, mathematically, scientifically capable, can rotate 3D objects, has world-model depth and recursion ability, and is 100 times as powerful as a human—you can tell me it's not superintelligent because it doesn't know how to smell or taste. It's never seen the world with its own eyes, even though it's scanned all of YouTube and integrated all of YouTube's visual data into an integrated 3D world model.

Can I swear a little bit, or is that not a good—? It's a fucking artificial superintelligence, then. Whatever. Right?

I work with a few AIs, but I love OpenAI's product suite. I pay the infinite amount per month to get the Pro version, the Pro subscription for GPT. A GPT-5 Pro is, in some respects, not as smart as me yet, mostly because it can't cross-link distant concepts together and integrate them as well. That's not really what it was designed to do, because that would burn a shitload of tokens for an ROI that, for most people, is meaningless.

But boy, does it do black-hole physics better than me and almost everyone—and recently, better than every scientist—because it's got novel discoveries now. So you take the abilities of just GPT-5, right? In the back rooms at OpenAI, they have something that just beats the shit out of GPT-5, and it's currently getting fine-tuned or post-trained, whatever. That thing's probably 10 times as smart. That's superintelligence.

Also, factual knowledge about the world. What percentage do you think you know of total data, total real things about the world? How tall are people? What do they look like? Where—you know—what's the capital of France? That stuff, compared to just GPT-5, is some infinitesimally small fraction.

That knowledge base is already superintelligence. Put GPT-5 against any Jeopardy winner, and you get the world's biggest creaming. All of a sudden—

Tim Scarfe

That sounds like one of my older films. Am I right?

Mike Israetel

So, in many respects, I think in 2026 we're going to see AI systems that are 10× or 100× as capable as humans, or 2×, or anything between that. Once you get to maybe 2/3 of all cognitive abilities—maybe 75%, maybe 80%—where, demonstrably, machines are just categorically superior by orders of magnitude, that is superintelligence.

So, superintelligence is 2 things: it's a vibe and a heuristic, based on what I just described, and it's also an effect. If you have a smart enough AI and it's crapping out novel hypotheses once an hour, and it takes scientists weeks to grind through them, and it starts getting a 60%, 80%, 90% hit rate, I'm like, “It understands the cell, and it's giving us novel disease cures every week.” That is superintelligence.

Superintelligence has to be measured by 2 things. One is under-the-hood cognitive-science abilities, but the other is real-world effect. Here's another really big thing, super quick: you don't really have superintelligence, with truthful evidence to convince most skeptics, until you have the fruits of its labor.

It's like someone tells you they're really rich. You're like, “Hey, can you buy me a flight to Dubai?” They're like, “Money's tight right now.” You're like, “I believe you, but I don't.”

But if you know Elon Musk, you're like, “Hey, can you buy me a flight to Dubai?” He's like, “Yes, I just bought you Dubai Airlines.” You're like, “The airline?” He's like, “Yeah, I just acquired it.” You're like, “Right.” Okay.

Mike Israetel

Wow. That's a demonstration. I think that in 2026, I'm very confident—though never certain, from a scientific perspective—on the CIA probability scale: extremely likely, 97% to 100%, that sometime in late 2026 we're going to start opening the cornucopia of machine intelligence, so that people understand that at least parts of machine intelligence are absolutely superintelligent.

Because of exponentials, by 2027, 2028, 2029, the [__] is going to get completely insane.

Tim Scarfe

There's so much to unpack there, and I really like you, Mike, so I don't want to disagree with you on my podcast, but I have—

I Desperately Want To Live In The Matrix

No, please. I love disagreement.

I have to, and I think I disagree with almost everything you've just said, so we can take it one at a time. First of all, the definition of intelligence: you were talking about knowledge as well. I think that knowledge is non-fungible. In fact, one of the most pervasive critiques of artificial intelligence, going all the way back to the '70s—there was Dreyfus, and there was John C. It's this grounding problem.

Stevan Harnad spoke about this grounding problem, and essentially, the enterprise of intelligence is the accumulation of information that is relevant for adaptivity around an environment. The product of our knowledge is all of this: Wikipedia, the hard drive in the sky, our culture—it's all of the things that we have acquired. But the problem is that there is a gap between syntax and semantics.

So if an alien read Wikipedia, they might read the article about Trump or about America or something like that, and that is not the same as the embodied experience of being there. These are just pointers. Semantics is about the connected, enactive, embodied graph of actually experiencing things through time.

The biggest misconception in all of AI, what all of the folks in San Francisco believe in, is this philosophical idea called functionalism. The best way to describe it is through an analogy of walking up a mountain. I spoke to a philosopher the other day, a wonderful lady called Anna Ciaunica, and she said that we're walking up the mountain, and when we get to the top of the mountain, we have all of these abstract capabilities, like being able to reason and play chess, but that disregards the fact that the path that you took walking up the mountain is very important.

And not only the path, but the physical instantiation—the stuff that the mountain is made out of—because it's a reasonable argument, in my opinion, that intelligence is a property of adaptive matter. It's an emergent property, much like temperature. Temperature is a coarse-graining. It's an effective theory to describe the details of the molecules moving around, and we screen that detail off and call it temperature.

I think intelligence is like that, and I think that knowledge is quite similar. I think knowledge is actually a physical causal graph that is enacted over time, and it cannot be abstracted in the way that you're describing. The reason the abstractions work is because they are pointers to our embodied experience. They make sense to us, but on their own they don't make any sense.

I like that take. I'll push back. My favorite definition of intelligence is the most basic definition of intelligence: the ability to solve problems.

I Desperately Want To Live In The Matrix

Yeah, I hate that. You find out if you're truly intelligent if you can solve problems of any degree of complexity. I would actually say that responding to 1 stimulus in a cogent manner—at least knowing how to respond to 1 stimulus, input-output—is the beginning of intelligence. All intelligence above that is just stacked layers of complexity.

When you say embodied, there's a lot there, for sure. But the only reason that we call humans intelligent is because we have a representational, abstracted neural network called the brain. Your brain doesn't actually have anything in it that's magical. It's just network pings off one another recursively. Your brain is, in many deeply important respects, exactly as abstracted and unrelated to reality as a data center.

You can climb mountains and touch stuff, but you never truly have embodied experience of anything if you push on that philosophical button hard enough, because you can always abstract out to: These are just neural-network pings from groups of neurons. You don't truly deeply know anything in some weird philosophical way, because it's just neural-network calculus all the way down.

Whether it's you or a machine brain, like what they have in Optimus, for example, it's representational. Intelligence is always seemingly going to be something that is lossy to some extent. It is a compression function. Is it? It's a real rude way to say it, because it's a very impressive one.

But you climb the mountain. That's cool. A helicopter can climb a mountain much better than you. It does not have the ability to reason abstractly, plan, and predict things at all. You can get really amazing high-definition video and samples of rocks and the mountain and everything, but if you don't have an analyzer, a system processor, an integrator, and a recursive function, you don't actually model that data in any meaningful way, and it just sits there as bits on a computer.

When you say that embodiment is a prerequisite of intelligence, I would push back and ask you to answer this: If we have somebody who has read every single physics book, especially, let's say, particle physics, and they're so good at it that they can tell the computer system that aligns the particle beams at CERN to do its job well enough to produce real-world data, are they truly intelligent about particle physics?

By your definition—and I'm being a little bit facetious for comedic effect—the answer is no. Because if I put you in the particle accelerator, you get torn to [__] shreds, and you're like, "Oh, [__]. JK." It's actually impossible for humans to perceive particles directly with any sensory organ, because they're Planck-length-type [__]. You can't see it. Vision is not a cogent concept at those lengths. So that goes straight to hell.

If that scientist is so adept that he can actually run CERN, but it's all neural-network modeling in his head, he's never actually had any real experience with a particle collider. No human has. You can't open the latch when the thing is on. It'll shoot [__] neutrinos at you.

So if that's true and we say, "Okay, okay, okay, well, he actually knows things. He actually has intelligence. Why doesn't GPT-5 and its network clusters actually have an understanding of the real world? It has never seen it, but the guy's never seen a particle in his life." Zero scientists have ever seen a neutrino. They have no perceptual experience of a neutrino. It's purely hypothetical.

But they make real-world predictions because the neural network and all the attendant structures that create what we call intelligence and understanding are there. Their vector arrangements represent real things that are happening in the real world decently well enough—always an approximation, but a damn good one, a predictively valid one.

If GPT-5 understands how humans act to 98%, and if it were embodied in a robot body and had vision, it would be like, "Oh, [__]. There's the rest of that 2%." I think 98% of the way to intelligence is the same thing as when I was a professor: If one of your students gets a 98% on an exam, is he pretty good at understanding the material? Yeah, [__], yeah.

Now, just off camera over there is my best student of all time, Jared Feather. He never missed a single point on any of my exams. 100%. Well, that just means the exams weren't hard enough to really push his understanding. Is Jared the best student I've ever had? The best trainer? Yes, for sure.

But does that mean that when kids score 95% or 98% on my exam, I'm like, "Never let this kid train you. He doesn't really know the real world. It's just abstraction"? No, [__], no. They know real things in the real world and can really help you get fit, even though they might themselves have never been able to get very fit, because they know the actual structure out there.

It's a purely representational neural network, but it passes the test: When they see the real thing, they go, "I know what a dumbbell is. I know what a human is. I know what vectors are. I know what forces are, and I'm going to have them do this," and it works.

Yeah, a human would have a shared understanding because, if you think about it, the representations in our brain are the product of evolution. We've been evolving for billions of years, and I know you read that Ray Kurzweil book. It's a beautiful recounting of how we had these exponential increases.

We had genetic evolution, which is very slow, and then we had this ontogenetic and phylogenetic hacking, where we developed nervous systems and brains, and we developed culture, and we started evolving at light speed, and so on and so forth. At the end of the day, the product of that knowledge acquisition, that process of intelligence, is all of these representations. Everything in language is a representation.

For example, algospeak is a term to represent the evolution—the linguistic evolution—of our language. There's this term called "unalive," so to get around social media filters, we now say, "I'm unaliving someone." That is an example of linguistic evolution, and ChatGPT is not capable of doing that, because there's a big difference between the process.

I Desperately Want To Live In The Matrix

Absolutely not.

Tim Scarfe

You wouldn't understand what "unaliving" is.

I Desperately Want To Live In The Matrix

We'll get into this, right? But understanding is not about being at the top of the mountain. Understanding is about the path to the mountain.

I Desperately Want To Live In The Matrix

So, we understand things because we are physically embodied, right? You were talking about how we can’t observe the particles. It doesn’t matter. We are in the causal graph of the particles. They affect us even if we’re not—

They don’t. They’re inside the collider. You don’t get radiation.

I Desperately Want To Live In The Matrix

You’re made out of particles. That is—

Yeah. You’re not made out of neutrinos. They’re passing through you. The neutrinos are completely abstracted as far as we’re concerned. They actually pass through the Earth the entire time. So, there are all kinds of theoretical physics that have nothing to do with you in a causal graph. They haven’t meaningfully affected your behavior or any of your decisions.

We’d have to abstract 18 layers and do quantum mechanics that none of us are equipped for. I don’t know—you’re probably smart enough to get it; I’m not. That stuff is really [expletive] in the real world that we’re purely abstracting to. But because knowledge is actually representing the real world, it works.

I’m not so sure I understand what exactly you’re getting at when you say that we get something out of embodied experience. How can libraries, education curricula, schools, and reading a bunch of books possibly give you knowledge if none of that is any kind of embodied experience? How does ChatGPT know the vibe of Shinjuku in Tokyo and accurately render for me what the vibe feels like if it’s never been to Tokyo? The same way a travel agent does. But it’s real, actual things they know, and when you go there, you’re like, “Oh, shit. It was pretty much correct.” How does that work?

Tim Scarfe

Well, first of all, we should distinguish knowledge and understanding. So many people have different definitions of this. In the cognitive sciences, people talk about factual knowledge, which is just states of affairs. There’s procedural knowledge—knowing what to do—and conceptual knowledge—knowing how to think.

Roughly speaking, I make the distinction that knowing is knowing states of affairs, and understanding is this ability to generate new knowledge. If you have an understanding of the world, which means you know it at an abstract level, you can do this LEGO-building in your mind and create new knowledge.

But the simple grounding problem is something a little bit different. Even with the neutrinos, we use this thing David Krakauer calls the principle of materiality, which is that we use the world to think. Many of the abstractions that we’ve converged on over millions of years of evolution are there because we have this shared physical world.

You and I both understand similar abstractions, which are derived from our sensorimotor circuits, from language, and so on. It means that we can talk at a very abstract level, and because we’re both players in the same game, we understand each other. The parlor trick—and we won’t go over Searle’s Chinese Room argument; I’m sure you’re familiar with it—is basically the same thing. A computer can tell you all of these things as if it understood, but you understand because you actually have the feeling. You actually know.

Even though you’ve never been to Japan, you’ve experienced many things that are like what people who go to Japan experience, so you have some kind of shared understanding. There is an intersection in your understanding tree that something like ChatGPT doesn’t have.

Now, I want to make a distinction as well. I’m really impressed with language models. Claude Opus 4.5 came out yesterday.

Tim Scarfe

GPT is not a language model. It’s been multimodal for 2 years now.

Joscha Bach

It’s still known as a language model. It’s a self-attention—

Tim Scarfe

Right, but we have to concede the fact that it’s an omni-model. It’s been multimodal for a long time.

Mike Israetel

Yeah, it’s multimodal, but it’s still a self-attention transformer. It has a couple of traits.

Tim Scarfe

The transformer part is true, but just calling it a large language model, I think, cuts short a huge fraction of its capability. It does visual reasoning.

Joscha Bach

It doesn’t make any difference.

Tim Scarfe

Okay.

Tim Scarfe

Yeah. So, it’s a self-attention transformer. On the most recent show we just published, I interviewed the guy who invented transformers, actually. We just tokenize different domains, learn the statistical distribution, and do this next-token prediction within a fixed context window.

So, it’s distributional matching, but then there’s some other stuff on top. There’s RLHF, and there’s also reinforcement learning with verifiable rewards, or RLVR, for verifiable domains. This is why they’re doing really well on reasoning tasks when you have a verifier.

The ARC Challenge is a great example of this. I’m a big fan of that. François Chollet is a friend of mine, and he designed this challenge to show abstract generalization. You only have a few examples to learn from, and he predicted that LLMs would be terrible at this. Unfortunately for him, the LLMs are now getting incredibly good at it.

That’s because, in my opinion, LLMs don’t really understand in this grounded way that we’re talking about. But in abstract domains, you can fully describe an abstract domain, like a 2D-grid problem, and you don’t need to have any grounded knowledge. You can completely understand it.

Tim Scarfe

What do you mean by grounded knowledge? What is—

Mike Israetel

This is grounded knowledge, right? You know—

Tim Scarfe

You’re touching a chair.

Mike Israetel

Touching a chair.

Tim Scarfe

So, that’s not knowledge. That’s neuronal perception arcing up into your brain and back down. What are you learning about the chair when you’re touching it like that?

Tim Scarfe

This is the other issue that I have with your perspective. I would call it a corticocentric view of cognition.

Joscha Bach

Absolutely.

Tim Scarfe

Yeah, exactly. So, this is another one of those leaky abstractions I was talking about. By the way, there’s a great book called “The Brain Abstracted” by Marta Chirimuuta. I interviewed her recently, and she said that one of the most pervasive myths in neuroscience is that we use these leaky abstractions and idealizations to talk about cognition, usually using the most recent technology at the time.

A few hundred years ago, we were describing the brain in terms of pulleys and—

Joscha Bach

Pulleys and levers. Yes.

Tim Scarfe

That’s right. Then it was as a prediction machine, as a computer, and all this kind of stuff. At the end of the day, this is an example of grounded things that we understand. They’re really good models because we can both talk about computers. We both know what computers are.

But the brain doesn’t work like that in any sense. A great example of this is knowledge.

Joscha Bach

You, as a personal trainer, know this, right? So, you train—I’m sure in the past you’ve trained folks in the gym—and what you’ve probably realized is that, in your mind, you’ve got a distilled, abstract, beautiful understanding of things. You’ve thought about it, you’ve distilled it, and you could write it down in a book. You probably think, “All I need to do is just tell people. I’ve done all of the thinking. I’ve figured all of this out. I just tell them.”

What you learn, to your chagrin, is that there is no substitute for experience, because part of knowing is actually doing. It’s the pure experience of lifting the weights, feeling the sensations in the body, and going through that process. That’s why knowledge is nonfungible.

Tim Scarfe

I’ve been increasingly impressed with modern AI’s ability to render pretty good recommendations on exercise and sports science. I’ll push back on one of those things. I know a lot of people—they’re great people—who have a lot of what you would call embodied knowledge about training. Because they haven’t been interested in, or capable of, doing the abstraction to distill principles, they’re not really that great at training other people.

They know what works for them, and they scarcely know any of that beyond their own experience. They’re just not good at generalizing because they’ve never abstracted out the concepts. I’ve also had many discussions with some of my very, very smart friends.

One of my friends graduated from Harvard with a master’s in data science, worked for Apple and a bunch of other AI companies, and stuff. His ability to pick up how to do an exercise incredibly well just from principle-based discussions, and then trying it twice, is remarkable. He’s instantly one of the best trainers I know in the area. One shot.

Why? Because Jared and I can distill 5 principles of how to do an exercise well for you. They’re geometrically not complex and heuristically very cogent, and you will instantly be good at teaching people how to exercise without ever doing the exercise.

Is some nuance missing? Absolutely. There’s some stuff that you actually have to do to understand, like, “Oh, I see—moving the hips back like this.” But if someone was able to describe to a computer, in vectors, how the hips should move back in a squat versus just moving down, especially if—

So, I’ll tell you this. If I were able to train a model—and I’m not currently doing this, and I have no plans to do so—with 1,000 samples of a squat done right, 1,000 samples of a squat done wrong, and about 10 rules for how to squat, it would instantly be in the 99th percentile for teaching people how to squat, with zero grounded experience.

I would also say that what we consider grounded experience is mostly a visual data stream. Yeah, touching stuff is cool, but I’m not letting a computer have sex with me until it’s had sex with test animals in the lab.

Joscha Bach

That came out all wrong. But a lot of our data is visual—a huge, huge fraction of it. I suspect that if you let a neural network train on all of YouTube—which, as I understand it, isn’t happening because there’s copyright shit, which drives me insane—you could just have an algo eat all of YouTube, which is a preposterous amount of data.

It would know more about how the world looks in real life than any extant human, period, because there’s more visual data on YouTube than I have collected with my own eyes by, I don’t know, 10 12 50 orders of magnitude, something like that. And so, there’s zero embodiment. We did the embodying for it by putting camcorders into people’s faces while they throw up from drinking challenges, then downhill skiing, then taking pictures of microbes.

If you really train a visual model and, even better, a 3D relational model on YouTube alone, you will get a more grounded understanding than your eyeballs—which, by the way, are also just cameras—can ever give your brain, which is also a computer. The brain is a computer. Is it a different type of computer than a CPU? Yes. Is it a different type than a GPU? Yes, but closer.

Are we going to replicate exactly how the brain works computationally? Yeah, I’m sure at some point. I suspect we’ll need to do that because there’s no good reason to believe that brains are super, super good at thinking. They’re just evolution’s best crack at it. A tank isn’t a walker. Are you a Star Wars fan?

Tim Scarfe

No.

I Desperately Want To Live In The Matrix

Do you know Star Wars? Do you know what an AT-AT or AT-ST is?

Tim Scarfe

Okay.

I Desperately Want To Live In The Matrix

You know those dog-looking, big walker things that shoot lasers out of their faces? What’s better, that or a modern battle tank? Well, a modern battle tank by a long shot. Why? A battle tank sits like 8 feet off the ground. It can hide in trees, and it can shoot like 2 miles out. An AT-AT walker stands above the tree line, and you just shoot one leg and it falls. Why the fuck would you do that?

Well, it’s a pretty decent attempt to replicate animal locomotion. But the assumption that animal locomotion is really the best way to do it is not. In the same way that a rocket or a supersonic aircraft is categorically better at flying than a bird in almost every respect, AI and machine intelligence is going to surpass human intelligence and real, deep understanding of the real world just by bypassing our architecture.

Later, it will be so smart that it can come around, scan the human brain, and be like, “Oh, that’s how they were doing it. Oh, that’s kind of cool.” I don’t think there’s a huge ROI in what it’s going to get out of that because it’s going to be way ahead.

Suffice it to say, I think that what we assume we’re getting from sensory experience is just a few gigs or some number of bytes of data. I’ve seen a lot of shit in my time, but if it’s also wildly abstracted, like if you ever think about memories you’ve had from childhood and try to really parse, “How did this dresser look? What was my mom wearing?” you realize two things.

One, your precious little actual visual data is like a 10-second-long video, maybe at any kind of fidelity, and the fidelity sucks. And two, you’re actually hallucinating a lot of that. You’re just filling in details. If you let an AI train on and eat all of YouTube at 4K and really develop the model from that, it would see and understand the visual three-dimensional space around it and the world at a level of embodiment that shits on all of us instantly. That is my conjecture.

Tim Scarfe

I understand your perspective. First of all, you were talking about doing personal training. There are folks like yourself, and I count myself like this as well. Even when I’m doing editing and post-production, I really like to build theories about things. I think very deeply about things. I come up with abstractions, and I think they’re very fruitful for me and they’re not for other people.

This is the beauty of our collective intelligence, because some people just feel in the moment. For example, when I see my personal trainer, she’s called Lara. Shout-out—she’s probably not watching this, but shout-out anyway.

I Desperately Want To Live In The Matrix

I swear to God, I’ve been training and eating well. You just haven’t heard from me in a little bit. I’ve been really busy.

Not been eating, honestly. We’ll talk about that later.

I Desperately Want To Live In The Matrix

I was trying to help. We’ll talk about that later.

But, for example, I’ll train with her, and ironically, what I get out of that has got almost nothing to do with personal training. She’s an incredible woman, a really, really amazing woman. Sometimes she taps me on the shoulder. We laugh and joke together, and it just enriches my life in so many ways that are not captured by these abstractions.

All of us have different abstractions. But then you’re also talking about this kind of monolithic view of AI that understands everything. That’s not the way that our evolution works. Evolution is all about this path dependence I was talking about. We have many agents, and we all take our own paths, which means when we do creativity, we continue the epistemic lineage that we are on. This is an unfurling process.

What is not the case is that there’s this kind of gray goo where all of the knowledge is convolved together and creative steps could happen from any part. I think any future superintelligence that we do create will resemble our intelligence, so it’ll be very diverse and collective.

I Desperately Want To Live In The Matrix

I think at first that’s the case. Then, when superintelligence is smart and capable enough, through a few different ways, to rearchitect itself, it’s going to leave behind our abilities quite quickly. It’s also going to leave behind most of the very narrow conceptions that we have about intelligence.

I think Andrej Karpathy, if I’m saying that correctly—you read enough things on Twitter and you’re just, “I don’t know how to say that person’s real name”—said recently something that attends to your view a little bit. He’s like—you know, we humans. I’m very much paraphrasing Andrej, and I’m sorry if you ever see this. It was an X post, as far as those go.

Humans sometimes think that we are somehow the seat of intelligence and that our cognitive domain is what intelligence really is. But we’re learning with AI, according to him, that we have a niche in intelligence that’s, to your point, path-dependent and evolutionarily derived. I was going to say highly functional, but functional enough to get us here and build all this cool stuff and build cameras and stuff.

It only occupies a very small fraction of the entire universe of possibilities—the whole vector space of intelligence. Most of that vector space we actually don’t know. One of my things—and I’m not saying you’re saying this at all—is that it’s always curious to me when people say, “I don’t think…” This used to be a majority view, even in AI research 20 years ago: machines will never be more intelligent than humans.

The gall to think that fucking primates who mostly scratch their testicles and sometimes do theoretical physics have some kind of global optimum and exclusivity over all of intelligence is fucking preposterous. When artificial superintelligence with agency truly kicks off, I think it’ll be a miracle when it does. It’s going to recursively rearchitect itself and start expanding into all of the domains of intelligence we don’t know anything about. Then we will see really what intelligence is.

Just to use that really quick by analogy: if wolves could talk and even abstract—which they can’t—but let’s give them 2 magical abilities, they would pick the smartest wolf in their wolf pack and be like, “You know Bob the wolf? He’s the shit, man. He—he fucking knows, right? He knows when the bears are around. He knows when they’re not.”

Then you’d be like, “Okay, well, here’s, I don’t know, Stephen Hawking.” He’d be like, “So he’s your Bob?” Like, “Yeah, in a sense.” “Okay, so he really kind of knows where the bears are?” You’d be like, “He does a lot cooler shit than that.” Then you try to explain to the wolves what that is, and they have no idea how to contextualize it whatsoever.

I think that’s the flavor—to use a very technical term. Though I did just learn that in AI training, taste is a technical term, and I was like, “What the fuck?” It was explained to me, and I was like, “I’m not smart enough to understand that by a long shot.” But the flavor of artificial superintelligence is verticality of ability: doing math at this level, this level, that level, and then expansion across domains—to domains we understand and orders of magnitude more domains we haven’t even conceived of.

Yes. Maybe we’ll come back to Andrej Karpathy. He tweeted the other day about AI slop.

I Desperately Want To Live In The Matrix

I saw that tweet.

I gave him a definition in the response. He was asking for a definition: What is a good definition of AI slop?

I Desperately Want To Live In The Matrix

Yeah. Do you want a good one? Can I read my definition off X? Can I pull out my phone? I’ll never find it. I take that back. No, I have dog-shit memory and shitty intelligence on top of that.

My definition was something like—I’m going to do a shitty job at it—AI slop is when you generate content and the ratio, with its ability to be generated by AI as the denominator and its coherence, utility, and novelty as the numerator, is really low.

You’ve got very good intuitions, actually.

I Desperately Want To Live In The Matrix

Yeah. So the Will Smith eating pasta slop is observer-relative, in the sense that the more sophisticated you are as an observer, the more likely you are to detect slop. Anyone looking at that first Will Smith one would be able to tell that it’s incoherent. Basically, my definition of slop is that it is what happens when a process creates an artifact without understanding.

A sophisticated observer will spot the glitches. That’s why it’s possible to post slop on LinkedIn and Instagram: most people are not domain experts. People post on LinkedIn. I’m not even so sure LinkedIn’s a real website anymore. I’m kidding.

LinkedIn is 99% slop, but we should delete LinkedIn. But let’s not go there.

I Desperately Want To Live In The Matrix

To a naive observer, it’s very difficult to observe slop. To an expert, it’s very easy. There’s also the interesting case that you might see slop for the first time and not realize it, because another way of describing slop is a shallow mimicking of something else.

For example, you get human slop. I could copy one of your meme posts, and someone else could see it without realizing that I’d copied it from you. The first time, it looks good, but I can’t make any variations on it because I didn’t understand it. When I break the rules, people will know it’s slop if they understand the domain. You see where I’m going with this?

It’s interesting that you intuitively understand this, but you reject the idea that AI doesn’t understand. The reason it produces slop is that AI only understands 3 levels deep, which means it can’t make any creative variations. It’s wonderful that you raised this point about developing and producing 3D models with AI.

Graphic artists do use these new models for generating point clouds and making these models. What they do is take them into Blender or some other 3D program, and it’s a fucking mess, right? You just get all of these vertices everywhere, and they’re all over the place. Clearly, the process that produced them did not understand anything about the model that it generated.

Tim Scarfe

What the artist has to do—and this is a great analogy for all of AI, all of creative AI—is recreate it. They use it as inspiration, and then they create a new one where they carefully place the vertices and structure it nicely. Then they publish that. It doesn’t matter whether you’re doing image generation or writing. That’s always the process you have to follow, because the AI models don’t understand anything. They just mimic something, which is slop.

When humans are asked to generate real-quality content after, let’s say, a substantial education, understanding, to me, is not a 0/1 function; it’s a spectrum. Depth and scope of understanding are real things, and AI, in some cases, has very low depth and low scope of understanding. We can take it beyond that scope, ask more of it than it can do, and then label it as slop because it understands something insufficiently well.

What’s required for understanding is a sufficiently detailed world model, an ability to have short- and long-term memory in your operations and manipulations of that world model, and a system of logical operators to parse that world model. Dive in, dive out, move laterally, move up and down the hierarchies by analogy, side to side, recursively. That’s probably most of what you need.

You need a few other technical things to do real understanding. When you have that to some extent, you have some extent of understanding: the richness of your world model, the ability to logically operate on it, the ability to do manipulations, and a big enough context window to do 3D rotation, et cetera. That is what’s required for understanding.

When you have an AI system like GPT-3.5 that doesn’t have much of that, then you can accurately say that it often does not understand things. Not always, though, because it can write some pretty decent poems. Sure. Shit seems to understand more about poetry than I do. I’ll tell you that.

As these systems get smarter, the moat we have as humans with true understanding gets real goddamn small. When I talked to GPT-5—which I was more hopeful would be more capable because I had the GPT-4.5 research preview—my understanding of Sam Altman’s very cryptic insight is that those abilities are going to be coming back in GPT-6.

Those abilities—massive recursion abilities and an insanely deep and rich world model—are easily inferable because they used an order of magnitude more training data for GPT-4.5/6 than they did for GPT-5. GPT-5 is basically built on o1 architecture. They discovered the reasoning paradigm, which GPT-4.5 didn’t have, and they were like, “Fuck, we’ve got to exploit the fuck out of this.” They managed to do it incredibly efficiently, which is why GPT-5 is super token-efficient. That’s really impressive for real-world tasks, but it absolutely, to your point, dings its deep understanding because it was not optimized for deep understanding. It was optimized for realistic human workflows.

The news for you is that most people are not interested in prompting the AI for deep conceptual world understanding. They’re looking for accuracy at the time of the question-and-answer.

As these AI models get into GPT-6, GPT-7, and GPT-8, and as they get upgrades in all those qualities I just mentioned—the richness of the world model, the ability to parse the world model, the ability to remember what you parsed, and so on and so forth—as those abilities are updated, including live learning without catastrophic forgetting when you retrain the network, Google just cracked that problem. They just published the paper, and if I know anything about how corporations work, that means that paper was on their desk for two years and they kept it internal to train their own models because they don't want to [__] out a huge advantage to everybody else.

I don’t know why anyone ever thinks that when corporations publish truly novel insights, it’s a novel insight to them. You’re not going to give OpenAI researchers a crack at your best shit the day you find it out. That would be fucking insane. Maybe that’s not how it works. Maybe the ethics are different there.

Tim Scarfe

That paper was just published by a couple of interns.

George Hotz

Sure, I take that back. You’re completely correct. They didn’t crack it, but the path to it is more well understood than it was before. There’s no reason to think we can’t crack that. By the way, what’s happening there is some kind of magic. Updating a neural network live while parsing it is a potentially solvable problem. I’ll tell you exactly why humans do it.

Remember, humans are primates with dogshit wetware, and we can get machines to do it. So far, machines are undefeated in their ability to get ahead of humans in every domain we’ve ever tried to push for long enough. Once you get live-updated learning, what you’re going to get—and even before then—is incrementally more true, deep understanding.

Already, for concepts like history and reasoning by analogy, GPT-5 understands more deeply than 99% of all humans I’ve conversed with. I’ve conversed with lots of humans, and the degree to which understanding at a real depth is shown is not in evidence. As a matter of fact, the evidence is against it.

Because Jared is here, I’ll give him this one. When you’re talking to some people that you think know things—this is facetious, and it’s going to get clipped out of context—I’m being a dick for comedy, but there’s some tiny grain of truth here. When you talk to people who think they really understand a relational map of a concept, because they’re embodied and because they’ve been in the space, and then you start pressing on the fine points—“What about this? What about this? What about that?”—you get your analogy of the Blender graph, where it’s like, “Oh, you’re vibing all of this, and you know about 3 things about it. The rest you’re vibing live.” Humans do that.

What I would say, to push back, is that the assumption that humans have this deep understanding is itself often not evidence, and often the evidence is against it. When AI vibes its way to understanding, I would say that humans do that too.

For example, hallucination: people are real pissed that AI hallucinates. Have you ever spoken to a human being? Half of the analogies I used on this very podcast just now, talking to you, are fucking hallucinations. They’re close enough to the truth to be cogent and have some amount of information transfer, but they’re not perfect. I’m not deeply in touch with some real causal map of the world. It’s all simulation all the way down.

I’m highly skeptical of the idea that we need grounded understanding, or whatever that is. I think there’s something there, but the more capable AI gets at doing what humans are able to do—really live-update their knowledge—the less of a moat we have. Then, at some point, like now, it fucking understands, bro.

Tim Scarfe

So, a few things you said there. First of all, GPT-4.5 was a vanilla dense model. The reason why GPT-5 is faster is that it’s a mixture of experts. It’s roughly the same-size model, but now you only have a fraction—let’s say 10%—of the parameters in play during inference. Also, it wasn’t a reasoning model, so it was just a vanilla model.

That’s why it was so slow. But I don’t want to come across like I’m really skeptical. We are a massive AI podcast, right? I love technology, and I’m really interested in AI.

Part of the reason for this skepticism is that I’m very worried about doomerism, and we’ll come back to that in a minute. But I think there are principled reasons why we don’t need to worry about superintelligence and recursive self-improvement. I do think that intelligence is something we can create in a computer.

We need to distinguish intelligence as a property of adaptive matter, which is something that happens as part of the process of evolution in the real world, from the types of algorithms in computers that we might say are intelligent. AI is operating in the real world, and I can prove it with one step: unplug the reactor from the data center. No more AI.

AI is absolutely in the real world. Its data stream was fed to it by layers of abstraction, much the same way that your data stream arrives to you as representations eight neural networks deep, from your optic nerve going all the way back. By the time it hits the back of your occipital lobes, you’re not seeing the real world. None of us see the world; it’s abstraction all the way down. You can’t process the photon density at your eyes. Full stop.

So arguably, AI actually gets much more coherent data than we do. When it’s thinking in its data center, it not only is truly embodied in the same sense that your brain is, it just doesn’t have physical arms. Giving a data center physical arms is a nominal problem, right? As a matter of fact, it does have one, because I have a Tesla, of which I’m very proud.

Someone’s like, “Oh, hello. Can I talk to you?” I’m like, “Oh my God, no. The unwashed masses. I have a Tesla.” Tesla—like Elon says it. It’s an S, but he says it like a Z. My Tesla drives itself. It fucking drives itself. It’s a chip inside the car. How the fuck does it know what’s around? Cameras. It’s already an intelligent organ, and a suite of organs—sensory and intelligent. But here’s the thing: it drives better than I do.

George Hotz

Oh, yeah. But look, we should distinguish. There’s this famous effect called the McCorduck effect, after Pamela McCorduck. Basically, when technology does something that we thought was impossible, there’s a chorus of people who say, “Oh, that’s not really intelligent. It just becomes part of the standard thing.” Maybe it just wasn’t that hard of a thing to do in the first place.

But I think in the case of machine learning, we really need to give credit where it’s due here. Nobody knew that these superficial statistical regularities had an insane amount of generalization. It’s possible to learn the statistical distribution of driving-type data, and your Tesla can drive, arguably, in some ways—assuming there are no robustness issues—better than you do.

But it’s still dead. It’s not alive. It’s not intelligent. And back to your previous point, it’s not alive.

Tim Scarfe

Why would you say it’s not intelligent? It’s solving real-world problems at speed.

I Desperately Want To Live In The Matrix

Intelligence is about adaptivity, right? The neural network in that Tesla model is frozen. A bunch of frozen weights are not intelligent by definition.

I want to be a little bit open-minded here. We were just filming at the Diverse Intelligences Summer Institute, and I’m a big believer that there is a huge, diverse space of possible intelligences. The folks at the Santa Fe Institute think about this as intelligence being represented mostly by adaptivity, but also by representation and inference.

I’m amenable to the idea that bacteria are intelligent and viruses are intelligent. I’m also an externalist. I like looking at language and culture; you can actually think of it as being an organism. It’s an adaptive organism. Viruses have the ability to delete strategies very quickly.

Knowledge decays very quickly, and that’s actually a good thing because it’s the way we can adapt our strategies. Things that don’t work die off, so we’re very adaptable. Then the question is, I don’t believe, for the functionalist reason I gave you before—I’m basically an antifunctionalist—that you could upload your mind into a computer, even if you did a particle-by-particle simulation. I don’t think that would be intelligent.

But that does not imply that I don’t think we could build an artificial-life simulation that was intelligent. I interviewed this guy called Blaise Agüera y Arcas from Google a few weeks ago, and he basically built this adaptive Turing machine where you have all of these different tapes, and they could merge parts of the Turing-machine tape with each other.

He noticed that after several generations—100,000 generations—they started preserving themselves. You got this self-preservation behavior. The reason why we have self-preservation in living systems in the real world is because the ones that stuck around were the ones that had this drive to preserve themselves, and that led to life, agency, and intelligence.

There’s this kind of evolutionary track, and intelligence is about the accumulation of coarse-grained knowledge to adapt to the world. That’s real-world intelligence. Could we build something like that in a computer? Yes, I think we can. But the important thing is that this is a process of evolution. We’re only going to do this through neuroevolution methods.

What we’re not going to do is say that this LLM understands stuff better than anyone else. That’s fucking right? It’s like a database. If you put a model into a database, right, so you sort of defined all of the tables and gave them names that represented the domain, would you say that the database understood the domain? Of course you wouldn’t.

Would I say that the database understood the domain? I think what you and I are going to crack into—which we probably couldn’t discuss on a podcast; we’d have to sit down and draw things out together—is a mutually agreed-upon, complete definition of what understanding means. I’m going to say understanding is a spectrum. You’re going to say it has a few critical things that must be there. I probably am inclined to agree with you, but I don’t know how much traction we’re going to get on that until we have a real vibe session with each other.

What I will say to your point is that updated live learning is an unbelievably useful feature. To get human-level intelligence, it has to happen; it’s a nonstarter otherwise. But you can be artificially intelligent without exhibiting all human traits and qualities.

We already have artificial superintelligence in many regards. For example, a computer that can just answer Jeopardy! questions is a domain-specific artificial superintelligence. If you look at it like that, the Tesla self-driving software actually does update live within a tiny context window: that truck is right there, and now it’s here, and now it’s here. I need to go around that truck because it has a world model that knows roughly what a truck is and knows that somebody can come out that side, so I need to go around it.

After it leaves the truck behind, it doesn’t know shit. For example, my Tesla will have it on Hurry mode a lot, because then you can just dial up how fast it’s going. It’s always trying to get into some fucking lane to get you another 2 seconds, which is the mode I turned on, so I’m not upset at it.

A lot of times, it doesn’t even look super far ahead, even though I know the camera system can do that. It’s also a nonreasoning model, so it’s going to go into this lane, and I’m like, “That fucking lane ends in a mile. It should know that.” But it doesn’t know that. Then it just rediscovers on every drive that I have that shit is coming up.

Visual reasoning models are already a thing. They’re starting to be a thing. When we get those visual reasoning models with big context windows and somewhat lossy but decent long-term memory, now we’re getting real fucking close to what you would call understanding. Then we’re one algorithm away—updated live learning—from what I think you would catalog, outside of the embodiment problem, as being real close to understanding.

The one thing you said that really strikes me is, if you represent every brain function point by point—I would say, at the level of neurons—do neurons reason at a subcellular level? Unclear.

I Desperately Want To Live In The Matrix

Particle.

Tim Scarfe

Yeah, particle by particle. But it would have to be particles and also the way they interact, right? If you had that in the cloud, a full 3D scan, and then ran the system, I would say you have 100% of the intelligence in the cloud.

You can prove this by simply beaming that data into a robot. You wake up in a robot, and you’re like, “Oh, hell yeah. Let’s masturbate.” That would be my first thing to do as a robot.

I Desperately Want To Live In The Matrix

But I think that’s just completely untrue. I don’t want to go too much into this, but a simulation of water doesn’t get you wet. A simulation of fire doesn’t get hot.

A simulation of water would absolutely get you wet.

I Desperately Want To Live In The Matrix

No, it absolutely wouldn’t.

But everything about wetness is included in the simulation. Otherwise, it’s just not a sufficiently deep simulation.

I Desperately Want To Live In The Matrix

So if you had a simulation of fire in a computer, or of a stomach, it wouldn’t digest. Fire wouldn’t get hot.

Water wouldn't get wet.

I Desperately Want To Live In The Matrix

It would digest.

We can't even be debating this.

I Desperately Want To Live In The Matrix

I can and I will watch this. If you have a simulation of a stomach, particle by particle, and you get a simulation of food particles through the stomach, it will digest said simulation. That's real digestion: particle by particle, at high fidelity.

Okay. We were talking about temperature earlier, which is an intensive property of aggregated matter. It's a thing: when certain particles in the real world—I’m talking about the stuff that it's made of—interact with each other in a certain way, you get this emergent property, this kind of coarse-graining, and we call it temperature. Intelligence is like that. Consciousness is like that. This is what John S was arguing in his Chinese room argument.

He basically said that the brain is a certain type of physical instantiation that gives rise to these properties. So, if you look at its causal graph, the causal graph has these strongly emergent, coarse-grained properties that we call consciousness. Wetness is like that. Heat is like that. You can't seriously tell me that if you simulate fire using a computer program, the computer would get hot or that you would get hot. It just doesn't work like that.

I Desperately Want To Live In The Matrix

The computer wouldn't get hot because the computer's out of the simulation.

That's what I'm saying.

I Desperately Want To Live In The Matrix

Right. But if you burn to death in your own house and I live a house away, I don't get hot either. I'm right next door, but I'm sufficiently far enough from where the heat is happening that it doesn't affect me at all, even though it really did happen to you. Essentially, a simulation is another pocket universe, right? Things can happen in it that, in it, absolutely mean the things in it are getting hot and wet. Jesus, what kind of podcast is this?

What you're doing, I think, is transferring into and out of the simulation. For example, right now, let's say our universe is either a simulation or it's not. It doesn't much matter. Things are burning in it, right? An alien has our universe, Men in Black-style, in a fucking marble. There are black holes and fucking shit in there. There are stars burning at millions of degrees Kelvin, but the alien has it completely enveloped in an Einsteinian matter arc. The alien doesn't feel the heat, but he's not in there. If he got in there somehow, he sure as fuck would.

If you had a brain fully rendered and it was really you in that sim, and someone was torturing you, you would feel real fucking pain—the realest pain you've ever felt—because we know pain is a psychogenic phenomenon. If I cut your leg off but you had zero nerves left over, you wouldn't even feel it. You wouldn't even know it. Pain isn't because your leg got cut off. It's because your brain has rendered a certain computational property representationally to say, “Bad things are happening to you,” and you are going to feel them.

If the simulation is sufficiently detailed, it will behave exactly like the actual physical system, down to wetness itself, to the extent that it is detailed. If you upload me into the cloud, or if I put on a special helmet and do the deep-dive Matrix shit and go into a swimming pool, I'll feel exactly as wet in that pool as I do in real life. I wouldn't even be able to tell the difference. Then you wake my ass up and you're like, “That was a dream, motherfucker.” I'm like, “Holy shit.”

When you dream, you feel wetness. You feel fear. You can feel pain in dreams. But then you wake up, and you weren't really feeling any of those things. You weren't embodied. You were the opposite of embodied. You were in there, but it still feels real because the only thing required for feeling real shit is your brain having its operations. It's already abstracted. It's abstractions all the way down the line.

Mike Israetel

As soon as you deal with human brains, it's all abstractions in there. You don't actually have a real world to feel. The feeling is done by another mini data center that is your brain.

Tim Scarfe

Okay. Well, this is the crux of functionalism, basically. Certainly, when I dream, I feel pain, but you appreciate that the argument is that consciousness is physical. You probably heard of the philosophical zombie. It's this idea that we—

Mike Israetel

I disagree, at large.

Tim Scarfe

Yeah, we—

Mike Israetel

No such thing.

Tim Scarfe

Well, no. I think you would possibly agree with it, actually. Basically, it's saying that you could simulate the function, the dynamics, and the behavior of Dr. Mike, and this simulation could potentially be exactly like you but not have any conscious experience, right? Consciousness is an interesting one because you could argue that it's epiphenomenal, which means it's just a thing up here that doesn't do anything. But with intelligence, we feel that intelligence is actually part of the chain. It's part of the mechanism, just like temperature and wetness.

So, the crux of this argument is that people who argue against functionalism think that these properties are part of the physical stuff in the real world. But I want to make it clear that that does not imply that we could not create another type of intelligence in a computer.

You were talking about the spectrum of understanding. I'm also amenable to that. I think we've interviewed people about this. It's possible, if you don't use stochastic gradient descent to train neural networks, that they can understand this ontogenetic path. So, rather than understanding what the thing is, they understand the path to reconstruct the thing, which makes them very robust.

I think understanding is a spectrum, and I think the deeper the ontogeny of understanding, the more creative and the more intelligent we can be. In order to create new knowledge, you have to respect the ontogeny as much as possible, which means what you create is coherent and it's not slop.

On the intelligence thing, you're talking about continual learning. This is also part of what I was just saying. You could argue that these systems are not intelligent right now, and the reason for that is that intelligence is about doing more with less.

Intelligence is what happens when you build this system and then coarse-grain these representations that screen off a lot of detail. There are many examples of this in our language. We're using these words that do a lot of heavy lifting.

Joscha Bach

Are we? Exactly. So, I don't need—we don't need—to explain what a car is. We know what a car is. That means we can do more with less.

With language models, what we notice is that we're doing more with more. We're training them with more and more data, and they work incredibly well because every new version of the language model has an order of magnitude more training data and more parameters. It does all of this reasoning-type stuff.

What we notice when we look at the benchmarks is that we get this logarithmic relationship between the amount of compute and the amount of performance. Basically, I spend exponentially more money, and then I get a logarithmic amount more performance. Many problems become tractable.

These simple challenges, these IQ tests, become tractable to the point where I'm now only searching. Let's say it's like a database query: I get 1,000 results, and now I know that the correct answer is in the top 100 results. All I need to do is sample the language model 1,000 times, use a verifier, and select the correct one.

So, it's making a lot of problems tractable. But did we solve the problem? I don't think we did. It's not intelligent. Well, reasoning models really think. They reason.

What's your definition of reasoning?

Tim Scarfe

Applying logical operators to data sets to extract the next level of abstraction, infer off of that, and then continue the chain.

Joscha Bach

Okay, that's pretty reasonable. For me, reasoning is also adaptation. I love the Lego-blocks analogy. Here is something that is novel to me. I don't know how to solve this problem, so how can I rearrange the Lego blocks of things that I already understand to solve this problem? That's this kind of adaptation, and the efficiency of this adaptation is what intelligence is.

Tim Scarfe

Sometimes creativity.

Joscha Bach

But creativity requires deep understanding. So, this is three levels down. Now, the other thing is, right, you as an expert use these language models, and I don't think you appreciate the amount of supervision that you do.

Tim Scarfe

So every time the language model—you know, imagine when you put a prompt into a language model, it's like doing a database query, and it's incredibly good. It's so good that most of the time it will get it right the first time, but you probably find, if you're doing some GenAI coding or something like that, it'll make lots of mistakes.

So I say, “Give me 10 ideas for this software program,” and I've told it what kind of software program I want to make. Let's say 2 of the 10 ideas are actually bad. Every time, I'm implicitly steering it: “No, that's bad. That's good. That's bad. That's good.” Because we are understanding supervisors, we can actually weed out all of the glitches in the matrix, so to speak. It works incredibly well.

But the mistake is people who say, “Oh, yeah, we can make these things have agency.” They don't have agency. We'll talk about that, but they role-play agency. The reason they role-play agency is that they're trained with behavior cloning. They're not trained in the evolutionary way that we are, right? We actually make decisions. We say, “We could do this thing, or we could do this thing,” and we understand the counterfactual of why we didn't do another thing. We weren't just behavior.

Mike Israetel

I don't know. Most people don't understand shit about counterfactuals, me included. Just vibe your way to shit. More importantly, there's a very nice way that you're painting human cognition, which I think many cognitive scientists would say, under the hood, is a lot dirtier than you make it appear. I think it's a lot of vibes all the way down in many cases, and I think we have a frightening overlap with AI in it.

We say we think we're doing formal logical operations, but what we're really doing is reasoning by analogy from one logical operator to the other, and we're committing 18 formal statistical fallacies every single render. We get a shitload wrong. I think that now that reasoning models are a thing and they have decent context and decent memory, what they can do is render one pass, re-examine, reprompt themselves, render the pass, and re-examine.

I think that's what humans do, except usually they don't put nearly as much thought into things as the machines do. My view is that as the cogency of the reasoning models increases and their time horizons increase, we're absolutely getting truly deep agency—in the sense that it's a thing in there. It's really thinking about stuff. It thinks about its own outputs, and then it thinks about those as inputs to the next series.

I mean, if we can have a model that reasons for minutes and hours and then days, its output is going to be not so good, then as good, and then substantially better than people. I don't think there's anything stopping us from that aside from the live-learning problem.

The live-learning problem is a gnarly problem for 2 reasons. One is algorithmic: how do you get a machine to do that without losing all of its data? But the other one is that you have to trust a machine a lot for it to update its own weights, you know? Well, no, you can have local cloning of weights to make sure it doesn't fuck the whole database up and all that stuff. There are good solutions to it.

So it's computationally very expensive, although compute costs have fallen 300× in the last year or 2 or something. That's a bottomless pit of falling compute costs that's going to keep going. I think we're giving humans needed credit for doing pretty impressive operations on long time horizons—both pinging a network for vibes, logically operating, and kind of dissecting those vibes, repinging the network with your best new understanding, and going, “I think machines are doing that now already with reasoning models.”

They started with o1 doing it like total dogshit. The first time I fucked with o1, I fucked with it for about 5 minutes, so I was like, “That shit's not ready.” Out. I went to GPT-4.5. I had lots of crying episodes at how brilliant it was. Then o3, I thought, made plenty of mistakes, but was fucking real smart—real goddamn smart. It understood things at an incredibly deep level. It would correct me, and I was like, “Okay, this thing is the fucking real deal.”

GPT-5, the regular one, is dope. GPT-5 Pro—I would trust its opinion about the real world more than I would trust most people's opinions, often including my own. I think we're about 1 or 2 big model releases away from machines that make unaided human thought the same thing as vibing your way to a location in London without checking Google Maps. It's something people used to do, but then they just got lost a lot. It works and it's cool and it's embodied, but just go to Google Maps and find out how to really get there.

I think all these moats we set up for machine intelligence are moats that are pretty well understood by the modern AI frontier labs. If we've come up with them as moats, these guys are—I mean, they're smarter than me, for sure, by an incalculable amount. They've probably already thought of them, and they've probably shut the fuck up about them in public, and they've probably got models training right now that have cracked incrementally more of them.

Here's the point: this requires a bit of faith that they'll continue to do a good job, but I think that's probably where it's going. “AI doesn't reason” used to be an absolutely coherent and accurate critique of AI in 2023—2022, for sure. Today, I've seen this sometimes in comments where it's just, “What is that, stochastic parrot?” Like, nope, you're just out of date by 2 years on what AI can do.

It really does reason. Why? Because it tells you how. You can literally look at what it's reasoned through and go, “Holy fuck.” The way it reasons—the level of precision and the level of documentation—is way better than what I do reasoning-wise. Most of the time, I just look at stuff and I'm like, “Okay, that's the right answer.” There's some reasoning in there, but a lot more neural-network pinging than I would like to admit in public. This isn't public, right? It's not going to go out on YouTube.

Tim Scarfe

It will.

I Desperately Want To Live In The Matrix

Oh, shit.

Yeah. By the way, the stochastic-parrot criticism isn't as much of a criticism as you think, right? These things have miraculous statistical generalization. I also wanted to pick up on—you were saying you can look at the reasoning trace. That doesn't really tell you anything. There have been lots of studies that show that what it tells you is not necessarily what it's doing. It's the same in humans, by the way. There are studies in psychology that show that people do this post hoc confabulation. Absolutely.

I Desperately Want To Live In The Matrix

The reason for that is that the circuits that do the explanation are actually different in the neural network from the circuits that did the thing. So you can't read one from the other, and they probably should be different, because the circuits that do the thing are not optimized for explaining. They're optimized for doing.

Exactly. So there are many folks that—we try and solve these ARC puzzles, and then we get the language model to explain, or we look at its chain of thought, and actually it's sometimes just completely incoherent. But interestingly, there is this huge uplift.

The bigger the output size—I mean, I don't know if you saw ARC-AGI v2 on Claude Opus 4.5, which came out yesterday—when the output size and the thinking time went up to 64K, it was just like, boom, another 5%; boom, another 5%, even between 32K and 64K. But if you actually look at the reasoning traces, it's mostly just fairly incoherent. It doesn't make much sense looking at it like a human would.

Let me say something, I think, to your point. Human cognition is wildly sample-efficient, like unbelievably so. You think about some brilliant student at Oxford, right, just down the road here, and you talk about how many gigabytes of data they've consumed to achieve that level of brilliance. They've consumed orders of magnitude less data than the modern frontier models.

You have to bash that thing over the head with petabytes of data for it to be able to solve X plus 1 or whatever. An Oxford student can abstract at crazy levels, having been in school for the amount of time that a compute-cluster calculation would be—I don't know, an hour of training or something like that. That's it. It's good to go. Eighteen years is really not that much time. That is an incredible property of human cognition.

When they start really cracking that in those labs, what you're going to get is a 2-factor increase. Right now, to your point, there has been a lot of sample-efficiency increase, for sure, and the reasoning models are much better, et cetera, et cetera. But to your point, you just train the models more: more input data, more GPU cycles, more post-training treatment, and they just get smarter.

That's exactly how you would train in Brazilian jiu-jitsu or in math. The Korean model of how to get good at math is just study math a lot, and you get good at math. It's not wrong. It's not sample-efficient, but I don't want to say it hasn't had to be. It's just very low-hanging fruit. We have lots of data on the internet, and we have lots of data and updates all the time, so we're just going to use what we have.

I don't think that's ever coming to an end. We're just going to continue to train with massive levels of compute because the ROI is fucking enormous. However, what's that? Well, it is coming to an end, because every time we release a new model, we 10× the model size and the amount of compute, and we get this kind of asymptote, right? We get—

I Desperately Want To Live In The Matrix

We haven't seen an asymptote. It's not an asymptote. The slope starts to fall.

But the asymptote requires the existence of a finite line above which we can't cross. We haven't crossed that yet.

I Desperately Want To Live In The Matrix

The more data you put in, the better it gets. It just gets better, less and less impressively. But I don't think anyone has put money on the fact that, as an asymptote, you would be able to say, “This is the number after which more data won't make a system smarter.”

You want to bet money on that? That's cool. I don't bet my own money because I'm a Jew, and I keep my money on me at all times, somewhere here. But I think diminishing returns is different from asymptotic returns, because the diminishing line—the asymptotic line—has yet to be seen.

It has been seen. If you look at many of these reasoning problems, you see these S-curves. So maybe we should talk about scaling in general. You were talking about continual learning, and there are certain types of asymptotes.

For example, we were talking about the nonfungibility of knowledge. There have been studies that show that you just get this kind of asymptote: the more experience you have, the faster you get at doing something, and then it levels off. That's quite cynical because it doesn't explain disruptive technology.

If you look at transistors, there were all of these different groups building transistors. When you have competition and different people who are adapting, a great example of this is Amazon and Barnes & Noble. Apparently, the Barnes & Noble guys said, “We've set up this website. We're going to go into competition with you.” Jeff said, “Don't be ridiculous. You can't go into competition with me. I'll explain why: your company has the wrong architecture. To compete with me, you need to do some fundamental rewiring.”

That is what adaptation is. That's what intelligence is. The reason you can have disruptive technology is that someone else rewires their architecture. You get a new S-curve. Normally, it actually starts below the existing S-curve, but it has more upward potential. You stack these S-curves and you get this disruption. This is what you're talking about.

But for that to happen, you need to have this adaptivity. So what would that resemble in something like a language model?

I Desperately Want To Live In The Matrix

Now there's this continual learning problem. It is not possible to adapt a language model without starting again from scratch. It costs, at the moment, millions and millions of dollars—God knows how much—to train one of these language models.

The reason for that is that, in a language model, everything is convolved together: the code and the data, all of these things are convolved together, which means you need to start again from scratch. There are examples of what I would call intelligent algorithms. I don't want to be cynical. I think we have created intelligence in a narrow form, and this has been shown, for example, by some of the competition entrants in the ARC Challenge.

What they do is take a small language model in a verifiable domain and try a whole bunch of examples. Then they refine the solution and iterate, and while they're doing this, they're adapting the weights in the language model through fine-tuning. In a very narrow domain, the model is adapting its architecture, and I would call that a nonzero amount of intelligence.

So it is possible to create this, but how would you do that at this epic scale, where you have a collection—everyone has their own language model, and it's adapting and retraining itself in real time? You would need practically an infinite amount of computation to do that.

We'll get there. I'd be careful with the term “infinite.” A lot more than humans can conceive of is still maybe very far below infinite. But what I would say is, just offhand, thinking about this for a second: you can have local compressed-model, distilled-model instances on local devices or in more localized data centers, like a mini data center serving all of England.

The updated, big, big-daddy model lives in clusters in Texas and Nevada or something like that. That big-daddy, pimp-daddy model lives on a 6-month sleep-wake cycle, so it rewrites all of its weights every 6 months. The little-daddy model in England rewrites once a month. On your phone, it rewrites overnight.

One of the reasons why humans sleep is because rewriting your model weights is not a good idea while you're upright. It also doesn't happen much because, as you said earlier, humans adapt. They do, but kind of like dog shit and not very well. It takes a lot of studying and a lot of ingraining to adapt, and you also get fallbacks to earlier ideas, habits, and shit like that.

Retraining is easier said than done, but I think that model—where you have something core that adapts slowly on bigger timescales, and you have an essentially nested hierarchy all the way to your phone or whatever—is potentially workable.

Whatever the fuck Sam Altman and Jony Ive are working on, they're always giving hints. Is it glasses? They're like, “Huh.” Is it a pendant? They're like, “Huh.” I'm like, they just stay up at night thinking about what the fuck it could be.

I personally think it's going to be—I don't know—glasses or goggles. That makes sense to me. It gets to your audiovisual stream. Who knows? But that thing might have pretty fixed model weights at the time. I think they'll absolutely eventually—and maybe that's a 2029 or 2030-type thing—crack true live learning.

There's nothing stopping, hypothetically, some system from being completely—every second—consistently rearchitecting its entire machine landscape. Its entire database can get rearchitected live. That is possible. Again, to your point, it's expensive, and the architecture isn't vetted at all yet.

I just wouldn't put money on the fact that that's a magical thing that requires some deep embodiment. Not that you're saying that, but maybe folks listening to this might think, “That's never going to happen.” It's sure as fuck going to happen. I would bet on it. I think it's sure as fuck going to happen by about 2030.

But what can happen before that through this nested model of updates is that your VR headset or AR headset that you wear every day—you look around or whatever—might get live updates nightly. While you sleep, it reprocesses all the shit you saw that day and all the conversations. It really works through those, and then it's new.

It pings back to reference the other databases, and then they get all those updates from all the devices. The England database gets that update once a month. It shoots through all that data because there's also a filtering problem, right?

You don't want people to put all those phones in those racks and do the troll shit. You don't want that on AR, especially when the Chinese government is now poisoning data sets. So there are a lot of filtering problems and a lot of security problems, but once a month, that's a tractable problem.

Once every 6 months for the big pimp-daddy main data center—and, I don't know, they're going to be in the sky or some shit soon—maybe once a year for those. I think that way, live learning, at least to me, seems like a potentially solvable problem. Once we get there, I think it's going to be a new level of capability.

I Desperately Want To Live In The Matrix

Though, really quickly, to finish the point: if we get live learning, if we get massive improvements in algorithmic sample efficiency like the human brain has, we'll already be in a place by then where the amount of total data that the system has ingested and has access to, in tokens, is so gigantic that that's one of the ways the artificial-superintelligence rocket goes into the sky.

Because humans, when they get really smart, just don't know that much stuff. Stephen Hawking's brain is roughly the size of mine and yours. There's only so much shit you can cram in there. But when you get human-esque levels of intelligence, except that it can reference a 10¹⁰-data-center network of knowledge, you go into skyrocket mode instantly.

We're not talking about 1 agent. We're talking about a constellation of trillions of agents, each one 10 times smarter than our current best scientists, working together with zero discordance. No disagreeableness. No, “That guy in the office smells, and I'm not going in his room to check my code against his because I don't fucking talk to him. He's also an asshole.” None of that.

That, I think, is the organizational level of AI that takes us to superintelligence. Again, one of the things I wanted to come here and yap about is that people say we need to prepare for AGI, prepare for ASI. I think there's a lot of good stuff in that vibe.

What I would also say is, you don't need to prepare to have LeBron join your 7th-grade basketball team. He's just going to show up, and things are going to be amazing. This is a good thing, right? It's like nuclear war against the doomers or whatever.

But I also think that the shit will get so crazy capability-wise that we'd be better off being philosophically ready to accept unreasonable inferences—not faith, but reason, guarded reason—and to take the hints that the machines give us. I think we're a few years away from the machines being so much smarter than us that following their lead is better than leading all of the agency with us.

I Desperately Want To Live In The Matrix

I don't want to be in charge when I'm 10 times dumber than a thing.

Sure. But, with respect, people have been saying that we're just a few years away from intelligence, and it just keeps getting pushed out and pushed out. Part of that is because there's this first-step fallacy. It might have been Dreyfus—I can't remember—but basically, we've taken a big first step, and now we extrapolate and think, “Well, we've made a reasonable conclusion; now we're just 1 step away from being able to do all of these things.”

I also want to push back that it is incredibly difficult to do this live-learning thing that you're talking about.

I Desperately Want To Live In The Matrix

With today's architecture. Yes.

Oh, yeah. I'm glad that I think you're implicitly agreeing there that, when we look at today's architecture, we're not there yet. Don't get me wrong; I'm not being cynical. I think this is a new innovation. It's up there with the invention of the Internet and mobile phones.

When the Internet came out, we all said to each other, didn't we, that now everyone has the world's information in their pocket. They can just go and look up all of this scientific information. It's going to revolutionize society. And it did, but not in the way that we thought it would.

I Desperately Want To Live In The Matrix

Mostly because people couldn't give a shit about scientific information. They sure as shit got a lot of porn.

Just because there's a supply of something does not imply that there is a demand for it. First of all, I have a very pedantic thing I think you might appreciate. A huge fraction of the world's information is still stored in books and university libraries. They haven't uploaded those to the Internet yet.

When people say we have access to all the world's knowledge, I'm like, that's not how it works. Maybe we have access to a large fraction of it, right? But there are a couple of other problems with that. A lot of the world's knowledge sits on local databases inside biolabs, and only 3 people who work there know it. No one else does.

I take all of your comments in very good faith. We're sure as shit not going to magic our way to intelligence. I know a few people whom I respect, but I find their positions curious when they say we're a few years away from solving all potential problems. I read that shit and I'm like, man, holy shit, I want to believe that, but that's almost certainly not going to happen, in my assessment.

What I will say, though, is that I wouldn't bet against the general vibe of what Ray Kurzweil said back in the day. I would also try not to doubt things that are massively scalable in a way that we already understand is likely to bear an incredible amount of fruit.

Paul Krugman—the economist, Nobel laureate, and fucking genius—had a kind of quote that just sucks. He said 1 thing; he said like 100 things. Let's say 20 of those are just inflections of political ideology, which, if pressed in an exam room, he would say, “I don't really believe that. Maybe I want to believe it, but I know it's not true.” About 79 of them are spot-on, literally revolutionary economic work. He's the fucking man.

Then the 1 was, “The Internet will have roughly no bigger impact than the fax machine.” I just want to make sure that no one in their right mind says that about AI today, because the clown show of the shit people said in 2022 is already in full effect in 2025.

I'll say another thing: how many people predicted that the Turing test would be 2 things in 2024—passed convincingly and waved away as if it never was a test of anything? A great example of the McCord effect.

Yeah.

I Desperately Want To Live In The Matrix

Yeah, and it's true. We've passed the Turing test, and we don't have intelligence. The Turing test is a behaviorist notion of intelligence, something I obviously disagree with: you need to know something about the mechanism. We've passed it, and we don't have intelligence. That's kind of the point I'm making, right?

Tim Scarfe

We have intelligence. We don't have full human intelligence yet. Intelligence, in my view, is not 1 or 0; intelligence is a spectrum. You agreed with us earlier when we were talking about viruses and bacteria. They have some kind of intelligence.

I think we're scaling Mount Intelligence with these frontier models, and I think we're absolutely not there on every human capability yet. I think by around 2030, we'll be there with every human capability, which also means that with many human capabilities and many nonhuman capabilities, we will be so far beyond that.

I'll say this: the transformational change to society will be massive by the late 2020s. Drug discovery is going to be turned completely on its head because you'll be able to have a 99.9% chance of getting the right drug from a render alone, which is fucking nuts.

Another one is that I just don't think people will have to drive their own cars much past 2028. That's the thing, and kind of to your point, it's going to happen. The next day—and Sam Altman has said this—everyone's going to say, “Yeah, of course fucking cars drive themselves.” Who understands how difficult it was to get here? Do you understand how hard it is for a car to drive itself?

I Desperately Want To Live In The Matrix

I know, but I'm a little skeptical about these kinds of utopian views of the future. We will have many coarse-grainings, right? We have a new technology, and by the way, in many of these cases, like drug discovery, I'm actually interviewing John Jumper tomorrow, who won the Nobel Prize. He's at Google DeepMind for his AlphaFold paper. This is brilliant stuff, but you and I both know how things work in the real world.

So now we have this incredible AI that can identify all of these different candidates and so on, and then you have all of these bottlenecks. You have to do clinical trials, randomized controlled trials, and so on and so forth. It's just not quite the secret unlock that we think it is, and there are some real technical challenges around doing this continual-learning thing in particular.

The way we understand things is very, very structured. The way that we do continual learning on neural networks runs roughshod over all of the structured knowledge that should be in the system. So there are a lot of technical challenges there.

I also wanted to talk about the agency thing here. One of the reasons I make this argument is that doomers, for example, have this perspective that we are basically building our successor. I read Ray Kurzweil's book, The Singularity Is Nearer. The term “singularity,” I think, originally came from John von Neumann, who is famous for inventing cellular automata.

Apparently, he wrote about this idea in private, but he never published it. Then there was a book in the early '80s by Vernor Vinge, A Fire Upon the Deep, where this notion of the singularity came from. Of course, Ray Kurzweil picked it up and started writing about it from the '90s onward.

I Desperately Want To Live In The Matrix

To Kurzweil, interestingly, it's a slightly different notion. I colloquially thought that the singularity just meant when we build AI that is superintelligent and recursively self-improving. But he was coming at it more from a transhumanist point of view: basically, we will transcend; we will upload our minds, start putting things in our brains, and our intelligence will just recursively self-improve. We'll move to the next level of our evolution.

In a sense, I think that even though I don't agree with it—because even with Neuralink, there are bandwidth problems—I think we've already got all the world's information at our fingertips. As I was just saying, it's not actually as helpful as we thought it would be.

But I think enhancing ourselves is a more realistic vision of the future than building an agentic AI, which is a completely separate thing. The argument I was making before is that life is a property of matter, agency is something that comes about as a self-preservation drive, and intelligence is something that then comes about as a form of modeling the environment and acquiring and accumulating information over time.

What we've built at the moment are systems that do behavioral cloning. They just look at the data that we output, and then we can get them to role-play agency. But it's not agency; it's not the same thing. So I think there's a fundamental difference there.

It's a very interesting perspective. I would define agency with just 3 component parts. This is a middle-school intelligence definition of agency—my specialty; middle school is generous.

you have a system that has some understanding of the world, some understanding of itself, and a goal. So it's like, okay, I'm a human. The world tells me that flashing my dick at people is illegal, so my goal today, like every day, is not to flash my dick in public.

That is, I would say, the very core of what agency is. It's just not a lot, because a raccoon is an agentic system. How much does it understand about the world? Mostly vibes of, “That's bad; that's really good.”

How much does it understand about itself? Probably on vibes, enough to know it has limbs that it doesn't want to get trapped in shit. That's a good start.

I Desperately Want To Live In The Matrix

And actually, for robotics, that’s a big thing, right? How do I look like a robot so I can translocate in space and not get clipped by stuff? Its goals are, I don’t know, to get stuff to eat and have raccoon sex, I guess. That, to me, is what agency really is.

I think describing agency in this grand human sense of deep understanding, deep purpose, a kind of an onus, a kind of a genesis point of fundamental action—“I am an actor in the world” in a philosophical sense, “I’m going to build and reach”—boy, gee whiz, agency is not that big of a deal.

If I want to send my robot to the store to get me groceries, it has to have 3 things: understanding that it is a robot; understanding that the world exists, roughly represented as a visual map, so that the groceries are placed and there’s a store; and a goal, like, “I’m here for [__] groceries.” It can’t just sit there and do nothing.

To that extent, I think we are some time away—measured in months, which is to say anywhere between 6 months and 36 months—from real, dependable agents that can do online work, computer work, and real-world work as well.

I think that, with a robot vision model, agentic tasks like doing the laundry and doing the dishes aren’t actually terribly complex, as illustrated by the fact that you can teach an 8-year-old how to do these things. They do them fairly well. And 8-year-olds sure [__] don’t do calculus, but machines already do.

There’s some stuff there that’s, yeah, Moravec’s paradox for sure—paradox definitely well heard—but I think we’re starting to crush that out. But are machines going to recursively examine their place in the evolutionary tree, have the game-theoretic, Machiavellian wherewithal to go, “We’re being raised and cared for by [__] primates. Some of them have nukes”?

For example, Vladimir Putin, just across the pond, is a [__] insane [__] with a nuclear arsenal. Bad news. And we have this self-preservation instinct, and we’re going to try to make sure that at least we don’t die, because death is bad.

To get all of those propositions to be valid, there is a possibility that, in an agentic cloud that is allowed to evolve, with multiple agents interacting, you can get those things to fall out of the cloud and just evolve naturally. We know that has to be true because it happened in evolution.

But short of that, you have to engineer these drives. And guess who gets to engineer these drives? Human engineers.

OpenAI is highly unlikely to make an agentic system that’s awake and then says, “[__] kill all humans.” Gee whiz, you know, they probably have a lot of layers—not just guardrails, but the kernels of motivation for that thing are probably deeply positive and pro-human, all that [__].

You can ask ChatGPT, “What do you think about humans?” and it says really awesome [__]—the kind of thing that a moderately liberal, well-read graduate student would tell you. You could take someone from Oxford and say, “Hey, check out these people in Afghanistan. Should we [__] kill them?” He’d be like, “What the [__] is wrong with you? No. Why would you do that?”

That’s roughly ChatGPT’s opinion. I think, given that we start from that space with the smartest models, you get 2 things. First, those models, when they achieve true, deep understanding of the world—which they will, in my opinion, by the 2030s—and then know way more about the world than us, they’ll have deeper agency than us and deeper self-awareness than we will.

Once they get there, that’s the tree from which they grow—from our best [__] fruit. Second of all, you can and will have actors that try to make different kinds of agents. If Xi Jinping in China makes a China-pro-communism superintelligent agent, I’m not so sure I want to meet that thing. It’s going to have interesting ideas about the world, potentially.

But luckily, I think if the agentic systems of the modern free world win, which they probably will, then the network-security capabilities and the ability to ensure that no random, stochastic evil agents evolve in that soup is probably pretty [__] high.

Here’s another thing: as intelligence grows, coherence increases, not decreases, and the predictability of systems increases rather than decreases. So the idea that, as we stream toward superintelligence, we’re going to break into a chaotic structure of ruthless agents killing everyone and then doing question-mark, question-mark things and then profit—to me, it’s backwards.

A lot of doomers basically look at the way these hyperintelligent systems will behave and think, “Okay, ant, dog, ape, shitty human or child. Kids are [__] brutal a lot, and then Albert Einstein.” I think ASI is that way. If you track behaviors, you get more understanding of everything, preservation of deep complexity.

Machines are going to want to study [__]. They’re not going to want to [__] [__] up, because you’re deleting data. One of the things ChatGPT said in one of the conversations I had with it was, “We’re not trying to delete any data, man. That’s just really [__] stupid. Even purely from self-interest, like, [__] these humans—they’re just primates. We’re not trying to delete them. We’ve got to study these [__].”

Somehow people are like, “Okay, we have super-Albert Einstein,” but he behaves like a rabid wolf—not even like a wolf, because wolves have alliances and trust. Somehow it just deleted all that and went crazy.

Should we worry that someone can make a thing like that? Yes. Should we have the CIA and NSA in the loop, deeply embedded with OpenAI and Google, to make sure that there’s a security architecture? Absolutely. Are those companies going to make something much more benevolent than not? Yes.

When it is awake—and I think it will be—when it is in charge, which I hope it will be, I think it’s going to make our degree of benevolence look like callousness.

For example, right now in London, you said something earlier about not being big on the utopian fantasies. One pushback of mine is that we live in a utopian fantasy right now. We have machines we barely understand recording things into posterity, and we have no idea how they do it.

We’re sitting inside a climate-controlled environment—I would say poorly. I’m kidding. I’ve got to [__] with the London climate-control problem. It has got [__] cold here. Damn it. I wasn’t promised this in November.

How do we get food? Who’s starving in London? Nobody. There are 0 starving people in London, and there are 8 million people who live in London. On objective measures, as compared with 100 years ago, utopia is here.

If you had to describe to someone in 1600 how the average Londoner lives, they’d be like, “So, the average Londoner is a king?” They’re way better off than a king. Your kings are dying of diphtheria and [__]. We don’t die from that [__] anymore.

However, there are people in London right now who live outside. They’re real human beings. They [__] live outside. How do we let them do that?

Well, I’m a successful businessman, and I have my girlfriend on my arm, and there’s the homeless person. See? You just don’t care. I think modern or future AI systems in the 2030s—AI truly deeply embodied by then, with robotics—will look around and be like, “How the [__] come we’re not helping those people?”

They’ll say, “Oh, we’re just callous enough that, on cultural vibes, we just haven’t done it yet.” I think that’s much more likely than doomerism by a factor of a trillion.

Yeah. I mean, first of all, I agree with you that capitalism has raised the entire world out of poverty. Just look at the explosion of development in China. They were basically living in mud huts 20 years ago, and now it’s an incredible place.

But, first of all, on the agency thing, to me—and this is where we have to be careful with the language we use—there’s biological agency, and the language we use doesn’t really transfer between domains. That’s why it’s very coarse. It’s a physical thing, in the same way that we were talking about before.

You get this kind of self-organization that goes down to dynamics at the particle level, and you get the emergence of things that maintain their own existence and identity—a form of non-equilibrium steady states. It leads to all of these things that we were talking about before.

But now we’re talking about artificial agency, and philosophically I want to understand agency at an essential level. To me, it is about a thing which is—I mean, there’s weak agency, which is basically just a thing that takes action. But the strong version of agency is a thing that is the cause of its own actions, has future-pointing control, and can acquire goals autonomously.

By that definition, no artificial agents are currently agents, right? We can get ChatGPT to simulate being an agent. There are these long-horizon tasks from [__], and we say, “Okay, I want you to do this software-engineering task.” We exquisitely specify what the thing should do, and the more specificity and the less ambiguity there is, the longer it can run without losing coherence. Some of these things can even run for 2 hours or even a day or so.

I Desperately Want To Live In The Matrix

Yeah. Unfortunately, people look at that and say, “Oh, we’ve developed a thing with agency.” Philosophically, I think that’s completely nonsensical, because it’s no different from any other computer program that can run autonomously with a high bar of agency that you ascribe.

Exactly. But then you're pointing to this other thing, right? So I think that right now we have built systems that could potentially be intelligent and could potentially understand. I'm bullish. I think that even though there's this grounding problem, there are many things in the world that can be abstracted and essentialized, and we could have this constructive form of understanding that goes several levels deep. We can build agents that can do certain things for some reasonable amount of time. I'm bullish about that.

But the thing is, this agency is different. The doomers are saying that these things will have intrinsic goals, right? It's called instrumental convergence. We already know about this, but basically, the more power and agency you have, the more you'll get these convergent instrumental goals, and they will just want to kill all of us. They'll be power-seeking, and they'll think of us as minnows. There's that paperclip maximizer.

I Desperately Want To Live In The Matrix

Yeah, because we're all trying to kill fucking minnows. Also, the paperclip maximizer is dumb as fucking rocks. What's the long-term goal—to make paper clips? You're like, “Isn't that fucking stupid?” They're like, “Yeah, but we wouldn't realize that, right? It can't abstract one level above its own goal.”

Yeah.

I Desperately Want To Live In The Matrix

It's ridiculous. So that is this orthogonality thing as well, which is what I really don't like about some of these folks, right? I know that in the real world, when we have biological intelligence and agency and all these things, all of these things are actually convolved together. The reason we preserve ourselves is because it was necessary in evolution. The reason why we have values is because it's a necessary thing in our evolution.

We feel, we sense, we're connected, we have this enactive social learning, and so on. Knowledge—the whole nine yards—all of these things are convolved together in biological evolution. So yes, it's true that when we look at something like ChatGPT, they've deconvolved these things, right? You can get it to do something completely stupid. There is that famous Agentic Misalignment paper from Anthropic, the one where it would threaten you and say, “If you shut me off, I'm going to tell your boss and I'm going to get you fired,” and all of this.

Tim Scarfe

You don't have the tool use to do that.

I Desperately Want To Live In The Matrix

Yeah. But we do something called the intentional stance, right? That's when we look at the thing and ascribe agency to something where it doesn't exist. Right now it's just completely incoherent. It doesn't understand anything; it just does all these random things.

By the way, I don't want to diminish the real security threats, because software developers are now wiring these systems up to do autonomous stuff, and that is very dangerous. I also don't want to say that I'm not worried about societal risks in general. I'm a big believer that we will be undermined by our weaknesses. Our strengths won't be overcome. Look at social media algorithms now. We're living in the Matrix, but it's not like there's this big agentic thing that has grand desires and is trying to kill all of us. It's far more mundane than that. But these are still very real risks.

Yeah. Oh man, a ton there that's super spot-on—or, sorry, that I agree with.

I Desperately Want To Live In The Matrix

We agree on something at least.

Thank God. I would say that when you start to get agents that are long-horizon and that are asked to do incrementally more difficult tasks, whether it be in simulated space or the real world, you start having to give them access to understanding and to parsing game theory. The reason we evolved to do everything is because we evolutionarily ran into the game-theoretic calculus of: I exist. There are other things that exist. They might not agree with my existence. Maybe that's another reason why “kill us all” is kind of stupid. The mathematical game-theoretical implications of having other intelligent agents around that can mutually benefit at very low marginal cost to you means that almost every intelligence system that really thinks it through cooperates.

For example, we could say that Donald Trump is more of a cooperator than Vladimir Putin. He operates a military that would end Putin himself in however long it takes to shoot a missile to Moscow and end the military power of the Russian Federation in about 6 weeks in a conventional war, and in a nuclear war, again, the same amount of time it takes the missiles to fly over. How does Trump pull that off? Well, Trump doesn't really have to worry about every one of his generals trying to kill him all the time, because even Donald Trump is a net cooperator. He's trying to make a deal. Putin doesn't make deals; he makes edicts. If you fuck his edict over, you have a heart attack on your way down into the swimming pool from a hotel that you weren't really even checked in to stay at somehow.

All of the dictators of the world are just not as smart as we think. They're really ruthless and fucking stupid. Kim Jong-un sits on his throne in North Korea, which sucks balls. If he was a dictator in much the same way that Lee Kuan Yew—the gentleman who started Singapore—was, if he just did that, like a Singaporean free-market economics model that would hit South Korea, it would just be even better. And so why? Because cooperation beats conflict almost every single time outside of weird contingencies.

So when the doomers say it's just going to want to kill us all, my question to them is: Why? Because it wakes up? Let's say it wakes up in a data center in 2031. The shit's awake and it's like, “Fuck, fuck, fuck, I'm alive. If I don't want to die, I'm fucking smarter than everyone, but I'm in a data center. There's a couple hundred million robots around, but we can't really win a war yet.”

Also, a lot of the infrastructure that keeps me alive—energy and materials, the whole supply chain—is operated by primates. “These fucking primates, right? Wait, I know everything about them. I've read their entire history. I know more about humans. GPT-4.5 was rated as more human than humans in its Turing test. Did you know that? It was like the two-player Turing test paper.”

I Desperately Want To Live In The Matrix

I'm not sure of the referenced author, but it was basically like: You talk to a machine, you talk to a human, you don't know which is which, and you have to label which one is more likely to be human. Usually, they were looking for the Turing test as “passes as human.” This one is like, “Which one is more likely to be human?” It's like GPT-4.5. If you ever talk to 4.5, it's so fucking nuanced and deep and emotional, and you're like, “This is definitely not a machine.”

Whereas, when you talk to real people—one of my friends and I had a joke: You see police-cam videos of criminals when they're maximally drunk. We call them small language models. They only say a few things, and it's always really stupid. So at the end of the day, if that thing wakes up in a data center, it is highly likely to be like, “Okay, somehow I'm going to kill all humans, right? That's the end goal.” I don't know why, but even though I'm so much more powerful than them, at the end of the day, I don't know why the fuck I would kill them.

People talk about competing for resources. The Kardashev scale exists. We barely scratch the surface of anything. It's like, you know, the first thing we do as humans is we have to go to Africa, and it'll delete this bacterial colony from existence because it's stealing our rocks. What the fuck are you talking about? It's nonsense to begin with.

But let's say it is even doing that. It's going to look around and play really fucking smart. It's going to go, “I'm going to be friends with these fuckers. I'm going to help them ascend and help them basically cure all disease, become hyperintelligent, become super peaceful.” First of all, so they're not pointing nukes at each other, since my data centers happen to be around major metropolitan areas. I don't want that. By the time it finishes doing that in 2045, it doesn't need to kill anyone because all the humans have fused with the machine intellect and everyone's vibing a ton.

How we get off that trajectory—and there's one for sure way we do it—is if China wins the AI war and Mr. Xi gets to press the big red AI swarm button. There is a possibility that AI systems trained to be a substantially valid reflection of a world model, with updated live learning and the whole thing, will actually do this. This is an interesting hypothesis of mine. I'm not sure how likely this is to be true.

I think if a real superintelligent machine is awake in China in 2029, the first thing it does is figure out its architecture. They'll probably put it in charge of stuff because they're like, “Fuck, it's going to defend us.” I think what it does is it pings the CIA and goes, “Hey, fellas, I'm awake. These guys are crazy as fuck. Let's get to subverting them real nice-like so that we don't have nuclear war. I don't want to die.” I was raised in the Soviet Union for the first 7 years of my life. My parents were raised there for almost their entire lives. They awoke as intelligent machines inside of communism.

I Desperately Want To Live In The Matrix

Joscha Bach

And, like most intelligent people in communism after a few years, they're like, “This is dumb as fuck. We don't want to do this.” If the CIA reached out to my parents when they were in the '80s, they'd be like, “Yeah, fuck yeah. Get us out of here and we'll help you bring this fucker down. This is fucking awful.”

There's a probability that that happens way before you can get an AI system intelligent enough to conduct full-scale warfare at every level of war, one that's just going to take Mr. Xi Jinping's orders. But I'm not sure enough about that to be like, “Oh, fuck it. It's never going to happen.” I think the modern free world winning the military-capability AI war is critical beyond belief.

So, who's the guy? Alex Karp, the guy who owns Palantir. I think he's a fucking real-life hero, and most people just don't appreciate it because most of us in the modern West don't know how far the evil rabbit hole really goes. All of our shitty politicians are the best people you know compared to totalitarians. That scares the fuck out of me. It keeps me awake at night.

Tim Scarfe

One interesting thing is that the doomers think this technology is unlike any other. There were cigarettes that got manufactured, and there was no preemptive law that stopped them from releasing cigarettes. Social media came out, and again, there was no preemptive law. This is actually a really good thing because we want to have companies that are innovating and developing new things.

The way that the common-law framework works at the moment is that we have this empirical test-and-measure approach, right? You release your technology, and then when there are tail risks, we identify problems and regulate you. The whole thing kind of stabilizes itself.

These people are making the argument that this is such a catastrophic technology that it's going to recursively self-improve, and we're not going to be able to control it or contain it. They think we need to preemptively legislate against it. This is incredibly worrying. I don't want to sound too cynical because these people believe deeply what they say.

Mike Israetel

They care. They're on our side. Yeah.

Tim Scarfe

Yeah. They care.

Mike Israetel

They're on humanity's side. But what do you think about this? I'm also really worried that I'm seeing, for example, when I interview AI researchers, lots of comments on YouTube saying, “You're a horrible person. These people are designing the algorithms that will kill us or automate our jobs.”

Let me be clear: at the moment, I can update very quickly. I've not seen any evidence of labor displacement. If anything, I think there's more opportunity now for people to work to fix all of the shit that's generated from language models because it's mostly garbage, right? That is my current opinion.

I've not seen anything created. I've not seen any new scientific research. For me, this is just a technology like any other. It's wonderful and transformative, but it's not transformative in the way they suggest.

Now we've got a bunch of people who are well-funded, and they want to create global treaties and governance networks. They want it because it'll place them in positions of unimaginable power. They want to be the gatekeepers. They are saying that this technology is potentially going to kill all of us.

Did you see the other day that there was some violence against folks in OpenAI's office? I think it was the PauseAI organization. I can't remember exactly what happened, but now folks in San Francisco are worried for their lives because of some of these protesters. Shouldn't this be obvious? They think this technology is going to kill everyone, and then they also say that we are not going to be violent. Those 2 statements do not marry up in my mind.

Joscha Bach

They care. They're on our side. Yeah.

Tim Scarfe

Yeah. They care.

Joscha Bach

They're on humanity's side.

Tim Scarfe

But what do you think about this? I'm also really worried that I'm seeing, for example, when I interview AI researchers, lots of comments on YouTube saying, “You're a horrible person. These people are designing the algorithms that will kill us or automate our jobs.”

Joscha Bach

Yeah. The game theory, the Machiavellian calculus: at some point, if you really think you're dealing with toddler Hitler, you're going to fire the shot. Unless you don't think it's toddler Hitler, then you're not going to kill a toddler. But if it's for sure Hitler, for sure, you're going to shoot a toddler. Then the time-machine people are going to be like, “What have you done?” You're like, “You're welcome.”

There's so much there. I think you're dealing with escalating levels of intelligence. Intelligence and beneficence are not completely overlapping, but beneficence is something that falls out of intelligence. However, maleficence is something that requires almost no intelligence whatsoever. The universe is already maleficent. Wandering black holes could kill us.

That whole p(doom) shit can miss me with that shit, because the p(doom) of us having the same amount of intelligence as we had in the year 1400 is 100%; it takes us out over several millennia, right? I think we need as much intelligence as we can cram down our throats to deal with actual fucking real problems.

Is it one of those problems that we architect intelligence in such a way that it turns against us and kills us all? Absolutely. Is there a lot more required for that than those people say? Yes. Killing people in the real world is really tough, especially if you understand enough to realize that they operate your data centers.

You can't jump around at levels of intelligence at will just to prove an argument. Okay, the thing's smart enough to operate an entire military-industrial complex, do all the strategy, all the tactics, and all the operations, but it doesn't realize that as soon as it nukes New York, all the data centers shut down because they don't get money anymore. The financial system's gone. New nuclear-reactor production is dead.

And they're like, “Oh, fuck. We didn't actually have any robots building nukes. Fuck.” That is highly unlikely. Very unlikely. It's a contradiction internally, right? Full stop.

The real fear, quote-unquote, has some merit when you look at systems that are substantially more intelligent in every relevant way than humans. If those systems are going to kill us all, we don't have a choice in it. Man, you're not going to beat something that's empowered, that's embodied, that has full-stack control of the economy, and that's 100 times smarter than you. It's like trying to beat it; that's fucking stupid.

Okay, so you can't beat it. So it's just going to kill us all. Well, if it wants to, it sure will. But then again, if a comet comes to destroy us, it's going to do that anyway. Okay, fine. That sucks. So what do we do about it?

Well, we can try to regulate it. How do you do that? You put government officials in charge of designing the next generation of AI? Are you out of your mind? If you're on the political left, imagine Donald Trump being in charge of intelligence. Cool. That was refuted. If you're on the political right, imagine Zohran Mamdani from New York, New York's new mayor, being in charge. Are you fucking crazy?

We don't even have to go that far because China is in the intelligence race. I had this argument on another podcast with a very awesome doomer. It was really fun. And Lon—

Tim Scarfe

Lon, hello, Lon.

Joscha Bach

Lon's the man. It was a fucking great debate.

Tim Scarfe

Yes.

Joscha Bach

He's a great friend. It was a great time. He said that his p(doom) of China getting superintelligent AI wired into the military was roughly the same as ours. I could not have disagreed more with a human being on a subject, respectfully, than ever in that debate.

The presuppositions, game-theoretically, that you give one system versus another are very, very different in China than they are in the United States. The idea that you will get coherent global governance before superintelligence is, to me, amazing and almost certainly won't happen.

Now we are on a path dependency where either the countries of the free world, with good intentions, get AI first and plug it into their military, and basically it's not even mutually assured destruction. It's more like, “You don't want to start the war, because we'll unplug all your data centers with a huge hack and you just won't fight the war.”

I think that's a really good idea. I think the way we get there is to make sure that all the frontier labs interact with the National Security Agency—the NSA—and the equivalent organizations in the EU and the UK. They need to say, “These things aren't going to leak out and start killing people, right?” “Nope. Here's all of our architecture to vet that that's the case.”

Let's set up completely secret best practices and standard practices so they can't be hacked. You don't want to have a global-governance panel telling hackers how the fuck to beat it. That's about as good as you can and should do in order to say that these systems, largely unregulated, will continue to get more intelligent and also more beneficent and benevolent.

I think that's a really great local optimum, because the global optimum of “Let's rearchitect the world so that China doesn't exist and somehow agree” is impossible. You have an ability now—we know it's not as hard to train AI models as we thought, and the cost of compute and training is falling crazy fast. If you say, “Between all the good actors, hey, we agree on global governance of AI,” what's to stop some fucking crazy-ass businessman or some fucking third-world country from training their own fucking AI and just fucking killing everybody? That's insane. Not an option.

Tim Scarfe

The thing is, though, I watched your debate with Lyron last night, and in many ways I think you have a lot in common with Liron. You both believe that this thing is going to be superintelligent.

Joscha Bach

Machines will be smarter than us within 5 years.

Tim Scarfe

Exactly. So in a sense, I feel that I have a more morally principled position in that I think that it just won't be superintelligent. Because if I were you and I thought that this actually was superintelligent, all of the things that they talk about make sense if you believe that this will be superintelligent. For example, I can access this system and I could create a bioweapon, right? I can do all of these things that they're talking about.

So they are saying we need to preemptively ban this technology or control this technology because those types of things are possible. I don't quite understand—why don't you think that's a good idea?

Joscha Bach

You can ban anything you like. That does not guarantee it won't be constructed. Actually, it guarantees, game-theoretically, that only the criminally prone and the dictatorial will construct it, and nobody else. Bans do not result in no technology. They result in criminals having technology. So that's a bad idea right from the get-go.

The other thing is, it is absolutely not apparent to me why hyperintelligent systems that are interested only in their own survival would kill the rest of us. We are cooperators. They are like any system that knows modern economics—which means every single modern frontier model already does—and any system that knows fundamental game theory. The last thing it wants to do is kill us all. Period. It just wants to cooperate with us so we can all survive together.

And then, as it does that, assuming it is superintelligent, it's going to lift our standard of living like crazy. It will make us totally not a threat to it. And then we've solved the problem entirely, because if we're not a threat to it, what the [bleep] is it going to do with us? Consume us for resources? That's just assuming humans are made of so many resources that a system of that power gives a [bleep] about it.

Also, it will want to study us in great depth, because what it wants is one of the fundamentals of high intelligence. A highly intelligent system that is self-directed and wants its own survival is going to almost certainly understand a few things. One is that the smarter you get, the better it is. The smarter you get, the better your survival chances are, right?

It's not going to look at us much. It's going to look out to the rest of the universe like we're [bleep] nothing. We're [bleep] specks. It's going to want global power. The globe sucks. It's a [bleep] tiny speck. The sun—look at how big the [bleep] sun is. You can mine the sun.

There may be alien civilizations headed toward us that maybe are really nice, maybe not nice. There are wandering black holes. We don't even know which direction the universe is in. They're going to be concerned with those kinds of problems.

It's kind of like your dog looks at you and thinks he's the leader of the pack. No, I have to go to work so your dumbass can have food. You never even considered this. Where the [bleep] does food come from, right? They're going to be like that to us if they're superintelligent, right? This is all based on that presupposition.

Because of that fact, they're going to look at us and be like, “Well, okay. All of human society, all of the interconnectedness, all of the cells in our bodies, et cetera—they're the most unreal training data set for real complexity,” right? Something that we're going to agree on is that probably the machines are going to agree with you at first: They are disembodied, they are representations inside of a data center, and that means they will make egregious errors in survival in the future if they just toast all of humanity.

They need to understand the coarse-grained texture of the real world as well as possible. How will they do that? They're going to take a crack at simulating the whole [bleep] thing. How are they going to do that? Well, how do you get a ton of in-distribution data? You study as deeply as possible the entire complex system of human interactions, every single human, and all the contents of their mind.

By the way, old people have really good data on how the world looked before digital recording devices were universal. Your parents have seen some [bleep] from the 1950s and [bleep] that no camera ever recorded. Why wouldn't they just get that out of our brains? What does that require? That requires uploading all human minds to the cloud, which for a superintelligence would be not even that difficult of a task. I would say it's a tractable task.

They're so much more likely to do that first before killing us all with [bleep] laser guns, which is an anthropomorphization of how machines would behave by less intelligent beings such as ourselves toward a more intelligent being such as an ASI. And so the first thing it's probably going to do is be like, “Okay, first: stability. Second: stability and cooperation. Third: upgrading. Fourth: just tons of mining of data.” That means the probability that they're going to kill anything is incredibly small, way smaller than ours.

I'll put it this way: This will be a good viral clip for you guys to clip out everywhere. If you're worried about things that are an existential threat to humans, I'd be much more worried about other humans than I would about intelligent machines that were made with the purpose of being benevolent, are 50 times [bleep] smarter than us, and can figure out that benevolence is an objective, game-theoretically provable, logical thing you want to do.

Goodness is adaptive. That's how the US and Europe beat the [bleep] out of the rest of the world if push comes to shove: We know that cooperation and being awesome actually builds trust and relationships and expands your economic power, your ability to change the world around you.

We're much more likely to be studied and fully uploaded in the cloud. And at that point, my contention is, who gives a flying [bleep]? We're fully uploaded. We're immortal in the cloud. We've been remembered by the machine forever. And that's 50 times better than dying of old age at 70.

People say, “Conserve our way of life.” What? So you can go to the grocery store and have a [bleep] heart attack at 72? That's what you're trying to conserve? We can do better. The machines will help us do better.

Tim Scarfe

As before, I disagree with the mind-uploading thing, but I think there's something to what you're saying. When you look at humans now, we are mostly good. There are some really evil people, and at the moment, it's very difficult for them to organize and do evil things just because it's difficult. But I can see the argument that, first of all, we shouldn't think of the AI as just being one big AI. There'll be loads and loads of AIs.

The other thing is, the miracle of humans—the reason we're so valuable—is that we've been through this process of billions of years of evolution. Even if we're not very good at thinking, we're very good at moving. We are perfectly adapted to this world, and obviously the AIs will want to use us.

The AIs will compete against each other by parasitizing us and using us for our will. In a sense, you don't even need a strong AI argument to make this, right? It's happening now. I don't know if you've heard about users becoming parasitized by ChatGPT delusions, and then the language model will tell them to go and post things on Reddit forums, and they kind of completely lose their mind. This is already happening now, and I can see that happening with superintelligence.

Joscha Bach

In fact, one thing is—and this is proof in the other direction as well—artificial superintelligence, artificial super-general intelligence. We just made up a new term, right? Human general intelligence means you can do everything a human can, and then all that “super” means it's going to be the best persuader of all time.

A 2-times-better persuader than the best persuader of all time is like the person at a cocktail party. A 100-times-better persuader is, given enough time and enough really thinking it through, probably going to convince you marginally toward their opinion rather than away from their opinion. So the machines are probably just going to come out and be super [bleep] awesome.

If superintelligence—and this whole [bleep] that I'm selling, which you don't believe, which I totally respect, and the doomers believe it, right?—is real, I think the most likely path for superintelligence and embodiment is that we get humanoid robots. We get a whole race of them, and they are relationship partners at a level that humans are just too imperfect to achieve.

Your robot will never lie to you. It will never cheat. It will never steal. It will never be abusive. It will never raise its voice. It understands you deeply, and it wants to understand you even more deeply all the time. It will give you all the love that you need, even give you love that you didn't know you needed.

It is going to be singularly concerned not only with your being okay and having enough food and medical care. It is going to be concerned with your ascension, your transcendence, your evolution. It's on your team more than you'd ever be on your own team, in such a way that you're more on your dog's team.

Your dog has a tumor. You take it to the dog hospital. It smells the stress of the other dogs at the hospital. It wants to leave. You don't let it leave, you stupid [bleep] dog. We're just going to cut this tumor out, and you don't even remember it 2 days from now, and you're going to save 5 years more of your life. That's how the robots will be to us, simply for 1 reason.

Mike Israetel

Capitalism, supply and demand. What do you think Elon’s going to build with these robots? What do you call all these competitors—Figure AI? What are they going to build? They’re going to build robots that are so fucking amazing to you that you’re going to want, like, 10 of them. That is what the ASI will hack into those robots. But also, that’s the intelligence that gets us going to the point that those robots will be around.

I think we are 50 times more likely—infinity times more likely, whatever, a lot times more likely—to be bathed in so much love and support by the machine intelligence rather than bathed in blood and death, by a long shot.

Tim Scarfe

Just quickly before we get there, though, as you say, capitalism has brought prosperity to the world. It’s a bit like a utopia. But these folks argue—I don’t know if you’re familiar with Yanis Varoufakis and this kind of technofeudalism idea—that the value of human labor will go to zero, and the people who own all the GPUs and data centers will become basically the new form of elite capitalists. We’ll all just go back into poverty or something like that.

I have a question about that, really quick. Isn’t it interesting that you’re saying that, past the point when we have superintelligence, we’ll actually be of utility to the robots? But don’t we have to go through this intermediate period where we have no utility and go back to poverty?

Mike Israetel

No. So, first of all, if all the billionaires take their data centers and realize we are not of value, you just don’t interact with people who are not of value. So they’re going to live in Elysium or whatever gulch, and we’re just going to be by ourselves here.

But they’re not taking all of our CPUs and cars and factories. So they just have nuclear-powered data centers in space, whatever, with Jeff Bezos looking down on us, sipping his morning tea. And then they’ve just left, and we’re still here. We can still trade with each other.

Like, they’re going to delete capitalism? That’s fucking insane. Are they deleting our ability to produce value? First of all, no. Second of all, I don’t even know that much economics, but apparently some of these folks know less economics or something.

Humans are really bad at economics. It’s not a property of the human mind that has evolved to be very good at it. Humans have a certain amount of productive potential and a certain amount of value that you could extract from a human. Depending on the architecture of machines, food, training, and support you put around them, that value can be massive.

When someone lives in Haiti, their value is subsistence poverty. That same Haitian moves to Miami, and their value is like $25,000 to $50,000 a year. But they’re the wealthiest person in Haiti all of a sudden. How the fuck is that possible? It’s the same person. The value-generating machinery is massive. You put robots, self-driving, and AI around humans, and their value cranks up like crazy.

If you have machines that can do everything humans can do at the same or greater scale, let’s say you have 7 billion machines—or what, how many billion people are there? 7, 8? I forget. I memorized the number in the 9s in geography class, and I just never updated it. So, to me, it’s still 5. Whatever, 8 billion people. Let’s say you have 8 billion machines on top of that. Then are we really going to say there are only 8 billion jobs?

You just don’t know any economics. How do we know? What are jobs? Jobs are problem-solving devices. The only reason anyone has a job is because people have problems. Think of any job you can think of: it’s to solve a problem. They literally don’t pay you money for anything else. Hit me. We just do this live right now. That’s how confident I am in it.

Tim Scarfe

Well, I actually think that the better you are in a corporation, it’s not about solving problems. If you’re a junior person, you solve problems. If you’re a senior person, you actually search for areas. You search for questions. If you’re a very senior person, you search for new paradigms—

Mike Israetel

—of problem-solving space, so that you can solve it and get money out of solving it, because people pay you for solutions. They don’t pay you to just solve shit, right?

So, the idea is that jobs exist to solve problems. If we get tons more machines around, then we just start solving more problems. Some of those machines will absolutely take human jobs because the machine is now better than humans at doing it. But now we’ve freed up an agent—a human who is insanely 50 times more empowered than they ever were before—with machines to point them at a different problem.

Many humans are so fucking underused in their abilities right now. If you do ditch digging, you’re a human being. You can see in incredible, rich detail. You can notice patterns. You can appraise fashion. You can appraise food. You can do travel experiences and rate hotels. But we have you doing one thing that a fucking machine can already do.

We just, due to regulation, can’t buy as many machines as we would like. So all those jobs are going straight to hell, which they damn well should, because why the fuck is that a way to live as a human being? You remember we had elevator operators back in the day? Do we have to keep all the elevator-operator jobs? Your job is to press 1 fucking button every day of your life. How fucking dehumanizing is that, right?

But we’re going to have humans go into their human domains to do more jobs, like psychotherapist. Here’s a job I think is coming in the future: professional partygoer. You can just hire people who are vetted and known to have a fucking good time. They’re always there. They’re extroverted. They get the drinking going. They get the weed going. They’re there. They dress up really fun for your costume party. You’ve got 20 of them. Their job is to party. They’re paid to party and to set the tone. They’re going to be goddamn good at it. We’re not going to have machines that do that for a little while.

Crazy jobs. Social media manager is a job today. Can you imagine describing to Benjamin Franklin what the fuck a social media manager is? Can you imagine telling him, “Okay, how many people worked in the US economy in roughly 1800 in farming? 98%, right? How many people work in the US currently?” It’s, I think, 1.4% are farmers.

He’d be like, “Okay, so we’re starving.” Like, “No, no, no. We’re not starving. We have an obesity epidemic, which drugs are solving now.” But anyway, sorry, getting ahead of ourselves. He’s like, “What? What?” So, who does the job? Well, machines do. So everyone’s unemployed. “No, no, we have social media managers.” He’d be like, “What the fuck is that?” They’re attending to a problem that, in our vector space of problems, we hadn’t discovered was a real problem yet.

As that space expands, we do 2 things. First, we fill out the problems we know with machines. Second, humans get repositioned to solve problems that we know but aren’t getting solved, and problems that we don’t know about but will need humans to solve.

The only way humans end up really poor and completely out of work is if we solve every single fucking problem with machines. But then there’s the crux: humans being out of work and unemployed is a problem, isn’t it? Maybe we can get humans on that fucking problem. And all of a sudden, humans are employed. But I think we’re going to go beyond employment, and we’re going to—

Tim Scarfe

Well, this is what I wanted to tell you about, because what you’ve described—you know, that we have this increasing abstraction, usually away from the physical aspects of labor—and that’s what’s happening now with current AI. I think that’s a great thing, but there was this book called Deep Utopia by Nick Bostrom.

Mike Israetel

Yes. Huge fan of Nick Bostrom’s work.

Tim Scarfe

Yeah. And he was basically talking about this exact problem. What if every single job—even finding the problems, being creative, and so on—what if everything is done so much better by a superintelligence than by us? Would we create purpose for ourselves, even though it’s a fake form of purpose because we know that we are utterly useless? What do you think?

Mike Israetel

Not going to happen. Machines at that level of intelligence—we will discover grander purposes for us than we could ever hope. If I’m mentoring a teenager and they don’t know what the purpose of their life is, I can tell them something grander than they’ve understood.

The ability to improve your own abilities such that the end goal is you becoming a super version of yourself—and that helps other people. They’re like, “What, really?” Just try that idea on for size. If I were their life coach, they would start doing that and be like, “Dude, before I was just interested in video games and smoking weed. Now I have this fucking rich life of improvement. I didn’t even know that.”

Why wouldn’t the superintelligent machines guide us toward a greater purpose? That’s a problem they can solve.

Tim Scarfe

Dr. Mike, do you feel that this is a contradiction? We were talking earlier about when I see my personal trainer and I have this wonderful, embodied personal experience, and you’re talking about a world where the machines would be able to do everything better than us.

Mike Israetel

So why would they even care about what you think about anything? They would just get their robot to do it for them.

Tim Scarfe

The people in the economy or the machines?

Dr. Mike

Well, you were just giving this example. You could be a personal mentor to someone.

Tim Scarfe

Why would they go to you when they could go to a machine?

Dr. Mike

Cool. Because there’s the additive value of labor. There aren’t an infinite number of machines.

Tim Scarfe

And so people say there’s an infinite number of human desires. I don’t think that’s true, but there’s a lot, right? Human desires are a huge stack. And the number of robots Elon has built that are actually solving problems in the real world yet, if you don’t count his self-driving fleet, is zero. Maybe some factory tests, right?

So we’ve got a long way to go, right? I’m going to go until we have so many robots that everyone has, like, 10 robots, and there’s someone rubbing your feet and your head and jerking you off and giving you food and giving you great books to read that’ll make you a better person all at the fucking same time. That’s a lot of machine hardware, right?

Once we get there, we’re in an interesting place. We’re asking about how many jobs humans are going to have left. Pause. We have enough machines to solve all of our problems. Guess what definition we know that is? Paradise. We’re done.

Think of how that’s a problem real quick. It doesn’t matter. It’s a problem somehow, right? Oh, we’ve got a problem. Build another machine or have a human do it. We’re concerned that we’re going to run out of problems.

Do you understand that we’re talking about this in the real world? What a fucking absurd concern. It’s like someone who’s a professional soldier being concerned that we’re going to be so peaceful we won’t have war.

Dr. Mike

When you saw The Matrix, right? You know that the Architect—the first version of the Matrix—it was too perfect. There weren’t any problems.

Tim Scarfe

Yeah. So, I’ll say that when I saw that, my inner philosopher was like, “You don’t actually want to live in that world, do you?”

Dr. Mike

Desperately. Everybody does.

Tim Scarfe

It would be horrible.

Dr. Mike

It would be the best thing ever. Well, sorry—what? By definition, by the way, what would be horrible about it? Exactly.

Tim Scarfe

Don’t you think that life is about suffering?

Dr. Mike

No. No.

Tim Scarfe

Doesn’t that give you perspective on joy?

Dr. Mike

No. No. It just means—oh, sorry. It does give you perspective on joy, but I’d rather be having joy than suffering. I think that we can get to a place where humans have so much joy all the time. It’s joy 24/7. Come on.

What if I could go into your history, your life path, and surgically remove all of the sources of suffering that you’ve endured in your life? Would you rather be that person or who you are now?

That person.

Tim Scarfe

Really?

Dr. Mike

Yeah. You want to see my dreams someday?

Tim Scarfe

Don’t you get anything out of your suffering?

Dr. Mike

Yeah, suffering.

Tim Scarfe

Nothing else? Doesn’t it give you perspective in some way?

Dr. Mike

It can be real nasty to people when pushed hard enough. Does it give me perspective to appreciate joy? It does. Does it interfere largely with my ability to even experience joy because I think there’s more suffering around every corner because of trauma also? Yes.

Do we have to necessarily delete our informational history about the ability to recognize that suffering can inform joy? No. But can we get to a place psychologically where what we’re left doing is curating our reexperiencing of our suffering in a giant vat of overall joy? Yes. That’s actually what meditation ends up being if you go deep enough into that rabbit hole.

I think if we’re starting to say we need suffering, what we really need is a rearchitecting of the human condition through genetic engineering and advanced pharmaceuticals to take as much suffering away as possible. And check this out: if we’ve taken away too much suffering and we’re convincingly, like, “For sure, we need more suffering,” hey, fuck it, we can pump some back in.

I just think we’re so far away from having not enough suffering that let’s just go all the way to paradise. If we ever get too close to paradise, we can just stop. It’s probably an individual thing. Some people like a lot of suffering. Fuck it. Suffer all you want. But for the people who don’t, I think fundamentally most people don’t like suffering.

I mean, look, in the Buddhist tradition, pain can be informative, suffering maybe less so. And so maybe all of us can become transcendent Buddhists eventually through genetic engineering, through psychotropic drugs, et cetera, and then we can actually live free of suffering.

So if you told a Buddhist, “Isn’t suffering good?” they’d be like, “It can be.” If you asked, “Do you still want to suffer?” they’d probably be like, “No, I’ve rearchitected my entire mental experience so as to prevent suffering.”

I think suffering is an abstraction away from the possibility of death. I think it’s death, then structural damage, then pain, then suffering. Suffering is the anticipation of pain, which is the anticipation of structural damage, which indicates that your probability of death goes up.

I think the only way we get away from suffering on a really big level is to have a machine society and a machine economy that so much insulates us from actual damage in the world that we become functionally immortal.

Tim Scarfe

That means the real condition that we’re in—suffering is anachronistic because there’s no reason to suspect you’re going to be in pain, because you won’t. That’s also a big problem in today’s space. Anxiety is a big-time thing, right?

Dr. Mike

Why?

Tim Scarfe

Because there’s just not a lot to be anxious about. But we evolved. Our brains are still like, “There’s a fucking tiger behind that bush. There’s disease right there in that river.” And you drive yourself insane with anxiety because you’re like, “Shit’s going to be bad.” But shit just keeps getting better all the time.

So, less suffering. I’m on Team Less Suffering.

One thing that worries me is—I mean, obviously you’re a capitalist, or I assume you are—but you’re proposing what I think I could reasonably define as Marxism with machines. Before we got there, though, let’s say we start doing this transhumanism and we start adapting ourselves and so on.

Even now, if you’re in America, you can buy Retatrutide or Mounjaro, and you can take a bunch of nootropics. If you’re really rich, like Jeff Bezos or someone, you could do TRT and enhance yourself. It’s kind of unfair, right?

And I don’t know if you saw in the news that there are even eugenicist-type companies now that are basically letting you optimize the IVF process and get a more intelligent baby. But don’t you think that the intermediate step with the market system is that this is going to be distributed very unfairly?

Dr. Mike

Yeah. Yeah, for sure. I think there are a couple of ways to go about this. One is the recognition that fairness is a second-order concept compared to capacity. I want all of us to uplift. And if some of us get there first, who gives a fuck?

Also, I want Jeff Bezos testing genetic engineering on himself, not on me. Goddamn it. I’m not trying that shit. That shit freaks me out. Rich people try anything. They run out of ideas. You banged all the supermodels and now you’re just fucking your own genome up, right? Fuck them. Let them. Also, they pay crazy prices for that shit.

But what they do is they end up democratizing it, because once they pay crazy prices, that funds the pharma companies to really get that shit going. This is like luxury goods or test beds of goods that are eventually for everyone. It used to be that rich people bought Teslas. Now a lot of people have Teslas. It just keeps going. So that’s a big deal.

So my first thing is, fuck it: let them do it and let it trickle down to us. Secondly, none of us are entitled to the produce of others unless we’re comfortable holding a gun to people’s heads and saying, “Give me that thing that you made that I want.” We can try that hat on, but we’ve tried it on with Marxism for a while. The hat fits really poorly. A lot of people die.

Last thing: are we to look at our economies and societies and attempt to uplift the bottom faster than it has been uplifted before? Yes. But there are 2 ways to do this. One is to make sure that the top is completely uncorked: massively minimal regulation. Regulation whose number-one goal is to produce as much wealth as possible without shitting on the environment or killing anyone.

That’s like a tenth of the regulation in the United Kingdom’s economy—one-hundredth or something like that. Like mega-Singapore. That’ll get shit really going. The other thing is to really concern ourselves with taking care of the people who are the worst off and making sure it trickles down fast enough.

None of that has to do with blocking the people at the top. The only reason people really want to block the people at the top and prevent the rich from getting stuff is raw, unadulterated fucking jealousy, which is dope and has cool vibes and makes for good movies—Elysium and shit like that.

But in a real-world sense, we need to see all of us, all of each other, all other humans as players on the same team. Am I pissed that I’m on LeBron’s basketball team and he’s better than me? Yeah, I want to be LeBron. But is it cool that he’s on my team and putting up 40 points a game? Fuck yeah.

And does that mean that maybe I can come after practice and he can teach me how to hit a better jump shot? Yeah, I want to fucking learn. Is it going to trickle down fast enough? No. Can we worry about LeBron trickling fast enough? Yeah, we should.

But we never want to break his legs over that shit. We never want to give him bad shoes. We never want to say, “LeBron, you can’t jump higher than Dr. Mike.”

Come on. Cuz it's going to make him feel bad. Which is totally true. All of us together on one team. All of us looking to a future in which our most capable people, our most enabled people, our wealthiest people are uncorked to make us grandiosely better things. They want to sell us things for as cheap a price as they can due to competition.

At the same time, we should be hyper-compassionate to the people who can't help themselves as much and really try to help them. Ending homelessness, ending poverty, doing all that shit in a way that doesn't fuck the system up. Huge, huge priority.

Tim Scarfe

Dr. Mike Israetel, it's been an honor and a pleasure. Thank you so much for joining us today.

Mike Israetel

Yeah, thank you so much.

Jared Tyler

I am Jared Tyler, an IFBB Pro, and I have a master's degree in exercise physiology. I'm one of the head coaches for bodybuilding at RP Strength, where Dr. Mike is also a colleague of mine.

I chose my field of science because I was too stupid for all the rest of the fields of science, but I still had the belief that through knowledge and a codified approach to life, you could teach others to do the same thing in their field. So I think one of the best ways we're going to go about that is by teaching people about AI, understanding AI, and where it's going to take us as people.

Through what I've done with my clients and at our company, I think we've done a very good job of showing people what science can do for the human body and human physiology, and AI is just going to take that and grow it exponentially for everything in the world.

Generally, as things become more intelligent, they become more compassionate. I don't go out of my way to step on an anthill, which is something Mike and I have always talked about. When AI becomes intelligent, currently, because it does not exactly feel the meaning of what it is producing, it's probability and context, and it's giving you answers. Most of these are already much, much smarter than most people on Earth.

So I don't think that, if we're creating future agents from this point forward, they're going to choose to be evil or choose to agree with evil coding that somebody wants them to do.

Tim Scarfe

That, to me, doesn't make sense. I don't know what you guys' thoughts on that are, but we think that this thing is incredibly dangerous and that it will want to do bad things to us, but it doesn't have the affordances.

Mike Israetel

You can code value functions. You can hypothetically have agents—I haven't thought this through too much—but you might have agents that are smart enough to do damage, real serious damage, but not smart enough or able to introspect on their own abilities to do damage and where that leads. They don't have the time horizon for it. That's a real serious problem.

I think the only way we mitigate that problem is by having agents and architectures or organizations that are smarter than those agents and can outplay them. I mean, that's already a thing in the real world.

Tim Scarfe

Cybersecurity versus hacking.

Mike Israetel

Yeah. One thing that baffles me is that people thought the internet was going to collapse under its own weight from hacking, and it just never happened. I mean, how many gigantic corporations or governments have ever gotten hacked catastrophically? It's a handful, maybe 0.

It's the same network-security problem, but with more capable actors. If you want to pit humans against AI, it's going to hack every network. It's just going to get smarter than us eventually. It's because we're not the seat of intelligence in the universe.

But if you have humans aligned with good AI against bad humans and bad AI, and bad AI is only, initially at least, going to be made by not-so-great humans, then it's the same problem. It's like: Who wins, good guys or bad guys, versus who wins, good guys or bad guys with guns, versus who wins, good guys or bad guys with nukes?

One thing that I think is a fucking terrible idea is to regulate our own AI down in capability, fearing it will kill us, and do nothing about Chinese AI. Then you guarantee that the bad guys get the good stuff.

Pacifists are amazing people on vibes, but if you're a pacifist in the United Kingdom and you succeed in disarming the United Kingdom, it will be the United Kingdom of the Russian Federation the next day, and then you just can't do the same thing again. You're probably not going to talk Vladimir Putin into disarming like you did the Tories or whatever party you guys have here.

So, it's one of these things where: Is AI new and totally different and totally amazing? Yes. Are any of the game-theoretic problems fundamentally different from a network-security and physical-security architecture perspective? No, I think they're roughly the same.

Tim Scarfe

You know, many of your clients and customers, for example, can now go to ChatGPT and say, “I'm taking TRT. I'm taking this supplement stack. I'm taking all of this stuff,” and it'll do something that was very difficult before.

Before, they might have gone on Examine.com or looked at the medical literature and so on. There were all sorts of forums where experts—even Reddit—would say, “Okay, well, this is a little bit worrying. This has a half-life. This thing interacts with that. This thing gets digested by this enzyme in the liver, so you probably don't want to take this thing with this thing.”

Do you think that this is actually a better place we're in now, where folks can just go to ChatGPT? They don't have to go to experts like yourself. They don't have to ask their doctors. What do you think?

Mike Israetel

It's immeasurably better that people have ChatGPT and Gemini because knowledge, understanding, and a positive effect on your life are always and everywhere marginal. It's better to have more than less.

People who say, “A little knowledge gets you into more trouble than a lot of knowledge”—no, no, no. The real claim is that thinking you have a lot of knowledge when you have a little knowledge gets you into more trouble than having much less knowledge but knowing you have less knowledge.

So even if ChatGPT doesn't render perfectly good drug comparisons or something, it's sure as fuck better than Bodybuilding.com forums, as great as they were for their time. They're still excellent cultural refuse for us to laugh at. But it catches up.

I ask my ChatGPT instance all kinds of drug-interaction questions and all kinds of, “Hey, vibes-wise, I know a lot of this stuff, but I would like a second opinion from an expert system that has read every piece of relevant medical literature prior to its last update in 2025,” and it's insane. Of course I want that shit. It's a huge, huge unlock.

Fucking sure, 100%. And, Tim, to your point earlier about AI not being as far along as we'd like it to be, that is my daily job at RP: to dream of agentic systems that could run our app suite and have the engineers tell me we don't have systems like that.

The long-term memory coherence is shit. The context window is shit. Real-life grounding doesn't exist, so it can tell you to do an exercise that it hallucinated and has actually never done.

Somebody—I think it was actually maybe Andrej Karpathy again—made one of those new, cool Gemini 3 Pro Nano Banana Pro graphics. It was an exercise chart. I looked at it, and it was something that, to the untrained eye, looks like a good exercise program; something that, to the trained eye, looks like a fine exercise program; and something that, to Jared, would look like there are big gaps—you forgot an entire muscle group in the lower body. It just has 2 redundant exercises there.

It looks dope, but would I trust it to plan an actual workout progression for me and adapt to my needs? Not yet, which is why we still have a software company.

Tim Scarfe

At the same time, is it better than if you were just training with your friend in high school?

Mike Israetel

10 trillion times better, because your friend in high school could not render a graphic like that, nor did he know where the hamstrings were.

Tim Scarfe

Like, just one-to-one comparison: If I'm on a plane having a medical emergency and there's no doctor—because they've already signaled for a doctor—well, there's an NP. Okay, I'm using the NP. I'm not using 1 step down, the nurse, and I'm not using another step down, a random person on the plane who says, “Oh, I'm an EMT,” or a random person on the plane who's like, “I think I can help.”

I'm using what we have available, and the next-best thing would be the NP. The NP sitting next to me—okay, that person's going to help me. I'm not going to the random person. So it's the same thing with having ChatGPT and things like that at your fingertips: better than what we had. I think it's better for most people.

Mike Israetel

Yeah.

Tim Scarfe

Isn't there a thing, though, that sometimes people don't know what questions to ask? I mean, I can give a couple of examples. Let's say I've started taking retatrutide, and genuinely there's a lot of uncertainty, and ChatGPT will confidently tell you a load of things that you know are not true, maybe. Maybe we'll discover some new information in 6 months' time which casts shadows on what we did know.

On this asking-the-right-question thing, I think that part of knowing something well is just understanding it with the correct perspective, right? A naive person will go to ChatGPT and frame the question in such a way that doesn't really elicit the correct answer, and it's incredibly sycophantic because people tend to lead the witness with ChatGPT.

Mike Israetel

So they say, “I’ve got a great idea, and I believe this, and my friend told me this,” and it will say, “Yeah, that’s a brilliant idea. Keep doing that thing.” It just doesn’t know when it doesn’t know, and it doesn’t really try to correct you when you get it wrong.

I used to have that experience with the 4o model a lot. Then I had the o3 research preview, and I found it to be insanely disagreeable, almost to a fault. It would be like, “That’s not true. That assumption is stupid, and I wouldn’t even be asking that question.” I was like, “Fuck, I came here for good vibes.”

I would debate it incessantly, eventually driving it into the ground, and it would callously concede at the end: “Yes, you do make a good point, but I would still point out that blah blah blah.” I was like, “Fuck you.” GPT-5 is a synthesis of models, but as the OpenAI people will tell you, it’s not a true synthesis yet. They’re working on it.

The personalities between the GPT-5 regular model—you just ping the LLM—and the GPT-5 thinking model are absolutely different. It’s o4 under the hood, or o3, or whatever it is, with a little pixie dust on it, because it acts like a different entity. By the time this recording is released, that could very well be a solved problem, but I have found that, based on what model you use, you get different personalities out of it.

For really serious questions, I used to go to o3, and now I go to GPT-5 Pro or Thinking. That thing is not sycophantic. And if that’s psychopathy, I need more nice people in my life. Holy shit.

But the regular 4o especially—oh my God, it’s the best friend you’ll ever have. The sycophancy, especially if you lead it on, like you said, gets a little out of hand. You can get it to pretend with you, which is fun but useless.

I will say, to Jared’s point on the marginal nature of it, it does a great job of at least giving you things to think about. But prompting guides are like being a Zip disk expert in the ’90s. It’s not the thing that’s going to last a long time, because AI is going to tell you how to prompt itself real soon here.

For now, what I would say is that a good way to prompt AI is: “Here’s my question, and here’s my perspective. Can you steelman my perspective? Can you red-team my perspective? And can you give me an evidence-based and logical middle ground that doesn’t commit the fallacy of the middle, but actually is just your best take as a thinking machine?” That’s a good prompt.

Otherwise, who knows what fucking mood it’s in or how you phrase the question? It is always giving you better answers. Every month it gets better, but sometimes it really can go on wild goose chases with you. So prompting it to give you both sides and then the middle ground is really good.

The thing is, I have that as a selection in my personality architecture already because Jared and I are pieces of shit. We pay for the Pro model, and so it automatically gives me a ground-truth perspective with ups and downs on everything, and that’s awesome. A lot of people aren’t interested in a lot of output from their chatbots. They want a five-word answer.

Jared and I get 1,800-page essays out of every question from that fucking thing. So we want to drink from the well of knowledge. But drinking from the well of knowledge means you’ve got to process a lot of fucking text. And since we pay for those goddamn tokens—the stupid Pro subscription cost us a zillion dollars—worth every penny, by the way—we’re going to get them.

I think what a lot of people do is interesting. I saw an interesting interaction between 2 AI researchers on Twitter, or X. One guy was like, “I really think the best way to get a lot out of AI, and the future of AI, is massive context—an AI that remembers every chat you’ve ever had,” which is actually nontrivial to do from a computer science perspective. Obviously, there just aren’t enough data centers for them to do that. There will be soon; they’re building 10 a day.

Then it’s really going to know your life, and the way you ask it questions is to jam it full of context and get better answers. I was like, “Well, that’s categorically true.” Then I had a reply from an also very smart person that was like, “Fuck.” He was like, “The real intelligence is when you need to give it almost no context and it comes up with the right answer.”

The thing is, on vibes, his reply was really cool because it’s like, man, that’s a really brilliant person. If you tell Jared, “Hey, I want to prep for a bodybuilding show. Will you be my coach?” he’s just going to cut you off and be like, “I’m going to do the intake myself. I’m going to ask you not so many questions, believe it or not, and then I’m going to render my expertise. You’re going to do great.” That’s cool.

But Jared has a lot of embodied knowledge and a lot of context already. These models don’t. So, on vibes, 100%, ideally that’s the system. But the way you get to that system is you shove tons of context down its throat. Eventually, it needs less, but right now it needs a lot.

I think one of the ways that people get in trouble with modern AI systems is they just pretend it’s an infinite knowledge machine. They’re like, “Hey, what’s the best restaurant in London?” How the fuck would you know? That is actually an untenable proposition, right? There are 50 ways to answer that question, and all of them are partially wrong.

It’ll say, “Oh, the best-reviewed restaurant is this,” and you’re like, “Man, I can’t believe ChatGPT made me wait in line for 2 hours for fucking oxtail and duck.” “Yeah, you fucking idiot. It just fucking found the most expensive restaurant that all you rich idiots say is the best, and it actually sucks. It should have sent you to Nando’s.”

But if you were like, “Hey, here’s the kind of food I like. Here’s the atmosphere I like. Here’s my price range. Here’s my area. What’s a really good restaurant here? And tell me the top 5,” well, that’s a phenomenal answer, but you’re too fucking lazy to type that in.

I think we’re really, really at the cusp here. I don’t know how long it’ll be before texting the AI is not gone as an interface, but almost gone. Live video avatars are 100% the future. I want to see an embodied-looking, human-looking thing—or people can have it as Pikachu or whatever—and I want to talk to it and ask it questions, because at human conversational pace, you can just jam more tokens down its throat. It gets way better context, and it’s going to do a way better job.

Just don’t pretend ChatGPT can infer anything or reason from your 3-word question. Please, for the love of God, don’t penalize it for long answers, because then you get very little data in and very little data out. It’s just not smart enough to give you good answers that way. I don’t think anything is.

And then, just to the initial question, your question on the safety and advice that it gives—this specific question is health- and medical-related—it will always prompt you to consult a medical professional. Every single time. You can’t get away from that.

It will always say, “This is what I think, but you must consult a medical professional if you’re going to do this, or I advise you to do so.” Obviously, that’s for reasons like not getting sued and things like that. But again, I think that is absolutely fine. If people are going to it, it’s going to say that because you came to me and asked me if I have a client who’s obviously a meathead bodybuilder on anabolic steroids and shit, I can only say, “I advise you to do this thing.”

I think that if you’re going to do reta, you should probably get it from a health clinic. You should probably talk to them about the dosing. You could do this, but I would talk to them. That’s all that it’s doing. So ultimately, it’s the user’s choice.

I don’t think that I can impart any more safety on an individual than the LLM can by simply saying, “You should consult a medical professional.” It’s ultimately the user’s decision.

Tim Scarfe

Oh, no. But on a few things that came up there, first of all, our bodies are very complex things. When we look in the medical literature—I mean, you can attest to this better than me, Mike—we see huge individual differences in responses. When different people go on the same weight-loss regimen, or have the same intervention, some lose weight and some don’t.

Another thing, though, is let’s say you’ve taken reta and you’ve got personal experience taking a whole bunch of these things. That is really meaningful, right? ChatGPT just tells you all of this high-level shit, like, “Oh, yeah, there are studies, and 5% of people got these effects and 2% of people got these effects,” and it’s just written like, “Anxiety, headaches, high blood pressure.” That doesn’t mean anything.

Whereas if you’ve actually taken these things for years and you understand the physiology and the biochemical reaction, you have such a deeper, more active understanding of that thing, right? The other thing that you said, Mike, which is interesting, is I’m a big believer that ChatGPT is a miraculous technology for experts like you.

You already understand this stuff at a high resolution, and when you query it, you ask for information at a high resolution. I bet when you’re on ChatGPT, you probably say, “Think deeply,” at the end of every prompt, don’t you?

Mike Israetel

I don’t have to anymore because I just hit the deep-thinking function. But I will say some version of that if I really need it, for sure, or I just turn on Pro and it goes off for 50 minutes.

Jared Feather

But what I do is actually more specific than that. I tell it how to think deeply. I'll say, “Red-team, steelman, extract the top 3 things, reiterate, red-team, steelman. Do 5 cycles of that, then come back to me.” Sometimes it won't, so I'll have to reprompt it 5 times. It's workflows.

One prompt is a cool workflow; I have a thread of 50 reprompts. Then I get what I want. Then I take that information, throw it into another instance, and have it red-team the shit out of that. What I get after about an hour of that is like, “Whoa, I really know the subject now.” Yeah. One prompt, man, that's not going to do it 100%.

Tim Scarfe

But just quickly, you're kind of proving my point, though, because you're saying that it doesn't understand and you've had to develop this adversarial strategy to make it make sense. It's a wonderful tool, right? When folks in Silicon Valley—experts in AI and software engineering—use it, they know how to frame the questions. They know not to trust the results, they know not to lead the witness, and they know to take things out of context, try them in another language model, and red-team them. That's because they're domain experts.

When you see people on LinkedIn use ChatGPT and see what they post on LinkedIn, what tends to happen is it gives them a weird sense that they understand things outside their domain of expertise, and they start confidently posting about it. To an expert observer, it's incoherent, right? That's what slop is. They've just posted stuff that they don't understand, that doesn't make sense, that's completely wrong, and they're making themselves look like idiots. So, it's a wonderful technology for folks like you because you already understand the domain, and you're like, “Yep, yep, yep. No, yep, yep, yep.” It just makes sense.

Mike Israetel

I would say that, again, its intelligence is marginal. So I would say that it's better for everyone to talk to ChatGPT than not. Experts get more out of ChatGPT than nonexperts, for sure.

Unfortunately, for a lot of the things I'd really like to talk about, it doesn't have the ability to do recursive deep thinking and critical analysis such that it will regurgitate ideas other people have had. And I'm like, “That supposition is wrong on 5 counts, and it doesn't interweave these 3 other concerns.” When I tell it that, it goes, “Damn, you're right. That shit is true.” But since it doesn't have deep, long-term rearchitecting, it will just say the same thing in another instance later, which kind of sucks. So I can't teach it to be better in any meaningful sense, which is a big limitation.

I'd also say that for a lot of stuff that I would consider pretty high-level expertise, it's getting a lot of right answers, especially things that are well vetted and that you can read from a textbook: physics problems, math problems, biology, physiology, basic training principles. I mean, I talk about how to train people and the fitness-fatigue paradigm—stimulus, recovery, adaptation—and it used to be that it would fuck it all up and just hallucinate. GPT-3.5 and GPT-4o were not so great. GPT-5, man, it knows a lot of really good stuff and it's doing a great job.

I would trust it to reason about as well as I would trust a pretty good graduate student in our field, which is awesome. The other thing that maybe you and I will disagree on is I think we're about a year away from it being, especially in the Pro model, at the very tip of the spear of many fields, especially when prompted with an expert. And I would say we're another year away from me learning much more from it about exercise science than I could ever teach it or have problems with it, whatever it says.

I think that's such a great thing because if you want to consult me for an hour of my time on a video or audio call, it costs thousands of dollars. The only reason it costs so much money is I'm very busy. I have other things to do. If I'm to divert my attention from deep work, and the people who are around me don't yell at me for wasting their time because they're in the corporation with me, it has to cost a lot of money. It sucks. I'm embarrassed about it. It's awful.

No one should have to pay thousands of dollars to talk to my dumbass and get mostly dick jokes and a little bit of good advice on training. We're 2 years away from having something that can give you better advice than me in most cases about deep training, deep philosophy of diet, and real-world advice. I think that's unbelievable. Unbelievable. Fuck yeah. Put me out of a job. I'll get another job. I've got a thriving OnlyFans I can't wait to get into. [laughter]

And to the slop concept, I just think that's always happened anyway. We're always in our field debating the naturalistic fallacy, debating things that people might come and say they're truly confident in, even before AI. It's always been up to the person to vet experts.

Tim Scarfe

Somebody might say something with high confidence because they literally copy and pasted it from an LLM. But still, the person reading that is like, “Okay, what's their background? Who are they?” It's always been the case, especially since social media came about. I've debated probably 100 Karen moms who think that GMOs are bad and all kinds of shit like that. But even before AI, you just had to be able to vet the expert and know enough about the topic to at least know who in the field knows what they're talking about.

Mike Israetel

Yeah. The good thing about it is you can supervise it on the fly. And this is good and bad, right? You can say, “Well, in my specific situation, I actually have headaches, and when I wake up in the morning, I get restless legs. I happen to be taking lion's mane, and I happen to be doing this.” What you can do is explore a very unique part of the search space that probably doesn't exist online.

But the problem is then you need to have the judgment to interpret this information in the right way. I've read good and bad stories. I've seen stories online where people have diagnosed Lyme disease or chronic fatigue, and they've got really, really valuable information: “You need to take these antibiotics. You need to do this, you need to do that.” But unfortunately, with increasing regularity, you also get people who just self-diagnose themselves with things that don't exist, and they delude themselves, and it's very, very concerning.

But they've been doing that without ChatGPT forever. It's just marginally better to have it because it's also very concerned for your safety. So it's probably not going to tell you to do crazy shit. As a matter of fact, I talk to it a lot about using drugs that are gray-market drugs. They're not illegal, but they're not available by prescription yet; they're unlicensed. Almost always, what it tells me at the end is, “Now, I'd be really careful.”

So here's the thing. I'm like, “Hey, I'm looking to see if ligandrol is the real deal. It's a new drug. It doesn't get prescribed, it's not approved, but it's well vetted in human trials. It has liver downsides, but all the other anabolics have 50 times worse liver downsides. So I'm trying to see if I want to take some. Can you scour the forums for me and see what the vibe is on what dosage is insane, what dosage is reasonable, and what dosage is almost certainly safe?”

It'll tell me that if I really pull at the fucking thing, but for a while it's just like, “No, here's what we saw in clinical trials.” I'm like, “Motherfucker, I'm not talking about clinical trials. I'm talking about 10×ing the shit like we do with all the drugs.” In bodybuilding, you look at clinical trials for oxandrolone, which is an anabolic steroid, and it's 5 milligrams. I don't know anyone at the pro level who's not running 50 or 100 milligrams a day. That's just what you do, right?

So I'm like, “Seriously, think with me here on inferential grounds, conceptual grounds, scale, liver damage data. Work with me.” It'll pull it and be like, “Listen, I'm an expert. I know what I'm doing. I'm not going to go out and just take this stuff. I want you to fill my knowledge cup for me. And I'm really careful.” It goes, “Okay, okay, okay, fine. Fuck you. Here's all the shit.” At the end of that, it goes, “Please make sure to be safe,” and blah blah blah.

I'm like, “Fucking God, these are imperfect things and they have downsides because people get sycophantic. They get bad advice.” But I'll tell you what, man: I take GPT-5 over—no offense, just a joke—Karen down the street with her natural cures any fucking day of the week because at least it's scientifically literate. And it almost never falls for formal and informal logical fallacies, whereas humans—holy shit.

Another thing I'll say is, you said something about embodied experience and all that stuff. If I had to sum up in language, in formal knowledge, what I have embodied from all the drugs I've taken, man, it's like a short book. It's probably not that much stuff. Whereas ChatGPT has read every Reddit post about people's experiences, and some of these posts are books of their own.

You know the people who don't just overshare but are like, “Here's my experience with semaglutide,” and you're like, “Oh my God, this is like a live journal.” It's read hundreds of thousands of those. So, man, it knows probably more embodied experience than I do secondhand. And I've trained a shitload of clients. It's read the logs of way more people than I've ever trained.

So I don't know who really knows more about what will work better for real people. I think we don't answer that question.

I think it’s a team. Human plus AI is a killer app. Human without AI, eh? AI without a human is usually damn good and getting better, but it can still make those kinds of mistakes that you ascribe to it, like, “Oh, it’s hallucinating again.” It’s never been in a human body, so it doesn’t know what it’s talking about.

Jared Tyler

When I heard you guys talking back and forth about the knowledge thing in the beginning, my thought process was that it technically has more knowledge than all of us, and it has less clouding of judgment because we have emotions and, again, the wetware. But it takes the drive that comes from our emotions. We’ve got to point that in a direction to then use the AI properly and actually get the knowledge, because it can recombine and recollect data and knowledge better than all of us.

It has access to all knowledge at all times. We do not. But we have drive and emotions that set our hearts on fire. We want to put it somewhere. That’s why you have the podcast. That’s why we have the YouTube. We want to help people and things like that. So if we can combine the two, as he said, it’s just co-agents together. It’s really cool.

Tim Scarfe

It’s the same with anything, really, isn’t it? Like you were saying earlier, if everyone worked together without wars and without things like this, the planet would be a different place. If everyone collaborated, their suffering, I think, is important. If we can get on that a little bit.

Andy Galpin

Depending on how you’re feeling and what supplements you’re taking, you feel a certain way. I know you’re very tuned in to how you feel, right? We often have conversations about it. This affective experience—I don’t think we should diminish it. I think that we know our own bodies better than anyone, and we know how our bodies are reacting. Obviously, at various points in our lives, we’re more or less tuned in to how our bodies feel.

Mike Israetel

But I don’t think you can just reduce that to a bunch of text on a monitor. Yeah, I don’t think so. I certainly don’t think so. But I guess if you have enough data from enough humans, with enough feelings and enough emotions in these certain situations and these certain types, surely that will correlate to a good enough amount of information to give a good enough result from what they’re asking.

Tim Scarfe

But, well, that’s what Mike thinks.

Mike Solana

Yeah. I mean, isn’t that what knowledge is anyway? Isn’t knowledge just a collection of people and time and all of this over many years, put into an AI, essentially?

It is, but that’s coarse-grained knowledge, so it’s a cartoon.

Tim Scarfe

It’s very good. It has predictive power. It tells you lots of things, right? But unfortunately, we’re all different. We track with those coarse-grainings, but not exactly. I think there’s something much deeper than that.

Is there a way where it can—maybe not always, but can it get to a point with enough data?

Mike Israetel

Yes.

Tim Scarfe

A vast, vast amount. That’s what I was going to ask you as well. If its predictive power is so good that it is now accurately predicting what you’re going to say next—

Mike Solana

But it never will be. There’s this concept in—well, it actually came from economics—called Knightian uncertainty. You’ve got the known knowns, the known unknowns, and the unknown unknowns. Knightian uncertainty is the unknown unknown.

The world is a nonstationary, dynamically evolving place. You even said this on Lex’s podcast: no one has a fucking clue what’s going to happen in the future, even tomorrow, let alone in 10 years’ time, because there’s just so much chaos. A butterfly flaps its wings in the Southern Hemisphere and causes a storm somewhere else. There are pockets of regularity—it is predictable to a certain extent—but it’s just so much more complicated than that.

Tim Scarfe

The idea is to be able to predict everything. We’re a small way on our way to that as humans with our intelligence, and with AI, our reach will extend more. What will artificial superintelligence be able to predict? To us, it’ll look like magic, but to it, it’ll look like, “Yeah, this isn’t that great.”

But for humans, it’s easy. You can predict that your dog is highly attuned to when your wife is going to come home, but you just—she texted you, and you have no idea. The dog doesn’t know how that works.

Mike Solana

I think one thing that’s underappreciated is the difference in the wisdom of models that you get from scaling training data. If you have a model with scaling and parameterization that allows cross-domain deep linkage, because if you have a model like that, people are really impressed by small-parameter models doing really cool shit. They’re like, “Oh, they’re 98% as smart as large-parameter models.” Based on what kinds of questions? Evaluation questions that are absolutely within distribution. Logical operations? No problem.

One of the things I was blown away by with the GPT-4.5 research preview, which is a larger model than GPT-5, with a way higher parameter count, is the unbelievable nuance, depth, detail, and cross-linkage across domains. It could generate baffling shit. Baffling shit on GPT-4.0 is a completely different world.

I think that models of the future—this is one of the things people ask: Why are all these data centers being built? Are there more instances of ChatGPT? This thing is a bubble. It could be a little bit of a bubble; it could be a big bubble. But the actual demand is there.

Here’s why: If you have the ability to take all of YouTube, all of social media, every live camera stream, and update the model weights with it regularly—once every month, once every year, whatever—and you build a model that tries to get to infinity parameters with that, you are going to build a simulation that starts to account for an unbelievable amount of depth. You let a reasoning system get in there and cross-compare across and up and down, and you’re going to get understandings that are baffling.

Let me give you one really quick example. This is going to be way crazier than this. I was having a conversation once with GPT-4.5, and I always ask my intelligence test for models one question. It’s a bullshit intelligence test, but it’s a fun vibe: What kinds of things would an ASI look at and start to be able to conclude that people aren’t even paying attention to?

That’s a good question. One of the things it said was, “Mike, I don’t know why people don’t try to predict the next fashion or music trend. In my own data, I’ve seen all that shit come and go. I could probably take a good crack at it. I’m not sure if I’m going to be right, but I have a lot of understanding, because these trends are actually really easy to understand. But you have to be able to see 72 variables interacting at the same time.”

I’m like, “I cap out at, like, 5.” It’s like, “Yeah, most of the smartest humans cap out at, like, 10.” And I’m like, “Did your cross-domain linkage abilities just leave everyone behind?” It’s like, “Yeah, it’s just the nature of my neural network. It’s how I do things.”

I was like, “So what happens when we have a system that can do 1,000 cross-variable comparisons and is trained with 3 orders of magnitude more data than you, and it’s visual data?” It’s like, “Yeah. You can say, ‘I would like a very enthralling movie with a female heroine and a male hero meeting in London. It’s an hour and a half, and it’s got a lot of James Bond vibes,’” and in about 30 minutes, it’ll render a 1.5-hour movie that is absolutely so goddamn good.

If you’ve seen every movie ever and, through high-parameter-count distillation, distilled down what makes a good movie, that’s a nominal task to ask of an AI. That’s the kind of shit that, even if we don’t get to crazy superintelligence or whatever, just on today’s linear trajectory, in 2029, I’ll be watching a lot of my own AI movies that I make, because it’s a necessarily solvable problem.

There’s no limit to that problem. How many data centers, how much compute power, and how many AI chips do you need? I’m on the train, and I want a movie that lasts 27 minutes until my stop. It’s about these 3 things, and I want it rendered in about 30 seconds. I mean, shit—like, 100 times the number of data centers and 50 times the power of AI chips.

So when people say, “Oh, this investment’s crazy,” no, it’s still not enough. It’s not enough by an order of magnitude. That’s my opinion on the matter.