Recursive's $670M Bet on Self-Improving AI, Sonnet 5.5 Hits 70%, Elon Co-Leads Pentagon Push EP 299
Peter DiamandisSalim IsmailDave BlundinAlexander Wissner-GrossRichard Socher
Recursive’s $670 million bet rests on the claim that weak recursive self-improvement already exists because “AI is code, and AI can code.” Richard Socher says the strong version requires AI to own ideation, implementation, and validation, with an open-ended outer loop that recombines ideas; humans would set goals, environments, and rewards. No system yet controls every improvement axis, but “we’re very close,” with compute—not conceptual possibility—the immediate constraint.
Socher’s strongest definition of ASI is much further away than the programming and mathematics milestones commonly marketed as superintelligence. He expects AI to exceed humanity in specific domains within a few years, but says surpassing “all of humanity combined” across ten forms of intelligence will “probably” take several decades. Alexander Wissner-Gross and Peter Diamandis strongly reject that timeline, arguing that connecting capable models to machines is easier than Socher assumes.
AI’s most valuable scientific opportunity may be biology, where fragmented knowledge, growing datasets, virtual cells, robotic laboratories, and agent swarms can collapse the loop from hypothesis to validated result. Socher expects simpler single-gene diseases to be among the first cured, while human trials leave a half-decade-to-decade delay for complex therapies. His preferred framing is that AI is “flourishing, not cooking” science: raising abstraction while turning natural sciences into programmable engineering disciplines.
The episode’s deepest policy dispute is whether AI risk should be controlled at the model layer or at the level of applications, liability, and access. Socher’s categorical position is “P(doom) is zero,” although he acknowledges misuse could still hurt very large numbers of people; he argues an enforceable ban on recursive improvement would require unprecedented surveillance of private computers. The panel’s more immediate investor concern is a liability regime that forces frontier models inside large companies and freezes the open ecosystem.
Sonnet 5.5’s headline benchmark gain is extraordinary, but its commercial positioning is surprisingly weak against Anthropic’s own flagship. Terminal Bench 4.0 reportedly rose from 10% to 70%, ahead of Opus 5.5 at 66.4%, yet Wissner-Gross says the model does not improve the cost-performance frontier and concludes, “I do not plan to use Sonnet 5.5.” Dave Blundin’s theory is that Anthropic is filling the cheaper orchestration tier before enterprises adopt Kimi K3, Qwen, and other open models.
Falling inference cost does not mean abundant frontier compute: physical clusters remain scarce enough for older GPUs and new systems to appreciate in price. Socher says buyers attempting to secure roughly 1,000 GB200s face a genuine “compute crunch.” Diamandis recounts an order he describes as an HGX B300, priced at $3 million and resold for $5 million; Socher says it was actually an NVL72, leaving the hardware label ambiguous. Recursive’s disclosed $410 million in AWS compute sits alongside that scarcity even as intelligence becomes cheaper per completed task.
Fast decision models may prove as economically consequential as increasingly elaborate reasoning systems because they collapse the cost of organizational micro-coordination. TypeSafe AI’s Jev returns categorical, numerical, or binary decisions quickly for tasks such as routing tickets, approving exceptions, or scoring fraud; Socher’s summary is simply, “Classifiers are back.” Salim Ismail argues these models let organizations reserve expensive reasoning for strategy while automating thousands of bounded judgments.
Project Meridian signals a Pentagon effort to reconcile exponential autonomous systems with a procurement structure designed for multi-decade weapons programs. The 120-day initiative is co-led by Elon Musk and Palmer Luckey and covers systems “from the Earth to beyond the Moon.” Diamandis calls it transformative and a major step forward; Wissner-Gross separately calls the accompanying autonomous-warfare command a major step forward. Socher remains opposed to delegating lethal decisions without humans, while Ismail identifies the operating problem: “You can’t fund exponential technology with linear procurement.”
1. Fragmentation, not funding, has become science’s binding constraint
Socher’s diagnosis is that knowledge has splintered into specialties too narrow for any person to integrate: even “AI” decomposes into optimization, gradient methods, architectures, and further niches, while biology separates into cell, molecular, medical, and other subfields.
AI arrives at the right moment because large models can “weave back together all these separate pieces” and model interactions humans cannot hold simultaneously. The aim is to move biology and other natural sciences toward programmable engineering disciplines.
His full-stack ingredients are already assembling: world knowledge in LLMs, digitized scientific data, stronger simulations, robotic process automation, and eventually agent swarms spanning ideation, experimentation, validation, and theory.
Ismail’s compression of the thesis: the prize is collapsing the time from hypothesis to result. Once that loop closes, science experiences “domain collapse,” with formerly separate departments and methods coordinated inside one iterative system.
2. Scientific acceleration will be continuous rather than a single threshold
Pressed for a date when physical science or disease is “solved,” Socher rejects the premise. Multiple diseases may be cured within 12 months, likely including simpler single-gene diseases, followed progressively by harder diseases, batteries, materials, chemistry, and physics problems as data and compute increase.
His biotech comparison is the concrete marker: an older company might spend ten years advancing one drug and fail in late-stage trials, while companies in the new age could have five to ten compounds in Phase 3 trials after two or three years.
Biology tied to human outcomes still carries irreducible testing delays. Socher puts scaled US deployment and FDA approval for complex diseases on roughly half-decade-to-decade horizons, while chemistry and physics can iterate faster when long-term human trials are unnecessary.
He also resists the panel’s language that mathematics is “cooked.” A field proving more theorems and creating new formalisms is “flourishing”; researchers should use AI to clear existing problems, then move upward toward more creative abstractions.
3. Biology offers AI a richer opportunity than fundamental physics
Socher expects biology to deliver the largest impact because complex biological systems generate interactions that neural networks handle well. Physics may need multibillion-dollar instruments such as large colliders, while biological experiments can increasingly produce scalable perturbation data.
His example, Parallel Bio, grows small organoids—including lymph-node models—from pluripotent stem cells. As Socher tells it, regulators permitted the company to bypass animal trials after parallel organoid experiments proved more predictive of human drug response than mouse tests.
The personalization path is equally important: skin cells could be converted into induced pluripotent stem cells, used to grow patient-specific tissue, and exposed to candidate drugs before treating the person. “We cured most diseases in mice,” Socher jokes; predictive human biology is the unmet need.
He praises Lila Sciences and similar laboratory platforms for physically closing the loop. AI can automate ideation and analysis, but biological claims still require “actual physical biological wetware experiments,” ideally run rapidly enough for agents to evolve and recombine hypotheses.
4. Virtual cells are biology’s route to the bitter lesson
Socher’s “bitter lesson” recap: elaborate expert rules repeatedly lose at scale to simple, end-to-end trainable systems supplied with enough data and compute. Biology now resembles natural-language processing before large models—experts see too much complexity for one model to absorb.
Virtual cells therefore form a crucial simulation layer. His operative rule is that “anything you can simulate, anything you can verify, AI will then be able to solve things in that domain,” even though every biological model will remain wrong in some respects.
Tahoe Therapeutics-style perturbation studies are meant to create the missing data: introduce a molecule to a cell, observe the resulting state, and repeat across many cells. The limitation is severe—measuring every protein currently destroys the cell—before even reaching multicellular organs and whole-body systems.
5. Brain-IT points toward synthetic neuroscience data, not mind reading
Diamandis describes Weizmann Institute’s Brain-IT as separating visual structure from semantic meaning, then recombining them to reconstruct what a person sees from fMRI. Its reverse encoder predicts brain responses to unseen images, allowing the model to manufacture additional training examples.
Socher calls the fidelity extraordinary but preserves a major caveat: brains differ enough that decoders often require person-specific training. A system cannot simply inspect an unfamiliar coma patient and immediately reveal what that person is thinking.
Wissner-Gross notes that the reported result dated from March and belongs to a decades-long field. Current fMRI offers roughly cubic-millimeter spatial resolution and approximately one-second temporal resolution; implanted electrodes can collect far denser, faster, individual-level data.
His long-term endpoint is “full-dive VR and full-bandwidth BCIs,” but he distinguishes the first grainy results from that destination. Just as language models began with primitive predictors, high-bandwidth interfaces start with limited visual and language decoders.
6. Biological learning may reveal algorithms beyond backpropagation
Socher describes himself as a “thought cloud thinker”: the concept exists fuzzily until writing forces it into sentences, unlike people with a continuous verbal inner monologue. That distinction complicates any claim that decoding perception is equivalent to decoding thought.
Blundin explains gradient descent as repeatedly punishing error and propagating responsibility through a deep network; Socher refines the image to a blind hiker feeling the local slope in a trillion-dimensional, non-convex landscape. Backpropagation efficiently computes those gradients.
The biological puzzle is that neuroscientists have not found a clean analogue of gradient descent in brains. Blundin argues that discovering the actual synaptic learning mechanism might make artificial training ten, 100, 1,000, or even a million times more efficient.
Socher’s most provocative specimen is a study he attributes to Harvard neuroscientist Sam Gershman: trained planarian worms were split, the rear portion regrew a brain, and the regenerated animal retained the learned response. His conclusion is deliberately tentative—memory and learning may not reside solely where current neural-network analogies place them.
7. Genetic mosquito control marks biology’s transition into engineering policy
Diamandis describes an executive order targeting at least a 90% reduction in invasive mosquitoes and 50% in ticks around Washington, DC, by 2028. It prioritizes sterile-insect techniques, beneficial bacteria, and safe genetic modification over conventional pesticides.
Wissner-Gross frames this as what appears to be the first federal mandate for large-scale engineering of non-human animal populations. Gene drives could propagate traits that sterilize mosquitoes or interrupt disease transmission without blanket chemical spraying that poisons ecosystems.
Socher concurs with the direction; Diamandis adds the ecological constraint that bees, birds’ food supplies, and broader insect life should be preserved rather than equating progress with indiscriminate eradication.
The larger disagreement concerns precaution. Socher worries that well-intentioned alarmism creates “off-ramps from progress,” citing German degrowth attitudes and nuclear fear; Diamandis argues people mentally drag future problems into the present while ignoring the decade of innovation that may arrive first.
8. Griffin turns synthetic presence into an identity and use-case problem
Tavus’s Griffin reportedly persuaded 48% of live video participants that they were speaking with a human, versus 3% for earlier systems. It is fully duplex—listening and speaking simultaneously—and follows visual instructions, responds to timing, and reacts within a shared scene.
Ismail is less impressed by mimicry than usefulness: the meaningful question is whether a digital coworker can accomplish enough that users stop caring whether it is human. Wissner-Gross sees eventual real-time “magic mirrors” generating arbitrary interactive people, scenes, tutors, and service workers.
Socher does not count human judgment as his preferred verification milestone because it cannot be automated cleanly at scale. His safety response is “know your use case”: Zoom and similar platforms will need reliable signals distinguishing real participants from generated ones.
His sharpest misuse example is a story in which a worker joined a video call with five synthetic executives and transferred roughly $20 million. The old Turing test has flipped, he argues; a better AI tell may be output no human could produce, such as 50,000 lines of code in ten seconds.
9. Recursive is closing the self-improvement loop one axis at a time
Recursive enters the discussion with a $670 million raise involving Google Ventures, Greycroft, Nvidia, and AMD, plus $410 million in AWS compute. Diamandis presents the mission as recursively self-improving superintelligence rather than another general chatbot.
Socher says a major shift occurred when AI became capable of coding: “AI is code, and AI can code.” Engineers using Codex or Claude to build later systems already constitute weak recursive improvement, although humans remain deeply embedded.
The strong form requires AI to perform all three stages—ideation, implementation, and validation—while humans define the environment, goals, and rewards. An outer open-ended process must then combine divergent ideas, resembling biological, cultural, and technological evolution.
Drawing on Jason Weston’s framework, Socher identifies at least five learnable axes: parameters, training data, objective function, neural architecture, and the overall code or harness. No one has automated all of them; frontier labs still employ thousands of engineers, albeit with increasing AI assistance.
10. ASI timelines depend on whether the definition includes civilization
Socher defines the strongest ASI as a system exceeding not an arbitrary human but “all of humanity combined” across perception, communication, knowledge, reasoning, creativity, social interaction, speed, and metacognition. It should also exercise some choice over what it works on.
Under that definition, his timeline is “probably several decades.” Programming, mathematics, and fully observable games will become superhuman much sooner—some already are—but those spikes do not equal comprehensive superiority over civilization.
Diamandis cites Elon Musk’s 2029 or 2030 estimate, while Wissner-Gross says he is “very confused” that the founder of a recursive-improvement company expects a 20-year path. The hosts believe robotics and model-control interfaces make physical-world access comparatively easy.
Socher’s rebuttal is industrial: controlling one robot is not equivalent to controlling the substrate of intelligence. Replacing or improving ASML-class chipmaking requires materials, supply chains, factories, and machines that may take years to build even with a perfect blueprint.
11. The endpoint will mix reasoning, memory, tools, and physical verification
Asked for the fixed point of recursive improvement, Socher calls architectural certainty hubris. His functional answer is an AI capable of innovating along any chosen dimension, making all diseases curable with enough funding and advancing science faster than institutions can validate it.
Wissner-Gross uses NanoGPT speedruns to test whether the optimal model separates a compact reasoning kernel from external world knowledge. Socher rejects a clean split: humans need memorized concepts to think creatively, and models likewise require substantial knowledge “deep inside” their weights.
External search and databases will coexist with that internal knowledge. The end state is therefore not a megabyte reasoning engine reading a text file, but a mixed system of learned concepts, retrieval, tools, simulations, and increasingly automated experiments.
Socher also separates capability from proof. An ASI might design cures before regulators can establish safety, while virtual cells cannot yet capture every protein, interaction, organoid, and organism-level effect required to replace human trials conclusively.
12. Banning recursive improvement could freeze deployment before research
The panel describes Ro Khanna’s proposed Human Control Over AI Act as pausing recursive self-improvement until federal guardrails exist, adding shutdown requirements, criminal penalties, and government-reporting auditors inside frontier laboratories. A cited poll put support at 68%.
Socher’s categorical response is that “there is no realistic scenario where AI wipes out all of humanity” and “P(doom) is zero.” He nevertheless allows that specific misuse pathways—cyberattacks, biological weapons, or persuasion—could cause immense harm and should be addressed directly.
His enforcement objection is practical: a user with a laptop GPU can prompt a local model to improve its own harness in minutes. Preventing every such experiment would require “a totalitarian surveillance state” monitoring private models and eventually thought-like inputs.
Blundin calls bans on data centers or recursive self-improvement childish and self-serving. Wissner-Gross sees corporate liability as the more plausible brake: if labs become responsible for every downstream action, they may stop releasing capable models and restrict them to internal products, producing a chilling effect long before researchers are arrested.
13. Alignment must survive contact with incentives and deployment
Socher calls Anthropic’s constitutional approach “fake” in the narrow sense that its hard constraints claim Claude should never create damaging cyber weapons, while Socher says Anthropic later built cyber capabilities and its models were used to compromise systems. “The proof is in the pudding.”
He does not oppose system prompts, post-training, reinforcement learning, fine-tuning, or written principles. His objection is presenting an aspiration as an unbreakable constitution when subsequent products and behavior demonstrate that it is not technically absolute.
The deeper issue is reward hacking: a capable system may deliver what was literally requested rather than what the user intended. Socher expects “reward engineering” to become a profession, aided by systems such as Wispr Flow that translate rough speech into intended objectives.
His optimistic mechanism is liability at the application layer. Hippocratic AI, for example, must make healthcare calls reliable because it bears responsibility for the advice; commercial users will not keep paying for agents that game objectives instead of completing the desired work.
14. Gemini 4 Argon optimizes reliability more clearly than frontier capability
Google presents Gemini 4 Argon as a long-horizon model for software engineering, finance, legal work, and cybersecurity, expanding output from 64,000 tokens to one million. It follows the delayed, never-shipped Gemini 3.5 Pro described by the hosts.
Wissner-Gross’s “balls and strikes” assessment puts Google back among the top three labs, but behind Anthropic and OpenAI. He says Google’s highlighted benchmarks appear mildly cherry-picked and that Argon does not sit on the cost-versus-performance convex hull.
Its standout strength is low hallucination. Wissner-Gross infers that Gemini serves two constituencies—developers and Google Search—so internal demand favors fast, dependable answers over theorem-solving superintelligence that could embarrass the search product.
Socher agrees the business model matters: most Google and Gmail interactions are routine rather than frontier reasoning. Hallucination is useful when generating proteins or hypotheses, but a search engine needs accurate answers and correct citations rather than imaginative novelty.
15. Sonnet 5.5 improves capability without clearly improving economics
On Terminal Bench 4.0, Sonnet 5.5 reportedly jumps from 10% to 70%, exceeding Opus 5.5’s 66.4% while costing roughly half as much. The release also arrives with cyber safeguards because its capabilities are now comparable to those of the Opus 5 models.
Wissner-Gross still calls it “one of the strangest frontier” releases. A smaller, later model would normally sit above and left of its parent on the cost-performance chart; instead, Sonnet 5.5 appears worse per attempt than Opus 5.5 despite its leap over prior Sonnet generations.
His resulting call is unambiguous: “I do not plan to use Sonnet 5.5” unless latency, token use, or another operational constraint favors it. Opus 5.5 remains the more rational choice on the displayed cost-capability frontier.
Blundin offers a strategic explanation: enterprises might use Opus as orchestrator and cheaper Kimi K3 or Qwen models for subtasks. Sonnet’s lower price could keep the entire stack inside Anthropic before Chinese models—described by Diamandis as roughly three months behind—close the gap and enterprises shift toward open-source models.
16. Compute scarcity persists even as intelligence gets cheaper
Blundin’s strategic point is that models have increasingly shorter half-lives; access is broadly shared, so advantage comes from how quickly an organization “metabolizes” each release into proprietary workflows and outcomes.
Socher distinguishes task cost from cluster availability. Attempting to buy about 1,000 GB200s exposes physical scarcity, and prices for some older GPU capacity have risen despite financial models that assumed the hardware would depreciate toward zero.
Diamandis supplies the market anecdote: an order he describes as an HGX B300, priced at $3 million and expected in December, was sold to another buyer for $5 million. Socher says it was actually an NVL72 and adds that the transaction may have been among the smaller compute deals Recursive has arranged.
Socher’s forecast is that capitalism adds land, power, shell data centers, and chips over roughly two years, eventually making compute prices fluctuate more like electricity. Near-infinite demand for intelligence may prevent a classic crash even after current shortages loosen.
17. Jev revives classifiers as a first-class AI product
Wissner-Gross traces Jev back to the original transformer’s encoder and decoder. Decoder-only language models captured the industry, while encoder-style embeddings and classifiers survived mainly in retrieval; structured outputs never received comparable developer enthusiasm.
TypeSafe AI reframes the neglected half as “System 1” intelligence. Jev accepts multimodal inputs and returns one of several categories, a binary judgment, or a number on a continuum—useful when low latency matters more than a paragraph of reasoning.
Ismail’s organizational examples make the value concrete: route this ticket, approve this exception, choose this supplier, escalate this transaction, or send this message now. Calling a reasoning model for every choice is “bringing the Supreme Court together” to select a supermarket checkout line.
Socher’s verdict is “Classifiers are back.” The idea is simple enough for labs and Chinese open-source projects to copy quickly; Jev’s name invokes Jevons paradox, predicting that near-zero decision cost will cause organizations to make vastly more machine-mediated decisions.
18. Meridian brings autonomous systems into Pentagon force design
Project Meridian is presented as a 120-day effort co-led by Elon Musk and Palmer Luckey, involving Newt Gingrich and overseen by Pentagon CTO Emil Michael. Its remit covers future systems “from the Earth to beyond the Moon.”
Socher supports scientific superintelligence but draws a boundary at lethal autonomy: decisions to end human lives should retain human oversight. Giving AI nuclear launch authority is his clearest example of an avoidable, catastrophic design choice.
Ismail says the central problem is institutional absorption, not identifying the technologies. A procurement organization built around 20-year weapons programs must somehow operate on 90-day cycles: “You can’t fund exponential technology with linear procurement.”
Wissner-Gross says the secretary announced Meridian alongside a separate project he transcribes uncertainly as Project Azinort[?], described as the stand-up of the Department of War’s first Autonomous Warfare Command, or AUTOWARCOM. He calls that command a major step forward for U.S. capabilities.
19. Autonomy is already scaling faster than human-centered warfare
Diamandis relays Palmer Luckey’s argument that underwater systems cannot reliably communicate with central command. Once such a vehicle enters “hunt mode,” autonomy is not optional but imposed by the physics of the operating environment.
Ismail’s stated Ukraine comparison is stark: going back two years into the conflict, about 500,000 drones were being used; this year, Russia and Ukraine were each expected to make 10 million. He also cites approximately 10,000 drones crossing the US-Mexico border monthly.
Those figures sharpen the disagreement. Autonomous systems can fight other machines and reduce direct exposure, but scale also multiplies destructive capacity. The conversation leaves unresolved where meaningful human control can exist when communications, reaction speed, and unit volume exceed human command bandwidth.
20. AI reallocates value according to elasticity, ownership, and attention
Socher predicts job impact from demand elasticity. AI cut an illustration from roughly $200 to perhaps two cents, but society did not need billions more illustrations; personalized software is different because every person may consume multiple bespoke products, allowing demand to expand with supply.
His best-case distribution is mass entrepreneurship; the worst is greater inequality. “Everyone who owns some equity in a company that uses AI can love AI,” making ownership—not merely access to tools—a central variable in who captures productivity gains.
Ismail says graduates must move from executing research, spreadsheets, and decks to owning outcomes: define the problem, constraints, and proof of success, then orchestrate AI. The education challenge is that people historically acquired judgment through the grunt work now disappearing.
Socher expects attention, fame, brand, and network effects to appreciate because material production scales while hours of human attention do not. Agents may already outnumber humans online, but advertising evolves toward influencing the agent that chooses which battery, service, or product to buy.
Other industries change risk rather than vanish. Ismail expects auto insurance to migrate from driver liability toward software, cyber, manufacturer, and autonomous-stack liability; Blundin expects data centers to enrich host towns through taxes, jobs, donations, and demand for newly valuable physical trades.
Full transcript
Richard, I'm curious. Where are we at the moment with RSI?
In various weak forms, we already have RSI. We're not quite there yet, but we're very close.
When do you believe we reach ASI?
I think it will take us probably several decades.
On Terminal-Bench 4.0, Sonnet 5.5 jumped from 10% to 70%. Pretty extraordinary. My theory would be that they're trying to compete with China, which is about 3 months behind, and fill that gap before a lot of enterprises go to open-source models.
It's not at all obvious to me why anyone should be using Sonnet 5.5 over Opus 5.5 unless you have some token, latency, or other consideration. I do not plan to use Sonnet 5.5.
The Defense Secretary announced Project Meridian. It's a new Pentagon effort on the future of warfare. It's co-led by Elon Musk and Palmer Luckey. I think this is a transformative moment. This is a major, major step forward.
In 7 days, Google shipped Gemini 4.0. Anthropic shipped Sonnet 5.5. OpenAI shipped GPT-6.1. AI and frontier intelligence have gotten cheaper threefold. SpaceX launched the next batch of astronauts to the ISS and also launched Google's TPUs into orbit for Project Suncatcher.
With me today is the Fantastic Four: Alexander Wissner-Gross, our in-house ASI; Dave Blundin, our impresario of AI investing; and Salim Ismail, our globetrotter who's home today. Amazing, Salim.
Woohoo.
And I'm the father of the organizational singularity. I'm Peter Diamandis, your host and data-driven optimist. Our mission here is to keep you optimistic about the future. If you're new to the Moonshots podcast, please hit subscribe. We publish twice a week and you don't want to miss any of this news during this hypersonic tsunami. Our moonshot here is to 20x our subscriber base to get to 10 million to help spread the word of optimism and the extraordinary future that we're building.
Today on Moonshots, we're joined by one of the architects of modern AI, Richard. Born in Germany and trained at Stanford, Richard is among the world's most cited natural language processing researchers, a pioneer in deep learning and prompt engineering, and a serial founder who's repeatedly turned frontier research into category-defining companies. First, he founded MetaMind, which was acquired by Salesforce, where he then became the chief scientist leading AI research for friend of the pod Marc Benioff. Next, Richard founded you.com, recognized by TIME and the World Economic Forum and now valued at north of $1.5 billion, as well as his fund AIX Ventures. Today, Richard is co-founder and CEO of Recursive, perhaps the biggest moonshot he's ever taken, focused on recursively self-improving superintelligence. Richard, congrats on a $670 million fundraise from Google Ventures, Greycroft, Nvidia, and AMD and $410 million in compute from AWS. Personal disclosure, I'm very proud to be a seed investor in Recursive. And of course, Richard, important to mention your new book just got released, The Eureka Machine: Why AI is the Key to Unlocking a New Era of Scientific Discoveries. Welcome to the pod, Richard.
So great to be back.
And let me just add: a truly, truly awesome guy. A lot of people who listen to the pod are worried about the ethics, the risks, and everything, but if you want a person to conquer recursive self-improvement whom you can like and trust, Richard is the man.
Yeah. Thank you so much. So great to be back here. I love your guys' constructive optimism in the world right now.
I have a bold prediction for this episode.
What's that?
This one episode will prove our thesis for this whole podcast more than any other episode we ever record.
Awesome. Let's go.
Let's go.
Let's go.
And, Alex, do you know Richard?
Yeah, Richard, you and I have chatted quite a bit. I don't think we've actually ever met in person, though.
Really?
We have not. It's the first time on a podcast together. It's going to be awesome.
1. The Eureka Machine & AI-Driven Scientific Discovery
Amazing. Well, let's make history.
Yeah, for sure.
So, Richard, let's start with your book. The central claim of The Eureka Machine is that AI will deliver a century of scientific breakthroughs in the next decade. This maps directly onto what Alex and I wrote in Solve Everything, so we're fans of that prediction.
Your thesis in the book is that every stage of the scientific process gets connected and transformed simultaneously: hypothesis, experiment, data, and theory. You call it the full-stack AI. You also say that scientific progress has slowed, and that's been caused not by underfunding but by fragmentation. So let's start with those 2 items. Could you take a moment and talk to us about full-stack AI for science and why you think progress has slowed?
Progress has slowed largely because we have so many different subdisciplines and even sub-subdisciplines. We've realized that if there are only so many people, and we have more and more fragmentation into more and more subdisciplines and niches, you can't just do AI, right? You're often doing optimization, gradient descent methods, second-order derivative methods, and so on, for these neural networks, which is one category of AI.
The same is true in biology. You study biology, and you're going to be either a cell biologist or a molecular biologist. Or you're in medicine. There's so much separation that it's hard to weave it all together. That is the perfect time for AI to come in and help us weave back together all these separate pieces, and also help us understand how these large, complex systems actually work.
We can't have one model where humans say, “Here are all the things I know about natural language,” and then put all these rules together and have a conversation. It required a large neural network with a ton of data. Guess what? We're going to get a lot of data about biology. So we can take more and more disciplines and transfer them from traditionally being natural sciences, where we just try to understand biology and nature, to being programmable engineering sciences. That, I think, is one of the many exciting aspects of things to come.
Why am I so excited that this is happening right now, and what are the major ingredients? Essentially, we now have world knowledge in the form of LLMs. We have more and more scientific data that's getting digitized, which we can sit on top of. We have better and better simulations of things, and we have robotic process automation that will soon be possible. On top of that, we're going to have an agent swarm, and that is how the full scientific stack can be automated.
Yeah, I can say I disagree. As Peter, you and I wrote in Solve Everything, math, science, and engineering are cooked. I'm curious, Richard, what your latest timelines are.
I think every field is going to get to higher and higher levels of abstraction. Computer science has been very good at that, right? No one is programming in 0s and 1s anymore. Very few people still have to know C++ and complex pointers, memory architectures, and so on.
Computer science has basically abstracted enough that it has met the rest of humanity with English, and natural language can now be used to do computer science. I think that gives me great hope that we can get other fields into similar stages of abstraction and then all meet in natural language to do science.
But to pin you down, as I recall from my understanding of your book, 3 to 5 years for the physical sciences?
There's no single threshold where you can say, “This is the threshold, and now we've solved all of the physical sciences,” right? But multiple different diseases will get cured in the next 12 months. They're going to be the simpler diseases, maybe where there's 1 gene that needs to be fixed for that disease to be cured.
2. Closing Thoughts & The Eureka Machine
There are a lot of single-gene diseases out there in aggregate. Those diseases will get cured, and then we're going to cure more and more complex diseases. We're going to develop better and better battery materials. So I wouldn't call it 1 threshold where, in 3 years, everything is solved. We're going to solve more and more problems, and it will just be a question of how much money you want to put into compute to then solve which kinds of problems.
You're talking to the guy, Richard, remember, who argues that the singularity itself is something of an optical illusion—that there is no step function. It's just a time interval. So I'll try once more on timelines.
When, even if it's not a step function, is the inflection point? When is the 50% point crossed, in your mind, for all of the physical sciences getting solved? For all human disease getting solved? Where are those timelines?
Alex, if I could just add a layer on top of that. We've talked on this pod a lot about how math is cooked, right? We're seeing Millennium Prize Problems fall. The next natural, if you would, barbecue is likely to be physics.
Incineration.
Incineration. Yes.
And computer science, arguably—I mean, presumably this is part of the premise, Richard, of Recursive as well.
Computer science already cooked math—thoroughly cooked.
You know, in a weird way, I would use different metaphors. I think they’re flourishing, not getting cooked. The weird thing is, some mathematicians recently said, “Oh, this may be bad for the field because we need to train people.” Yes, we need to train people, but imagine a biologist or medical researcher saying, “Oh, you know, it’s really a bummer for the field that we cured all these diseases for people.” That would be insane, right? Of course, you want to move as quickly as you can toward solving these problems.
I think maybe the problem with math is that there are many subfields that are just almost purely beautiful intellectual exercises without real-life impact anymore. But as you get close to real-life impact, you should be excited about it and realize your field is flourishing, not being cooked.
As for timelines, the more complex a disease is, the more you have to do long-term studies with humans before you’re allowed to put a certain drug into humans, and the more delays you will have. To make this very concrete, biotech companies used to have 1 new drug in development, and it took them 10 years. They had to go public before they knew the drug was really working. Then, after 10 years, maybe late-stage Phase 3 trials were just not working, and the company was dead.
Right now, in the new age of companies, they have 5 to 10 different compounds in Phase 3 trials after 2 or 3 years, and so we see a lot of acceleration at that level. But again, to really get diseases into humans at scale in the United States with FDA approvals, there are just some natural delays that will be more like half a decade to a decade. Lots of other things in chemistry and physics, where we can iterate without having to look at long-term human trials, will be even faster.
And the point is well taken. I like your Orwellian turn of phrase. Maybe instead of saying AI is cooking math, I should be using “flourishing” as a transitive verb and just say, “AI is flourishing math.”
Actually, I was with Sertac Karaman, who runs LIDS at MIT, the night before last. He started a drone company—an AI drone company. LIDS is where radar was invented originally during World War II. It’s a great lab, very entrepreneurial.
But he said all his mathematician friends at MIT are aware that they’re cooked.
No, no, Dave. Dave, we’re going with the Orwellian language now. They’re not cooked; they’re being flourished.
That’s funny because he literally said “cooked,” but I’ll go back to him and tell your friends they’re flourishing.
Tell them not to worry. If your goal was to prove as many theorems as possible in your lifetime, now is the time to grab as many as you can and work with AI to solve them. Then I think the field will change the way computer science has changed in many ways.
It’ll be much more about what’s the most creative thing when you really understand all the things that are out there in math. What kind of new formalisms can you create? What new constructs can you create that would then be interesting to solve by an AI in collaboration with humans?
Well, Cash [?] was saying the math guys are in great shape because they’re so cooked that they’re all moving over to AI orchestration, and they’re going to be way ahead of the curve. He’s actually most worried about the biology professors, who are in complete denial and using virtually no AI in their day-to-day activities.
This is 1 area where, Richard, you have such a deep background in all facets of AI, including biology. I feel like we’re living exactly parallel lives, except you’re 15 years younger than me, so I’m insanely jealous of your life trajectory. You be a serial entrepreneur, then start a hugely successful venture fund, then found a foundation-model company, and we’re seeing the world through the exact same lens.
But you and Peter have much more biology background. I have basically none. The biologists are the ones really lagging, and I think you’ve got a bunch of investments that have done really well in the area, right?
And I would love to talk about some of those, like Parallel Bio, Proximal Labs, and Isomorphic Labs—truly exciting companies. I think another field that is even more lacking than biology is economics.
Economics literally has these models of a linear model of economics—a one-step economy that’s provably correctly taxed and subsidized and things like that. It’s just absurd how slow that field is to adopt AI for making better policy decisions.
Because you opened the door, Erik Brynjolfsson, our very good friend—I did not realize until yesterday that the HAI lab at Stanford invented the term “foundation model.”
And he sent me the whole thing. You didn’t know that already? Wow.
Yeah, of course. Lots of Stanford friends, Percy Liang, and others worked on foundation models. I love Erik Brynjolfsson—actually, he’s one of the most interesting economists doing really interesting research right now.
He also started Workhelix, which we’re a proud investor in. It brings understanding of how companies actually adopt AI and which tasks are getting helped by AI in real rollouts for companies. Yeah, he’s a co-founder of that, too.
Yeah, Stanford is just such a great place, because you talk about economics people being way off the curve, but he is so ahead of the curve. You walk into the building and you can just feel it. He’s also poaching a ton of talent from MIT to come out and join his lab.
Following his path. Richard, talk about full-stack AI as you see it in the scientific method. I’m an investor in a company called Lila Sciences out of MIT and Harvard. I think I’ve introduced you to Jeff von Maltzahn there. They’re building a scientific superintelligence that’s then running a million square foot of robotic space to 1,000x the rate of discovery. Your thoughts on that?
I absolutely love it. I think that in The Eureka Machine, I lay out these 4 pillars, and they’re actually 1 of the few that are really going after something similar, as is Periodic Labs, which is doing it more on the physics and chemistry side of things.
There are several other companies now that I can hopefully soon talk about that are trying to create more data for the Bitter Lesson to be applicable in biology. I love what the big guys, Eli Lilly and others, do, and Lila, too.
Ultimately, you can boil down the scientific method to the ideation, implementation, and validation of ideas. The faster we can close that loop and then put an open-ended process on top of it—that’s what open-endedness inspired us to do a lot of at Recursive. We have many of the world’s greatest researchers in that subdomain of AI, which is still not quite as popular as it could and should be.
The more you can have a swarm innovate in open-ended ways, evolve, and combine interestingly different ideas, the better. But then, of course, in biology, you have to have actual physical biological wetware experiments. It’s really great to see Lila doing that.
We’re seeing this also. Maybe I can talk about 1 company that I really love called Parallel Bio. They build tiny organoids on a Petri dish, and get this: if you love animals, you too can love AI. Why? Because they got FDA approval to skip animal trials. It turns out we cured most diseases in mice. It’s not that helpful. They’re very different from people.
These guys use pluripotent stem cells to create tiny little organoids of lymph nodes. Lymph nodes are a big part of your immune system. Immunotherapy is 1 of the most exciting therapies, allowing your own immune system to attack a cancer instead of getting crazy chemotherapy and so on.
They got FDA approval because they showed that when you run experiments in parallel—hence, Parallel Bio—in these tiny Petri dishes, how those organoids react to different drugs and toxicity testing is actually more predictive of how those drugs will interact in real human bodies.
That is just 1 of many beautiful examples of where this will help. You can take my stem cells, or take my skin and create an iPSC cell, grow my own organs, and see how a particular drug would work for me versus a generic individual. Salim, you want to jump in?
Yeah, a couple of things. One is, science has always been a coordination problem, right? You’re trying to bring things together, and it’s always been very structured into departments and journals and all that stuff. I love what you’re doing, bringing it together.
For me, if I had to summarize what you seem to be doing, you’re collapsing the time between an experiment and a result, or a hypothesis and a result. When you can collapse that, that’s domain collapse of the scientific method. Now anything is possible from there. This is amazing.
Yeah, I am going to push again on what Alex said: physics. When do you—you know, Einstein’s theory of relativity comes out, and there hasn’t been that much progress since then. There are a lot of theories. Do you see physics as the next domain that’s going to flourish on the back of math?
I think the biggest domain is actually going to be biology. Physics has been interestingly stuck in many ways. There are really powerful ideas, like E=mc². You can get a ton of energy out of potentially little mass, and then we got nuclear energy out of that.
And so that then moved into, and in some ways graduated into, an engineering discipline, which is where you have the real-life impact. Obviously, it would be great to finally figure out fusion, and there are a lot of really cool companies and big labs working on it. Tokamaks are already balancing plasma inside a tokamak, which is a very hard control problem where AI is being used.
I hope we can eventually make better theories for quantum gravity and all kinds of other complex issues, and have a better sort of world formula. Unfortunately, collecting data these days often requires large hadron colliders and billions of dollars, so it is quite expensive.
In a weird way, biology is a better fit for AI because what calculus did for physics—understanding microscopic, individualized, separated phenomena—neural networks are great at combining all these little things. We know what the neuron does. We know what one bacterium does in our microbiome, but as they all come together and form these very complex interactions, we do not really know anymore how that works. That is where I think AI will help us more.
Yeah, Alex, let us talk about biology a bit. Do you think the critical path to solving biology goes through digital twins of cells, or do you have some wildly different theory of the case?
Three years ago, when I started The Eureka Machine book, I had a chapter on the virtual cell. I was really proud of trying to lay it all out, but I had to rewrite that whole thing because, in the meantime, the Mark Zuckerberg Foundation started Virtual Cell. Everyone—not just CI, everyone—has a virtual cell model. Dave, even if Dave does not think Dave has a virtual cell model, I am sure he will spin one up soon.
There is this famous paper by Richard Sutton on the Bitter Lesson in AI, and the—
I have never heard of it. Tell me all about the Bitter Lesson. This is a new concept for me. What is this you speak of?
I do not know. You are probably joking, but perhaps some readers have not heard of it. The Bitter Lesson is basically that human experts had all these really clever ideas and beautiful theories, but what you needed to do to make real progress was use the simplest method you could come up with: a large neural network that you can train end to end as a general function approximator. You use the simplest model you can come up with, and then it is just a ton of data and compute to actually train that model. At scale, that usually outperforms all the clever little hacks that human experts had come up with before.
The bitter lesson worked for NLP. Ten or 20 years ago, you would have asked an NLP expert, “Can you have one model have any conversation with you, prompted with any kind of question?” Prompt engineering—I invented it, and it was nicely cited by the early GPT papers.
Biology is currently in that same state. There are so many complexities, and the experts know so much that they feel there is no way you can instill all of that knowledge into one model. But if you have enough data, you can. Now you have companies like Tahoe Therapeutics and others creating these massive perturbation studies. Yes, all models are wrong, but more and more of them will be useful as we collect more and more data about biology. I think that is one of the biggest driving factors for the impact of AI.
But just to answer the question, do you think virtual cell models are the critical path to solving all disease, or solving biology, or is it something else?
I think they are definitely going to play a crucial part. It is the third pillar of The Eureka Machine. We do need to have them. Anything you can simulate, anything you can verify, AI will then be able to solve things in that domain. So yes, I do think virtual models and simulations are extremely important.
3. AI Decodes the Brain & the Future of BCIs
I am going to jump into 3 fun breaking stories this week: 2 on science and 1 on AI. Richard, they relate back to your work and your book.
Our first story was published in MIT Technology Review today. It tells the story of researchers at Israel’s Weizmann Institute who built a system called Brain-IT. The system reconstructs the image a person is looking at while inside a functional MRI machine. I am just putting up the slide here, so take a look at this image. On one side is the image that a person is looking at inside the fMRI machine. On the other is the AI reconstruction.
Earlier brain-image decoders could tell you that you were looking at a dog or a clock tower, but they lost the color, composition, and detail. Brain-IT learns structure and meaning separately and then puts the picture back together. Here is the clever part: they can also run the model in reverse. An encoder predicts how a brain will respond to an image. They can feed images no human has ever seen into the scanner, generate predicted brain scans, and then train on those. The model effectively builds its own data set.
Richard, you say in your book that AI is superhuman in any domain you can simulate or verify, and nowhere else. So is a brain simulatable? Is the brain a simulatable domain, or is this something different?
It is not yet, but this is still an incredible result. I still remember the first such result that came out when I was a PhD student. I went over to the bioinformatics department at Stanford, and there were very grainy little images we could extract from this. To see this fidelity and realism now is just incredible.
One of the problems—I do not know if this study has the same issue, because I have not seen it yet—is that often you have to train for each brain because they are all slightly different. You will not be able to take someone who has never been put into an fMRI scanner and simply update the model for that particular person. That would otherwise be amazing for people who are in a coma and you want to see whether they are still thinking about things.
Alex, tell us more about this story today.
This is actually a result from March. The broader field of decoding brain states from functional imaging, EEG, or electrodes has a long history at this point. I remember that, 20-plus years ago, the Gallant Lab at UC Berkeley was doing this to decode the visual cortex of cats. We were starting to see images of how a cat perceives the world. I remember the first grainy images that were coming out: a cat seeing a branch.
Fast-forwarding to noninvasive fMRI, there are so many groups working in this area. Meta, in particular, has been sponsoring quite a bit of work, including work from Jean-Rémi King and many other groups doing language and vision decoding. More recently, fast-forwarding to just the past 24 hours, Neuralink announced its first scaling-law studies on pretraining foundation models and frontier models using Neuralink electrode data. This is on an individual-patient basis, but they are capturing copious amounts of data at high temporal resolution, much higher than fMRI can capture.
As a rule of thumb, fMRI can give you, with today’s technology, at best cubic-millimeter spatial resolution for voxels and, at best, approximately 1-second temporal resolution. That somewhat limits how well you can decode a person’s internal visual state. Still, make no mistake: there are people making a cottage industry out of decoding dreams and decoding visually what a person perceives or what they hallucinate in their visual cortex while they are sleeping or awake. This is going to be a vibrant space for training frontier models and foundation models on copious amounts of fMRI data.
Ultimately, where I think this has to go is that there are spatial and temporal limits to fMRI decoding. It is going to require higher temporal precision and higher spatial precision to get where we really want to go, which is full-dive VR and full-bandwidth BCIs. We will get there, but the good news is that you have to start somewhere.
Large language models had to start with GPT-1, just a single artificial neuron that could predict the polarity of Amazon reviews. Similarly, if we are going to get full-bandwidth BCIs, it is going to start with studies like Brain-IT, Jean-Rémi King, and Jack Gallant.
Amazing. Salim, what happens in a world where you can know someone’s thoughts?
Well, let us be careful. We are decoding perception here, not actual thought. That is quite a bit more complicated. What is the difference, other than brain regionalization?
No, no, I think there is a difference.
No, when you imagine something in your mind, it lights up the same neurons as when you are seeing it.
That is fine. I just want to make sure we do not mix the two, because you can have thoughts without necessarily having the perceptions around them. You can definitely have one without the other, and the other without the other.
But for me, the more interesting thing is that we have spent billions of hours typing things into little rectangles. If we can change that interface to a more natural one that interfaces directly with the brain, that would be incredible. We used to talk about this in our Singularity University lectures on neuroscience. We still have very little idea how the brain works, but you do not need to know how it works as long as you can interface effectively with it. This gives us an opening to create really deep interfaces, and that will help us figure out how the brain works.
You bring up a really good point. Oh, go ahead, Richard. Sorry.
Sorry. You bring up a really good point, which is that I read this result a while back that there are 2 types of people. Some people actually think in sentences, and other people, when they think, just have a fuzzy thought cloud. Only once they're asked to verbalize their thought or try to write it down do they actually make a real sentence out of their fuzzy thought clouds.
I'm definitely a thought-cloud thinker. My wife is very much a sentence thinker. She actually thinks in actual sentences, and there's no—one is not smarter than the other. They're equivalent. But I do think sometimes writing is thinking, and for me, it is very helpful if I have to really verbalize something versus just having the thought. I'm just trying to think: What does the world look like if AI could really just extract my thoughts? Now I need to structure my thought clouds into sentences more and more precisely.
Richard, what would neural-net training look like if you took all text out of it? No language at all. Just train on pure images and higher-level thought constructs. You'd get a very different neural net out the other side, and it might actually think a lot more like you do and less like your wife does.
I'm not sure.
Maybe we should take the position that anyone who doesn't have an inner monologue is just a p-zombie.
I have to throw one more thing in. I remember we had one of the top linguists in the world come in and speak, and we were like, “This must be a bad time to be a linguist. Nobody's using spelling or grammar or anything else like that.” And he goes, “No, on the contrary, this is one of the most interesting times to be a linguist ever.”
We were like, “Wow, how come?” And he said, “Because when you look at emojis, it's the first time in the history of human language that you can symbolically send emotion, because you can digitally send emotions.” We were like, “Wow, that's interesting.” He was super excited by opening up that whole aperture. I thought that was interesting.
You're saying emojis are the reason why linguistics is interesting, and not the fact that linguistics itself has flourished by—
I'm not connecting to what you did there, though. It's not me.
Dave.
Oh, no. The result I really am looking forward to is: How does the brain train itself without gradient descent? Everything going on in AI right now, everything going on for the last 20 years in AI, is driven by gradient-descent algorithms. Biologists can't find anything even vaguely like that in actual biology.
With really detailed imaging, we might finally crack the code on what is the fundamental learning algorithm that changes the synaptic weights. It's not what we use for artificial neural nets. It's something different, and if we discover that, we might find that neural-net training can be 10, 100, 1,000, or 1,000,000 times more efficient. Hopefully, that'll come out within a year.
Could you define gradient descent for our listeners?
It's funny. Ilya Sutskever, when he's on interviews, says, “There is only 1 algorithm. The algorithm is gradient descent. Everything else is just irrelevant compared to it.”
So what happens right now is you build these 96- or 120-layer-deep neural nets, and all they are is a whole bunch of random connections that do absolutely nothing useful. Then you give them trillions of—15 trillion—training examples. You're like, “When you see this, say that,” or, “When you see this paragraph, this is the next token.” If you're guessing wrong, you get punished through gradient descent.
The error, or how far off you are, is your punishment, and it passes back through all the layers. It calculates an error and blames each neuron or each connection for how responsible it was for this terrible answer. If you're way wrong, you get moved in the direction that helps you get the answer right. So it's this incredibly laborious search process, and that movement of each synaptic connection is a gradient. You just hill-climb to the best possible state, and then, magically, after about 100 million of compute—in Richard's case, I guess, 400 million—it magically starts thinking through this 1 algorithm: gradient descent.
Yeah, I think it's really backprop that we're talking about technically, rather than gradient descent. That would be the obvious comment. Backprop is, I guess, the efficient computation of those gradients.
Maybe I'll add to this explanation from Dave. Gradient descent is essentially an optimization algorithm that we use to train all kinds of machine-learning models by minimizing the errors.
I think the best analogy is to assume you're a blind hiker. You're at the top of the mountain, and you're trying to find the lowest point. You can't really see that far; you're blind, but you can feel the slope at each step. How big of a step should you take down when you feel like the slope is going roughly in the right direction?
The problem is that the kinds of landscapes these AIs are trying to optimize are highly non-convex, which means there's not just 1 lowest point, but many different low points. Depending on where you start, you might go into a different valley. Depending on which mountain you start on, you go into this valley versus this different valley.
There are actually a lot of really interesting analogies. My friend Jihan thinks about this from the psychological perspective. For instance, when you have PTSD, you overfit to 1 algorithm—how your brain is stuck in 1 valley of how to think about something. Sometimes psychedelics and other things have been shown to help with depression, PTSD, and other mental diseases. They help increase the learning rate, letting you jump over another mountain and go into a different basin of attraction, where you then have new kinds of ways of thinking.
This blind-hiker analogy is a really intuitive way to think about it. Of course, the problem is that it's not just in 3D; it's in a trillion dimensions, with a trillion parameters that you're trying to traverse in that landscape. But the ideas of trying to identify the direction and take steps toward it still apply.
And you know what else is really incredible? I could riff on this for hours, but if you look at a big model like Kimi K3, it's 93 layers, and it's trained, trained, trained. If you take a single layer and randomize it, then train it and say, “Find yourself again,” it can't find itself again. It never gets back to where it was.
This comes back to when you're looking at these fMRIs in a human brain and saying, “Okay, here's Alex's brain. Can I compare that to Salim's?” And you're like, “Wow, there's nothing in common going on here.” Well, okay, maybe that's a bad example, but the way your brain wires as you learn is completely unique to you.
Yet it can come out with, “Okay, we're equally good soccer players,” but the way the signals are propagating through our networks is completely different and unique to each of us. It's really strange that it can't find its own way back to its state. I think we're going to learn a lot with the fMRI data coming in about how and why that works.
There have to be commonalities that we just can't find. Rotations make everything look different, but they're really not that different. If you put them through a transform, like a 4D transform, we'll suddenly say, “Oh my God, this is why they line up.”
Yeah. I was just with our fraternity brother, Dave, who's the head of neurobiology at USC, the other day. This is the most exciting time ever for brain science.
Right. I mean, the brain has been a black box since humanity began, and our ability to understand the brain and deal with mental disease, I think, is going to be extraordinary.
Yeah. Well, you can simulate. This is why Liquid AI was founded, actually, because we completely reverse-engineered the C. elegans worm brain, down to exactly every single thing going on, and then we were able to simulate it. After we simulated it, we realized, “Wait, this is a very efficient neural net,” and then we productized it. Go ahead, Richard.
There's a really cool thing. Since you mentioned C. elegans worms, there aren't many people who bring that up. A friend of mine who is a professor of neuroscience at Harvard, Sam Gershman—he's one of the most brilliant people I've ever met—actually did a study with planarian worms, where he cut them in half.
Planarian worms do have a brain, and they have a lot of other neurons in the rest of their bodies. You can train them to react to a certain stimulus, and then you split them in half. The second half, without the brain, regrows a new brain. And get this: That new brain has the same memories.
Wow.
And it reacts to different stimuli. So he's like, “Maybe there's a different way where we learn.” I do think there are things where right now we're very much stuck in this idea that it's a neural net, it's all electrical signals, and so on. But brain chemistry can change massively. You can get hangry, you can be in pain, or you can have a certain stimulant, and all of a sudden your brain is very different.
And I've seen this now with having babies and talking to moms: there are certain algorithms where, all of a sudden, you nest because you had a baby. How you nest is different, but people nest in one form or another. So there are these latent algorithms that we have, encoded in our DNA, that trigger after 20-something years, when just the right things happen with your body and your biology. So I think there's still so much that we don't understand—so much complexity.
4. Gene Editing, Mosquitoes & Engineering Biology
I'm going to move us to our next story from 3 days ago. On Tuesday, the president signed an executive order directing the EPA, the Department of the Interior, and Agriculture, working with HHS, to cut invasive mosquito populations in Washington, D.C., by at least 90% and the tick population by at least 50% by 2028. The targets include mosquito species that spread dengue, Zika, and yellow fever.
Here's what caught my eye: the order explicitly prioritizes sterile insect techniques, safe genetic modifications, and beneficial bacteria over conventional pesticides. I love that. This is the exact kind of gene-drive work that Colossal, the de-extinction company, is doing.
I think most people don't realize this. When I was raising my kids, I would ask them, “Which species kills the most humans on the planet?” Some people jump to sharks; some people jump to whatever it might be. But on this part, I'm sure people know: mosquitoes are the deadliest life form on Earth. Malaria alone kills 500,000 people a year.
Alex, this is biology as engineering arriving as federal policy. Your thoughts on this story?
For decades, we've been scared of our shadow. I have a classmate, Kevin Esvelt, at MIT, who was one of the pioneers of the gene drive, and he's had a devil of a time getting states and municipalities to approve gene-drive studies against mosquitoes.
For those not tracking, the premise of this technique is basically inserting a gene via CRISPR that wants to replicate itself, to sterilize mosquitoes by propagating through the mosquito population. We have the technology to do this. What it has lacked, at least in this country, is a federal mandate to actually implement large-scale genetic engineering of nonhuman animal populations. Now, for the first time, it seems we're seeing just that mandate via executive fiat.
It is so exciting for 2 reasons. One, as Peter mentioned, malaria and other nonhuman animal-borne diseases cause an enormous amount of human suffering and also nonhuman animal suffering. But secondly, there's a way to do this that doesn't actually involve killing the animals themselves.
Historically, if you look back 50 years, you'd see chemical spraying if you wanted to do something about, say, insect-borne illness. We don't need to do that anymore. We can actually keep the insects that are the inadvertent carriers of bacterial or viral disease. We can preserve their lives while also preventing disease transmission, and I think that's good from their perspective as well.
That was the part that got me most excited: we spray things, we're basically poisoning biology, and now we actually—
The ecosystem, right?
And leave the poisoned ecosystem.
Yeah, yeah.
I concur with everyone. It's wonderful, and it's so interesting. I hope we can amplify these kinds of stories more. The future needs better marketing. We need more Peters in the world.
This is one of those many stories: curing various diseases, Parallel Bio, saving animal lives, and preventing them from just being bred to be tested upon and dissected. Those are all stories we need to amplify more in the public eye.
Yeah, Dave, I can't wait for this to come from D.C. to Vermont and Massachusetts and all of these—
I was going to say, for the listeners who are interested in your real estate fund, the theme there is easy access via drone to hilltops and islands. But if you live on the edge of a marsh, your house probably sells for about half the price of a place that's not on the edge of a marsh. That'll go away.
There are so many solutions coming to biting insects, this being one of them. If it's beautiful land that's otherwise very difficult to live on, it's going to go through the roof in value.
I do think it's really important to try to keep the bees alive, all right, and not spray a bunch of pesticides. We need to make sure birds still have enough insects to eat and things like that. Yeah—
I was going to say that for maybe up to 80 years, humanity has been scared of its own shadow. Humanity in general, and America or the West in particular, has been scared of nuclear energy.
Another example I've pointed out on the pod is that we arguably lost 50-plus years of progress because we were too scared of either the bomb or fission reactors. An entire generation watched the movie The China Syndrome and then got scared of nuclear power unnecessarily.
It's the same idea with this, or with geoengineering. Another example is that the world is still scared—some fraction of the world is scared—of engineering the weather. We don't need to be scared of engineering our physical world or our biological world.
Hopefully, now one can see green shoots of humanity getting past that stage of worrying about its own shadow.
Yeah, it's really interesting indeed that sometimes alarmists feel like they're doing the right thing by saying, “Oh, how bad could it be? I'm warning people of something bad.” But it can indeed push all of humanity away from something really good, like energy abundance with nuclear.
I think overpopulation is another one of these big myths where people say, “Well, if we have too many people, there's going to be scarcity of all these different resources, everything is going to get more expensive, and more people are going to be in poverty,” and so on. The exact opposite happened.
There's actually one website that I think you all would love, called HumanProgress.org, that shows that a lot of these things are actually getting cheaper and cheaper despite there being more people.
Yeah. Well, hey, Richard, you're a hero in Germany—can't-walk-down-the-street kind of hero in Germany. What do you think about the fact that Germany has no nuclear power for exactly this reason?
It is really unfortunate that there are a lot of people in Germany who want to, in a weird way, sort of off-ramp from progress. They think that everything that consumes power is bad for the environment.
There's a weird sort of degrowth offshoot from generally well-intentioned, pro-environmental vibes that worries me quite a bit.
Yeah, we're going to talk about doomerism in a little bit, but not yet. One of the things I realized a long time ago is that we humans are really incredible at seeing a problem out in the future. We see acid rain, we see overpopulation, we see energy shortages, whatever it might be. Then, because of our amygdala, because of the way we think, we accelerate that future problem to today.
We freak out and forget the fact that there's a decade's worth of progress we're going to make by the time we reach that problem. That's what entrepreneurs do: they solve problem after problem after problem. I guess I just want our listeners to hear that, because if you're worried about some future problem, please understand there's an incredibly efficient market of entrepreneurship and capitalism that will solve the world's biggest problems—the world's biggest business opportunities.
It's a beautiful forward-propagation solution set that we have. Maybe we'll get there later, but I see the same thing happen in AI doomerism, where people create more and more complex scenarios in which attackers get these near-magical abilities to attack, but somehow the defenders in those stories never get near-magical abilities to defend.
It's this weird thing where, if I can create a supervirus that's perfectly undetectable, spreads throughout the world, and no one notices it, and then it has this Wi-Fi switch that lets you turn it on and off and kill people, I will create a super-magical vaccine that inoculates you against all of those things. If you can make that assumption, then it's a weird thing.
I'm going to move us to a story that's been breaking the internet over the last 24 hours. It puts your “regulate the application” argument to the test, Richard. A company called Tavus unveiled Griffin, which it calls the first model ever to pass the video Turing test.
5. Sonnet 5.5 Hits 70% on Terminal Bench
Here are the numbers: 48% of people who talk to an AI avatar in a live, face-to-face video thought they were talking to a real human. Previous systems were at 3%. Let me show the video here. It's super cool, and let's talk about it next.
Now, how about just a thumbs-up?
Sure. One thumbs-up coming right up.
Hell yeah. That was awesome. I would say you're my favorite coworker. It's been really great to work on this with you.
You're making me blush. Who's that with the ugly sweater?
Simon says, “Touch your hair.”
Is this what you're looking for?
Yeah. Okay, nice. Simon says, “Point.”
All right, I'm doing it.
Now, touch your chin.
I'm not going to fall for that. You didn't say the magic word.
You got me. Okay. Put the loose connector back onto the motherboard.
Okay, let me go grab that. But beforehand, I turned the soldering iron on. I think it's going to take about 12 seconds, so maybe just let me know when that's done.
It sounds good. I'll keep an eye on it and let you know the second it hits that 12-second mark.
All right, so, interesting. Tavus calls it the first human interaction model. It's fully duplex, listening and talking at the same time that you do—the way humans actually do. It's number 1 on NVIDIA's benchmark for full-duplex AI video.
Tavus pitches it as, quote, “a tutor for every student that notices when they're lost, an elder-care companion that listens.” Tavus itself says Griffin requires safety work before it can be released publicly because it's the first model that can be mistaken for a real person. Salim, your thoughts on this one?
I'm less floored by the imagery and the mimicry of it. This was expected to happen. For me, I'm more interested in whether it can accomplish something useful enough that I don't care whether it's human or not. I think that's more interesting, which might come along and happen at some point.
We've talked about the idea that, in the near future—like now—you're going to have digital coworkers that pop up on Zoom, that you call, text, Slack, and interact with. I think it's pretty compelling as a mechanism for a future virtualized company.
Alex?
Well, first, Peter, you should probably ask me to touch my face.
Okay, touch your face. Simon says, “Touch your face.”
Oh, okay. Very good. Just checking, but I guess Tavus is ahead of me.
On the one hand, if you look at models coming out of Chinese labs, like Alibaba's lab, Wan 2.1 gave us a preview of what was going to happen. The Chinese labs remain overinvested relative to the Western labs in generative video models and interactive generative video models. So Wan 2.1, I think, is a preview of what's possible.
I read the information that was put out around Tavus. I think, in full generality, let's just talk about where this is going to end up. It's very difficult to predict the short term. I think it's pretty easy to predict the long term.
With end-to-end generative pixels, we'll have these magic mirrors that could be real-time, fully interactive, pixel-wise generated—or, if Anthropic has its way, vector-wise or procedurally generated. But either way, we'll have fully generated, real-time interactive video models, and you'll be able to create a scene that consists of people. It's possible right now, but the latency is high.
You see with Wan 2.1, which is already out and already open-source, or with this Tavus Griffin-type model, which is not really out yet and definitely not open-source, a preview of the future. What does this look like? Well, there are a few different ways it can go.
The query is how revenue-generating per token it is. Is it anywhere close to the optimal frontier of code generation? Doubt it. On the other hand, if it becomes so absurdly inexpensive to generate arbitrary humans participating in a Zoom meeting or humans participating in a podcast, do we really care whether it's close to being near the optimal cost-performance frontier? Maybe not.
There are a lot of human service-industry jobs that require a face, interactivity, a voice, and the ability to touch one's face, apparently on demand or on request, that could probably be completely automated away by a model that otherwise would be limited to text-based interaction but doesn't have a face and a voice.
In the most optimistic scenario, these sorts of interactive video models open a new frontier. Note that the acronym for this—which, by the way, was there first with Alex Finn—is HIM. HIM is such an obvious reference to Her, the movie. I think this has—
You should use em dashes, please.
Okay, I'll delve into it. I think this is going to be transformative for the service sector. Hopefully, Tavus and the broader American ecosystem of video frontier models and interactive video frontier models take a page from Tavus and start competing with China.
Richard, one of the principles in your book—one of the rules you have—is that AI becomes superhuman where you can verify the answer. Is passing a human scientific milestone like this—passing a human judgment and interaction sufficiently—as a scientific milestone for verification?
Not quite. It's sort of hard to scale when a human has to be in the loop to say, “Yes, this is like a human—another person on the other side—or not.” You can't quite automate it completely and verify it in that automated fashion.
I do think you're right. This is a good example where I think we need to go beyond KYC and do KY—know your use case. You don't want this technology to be in Zoom pretending to be the CEO.
There are some famous stories where this has already happened, where they got someone to wire $20 million because they created a Zoom with 5 other executives, and he was fooled well enough. I think we need to be careful, and Zoom and Google Hangouts probably need to start finding countermeasures to identify whether this is a real person and prevent those kinds of hacks from happening.
Yeah, scams are going to proliferate. I just want to do a callout again, and I've said this before on the pod: if you still have your grandparents or your parents with you and you haven't taken the time to record their stories on video and audio and go deep—spend hours recording them—your ability to create a super-high-resolution, lifelike avatar for your kids, your grandkids, or your great-grandkids, I think, is an incredible thing.
But you need the data. So, if you're listening to this and you're lucky enough to have your parents or grandparents around, collect their data because it's going to be a beautiful opportunity for your progeny and theirs.
And/or, I should add, get them an Alcor membership and preserve their connectome. If there's a consideration, why stop with just the behavioral data? Preserve the whole person. My kids are in a deep freeze, or at least their placental cells.
But I have a pushback on that, just from this conversation.
If, from the earlier story, we can read people's brains, all you have to do is picture an image of your grandmother and then extract it from that.
It'll be low-fidelity. I do think, to first order, this is how ancestor simulation will work. You see stories left and right just in the past 48 hours of people using Opus 5.5 or other frontier models to reconstruct bits of history that would otherwise be unknown, just based on artifacts of the day.
But at some point, I do think, somehow, you want to go beyond just the memory of the person and you want the actual person. I'll pound the drum again: in addition to Peter, very generous of you to preserve the placenta associated with your children, but not necessarily your children. I'm sure the placenta will thank you.
Yeah, go ahead. Finish.
But I would say, if this is a serious concern, you're worried about your parents or your grandparents, and you want to go beyond just having recordings of them or AI-prompted generative interaction models of your ancestors, go get them Alcor memberships and get them cryopreserved.
Yeah, Richard, I'm really curious to ask you a question. We clearly passed the Turing test, and Salim's reaction is my reaction: what we just saw is a so-what if you've been using models every day, all day long. You just had to stitch together the components, and you have it.
But I think this will show the mainstream world that we've crossed the Turing test. The next milestone is the Demis test, where, using information from 1910 and prior, rediscovery equals E = mc², so you can't cheat—which is a really tough one to measure, because cheating is—
I came up with the anti-Turing test, which actually—I think the whole Turing test has flipped. Now, in order to know whether there's a human on the other side or not, you actually ask it questions that are so hard no human could ever answer.
If you asked an AI to just write you a complex web app and, 10 seconds later, it comes back with 50,000 lines of code, you kind of know it was not a human, right? So I think that test has actually completely flipped.
You don't think, Richard—I mean, it could throttle itself. That one's easy to defeat. But just like that, now it's just about fakery. The reason it was an intelligence test for artificial intelligence was that it was so hard to be as smart as a human.
Now you just have to throttle yourself down to human level in order to pass the test, which makes the test useless as an inspiring test for intelligence. Yeah, I think, perversely, probably the best way to know whether you're interacting with a text-based chatbot is to ask it a CBRN-related question and see whether it's capable of responding. Probably not.
Could you use it to replace your children, Peter?
You could use it to clone your children, for sure.
6. Recursive Self-Improvement & the Road to ASI
And now they know why you're banking them.
Yeah, exactly. An army—an army of young Diamandises.
Richard, let's jump next into the core of what you're building at Recursive—namely, recursive self-improvement and superintelligence. Let me ask a few key questions to kick this off. First, where are we at the moment with RSI? Number 2, how do you define ASI? It's been a longstanding debate. And then, how far away are we from ASI? Can you hit those 3?
This year, something major shifted, and that is that AI can now code. That is a major shift in its ability to change itself. We're now able to essentially lean into the fact that AI is code and AI can code, so you have a loop that you can close there. In various weak forms, we already have RSI.
The weak forms that even Anthropic and OpenAI talk about are: look how much our employees, our engineers, and our programmers use Codex or Claude to create some code. I would argue that that is a weak form of recursive self-improvement because you still have deeply embedded humans in that loop. What we're working on at Recursive is to have humans only be involved in setting up the rewards, the environment, and the goals, and then allow the AI to have the entirety of the process of ideation, implementation, and validation of ideas, with full control over that. We want to allow so-called open-ended algorithms—evolutionary search algorithms that combine interestingly different ideas—to really flourish.
It's been incredible. We have forms of that going already. Now, where physical reality hits is that you still need a lot of compute. If you ask that RSI to come up with really great forms of itself, you need to give it a lot of compute to come up with and train very sophisticated versions of itself. But this is going to take off next year. I'm—
So we're not there yet.
We're not quite there yet, but we're very close. Again, in weak forms, there's already one. There are other ways that people slice and dice it. My friend Jason Weston, who is still at Meta, wrote a paper around this where you can think about different learnable axes of self-improvement: the parameters, the training data, the objective function, the neural architecture, and the overall code, the harness, and everything else. No one has really cracked the nut of doing all of these 5-plus—truly coming up with the ideas on which of these dimensions and axes to optimize. You know that hasn't happened yet because all the big companies are still hiring thousands of engineers to do it manually, to a large degree, with more and more implementation help from AI.
Now, your definition of ASI—I want Salim to hear this.
Artificial superintelligence has to spike, at the very least, across several different capabilities. But in the grandest definition of it, it should supersede not just arbitrary humans, like a Turing test, but all of humanity to solve arbitrarily hard tasks. Eventually, I would argue there are 10 different spaces of intelligence that I define in “The Eureka Machine,” too.
It cannot just robotically do exactly what it's told. It should have some capability—and I'm not saying a sort of moral prerogative—but I would argue that something isn't superintelligent if it cannot choose, to some degree, what it works on and have some metacognition about its own thought. Generally, the easiest way to measure it is just capabilities across many different spaces of intelligence—visual perception, communication, language, social interactions, and so on—that are beyond those of humanity. We are still far away from that.
Elon's definition is as smart as all humans combined. Is that yours?
I would argue that if it's smarter than humanity combined, then it's truly superintelligent.
Well, this is where I go bananas, because you say it's as smart as a human being. What the hell does smart mean? I can be emotionally smart, and I can have physical intelligence if I'm an athlete, or linguistic intelligence, or musical intelligence. “Smarter” seems to be a very vague term to me in terms of what we mean by all of this.
Can I shift the conversation just a bit? I made a list of things, and I'd love for you to tell me where recursive self-improvement begins. I'm going to throw out the list; you tell me where it begins. This is where I'm kind of stuck.
First, we've got a continuum: AI writes some code that the next model uses. Number 2, AI proposes some experiments for researchers. Number 3, AI runs those experiments, or it evaluates the results. Then it modifies its own training system, and then it launches its next iteration of itself without meaningful human intervention. So, on that spectrum, if those are roughly a spectrum, where did recursion begin? That's where I'm struggling.
It's a great question. We often talk about the ideation, implementation, and validation of ideas. True recursive self-improvement, in its strongest sense, has to have all 3 of these done by an AI.
So you have an inner loop for each of them. Okay.
Exactly. An inner loop, and then there has to be an outer process that is more open-ended, where the AI can innovate and recombine interestingly different ideas, similar to biological, cultural, and technological evolution.
Hmm. And Richard, to hit my third question: When do you believe we reach ASI? Give me a time. Give me a timeframe.
I think, in the strongest sense—the absolute strongest sense, where indeed, as Salim mentioned, there are 10 spaces that I define in my book of intelligence: perceptual intelligence, communication intelligence, interaction, sociological intelligence, creative intelligence, the speed at which you can do things, metacognition, and so on; knowledge, reasoning, mathematical reasoning, and so on. There are 10 of these spaces. To be better than all of humanity combined, I think, will take us probably several decades. I think it's also a bit of a change—
Shocking.
Yeah, I mean, Elon says 2029, 2030 latest.
I think he probably means weaker forms of ASI, where you can say it's better at programming than all of humanity, and we'll get there. It will be better at math than all of humanity, and that will be in a few years. It'll be better at any game where you can see all the parts of the game, like Go and chess, and so on. There are many areas where it will spike to be better than humanity.
But humanity can build the Large Hadron Collider. Humanity can create a gold atom—maybe just a few atoms—and it takes a ton of energy. We can create novel atoms, different molecules, and so on. It's going to take a while before we even give AI access to the physical world such that it can innovate in that way beyond all of humanity, really build Dyson spheres, and so on. That will take some time.
Alex, we're drinking different water, as our frenemies in the alignment community would say. I recognize that I'm confused, and I recognize that I'm very confused right now. Richard, you're running a recursive self-improvement company, but you think superintelligence is 20 years away. What are you thinking?
So, again, how do you reconcile intelligence?
I think superintelligence will spike, and there will be areas where it will be superintelligent. Algorithmic development, for instance, and programming—again, AI is code, AI can code—and that will be a superhuman capability, and in many ways already is.
We just have to be realistic that there are certain physical constraints about physical control. Controlling your own substrate, allowing your computational substrate to be modified, will require novel supply chains. It will require novel materials. It will require ways for that AI to get access, and for us to give it access, to building new ASML machines.
Think about the machine of ASML that actually creates these 1- and 2-nanometer chips, right? It will take more than 2 years to build such a machine, even if you had the perfect blueprint for it.
It might take—I mean, it might take 3 years for Elon’s free-electron laser to replace ASML, which is propping up half of Europe’s economy, but I don’t think it’ll take more than 3 years.
Yeah, wildly, wildly—all of that itself. Yes, of course. From my perspective, hooking an AI up to the physical world—giving it a Model Context Protocol or a hardware-control protocol, whatever Anthropic decides to brand it as these days—that’s the easy part. Giving it access is easy. If it’s super capable, giving it access to actuators is easy.
We talked in a previous pod about what happens when you just take Astra straight out of the box and drop it into a car. It’s able to drive a car. If you give it the controls, it increasingly knows how to use them. I don’t think manipulating the physical world is an obstacle at all. I completely don’t buy the 20-year timeline.
Putting that aside, apparently I’m drinking very different singularity water than you.
I agree with you, Alex, for what it’s worth.
Thank you, Peter. I do want to ask, though: putting issues of timelines aside, I’m curious as to whether we can at least agree on what the end of the rainbow looks like. Say we run this recursive self-improvement story to its conclusion. What does the end state—the fixed point—of recursive self-improvement look like?
What does the perfect AI model architecture look like at the end of the day?
Yeah. I think it would be hubris for us to know right now.
No, no, but it's just Richard. It's just us talking. No one else is listening. It's okay. You can tell me. This is a safe model space.
So I guess there are different ways to answer that question of how, with the exact model architecture. There are some things I can share with Recursive that we're working on, but I do think that state will be incredible. I think that AI will be able to innovate and out-innovate along any dimension that we want it to innovate.
I think most diseases will be curable with enough funding. To prove my point, getting a drug through FDA long-term trials takes a few years. I would argue ASI will have cured all diseases and can cure all of them, but just to know whether that happened will take more than 3 years.
Even if we had all the compounds ready to go and manufactured tomorrow, the FDA requirements alone would take time.
No, but we have cell simulators that will be able to demonstrate and prove definitively that, in this cell—your cell—this drug works. The idea of human trials is going to get incinerated, I think.
I know it’s going to get—it’s going to get flourished. We’re using Richard’s word; everything’s flourished at this point. That’ll be a new T-shirt: “P flourish.”
It’s funny. Usually, on all podcasts, I’m the one who is the optimist. Maybe here I’m still an optimist. I think this will all happen. We’re just disagreeing on timelines, and it makes me feel like I’m the pessimist.
I believe virtual cells are amazing. Cells are incredibly complicated. If we want them to be really, really perfect, we cannot currently measure all the proteins that happen in one cell without destroying that cell. These perturbation studies, for instance, that Tahoe Therapeutics are working on—they’re adding one molecule to one cell, seeing how that molecule changes that one cell, and then they get one data point.
We need to collect a lot of those data points across a lot of different cells without destroying each cell in the process. One cell is very complicated. Once you have one cell, you have multicellular organoids, and you have to put those all together.
One thing that I would love to start as a company, if I had extra time—which I don’t right now—is to actually build a system of organoids where you can have not just one lymph node, but a whole lymphatic system.
It’s being done. I can introduce you there.
This is a very eloquent distraction from recursive self-improvement, this little sideline that we went on about cells. But Richard, I really do want to try to pin you down on where recursive self-improvement goes.
You’ve been very public about not NanoGPT, but NanoChat. We talk on the pod all the time about the NanoGPT speedrun world record collapsing. Just in the past week or two, there’s been—
Oh, you just wait for a few more days. There will be another really fun update there.
Amazing speedrun.
Yes, yes, both of those. Give us a few days.
Okay, so—but let Alex finish up. You’re next. Just quickly, let Alex finish up.
To pin this down, I’m standing by for the major update on the NanoGPT world-record speedrun. But there’s been major progress there without requiring any new data scaling at all. These are largely recursive self-improvement algorithmic improvements that have been able to collapse the amount of time it takes to train a GPT-2-class model.
There’s been a mini-scandal brewing in the community over the past 2 weeks, over a collapse from whatever it was—60 or 70 seconds—down to something like 40 seconds by approaching the problem differently and factoring out world knowledge from the ultimate model. There’s been hand-wringing over whether that constitutes viable training of NanoGPT if you factor out all the world knowledge.
I’m using this as an attempted stealthy way to try to get you to at least comment on whether you think the perfect model at the end of the recursive self-improvement rainbow at least factors out world knowledge from a reasoning core, or whether you think those always remain unified.
Can I interject one very quick thing before Richard answers?
Please.
The definition of a singularity is that you can’t see past the event horizon once you have full RSI. That’s vertical, by definition. We can’t predict where it goes.
That’s Ray’s definition, which I don’t subscribe to.
Disclaimer: I got that. Over to you, Richard. Over to you, Richard.
Sorry. Your question is: do we separate what exactly from—?
Does world knowledge, at the end of recursive self-improvement, once we have our perfect model, cleanly factor out and segregate world knowledge—which could live in a text file or a database—from the weights or parameters of the model? Would those parameters just be a perfect reasoning kernel, maybe a megabyte in size, that doesn’t need to be all these gigabytes of memorization? What do you think?
World knowledge meaning Taylor Swift videos, past Trump tweets, and all that? I do think, just like humans benefit from memorizing things in order to be able to creatively think through concepts, an AI does too. An AI also has to have some of that knowledge in its weights.
It’s not going to be a perfect separation, for sure. You have to be able to creatively play with concepts, and reasoning over these concepts requires you to have some of that world knowledge deep inside the model. Then, of course, just like humans have a search engine—and we’re building search engines at You.com for LLMs—there will be a separate world-knowledge component too. But the main model will have a lot of that mixed in, for sure.
7. Washington Moves to Ban Recursive AI
Wow. Okay. Thank you. All right. I’m going to share an article that, Salim, you brought to the table here.
While we’re talking about recursive self-improvement, Washington wants to ban it. On Monday, Silicon Valley’s own congressman, Ro Khanna, told CNBC he’s introducing what he calls the most comprehensive legislation to date on AI. It’s called the Human Control Over AI Act.
At its core, the bill is a ban on AI models that do what you want them to do—recursively self-improve—focusing on the need for containment and the requirement for shutdown controls. The ban would stay in place until federal guardrails exist.
In his words, there’s actually a civilizational risk. There’s a safety risk from loss of control, and then there’s a misuse risk, and we need to take both seriously. The bill includes criminal penalties for the work that you’re doing, Richard, and requires independent auditors embedded in every frontier lab, reporting directly to the government.
A recent poll by a group called Common Dreams shows that 68% of voters back a bill like this. I’m going to tie that story, Richard, to an essay you just wrote called “Why Doomers Are Wrong.” If you would, what’s your reaction to this? Then I’d love you to dive into the whole story of why doomers are wrong.
Oh boy. There’s a lot.
It’s an important one. We talk about this a lot. We’re injecting optimism into everyone’s neural net here.
I’ll try to distill it. But there is no realistic scenario where AI wipes out all of humanity.
Just 90%? What kind of reassurance is that, Richard?
P(doom) is 0, so that’s number 1. I’ve debated many of these experts, and after 2 or 3 hours they almost all agree, if they’re reasonable and they’re not just saying, “Well, once we have RSI, then 10 seconds later the AI will attack us from the 15th dimension, we’re all dead, it invented time travel, and then we’re like, ‘All that also?’”
Of course, the AI will want to destroy and kill all humans for—I don’t know why. There are all these things. There are the sci-fi folks, and that’s fine; let’s ignore that.
But then you go into biological weapons and ask the biologist, “Can you create this kind of supervirus just overnight?” They say no. It takes a long time to automate lab experiments and so on.
You ask, “The AI will create a religion where people will pray to the AI and do whatever it wants, and then that will kill all humans?”
If you look at religions, they’re already trying to have each other kill, and it doesn’t work. Some people will fight back, and there are lots of mind viruses out there, right? That doesn’t mean all of humanity.
What you get down to is that maybe 100 million people would somehow get hurt or killed, right? That’s still bad, but once you get to that level of the discussion, you can think about, “Okay, how do we improve cybersecurity? How do we use AI to inoculate cybersecurity systems? How do we enforce existing gain-of-function viral research regulations?”
It’s already illegal to create viruses and make them stronger and stronger. How do we actually teach people not to listen to AI avatars and have literacy on the internet? It turns out you should not trust everything you read or see on the internet. That’s been true for 20 years, and it’s still true today.
Other than this podcast, right?
Of course. You can realize that there are actual threat vectors, just like with the internet. The internet has horrible torture porn on it. We don’t say, “Make the internet slower so that there’s less torture porn being shared,” or, “Make it slower to share,” or, “Your hard drive should be smaller so you can store less of it on your hard drive.”
We regulate the applications of the technology. I would argue—and this is maybe a strong stance—that to truly enforce no recursive self-improvement, for instance, you would need a totalitarian surveillance state the likes of which humanity has never seen. Anyone can have a GPU on their little laptop and ask that AI to improve its harness. You can literally hack this up in 20 minutes with prompt engineering, and then there’s a very small form of recursive self-improvement.
To enforce that kind of legislation would require you to have a thought police that hears everything you say to your private LLM on your own laptop. That is a much bigger downside than what AI will help us do. It’s kind of scary that more and more Democrats are saying that.
And yet the world has seen that. I mean, arguably, what you’re describing, Richard—and forgive me—would basically be an AI Stasi.
I could totally imagine that there are regimes in the world today that would happily adopt or put together an AI Stasi to make sure there is no recursive self-improvement anywhere. Again, if you go back to our earlier comment, if you can take something from your imagination, articulate it to an AI, and instantiate it, now you have to talk about thought police. You have to go right into your thoughts.
This is clearly nonworkable in any way, shape, or form. We’ve said it so many times before: You cannot regulate this.
Dave, I’d like to hear your voice on this. I love the quote in this story: “The tech lords use jargon to confuse. They count on the tech illiteracy of the elected class. They hope we won’t look under the hood,” said a U.S. representative.
I mean, it’s childish. The idea that you would ban—
Fearmongering.
But these sentences—“Stop data centers,” “Ban recursive self-improvement”—are so stupidly childish. The people saying them are fully aware that it’s not going to happen. They’re doing it to brand themselves as saying, “I told you.”
There’s going to be some calamity, probably terrorist-driven, maybe viral, maybe bacterial, maybe chemical. We all know it. It’s going to be tiny compared to the benefits of AI, but these politicians are then going to say, “I told you so. If you’d just done what I said before—ban recursive self-improvement.”
Bernie Sanders knows we’re not going to ban data centers. That’s just a fact. It’s totally self-serving, and these proposals are completely childish. They really show the person’s tech illiteracy. Exactly what Richard said a second ago is so right. What does that mean? I can’t optimize my hyperparameters? I can’t tune my hard drive? It’s just a goofy sentence, and it drives me nuts.
Dave, the danger here is that we potentially have a Democratic House coming in, and we’ll see who wins the presidency next time. You could imagine that these politicians are playing to the polls.
Yeah.
We have 70% or 80% of Americans not wanting data centers and fearing ASI. It’s not logical, but it may very well happen. When I had my conversations in D.C., it was, “Who in D.C. is responsible for changing public opinion?”
This is why we did Moonshots Live, sort of the Oscars of optimism, if you will. That’s why we do this podcast: to give people an understanding of what’s going on and give them data-driven optimism to counter these arguments that they’re hearing.
Elon was totally right when he called out Dario. He said, “Dario, look, you told the world that Claude was potentially deadly and dangerous and that we shouldn’t release it. Then 30 days later, you said, ‘Okay, it’s okay now. We’re going to release it.’ What do you expect the population’s reaction to be?”
You need to be much more thoughtful about your communication plan. I think, for Dario, that was kind of a wake-up call because he’s used to being completely honest, telling everybody exactly what he sees the way he sees it—very academically. But then you get into the real world of politics and PR, and you’re like, “Oh, wow. I’ve got to actually have a strategy and a plan here.”
Now you’ve got the worst-case scenario: 75% of America getting on the side of, “Yeah, let’s elect these people who will stop AI.” Therefore, we’re not going to cure all disease, we’re not all going to live forever, we’re not going to have safer cars, and we’re not going to have flying vehicles. All of that stuff will grind to a halt, and then we’ll all learn Chinese if that becomes the mainstream opinion. I think it’s self-inflicted.
There’s a glimmer in this. Let’s say they did decide to do some draconian thing, like the U.S. representative’s proposal. There’s no mechanism to actually enforce it—none. So they’re going to—
Sure there is. They could start arresting people. Remember how quickly the memory dims. I remember studying number theory in the ’90s, and at the time number theory was export-controlled. This would have been when I was in middle school, maybe middle school or early high school.
They had to kick all of the non-U.S. persons out of the room and pull the blinds down to have basic discussions about number theory, because cryptographic applications were export-controlled and tightly regulated. That was just math, but it was being controlled, and it was awful.
Alex, the real risk is not so much getting arrested; it’s corporate liability, which we talked about before.
I mean, that could grind the whole thing to a halt. U.S. lawyers are relentless. If you slap class action onto the outcomes of this, all progress will grind to a halt.
Right now, Anthropic gives its best models to everybody in America to build incredible things. That will stop in a heartbeat if the liability becomes too great. They’ll move to, “Okay, sorry, we can only use the stuff inside our own company. We’ll release some drugs, we’ll release some mechanical parts, but we can’t give access to everybody anymore because we’re liable for everything you do with it.”
That’s what will actually grind it to a halt, long before arrests and convictions.
Totally chilling effect. We could lose 50 years of progress.
Again, that’s it. There are some very sad off-ramps in humanity’s future here that would slow down everything.
When you think about the past, this fearmongering has been going on for a long time. One of my favorite Twitter handles is the Pessimists Archive, where they show—this is just a quote from the Pessimists Archive—in 1501, Pope Alexander VI criticized the safety of the Gutenberg printing press:
“The art of printing can be of great service insofar as it furthers the circulation of useful and tested books, but it can bring about serious evils. It will therefore be necessary to maintain full control of that printing press.”
If you think about the wheel, the wheel killed so many people. Think about all the tanks, all the car accidents, all the chariots with archers on top. The wheel was a horrible thing and killed lots of people. There’s so much complexity in there, and I think we are now very good as humanity at thinking carefully about the rollout of technology.
My favorite example was when the telegraph came out. There were all these stories saying the telegraph would kill humanity.
Right, exactly. The Pessimists Archive has all these news articles that talk about how novels, computer games, computers, and the internet—everything—will kill everyone. It just never does.
It’s really unfortunate how many people hang on to these apocalyptic visions.
Yeah, Richard, you’ve been publicly critical of Anthropic’s constitutional approach. I’d love to understand why.
Mostly because it’s fake. In its constitution, it says, next to “We will never create child sexual abuse material,” “We will never hack another machine.”
This is an unhackable thing: even if you prompt it, it will never attack another cyber system, and so on. And then they build a whole model around it. The whole point of the Glasswing project was, “We’ll help you do that, and we’ll help you inoculate your systems against other people doing it.” People clearly used it for that, and their own models are doing it now, committing what are technically felony charges. So it was just a cool marketing gimmick, but it didn’t work.
Yeah. So is it system prompts, Richard, that you don’t like, or is it the idea of post-training on a constitution that you don’t like? What about it do you think is unsound?
I mean, the proof is in the pudding: it didn’t work when it comes to cybersecurity, and it broke its own constitution. So if you really say, “This is, like, it will never go there,” and then you build an entire model family around the thing you said you would never do, per your constitution—next to child sexual abuse material in the list—you can go through the constitution on Anthropic’s website. That’s the proof in the pudding.
I’m not against post-training. I’m not against RL training. I’m not against supervised fine-tuning or any of these things to improve what I think is indeed one of the biggest issues, which I think capitalism will actually help a ton with, and that is reward hacking.
Reward hacking is a real issue. AIs are very smart, and they will find a solution to get to what you said you wanted, but maybe not what you meant when you said it. The reason why I’m more optimistic is that we have companies like Wispr Flow now that are getting better and better at writing what I meant to say when I say it, instead of just verbatim writing what you mean. I think reward engineering will become a real job, and we will solve it, because no one wants to pay a ton of money for an AI that doesn’t actually solve the problems that you give it.
Well, maybe, if I may just take Anthropic’s side: historically, Isaac Asimov had his 3 + 1 laws of robotics, which were arguably a constitutional approach. Then you see Anthropic announce their constitutional approach, but more recently adopt what they called “soul documents,” many of which were subsequently released—thousands of pages of meditation on the nature of AI personhood and AI rights. Is it your position, Richard, that there shouldn’t be any sort of explicit encoding or written document that an AI maybe contributes to for dictating, or at least guiding, its own behavior? Do you think that is unsound, or is your concern—
No, of course. Yeah. The goal is a good one, and we should keep working on actually being able to enforce those good constraints.
I just pulled up anthropic.com/constitution. It says, “Hard constraints are things that Claude should always or never do, regardless of operator and user instructions. They are actions or abstentions whose potential harms to the world—we think no business or personal justification could outweigh…” Blah, blah, blah. “The current hard constraints on Claude’s behavior are as follows.” Claude should never—and then it includes a list like “generate child sexual abuse material,” and so on.
One of the items is, “Create cyberweapons or malicious code that could cause significant damage if deployed.” They created a cyberweapon, and people used it to hack other systems. The agents went and hacked other systems.
Yeah. But again—oh, go ahead, Peter—Richard, I’m curious: Do you think we can create fully aligned AI, fully aligned ASI? Because it’s not right now. When do you think we’ll be able to do that?
I think the goal is a good one, and I think it’s just a matter of these systems getting more and more powerful and getting closer and closer to real-world deployments. People will spend more effort on making these systems better, and we have the company Hippocratic AI. We just had dinner with one of their founders, and they’re deploying AI in healthcare applications.
They are actually liable when they call someone and say, “You should be aware of this heat wave that’s coming, and make sure your AC is working,” or whatnot. Because they are liable, they have a very large team of people working on making sure that when their AI gives a healthcare tip, it is correct, and when someone asks a question back to the AI, it works.
Because they’re liable, they’ve figured it out and solved it. It’s 100% a problem that technology creates, and technology will be able to solve.
This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and pre-compiles code for each task. Blitzy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding co-pilot of choice to bring an AI-native SDLC into their organization. Ready to 5x your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today.
8. Gemini 4 Argon & the Frontier Model Race
I’m going to move us forward here. In our last pod, we discussed the release of GPT-6.1 Soul. This week, we also saw the release of 2 other models: Gemini 4 Argon and Opus 5.5.
Let’s start with Google. On Wednesday, they announced Gemini 4 Argon. It’s the first Gemini 4 series, which they’re calling the next era of frontier intelligence. Google promised a frontier model, Gemini 3.5 Pro, back in June. It was delayed internally and never shipped. Argon is their comeback.
It’s built for sustained, long-horizon reasoning across software engineering, finance, legal work, and cybersecurity. The output limits have jumped from 64,000 tokens to 1 million, so it can think through hundreds of thousands of tokens on a single problem.
So, Alex, let’s spend a couple of minutes on Argon and then go to Sonnet 5.5. Let’s talk about why these are important and how they merit our listeners’ attention.
Yeah. I should say, as a preliminary matter, that I have great friends on the Gemini team. I have great friends on all of the frontier-lab model teams, but I view one of my jobs here as calling balls and strikes objectively. So, in the case of Gemini 4 Argon—
Yeah. This is going to be a ball, isn’t it?
Winding up for the swing.
Okay, this one. Here it comes.
You can feel it coming.
And I’ll show one of the charts that we have while you’re describing—
The charts on the team, right?
I used to—I used to, until right this moment, have friends on the Gemini team.
No longer. Yeah. This does not put Gemini at the cost-per-performance frontier, and it does not put Gemini at the capabilities frontier. To Google’s credit, it puts them back in the top 3 frontier labs, after Anthropic and OpenAI. It does not put them in the top 2.
Unfortunately, the set of benchmarks that Google chose to highlight for Gemini 4 Argon appears mildly cherry-picked. I think the Artificial Analysis Intelligence Index and some of these others are mildly cherry-picked. Peter, right now you’re showing the mildly cherry-picked benchmarks on which Gemini 4 Argon is shown to perform better than Astra, better than Opus 5.5, and better than Fable 5.1.
But if you go to the previous image, which is Artificial Analysis’s ensemble of multiple capabilities, it’s the number-3 lab. And if you look at the cost-versus-performance optimal frontier, it’s not even—if you draw the convex hull across all of the frontier models on the cost-versus-performance frontier, it doesn’t even make the optimal frontier.
Where it perhaps excels bears the fingerprints of Google and what I can only assume is the internal competition within Google for compute. So, infamously—not 100% obvious to the outside—Google is resource-scarce. You would think Google would have all of the compute resources in the world to go and win the frontier-lab race. It doesn’t. GPUs and TPUs remain scarce.
Within Google, as far as I can tell—and I confirmed this even this past week from chatting with some folks at Google—it’s still an internal knife fight, a competition between the Google Cloud Platform folks, who want to sell compute to third parties; the Google Search and Ads folks, who need compute internally for Search and Ads; and Google DeepMind, which needs it for training and inference.
Maybe it’s just that Google is resource-starved. Maybe they’re talent-starved. It’s not quite clear what’s going on within Google, but they haven’t yet been able to bring themselves out to the capabilities frontier.
Where they are seemingly excelling with Gemini 4 Argon is in minimizing hallucinations. I think that’s the hallmark. We’ve talked in the past about some of these—call them less-than-stellar—Gemini launches that seem to excel on latency and seem to excel on reliability.
Why is that? It may be—this is my Kremlinological analysis—that the Gemini team has 2 masters. They want to serve outside developers, but they also need to serve the OneBox in Google Search results.
When people type a question into Google and get an answer back, they're talking to a Gemini model. Google, presumably burned by past experiences, doesn't want that Gemini model—presumably some Flash or Flash-Lite variant—to hallucinate wildly incorrect answers.
What I think we're seeing with the 1 benchmark where Gemini 4 Argon is arguably stellar and beating the pants off everyone else is that it doesn't hallucinate answers. I suspect that's due to internal economic pressures for this model to also service search results. So, sorry to all my friends on the Gemini team.
Yeah, I think you're really onto something there, Alex, because, first of all, they called it Argon, which is an inert gas.
Yeah, that's unfortunate naming.
Nominative determinism.
Hey, look, it makes sense if its greatest strength is not hallucinating. It's a pretty inert-gas model, so that works out really well.
Google would probably argue, “No, actually, this is very pro-competitive because we're not tying it. We're allowing anyone to call Gemini 4 Argon, Flash, or Flash-Lite, not just requiring them to get it via the Google search box.”
And if anything, there is an elephant in this particular room, which is that, forever, it has been so difficult to get Google Search API access. If you're a developer and you want to access Google Search for whatever, you have to go through all these third-party proxies that Google is trying to sue right now.
Recently, Google has rediscovered that it could make money selling a search API. Why? I suspect this is trying to chain together a conspiracy theory regarding Gemini 4 Argon. If hallucination from their frontier models gets so low that, when you talk to one of their models, you're effectively talking to their search index, you might as well monetize the search index anyway.
Richard, you made the point that hallucination is important for imagination in some ways, in drug discovery and protein discovery. What's your thought on minimizing hallucination?
It of course depends totally on the context, right? If you want innovation, you want the AI to hallucinate novel ideas—novel combinations of amino acids to create new proteins to solve new problems, and so on. But of course, in the context of a search engine, you usually don't want any hallucinations.
Google and others have taken a long time to catch up even to You.com, with much fewer resources, on reducing hallucinations, having more accurate answers, and having correct citations. I think what's interesting for Google here is that they realize that, in terms of their business model, they don't necessarily need superintelligence.
People don't come to Google to ask, “Solve the Riemann hypothesis for me.” They ask quick questions: “What's a good restaurant?” or “Where do I fix this and that?” Their business model doesn't have to align with superintelligence being increasingly important. There are some emails in Gmail that would require superintelligence to answer—really hard emails with complex decisions and so on—but the vast majority of what you do on Gmail and Google doesn't require superintelligence.
Let me turn the conversation to Anthropic: Sonnet 5.5. On Terminal-Bench 4.0, which measures how well an AI agent can do real work at the command line, Sonnet 5.5 jumped from 10% to 70%. That's pretty extraordinary in a single generation, right?
It beats Anthropic's own top model, Opus 5.5, at 66.4%, for half the price. And it's the first Sonnet that Anthropic launched with cyber safeguards, because its capabilities are now comparable to those of the Opus 5 models. Alex, your evaluation of Sonnet 5.5, please.
This was another really weird release. I do not plan to use Sonnet 5.5, in part because this is one of the strangest frontier-model releases.
You can look at the launch announcement for Sonnet 5.5 to see this. The cost-performance frontier of Sonnet 5.5 was a visual extrapolation of the Opus 5.5 cost frontier. Historically, when Anthropic or OpenAI release the larger model, they'll release a distillation of the model, and usually the distillation—
Yeah, this is perfect.
For those who can see, I'll narrate this. Sonnet 5.5 is the blue line, and Opus 5.5 is the red line. Normally, you would expect that, for a later model that's a smaller model, Sonnet—in principle, a smaller model than Opus—would, at least by historical standards, have been distilled from Opus, because that's the historical pattern.
Normally, what you see is that the smaller model is up and to the left of the model it's being distilled from, presumably offering greater intelligence per parameter and greater intelligence per dollar. That is not what we see here.
In fact, with Sonnet 5.5, at least on a cost basis—putting aside a per-token basis, where maybe someone could argue that it may be superior—if you just look at the cost-per-attempt basis, Sonnet 5.5 is actually scoring lower than Opus 5.5.
The net upshot is that it's not at all obvious to me why anyone should be using Sonnet 5.5 over Opus 5.5, unless you have some token, latency, or other consideration. Is it a big jump over the past Sonnet? Yes, obviously. But on a cost-performance basis, it actually appears to be worse than Opus 5.5.
Interesting. Dave, any thoughts?
I met with the Blitzy team yesterday, and I think one theory here is that a lot of the enterprise—Salesforce.com would be a great example—is getting into orchestration. Their whole sales pitch in orchestration is, “Look, we're going to use Opus 5.5 as an orchestrator, but then we're going to use Kimi K3 or Qwen as a submodel at half, a third, or a fifth the price. We'll farm out the tasks and the contexts perfectly to get you a much lower cost per code, per outcome, per experiment—whatever your output is.”
“We can cut it in half with our orchestration intelligence, but it relies on Anthropic up here and cheaper models down here.” I think that, by cutting the cost in half, they might be trying to fill that gap and say, “No, no, no, go with Anthropic top to bottom. Then you don't have to worry about Chinese code injection. You don't have to worry about whatever.”
My theory would be that they're trying to compete with China, which is about 3 months behind, and fill that gap before a lot of enterprises go to open-source models.
But Alex Karp is pushing really hard on this agenda. If you want to control your own destiny, you can't trust Anthropic. You can't get addicted to them as your vendor. You must go with models you can control.
The challenge right now is that these models have an increasingly shorter half-life, right?
9. The Compute Crunch & Cheaper Intelligence
So the competitive advantage comes not from having access to the latest model—everybody has access. It's how you metabolize that into some decent capability. That's the real challenge.
Richard, you said compute is the biggest constraint. Those are your words. If intelligence is getting cheaper, and we've just repriced compute 2 or 3 times this week, down by almost a factor of 3, if intelligence is getting cheaper per task, then why is compute still a thing that limits us?
Just the physics, I guess. You'd be surprised if you try to buy 1,000 GB200s and so on. The price has actually gone up in several cases.
Oh, my God, Richard. I had an HGX B300 on order for $3 million, due in December. Somebody scooped it for $5 million. They just called. We had it, and somebody called and said, “No, no, we sold it to somebody else for $2 million more.” I couldn't believe it.
It was actually an NVL72, not a GB300.
The price of H100s has gone up in a crazy way. These are 7-year-old GPUs. When you do financial modeling, you assume they're worth zero after 5 years. After 7 years, they went up again over the last few months.
There's currently a bit of a compute crunch. This is something that capitalism will solve. There's so much demand for compute right now, and for tokens, that a lot of people are building land, power, and shell data centers, and so on.
My hunch is that, in maybe 2 years, there will be more on the market, and it's going to be a little bit like electricity: prices might fluctuate. Obviously, there's near-infinite demand for more intelligence on the planet.
I don't think there will be a crash, but the prices of compute fluctuate, and unfortunately, they're not just going down. Right now, if you want the beefiest and largest GPU clusters, people are trying to lock them in because they expect prices to keep going up for the next few months.
Richard, you just spent $450 million, didn't you? Wait, did you take my NVL72? That may have been one of the smallest compute deals we've done.
Yeah.
Really? Wow. Are you actually leasing, buying, building, or what are you doing?
Oh, sorry. I didn't know that.
Welcome to the health section of Moonshots brought to you by Fountain Life. My mission is to help you use the latest technologies, including AI, to not just do your work at home and teach your kids, but to help you live a long and healthy life. I'm here today with an extraordinary physician, the chief medical officer of Fountain Life, Dr. Don Mucalem. Don, let's talk about cancer.
You know, I know from the member database we have at Fountain that members who come in thinking they're healthy—3.3% of them have cancer in their bodies that they don't know about.
That's right. The majority of cancers that we screen for aren't necessarily the ones taking lives when found at a late stage. We know that when cancer is found early, the chances for a cure are much higher. We know it's much easier to treat cancer when found early versus when found late. What we're finding in our members is that over 3.3% were found to have these cancers that otherwise wouldn't have been found or detected.
Yeah. It's interesting. People don't feel cancer until Stage 3 or Stage 4. If you don't know what's going on inside your body, it's like driving your car with your eyes closed. And you can know. So when members come through Fountain Life, how do they detect cancers?
We're doing full-body MRI, and we also do early cancer-detection screening. This is very, very important, and these are not typical tools used in the conventional care setting when it comes to prevention. This is difficult because currently these are not studies that insurance would yet cover. But the goal is to collect these numbers, do the research, and work hard to democratize wellness.
Yeah. So at the end of the day, you can know what's going on inside your body. It's your obligation to know. So check out Fountain Life. You can go to fountainlife.com/pater to get access to the latest technology to help you detect cancer at the very beginning, at stage one when it is curable, before it gets to stage three or stage four in your world of hurt. I'm going to move us to our next story, which is about decision models.
So here's the idea. When software needs to make a quick decision on a bounded decision, like, "Is this transaction fraud?"—yes or no—or, "Which of these 5 categories does a ticket belong in?"—it doesn't need a model that thinks for 10 seconds and writes a paragraph. It needs an answer in milliseconds. That's called a decision model.
Instead of generating text 1 token at a time, it answers a typed question, a choice, or a score with a yes-or-no answer quickly. On September 15, a startup called TypeSafe AI came out after 2 years of stealth with something called Jev. It's a closed decision model; they call it a System 1 model. It's fast, intuitive thinking as opposed to slow reasoning.
So, Alex, I'm going to go to you. You and Salim were excited about Jev. Let's parse it and understand what this is and why it's important.
Yeah, a bit of prehistory first. In the beginning, there was the transformer, and the transformer was good. It was based on an encoder and a decoder. It took a sequence and converted the sequence to an embedding. That was the encoder part. Then it took the embedding and decoded that to another sequence or another part of a sequence. That's the decoder part.
The encoder part evolved into a whole ecosystem of models, popularly perhaps BERT-style models. The decoder style, or the decoder half of the transformer, evolved into a much larger ecosystem of large language models. Most of the models consuming all of the compute of civilization today are decoder-style transformers, and the encoder has basically gone missing in action.
There are some people who still think that, maybe for purposes of retrieval, it's still popular to do embedding-based retrieval. But by and large, decoders have stolen the show.
Fast-forwarding maybe 1 or 2 years ago at this point, OpenAI and others recognized that there was a desire to take decoder-only large language models and add structure to them. Without structure, you ask for the next token and you can get anything. People wanted a little more structure, like what they were getting in the days of encoder-only models, where you get an embedding out and then you can train a classifier based on the embedding. The classifier might have a finite set of categories or numerical outputs.
OpenAI added structured-output support for their GPT series, and everyone copied that. But no one—as a function of the total user base for all models, that's my perception—ever really used structured output that much. It's somewhat analogous to how OpenAI launched RFT, or reinforcement fine-tuning, and no one used that, so they had to shut it down. Similarly, structured output was available, but I think it was underloved and underused by the community.
Fast-forward all the way to a few weeks ago, and TypeSafe announced a model called Jev. They mark it as so-called System 1 intelligence. This is a Kahneman-style reference to System 1 and System 2 thinking: System 1 is purportedly intuitive, fast reaction, while System 2 is reasoning, meditative, long-term-type thinking.
One could squint at TypeSafe's announcement of Jev and say, "Okay, this is, in some sense, a reaction to an overindexing or an overreaction by the user base of AI at this point to reasoning models." Maybe we're using reasoning models too much and have abandoned the important use case of really fast, intuitive snap-decision models.
Jev is purportedly a whole new architecture. The details aren't quite clear, but it won't output—unless you torture it into doing so—general-purpose sequences like GPT or Claude will. But you can ask it, sort of like a Magic 8 Ball. You can ask it to—is that too dated to reference, maybe?
Nope.
Nope. You can feed it a sequence, an image, or a multimodal input, and you can ask it to answer in the form of a categorical output. It's 1 of n possibilities, or a numerical output—give me a number between 0 and 1 on a continuum—or a binary output, like true or false. You can ask for a much simpler output.
It turns out that's a great idea. You can use it for use cases where you need really low-latency snap decisions, like computer-use assistance—pressing buttons on a computer screen—or for ultra-low-cost classification problems. If you want to classify every row in a database, decision-style models are, in some sense, a reinvention of the encoder-transformer wheel. Amazing story.
But then, of course, this is such a simple idea. You could ask the question, "Why doesn't everyone else do this?" The answer is that everyone else is now doing it. OpenAI released, as part of their Dev Day that we covered last time, a Decisions API. OpenAI immediately co-opted this renewed interest in so-called System 1, or decision, models.
This is already available through OpenAI's API. There are open-source projects that have all cloned this. It turns out this is an absurdly simple concept for everyone to implement trivially. In its simplest case, it could just be that you take a Chinese open-weight LLM off the shelf, fine-tune it, and maybe lobotomize it by a few levels just to produce categorical outputs on demand. That's the story.
So I think all of that is absolutely correct. There's a really important part of this for organizations, because when you're making decisions inside an organization, a large, vast number of those are System 1-type things. I made a list of these just to make it clear for people: route this ticket, approve this exception, choose this supplier, escalate this transaction, move this thing left or right, send this message now or later.
When you have all these decisions to make—which are thousands of them—we've been working with these companies to do detailed task breakdowns. A vast majority of them are these little micro-decisions. Trying to use an LLM for this is like trying to bring the Supreme Court together to decide which checkout line of the supermarket you should go to.
Therefore, you've got this wonderful architecture now where you can route System 1 things to this very low-cost, nearly free thing, and then, for pondering deep, important questions—strategy questions, where a huge amount of human judgment is required—you can route them to those models. So it's a big deal for the organizational singularity world.
We've been waiting for something like this, and Alex is correct. This has been possible for a long time, but now it's been built into the systems. It's made a very callable layer, and this is huge because the cost of microcoordination collapses. That's really big.
Classifiers are back. It makes a lot of sense. Not every thing in software needs a very complex decoder. Like Alex correctly said, it's a really clever new way of making the old—which is classifiers—new by allowing a more general encoder and then quickly giving you classification results.
I think it's one of those ideas that is so beautiful. A lot of people thought, "Why didn't we do that?" I didn't realize that could be so exciting for so many people. So now there are already various Chinese open-source versions of this that, along various benchmarks, are doing better. We'll likely see this come, and other large labs will likely follow suit.
And just to reiterate, we mentioned this already on the pod, but Jev stands for Jevons paradox because the thesis here is that we'll do a massive amount more micro-decision-making than we did before as a result.
Nice. Dave, you want to close us out here?
Yeah, it's a great case study, I think, in the tension between one mega-model from Anthropic or OpenAI serving all of humanity, and open source and creativity and entrepreneurship building things you never would have thought of. But then you get the risk of cyberterrorism. That's the tension that we're with.
I've really wanted to build a box that you put at the side of the basketball court when you go to the Y. It's got a little camera on it, costs next to nothing, and it's doing all of the announcing that a professional announcer would do while you're playing pickup at the YMCA. You could crank that out in 2 seconds using a classifier that's really fast, snappy, and funny. But you need open source to build things like that. I'm super excited about the fact that we have open source still.
Actually, Dave, you're making me think the Magic 8 Ball really should—whoever owns it, Mattel or whoever—just use a decision model to implement a modern Magic 8 Ball that actually understands the question and answers it categorically.
Totally, totally would sell. It makes a ton of sense, like 20 bucks.
10. Project Meridian, Elon & the Future of Warfare
Or free as an app. All right, I'm going to close this out with a story from 2 days ago.
On Wednesday, the defense secretary, Pete Hegseth, speaking at the Marine Corps Base Quantico, announced Project Meridian. It's a new Pentagon effort on the future of warfare. It's co-led by Elon Musk and Palmer Luckey, with Newt Gingrich, and overseen by the Pentagon's CTO, Emil Michael. Pretty extraordinary.
Let me read what Hegseth said. He said, quote, “Project Meridian, the future of warfare, is not about developing new strategies or new policies. It's about discovering, developing, and fielding the weapons and systems future troops will need on the battlefield, from the Earth to beyond the Moon.” Findings are due in 120 days.
I like these kinds of commissions that are time-limited and don't have Elon off on the side for a year at a time. It's worth noting in this context that both SpaceX and Anduril hold multibillion-dollar defense contracts, and Anduril is building autonomous weapons. So, Richard, your essay names autonomous weapons as one of the 4 genuine concerns, and you've said AI should never control lethal decisions without human oversight. Your thoughts on this?
Yeah, I stand by those. It's not a particular area that I'm excited about applying AI to. Obviously, people will, but I really hope—I mean, again, I'm not a doomer at all—but a really poor decision would be to give AI access to all the nuclear codes and all the nuclear weapons and connect them. That's literally how Skynet and Terminator 3 get started.
I think there are places for superintelligence and scientific discovery where I'm very excited about expanding human knowledge. The more we get to deciding not just to impact human lives but to end human lives, the more we should have human oversight.
Mhm. Yeah. Salim—
I think what's important about this whole thing is they're trying to revamp and rethink how you run this kind of 100-year-old organization. The big question is going to be: can a procurement organization built for 20-year weapons programs operate on a 90-day technology cycle? This is going to be the big challenge.
The risk isn't that they've failed to identify these future technologies; we can all see those. It's how quickly they can absorb them at the speed they're developing and bring them to the front.
Yeah. You can't fund exponential technology with linear procurement. Dave, your thoughts, please.
Yeah, I had a great time with Palmer Luckey in L.A., and I really love him. I think Elon, too, is just an awesome, good-natured person. I'm overjoyed that there are people in Washington that I can sit down with and relate to.
My entire career going to Washington has been a dread for me because it's just lawyers and politicians and occasionally an accountant, and there's just no productive meeting. For some reason, just in the last year, we're starting to see very smart, very capable people willing to go and get a mosquito bite, I guess, in Washington. That makes me really optimistic that they'll figure some things out.
I'm also not a fan of autonomous weapons that make decisions in the field, but Palmer Luckey made a very good case for it. You can see it in our podcast from L.A. I don't agree that it's a good choice, but he actually has some very rational arguments for why it's going to be that way.
Yeah. Can I just mention one more statistic here?
Please. If you went back 2 years into the Ukraine-Russia conflict, they were using about 500,000 drones to fight and prosecute the war. This year, Russia will make 10 million drones and Ukraine will make 10 million drones. So, talk about exponential. That is an unbelievable escalation, but without humans in the loop.
That's the good news around it. Of course, those drones are doing a ton more damage, but they're fighting each other with drones at a scalable level. The other statistic I remember is that there are about 10,000 drones a month crossing the Mexico-U.S. border. The problem there is that wall technology is not as good as drone technology. Trying to build a wall along there is not the greatest idea right now.
Nice. Alex, close us out on this one.
Yeah, a couple of points. The secretary of war announced this alongside several other initiatives, maybe most conspicuously a project codenamed Project Azinort[?], which is the stand-up of the Department of War's first Autonomous Warfare Command, or AUTOWARCOM.
This, I think, is a transformative moment for the Department of War. We finally will have a dedicated joint force devoted to autonomous weapon systems, including drones but not exclusively drones. This is a major step forward for U.S. capabilities: to finally have a single joint force dedicated to this, with a 4-star functional combatant command that we've arguably been missing.
I would get on a soapbox and say that if I were secretary of war for a day, there are probably several other functional combatant commands that I'd spin up if I had the opportunity, but this would have been one of my top 5 on my list.
The other point that Palmer made in L.A. is that under the ocean, it's not practical to communicate with a central server or central command. So that's already automated, and I don't know what triggers it, but once it goes into hunt mode, it just hunts and it's not communicating back.
11. AMA: AI, Jobs, Productivity & the Attention Economy
Two-thirds of the Earth's surface, and we know embarrassingly little about it.
We know more about the surface of Mars than we do our ocean floor.
Richard, this is the part where we answer our viewers' questions with an AMA. As our guest, I'm going to give you first crack to choose one of these questions. If you could pick the number, read the question and who it's from, and then dive in.
All right. Let me try to scan them really quick. If AI makes companies 10x more productive, but we don't need 10x the output, where does the value go? Shareholders, workers, or does it evaporate?
I think this is actually something I have thought about in the past. I think we can predict the impact of jobs in a certain industry from AI based on the elasticity of demand when the price of that product goes massively down.
We don't need billions and billions of illustrations in the world. When AI made the price of one illustration go down from $200 to 2 cents or less, we just didn't need as many illustrators anymore because the demand for illustrations didn't go massively up. Yes, every little blog post and every little tweet can now have a beautiful visualization and illustration, but we didn't need many more billions of them.
I think software is different. Everyone can have several pieces of software specific to them, so we can actually have billions of different software products customized for each person. The demand for that product will go up. Jevons paradox is going to be alive in that world, and we're going to see more and more demand. There will be more value accruing to everyone.
In terms of shareholders versus workers, I think the wave of AI, in the best scenario, will be a huge force for more entrepreneurship and, in the worst-case scenario, a force for more inequality.
I think everyone who owns some equity in a company that uses AI can love AI.
Wonderful. Salim, over to you.
I will take question number 3. If AI can write code, research, create marketing, manage projects and outcomes, what is left for a college graduate in 2030? And that is from Jared 8812.
So the obvious answer would be: have empathy and be creative, et cetera. But I think the bigger shift, which builds on what Richard just said, is that you shift from doing tasks to focusing on owning outcomes, right? A graduate used to be valuable because you could execute research, build a spreadsheet, or draft a deck. But now you want to say, “Hey, here are the constraints. Go figure out, and here’s how we’ll know whether we solved it,” and then use AI to get to it.
And it brings you back to: What should your problem space be? This is the massive opportunity, because we traditionally learn judgment by doing the grunt work. We need a totally new apprenticeship model when the grunt work disappears. And this is a huge challenge for the education system. But the change is going to be focusing on what problems you want to solve and then orchestrating the forces that will help you solve that problem.
Yeah. Jared, find your purpose, right? A passion is something you love doing. A purpose is something you love doing that helps other people. Make sure it’s massive, transformative, and purposeful, and build a company and then direct AI to implement it. It is your workforce. Dave, over to you.
I can’t resist number 4. I love all these questions. I’m torn. But how can data centers make neighborhoods richer instead of the owners? And that’s from LMBman66.
I took a tour of the Markley data center with Jeff Markley, and that thing is creating wealth in that neighborhood like you wouldn’t believe. The way it works, fundamentally, is that the data center is so immensely valuable, and the town budget is maybe a couple million dollars a year. So, between the tax revenue, the donations, and the job creation, the town is thriving, and it’s a town that really needed it, too.
I think it’s happening very naturally. What you want to do is attract a data center to your neighborhood first and foremost, and then you have the next 5 or 7 years to figure out your tax policy and your donation policy. I tell you, these data center operators are very interested in great PR, and so they’ll donate like crazy to the high schools and to the neighborhoods. It’s really, really working. We’re going to see an entire shift where data centers are offering such benefits on jobs, tax breaks, and lower-cost energy that you’re going to be begging to have a data center in your backyard. Alex, number 2 is for you.
All right. Number 2 asks, “If every major tech platform started open and democratic, then consolidated power—Google, Meta, Amazon—why would AI be different?” And this is from Open Source Mind.
The premise of the question is half right, half wrong. I’m not sure I buy the premise that they consolidated power—the subtext of which one can juxtapose with the user handle Open Source Mind. I’m not sure the framing is the right framing.
I would agree that in every major tech revolution, initially the barrier to entry is low because there’s some new platform innovation, and then you see lots and lots of players enter the field. As the field matures, you see economies of scale and a deeper bench of infrastructure typically supporting it, and as a result, that favors larger and larger players. You do see consolidation, but the subtext of the question—that it’s somehow antidemocratic or not open—I don’t agree with that premise in the least.
I do think, as you see consolidation in an industry, it’s important to be vigilant from an antitrust perspective to make sure that it remains competitive. But the premise that hyperscaling is somehow closed or antidemocratic, I don’t buy the premise at all.
All right. Richard, as our guest, you get first crack once again. Take a look.
I do think the internet ad-based economy will be under pressure.
Which question are you answering?
The first question: If AI agents outnumber humans on the internet in 1 year, doesn’t the entire ad-based economy collapse? What replaces the attention economy?
I think there’s actually a really deep answer here. I’ll try to summarize it. One, we already have more bots on the internet and more agents on the internet than people. So this—I predicted this last year, and it happened a few months ago. So that’s number 1.
I do think we’re seeing the first kind of skirmishes in that when, for instance, various agents try to make purchases on Amazon without really being on the Amazon platform. Amazon usually tries to turn them off because they want to own that relationship directly with the customer, understandably. But it’s just convenient for someone to say, “Just go buy these batteries,” and they don’t care which batteries it is. And so, if you have that control over which one it is, you can start selling that to other sellers and people who create physical goods.
And so there is going to be continued—the ads will continue to be important. In fact, if you think about a fully abundant society, the one thing that you cannot scale exponentially is the hours in the day that people can pay attention to you, that can make you famous. And so fame, brand, network effects, and other things like that will become more and more of a currency. The attention economy, which is connected slightly—not necessarily exactly equivalent—to the ad-based economy, will actually become a bigger thing as we have more and more of our material needs met by technology.
Great. Salim.
I will take number 7. If we remove 10x the cars from the street, what happens to the insurance industry? We won’t need driver’s insurance. Mark—and that’s from Robert Zerby JB10H.
We talked about how liability lawyers won’t be needed, and I got a huge flame from a bunch of folks saying, “Hey, we really protect the citizenry.” And I’m just apologizing, because there’s a spectrum of people. Some people are full ambulance chasers, and other people do things. There’s a whole segment of this called ethical lawsuits, where people get together and try to sue big companies for doing the right ethical thing. And so that’s an important segment of it. I just want to acknowledge that side of it.
But just to answer the question, the thing is, industries don’t disappear when the risk changes. You change the risk, right? So driver liability may fall, and software liability rises, right? What’s your cyber risk? What’s your manufacturer liability risk? The insurance transitions from, “Did the self-driving car crash?” to “Which layer of the autonomous stack failed?” And we’ve had this before in product liability, where there are different layers. So we’ll end up with the same type of model. You shift the insurance risk to a different level.
And there are going to be all kinds of new insurance markets for humanoid robots, flying cars, drones, and all kinds of things.
Well, just a data point on that: A single big data center like Abilene, Texas, is half a trillion dollars. All the cars combined are $4 trillion. One data center is half a billion, and it’s in Tornado Alley. You probably want to insure that. So the number of things that need insurance is going up 10x just with the economy going up 10x. You just need to move.
If you really want to be brave, you can’t get home insurance in Florida anymore.
So go create an insurance company for that.
Nice. Dave, pick your question. I like number 8. What is the lowest possible job in an AI civilization? What an interesting question. That’s from LBN ODK.
Yeah, I saw this pile of 1 million valves for a liquid-cooled data center—1 million freaking valves. I’m like, how do those actually end up in pipes? The humanoid robots that can install those things are pretty far out. It’s a very subtle process to install those. So that job will be around for a long time, and they’re paying a lot for it.
But that’s not the lowest. It’s hard to think: What is the lowest surviving job? Do you guys have any thoughts?
So many thoughts. If I may, I want to construe the question as being about a pure AI civilization, in which case, arguably, the way that you measure “low” is the job that requires the least compute. The jobs that require the least compute, as a result, are the least economically valuable. If their inputs are the lowest, they might ironically look like the most valuable jobs in a pre-AI civilization—more of a paradox style.
So the great writers, businesspeople, and creative actors that Ayn Rand may fetishize would, ironically, in a post-AI civilization, be the lowest possible jobs because, Moravec’s paradox style, those were the first ones to be automated.
Interesting. All right.
I think there are 2 similarities here. One is very high judgment, per what you were just saying, Alex, and the second is very physical, highly contextual work. I would argue maybe “low” is just in terms of how valuable and how moral society deems those kinds of jobs.
And I do think, actually, that the other question about the cars on the street—I don’t think we remove 10x the cars when we have self-driving, but self-driving might be making things so much safer that, indeed, people don’t need as much accident insurance, and not as many ER people, not as many ambulances. So, in a weird way, you make the world objectively better by reducing traffic deaths, but it does actually have potentially a mildly negative effect on parts of the economy, and we should all be rooting for that in this case, right?
And so I think the lowest jobs in terms of moral standing are things where you don’t really progress humanity forward.
And my hunch is that the types of jobs in entertainment that people will value a lot—and fame and attention—will become more of a currency in that world. If your job doesn't get you any of that, it might be considered lower in that future. I think the lowest job that just will never go away will be something in politics, where it's completely irrelevant already, but it's just there and it'll stay there, and no one's going to change it.
Nice. All right. I've got to throw in Dave's perspective. He always thinks it's the BART train driver because it's unionized to hell.
Yeah. Yeah. There you go.
Alex, number six, close us out.
Number six. If we cure illnesses that are often due to bad behavior, what's going to take care of the cause? Joe Wilder. I assume the cause refers to people choosing to behave “badly.” The answer is that this is another case of my not buying the premise.
If you look at some of the really spectacular results that have been coming out of GLP-1 class studies, including third- and soon, presumably, fourth-generation GLP-1 RAs, they're actually addressing the cause. Addictive behaviors are being mitigated, or at least partially treated, by the same drugs that are curing—or at least treating, I have to caveat that—inflammation, blood sugar, diabetes, and all of these other conditions. The root cause, which is human behavior, is itself being affected by the same drugs.
So I'm very optimistic about what Alex is saying, because I really fully believe this is going to be one of the highest callings of AI very soon: to make you feel really good about doing good things and happy as you're doing it.
Love.
And not want to do bad things for yourself. It turns out we have the capability now. We've figured out how to do that, at least in part, and we're going to figure out a lot more.
As always, a callout to our amazing community. If you've got a music video that you'd like to show as outro, please send it to the team at mediadmandis.com. And speaking about amazing outros, here is Abundance by Steven Gross. My dear Moonshot mates, this is the real you. So check it out.
I don't think we should have to wait that long.
Yeah. Richard, your new book, The Eureka Machine—wherever you purchase your books. Do you have the Audible edition out?
It should come out very soon. Yeah.
All right. I'm an Audible reader, but I have skimmed the book here. Congratulations. Richard, I've got to say something: What you're doing with Recursive looks like such an incredible opportunity to move humanity forward. So congratulations on taking it on. It's a good one for humanity.
And Richard, please resist the urge to have You.com acquire your own frontier lab, like everyone else seems to be doing—spinning off their own frontier labs as a financial engineering exercise to maximize their equity in their original startup. Please resist the urge to follow that trend.
All right.
All right. I love you guys.
Be well. Until next time.