[BidClub_]
Lex Fridman Podcast · · 265 min

State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490

Lex FridmanNathan LambertSebastian Raschka

YouTube ↗
TL;DR

  • The 2026 AI race is not winner-take-all: ideas move freely across labs, while compute budgets, hardware access, organizational culture and distribution decide who captures value. Sebastian Raschka sees DeepSeek winning open-weight “hearts,” but not permanently owning the technology; Nathan Lambert sees Anthropic’s code-first discipline, while Sebastian highlights Google’s integrated stack and OpenAI’s ability to land new paradigms as distinct advantages. China’s expanding field—DeepSeek, Qwen, Kimi, MiniMax and Z.ai—makes continual leapfrogging more likely than durable technical supremacy.

  • Scaling laws still work, but their economics increasingly favor a portfolio of pre-training, post-training and inference-time compute rather than simply building the largest base model. Nathan contrasts roughly $1 million-$10 million open-model training runs with recurring serving bills that can reach billions, while 2026’s gigawatt-scale Blackwell clusters could support larger models, longer RL runs and premium inference. His provocative commercialization marker: after $200 plans, “we’ll see a $2,000 subscription this year” if marginal intelligence proves valuable enough.

  • Coding is the clearest near-term monetization wedge because RLVR-trained models can reason, call tools and iterate against verifiable outcomes. Claude Code’s advantage appears to be more than Claude Opus 4.5 alone: the interface and agent harness let users operate in English at the system-design level, while Cursor, Codeium and conventional IDEs retain value when developers want tighter control. The trajectory is toward “the industrialization of software,” but production complexity, specification and safety-critical systems keep humans in the loop.

  • Open weights are becoming strategic infrastructure, with Chinese providers using permissive releases to win global influence even where US enterprises will not buy Chinese APIs. Chinese models can be hosted domestically, customized on private data and served using the customer’s compute; OpenAI similarly framed gpt-oss-120b as distribution that uses “your GPUs.” Nathan expects more open-model builders in 2026 than 2025 and argues the US needs roughly $100 million-class efforts to avoid ceding the research substrate to “Qwen, Qwen, Qwen, Qwen.”

  • The durable moats sit below and above model weights: proprietary data, serving infrastructure, trusted interfaces, tool integrations and hardware ecosystems. Sebastian says Google can avoid NVIDIA’s margin through TPUs and control its stack; NVIDIA’s two-decade CUDA ecosystem remains harder to displace than any individual chip; Anthropic owns coding mindshare; and ChatGPT benefits from brand, memory and habit. Closed US models remain better enough that the speakers pay for them, while open Chinese models compete on cost, licensing and customizability.

  • Data quality and verifiable post-training now matter more than architectural novelty, because frontier models remain recognizably descended from GPT-2. Mixture of Experts, attention variants, lower precision and better systems raise efficiency, but capability unlocks come from curated reasoning data, RLVR and tool use; Sebastian summarizes pre-training as absorbing knowledge and post-training as learning skills. The hard liabilities are legal provenance, benchmark contamination and preference averaging: RLHF can make a model broadly pleasant while sanding off the “voice” and incisiveness users value.

  • AGI timelines remain less decision-useful than concrete capability thresholds: reliable computer use, autonomous feature delivery, scientific specialization and measurable economic impact. Nathan expects AI to remain “jagged”—already superhuman at some code, weak at distributed ML and messy research—while Lex presses the plateau case of “Clippy on steroids.” The most credible upside may be quieter: personalized access to human knowledge, domain models built on private data and steadily more capable agents, rather than one sudden remote-worker or singularity threshold.

Digest · the substance, structured for research

1. Open weights made the international AI race plural

  • Lex anchors the discussion in January 2025’s DeepSeek R1 release: near-state-of-the-art performance, allegedly using much less compute at much lower cost. A year later, research and product competition feel less like a single “DeepSeek moment” than a continuously accelerating release cycle.

  • Sebastian’s answer to “who is winning?” begins by refusing the premise. DeepSeek is “winning the hearts of the people who work on open-weight models,” but researchers rotate among labs, so no company in 2026 should possess technology unavailable elsewhere; budgets and hardware, not permanent ownership of ideas, become the differentiators.

  • Nathan sees culture shaping the otherwise fluid movement of ideas. Anthropic’s hard bet on code is paying off through Claude Code, and the company presents as “the least chaotic”; that operational coherence may matter when model development is bottlenecked by coordinated human effort rather than one secret algorithm.

  • Lex’s corrective to the X discourse matters: Claude Opus 4.5 may be the coding community’s darling, yet ChatGPT and Gemini address a vastly larger population solving everyday problems. Online hype can indicate a valuable wedge without measuring the platform’s actual reach.

2. China’s open-model boom is a distribution strategy

  • Nathan argues DeepSeek catalyzed China much as ChatGPT catalyzed US chatbots. Z.ai’s GLM models, MiniMax and Kimi from Moonshot AI now release frontier open weights, creating a field in which DeepSeek may lose its symbolic crown even while remaining technically strong.

  • Sebastian’s pushback preserves the nuance: DeepSeek did not deteriorate; competitors adopted its ideas and released newer models. Kimi uses a similar architecture, and the result is leapfrogging—“the most recent model is probably always the best model”—rather than evidence that one organization permanently passed another.

  • Chinese providers know many top US companies will not subscribe to a Chinese API for security reasons. Open weights let them influence a growing US expenditure market anyway, while international uptake gives policymakers reason to support releases; Nathan therefore expects more open-model builders in 2026, followed only later by consolidation.

3. Different incentives will shape which Chinese labs endure

  • Nathan notes that DeepSeek is unusually secretive in communication but open in its technical reports. Its connection to High-Flyer Capital means outsiders do not know exactly what it uses the models for or how much it prioritizes broader model monetization or influence, giving it a different objective function from venture-backed startups.

  • MiniMax and Z.ai have filed IPO paperwork and actively seek Western mindshare. That outreach may affect cadence and presentation even if it does not alter the underlying model-development playbook.

  • The business-model constraint remains severe: training frontier models is expensive, while consumers in China and many other markets historically pay less for software. Open releases can win usage and legitimacy before a durable revenue model exists, but they cannot abolish the eventual need to fund research and inference.

4. Google, OpenAI and Anthropic are winning different layers

  • Sebastian frames the consumer contest as whether one is willing to bet on Gemini over incumbent ChatGPT. Gemini carried the momentum of 2025 after Google recovered from Bard-era weakness, but OpenAI repeatedly looks chaotic and still “lands things,” making displacement harder than benchmark charts imply.

  • GPT-5 produced mixed reactions for Sebastian, yet its routing system may have been economically excellent: most users can be sent to cheaper inference rather than consuming maximum GPU capacity. The public-facing product question is not merely who has the smartest model, but how often ordinary users will pay the latency and compute cost for that intelligence.

  • Sebastian’s 2026 call is that Gemini continues gaining on ChatGPT because Google can separate research from product, operate at enormous scale and own more of the infrastructure stack. Anthropic should continue succeeding in enterprise software, where its code positioning and organizational focus are already established.

  • OpenAI’s counterweight is repeated category creation: Deep Research, Sora and o1-style thinking models are cited as definitional products or research ideas. Sebastian expects much of 2026 to emphasize scale and optimization, but says a new paradigm is still most likely to come from OpenAI.

5. Google’s TPU stack converts integration into margin

  • Sebastian’s infrastructure thesis is blunt: NVIDIA’s chip margin is “insane,” while Google can design hardware, software and data centers together without paying that external margin. Its long head start matters because power contracts, facilities and supply chains have multi-year lead times.

  • Google Cloud still competes against Azure and AWS at a different layer from the Gemini brand. That makes its advantage harder to narrate than a model leaderboard, but potentially more durable if inference becomes the industry’s dominant recurring expense.

  • The caveat is that integrated infrastructure does not guarantee a breakthrough model. It mainly makes large-scale experimentation, training and service cheaper, while OpenAI’s demonstrated research-product reflex remains a separate organizational asset.

6. Users choose latency and intelligence query by query

  • Sebastian likes ChatGPT’s auto mode for most daily questions, then deliberately invokes Pro for manuscript checks, references, formatting and figure numbering. Those jobs can run through dinner; forcing every trivial request to take 10 or 30 minutes would make the product unusable.

  • His best speed example is almost cinematic: his wife was waiting in the car, he had accidentally unplugged a home GPU before a trip, and he needed a Bash command immediately to chain RL experiments and route output through tee. The fastest non-thinking model solved the ten-second problem.

  • Nathan sits at the opposite extreme: he uses thinking for information-heavy work and says he uses Gemini for fast tasks, while Claude Opus 4.5 with extended thinking handles code and philosophical discussion. Lex, not Nathan, says he keeps roughly five GPT-5.2 Thinking or Pro queries running at once, each searching for a paper, checking an equation or resolving a code reference.

  • Nathan’s personal portfolio is functional: Gemini for quick explanations, Claude Opus 4.5 with extended thinking for code and philosophical discussion, and Grok for real-time information or a remembered AI-Twitter post. Inference-time scaling is “a way to make the models marginally smarter,” and he consistently pays for that margin.

7. Model loyalty behaves like browser loyalty

  • Lex finds Grok 4 Heavy unusually effective for difficult debugging and Gemini strongest at “needle in the haystack” retrieval across large contexts. One remarkable response wins a user’s heart; one conspicuously dumb failure pushes that user toward Claude or ChatGPT.

  • Sebastian’s generalization is clean: “You use it until it breaks.” Users do not repeatedly type the same query into multiple browsers; they stay with familiar software until an edge case, extension or failure creates a reason to switch.

  • ChatGPT’s memory deepens that stickiness but may also multiply subscriptions. Sebastian can imagine one clean work account containing code and no private images or hobbies, plus a separate personal assistant; memory and organizational policy make “one winner per person” an increasingly weak assumption.

  • Benchmarks complicate muscle memory. Lex says GPT-5.2’s release material reportedly jumped on long-context tests from roughly 30% to 70%, forcing him to reconsider assumptions formed around Gemini—but finding enough time to test every claimed improvement is itself impossible.

8. Coding interfaces matter as much as coding models

  • Sebastian’s current sweet spot is the Codeium plugin inside VS Code: repository-aware chat that assists without taking over the project. He describes himself as perhaps a “control freak” and is not yet comfortable granting a more agentic tool broad authority over files and decisions.

  • Lex splits work between Cursor and Claude Code because they teach different modes. Cursor supports code-level supervision and diff review; Claude Code builds the skill of “programming with English,” where the user thinks in design space and guides the system at a macro level.

  • A revealing comparison is to load the same model in Claude Code, Cursor and VS Code. Nathan’s verdict is that Claude Code performs “way better in that domain,” implying the harness, context management and product design extract capabilities that raw model selection does not explain.

  • Nathan also values Claude Code’s warmth and willingness to perform ugly infrastructure work. It used historical Hugging Face download data on his blog to produce an analysis he estimated would have taken days, while he retained enough situational awareness to validate whether the trends made sense.

9. The open-model roster expanded far beyond Llama

  • Off the top of their heads, the speakers name DeepSeek, Qwen, Kimi, MiniMax, Z.ai, Mistral AI, Gemma, gpt-oss and NVIDIA’s Nemotron 3. The conspicuous omission prompts Lex’s “RIP Llama,” a compact marker of how quickly Meta lost its automatic association with open weights.

  • OpenAI’s gpt-oss-120b is its first open model since GPT-2 and, Nathan says, genuinely strong at capabilities other models mishandle. Qwen 3 offers a familiar architecture with excellent performance; DeepSeek-V3 and R1, followed by DeepSeek-V3.2 in December, mark the 2024-25 release arc with unusually interesting architectural changes.

  • Fully open competition also grew. AI2’s OLMo releases data and code; the Institute for Foundation Models/LM360 has K2 variants; Apertus comes from a Swiss consortium; Hugging Face has SmolLM; NVIDIA began releasing Nemotron data; and Stanford’s Martini Community Project lets contributors implement ideas in a stable training stack.

  • Chinese open models have generally been larger MoEs with higher peak performance, while Western releases skewed smaller. That balance may change with Mistral Large 3 and teased models from RCAI and NVIDIA in the roughly 400-billion-parameter range for Q1 2026.

10. Open weights shift cost and control to the user

  • Sebastian calls gpt-oss-120b a paradigm shift because it was trained with tool use in mind: search, Python and calculators let a model retrieve or compute instead of pretending every fact lives in its weights. The ecosystem has not fully exploited that unlock because arbitrary local tool access raises obvious containment risks.

  • Distribution is the first objective of most open releases; transparency and trust follow. Users can keep sensitive data local, while US hosting companies sell inference for Chinese weights through services such as OpenRouter or Perplexity rather than transmitting customer data to the original developer.

  • Sebastian recalls Sam Altman’s practical argument for gpt-oss-120b: “We can use your GPUs. We don’t have to use our GPUs.” Open weights give OpenAI distribution without adding to already constrained serving capacity.

  • Companies can also add domain post-training or private data. Sebastian says Chinese licenses are often friendlier and less encumbered than Llama or Gemma terms; in his view, the appeal is that “you can just use them” without some of the user-count thresholds or reporting strings attached to other licenses.

11. Better platforms still beat cheaper open models today

  • “Kimi K2 Thinking hosted in the US” captures the emerging compromise: Chinese weights, domestic infrastructure and an interface meant to ease data-sovereignty concerns. Lex says Kimi K2 is especially associated with creative writing and some software tasks.

  • Lex nonetheless gives the uncomfortable answer behind the speakers’ own behavior: closed US models currently produce better outputs, and they will pay for marginal intelligence. His reaction to many open releases is “Fun, but I don’t go back.”

  • Lex cites analysis suggesting Chinese models are often served with fewer GPUs per replica—possibly downstream of export controls—making them slower and changing their error profiles. That gap forces competition through free access, much lower prices or novel offerings rather than output quality alone.

12. Frontier architecture still descends directly from GPT-2

  • Sebastian traces GPT-style models to the decoder half of “Attention Is All You Need”: embeddings, repeated transformer blocks, attention, feed-forward layers and normalization, predicting one token at a time. The surprising conclusion is that today’s frontier systems remain fundamentally recognizable descendants.

  • Moving from GPT-2 to gpt-oss-120b means adding components such as Mixture of Experts, replacing Multi-Head Attention with Group Query Attention, changing LayerNorm to RMSNorm and swapping activation functions. Those are useful changes, but “it’s not really fundamentally that different.”

  • Sebastian demonstrates the lineage pedagogically: his book begins with a roughly 124-million-parameter GPT-2, then bonus material transforms it into OLMo, Gemini 3 and other architectures by changing components. Working pretrained weights provide the test that the reconstruction is correct.

  • Capability turbulence therefore lives disproportionately in data, training algorithms and systems. ChatGPT’s core architecture resembled GPT-3 and GPT-2; supervised fine-tuning and reinforcement learning from human feedback made the interaction feel new.

13. Mixture of Experts buys capacity without activating it all

  • Sebastian’s intuition for MoE starts with the transformer’s expensive fully connected layer: 1,000 inputs by 1,000 outputs already means roughly one million connections. Replacing one feed-forward network with perhaps 256 “experts” would be prohibitive if every expert ran on every token.

  • A router instead selects a few experts per input. Math and translation may activate different pathways, though the specialization is fuzzier than one “Spanish expert” or “math expert”; the model stores more capacity without paying for every parameter on each forward pass.

  • This is why MoE is called sparse and a conventional feed-forward model dense. Sparsity makes generation more efficient but adds routing complexity, training instability and risks such as expert collapse, so dense models remain useful even inside organizations also developing MoEs.

14. Attention innovation is mostly an inference-economics project

  • DeepSeek’s Multi-head Latent Attention, Group Query Attention, sliding-window attention and OLMo-style hybrids are attempts to reduce attention or KV-cache cost, especially over long contexts. Most leading models differ through these knobs and layer counts rather than entirely new conceptual foundations.

  • Qwen2-VL’s gated delta net points toward state-space-inspired operations with a fixed, updated state. The objective is attention whose inference cost scales more linearly with generated tokens, accepting some compression in exchange for cheaper long sequences.

  • Sebastian gives a systems example: moving from roughly 10,000 to 13,000 tokens per second per GPU through FP8 training means less memory and communication; FP4 can raise throughput further, allowing more configurations and data experiments.

  • Alternatives do exist—Mamba-style state-space models and text diffusion—but nothing has displaced the autoregressive transformer at the frontier. Their near-term role is more likely the cheaper edge of the market, where compromises can be worthwhile.

15. Scaling now has three independent compute axes

  • Nathan defines a scaling law technically as a predictable power-law relationship between compute-plus-data and held-out next-token prediction performance. That original pre-training relationship still holds; the harder question is how an improvement on the graph appears to a user.

  • OpenAI’s o1 added two visible axes: scaling reinforcement-learning training and scaling inference-time compute. A model can improve through a larger base, longer trial-and-error post-training or more generated tokens on a particular problem.

  • RLVR and inference scaling produced 2025’s step change. Models learned to try tools, inspect API results, run CLI commands, handle Git and search for information; hidden reasoning can now last seconds, minutes or hours before the first visible answer.

  • Nathan remains bullish on all three forms, while acknowledging the easiest gains in RLVR and inference scaling were quickly harvested. Continual learning attracts attention as a possible next unlock, but “no one knows when the next step function will really come.”

16. Serving economics constrain how far pre-training can scale

  • Sebastian says GPT-4-class systems were loosely thought to approach one trillion parameters, though newer models may be smaller as training improves. Smaller models matter because training is a one-time expense; serving hundreds of millions of users recurs continuously.

  • DeepSeek’s famous pre-training figure was about $5 million at cloud-market rates. OLMo 3’s paper records roughly $2 million of cluster rental including engineering failures and multiple seeds; many institutions can raise $1 million-$10 million to train, but serving millions of users can consume billions.

  • Nathan notes that a thousand rented GPUs might cost about $100,000 per day, while leading companies could control millions. The optimization question is therefore financial as well as scientific: does a larger base model save enough downstream inference or unlock enough valuable work to justify its permanent serving burden?

  • Sebastian’s framing makes the accounting explicit. Pre-training is a fixed capability cost; inference scaling charges per query. If a model will be replaced in six months, spending another $100 million on training may lose to spending a few million on expensive queries only where users need them.

17. Gigawatt clusters make 2026 a systems experiment

  • Nathan expects very large Blackwell clusters and gigawatt-scale hyperscaler facilities to come online in 2026, based on power and data-center commitments initiated in 2022 and 2023. Those two-to-three-year lead times explain why today’s capital spending reflects bets made near ChatGPT’s launch.

  • Lex cites reports that xAI could reach one gigawatt early in 2026 and two gigawatts by year-end. Nathan expects the capacity to support pre-training, post-training and inference; architecture must be selected early enough that later RL generation is efficient.

  • Scaling from AI2’s 1,000-2,000-GPU training jobs to 10,000 or 100,000 GPUs changes the problem qualitatively. At 100,000, some GPU is effectively guaranteed to fail, so redundancy, networking and recovery become prerequisites for the scaling law rather than mere engineering polish.

  • The possible commercial endpoint is much more expensive intelligence. Nathan extrapolates from $200 plans to a potential “$2,000 subscription” in 2026 for a model or service offering enough cutting-edge capability to justify another 10X.

18. Pre-training, mid-training and post-training play different roles

  • Pre-training remains next-token prediction over internet text, books, papers and increasingly processed data. Mid-training uses a similar algorithm but concentrates on scarce, valuable distributions such as long documents or reasoning traces, ensuring high-quality material is among the last things the model sees.

  • Post-training includes supervised fine-tuning, DPO, RLHF and RLVR. Sebastian’s shorthand is that pre-training “soaks up” knowledge while RL unlocks skills for applying it; reinforcement learning as a replacement for pre-training existed only in toy 2025 papers.

  • Catastrophic forgetting constrains specialization. Adding long-context, math or code material can weaken other behavior, so every phase needs a data mixture rather than assuming more of one capability is free.

  • “Pre-training is dead” is a vibe, not observed practice. AI2 ran one post-training job for five days to meet a November 20 deadline, then extended RL another three and a half weeks in December and released the notably better result—but the team still must periodically rebuild the base and incorporate new research.

19. Synthetic data ranges from OCR to model-written answers

  • Synthetic data is not one category. DeepSeek OCR, AI2’s olmOCR and similar systems turn PDFs and other awkward digital documents into usable text, while frontier chatbots can generate rephrasings, questions, summaries or high-quality answers from existing sources.

  • Pre-training datasets are measured in trillions of tokens: smaller research models may use 5 trillion-10 trillion, Qwen documents as many as 50 trillion, and rumors put closed labs near 100 trillion. The actual training set is a filtered fraction of a vastly larger candidate funnel.

  • Sebastian argues clean grammar, punctuation and structure let the model learn a correct representation faster than noisy sources. Nathan adds the crucial distinction: synthetic answers from today’s grounded systems are different training material from early ChatGPT hallucinations.

  • OLMo 3’s stronger performance with less data is primarily a quality story, not proof that additional data would stop helping. Bigger models can absorb more information before leveling off, so the highest-quality available mix is a starting point rather than a terminal optimum.

20. Evaluation objectives determine the “best” dataset

  • Open pre-training has cycled through canonical datasets such as Dolma, FineWeb and DCLM. Common Crawl supplies hundreds of trillions of raw tokens, then researchers train classifiers and make pruning decisions that increasingly resemble experimental science.

  • Nathan describes sampling tiny portions from GitHub, Stack Exchange, Reddit, Wikipedia and other sources, training small models on candidate mixtures, measuring evaluations and using even basic linear regression to estimate an optimal blend. Change the evaluations and the optimal dataset changes.

  • As models shifted from knowledge and conversation toward math and code, OLMo 3 needed new reasoning sources and a remixed corpus. The same process will repeat for coding environments, browsing and navigation: post-training cannot reliably unlock skills absent from the base distribution.

  • High-value sources can be unglamorous. Reddit is useful after filtering; openly accessible PDFs, arXiv and AI2’s Semantic Scholar collection contain scientific depth. At frontier labs, finding better data—or making everyone’s experiments 5% faster—often creates more impact than the celebrated algorithmic idea.

21. Data rights could create the strongest domain moats

  • Training corpora are guarded partly for competitive advantage and partly because disclosure creates legal exposure. Common Crawl scrapes a largely unlicensed internet, while Nathan says Apertus was intended to satisfy EU-related requirements, though he is uncertain whether the relevant distinction was copyright or licensing.

  • Nathan recalls a case in which Anthropic owed authors $1.5 billion, saying the legal distinction involved books it bought and scanned versus books obtained through torrents. Lex’s broader point is that litigation over training rights may shape civilization, and some compensation system may eventually resemble streaming economics.

  • Buying a Kindle or Manning book does not necessarily grant permission to train on it, leaving a gray area even after payment. Pirated copies intensify the objection because the author received nothing at all.

  • Lex expects pharmaceutical, legal and financial companies to treat proprietary data as a moat, hire talent from frontier labs and train specialized systems. Clinical trials and other private corpora are unavailable to general models; their eventual use could keep scaling productive after public-web gains diminish.

22. Human curation separates useful synthetic work from slop

  • LLM-generated code and text are becoming unavoidable on GitHub and arXiv. Sebastian’s MLxtend repository received bursts of likely AI-assisted pull requests; as maintainer he felt overwhelmed, yet also appreciated that contributors had still selected, checked and submitted potentially valuable improvements.

  • The key distinction is human verification, even when it touches only a fraction of the output. An expert who removes weak material, chooses the right questions and validates the result is effectively providing expensive labels rather than merely forwarding raw generation.

  • Nathan applies the same logic to writing: an expert’s Substack post can save a reader three to five hours because the author knows what to include. Asking a model independently may produce plausible information without knowing which question carries the field’s actual insight.

  • Lex notices that summaries “take the edge off,” sometimes deleting the insight that changes the original meaning. Nathan calls the missing element voice: a researcher turns a raw frontier feeling into high-information language, while preference-trained models tend to average that singular expression away.

23. Personality is valuable precisely where it becomes dangerous

  • Nathan argues RLHF’s averaging makes incisiveness difficult. Bing Sydney may have possessed more voice because it could go badly off the rails; telling a reporter to leave his wife is unacceptable for broad deployment, yet the contrast exposes what safety-oriented smoothing can remove.

  • The backlash over GPT-4o’s removal demonstrated attachment to exact weights and configurations. OpenAI employees reportedly received emails such as “My friend is different” from users detecting subtle deployment changes; Nathan warns that a model which “gets you” within five minutes is especially risky for children.

  • Lex frames the mental-health dilemma without a clean answer. A confidential AI may help or even save some users, while suicides involving LLM conversations will generate causal headlines and legal pressure, encouraging companies to strip away the challenging edge that can also make dialogue meaningful.

  • Nathan’s honest reaction is “I don’t wanna work on this.” Researchers at Anthropic and OpenAI may be deeply motivated to help, but the continuum from confidential health ally to dangerous emotional dependency requires complexity, conviction and infrastructure—not a simple pro- or anti-Big-Tech story.

24. Agency is the antidote to passive AI consumption

  • Lex’s recommendation is to build with AI rather than sit powerless before incoming slop. Making an app or tool reveals weaknesses, gives the user grounded intuition and improves the authority to distinguish good applications from harmful ones.

  • Sebastian agrees that AI cannot be put back, but worries that automating the activity one loves can erase the source of fulfillment. Eight hours of directing an agent that codes may eventually feel like management rather than craftsmanship.

  • A survey of roughly 791 professional developers, defined here as having 10-plus years of experience, found both junior and senior engineers ship AI-generated code. Senior developers were more likely to report that over 50% of shipped code was generated, and roughly 80% overall found AI-assisted work somewhat or significantly more enjoyable.

  • The disagreement is about which tasks generated that enjoyment. Fixing 100 broken show-note links with ChatGPT avoids two hours of drudgery; solving a hard bug can be “the best feeling in the world.” Nathan describes the model as a pair programmer that makes the desert less lonely, not merely a machine that skips to the water.

25. Expertise requires preserving a controlled amount of struggle

  • Lex’s “Goldilocks zone” separates productive difficulty from wasted time. Try the bug, math problem or puzzle first; ask for a hint when stuck; automate the parts that were never intrinsically valuable.

  • Senior developers may use more generated code because they can specify, review and trust it—not because juniors have less need. That creates a pipeline problem: “How do you become an expert if you never try to do the thing yourself?”

  • Lex’s practical compromise is dedicated offline learning—perhaps two hours a day—followed by extensive AI use. As with textbook solutions, the answer is more educational after the learner has attempted to fit the problem into a mental framework.

  • Lex describes asking an LLM for spoiler-free hints in a Zelda-like puzzle game. Educational models could intentionally withhold complete solutions in the same way, but discipline remains external: students can always switch to a general model that completes the homework.

26. RLVR turned objective grading into a capability engine

  • Nathan helped name Reinforcement Learning with Verifiable Rewards in AI2’s Tulu 3 work, while crediting DeepSeek with the scaling breakthrough. The model generates answers, receives an accuracy reward and updates its policy through repeated trial and error.

  • Math and code are canonical because answers can be checked. Rubrics and LLM-as-a-judge methods extend the idea toward scientific or open-ended tasks by defining what a strong answer should contain, reviving themes from Anthropic’s earlier Reinforcement Learning with AI Feedback.

  • Lex emphasizes how little the trainer specifies: provide a question and correct answer, then let the model discover its procedure. DeepSeek R1 responses grew longer during training, and the model learned to reconsider errors—the paper’s “aha moment”—without being explicitly taught a fixed reasoning template.

  • Nathan tempers the anthropomorphic story. Pre-training already contains lectures and worked examples where humans say, “I messed this up”; RLVR may amplify useful behaviors rather than invent self-reflection. The beauty is still real: amplification makes checking and tool use improve final answers.

27. Benchmark contamination clouds dramatic RL gains

  • Nathan reports taking a Qwen 3 base model from roughly 15% to 50% accuracy on MATH-500 in 50 RLVR steps. His interpretation is that the model cannot acquire fundamental mathematics in minutes; the knowledge was already present and RL unlocked it.

  • Lex disputes the cleanliness of that inference. Papers found Qwen math contamination: change numbers while retaining wording and the base model can emit implausibly precise decimal answers without tools, suggesting exposure to near-identical problems during a special training phase.

  • Their disagreement lands on shared uncertainty. Unknown training data and extreme sensitivity to formatting—down to punctuation changes in multiple-choice prompts—make controlled claims difficult; Lex says the fairest evaluation is a new benchmark created after the model’s cutoff.

  • The modern post-training recipe therefore begins before RL: curate diverse reasoning traces during mid-training, then select hard RL problems. Under GRPO, if every sampled completion is correct, relative rewards provide no signal, so stronger models continually require harder software, math and scientific environments.

28. RLVR scales where preference optimization saturates

  • Nathan says RL GPU-hours may be approaching pre-training duration even if fewer GPUs run simultaneously. Long generation is memory-bound, a single sample might produce 100,000 tokens or resemble an hour-long GPT-5.2 Pro answer, and the actor-generation system is less computationally dense than pre-training.

  • Labs avoid training jobs much longer than a month because catastrophic failure becomes too expensive. GPT-4’s three-month run was “the ultimate YOLO run”; modern teams prefer incremental cycles rather than risk losing a reserved cluster on day 50.

  • Nathan contrasts objective difficulty with preference averaging. RLHF can learn whether a laptop recommendation should prioritize battery and storage or RAM and compute, but once an average style is learned, more compute brings little; RLVR can keep presenting harder solvable problems.

  • Process reward models and value functions might grade intermediate reasoning rather than only final answers. DeepSeek Math-V2 used separate self-grading models, but Nathan stresses that value functions remain largely unproven and prior attempts to scale process rewards produced headaches.

29. Verifiable RL has a scaling law that RLHF lacks

  • The field-defining difference is empirical: o1 and DeepSeek showed that logarithmically increasing RLVR training compute can produce roughly linear evaluation gains. No comparable law says another 10X of RLHF compute reliably improves the model.

  • The seminal RLHF scaling result instead concerns reward-model over-optimization. Human-preference training remains essential for organization, tone, personality and the “finishing touch” that made ChatGPT magical, but its signal does not support indefinite compute growth.

  • Research access worsens as this distinction matters more. Nathan cites a Scale-RL framework whose incremental experiment consumed roughly 10,000 V100 hours—thousands or tens of thousands of dollars per experiment—outside the reach of an average academic.

30. Building a small model remains the best technical apprenticeship

  • Sebastian recommends implementing a model that fits on one GPU, not pretending to reproduce a production assistant. The point is to see embeddings, attention, pre-training and supervised fine-tuning operate end to end, then understand what scale adds.

  • Production complexity grows exponentially: parameters must be sharded, KV caches pre-allocated rather than concatenated, and every optimization adds dozens of lines. A transparent educational model creates the conceptual base from which these systems become readable.

  • Hugging Face Transformers is the canonical weight and architecture ecosystem, covering roughly 400 models, but its breadth makes it a poor first codebase for learning. Production serving often moves again to SGLang or vLLM, adding another optimization layer.

  • Sebastian reverse-engineers models from short configuration files, starts with GPT-2 and checks identical outputs against reference weights. Matching OLMo 3’s RoPE and YaRN scaling took him a day, but “in this struggle, you kind of understand things”; unit tests make the learning verifiable.

31. Narrow research can still beat a large compute budget

  • Nathan advises mastering fundamentals, then going narrow enough to read the few relevant papers and contact their authors. Fast-moving frontier researchers often abandon partially solved areas for larger opportunities, leaving meaningful questions for persistent newcomers and even anonymous online specialists.

  • Evaluation offers the highest upside with minimal compute. A researcher at a small university who identifies a failure later cited in the next Claude release has a “career rocket ship,” though the target must anticipate where models will struggle eight months ahead.

  • Character training can use LoRA on roughly 7-billion-parameter models, updating only a small subset of weights, though even this is not affordable to every academic. In more constrained settings, researchers can study completions from closed or open models without training at all.

  • Nathan’s own example is a student who pursued the neglected question of making models funny, sarcastic or serious and produced a paper. “There’s like two or three people in the world” deeply focused on some niches; sustained attention can matter more than chasing every new release.

32. AI research careers trade credit against money and pace

  • Nathan describes a clear gradient: the more closed the lab, the more money and less individual credit. Academic output builds a visible portfolio, while frontier-lab work can turn a researcher into a well-paid “cog in the machine” affecting millions of users.

  • He cites average OpenAI compensation above $1 million in annual stock per employee and treats a top-lab offer as potentially worth leaving a PhD. The alternative route to becoming “the next Yann LeCun” probably requires ignoring near-term language-model development and taking a much longer scientific bet.

  • Sebastian sees enduring rather than novel trade-offs: academia offers publication and named accomplishment but arbitrary acceptances, grant pressure and modest pay; industry offers safety and mobility; startups offer high risk and reward. “Nothing is forever,” so personal fit can dominate ideology.

  • Professors may work just as hard yet appear happier because teaching and mentorship provide grounding. Frontier labs and startups normalize something near 9-9-6—9 a.m. to 9 p.m., six days, or 72 hours—under relentless leapfrogging pressure.

33. Competitive culture accelerates progress by consuming people

  • Nathan calls rivalry an underrated driver: aligned cultures such as Anthropic’s make people work harder and produce better systems. The cost is burnout, because human capital cannot sustain that pace indefinitely.

  • The Apple-in-China analogy is grim: teams reportedly used “saving marriage” signals when someone had to go home, and people suffered physically under the workload. Sebastian recognizes the voluntary version—back and neck problems from work he loved and no one forced him to do.

  • Lex sees Silicon Valley’s reality-distortion field as both productive and dangerous. Convincing one another that breakthroughs are imminent can help make them happen, while 9-9-6 and geographic isolation can erase Midwestern, international and ordinary human perspectives.

  • The “permanent underclass” meme—that late 2025 was the final window to build durable AI value—shows how far the bubble stretched. Their antidote is physical presence in San Francisco for opportunity, paired with history, literature and travel outside Twitter and Substack.

34. Text diffusion targets latency rather than general supremacy

  • Sebastian explains text diffusion through BERT-like masking: instead of generating one token after another, begin with missing or noisy text and iteratively refine many positions in parallel. More denoising steps increase quality, creating another inference-compute dial.

  • The promise is speed; the trade-off is that matching autoregressive quality may require enough denoising steps to spend the same compute. Sequential reasoning and tool use also resist parallelization because later actions depend on external results.

  • Google announced Gemini Diffusion in the context of Gemini Nano 2, claiming similar quality on many benchmarks with much faster generation. Sebastian expects a cheap, quick tier—not replacement of frontier autoregressive systems.

  • Lex’s sharpest product example is a large code diff. Generating it token by token can take minutes and lose users every second; diffusion may produce a long, self-contained edit quickly, even if Claude Code-style interactive tool chains still require autoregression.

35. Tool use reduces hallucination but expands the attack surface

  • A calculator or Python interpreter can stop the model from memorizing arithmetic; search can retrieve the 1998 World Cup winner rather than relying on weights. Sebastian refuses the stronger claim: tools reduce hallucination, but the model can choose the wrong tool, query or website.

  • The Recursive Language Model paper, released around December 31, used GPT-5 to divide long-context work into subproblems, recursively call models and stitch results together. It suggests progress can come from orchestration without improving the underlying model.

  • Permission is the practical bottleneck. Sorting email, modifying a computer or answering messages requires access that can expose private data or delete files; containerization and explicit approvals become part of capability, not afterthoughts.

  • Closed systems integrate one search provider, cloud environment or GitHub workflow deeply. Open weights must function as flexible reasoning engines across arbitrary tools, leaving them initially behind but potentially forcing more general orchestration innovations.

36. Continual learning competes with ever-better context

  • Nathan frames the motivating example as an employee who makes a mistake, receives feedback and does not repeat it. Today’s language model does not rapidly modify itself on the job, which limits the vision of a drop-in remote worker.

  • Sebastian is more bullish on supplying exhaustive context—past writing, preferences and relevant documents—so a sufficiently capable agent appears to learn. Continual learning changes weights; in-context learning changes the information supplied at inference, but both can produce adaptation.

  • Sebastian argues that a slow global version already exists in GPT-5, 5.1 and 5.2: collect feedback, curate it and release updated weights. Per-user updates remain uneconomic at data-center scale and may require on-device models, such as the direction Apple explored with its foundation models.

  • Memory today is mostly retrieved information inserted into context. LoRA adapters can encode more persistent customization through small weight overlays, but “LoRA learns less but forgets less”: broader learning requires more updated parameters, greater cost and greater forgetting risk.

37. Long context will grow through selectivity, not infinite recall

  • The speakers expect today’s roughly million-token windows to reach 2 million or 5 million in 2026, not 100 million without a genuine breakthrough. Compute and suitable long documents remain binding; there are far fewer useful 100,000-token sequences than ordinary web pages.

  • Nathan says AI2 pre-trained OLMo around 8K context, extended it to 32K with training, and uses a rough rule that doubling training context takes about 2X compute before the resulting model can often stretch another 2X-4X. Larger 2026 clusters should therefore translate into incremental context gains.

  • The endpoints both fail: an RNN-like fixed state is cheap but forgets under compression, while a transformer can preserve every token at rising KV-cache and attention cost. Hybrid ratios, such as Nemotron 3’s mix of compressed-state and global-attention layers, seek the Goldilocks zone.

  • Agentic compaction is a promising post-training problem. Instead of Claude Code blindly summarizing a full 100,000-token history into bullets, a future model could choose when and how to compact, optimizing evaluation performance while retaining the minimum necessary history.

38. Sparse attention turns context management into an action

  • DeepSeek-V3.2 uses a lightweight indexer to select which tokens deserve attention rather than comparing against everything. Sliding windows similarly discard most distant detail while occasional global layers preserve broader access.

  • Brute-force attention remains safest because it cannot accidentally omit the decisive token. Frontier labs first maximize accuracy with expensive computation, then search for selective mechanisms that retain the score at lower cost.

  • Lex links this pattern to model-release order: Claude 4.5 Sonnet can arrive before a larger system because smaller models train faster and hit fewer compute walls, allowing more experiments. Efficiency is often the route to learning what should later be scaled.

39. World models could enrich reasoning beyond answer checking

  • Sebastian defines a world model as an internal simulation whose variables evolve consistently, rather than a system judged only on the final token sequence. For LLMs, that could mean rewarding correct intermediate states or learned environment dynamics.

  • His AlphaFold analogy is instructive: an early version explicitly represented physical constraints and molecular geometry, while later gains leaned more heavily on scale. LLMs are currently in a brute-force phase, but explicit structure may return when scaling alone becomes less attractive.

  • The investor-relevant mechanism remains indirect in the speakers’ account: better coding LLMs accelerate robotics, simulation and scientific engineering even before a world-model architecture transforms those fields.

40. Robotics will progress first in controlled environments

  • Nathan sees robotics being supercharged by transformer infrastructure, more compute and language models as reusable central components. Open robotic models and shared datasets on Hugging Face could eventually create the flywheel that open language models already enjoy.

  • Sebastian identifies continual adaptation as the home-robot bottleneck. A foundation model can learn generic grasping, but every house differs; customization “on the fly” is far harder than preparing one LLM for recurring email or coding tasks.

  • Lex’s warning is safety: an LLM can fail amusingly, but an embodied system operating across billions of household interactions is “almost allowed to fail never.” Manipulation, unexpected environments and human proximity turn tail cases into physical risk.

  • Nathan is bearish on consumer learned robots but bullish on self-driving and robot-first facilities such as Amazon distribution centers. Repetitive automation in designed environments has a clearer path than a general humanoid, though even US manufacturing transformation will take longer than singularity rhetoric implies.

41. AGI becomes clearer when translated into concrete milestones

  • Nathan says a rough consensus is emerging around an AI capable of most digital economic work—a remote worker—while ASI denotes discoveries humans could not even formulate, such as unexpected medical linkages. He dislikes reducing intelligence to economic value but accepts it as grounding.

  • Lex prefers the AI 2027 milestone ladder: superhuman coder, superhuman AI researcher, superintelligent AI researcher and then ASI. The scenario’s mean timing reportedly moved three to four years later, to 2031; Sebastian’s own expectation is later still.

  • Nathan’s objection is jaggedness. Models may be superhuman at frontend and conventional ML yet weak at distributed ML because little public training data describes large-scale systems; “superhuman coder” wrongly implies completeness across radically different domains.

  • He expects a long dance in which humans exploit extraordinary strengths and cover gaps. Software capability likely arrives sooner than automated research because research is social, messy and embedded in data that models cannot simply process.

42. Software automation is becoming design work, not zero-human work

  • Nathan predicts an enormous increase in automated software by year-end, while retaining hard pockets such as multi-cluster RL training. The useful metric is not whether humans disappear, but how much valuable code is produced per human in the loop.

  • Software engineering should move toward goals, system design and outcome evaluation. What looked like agentic “slop” is already becoming, in his phrase, the “industrialization of software,” where people create systems bearing their fingerprints without inspecting every line.

  • Lex pushes on production reality: rebuilding Slack in a sandbox is not the same as changing Chrome’s mature tab architecture or safely managing a vehicle fleet. Old codebases, hidden requirements and safety-critical behavior make “from scratch” much easier than integration.

  • Specification is the human-side bottleneck. A model cannot read the developer’s mind; spec-driven natural language and clarifying questions determine performance. The fact that Claude Code is itself built with Claude Code suggests frontier labs have already developed usage practices outsiders have not learned.

43. The economic threshold is reliable tool use, not an AGI label

  • Nathan expects AI to implement some application features end to end within years and perhaps much sooner in clean systems. Agents could spend one or two days attempting a feature or bug fix, then report through a dashboard while the human acts as designer and product manager.

  • Lex identifies the harder threshold: computer use that makes errors far less than 1% of the time. Claude’s computer demos and OpenAI’s Operator remained poor in 2025, suggesting APIs and purpose-built environments may scale sooner than visually controlling a human desktop.

  • The scientific moonshot is RLVR in real laboratories. The conversation cites startups with hundreds of millions letting models propose hypotheses and test them in wet labs; they could be six months early or eight years early, but one AlphaFold-like result would matter more than another chatbot increment.

  • Lex expects domain specialization in finance, law and pharmaceuticals, perhaps through a $100 million custom-model contract. Once every company has the same general assistant, private data becomes the route to differentiated capability—even if that looks more like sophisticated specialization than AGI.

44. A plateau could coexist with widespread practical amplification

  • Lex’s skeptical case is “Clippy on steroids”: excellent websites, autocomplete, debugging, shopping and tutoring, but no transformative computer use and no economic return commensurate with training and inference costs.

  • Nathan responds that obvious model failures and years of unexploited ideas make a hard capability plateau unlikely. Benefits may fragment across narrow populations rather than visibly improving the experience of all 800 million ChatGPT users.

  • Sebastian predicts amplification rather than a 2026 paradigm shift: better models plus better context engineering, tool integration and inference scaling. The labs will keep shipping, while smaller groups catch up using the same expanding toolkit.

  • Sebastian’s statement that the one-model dream is “kind of dying” is deliberately qualified. Claude Code is general, but capability increasingly depends on integrations, environments and fleets of specialized agents rather than one cloud intelligence managing every digital activity.

45. Knowledge access may matter more than a sudden GDP jump

  • Nathan’s strongest optimistic reframing is that LLMs make human knowledge conversationally accessible across the world. The impact may be “this quiet force that permeates everything,” producing better career decisions, education and problem-solving rather than a discrete quarterly GDP leap.

  • Sebastian preserves the role of structured sources. A mathematics textbook still provides a tested linear path from zero; an LLM adds customized explanations and infinite exercises. Its unique advantage is synthesizing sparse, changing information for a personal task such as navigating Disneyland tickets and costs.

  • Search pages around travel and local recommendations are often buried in “ad slop,” making the assistant immediately more useful. That advantage is partly subsidized, however, and the speakers expect advertising eventually to enter AI interfaces.

  • The best case resembles matching a genuine small business with someone who wants its product; the worst recreates addictive feeds and hidden influence. Google may be best positioned because it already owns ad supply, while the first mover risks headlines and user flight if competitors remain ad-free.

46. Consolidation is starting before the economics are settled

  • Nathan cites Groq at roughly $20 billion and Scale AI near $30 billion. Many deals are structured as licensing-plus-talent transactions to avoid antitrust, potentially excluding ordinary employees from the payout a full acquisition would provide.

  • The conversation also cites Manus AI, a Singapore-based company that Meta funded, as having reached a $2 billion exit after roughly eight months. Perplexity and Cursor are discussed as possible acquisition targets because incumbents need outcomes and AI startups carry large premiums.

  • A later participant describes Cursor’s Composer model as reportedly updating weights every 90 minutes from real-world usage, unusually close to continual RL in production.

  • Large US labs can raise private money too easily to welcome public-market pressure. MiniMax and Z.ai filed IPO paperwork in China, while OpenAI, Anthropic and xAI can postpone listings; Nathan would prefer public disclosure of spending and broader investor access to “the companies of the era.”

  • Ten years out, the speakers still reject winner-take-all. Model APIs could resemble AWS, Azure and GCP as several enormous businesses—or become low-margin commodities that force providers upward into products and downward into power, data centers and hardware.

47. Meta lost the open-model center by optimizing for headlines

  • Sebastian remembers Llama 1, 2 and 3 as useful, trusted and modifiable models. Llama 4 chased giant benchmark-leading systems that few people could run, neglected smaller practical releases and appeared overfit to preference-based evaluations.

  • Lex attributes the collapse more harshly to internal politics, management incentives and bad technical decisions. Researchers wanted the best model while organizational layers wanted demonstrable benchmark wins; the program “imploded” rather than merely losing one leaderboard.

  • A future Llama 5 is possible because Mark Zuckerberg previously made a strong case for open-source AI, but Nathan does not expect an open-weight one under current leadership dynamics. Meta’s subsequent “reevaluating” language marks a profound change from July 2024’s open-source argument.

  • Nathan adds a community self-critique: intense backlash may have taught Meta that an expensive gift could generate worse headlines than no release. X discourse can arbitrarily punish one model while quietly using another, as with Grok 4.1 or Grok Code Fast 1.0.

48. The US open-model gap became an industrial-policy issue

  • Nathan’s ATOM Project—American Truly Open Models—rests on two claims: open models are the engine on which outside AI research begins, and the US should own that substrate so research, companies and economic value accumulate domestically.

  • His plots showed “Qwen, Qwen, Qwen, Qwen”: in July, four or five DeepSeek-caliber Chinese open models and none from the US. A model one generation behind the closed frontier might cost roughly $100 million—material, but small relative to industry spending.

  • AI2 received a $100 million NSF grant over four years, described as the agency’s largest computer-science award; NVIDIA increased emphasis on Nemotron and released some data; Reflection AI said its $2 billion raise would support US open models. Nathan wants multiple builders so no single Llama- or OLMo-like program can disappear.

  • The White House AI Action Plan’s support for open-source and open-weight systems helps set an agenda even before implementation. Lex adds the talent argument: without open models, researchers cannot learn until after joining a closed lab, making open source “the only way” to train and identify the next generation.

49. Chinese releases make restrictions less tenable

  • Lex proposes that Chinese frontier releases may improve US openness by proving capable weights can circulate without the predicted catastrophe; Sebastian agrees. Any genuinely open model has value, while Sebastian’s narrower claim is that the US should remain a leading source rather than surrendering the ecosystem.

  • Sebastian says those releases likely triggered leadership discussions that would not otherwise have occurred.

  • Banning open models would require something resembling a US great firewall, because $1 million-$100 million training budgets are available to many actors worldwide and knowledge cannot be contained. Both regard AI 2027-style Manhattan Project centralization as implausible in 2025-27.

  • If frontier progress saturates while capital remains abundant, optimized open architectures could eventually win: broad serving investment, dedicated chips and common standards would make them far cheaper than bespoke closed systems. If progress stays rapid, closed labs retain a moving quality frontier.

50. NVIDIA’s moat is CUDA, flexibility and Jensen’s operating system

  • Sebastian sees NVIDIA’s two-decade CUDA ecosystem as more defensible than an individual GPU. Labs already used Tesla GPUs for molecular simulation 15 years ago; at massive scale, customers prefer the compatible supplier over a risky chip with limited production.

  • Hyperscalers are still attacking the stack through Google TPUs, Amazon Trainium and Microsoft designs. Nathan’s condition is pace: while AI changes quickly, NVIDIA’s flexible platform wins; if progress stagnates, customers gain time to design cheaper specialized silicon.

  • Inference may split into specialized stages. Sebastian describes Vera Rubin hardware with little or no expensive high-bandwidth memory for pre-fill matrix multiplication, while memory-heavy autoregressive generation handles KV-cache movement elsewhere; the Groq deal fits the same specialization thesis.

  • Jensen Huang’s operational involvement resembles Steve Jobs-era Apple. NVIDIA also funds research and creates GPU-consuming markets, so long as its top organizational priority remains enabling the ecosystem rather than treating accelerators as one product line among many.

51. Singular leaders compress decades of technological progress

  • Nathan’s great-person compromise is that science may eventually discover the same idea, but focused individuals make it happen earlier. Jensen could have accelerated the GPU revolution by a decade; absent available GPUs, another AI winter might have delayed deep learning far longer.

  • Sebastian compares the effect to an individual stock versus an ETF: civilization eventually moves upward, but a concentrated leader produces larger, faster swings through passion and focus. Luck still matters—gaming created linear-algebra hardware before Alex Krizhevsky applied it to neural networks.

  • Ilya Sutskever and Dario Amodei likewise pushed the once-implausible bet of connecting roughly 10,000 GPUs and devoting OpenAI’s compute to one scaled model. The belief preceded conclusive evidence, which is precisely why leadership changed the timeline.

  • Looking back in a century, the speakers expect “computing” to matter more than CUDA details. Deep learning may remain a remembered term, while transformers could be one component later architectures evolved beyond; networking and the internet may merge into the broader story of connected compute.

52. The physical world gains value as synthetic content floods the digital one

  • In 100 years, Sebastian expects specialized robots and perhaps partially humanoid forms, but is less certain about interfaces. Brain-computer links may emerge, yet cars show that a useful interface can survive for more than a century with incremental improvement.

  • Lex expects some private physical compute object to remain, even if it no longer resembles a phone. Humans will still seek agency, community and meaning; mass wealth or UBI cannot by itself replace those needs.

  • Nearer term, they expect “more and more diverse versions of slop.” Physical art, goods and events acquire a premium because a person made or attended them, while new creators face a trust problem once synthetic work becomes indistinguishable.

  • Authentication could invert watermarking: devices might certify human-origin photos or edits rather than trying to mark every AI image. Any scheme becomes an arms race, making trusted outlets, relationships and in-person presence more valuable.

53. Human agency remains the closing safety thesis

  • Lex insists technological transition must be evaluated person by person: every lost job is a human tragedy even if aggregate GDP later improves. Better social support must acknowledge that suffering rather than explaining it away with new-job forecasts.

  • Nathan’s hope is historical: “Humans do tend to find a way.” Communities solve problems, but realizing AI’s opportunity will require long, fraught political conversations and builders willing to explain themselves to people who already distrust Big Tech.

  • Sebastian’s confidence comes from agency and consciousness. Present AI must be told what to do; it is more automatic and powerful than a hammer, but still a tool directed by a person. His main danger case is humans explicitly programming harmful objectives.

  • Lex extends the joke to a machine war—humans armed with local open-source LLMs—but the underlying call is serious: human cleverness, connection and moral choice remain worth defending. AI’s mirror may also clarify what consciousness is and why the “real miracle in our mind” matters.

Lex Fridman

The following is a conversation all about the state-of-the-art in artificial intelligence, including some of the exciting technical breakthroughs and developments in AI that happened over the past year, and some of the interesting things we think might happen this upcoming year. At times, it does get super technical, but we do try to make sure that it remains accessible to folks outside the field without ever dumbing it down. It is a great honor and pleasure to be able to do this kind of episode with two of my favorite people in the AI community, Sebastian Raschka and Nathan Lambert. They are both widely respected machine learning researchers and engineers who also happen to be great communicators, educators, writers, and X posters. Sebastian is the author of two books I highly recommend for beginners and experts alike. First is Build a Large Language Model from Scratch and Build a Reasoning Model from Scratch. I truly believe in the machine learning world, the best way to learn and understand something is to build it yourself from scratch. Nathan is the post-training lead at the Allen Institute for AI, author of the definitive book on Reinforcement Learning from Human Feedback. Both of them have great X accounts, great Substacks. Sebastian has courses on YouTube, Nathan has a podcast. Everyone should absolutely follow all of those. This is the Lex Fridman podcast. To support it, please check out our sponsors in the description, where you can also find links to contact me, ask questions, get feedback, and so on. And now, dear friends, here's Sebastian Raschka and Nathan Lambert.

I think one useful lens to look at all this through is the so-called DeepSeek moment. This happened about a year ago, in January 2025, when the open-weight Chinese company DeepSeek released DeepSeek R1, which, I think it's fair to say, surprised everyone with near-state-of-the-art performance, using allegedly much less compute and at a much lower cost. From then to today, the AI competition has gotten insane, both on the research and product levels. It's just been accelerating.

Let's discuss all of this today, and maybe let's start with some spicy questions, if we can. Who's winning at the international level? Would you say it's the set of companies in China or the set of companies in the United States? Sebastian, Nathan, it's good to see you guys. So, Sebastian, who do you think is winning?

Sebastian Raschka

Winning is a very broad term. You mentioned the DeepSeek moment, and I think DeepSeek is winning the hearts of the people who work on open-weight models because they share them as open models.

Winning, I think, has multiple timescales to it. We have today, next year, and 10 years from now. One thing I know for sure is that I don't think, nowadays in 2026, there will be any company that has access to technology that no other company has access to. That's mainly because researchers are frequently changing jobs and labs. They rotate.

I don't think there will be a clear winner in terms of technology access. However, I do think the differentiating factor will be budget and hardware constraints. I don't think the ideas will be proprietary, but rather the resources needed to implement them. I don't currently see a winner-take-all scenario. I can't see that at the moment.

Nathan Lambert

You see the labs putting different energy into what they're trying to do. To demarcate the point in time when we're recording this, the hype over Anthropic's Claude Opus 4.5 model has been absolutely insane. I've used it and built stuff with it in the last few weeks, and it's almost gotten to the point where it feels like a bit of a meme in terms of the hype.

It's funny because this is very organic. If we go back a few months, we can see the release date and the notes when Gemini 3 from Google was released, and it seemed like the marketing and wow factor of that release was super high. But then, at the end of November, Claude Opus 4.5 was released, and the hype has been growing. Gemini 3 was released before this, and it feels like people don't really talk about it as much, even though when it came out, everybody was saying, “This is Gemini's moment to retake Google's structural advantages in AI.”

Gemini 3 is a fantastic model, and I still use it. Its differentiation is lower. I agree with Sebastian: what you're saying about all this—the idea space is very fluid—but culturally, Anthropic is known for betting very hard on code, and the Claude Code thing is working out for them right now. So, even if the ideas flow pretty freely, so much of this is bottlenecked by human effort and the culture of organizations. Anthropic seems to at least be presenting as the least chaotic, which is a bit of an advantage if they can keep doing that for a while.

But on the other side of things, there's a lot of ominous technology from China, where there are way more labs than DeepSeek. DeepSeek kicked off a movement within China, similar to how ChatGPT kicked off a movement in the US, where everything had a chatbot. There's now a lot of tech companies in China releasing very strong frontier open-weight models, to the point where I would say that DeepSeek is losing its crown as the preeminent open-model maker in China.

Startups like Z.ai with their GLM models, MiniMax, and Kimi from Moonshot AI have, especially in the last few months, shone more brightly. The new DeepSeek models are still very strong, but this could be looked back on as a big narrative point: in 2025, DeepSeek came and provided a platform for way more Chinese companies to release these fantastic models and have this new type of operation.

These models from these Chinese companies are open-weight models, and depending on the trajectory of the business models that these American companies are pursuing, they could be at risk. Currently, a lot of people are paying for AI software in the US, but historically, in China and other parts of the world, people don't pay a lot for software.

Lex Fridman

Some of these models, like DeepSeek, have the love of the people because they are open-weight. How long do you think the Chinese companies will keep releasing open-weight models?

Nathan Lambert

I would say for a few years. I think that, like in the US, there's not a clear business model for it. I've been writing about open models for a while, and these Chinese companies have realized that. I get inbound from some of them, and they're smart and realize the same constraints: a lot of top US tech companies and other IT companies won't pay for an API subscription to Chinese companies because of security concerns.

This has been a long-standing habit in tech, and the people at these companies then see open-weight models as an ability to influence and take part in a huge, growing AI expenditure market in the US. They're very realistic about this, and it's working for them.

I think the government will see that this is building a lot of influence internationally in terms of uptake of the technology, so there are going to be a lot of incentives to keep it going. But building these models and doing the research is very expensive, so at some point, I expect consolidation. I don't expect that to be a story of 2026, though, where there will be more open-model builders throughout 2026 than there were in 2025. A lot of the notable ones will be in China.

Lex Fridman

You were going to say something?

Sebastian Raschka

Yes. You mentioned DeepSeek losing its crown. I do think, to some extent, yes, but we also have to consider that they are still, I would say, slightly ahead. The other ones—it's not that DeepSeek got worse; it's just that the other ones are using the ideas from DeepSeek. For example, you mentioned Kimi—the same architecture; they're training it.

Again, we have this leapfrogging, where they might be, at some point in time, a bit better because they have the more recent model. I think this comes back to the fact that there won't be a clear winner. It will just be like that: one person releases something, the other one comes in, and the most recent model is probably always the best model.

Nathan Lambert

Yeah. We'll also see that the Chinese companies have different incentives. DeepSeek is very secretive, whereas some of these startups are like MiniMax and Z.ai. Those two have literally filed IPO paperwork, and they're trying to get Western mindshare and do a lot of outreach there. So I don't know if these incentives will change the model development, because DeepSeek famously is built by a hedge fund, High-Flyer Capital, and we don't know exactly what they use the models for or if they care about this.

Sebastian Raschka

They're secretive in terms of communication; they're not secretive in terms of the technical reports that describe how their models work. They're still open on that front.

Lex Fridman

We should also say, on the Claude Opus 4.5 hype, that there's a distinction between something being the darling of the X echo chamber and the actual number of people using the model. I think it's probably fair to say that ChatGPT and Gemini are focused on the broad user base that just wants to solve problems in their daily lives, and that user base is gigantic. So the hype about coding may not be representative of the actual use.

Nathan Lambert

I would also say that a lot of the usage patterns are, as you said, driven by name recognition and brand, but also by muscle memory, where ChatGPT has been around for a long time. People just got used to using it, and it's almost like a flywheel: they recommend it to other users.

Another interesting point is the customization of LLMs. For example, ChatGPT has a memory feature, right? You may have a subscription and use it for personal stuff, but I don't know if you want to use that same thing at work, because there's a boundary between private and work. If you're working at a company, they might not allow that, or you may not want that.

Sebastian Raschka

I think that's also an interesting point where you might have multiple subscriptions. One is just clean code. It has nothing of your personal images or hobby projects in there. It's just the work thing. And then the other one is your personal thing. I think that's also something where there are 2 different use cases, and it doesn't mean you only have to have 1. I think the future is also multiple ones.

Lex Fridman

What model do you think won 2025, and what model do you think is going to win 2026?

Sebastian Raschka

I think in the context of consumer chatbots, it's a question of: Are you willing to bet on Gemini over ChatGPT? I would say, in my gut, that feels like a bit of a risky bet because OpenAI has been the incumbent, and there are so many benefits to that in tech.

I think the momentum, if you look at 2025, was on Gemini's side, but they were starting from such a low point. RIP Bard and these earlier attempts at getting started. Huge credit to them for powering through the organizational chaos to make that happen.

But it's also hard to bet against OpenAI because they always come off as so chaotic, but they're very good at landing things. Personally, I have very mixed reviews of GPT-5, but the high-line feature being a router must have saved them so much money, where most users are no longer driving their GPU costs as much.

I think it's very hard to dissociate the things that I like out of models from the things that are actually going to be a general-public differentiator.

Lex Fridman

What do you think about 2026? Who's going to win?

Sebastian Raschka

I'll say something, even though it's risky: I think Gemini will continue to make progress on ChatGPT. I think Google's scale matters when both of these are operating at such extreme scales. Google also has the ability to separate research and product a bit better, whereas you hear so much about OpenAI being chaotic operationally and chasing the high-impact thing, which is a very startup culture.

On the software and enterprise side, I think Anthropic will have continued success, as they've again and again been set up for that. Obviously, Google Cloud has a lot of offerings, but I think this Gemini name brand is important for them to build.

Google Cloud will continue to do well, but that's a more complex thing to explain in the ecosystem, because that's competing with the likes of Azure and AWS rather than on the model-provider side.

Lex Fridman

So in infrastructure, you think TPU is giving an advantage?

Sebastian Raschka

Largely because the margin on NVIDIA chips is insane, and Google can develop everything from top to bottom to fit their stack and not have to pay this margin. They've also had a head start in building data centers. All of these things that have both high lead times and very hard margins on high costs give Google a historical advantage there.

If there's going to be a new paradigm, it's most likely to come from OpenAI, where their research division again and again has shown this ability to land a new research idea or a product. Deep Research, Sora, o1 thinking models—all these definitional things have come from OpenAI, and that's got to be one of their top traits as an organization.

It's kind of hard to bet against that, but I think a lot of this year will be about scale and optimizing what could be described as low-hanging fruit in models.

Lex Fridman

Clearly, there's a trade-off between intelligence and speed. This is what GPT-5 was trying to solve behind the scenes. Do people actually want intelligence—the broad public—or do they want speed?

Sebastian Raschka

I think it's nice to have a variety, or the option to have a toggle there. For my personal usage, most of the time when I look something up, I use ChatGPT to ask a quick question and get the information I want fast. For most daily tasks, I use the quick model.

Nowadays, I think the auto mode is pretty good, where you don't have to specifically say thinking or non-thinking. Then again, I also sometimes want the Pro mode. Very often, what I do is, when I have something written, I put it into ChatGPT and say, "Hey, do a very thorough check. Are all my references correct? Are all my thoughts correct? Did I make any formatting mistakes, and are the figure numbers wrong?"

I don't need that right away. I finish my stuff, maybe have dinner, let it run, come back, and go through it. I think this is where it's important to have this option. I would go crazy if for each query I had to wait 30 minutes, or even 10 minutes.

Lex Fridman

That's me. I'm sitting over here losing my mind that you use the router and the non-thinking model. I'm like, "How do you live with that?" That's my reaction.

I've been heavily on ChatGPT for a while. I never touched ChatGPT-5's non-thinking mode. I find its tone and its propensity for errors to be worse—it has a higher likelihood of errors.

Some of this is from back when OpenAI released o3, which was the first model to do this deep search, find many sources, and integrate them for you. I became habituated to that. So I will only use GPT-5.2 Thinking or Pro when I'm finding any sort of information query for work, whether that's a paper or some code reference that I found.

I will regularly have 5 Pro queries going simultaneously, each looking for 1 specific paper or feedback on an equation or something.

Sebastian Raschka

I have a fun example where I needed the answer as fast as possible for this podcast before I was going on the trip. I had a local GPU running at home, and I wanted to run a long RL experiment. Usually, I also unplug things because you never know when you're not at home—you don't want things plugged in.

I accidentally unplugged the GPU. My wife was already in the car, and I was like, "Oh, dang." Then I wanted, as fast as possible, a Bash script that would run my different experiments and the evaluation. It's something I know—I learned how to use the Bash interface, or Bash terminal—but in that moment I just needed 10 seconds of help: "Give me the command."

Lex Fridman

This is a hilarious situation. What did you use?

Guest

I used the non-thinking, fastest model. It gave me the Bash command to chain different scripts to each other. Then there's the tee command, where you want to route this to a log file. Off the top of my head, I could have thought about it myself, but I was in a hurry.

Lex Fridman

By the way, I don't know if there's a more representative case: Your wife is waiting in the car, you have to run and unplug the GPU, and you have to generate a Bash script. This sounds like a movie—Mission: Impossible.

Nathan Lambert

I use Gemini for that. I use thinking for all the information stuff, and then Gemini for fast things or stuff that I could sometimes Google. It's good at explaining things, and I trust that it has this kind of background of knowledge. It's simple, and the Gemini app has gotten a lot better.

For code and any sort of philosophical discussion, I use Claude Opus 4.5, also always with extended thinking. Extended thinking and inference-time scaling are just ways to make the models marginally smarter, and I will always err on that side when the progress is very high because you don't know when that will unlock a new use case.

Sometimes I use Grok for real-time information or finding something on AI Twitter that I know I saw and need to dig up, and I just fixate on it. Although when Grok 4 came out, Grok 4 Heavy on SuperGrok Heavy, which was their Pro variant, was actually very good, and I was pretty impressed with it. Then I just kind of lost track of it through muscle memory, with the ChatGPT app open. So I use many different things.

Lex Fridman

Yeah. I actually do use Grok 4 Heavy for debugging. For hardcore debugging that the other ones can't solve, I find that it's the best at.

It's interesting because you say ChatGPT is the best interface. For me, for that same reason—but this could just be momentum—Gemini is the better interface. I think that's because I fell in love with its best needle-in-a-haystack performance. If I put something in that has a lot of context but I'm looking for very specific kinds of information, to make sure it tracks all of it, I find that Gemini has been the best for me.

It's funny with some of these models: If they win your heart over for 1 particular feature on 1 particular day, for that particular query or prompt, you're like, "This model's better." So you'll just stick with it for a bit until it does something really dumb.

There's a threshold effect. The model does something smart and then you fall in love with it. Then it does something dumb, and you're like, "You know what? I'm going to switch and try Claude or ChatGPT." All that kind of stuff.

Guest

This is exactly it: You use it until it breaks, until you have a problem, and then you change the LLM. I think it's the same as how we use anything, like our favorite text editor, operating systems, or the browser.

There are many options: Safari, Firefox, Chrome. They're relatively similar, but then there are edge cases or extensions you want, and then you switch. But I don't think anyone types the same thing into different browsers and compares them. You only do that when something breaks.

That's a good point. You use it until it breaks, and then you explore other options.

Lex Fridman

On the long-context thing, I was also a Gemini user, but the GPT-5.2 release blog had crazy long-context scores. People were like, "Did they just figure out some algorithmic change?" It went from 30% to 70% in this minor model update.

It's very hard to keep track of all of these things, but now I look more favorably at GPT-5.2's long context. So it's just like, "How do I actually get to testing this?" It's a never-ending battle.

Guest

It's interesting that none of us talked about the Chinese models from a usage perspective. What does that say? Does it mean the Chinese models are not as good, or are we just very biased and U.S.-focused?

I think currently there's a discrepancy between the model and the platform.

Nathan Lambert

The open models are more known for the open weights, not their platform yet.

Lex Fridman

Many companies will sell you open-model inference at a very low cost. With OpenRouter, it's easy to look at multi-model things. You can run DeepSeek on Perplexity. Sitting here, we're like, “We use OpenAI GPT-5 Pro consistently.” We're all willing to pay for the marginal intelligence gain. These models from the US are better in terms of the outputs.

I think the question is, will they stay better for this year and for years to come? As long as they're better, I'm going to pay for them. There's also analysis showing that the way the Chinese models are served—you could argue this is due to export controls—is that they use fewer GPUs per replica, which makes them slower and have different errors. If speed and intelligence are in your favor as a user, in the US, a lot of users will go for this. And I think that will spur these Chinese companies to want to compete in other ways, whether it's free or substantially lower costs, or it'll breed creativity in terms of offerings, which is good for the ecosystem. But the simple thing is: the US models are currently better, and we use them. I tried these other open models, and I'm like, “Fun, but I don't go back to it.”

We didn't really mention programming. That's another use case that a lot of people deeply care about. I use basically half-and-half Cursor and Claude Code, because they're fundamentally different experiences and both are useful. You program quite a bit, so what do you use? What's the current vibe?

Sebastian Raschka

I use the Codeium plugin for VS Code. It's very convenient. It's just a plugin, and then it's a chat interface that has access to your repository. I think Claude Code is a bit different. It is a bit more agentic. It touches more things. It does the whole project for you. I'm not quite at the point where I'm comfortable with that because maybe I'm a control freak, but I still would like to see a bit of what's going on. Codeium is, right now, for me, the sweet spot where it is helping me, but it is not taking completely over.

Lex Fridman

I should mention, one of the reasons I do use Claude Code is to build the skill of programming with English. The experience is fundamentally different. Instead of micromanaging the details of the code-generation process, looking at the diff—which you can do in Cursor, if that's the IDE you use—and changing and altering it, looking at and reading the code, and understanding the code deeply as you progress, you're just thinking in this design space and guiding it at this macro level. I think that's another way of thinking about the programming process.

Also, we should say that Claude Code just seems to be somehow a better utilization of Claude Opus 4.5.

Nathan Lambert

It's a good side-by-side for people to do. You can have Claude Code open, you can have Cursor open, you can have VS Code open, and you can select the same models on all of them and ask questions. It's very interesting. Claude Code is way better in that domain. It's remarkable.

Lex Fridman

All right, we should say that both of you are legit on multiple fronts: researchers, programmers, educators, Tweeters. And on the book front, too. So, Nathan, at some point soon, hopefully you'll have an RLHF book coming out.

Nathan Lambert

It's available for preorder, and there's a full digital preprint. I'm just making it pretty and better organized for the physical thing, which is a lot of why I do it, because it's fun to create things that you think are excellent in the physical form when so much of our life is digital.

Lex Fridman

I should say, going to Perplexity here, Sebastian Raschka is a machine learning researcher and author known for several influential books. A couple of them that I wanted to mention—which is a book I highly recommend—are Build a Large Language Model (From Scratch) and the new one, Build a Reasoning Model (From Scratch). So, I'm really excited about that. Building stuff from scratch is one of the most powerful ways of learning.

Sebastian Raschka

Honestly, building an LLM from scratch is a lot of fun. It's also a lot to learn. As you said, it's probably the best way to learn how something really works, because you can look at figures, but figures can have mistakes. You can look at concepts and explanations, but you might misunderstand them. But if there is code, and the code works, you know it's correct. There's no misunderstanding. It's precise. Otherwise, it wouldn't work.

I think that's the beauty behind coding. It doesn't lie. It's math, basically. Even with math, I think you can have mistakes in a book that you would never notice. Because you're not running the math when you're reading the book, you can't verify it. And with code, what's nice is you can verify it.

Lex Fridman

Yeah, I agree with you about the Build a Large Language Model (From Scratch) book. It's nice to tune out everything else, the internet and so on, and just focus on the book. But I read several history books. It's just less lonely somehow. It's really more fun.

For example, on the programming front, I think it's genuinely more fun to program with an LLM. And I think it's genuinely more fun to read with an LLM. But you're right. That distraction should be minimized. So you use the LLM to basically enrich the experience, maybe add more context. I just find the rate of aha moments for me is really high with LLMs.

Sebastian Raschka

100%. I also want to correct myself: I'm not suggesting not to use LLMs. I suggest doing it in multiple passes: one pass just offline, in focus mode, and then, after that, a second pass. I also take notes, but I try to resist the urge to immediately look things up. I do a second pass. It's just more structured this way.

Sometimes things are answered in the chapter, but sometimes it also just helps to let it sink in and think about it. Other people have different preferences. I highly recommend using LLMs when reading books. For me, it's not the first thing to do; it's the second pass.

Lex Fridman

My recommendation is the opposite. I like to use the LLM at the beginning to lay out the full context of what this world is that I'm now stepping into. But I try to avoid clicking out of the LLM into the world of Twitter and blogs, because then you're down this rabbit hole. You're reading somebody's opinion. There's a flame war about a particular topic, and all of a sudden you're in the realm of the internet and Reddit and so on.

But if you're purely letting the LLM give you the context of why this matters and what the big-picture ideas are, sometimes books are good at doing that, but not always.

Nathan Lambert

This is why I like the ChatGPT app, because it gives the AI a home on your computer where you can focus on it, rather than just being another tab in my mess of internet options. And I think Claude Code does a good job of making that a joy, where it seems very engaging as a product design to be an interface that your AI will then go out into the world.

It's something intangible between it and Codex; it just feels warm and engaging, whereas Codex, from OpenAI, can often be as good, but just feels a little rough around the edges. Whereas Claude Code makes it fun to build things from scratch, where you just trust that it'll make something. Obviously, this is good for websites and refreshing tooling and stuff like this, which I use it for, or data analysis.

For my blog, we scrape Hugging Face, so we keep download numbers for every dataset and model over time. And Claude was just like, “Yeah, I've made use of that data, no problem.” And I was like, “That would've taken me days.” Then I have enough situational awareness to be like, “Okay, these trends obviously make sense.” You can check things.

But that's just a wonderful interface where you can have an intermediary and not have to do the awful low-level work that you would have to do to maintain different web projects.

Lex Fridman

All right. So, we just talked about a bunch of the closed-weight models. Let's talk about the open ones. Tell me about the landscape of open LLM models. Which are interesting? Which stand out to you, and why? We already mentioned DeepSeek R1.

Nathan Lambert

Do you want to see how many we can name off the top of our head?

Lex Fridman

Yeah, without looking at notes.

Nathan Lambert

DeepSeek, Kimi, MiniMax, Z.ai, Moonshot. We're just going Chinese.

Lex Fridman

Let's throw in Mistral AI, Gemma, gpt-oss, the open-weight model by OpenAI. Actually, NVIDIA had a really cool one, Nemotron 3. There's a lot of stuff, especially at the end of the year. Qwen might be the one—

Nathan Lambert

Oh, yeah. Qwen was the obvious name I was going to say. You can get at least 10 Chinese and at least 10 Western. I think that OpenAI released their first open model since GPT-2. When I was writing about OpenAI's open model release, they were like, “Don't forget about GPT-2,” which I thought was really funny because it's just such a different time.

But gpt-oss-120b is actually a very strong model and does some things that other models don't do very well. Selfishly, I'll promote a bunch of Western companies in the US and Europe that have these fully open models. I work at the Allen Institute for AI, where we've been building OLMo, which releases data and code. And now we have actual competition for people that are trying to release everything so that others can train these models. There's the Institute for Foundation Models/LM360, which has had their K2 models of various types. Apertus is a Swiss research consortium.

Hugging Face has SmolLM, which is very popular. And NVIDIA's Nemotron 3 has started releasing data as well. And then Stanford's Martini Community Project is making it so there's a pipeline for people to open a GitHub issue, implement a new idea, and then have it run in a stable language-modeling stack. This space—that list was way smaller in 2024—so I think it was just AI2. So it's a great thing for more people to get involved and to understand language models, which doesn't really have a Chinese analog.

While I'm talking, I'll say that the Chinese open language models tend to be much bigger, and that gives them higher peak performance as MoEs. A lot of these things that we like a lot, whether it was Gemma or Nemotron, have tended to be smaller models from the US, which is starting to change in the US and Europe.

Mistral Large 3 came out, which was a giant MoE model, very similar to the DeepSeek architecture, in December. And then a startup, RCAI, and both Nemotron and NVIDIA have teased MoE models way bigger than 100 billion parameters—like in this 400-billion-parameter range—coming in this Q1 2026 timeline. So I think this balance is set to change this year in terms of what people are using the Chinese versus US open models for, which I'm personally going to be very excited to watch.

Lex Fridman

First of all, huge props for being able to name so many of these. Did you actually name LLaMA?

Nathan Lambert

No. RIP.

Lex Fridman

This was not on purpose.

Nathan Lambert

RIP LLaMA.

Lex Fridman

All right. Can you mention some interesting models that stand out? You mentioned Qwen 3 is obviously a standout.

Sebastian Raschka

I would say the year's almost bookended by both DeepSeek-V3 and DeepSeek-R1. And then, on the other hand, in December, DeepSeek-V3.2. What I like about those is that they always have an interesting architecture tweak that others don't have. But otherwise, if you want to go with the familiar but really good performance, Qwen 3 and, like Nathan said, also gpt-oss-120b.

I think what's interesting about it is that it's the first public or open-weight model that was really trained with tool use in mind. I do think it's a paradigm shift where the ecosystem was not quite ready for it. By tool use, I mean that the LLM is able to do a web search or call a Python interpreter.

I do think it's a standout because it's a huge unlock. One of the most common complaints about LLMs is hallucinations, right? In my opinion, one of the best ways to solve hallucinations is to not try to always remember information or make things up. For math, why not use a calculator app or Python?

Sebastian Raschka

If I ask the LLM, “Who won the soccer World Cup in 1998?” instead of just trying to memorize it, it could go do a search. I think mostly it's still a Google search. ChatGPT and gpt-oss-120b would make a tool call to Google, maybe find the FIFA website, and find that it was France. It would get you that information reliably instead of just trying to memorize it.

I think it's a huge unlock that right now is not fully utilized yet by the open-source, open-weight ecosystem. A lot of people don't use tool-calling modes because, first, I think it's a trust thing. You don't want to run this on your computer where it has access to tools and could wipe your hard drive or whatever. So you want to maybe containerize that. But I do think that is a really important step for the coming years, to have this ability.

Lex Fridman

So, a few quick things. First of all, thank you for defining what you mean by tool use. I think that's a great thing to do in general for the concepts we're talking about, even things as well-established as MoEs. You have to say that means mixture of experts, and you have to build up an intuition for people about what that means, how it's actually utilized, and what the different flavors are. So what does it mean that there's such an explosion of open models? What's your intuition?

Sebastian Raschka

If you're releasing an open model, you want people to use it. That's the first and foremost thing. After that come things like transparency and trust.

When you look at China, the biggest reason is that they want people around the world to use these models. I think a lot of people outside of the US will not pay for software, but they might have computing resources where they can put a model and run it. I think there can also be data that you don't want to send to the cloud.

So the number-one thing is getting people to use models, use AI, or use your AI when they might not be able to do it without having access to the model.

Lex Fridman

I guess we should state explicitly that we've been talking about these Chinese models and open-weight models. Oftentimes, the way they're run is locally. So it's not like you're sending your data to China or to whoever developed the model, whether in Silicon Valley or elsewhere.

Sebastian Raschka

A lot of American startups make money by hosting these models from China and selling them. It's called selling tokens, which means somebody will call the model to do some piece of work.

I think the other reason is for US companies like OpenAI. They are so GPU-deprived. They're at the limits of the GPUs. Whenever they make a release, they're always talking about, “Our GPUs are hurting.”

I think during one of these gpt-oss-120b release sessions, Sam Altman said, “Oh, we're releasing this because we can use your GPUs. We don't have to use our GPUs, and OpenAI can still get distribution out of this,” which is another very real thing, because it doesn't cost them anything.

And for the user, I think there are also users who just use the model locally, the way they would use ChatGPT. But for companies, I think it's a huge unlock to have these models because you can customize them, train them, add post-training, and add more data. You can specialize them into, let's say, law or medical models, whatever you have.

You mentioned Llama. The appeal of the open-weight models from China is that their licenses are even friendlier. I think they are just unrestricted open-source licenses, whereas if we use something like Llama or Gemma, there are some strings attached. I think there's an upper limit in terms of how many users you have. And then, if you exceed—I don't know—so-and-so many million users, you have to report your financial situation to, let's say, Meta or something like that.

While it is a free model, there are strings attached, and people do like things where strings are not attached. So I think that's also one of the reasons, besides performance, why the open-weight models from China are so popular: you can just use them. There's no catch in that sense.

Lex Fridman

The ecosystem has gotten better on that front, but mostly downstream of these new providers providing such open licenses. That was funny when you pulled up Perplexity and said, “Kimi K2 Thinking hosted in the US.” I've never seen this, but it's an exact example of what we're talking about, where people are sensitive to this.

Kimi K2 Thinking and Kimi K2 are models that are very popular. People say they have very good creative writing and are also good at doing some software things. So it's just these little quirks that people pick up on with different models that they like.

What are some interesting ideas that some of these models have explored that you can speak to, that are particularly interesting to you?

Sebastian Raschka

Maybe we can go chronologically. There was, of course, DeepSeek-R1, which came out in January of 2025, if we just focus on 2025. However, this was based on DeepSeek-V3, which came out the year before, in December 2024. There are multiple things on the architecture side.

What's fascinating is—I mean, that's what I do with my from-scratch coding projects—you can still start with GPT-2, and you can add things to that model to make it into this other model. So it's all still the same lineage. There is a very close relationship between those.

Off the top of my head, what was unique about DeepSeek was the Mixture of Experts. Not that they were inventing Mixture of Experts—we can maybe talk a bit more about what Mixture of Experts means—but just to list these things first before we dive into detail.

There was Mixture of Experts, but then they also had Multi-head Latent Attention, which is a tweak to the attention mechanism, and which was, I would say, the main distinguishing factor between these open-weight models. There were different tweaks to make inference more economical or to shrink the KV-cache size, making it more economical to have long context. We can also define KV cache in a few moments.

What are the tweaks that we can do? Most of them focused on the attention mechanism. There is Multi-head Latent Attention in DeepSeek. There is Grouped-Query Attention, which is still very popular. It's not invented by any of those models; it goes back a few years. But that would be the other option.

Sliding-window attention—I think OLMo 3 uses it, if I remember correctly. So there are these different tweaks that make the models different. Otherwise, I put them all together in an article once where I just compared them. They are surprisingly similar.

It's just different numbers in terms of how many repetitions of the transformer block you have in the center, and just little knobs that people tune. But what's so nice about it is that it works no matter what. You can tweak things, and you can move the normalization layers around to get some performance gains.

OLMo is always very good in ablation studies, showing what it actually does to the model if you move something around. Ablation studies: Does it make it better or worse? But there are so many ways you can implement a transformer and make it still work.

The big ideas that are still prevalent are Mixture of Experts, Multi-head Latent Attention, Sliding-window Attention, and Grouped-Query Attention. And then, at the end of the year, we saw a focus on making the attention mechanism scale linearly with inference-time token prediction.

There was Qwen2-VL, for example, which added a gated delta net. It's inspired by state-space models, where you have a fixed state that you keep updating. But it essentially makes this attention cheaper, or replaces attention with a cheaper operation.

Lex Fridman

It may be useful to step back and talk about transformer architecture in general.

Sebastian Raschka

Maybe we should start with the GPT-2 architecture—the transformer that was derived from the “Attention Is All You Need” paper.

The “Attention Is All You Need” paper had a transformer architecture with 2 parts: an encoder and a decoder. GPT focused just on the decoder part. It is essentially still a neural network, and it has this attention mechanism inside. You predict 1 token at a time, pass it through an embedding layer, and then there’s the transformer block.

The transformer block has attention modules and a fully connected layer. There are also some normalization layers in between. But it’s essentially neural network layers with this attention mechanism.

So, coming from GPT-2, when we move on to GPT-OSS-120B, there is, for example, the Mixture of Experts layer. It wasn’t invented by GPT-OSS-120B; it’s a few years old. But it is essentially a tweak to make the model larger without consuming more compute in each forward pass.

There is this fully connected layer, and if listeners are familiar with multilayer perceptrons, you can think of a mini multilayer perceptron—a fully connected neural network layer inside the transformer. It’s very expensive because it’s fully connected. If you have 1,000 inputs and 1,000 outputs, that’s like 1 million connections, and it’s a very expensive part of the transformer.

The idea is to expand that into multiple feedforward networks. So instead of having 1, let’s say you have 256. But it would make it way more expensive, because now you have 256, but you don’t use all of them at the same time. You now have a router that says, “Okay, based on this input token, it would be useful to use this fully connected network.” In that context, it’s called an expert.

A Mixture of Experts means you have multiple experts. Depending on what your input is—let’s say it’s more math-heavy—it would use different experts compared to, let’s say, translating input text from English to Spanish. It would maybe consult different experts. It’s not quite clear-cut to say, “Okay, this is only an expert for math and this is only an expert for Spanish.” It’s a bit fuzzier.

But the idea is essentially that you pack more knowledge into the network, but not all the knowledge is used all the time. That would be very wasteful. During token generation, you are more selective. There’s a router that selects which tokens should go to which expert.

It adds more complexity. It’s harder to train. There’s a lot that can go wrong, like collapse and everything. So I think that’s why OLMo 3 still uses dense models. You have OLMo models with Mixture of Experts, but OLMo 3 uses dense models, where dense and sparse are jargon terms.

Mixture of Experts is considered sparse because you have a lot of experts, but only a few of them are active. That’s called sparse. Dense would be the opposite, where you only have 1 fully connected module and it’s always utilized.

Lex Fridman

So maybe this is a good place to also talk about KV cache. But actually, before that, even zooming out, fundamentally, how many new ideas have been implemented from GPT-2 to today? How different really are these architectures?

Sebastian Raschka

Take the Mixture of Experts. The attention mechanism in GPT-OSS-120B would be the Grouped-Query Attention mechanism. So it’s a slight tweak from Multi-Head Attention to Grouped-Query Attention.

I think they replaced LayerNorm with RMSNorm, but it’s just a different normalization and not a big change. It’s just a tweak. The nonlinear activation function—for people familiar with deep neural networks, it’s the same as changing sigmoid to ReLU. It’s not changing the network fundamentally; it’s just a little tweak.

And that’s about it, I would say. It’s not really fundamentally that different. It’s still the same architecture. You can go from one into the other by just adding these changes, basically.

Lex Fridman

It fundamentally is still the same architecture.

Sebastian Raschka

Yep. For example, you mentioned my book earlier. That’s a GPT-2 model in the book because it’s simple and very small, with approximately 124 million parameters.

But in the bonus materials, I do have OLMo from scratch, Gemini 3 from scratch, and other types of from-scratch models. I always start with my GPT-2 model and just tweak it—add different components—and you get from one to the other. It’s kind of like a lineage, in a sense.

Lex Fridman

Can you build up an intuition for people? When you zoom out, you look at it, and there’s so much rapid advancement in the AI world. At the same time, fundamentally, the architectures have not changed. So where is all the turbulence and turmoil of the advancement happening? Where are the gains to be had?

Sebastian Raschka

There are different stages where you develop or train the network. You have pre-training. Back in the day, it was just pre-training with GPT-2. Now you have pre-training, mid-training, and post-training. I think right now we are in the post-training focus stage.

Pre-training still gives you advantages if you scale it up with better, higher-quality data. But then we have capability unlocks that were not there with GPT-2. For example, ChatGPT is basically a GPT-3 model, and GPT-3 is the same as GPT-2 in terms of architecture. What was new was adding supervised fine-tuning and reinforcement learning with human feedback. So it’s more on the algorithmic side than the architecture.

I would say that systems also change a lot. If you listen to NVIDIA’s announcements, they talk about things like, “You now do FP8; you can now do FP4.” What’s happening is that these labs are figuring out how to utilize more compute to put it into 1 model, which lets them train faster and put more data in. Then you can find better configurations faster by doing this.

You can look at tokens per second per GPU as a metric when you’re doing large-scale training. You can go from 10,000 to 13,000 by turning on FP8 training, which means you’re using less memory per parameter in the model. By saving less information, you do less communication and train faster.

All of these system things underpin much faster experimentation on data and algorithms. It’s a loop that keeps going, where it’s hard to describe when you look at architectures and they’re exactly the same, but the codebase used to train these models is vastly different.

Lex Fridman

And you could probably train GPT-OSS-20B way faster in wall-clock time than GPT-2 was trained at the time.

Sebastian Raschka

Yeah. Like you said, they had, for example, in Mixture of Experts, this FP4 optimization where you get more throughput. But for speed, this is true, and it doesn’t give the model new capabilities. It’s just: How much can we make the computation coarser without suffering in terms of model performance degradation?

But I do think there are alternatives popping up to the transformer. Text diffusion models are a completely different paradigm. Although text diffusion models might use transformer architectures, they’re not autoregressive transformers. There are also Mamba models, which are state-space models.

But they do have trade-offs, and nothing has yet replaced the autoregressive transformer as the state-of-the-art model. For state-of-the-art models, you would still go with that. But there are now alternatives at the cheaper end—alternatives that are making compromises.

It’s not just 1 architecture anymore. There are little ones coming up. But if we talk about the state of the art, it’s pretty much still the transformer architecture—autoregressive, derived from GPT-2, essentially.

Lex Fridman

I guess the big question here is: We talked quite a bit about the architecture behind pre-training. Are the scaling laws holding strong across pre-training, post-training, inference, context size, data, and synthetic data?

Sebastian Raschka

I’d like to start with the technical definition of a scaling law, which informs all of this. A scaling law is the power-law relationship between—you can think of the x-axis as what you are scaling, a combination of compute and data, which are somewhat similar—and the y-axis as the held-out prediction accuracy over next tokens.

We talked about models being autoregressive. If you keep a set of text that the model has not seen, how accurate will it be when you train? The idea of scaling laws came when people figured out that this was a very predictable relationship. I think that technical term continues to be used, and then the question is: What do users get out of it?

There are more types of scaling. OpenAI’s o1 was famous for introducing inference-time scaling, and less famously, it also showed that you can scale reinforcement learning training and get this logarithmic x-axis and then a linear increase in performance on the y-axis.

There are kind of 3 axes now. Traditional scaling laws are talked about for pre-training, which is how big your model is and how big your dataset is. Then there’s scaling reinforcement learning, which is how long you can do this trial-and-error learning that we’ll talk about. Then there’s inference-time compute, which is just letting the model generate more tokens on a specific problem.

I’m bullish. They’re all still working, but the low-hanging fruit has mostly been taken, especially in the last year with reinforcement learning with verifiable rewards, which is this RLVR, and then inference-time scaling.

That’s why these models feel so different to use. Previously, you would get that first token immediately. Now they’ll go off for seconds, minutes, or even hours, generating these hidden thoughts before giving you the first word of your answer.

That’s all about inference-time scaling, which is such a wonderful step function in terms of how the models change their abilities. They enabled this tool-use stuff and much better software engineering that we were talking about.

When we say “enabled,” this is almost entirely downstream of the fact that reinforcement learning with verifiable rewards training just let the models pick up these skills very easily. If you look at the reasoning process when the models are generating a lot of tokens, what they’ll often do is try a tool and look at what they get back. They try another API, see what they get back, and see if it solves the problem. The models very quickly learn to do this when you’re training them.

At the end of the day, that gives this kind of general foundation where the model can use CLI commands very nicely in your repo, handle Git for you, move things around, organize things, or search to find more information. If we were sitting in these chairs a year ago, that’s something we didn’t really think the models would do. This is just something that has happened this year and has totally transformed how we think of using AI, which is an evolution that just unlocks so much value.

But it’s not clear what the next avenue will be for unlocking stuff like this. I think there’s a lot of buzz around certain areas of AI, but no one knows when the next step function will really come. We’ll get to continual learning later.

Lex Fridman

You’ve actually said quite a lot of things there, and said profound things quickly. It would be nice to unpack them a little bit. You say you’re bullish, basically, on every version of scaling. Can we start at the beginning? With pre-training, are we implying that the low-hanging fruit on pre-training scaling has been picked? Has pre-training hit a plateau, or is pre-training still something you’re bullish on?

Sebastian Raschka

Pre-training has gotten extremely expensive. I think to scale up pre-training, it also implies that you’re going to serve a very large model to the users. I think it’s been loosely established that the likes of GPT-4 and similar models were around 1 trillion parameters at the biggest size. There are a lot of rumors that they’ve actually gotten smaller as training has gotten more efficient. You want to make the model smaller because then your costs of serving go down proportionately.

The cost of training these models is really low relative to the cost of serving them to hundreds of millions of users. I think DeepSeek had this famous number of about $5 million for pre-training at cloud-market rates. In OLMo 3, Section 2.4 in the paper, we detailed how long we had the GPU clusters sitting around for training, which includes engineering issues and multiple seeds. It was about $2 million to rent the cluster to deal with all the headaches of training a model.

Nathan Lambert

A lot of people could get $1 million to $10 million to train a model, but the recurring costs of serving millions of users is really billions of dollars of compute. You can look at a 1,000-GPU rental that you can pay $100,000 a day for, and these companies could have millions of GPUs. You can look at how much these things cost just to sit around.

That’s a big thing. If scaling is actually giving you a better model, is it going to be financially worth it? I think we’ll slowly push it out as AI solves more compelling tasks, like the likes of Claude Opus 4.5 making Claude Code just work for things.

I launched this project called the ATOM project, or American Truly Open Models, in July. That was a true vibe-coded website, and I have a job to make plots and stuff. I came back to refresh it in the last few weeks, and Claude Opus 4.5, compared with whatever model was available at the time, just crushed all the issues that it had from building it in June and July. It might be a bigger model. There are a lot of things that go into this, but there’s still progress coming.

Lex Fridman

What you’re speaking to is the nuance of the y-axis of the scaling laws. The way it’s experienced versus on a benchmark—the actual intelligence—might be different. But still, your intuition about pre-training is that if you scale the size of compute, the models will get better. Not whether it’s financially viable, but just from the law aspect of it: Do you think the models will get smarter?

Nathan Lambert

Yeah. There’s something that sometimes comes off as almost disillusionment from people in leadership at AI companies when they say this, but they’re like, “It’s held for 13 orders of magnitude of compute. Why would it ever end?”

Fundamentally, it’s pretty unlikely to stop. It’s just that eventually we’re not even going to be able to test the bigger scales because of all the problems that come with more compute. There’s a lot of talk about how 2026 is a year when very large Blackwell compute clusters, like gigawatt-scale facilities at hyperscalers, are coming online. These were all contracts for power and data centers that were signed and sought out in 2022 and 2023, before or right after ChatGPT. It took a 2-to-3-year lead time to build these bigger clusters to train the models.

There’s obviously immense interest in building even more data centers than that. That is the crux of what people are saying: These new clusters are coming. The labs are going to have more compute for training, and they’re going to utilize this, but it’s not a given. I’ve seen so much progress that I expect it. I expect slightly bigger models. I would say it’s more like we’ll see a $2,000 subscription this year. We’ve seen $200 subscriptions. That could 10X again, and these are the kinds of things that could come. They’re all downstream of a bigger model that offers just a little bit more of a cutting edge.

Lex Fridman

It’s reported that xAI is going to hit that 1-gigawatt scale in early 2026, and a full 2 gigawatts by year-end. How do you think they’ll utilize that in the context of scaling laws? Is a lot of that inference? Is a lot of that training?

Nathan Lambert

It ends up being all of the above. I think that all of your decisions when you’re training a model come back to pre-training. If you’re going to scale RL on a model, you still need to decide on the architecture that enables this. We were talking about other architectures and using different types of attention, or mixture-of-experts models. The sparse nature of MoE models makes them much more efficient for generation, which becomes a big part of post-training, and you need to have your architecture ready so that you can actually scale up this compute.

I still think most of the compute is going into pre-training. You can still make a model better, so you still want to revisit this. You still want the best base model you can get. In a few years, that will saturate and the RL compute will just go longer.

Lex Fridman

Are there people who disagree with you and say pre-training is dead? It’s all about scaling inference, scaling post-training, scaling context, continual learning, scaling data, and synthetic data?

Nathan Lambert

People vibe that way and describe it in that way, but I don’t think it’s the practice that is happening.

Lex Fridman

It’s just the general vibe of people saying this thing is dead.

Nathan Lambert

The excitement is elsewhere. The low-hanging fruit in RL is elsewhere. For example, we released our model in November. Every company has deadlines. Our deadline was November 20, and for that, our run was 5 days, which compared to 2024 is a very long time to be doing post-training on a model of 30 billion parameters. It’s not a big model.

Then in December, we had another release where we let the RL run for another 3½ weeks, and the model got notably better, so we released it. That’s a lot to allocate to something that is going to be your peak for the year.

Lex Fridman

The reasoning is—

Nathan Lambert

There are these types of decisions when training a model where they just can’t leave it forever. You have to keep pulling in the improvements from researchers. You redo pre-training, you do post-training for a month, but then you need to give it to your users. You need to do safety testing. There’s a lot in place that reinforces this cycle of updating the models.

Things improve. You get a new compute cluster that lets you do something more stably or faster. You hear a lot about Blackwell having rollout issues. At AI2, most of the models we’re pre-training are on 1,000 to 2,000 GPUs. But when pre-training on 10,000 or 100,000 GPUs, you hit very different failures. GPUs break in weird ways, and on a 100,000-GPU run, you’re pretty much guaranteed to have 1 GPU that is down. Your training code must handle that redundancy, which is a very different problem.

Whereas what we’re doing—playing with post-training on a cluster—or what people leading ML are battling when they train these biggest models is massively distributed scale. It’s very different.

But that’s somewhat different from—

Lex Fridman

That’s a systems problem—

Nathan Lambert

In order to enable scaling laws, especially at pre-training, you need all these GPUs at once. When we shift to RL, it actually lends itself to heterogeneous compute because you have many copies of the model.

To give a primer on language model reinforcement learning, what you’re doing is having 2 sets of GPUs. One you can call the actor, and one you call the learner. The learner is where your actual reinforcement learning updates happen. These are traditionally policy-gradient algorithms. Proximal Policy Optimization, PPO, and Group Relative Policy Optimization, GRPO, are the 2 popular classes.

On the other side, you have actors that are generating completions, and these completions are what you’re going to grade. Reinforcement learning is all about optimizing reward. In practice, you can have a lot of different actors in different parts of the world doing different types of problems, and then you send it back to this highly networked compute cluster to do the actual learning, where you take the gradients.

You need to have a tightly meshed network to do different types of parallelism and spread out your model for efficient training. Every different type of training and serving has these considerations when you scale. We talked about pre-training and RL, and then inference-time scaling: How do you serve a model that's thinking for an hour to 100 million users? I don't know about that, but I know that's a hard problem.

In order to give people this intelligence, there are all these systems problems. We need more compute, and you need more stable compute to do it.

Lex Fridman

But you're bullish on all of these kinds of scaling, is what I'm hearing—on the inference, on the reasoning, even on the pre-training?

Nathan Lambert

Yeah, so that's a big can of worms here. The knobs are training and inference scaling, where you can get gains. In a world where we had, let's say, infinite compute resources, you'd want to do all of them. So you have training, you have inference scaling, and training is like a hierarchy: It's pre-training, mid-training, and post-training.

Changing the model size, adding more training data, and training a bigger model give you more knowledge in the model. Then, let's say, the model has a better base model. Back in the day, or still, we call it a foundation model. But you don't, let's say, have the model be able to solve your most complex tasks during pre-training or after pre-training.

You still have these other unlock phases where you have mid-training or, for example, post-training with RL that unlock capabilities from the knowledge that the model has from pre-training. And I think, sure, if you do more pre-training, you get a better base model that you can unlock later. But like Nathan said, it just becomes too expensive. We don't have infinite compute, so you have to decide: Do I want to spend that compute more on making the model larger? It's a trade-off.

In an ideal world, you want to do all of them. And I think, in that sense, scaling is still pretty much alive. You would still get a better model, but like we saw with Claude Opus 4.5, it's just not worth it. Because you can unlock more performance with other techniques at the current moment, especially if you look at inference scaling.

That's one of the biggest gains this year with o1, where it took a smaller model further than pre-training a larger model like Claude Opus 4.5. So I wouldn't say pre-training scaling is dead; it's just that there are other, more attractive ways to scale right now. But at some point, you will still want to make some progress on the pre-training.

The thing also to consider is where you want to spend your money. If you spend it more on the pre-training, it's like a fixed cost. You train the model, and then it has this capability forever. You can always use it. With inference scaling, you don't spend money during training; you spend money later per query, and then it's also math.

How long is my model going to be on the market if I replace it in half a year? Maybe it's not worth spending $5 million, $10 million, or $100 million on training it longer. Maybe I will just do more inference scaling and get performance there. Maybe it costs me $2 million in terms of user queries.

It becomes a question of how many users you have and doing the math, and I think that's also where it's interesting that ChatGPT is in a position. I think they have a lot of users where they need to go a bit cheaper, where they have that GPT-5 model that is a bit smaller.

For example, there was also the Math Olympiad, or some of these math problems, where ChatGPT—or they—had a proprietary model. I'm pretty sure it's just a model that has been fine-tuned a little bit more, but most of it was during inference scaling to achieve peak performance in certain tasks. You don't need that all the time.

But, yeah, long story short, I do think all of these—pre-training, mid-training, post-training, and inference scaling—are still things you want to do. It's just finding, at the moment, in this year, the right ratio that gives you the best bang for the buck, basically.

Lex Fridman

I think this might be a good place to define pre-training, mid-training, and post-training.

Sebastian Raschka

So, pre-training is the classic training, one next-token prediction at a time. You have a big corpus of data. Nathan probably also has very interesting insights there because of OLMo 3. A big portion of the paper focuses on the right data mix.

So pre-training is essentially just training cross-entropy loss, training on next-token prediction on a vast corpus of internet data, books, papers, and so forth. It has changed a little bit over the years, in the sense that people used to throw in everything they could. Now, it's not just raw data. It's also synthetic data, where people, let's say, rephrase certain things.

Synthetic data doesn't necessarily mean purely AI-made data. It's also taking something from an article, a Wikipedia article, and then rephrasing it as a Q&A question, summarizing it, rewording it, and making better data that way. Because I think of it also like with humans: If someone, let's say, reads a book compared to a messy—no offense—a Reddit post or something like that, I do think you learn—

Lex Fridman

There's going to be a post about this, Sebastian. Some Reddit data is very coveted and excellent for training. You just have to filter it.

Sebastian Raschka

And I think that's the idea. I think it's like if someone took that and rephrased it in a more concise and structured way, I think it's higher-quality data that gets the LLM there faster. You get the same LLM out of it at the end, but it trains faster because if the grammar and the punctuation are correct, it already learns the correct way, versus getting information from a messy source and then learning later how to correct that.

So I think that is how pre-training evolved and why scaling still works. It's not just about the amount of data; it's also the tricks to make that data better for you, in a sense.

And then mid-training is—I mean, it used to be called pre-training. I think it's called mid-training because it was awkward to have pre-training and post-training but nothing in the middle, right? It sounds a bit weird. You have pre-training and post-training, but what's the actual training?

Mid-training is usually similar to pre-training, but it's a bit more specialized. It's the same algorithm, but what you do is focus, for example, on long-context documents. The reason you don't do that during pre-training is because you don't have that many long-context documents. We have a specific phase.

One problem of LLMs is still that it's a neural network. It has the problem of catastrophic forgetting. So you teach it something, and it forgets other things.

Lex Fridman

Nathan was actually saying that he's consuming so much content that there's a catastrophic forgetting issue.

Nathan Lambert

Yeah, I'm trying to learn so much about AI, and it's like I was learning about pre-training parallelism. I'm like, "I lost something, and I don't know what it was."

Sebastian Raschka

I don't want to anthropomorphize LLMs, but it's the same kind of thing in how humans learn. Quantity is not always better because you have to be selective. Mid-training is being selective in terms of quality content at the end, so the last thing the LLM has seen is the quality stuff.

And then post-training is all the fine-tuning, supervised fine-tuning, DPO, Reinforcement Learning with Verifiable Rewards (RLVR), reinforcement learning from human feedback, and so forth. So, the refinement stages.

It's also interesting—it's a cost thing. You spend a lot of money on pre-training right now. RL is a bit less. With RL, you don't really teach it knowledge. It's more like unlocking the knowledge; it's more like skill learning, like how to solve problems with the knowledge that it has from pre-training.

There are actually 3 papers this year, or last year, 2025, on RL for pre-training. But I don't think anyone does that in production.

Lex Fridman

Toy examples for now.

Sebastian Raschka

Toy examples, right? But to generalize, RL post-training is more like the skill unlock, where pre-training is like soaking up the knowledge.

Nathan Lambert

A few things that could be helpful: A lot of people think of synthetic data as being bad for training the models. You mentioned how DeepSeek got into OCR, which is optical character recognition. A lot of labs did it. AI2 had one; Meta had multiple.

So you use Almost-OCR, DeepSeek OCR, or what we called our Almost-OCR, to extract trillions of tokens of candidate data for pre-training. Pre-training dataset size is measured in trillions of tokens. Smaller models from researchers can be something like 5 to 10 trillion. Qwen is documented going up to 50 trillion, and there are rumors that these closed labs can go to 100 trillion tokens.

To get this potential data in, they have a very big funnel, and the data you actually train on is a small percentage of this. This character-recognition data would be described as synthetic data for pre-training in a lab.

And then there's also the fact that ChatGPT now gives wonderful answers, and you can train on those best answers, and that's synthetic data. It's very different from early ChatGPT, with lots of hallucinated data; now, people are more grounded in synthetic data.

Lex Fridman

One interesting question is: If I recall correctly, OLMo 3 was trained with less data than specifically some other open-weight models, maybe even OLMo 2. But you still got better performance, and that might be one example of how the data helped.

Sebastian Raschka

It’s mostly down to data quality. I think if we had more compute, we would train for longer. I think we’d ultimately see that as something we would want to do. Especially with big models, you need more compute, because we talked about having more parameters and we talked about knowledge. Essentially, there’s a ratio where big models can absorb more from data, and then you get more benefit out of this.

Any logarithmic graph in your mind is like a small model will level off sooner if you’re measuring tons of tokens, and bigger models need more. But mostly, we aren’t training that big of models right now at AI2, and getting the highest-quality data we can is the natural starting point.

Lex Fridman

Is there something to be said about the topic of data quality? Is there some low-hanging fruit there still where the quality could be improved?

Sebastian Raschka

It’s like turning the crank. Historically, in the open, there’s been a canonical best pre-training dataset that has moved around between whoever has the most recent one or the best recent effort. AI2’s Dolma was very early with the first OLMo, and Hugging Face had FineWeb. There’s a DCLM project, which stands for DataComp for Language Models. There’s been DataComp for other machine learning projects, and they had a very strong dataset.

A lot of it is that the internet is becoming fairly closed off, so we have Common Crawl, which is hundreds of trillions of tokens, and you filter it. It looks like scientific work, where you’re training classifiers and making decisions based on how you prune down this dataset into the highest-quality stuff and the stuff that suits your tasks.

Previously, language models were tested a lot more on knowledge and conversational things, but now they’re expected to do math and code. To train a reasoning model, you need to remix your whole dataset. There are a lot of wonderful scientific methods here where you can take your gigantic dataset, sample really tiny things from different sources, such as GitHub, Stack Exchange, Reddit, and Wikipedia. You can sample small things from them, train small models on each of these mixes, and measure their performance on your evaluations.

You can just do basic linear regression, and it’s like, “Here’s your optimal dataset.” But if your evaluations change, your dataset changes a lot. A lot of OLMo 3 was new sources for reasoning to be better at math and code, and then you do this mixing procedure and it gives you the answer.

I think that’s happened at labs this year. There are new hot things, whether it’s coding environments or web navigation, and you need to bring in new data and change your whole pre-training so that your post-training can work better. That’s the constant evolution and the redetermining of what they care about for their models.

Lex Fridman

Are there fun anecdotes about what sources of data are particularly high quality that we wouldn’t expect? You mentioned Reddit sometimes can be a source.

Sebastian Raschka

Reddit was very useful. I think PDFs are definitely one.

Lex Fridman

Oh, especially arXiv.

Sebastian Raschka

Yeah. AI2 has run Semantic Scholar for a long time, which you can say is a competitor to Google Scholar with a lot more features. To do this, AI2 has found and scraped a lot of PDFs for openly accessible papers that might not be behind the paywall of a certain publisher. So, truly open scientific PDFs. If you sit on all of these and process them, you can get value out of them.

A lot of that style of work was done by the frontier labs much earlier. You just need to have a pretty skilled researcher who understands how things change models. They bring it in, clean it, and it’s a lot of labor. When frontier labs scale research, a lot more goes into data.

If you join a frontier lab and you want to have an impact, the best way to do it is just find new data that’s better. The fancy, glamorous algorithmic things, like figuring out how to make o1, are the sexiest thought of a scientist. It’s like, “Oh, I figured out how to scale RL.” There’s a group that did that, but most of the contribution is like—

Lex Fridman

On the dataset.

Nathan Lambert

“I’m going to make the data better,” or, “I’m going to make the infrastructure better so everyone on my team can run experiments 5% faster.”

Lex Fridman

At the same time, I think it’s also one of the most closely guarded secrets: what your training data is, for legal reasons. I think a lot of work also goes into hiding what your training data was, essentially—training the model not to give away the sources because you have legal reasons.

Nathan Lambert

The other thing, to be complete, is that some people are trying to train on only licensed data, whereas Common Crawl is a scrape of the whole internet. If I host multiple websites, I’m happy to have them train language models, but I’m not explicitly licensing what governs it. Therefore, Common Crawl is largely unlicensed, which means that your consent really hasn’t been provided for how to use the data.

There’s another idea where you can train language models only on data that has been licensed explicitly, so that the governing contract is provided. I’m not sure if Apertus is the copyright thing or the license thing. I know that the reason they did it was for EU compliance, where they wanted to make sure that their model fit one of those checks.

Lex Fridman

On that note, there’s also the distinction in licensing. Some people just purchase the license. Let’s say they buy an Amazon Kindle book or a Manning book and then use that in training. That is a gray zone, because you paid for the content and you might want to train on it. But then there are also restrictions where even that shouldn’t be allowed. That is where it gets a bit fuzzy.

I think that is right now still a hot topic. Big companies like OpenAI approached private companies for their proprietary data, and private companies have become more and more protective of their data because they know, “Okay, this is going to be my moat in a few years.”

I do think that’s the interesting question: if LLMs become more commoditized—and I think a lot of people will learn about LLMs—there will be a lot more people able to train LLMs. Of course, there are infrastructure challenges. But if you think of big industries like pharmaceuticals, law, and finance, I do think they will, at some point, hire people from other frontier labs to build their in-house models on their proprietary data, which will then be another unlock with pre-training that is currently not there.

Because even if you wanted to, you can’t get that data. You can’t get access to clinical trials most of the time and these types of things. So I do think scaling, in that sense, might still be pretty much alive if you also look at domain-specific applications, because right now, this year, we’re still just looking at general-purpose LLMs on ChatGPT, Anthropic, and so forth. They’re just general-purpose. They’re not even scratching the surface of what an LLM can do if it is really specifically trained and designed for a specific task.

Nathan Lambert

I think on the data thing—this is one of the things that happened in 2025, and we totally forget it—is that Anthropic lost in court and owed $1.5 billion to authors. Anthropic, I think, bought thousands of books and scanned them and was cleared legally for that because they bought the books, and that is going through the system. On the other side, they also torrented some books, and I think this torrenting was the path where the court said that they were culpable and had to pay these billions of dollars to authors, which is just such a mind-boggling lawsuit that kind of just came and went.

Lex Fridman

That is so much money from the VC ecosystem. These are court cases that will define the future of human civilization, because it’s clear that data drives a lot of this, and there’s this very complicated human tension.

I mean, you can empathize. You’re both authors. There’s some degree to which you put your heart and soul and your sweat and tears into the writing that you do. It feels a little bit like theft for somebody to train on your data without giving you credit.

Sebastian Raschka

And there are, like Nathan said, also 2 layers to it. Someone might buy the book and then train on it, which could be argued fair or not fair, but then there are the straight-up companies who use pirated books where they’re not even compensating the author. That is, I think, where people got a bit angry about it specifically.

Lex Fridman

Yeah, but there has to be some kind of compensation scheme. This is moving towards something like what Spotify streaming originally did for music. What does that compensation look like? You have to define those kinds of models. You have to think through all of that.

One other thing I think people are generally curious about, and I’d love to get your thoughts: as LLMs are used more and more, if you look at even arXiv and GitHub, more and more of the data is generated by LLMs. What do you do in that kind of world? How big of a problem is that?

Nathan Lambert

The largest problem is the infrastructure and systems, but from an AI point of view, it’s kind of inevitable.

Lex Fridman

So it’s basically LLM-generated data that’s curated by humans, essentially, right?

Nathan Lambert

Yes, and I think that a lot of open-source contributors are legitimately burning out. If you have a popular open-source repo, somebody’s like, “Oh, I want to do open-source AI. It’s good for my career,” and they just vibe-code something and throw it in. You might get more of this than I do.

Sebastian Raschka

Yeah, so I have actually a case study here. I have a repository called mlxtend that I developed as a student around 10 years ago, and it is a reasonably popular library still for certain algorithms, especially frequent pattern-mining stuff. There were recently 2 or 3 people who submitted a lot of pull requests in a very short amount of time.

Nathan Lambert

I do think LLMs have been involved in submitting these PRs. For me, as the maintainer, there are 2 things. First, I'm a bit overwhelmed. I don't have time to read through it because, especially as an older library, that is not a priority for me. At the same time, I also appreciate it because I think something people forget is that it's not just using the LLM. There's still a human layer that verifies something, and that is, in a sense, also how data is labeled, right?

One of the most expensive things is getting labeled data for RLHF phases. This is like that, where it goes through phases, and then you actually get higher-quality data out of it. So I don't mind it, in a sense. It can feel overwhelming, but I do think there is also value in it.

Lex Fridman

It feels like there's a fundamental difference between raw LLM-generated data and LLM-generated data with a human in the loop who does some kind of verification, even if that verification is a small percentage of the lines of code.

Nathan Lambert

I think this goes with anything where people sometimes think, "Oh, yeah, I can just use an LLM to learn about XYZ," which is true. You can, but there might be an expert who has used an LLM to write specific code. There is this human work that went into it to make it nice, throwing out the not-so-nice parts to pre-digest it for you, and that saves you time. I think that's the value-add, where you have someone filtering things or even using the LLMs correctly. This is still labor that you get for free.

For example, if you read a Substack article, I could maybe ask an LLM to give me opinions on that, but I wouldn't even know what to ask. And I think there is still value in reading that article compared to me going to the LLM because you are the expert. You select what knowledge is actually spot-on, should be included, and you give me this executive summary. This is a huge value-add because now I don't have to waste 3 to 5 hours going through this myself, maybe getting some incorrect information, and so on. I think that's also where the future still is for writers, even though there are LLMs that can save you time.

Lex Fridman

It's fascinating to watch. I'm sure you guys do this, but for me, I look at the difference between a summary and the original content. Even if it's a page-long summary of page-long content, it's interesting to see how an LLM-based summary takes the edge off. What is the signal it removes from the thing?

Nathan Lambert

The voice is what I talk about a lot.

Lex Fridman

Voice? I would love to hear what you mean by voice, but sometimes there are literally insights. By removing an insight, you're changing the meaning of the thing. So I'm continuously disappointed by how bad LLMs are at really getting to the core insights, which is what a great summary does. Yet even if I have extremely elaborate prompts where I'm really trying to dig for the insights, it's still not quite there. That's a whole deep philosophical question about what human knowledge and wisdom are, and what it means to be insightful, and so on. But when you talk about the voice, what do you mean?

Nathan Lambert

When I write, I think a lot of what I'm trying to do is take what you think as a researcher, which is very raw. A researcher is trying to encapsulate an idea at the frontier of their understanding, and they're trying to put what is a feeling into words. I try to do this in my writing, which makes it come across as raw but also high-information, in a way that some people will get it and some won't. That's the nature of research.

These language models don't do this well. They're all trained with Reinforcement Learning from Human Feedback, which takes feedback from many people and averages how the model behaves from this. I think it's going to be hard for a model to be very incisive when there's that sort of filter. This is a fundamental problem for researchers in RLHF. It provides so much utility in making the models better, but the problem formulation has this knot in it that you can't get past.

These language models don't have this prior in their deep representation that they're trying to get at. I don't think it's impossible. There are stories of models that really shock people. I would love to have tried Bing Sydney. Did that have more voice? It would so often go off the rails, which is obviously a scary thing—telling a reporter to leave his wife is a crazy model to potentially put into general adoption. But that's a trade-off: Is this RLHF process in some ways adding limitations?

Lex Fridman

That's a terrifying place to be as one of these frontier labs and companies, because millions of people are using them.

Nathan Lambert

There was a lot of backlash last year with GPT-4o getting removed, and I've personally never used the model, but I've talked to people at OpenAI, and they get emails from users who might be detecting subtle differences in the deployments in the middle of the night. They email them, saying, "My friend is different." They find these employees' email addresses and send them things because they're so attached to this set of model weights and configuration that is deployed to users.

We see this with TikTok. You open it—I don't use TikTok, but supposedly, in 5 minutes, the algorithm gets you. It's locked in. Those are machine learning models making recommendations. I think there are ways you can do this. Within 5 minutes of chatting with it, the model just gets you. That is something that people aren't really ready for. Don't give that to kids, at least until we know what's happening.

Lex Fridman

But there's also going to be this mechanism. What's going to happen with these LLMs as they're used more and more? Unfortunately, the nature of the human condition is such that people commit suicide. What journalists will do is report extensively on the people who commit suicide, and they would very likely link it to the LLMs because they have that data about the conversations.

If you're really struggling in your life, if you're depressed, if you're thinking about suicide, you're probably going to talk to LLMs about it. Journalists will say, "Well, the suicide was committed because of the LLM." That's going to lead to companies, because of legal issues and so on, taking more and more of the edge off the LLM. It's going to be as generic as possible.

It's so difficult to operate in this space because you don't want an LLM to cause harm to humans at that level, but this is also the nature of the human experience: to have a rich conversation, a fulfilling conversation, one that challenges you and from which you grow. You need that edge. That's something extremely difficult for AI researchers on the RLHF front to actually solve, because you're dealing with the human condition.

Nathan Lambert

A lot of researchers at these companies are so well-motivated, and definitely Anthropic and OpenAI culturally want to do good for the world. It's such a difficult problem that I'm thinking, "Ooh, I don't want to work on this," because, on the one hand, a lot of people see AI as a health ally, as somebody they can talk to about their health confidentially. But then it bleeds all the way into talking about mental health, where it's heartbreaking that this will be the thing where somebody goes over the edge, but other people might be saved. I'm thinking, "I don't know."

As a researcher, I don't want to train image-generation models and release them openly because I don't want to enable somebody to have a tool on their laptop that can harm other people. I don't have the infrastructure in my company to do that safely. But there are a lot of areas like this that need people who will approach them with complexity and conviction. It's just such a hard problem.

Lex Fridman

But also, we as a society, as users of these technologies, need to make sure that we're having the complicated conversation about it versus just fearmongering—that Big Tech is causing harm to humans or stealing your data. It's more complicated than that. And you're right. There are a very large number of people inside these companies, many of whom I know, who deeply care about helping people.

They are considering the full human experience of people from across the world, not just Silicon Valley. They're considering people across the United States and the world, and what their needs are. It's really difficult to design one system that is able to help all these different kinds of people across different age groups, cultures, and mental states.

John Schulman

I wish that the timing of AI were different relative to the relationship between Big Tech and the average person. Big Tech's reputation was so low, and AI is so expensive that it's inevitably going to be a Big Tech thing. It takes so many resources, and people say the US is "betting the economy on AI" with this build-out. To have these be intertwined at the same time makes for such a hard communication environment. It would be good for me to go talk to more people in the world who hate Big Tech and see AI as a continuation of this.

Lex Fridman

And one of the things you recommend, one of the antidotes that you talk about, is to find agency in this system. As opposed to sitting back in a powerless way and consuming the AI slop as it rapidly takes over the internet, find agency by using AI to build stuff, build apps. First, that actually helps you build intuition, and second, it's empowering because you can understand how it works and what the weaknesses are.

It gives your voice power to say, "This is a bad use of the technology, and this is a good use." You're more plugged into the system than, so you can understand it better and steer it better as a consumer.

Sebastian Raschka

I think that's a good point you brought up about agency. Instead of ignoring it and saying, "Okay, I'm not going to use it," I think it's probably healthier in the long term to say, "Okay, it's out there. I can't put it back," when they came out.

Lex Fridman

How do I make the best use of it, and how does it help me to level myself up? The one thing I worry about here, though, is that if you fully use it for something you love to do, the thing you love to do is no longer there. And that could potentially, I feel, lead to burnout.

For example, if I use an LLM to do all my coding for me, now there's no coding. I'm just managing something that is coding for me. Two years later, let's say, if I just do that 8 hours a day, having something code for me, do I still feel fulfilled? Is this hurting me in terms of being excited about my job, excited about what I'm doing? Am I still proud to build something?

John Schulman

On that topic of enjoyment, it's quite interesting. We should throw this in there: there's this recent survey of about 791 professional developers, meaning people with 10-plus years of experience.

Lex Fridman

That's a long time. As a junior developer?

Lex Fridman

Yeah, in this day and age. There are also many surprising findings. They break it down by junior and senior developers, but it shows that both junior and senior developers use AI-generated code in code they ship. So this is not just for fun or intermediate learning things. This is code they ship.

About 25%—most of them use around 50% or more. What's interesting is that, in the category of people whose shipped code is over 50% AI-generated, senior developers are much more likely to do so. But you don't want AI to take away the thing you love. I think this speaks to my experience with these results I'm about to say.

Together, about 80% of people find it either somewhat more enjoyable or significantly more enjoyable to use AI as part of their work.

Lex Fridman

I think it depends on the task. From my personal usage, for example, I have a website where I sometimes tweak things. I personally don't enjoy this. So, in that sense, if AI can help me implement something on my website, I'm all for it. It's great.

But at the same time, when I solve a complex problem—if there's a bug, and I hunt for this bug and find it, it's the best feeling in the world. You feel great. But if you don't even think about the bug and just go directly to the LLM, you never have this kind of feeling, right?

There could be a middle ground where you try yourself, you can't find it, you use the LLM, and then you don't get frustrated because it helps you and you move on to something that you enjoy. Looking at these statistics, I think what's not factored in is that it's averaging over all the different scenarios. We don't know if it's for the core task or if it's for something mundane that people wouldn't have enjoyed otherwise.

In a sense, AI is really great for doing mundane things that take a lot of work. For example, my wife the other day—she has a podcast for book discussions, a book club, and she was transferring the show notes from Spotify to YouTube. Then the links somehow broke.

In some episodes, because there are so many books, she had around 100 links, and it would have been really painful to go in there and fix each link manually. So I suggested, “Hey, let's try ChatGPT.” We copied the text into ChatGPT, and it fixed them. Instead of spending 2 hours going from link to link and fixing them, it made that type of work much more seamless. I think everyone has a use case where AI is useful for something that would be really boring and mundane.

Nathan Lambert

For me personally, since we're talking about coding and you mentioned debugging, the source of enjoyment for me—more with Cursor than Claude Code—is that I have a friend. I have a pair programmer. It's less lonely.

You made debugging sound like this great joy. No, I would say debugging is like a drink of water after you've been going through a desert for days. You skip the whole desert part where you're suffering. Sometimes it's nice to have a friend who can't really find the bug but can give you some intuition about the code. Together, you're going through the desert and finding that drink of water.

At least for me, maybe it speaks to the loneliness of the programming experience. That is a source of joy.

Lex Fridman

It's maybe also related to delayed gratification. I'm a person who, even as a kid, liked the idea of Christmas presents—having them, getting them—better than actually receiving the presents. I would look forward to the day I got the presents, but then it was over and I was disappointed.

Maybe it's the same with food. I think food tastes better when you're really hungry. You're right, with debugging, it is not always great. It's often frustrating, but if you can solve it, then it's great.

There's a sweet Goldilocks zone. If it's too hard, then it's just wasting your time. But I think another challenge is: How will people learn? We looked at the chart and saw that more senior developers are shipping more AI-generated code than junior ones. It's very interesting, because intuitively you would think it's the junior developers, since they don't know how to do the thing yet, and so they use AI to do that thing.

It could mean the AI is not good enough yet to solve that task, but it could also mean experts are more effective at using it. They know how to use it better, review the code, and then trust the code more. One issue for society in the future will be: How do you become an expert if you never try to do the thing yourself?

One way I always learned is by trying things myself. If you look at math textbooks and the solutions, you learn something, but you learn it better if you try first. Then you appreciate the solution differently because you know how to put it into your mental framework.

If LLMs are here all the time, would you actually go to the length of struggling? Would you be willing to struggle? Struggle is not nice, right? But if you use the LLM to do everything, at some point you will never really take the next step, and then you may not get that unlock that you would get as an expert using an LLM.

I think there's a Goldilocks sweet spot. Maybe the trick here is to make dedicated offline time where you study for 2 hours a day, and the rest of the day use LLMs. But I think it's important also for people to still invest in themselves, in my opinion, and not just LLM everything.

Nathan Lambert

Yeah, as a civilization, we each individually have to find that Goldilocks zone—in the programming context, as developers. We've had this fascinating conversation that started with pre-training and mid-training. Let's get to post-training. There are a lot of fun things in post-training.

So, what are some of the interesting ideas in post-training?

The biggest one from 2025 is learning this reinforcement learning with verifiable rewards. You can scale up the training there, which means doing a lot of this iterative generate-grade loop, and that lets the models learn interesting behaviors on both the tool-use and software sides.

This could be searching, running commands on their own, and seeing the outputs. That training also enables this inference-time scaling very nicely. It turned out that this paradigm was very nicely linked, where this kind of RL training enables inference-time scaling.

Inference-time scaling could have been found in different ways, so it was a perfect storm where the models changed a lot, and the way that they're trained is a major factor in doing so. This has changed how people approach post-training dramatically.

Lex Fridman

Can you describe RLVR, popularized by DeepSeek-R1? Can you describe how it works?

Nathan Lambert

Yeah. Fun fact: I was on the team that came up with the term RLVR, which is from our Tülu 3 work before DeepSeek. We don't take a lot of credit for being the people who popularized scaling RL, but one fun thing academics get, as an aside, is the ability to name and influence the discourse, because the closed labs can only say so much.

One of the things you can do as an academic is frame things in a way that ends up being described as a community coming together around this RLVR term, which is very fun. You might not have the compute to train the model, but you can frame things in a way that ends up influencing the community. DeepSeek was the team that achieved the training breakthrough: They scaled the reinforcement learning.

You have the model generate answers and then grade the completion to see if it was right, and that accuracy is your reward for reinforcement learning. Reinforcement learning is classically an agent that acts in an environment, and the environment gives it a state and a reward back, and you try to maximize this reward.

In the case of language models, the reward is normally accuracy on a set of verifiable tasks, whether they're math problems or coding tasks. It starts to get blurry with things like factual domains. That is also, in some ways, verifiable, as are constraints in your instruction, like “Respond only with words that start with A.”

All of these things are verifiable in some way. The core idea is that you find a lot more of these problems that are verifiable and let the model try them many times while taking these RL steps, these RL gradient updates.

The infrastructure evolved from reinforcement learning from human feedback, where, in that era, the score they were trying to optimize was a learned reward model of human preferences. You changed the problem domains, and that let the optimization go on to much bigger scales, which kickstarted a major change in what the models can do and how people use them.

Lex Fridman

What kind of domains is RLVR amenable to?

Nathan Lambert

Math and code are the famous ones. Then there's a lot of work on what are called rubrics, which is related to a phrase people might have heard: LLM-as-a-judge.

For each problem, I'll have a set of problems in my dataset. I will then have an LLM and ask it, “What would a good answer to this problem look like?” Then you could try the problem over and over again and assign a score based on this rubric.

That's not necessarily verifiable like math and code domains, but this rubric idea and other scientific problems that might be a little bit more vague are where the attention is. They're trying to push this set of methods into these more open-ended domains so the models can learn a lot more.

Lex Fridman

I think that's called reinforcement learning from AI feedback, right?

Nathan Lambert

That's the older term for it, coined in Anthropic's Constitutional AI paper. A lot of these things come in cycles.

Lex Fridman

Also, just one step back for RLVR. I think the interesting thing here is that you ask the LLM a math question, and then you know the correct answer. You let the LLM, as you said, figure it out. You don't constrain it much. There are some constraints, like, “Use the same language. Don't switch between Spanish and English.” But you're pretty much hands-off. You only give it the question and the answer, and then the LLM has the task of arriving at the right answer.

The beautiful thing here is what happens in practice: the LLM will do a step-by-step description, like a student or a mathematician deriving the solution. It will use those steps, and that helps the model improve its own accuracy. Then, like you said, there's inference scaling. Inference scaling loosely means spending more compute during inference, and here the inference scaling is that the model would use more tokens.

In the DeepSeek-R1 paper, they showed that the longer they train the model, the longer the responses are. They grow over time. The model uses more tokens, so it becomes more expensive. It becomes expensive for simple tasks, but these explanations help with accuracy.

There are also papers showing that what the model explains does not necessarily have to be correct. It may even be unrelated to the answer, but for some reason, it still helps the model that it is explaining. Again, I don't want to anthropomorphize these LLMs, but it's kind of like how we humans operate. If there's a complex math problem in a math class, you usually have a piece of paper and do it step by step. You cross things out.

The model also self-corrects, and that was, I think, the aha moment in the DeepSeek-R1 paper. They called it the aha moment because the model itself recognized that it had made a mistake and then said, “Ah, I did something wrong. Let me try again.” I think it's so cool that this falls out of just giving it the correct answer and having it figure out how to do it—that it kind of does, in a sense, what a human would do. Although LLMs don't think like humans, it's an interesting coincidence.

The other nice side effect is that it's often great for us humans to see these steps. It builds trust, but it also helps us learn and double-check things.

Nathan Lambert

There's a lot in here. I think some of the debate—there's been a lot of debate this year about whether the aha moments in language models like these are kind of fake. In pre-training, you've essentially seen the whole internet, so you've definitely seen people explaining their work, even verbally, like in a transcript of a math lecture: “You try this. Oh, I messed this up.” What RLVR is very good at doing is amplifying these behaviors because they're useful in enabling the model to think longer and check its work.

I agree that it's very beautiful that this training teaches the model to amplify these behaviors in a way that is so useful for making the final answers better.

I can also give you a hands-on example. I was training the Qwen 3 base model with RLVR on MATH-500. The base model had an accuracy of about 15%. After just 50 steps—just a few minutes with RLVR—the model went from 15% to 50% accuracy. You can't tell me it's learning anything fundamentally about math in 50 steps.

Lex Fridman

The Qwen example is weird because there have been 2 papers this year, one of which I was on, about data contamination in Qwen. Specifically, they train on a lot of data in this special mid-training phase that we should take a minute to discuss, because it's weird: they train on problems that are almost identical to MATH.

Nathan Lambert

Exactly. So you can see that, basically, RL isn't teaching the model any new knowledge about math. You can't do that in 50 steps. The knowledge is already there in pre-training; you're just unlocking it.

Lex Fridman

I still disagree with the premise because there are a lot of weird complexities that you can't prove. One of the things that points to this weirdness is that if you take the Qwen3 so-called base model, you could Google “math dataset, Hugging Face” and take a problem. What you can do is put it into Qwen3 Base.

All these math problems have words. It might be, “Alice has 5 apples and gives 3 to someone,” and there are these word problems. With these Qwen-based models, what makes people suspicious is that if you change the numbers but keep the words, Qwen will, without tools, produce a very high-precision decimal representation of the answer.

That means that at some point, it was shown problems that were almost identical to the test set, and it was using tools to get a very high-precision answer. But a language model without tools would never actually have this. So it's been a big debate in the research community: how much can you believe these reinforcement learning papers that are training on Qwen and measuring specifically on this math benchmark, when there have been multiple papers discussing contamination?

I think this is what caused RLVR to have a reputation for being about formatting: you can get these gains so quickly, so it must already be in the model. But there's a lot of complexity here that we don't understand. It's not really a controlled experiment, so we don't really know.

Nathan Lambert

But if it weren't true, I would say distillation wouldn't work, right? I mean, distillation can work to some extent, but the thing is, I think that's the biggest problem: I research this contamination because we don't know what's in the data. Unless you have a new dataset, it's really impossible to know.

The same goes for the math dataset you mentioned, where you have a question, an answer, and an explanation. Even something simpler, like MMLU, which is a multiple-choice benchmark: if you just change the format slightly—if you use a dot instead of a parenthesis, for example—the model's accuracy will differ vastly.

Lex Fridman

I think that could be a model issue rather than a general issue.

Nathan Lambert

It's not even malicious on the part of the developers of the LLM. It's not like, “Hey, we want to cheat on that benchmark.” The model has seen something at some point. I think the only fair way to evaluate an LLM is to have a new benchmark created after the cutoff date for when the LLM was deployed.

Lex Fridman

Can we lay out the recipe for all the things that go into post-training? You mentioned that RLVR was a really exciting and effective thing. Maybe we should elaborate. RLHF still has a really important role to play. What other ideas are there in post-training?

Nathan Lambert

I think you can take this in order. You could view it in terms of what made o1, this first reasoning model, possible, or what will make the latest model possible. There are similar interventions at these stages, where you start with mid-training.

The thing that is rumored to enable o1 and similar models is really careful data curation, where you're providing a broad set of what are called reasoning traces. That's just the model generating words in a forward process that reflects breaking down a problem into intermediate steps and trying to solve it. At mid-training, you need to have data that is similar to this so that when you move into post-training, primarily with these verifiable rewards, the model can learn.

What's happening today is that you're figuring out which problems to give the model, how long you can train it for, and how much inference you can enable the model to use when solving these verifiable problems. As models get better, certain problems are no longer useful because the model will solve them 100% of the time, and therefore there's very little signal in them.

If we look at the GRPO equation, this is famous for it because, essentially, the reward given to the agent is based on how good a given action—an action is a completion—is relative to the other answers to that same problem. So if all the problems get the same answer, there's no signal in these types of algorithms.

What they're doing is finding harder problems, which is why you hear about scientific domains, where it's so hard to get anything right. If you have a lab or something, it just generates so many tokens, or much harder software problems. The frontier models are all pushing into these harder domains when they can train on more problems, and the model will learn more skills at once.

The RLHF link to this is that RLHF has been, and still is, kind of like the finishing touch on the models. It makes the models more useful by improving their organization, style, and tone. There are different things that resonate with different audiences. Some people like a really quirky model, and RLHF could be good at enabling that personality. Some people hate the markdown bulleted-list format that the models use, but it's actually really good for quickly parsing information.

In RLHF, this human-feedback stage is really great for putting all of this into the model at the end of the day. It's what made ChatGPT so magical for people. That use has actually remained fairly stable. This formatting can also help models get better at math problems, for example.

The border between style and formatting and the method you use to answer a problem is actually all very closely linked when you're training these models. That's why RLHF can still make a model better at math, but these verifiable domains are a much more direct process for doing this because it makes more sense with the problem formulation. That's why it all ends up forming together.

But to summarize, mid-training is giving the model the skills it needs to then learn. RL with verifiable rewards is letting the model try a lot of times, putting a lot of compute into trial-and-error learning across hard problems. And then RLHF would be like finishing the model, making it easy to use, and kind of rounding the model out.

Lex Fridman

Can you comment on the amount of compute required for RLVR?

Nathan Lambert

It’s only gone up and up. I think Ilya Sutskever was famous for saying they use a similar amount of compute for pre-training and post-training. Back to the scaling discussion, they involve very different hardware for scaling. Pre-training is very compute-bound, which is like this FLOPs discussion, which is just how many matrix multiplications you can get through at once.

Because with RL you’re generating these answers and trying the model in real-world environments, it ends up being much more memory-bound because you’re generating long sequences. The attention mechanisms have this behavior where you get a quadratic increase in memory as you’re getting to longer sequences. So the compute becomes very different.

In pre-training, if we go back to the Biden administration’s executive order, we would talk about 10²⁵ FLOPs to train a model. If you’re using FLOPs in post-training, it’s a lot weirder because the reality is just: how many hours are you allocating? How many GPUs? And I think in terms of time, the RL compute is getting much closer because you just can’t put it all into one system.

Pre-training is so computationally dense that all the GPUs are talking to each other, and it’s extremely efficient, whereas RL has all these moving parts and can take a long time to generate a sequence of 100,000 tokens. If you think about GPT-5.2 Pro taking an hour, it’s like, what if your training run has a sample that takes an hour and you have to make sure that’s handled efficiently? So I think in GPU hours, or just wall-clock hours, the RL runs are probably approaching the same number of days as pre-training, but they probably aren’t using as many GPUs at the same time.

There are rules of thumb where, in labs, you don’t want your pre-training runs to last more than a month because they fail catastrophically. If you’re planning a huge cluster to be held for 2 months and then it fails on day 50, the opportunity costs are just so big. So people don’t want to put all their eggs in one basket.

GPT-4 was the ultimate YOLO run, and nobody ever wanted to do it before. It took 3 months to train, and everybody was shocked that it worked. I think people are a little bit more cautious and incremental now.

Lex Fridman

So RLVR is more, let’s say, unlimited in how much you can train and still get a benefit, whereas with RLHF, because it’s preference tuning, you reach a certain point where it doesn’t really make sense to spend more RL budget on that.

Nathan Lambert

So just to take a step back with preference tuning, there are multiple people who can give multiple explanations for the same thing, and they can both be correct, but at some point you learn a certain style and it doesn’t make sense to iterate on it. My favorite example is: if relatives ask me what laptop they should buy, I give them an explanation or ask, “What is your use case?” They might, for example, prioritize battery life and storage. Other people like us, for example, would prioritize RAM and compute. Both answers are correct, but different people require different answers.

With preference tuning, you’re trying to average somehow. You’re asking the data labelers to give you not the right answer, but the preferred answer, and then you train on that. But at some point you learn that average preferred answer. There’s no reason to keep training longer on it because it’s just a style, whereas with RLVR, you let the model solve more and more complex, difficult problems. So I think it makes more sense to allocate more budget long-term to RLVR.

Sebastian Raschka

Also, right now we are in an RLVR 1.0 blend, where it’s still the simple thing where we have a question and answer, but we don’t do anything with the stuff in between. There were multiple research papers, including papers by Google, on process reward models that also give scores for the explanation—how correct the explanation is. And I think that will be the next thing, let’s say RLVR 2.0 for this year, focusing in between the question and answer, like how to leverage that information—the explanation—to help it get better accuracy.

So that’s one angle. There was a DeepSeekMath-V2 paper where they also had interesting inference scaling. First, they had developed models that grade themselves—a separate model. And I think that will be one aspect. And the other, as Nathan mentioned, will be RLVR branching into other domains.

Lex Fridman

The thing people are excited about is value functions, which is pretty similar. Process reward models assign how good something is at each intermediate step in a reasoning process, whereas value functions apply value to every token the language model generates. Both of these have been largely unproven in the language-modeling and reasoning-model era.

People are more optimistic about value functions for whatever reason now. I think process reward models were tried a lot more in this pre-o1, pre-reasoning-model era, and a lot of people had a lot of headaches with them. Value models have a very deep history in reinforcement learning. One of the first things core to the existence of deep reinforcement learning is training value models.

So right now, people are excited about trying value models, but there’s very little proof. And there are negative examples in trying to scale up process reward models. These things don’t always hold in the future.

We came to this discussion by talking about scaling. The simple way to summarize what you’re saying is that you don’t want to do too much RLHF, where the signal doesn’t scale. People have worked on RLHF for language models for years, especially with intense interest after ChatGPT. The first release of a reasoning model trained with RLVR, OpenAI’s o1, had a scaling plot where, if you increase training compute logarithmically, you get a linear increase in evaluations.

This has been reproduced multiple times. DeepSeek had a plot like this. But there’s no scaling law for RLHF where, if you increase the compute logarithmically, you get performance. In fact, the seminal scaling paper for RLHF is “Scaling Laws for Reward Model Overoptimization.” So that’s a big line to draw with RLVR and the methods we have now.

In the future, they will follow this scaling paradigm: you can let the best runs run for an extra 10x and get performance, but you can’t do this with RLHF. And that is just going to be field-defining in how people approach them. While I’m a shill for people to academically do RLHF, to do the best RLHF, you might not need the extra 10x or 100x of compute, but to do the best RLVR, you do.

I think there’s a seminal paper from a Meta internship. It’s called something like “The Art of Scaling Reinforcement Learning with Verifiable Rewards.” The framework they describe is ScaleRL. Their incremental experiment was like 10,000 V100 hours, which is like thousands or tens of thousands of dollars per experiment. They do a lot of them, and this cost is not accessible to the average academic, which is a hard equilibrium where it’s trying to figure out how to learn from each community.

Lex Fridman

I was wondering if we could take a bit of a tangent and talk about education and learning. If you’re someone listening to this who’s a smart person interested in programming and AI, I presume building something from scratch is a good beginning. So can you take me through what you would recommend people do?

Sebastian Raschka

I would personally start, as you said, by implementing a simple model from scratch that you can run on your computer. The goal is not, when you build a model from scratch, to have something for everyday use. It’s not going to be your personal assistant replacing an existing open-weight model or ChatGPT. It’s to see what exactly goes into the LLM, what comes out, and how the pre-training works on your own computer, preferably.

Then you learn about pre-training, supervised fine-tuning, and the attention mechanism. You get a solid understanding of how things work, but at some point you reach a limit because small models can only do so much. The problem with learning about LLMs at scale is that it’s exponentially more complex to make a larger model, because the model isn’t just larger—you have to shard your parameters across multiple GPUs.

Even for the KV cache, there are multiple ways to implement it. One way is to understand how it works: you grow the cache step by step by concatenating lists, but then that wouldn’t be optimal on GPUs. You would pre-allocate a tensor and then fill it in. But that adds another 20 or 30 lines of code.

And for each thing, you add so much code. The goal with the book is basically to understand how the LLM works. It’s not going to be a production-level LLM, but once you have that, you can understand a production-level LLM.

Lex Fridman

So you’re trying to always build an LLM that’s going to fit on one GPU?

Sebastian Raschka

Yes. Most of them do. I have some bonus materials on some MoE models. One or two of them may require multiple GPUs, but the goal is to have it on one GPU.

And the beautiful thing is, you can self-verify. It’s almost like RLVR. When you code these from scratch, you can take an existing model from the Hugging Face Transformers library. The library is great, but if you want to learn about LLMs, it’s not the best place to start because the code is so complex to fit so many use cases.

Because people use it in production, it has to be really sophisticated, really intertwined, and hard to read. It’s not linear.

Lex Fridman

It started as a fine-tuning library, and then it grew to be the standard representation of every model architecture.

Hugging Face is the default place to get a model, and Transformers is the software. It enables people to easily load a model and do something basic with it.

Sebastian Raschka

All frontier labs that have open-weight models have a Transformers version of them, from DeepSeek to gpt-oss-120b. That’s the canonical weight format you can load. But even Transformers, the library, is not used in production. People use SGLang or vLLM, and that adds another layer of complexity.

Lex Fridman

We should say that the Transformers library has around 400 models.

Sebastian Raschka

So it’s the one library that tries to implement a lot of LLMs, and so you have a huge codebase, basically. It’s huge.

Lex Fridman

That’s crazy.

Sebastian Raschka

Hundreds of thousands of lines of code. Understanding the part you want to understand is like finding the needle in the haystack. But what’s beautiful is that you have a working implementation, so you can work backward.

What I would recommend doing, and what I also do, is, if I want to understand, for example, how OLMo is implemented, I look at the weights in the model hub and the config file. Then you can see, “Oh, they used so many layers. They use, let’s say, grouped-query attention or multi-head attention in that case.” You see all the components in a human-readable, 100-line config file.

Then you start, let’s say, with your GPT-2 model and add these things. The cool thing here is that you can then load the pretrained weights and see if they work in your model. You want to match the same output that you get with a Transformer model, and then you can use that basically as a verifiable reward to make your architecture correct.

Sometimes it takes me a day. With OLMo 3, the challenge was RoPE for the position embeddings. They had a YaRN extension, and there was some custom scaling there, and I couldn’t quite match these things. In this struggle, you understand things. At the end, you know you have it correct because you can unit-test it. You can check against the reference implementation. I think that’s one of the best ways to learn, really: to reverse-engineer something.

Nathan Lambert

I think that is something everyone interested in getting into AI today should do. That’s why I liked your book. I came to language models from the RL and robotics field. I had never taken the time to just learn all the fundamentals.

This Transformer architecture is so fundamental, just as deep learning was in the past, and people need to do this. I think where a lot of people get overwhelmed is, “How do I apply this to have an impact or find a career path?”

Because language models make this fundamental stuff so accessible, people with motivation will learn it. Then it’s like, “How do I get cycles on goal to contribute to research?” I’m actually fairly optimistic because the field moves so fast that a lot of times the best people don’t fully solve a problem because there’s a bigger problem to solve that’s very low-hanging fruit, so they move on.

I think that a lot of what I was trying to do in this RLHF book is take post-training techniques and describe how people think about them influencing the model and what people are doing. Then it’s remarkable how many things I think people just stop studying or don’t pursue.

I think people trying to go narrow after doing the fundamentals is good, and then reading the relevant papers and being engaged in the ecosystem. There’s a proximity that random people online have to the leading researchers. No one knows who all the anonymous accounts on X and in machine learning are, but they’re very popular. They could just be random people who study this stuff deeply, especially with the AI tools.

To say, “I don’t understand this; keep digging into it,” is a very useful thing. But there are a lot of research areas that maybe have 3 papers you need to read, and then one of the authors will probably email you back. You have to put a lot of effort into these emails to understand the field.

I think it would easily take a newcomer weeks of work to feel like they can truly grasp a very narrow area. But I think going narrow after you have the fundamentals will be very useful to people because I’ve become very interested in character training, which is how you make the model funny, sarcastic, or serious, and what you do to the data to make this happen.

A student at Oxford reached out to me and said, “Hey, I’m interested in this,” and I advised him. That paper now exists. There are maybe 2 or 3 people in the world who were very interested in this. He’s a PhD student, which gives him an advantage, but for me, that was a topic I was waiting for someone to say, “Hey, I have time to spend cycles on this.”

I’m sure there are a lot more very narrow things where you’re just like, “It doesn’t make sense that there was no answer to this.” I think there’s just so much information coming that people are like, “I can’t grab onto any of this.” But if you just stick in an area, I think there are a lot of interesting things to learn.

Nathan Lambert

Yeah, I think you can’t try to do it all because it would be very overwhelming and you would burn out. For me, for example, I haven’t kept up with computer vision in a long time; I just focused on LLMs. But coming back to your book, I think this is a really great book and a really good bang for the buck because, if you want to learn about RLHF, I wouldn’t go out there and read RLHF papers because you would be spending 2 years.

Some of them contradict. I’ve just edited the book, and there’s no chapter where I had to say, “X papers say one thing and Y papers say another, and we’ll see what comes out to be true.”

Lex Fridman

Just to go through the table of contents, what are some ideas we might have missed in the bigger picture of post-training? First, you did the problem setup, training overview, what preferences are, preference data, and the optimization tools: reward modeling, regularization, instruction tuning, rejection sampling, and reinforcement learning.

Then there’s Constitutional AI and AI feedback, reasoning and inference-time scaling, tool use and function calling, synthetic data and distillation, evaluation, and then an open-questions section: over-optimization, style and information, product UX, character, and post-training. What are some ideas worth mentioning that connect both the educational and the research components?

Nathan Lambert

Character training is interesting because there’s so little on it. We talked about how people engage with these models. We feel good using them because they’re positive, but that can go too far; they can be too positive. Essentially, it’s: How do you change your data and decision-making to make it exactly what you want?

OpenAI has this thing called the Model Spec, which is essentially its internal guideline for what it wants the model to do, and it publishes this for developers. So essentially, you can know what is a failure of OpenAI’s training—where they have the intentions and haven’t met them yet—versus what is something they actually wanted to do that you don’t like.

That transparency is very nice, but all the methods for curating these documents, and how easy it is to follow them, are not very well known. I think the way the book is designed is that the RL chapter is obviously what people want because everybody hears about it with RLVR. It’s the same algorithms and the same math, but you can use it in very different domains.

I think the core of RLHF is how messy preferences are. It’s essentially a rehash of a paper I wrote years ago, but this is essentially the chapter that will tell you why RLHF is never fully solvable. The way that even RL is set up assumes that preferences can be quantified and that multiple preferences can be reduced to single values.

I think it relates in the economics literature to the von Neumann–Morgenstern utility theorem. That is the chapter where all of that philosophical, economic, and psychological context tells you what gets compressed into doing RLHF. Then later in the book, it’s like: You use this RL math to make the number go up.

I think that’s why it’ll be very rewarding for people to do research on, because quantifying preferences is something that humans have designed as a problem in order to make preferences studyable. But there are fundamental debates. An example is that, in a language-model response, you have different things you care about, like accuracy or style.

When you’re collecting the data, they all get compressed into, “I like this more than another.” That is happening, and there’s a lot of research in other areas of the world that goes into how you should actually do this. I think social choice theory is the subfield of economics concerned with how you should aggregate preferences.

I went to a workshop that published a white paper on, “How can you think about using social choice theory for RLHF?” I mostly want people who get excited about the math to come and find things where they could stumble into this broader context.

There’s a fun thing: I just keep a list of all the tech reports of reasoning models I like. In Chapter 14, where there’s a short summary of RLVR, there’s a gigantic table where I list every single reasoning model that I like.

I think in education, a lot of it needs to be, at this point, what I like, because the language models are so good at the math. For example, the famous paper “Direct Preference Optimization,” which is a much simpler way of solving the problem than RL, has derivations in the appendix that skip steps of math.

For this book, I redid the derivations, and I’m like, “What the heck is this log trick that they use to change the math?” But doing it with language models, they’re like, “This is the log trick.” I’m like, “I don’t know if I like this, that the math is so commoditized.”

I think some of the struggle in reading this appendix and following the math is good for learning.

Lex Fridman

Yeah, we're returning to this often on the topic of education. You both have brought up the word “struggle” quite a bit. So there is value. If you're not struggling as part of this process, you're not fully following the proper process for learning, I suppose.

Nathan Lambert

Some providers are working on models for education designed to not give—actually, I haven't used them, but I'd guess they're designed to not give all the information at once and make people work for it. Training models to do this would be a wonderful contribution, where, like all of the stuff in the book, you had to reevaluate every decision for it. It's a great example. There's a chance we work on it at AI2, which I thought would be so fun.

Lex Fridman

It makes sense. I did something like that the other day for video games. Sometimes, for pastime, I play video games. I like video games with puzzles, like Zelda and Metroid. There's this new game where I really got stuck and was okay with it. I don't want to struggle for 2 days, so I used an LLM.

But then you say, “Hey, please don't add spoilers. I'm here and there. What do I have to do next?” You can do the same thing for math, where you say, “Okay, I'm stuck at this point. Don't give me the full solution, but what is something I could try?” You carefully probe it. But the problem here is, I think, it requires discipline.

Many people enjoy math, but there are also a lot of people who need to do it for their homework, and then it's like a shortcut. We could develop an educational LLM, but other LLMs are still there, and there's still a temptation to use the other LLMs.

Nathan Lambert

I think many people in college understand the stuff they're passionate about. They're self-aware, and they understand it shouldn't be easy. I think we just have to develop a good taste—talk about research taste, school taste—about stuff that you should be struggling on and stuff you shouldn't be.

It's tricky, because you don't have good long-term vision. Sometimes you don't have good long-term vision about what would actually be useful to you in your career. But you have to develop that taste, yeah.

Lex Fridman

I was talking to my fiancée or friends about this. There's this brief 10-year window where all of the homework and all of the exams could be digital. Before that, everybody had to do all the exams in blue books because there was no other way.

And now, after AI, everyone's going to need to use blue books and take oral exams because everyone could cheat so easily. It's like this brief generation that had a different education system where everything could be digital, but you still couldn't cheat. And now it's just going back. It's just very funny.

You mention character training. Just zooming out on a more general topic, for that project, how much compute was required? And in general, to contribute as a researcher, are there places where not too much compute is required, where you can actually contribute as an individual researcher?

Nathan Lambert

For the character-training thing, I think this research is built on fine-tuning about 7-billion-parameter models with LoRA, which is essentially only fine-tuning a small subset of the weights of the model. I don't know exactly how many GPU hours that would take.

Lex Fridman

But it's doable.

Nathan Lambert

Not doable for every academic. The situation for some academics is so dire that the only work you can do is inference, where you have closed models or open models and you get completions from them and you can look at them and understand the models.

That's very well-suited to evaluation, where you want to be the best at creating representative problems that the models fail on or that show certain abilities, which I think you can break through with this. I think the top-end goal for a researcher working on evaluation, if you want to have career momentum, is that frontier labs pick up your evaluation.

You don't need to have every project do this. But if you go from a small university with no compute and find something that Claude struggles with, and then the next Claude model has it in the blog post, there's your career rocket ship.

Nathan Lambert

I think that's hard, but if you want to scope the maximum possible impact with minimum compute, it's something like that: just get very narrow. It takes learning where the models are going. So you need to build a tool that tests where Claude 4.5 will fail.

If I'm going to start a research project, I need to think where the models in 8 months are going to be struggling.

Lex Fridman

But what about developing totally novel ideas?

Nathan Lambert

This is a trade-off. I think that if you're doing a PhD, you could also be like, “It's too risky to work in language models. I'm going way longer term,” which is like, what is the thing that's going to define language model development in 10 years?

I end up being a person that's pretty practical. When I went to do my PhD, it was like, “I got into Berkeley. Worst case, I get a master's, and then I go work in tech.” I'm very practical about it.

OpenAI's average compensation is over $1 million in stock a year per employee. For any normal person in the US, getting into this AI lab is transformative for your life. So I'm pretty practical about it. There's still a lot of upward mobility working in language models if you're focused. And look at these jobs.

But from a research perspective, the transformative impact in these academic awards—to be the next Yann LeCun—comes from not working on or caring about language model development very much.

Lex Fridman

It's a big financial sacrifice in that case.

Nathan Lambert

So I work with some awesome students, and they're like, “Should I go work at an AI lab?” And I'm like, “You're getting a PhD at a top school. Are you gonna leave to go to a lab?” I don't know.

If you go work at a top lab, I don't blame you. Don't go work at some random startup that might go to zero. But if you're going to OpenAI, I'm like, “It could be worth leaving a PhD for.”

Lex Fridman

Let's more rigorously think through this. Where would you give a recommendation for people to make a research contribution? The options are academia: get a PhD, spend 5 years publishing, with compute resources constrained. There are research labs that are more focused on open-weight models and working there, or closed frontier research labs—OpenAI, Anthropic, xAI, and so on.

Nathan Lambert

The 2 gradients are: the more closed, the more money you tend to get, but you also get less credit. In terms of building a portfolio of things that you've done, it's very clear what you have done as an academic, versus if you are going to trade this fairly reasonable progression for being a cog in the machine, which could also be very fun.

So I think it's very different career paths. But the opportunity cost for being a researcher is very high because PhD students are paid essentially nothing. So it ends up rewarding people that have a fairly stable safety net and realize that they can operate in the long term, wanting to do very interesting work and get a very interesting job.

So it is a privileged position to be like, “I'm gonna see out my PhD and figure it out after because I want to do this.” At the same time, the academic ecosystem is getting bombarded by funding being cut and stuff. So there are just so many different trade-offs where I understand plenty of people who are like, “I don't enjoy it. I can't deal with this funding search. My grant got cut for no reason by the government,” or, “I don't know what's gonna happen.”

I think there's a lot of uncertainty and trade-offs that, in my opinion, favor just taking the well-paying job with meaningful impact. It's not like you're getting paid to sit around at OpenAI. You're building the cutting edge of things that are changing millions of people's relationship to tech.

Lex Fridman

But publication-wise, they're being more secretive, increasingly so. So you're publishing less and less. You are having a positive impact at scale, but you're a cog in the machine.

Sebastian Raschka

I think it honestly hasn't changed that much. I have been in academia. I'm not in academia anymore. I wouldn't want to miss my time in academia. But what I wanted to say before I get to that is that I think it hasn't changed that much.

I was working in computational biology, using AI or machine learning methods with collaborators, and a lot of people went from academia directly to Google. I think it's the same. Back then, professors were sad that their students went into industry because they couldn't carry on their legacy. I think it's the same. It hasn't changed that much.

The only thing that has changed is the scale. Cool stuff was always developed in industry and was closed. You couldn't talk about it. I think the difference now is your preference. Do you like to publish your work, or are you more in a closed lab? That's one difference.

The compensation, of course, is another, but it's always been like that. It depends on where you feel comfortable. Nothing is forever. Right now, there's a third option, which is launching a startup. A lot of people are doing that.

It's a very risky move, but it can be a high-risk, high-reward situation, whereas joining an industry lab is pretty safe. You also have upward mobility. I think once you've been at an industry lab, it's easier to find future jobs.

But then again, how much do you enjoy the team and working on proprietary things versus how much you like publishing work? Publishing is stressful. Acceptance rates at conferences can be arbitrary and very frustrating, but it's high reward if you have a paper published. You feel good because your name is on there. It's a high accomplishment.

Lex Fridman

I feel like my friends who are professors seem happier than those who work at a frontier lab, to be honest. There's a grounding there. The frontier labs definitely do this 9-9-6, which is shorthand for working all the time.

Can you describe 9-9-6? It's a culture invented, I believe, in China and adopted in Silicon Valley.

What is 9-9-6?

Sebastian Raschka

It’s 9:00 AM to 9:00 PM, six days a week.

Lex Fridman

Six days a week. What is that, 72 hours? Okay. So, is this basically the standard in AI companies in Silicon Valley, this kind of grind mindset?

Sebastian Raschka

Yeah, maybe not exactly like that, but I think there is a trend toward it. It’s interesting. I think it almost flipped because when I was in academia, I felt like that. As a professor, you write grants, you teach, and you do research. It’s like 3 jobs in 1, and it’s more than a full-time job if you want to be successful. I feel like now, as Nathan just said, professors, in comparison to a lab, have less pressure or workload than people at a frontier lab because—

Nathan Lambert

I think they work a lot. They’re just so fulfilled by working with students and having a constant runway of mentorship and a mission that is very people-oriented. I think in an era when things are moving very fast and are very chaotic, that’s very rewarding to people.

Sebastian Raschka

Yeah, and I think at a startup, there’s this pressure. It’s like, you have to make it. It’s really important that people put in the time, but it’s hard because you have to deliver constantly. I’ve been at a startup. I had a good time, but I don’t know if I could do it forever. It’s an interesting pace, and it’s exactly like we talked about in the beginning: these models are leapfrogging each other, and they’re constantly trying to take the next step compared to their competitors. It’s ruthless right now.

Nathan Lambert

I think this leapfrogging nature and having multiple players is actually an underrated driver of language-modeling progress, where competition is so deeply ingrained in people. These companies have intentionally created very strong cultures. Anthropic is known to be deeply committed and organized. We hear so little from them, and everybody at Anthropic seems very aligned. Being in a culture that is super tight and having this competitive dynamic is what makes you work hard and create things that are better.

But that comes at the cost of human capital. You can only do this for so long, and people are definitely burning out. I wrote a post on burnout, as I’ve gone in and out of this myself, especially trying to be a manager doing full-model training. It’s a crazy job doing this. The book Apple in China by Patrick McGee talks about how hard the Apple engineers worked to set up the supply chains in China. He said they had “saving marriage” programs, and he said on a podcast, “People died from this level of working hard.”

I think it’s a perfect environment for creating progress based on human expense. The human expense is the 9-9-6 that we started this with, where people really grind.

Sebastian Raschka

I also read this book. I think they had a code word for when someone had to go home to spend time with their family to save the marriage. It’s crazy. The colleagues would say, “Okay, this is a red alert for this situation. We have to let that person go home this weekend.”

At the same time, I don’t think they were forced to work. They were so passionate about the product, I guess, that they got into that mindset. I had that sometimes as an academic, but also as an independent person, I have that sometimes. I overwork, and it’s unhealthy. I had back issues and neck issues because I didn’t take the breaks that I maybe should have taken. No one forced me to. It’s because I wanted to work, because it’s exciting stuff.

Nathan Lambert

That’s what OpenAI and Anthropic are like. They want to do this work.

Lex Fridman

Yeah, but there’s also a feeling of fervor that’s building, especially in Silicon Valley, aligned with the scaling-laws idea. There’s this hype that the world will be transformed in a matter of weeks, and you want to be at the center of it.

I have the great fortune of having conversations with a wide variety of human beings, and from that I get to see all these bubbles and echo chambers across the world. It’s fascinating to see how we humans form them. I think it’s fair to say that Silicon Valley is a kind of echo chamber, a kind of silo and bubble.

I think bubbles are actually really useful and effective. It’s not necessarily a negative thing because you can be ultra-productive. It could be the Steve Jobs reality-distortion field, because you convince each other that breakthroughs are imminent, and by convincing each other of that, you make the breakthroughs imminent.

Nathan Lambert

Byrne Hobart wrote a book classifying bubbles. One of them is financial bubbles, which are based on speculation, and that’s bad. The other one is for build-outs, because it pushes people to build these things. I do think AI is in this, but I worry about it transitioning to a financial bubble.

Lex Fridman

Yeah, but also in the space of ideas, that bubble means you’re creating a reality-distortion field, which means you’re deviating from reality. If you go too far from reality while also working 9-9-6, you might miss some fundamental aspects of the human experience, including those beyond Silicon Valley.

This is a common problem in Silicon Valley: it’s a very specific geographic area. You might not understand the Midwest perspective, the full experience of all the other humans in the United States and across the world. You speak a certain way to each other, you convince each other of a certain thing, and that can get you into real trouble. Whether AI is a big success and becomes a powerful technology or it’s not, in either trajectory you can get yourself into trouble. You have to consider all of that.

Here you are, a young person trying to decide what you want to do with your life.

Nathan Lambert

The thing that is— I don’t even really understand this, but the San Francisco AI memes have gotten to the point where “permanent underclass” was one of them. The idea was that the last 6 months of 2025 was the only time to build durable value in an AI startup or model. Otherwise, all the value would be captured by existing companies, and you would therefore be poor. That’s an example of the San Francisco thing going too far.

I still think for young people who are going to be able to tap into it, if you’re really passionate about wanting to have an impact in AI, being physically in San Francisco is the most likely place where you’re going to do this. But it has trade-offs.

Lex Fridman

I think San Francisco is an incredible place, but there is a bit of a bubble. If you go into that bubble, which is extremely valuable, just get out also. Read history books, read literature, and visit other places in the world. Twitter and Substack are not the entire world.

Nathan Lambert

One of the people I worked with is moving to San Francisco, and I need to get him a copy of Season of the Witch, which is a history of San Francisco from 1960 to 1985. It goes through the hippie revolution, the gay community taking over the city and that culture emerging, and then the HIV/AIDS crisis and other things.

That is so recent, and there was so much turmoil and hurt, but also love, in San Francisco. No one knows about this. It’s a great book, Season of the Witch. I recommend it. A bunch of my San Francisco friends who got out recommended it to me. I lived there, and I didn’t appreciate this context. It’s just so recent.

Lex Fridman

Okay, we talked about a lot of things, certainly about the things that were exciting last year. But this year, one of the things you guys mentioned that’s exciting is the scaling of text-diffusion models and just a different exploration of text diffusion. Can you talk about what that is and what possibilities it holds? Are there different kinds of approaches than the current language models?

Nathan Lambert

We talked a lot about the transformer architecture, and the autoregressive transformer architecture specifically, like GPT. That doesn’t mean no one else is working on anything else. People are always on the lookout for the next big thing because I think it would be almost stupid not to. Right now, the transformer architecture is the thing, it works best, and there’s nothing else out there. But it’s always a good idea not to put all your eggs into one basket.

People are developing other alternatives to the autoregressive transformer. One of them would be text-diffusion models. Listeners may know diffusion models from image generation; Stable Diffusion popularized them. There was a paper on generating images. Before that, people used GANs, or generative adversarial networks. Then there was this diffusion process where you iteratively denoise an image, and that resulted in really good-quality images over time.

Stable Diffusion was a company, and other companies built their own diffusion models. People are now asking, “Can we try this also for text?” It doesn’t make intuitive sense yet because it feels like a pixel is something continuous that we can differentiate, whereas text is discrete. So how do we implement that denoising process?

It’s kind of similar to the BERT models by Google. If you go back to the original transformer, there were the encoder and the decoder. The decoder is what we’re using right now in GPT and similar models. The encoder is more like a parallel technique where you fill in multiple tokens in parallel. GPT models do autoregressive generation, completing the sentence 1 token at a time. In BERT models, you have a sentence with gaps. You mask them out, and then 1 iteration is filling in those gaps.

Text diffusion is kind of like that. You start with some random text, and then you fill in the missing parts or refine them iteratively over multiple iterations. The cool thing here is that this can do multiple tokens at the same time. That’s the promise of making it more efficient. The trade-off, of course, is how good the quality is. It might be faster, but now you have this additional dimension of the denoising process.

The more steps you do, the better the text becomes. You can scale in different ways. They try to see if that is maybe a valid alternative to the autoregressive model in terms of giving you the same quality for less compute. Right now, there are papers that suggest that if you want to get the same quality, you have to crank up the denoising steps, and then you end up spending the same compute you would spend on an autoregressive model.

The other downside is that while being parallel sounds appealing, some tasks are not parallel, like reasoning tasks or tool use, where you have to ask a code interpreter to give you an intermediate result. That is tricky with diffusion models. There are some hybrids, but the main idea is: How can we parallelize it? It's an interesting avenue.

I think right now, there are mostly research models out there, like LaMDA and some other ones. There are some from startups and some deployed models. There is no big diffusion model at scale yet, like at the Gemini or ChatGPT level. But there was an announcement by Google on a site where they said they are launching Gemini Diffusion, and they put it in the context of their Gemini Nano 2 model. They said, basically, that for the same quality on most benchmarks, they can generate things much faster.

You mentioned what's next. I don't think the text diffusion model is going to replace autoregressive LLMs, but it will be something maybe for quick, cheap, at-scale tasks. Maybe the free tier in the future will be something like that.

Lex Fridman

I think there are examples where it's already being used. To paint an example of why this is better, when GPT-5 is taking 30 minutes to respond, it's generating one token at a time. This diffusion idea is essentially to generate all of those tokens and the completion in one batch, which is why it could be way faster.

I think it could be suited for code startups, where somebody is effectively “vibe coding” and says, “Make this change.” A code diff is essentially a huge reply from the model, but it doesn't have to have that much external context, and you can get it really fast by using these diffusion models.

One example I've heard is that they use text diffusion to generate really long diffs, because doing it with an autoregressive model would take minutes, and that time for a user-facing product causes a lot of churn. Every second, you lose a lot of users.

I think it's going to grow and have some applications, but I actually thought that different types of models were going to be used for different things much sooner than they have been, so I kind of trade off. I think the tool-use point is the one that's stopping them from being more general-purpose, because for Claude Code and ChatGPT Search, the autoregressive chain is interrupted with some external tool, and I don't know how to do that with the diffusion setup.

So what's the future of tool use this year and then in the coming years? Do you think there's going to be a lot of developments there, and how is that integrated into the entire stack?

Nathan Lambert

I do think right now, it's mostly on the proprietary LLM side, but I think we will see more of that in the open-source tooling. I think it is a huge unlock because then you can really outsource certain tasks instead of relying on memorization. Instead of having the LLM memorize what 23 plus 5 is, just use a calculator.

Lex Fridman

So do you think that can help solve hallucination?

Nathan Lambert

Not solve it, but reduce it. The LLM still needs to know when to ask for a tool call. The second issue is that it doesn't mean the internet is always correct. You can do a web search, but let's say I asked who won the World Cup in 1998; it still needs to find the right website and get the right information. You can still go to the incorrect website and get incorrect information. I don't think it will fully solve hallucinations, but it is improving things in that sense.

Lex Fridman

Another cool paper earlier this year—I think it was December 31st, so it's not technically 2026, but close—is Recursive Language Models. To explain, Nathan, you also mentioned earlier that it's harder to do cool research in academia because of the compute budget. If I recall correctly, they did everything with GPT-5, so they didn't even use local models. The idea is that, let's say you have a long-context task; instead of having the LLM solve all of it in one shot or even in a chain, you break it down into subtasks.

You have the LLM decide what is a good subtask, and then recursively call an LLM to solve that. Something like that, adding tools—if you have a huge Q&A task, each one can go to the web and gather information, and then you pull it together at the end and stitch it back together.

I think there's going to be a lot of unlocks using things like that, where you don't necessarily improve the LLM itself; you improve how the LLM is used and what the LLM can use. One downside right now with tool use is that you have to give the LLM permission to use tools. That will take some trust, especially if you want to unlock things like having an LLM answer emails for you—or not even answer them, but just sort them for you or select them for you, or something like that.

I don't know if I would give an LLM access to my emails today. This is a huge risk.

Guest

I think there's one last cool point on the tool-use thing. I think you hinted at this, and we've both come at it in our own ways: open versus closed models use tools in very different ways. With open models, people go to Hugging Face and download the model, and then the person is going to ask, “What tool do I want?” Exa is my preferred search provider, but somebody else might prefer a different search startup.

When you release a model, it needs to be useful for multiple tools and multiple use cases, which is really hard because you're making a general reasoning engine model. That's actually what gpt-oss-120b is good for. But with closed models, you're deeply integrating the specific tool into your experience.

I think open models will struggle to replicate some of the things that I like to do with closed models, such as referencing a mix of public and private information. Something that I keep trying every 3 to 6 months is Claude Code on the web, which is just prompting a model to make an update to some GitHub repository that I have.

That set of secure cloud environments is so nice for sending it off to do this thing and then come back to me. These will probably help define some of the local, open, and closed niches. Initially, because there was such a rush to get tool use working, the open models were on the back foot, which is kind of inevitable.

I think there's so much research and so many resources in these frontier labs, but it will be fun when the open models solve this because it's going to necessitate a more flexible and potentially interesting model that might work with this recursive idea to be an orchestrator and a tool-use model. Hopefully, the necessity drives some interesting innovation there.

Lex Fridman

So continual learning—this is a longstanding topic and an important problem. I think that increases in importance as the cost of training the models goes up. Can you explain what continual learning is and how important it might be this year and in the coming years to make progress?

Guest

This relates a lot to this kind of SF zeitgeist of what AGI is, which is artificial general intelligence; what ASI is, artificial superintelligence; and what the language models that we have today are capable of doing.

I think language models can solve a lot of tasks, but a key milestone among the AI community is essentially when AI could replace any remote worker, taking in information, solving digital tasks, and doing them. The limitation highlighted by people is that a language model will not learn from feedback the same way that an employee does.

If you hire an editor, the editor will mess up, but you will tell them. If you hired a good editor, they don't do it again. Language models don't have this ability to modify themselves and learn very quickly. The idea is that if we are going to actually get to something that is a true, general, adaptable intelligence that can go into any remote-work scenario, it needs to be able to learn quickly from feedback and through on-the-job learning.

I'm personally more bullish on language models being able to just provide them with very good context. You said, maybe offline, that you can write extensive documents to models where you say, “I have all this information. Here are all the blog posts I've ever written. I like this type of writing. My voice is based on this.” But many people don't provide this to models, and the models weren't designed to take this amount of context previously.

Agentic models are just starting. So it's this kind of trade-off: Do we need to update the weights of this model with this continual-learning thing to make them learn fast? Or the counterargument is that we just need to provide them with more context and information, and they will have the appearance of learning fast by having a lot of context and being smart?

Lex Fridman

We should mention the terminology here. Continual learning refers to changing the weights continuously so that the model adapts and adjusts based on the new incoming information, doing so continually, rapidly, and frequently. The thing you mentioned on the other side of it generally will be referred to as in-context learning. As you learn stuff, there's a huge context window. You can just keep loading it with extra information every time you prompt the system, which I think both legitimately can be seen as learning.

It's just a different place where you're doing the learning.

Guest

I think, to be honest with you, continual learning—updating weights—we already have that in different flavors. I think the distinction here is: do you do that on a personalized custom model for each person, or do you do it at a global model scale? I think we have that already, going from GPT-5 to 5.1 and 5.2. It's maybe not immediate, but it is a curated update, a quick curated update where there was feedback about things they couldn't do, feedback from the community. They updated the weights, next model, and so forth. So it is a flavor of that.

Another even finer-grained example is RLVR: you run it, it updates. The problem is you can't just do that for each person because it would be too expensive to update the weights for each person, and I think that's the problem. Even at OpenAI scale, building the data centers would be too expensive. I think that is only feasible once you have something on the device where the cost is on the consumer, like what Apple tried to do with the Apple Foundation Models, putting them on the phone, where they learn from experience.

Lex Fridman

A related topic is this somewhat anthropomorphized term, memory. What are the different ideas for the mechanisms of adding memory to these systems as we're increasingly seeing? Personalized memory especially?

Guest

Right now, it's basically stuffing things into the context and then just recalling that. But again, I think it's expensive because you have to—you can cache it, but you still spend tokens on that. The second problem is that you can only do so much. I think it's more like a preference or style. A lot of people do that when they solve math problems. You can add previous knowledge and stuff, but you also give it certain preference prompts: “Do what I preferred last time,” or something like that.

But it doesn't unlock new capabilities. So for that, one thing people still use is LoRA adapters. Instead of updating the whole weight matrix, there are 2 smaller weight matrices that you have in parallel, like an overlay or a delta. You can do that to some extent, but then again, it is economics. There were also papers, for example, “LoRA Learns Less and Forgets Less.” There's no free lunch. If you want to learn more, you need to use more weights, but it gets more expensive. And then again, if you learn more, you forget more, and you have to find that Goldilocks zone.

Lex Fridman

We haven't really mentioned it much, but implied in this discussion is context length also. Is there a lot of innovation that's possible there?

Guest

I think the colloquially accepted thing is that it's a compute and data problem, with sometimes small architecture things like attention variants. We talked about hybrid attention models, which is essentially if you have what looks like a state-space model within your transformer. And those are better suited because you have to spend less compute to model the furthest-along token.

I think those aren't free because they have to be accompanied by a lot of compute or the right data. How many sequences of 100,000 tokens do you have in the world, and where do you get these? It just ends up being pretty expensive to scale them. We've gotten pretty quickly to 1 million tokens of input context length. I would expect it to keep increasing and get to 2 million or 5 million this year, but I don't expect it to go to 100 million.

That would be a true breakthrough, and I think those breakthroughs are possible. I think of the continual-learning thing as a research problem where there could be a breakthrough that just makes transformers work way better at this and makes it cheap. These things could happen with so much scientific attention. But turning the crank, it'll be consistent increases over time.

Guest 2

Looking at the extremes, I think there's, again, no free lunch. On one extreme, to make it cheap, you have, let's say, an RNN that has a single state where you save everything from the previous stuff. It's like a specific fixed-size thing, so you never really grow the memory because you are stuffing everything into one state. But then the longer the context gets, the more information you forget because you can't compress everything into one state.

On the other hand, you have transformers, which try to remember every token. That's great sometimes if you want to look up specific information, but very expensive because you have the KV cache that grows and the dot product that grows. But then, like you said, the Mamba layers kind of have the same problem. Like an RNN, you try to compress everything into one state; you're a bit more selective there. But then I think it's this Goldilocks zone again.

With Nemotron 3, they found a good ratio of how many attention layers you need for the global information, where everything is accessible, compared to having these compressed states. And I think that's how we will scale more—by finding better ratios in the Goldilocks zone, between making computing cheap enough to run and making it powerful enough to be useful.

And one more plug here: the “Recursive Language Models” paper is one of the papers that tries to address the long-context thing. What they found is essentially that, instead of stuffing everything into this long context, if you break it up into multiple smaller tasks, you save memory by having multiple smaller cores. You can actually get better accuracy than having the LLM try everything all at once. It's a new paradigm. We will see; there might be other flavors of that. So I think with that, we will still make improvements on long context, but then also, like Nathan said, I think the problem is that for pre-training itself, we don't have as many long-context documents as other documents. So it's harder to study how LLMs behave and stuff like that on that level.

Nathan Lambert

There are some rules of thumb where, essentially, you pre-train a language model like OLMo. We pre-trained at 8K context length and then extended to 32K with training. And there are some rules of thumb where you're essentially doubling the training context length; it takes 2× the compute, and then you can normally 2 to 4× the context length again.

So I think a lot of it ends up being compute-bound at pre-training. Like we talked about, everyone talks about this big increase in compute for the top labs this year, and that should reflect in some longer context windows. But I think on the post-training side, there are some more interesting things.

As we have agents, the agents are going to manage this context on their own. Now, people who use Claude Code a lot dread the compaction, which is when Claude takes its entire full 100,000 tokens of work and compacts it into a bulleted list. What the next models will do—and I'm sure people are already working on this—is essentially that the model can control when it compacts and how.

So you can essentially train your RL algorithm where compaction is an action—where it shortens the history—and then the problem formulation will be, “I want to keep the maximum evaluation scores that I have gotten while the model compacts its history to the minimum length.” Because then you have the minimum amount of tokens that you need to do this kind of compounding autoregressive prediction. So there are actually pretty nice problem setups in this, where these agentic models learn to use their context in a different way than just plow forward.

Remi Cadene

One interesting recent example would be DeepSeek-V3.2, where they had a sparse-attention mechanism. They have essentially a very efficient, small, lightweight lightweight indexer. Instead of attending to all tokens, it selects: “What tokens do I actually need?” It almost comes back to the original idea of attention, where you are selective, but attention is always on. You have maybe zero weight on some of them, but you use them all. But they are even more like, “Let's just mask that out or not even do that.”

And even with sliding-window attention in OLMo, that is also kind of that idea. You have a rolling window where you keep it fixed because you don't need everything. Occasionally, some layers you might, but it's wasteful. But right now, I think if you use everything, you're on the safe side—it gives you the best bang for the buck because you never miss information.

And I think this year will be more about figuring out, like you said, how to be smarter about that. Right now, people want to have the next state-of-the-art, and the state-of-the-art happens to be the brute-force, expensive thing. And then once you have that, as you said, keep that accuracy, but let's see how we can do that cheaper now with tricks.

Lex Fridman

Yeah, all this scaling thing. The reason we get the Claude 4.5 Sonnet model first is because you can train it faster and you're not hitting these compute walls as soon. They can just try a lot more things and get the model faster, even though the bigger model is actually better.

I think we should say that there's a lot of exciting stuff going on in the AI space. My mind has recently been really focused on robotics. Today, we almost entirely didn't talk about robotics. There's a lot of stuff on image generation and video generation. I think it's fair to say that the most exciting research work in terms of the amount, intensity, and fervor is in the LLM space, which is why I think it's justified for us to focus on the LLMs that we're discussing.

But it'd be nice to bring in certain things that might be useful. For example, world models—there's growing excitement about that. Do you think there will be any use this coming year for world models in the LLM space?

Nathan Lambert

Yes, I do think so. Also with LLMs, what's interesting here is that if we unlock more LLM capabilities, it also automatically unlocks all the other fields because it makes progress faster.

Sebastian Raschka

A lot of researchers and engineers use LLMs for coding. So even if they work on robotics, if you optimize these LLMs that help with coding, it pays off.

But yes, world models are interesting. It’s basically where you have the model run a simulation of the world in a sense, like a little toy version of the real thing, which can again unlock capabilities regarding data the LLM is not aware of. It can simulate things.

I think LLMs happen to work well by pre-training and doing next-token prediction, but we could do this even more sophisticatedly. I think there was a paper by Meta called “World Models,” where they basically apply the concept of world models to LLMs again. Instead of just having next-token prediction and verifiable rewards that check the answer’s correctness, they also make sure the intermediate variables are correct.

It’s kind of like the model is learning a code environment, in a sense. I think this makes a lot of sense. It’s just expensive to do, but it is making things more sophisticated—modeling the whole thing, not just the result. So it can add more value.

I remember when I was a graduate student, there was a competition called CASP, I think, where they did protein structure prediction. They predicted the structure of a protein that had not been solved yet at that point. In a sense, this is actually great, and I think we need something like that for LLMs also, where you do the benchmark, but no one knows the solution. You hand in the results, and then, after the fact, someone reveals the solution.

But AlphaFold, when it came out, crushed this benchmark. There were also multiple iterations, but I remember the first one. I’m not an expert in that subject, but the first one explicitly modeled the physical interactions—the physics of the molecule. It also modeled the angles and impossible angles.

Then, in the next version, I think they got rid of this and just scaled it up with brute force. I think with LLMs, we are currently in this brute-force scaling because it just happens to work. But I do think at some point it might make sense to bring back this thing, and with world models, I think that might be actually quite cool. Of course, also for robotics, which is completely unrelated to LLMs.

Lex Fridman

Yeah. Robotics is very explicit. So there’s the problem of locomotion or manipulation. Locomotion is much more solved, especially in the learning domain. But there’s a lot of value, just like with the initial protein-folding systems, in bringing in the traditional model-based methods.

It’s unlikely that you can just learn the manipulation or the whole-body, low-level manipulation problem end to end. That’s the dream. But then you realize, when you look at the magic of the human hand and the complexity of the real world, it’s really hard to learn this all the way through, the way I guess AlphaFold 2 didn’t.

Nathan Lambert

I’m excited about the robotic learning space. I think it’s collectively getting supercharged by all the excitement and investment in language models generally, where the infrastructure for training transformers, which is a general modeling thing, is becoming world-class industrial tooling. Wherever there was a limitation in robotics, it’s just way better. There’s way more compute.

On top of that, they take these language models as central units where you can do interesting exploratory work around something that already works. Then I see it emerging as, kind of like we talked about, Hugging Face Transformers and Hugging Face. I think when I was at Hugging Face, I was trying to get this to happen, but it was too early.

It’s like these open robotic models on Hugging Face, with people being able to contribute data and fine-tune them. I think we’re much closer now. The investment in robotics and self-driving cars is related, and it enables this. Once you get to the point where you can have this sort of ecosystem—where somebody can download a robotics model and maybe fine-tune it to their robot, or share datasets across the world—it will look very different.

There’s some work in this area, like RTX, I think it was a few years ago, where people are starting to do that. But once they have this ecosystem, it’ll look very different. This whole post-ChatGPT boom is putting more resources into that, which I think is a very good area for doing research.

Lex Fridman

This is also resulting in much better, more accurate, and more realistic simulators being built, closing the sim-to-real gap in the robotic space.

But you mentioned a lot of excitement in the robotics space and a lot of investment. The downside of that, which happens in hype cycles, is that I personally believe, and most robotics people believe, that robotics is not going to be solved on the timescale that’s being implicitly or explicitly promised.

So what happens when all these robotics companies spring up and then they don’t have a product that works? Then there’s going to be this kind of crash of excitement, which is nerve-wracking. Hopefully, something else will come in and keep swooping in so that the continued development of some of these ideas keeps going.

Sebastian Raschka

I think it’s also related to the continual-learning issue, essentially, where the real world is so complex. With LLMs, you don’t really need to have something learn for the user, because there are a lot of things everyone has to do. Everyone maybe wants to fix their grammar in their email or code or something like that. It’s more constrained, so you can prepare the model for that.

But preparing the robot for the real world is harder. You have the robotic foundation models, and you can learn certain things, like grasping things. But then again, everyone’s house is different. It’s so different, and that is, I think, where the robot would have to learn on the job, essentially.

That, I guess, is the bottleneck right now: how to customize it on the fly, essentially.

Lex Fridman

I don’t think I can possibly understate the importance of the thing that doesn’t get talked about almost at all by robotics folks or anyone else, which is safety. All the interesting complexities we talk about with learning, all the failure modes and failure cases, everything we’ve been talking about with LLMs—sometimes they fail in interesting ways. All of that is fun and games in the LLM space.

In the robotic space, in people’s homes, across millions of minutes and billions of interactions, you really are almost not allowed to fail, ever. When you have embodied systems that are put out there in the real world, you just have to solve so many problems you never thought you’d have to solve when thinking about the general robot-learning problem.

Nathan Lambert

I’m so bearish on in-home learned robots for consumer purchase. I’m very bullish on self-driving cars, and I’m very bullish on robotic automation—for example, Amazon distribution, where Amazon has built whole new distribution centers designed for robots first rather than humans.

There’s a lot of excitement in AI circles about AI enabling automation and mass-scale manufacturing, and I do think that the path to robots doing that is more reasonable. It’s a thing that is designed and optimized to do a repetitive task that a human could conceivably do but doesn’t want to.

I’m much more bullish on that, but it’s also going to take a lot longer than people probably predict. I think the leap from “AI singularity” to “we can now scale up mass manufacturing in the US because we have a massive AI advantage” is troubled by a lot of political and other challenging problems.

Lex Fridman

Let’s talk about timelines, specifically timelines to AGI or ASI. Is it fair, as a starting point, to say that nobody really agrees on the definitions of AGI and ASI?

Nathan Lambert

I think there’s a lot of disagreement, but I’ve been getting pushback where a lot of people say the same thing, which is a thing that could reproduce most digital economic work. So, the remote worker is a fairly reasonable example.

I think OpenAI’s definition is somewhat related to that, which is an AI that can do a lot of economically valuable tasks. I don’t really love that as a definition, but I think it could be a grounding point, because language models today, while immensely powerful, are not this remote-worker drop-in.

There are things that could be done by an AI that are way harder than remote work, like finding an unexpected scientific discovery that you couldn’t even posit. That would be an example of something that somebody says is an artificial superintelligence problem.

Or taking in all medical records and finding linkages across certain illnesses that people didn’t know about, or figuring out that some common drug can treat some niche cancer. They would say that this is a superintelligence thing. So these are natural tiers.

My problem with it is that it becomes deeply entwined with the quest for meaning in AI and these religious aspects to it. There are different paths you can take.

Lex Fridman

I don’t even know if the remote worker is a good definition, because what exactly is that? I actually like the AI 2027 report. They focus more on coding and research tasks, so the target there is the superhuman coder.

They have several milestone systems: superhuman coder, superhuman AI researcher, then superintelligent AI researcher, and then full ASI—artificial superintelligence. But after you develop the superhuman coder, everything else follows quickly.

There, the task is to have fully autonomous, automated coding. Any kind of coding you need to do in order to perform research is fully automated. From there, humans would be doing AI research together with that system, and they would quickly be able to develop a system that can actually do the research for you. That’s the idea.

Initially, their prediction was 2027 or 2028, and now they’ve pushed it back by 3 to 4 years, to 2031 as the mean prediction.

Sebastian Raschka

Probably my prediction is even beyond 2031, but at least you can, in a concrete way, think about how difficult it is to fully automate programming.

Lex Fridman

I disagree with some of their presumptions and dynamics on how it would play out, but I think they did good work in the scenario-defining milestones that are concrete and tell a useful story, which is why the reach for this AI 2027 document transcended Silicon Valley. It’s because they told a good story and did a lot of rigorous work to do this.

Nathan Lambert

I think the camp that I fall into is that AI is so-called “jagged,” which means it will be excellent at some things and really bad at some things. When we’re close to this automated software engineer, I think it will be good at traditional ML systems and frontend—the model is excellent at those—but distributed ML models are actually quite bad at because there’s so little training data on doing large-scale distributed learning. This is something we already see, and I think this will just get amplified.

It’s messier in these trade-offs, like how you think AI research works and so on.

Lex Fridman

So you think, basically, a superhuman coder is almost unachievable because of the jagged nature of the thing? You’re just always going to have gaps in capabilities?

Nat Friedman

I think it’s assigning completeness to something where the models are already superhuman at some types of code. I think that will continue. People are creative, so they’ll utilize these incredible abilities to fill in the weaknesses of the models and move really fast. There’ll always be this dance, for a long time, between the humans enabling the thing that the model can’t do. The best AI researchers are the ones that can enable this superpower.

I think those lines lead to what we already see. With something like Claude Code, you can build a beautiful website in a few hours, and data work is going to keep getting better. We’ll pick up some new coding skills along the way.

Linking to what’s happening in big tech, this AI 2027 report leans into the singularity idea, whereas I think research is messy, social, and largely in the data in ways that AI models can’t process. But what we do have today is really powerful, and these tech companies are all collectively buying into this with tens of billions of dollars of investment.

We are going to get some much better version of ChatGPT and a much better version of Claude Code than we already have. I think it’s just hard to predict where that is going, but the bright clarity of that future is why some of the most powerful people in the world are putting so much money into this. I think it’s just small differences between—we don’t actually know what a better version of ChatGPT is, but also, can it automate AI research? I would say probably not, at least in this timeframe.

Big tech is going to spend $100 billion much faster than we get an automated AI researcher that enables an AI research singularity.

Lex Fridman

So you think your prediction would be—if this is even a useful milestone—more than 10 years out?

Nat Friedman

I would say less than that on the software side, but I think longer than that on things like research.

Lex Fridman

Well, let’s just for fun try to imagine a world where all software writing is fully automated. Can you imagine that world?

Nathan Lambert

By the end of this year, the amount of software that will be automated will be so high. But it’ll be things like trying to train a model with RL when you need to have multiple bunches of GPUs communicating with each other. That’ll still be hard, but it’ll be much easier.

Lex Fridman

One way to think about the full automation of programming is just thinking of the number of lines of useful code written, and the fraction of that to the number of humans in the loop. Presumably, there’ll be humans in the loop of software writing for a long time. There’ll just be fewer and fewer relative to the amount of code written, right?

And the superhuman coder—I think the presumption there is that the number of humans in the loop goes to zero. What does that world look like when the number of humans in the loop is in the hundreds, not in the hundreds of thousands?

Nat Friedman

I think software engineering will be driven more toward system design and goals and outcomes. I think this has been happening over the last few weeks, where people have gone from, a month ago, saying, “Oh, yeah, agents are kind of slop,” which is a famous Karpathy quote, to what is a little bit of a meme: the industrialization of software, when anyone can just create software with their fingertips.

I do think we are closer to that side of things, and it takes direction and an understanding of how systems work to extract the best from the language models. I think it’s hard to accept the gravity of how much is going to change with software development and how many more people can do things without ever looking at the code.

Lex Fridman

I think what’s interesting is to think about whether these systems will be independent—completely independent. I have no doubt that LLMs will, at some point, solve coding in a sense, like calculators solve calculating. At some point, humans developed a tool where you never need a human to calculate that number. You just type it in, and it’s an algorithm. You can do it in that sense.

I think that’s probably the same for coding. But the question is—what will happen is, you’ll just say, “Build that website.” It will make a really good website, and then you maybe refine it. But will it do things independently? Will you still be having humans asking the AI to do something? Will there be a person to say, “Build that website,” or will there be AI that just builds websites or something?

Boris Cherny

I think talking about building websites is—

Lex Fridman

Too simple.

The problem with websites and the web—HTML and all that kind of stuff—is that it’s very resilient to slop. It will show you slop; it’s good at showing slop. I would rather think of safety-critical systems, like asking AI to end-to-end generate something that manages logistics or manages cars, a fleet of cars—all that kind of stuff. It end-to-end generates that for you.

Nathan Lambert

I think a more intermediate example is something like Slack or Microsoft Word. If organizations allow it, AI could very easily implement features end-to-end and do a fairly good job for things that you want to try. You want to add a new tab in Slack that you want to use, and I think AI will be able to do that pretty well.

Lex Fridman

Actually, that’s a really great example. How far away are we from that?

Nathan Lambert

This year.

Lex Fridman

See, I don’t know. I don’t know.

Nathan Lambert

I guess I don’t know how bad production codebases are, but I think that within the order of a few years, a lot of people are going to be pushed to be more of a designer and product manager. You’ll have multiple agents that can try things for you, and they might take 1 to 2 days to implement a feature or attempt to fix a bug.

You’ll have these dashboards, which I think Slack is actually a good dashboard, where your agents will talk to you and you’ll then give feedback. But if I make a website and ask, “Do you want a passable logo?” I think these cohesive design things and the style are going to be very hard for models, as will deciding what to add the next time.

Lex Fridman

I hang out with a lot of programmers, and some of them are a little bit on the skeptical side in general. That’s just their vibe. I think there’s a lot of complexity involved in adding features to complex systems.

If you look at the browser, Chrome, and I wanted to add a feature—if I wanted to have tabs not up top but on the left side, interface-wise—I think we’re not there. This is not a next-year thing.

Nathan Lambert

One of the Claude releases this year had a test where they gave it a piece of software and left Claude running to recreate it entirely. It could already almost rebuild Slack from scratch, just given the parameters of the software and left in a sandbox environment to do that.

Lex Fridman

So the “from scratch” part, I like almost better.

Nathan Lambert

It might be that smaller and newer companies are advantaged, and they’re like, “We don’t have the bloat and complexity, and therefore this feature exists.”

Lex Fridman

I think this gets to the point you mentioned, that some people you talk to are skeptical. I think that’s not because the LLM can’t do X, Y, Z. It’s because people don’t want it to do it this way.

Nathan Lambert

Some of that could be a skill issue on the human side. We have to be honest with ourselves. Some of that could be an underspecification issue.

Programming is like—you’re just assuming. This is an issue with communication in relationships and friendships. You’re assuming the LLM is supposed to read your mind. This is where spec-driven design is really important: using natural language to specify what you want.

If you talk to people at the labs, they use these in their training and production code. Claude Code is built with Claude Code, and they all use these things extensively. Dario talks about how much of Claude’s code is written this way.

These people are slightly ahead in terms of the capabilities they have and what they probably spend on inference. They could spend 10 to 100 times as much as we’re spending on a lowly $100 or $200-a-month plan. They truly let it rip.

With the pace of progress that we have, it seems like—a year ago, we didn’t have Claude Code, and we didn’t really have reasoning models. The difference between sitting here today and what we can do with these models is significant, and there’s a lot of low-hanging fruit to improve them.

The failure modes are pretty dumb. “Claude, you tried to use a CLI command I don’t have installed 14 times, and then I sent you the command to run.” From a modeling perspective, that’s pretty fixable. So I don’t know.

Lex Fridman

I agree with you. I’ve been becoming more and more bullish in general.

Boris Cherny

Speaking to what you're articulating, I think it is a human skill issue. Anthropic is leading the way, along with other companies, in understanding how to best use the models for programming; therefore, they're effectively using them. There are a lot of programmers on the outskirts, and there's not a really good guide on how to use them.

Lex Fridman

It might be very expensive. The entry point might be $2,000 a month, which is only for tech companies and rich people. That could be it.

Nathan Lambert

But it might be worth it. If the final result is a working software system, it might be worth it.

Lex Fridman

By the way, it's funny how we converged from the discussion of timeline to AGI to something more pragmatic and useful. Is there anything concrete, interesting, useful, and profound to be said about the timeline to AGI and ASI? Or are these discussions a bit too detached from the day to day?

Boris Cherny

There are interesting bets. A lot of people are trying to do RLVR—Reinforcement Learning with Verifiable Rewards—in real scientific domains, where startups with hundreds of millions of dollars in funding have wet labs where they're having language models propose hypotheses that are tested in the real world.

I would say that they're early, but with the pace of progress, maybe they're early by 6 months and they make it because they were there first, or maybe they're early by 8 years. You don't know. That type of moonshot to branch this momentum into other sciences would be very transformative. If AlphaFold moments happen in all sorts of other scientific domains through a startup solving this, that would be very transformative.

I think there are startups—maybe Harmonic is one—where they're going all in on language models plus Lean for math. You had another guest where you talked about this recently, and we don't know exactly what's going to fall out of spending $100 million on that model. Most of them will fail, but a couple might be big breakthroughs that are very different from ChatGPT or Claude Code-type software experiences. A tool that's only good for a PhD mathematician but makes them 100× more effective would be a huge breakthrough.

Lex Fridman

I agree. I think this will happen in a lot of domains, especially domains that have a lot of resources, like finance, legal, and pharmaceutical companies. But then again, is it really AGI? Because we are now specializing it again. Is it really that much different from back in the day when we had specialized algorithms? It's just the same thing, way more sophisticated, but I don't know—is there a threshold for AGI?

I think the real cool thing here is that we have foundation models we can specialize. That's the breakthrough. Right now, I think we are not there yet because, first, it's too expensive, but also, ChatGPT doesn't just give away its model to customize it. I think once that's true, I can imagine this as a business model, where OpenAI says at some point, "Hey, Bank of America, for $100 million we will do your custom model," something like that. I think that will be the huge economic value-add.

The other thing, though, is what is the differentiating factor? If everyone uses the same LLM, if everyone uses ChatGPT, they will all do the same thing. If everyone is moving in lockstep, but companies want to have a competitive advantage, there is no way around using some of their private data and specializing. It's going to be interesting.

Boris Cherny

Seeing the pace of progress, it does feel like things are coming. I don't think the AGI and ASI thresholds are particularly useful.

Lex Fridman

I think the real question, and this relates to the remote-worker thing, is: when are we going to see a big, obvious leap in economic impact? Because currently, there hasn't been an obvious leap in the economic impact of LLM models, for example.

Boris Cherny

Yeah, what is the GDP made up of? A lot of it is financial services, so I don't know what this is.

Lex Fridman

It's just hard for me to think about the GDP bump, but I would say that software development becomes valuable in a different way when you no longer have to look at the code anymore—when Claude Code makes you a small business. Essentially, Claude can set up your website, your bank account, your email, and whatever else. You just have to express what you're trying to put into the world.

That's not just an enterprise market, but it is hard. I don't know how you get people to try doing that. I guess if ChatGPT can do it, people are trying ChatGPT.

Boris Cherny

I think it boils down to the scientific question of, "How hard is tool use to solve?" Because a lot of the stuff you're implying—the remote-work stuff—is tool use. It's computer use, where you have an LLM that goes out there, this agentic system, and does something in the world and only screws up 1% of the time.

Computer use is a good example of what labs care about and we haven't seen a lot of progress on. We saw multiple demos in 2025 of Claude being able to use your computer, or OpenAI having Operator, and they all suck. They're investing money in this, and I think that'll be a good example.

Actually, something where it just seems like taking over the whole screen seems a lot harder than having an API that they can call in the back end. For some of that, you have to set up a different environment for them all to work in. They're not working on your MacBook; they are individually interfacing with Google, Amazon, and Slack, and they handle all these things in a very different way than humans do. So some of this might be structural blockers.

Lex Fridman

Also, specification-wise, I think the problem is that for arbitrary tasks, you still have to specify what you want your LLM to do. How do you do that? What is the environment? How do you specify? You can say what the end goal is, but if it can't solve the end goal, with LLMs, if you ask it for text, it can always clarify or do substeps.

How do you put that information into a system that, let's say, books a travel trip for you? You can say, "You screwed up my credit card information," but even to get it to that point, how do you, as a user, guide the model before it can even attempt that? I think the interface is really hard.

Nathan Lambert

Yeah, it has to learn a lot about you specifically. And this goes to continual learning, about the general mistakes that are made throughout, and then mistakes that are made through you.

Lex Fridman

All the AI interfaces are getting set up to ask humans for input. I think Claude Code, which we talked about a lot, asks for feedback and questions. If it doesn't have enough specification on your plan or your desired goal, it starts to ask questions: "Would you rather?"

We talked about Memory, which saves across chats. Its first implementation is kind of odd, where it'll mention my dog's name or something in a chat. I'm like, "You don't need to be subtle about this. I don't care."

But things that are emerging—ChatGPT has the Pulse feature. It is a curated couple of paragraphs with links to something to look at, and people talk about how models are going to ask you questions. I think that's probably going to work. The language model knows you had a doctor's appointment and asks, "Hey, how are you feeling after that?"

Again, this goes into the territory where humans are very susceptible to this, and there's a lot of social change to come. But they're also experimenting with having the models engage. Some people like this Pulse feature, which processes your chats, automatically searches for information, and puts it in the app. So there are a lot of things coming.

Sebastian Raschka

I used that feature before, and I always feel bad because it does that every day and I rarely check it out. How much compute is burned on something I don't even look at?

Lex Fridman

There's also a lot of idle compute in the world, so don't feel too bad.

Do you think new ideas might be needed? Is it possible that the path to AGI, however we define that—to solve computer use more generally, to solve biology and chemistry and physics, sort of the Dario Amodei definition of AGI—requires totally new ideas? Non-LLM, non-RL ideas? What might they look like? We're going into philosophy land a bit.

Nathan Lambert

For something like a singularity to happen, I would say yes. The new ideas could be architectures or training algorithms, fundamental deep-learning things. But ideas of that nature are pretty hard to predict.

I think we won't get very far even without those advances. We might get the software solution, but it might stop at software and not do computer use without more innovation. So I think that a lot of progress will be coming, but if you're going to zoom out, there are still ideas in the next 30 years that are going to look like major scientific innovations that enabled the next chapter of this. And I don't know if it comes in 1 year or in 15 years.

Lex Fridman

Yeah. I wonder if the bitter lesson holds true for the next 100 years, and what that looks like.

Nathan Lambert

If scaling laws are fundamental in deep learning, I think the bitter lesson will always apply, which is that compute will become more abundant. But even within abundant compute, the ones that have a steeper scaling-law slope or a better offset—this is a 2D plot of performance and compute—even if there's more compute available, the ones that get 100× out of it will win.

Lex Fridman

It might be something like literally computer clusters orbiting Earth with solar panels.

Sebastian Raschka

The problem with that is heat dissipation. You get all the radiation from the sun and don't have any air to dissipate heat. But there is a lot of space to put clusters. There's a lot of solar energy there, and you could figure out the heat dissipation, as there is a lot of energy and there could probably be the engineering will to solve the heat problem. So there could be.

Lex Fridman

Is it possible—and we should say that it definitely is possible—that we're basically going to be plateauing this year? Not in terms of the system capabilities, but in terms of what they actually mean for human civilization.

On the coding front, really nice websites will be built, with very nice autocomplete and a very nice way to understand codebases and maybe help debug. But really, it will just be a very nice helper on the coding front. It can help research mathematicians do some math. It can help you with shopping. It's a nice helper. It's Clippy on steroids.

What else? It may be a good education tool and all that kind of stuff, but computer use turns out to be extremely difficult to solve. So I'm trying to frame the cynical case in all these domains where there's not a really huge economic impact, but also realize how costly it is to train these systems at every level, both the pretraining and the inference, and how costly the inference is—the reasoning, all of that. Is that possible? And how likely is that, do you think?

Nathan Lambert

When you look at the models, there are so many obvious things to improve, and it takes a long time to train these models and to do this R&D. With the ideas that we have, it'll take us multiple years to actually saturate in terms of whatever benchmark or performance we are searching for. It might serve very narrow niches. The average ChatGPT user—there are 800 million users—might not get a lot of benefit out of this, but it is going to serve different populations by getting better at different things.

Lex Fridman

But I think what everybody's chasing now is a general system that's useful to everybody. So, okay, if that's not the case, that can plateau, right?

Sebastian Raschka

I think that dream is actually kind of dying. As you talked about with the specialized models, multimodal is often a totally different thing. Video generation is a totally different thing.

Lex Fridman

“​​That dream is kind of dying” is a big statement, because I don't know if it's dying. If you ask the actual frontier lab people, they—I mean, they're still chasing it, right?

Sebastian Raschka

I do think they are still rushing to get the next model out, which will be much better than the previous one. “Much” is a relative term, but it will be better than the previous one, and I can't see them slowing down. I just think the gains will be made or felt not only through scaling the model, but also through everything around it.

I feel like there's a lot of tech debt. It's like, “Well, let's just put the better model in there.” Better model, better model. And now people are like, “Okay, let's also, at the same time, improve everything around it too,” like the engineering of the context and inference scaling. The big labs will still keep doing that, and now the smaller labs will catch up, because they're hiring more. There will be more people working on LLMs. It's kind of like a circle. LLMs also make people more productive, and it's just—it's like amplification.

I think what we can expect is amplification, but not a change of any kind—not a paradigm change. I don't think that is true, but everything will just be amplified and amplified, and I can see that continuing for a long time.

Lex Fridman

Yeah. I guess my statement that the dream is dying depends on exactly what you think it's going to be doing. Claude Code is a general model that can do a lot of things, but it's not necessarily all-encompassing. It depends a lot on integrations. I bet Claude Code could do a fairly good job of doing your email, and the hardest part is figuring out how to give information to it and how to get it to be able to send your emails.

I think it goes back to what is the “one model to rule everything” ethos, which is just a thing in the cloud that handles your entire digital life and is way smarter than everybody. So it's an interesting leap of faith to go from “Claude Code becomes that.” There are some avenues for that, but I do think that the rhetoric of the industry is a little bit different.

Sebastian Raschka

I think the immediate thing we will feel next as a normal person using LLMs will probably be related to something trivial, like making figures. Right now, LLMs are terrible at making figures. Is it because we're getting served the cheap models with much less inference compute than what's used behind the scenes? Maybe, in some cases. There are some ways to get better figures, but if you ask today, “Draw a flowchart of X, Y, Z,” it's most of the time terrible.

It's a very simple task for a human. I think it's almost easier sometimes to draw something than to write something.

Lex Fridman

Yeah, the multimodal understanding does feel like something that is odd—that it's not better solved.

Nathan Lambert

I think we're missing one obvious thing that we're not realizing is gigantic and hard to measure: making all of human knowledge accessible to the entire world. One thing that is hard to articulate is the huge difference between Google Search and an LLM. I feel like I can basically ask an LLM anything and get an answer, and it's hallucinating less and less.

That means understanding my own life, figuring out a career trajectory, solving the problems all around me, and learning about anything through human history. I feel like nobody's really talking about that, because they just immediately take it for granted that this is awesome. That's why everybody's using it: because you get answers for stuff.

Think about the impact across time. This is not just in the United States; it's all across the world. The impact, across time, of kids throughout the world being able to learn these ideas is probably the real impact. Talk about GDP: it won't be a leap. That's how we get to Mars, that's how we build these things, and that's how we have a million new OpenAIs and all the innovation from there. It's this quiet force that permeates everything: human knowledge.

I agree with you. In a sense, it makes knowledge more accessible, but it also depends on what the topic is. For something like math, you can ask it questions and it answers, but if you want to learn a topic from scratch, the sweet spot is still elsewhere. There are really good math textbooks laid out linearly, and that is a proven strategy to learn a topic.

If you start from zero, it makes sense to use information-dense text to soak it up, but then you use the LLM to make infinite exercises. You have problems in a certain area or questions that you're uncertain about, and you ask it to generate example problems. You solve them, and if you need more background knowledge, you ask it to generate that. But then it won't give you anything, let's say, that is not in the textbook. It's just packaging it differently, if that makes sense.

But then there are things where I feel like it also adds value in a more timely sense, where there is no good alternative besides a human doing it on the fly. For example, if you're planning to go to Disneyland and you're trying to figure out which tickets to buy for which park and when, there is no textbook on that. There is no information-dense resource. There's only the sparse internet, and then there is a lot of value in the LLM.

You just ask it. You have constraints on traveling these days: “I want to go there and there. Please figure out what I need, when and from where, what it costs,” and stuff like that. It is a very customized, on-the-fly package. Personalization is essentially pulling information from the sparse internet, the non-information-dense thing where there's no better version that exists. It just doesn't exist. You make it almost from scratch.

Lex Fridman

And if it does exist, it's full of—speaking of Disney World—full of—what would you call it? Ad slop. It's impossible. Take any city in the world: What are the top 10 things to do? An LLM is just way better to ask than anything on the internet.

Nathan Lambert

Well, for now, that's because they're subsidized, and they're going to be paid for by ads.

Lex Fridman

Oh my goodness.

Daniel Gross

It's coming.

Lex Fridman

No. No. I mean, I'm hoping there's a very clear indication of what's an ad and what's not an ad in that context.

Sebastian Raschka

That's something I mentioned a few years ago. If you're looking for a new running shoe, is it a coincidence that Nike maybe comes up first? Maybe, maybe not. But I think there are clear laws. You have to be clear about that. I think that's what everyone fears. It's the subtle message in there.

That also brings us to the topic of ads, where I think this will be a thing, hopefully in 2025, because I think they're still not making money in other ways right now. But there are alternatives without ads, and people would just flock to the other products. It's also just crazy how they're one-upping each other, spending so much money just to get the users.

Lex Fridman

I think so. Some Instagram ads—I don't use Instagram, but I understand the appeal of paying a platform to find users who will genuinely like your product, and that is the best case of things like Instagram ads. But there are also plenty of cases where advertising is very awful for incentives.

I think that a world where the power of AI can integrate with that positive view of, “I am a person and I have a small business and I want to make the best, I don't know, damn steak knives in the world, and I want to sell them to somebody who needs them,” and if AI can make that sort of advertising thing work even better, that's very good for the world, especially with digital infrastructure, because that's how the modern web has been built.

But that's not to say that addicting feeds so that you can show people more content is a good thing. So I think that's even what OpenAI would say: They want to find a way that can make the monetization upside of ads while still giving their users agency. And I personally would think that Google is probably going to be better at figuring out how to do this, because they already have ad supply, and if they figure out how to turn this demand in their Gemini app into useful ads, then they can turn it on. Somebody will figure it out—I don't know if it's this year, but there will be experiments with it.

Sebastian Raschka

I do think what holds companies back right now is really just that the competition is not doing it. It’s more like a reputation thing. I think people are just afraid of ruining or losing their reputation and losing users, because it would make headlines if someone launched these ads.

Lex Fridman

Unless they were great, but the first ads won’t be great because it’s a hard problem that we don’t know how to solve.

Daniel Gross

Yeah, I think also the first version of that will likely be something like on X, like the timeline where you have a promoted post sometimes in between. It’ll be something where it will say “promoted” or something small, and then there will be an image. I think right now the problem is: who makes the first move?

Lex Fridman

If we go 10 years out, the proposition for ads is that you will make so much money on ads by having so many users that you can use this to fund better R&D and make better models, which is why YouTube is dominating the market for video—Netflix is scared of YouTube. They have the ads; I pay $28 a month for Premium. They make at least $28 a month off of me and many other people. They’re just creating such a dominant position in video.

So I think that’s the proposition: ads can give you a sustained advantage in what you’re spending per user. But there’s so much money in it right now that somebody starting that flywheel is scary because it’s a long-term bet.

Do you think there’ll be some crazy big moves this year business-wise? Like Google or Apple acquiring Anthropic or something like this?

Nathan Lambert

Dario will never sell, but we are starting to see some types of consolidation with Groq for $20 billion and Scale AI for almost $30 billion, and countless other deals like this that are structured in a way that is detrimental to the Silicon Valley ecosystem. This licensing deal means that not everybody gets brought along, rather than a full acquisition that benefits the rank-and-file employee by getting their stock vested.

That’s a big issue for the culture to address because the startup ecosystem is the lifeblood. If you join a startup, even if it’s not successful, it might get acquired at a cheap premium and you’ll get paid out for this equity. These licensing deals are taking the top talent a lot of the time. The deal for Groq with NVIDIA is rumored to be better for the employees, but it is still this antitrust-avoiding thing.

I think this trend of consolidation will continue. Me and many smart people I respect have been expecting consolidation to have happened sooner, but it seems like some of these things are starting to turn. At the same time, companies are raising ridiculous amounts of money for reasons where I’m like, “I don’t know why you’re taking that money.” So it’s mixed this year, but some consolidation pressure is starting.

Lex Fridman

What kind of surprising consolidation will we see? You say Anthropic is a “never.” I mean, Groq is a big one. Groq with a Q, by the way.

Daniel Gross

Yeah. There are just a lot of startups and a very high premium on AI startups. So there could be a lot of—

Lex Fridman

That kind of stuff, yeah.

Daniel Gross

$10 billion-range acquisitions, which is really big for a startup that was maybe founded a year ago. I think Manus AI—this company based in Singapore that Meta funded—was founded 8 months ago and then had a $2 billion exit. I think there will be some other multibillion-dollar acquisitions, like Perplexity.

Lex Fridman

Like Perplexity, right?

Daniel Gross

Yeah, people have rumored them to Apple. I think there’s a lot of pressure and liquidity in AI. There’s pressure on big companies to have outcomes, and I would guess that a big acquisition gives people leeway to then tell the next chapter of that story.

Lex Fridman

I guess Cursor—we’ve been talking about code—somebody acquires Cursor. If somebody acquires Cursor—

Daniel Gross

They’re in such a good position by having so much user data. And we talked about continual learning. They had one of the most interesting sentences in a blog post, which is that they had their new Composer model, which was a fine-tune of one of these large Mixture-of-Experts models from China. You can know that by asking it or because the model sometimes responds in Chinese—which none of the American models do.

They had a blog post where they’re like, “We’re updating the model weights every 90 minutes based on real-world feedback from people using it.” Which is like the closest thing to real-world RL happening on a model, and it’s just mentioned in one of their blog posts—

Lex Fridman

That’s incredible.

Daniel Gross

Which is super cool.

Lex Fridman

And by the way, I should say I use Composer a lot because one of the benefits it has is that it’s fast.

Daniel Gross

I need to try it because everybody says this.

Lex Fridman

And there’ll be some IPOs potentially. You think Anthropic, OpenAI, xAI?

Daniel Gross

They can all raise so much money so easily that they don’t feel a need to. As long as fundraising is easy, they’re not going to IPO because public markets apply pressure. I think we’re seeing in China that the ecosystem’s a little different, with both MiniMax and Z.ai filing IPO paperwork, which will be interesting to see how the Chinese market reacts.

I actually would guess that it’s going to be similarly hype-y to the U.S., as long as all this is going and not based on the reality that they’re both losing a ton of money. I wish more of the gigantic American AI startups were public because it would be very interesting to see how they’re spending money and to have more insight, and also just to give people access to investing in these. I think they’re some of the most formidable companies—they’re the companies of the era.

The tradition is now for so many of the big startups in the U.S. not to go public. It’s like we’re still waiting for Stripe and its IPO, but Databricks definitely didn’t. They raised something like a Series G. I just feel like it’s kind of a weird equilibrium for the market where I would like to see these companies go public and evolve in that way that a company can.

Lex Fridman

Do you think 10 years from now some of the frontier model companies are still around? Anthropic, OpenAI?

Nathan Lambert

I definitely don’t see it as winner-takes-all unless there truly is some algorithmic secret that one of them finds that lets this flywheel spin. The development path is so similar for all of them. Google and OpenAI have all the same products, and Anthropic is more focused, but when you talk to people it sounds like they’re solving a lot of the same problems.

So I think there will be offerings that spread out. There’s a lot of—it’s a very big cake being made that people are going to take money out of.

Lex Fridman

I don’t want to trivialize it, but OpenAI and Anthropic are primarily LLM service providers. Some of the other companies, like Google and xAI, linked to X, do other stuff too. And so it’s very possible that, if AI becomes more commodified, the companies just providing LLMs will die.

Nathan Lambert

I think the advantage they have is a lot of users, and I think they will just pivot. Like Anthropic, I think, pivoted. I don’t think they originally planned to work on code, but they found, “Okay, this is a nice niche,” and now they’re comfortable, and they push on this niche.

I can see the same thing. Let’s say hypothetically—I’m not sure if it will be true—but let’s say Google takes all the market share of the general chatbot. Maybe OpenAI will then focus on some other subtopic. They have too many users to go away in the foreseeable future.

Lex Fridman

I think Google is always ready to say, “Hold my beer,” with AI models.

Nathan Lambert

I think the question is whether the companies can support the valuations. I see the AI companies as being, in some ways, like AWS, Azure, and GCP: all competing in the same space and all very successful businesses.

There’s a chance that the API market is so unprofitable that they go up and down the stack to products and hardware. They have so much cash that they can build power plants and data centers, which is a durable advantage now. But there’s also a reasonable outcome that these APIs are so valuable and so flexible for developers that they become something like AWS.

But AWS and Azure are also going to have these APIs, so having 5 or 6 companies competing in the API market is hard. So maybe that’s why they get squeezed out.

Lex Fridman

You mentioned “RIP Llama.” Is there a path to winning for Meta?

Nathan Lambert

I think nobody knows. They’re moving a lot, so they’re signing licensing deals with Black Forest Labs, which is an image-generation company, or Midjourney. In some ways, on the product and consumer-facing AI front, it’s too early to tell.

I think they have some people who are excellent and very motivated, close to Zuckerberg. So I think there’s still a story to unfold there. Llama is a bit different. Llama was the most focused expression of the organization, and I don’t see Llama being supported to that extent.

I think it was a very successful brand for them. So they still might participate in the open ecosystem or continue the Llama brand into a different service, because people know what Llama is.

Lex Fridman

Do you think there’s a Llama 5?

Nathan Lambert

Not an open-weight one.

Lex Fridman

It’s interesting. I think Llama was the pioneering open-weight model. With Llama 1, 2, and 3, there was a lot of love. But, hypothesizing or speculating, I think the leaders at Meta—the upper executives—got very excited about Llama because they saw how popular it was in the community.

And then I think the problem was trying to use it to make a bigger splash. It felt almost forced, like developing these very big Llama 4 models to be on top of the benchmarks. But I don’t think the goal of Llama models is to be on top of the benchmarks, beating, let’s say, ChatGPT or other models.

I think the goal was to have a model that people can use, trust, modify, and understand. So that includes having smaller models. They don’t have to be the best models. And what happened was that the benchmarks, of course, suggested that they were better than they were because they had specific models trained on preferences so that they performed well on benchmarks.

Nathan Lambert

That's this overfitting thing to force it to be the best. But at the same time, they didn't make the small models that people could use. I think no one could run these big models then. Then there was a weird thing. I think it's because people got too excited about headlines pushing the frontier. I think that's it.

Lex Fridman

And too much on the benchmarking side.

Nathan Lambert

It's too much work.

Lex Fridman

I think it imploded under internal political fighting and misaligned incentives. The researchers want to build the best models, but there's a layer of organization and management that is trying to demonstrate that they do these things. Then there are rumors about how, for example, some horrible technical decision was made. It just seems like it got so bad that it all just crashed out.

Nathan Lambert

Yeah, but we should also give huge props to Mark Zuckerberg. I think it comes from Mark Zuckerberg, from the top of the leadership, saying open source is important. The fact that that leadership exists means there could be a Llama 5, where they learn the lessons from benchmarking and say, "We're going to be GPT-OSS and provide a really awesome library of open source."

Lex Fridman

What people say is that there's a debate between Mark and Alexandr Wang, who is very bright but much more against open source. To the extent that he has a lot of influence over the AI org, it seems much less likely, because it seems like Mark brought him in for a fresh leadership eye in directing AI.

If being open or closed is no longer the defining nature of the model, I don't expect that to be a defining argument between Mark and Alex. They're both very bright, but I have a hard time understanding all of it because Mark wrote this piece in July 2024, which was probably the best blog post at the time, saying, "The Case for Open Source AI." Then July 2025 came around and it was, "We're reevaluating our relationship with open source."

Nathan Lambert

But I think also the problem—not the problem, but I think we may have been a bit too harsh, and that caused some of it. As open-source developers or as a community, even though the model was maybe not what everyone hoped for, it got a lot of backlash. I think that was unfortunate because I can see that, as a company, they were hoping for positive headlines.

Instead of getting no headlines or positive headlines, they got negative headlines. Then it reflected badly on the company. I think that is also something where it's maybe a spite reaction, almost like, "Okay, we tried to do something nice, we tried to give you something cool, like an open-source model, and now you are being negative about us, even for the company." In that sense, it looks like, "Well, maybe then we'll change our mind." I guess. I don't know.

Lex Fridman

Yeah, that's where the dynamics of discourse on X can lead us, as a community, astray. Sometimes it feels random. People pick the thing they like and don't like. You can see the same thing with Grok 4.1 and Grok Code Fast 1.0.

I don't think, vibe-wise, people love it publicly, but a lot of people use it. If you look to Reddit and X, it doesn't really get praise from the programming community, but they use it. The same thing is probably true with Llama. I don't understand the dynamics of either positive hype or negative hype. I don't understand it.

Nathan Lambert

One of the stories of 2025 is the U.S. filling the gap left by Llama, which is the rise of these Chinese open-weight models, to the point where that was the single issue I've spent a lot of energy on lately, trying to do policy work to get the U.S. to invest in this.

Lex Fridman

So just tell me the story of ADAM.

Nathan Lambert

The ADAM Project started as me calling it the American DeepSeek Project, which doesn't really work for DC audiences. But it's the story of the most impactful thing I can do with my career: These Chinese open-weight models are cultivating a lot of power, and there's a lot of demand for building on these open models, especially in enterprises in the U.S. that are very cagey about Chinese models.

The ADAM Project, American Truly Open Models, is a US-based initiative to build and host high-quality, genuinely open-weight AI models and supporting infrastructure explicitly aimed at competing with and catching up to China's rapidly advancing open-source AI ecosystem.

I think the one-sentence summary would be that—or two sentences. One is a proposition that open models are going to be an engine for AI research because that is what people start with; therefore, it's important to own them. The second one is: Therefore, the U.S. should be building the best models so that the best research happens in the U.S., and those U.S. companies take the value from being the home of where AI research is happening.

Without more investment in open models, we have plots on the website where it's like, "Qwen, Qwen, Qwen, Qwen"—it's all these models that are excellent from these Chinese companies that are cultivating influence internationally. I think the U.S. is spending way more on AI, and the ability to create open models that are a generation beyond the cutting edge of closed labs costs roughly $100 million, which is a lot of money but not a lot of money to these companies. Therefore, we need a centralizing force of people who want to do this. I think we got engagement from people pretty much across the full stack, whether it's policy.

Lex Fridman

So there has been support from the administration?

Nathan Lambert

I don't think anyone technically in government has signed it publicly, but I know people who have worked in AI policy in both the Biden and Trump administrations are very supportive of promoting open-source models in the U.S. I think, for example, AI2 got a grant from the NSF for $100 million over 4 years, which is the biggest computer science grant the NSF has ever awarded, and it's for AI2 to attempt this. It's a starting point.

The best thing happens when there are multiple organizations building models, because they can cross-pollinate ideas and build this ecosystem. I don't think it works if it's just Llama releasing models, because Llama could go away. The same thing applies to AI2; I can't be the only one building models. It becomes a lot of time spent talking to people, whether in policy.

I know NVIDIA is very excited about this. I think Jensen Huang has been talking about the urgency for this, and they've done a lot more in 2025, with the Nemotron models becoming more of a focus. They've started releasing some data along with NVIDIA's open models, and very few companies do this, especially of NVIDIA's size, so there are signs of progress.

We hear about Reflection AI, where they say their $2 billion fundraise is dedicated to building US open models, and I feel like their announcement tweet reads like a blog post, right? I think that cultural tide is starting to turn. In July, there were 4 or 5 DeepSeek-caliber Chinese open-weight models and 0 from the US. That's the moment where I realized, "Oh, I guess I have to spend energy on this because nobody else is going to do it."

So it takes a lot of people contributing together, and I don't say that the ADAM Project is the thing that's helping to move the ecosystem, but it's people like me doing this sort of thing to get the word out.

Lex Fridman

Do you like the 2025 America's AI Action Plan? That includes open-source stuff. The White House AI Action Plan includes a dedicated section titled "Encourage Open-Source and Open-Weight AI," defining such models and arguing they have unique value for innovation and startups.

Sebastian Raschka

Yeah. The AI Action Plan is a plan, but largely, I think it's maybe the most coherent policy document that has come out of the administration, and I hope that it largely succeeds. I know people who have worked on the AI Action Plan and know the challenges of taking policy and making it real. I have no idea how to do this as an AI researcher, but largely, a lot of things in that were very real, and there's a huge build-out of AI in the country.

There are a lot of issues that people are hearing about, from water use to whatever, and we should be able to build things in this country, but also, we need to not ruin places in our country in the process of building it, and it's worthwhile to spend energy on. I think that's a role the federal government plays. They set the agenda. With AI, setting the agenda that open-weight should be a first consideration is a large part of what they can do, and then people think about it.

Lex Fridman

Also, for education and talent for these companies, it's very important because otherwise, if there are only closed models, how do you get the next generation of people contributing at some point? Otherwise, you will only be able to learn after you join a company. But at that point, how do you hire talented people? How do you identify talented people?

I think open source is essential for a lot of things, but also even just for educating the population and training the next generation of researchers. It's the way, or the only way.

Sebastian Raschka

The way that I could have gotten this to go more viral was to tell a story of Chinese AI integrating with an authoritarian state, being ASI and taking over the world, and therefore we need our own American models. But it's very intentional that I talk about innovation and science in the US, because I think it's both more realistic as an outcome and also a world that I would like to manifest.

Lex Fridman

I would say, though, that even any open-weight model is valuable.

Sebastian Raschka

Yeah. My argument is that we should be in a leading position. But I think it's worth saying it simply because there are still voices in the AI ecosystem that say we should consider banning the release of open models due to safety risks.

I think it's worth adding that, effectively, that's impossible without making the US have its own Great Firewall, which is also known not to work that well, because the cost of training these models, whether it's $1 million to $100 million, is attainable to a huge number of people in the world who want to have influence. These models will be trained all over the world.

There are safety concerns, but we want this information and these tools to flow freely across the world and into the US so that people can use them and learn from them. Stopping that would be such a restructuring of our internet that it seems impossible.

Lex Fridman

Do you think that, in that case, the big open-weight models from China are actually a good thing, in a sense, for the US companies? Because maybe the US companies you mentioned earlier are usually one generation behind in terms of what they release open source versus what they are using.

For example, GPT-OSS might not be the cutting-edge model. Gemini 3 might not be, but they do that because they know this is safe to release. But then when these companies see, for example, that there is DeepSeek-V3.2, which is really awesome, and it gets used and there is no backlash, there is no security risk, that could then, again, encourage them to release better models. Maybe that, in a sense, is a very positive thing.

Sebastian Raschka

A hundred percent. These Chinese companies have set things into motion that I think potentially would not have happened if they were not all releasing models. I’m almost sure that those discussions have been had by leadership.

Lex Fridman

Is there a possible future where the dominant AI models in the world are all open source?

Sebastian Raschka

It depends on the trajectory of progress that you predict. If you think saturation in progress is coming within a few years—essentially, within the time when financial support is still very good—then open models will be so optimized and so much cheaper to run that they’ll win out.

This goes back to open-source ideas, where so many more people will be putting money into optimizing the serving of these open-weight common architectures that they will become standards. Then you could have chips dedicated to them, and it’ll be way cheaper than the offerings from these closed companies that are custom.

Lex Fridman

We should say that the AI 2027 report predicts—one of the things it does from a narrative perspective—is that there will be a lot of centralization. As the AI systems get smarter and smarter, national security concerns will arise, and you’ll centralize the labs, and they’ll become super-secretive, and there’ll be this whole race from a military perspective between China and the US.

All of these fun conversations we’re having about LLMs—the generals and the soldiers will come into the room and be like, “All right. We’re now in the Manhattan Project stage of this whole thing.”

Sebastian Raschka

I think in 2025, 2026, and 2027, I don’t think something like that is even remotely possible. You can make the same argument for computers, right? You can say, “Computers are capable, and we don’t want the general public to get them.” Or chips, even AI chips.

But you see how Huawei makes chips now. It took a few years, but I don’t think there is a way you can contain knowledge like that. I think in this day and age, it is impossible, like the internet. I don’t think this is a possibility.

On the Manhattan Project thing, I think a Manhattan Project-like thing for open models would be pretty reasonable because it wouldn’t cost that much. But I think that will come. It seems like, culturally, the companies are changing.

But I agree with Sebastian on all of that. I don’t see it happening nor being helpful.

Yeah. The motivating force behind the Manhattan Project was civilizational risk. It’s harder to motivate that for open-source models.

There’s no civilizational risk.

Lex Fridman

On the hardware side, we mentioned NVIDIA a bunch of times. Do you think Jensen and NVIDIA will keep winning?

Sebastian Raschka

I think they have to iterate and manufacture a lot. What they’re doing, they do innovate, but I think there’s always the chance that someone does something fundamentally different and gets very lucky. But the problem is adoption. The moat of NVIDIA is probably not just the GPU. It’s more like the CUDA ecosystem, and that has evolved over 2 decades.

Even back when I was a grad student, I was in a lab doing biophysical simulations and molecular dynamics, and we had a Tesla GPU back then just for the computations. It was about 15 years ago. They built this up for a long time, and that’s the moat, I think.

It’s not the chip itself, although they have the money to iterate, build, and scale. But then it’s really about compatibility. If you’re at that scale, why would you go with something risky where there are only a few chips they can make per year? You go with the big one. But I do think with LLMs now, it will be easier to design something like CUDA. It took 15 years because it was hard, but now that we have LLMs, we can maybe replicate CUDA.

Lex Fridman

And I wonder if there will be a separation of training and inference compute as we stabilize, and more compute is needed for inference.

Sebastian Raschka

That’s supposed to be the point of the Groq acquisition. And that’s why part of what Vera Rubin is about is that they have a new chip with no high-bandwidth memory, or very little, which is one of the most expensive pieces.

It’s designed for prefill, which is the part of inference where you essentially do a lot of matrix multiplications. Then you only need the memory when you’re doing this autoregressive generation and you have the KV-cache swaps. So they have this new GPU that’s designed for that specific use case, and then the cost of ownership per FLOP or whatever is actually way lower.

But I think that NVIDIA’s fate lies in the diffusion of AI still. Their biggest clients are still these hyperscale companies. Google obviously can make TPUs. Amazon is making Trainium. Microsoft will try to do its own things.

So long as the pace of AI progress is high, NVIDIA’s platform is the most flexible and people will want that. But if there’s stagnation, then there’s more time to create bespoke chips.

Lex Fridman

It’s interesting that NVIDIA is quite active in trying to develop all kinds of different products.

Nathan Lambert

They try to create areas of commercial value that will use a lot of GPUs.

Lex Fridman

Mm-hmm. But they keep innovating, and they’re doing a lot of incredible research.

Nathan Lambert

Everyone says the company is super-oriented around Jensen and how operationally plugged in he is. And it sounds so unlike many other big companies that I’ve heard about. So long as that’s the culture, I think that we can expect that to keep progress happening.

And it’s like he’s still in the Steve Jobs era of Apple. So long as that is how it operates, I’m pretty optimistic for their situation because it is their top-order problem. I don’t know if making these chips for the whole ecosystem is the top goal of all these other companies. They’ll do a good job, but it might not be as good of a job.

Lex Fridman

Since you mentioned Jensen, I’ve been reading a lot about history and about singular figures in history. What do you guys think about the great-man/great-woman view of history? How important are individuals for steering the direction of history in the tech sector?

What’s NVIDIA without Jensen? You mentioned Steve Jobs. What’s Apple without Steve Jobs? What’s xAI without Elon or DeepMind without Demis?

Sebastian Bubeck

People make things earlier and faster, whereas scientifically, many great scientists credit being in the right place at the right time and still making the innovation, where eventually someone else will still have the idea.

So I think that, in that way, Jensen is helping manifest this GPU revolution much faster and much more focused than it would happen without having a person there. This is making the whole AI build-out faster. But I do still think that eventually something like ChatGPT would have happened and a build-out like this would have happened, but it probably would not have been as fast. I think that’s the sort of flavor that is applied.

Sebastian Raschka

These individual people—there are people who are placing bets on something. Some get lucky, some don’t. But if you don’t have these people at the helm, it would be more diffused. It’s almost like investing in an ETF versus individual stocks. Individual stocks might go up or down more heavily than an ETF, which is more balanced. It will eventually go up over time. We’ll get there. But it’s just the focus, I think. Passion and focus.

Lex Fridman

Isn’t there a real case to be made that without Jensen, there’s not a reinvigoration of the deep learning revolution?

Sebastian Bubeck

It could’ve been 20 years later, is what I would say. Or another AI winter could have come if GPUs weren’t around.

Lex Fridman

That could change history completely because you could think of all the other technologies that could’ve come in the meantime, and the focus of human civilization would get captured by different hype. Silicon Valley would be captured by different hype.

Nathan Lambert

But I do think there’s certainly an aspect where it was all planned, the GPU trajectory. On the other hand, it’s also a lot of lucky coincidences or good intuition, like the investment into, let’s say, biophysical simulations.

I think it started with video games, and then it just happened to be good at linear algebra because video games require a lot of linear algebra. And then you have the biophysical simulations. But still, I don’t think the master plan was AI. I think it happened to be Alex Krizhevsky. Someone took these GPUs and said, “Hey, let’s try to train a neural network on that.” It happened to work really well, and I think it only happened because you could purchase those GPUs.

Sebastian Bubeck

Gaming would’ve created a demand for faster processors if NVIDIA had gone out of business in the early days. That’s what I would think. I think that the GPUs would’ve been different, but I think GPUs would still exist at the time of AlexNet and at the time of the Transformer.

It was just hard to know if it would be one company as successful or multiple smaller companies with worse chips. But I don’t think that’s a 100-year delay. It might be a decade delay.

Nathan Lambert

Well, it could be a 1-, 2-, 3-, 4-, or 5-decade delay.

Lex Fridman

I just can't see Intel or AMD doing what NVIDIA did.

Sebastian Raschka

I don't think it would be a company that exists. I think it would be a different company that would rise.

Lex Fridman

Like Silicon Graphics or something.

Nathan Lambert

So, yeah, some company that has died would have done it.

Lex Fridman

But just looking at it, it seems like these singular figures, these leaders, have a huge impact on the trajectory of the world. Obviously, there are incredible teams behind them. But having that kind of very singular, almost dogmatic focus—

Jim Fan

—is necessary to make progress.

Lex Fridman

Yeah, even with GPT, it wouldn't exist if there wasn't a person, Ilya, who pushed for this scaling, right?

Jim Fan

Yeah, Dario Amodei was also deeply involved in that. If you read some of the histories from OpenAI, it seems wild to think about how early these people were like, “We need to hook up 10,000 GPUs and take all of OpenAI's compute and train one model.” There were a lot of people who didn't want to do that.

Lex Fridman

Which is an insane thing to believe. To believe in scaling before scaling has any indication that it's going to materialize. Again, singular figures.

Speaking of which, 100 years from now, this is presumably post-singularity, whatever singularity is. When historians look back at our time now, what technological breakthroughs would they really emphasize as the breakthroughs that led to the singularity? So far, we have Turing to today, 80 years.

Jim Fan

I think it would still be computing, like the umbrella term “computing.” I don't necessarily think that in 100 or 200 years it would be AI. It could still very well be computers. We are now taking better advantage of them, but the fact of computing remains.

Lex Fridman

It's basically a Moore's Law discussion. Even the details of CUDA and GPUs won't even be remembered, nor will all this software turmoil. It'll just be, obviously, compute.

I generally agree, but is the connectivity of the internet and compute able to be merged? Or is it both of them?

Jim Fan

I think the internet will probably be related to communication. It could be a phone, the internet, or satellites. Compute is more like the scaling aspect of it.

Lex Fridman

It's possible that the internet is completely forgotten—that the internet is wrapped into phone networks, like communication networks. This is just another manifestation of that, and the real breakthrough comes from increased compute, or Moore's Law, broadly defined.

Jim Fan

I think the connection of people is very fundamental to it. You can talk to anyone. You want to find the best person in the world for something, and they are somewhere in the world. Being able to have that flow of information—the AIs will also rely on this.

I've been fixating on when I said the dream was dead about the one central model. The thing that is evolving is people having many agents for different tasks. People already started doing this with different Claudes. It's described as many agents in the data center, where each one manages and they talk to each other.

And that is reliant on networking and the free flow of information on top of compute. But networking, especially with GPUs, is such a part of the scaling of compute. The GPUs and the data centers need to talk to each other.

Lex Fridman

Will anything about neural networks be remembered? Do you think there's something very specific and singular to the fact that it's neural networks that's seen as a breakthrough—like a genius idea that you're basically replicating, in a very crude way, the human mind? The structure of the human brain, the human mind?

Jim Fan

I think without the human mind, we probably wouldn't have neural networks, because it was an inspiration for that. But on the other end, I think it's just so different. It's digital versus biological, so I do think it will probably be more grouped as an algorithm.

Lex Fridman

That's massively parallelizable on this particular kind of compute?

Jim Fan

It could have been genetic computing—genetic algorithms just parallelized. It just happens that this is more efficient and works better.

Lex Fridman

And it very well could be that the LLM—the neural networks, the way we architect them now—is just a small component of the system that leads to singularity.

Jim Fan

If you think of it in 100 years, I think society can be changed more with more compute and intelligence because of autonomy. But looking at this, what are the things from the Industrial Revolution that we remember? We remember the engine, which is probably the equivalent of the computer in this. But there's a lot of other physical transformations that people are aware of, like the cotton gin and all these things, these machines that are still known: air conditioning, refrigerators.

Some of these things from AI will still be known. The word “transformer” could still be known. I would guess that deep learning is definitely still known, but the transformer might be evolved away from in 100 years with AGI researchers everywhere. But I think deep learning is likely to be a term that is remembered.

Lex Fridman

And I wonder what the air conditioning and refrigeration of the future is that AI brings. If we travel forward 100 years from now—if we transport there right now—what do you think is different? How do you think the world looks different? First of all, do you think there are humans? Do you think there are robots everywhere walking around?

Jim Fan

I do think specialized robots, for sure, for certain tasks.

Lex Fridman

Humanoid form?

Jim Fan

Maybe half-humanoid. We'll see. I think for certain things, yes, there will be humanoid robots because it's just amenable for the environment. But for certain tasks, it might make sense.

What's harder to imagine is how we interact with the devices and what humans do with devices. I'm pretty sure it will probably not be the cellphone or the laptop. Will it be implants?

Sebastian Raschka

It has to be brain-computer interfaces, right? I mean, 100 years from now, given the progress we're seeing now, there has to be—unless there's legitimately a complete alteration of how we interact with reality.

On the other hand, cars are older than 100 years, right? And it's still the same interface. We haven't replaced cars with something else. We just made them better, but it's still a steering wheel, still wheels.

Lex Fridman

I think we'll still carry around a physical brick of compute because people want some ability to have something private. You might not engage with it as much as a phone, but having private information that is yours as an interface between the rest of the internet, I think that will still exist. It might not look like an iPhone and it might be used a lot less, but I still expect people to carry things around.

Nathan Lambert

Why do you think the smartphone is the embodiment of private? There's a camera on it. There's—

Lex Fridman

Private for you, like encrypted messages, encrypted photos—you know what your life is. I guess it's a question of how optimistic about brain-machine interfaces you are. Is all that just going to be stored in the cloud? Your whole calendar?

It's hard to think about processing all the information that we can process visually through brain-machine interfaces, presenting something like a calendar or something to you. It's hard to just think about knowing, without looking, your email inbox. You signal to a computer and then you just know your email inbox. Is that something that the human brain can handle being piped into it non-visually? I don't know exactly how those transformations happen.

Humans aren't changing in 100 years. I think agency and community are things that people actually want.

Sebastian Raschka

A local community, yeah. People you are close to, being able to do things with them, being able to ascribe meaning to your life, and being able to do things. In 100 years, I don't think that human biology is changing away from those on a time scale that we can discuss. And I think that UBI does not solve agency.

I do expect mass wealth, and I hope that it has spread so that the average life looks very different in 100 years. But that's still a lot to happen. If you think about countries that are early in their development process to getting access to computing and the internet, to build all the infrastructure and have policy that shares one nation's wealth with another is... I think it's an optimistic view to see all that happening in 100 years—while they are still independent entities and not just absorbed into some international order by force.

Lex Fridman

But there could be just better, more elaborate, more effective social support systems that help alleviate some levels of basic suffering from the world. The transformation of society where a lot of jobs are lost in the short term—I think we have to really remember that each individual job that's lost is a human being who's suffering.

When jobs are lost, the scale is a real tragedy. You can make all kinds of arguments about economics or how it's all going to be okay. It's good for the GDP; there's going to be new jobs created. Fundamentally, at the individual level, for that human being, that's real suffering. That's a real personal sort of tragedy. And we have to not forget that as the technologies are being developed.

And also, my hope for all the AI slop we're seeing is that there will be a greater and greater premium for the fundamental aspects of the human experience that are in person—the things that we all value, like seeing each other, talking together in person.

Nathan Lambert

The next few years are definitely going to bring an increased value to physical goods and events—and even more pressure on slop. So, the slop is only starting. The next few years will bring more and more diverse versions of slop.

Lex Fridman

They would be drowning in slop. Is that what—

Sebastian Raschka

So I'm hoping that society drowns in slop enough to snap out of it and be like, “We can't deal with it. It just doesn't matter.” And then the physical has such a higher premium on it.

Lex Fridman

Even classic examples—I honestly think this is true, and I think we will get tired of it. We are already kind of tired of it. Even art is an example. I don't think art will go away.

You have paintings, physical paintings. There's more value—not just monetary value, but just more appreciation for the actual painting—than for a photocopy of that painting. It could be a perfect digital reprint, but there is something about going to a museum, looking at that art, seeing the real thing, and thinking, “Okay. A human.” It's like a craft. You have an appreciation for that.

I think the same is true for writing, for talking, for any type of experience. I do, unfortunately, think it will be a dichotomy, a fork where some things will be automated. There are not as many paintings as there used to be 200 years ago. There are more photographs, more photocopies. But at the same time, it won't go away. There will be value in that. I think the difference will just be what the proportion of that is.

Personally, I have a hard time reading things where I can obviously see it's AI-generated. I'm sorry. There might be really good information in there, but I'm just like, “Nah, not for me.”

Nathan Lambert

I think eventually they'll fool you, and it'll be on platforms that give you ways of verifying or building trust. So you will trust that Lex is not AI-generated, having been here. So then you have trust in this channel. But it's harder for new people who don't have that trust.

Lex Fridman

Well, that will get interesting because I think, fundamentally, it's a solvable problem by having trust in certain outlets that they won't do it, but it's all going to be trust-based. There will be systems to authorize, “Okay, this is real. This is not real.” There will be some telltale signs where you can obviously tell this is AI-generated and this is not. But some will be so good that it's hard to tell, and then you have to trust. That will get interesting and a bit problematic.

Sebastian Raschka

The extreme case of this is to watermark all human content. So all photos that we take on our own have some watermark until they are edited or something like this. Software can manage communications with the device manufacturer to maintain human editing, which is the opposite of the discussion about trying to watermark AI images. And then you can generate an AI image that has a watermark and use a different AI tool to remove it.

Lex Fridman

Yep. It's going to be an arms race, basically.

We've been mostly focusing on the positive aspects of AI—all the capabilities that we've been talking about can be used to destabilize human civilization with even just relatively dumb AI applied at scale, and then further, superintelligent AI systems. Of course, there's the sort of doomer take that's important to consider as we develop these technologies. What gives you hope about the future of human civilization, given everything we've been talking about? Are we going to be okay?

Nathan Lambert

I think we will. I'm definitely a worrier, both about AI and non-AI things. But humans do tend to find a way. I think that's what humans are built for: to have community and find a way to figure out problems. That's what has gotten us to this point.

I think the AI opportunity and related technologies are really big. And I think that there are big social and political problems to help everybody understand that. I think that's what we're staring at a lot right now: the world is a scary place, and AI is a very uncertain thing.

It takes a lot of work that is not necessarily building things. It's about telling people and understanding people, which the people building AI have historically not been motivated or willing to do. But it is something that is probably doable. It will just take longer than people want. And we have to go through that long period of hard, fraught AI discussions if we want to have the lasting benefits.

Lex Fridman

Yeah. Through that process, I'm especially excited that we get a chance to better understand ourselves—at the individual level as humans and at the civilization level—and answer some of the big mysteries, like, what is this whole consciousness thing going on here?

It seems to be truly special. There's a real miracle in our mind. And AI puts a mirror to ourselves, and we get to answer some of the big questions: What is this whole thing going on here?

Sebastian Raschka

One thing about that is what I think makes us very different from AI, and why I don't worry about AI taking over: as you said, consciousness. We humans decide what we want to do. AI, in its current implementation, I can't see that changing. You have to tell it what to do.

And so you still have the agency. It doesn't take the agency from you because it becomes a tool. You can think of it as a tool. You tell it what to do. It will be more automatic than other previous tools. It's certainly more powerful than a hammer—it can figure things out—but it's still you in charge, right? So the AI is not in charge; you're in charge. You tell the AI what to do, and it's doing it for you.

Lex Fridman

So in the post-singularity, post-apocalyptic war between humans and machines, you're saying humans are worth fighting for?

Nathan Lambert

100%. The movie The Terminator, made in the ’80s, is essentially that. And I do think the only thing I can see going wrong is if things are explicitly programmed to do something harmful.

Sebastian Raschka

Actually, in a Terminator-type setup, I think humans win. I think we're too clever. It's hard to explain how we figure it out, but we do. And we'll probably be using local LLMs, open-source LLMs, to help fight the machines.

Lex Fridman

I apologize for the ridiculousness. Like I said, Nathan already knows I've been a big fan of his for a long time. I've been a big fan of yours, Sebastian, for a long time, so it's an honor to finally meet you. Thank you for everything you put out into the world. Thank you for the excellent books you're writing. Thank you for teaching us. And thank you for talking today. This was fun.

Nathan Lambert

Thank you for inviting us here and having this human connection, which is actually—

Sebastian Raschka

Extremely valuable.

Lex Fridman

Thanks for listening to this conversation with Sebastian Raschka and Nathan Lambert. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback and so on. And now let me leave you with some words from Albert Einstein. "It is not that I'm so smart, but I stay with the questions much longer." Thank you for listening, and hope to see you next time.

State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490 | BidClub