《Hard Fork》直播，第3部分：对 AI 未来的不同构想

Well, Casey, we are still on our annual summer vacation, and can you believe there is yet more amazing stuff from Hard Fork Live that we have not shared with our podcast listeners?

There is. In particular, we had a really fun discussion at the event between Daniel Kokotajlo and Sayash Kapoor, who have somewhat different views of how fast the AI conversation is going to go. We've heard them debate before. We wanted to have an updated discussion with them now that it's been almost a year since the last time they had it. So I think you'll really enjoy hearing what they have to say about that.

We also had the great podcaster Dwarkesh Patel stop by and hang out with us a bit, telling us a little bit about what is on his mind. And just to round it out, we took some live Q&A and heard what was on the minds of our audience after a spectacular Hard Fork Live 2.

So these are all conversations that I would classify in sort of the same bucket of insider sense-making: people who are deeply enmeshed in the AI scene in San Francisco, trying to understand and explain to the outside world what is going on, the pace of progress, and the trajectory of these models. Sayash, Daniel, and Dwarkesh are among the 3 most gifted people I have ever heard try to explain this stuff to an outside world that doesn't always know exactly what's going on.

It's a great set of conversations. We think you'll really enjoy it. This is our final installment of our episodes from Hard Fork Live 2.

We will be back in 2 weeks to our regularly scheduled Hard Fork programming. In the meantime, enjoy your summer. Wear sunscreen.

This next segment I am so excited for because we're going to have a conversation with 2 people who have very different views about how AI is going right now.

Yes. We have Daniel Kokotajlo with us tonight. He is the co-author of AI 2027, a report that many of you, I'm sure, have read. This came out in 2025 and laid out a vivid scenario, or account, of how AI could fundamentally upend the world, achieving tasks like autonomous coding and R&D. He's since updated that prediction a few times. We'll ask him about that.

And he'll be joined by Sayash Kapoor, who is an AI researcher at Princeton with a very different view of the future. He's the co-author of AI as Normal Technology, which looks at evidence that AI is much like previous technologies that have upended the economy, technologies that take a long time to diffuse through society.

We've invited them both here tonight because we saw, last year, a very interesting debate that the 2 of them had at an AI conference called The Curve. We thought it was so interesting that we decided to bring them back tonight and hear how their views have evolved since then, where they continue to disagree, and where they might agree now. So please give a warm welcome to Daniel Kokotajlo and Sayash Kapoor.

Is he with us? Hey.

Hey, Daniel.

Hey.

Hey, Sayash. All right. So Daniel, Kevin mentioned this up top. You have updated your timelines a few times since you first published AI 2027. Give us the most up-to-date view of your thinking. What's your best estimate for when we will achieve AI models that can do their own AI R&D?

Probably 50 percent by late 2028.

Okay. That's soon.

Yeah.

I'm thinking about the calendar. That's 2 years.

Yeah. That's a little bit later than Anthropic expects, I think. Things take longer than you plan for.

Which is a point that Sayash makes sometimes. Sayash, can you summarize where your views are today? My sense is that you do not believe in the sudden takeoff scenario that some other observers believe in.

That's exactly right. I think the main reason for that is this disagreement boils down to whether the bottlenecks to this intelligence explosion, the bottlenecks to automating R&D, are all computational, or whether they rely on real-world bottlenecks that will be really hard to automate away.

This is one place where we disagree. I think that in a lot of domains, making these advances won't be as easy as it has been in coding. And to really get to artificial superintelligence, you need to cover all of these different domains. You need sample efficiency across the board, which is much easier to do in a field like programming, where you have these simulators, these virtual environments, but much harder to do in the real world.

Some evidence bears this out. Adoption of AI systems has indeed been far slower in other domains than in coding.

So give us an example of what these bottlenecks are. Because I talk to a lot of AI researchers, and the way they make it sound to me is, “Look, eventually the model just gets good enough, and then it's game over.”

You're saying that there's something that exists called the real world, and I'd like to hear more about it.

To be honest, I think these are just 2 independent, self-consistent worldviews about the future of AI. And the reason that Daniel and I have had such productive conversations is that we're basically trying to figure out where these worldviews differ.

Speaking of Daniel's actions and Daniel's predictions, they are entirely self-consistent with the worldview that we'll get to AI systems at this point. Unfortunately, in order to get evidence one way or the other, we need to actually carry out lots of evaluations. We need evaluations to be of a much higher standard than we have today.

To give you one example of a bottleneck, the other day I was talking to a lawyer friend of mine, and he uses these tools. He's very bullish about them. But what has turned out to be the case is, as he started using these tools for bigger and bigger tasks, the rate of hallucinations, the rate of unreliable outputs, has remained the same.

Hmm.

Right? It's not because the AI systems haven't gotten better. They indeed have. They are so much better today than they were just a year ago. But the fact is that the tasks that you can do with these systems are actually bounded by the rate of hallucinations or by reliability, and that's one place where AI systems continue to struggle.

In a domain like software engineering, where you have this instant feedback loop, where you can actually run the code and see what the output would be, it's a much easier bottleneck to address than something like the law, where even the right answer is not obvious to a domain expert. Domain experts can reasonably differ in the approach that they take.

So this is just one example of a bottleneck in a domain where the right answer can be a bit more subjective than in coding.

Daniel, I think when AI 2027 first came out, there were some people who dismissed it as speculation or scary science fiction. That was a term that some people were throwing around a lot. I reported on this. I talked to you and your co-authors then. I know that you grounded this in real forecasting work, in months of trying to figure out what would happen as the technology got better.

And I will say that a lot of that has come true already. You predicted in AI 2027 that we would start to see large parts of coding become automated. That much has come true. I was reading today that someone was copying and pasting something that you had written about frontier labs restricting the use of their models for frontier LLM development, something that has happened this week with Claude Fable.

What are the things that you think will happen if your scenario continues to mostly hold for, let's call it, the rest of 2026? What are we going to see this year?

We're not going to see an intelligence explosion this year. In the scenario, that happens next year.

That was close.

So that's nice. I think—

Intelligence explosion being recursive self-improvement leading to—

That's right.

—a sort of out-of-control, runaway superhuman AI.

Or, to put it another way, just fully automating the AI research process, causing AI research to happen even faster than it currently happens. And it's currently happening at a very fast rate compared with many other technologies.

But, yeah, I would say the coding agents are just going to get better and better, and maybe a year from now, maybe 2 years from now, they will be good enough that you can say they've automated coding fully. They haven't fully automated coding yet, but maybe in a year or 2 they'll have fully automated coding, at which point the bottleneck will be research taste and management and all the other aspects of the AI research process besides the actual coding.

Then the companies are going to turn toward resolving those bottlenecks and teaching their AIs to do those skills as well. That's going to take some time, but it's going to go by faster than you might think when all the coding has been automated.

Once they've finished doing those things, they won't have superintelligence immediately. The first AI system that can do the complete AI research process probably won't be able to do various other things. But once they've fully automated the AI research process, things will probably go faster and faster, and then the type of system that can do absolutely everything is probably not far off.

Sayash, do you believe this sort of recursive self-improvement is possible?

In some sense, I think the process of recursive self-improvement started 6 decades ago. In fact, the entire history of computing has been one where we develop tools that then aid us in the development of better tools. We've developed compilers that have allowed us to be 2 orders of magnitude better at programming. We've developed frameworks on top of that. We've developed entire systems and libraries that allow us to do things that would frankly take an experienced software engineer years or decades if they were using assembly language.

So I think, in some sense, this loop has already been kick-started. This loop is something that the entire history of computing bears out. What I disagree with in terms of Daniel's predictions is whether this process will naturally lead us to a point where we develop the automated AI R&D researcher, or whether humans will continue to have this edge and teams of humans with AI will continue to outperform AI alone, and whether this process will lead to artificial superintelligence.

I actually think that it's a very plausible scenario that we get this sort of recursive self-improvement, that AI systems do indeed continue performing better and better at AI research tasks. But the endpoint of that need not be ASI. The endpoint could just be far more capable models than we have today, perhaps following the trend of previous technologies and yet not reaching the point where we have these systems that outperform the top human experts on everything, which is, I believe, the definition of ASI.

Perhaps we should talk about the point of agreement.

Yeah.

What's the point of agreement?

We wrote this blog post together, the authors of “AI as a Normal Technology” and “AI 2027,” where we talked about the things that we agree on. Correct me if I'm misstating it, but roughly speaking, we talk about what you might call strong AGI, or “humans in the cloud”—AIs that can do all the cognitive tasks, or the tasks you can do at your computer, as well as professional humans or as well as the best professional humans.

I guess the headline is: I agree that AIs that aren't that powerful are still normal technologies, and they agree that AIs that are that powerful are not normal technologies.

Exactly. The normal technology thesis sort of stops being accurate or helpful in a world where we have humans in the cloud, let's say.

The reason that we spend this time talking about recursive self-improvement is that RSI is kind of the moment that observers believe is the scariest moment in the development of AI, right? It becomes ever harder to control. So how far away are we from it, and is it possible? I think those are probably 2 of the most important questions that we will ever ask on the podcast.

Having heard what sounded to me like very sensible objections to why it may not be possible anytime soon, and understanding, Daniel, why you do think it's possible, I'm curious: At the very least, do you hope Sayash is right? Would you breathe a sigh of relief?

Oh, yeah.

Would you breathe a sigh of relief?

I would love it if he were right.

Okay.

Yeah.

Okay.

Thank you, Daniel.

But what do you see that makes you think that he's not right?

I think I've tried to spend some time thinking about what the barriers or bottlenecks could be that might block Anthropic from succeeding in its stated plans, and none of them really seem that strong to me, basically.

We can go through them bit by bit. Data efficiency, for example: It does seem like AIs currently are less data-efficient than humans, but that also seems like something that companies could probably make rapid progress on if they tried. Separately, it may not actually be that important for automating the AI research process. It might be that you can 99 percent automate the AI research process without getting that data efficiency to the human level.

Even though that's not quite there, 99 percent automation would speed things up quite a lot, which would then allow you to do a decade or 2 decades' worth of research in a year, perhaps. Those are my 2 arguments for why it seems like we're getting pretty close.

Another argument, a sort of meta-argument that I would make, is that there's been a long history of AI scientists and other commentators making claims about what AIs can't do—various walls that deep learning is going to hit—and they just keep getting smashed through almost as soon as people make the claims. I feel like that's probably what's going to happen with data efficiency, for example.

Let's pause there, because that actually seems really important to me. That's been my observation as well, and it's why I am more inclined to believe the labs when they make grand pronouncements, right?

Sayash, I'm curious: What is your relationship to that? You've also seen these models come along and blow away the benchmarks, seen the evals get saturated, and watched us have to make new ones. In fact, you've been making your own evals because the old ones got saturated.

We've worked on several evals that, for example, Anthropic has used, and that were saturated with the release of Opus 4.5. We were the first ones to say, “Look, this is solved now.”

I think this progress will continue. As long as we can specify things well enough, we'll continue to build AI systems that can solve those tasks. Where I differ, perhaps, is whether the natural endpoint of this process is something like solving data efficiency.

I'm skeptical about that for a couple of reasons. First, sample efficiency, or data efficiency, is not the only bottleneck to getting what we called humans in the cloud earlier. If you look at past progress in AI, we've continued to develop these more general systems. But at any given level of generality, we've been really bad at predicting what the bottlenecks to the next level are. We've been really bad at knowing when we solve those bottlenecks and what underlying transformative breakthroughs are needed to solve them.

As evidence of that, perhaps we can take the transformer moment, and before that we can take all of the skepticism about neural networks that pervaded the AI research community. That took a matter of a few years until the community pivoted, and now everyone is all in on transformers. But perhaps that's not the right architectural choice either. Perhaps we're yet to discover these new architectures that would allow us to make these data-efficient AI systems. Perhaps those will still not be enough to get us to the point where we have the sample efficiency of humans in the cloud.

That's the broad stroke of things. I think the AI community in general has been really accurate about near-term predictions about things that are within the event horizon, so to say, and has been really bad at predicting transformative shifts that change the entire research paradigm.

Credit where credit is due: I think Daniel was one of the few people who got some things right in his report from 2021—was it?—about what 2025 looks like. But in general, I would say the community has a very poor track record.

Say more. What's a prediction that they made that just wasn't true at all?

Come again?

What is a prediction that the AI industry made that just was not true at all?

I guess the entire skepticism about neural networks. From the 1990s to the 2010s, the entire AI community dismissed neural networks as a joke, basically. You could count the number of researchers who took you seriously if you worked on neural networks on 2 hands.

It was only through the persistence of a few people like Fei-Fei Li, who released a big dataset that led to the deep learning revolution, and Yoshua Bengio, Yann LeCun, and Geoffrey Hinton, who later went on to win the Turing Award for their work on deep learning, that this subfield persisted and eventually was able to disprove the claims of skeptics.

In the same way, I think the AI community might be herding too much around, let's say, transformer-based models right now, perhaps at the expense of other transformative improvements—breakthrough improvements that are being sidelined because of the community's single-minded focus on them.

I think an experience that you both have in common, and that Casey and I also share, is writing things that we think are very measured, careful, and precise, and then just having people interpret them in the wildest possible ways.

You both published your breakout essays and scenarios, and both of them were immediately seized on by these polarized camps. David Sax, the former White House adviser, was posting things about AI being a normal technology, sort of agreeing with you and taking issue with you for changing your forecast. “Oh my God, the doomers are backed into a corner now.” Gary Marcus, J.D. Vance, Bernie Sanders and all kinds of people have used your arguments in support of whatever they already believed. How has that been, watching your work ripple out in ways that maybe aren’t what you expected?

I’ll go first. It’s been a sort of leap of faith in humanity. At OpenAI, I was doing scenario forecasts like this, too—much smaller, low-effort versions. But they were just for internal use only. I wouldn’t have been allowed to publish them.

It seemed to me that the world really needs to wake up to AI and what’s coming, and start thinking more seriously about it. The discourse is not necessarily so great, and there are lots of terrible people and lots of terrible takes. It’s very chaotic and confusing. But we at AI Futures Project are making a bet that we should say what we think is coming. We should be clear, articulate and explain our reasoning.

The discourse will get rolling. Lots of people will say lots of things. Hopefully, in the end, it will converge toward the truth. Hopefully, in the end, it will converge toward better decision-making on average. We’ll see what happens. I have faith.

Sayash?

I guess the biggest surprise for me was how few people read things in depth.

Yeah.

It was honestly shocking. In the first line of the essay, we compare AI to the internet or perhaps the electrical revolution. We talk about AI’s impact as being on par with perhaps the First Industrial Revolution, and people sometimes put us in the same camp as Gary Marcus, which is honestly shocking.

But, one level deeper, I think it has been really nice to see these intellectual communities use these essays to advance their thinking. Perhaps the biggest surprise to me was the fact that our essay, and perhaps both of our essays, were taken so seriously by people who are thinking deeply about the future of AI. That was really heartwarming.

Looking back, have you ever had second thoughts about using the adjective “normal” to describe AI? I read your writing, and I think it’s beautifully argued. I share it widely with folks to help them explore reasons why AI may diffuse more slowly than other people think. And yet I have never really thought that AI was all that normal. You know what I mean?

I do understand that. I guess part of it is the fact that we have been in these cycles of discourse where, at least, the people who are thinking seriously about AI take it for granted that AI is transformative, and we do, too.

Within that discourse, there’s this huge spectrum of opinions. Even just between the 2 of us, I think AI will be as impactful as the internet. Daniel perhaps thinks this is the most important invention in the history of humanity. How do you put yourselves on that spectrum?

This was the debate that we felt was really worth having. We’re not interested in the takes of people who think there’s nothing to see here. We actively distance ourselves from that, let’s say in the first paragraph of the essay and in a lot of our writing. I think this is the debate that’s worth having.

Within the context of this debate, I feel like it’s a fair description of where we lie on the spectrum. I don’t know if you agree, Daniel, but I think it’s also been helpful between us to clarify where we stand on this technology. To say that today’s AI is normal technology is a really powerful statement.

Of course, this doesn’t discount the importance of the technology. It does not discount the importance of taking its societal impact seriously. But it does put things into perspective compared with the view that Daniel perhaps has about the future of AI.

AI 2027 warns us that these very disruptive changes are coming soon, so it has a natural set of policy responses that we might want to see in response to that. What is the right policy response to AI if it is normal technology and it’s going to take longer than Daniel says?

One thing that I don’t know if you’ll find surprising, but perhaps many people here will find surprising, is that Daniel and I share a lot of common ground when it comes to policy responses. Both of us value transparency immensely. Both of us value the ability of external third parties to see what’s going on inside companies.

In fact, we were just talking backstage about Anthropic’s release of Claude Fable-5 and the fact that the model is purposefully degraded for tasks involving AI R&D. I think I speak for both of us when I say that this is a very dangerous precedent. We shouldn’t be fine-tuning our models in such a way that they lie to customers. Companies shouldn’t be allowed to do this. They should act in good faith.

That’s the sort of thing where we have a lot of policy agreement. I do think there are areas where we diverge. For example, in more aggressive scenarios, you might want a conditional slowdown. You might want companies to pause.

Whereas when you consider AI a normal technology, the benefits of diffusion of AI and the development of more capable AI systems perhaps outweigh the risks a little bit more, at least in the near term.

It was funny when I spoke to Thomas, who’s another one of the co-authors of AI 2027. We spent hours trying to figure out where on the timelines we actually disagree. It was funny because we couldn’t find any near-term disagreements.

We wrote this blog post together where we say that I agree completely with the events of AI 2027, or at least find them plausible, until the end of 2026, which is a long time. We wrote this last year. In some sense, I think there is much more common ground in terms of policy than you might think.

You guys are being much too agreeable. Daniel, what is something you are worried about more than Sayash is? Then I’ll ask the same question of Sayash. What is an AI risk that concerns you more than you think it concerns Sayash?

In general, strong AGI or superintelligence, that sort of thing. The main one would be loss of control. Number 2 would be concentration of power. There’s a whole bunch of other ones besides that, but I’ll stop there. I can elaborate if you like.

Those seem pretty bad.

Yeah.

Sayash, what about you?

Actually, this is another thing we were just talking about backstage. I was surprised to hear that we disagree far more, or that I’m far more concerned about military uses of AI than Daniel is.

It’s on the list. It’s just—

Perhaps, yeah. It’s true.

It’s just a couple of notches down.

As you both know, in the essay we explicitly carved out military AI because we felt like we weren’t the right people to comment on it. People who are experts on this, like Michael Horowitz, have used our frame to argue that military AI, at least today, is a normal technology in his view as well.

But frankly, the actions that are being taken by countries worldwide, by nation-states, are pretty damn alarming. We shouldn’t take it for granted that companies or countries can use kill bots. That is not something that requires further technological investment, either. It’s not something where we have any technical bottlenecks. We can use off-the-shelf computer vision libraries to basically build killer robots today.

It’s actually something where we need to exercise a lot of agency, and I’m not really positive about where things are going right now on that front.

I truly believe that whatever is about to happen to us lies somewhere in between the views of these 2 people, so we will continue to pay very close attention to your work. Thank you so much, Daniel and Sayash.

Thank you.

Thank you for joining us.

Thanks, guys. That was fun. Thank you.

Thank you.

Yep. Thank you.

Thanks, guys.

It was great.

One thing we know for sure is that no matter what happens with the future of AI, it will be extremely fun to talk about robots.

Yes. So we have already shown you, I think, more than 10 robots tonight, including members of our robot choir. But we have one more very special robot guest tonight. We are about to bring on George Eakins. He is the director of engineering at Tobor Life AI, a robotics company in Silicon Valley that is one of the leading distributors of humanoid robots, specifically these Unitree robots from China. And we are going to be joined by George and Toby the robot. George and Toby, come on out.

Thanks for having me.

Good to see you.

George. Good to see you.

You’re a very convincing humanoid. Oh, no, wait, that’s Toby.

Do we shake hands? Okay. We’ll try it. There.

Hi. Well, short king.

It’s great.

Wow.

I appreciate the weak grip strength. It gives me comfort.

Yeah, it’s sort of like a dead fish handshake.

Yeah. Now he is advancing on me. All right.

Oh, okay.

Okay.

Wow.

Yeah.

Now we’re going to talk about all the things that Toby and his brethren can do, but we heard that Toby can actually dance. Is that true?

That is the case.

Okay. Can we see that? Toby, can you dance for us?

We—

Dan, will you help us out? Hit it, DJ. Oh, Jesus Christ.

Listen, we’ve all been there. Sometimes you just dance to the drop. This robot left it all on the dance floor, ladies and gentlemen. It could have been operator error. Thank you. Thank you, Toby, for your sacrifice. You will not be forgotten.

We’ll add you to the in memoriam next year. Now, is Toby capable of standing up?

Is he okay? Yeah, probably just a misclick on the controller and—oh. Oh, okay. He’s not responding to us right now. We’re so back. We’re so back. He’s absolutely fine. They’re quite durable.

Oh, my God. That was not in the script. No. I’m sorry we’ve traumatized our audience here tonight. I’m so sorry.

Now, George, were you the choreographer on that, or—nope.

Okay. Well, it was great choreography.

George, what is the use case for these other than doing dance demos and sometimes falling over? Who is buying and renting these humanoid robots from your company, and what are they doing with them?

George Eakins

Well, right now, the early market for the humanoids is the research market. People want to collect a lot of data. You guys had the NEO folks on, specifically Bernt, right? And they’re deploying the humanoids into households to try to collect a lot of data in households. People with Unitree robots are also targeting different use cases. Different companies are pursuing different verticals with them and trying to get big data sets and train models on these humanoids.

There is also a set of robots that we sell, which are more reliable and more industrial right now, called quadrupeds, and probably easier just to remember them as the dog robots. You can put LiDAR on them, you can put different sensors—

Put a mask of Mark Zuckerberg or Elon Musk on them. We saw that earlier tonight, yes.

George Eakins

I forgot about that.

Yeah.

George Eakins

Somehow I forgot about that. But they are practical for inspection use cases or security patrols. So those are being pushed out into industry and applications more, and these are on the edge of research and acquiring data to build policies.

How much does one of these cost?

George Ekins

They range in cost. If you want one to just dance around, I don’t remember the exact figure on the low-level dancing ones, but they’re less than the ones that you could put dexterous hands on and then collect manipulation data with on tasks. So you collect data from doing tasks with them.

So, more or less than $10,000?

George Ekins

More.

More.

George Ekins

More.

Okay.

George Ekins

That’s a great question.

Okay.

George Ekins

The ones I was getting to are in the $50,000 to $70,000 range.

Okay.

George Ekins

The ones with the hands.

So, a mid-range sports car.

George Ekins

Yes.

Yeah. All right. I have to say, it did not inspire a lot of confidence in me to learn that the primary use case for these robots is data collection. I think the vision is that these things—as we saw when we talked with Bernt from 1X about their robot, and as we’re hearing about these Unitree robots—the dream is that these things will just be in your house and will be doing chores for you: folding laundry, doing the dishes, cleaning the house. What is the timeline for that? Do you think that is realistic? Should people be pre-ordering now in hopes of automating their chores forever? Where are we on the chore spectrum?

George Ekins

I think Bernt’s very optimistic. I’d put it a few more years out than he would in terms of being in your house. But in terms of maybe operating in an industrial setting, where they can maybe load up a fabricator or something with a material or a part, I think that’s in the next couple years. And there are actually early implementations of that by Figure and Unitree—Unitree in their factory, Figure in the BMW factory. So people are doing that with these, but the widespread adoption, I believe, in the next couple years will happen in those settings.

Let me ask one question about the data collection. Some security researchers have claimed that Unitree robots might have a backdoor that could allow remote users to control or monitor what they’re seeing. Can Toby send the data to China?

George Ekins

So they do send logging data to China, just like every other Chinese thing that you can own, like a computer or any other computer-chip-based thing that connects to the internet and sends logging data. They send that, but they don’t actually—there hasn’t been an established thing that sends camera data or telemetry data of the joints to China. So there are things that people will be like, “Oh, it sends data to China.” It’s like, yeah, and your computer sends data to Microsoft, and it’s because your computer crashed and it needs to send data to Microsoft.

Right. I think the difference is, in this case, that Unitree is a Chinese company, and some members of Congress have become very worried about the fact that these are now being sold in the United States. Some have even proposed banning the importation of these specific Unitree robots. How likely do you think that is, and would that be a big hit to your business? What’s your plan if they ban these?

George Ekins

It would certainly be problematic. I wouldn’t like that. If they’re going to ban all Chinese humanoid robots, I wouldn’t be too stoked on that. So I don’t have much more to say.

Well, much to consider. Before we let you go, does Toby maybe have one more cool routine he could show us?

George Ekins

Yes, he does.

Take it easy. All right. DJ Dan, will you help us out again?

George Ekins

It’s great.

This is like what happened the last time Casey had a Long Island iced tea at the club.

George Ekins

All right.

Okay.

George Ekins

Fascinating.

George and Toby—

George Ekins

George and Toby—

Thank you.

George Ekins

Thanks for joining us.

Thank you.

George Ekins

Thank you.

Ah.

George Ekins

You’re so good. I believe in you.

All right, gang. We are in the home stretch, but we had one more friend of the pod who we just wanted to bring on and have a little bit of fun with before the end of the show.

Yes, our next guest is friend of the pod, YouTuber, and podcast sensation Dwarkesh Patel. Dwarkesh, come on out.

What's up, guys? Good to see you.

Hello.

How's it going?

All right.

How am I supposed to follow a robot dancing?

You could fall over.

You could just face-plant. That'd be great.

Yeah.

Dwarkesh, it's been a hell of a year for you. You are firing on all cylinders, doing interviews with Jensen Huang and other tech luminaries. You've got a new blackboard series that teaches people extremely dense and esoteric concepts in AI. You also got profiled in The New York Times in April, and they made a big deal of you and the media empire that you are building here.

I don't really have a question about that. I'm just kind of in awe of what you have managed to build, and I'm curious what you hear when you hear the conversation about AI 2027 versus AI and normal technology. Where are you on the spectrum of “Everything is changing, the scaling laws are holding” to “Maybe things are slowing down and we don't quite have the breakthrough ideas yet to get to AGI?”

I think fundamentally the scary thing is we realize just how far we are from human intelligence, yet these models are so powerful. That raises the obvious question: When they not only have the current advantages that they do—that they can think thousands of times faster and have a greater ability to absorb knowledge across a wide variety of domains—what happens?

If anybody's used these models for coding work or any sort of computer-use work, you must have experienced this. Then you think, well, there's this huge overhang where humans are able to learn about new things literally a million times faster. If you think about how much information you see from birth to adulthood versus what these models see, we're capable of retaining information across sessions. We're learning on the job. We're not just first-day-on-the-job the way these models are experiencing things.

I think the really scary thing is that we know there's a big difference between where these models are currently and where human intelligence lies. We're making really fast progress toward human intelligence. Already, these things are so capable. What happens when they not only have their inherent advantages because they're digital minds, but also have all our advantages?

You've written and spoken before about how you've tried and failed to automate parts of your own production process—

Yeah.

—with your podcast and your YouTube show, and how hard it's been to get rid of some of the sticky human processes there. Are you having better luck with newer models? Is your operation more AI than it was 6 months ago?

Most of the tokens I see in a given day are produced by AI, so I can't really come here and say, “No, AI is not making me more productive,” or, “I'm not using it in a significant way.”

I do think people underrate how hard it is to automate jobs. People underrate how much it takes to do every single thing a human—even a white-collar worker—might be doing. At the same time, you guys must be finding this as well: The ability to triage huge amounts of information, which is a large part of my job, has just gotten way better. How have you guys been finding these models?

I mean, sort of the same. I do feel like with each of the big leaps in model capability, they become better at tasks that are quite useful in, for example, preparing for a podcast.

If we're sitting down with a guest that I'm not that familiar with, I can say, “Go out and prepare a briefing document for me about this person, and give me some interesting directions to maybe take the conversation based on things they've said in public in the last 3 months.” That's absolutely a job that I could have hired for, and now I can get it in about 4 minutes on my computer.

So that's really useful. Does it make me more productive? Yes. But do I work less or use the computer less? No.

I'm finding something similar. I want to use these models to automate a lot of my life, and I've been very successful at doing some pieces of it. But there are just things that—

Now, the primary feeling I had when I got access to Claude Fable yesterday was, “I am too dumb to use this thing.” I actually don't know what I would prompt it to do that a previous model would not have been able to do.

But I'm not building RL environments. I'm not overseeing training runs. So what is the use for you as a media figure and podcaster? What is the thing that you wish the models could do that they can't currently?

First of all, every time I'm bearish or say something bearish about the models, I put it in the context that we're living in an absurd timeline, and I am reacting to my close friends who are just—well, you just had some of them on, and they're talking about the singularity in 2 years.

I feel like we're so used to what these models are capable of currently that we ask these questions like, “What is it that they can't do? Aren't they clearly already AGI?” It's like, no, we all have jobs. That wouldn't happen in a world with AGI, right?

Just get them to do something pretty basic. For example, I'm negotiating with a sponsor for next season or something, and they ask for the back-and-forth there with the relevant context about how we think about our business and stuff. That's probably a 1-hour-horizon task for me or my general manager. The models couldn't do it at all.

Or, let's say, book a show in another city—book an event like this, right? There are a lot of people who are involved in this. What part of it could the models do reliably? I've—anyways, all this to say, I think people really underrate the range of human, even white-collar, work.

I mean, it seems to me like it might be very helpful in a negotiation, though. Particularly, you're not in this position, but maybe you're just starting a new podcast and you have some interest from a sponsor, and you say, “Go tell me something about this market, and what's the best place to get started?”

For sure.

I could see it compressing that into a much smaller problem, but to your point, somebody still has to do the rest of the job.

Yeah, that's right. They can't do something on a computer you might want them to do, right? It's actually quite interesting: Why are they so bad at computer use, given that it's an extremely verifiable domain?

I think that actually goes to show you that it's not just about verifiability. It's about the ability to—the environment has to be one which allows you to deterministically run many parallel rollouts at the same time. If you try to do that on Amazon, Andy Jassy will just shut your ass down.

They have to build clones of every single website, because it takes a ton of data in the relevant domain for these models to become competent at learning how Amazon works or Slack works. So you have to build clones of those things. That's very labor-intensive.

I think we'll make progress on that as well. But yeah.

One of the issues that you really brought to the forefront of the industry's conversation over the past year, I would say, has been the failure of these models when it comes to continuous learning, right?

It's often observed that a good LLM might be better on day 1 than an intern, but the intern is almost always better after 2 weeks because they've been able to learn. Are you still as convinced that this is going to be a major hiccup to getting us all the way to AGI, or have recent developments—maybe any new models—changed the way you think about that?

There's a big crux in how people think about how these models will evolve. One side of the discussion says you need some way in which, between sessions for a given user, the weights themselves are updating.

If you think about the way humans learn, there's not—you know, you're way better at your job than you were the first day you were on your job. People often say an employee's not net productive until 6 months on the job.

What is happening in that time? It's not like you're building up this intensely accurate episodic recall of every single thing that has happened to you over the 6 months, which is what in-context learning is like. That just grows linearly in size as you spend more time on the job. There's some distillation back into a higher-level abstraction that's happening over time, and so the real question is, does there need to be an updating that happens back in the weights?

Some people say, well, no. Basically, you'll get to a point where these models are spending 6 months on the job, and that 6 months is happening in context. We're going to train them in such a big variety of RL environments that they'll learn how to adapt to any given situation you put them in. My question with something like this is, I think that might be enough to get these labs to, like, $1 trillion in revenue or something—truly ludicrous outcomes. I'm concerned about, or also interested in, what role this plays in getting us to superintelligence or something like that.

One question you could ask is, how would you build something that is as good as Henry Kissinger at politics? There's no relevant training environment for that you can run in a data center, and so you do need something that can learn that on the fly. Maybe just by doing enough RLVR, you build something that can pick up whatever Kissinger picked up throughout his life through interacting with the world. Maybe not.

You know, the headline coming out of this talk is going to be, “Dworkin Says Henry Kissinger is Good at Politics.” So I'm just preparing you for that.

LBJ or whatever. The example doesn't matter.

Okay.

You know what I'm saying.

Interesting. You have a very old soul. All your references are to the mid-20th century. You live in San Francisco with Sholto Douglas, a researcher at Anthropic, and Dylan Patel, of SemiAnalysis, a very influential semiconductor newsletter. You guys are—

Have you seen the rent, man? I gotta split it.

Well, that's my question. SemiAnalysis is reportedly making something like $100 million a year in revenue. Anthropic is obviously very valuable. At what point are you guys rich enough to not need roommates?

The problem is everybody else in San Francisco is also getting so rich. And so the housing is increasing at the same rate that our net worth is increasing. We're never escaping this.

One knock that I sometimes hear on the San Francisco AI scene is that it's all very clubby and insular, that there aren't a lot of people who are doing the work of holding people to account or being appropriately skeptical. One detail in The New York Times profile of you was that you sometimes invest in companies whose CEOs or leaders you interview. Do you think that journalists and other more conventional media people have the wrong framework for thinking about conflicts of interest, or do you just think you're doing something different?

I totally see the rationale for journalistic policies that say you're not allowed to have any sort of financial entanglement with the company that you're covering or whatever. I think at the end of the day, I hope the product speaks for itself, and that if you watch an interview I do with a CEO or an executive, you hopefully feel like I ask the relevant questions. Look, I also don't try to steelman some objection that I don't have.

But when I do think that they're not making sense, I try to say so. I hope that that, in and of itself, speaks for the interview.

Who's your white whale? Who's the guest that you wish you could book that has not agreed to come on?

Robert Caro. Can you make this happen?

Robert Caro? Okay. Robert, if you're out there, go on our podcast.

I will say—

Come on Hard Fork first.

Yeah, I will say that Robert Caro was also famously Conan O'Brien's white whale, and Conan O'Brien never got him on the show.

No, he got him on.

Did he?

Yeah, on Conan O'Brien Needs a Friend.

All right. He just fact-checked my ass.

Yeah. Well, Dorkest, the podcast and the show is amazing. I learn so much from it. I listen to every episode, and I understand about 80% of it now, which is up from about 20%. So I'm learning along with your audience, and we thank you for all the work you do. It's a great show.

Thank you, Dorkest. Great seeing you guys.

Thank you. Good to see you. Thanks for having us.

Thank you.

Yeah.

All right.

Okay. Well, friends, we are almost there at the finish line. But before we go, we wanted to take some questions. If any of you have questions for us, we will spend a few minutes answering them. We have mic runners upstairs and downstairs, so raise your hand. Someone will approach you with a mic. Anything—we're an open book. You can ask us about it all. It's like a YouTube comment section, but in real life.

This one right here.

Guest

Hi, my name is Dallon. I'm here with my brother from Utah. What happened to the Fediverse? The Forkverse, I should say. What's—

The Forkverse was, of course, our effort to build a social network in a federated way and show people what it would be like to be part of a social network that wasn't owned by a giant corporation. I think it just ran into the challenge that any social product does, which is that if you're not constantly bringing in new users, its default state is to shrink. We've been in discussions recently about what the future of it is.

I think it was a fun experiment, but we didn't really have that strong of an idea of what was going to happen after we started it, and so we're now living with the consequences of that. Yeah.

Balcony, do we have anyone in the balcony?

Oh.

Yes.

Guest 2

Hi, Kevin and Casey. I was wondering why we're not hearing more from executives like Satya and other tech leaders who are restructuring their companies around the premise of AI. They just don't seem to want to engage with that premise when you ask them. What do you think that's about?

I mean, I think there's a lot of conflicting incentives here, right? There are some companies that really want you to know how much they are using AI and how much more productive they are getting and how many workers they are laying off, and sometimes that's real and sometimes it might just be covering for some over-hiring they did a couple of years ago.

I think that's going to flip at some point, where companies will not want to advertise the fact that they are restructuring around AI. Right now, there is still this weird market premium for that. I think that will continue for as long as the market premium lasts, and then it'll be like, “We're just going to sweep it under the rug and hide it. And if we're going to lay people off to replace them with AI, we're going to call it something else, because we don't want to deal with the backlash.”

I think that really hasn't happened yet, which has been a surprise to me. What about you?

No, I agree with that. And in the interest of answering as many questions as possible, I think we should move on to the next one. One right here.

Guest 3

Hi, my name is Ena. I work at Quizlet. If you've gone to school in the last 20 years, you've heard of Quizlet. If you haven't, what? Education is obviously being radically changed, but what people need to learn and the fact that you need to learn don't really change. So I'm curious: if Quizlet were to start everything from the ground up tomorrow, what do you think we should build?

I mean, that is really challenging. Kevin and I get a chance to go speak in schools from time to time, and I think what we find is people who are doing their absolute best to introduce fairly incremental change and see what happens. There's just tremendous uncertainty right now.

School is typically trying to educate you for a fixed target. When I went to journalism school, it was like, “Well, if I get these skills, then I can have this kind of job.” I think we're not able to ask any guest on this stage about anything longer than a 2-year timeline, because none of them credibly have anything to say about that. So how do you educate a 5-year-old so they'll be prepared for the world when they're 18? Good luck.

What an inspiring message. Thank you. All right. Let's take a couple more. Yes, up there in the balcony.

Guest 4

Okay. Can you hear me?

Yeah.

Guest 4

Okay, great. Hi, I'm Liz.

Hi, Liz.

Guest 4

Okay. So, 2 real legitimate questions. Number 1, what are we wearing now that Allbirds is under? Okay. And 2, I work as a regulator. I work for the state of California. I do privacy regulation. My question is, if you were to take a stab at what privacy would be in the AI-native world, how are you going to protect your digital selves, either your own or your friends? What are we going to do when it's all owned in 1 walled universe?

Yeah, I mean, my hope is just that that is not the case.

We asked Cindy about that tonight. I think there is a lot of logic in having some kind of privilege-like system that protects certain kinds of conversations that you would have with a chatbot, the same way that a conversation with a lawyer might be protected. But I also think there’s a lot of wisdom in what she said: What systems can we build that would ensure that that sort of data never makes it into the hands of a big corporation?

I think we should outlaw data brokers. Next question.

Oh, yeah. Outlaw data brokers. That’s a good one. What’s that?

Guest

Where do you guys get your shoes?

Oh, yeah, and where do you get your shoes, Kev?

These are from Quints. That was not sponsored content. Yours are better, though.

I got these from online, unspecified. I honestly don’t remember, but I can look into it. I’ll figure it out by the reception. How’s that? All right, just a couple more.

Guest

I’m a software engineer, so take this for what it’s worth. There’s been some talk about lots of people being afraid of jobs going away, and then you hear other people saying, “Oh, there’s tons of hiring going on.” That’s what I see. I see a lot of hiring going on, but it’s all for senior engineers, for people who know how to fact-check the models or how to architect and combine the things that they can do really fast. What’s happening with the entry-level folks? It seems like that is a real problem.

Yeah, so I’ve talked to a couple of labor economists about this within the past couple weeks, and they have said, believe it or not, things were actually much worse during the Great Financial Crisis, and the circumstances that we’re seeing today don’t approach that at all. Now, maybe they will eventually, but one labor economist I talked to, Catherine Anne Edwards, was telling me that some people sometimes forget that your first job just sucks and has nothing to do with the thing you actually want to do. And so she’s encouraging younger folks to manage their expectations, which is also not a very inspiring message.

I think we can do 1 more question, so let’s have the last question. Yes.

Guest

Hey there. My name is Kevin. Oh.

Great name.

Guest

Are we good? Yes, my name’s Kevin, and what is your optimistic view over here in the middle if you’re looking out? What is your optimistic view on AI for about 3 years out, 2 to 3 years out? Just curious to get y’all’s take.

Yeah.

My optimism is around the acceleration of science and medicine. This is really a place I care a lot about. I don’t know if any of you saw the cheering at the conference the other week where they announced that they had created a new breakthrough therapy for pancreatic cancer. I want there to be many, many more of those very soon. Yeah, thank you.

So that is my case for optimism: We muddle through the transition from the old jobs to the new jobs, we deal with the safety risks that are really extreme, and then we just accelerate the hell out of the things that make people’s lives healthier and longer and allow us to flourish.

Yeah, that’s my number 1. But 2 more I would throw in there are: AI is amazing for learning, and AI is amazing for building, and it’s fun to learn, and it is fun to build. If I were in school right now, I froth at the mouth thinking of what it would have been like to take my AP exams in a world where I could have ChatGPT generate infinite quizzes for me to do.

Kevin and I have talked a lot on the show about vibe coding in the past year. I’ve been making new projects this week and annoying my fiancé and making him come see them, even though they’re just pure slop. But it is fun to make things in AI—

It is fun to annoy your partner with random AI stuff that you build.

Yeah.

We are gonna stop it there so that we can get to the reception.

We’ll see you all at the reception. Thank you—

Thank you.

—so much for coming. Thank you. We love you.

We love you.