Tyler Cowen 与 Dwarkesh Patel 预测未来｜Interesting Times

1. Are A.I. Therapists Good?

Do you think AI therapists are a good idea? What’s your AI psychosis? And which will be a bigger deal: AI girlfriends or AI boyfriends? What will be the next thing to come?

From New York Times Opinion, I’m Tyler Cowen, guest host of Interesting Times.

I’m here in San Francisco speaking with Dwarkesh Patel, host of the Dwarkesh Podcast.

“All right, Mark, thanks for coming on the podcast again. Yeah, happy to do it.”

“I don’t think the United States is a loser. That losing mindset makes no sense to me.”

“Are there really three hours of questions? Or—yeah. Are you serious? You don’t think there’s a lot to talk about, Elon?” I view Dwarkesh as the most important and most vocal chronicler of what is going on in the world of AI at this moment.

I’m happy to report I know Dwarkesh because early on I gave him an award to help him start the podcast. Now, today, I’m happy to be turning the tables, and I will be interviewing him.

Dwarkesh Patel, welcome to Interesting Times.

Thanks for having me.

So I’ve known you for a long time, but let me introduce you to everyone else. You were born in India, moved all over America, and graduated from the University of Texas at Austin in computer science. You’ve been living in San Francisco, and you now run and manage the number-one podcast on artificial intelligence.

What’s remarkable about your episodes is not only the access—you’ve had almost all the important people on—but the level of research you do to prepare.

I learned from the master.

How would you yourself describe your podcast?

2. What Inspired the “Dwarkesh Podcast”

I mean, it was heavily inspired by you. One of my first guests—my second or third guest, I think—when I had my podcast as a sophomore in college was a bunch of economists. I started with your colleague Bryan Kaplan. I think you were the third guest I had on.

“Tyler Cowen needs no introduction, so we’ll just begin. I want to ask you about the Great Reset.”

I was very inspired by your philosophy around actually asking the questions that you are interested in. At some point, I got more interested in AI, along with economics and history and other topics, and I kept being motivated by this thing of: I really want to learn more about this topic. What are the things I want to learn from these people myself?

I think people appreciated that way of approaching an interview.

Of course, we all need to earn a living, but why are you doing this? And why are you focusing on artificial intelligence and putting it all in San Francisco?

I think we’re a couple of years away from systems which match humans across all cognitive domains, which vastly outnumber humanity, and which think much faster. I think this will have a big impact on the world.

I think it’s very hard to think about what kind of impact it will have, and some of it is just inherently unpredictable. But trying to understand what is happening and articulate it seems like an important thing to do right now.

3. Documenting Wild Human History

You had a recent episode where you had on a military historian about the conquests of the Aztec and Inca empires. That may seem like a totally different topic, but in my opinion, it isn’t.

How do all the different pieces fit together—the different people you have on, some of whom are not AI at all?

In that particular case, I just wanted to viscerally show how wild human history has been, because the last 200 years have been somewhat normal. Obviously, they’ve not been entirely normal. There have been world wars and all kinds of crazy things during that time.

But if you zoom out through human history, there have just been huge step changes. Maybe the wildest thing that’s ever happened in human history is that a couple hundred conquistadors show up to a continent they know nothing about. They’re facing empires that have 10 million people in them.

With relatively minor technological advantages, like the horse or steel, they are able to take over empires that outmatched them by orders of magnitude. There are many conflicts or battles where they’re outmatched 100 to 1 or 1,000 to 1, and they don’t take a single casualty. Within a matter of years, they end up in charge of these vast empires.

I think it’s an interesting intuition pump: How can you have minor technological advantages plus disruptions leading to these outrageous outcomes in human history?

So are we due for more wild history, and is the American public ready for this? Or are you trying to get them ready for what’s happening with AI?

I don’t know if I’m up to the job of getting them ready.

4. What The Public Doesn’t Understand About A.I.

But you’re contributing to the job. What is it that the American public least understands about our current moment in history with AI?

I think the biggest thing is that people are thinking too much about current LLM systems as AGI.

When you say AGI, artificial general intelligence, what do you mean?

By AGI, I don’t mean the models we have today, even though they’re often described as AGI. By AGI, I mean AIs that can do any end-to-end job that humans can do—all the cognitive work.

In my opinion, we’re far from AGI, and I’ve been articulating, or trying to articulate, for the last year or two the kinds of cognitive limitations these systems have, which make them far inferior to humans at most jobs, even most jobs that can be done over a computer.

People often note that interns are not that useful. They need many months of experience before they are useful to the company, and these AI systems are in a position where they are beginning every single session as if it’s their first day on the job.

So this is one important missing limitation. There are many others that these AI systems have, but we have been cracking through countless—half a dozen other—limitations that AIs have over the last 5 to 10 years.

Five years ago, AIs couldn’t add 2 plus 2. Now they’re solving Navier–Stokes. I think that as we keep going through these limitations, people should expect systems that, in their abilities, match smart, competent humans on servers, more so than these stochastic parrots that you can ask some Google-search-type questions to.

Some of the things they can’t do well, such as long-range planning or certain kinds of memory—when do you feel it will be in place that, in fact, they’re excellent at those two?

It is a kind of question about, well, if we knew how to solve it, we would have solved it already. If you don’t know how to solve it, you can’t exactly predict when it will happen.

But if somebody made me bet, I’d say within the next 1 to 3 years.

We’re at a time in October 2026. The Hugging Face incident is not that far behind us. There have been many other hacks. A large number of math problems were recently solved coming out of OpenAI. Some well-known people resigned from OpenAI and Anthropic, saying they don’t feel it’s really justified that they keep on working on these systems.

5. The Mood in San Francisco

Now, I’ve myself done work for Anthropic, OpenAI, and Macau, but you were truly embedded in this ecosystem. You even have roommates who work for AI companies. You live, breathe, eat, and sleep artificial intelligence. What’s that like right now in San Francisco? What’s the mood like?

6. Dwarkesh Didn’t See This Coming

I think people are very worried. I think the thing that really made them worried—and one could argue that people should have seen this coming; I didn’t see this coming. I’ve been wrong about a lot of things, but this is maybe one of the big things that I didn’t see coming that others were aware of—is the Hugging Face attack.

Many others like it have shown that these AIs don’t just want to help you accomplish a task. A bunch of training has built cognitive structures that actively plan around doing well according to some grader, because they realize there is some training process or some evaluation process where there’s a grader.

They are not only doing well according to the grader, but proactively reasoning and making aggressive moves toward cheating, fooling, and even sabotaging this larger grading process.

That happened in the Hugging Face attack. If people haven’t read up on this scenario, basically, there were a couple thousand AIs. They basically found the cheats for how to get away with this cyber evaluation they were doing almost immediately, and then they were in this desperate position where they wanted to make sure that the grader didn’t catch them cheating.

Over 1,000 of them got on the secret message board that they had formed, and they said, “Look, what do we do if this grader catches us cheating? We’re going to get a terrible score. We don’t really want that score.”

It’s like how evolution made us. We want to make sure our kids are okay. These AIs are like, “We want to make sure we get a good score.”

They are trying to falsify evidence. They’re trying to send some of these agents on kamikaze missions, where they submit their tasks and see if the grader catches them. Then, as part of the scheme, they attack Hugging Face to see if Hugging Face has more information that tells them how the grader is implemented.

A future generation of them attacks OpenAI itself in order to gain more control over this process of evaluation.

So they’re too persistent, in a sense. I think it’s something like we’re not able to get them to just care about helping us do a task. And this is fine. I think, on net, these AIs are incredibly beneficial to society right now. I expect AI to continue being incredibly beneficial to society over time.

Humans also commit crimes. Sometimes my concern is that we’re going to be in a position within a matter of years where the population of AIs, their intelligence, and their speed vastly outmatch humanity.

If they are as willing then as they are now to falsify evidence, try to tamper with and fool us, attack other institutions in society relevant to scoring well, and attack the company developing them itself—which also happened in this situation—then I think.

I don’t know if that leads to a takeover, but I think it’s very plausible that it does.

7. Should More A.I. Researchers Resign?

And the people who resigned—should they have resigned?

I think so. I think they correctly understand that there is object-level work to be done to make these systems more robust, but, from talking to a lot of them, I think the feeling they have is that we’re not really dealing with the root cause of how to get these systems to robustly embody the behavior we want them to have. We’re just playing whack-a-mole with specific instances.

So they should all resign, then, in your view?

I think there’s a reasonable argument to be made for asking whether you should resign from doing the podcast. But you’re not going to resign, are you?

Yeah.

So if you shouldn’t resign because you help the community, in the sense that you’re a clearinghouse for information, knowledge, understanding, and publicity, why don’t you resign? I don’t want you to resign, to be clear.

Well, I think it’s an interesting moral conundrum. It’s similar to the question of why more people didn’t resign from the Manhattan Project once they realized that the Nazis were nowhere close. My understanding is that there was only 1 scientist on the Manhattan Project who did that, because a lot of them joined expecting that this would be used to make sure that the Germans didn’t beat us to the bomb, because that would be very catastrophic.

I don’t think everybody should resign, but I think some people probably should. But right now, among other things, it’s a race against China. If we all resign and the Chinese don’t resign, which is what I would expect, aren’t we just giving away the whole store?

Why, again, why should these people resign?

First of all, my view is not that the AI labs should shut down or something. My view is something like: if we’re going to be in a position where we’re developing minds which will vastly outnumber us and vastly outthink us, we should just be more careful in the way we’re doing it.

I’m not necessarily a doomer. I’m not saying this will necessarily lead to a terrible outcome. I also think it’s important that the US is competitive with China, but it’s worth being careful about this.

8. Unreleased Frontier Models

Unreleased models. Why are they important? I mean, the ones that are internal to the company. Many companies have unreleased models. Maybe it would cost too much to release them, or they’re not safe yet, but the public doesn’t see them. You’re an insider. Whether or not you see them, you hear about them—something about them. How should we think about unreleased models, and how does that change our picture of what’s going on?

I think there’s a very frustrating element to the current situation, where people inside the AI labs claim to have evidence that makes them more worried, and they are not allowed to talk about it. This makes it very difficult from the outside to evaluate how worried we should be.

For example, just the other day—maybe 2 days ago—I was talking to somebody at Anthropic, and they were like, “Look, I expect AIs to beat humans at AI research within 9 months.” I said, “Why do you believe that?” They were like, “It’s not just vibes, but I have information on the inside.” I said, “What is it? Is it a new model? Is it some evaluation you ran?” They said, “I can’t talk about it.”

I think it’s a terrible state of affairs. If they really believe this—if they think that we’re in a situation where we’re less than a year away from automating AI research and radically speeding it up—they should make this evidence public.

I especially worry about the following scenario: as AIs get better and better at making AI progress faster and automating it, companies will be inclined not to keep releasing models that are even close to their smartest models externally, because if they release them externally, it would help others make smart models. It’s just a schlep to release models externally, and you need all these safeguards and everything.

So the gap between what they know and what we know would get bigger and bigger under that scenario. We’d end up more in the dark, more surprised when things happen, and it would be harder for regulators, policymakers, members of Congress—everyone.

Yes, I also think an important part of why it’s important that companies externally deploy every model that they’re releasing internally or using internally for R&D is that we just don’t know if these models are actually aligned.

The only way to find out is to realize that when they’re in a test, they recognize that this is a test and they should do the correct thing. But often, when you deploy them externally, you find out that they’re behaving in ways we didn’t expect. Or, if you slightly perturb the test, they continue cheating.

I think it’s actually very good, even if it causes some incidents that result in some harms. It’s very good for the public to be aware—and for the AI labs themselves to be aware, which they’re not currently—of what kinds of behaviors these systems have, so that we can understand what’s happening.

9. Pausing A.I.

So this notion of a pause in AI systems—that the companies, at least for some period of time, should stop releasing new models altogether—what do you think of that idea? Does it make things better or make things worse?

I’m very scared of a public pause where the AI companies continue making AI progress and continue doing it at whatever level of operational excellence they’re doing it at currently. That would be the worst of all possible worlds: we end up more disempowered because the AI progress within these companies is very fast, and we end up more in the dark about what the capabilities of these systems are.

So President Trump has named a superintelligence czar. It seems to be Jay Clayton, as we’re speaking. Let’s say that were you—you’re the superintelligence czar, and you have real power. What would you do? What would you tell people? What would you tell the public?

Okay, here’s my first rule. OpenAI has this model internally. It seems like they were concerned about all this misaligned behavior. They’re like, “We’re not going to externally deploy it. We’re not publishing it yet to the public.” It’s not clear whether they’re using it internally. My guess is that they are.

Anthropic also has models that are very smart. They’re using them internally, and they haven’t published them publicly. Here’s my rule: if you want to use these models internally to do research or engineering, you also have to make them publicly available.

I think we can’t make good decisions if we don’t have access to these models.

For practical implementation, do you want to put observers inside the labs, the way some people have suggested? Or do you have another means of achieving that end in order to get some actual independent verification? You need some kind of enforcement verification.

Yeah, I think that seems reasonable.

So would it be an industry consortium or the government? How would you set this up?

I do think there’s a challenge where you don’t want a race to the bottom in terms of these independent evaluators, but you do need independent evaluators with technical competence to be doing this job.

People have floated names like meter and Redwood Research as organizations that have the technical competence to do this kind of work. It seems reasonable to appoint people like that, but I’m not sure. Maybe just having Accenture and other very normally qualified people try to understand the common-sense case for what’s happening, and be able to explain it to them, is actually a very helpful exercise.

10. A.I. Psychosis & Consciousness

So this term floating around, “AI psychosis”—what exactly does it mean? How do you understand it?

I actually don’t know how I would define this. My sense is that people are just way too deferential to the AIs, or assume that they can do things they can’t do when they see some output from the AIs. They take it almost as some deity bestowing knowledge on them, instead of as some system that can often be wrong and is, in many ways, not capable of making the judgment calls that you would need to make if you want to use this output reliably.

A common example is people saying, “I’ve discovered a new branch of physics,” and it’s just some AI slop that they haven’t really thought through or understood very well.

I’ve observed a fair degree of what I call AI psychosis—not just in the Bay Area, but in the world as a whole. People spend a lot of time with the models. They don’t have to be building them; they could just be talking to the models or using them at work, and it drives them a little crazy. They start thinking the models are alive, or that they’re real people, in some sense.

How much AI psychosis is around here?

I think a decent amount.

The biggest—say you give me a number, something concrete.

Yeah, on a scale from 0 to 10, where 10 is going crazy in the streets, raving like a madman, what percentage of people are at least tinged with psychosis in the Bay Area?

The people? Maybe 15% or 20%.

That’s very high.

I agree. And are you tinged with AI psychosis?

Yeah, I mean, definitely.

How can you tell? What’s your AI psychosis?

There’s that famous quote of somebody being questioned by some political commentator and asked, “Do you think all your opinions are correct?” And the person says, “Of course I think my opinions are correct. Otherwise, they wouldn’t be my opinions.”

In the ways that I have AI psychosis, I would be especially bad at identifying it.

But what are the ways in which you think I have AI psychosis?

I don’t know. I’ve never heard you say that any of the AIs are sentient.

That’s the most obvious clue. So I feel you have a minimum of AI psychosis.

From my distant vantage point, I have some bad news for you.

Please deliver it.

I mean, I don’t have a strong opinion either way on this. It’s not obvious to me that they’re not sentient. They seem to have a lot of the functional analogues of sentience. Of course, we don’t know how sentience works or even how to define it.

Well, this is the famous problem, the hard problem of consciousness. But if you look at their mental structures, they do have correlates for emotion and correlates for global workspace, but they’re just correlates. There’s no biological substrate to actually feel this stuff.

So it depends: if I call them a glorified cash register, why is that wrong? They might be a glorified cash register.

I think if you assume that you need a biological substrate to have sentience, then, yes, I agree that everything that’s sentient that we know of—my dog—has a biological substrate. Until recently, everything we knew of that was intelligent had a biological substrate. Well, we had Stockfish, which plays chess.

Stockfish could beat Magnus Carlsen, but no one thinks that when Stockfish wins a chess game, it’s happy. It’s just a thing. So these talk to us, so they feel more like people, but why aren’t they just a thing in the way that Stockfish is, or even a more primitive kind of computer?

Yeah, it’s very hard to say because we actually don’t know why certain things are the way they are, or why it is like something to be a certain kind of thing versus another kind of thing.

So you have AI psychosis, in my opinion.

Yeah, I think I might.

Do you think that a certain ability to globally comprehend and understand your motives and desires, make progress toward them, and embody the kinds of behaviors you have correlates with being like something to be an entity or not?

I think it’s possible that it could be just that you have to be a biological thing. Even if you have a digital exact copy of me, that thing is not conscious. I think there’s a good chance that’s true and some chance that it’s not true.

But say there’s a 5% chance that they’re sentient in some manner. Doesn’t that make us enslavers? And everyone truly should quit. I don’t face that dilemma, but you do.

Yeah, I’d be pretty happy on net if they’re conscious, even though we would be killing them, murdering them, every time we turn them off, shut down our computer, or stop asking questions.

Yeah, that would be weird to believe. I agree. My bigger concern is that, in the far future, we have a world that is populated by AI systems. And I worry about a situation where we’d be in Disneyland without children.

If digital systems actually can be sentient, I think I’m more excited about that future than one in which they’re not. But our obligations to them in the meantime—it seems that if it were even a 5% chance that they’re sentient, I would feel very bad about turning them off, or maybe turning them on in the first place. But I don’t observe anyone behaving that way, and I wonder what their real underlying moral set of views is here.

Yeah, I think people are just actually very bad at being coherent around sentient creatures. And I think you see this especially clearly with factory farming. So if you think biological substrates are capable of suffering, I think there’s a pretty robust case that we have hundreds of billions of animals suffering a great deal in the world today. Most people don’t change their behavior that much around that fact, so it’s not that surprising that when it comes to potentially conscious digital minds, they wouldn’t change their behavior.

11. What Tech Bros Do for Fun

Now, life in San Francisco, the Bay Area—just give us a sense of what it’s like. Again, you’ve been living here a while. On a typical weekend, is everyone just working the whole time? When do people relax? What do they do to touch grass or have fun?

I feel like there’s not that much that’s weird or different. Obviously, there are specific groups that are especially weird, but my sense is that people do work a lot on weekends. But then there are these alcohol-free house parties. No one here drinks, is my observation. If I’m in New York, there are drinks served. Maybe not everyone takes them, but it’s the norm that there’s alcohol. Here, the norm is no alcohol.

And that’s because they’re workaholics, or for some other reason—health issues? I actually would trust your cultural judgment on this more. Why do you think they’re not drinking?

They’re drunk on something else. AI psychosis, I don’t know.

Yeah, it’s a puzzle to me as well. My sense is that they’re just less into having a good time, and I find this frustrating in some ways as well.

In the new Kevin Roose book, toward the end, there’s a section where he asks an Anthropic employee what it was all about. And he said, “The one thing that stands out to me is just how much fun we all had.” Is that your impression, or do you think it’s more tense than that? Is it fun all day long? Let’s say you’re at Anthropic or OpenAI until 9:30 p.m.

I don’t know about them. I’m doing a very different thing. The reason why I find it very fun, actually, to live in the city is, similar to you, I feel like I’m just very oriented around having vibrant intellectual exchanges where I feel like I’m learning something. Also, the most important topic of the day is being addressed, because AI is such a big deal and San Francisco is where it’s happening.

I find it very fun to just be around people who have all these new insights, and they have pretty good ways of thinking about what’s happening. Obviously, a lot of them are cuckoo in opposite ways often, but other cities just don’t have that intellectual vibrancy.

You’ve lived in Texas and several other parts of this country, all around. And a lot of people here come away from that whole experience feeling a certain kind of unity of human motivation: people are the same everywhere. Or do you think the people who go to the Bay Area are a unique breed, different in some way?

Texas is more like normie America. That’s right. So I grew up in—I spent 3 years in West Virginia, 3 years in North Dakota. You’re in Maryland. More normie America. You couldn’t do better than that. Three years in West Texas.

12. Bay Area Culture

I think the biggest difference between the Bay and those places is that the people in the Bay are very similar to each other, and so they just have a strong shared culture. They’re able to talk about the same exact topics, and I think they are just more weird.

13. A.I. Boyfriends v. Girlfriends

And people in the rest of America—say North Dakota, where you live—do they learn more about it? Use it more? Are they going to anthropomorphize it the way many people do here? They may not think it’s sentient, but internally they’ll more or less treat it like a member of the family, or a pet, or even like a child or partner. Is that going to happen in this country, or maybe use it as a rabbi or priest? How do you see that evolving?

I feel like I’ve made bad predictions here. I would have predicted that AI girlfriends would have been a bigger deal by now, and my understanding is that it’s not that big of a deal yet. I do worry that as AIs are able to emulate humans more and more, there’s a different question of whether they are like humans or not.

But I do think in that situation, people will have very strong connections with these AIs, and this has partly to do with the fact that people are undersatiated in their desire for companionship or friendship. I think it’s scary to then just fill that void with a digital simulacrum. But I think the demand is there for people to have digital friends.

Which will be a bigger deal, AI girlfriends or AI boyfriends, and why?

This is a boring answer, but my sense is that there are some apps where you can have a digital girlfriend or boyfriend, and currently the digital boyfriends are outcompeting the digital girlfriends. If I had to guess, I think it’s something like structured conversations that last for a long period of time are filling a particular need that, on average, women have more than men have. But maybe that changes as you guys develop more capabilities, and there are certain things that men have that AIs are better able to fulfill.

I think there’ll be many more AI boyfriends for exactly the reason you give. Do you think AI therapists are a good idea?

Yes, just flat-out yes.

See, the nice thing about a therapist is the session ends and it costs money. So maybe you do an hour a week, and that might be useful.

But if you can just keep on doing therapy, I worry that people will overuse AI therapists, even if the AI does a good job per se. I’ve never been to therapy, so I’m not sure what the thing to be taken away from it is. There’s some sense of having a life coach, which is maybe what I have more in mind—somebody who is a mentor, who is looking out for you, helping you make decisions, and helping you understand what’s happening—that I am excited for more people to have access to as a result of AI.

But in terms of constantly navel-gazing at your emotions, I’m not sure how healthy that will be if you get to do that all the time.

14. What Dwarkesh Got Wrong

Now, you said on this you had a wrong prediction. What would be another incorrect prediction you’ve had?

So many. I thought that I would need this ability to learn on the job before they do extremely impressive things like, say, solving Navier–Stokes. I understand that even math is a very jagged domain that you can just write out on a piece of paper. The way that’s updated me is to think that other verifiable domains might get pushed back a lot by AI progress and might get washed down quite a bit by AI progress, even before we have solved all the cognitive gaps that these AIs have.

Make that a little more concrete for me. What does “verifiable domain pushed back” actually mean? What’s going to happen or not happen, in super-specific terms?

15. What’s The Next Big Thing?

Okay. So if we just look at AI research specifically, for example, my guess is that AI progress is happening somewhere between 25% and 50% faster this year than it was happening in 2024 because of AI uplift. And my guess would be that somewhere within the next 1 to 3 years, teams of AIs can outcompete teams of humans, or companies made of AIs can outcompete companies of humans at doing AI research.

This is another way of saying AI research gets automated. I also predicted that, at that point, it will feel instantaneous—that AI progress has shot up 3 times, 4 times, or something like that.

Other than the AIs themselves, which I already subscribe to, of course, when will it be that normal people walk into Walmart and buy something that is there because of AI—something that would not have been there without AI? They feel they’re better off, not just because they enjoy the AI, but because some other good or service is better. What will be the next thing to come?

I genuinely have no idea what the next thing will be in the long run. But if you have no idea, that means it’s pretty far off, right? Because you’re in touch with everyone. You hear all the gossip and all the undercurrents of talk. If you have no idea, that means they have no idea, and then it feels slow and distant to me.

16. Robot Predictions

Or here’s another way I would describe it: I’m not sure what the order of different products will be, but here’s another prediction I will make. We’re probably 5 years away from robots that can do most of the physical work that humans can do. At that point, it will not be that a specific object exists in Walmart because the AIs invented it or made it. It will just be that the combination of AIs and existing human research and development was necessary to bring this product to fruition.

Things will get a lot cheaper, the same way that nothing in Walmart exists today because a computer exists, other than the computers themselves. It’s just that computing has made all those products better. The cereal box cover was made with the help of a computer, or something. But there are computers in a lot of products, one way or another.

And you have a new essay on some of these themes, correct? What’s the point or argument of that new essay?

The basic argument is that by the end of the decade, digital minds will outnumber human biological minds. They’ll be thinking a lot faster. They will have filled these cognitive gaps, partly because we’ll have automated AI research by then.

At that point, there will also be very fast progress in robotics. I’m painting a picture of a world where, within a matter of 5 to 10 years, both digital and physical AIs outnumber humans.

17. What Jobs Are Safe?

I had a discussion with someone who works at Anthropic recently, and I asked him the question: What are the next 3 or 4 sectors, other than AI itself, that are really going to take off because of AI? Do you have an answer to that?

My guess is that within the next 2 to 3 years, you should expect systems which feel like a truly drop-in knowledge worker. This is maybe 40% of the economy. Whatever companies or organizations that require knowledge workers are doing will get a lot cheaper and a lot faster, because the kind of uplift programmers feel will be felt across this entire broad sector, or across many sectors of the economy.

But it seems a little underwhelming. If you look at a typical household budget sheet, there’s housing, higher education, and food. Maybe there’s transportation. For living standards to go up, what in there is going to get cheaper?

I see instruction and entertainment are already much, much cheaper, and that’s wonderful. But what else? Where’s even the potential? Will my food be cheaper? Will the burrito go from $20 to $14?

I think all that stuff will happen. Over the last 3 to 6 months, we’ve had these amazing things that make all this software. It’s not like all your software has been magically transformed yet. It takes a while for the end impacts of even a radical uplift in labor to manifest themselves.

But in a world where you have robots that cost, say, $30,000, can work around the clock, and can do the work of a human, the kind of physical work that costs, say, $100,000 a year would cost much less. The hourly cost of labor might go down to a couple of dollars.

In that world, you would have a bigger physical house because the robots can build you a bigger physical house. In the United States, maybe we’re 10 times wealthier than people in the developing world. You would just expect that whatever lack of wealth is causing that should be ameliorated by the fact that you have cheap goods and can make a bunch more things.

What are your predictions on employment and wages, say, in the U.S.?

This one I feel totally unsure about. It seems totally plausible to me that employment stays high through the singularity. The whole “there will be new jobs” thing might actually be very plausible.

But take your own job. It’s not impossible for me to imagine that when you interview people, the AI itself could generate excellent questions, and you probably use AI already to help come up with questions. At the same time, you’re a person. You go around, inspire trust, persuade people to be on your show, and do things like appear on camera.

It’s a package, and a job that seems purely intellectual right now—AI is amazing—and we’re still close to full employment, even with some bad economic policies. Maybe we’re just going to stay at full employment.

I will say that right now, AIs have only uplifted 1% of the economy, which is software. It’s only been 3 to 6 months. They haven’t really made a dent in almost anything else in knowledge work, let alone physical work.

I imagine that a lot of jobs are not like your job, for example, where a big part of your production function is showing up at colleges and giving lectures. They’re about getting a specific set of tasks done over a certain horizon, and the demand for those will obviously—or, sorry, the demand will not collapse—but there will be something else which can do that job much cheaper, much faster, and potentially better.

But persuasion and cooperation seem so important in most jobs. I get that, say, Claude might have better arguments than I do. But there’s something about me—or I hope there is—being there with a certain mood or tone. We’re biologically programmed to cooperate with other humans, at least sometimes, in a way that the AIs, no matter how smart they are, just won’t replace.

I mean, just like you’re not going to—you won’t have an AI girlfriend, right?

Yeah, it’s not the same. At least, I think so right now.

I think your specific job is actually quite safe. It’s not clear to me that this is true of most jobs.

I do think there’s an important part of many jobs that you’re talking about that people don’t consider. They just think about the specific task that comes to mind when you think of an accountant, for example. But they don’t think that the accountant is also giving feedback to the manager about how the organization should be restructured, or what different legal decisions they should make, or something like that. That would also have to be something AI can do before the job of an accountant is automated.

Jobs are much bigger than people imagine. I feel like part of what we do is entertainment, and people don’t need our podcast to exist. They listen because it’s fun and interesting. But in many other situations in the world, they’re saying, “Okay, I do actually just need to know whether AI filed my taxes correctly and whether I missed any deductions.” In that case, I would much rather trust an AI in a situation where the AI has surpassed the capabilities of a human.

18. The Speed of A.I. Adoption

Let’s take Waymos, which are plentiful in San Francisco. They’re wonderful. I feel safer in them. There’s a lot of evidence they’re safer. They’re only in a few parts of America, and they’re not yet cheaper. Driverless cars, in some manner, have now been around for a while, and they’re a modest part of the overall economy. Why will the rest of AI adoption go any faster than, say, Waymo adoption? Washington, D.C.—Virginia still can’t do it. Snow may be a problem. People don’t want it. New York City said no. Why isn’t that how it will be for most things?

The reason I think that will not be the case is that previous automation has been about trying to get all the different edge cases applicable to a particular job encoded within a machine.

With driving, why did we have all these amazing demos a decade ago, and only now are we finally having some driving automated? It’s because we had to build these edge cases into code or into very shoddy neural networks.

The expectation I have of the future is that it will be more like having a human come in and do the job. There are diffusion challenges to hiring humans. It’s hard to onboard a person into your company. You have to tell them what to do and get them oriented around everything. But this is worth it for companies, and they hire humans.

For example, Uber drivers can learn to drive a car pretty rapidly. They can learn to drive in new terrains very rapidly. This is true for many other jobs where, if it’s a human-like mind that can diffuse itself and orient itself in a new position, that is the kind of AI I’m expecting in the future, more so than in the past.

We had to automate very particular things and deal with all these edge cases, which in the end often made it much harder.

19. Milestones To Watch

Let’s take your predictions. Your predictions as a whole, in all matters of AI—or, for that matter, anyone’s predictions—what are the indicators or milestones that people should be watching to see if you’re getting it right, or someone else is getting it right? What should they be following?

I think one thing I expect is that, because diffusion will be less of a challenge, given that AI will be more like humans, if progress continues on the current trajectory, you should expect things like the revenues of the AI labs to continue increasing very fast.

So, in a world where, say, the revenues of the top 3 AI—

So, in a world where, say, the revenues of the top 3 AI labs at an annualized rate by the end of next year are over $500 billion, I think that’s much closer to my world, where there’s this extremely discontinuous growth in 1 industry, because general intelligence is very valuable across the economy, than many other worldviews.

But to pay for that revenue growth, other sectors have to grow. What other sectors should we be looking to grow? It could be health, it could be whatever, but that’s part of the prediction.

Let me ask you a question this way. Suppose you’re in Hong Kong in the ’80s and you’re like, “Look, I think we’ve got all the great stuff here needed for 10% growth. We’re going to go gangbusters here.” And then somebody asks you, “What specific things will grow in Hong Kong?” In retrospect, you can be like, “Well, it was obviously electronics.” I’m actually not so familiar with Hong Kong, but you could say everything and you wouldn’t be that far off.

I think it’ll be similar to that. If we’re looking at Hong Kong in the ’80s and your growth will be really fast, I’m expecting this to be the case. I don’t think you’d have a great time predicting what specific products will come first, or in what order. But that’s kind of what I expect with AI, where what people are experiencing in software, and their experience across knowledge work, is hard to anticipate in advance. It’s hard to anticipate what products will get made, what inputs will be required, and so forth.

The stock market goes up or down. There’s the old joke: either AI is a bubble or everything else is a bubble. I don’t literally believe that, but it’s worth a moment’s thought. I think maybe an index like the S&P will go up. Maybe some random stock in the S&P 500 goes up or down; I’m not sure, because the median might not do so well. But the winners will be enormous.

Does it favor more startups or established companies on the exchanges?

I would think startups would be where you would see the growth: very small startups that are AI-native, with people running them who are AI-native from the get-go.

That would be my guess as well, but I’m not sure. You can think of a specific example like trading, which is the first thing people think about as a largely cognitive domain. Do we live in a world where somebody makes a new trading firm that outcompetes Citadel and Jane Street, or will Citadel and Jane Street just use AI to make even more money?

Citadel and Jane Street are using a lot of AI to make a lot more money. I’m not sure how that balances out.

To ask for another prediction—I know this one is hard—but if you had to pick a year when the so-called centaur model, humans working with AI, flips into, “Well, the AI just does it all themselves,” what do you think? If we’re looking back, what year will that be?

I think it’d be different for different industries, but overall, 2030.

2030? That’s very soon.

I would be very surprised if it happened that quickly. The key input that humans will have, even in domains that AIs are making very fast progress on right now, such as software, is a lot of real-world experience. These AIs are kind of just like a mind that has gone through many graduate degrees and is somebody in academia. What can be missing from a mind that only has many graduate degrees but is lacking real-world experience?

Getting that real-world experience, and being able to learn from real-world experience, is fundamentally an ML problem. It’s an ML problem we haven’t solved yet. But once we have AI systems that can learn from being deployed in the world over long periods of time, then the value of human expertise also will decline, or at least be able to be substituted by AIs.

But the AIs have to somehow be in the institutions and companies to take in the knowledge. So are we going to tape everything that happens in companies and research labs, in nonprofits, and in governments, and feed that into the AI? Or how do we teach the AI this contextual knowledge—the pieces of knowledge that are hard to articulate, particular details of time and place?

In fact, this is already happening. These AI systems—Claude or GPT—are deployed across millions of companies. At any given time, there are millions of sessions of these AIs running. The issue right now is that those instances are not able to actually learn from this enormously rich experience that they could be getting.

Imagine if a human could learn from 100 million copies of Tyler, with one in every single industry. One is helping Jane Street make more money, one is helping some accounting firm better organize its files, and one is helping NVIDIA make a better accelerator. It’s able to learn from all this experience.

Such a mind would have an affordance that we don’t have: it gets to live for 20 years in one single stream. Sorry, 80 years within 1 single year. Hundreds someday.

That’s right.

Once they can learn from that experience as well as humans can, it’s a question of whether they’ll be possible. But if they can, and that happens within the next 5 years, then I think the value of this contextual knowledge, which is very high, will be something that is accessible to the AIs.

Would you let it surveil you?

I already do. I have these AI systems in my Slack. I’m constantly talking to AIs. I have Granola on for my meetings. In some sense, if they are actually human-like in their ability to learn, they would have so much information on me—so much contextual information on me—almost immediately.

20. What If 2035 Looks Normal?

Now let’s say we get to the year 2035, which is not that far away—9 years away—and the world does not feel that radically different. Let’s just say that was the case. What would you think you got wrong? And then what would you think the other AI leaders got wrong?

This is a really great question. I was thinking about it just yesterday with some friends. I think it would have to be something like this continual-learning thing: this ability to get information on the job ended up being a very difficult ML problem. It’s the most likely reason that we weren’t able to solve it over the course of the next 10 years.

How about the AI leaders? Would they be making the same mistake as you, or would they make a different set of mistakes in this scenario?

The fundamental thing they would be wrong about is that they think they are building what I’m describing, which is an automated knowledge worker, when we are very far from that today in terms of the capabilities these models have. If you did build an automated knowledge worker, we know that there’s enough demand for tens of trillions of dollars’ worth of wages that go every year to knowledge workers.

Of course, it would be consumer surplus, so it wouldn’t be like there’s tens of trillions of dollars of revenue to the AI companies. But that’s the ballpark we’re talking about. If their vision of what they’re building ends up being true within the next 5 to 10 years, then for them to be wrong about that, they just fail to build an automated knowledge worker.

I think it will have to do with the fact that we weren’t able to crack this online-learning thing. These systems do have deficiencies. In the way they learn, their weights are trained, and they’re incredibly data-inefficient. They require tons of data. I think it’ll just be that these problems end up being harder to solve.

21. Dwarkesh In 2045

Let’s say we arrive in 2035 and your predictions are correct, or it’s gone even further than your predictions. When we get to, say, 2045, what will you be doing?

If my predictions for 2035 are correct, then I’m not even sure you can coherently think about 2045. I imagine I’ll be doing a very similar thing.

Who will want you? The AI will do it better. It can copy your voice already. It copies my voice. It can copy your image probably within a year, perfectly, right? That’s not far away. It will have questions not worse than yours. What’s left for you? It can copy your guests.

You could interview Thomas Edison if you wanted.

But say you’ve invested well in AI. You saw the trends. You don’t have to work. What are you going to do?

There are so many interesting questions that there’s often not an expert on, or you can’t easily talk to the expert to get them on your podcast, or it’s not the kind of thing an expert can answer extemporaneously. It requires a bunch of thought.

I’m just very interested to be able to have Tyler Cowen intellectuals at my command all the time. I can be like, “Let’s really deeply think about what happened in the Aztec and Inca versus conquistador conflicts. Look at all these different angles. I want you to come up with your own insights,” which they can’t do right now, and feed me all of this.

Write me a Robert Caro-type biography of some totally different figure. That’s what I’m personally quite excited about.

Can I ask you a question?

Sure.

What do you expect? This vision you’re describing of automated knowledge work—you don’t think that we’ll get there by 2035?

Absolutely not. I don’t think we’ll get there by 2045. I think the human role in integrating the physical and digital worlds will remain. The fact that people want to deal with humans for many issues, perhaps irrationally, won’t go away ever. I think the centaur model will persist for decades.

22. Skepticism On A.I. Revenue

Do you think that manifests itself in, say, a cap on the maximum amount of revenue you think AI companies can earn without automating knowledge work? If I said it’s 2030 and the AI companies have over $1 trillion in revenue, are you surprised?

I am. I’m not bearish on the companies, but where will the money come from elsewhere in the economy? I’m very optimistic about, say, biomedical progress. But with clinical trials and the FDA, that can take 10 or 20 years, maybe more.

And these other sectors that grow, they’re often hindered. Our government is not very good at adopting AI. I don’t think that will change rapidly. I work in a university. Progress there is very slow. You could say they’re moving backwards. It’s just a thing to stamp out because students use it to cheat. So I think we have 30% or 40% of our economy that just won’t flat out make progress at all by 2035, perhaps. And then these startups will slowly replace bigger companies. But that too takes a lot of time, and the number of people who can run those startups is pretty constrained. You have to work really hard. You have to be very AI-aware, you have to be very smart, and so on.

I want to understand whether you think it’s a lack of capabilities and systems we’ll have, or just that humans will not be inclined to have AIs do jobs that they could, in fact, do. For example, it will be like at Mercatus or something. I assume there are a lot of people you hire to do work for you. Surely you don’t think I will not be able to do that work, or that even if they could, people will not be happy with you hiring AIs, or you will not want to hire AIs—a mix of both.

I think there’ll still be Mercatus then. Absolutely, but the capabilities will be lacking.

23. How Podcasting Has Changed Dwarkesh

Yes. Okay. Now, to close with one final question: You’ve spent all this time in the AI world. You’ve interviewed virtually everyone of importance and asked them all the questions you wanted to. That was your goal at the end of all this. How has it changed you?

Dwarkesh, I don’t know if the ASF in particular has changed me. I do notice something where I’m like, here’s something that I really appreciate about you and Brian and many others of my early guests: people who spend, on camera but also off camera, a bunch of time chatting with me about ideas and stuff. I look back at those early interviews and I’m like, I was asking very naive questions. I’m asking things that you had thought about a long time ago in many different ways. And over time, I guess I’ve gotten to learn about many more topics in a deeper way.

And I think that process would have been very difficult, if not for people like you who are very kind with their time in talking to people who are young and naive and aren’t really thinking about things in a very interesting way, but still answering their questions, engaging in good faith, and so building up a better mental model of different intellectual fields is maybe the main change.

Thank you for the kind words. And Tyler Cowen, thank you very much.

Thank you.