Dwarkesh Patel 与 Noah Smith 谈 AGI 与经济

Are you dubious of the trope that labor provides meaning, and if people don't have a clear sense of labor, it will be very difficult for them to obtain alternative sources of meaning?

Humans have just adapted to so much: the Agricultural Revolution, the Industrial Revolution, the growth of states. Once in a while, a communist or fascist regime will come around, or something like the idea that being free and having millions of dollars is the thing that finally gets us. I'm just suspicious of that.

Dwarkesh, Noah, welcome. This is our first podcast ever as a trio.

Yes. I'm very excited.

So, Dwarkesh, it's almost as if you came up with the scaling era. It's almost like you're a future historian. You're sort of telling the history as it's being written. So it's only appropriate to ask you: What is your definition of AGI, and how has that evolved over time? Some people say superintelligence. Break it down for us.

I feel like I'm 5 decades too young to be a historian. You have to be in your 80s or something.

But we're living in history right now.

The ultimate definition is: Can it do almost any job—98% of jobs, at least as well, as fast, and as cheaply as a human? I think the definition that's often useful for near-term debates is whether it can automate 95% of white-collar work, because there's a clear path to get to that, whereas robotics has a long tail of things you have to do in the physical world, and robotics is slower. So, automated white-collar work.

That's interesting, because it's an economic definition. It's not a definition about how it thinks or how it reasons. It's about what it can do.

Yeah. We've been surprised by what capabilities have come first in AI. They can reason already, yet they seem to lack the economic value we would have assumed would correspond to that level of capability. This thing can reason, but it's making OpenAI $10 billion a year, and McDonald's and Kohl's make more than $10 billion a year, right?

Clearly, there are more things relevant to automating entire jobs than we previously assumed. We don't know what all those things are, but once they can automate them, then it's AGI.

And so, when Ilya or Meta is using the word “superintelligence,” what do they mean? Do they mean the same thing or something totally different?

I'm not sure what they mean. There's a spectrum between God and just something that thinks like a human, but much faster.

Do you have a sense of what you think they mean?

God. I think they probably mean something they would worship as a god.

Yeah. And so, when Tyler says, “We've achieved AGI,” and you differ from him, where's the tangible difference there?

I'm just noticing that if there were a human working for me, they could do things for me that these models cannot do. I'm not talking about something super advanced. I have transcripts for my podcast. I want you to rewrite them the way a human would. Then I'll give you feedback about what you messed up, and I want you to integrate that feedback as you get better over time. You learn my preferences. You learn my content.

They actually don't—they can't learn over the course of 6 months how to become a better editor for me or how to become a better transcriptionist for me. Since a human I hire would be able to do this and they can't, therefore it's not AGI.

I am a natural general intelligence. You are a natural general intelligence. But we cannot easily do each other's jobs, even though our jobs are fairly similar. Put me in the Dwarkesh podcast and I could not interview people nearly so well. If you had to write Substack articles several times a week on economics, you might not do as well.

So we're general intelligences, and we're not exactly substitutable. Why should we use substitutability as the criterion for AGI?

What else is it that we want them to do? I think with humans, we have more of a sense that there's some other human who theoretically could do what you would do. A particular copy of a model might be, say, fine-tuned to do a particular job, and it would be fair to ask: Why expect this particular fine-tune to be able to do any job in the economy?

But then there's a question of, well, there are many different models in the world, and each model might have many different fine-tunes or many different instances. Any one of them should be able to do a particular white-collar job for it to count as AGI. I'm not saying that every AGI should be able to do every single job. I'm saying that some artificial intelligence should be able to do this job for this model to count as AGI.

Okay, so let's take another similar example. Let's take Star Trek. You have Spock; he's very logical. He can do stuff that Kirk and whoever else can't do, but then those guys can do stuff that Spock can't do, like get in touch with their emotions and intuition. They're both general intelligences, but they're alien to each other.

AI feels alien to me. It sometimes talks just like us. It was built off our thoughts, obviously, but sometimes it talks just like us and sometimes it's very alien. Should we ever expect that to change such that it's no longer an alien intelligence?

I think it'll continue to be alien, but I think eventually we will gain capabilities that are necessary to unlock the trillions of dollars of economic value implied by automating human labor, which these models are clearly not generating right now. You could say, well, if we just substituted jobs right now, immediately, there'd be a huge productivity dip, but over time we would learn to do them better.

Maybe a better example is just that you hire people to do things for you. I don't know if you actually hire people.

Okay, okay. Why do you still have to do that rather than hiring an AI? I have many roles where an AI might be generating hundreds of dollars of value for me a month, but humans are generating thousands or tens of thousands of dollars of value for me a month. Why is that the case?

I think it's just that AI is lacking these capabilities. Humans have these capabilities.

Is the main thing missing, in your view, continual learning? The reason humans are so valuable is not just their raw intellect. It's not mainly their raw intellect, although that's important. It's their ability to build up context, interrogate their own failures, and pick up small efficiencies and improvements as they practice a task.

Whereas with an AI model, its understanding of your problem, your business, will be expunged by the end of a session. You're just starting off at the baseline of the model. With a human, you have to train them over many months to make them useful employees. What will need to happen for that to change? What needs to change in order for us to develop—for AI to develop—a capability like that?

I probably wouldn't be a podcaster if I had the answer to that question. It just seems to me that a lot of the modalities we have today to teach LLMs stuff do not constitute this kind of continual learning. For example, making the system prompt better is not the kind of continual learning or on-the-job training that my human employees experience, and RL fine-tuning is not this.

I don't know what the solution to this looks like. It's precisely because I don't have an obvious solution that I think we're many years away.

Okay, so here's my question about replacing jobs. It seems to me that it's partly about demand. For example, suppose that AI has already replaced my job, or can replace my job. Suppose that anyone who fires up ChatGPT or whatever model they want and says, “Search the web, find the most interesting topics that people are talking about in economics, and write me an insightful post telling me some cool new thing I should think about,” and they just do that every day. Then they get a better blog than Noahpinion.

I don't know if that's happened yet. I mean, I've tried that, and I don't like it as much. But suppose that most people would like it as much, and so my job has been automated and people just don't realize it. Or people have this idea in their mind of, “Is it really a human?” Then, as generational turnover happens, young people won't care about reading a human; they'll care about reading an AI.

In terms of functional capabilities, it's already there, but in terms of demand, it's not there. How much of that could there be?

I expect there will be much less of that than people assume if you just look at the example of Waymo versus Uber. You could previously have had this idea that people would hesitate to take automated rides, and in fact, in the cities where it's been deployed, people love this product, despite the fact that you had to wait 20 minutes because the demand is so high.

It still has some glitches to iron out, but there's just the seamlessness of using machines to do things for you. The fact that it can be personalized to you and can happen immediately. One thing people would say is, “Doctors and lawyers will set up guilds, so you won't be able to consult.”

I think there might be guilds governing who can call themselves a doctor or a lawyer, but if there’s genuinely as good medical advice as from a real doctor, the experience of just talking to a chatbot rather than spending 3 hours in a waiting room is so much better.

I think a lot of sectors of the economy look like this, where we’re assuming people will care about having a human, but in fact they will not if you assume that AI will genuinely have the capabilities that the human brings to bear.

So, it’s interesting: AI is better for diagnosis on a lot of things than humans, right? But something about having humans to follow up with makes me want to check with a human after I’ve gotten a diagnosis from an AI on something.

That might vary by job. Cars may be one thing, but maybe it is about capabilities. I can’t say; I’m just saying that everybody seems to think AI is a perfect substitute for humans, that it should be one, and that it will be one.

Everyone seems to think of it in that case. However, every other tool that’s ever been made—every other technological tool—was a complement to humans. It could do some things humans could do. Maybe it could do anything humans could do, but at different relative costs, different relative prices.

So you’d have humans do some things and the tool do other things, and you’d have this complementarity between the 2. Yet when people talk about AI and think about AI, they essentially never seem to think in these terms. They always seem to think in terms of perfect substitutability.

I’m trying to get to the bottom of why people insist on always thinking in terms of perfect substitutability when every other tool has been complementary in the end.

Human labor is also complementary to other human labor, right? There are increasing returns to scale. But that doesn’t mean Microsoft has to hire some number of software engineers and won’t care about what those software engineers cost. It will go to markets where it can get the highest performance for the relative value the software engineers are bringing.

I think it’ll be a similar story with AI labor and human labor. AI labor has the benefit of having extremely low subsistence wages. The marginal cost of keeping an NVIDIA H100 running is much lower than the cost of keeping a human alive for a year.

Now, would you say you’re AGI-pilled too, in the sense that Dwarkesh described the term? We’ve talked a little bit about AI’s effect on labor. Why are you perhaps a little bullish that there will be plenty for humans to do and that it’ll be more complementary?

What is AGI-pilled?

We just believe, in the sense that Dwarkesh described, that it will automate a huge swath of the economy.

I mean labor. I’m very unwilling to say, “Here’s something technology will never be able to do.” That always seems like a bad bet. Here are 2 things people have been saying since the beginning of the Industrial Revolution, neither of which has ever remotely come close to being true, even in specific subdomains. The first one is, “Here’s a thing technology will never be able to do.” The second one is, “Human labor will be made obsolete.”

People have been saying those 2 things over and over, and it’s never been true. That doesn’t mean it could never be true. Sometimes something happens that never happened before, such as the Industrial Revolution itself. You have this hockey stick where suddenly it’s like, “Oh, we’ll never get rich. We’ll never get rich. Oh, we’re rich.” Sometimes that happens; the unprecedented can happen.

However, I’m always wary because I’ve seen it said so many times. Within just the last 10 years or so, I’ve seen a couple of predictions spectacularly fail.

For example, in 2015, 10 years ago, I was sitting in the Bloomberg office in New York, and my colleague—I won’t name him—was physically yelling at me that truck drivers were in trouble. He said truck drivers were all going to be put out of a job by self-driving trucks, and that this was going to devastate a sector of the economy, the working class, blue-collar labor, and so on.

At the same time, I was reading the sci-fi short stories of the year, whatever, and there were 2 stories in the same year about truckers being mass-unemployed by self-driving trucks. Then, 10 years later, there’s a trucker shortage, and the number of truckers we hire is higher than ever.

I’m not saying truckers will never be automated. They may. However, I’m saying that was a spectacularly wrong prediction. You also got Geoffrey Hinton’s prediction that radiologists would be unemployed within a certain time frame. By that time, radiologist wages were higher than ever, and employment was higher than ever.

I’m not smugly sitting here and saying there’s a law of the universe that says you’ll never see this kind of mass unemployment. Encyclopedia salespeople were mass-unemployed by the internet. We’ve seen it happen in real life.

But these predictions keep coming out wrong. I’m trying to figure out why. Why do they keep coming out wrong? Is it simply that people overestimate progress in technical capabilities, or are there complementarities that people can’t imagine from the neat division of tasks or the standard mental division of tasks?

I think the problem has been that people underestimate how many things are truly needed to automate human labor. They think, “We’ve got reasoning, and now this is what it takes to take over a job.” I think in fact there’s much more to a job than is assumed. That’s why I wrote this blog post where I’m like, “Look, it’s not a couple of years away. It might be longer than that.”

Then there’s another question: by 2100, will there be jobs that humans are doing? If you zoom out long enough, will we ever be able to make machines that can think and do physical labor at least as cheaply and as well as humans can?

Fundamentally, the big advantage they have is that we can keep building more of them, right? So we make as many of those machines as the cost of producing them—or, sorry, the value they generate—equals the cost of producing them.

And the cost will continue to go down.

Right? Yeah. And it will be lower than the cost of keeping a human alive. So even if a human could do the exact same labor, a human needs a lot of stuff to stay alive, let alone to grow a human—everything.

An H100 costs $40,000 today. The yearly cost of running it is thousands of dollars. We can just buy more H100s, right? If we currently had the algorithm for AGI, we could run it on an H100.

So however big the demand is—the latent demand, right?

That’s unlocked by the more supply. We just increase the supply basically to meet that demand. So first, when AGI is here—

What does the world look like?

Yeah, I think if we have chatbots that can answer hard math questions, I don’t expect the world to look that different, because the fraction of economic value generated by math is extremely small. But there are other jobs that are much more mundane than “PhD intelligence” that a chatbot just cannot do, right? A chatbot cannot edit videos for me.

Once those are automated, I actually expect a pretty crazy world, because the big bottleneck to growth has been that human population can only increase at this slow clip. In fact, one of the reasons growth has slowed since the 1970s is that, in developing countries, the population has plateaued.

With AI, capital and labor are functionally equivalent, right? You can build more data centers or more robot factories, and they can do real work or build more robot factories. You can have this explosive dynamic, and once we get that loop closed, I think it would just be 20%-plus growth.

Do you see that as feasible, possible—20% growth? Tyler, I believe, said 5%, right?

5% more than the steady state.

5% more.

And what’s the argument for that?

For Tyler’s argument: bottlenecks. I think the problem with the argument is that there are always bottlenecks, right? You could have said before the Industrial Revolution, “We will never 10x the rate of growth because there will be bottlenecks,” and that doesn’t tell you anything. Empirically, you have to look at the fraction of the economy that will be bottlenecked and what fraction won’t, and then actually derive the rate of growth.

The fact that there are bottlenecks doesn't tell you how fast growth will be.

Is he mostly referring to regulation?

Yeah, and just that we live in a fallen world and people will have to use the AIs, and there—yeah, things like that.

Who'll be buying all the stuff? So, background in economics: GDP is what people are willing to pay for, right? Who will be buying the stuff in a world where we get 20% growth?

First of all, I don't know. You could have said in 10,000 BC, “The economy is going to be a billion times bigger in 10,000 years.” What does it mean to produce a billion times more stuff than we're producing right now? Who is buying all this stuff? You just can't predict that in advance.

In the 1700s, I could tell you exactly who was buying stuff, which was everybody—peasants. When, in fact, people wrote these things around 1900 about what the world would look like in 100 years and what we'd have, they didn't get exactly the right things right. But they correctly identified that it would be regular consumers who would be buying all these things—regular people—and that came true. It was obvious.

But here's my point: suppose that 99% of people do not have a job and are not getting paid an income, and all the money is going to sort of Sam Altman, Elon Musk, and 5 other guys. And they're captive AIs that they own because, for some reason, our property-rights system still exists. But okay, suppose that's the future we're contemplating, right?

So 99% of people, or more, don't have any job. They don't have any income. They're out on the street. And yet you're saying 20% growth a year. That growth is defined by consumers paying for things.

I wouldn't define it just as people. I would define it as—I mean, I assume agents.

Yeah. No, that doesn't count as GDP. Only final goods.

Okay. So we're launching Dyson spheres. We're not allowed to count that because the AIs are doing it? I mean, I want to know what the solar system will look like. I don't care about the semantics of that. I think the better way to capture what is physically happening is to ask: Why will they do any of that?

One argument is simply that if there's any agent, AI or human, who cares about colonizing the galaxy, even if 99% of agents don't care about that, if 1 agent cares, they can go do it. Colonizing the galaxy is a lot of growth because the galaxy is really big, right? So it's very easy for me to imagine that if Sam Altman decides to launch the probes, breaking down Mars and sending out the von Neumann probes generates 20% growth.

I think what you're getting at here is that AI will have to have property rights. AI agents will have to control resources.

Even if—I guess it depends on what you mean by autonomous. Today, we already have computer programs that have autonomous use of resources, right?

Okay, but the program goes off and colonizes the solar system. It's not like a dude telling it, “Colonize the solar system now,” and doing all this stuff. It's like the AI has made the decision to do it, and Sam Altman is sitting back there saying, “Oh, well, you know.”

I'm just saying this is not a crux. Sam Altman could say it, or the AI could say it. If some agent cares about this and it's not stopped from doing it, you can physically see where the 20% growth is coming from.

Let me make this a little more concrete. Suppose that AI is going to produce a bounty of the things that humans desire, and that's going to be what growth is. How will it get to the humans if the humans don't have a job? And if the humans don't have a job, why will AI be producing them? In other words, if there are no consumers to buy my cars, why am I building cars?

You might be assuming there's some UBI.

No, I don't need to assume that. Although, I mean—

Let's assume there isn't that.

Yes. I don't need to assume that. It seems like you're saying, look, if 99% of consumers are no longer consumers, where's this economic activity coming from?

Yeah. And I'm just saying, okay, if 1 person cares about colonizing the galaxy, that's generating a lot of demand. It takes a lot of stuff to colonize the galaxy. This isn't a world where everybody is roughly contributing equivalent amounts of demand. The potential for 1 person alone to generate this demand is so high that—

So Sam Altman tells his infinite army of robots to colonize the galaxy, we count that as consumption, we put a value on it, and that's GDP.

Yeah. Or it might be investment. Maybe he's going to defer his consumption. I'm trying to see what happens after he colonizes the galaxy.

I'm not saying this is the world I want. I'm just saying, think about it physically. If you're colonizing the galaxy, which you can potentially do after AGI—I mean, I'm not saying it'll happen tomorrow after AGI, right? But this is a thing that's physically possible. Is that growth? Something's happening that's explosive.

Maybe the thing is that it's a very weird world. It doesn't look like the kind of economy we've ever had, right? We created the notion of GDP to represent people exchanging money for goods and services, people basically exchanging their labor for goods, exchanging the value of their labor for goods and services.

At a fundamental level, that's what GDP is. We're envisioning a radical shift in what GDP means, to a sort of internal pricing that a few overlords set for the things that their AI agents want to do. And that's incredibly different from what we've called GDP in the past.

I mean, I think the economy will be incredibly different from what it was in the past. I want to say that I'm not saying this is the modal world. There are a couple of reasons why this might not end up happening.

One is, even if your labor is not worth that much, the property you own is potentially worth a lot, right? If you own the S&P 500 and there's been explosive growth, you're a multimillionaire. Or the land you have is worth a lot if the AI can make such good use of that land to build the space probes, assuming our system of property rights continues into this regime.

Second, in many cases it's hard to ascribe how much economic growth there has been over very long periods of time. For example, over 500 years, if you're comparing the basket of goods that we can produce as an economy today versus 500 years ago, it's not clear how you compare them.

We have antibiotics today. I wouldn't want to go back 500 years for any amount of money because they don't have antibiotics and I might die, and it'll just suck. There's actually no amount of money you could pay me to live in 1500 rather than live today.

And so, if we have those qualities of goods for normal people—if you can live forever, have euphoria drugs, whatever—these are things we can imagine now. Hopefully, it'll be even more compelling than that. Then it's easy to imagine, okay, it makes sense why this stuff is worth way more than what the world economy can produce, even for normal people today, right?

Yeah. And so I guess I'm just thinking about this thing that economists really struggled with in the early 20th century. It's this idea that we had the capacity to expand production, expand production, expand production. And then companies competed their profits to zero, the profits crashed, and nobody wanted to expand production anymore because they weren't making any profit.

We're seeing this happen again in China right now with overproduction. We're seeing BYD having to take loans from its suppliers just to stay financially afloat, even though it's the best car company in the world, because the Chinese government has paid a million other car companies to compete with BYD. And so you compete, you overproduce. You have this overproduction.

The question is: the solution was to expand consumption. This is the solution people are recommending for China now, to expand consumption so that you can refuel that wheel. The profits from this go negative, that makes the GDP contribution go to zero, and basically OpenAI, Anthropic, xAI, and whatever will just be sitting there saying, “Why am I doing this again? No one's buying this shit.”

At that point, it seems like there will be corporate pressure on the government to do something to redistribute purchasing power so that they don't compete their profits to negative, and so they have some reason to create more economic activity so they can take a slice of it, which is essentially what happened in the early 20th century.

Yeah, I disagree with this.

I'm not saying this will happen. I'm saying that would be the analogue.

I disagree. Even as a libertarian, I would prefer significant amounts of redistribution in this world, because the libertarian argument doesn't make sense if there's no way you could physically pick yourself up by the bootstraps—your labor is not worth anything.

Or your labor is worth less than subsistence calories or whatever, which is a more relevant thing. But I don't think this is analogous to the situation in China. I think what's happening in China is more due to the system of financial repression, which redistributes money, and currency manipulation, which basically redistributes ordinary people's money toward producing 1 EV maker in every single province. So it is the market distortion that the government is creating that causes this overproduction.

We can go into what the analogous thing in the AI case looks like, but I think if there isn't some market distortion, people will use AI where it has the highest rate of return. If it's not space colonization, there will be longevity drugs or whatever.

I'm just saying, why would I invest all this money into AI-producing stuff? Why would I invest the massive hundreds of billions or trillions of dollars into producing stuff for people who are all going to be out of a job and won't be able to buy it?

But again, I don't think you'll be producing it for them. I think you'd be producing it for whoever has some stuff in the world, so somebody will have stuff. Maybe it's the AIs; maybe it's Sam Altman. You're producing it for whoever has the capability to buy your stuff.

And will they want AI? I'm just saying AI can do so many things, least of which is colonizing the galaxy. People will be willing to pay a lot for stuff in the galaxy, right?

Right? I'm just trying to get this straight in my head: What does this economy look like? I'm seeing a picture of the trillions of dollars needed to build out all these data centers being spent not for profit—not to make money from a consumer economy for the creators of the AI—but to satisfy the whims of a few robot lords to colonize the galaxy.

I think you're making 2 different points, and they're getting wrapped into 1. I'm saying yes—important word. So there's one about whether you expect the robot-overlord world to happen, and I'm saying no. Actually, first of all, I expect redistribution to happen. I hope it happens, but even if it doesn't, I don't think it will happen because corporations want redistribution to happen. I think it would be good for it to happen for independent reasons, but I don't buy this argument that corporations will be like, “We need somebody to buy our AI; therefore, we need to give the money to ordinary consumers.”

Do you believe broad-based asset ownership will create a whole lot of broad-based consumer demand even in the absence of labor income?

I'm honestly not sure. I don't have a super strong opinion, but I think that's plausible. Independent of that, I'm like, okay, even if that demand doesn't exist, just the things you can do with a new frontier of technology—as long as 1 person wants it, there's so much room to do things. Space colonization is an obvious example.

A lot of money.

Right, there's obvious demand for the things that AI will be able to produce, right? One of the things it can produce is colonizing a galaxy.

Right, exactly. But the question is: Who? I can see a paperclip maximizer—an autonomous intelligence—colonizing the galaxy.

That's a lot of growth.

That is. By the way, I would like to say that I am a paperclip maximizer. I am the real paperclip maximizer. I want to maximize rabbits in the galaxy. I want to turn the entire galaxy into fluffy rabbits. That's my goal, and so my goal with AGI is to enlist AGI to help me in this goal, but then align it toward rabbits. But anyway—

Get these in front of the OpenAI board of directors.

I know. I like that. The objective of the social welfare function is floofiness.

But I guess my point here is, as long as AI still doesn't have property rights and it's humans making all the economic decisions—be it Sam Altman and Elon Musk or, you know, you and me—then that really matters for what gets done.

The money needed to build all these massive data centers is currently a lot of money. It's a ton of money required to build these data centers, and that money need will not go away. We can't just say, “Oh, cost goes to 0,” because we can say unit cost goes to 0, but total cost doesn't go to 0.

Nor has it. It has increased. The total spend on data centers has increased, and I think everyone expects it to increase for the foreseeable future. The question is: Is that money being spent because AI companies expect to reap benefits from consumers like you and me? Or to what extent is it that? And to what extent is it that Sam Altman feels like doing some crazy stuff and Sam Altman's just godlike richer than everybody else?

So Sam Altman is actually consuming when he builds those data centers. He is building those data centers so that he can indulge his godlike whims.

I mean, I think that's more plausible than either a single godlike person being able to direct the whole economy or there being broad-based consumer demand from every person.

For example, these are extremes.

I think more plausible is that AIs will be integrated through all the firms in the economy. A firm can have property. Firms will be largely run by AIs, even though there's nominally a human board of directors—and it might not even be nominal. Maybe the AIs are aligned and genuinely give the board of directors an accurate summary of what's happening.

But day to day, they're being run by AIs, and firms can have property rights. Firms can demand things.

So say all you have is a board of directors and AI.

Yeah. Okay.

I mean, in the ideal world.

Okay. So then what we're basically looking at is that the labor share of income goes to 0, or something approaching that, depending on how you define the labor and capital share of income. Capital income is distributed highly unevenly—more unevenly than labor income—but it's still distributed reasonably broadly. I have capital income; you have capital income.

So at that point, we have just an extremely unequal society where owners get everything and workers get nothing, and then we have to figure out what to do about that.

Yeah, 100%. The hopeful situation here is the way our society currently treats retirees and old people who are not generating any economic value anymore. If you just look at the percentage of your paycheck that's basically being transferred to old people, it's like 25% or something.

You're willing to do this because they have a lot of political power. They've used that political power in order to lock in these advantages. They're not so overwhelming that you're like, “I'm going to go to Costa Rica instead.” You're like, “Okay, I had to pay this money. I had to pay this concession. I'll do it.”

Hopefully humans can occupy that sort of position—can be in a similar position to this massive AI economy that old people today have in today's economy.

All right.

What do humans do? Let's say they get some money. They have enough to live. How do they spend their time? Is it art, religion, poetry, drugs?

Broadcasting. It's the final job.

Yeah, we're ahead of the curve here. Or we're the last man of history. Wait, so here's an idea: How about a sovereign wealth fund?

Okay, sovereign wealth fund. We tax Sam Altman and Elon Musk.

We're using Sam as a metaphor here. He's a friend of the firm, you know.

Yeah, yeah, yeah. We tax him. We tax Mark, and then we use their money to buy shares in the things that those people have. So they get their money back because we're buying the shares back from them. Okay, okay. And then we hire them.

Only the friends of the show will be taxed.

We hire a number of firms, including a16z, and pay them 2 and 20 or whatever to manage the investment in AI stuff on behalf of the humans. Then the humans become broad-based, sort of index-fund shareholders, or shareholders in whatever you guys choose to invest in. You take a cut, and this could be the future economy.

This is what my PhD adviser, Miles Kimball, has suggested. This is what the socialist Matt Bruenig has suggested. And this is what Alaska actually does with oil.

Capitalists like it, socialists like it, Alaska likes it.

I think sovereign oil funds generally have a bad track record. There are some exceptions that have managed to use their wealth well, like Norway or Alaska, but there are just these political-economy problems that come up when there's this tight connection between the investment—which should theoretically be just the highest rate of return—and politicians.

So I don't have a strong alternative. Ideally, you just let the market decide how the investment should happen, and then you can take a tax. But then exactly where does that tax happen? I haven't thought it through. But I wouldn't want the government influencing where that investment happens.

But I want the government taking a significant share of the returns of that investment.

Yeah. Are you dubious of the trope that labor provides meaning, and if people don't have a clear sense of labor, it will be very difficult for them to obtain alternative sources of meaning? Or is that kind of a capitalist sort of stroke that isn't necessarily true?

My suspicion is that humans have just adapted to so much: the Agricultural Revolution, the Industrial Revolution, the growth of states. Once in a while, a communist or fascist regime will come around or something. The idea that being free and having millions of dollars is the thing that finally gets us—I'm just suspicious of that.

By the way, do we not disagree about the thing I'm saying? Once we get AGI, humans will not have high-paying jobs. Do we disagree about this?

Yeah.

I think humans may have high-paying jobs because of comparative advantage. The key here is that if there's some AI-specific resource constraint that doesn't apply to humans, then the law of comparative advantage takes over, and humans get high-paying jobs even though AI would be better at any specific thing than a human, because there's some sort of aggregate constraint.

The example I always use, of course, is Marc Andreessen, who is the fastest typist I have ever seen in my life and yet does not do his own typing. Because there's a Marc Andreessen-specific aggregate constraint—there's only 1 of him—he hasn't taken all the secretaries' typing jobs, because he has better things to do. If there's some sort of AI-specific resource constraint that hits, then humans could have high-paying jobs. I'm not saying there will be.

Yeah.

And I'm not saying there won't be. I'm saying I don't know if there is one.

Yeah.

The reason I find that implausible is that I think it will be true in the short term, because right now there are 10 million H100 equivalents in the world. In a couple of years, there might be 100 million. An H100 has the same amount of FLOPs as a human brain, so theoretically they're as good as a brain if you had the right algorithm. There is a lower population of AIs, even if you had AGI right now, than humans.

But the key difference is that, in the long run, you can just keep increasing the supply of compute or robots. If an H100 costs a couple thousand dollars a year to run, but the value of an extra year of intellectual work is still $100,000, then you'd say, “Look, we've saturated all the H100s and we still have to pay a human $100,000 because there's still so much intellectual work to do in that world.”

The return on buying another H100—an H100 costs $40,000, and in a year that H100 will pay you over a 200% return, right?—means you'll just keep expanding that supply of compute until, basically, the H100 plus depreciation plus running costs is the same as an extra year of labor. In that world, that's much lower than human subsistence, so comparative advantage is totally consistent with human wages just being below subsistence.

It is, but that comes from the common resource consumption. If basically all of the land and energy that could be used to feed, clothe, and shelter humans gets appropriated by H100s, then that is the case. However, if you pass a law saying this land is reserved for growing human food, and if we were actually to pass a simple law saying that these resources are reserved for humans, then—

But at that point, comparative advantage has nothing to do with this. The only reason the system works is that you are basically transferring resources to humans. You've come up with a sort of intricate way to transfer resources to humans: you're just saying, “This resource is for you. You have this land, and therefore you can survive.” This is just an inefficient way to allocate resources to humans.

It's true that it is an inefficient way.

That has nothing to do with—I think people will hear this comparative advantage and be like, “Oh, there's some intrinsic reason—”

Take UBI instead.

Yeah. Okay.

Yeah.

Yeah. I mean, sure, but then again, we typically do not see the first-best, most efficient political solution implemented for things like redistribution in the real world. Redistribution happens via things like the minimum wage or letting the AMA decide how many doctors there are going to be. Redistribution in the real world is not always the most efficient thing.

I'm just saying that comparative advantage—if you're talking about whether humans will actually continue to get high-paid work, yes or no—it depends on political decisions. It may depend on physical constraints that will happen.

But the high-paid jobs are literally because, as you have said, there must be high-paying jobs politically. I understand that in this case you've said it in an indirect way, but you still said it, right?

That's right. You're absolutely right.

Yeah, you're not wrong. Or I guess it's incredibly different from what somebody might assume. It has almost nothing to do with the comparative advantage argument.

Okay, sure. But that's true of a lot of jobs that exist now. I'm not sure about university professors—there are a lot of those jobs—or credit-rating agencies. There are a lot of things where we could probably wring out some significant TFP growth, more or less, by eliminating those things, but we don't, because our politics is a vetocracy. I think this is one of Tyler's points.

Yeah. I do think it's important to point out in advance that it would be better if we just bit the bullet about AGI so that, instead of doing redistribution by expanding Medicaid—and Medicaid can't procure all the amazing services that AI will create—it would be better if we just said, “Look, this is coming.”

I'm not saying we should do a UBI today, but in the long run, if all human wages go to zero or below subsistence, the only way to deal with that is through some kind of UBI. Otherwise, if you happen to sue OpenAI, you get a trillion-dollar settlement; otherwise, you're kind of screwed, right?

Some people said the bear case for UBI was something around COVID as an example. You gave people a bunch of money, and what did they go do? Go riot in the streets. I'm teasing, but are people going to use that money in an effective way?

I mean, that was literally what happened.

Yeah. So is UBI the form that you would think is the most effective method?

The reason I favor UBI is that, in a future world with explosive growth, we're going to see so many new kinds of goods and services that will be possible but aren't available today. Distributing just a basket of goods is inferior to saying, “If we solve aging, here's some fraction of GDP. Go spend your tens of millions partly on buying this aging cure, or whatever this new thing that AI enables,” rather than, “Here's a food-stamps equivalent of the AGI world that you can have access to.”

Of course, this discussion may be academic, because I believe that you said that when we got phones, the world would look the same. I mean, no, it doesn't. Phones have destroyed the human race.

The fertility crash that's happening all around the world means nobody has replacement-level fertility. Fertility is going far below replacement everywhere because of technology. The human race does not have a collective desire to perpetuate itself. Yes, we're going to get lonely, but we'll have company through AI, the internet, and social media, until there's just a few of us and we dwindle and dwindle.

Technology has already destroyed the human race, and basically UBI is just keeping us around on life support for a little while while that plays out.

Is that the phone or the pill, or—

Well, no, it's the phone. The pill and other things, like women's education, lowered fertility quite a bit, but some countries were still at replacement level; some were still around replacement level. The crash we've seen since everybody got phones is epic and just unbounded.

I have a take about this. I do think so far there's been a lot of negative effects from widespread TikTok use or whatever that we're still learning about. I am somewhat optimistic that, in the long run, there's an optimistic vision here that could work.

Right now, it's impossible for Steven Spielberg to make every single TikTok and direct it in a really compelling way that's genuine content, not just video games at the bottom and some music video at the top. In the future, it might genuinely be possible to give every single person their own dedicated Steven Spielberg and create incredibly compelling, long narrative arcs that include other people they know, et cetera.

Oh, yeah. In the long run, I'm like, maybe this happens.

I don't think TikTok is the best possible medium.

No. But I also don't think TikTok is unique in destroying the human race. I think interacting online instead of interacting in person—that's how you make your money. That's the great—

How do you make your money? Go ahead?

I agree.

We're all making money, destroying our species. But that's—

You don't think it's isolated to dating apps?

No, I'm saying, why did humans perpetuate the human species? It was not because they wanted to see the human species perpetuated. It was because it's like, “Oops, I had sex and there came a baby.” And that's done. We've severed that, and that is the end. We did not evolve to want our species to continue.

Right. But you're saying the reason we're not having babies is because we can make friends on the internet. But is it that dating apps have created just a much more efficient market, and thus there isn't pair-bonding?

I don't know. People are having less sex. If Elon gets his way, everybody will just sit there gooning to some sort of Grok companion.

The goon apocalypse seems upon us.

No, but is this available right now?

What's the website?

Oh, no. Anyway, this podcast got silly. But I guess the point is that the idea of a humanity that just keeps increasing in numbers and spreading out to the galaxy—I don't see a lot of evidence that that is in our future, or that we have to go to great lengths to make sure that future is compatible with AGI, because I don't think it's happening in any case, AGI or none.

By the way, not to cope too hard, but in a world where AGI happens, how important is population? How important is sort of increasing population?

I mean, population has so far been the decisive factor in terms of which countries are powerful. The reason China could take over Taiwan, if the U.S. were not involved, is just that there are 1.4 billion Chinese people and 20 million Taiwanese people.

Now, if in the future your effective labor supply is largely AIs, then this dynamic just means that your inference capacity is literally your geopolitical power. Right. Correct.

I want to shift to the short term a bit. You've had some people on the podcast. You have the AI 2027 folks who believe that AI is perhaps 2 years away. I think they updated to 3 years away. And then you've also had some folks on who have said it's not for 30-something years. Maybe you could steelman both arguments and then share where you net out.

Yeah. So, if I'm steelmanning them, it's that, look, if you just look at the progress over the last few years, it's reasoning. Aristotle is like, “The thing that makes humans human is reasoning.” And it was not that hard, right? Train on math and code problems, have it think for a second, and you get reasoning. That's crazy. So what is the secret thing that we won't get?

Right.

Can I ask a stupid question? Why were things like o3-type models called reasoning models, but GPT-4o is not called a reasoning model? What are they doing differently that's reasoning?

One, I think GPT-3 can technically do a lot of things GPT-4 can, but GPT-4 just does them way more reliably. And I think this is even more true of reasoning models relative to GPT-4o, where GPT-4o can solve math problems—and, in fact, modern-day GPT-4o has probably been trained a lot on math and code—but the original GPT-4 just wasn't trained that much on math and code problems.

So it didn't have whatever meta-circuits exist for how to backtrack, how to be like, “Wait, but I'm on the wrong track. I have to go back. I have to pursue the solution this way.” Algorithmically, I have an okay idea of what a reasoning model does that non-reasoning models don't.

But in terms of how that maps to a thing that we call reasoning, what is the definition of what it means to reason that these people are using? The operational definition here—I don't understand that myself.

I mean, GPT-4o can't get a gold medal at the IMO.

Okay. But I can reason, and I can't get a gold medal at the IMO.

But I can reason.

Yeah. I can't get a gold either, but I don't think I can reason as well as a math Olympiad competitor, at least in the relevant domain. I agree that reasoning is not just about mathematics.

But this is true of any word you come up with. The zebra—what is the thing that is a mixture of a zebra and a giraffe, and they have a baby? Is that a zebra still? I agree there are edge cases to everything, but there's a general conceptual category of zebra, and I think there's a general conceptual category of reasoning.

Okay. I'm just wondering what it is.

No, I'm saying, when you have a checkout clerk, right? That checkout clerk would look at an IMO problem and be like, “What?” But then you have a checkout clerk and you're like, “Okay, so you put the thing on this shelf, and therefore someone has looked for it and didn't find it. So something else must have happened.”

But I think a reasoning model will be more reliable and better at solving that kind of problem.

Okay, okay. So you're steelmanning the AI 2027 view.

Yes. Basically, a lot of things we previously thought were hard have just been incredibly easy. So whatever additional bottlenecks you're anticipating—whether it's continual learning, on-the-job training, or computer use—this is just going to be the kind of thing where, in advance, it's like, “How would we solve this?” And then deep learning just works so well that we try to train it to do that, and then it'll work.

The longer-timelines people will say—there's a sort of longer argument. Basically, the things we think of as very difficult and requiring intelligence have been some of the things that machines have gotten first. We got just adding numbers together in the 1940s and 1950s. Reasoning might be another one of those things where we think of it as the apogee of human abilities, but in fact it's only been recently optimized by evolution over the last few million years.

Whereas things like just moving about in the world, having common sense, and having this long-term memory—evolution has spent hundreds of millions, if not billions, of years optimizing those kinds of things. Those might be much harder to build into these AI models.

I mean, the reasoning models still go off in these crazy hallucinations. They'll never admit they were wrong and will just gaslight you infinitely about some crap they made up. They'll still have trouble knowing truth from falsehood.

Yeah. I've met a couple of humans who don't seem to be able to know truth from falsehood. They're weird, you know.

But o3 sometimes does this. I mean, I think it's a question: do they hallucinate more than the average person? I think no less.

They can hallucinate, meaning getting something wrong, and when you push them on it, they're like, “No, whatever.” Eventually, they'll admit it if they're clearly wrong. I think they're actually more reliable than the average human.

But the thing about the average human is you can get the average human to not do that, right, with the right consequences. And maybe with AI, we haven't found the right reinforcement-learning function or whatever to get them not to do that.

Okay, now let's get to the view that it's 30 years away. Basically, what's that view?

Oh, just this thing that reasoning is relatively easy in comparison to—forget about robotics, which is just, you know, evolution spent billions of years trying to get robotics to work. And there's other things involved with tracking the long-run state of a lion. A lion can follow prey for a month or something, but these models can't do a job for a month. These kinds of things are actually much more complicated than even reasoning.

And where you've netted out is it's either going to happen in a few years or not for quite some time.

Yeah, when you explain this to me—

Basically, the progress in AI that we've seen over the last decade has been largely driven by stupendous increases in compute. The compute used in training a frontier system has grown 4x a year for, I think, the last decade. Over 4 years, that's 160x, so over the course of a decade, that's hundreds of thousands of times more compute.

That physically cannot continue. Right now, we're spending 1.2% of GDP or something on data centers. Not all of that is returning, of course, but what would it mean to continue this for another decade?

For maybe 5 more years, you could keep increasing the share of energy that we're spending on training data centers, or the fraction of TSMC's leading-edge-node wafers that we dedicate to making AI chips, or even the fraction of GDP that we can dedicate to AI training.

But at some point, you can’t keep this 4× trend going year after year. After that point, it has to come from new ideas: here’s a new way we could train a model.

And by the way, when I was writing that comparative advantage post and thinking about AI-specific aggregate constraints and resource constraints, this is what I was thinking of, actually.

That expansion of compute has to slow down.

But I don’t know how much that matters.

Well, that’s for training. For the labor, inference will also use the same bucket of compute.

It is the case that, for the amount of compute it costs to train a system, if you set up a cluster to train a system, you can usually run 100,000 copies of that model at typical token speeds on that same cluster. That’s still obviously not billions, but if we’ve got all this computing—these huge systems—in the future, it would still allow us to sustain a population of hundreds of millions, if not billions, of AIs. And at that point, maybe we’ll still want more AIs, but—

What does a single AI mean in this instance?

A copy of a model. When you’re talking to Claude, it’s a single instance.

Okay.

That’s talking to you.

Yeah, so instances.

Yeah. Yeah, okay.

So what’s going to determine whether it’s in a few years or not?

Right now, we’re basically riding the wave of this extra compute. That’s why AI is getting better every year, mostly. In terms of the contribution of new algorithms, that’s a smaller fraction of the progress that’s explained by them. So if we’ve just got this rocket, how high will it take us, and does it get us to space or not? If it doesn’t, then we just have to rely on algorithmic progress, which has been—

Slower. Yeah, I think it’s—

But you think it might get us to space?

Yeah. I think there’s a chance that continual learning is also— I had this whole theory about how it’s so hard and how do you slot it in, and then they’re like, “I trained it to do this.” What are we talking about here?

Yeah.

That leads into another thing that I’ve thought about, which is how poor our track record for making predictions about the future of AI has been. The first time you and I hung out—I don’t know if you remember—this was with Leopold.

Yeah. Oh, really? Yeah, it was at your old house.

And Leopold was just pronouncing a whole bunch of pronouncements from the couch.

And he released that big “Situational Awareness: The Decade Ahead” essay. How long ago was that? A year and a half?

Yeah.

Yeah. I would say that already most of the things he predicted have been invalidated or made irrelevant in the last year and a half, especially all the stuff about competition with China.

It turns out distillation was able to get them a whole lot of things that he never predicted.

It turns out that so many of the things—other than just the idea that AI would keep getting better, which he predicts and a lot of people predict—so many of the specific predictions about U.S. capabilities and Chinese capabilities, what the bottlenecks would be, and how we could compete with China, have all been proven wrong since.

I think this is actually an interesting trend in the history of science. Some of the scientists who were the smartest in thinking about the progression of the atomic bomb or the progression of physics had these ideas like, “We’ll have to have a one-world government after World War II.” I’m talking about after World War II. “There’s no other way we can deal with this new technology.”

Relative to the technological predictions, Leo’s main way of being wrong was that it didn’t take breaking the servers in order to learn how o3 or something works. It was just publicly seeing that you could use the model and learn what it knows. Just knowing that a reasoning model works, being able to use it, and seeing how fast it outputs tokens will teach you a lot about how big the model is. You learn a lot just from publicly using a model and knowing a thing is possible.

He has been right in one big way. He identified 3 key things that would be required to get us from GPT-4 to an AGI-like system: being able to think, so test-time compute; onboarding in terms of the workplace; and computer use.

Did you talk about test-time compute?

Yeah. It was one of his 3 big unhobblings.

Right.

I’m getting things done.

This goes back to your theory that phones are destroying us.

That is an update toward the idea that AI is now on this trend to be a super-useful assistant that’s already helping us make the research process of training AI much faster, and that this will just be a feedback loop and become exponential.

I have other independent reasons. I’m, like, 20% confident that we’ll have some sort of intelligence explosion.

What’s your take on the model of automating AI research as the path to AGI?

The METR uplift paper, contrary to expectations, found that whenever senior developers working in repositories they understood well used AI, they were actually slowed down by 20%.

Yeah, I did see that.

Whereas they themselves thought that they were sped up by 20%.

Right, and so there are a bunch of things—

I’m getting things done.

This goes back to your theory that phones are destroying us.

That is an update toward the idea that AI is not just going to be a super-useful assistant that’s already helping us make the slow process of training AI much faster, and that this will just be a feedback loop and become exponential.

I have other independent reasons. I’m, like, 20% confident that we’ll have some sort of intelligence explosion.

One of the other labeled predictions was nationalization. Is that something you could potentially foresee in the next few years?

I don’t think it’s politically plausible. Especially given this administration, I don’t think it’s desirable. First, I think it would drastically slow down AI progress, because this is not 1945 America. Also, building an atomic bomb is a way easier project than building AGI.

But China has quasi-nationalized most of its companies. I mean, it doesn’t control BYD’s day-to-day decisions about what to build. But if China says, “Do this,” BYD does it, as does every Chinese company.

I mean, that’s kind of the relationship American companies have with the U.S. government as well.

You think so?

I mean, somewhat. The big difference is, what do we mean by nationalization? There’s one thing in which there’s a party cadre who is—

In your company.

Exactly. There’s another in which each province is just pouring a bunch of money into building its own competitor to BYD in this potentially wasteful way. That distributed, competitive process seems like the opposite of nationalization to me. When people imagine AGI nationalization, I don’t think they’re saying that Montana will have its AGI and Wyoming will have its AGI, and they’ll all compete against each other. I think they imagine that all the labs will merge, which is actually the opposite of how China does industrial policy.

But then you do think that the American government, basically, if it says, “Do this,” xAI and OpenAI will do it?

No, actually, I think in that way, obviously, the Chinese system and the U.S. system are more different.

Although it has been interesting to see how different lab leaders have changed their tweets in the aftermath of the election.

More bullish on open source.

And didn’t Sam have a thing where— I think previously he said that AI would take jobs, and how do we deal with this? Then didn’t he recently say something at a panel where President Trump was correct that AI would create jobs or something? I don’t think that, in the long run, you believe this.

The reason why humans should be excited about even their jobs being taken is that they’ll be so rich. Why do they even need them?

Yeah.

Much richer than they are now, modulo this redistribution—and not screwing it over with some guild-like thing.

Yeah.

You mentioned the atomic bomb, and we also mentioned off-camera that you don’t think the nuke is a good comparison for what happens when a lab figures out AGI. How does it play out? If a lab figures out AGI, what then happens? Is there a huge advantage if one country has it first, or if one lab has it first? Do they dominate? What does it—

I think it’s less like the nuclear bomb, where there’s a self-contained technology that is so obviously relevant specifically to offensive capability. You can say there’s nuclear power as well, but nuclear power is just this very self-contained thing, whereas intelligence is much more like the Industrial Revolution, where there isn’t 1 machine that is the Industrial Revolution. It’s just this broader process of growth and automation and so forth.

So Brad DeLong’s right and Robert Gordon is wrong.

If Robert Gordon said there are only 4 things—just 4 big things, really—and Brad DeLong is like, “No, it’s a process of discovering it.” So, anyway—

Interesting.

And what were Rob’s 4 things again?

Oh, I mean electricity—

The internal combustion engine, steam power, and then—what was the fourth one? Maybe plumbing.

Right?

I think that was the fourth one.

Yeah. Or even in that case, maybe that’s closer to how I think about it: You needed so many complementary innovations in order for things to work together. Internal combustion engines were invented in the 1870s, and Drake found the oil well in Pennsylvania in the 1850s. Obviously, it takes a bunch of complementary innovations before these 2 things can merge, before they’re just using the oil for kerosene to light lamps.

But regardless, if it’s this kind of process, it was the case that many countries achieved industrialization before other countries. China was dismembered and went through a terrible century because the Qing dynasty wasn’t up to date on the industrialization stuff. Much smaller countries were able to dominate it. But that is not like we developed the atom bomb first and now we can just say we have a decisive advantage.

Because it was us. If that had been Nazi Germany or the Soviet Union, it would have gone differently.

Yeah.

How do you see the US-China competition playing out in terms of AI?

I genuinely don’t know. I think it’s possible that there could be some positive outcome for both. It’s not like a nuclear weapon, where both countries can just adopt AI. There is this dynamic where, if you have higher inference capacity, not only can you deploy AIs faster and have more economic value generated, but you can have a single copy—sorry, a single model—learn from the experience of all of its copies, and you can have this broadly deployed intelligence explosion.

So I think it really matters to get to that discontinuity first. I don’t have a sense of at what point, if ever, it is treated as the main geopolitical issue that countries are prioritizing. I also—from the misalignment stuff—the main thing I worry about is the AI playing us off each other rather than us playing the AIs off each other.

You mean AI just telling us all to hate each other, the way Russian trolls currently tell us all to hate each other?

More so like the way the East India Company was able to play different provinces in India off each other and ultimately, at some point, you realize, “Okay, they control India.”

You could have a scenario like—okay, think about the conquistadors: a couple hundred people show up to your border and take over an empire of 10 million people. This happened not once; it happened 2 or 3 times. So why was this possible? Well, the Aztecs and the Incas weren’t communicating with each other. They didn’t even know the other empire existed.

Whereas Cortés learns from the subjugation of Cuba, and then he takes over the Aztecs. Pizarro learns from the subjugation of the Aztecs and takes over the Incas. They’re able to learn, “Okay, you take the emperor hostage, and then this is the strategy you employ,” et cetera. It’s interesting: The Aztecs and Incas never met each other, and that worked both times, sort of.

Yeah.

Like, that’s interesting—that these totally disconnected civilizations both had similar vulnerabilities.

Yeah.

I mean, it was literally the exact same playbook.

One could say that the Spanish, on their side, had “Guns, Germs, and Steel.” But how could this have turned out differently if the Aztecs had learned this and then told the Incas—I mean, they weren’t in contact, but if there were some way for them to communicate—“Here’s how you take down a horse”?

Right. AI is trying to trick you into doing this. Watch out.

Yeah, exactly. That would require a level of trust. I’m not sure it’s plausible, but that’s the optimal thing that would happen.

At the lab level, do you think it’s multipolar, or is there consolidation? And who’s your bet to win?

I’ve been surprised. You would expect, over time, as the cost of competing at the frontier has increased, there to be fewer players at the frontier. This is what we’ve seen in the semiconductor companies, right? It gets more expensive over time; there are fewer players. There’s now maybe 1 company that’s at the frontier in terms of global semiconductor manufacturing.

We’ve seen the opposite trend in AI, where there are more competitors today than there were a year ago, even though it’s gotten more expensive. I don’t know where the equilibrium here is, because the cost of training these models is still much less than the value they generate. So I think it would still make sense to 10× the amount of investment for somebody new to come into this field and 10× the amount of investment. Do you have a take on where the equilibrium is?

Well, I mean, it has to do with entry barriers. Basically, it’s all about entry barriers. The question is, if I just decide to plunk down this amount of money—if the only entry barrier is fixed costs, I’d say we have such a good system for loaning people money that that’s not going to be that big a deal.

But if there are entry barriers that have to do with, “If you make the best AI, it gets even better,” then why enter? That’s the big question. I don’t actually know the answer to that question.

Yeah, there’s a broad question we ask in general: What are the network effects here? And what is the utility? It seems often to be brand.

Yeah. Yeah. I’m not sure that’s a network effect, but brand—everybody just sort of—OpenAI’s ChatGPT is the Kleenex of AI. Kleenex is actually a tissue, but we call it a Kleenex because there was a company called Kleenex.

Where are you going with this? Are we making a point?

Oh, no.

Well, no, I’m just saying it’s—or what’s another example? Xerox. Yeah—

You Xerox this thing. Xerox is just 1 company that makes a copier, right? Not even the biggest, but everybody knows that it’s Xerox. So ChatGPT gets massive rents from the fact that everyone just says, “I’ll use AI.” “What’s an AI?” “ChatGPT. I’ll use it.” Brand is the most important thing.

But I think that’s mostly due to the fact that—

So far.

This key capability of learning on the job has not been unlocked.

I was saying that could be a technological network effect that could supersede the brand effect, possibly.

Yeah. Yeah. And I think that will have to be unlocked before most of the economic value of these models can be unlocked. By the point these labs are—they’re already worth hundreds of billions, but by the point they’re generating hundreds of billions of dollars a year, or maybe trillions of dollars a year, they will have had to come up with this thing, which will be a bigger advantage, in my opinion, than brand network effects.

Is Zuck throwing away money, wasting it on hiring all the—

No, I think it’s—I mean, people have been saying, “Look, the messaging could have been better,” or whatever. I think it’s just much better to have worse messaging or something, but then not sleepwalk toward losing.

Also, if you just think about—okay, if you pay an employee $100 million, and they’re a great AI researcher, and they make your compute—your training or your inference—1% more efficient, Zuck is spending on the order of $80 billion a year on compute. If that’s made 1% more efficient, that’s easily worth $100 million. $100 million is below the break-even point for this extra researcher. So the real question is why we haven’t hit that break-even point yet.

And if we, as podcasters, encourage 1 researcher to join Meta, I mean, what’s the—how do you put a price on that?

Yes.

Do you have any last words for the audience based on our conversation?

I don’t know. I read your stuff a bunch. It’s great to actually just talk in person.

Thanks, man. Yeah. I have to come up with an English-language book so I can do the podcast.

I've written a Japanese-language book published in Japan, but I have to write my English-language book so I can do the Dwarkesh podcast. It's one of my dreams.

Amazing. Amazing. No, Dwarkesh, thank you so much for coming on. It's been great.

Awesome. Thanks, Erik.