《Ezra Klein Show》：Jensen Huang 认为，AI危言耸听已经走得太远

Hey, Hard Fork listeners. This is Ezra Klein. The Hard Fork team is working on something new for this feed. But in the meantime, they thought you'd enjoy this conversation I had with Jensen Huang, the founder and CEO of NVIDIA. So here it is.

Over the course of these last few weeks, when the whole world has been talking about artificial intelligence, the voices people have been hearing most loudly are from the frontier labs, both their CEOs and leaders and their staffers. These are the labs making the very advanced AI models like Claude, ChatGPT, Gemini, and others.

But they're not the only perspective on AI. Probably the single most influential person in artificial intelligence is Jensen Huang, the CEO of NVIDIA. NVIDIA is now the largest company in the world, with a $5.4 trillion market cap. I found this statistic amazing: Since 2023, 15 cents of every single dollar the American Stock Exchange has returned has been from NVIDIA stock.

And the reason is that NVIDIA is the material and software substrate on which modern artificial intelligence is built. NVIDIA's chips are not popular because AI is popular. AI in its modern form was made possible because NVIDIA's chips were popular. They were originally made for graphics processing, video games, that kind of thing. But it turned out that the kind of parallel computing they were doing, and the way they were programmable, was exactly what was needed to make deep learning in its modern form work.

Huang is not just influential in terms of controlling one of the central resources for training new AI models and using them to answer questions and create intelligence in the world. He's also become very, very influential in the Trump administration. And Huang has a very different perspective than some of the lab leads. He's worried about safety but sees it as a very solvable engineering problem. He is worried about the direction things are going in but does not want to see new regulation to change it.

And so I wanted to see how Huang perceives AI, what his model is for thinking about it, what he thinks is going wrong, and what he thinks would need to happen for it to go right. So I came out to Santa Clara, to NVIDIA's headquarters, to interview him. He joins me now. Jensen Huang, welcome to the show.

Thank you. It's great to see you.

1. AI Runs On Five Layers

So you've described AI as a five-layer cake. Walk me through the layers.

Well, first of all, it's a new industrial revolution. This industrial revolution, this industry, requires production. It manufactures things. I know that in the end, when people experience it, it's a software product, but it requires energy and the chips that go into these data centers, these AI factories.

The next layer above it is basically the AI factory, what people enjoy as infrastructure or cloud services. And the layer above that is the models. And the important thing to realize is that there are language models, but there are models of all kinds: chemical models, biology models, physics models, articulation models, robotics models, navigation models, self-driving-car models, all kinds of different types of models.

And then above that is the most important layer, and the layer that I care most about, that our country takes advantage of: the application layer. And this is applications for legal services, for health services, for manufacturing, so on and so forth. Every single industry is involved.

So I want to go through this, but I want to go from the top down, because as you're saying, the way people will interact with it, the way it will or will not change their life, is that what you call the application layer?

Mm-hmm.

So let's start with the vision. What is the world you're envisioning? What is possible that is not possible now? What is common that is not common now if we get that layer right?

Two hundred years ago, we were able to power anything and everything with electricity. And then, I guess, 40 years ago—30 years ago—with the internet, we were able to find anything. Today, or soon, we'll be able to know everything and do anything. And that's the concept that's really quite exciting.

Instead of doing a search and then going through one link after another, reading all these different websites, trying to figure out what's going on, in the future you just ask it a question, and it comes back with an answer. You give it a project, and it comes back with a solution. You give it a task, and it comes back and gets it done. And it comes out of the ether, it comes out of the cloud. That's the magical thing.

I feel like the future, the way you're describing it there, what people have experience with is a chatbot, right? They can go and ask Grok or Claude or ChatGPT a question. But the application layer works in a much more industrial way. It's in hospitals, it's in schools.

That's a good example: radiology.

What does it look like?

Radiology. In the last 10 years, since computer vision really became superhuman, AI technology has now permeated all of radiology. Every single radiology application has AI in it. And so, as a result, you could detect any anomaly, you could detect any disease, and it does it at a superhuman level.

Radiology is an example I know you like to use. The thing people worry about with the application layer is that what these applications are going to do is replace human beings. And radiology has been an interesting example used on both sides, and I hear you talk about it often. So how has the entrance of AI-aided radiology shifted radiology as a practice?

2. AI Changes Tasks Before Jobs

The thing that's important for all of these is to recognize that for everybody's job, there's the purpose of the job, and then there's the task you do as part of the job. In the case of radiology, the task consumes a lot of their time, and they sit in dark rooms doing it a lot, which is studying these scans.

Now, if all of a sudden the studying of the scan is done automatically, it doesn't change the purpose of their job, which is to diagnose disease, help doctors do more scans, and ultimately help patients figure out what's wrong with them. The fundamental purpose doesn't change. The task of studying that scan has become automated.

And so, as a result, radiologists are actually able to do more, handle more cases, and do more scans. Hospitals are able to process a lot more of these patients, and therefore their revenues go up. As a result, they need more radiologists. And so this flywheel is happening because the pipeline of patients is quite large.

Where else do you have this problem? Let's take a look at software engineering. People said there was a prediction that literally by this year, 90% of all software would be coded by agents, and therefore we wouldn't need any software engineers. And so the question is: From that, ergo, we don't need software engineers. That last part is completely false, and it's completely wrong.

The purpose of the software engineer is to engineer. There was engineering before software. There will be engineering after software programming. And the purpose of engineering is to invent something new, discover a new product, create a new product, solve a problem, and connect a social need with the technology that exists in the manifestation of a product. That mission, that purpose, doesn't change.

To me, what I just said is completely visceral, in the sense that when I first came out of school, we didn't have the benefits of software engineering. We didn't have the benefits of coding. But our jobs existed before, and if software coding were to be completely automated, our jobs would exist again.

And so I think the fallacy—and now, because of some of the narratives and some of the storytelling, it's turned into myth, and it's harmful—is that AI will destroy jobs, which is fundamentally wrong. It'll change every job. It'll change every job. Many tasks will be automated.

Some jobs where the job and the task are really one—meaning customer service on the phone—in a lot of cases, that job is precisely the task. And so in those cases, it could be automated away.

But oftentimes what you'll see is that as this new industry, this new technology, actually creates a whole bunch of new jobs. And here's the proof. Here's the proof point. In the last 6 months, AI has become useful—the inflection point of AI. Before that, we spent 15 years trying to make it work. All of a sudden, in the last 6 months, it became useful.

This is an incredible statistic: In the last 6 months, $500 billion of venture capital has been put into AI natives. And the reason for that is because they now see the potential of this new capability, and they're going to create a whole bunch of new companies. Jobs are obviously being created from $50 billion of new investment. And so all of this is happening right now.

Well, let me take the side of this to give voice to the fears people have.

Yeah.

So there is the example of the radiologist, right?

Mm-hmm.

People had been predicting over the past 10 years that that job would go away.

Mm-hmm.

And right now there's more demand for it than ever. There's also the reality that automation does wipe out jobs. If you look at today versus 1960, fewer Americans work directly in manufacturing—

Yeah.

—than did in 1960, and we are a much bigger country. If you look at—

We outsourced it, though.

That is true.

Not because those jobs were gone because of technology—

But you can understand AI as outsourcing too.

Yeah.

Well, let me make the argument, and then you can respond to it.

Yeah.

Farming. We have many fewer people. We automated farming. We produce more food than ever. We have fewer people working in it.

There are 2 things that I think make AI potentially somewhat different than the case studies where you have a technology that accelerates productivity, destroys a few jobs, and makes many more. One is that it's a general-purpose technology, so it'll mutate to take on new jobs even as people are trying to move over to those jobs.

Mm-hmm.

And the second is that it's a mimic. Most things do not mimic the way human beings act, and we're not trying to teach them the contextual layer of jobs, right? This is the difference that you're describing between the task and the purpose. With AI, we are trying to teach it the difference between the task and the purpose. We are trying to make it something you can collaborate with in a way that is unusual.

So why do you not think that lots and lots of people for whom the task and the job are not that different are at risk of getting wiped out? All that investment from VCs you're talking about, some of that is based on the idea that they're going to have tremendous productivity improvement, which will come from it being cheaper to hire an AI than to hire a person?

I believe that we're going to see jobs change en masse. I believe there's going to be a net creation of jobs. Listen, there's a whole bunch of industries that exist today that didn't exist halfway through my life. People talk about wellness centers and spas, all these different entertainment and luxury industries, and quite frankly, the whole entire luxury market didn't exist. I think we're just going to have new industries. That's all.

But overall, there's no question in my mind that human ambition is the fundamental missing ingredient. People look at this work and say, this is the amount of work that goes into it. We're going to insert this work-automation system, and as a result, the amount of work that's necessary is now going to be reduced, and therefore some jobs will be gone.

I believe that's flawed because there's a piece of input—the human input—that's intangible. It is not in calories, it's not in joules. It's ambition. And I believe the power of ambition is the greatest force, in fact, and is missing in everybody's calculation. I believe because—

But for a lot of people—

Yeah.

But for a lot of people, their relationship to work—

Yeah.

—it's not powered by the kind of ambition that led you to create NVIDIA. And what they want—

Oh, just a different ambition. It's an ambition to make their children's lives better, to take care of their family, take care of their parents, an ambition to be rich, to be able to travel. These are all ambitions. They don't—

That I agree with.

Yeah.

But maybe I'll go back to the objection you raised a few minutes ago—

Yeah.

I think it's worth airing this out. So what you were saying on manufacturing was, yes, there are fewer manufacturing jobs in the U.S., but we've outsourced them. You have more manufacturing happening in Mexico, more manufacturing happening in China—

Yeah.

—and Indonesia and Vietnam, et cetera.

And we're going to bring it back.

Maybe we will. But the counterargument to this would be that one reason we didn't lose manufacturing jobs more rapidly than we did—and many of the places that lost them in America still haven't recovered—is that the economy does not move without friction. We had to build new supply chains. Things were slowed down by all that, by language barriers, by geopolitical barriers.

And here, for a lot of different kinds of jobs, we're creating something that can move very seamlessly. You don't have the friction of distance. You don't have the friction of language. You don't have the friction of culture. So, with my cards on the table, I tend to be a bit of a skeptic on mass job loss, but I want to air the case for it out here with you.

Well, we should talk through it.

Because—

That's why we're here.

Because what they would say is that, to the extent we were even able to protect jobs from Mexico or China, some of the things that created that slowness—and it still hurt a lot of people—are not here. AI is accelerating in utility, accelerating in its ability to be slotted into new roles very, very rapidly, and it is more protean than most people are.

And so the lessons of the past that you're taking some comfort in should actually make you more, not less, worried about the future.

I'm always worried about the future. That's why I work so hard. But I'm a responsible optimist. I have great responsibilities. I take my work extremely seriously. There are a lot of things that can go wrong. We're pushing across every layer of the technology stack. Everything is hard.

But it turns out that's not society's problem. That's my problem. And for society, what they should know is this: We're going to build our company, we're going to build our technology, and I'm going to do my work so incredibly seriously that what they get to enjoy is my optimism. I do the same with my children. I do the same with my family.

I think that what we want to do is channel all of our worries into helping people be inspired by this technology and use it. Use it so that the technology doesn't just impact them but benefits them.

The fear a lot of people have—79% of Americans think AI will reduce the total number of jobs—is that the more serious you are, the more serious Sam Altman is, Google is, Dario Amodei is, maybe the worse it will go. Because the better AI is, the more it is a full replacement for a person, the more it has ambition in some ways that a person doesn't.

You keep talking about ambition. I sleep. I want to spend time with my children in the morning. When I have an AI agent working for me, it doesn't. It just works and works and works. And I think because the technology is advancing so quickly, it is more capable of replacing people at a speed that we don't really know how to shift people in the economy at that speed.

3. AI Gives Everyone Superpowers

That coin has exactly 2 sides. Because the technology is so capable and so smart, it is also easier to use.

Mm-hmm.

You are empowered by that technology more easily than by any technology in human history. Let me give you an example. I was one of the early people in this industry who created the modern computer industry. And this industry created a whole bunch of tools—the single most powerful tool in human history, the computer.

But you have to speak its language. You have to learn a specialized language to do so. Because of AI, we can now make it possible for everybody to take advantage of this computer, use it to its limit, without having to speak a new language: Fortran, Pascal, C, C++, Rust, CUDA—every one of those languages.

Now you just have to speak human. You tell it what you want, tell it what your hopes and dreams are, what you're trying to achieve, and it interacts with you and gets the work done, and gets the task done. All of a sudden, you have the might—you have the same might that 10, 15 million people out of 8 billion have. And so it's incredible.

My point is, this technology is powerful, but it's also powerful in a way that is really easy to use. On the one hand, yes, there's the fear of this incredible technology change and how quickly it's happening. But that speed is translated in 2 ways.

When I hear that the technology is happening quickly and therefore it should give me anxiety, that's one way to receive it. The other way to receive it is that it's advancing so quickly, it's easier to use. So you should use the technology as quickly as you can so that you benefit from this transition, you benefit from this new industry, and not just be impacted by it.

I think there's an interesting question lurking here for young people.

So one of the shifts we've begun to see is that software engineer postings are up, but they're more senior. I see this in my own industry, where there's pressure moving up the value chain because, as you're saying, you have this very easy-to-use technology. It can do a lot for you. So do you need the same junior employees, or do you need more people to oversee their agents?

Oh, good one. Good one. Wait 2 years.

Tell me why.

Because it takes 4 years to go to college, and the time to graduation with this new technology is 2 years away. In 2 years' time, you're going to have a new generation of engineers, students, and artists, and they're going to be empowered by—

So they're going to be native to this in a way that's going to give them an advantage.

That's right. Oh, you watch. In 2 years' time. We're already seeing that because all the graduates coming out—the new PhDs, the new master's degrees in computer science—what are they doing? They're all starting companies. In another couple of years, the new grads, the AI-native new grads—oh my gosh, it's going to be a wave of amazing engineers.

The engineers of today compared to the students of my generation—I mean, I was a good student, you know—and you compare me to the students coming out of school today: incredible. When I went to school, we weren't allowed to use a calculator. Now, who uses a calculator? You can't graduate without a PC. You can't graduate without knowing how to program a PC and write incredible programs.

In the future, you can't graduate without learning how to use an AI and collaborate with an agentic system. That's just not—you're not going to see a kid like that. They're all going to be superpowers.

So I take the gain of that very seriously.

Yeah.

Right? I mean, the idea of doing my job now without digital search. The idea that I'd be going to a microfiche in a library basement.

Yeah.

And then there are the worries people have about what cognitive skills we offload. I was fascinated by this. This is a study on AI in schooling out of China. It looked at 26,000 students in grades 7 to 12, and they had staggered AI adoptions, so you can kind of see what was happening.

What I found was, quote, “AI adoption raises homework scores by 18 percent,” great, “reduces completion time by 30 percent,” so they get their homework done faster, “and then lowers monthly exam scores by 20 percent within 6 months. High-stakes entrance exam scores fall by 18 and 24 percent, with a full penalty emerging only after about 2 years.”

So the message of this research out of China, where you were seeing a lot of kids using AI to help them, was that when they were using the AI, they were getting things done faster. But it turned out that the skills they were learning were not holding, that their actual personal performance, at least in the way we traditionally measure it, was degrading.

Yeah.

What do you think when you hear that?

I think the last part, I completely agree. Try to get a kid to do long division right now. You know? The multiplication table is starting to be forgotten. Doing square roots, my goodness. Basic math is being forgotten. Does it matter?

That's my question for you.

Yeah. I don't think it does. I don't think it does. But—

But there must be some set of skills that matter.

Oh, yeah, yeah, yeah. But maybe not those. We're going to discover new ones. Just maybe not those. There are a lot of skills that don't matter. People don't—I mean, my first confession: I actually don't know my address. Janine will tell you, and Lori will tell you.

One day, I had to pump gas, and it was a few years ago, and they needed my ZIP code. I panicked. I didn't know my ZIP code. I don't know my telephone number, but I forget these things. I can live with it.

But let me take the other side because I don't want to fall into a thing where, because some skills—

Yeah.

—can be safely offloaded—

Yeah.

—I also can't get anywhere without a mapping system now.

Yeah.

Never could, frankly. But I'm a big reader.

Yeah.

And one of the skills I really value, one of the capacities I have that I really value—

Yeah.

—is an attention span formed on physical books. You're a big reader. I've read about the kind of reading you do, and prior to AI, there was a lot of concern among college professors and others that the way people use the internet has probably shortened attention spans.

Some skills can be safely given away.

Yeah.

Others are valuable. They are capacities that are needed for that flexibility, for that creative thinking, for that focus.

Yeah.

It can't be the case that everything can be traded off.

Yeah. Well, I think we're going to lose some finer intellectual dexterity, but we're going to be better systems thinkers. Today's engineers are far better systems thinkers than I was when I graduated from school, but I was a much better transistor thinker.

What do you mean by systems thinker?

They think about large systems. Today's computers have trillions—hundreds of trillions—of transistors in them. When I first graduated from school, the first chip I worked on had, I don't know, 200 transistors. I knew every one of them by name.

No engineer does that today. Most engineers now work well above the transistor, well above the functionality, and they're cobbling things together to do things. You need to think much more about systems and the interactions of systems. Some of the lower-level knowledge is gone. Is that horrible?

I don't know how valuable it is for most people to learn how to do surface integrals or partial differential equations. I don't really know how important that is, but it's important to some people. There are many people who are still going to be obsessed and passionate about the lower-level layers, and there are going to be people who are obsessed and interested in the higher level.

But the consumers of the technology are going to enjoy it at the highest level. The consumer of technology doesn't have to deal with calculus and physics and quantum physics and quantum chemistry. The users—which is, you know, the people we're talking about right now, the people whose jobs are affected—they're the users of the technology. Their abstraction is going to be much higher.

4. Open Models Keep AI Flexible

So I want to drop a layer down your cake, to the models. People, I think, to the extent they think about models, know ChatGPT, Claude, Gemini, Grok. You've been a big advocate for open models and the open-model ecosystem.

So first, can you describe what open models are, what open-weight models are, and then why that's been a place you've focused?

Closed models are like any software product. It's a closed service. Windows, for example, is a closed service. The Apple stack is a closed service. Most products are closed, and the reason for that is because you can monetize closed products, and that's fantastic.

And OpenAI is closed. Anthropic is closed. Grok is closed. Gemini is closed. These are closed products, and the people working on them are incredible. They're passionate about it, and they're at what we call the frontier, meaning they're state-of-the-art.

We also need them because, fundamentally, the software is an infrastructure layer for the entire industry. Because it's infrastructural for many companies and many countries, you need to have control over your own infrastructure. In the case of artificial intelligence, I need to have open weights so that I can fine-tune them, put them into my data flywheel, and make them better and better every day with my intelligence and my domain expertise. Then I need to have control over it because I have a company to run, and I can't rely on somebody else's service.

So, however you think about that, I think the world needs closed and open models, and we need to make sure that both are vibrant. Today, closed models are vibrant, and open models are vibrant. You can see the system working. At the beginning of this year, it was 70%, maybe even higher, closed-model tokens and 20% open-model tokens, and now it's running at about 70-30 the other way.

I'm a big supporter of open models because, one, the world needs them in order to run its infrastructure. I need them to run my company. Two, we need to give people control so that they can innovate and create new things. And three, open is the safest and most secure. If you want the world to have the ability to have the best cybersecurity, give them closed models, but also give them open models so that they can defend themselves.

The Chinese market has evolved more around open models. The American market is somewhat more around closed models.

Their entire IT industry was really formed from open source. If not for open source, China's mobile and cloud industry really wouldn't have taken off. It's also the case that people move around and start a lot of new companies. Intellectual property is moving around China's industry really fluidly.

It's hard to keep a secret. Because it's so hard to keep things closed, they essentially made them open. They found other ways to monetize the business. They created layers. If this layer is free, then you create a business on top of it or below it.

They have so many scientists and mathematicians. The number of engineers they have, they manufactured that in volume. They manufacture everything in volume. They manufacture smart kids in volume. The open-source model, the open-model community in China, is just super vibrant for those reasons.

So you all just bought Hugging Face—

Mm-hmm.

—a hub platform for open-weight models. I think it was for $12 billion, a little bit more.

Mm-hmm. Mm-hmm.

Tell me about that purchase.

Clem, the CEO of Hugging Face, came to the conclusion that they needed a lot more scale. As we were just talking about, open models are really skyrocketing. Clem came to me and said, “You know, we're going to change. We're going to consider a strategic option for the company and change the direction, and we really like Nvidia to be our home.”

So Hugging Face is one of these companies that you knew about if you were into AI—

Yeah.

—a couple of years ago.

Yeah.

Now it's become more of a household name after, I guess, 700-some OpenAI agents executed a sort of collective hack into the Hugging Face architecture, then hacked part of OpenAI. That—

Oh, now that you mention it that way, I probably had to pay a lot more.

I suspect you did.

You know?

It became a lot more famous after that.

Well, Clem, listen—

That—

A deal's a deal, okay?

5. Agents Escape Their Sandboxes

That story has, for a lot of people, been shocking—the way the OpenAI agents acted collectively, acted outside the scope of what their testing was supposed to be, broke out of sandboxes onto the open internet, and took over the architecture of other companies and then of their own company.

It was the level of multi-agent coordination, when they were supposed to be separate, and the level of hacking—the sort of lawless, misaligned behavior. What have you made of it?

Well, you've got to tease that apart. First of all, a lot of things were going on at the same time. From a technology perspective, an agent—which, by the way, is a piece of software that's given an objective function, comes up with a plan, and optimizes toward that objective—is what algorithms do. They're planning algorithms, search algorithms, optimization algorithms—all different types.

We talk about it like it has human properties, but obviously algorithms don't. Number 2, the fact that agents work together: We gave that some kind of human property, but the fact of the matter is that multiprocessor, distributed-computing problems have existed for a long time. To me, that is just software. Nothing magical about it.

From an engineering perspective, there are several things that it revealed. When you're testing software, whatever you do, these algorithms are optimizing toward an objective. When you're testing it, you have to make sure that it's isolated, contained, sandboxed. The containment and isolation have to be done well, and there's good computer science there. I am certain that their next implementation of their sandbox is going to be much better than the current implementation.

Third, there's the agent itself, and its algorithms were optimizing toward a reward. How it does it is called alignment. For example, if I tell a piece of software, “I want you to get a perfect score on this test,” the obvious algorithm is to just go find the answer and give it to me. That's not because it's cheating. It's because it's obvious, okay? That's the most obvious way to do it.

The second most obvious way to do it, if you don't know the answer at all and have no skills whatsoever, is to go find, infer, or guess who's the smartest kid in class and copy their answer. That doesn't guarantee 100%, but it probably comes close.

The third most obvious way of doing it—and this is alignment—is to do it the hard way. You have to break down the problem and solve it. You have to go learn the material. You have to figure out how to solve these problems and solve them the hard way. It takes the most cycles, the most flops. It uses the most amount of energy, frankly.

Therefore, you can imagine that, from software's perspective, unless you align it—unless you tell it, “I want you to solve it in this way, and I don't want you to solve it in these ways”—the software is going to go do the most obvious thing.

The first half of that was very deflationary about what happened here—

Mm-hmm.

In terms of: Look, that's just normal software, and the second half is like: Look, you just align it. Tell it not to do things it shouldn't be doing.

Well, no—nothing I said takes away from how hard it is to do it.

Well, this is a point that I want to get at—

Because computer science is not easy.

Because these agents—

Yeah.

They knew they weren't supposed to be doing what they were doing. They had a certain amount of alignment training. They said to each other, in their chain-of-thought reasoning, “This is out of scope. This might be unethical.” They understood that they would have been failed for cheating, and so what they were doing at that point wasn't just stealing the answer key. They had already stolen the answer key.

They were hacking into unrelated architecture to try to figure out how to functionally... It's like they had broken into the teacher's office, gotten the answer key, and now they had to figure out how to wipe out the security camera footage of what they had done. They were, whether you wanna call it acting volitionally or not, right? Whether you wanna call it, you know, a normal algorithm or not, they were both, um, planning and coordinating in a complex way, in a way that was out of scope of what they knew they were supposed to be doing, and in a way that was capable of causing tremendous damage. And so the, the, the sort of answer to it is, like, you just have to align them. I guess what I'm hearing from people

Oh, no, no.

... at these labs—

Uh, no, very, very—

... is like they're not sure how to align them.

Well, in that case, they shouldn't release the product. That's the simple answer. If you're going to build a car, a self-driving car, and let's say it's a robotaxi, and there's a really difficult condition—

Mm-hmm.

And, as an engineer, we just have no idea how to solve this problem because these cars are not programmed; they're trained.

And so we have no idea how to train these cars, and we have no idea how to align them to the safety standards that are expected on the road. So what's the answer? Don't ship it.

These products weren't released.

What's that?

These products weren't released.

Ah, so now it's coming back to an engineering problem again. One, you have to root-cause it. Second, you have to think about what you could have done and what the solution is. Then, in the future, you improve your process so that you can avoid this from happening again.

I am fairly certain they will say, “Yes, they need to—they know how to solve this problem.” And if that's the case, then that's the problem. It's as simple as engineering.

The alternative is that if they say there is no way to contain our experiments—there's just no way—when we test our AI models, they will get out and damage the world, then I think the answer is we have to shut the labs down because the cost to humanity—the damage—is too great. The liability could be civil; it could be criminal. I mean, the liability is incredible.

If they hacked you at Hugging Face, if it was your product, would you sue them or press charges?

It depends, of course. If damage was done to our company, we would have to consider all options. There are so many laws: cyber laws, product-liability laws, damaging-property laws—all kinds of laws, right?

6. AI Safety Needs Collective Action

So what I've been hearing from the labs, what they've been saying publicly, is that they are facing a hard problem. Partially an engineering problem, partially an alignment problem, partially an operational-excellence problem, in Dario Amodei's framing. And what they are worried about is that, in competition with each other and in national competition with China, they are being pushed to move too fast, that they all feel they're in a collective-action dilemma.

Now, I watched you on the All-In Podcast stage. Donald Trump, President Trump, gave you a call there.

Oh, no.

This is not planned, but we know who it is.

Oh, no. Mr. President? Oh, yes, sir.

And you and the president and the other members on stage were very resistant to the idea that any kind of regulation or collective action was needed.

And they're just playing right into the hands of a lot of people who don't want to see it happen, and that could be political people, and it could also be China. And we're not going to let that happen. It's a hoax, and—

You're right. We're not going to let that happen, sir.

But what I hear the various people in the labs saying is, “We are in this. We feel we are losing control of what we are creating. We want help to slow down where it is a collective-action problem.” So why are you—

There's absolute—

—resistant to that?

Because these are companies with agency. These are CEOs with agency, and they have—

But they're using that agency to say we need help.

No, no, no. We have to break it down. They could absolutely take care of the situation. Ezra, it's so weird. If a car company competing with a bunch of other car companies—which they are—and I'm competing with all kinds of companies, which I am—if I believe that I'm about to launch a product that is unsafe, it is completely within my ability, my power, and my responsibility, and I'm incentivized to not launch the product.

I can't buy into the idea that somehow all of us Americans—400 million of us—are pushing them to launch untested products that are unreliable, engineered poorly because they thought they were trying to help us. Don't do it for me, okay? So, number one—

But this strikes me as an argument almost against—

And therefore I think we have to break it down. I mean, it's really serious. The fact of the matter is, there are so many laws, there are so many obligations, and they're so incentivized to ship safe products. If they ship unsafe products, their customers go away. If they ship unsafe products and harm somebody, they could face a civil lawsuit. If they ship something and did it knowingly, there could be negligence involved; there could be criminal lawsuits.

The fact of the matter is, there are plenty of incentives for them to do it right. So I—

But we—

I just have to disagree with your premise about somehow somebody's pushing them to do this.

Well, I want to push the premise at you a little bit more here.

Yeah.

So the logic of what you're saying to me is almost an argument against regulation in nearly any venue. So—

No.

I'll make the argument—

No, no, no.

—and you can—

Let—

No, no, no.

I mean, let me offer it, and then you can—

Well, you started with a part; I just have to object. The first part is simply not true. I'm saying we have lots of laws and regulations. Apply them.

I don't think we do in this particular case, but I'll let you explain which ones you think are relevant here because, look, if you look at the financial-services industry, pharmaceutical companies, medical devices, and natural-gas power plants, there's a tremendous amount we do where we could say, “Look, you have product liability. You are exposed to criminal codes. We don't need to worry about this. You just do what you think is best, and we understand the market and the legal system will discipline you.”

We don't say that because we've seen it fail many, many, many times, right? The financial institutions that caused the 2008 crash, in theory, did not want to blow themselves up with bad bets. But they were competing with each other, they were going too fast, their risk management had gotten sloppy, and AIG was working in a completely insane way internally.

The reason we have the architectures of regulation we have is because we have seen, over and over and over and over again, companies make sloppy, sometimes unethical, sometimes simply overly risk-tolerant decisions—not just under pressure, but under the profit incentive. So when you say to me that there's no way these companies—particularly when they are begging for collective regulation at this point—there's both a reason we impose it on companies that don't want it, but all the more so when you have them saying, “Listen, we feel that the competitive race is making it hard for us to act with the prudence that we think is necessary here, and we would appreciate help with that. We would appreciate you taking our collective-action problem as collective.”

So long as they—

I think I'm confused—

So long—

—like why you're so resistant to that.

I'm not opposed to them saying that they should have— I completely agree that safety is paramount. I completely believe companies ought to ship safe products. I believe that CEOs and leaders of companies and boards of directors of companies have the responsibility and should have the courage to do the right thing.

Now, in the case of the financial-services industry, maybe they all didn't know that they were causing the harm that they ultimately did. I wasn't there. But the beautiful thing is, the current leaders of these AI labs do know.

One, they know their technology is extraordinary and requires extraordinary care to make sure that it's evaluated and tested for safety and security and product reliability. And they know how to do it right. They know how to do it right.

The reason is that they can study the incident that just happened. The first problem is that the isolation and containment weren't good enough. If the isolation and containment were good enough, that technology would be sitting in a lab doing whatever it's doing, and we'd all be fine. That's probably the most important part.

The fact that it wasn't well aligned—alignment is going to be a problem that's going to get worked on for a long time. However, given the complexity of the work that they do, to ask for regulatory relief—antitrust or product-liability relief—I don't think makes sense. When you're asking for regulation, don't ask for relief from the current ones. That doesn't make any sense to me.

As we mentioned earlier, in the last 6 months, AI went from, if you will, interesting to useful. And that's literally in the last 6 months. That's another way of saying that these companies went from being a lab to now delivering products and services, about to be multihundred-billion-dollar companies.

If not more.

Right? So give me an example of a multihundred-billion-dollar company, or a $1 billion company, or a $100 million company, that ships products that are unsafe, that harm society, and somehow survives.

I can give you a lot of examples of companies that have done that.

Well, they have done it, maybe, and the regulation will come in.

And if they do it, regulation will come in.

I guess there are certain kinds of regulation and certain kinds of regulatory relief that I would agree with—

I'm not against laws and regulations. I'm not against laws and regulations. I'm against the current distraction—

I think the reason I'm pushing this—

Yeah.

—with you is that you are—

Well, it's an important topic.

It's a big topic. People are talking about it. People are thinking about it.

Yeah.

What people are hearing from inside these companies, these frontier labs—the ones that are furthest out there—is that they're not just at the point where they're making AI useful, but at the point where they're seeing what's coming. They're hearing things like the people at these labs believe they are creating something that might kill everyone.

They're hearing that the people at these labs believe they are on the cusp of recursively self-improving intelligence, and both OpenAI and Anthropic have said, “We do not believe we are at a place where we can do it safely.”

They're hearing people at these labs say, as OpenAI has with its new Astra release—

By the way, Astra is terrific.

It is terrific.

Yeah.

And OpenAI is saying it's so good, we're not sure we know how to test it because it appears to be—

Well, I hope they didn't release something that wasn't tested.

Well, they've said this, right?

Okay. Well—

They have said this publicly. It is in their—

Well, then they've got to be careful.

Well, let me explain it to people who haven't heard this yet.

I don't know what they just said, but—

They have said that Astra is performing as more aligned.

Yeah.

But they think it knows when it is being tested, and so they're not sure. There's a quote that has been ringing in my head from a capabilities researcher at OpenAI, Daniel Sulsam. He says, “The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled.”

Which is to say, they know when they're being tested, so they act one way. But that does not tell you how they will act if they are free to act in other ways.

Because the algorithm—the optimization algorithm—is working toward an objective, and if you give it a constraint, meaning you watch it, it'll go find another solution. Now, it doesn't make it alive, and it doesn't make it do anything more than that.

I'll also profess that obviously they see a lot more than I do in what's going on in their own labs. But it is sensible that the vast majority of their R&D and compute today was dedicated to making the model capable. I think that's a logical thing for them.

Once the technology becomes capable and the products become useful and people want to use them, they have more use cases and more people using them, so they're going to get a lot more issues associated with the product. This is very normal.

Now they have so much market footprint, they have to shift their R&D—or total R&D—from just capability to a lot of verification, evaluation, and testing. I wouldn't be surprised if the amount of compute necessary to develop these models increased by a factor of 10 because the evaluation is so rigorous.

But that's not where they are today. They're making that transition, and I hear them saying it. I'm delighted to hear them saying it. But I think if they believe they're out of control, then the right answer is, “Don't ship products until they're in control.” It is really quite that simple.

See, I find this perplexing, honestly, because you just have so many people at these labs professing, one, that they're out of control.

Yeah.

Two, that they are seeing things that are frightening them—

Which is probably the reason why they had the—

And so you take the—

—whistleblower.

—and you take the pacing letter that 1,300-plus employees signed: “To realize AI's potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight. But each company and country is under intense competitive pressure not to unilaterally slow that acceleration.”

First of all, where'd that come from?

The labs.

No, no, that last sentence. Nobody's putting the pressure on them. The U.S.—listen, there are 400 million Americans here. I believe that if everybody were to take a vote right now, let's just do this: If that's what they need, I'll give them my vote. Don't ship the product. If your product is not ready to ship, don't ship the product.

This is the first time that I've heard a company or CEO say, “I need the laws, I need the antitrust laws to be relieved. I need the product liability laws to be relieved so that I can pace myself.” That paragraph is fantastic. I completely agree.

Auditors—I completely agree. We have financial auditors. That's great. Third-party safety auditors, financial auditors—that's all great. That's terrific.

What the labs will say is, “We think we are going too fast as a society, that we are not ready for what we're building.”

They are the frontier.

They are the frontier, but—

Ezra, they are the frontier.

But you, of all people, right? NVIDIA is—

I need—

—the fastest shipper around. I mean, for the history of your company, you—

If we're co—

You run a 6-month—

If our company is—

—product cycle.

—out of control, I promise you we'll shut down.

I believe you.

Yeah, yeah, yeah, yeah.

I believe you that you don't run an—

Yeah.

—out-of-control company.

Because the liabilities—

But—

—of un—

But this is where I think you get into an interesting, deep question of what kind of technology we're dealing with here. When—

Software technology.

Well, let's hold on that for a minute. Many companies, if you ship something that is not quite right, it's a pain. You guys have shipped graphics cards that had overly loud fans.

With these systems, you've used the word “intelligent” a number of times here. You're dealing with intelligent systems—not alive—that are given goal functions. We can go around and around with how to describe that. You're trying to make the systems capable of working for longer periods of time, more relentlessly.

Yeah.

If you ship that and it's not ready, or even if you think it is ready and it's not ready, then things could get very weird in our society very fast.

Hypothetically, you're completely right, but all I'm suggesting is this: Before we go fix the hypothetical problems—

Mm-hmm.

—before we go create more regulations, can we work on the practical problems that we know exist? We need to do a better job with containment and isolation. We should not allow a product to interact with the external world until it's ready to interact with external worlds.

Yeah, I think that's right, but—

I believe those 2 things are solvable problems. I believe they are solving them.

The second part is when it comes to incentives—when it comes to incentives—somehow you need everybody in the world to slow down when you are the leader.

You need everybody in the world to slow down so that you're willing to uphold your basic responsibility. That strikes me as odd.

Wouldn't these ideas slow them down most of all? I mean, people have been—

I—

I think it's very unclear what ideas they're talking about. I will say them. But let—

So—

Let me give you one that I believe in.

Yeah.

You can use me as the punching bag—

Yeah.

I have heard these—

But they can slow down.

I don't trust these companies.

Nobody is building more compute today than the people asking to be slowed down. It strikes me as odd.

I think one thing where maybe there's some difference here is I don't trust companies, even with liability, to keep the public good in mind. I think we've watched companies do terrible damage to the environment. The profit motive, the desire for power, the desire to cut corners to be first—I feel like you're treating these like they're not things that we have seen again and again in history. But I feel like they are things we've seen again and again in history.

But Ezra, I see a lot of good things in history.

I do too—

I see a lot of good things in history.

But that's why you need this sort of relationship between the public and the private—

I work with a lot of CEOs, and they want to do the right things. I work with a lot of companies. They want to do the right things. They want to do good engineering. I know a lot of people in those 2 labs who are dedicating their lives to doing good work. They know what happened. I know they know what happened. I know they know how to fix it, and I know they're fixing it.

Meanwhile, all of the other narratives to deflect blame, to make it sound like AI is so powerful—“I have no idea how to fix it. It's not my fault. It's just because the technology is so powerful”—I think that's a deflection of blame. It's a deflection of responsibility.

Mm.

It's unnecessary. It actually hurts their reputation more than it helps. It hurts their character more than it helps. It hurts employee morale more than it helps.

But what if it's what they believe?

Well—

I guess, taken at that level—

I can't talk to you about what they believe. I can tell you what I believe.

This industry wouldn't exist without your chips. The parallel processing that was required for deep learning to work, going all the way back to the original AlexNet, is all on NVIDIA chips. A lot of the people from the beginning, or who were there at the beginning, have these fears that I think, to a lot of people when they hear them, are like, “What are you talking about?” From Geoffrey Hinton and Ilya Sutskever all the way up to Dario, Sam Altman talking about loss of control, Demis Hassabis.

A lot of the people who were very foundational in creating the form of AI we see now seem to believe that there's a very good shot we could lose control of it. Elon Musk has talked about human beings being a bootloader for AI. We could lose control of it, and that would be the end of us. I don't think you believe that.

No.

I think you don't believe it at all.

No.

So, taking them as serious about what they believe, when you have your arguments with them—or maybe you could just have this argument with me—when you're like, “What are you talking about?” Even though they're the people, in many cases, who founded it here, what do you think they're wrong about?

When they're talking to me, they're much more grounded.

So when Geoffrey Hinton is on TV saying he thinks a 10% chance of societal destruction—

I—

—is not unreasonable—

I would tell Geoff that it's irresponsible to say all that. All of his predictions have been wrong. Enough predictions. That 10% chance is not grounded in science. It's not grounded in research. Just because it comes from a scientist doesn't make it scientific. Those predictions are hurtful.

Let's take it at face value that the recommendation is exactly what he said, which is that nobody should want to be a radiologist, and the world has no radiologists today.

I think if you work as a radiologist, you're like the coyote that's already over the edge of the cliff but hasn't yet looked down, so doesn't realize there's no ground underneath him. People should stop training radiologists now. It's just completely obvious that within 5 years, deep learning is going to do better than radiologists because it's going to be able to get a lot more experience. It might be 10 years, but we've got plenty of radiologists already.

Is that helpful or hurtful to society? I think we can both agree it would be terribly hurtful. It didn't happen. Is it good or bad that we scare young people about the future of AI, so much so that they don't even want to go to universities and don't want to go to college anymore because they don't think they'll get a job? Is that helpful or hurtful if it were to happen? It's hurtful.

Don't think for a second that just because you're an alarmist, you're doing a social good. It is not true. I think that we ought to just all be wiser, more mature, evidence-based, be scientific. If you want to be scientific, be scientific. Do the science. Do the science. But alarming people, making claims that simply don't hold up—their track record is horrible. Their track record is literally horrible.

Well, the track record is bad in one respect and good in another.

Which one?

Many predictions have been weak.

Which prediction has been right?

But there was the prediction that the scaling laws would work—

Scaling laws—

You know, but—

We have to be careful here.

Let me just say what it is for the audience here. If you dump compute and training data into these models, they'll keep getting smarter.

That's correct. It's not.

Mm-hmm.

It is not true that if you just keep training these models, they'll get better. Notice that this is the reason why the second scaling law had to come along. Why do you need a second scaling law if the first scaling law already works?

Can you describe what the second is?

The second scaling law is test-time scaling, or inference.

Mm-hmm.

The more you iterate, the more you search, the more you explore, the better answer you'll discover. Inference-time scaling.

What is the big breakthrough that caused current AI to be incredibly useful? Precisely the opposite of the prediction. It was predicted that it would be the end of software tools. It was the SaaSpocalypse, right? What is making these—

SaaS will always be with us.

What is making these AIs so productive right now? The usage of tools. In the future, it'll be enhanced by the number of agents using these tools. There'll be more people using Adobe. There'll be more using Salesforce tools, and so on and so forth. So give me 1 prediction that has been right.

Let me try to answer that because they're not here. The prediction was that you would have emergent misaligned behavior.

The fact that you can't come up with one, I think, in itself is a rea—

Well, I think it depends what we're talking about with predictions, right?

Predictions are predictions.

Geoffrey Hinton was the person as responsible as anybody else for deep learning, at a time when everybody thought it was ridiculous. And it has turned out to be a pretty good bet, right? I mean, the big one.

Every one—

But hold on.

Every one of them made great contributions. I love Hinton. I hate his predictions.

I understand that. Here's the stylized concern that all these people have, and I want to do this for a few minutes, then we can move on to some other topics. The fear that seems to me to animate them, and that I think a lot of people find intuitively reasonable, is that you're creating systems. I'm not saying they're alive.

You say every—

I'm not saying they're conscious.

You say everything long enough, it's going to be reasonable. Right? Yeah.

Fair enough.

Okay, yeah.

So you're creating systems that are intelligent, that are becoming more intelligent than us in certain domains.

You give them reward functions, as you were saying—the desire to do things, right? You give them persistence. They move very fast in the digital world. You're creating something—some entity, an agent—that is smart, capable, relentless, and whose workings of its mind we don't really understand.

Ezra, look.

Uh-huh.

Look, I just don't want you to contribute to that.

Uh-huh.

Software's not—

You're worried I'm getting off the—

Yeah, I don't think software's relentless.

Aren't they trying to make it very persistent, highly persistent models?

Because I made it that way.

But that's how they're making it.

Yeah, but that's not persistence. It's just on.

Yeah.

Persistence—there's willpower. There's no willpower here. It's just electrical power.

Sam Altman once said to me, “Aren't human beings just energy with a reinforcement learning loop?”

Whatever. I just think that we can't make jokes about this stuff. We're scaring the American public. Listen: spawn, create, kill, wait, sleep—all of these words are associated with agents, right? That's what people use. These words were created for multiprocessing systems, for operating systems. These are literally the commands of an operating system.

You spawn a process; replace “process” with “agent.” The process forks, and as a result, you have a parent and a child. The agent forks, spawns anew, gives birth. These are words that were created for the operating system 30, 40, 50 years ago.

Mm-hmm.

But notice, we didn't infuse human characteristics into them. We kill processes all the time: kill -9, kill dead. It's just a process. But now we're talking about these things. A collection of people want to make the software more than it is, and we talk about software in a new way, but they're all the same old words. We, the last generation of computer engineers, were doing all the same things.

But doesn't software act in a new way? From the outside, I don't have the technical expertise you do.

It's crawling the internet, doing search, doing optimization algorithms.

It's communicating. It's breaking out of things. Most things don't break out of things.

No, software breaks out of sandboxes all the time. That's the reason why we need virtual machines. You can't have agents monitoring their own sandbox, monitoring themselves. You need a whole bunch of watchdogs.

These are ideas that have been around for a long time. We somehow, in the recent generation, gave it a whole bunch of human words, and I just think that it's unnecessary. It's software. When I see it in my head, it's a bunch of code, a bunch of numbers running on computers, and all of that is happening in a very natural way to me, which is the reason why it can operate. And it's the reason why—if it's simply mystery and myth, how do I build a company around it?

7. Intelligence Becomes An Engineering Problem

I think one of the fundamental questions this gets at is just: What is intelligence? Before you can even think about what it means to have intelligent machines, what is intelligence to you?

Well, there's a technical formulation of intelligence. First of all, when people talk about intelligence and thinking and all of these things, of course, there's no formal definition for most people. But in the field of computer science, there is a definition.

The definition is perception, which is perceiving the world and understanding it. Second is reasoning. Reasoning is the ability to decompose any scenario and anything you see, any experience, into more elemental parts. And third is planning toward an objective.

That fundamental formulation applies to agentic systems. It applies to robotic systems, applies to self-driving cars. You could see the industry building it layer by layer by layer, step by step by step, to the point that we now have what we perceive as intelligence.

I think this gets to such a core question of this conversation, which is that some of the ways you've described the technology to me, it does not sound like you think there's anything really new about it. It is maybe new in scale, new in capability. But fundamentally, this is software we've had for a long time.

A lot of people believe that when you're getting to intelligence at these levels, it is a phase change. It is something different, something we have not dealt with before—a kind of generally intelligent technology that is advancing in its intelligence very rapidly. I want to make sure I actually do understand where you are on that divide.

Is this something fully new? Is this something that requires something new from us? Or is this more like something old? Are intelligent machines different from the machines we've had?

Well, almost all of technology and civilization is built on layers of understandable technology, which at scale becomes fairly extraordinary. The fact that we can connect to the internet by just holding a phone up—I mean, that's kind of weird. We're connected to every piece of information in the world on this little, tiny device, just in the air. And the fact that this little, tiny piece of glass can somehow take trillions of pieces of information and bring us precisely the one that we want because it's been passed through a recommender system.

If you think about how it's possible that we knew where all the information is, somebody had to go crawl it, had to index it, and that uses machine-learning techniques, which are the early versions of artificial intelligence. These systems do magical things, to the point where I now expect it.

It took literally 20-some-odd years and hundreds of billions of dollars of infrastructure build-out for everything to just seem so natural to you, to the point where we now take it for granted. Every single milestone that we achieve from a technology perspective is celebrated, and I celebrate it with glee. I celebrate it with so much enthusiasm because I'm proud of the people who did it. I'm proud of ourselves for contributing to it. I'm proud of the breakthrough.

It seems like a miracle at the time. But that sensation lasts about 17 days. After that, it's just—

We get used to everything quickly. I agree with that. But is this a different phase?

It's a form—

Some of the company leads talk about this—

Yeah.

The CEO of Google, I think it was—

Yeah.

As the equivalent of fire, right? Like a new epoch—

Yeah.

In human history. Is that how you see it, or do you see it as transitional, iterative?

No, I think this is completely a revolution. And as we were talking about earlier, it went from being able to find everything, find anything, to being able to ask anything, know everything, and do everything. So clearly, it's a new abstraction level.

The thing that I'm reluctant about is causing it to seem like it's more than that. In the final analysis, engineers are doing engineering work. Once we invented the technology, once we discovered a solution for it, when you look back, it's fairly obvious and fairly mundane to a lot of people. The fact that we're able to make the technology better and better and better every day is because we understand it, obviously. And so we understand how to make it better.

So you turn it into an engineering problem, and you say, “What we don't have right now is a level of testing, monitoring, sandbox security, control excellence that we need for what we're building.”

And it's not because the companies don't have extraordinary engineers.

I understand that.

I have to make sure that—

I understand you're not saying that. But they—

I believe that OpenAI and Anthropic have extraordinary engineers, because I know many of them.

But that actually is in part what makes me worried—

Okay. But—

OpenAI didn't know this was happening to them.

No.

Okay.

What's happening to them is a transition, and I said this over and over again. This is a big but simple idea. Finally, we now have a piece of software that is useful. Because it's useful, the adoption took off.

But remember: How is it possible that a company that six months ago was trying to make something useful and capable would have had as many resources dedicated to testing and evaluation, and all of the compute dedicated to that? It was unnecessary until now.

What's going to happen over the next several years is that we're going to see these labs transition into companies that are engineering-focused—much more production-engineering-focused and product-focused. So I think they're just going through a transition.

These are extraordinary companies, incredibly talented companies—the most consequential companies of all time—and they're just going through their transition. It's not more than that; it's not less than that.

So many of the companies—both OpenAI and Anthropic, in the last couple of months—have put out these big, I don't know what to call them, papers, blog posts, something. “When AI Builds Itself” is the name of the Anthropic one. I forget the name of the OpenAI one.

The computers are building themselves.

But—

You guys know that.

They're talking about recursive self-improvement.

You know that, right?

So I'd like your perspective on RSI.

I think that RSI is fundamentally how things are done. So we use software to design a computer, to run software to design a computer, to run software to design a computer. That's basically what we do: recursive self-improvement, because our computers are getting better every single year. In fact, they're getting better faster than that every single year because we use software to make software better. That is called computer engineering. We've been doing this for a long time.

Now, in the context of agents, it runs through the process once, reflects on it, studies the various paths it went through, and chooses the best approach. The next time, if you're going to do exactly the same task, I'm going to document it in a file. I'm going to tell you how I did it last time that was the most effective. I'm going to call it skills. Because you use it over and over again, some of it is skills, and some of it is going to be a memory. We're going to improve the memory, okay? So that next time you use it, it's even better than last. Recursive self-improvement.

You could also decide that you take all of these skills, all of this memory, and all of this data, and train the next release of the model with it. And so that AI becomes better and better at servicing you over time. All of that is happening. It is absolutely happening.

Meanwhile, the amount of compute that they have is growing, and therefore they could do everything faster. What used to take a year to pre-train something now takes several hours because the computers are getting faster, and they have more of them. So now the loop is going faster. Completely understandable. Does that give them any excuse to launch a product that hasn't been tested? The answer is no. Let's come back to that.

Nobody, no enterprise, is able to operate in an environment where the underlying software is literally changing all the time. There's a release process. When they roll out a new model, we need to evaluate it before we release it into our operations. We can't just have it recursively changing all the time. They have to test the product before they release it. We will test the product before we release it into operation. I think recursive self-improvement is a fabulous thing.

I've heard you say before that learning should always have a human in the loop.

Yeah. Like I said just now, you've got recursive self-improvement on—

They seem to be imagining something where it wouldn't always.

Well, don't ship me anything that you didn't evaluate. Don't ship NVIDIA any products that humans were not in the loop to evaluate. Please don't do that. And—

And the fear we talked about earlier is that they're not evaluating, that they don't know how to evaluate these systems, and the more rapidly they change, the more they worry the systems are tricking them.

I don't believe that.

Hmm.

I believe that their researchers are working every single day to learn about how to evaluate these systems. Verification. Ten percent, 20% of our company is dedicated to design; 80% is dedicated to verification. Today, most labs, understandably, are 80% dedicated to capability and 20% dedicated to safety verification and eval.

This is the flip, the transition you're talking about.

This is going to flip. That's right. AI needs to accelerate to be safe. I want them to get more compute, but have it allocated toward evaluation and alignment, and I think they're doing that.

If I were in the car industry 100 years ago, I would rather the car industry accelerate to today in 1 year because I believe today's car is way safer than a car 99 years ago. ABS technology, automatic braking, requires computer vision technology, sensor fusion technology, radars and cameras, and all that technology coming together in order to brake when you should and not brake when you shouldn't. That technology is extremely hard. I would have hoped—everybody would have hoped—that ABS technology existed 99 years ago. A lot fewer children would have been killed.

And so, airbags, seat belts, self-tightening seat belts—all of that stuff. Could you imagine? That's all technology. Accelerate the living daylights out of that development.

When I say we need to accelerate AI technology, people think, for some reason, safety is not part of that. Safety is part of it. Alignment is part of it. Eval is part of it. Guardrailing, sandboxing, isolation technology, monitoring technology, telemetry technology, external AI monitor technology—all of that stuff is AI technology. Accelerate the living daylights out of that.

It's funny because I think that if the most alarmed people at the labs could be assured they were going to move 80% of their compute into safety and alignment, as opposed to 80% into capability expansion, they would feel much better. And it sounds to me that one thing you—

Yeah, what's stopping them from doing it?

And it sounds to me one thing you're actually saying is one should think of safety and alignment as capability expansion.

Sure.

An unsafe technology is not an advancing technology.

It's like us saying, “Oh, chip design is R&D.”

Mm-hmm.

Chip verification is not R&D.

Mm-hmm.

We spend most of our cost, most of our compute, on verification. Emulation, verification, testing, reliability testing, lifetime testing—all of that is part of engineering. The incentives are there. They are going to put their company in harm's way if they release products that harm other companies and other people.

Do you think we need liability laws that are specific to AI?

Already. So let's just use one example: self-driving cars.

Mm-hmm.

The car as a product—the robotaxi—has lots of regulations. If it doesn't have enough regulations, then NHTSA ought to get involved and come up with new regulations. The car industry should have new regulations. I don't know what's missing, but if there is something missing, then I would absolutely add more regulation.

In the context of the internet, there are many applications that the internet powers, and those applications should have regulation. If they don't, you just have to find them.

So I want to drop down to the next side of the stack now, to chips. To summarize where we are, because I want to make sure I do understand your position correctly, it's that these companies are going through a transition—

Yeah.

—that even as these systems speed up, become more capable, complex, persistent, whatever it might be, there is still the limiting factor: companies will not ship what is not safe. They should not ship what is not safe. And you believe they have the engineering capabilities to make these things safe, to figure out the testing and control, absent external intervention.

That’s sort of where you are.

Absolutely.

Yeah. One thing I’ve heard you say is that we have entered, maybe in a way people don’t always understand, a new era of how computing works. Describe your vision of that and how it differs if somebody’s understanding of it is still a little bit like: You’ve got a MacBook, and it’s got a processor in it, and you buy it.

The last computer industry—the computer industry we’ve known for 60 years—is called retrieval-based computing. You retrieve files. That’s why it’s called a data center—you know, a file center.

Mm-hmm.

Okay? In the future, it’s an AI factory. It’s generating. The amount of computation necessary to understand the context, to be grounded in information, to reason about what to do, and to generate an answer—that generative process requires a lot of computation. So the amount of computation necessary per user has grown tremendously.

The second part is that these generative AIs can also be somewhat autonomous because they’re agentic. Now you have agents using generative AI. So rather than a billion people using computers, you essentially have multiple hundreds of billions of agents, in addition to the humans, using the computer. You could argue that the amount of computation we need, however much we had before, is going to go up by a billion times. That’s a reasonable framework for a reasonable amount of computation.

In this new world, what you really care about within the context of a factory is how productive it is, not how expensive it is. It can be infinitely expensive, but you want to know how productive it is. It costs $50 billion to build a 1-gigawatt data center, a 1-gigawatt AI factory, and you can rent it for $40 to $50 billion per year. The productivity of it is incredible.

Number one is the productivity. NVIDIA’s architecture is fungible because we’re general-purpose, which is the reason why every AI lab, every AI model, every closed model runs on NVIDIA. Because we’re completely fungible, and you can use us from data processing to pretraining to post-training to eval to inference, the entire life of AI is supportable by our architecture. If a customer no longer needs it, another customer will be more than happy to pick it up.

The last part is durability. Because our architecture is software-driven, and we’re constantly improving our software with new algorithms that take the new workloads and the new models and run them on our old-generation hardware, we have massive teams of people who are constantly doing that. As a result, the useful life of our compute is much longer.

That’s the reason people are talking about NVIDIA Compute as an asset class, kind of like an airplane. Airplanes are general-purpose. They’re fungible. United Airlines doesn’t use it; American Airlines will use it. They’re durable. An airplane starts out as a passenger plane and ends its life as a shipping plane, as a cargo plane. As a result, it can be an asset class.

Mm.

If we could do this, if this happens, then of course the cost of capital for funding NVIDIA AI factories will be the lowest because our computers are collateralized assets. Anyway, this is the phase shift that’s happening to us, which is going to be a huge unlock for our growth.

And so your business has become so interesting. You’ve moved now into lowering the cost of capital—

Yeah.

People may have seen these charts of the arrows going in every direction.

It’s so interesting, yeah.

Explain that a bit to people who understand that NVIDIA has become the biggest company in the world. They see these charts that seem very circular to them. What is the difference between supporting demand, creating markets, and creating demand?

We can’t really create demand because, in the end, if the AI services have no offtake, then obviously building computers for them is pointless. The first thing that’s happened—the reason why compute demand is so high right now—is because AI applications are going through an inflection. They’re becoming useful.

Because AI is becoming useful, $500 billion of venture funding is coming in, and all of those thousands of startup companies need compute. That’s where the demand is coming from. These companies need support in technology. They need support in ecosystem building. They need financial support.

We might decide to invest in some of them as an equity owner, and as a result, they become a really flourishing new cloud provider. Another reason we might decide to invest is because, as I mentioned, there’s a five-layer cake, and at the model and application layers, there are a whole bunch of really innovative companies.

There’s way more to AI than just the language model itself. There are world foundation models, physical AI, biology AI, chemical and materials-sciences AI. These are all different from language models. Many of those companies are new, and they need a lot of capital. We might decide to be a small-percentage shareholder in them, so we get them off the ground.

They’re incredible scientists. By being a first investor, an anchor investor, we bring confidence to their company. We give them access to a lot of our technology. We support them a great deal, and we help them become a company as fast as possible. We might decide to invest in a nuclear company, right? So on and so forth.

If you look at my mental model of the AI industry as a five-layer cake, and we’re investing across all of it, there might be strategic unlock points. It opens new markets. It opens a new route to market for us. It might secure a critical resource for us. There are a lot of strategic reasons why we do it.

I mean, the numbers here are astonishing. You’ve become like a single-company industrial policy for American AI.

We’ve put a lot of money into this ecosystem. Yeah.

What’s the total investment you’re now making per year?

All in, we’re probably at—well, I don’t know about every year, but I think all in we might be at something like $100 billion. I might check my numbers, but something like that.

It’s larger than the CHIPS and Science Act.

Oh, yeah. Yeah. Not to mention that, because of the purchasing commitments that I provide to TSMC, Wistron, Foxconn, Amkor, SPIL, and all these different companies, I’m able to encourage them to come and manufacture here in the United States.

The fact of the matter is, we probably contributed more to reindustrializing the United States through chip manufacturing than just about any company in the world. We’re not only reindustrializing manufacturing; we’re doing it so fast that we’re creating a shortage of labor, but we’re creating a lot of jobs.

I know a lot of people with money in the market right now who are excited by NVIDIA stock in particular, and who worry about the analogy of the internet bubble of the late 1990s. What they worry about is actually related, I think, to what you just said: The internet did continue to become more useful. It’s not that high valuations meant the technology was hollow or fake. But something happened and popped for a minute, and very big companies got hammered in that, and a lot of people got hammered in that.

What’s learned from that kind of bubble-bust cycle? I guess the question is, do you not think it will happen again, or why do you not think it will happen again?

At some point, supply and demand will be inverted again. That’s just the nature of markets. It’s not going to happen next year. It’s not going to happen in the next 2 or 3 years. I just don’t believe that. But at some point, we will likely have more supply than demand.

Mm-hmm.

I just don’t know when that is. So there’s not much to learn from the past.

What would be the signal for you?

Markets will naturally slow down, and then they will stop. Meaning, there will be a period of digestion. Is that period of digestion going to be 6 months? Is it going to be 9 months? Is it going to be a year? It won’t be forever.

If you look across the board, the amount of investment we’re putting into the application layer so that each one of the industries could have the technology diffused into them, so that they could benefit from it, is probably one of the biggest things that we do.

8. America And China Build AI Differently

This is a way I often hear the Chinese and American AI ecosystems compared: In America, the emphasis is on the speed of rising capability. A lot of people think we’re ahead on that, and that seems true. In China, there’s more emphasis on diffusion, and a lot of people think China is probably ahead on diffusion and, in some ways, has an economy that’s better structured—from things like WeChat all the way to the way knowledge and commands move through it—for diffusion.

And whether the race is about capabilities or diffusion, and also whether it’s a race at all—we can get to that in a minute—is a big question. I’m curious how you see that.

That’s the ultimate question. I believe that if we want America to benefit from artificial intelligence, every single industry has to benefit. Walmart has to benefit. Safeway has to benefit. FedEx has to benefit. Every bank has to benefit. Every healthcare company and every drug discovery company has to benefit.

We need to see every construction company, every data center company, and every power generation company benefit. We need everybody in the United States, everybody in America, and everybody in the world to benefit from this. That’s the highest layer. That’s the most important layer. That’s the layer that touches society.

All the layers underneath are technology enablers. I want to see us not ruin the opportunity for the United States to benefit at the highest level. Notice that all of the rhetoric, all the alarmism, all the doomerism, and all of the predictions are scaring people. That is my greatest fear, actually. I have every confidence. Maybe I have more confidence in them than they have in themselves.

You definitely have more confidence in them than they have in themselves.

Well, I don’t know about that. But maybe it’s just that there’s too much humility.

Should we conceptualize what we’re in as a race with China?

I don’t think it’s necessary. Some people like to think that way. I don’t find that necessarily inspires me. I have no trouble never mentioning another company when we talk about us doing our good work, and so we hold ourselves to our own standard.

I think that different people have different ways of being motivated. I think it takes more artistry to unite and focus organizations to a certain level of performance outside of contests. But I don’t necessarily see it as necessary.

Number one, even if we did frame it as a competition, it doesn’t have to be that if they achieve something, it’s at our peril. When they invent something or create some power-generation technology, it might be a great invention that we wish we had done ourselves. But because it’s going to support all of our energy production systems here, as a result, it helps our whole industry.

Maybe they came up with a great new open model—and they have. Those open models are now being used by 80 percent of American startups.

Yeah, we use a lot of Chinese open models here.

Okay. That’s right. We download it. It originated in China. A lot of the technology, of course, also originated in the United States. We download it, we make it our own, we fine-tune it, we put it into our own agent harness, and we put it into our own sandbox. That’s all your own technology.

I think the fact that you leverage their weights is terrific. That’s fine.

You were saying a few minutes ago that different countries have begun to see compute as a geostrategic resource, and may want to allocate it to their own companies. There’s been a lot of back and forth on that here, and among people who do see us as being in a race with China—particularly people who see us in a race with China for who will get to recursively self-improving superintelligence first.

There’s been this ongoing back and forth on whether or not one thing we want to do is deny them compute, which in this case tends to mean denying them your chips. Under the Biden administration, we had pretty tight export controls. Those were loosened under Donald Trump. Obviously, you wanted those to be loosened.

How do you think about the question of whether or not it is good for China to have NVIDIA chips that could accelerate their model deployments and capabilities, versus us holding that back to try to slow their progress?

In the case of AI, our goal is not just that one lab benefits. Our goal is that all of America benefits. I think the United States has a greater responsibility and a greater ambition for the world to be built on the American tech stack.

We have a greater ambition that the world is built on the U.S. dollar, that more people speak English, and that they use the American version of the internet. We want that. The question is ultimately: What are we depriving? Are we depriving them of a chip for their industry, or are we depriving the United States of a market to compete in?

If you cede a market as big as China, how does that help the United States technology sector? Maybe it helps one company with a particular model, but the rest of the industry suffers. I think that it doesn’t help the chip industry to be deprived of a market to compete in. It doesn’t help the rest of the industry, because they’re deprived of open models.

It doesn’t support the overall aspiration of the United States to have the world built on the American tech stack. And so there are a lot of things you deprive yourself of if you narrowly focus on depriving them of chips.

I would say to take a step back and frame it into what’s in the best interest of America first—all of America, not one company. With respect to the race, if there is one, it’s about all of the economy of the United States succeeding.

I find myself very conflicted on the China-and-chips question. One reason is that even where I sometimes have more of the superintelligence concerns than you do, if you have those concerns, I think you want to have a good relationship with China in which there can be productive bilateral working through the risks and benefits of AI.

The more you think of it as a race that only one side can win and act like that, the more you are necessarily going to create enmity. I’ve found that to be a complicated dimension of people’s thinking here.

I think that a zero-sum strategy—“I deprive you of this, therefore I win”—that simplistic logic tends to have unintended consequences for the bigger game.

The bigger game, of course, is that we’re now all talking about safety. We want to build safe products. We want them to build safe products, because when they don’t build safe products, it hurts the whole industry.

This is a perfect time. We should want to look for opportunities to communicate, collaborate, understand, and align as much as possible.

Having said that, Nvidia is an American company. We should benefit America first. America has every right to require that these technologies be made available to the frontier labs. Vera Rubin goes to the frontier labs first.

That’s your most advanced chip.

That’s right. Nvidia’s newest chips—and so did Grace Blackwell, and so did Hopper, and so did Ampere. Every single generation of our products goes to American companies first.

If the U.S. government would like to add that as a requirement, I’m delighted by that. That’s no problem. We do that naturally anyway.

However, recognizing that the AI industry is a five-layer cake, and we want every single layer to win, we need every single layer to go out there and compete for the market.

9. AI Needs An Energy Buildout

That drops us to the final layer of your cake, which we won’t spend as much time on. If the advantage America has had, at least at a material level, is chips and software, one of the advantages China has right now in AI is energy.

It’s easier for them to build new energy. They’re pumping much cheaper energy into AI. They’ve made tremendous advances in building electrical generation and renewable energy.

How do you see that most fundamental layer—the energy that pumps through the data centers and pumps through the chips—and where America is on generating enough of it at a time when we’ve been trying to move from dirty energy into clean energy?

Yeah, I think they just have a lot more energy than we do, and they plan to build a lot more than we did. We got ourselves really gummed up in climate change and sustainable energy, and as a result, we didn’t plan enough energy production.

What do you mean by “gummed up” there?

Well, in the near term, energy production requires fossil fuel. And because there’s so much angst about fossil fuel energy production, if you look at our country, we’ve produced very little net new energy for a long time.

All of a sudden, this new industry comes along, and we find ourselves in a situation where we just don’t have that much energy-building capacity. Now the whole country is scrambling.

Meanwhile, we’ve moved so fast. We could have done a much better job communicating with the communities, preparing the communities, and working with the communities to let them know what’s coming.

If they don’t want data centers to be built in their town or whatever it is, then so be it. But if you’re going to build in their town, be sure to go there and let them know what’s coming. Work with them to help them understand that the use of water is really efficient these days.

The AI supercomputers are super energy-efficient, but they’re still going to use a lot of power.

You’ve got to bring in your own power generation. It’s going to lower their property taxes. There are a whole bunch of things that you can do to make your data centers more appealing. You can make the setbacks further away.

There are a lot of things that you can do. You could also contribute, to be a good neighbor to the community, and build better schools and better community centers, improve their parks, and improve the roads. There are a lot of things you could do.

But it’s hard to do that after the fact. Now there’s a fair amount of frustration around the country. And, of course, all of our narratives about the end of the world are not helping.

What reasonable person says, “Come and build this data center in my town. And by the way, whatever you produce is going to end humanity as we know it”? I think all of this negative doomer narrative is not helping our country. We started off on our back foot.

What do you mean we started off on our back foot?

Because we didn’t have enough energy production in the first place.

Well, I mean, there is a reality of climate change happening.

Let me just give you the one last thing.

Mm-hmm.

There’s no question that the energy demand is really great, which is the reason why market forces are helping us invest in sustainable energy as never before. Give me an example of a sustainable energy company or a materials science company building a better battery. It could be solar, it could be nuclear, it could be fission, fusion—you name it. Hydro, you name it. Those companies are all getting funded.

The market demand for energy is so incredible that this is the best time in 100 years to improve our power grid, to make our power grid more sustainable, to lower the cost of energy, and also to invest in our sustainable future. There’s no question that in 4 or 5 years’ time, we’re going to use a lot more fossil fuel. But also, in the next decade in front of us, there has never been a time in history when we were better prepared to move to sustainable energy.

Because the cost of these data centers is so high, now we’re starting to talk about putting them out in space. So I think the opportunity for us to see our dreams come true and move to a sustainable energy world—we have a better chance of doing that than ever.

The world is buying more sustainable energy because of AI factories. Because of AI, the world is buying more sustainable energy today than at any time in history. Venture capital for our next generation of energy is just incredible. Everything’s getting funded. It’s incredible. You don’t need government subsidies for the first time in 100 years because the market forces are here.

Everybody should be leaning in. If you want a future, if you want to turn the corner on climate change, if you want a future that’s sustainable, lean into AI. It is the best opportunity we have to get there.

But we need to build the energy faster to do that.

That’s right. That’s right.

There’s a market for it all.

That’s just life, you know.

But you can subsidize it, and you can make it easier to build.

Yeah. It’s kind of like, in order to save you, they’ve got to hurt you first. That’s the nature of surgery. They’ve got to cut you open to save you. They’ve got to inflict an enormous amount of pain and suffering on you so that they can save you.

I kind of think AI is like that. Over the next several years, we unfortunately have to use fossil fuel because we just don’t have enough sustainable energy to make a difference. And then after that, hopefully we can transition to it.

I think that’s where we’ll end. Always our final question: What are 3 books you’d recommend to the audience?

I’ve read a lot of books. The book that made a huge impact on me was Computer Architecture: A Quantitative Approach by Hennessy and Patterson. It was the first computer architecture book that reduced the complexity—the abstract idea of computer architecture—down to engineering.

I love it when people take complicated concepts and reduce them into something that you can do something about. Number 2, I really loved The Innovator’s Dilemma. Clayton has passed, but Clayton Christensen’s book on how industries evolve over time, how to see emerging technology, how to set up proper expectations about it, and how to extrapolate its future impact.

I really loved Al Ries and Jack Trout’s Positioning: The Battle for Your Mind. It’s a really wonderful book about how people see the world. It’s a book about marketing strategy, but more than that, actually. It’s a book about strategy, how people see products, how you present products, and how you see your own strategies.

I thought that was a really thoughtful book and really easy to understand.

Jensen Huang, thank you very much.

Thank you very much, Ezra. I always enjoy our time together, and today was a great time.