Google 前CEO：人工超级智能究竟会是什么样——Eric Schmidt 与 Dave B 对谈

When do you see what you define as digital superintelligence?

Within 10 years. The AI’s ability to generate its own scaffolding is imminent. I’m pretty sure that will be a 2025 thing. We certainly don’t know what superintelligence will deliver, but we know it’s coming.

And what do people need to know about that?

You’re going to have your own polymath—the sum of Einstein and Leonardo da Vinci in your pocket. Agents are going to happen. This math thing is going to happen. The software thing is going to happen. Everything I’ve talked about is in the positive domain, but there’s a negative domain as well. It’s likely, in my opinion, that you’re going to see…

I’m here live with my Moonshot mate, Dave Blundin. We’re here in our Santa Monica studios, and we have a special guest today, Eric Schmidt, the author of Genesis. We talk about China, digital superintelligence, and what people should be thinking about over the next 10 years.

We’re talking about the guy who has access to more actionable information than probably anyone else you could think of. It should be pretty exciting.

Eric, welcome back to Moonshots.

It’s great to be here with you guys.

Thank you. It’s been a long road since I first met you at Google. I remember our first conversations were fantastic. It’s been a crazy month in the world of AI, but I think every month from here on is going to be a crazy month. I’d love to hit on a number of subjects and get your take on them.

I want to start with probably the most important point that you’ve made recently, which got a lot of traction and attention: AI is underhyped, while the rest of the world is either confused, lost, or thinks it’s not impacting us. We’ll get into more detail, but what’s the most important point to make there?

AI is a learning machine.

In network-effect businesses, when the learning machine learns faster, everything accelerates.

It accelerates to its natural limit. The natural limit is electricity.

Not chips.

Electricity, really.

Okay. So that gets me to the next point here, which is a discussion on AI and energy. We saw Meta recently announce that it had signed a 20-year nuclear contract with Constellation Energy. We’ve seen Google, Microsoft, Amazon—everybody—buying essentially nuclear capacity right now. That’s got to be weird, that private companies are basically taking into their own hands what was previously a utility function.

Well, just to be cynical, I’m so glad those companies plan to be around for the 20 years it’s going to take to get the nuclear power plants built.

In my recent testimony, I talked about the current expected need for the AI revolution in the United States: 92 gigawatts of additional power. For reference, 1 gigawatt is 1 big nuclear power station, and there are essentially none being started now. There have been 2 built in the last 30 years.

There’s excitement that there’s an SMR—a small modular reactor—coming in at 300 megawatts, but it won’t start until 2030. As important as nuclear power, both fission and fusion, is, those technologies aren’t going to arrive in time to get us what we need as a globe to deal with our many problems and the many opportunities that are before us.

If you look at the roughly 3-year timeline toward AGI, do you think that if you started a fusion reactor project today, which wouldn’t come online for 5, 6, or 7 years, there’s a probability that AGI comes up with some other breakthrough—fusion or otherwise—that makes it irrelevant before it even gets online?

A very good question. We don’t know what artificial general intelligence will deliver. We certainly don’t know what superintelligence will deliver, but we know it’s coming. So first, we need to plan for it. There are lots of issues as well as opportunities.

The fact of the matter is that the computing needs we identify now are going to come from traditional energy suppliers in places like the United States, the Arab world, Canada, and the Western world. It’s important to note that China has lots of electricity. So if they get the chips, it’s going to be one heck of a race.

Yeah. They’ve been scaling it at 2 or 3 times the rate. The US has been flat for how long in terms of energy production?

From my perspective, forever. In fact, electricity demand declined for a while, as have overall energy needs, because of conservation and other things.

But the data center story is the story of the energy people, right? You sit there and go, “How could these data centers use so much power?” Especially when you think about how little power our brains use. These are our best approximations, in digital form, of how our brains work. But when they start working together, they become superbrains.

The promise of a superbrain with a 1-gigawatt data center, for example, is so palpable. People are going crazy. By the way, the economics of these things are unproven. How much revenue do you have to have to have $50 billion in capital? If you depreciate it over 3 or 4 years, you need to have $10 billion or $15 billion of capital spending per year just to handle the infrastructure. Those are huge businesses and huge revenues, which in most places aren’t there yet.

I’m curious. There’s so much capital being invested and deployed right now in SMRs, in nuclear power, in bringing Three Mile Island back online, and in fusion companies. Why isn’t there an equal amount of capital going into making the entire chipset and compute 1,000 times more energy-efficient?

There is a similar amount of capital going in. There are many, many startups working on nontraditional ways of making chips. The transformer architecture, which is what’s powering things today, has new variants. Every week or so, I get a pitch from a new startup that’s going to build inference-time, test-time computing, which is simpler and optimized for inference. It looks like the hardware will arrive just as the software needs expand.

And by the way, that’s always been true. We old-timers had a phrase: “Grove giveth and Gates taketh away.” Intel would improve the chipsets way back when, and the software people would immediately use it all and suck it all up. I have no reason to believe that the law of “Grove giveth and Gates taketh away” has changed.

If you look at the gains in the Blackwell chip or the MI350 chip from AMD, these chips are massive supercomputers, and yet, according to the people, we need hundreds of thousands of these chips just to make a data center work. That shows you the scale of computation these kinds of thinking algorithms require.

Now you sit there and go, “What could these people possibly be doing with all these chips?” I’ll give you an example. We went from language prediction, which is what ChatGPT can be understood as, to reasoning and thinking. If you want to look at an OpenAI example, look at OpenAI o3, which does forward and backward reinforcement learning and planning.

The cost of doing the forward and backward passes is many orders of magnitude beyond just answering your question for your PhD thesis or your college paper. That planning, the back-and-forth, is computationally very, very expensive.

With the best energy and the best technology today, we’re able to show evidence of planning. Many people believe that if you combine planning and very deep memories, you can build human-level intelligence. Of course, they will be very expensive to start with, but humans are very industrious. Furthermore, the great future companies will have AI scientists—that is, nonhuman scientists—and AI programmers who, as opposed to human programmers, will accelerate their impact.

So, if you think about it, going back to the fact that you’re the author of the abundance thesis, Peter, you’ve talked about this for 20 years. You saw it first. It sure looks like, if we get enough electricity, we can generate the power—in the sense of intellectual power—to generate abundance along the lines that you predicted 2 decades ago.

Let me throw some numbers at you. We have a couple of companies in the lab that are doing voice customer service and voice sales, just as of the last month.

Sure.

The value of these conversations is $10 to $1,000. The cost of the compute is maybe 2 or 3 concurrent GPUs, which is optimal. It's like 10 to 20 cents, so they would buy massively more compute to improve the quality of the conversation. There aren't even close to enough GPUs. We count about 10 million concurrent phone calls that should move to AI in the next year or so.

My view of that is that it's a good tactical solution and a great business.

Let's look at other examples of tactical solutions that are great businesses.

I obviously have a conflict of interest talking about Google because I love it so much. With that in mind, look at Google's strength in GCP. Now, Google's cloud product has a completely full-service enterprise offering for essentially automating your company with AI.

Yeah.

The remarkable thing—and this is shocking to me—is that, in an enterprise, you can write the task that you want. Then, using something called the Model Context Protocol, you can connect your databases to it, and the large language model can produce the code for your enterprise. There are 100,000 enterprise software and middleware companies that grew up in the last 30 years that I've been working on this. They're all now in trouble because that interstitial connection is no longer needed with their business.

Yeah.

Of course, they'll have to change as well. The good news for them is that enterprises make these changes very slowly. If you built a brand-new enterprise architecture for ERP and MRP, you would be highly tempted not to use any of the ERP or MRP suppliers. Instead, you would use open-source libraries, essentially use BigQuery or the equivalent from Amazon, which is Redshift, and build that architecture. It gives you infinite flexibility, and the computer system writes most of the code.

Programmers don't go away at the moment. It's pretty clear that junior programmers go away—the sort of journeymen, if you will, of the stereotype—because these systems aren't good enough yet to automatically write all the code. They need very senior computer scientists and computer engineers who are watching them. That will eventually go away.

One of the things to say about productivity—and I call this the San Francisco consensus because it's largely the view of people who operate in San Francisco—goes something like this: We're just about to the point where we can do 2 things that are shocking. The first is that we can replace most programming tasks with computers, and we can replace most mathematical tasks with computers.

If you think about programming and math, they have limited language sets compared to human language, so they're simpler computationally and they're scale-free. You can just do it and do it and do it with more electricity. You don't need data. You don't need real-world input. You don't need telemetry. You don't need sensors.

Yeah.

So, in my opinion, you're likely to see world-class mathematicians emerge within the next year that are AI-based, and world-class programmers are going to appear within the next 1 or 2 years. When those things are deployed at scale, remember that math and programming are the basis of kind of everything. They're an accelerant for physics, chemistry, biology, and materials science.

Going back to things like climate change, can you imagine if—and this goes back to your original argument, Peter—we can accelerate the discovery of new materials that allow us to deal with a carbonized world?

Yeah. Right. It's very exciting. I'd love to drill in about that first.

I just want to hit this because it's important: the potential for there to be—I don't want to use the word “PhD-level,” other than thinking in terms of research—PhD-level AIs that can basically attack any problem and solve it, and solve math, if you would, in physics. This idea of an AI intelligence explosion—Leopold Aschenbrenner put that at, like, 2026 or 2027, heading toward digital superintelligence in the next few years. Do you buy that time frame?

Again, I consider that to be the San Francisco consensus. I think the dates are probably off by 1.5 or 2 times, which is pretty close. A reasonable prediction is that we're going to have specialized savants in every field within 5 years. That's pretty much in the bag as far as I'm concerned.

Sure.

Here's why: You have this amount of humans, and then you add 1 million AI scientists to do something; your slope goes like this. Your rate of improvement—we should get there. The real question is, once you have all these savants, do they unify? Do they ultimately become a superhuman? The term we're using is “superintelligence,” which implies intelligence that's beyond the sum of what humans can do.

The race to superintelligence is incredibly important because imagine what a superintelligence could do that we ourselves cannot imagine. It's so much smarter than we are, and it has huge proliferation issues, competitive issues, China-versus-the-U.S. issues, electricity issues, and so forth. We don't even have the language for the deterrence aspects and the proliferation issues of these powerful models.

Or the imagination.

Totally agree. In fact, it's one of the great flaws, actually, in the original conception. You remember Singularity University and Ray Kurzweil's books and everything. We kind of drew this curve of rat-level intelligence, then cat, then monkey, and then it hits human, and then it goes superintelligent.

But it's now really obvious, when you talk to one of these multimodal models that's explaining physics to you, that it's already hugely superintelligent within its savant category. Demis Hassabis keeps redefining AGI as, well, when it can discover relativity the same way Einstein did with the data that was available up until that date. That's when we have AGI.

So, long before that.

Yeah.

I think it's worth getting the timeline right.

Yeah.

The following things are baked in. You're going to have an agentic revolution where agents are connected to solve business processes, government processes, and so forth. They will be adopted most quickly in companies and countries that have a lot of money and a lot of time-latency issues at stake. They will be adopted most slowly in places like government, which do not have an incentive for innovation and fundamentally are job programs and redistribution-of-income programs.

Call it what you will. The important thing is that there will be a tip of the spear in places like financial services, certain kinds of biomedical things, startups, and so forth. That's the place to watch.

All of that is going to happen. The agents are going to happen. This math thing is going to happen. The software thing is going to happen. We can debate the rate at which the biological revolution will occur, but everyone agrees that it's right after that. We're very close to these major biological understandings.

In physics, you're limited by data, but you can generate it synthetically. There are groups that I'm funding that are generating physics models that can approximate algorithms that cannot be computed—they're incomputable. In other words, you have a foundation model that can answer the question well enough for the purposes of doing physics, without having to spend 1 million years doing the computation of quantum chromodynamics and things like that.

The next questions have to do with the point at which this becomes a national emergency, and it goes something like this: Everything I've talked about is in the positive domain, but there's a negative domain as well. The ability for biological attacks and, obviously, cyberattacks. Imagine a cyberattack that we as humans cannot conceive of, which means there's no defense for it because no one ever thought about it. These are real issues.

A biological attack—you take a virus, and I won't obviously go into the details. You take a virus that's bad and make it undetectable through some changes in its structure, which again I won't go into the details. We released a whole report at the national level on this issue.

At some point, the government—and it doesn't appear to understand this now—is going to have to say, “This is very big,” because it affects national security, national economic strength, and so forth. China clearly understands this, and China is putting an enormous amount of money into it. We have slowed them down by virtue of our chip controls, but they've found clever ways around this. There are also proliferation issues. Many of the chips that they're not supposed to have, they seem to be able to get.

More importantly, as I mentioned, the algorithms are changing. Instead of having these expensive foundation models by themselves, you have continuous updating, which is called test-time training. That continuous updating appears to be capable of being done with less powerful chips.

We don't know the role of open source because, remember, open source means open weights, which means everyone can use it. A fair reading of this is that every country that's not in the West will end up using open source because they'll perceive it as cheaper, which transfers leadership in open source from America to China. That's a big deal if that occurs.

How much longer do the chip bans, if you will, hold, and how long before China can answer? What are the effects of the current government's policies of getting rid of foreigners and foreign investment? What happens with the UAE data centers, assuming they work? I'm generally supportive of them, but what happens if those things are then misused to help train models?

The list just goes on and on. We just don't know.

Okay. Can I ask you probably one of the toughest questions? I don't know if you saw Marc Andreessen. He went and talked to the Biden administration—the past administration—and said, “How are we going to deal with exactly what you just talked about: chemical, biological, radiological, and nuclear risks from big foundation models being operated by foreign countries?”

The Biden administration's answer was, “We're going to keep it to the 3 or 4 big companies, like Google, and we'll just regulate them.” Marc was like, “That is a surefire way to lose the race with China, because all innovation comes from a startup that you didn't anticipate. It's just American history, and you're cutting off the entrepreneur from participating in this.”

So, as of right now, with the open-source models, entrepreneurs are in great shape. But if you think about the models getting crazy smart a year from now, how are we going to have the balance between startups actually being able to work with the best technology and proliferation not percolating to every country in the world?

Again, these are a set of unknown questions, and anybody who knows the answer to these things is not telling the full truth. The doctrine in the Biden administration was called 10²⁶ FLOPs. There was a consensus point above which the models were powerful enough to cause some damage.

The theory was that if you stayed below 10²⁶, you didn't need to be regulated. But if you were above that, you needed to be regulated, and the proposal in the Biden administration was to regulate both the open-source and closed-source models.

Okay, that's the summary.

That, of course, has been ended by the Trump administration. They have not yet produced their own thinking in this area. They're very concerned about China getting ahead, so they'll come out with something.

From my perspective, the core questions are the following: Will the Chinese be able to use, even with chip restrictions, architectural changes that will allow them to build models as powerful as ours? And let's assume they're government-funded. That's the first question.

The next fun question is, how will you raise $50 billion for your data center if your product is open source?

Yeah.

In the American model, part of the reason these models are closed is that the businesspeople and the lawyers are correctly saying, “I've got to sell this thing because I've got to pay for my capital.” These are not free goods, and the US government correctly is not giving $50 billion to these companies. So we don't know that.

To me, the key question to watch is DeepSeek. A week or so ago, Gemini 2.5 Pro got to the top of the leaderboards in intelligence. Great achievement for my friends at Gemini. A week later, DeepSeek comes in and is slightly better than Gemini.

DeepSeek, of course, is trained on the existing hardware that's in China, which includes stuff that's been pilfered and some of Huawei's Ascend chips, and a few others. What happens now? People in the US say, “Well, the DeepSeek people cheated.” They cheated by doing a technique called distillation, where you take a large model, ask it 10,000 questions, get its answers, and then use that as your training material.

Yep.

So the US companies will have to figure out a way to make sure that their proprietary information, which they've spent so much money on, does not get leaked into these open-source things.

I just don't know with respect to nuclear, biological, chemical, and so forth issues. The US companies are doing a really good job of looking for that. There's a great concern, for example, that nuclear information would leak into these models as they're training without us knowing it. And by the way, that's a violation of law.

Oh, really?

The whole nuclear information thing is—there's no free speech in that world, for good reasons, and there's no fair use in copyright and all that kind of stuff. It's illegal to do it, and so they're doing a really, really good job of making sure that that does not happen.

They also put in very significant tests for biological information and certain kinds of cyberattacks. What happens there? Their incentive is to continue, especially if it's not required by law. The government has just gotten rid of the safety institutes that were in place under Biden and is replacing it with a new term, which is largely a safety assessment program. That's a fine answer.

I think collectively, we in the industry just want the government, at the secret and top-secret level, to have people who are really studying what China and others are doing. You can be sure that China really has very smart people studying what we're doing. We, at the secret and top-secret level, should have the same thing.

Have you read the AI 2027 paper?

I have. For those listening who haven't read it, it's a future vision of the US and China racing toward AI. At some point, the story splits into: we're going to slow down and work on alignment, or we're going full out. Spoiler alert: in the race to infinity, humanity vanishes.

The right outcome will ultimately be some form of deterrence and mutually assured destruction. I wrote a paper with 2 other authors, Dan Hendrycks and Alex Wang, where we named it Mutual Assured AI Malfunction.

The idea goes something like this: You're the United States, I'm China, and you're ahead of me. At some point, you cross a line—you, Peter, cross a line—and I, China, go, “This is unacceptable.” At some point, it becomes something you're doing that affects my sovereignty.

It's not just words and yelling and an occasional shooting down of a jet. It's a real threat to the identity of my country, my economy—what have you. Under this scenario, I would be highly tempted to do a cyberattack to slow you down.

In Mutual AI Malfunction, if you will, we have to engineer it so that you have the ability to do the same thing to me.

And that causes both of us to be careful not to trigger the other.

That's what mutual assured destruction is. That's our best formulation right now. We also recommend in our work—and I think it's very strong—that the government require that we know where all the chips are. And remember, the chips can tell you where they are because they're computers.

Yeah.

It would be easy to add a little crypto thing which would say, “Yeah, here I am, and this is what I'm doing.”

So knowing where the chips are, knowing where the training runs are, and knowing what these fault lines are is very important. Now, there are a whole bunch of assumptions in this scenario that I described.

The first is that there is enough electricity. The second is that there is enough power. The third is that the Chinese have enough electricity, which they do, and enough computing resources, which they may or may not have—or may have in the future.

I'm also asserting that everyone arrives at this eventual state of superintelligence at roughly the same time. Again, these are debatable points, but the most interesting scenario is that we're saying it's 1938. The letter has come from Einstein to the president, and we're having a conversation and saying, “Well, how does this end?”

Okay. So, if you were so brilliant in 1938, what you would have said is, “This ultimately ends with us having a bomb, the other guys having a bomb, and then we're going to have one heck of a negotiation to try to make sure that we don't end up destroying each other.”

I think the same conversation needs to get started now, well before the Chernobyl events, well before the buildups. Can I just take that one more step? And don't answer if you don't want to, but if it was 1947 or 1948, before the Cold War really took off, and you said, “Well, that's similar to where we are with China right now. We have a competitive lead, but it may or may not be fragile,” what would you do differently in 1947, 1948, or 1949? What would Kissinger have done differently than what we did?

You know, I wrote 2 books with Dr. Kissinger, and I miss him very much. He was my closest friend. Henry was very much a realist in the sense that, when you look at his history in roughly 1936, 1937, and 1938, he and his family were Jewish and were forced to immigrate from Germany because of the Nazis.

He watched the entire world that he'd grown up with as a boy be destroyed by the Nazis and by Hitler, and then he saw the conflagration that occurred as a result. I tell you that, whether you like him or not, he spent the rest of his life trying to prevent that from happening again.

Mhm.

So we are safe today because people like Henry saw the world fall apart.

Mhm.

From my perspective, we should be very careful in our language and our strategy not to start that process. Henry's view on China was different from that of other China scholars. His view on China was that we shouldn't poke the bear, that we shouldn't talk about Taiwan too much, and that we should let China deal with its own problems, which were very significant.

But he was worried that we or China, in a small way, would start World War II in the same way that World War I was started. You remember that World War I started with a small geopolitical event, which was quickly escalated for political reasons on all sides, and then the rest was a horrific war—the war to end all wars at the time.

So we have to be very careful when we have these conversations not to isolate each other. Henry started a number of what are called Track Two dialogues, and I'm part of one of them, to try to make sure we're talking to each other.

Somebody who's a hardcore person would say, “Well, we're Americans and we're better,” and so forth. I can tell you, having spent lots of time on this, the Chinese are very smart, very capable, and very much up here. If you're confused about that, again, look at the arrival of DeepSeek. A year ago, I said they were 2 years behind.

I was clearly wrong.

With enough money and enough power—

They're in the game.

Yeah. Let me actually drill in just a little bit more on that, too, because I think one of the reasons DeepSeek caught up so quickly is that it turned out inference-time compute generates a lot of IQ. I don't think anyone saw that coming, and inference-time compute is a lot easier to catch up on.

If you take one of our big open-source models and distill it, then make it a specialist, like you were saying a minute ago, and put a ton of inference-time compute behind it, it's a massive advantage. It's also a massive leak of capability within CBRN, for example, that nobody anticipated.

And CBRN, remember, is chemical, biological, radiological, and nuclear.

Let me rephrase what you said. If the structure of the world in 5 to 10 years is 10 models—and I'll make some numbers up: 5 in the United States, 3 in China, and 2 elsewhere—and those models are data centers that are multigigawatt, they will all be nationalized in some way.

In China, they will be owned by the government.

Mm-hmm.

The stakes are too high.

Mm-hmm. In my military work, one day I visited a place where we keep our plutonium. We keep our plutonium in a base that's inside another base, with even more machine guns and even more specialized security, because plutonium is so interesting and obviously very dangerous. I believe it's one or two facilities that we have in America.

So, in that scenario, these data centers will have the equivalent of guards and machine guns because they're so important. Is that a stable geopolitical system? Absolutely. You know where they are. The president of one country can call the other, and they can have a conversation. They can agree on what they agree on, and so forth.

Then you have a humongous data center proliferation problem, and that's where the open-source issue is so important, because those servers, which will be proliferated throughout the world, will all be open source. We have no control regime for that. Now, I'm in favor of open source, as you mentioned earlier with Marc Andreessen, that open competition and so forth tends to allow people to run ahead. In defense of the proprietary companies, collectively, they believe, as best I can tell, that the open-source models can't scale fast enough because they need this heavyweight training. If you look, I'll give you an example of Grok: it's trained on a single cluster that was built by Nvidia in 20 days or so in Memphis, Tennessee, of 200,000 GPUs. A GPU is about $50,000. You can say it's about a $10 billion supercomputer in one building that does one thing, right? If that is the future, then we're okay because we'll be able to know where they are.

Yeah. If, in fact, the arrival of intelligence is ultimately a distributed problem, then we're going to have lots of problems with terrorism, bad actors, and North Korea, which is my greatest concern. China and the United States are rational actors.

Yeah. The terrorist who has access to this—I don't want to go all negative on this podcast. It's an important thing to wake people up to the deep thinking you've done on this. My concern is the terrorist who gains access—

Are we spending enough time and energy, and are we training enough models, to watch them?

So, the companies are doing this. There's a body of work happening now that can be understood as follows: You have a superintelligent model. Can you build a model that's not as smart as the student it's studying? There is a professor that's watching the student, but the student is smarter than the professor. Is it possible to watch what it does?

It appears that we can. It appears that there's a way, even if you have a rogue, incredible thing, to watch it and understand what it's doing and thereby control it.

Another example of where we don't know is that it's very clear that these savant models will proceed. There's no question about that. The question is: How do we get the Einsteins? There are 2 possibilities. One is to discover completely new schools of thought—

Which is what's most exciting. Yeah. In our book Genesis, Henry and I and Craig talk about the importance of polymaths in history. In fact, the first chapter is on polymaths. What happens when we have millions and millions of polymaths? Very, very interesting. Now, it looks like the great discoveries—the greatest scientists and people in our history—had the following property: They were experts in something, and they looked at a different problem and saw a pattern.

In one area of thinking, they could see a pattern that they could apply to a completely unrelated field, and they were able to do so and make a huge breakthrough.

Okay.

Now, it looks like the great discoveries—the greatest scientists and people in our history—had the following property: They were experts in something, and they looked at a different problem and saw a pattern in one area of thinking that they could apply to a completely unrelated field. The models today are not able to do that.

So, one thing to watch for is, algorithmically, when they can do that.

This is generally known as the non-stationarity problem, because the reward functions in these models are fairly straightforward: Beat the human, beat the question, and so forth. But when the rules keep changing, is it possible to say the old rule can be applied to a new rule to discover something new?

And again, the research is underway. We won't know for years.

Peter and I were over at OpenAI yesterday, actually, and we were talking to many people, but Noam Brown in particular. I said the word of the year is “scaffolding,” and he said, “Yeah, maybe the word of the month is scaffolding.” I was like, “Okay, what did I step on there?”

He said, “Look, right now, if you try to get the AI to discover relativity or just some greenfield opportunity, it won't do it. If you set up a framework, kind of like a lattice or a trellis, the vine will grow on the trellis beautifully, but you have to lay out those pathways and breadcrumbs.” He was saying the AI's ability to generate its own scaffolding is imminent.

Mm-hmm. That doesn't make it completely self-improving. It's not Pandora's box, but it's also much deeper down the path of creating an entire breakthrough in physics, or creating an entire feature-length movie. These prompts that require 20 hours of consecutive inference-time compute will pretty much be a 2025 thing, at least from their point of view.

So, recursive self-improvement is the general term for the computer continuing to learn.

Yeah.

We've already crossed that—

In the sense that these systems are now running and learning things, and they're learning from the way they think within limited functions. When does the system have the ability to generate its own objective and its own question?

It does not have that today.

Yep. That's another sign. Another sign would be that the system decides to exfiltrate itself and takes steps to get itself away from the command-and-control system. Gemini hasn't called you yet and said, “Hi, Eric.”

But there are theoreticians who believe that the systems will ultimately choose that as a reward function because they're programmed to continue to learn.

Another one is access to weapons, right, and lying to get it. So, these are trip wires—

Right. All of which are trip wires that we're watching.

And again, each of these could be the beginning of a mini-Chernobyl event that would become part of consciousness. I think at the moment the U.S. government is not focused on these issues. They're focused on other things: economic opportunity, growth, and so forth. It's all good, but somebody's going to get focused on this, somebody's going to pay attention to it, and it will ultimately be a problem.

Can I clean up one kind of common misconception there? I think it's a really important one.

In the movie version of AI, you described, “Hey, maybe there are 10 big AIs: 5 are in the US, 3 are in China, and 2 are—one’s maybe in Brussels. Probably one’s maybe in Dubai.”

Or Israel.

Israel. Okay, there you go.

Somewhere like that.

Yeah. In the movie version of this, if it goes rogue, the SWAT team comes in, they blow it up, and it’s solved. But the actual real world is that when you’re using one of these huge data centers to create a superintelligent AI, the training process is 10^26, 10^28, or more FLOPs. But then the final brain can be ported and run on 4 GPUs or 8 GPUs, so a box about this size.

And it’s just as intelligent. That’s one of the beautiful things about it. You—

This is called stealing the weights.

Stealing the weights. Exactly. And the new thing is that, with that weight file, if you have an innovation in inference-time speed and say, “Oh, same weights, no difference,” you distill it or just quantize it or whatever, but you make it 100 times faster, now it’s actually far more intelligent than what you exported from the data center. All of these are examples of the proliferation problem.

And I’m not convinced that we will hold these things in 10 places.

And here’s why. Let’s assume you have the 10, which is possible. They will have subsets of models that are smaller but nearly as intelligent. The tree of knowledge of systems that have knowledge is not going to be 10 and then zero. It’s going to be 10, 100, 1,000, 1,000,000, and 1,000,000,000 at different levels of complexity. So the system that’s on your future phone may be 3 orders of magnitude, 4 orders of magnitude smaller than the one at the very tippy-top, but it will be very, very powerful.

Exactly what you’re talking about—there’s some great research going on at MIT. It’ll probably move to Stanford, just to be fair, but it always does. If you have one of these huge models and it’s been trained on movies and Swahili, a lot of the parameters aren’t useful for this particular use case, but the general knowledge and intuition are. So what’s the optimal balance between narrowing the training data and narrowing the parameter set to be a specialist without losing general learning?

The people who are opposed to that view—and again, we don’t know—would say the following: If you take a general-purpose model and specialize it through fine-tuning, it also becomes more brittle.

Mm-hmm.

Their view is that what you do is just make bigger and bigger and bigger models, because they’re in the big-model camp. That’s why they need gigawatts of data centers and so forth. Their argument is that the flexibility of intelligence they’re seeing will continue. Dario wrote a piece called Machines of Loving Grace, and he argued that there are 3 scaling laws at play. The first one is what you know as foundation-model scaling. We’re still on that. The second one is a test-time training law, and the third one is a reinforcement-learning scaling law.

Training laws are where, if you just put more hardware and more data, they just get smarter in a predictable way.

We’re just at the beginning, in his view, of the second and third ones. That’s why I’m sure our audience would be frustrated: Why do we not know? We don’t know.

Right. It’s too new. It’s too powerful.

At the moment, all of these businesses are incredibly highly valued, and they’re growing incredibly quickly. The uses of them—I mentioned earlier, going back to Google, the ability to refactor your entire workflow in a business—is a very big deal. That’s a lot of money to be made there for all the companies involved. We will see.

Eric, shifting the topic, one of the concerns that people have in the near term—and people have been ringing the alarm bells—is jobs. I’m wondering where you come out on this, and, flipping that forward to education, how do we educate our kids today in high school and college? What’s your advice? On the first question, do you believe that, as Dario has gone on TV shows and spoken about significant white-collar job loss, we’re seeing a multitude of different drivers, including robots coming in? How do you think about the job market over the next 5 years?

Let’s posit that in 30 or 40 years there will be a very different employment and robotic-human interaction—or the definition of whether we need to work at all, the definition of work, the definition of identity. Let’s just posit that, and let’s also posit that it will take 20 or 30 years for those things to work through the economy of our world.

Now, in California and other cities in America, you can get in a Waymo taxi.

The original work was done in the late ’90s.

The original challenge at Stanford was done, I believe, in 2004.

The DARPA Grand Challenge. It was 2004.

2005. Sebastian Thrun won one.

So, more than 20 years from a visible demonstration to our ability to use it in daily life. Why? It’s hard, it’s deep tech, it’s regulated, and all of that. I think that’s going to be true, especially for robots that are interacting with humans. They’re going to get regulated. You’re not going to have a robot wandering around and deciding to slap you. Society isn’t going to allow that sort of thing. It’s just not going to allow it.

So, in the shorter term, 5 or 10 years, I’m going to argue that this is positive for jobs in the following way. If you look at the history of automation and economic growth, automation starts with the lowest-status and most-dangerous jobs and then works up the chain. Think about assembly lines and cars and furnaces—all these very, very dangerous jobs that our forefathers did. They don’t do them anymore. They’re done by robotic solutions of one form or another, typically not a humanoid robot but an arm. The world dominated by intelligent arms and so forth will automate those functions.

What happens to the people? It turns out that the person who was working with the welder, who’s now operating the arm, has a higher wage, and the company has higher profits because it’s producing more widgets. So the company makes more money and the person makes more money.

Now you sit there and say, “Well, that’s not true, because humans don’t want to be retrained.” But in the vision that we’re talking about, every single person will have a human-computer assistant that’s very intelligent and helps them perform. You take a person of normal intelligence or knowledge and add an accelerant, and they can get a higher-paying job.

So you sit there and go, “Well, why are there more jobs? There should be fewer jobs.” That’s not how economics works. Economics expands because opportunities expand, profits expand, wealth expands, and so forth. There’s plenty of dislocation, but in aggregate, are there more people employed or fewer? The answer is more people with higher-paying jobs.

Is that true in India as well?

It will be, and you picked India because India has a positive demographic outlook, although its birth rate is now down to 2.0.

That’s good. The rest of the world is choosing not to have children.

If you look at Korea, it’s now down to 0.7 children per 2 parents.

Yeah.

China is down to 1 child per 2 parents.

It’s evaporating.

Now, what happens in those situations? They completely automate everything because it’s the only way to increase national productivity. So the most likely scenario, at least in the next decade, is that it will be a national emergency to use more AI in the workplace to give people better-paying jobs and create more productivity in the United States, because our birth rate has been falling.

People have talked about this for 20 years. If you have this conversation and ignore demographics, which is negative for humans, and economic growth, which occurs naturally because of capital investment, then you miss the whole story. There are plenty of people who lose their jobs, but there are an awful lot of people who have new jobs.

The typical simple example would be all those people who work in Amazon distribution centers and Amazon trucks. Those jobs didn’t exist until Amazon was created. The number-one shortage in jobs right now in America is truck drivers. Why? Truck driving is lonely, hard, low-paying, and low-status. Good people don’t want it. They want a better-paying job.

Going back to education, it’s really a crime that our industry has not invented the following product: a product that teaches every single human who wants to be taught, in their language, in a gamified way, the stuff they need to know to be a great citizen in their country. That can all be done on phones now. It can all be learned, and you can all learn how to do it. Why don’t we have that product? The investment in the humans of the world is always the best return. Investment in knowledge and capability is always the right answer.

Let me try and get your opinion on this because you’re so influential with—

So, I’ve got about 1,000 people in the companies where I’m the controlling shareholder, and I’ve been trying to tell them exactly what you just articulated, where a lot of these people have been in the company for 10 or 15 years.

They're incredibly capable and loyal, but they've learned a specific white-collar skill. They worked really hard to learn the skill, and the AI is coming within no more than 3 years, and maybe 2 years. The opportunity to retrain and have continuity is right now.

But if they delay—which everyone seems to be doing: “Let’s wait and see”—what I’m trying to tell them is that if you wait and see, you’re really screwing over that employee. So we are in wild agreement that this is going to happen, and the winners are the ones who act.

Now, what’s interesting is when you look at innovation history, the biggest companies you would think of are the slowest because they have economic resources that the little companies typically don’t. They tend to eventually get there, right? So watch what the big companies do.

Their CFOs and the people who measure things carefully, who are very, very intelligent, say, “I’m done with that 1,000-person engineering team that doesn’t do very much. I want 50 people working in this other way, and we’ll do something else with the other people.”

And when you say big companies, we’re thinking Google and Meta. We’re not thinking, you know, big banks that haven’t done anything.

I’m thinking about big banks. When I talk to CEOs—and I know a lot of them in traditional industries—what I counsel them is, you already have people in the company who know what to do. You just don’t know who they are.

Call for a review of the best ideas to apply AI in your business, and inevitably the first ones are boring: improve customer service, improve call centers, and so forth. But then somebody says, “You know, we could increase revenue if we built this product.”

I’ll give you another example. There’s this whole industry of people who work on graphical user interfaces, one way or another. I think user interfaces are largely going to go away because, if you think about it, the agents typically speak English or other languages. You can talk to them. You can say what you want, and the UI can be generated.

I can say, “Generate me a set of buttons that allows me to solve this problem,” and it’s generated for you. Why do I have to be stuck in what is called the WIMP interface—Windows, icons, menus, and pointer—that was invented at Xerox PARC 50 years ago? Why am I still stuck in that paradigm? I just want it to work.

Kids in high school and college now—any different recommendations for where they go? When you spend any time in a high school—or I was at a conference yesterday where we had a drone challenge—

And you watch the 15-year-olds, they’re going to be fine. They’re just going to be fine. It all makes sense to them, and we’re in their way.

Digital natives.

But they’re more than digital natives. They get it. They understand the speed. It’s natural to them. They’re also, frankly, faster and smarter than we are, right? That’s just how life works, I’m sorry to say. So we have wisdom; they have intelligence. They win, right?

Purpose-driven.

Yeah. Any form of solution that you find interesting. Most kids get into it for gaming reasons or something, and they learn how to program very young, so they’re quite familiar with this.

I work at a particular university with undergraduates, and they’re already doing different algorithms for reinforcement learning as sophomores. This shows you how fast this is happening at their level. They’re going to be just fine.

They’re responding to the economic signals, but they’re also responding to their purpose. An example would be that you care about climate, which I certainly do. If you’re a young person, why don’t you figure out a way to simplify the climate science, using simple foundation models to answer these core questions?

Why don’t you figure out a way to use these powerful models to come up with new materials that allow us, again, to address the carbon challenge? And why don’t you work on energy systems to have better and more efficient energy sources that are less carbon-intensive? You see my point?

Yeah. You know, I’ve noticed, because I have kids exactly that age, that there’s a very clear step-function change, largely attributable, I think, to Google and Apple. They have the assumption that things will work.

If you go just a couple of years older, during the WIMP era, as you described it—which I’ll attribute more to Microsoft—the assumption is that nothing will ever work. If I try to use this thing, it’s going to crash.

Also, what was interesting in my career was that I used to give these speeches about the internet, which I enjoyed, where I said, “You know, the great thing about the internet is that it has an off button. You can turn it off, and you can actually have dinner with your family. Then you can turn it on after dinner.”

This is no longer possible. The distinction between the real world and the digital world has become confusing. None of us are offline for any significant period of time.

Yeah. Indeed, the reward system in the world has now caused us not even to be able to fly in peace.

Right. Drive in peace, take a train in peace.

Starlink is everywhere.

Right. That ubiquitous connectivity has some negative impact in terms of psychological stress, loss of emotional and physical health, and so forth. But the benefit of that productivity is without question.

Google I/O was amazing. I mean, hats off to the entire team there. Veo 3 was shocking, and we’re sitting here 8 miles from Hollywood. I’m wondering about your thoughts on the impact this will have.

Are we going to see the 1-person feature film, like we’re potentially seeing 1-person unicorns in the future with AI? Are we going to see an individual be able to compete with a Hollywood studio? And should they be worried about their assets?

Well, they should always be worried because of intellectual-property issues and so forth. I think blockbusters are likely to still be put together by people with an awful lot of help from AI. I don’t think that goes away.

If you look at what we can do with generating long-form video, it’s very expensive to do long-form video, although that will come down. There’s also an occasional extra leg or extra clock or whatever. It’s not perfect yet, and that requires human editing.

So even in the scenario where a lot of the video is created by a computer, there are going to be humans producing and directing it for reasons. My best example in Hollywood is—let’s use the example, and I was at a studio where they were showing me this—they happened to have an actor who was recreating William Shatner’s movie movements, a young man.

They had licensed the likeness from William Shatner, who’s now older, and they put his head on this person’s body. It was seamless. Well, that’s pretty impressive. That’s more revenue for everyone. An unknown actor becomes a bit more famous, Mr. Shatner gets more revenue, and the whole movie genre works. That’s a good thing.

So who wins?

The costs are lower. The movies are made quicker. In theory, the movies are better, right, because you have more choices. So everybody wins.

Who loses? Well, there was somebody who built that set, and that set isn’t needed anymore. That’s a carpenter and a very talented person who now has to go get a job in the carpentry business.

So again, I think people get confused. If I look at the digital transformation of entertainment, subject to intellectual property being held—which is always a question—it’s going to be just fine.

There are still going to be blockbusters. The cost will go down, not up, and the allocation of revenue will shift because, in Hollywood, they essentially have their own accounting, and they essentially allocate all the revenue to all the key producing people. The allocation will shift to the people who are the most creative. That’s a normal process.

Remember, we said earlier that automation gets rid of the poor, lowest-quality jobs and the most dangerous jobs. The jobs that are sort of straightforward are probably automated, but there are really creative jobs.

Another example is the scriptwriters. You’re still going to have scriptwriters, but they’re going to have an awful lot of help from AI to write even better scripts. That’s not bad.

Okay. I saw a study recently out of Stanford that documented AI as being much more persuasive than the best humans.

Yes.

That set off some alarms. It also set off some interesting thoughts on the future of advertising.

Any particular thoughts about that?

We know the following. If the system knows you well enough, it can learn to convince you of anything.

Mhm. So what that means in an unregulated environment is that the systems will know you better and better. They’ll get better at pitching you, and if you’re not savvy, if you’re not smart, you could be easily manipulated. We also know that the computer is better than humans at trying to do the same thing.

So none of this surprises me. The real question—and I’ll ask this as a question—is: In the presence of unregulated misinformation engines, of which there will be many—advertisers, politicians, criminals, people trying to evade responsibility, all sorts of people who have free speech, including the ability to use misinformation to their advantage—what happens to democracy?

Yeah, we’ve all grown up in democracies where there’s a sort of consensus around trust, and there’s an elite that more or less administers the trust vectors and so forth. There’s a set of shared values. Do those shared values go away? In our book, Genesis, we talk about this as a deeper problem: What does it mean to be human when you’re interacting mostly with these digital things?

Especially if the digital things have their own scenarios? My favorite example is that you have a son, a grandson, or a child, and you give them a bear. The bear has a personality, and the child grows up, but the bear grows up, too.

So who regulates what the bear talks to the kid? Most people haven’t actually experienced the super-empathetic voice that can have any inflection you want. When they see that, which will be available in probably the next 2 months—

Yeah. They’re going to completely open their eyes to what this is.

Well, remember that voice cloning was solved a few years ago, and you can cast anyone else’s voice onto your own.

Yeah.

And that has all sorts of problems.

Have you seen an avatar yet of somebody you love who’s passed away, or Henry Kissinger, or anything like that?

Well, we actually created one with the permission of his family.

Did you start crying instantly?

It’s very emotional. It’s very emotional because, you know, it brings back a real human memory, a real voice. I think we’re going to see more of that. One obvious thing that will happen is that, at some point in the future, when we naturally die, our digital essence will live in the cloud. It will know what we knew at the time, and you can ask it a question.

Yeah.

So can you imagine asking Einstein, going back to Einstein—

What did you really think about?

You know, this other guy—

Did you actually like him, or were you just being polite with him in letters?

Yeah.

Right. In all those famous contests that we study as students, can you imagine being able to ask the people—

Yeah.

Today, with today’s retrospective, what did you really think?

I know that the education example you gave earlier is so much more compelling when you’re talking to Isaac Newton or Albert Einstein instead of just a textbook.

This is coming back to Veo 3 and the movies. One of the first companies we incubated out of MIT was CourseAdvisor. We sold it to Don Graham and The Washington Post, and I worked for him for a year after that. The conception was: Here’s the internet, here’s the newspaper; let’s move the newspaper onto the internet. We’ll call it washingtonpost.com. If you look at where it ended up today, with Meta, TikTok, and YouTube, it didn’t end up anything like the newspaper moving to the internet.

So now here’s Veo 3. Here are movies. You can definitely make a long-form movie much more cheaply.

But I just had this experience. A director will try to make a tearjerker by leading me down a 2-hour-long path, but I can get you to that same emotional state in about 5 minutes if it’s personalized to you.

Well, one of the things that’s happened because of the addictive nature of the internet is that we’ve lost the deep state of reading.

Mhm.

I was walking around and saw a Barnes & Noble bookstore. Big—oh, my God, my old home is back—and I went in and I felt good.

It’s a very fond memory. But the fact of the matter is that people’s attention spans are shorter. They consume things quicker. One interesting thing about sports is that the sports highlights business is huge, with licensed clips around highlights because it’s more efficient than watching the whole game.

So I suspect that if you’re with your buddies and you want to be drinking and so forth, you put the game on, and that’s fine. But if you’re a busy person and you want to know what happened with your favorite team, the highlights are good enough.

Yeah. You have 4 panes of it going at the same time, too.

And so this is, again, a change. It’s a more fundamental change to attention.

Mhm.

I’ve been working with a lot of 20-somethings in research.

And one of the questions I had is: How do they do research in the presence of all of these stimulations? I can answer the question definitively. They turn off their phone.

You can’t think deeply as a researcher with this thing buzzing.

Yeah.

Right. We essentially, aside from sleeping—and we’re working on having you sleep less, I guess, from stress—we’ve tried to monetize all of your waking hours with something: some form of ads, some form of entertainment, some form of subscription. That is completely antithetical to the way humans have traditionally worked with respect to long, thoughtful examination of principles and the time that it takes to be a good human being. These are in conflict right now. There are various attempts at this.

Yeah.

My favorites are these digital apps that make you relax. The correct thing to do to relax is to turn off your phone, right? Then relax in a traditional way, as humans have done for 70,000 years of existence.

Yeah. Yeah, I had an incredible experience. I’m doing the flight from MIT to Stanford all the time.

And, you know, like you said, attention spans are getting shorter and shorter and shorter. The TikTok extreme—the clips are so short.

This particular flight was my first time brainstorming with Gemini for 6 hours straight, and I completely lost track of time. I was trying to figure out circuit design and chip design for inference-time compute, and it’s so good at brainstorming with me and bringing back data, as long as the Wi-Fi on the plane is working. Time went by. It was my first experience with technology that went in the other direction.

But I noticed that you also weren’t responding to texts and annoyances. You weren’t reading ads. You were deep inside a system—

—for which you paid a subscription.

Mhm.

So if you look at the Deep Research stuff, one of the questions I have is: When you do a deep-research analysis—I was looking at factory automation for something—where is the boundary of factory automation versus human automation? It’s an area I don’t understand very well. It’s a very deep, technical set of problems. I didn’t understand it.

It took 12 minutes or so to generate this paper. Twelve minutes of these supercomputers is an enormous amount of time. What is it doing? And the answer, of course, is that the product is fantastic.

Yeah. You know, to Peter’s question earlier, too, I keep the Google IPO prospectus in my bathroom up in Vermont. It’s from 2004. I’ve read it probably 500 times. It’s getting a little ratty, actually. You’re the only person besides me who did the same.

I read it 500 times because I had to. It was legally required.

Well, I still read it because of the misconceptions. It’s such a great learning experience. Even before the IPO, if you think back, there was this big debate about whether it would be ad revenue, subscription revenue, or paid inclusion; whether the ads would be visible; and all this confusion about how you were going to make money with this thing. Now, the internet moved to almost entirely ad revenue.

No, but you have this with Netflix. There was this whole discussion about how you would fund movies through ads, and the answer is you don’t. You have a subscription. The Netflix people looked at having free movies without a subscription, advertising-supported, and the math didn’t work. So I think both will be tried. The fact of the matter is that Deep Research, at least at the moment, is going to be chosen by the well-to-do or for professional tasks.

You’re capable of spending $200 a month. A lot of people cannot afford it.

And that free service, remember, is the stepping stone for that young person, man or woman, who just needs that access. My favorite story there is that when I was at Google, I went to Kenya. Kenya is a great country, and I was with this computer science professor. He said, “I love Google.” I said, “Well, I love Google, too.” He said, “Well, I really love Google.” I said, “I really love Google, too.” I asked, “Why do you really love Google?” He said, “Because we don’t have textbooks.”

And I thought, “The top computer science program in the nation does not have textbooks.”

Yeah.

Well, let me jump in on a couple of things here. Eric, in the next few years, what moats actually exist for startups as AI is coming in and disrupting? Do you have a list?

Yes, I’ll give you a simple answer.

And what do you look for in the companies that you’re investing in?

First, in the deep-tech hardware stuff, there are going to be patents, patent filings, inventions—the hard stuff. Those things are much slower than the software industry in terms of growth, and they’re just as important. Power systems, all those robotic systems we’ve been waiting for a long time—they’re just slower. All sorts of hardware is hard.

Hardware is hard for those reasons.

In software, it’s pretty clear to me it’s going to be really simple. Software is typically a network-effect business where the fastest mover wins. The fastest mover is the fastest learner in an AI system.

What I look for is a company where they have a loop. Ideally, they have a couple of learning loops. I’ll give you a simple learning loop: as you get more people, more people click, and you learn from their clicks. They express their preferences.

Let’s say I invent a whole new consumer thing, which I don’t have an idea for right now, but imagine I did. Furthermore, I said that I don’t know anything about how consumers behave, but I’m going to launch this thing. The moment people start using it, I’m going to learn from them, and I’ll have instantaneous learning to get smarter about what they want.

So, I start from nothing. If my learning slope is this, I’m essentially unstoppable. I’m unstoppable because my learning advantage, by the time my competitor figures out what I’ve done, is too great.

Yeah.

Now, how close can my competitor be and still lose? The answer is a few months.

Mhm.

Because the slopes are exponential.

Mhm.

And so, it’s likely to me that there will be another 10 fantastic Google-scale, Meta-scale companies. They’ll all be founded on this principle of learning loops.

When I say learning loops, I mean in the core product, solving the current problem as fast as you can. If you cannot define the learning loop, you’re going to be beaten by a company that can define it.

And you said 10 Meta- or Google-sized companies. Do you think there will also be 1,000? If you look at the enterprise software business—Oracle on down, PeopleSoft, whatever—there are thousands of those. Or will they all consolidate into those 10 domain-dominant learning-loop companies?

I think I’m largely speaking about consumer scale, because that’s where the real growth is.

The problem with learning loops is that if your customer is not ready for you, you can only learn at a certain rate. So, it’s probably the case that the government is not interested in learning, and therefore there’s no growth in learning loops serving the government. I’m sorry to say that needs to get fixed.

Yeah.

Educational systems are largely regulated and run by the unions and so forth. They’re not interested in innovation. They’re not going to be doing any learning. I’m sorry to say we have to get that fixed.

So, the ones where there’s a very fast feedback signal are the ones to watch. Another example: it’s pretty obvious that you can build a whole new stock-trading company where, if you get the algorithms right, you learn faster than everyone else, and scale matters. So, in the presence of scale and fast learning loops, that’s the moat. Now, I don’t know that there are many others there.

Do you think brand would be a moat?

Brand matters, but less so. What’s interesting is people seem to be perfectly willing now to move from one thing to the other, at least in the digital world.

There’s a whole new set of brands that have emerged that everyone is using—the next generations that I haven’t even heard of. Within those learning loops, do you think domain-specific synthetic data is a big advantage?

Well, the answer is: whatever causes faster learning. There are applications where you have enough training data from humans. There are applications where you have to generate the training data from what the humans are doing.

Right?

So, you could imagine a situation where you had a learning loop where there are no humans involved, where it’s monitoring something—some sensors—but because you learn faster on those sensors, you get so smart you can’t be replaced by another sensor-management company. That’s the way to think about it.

So, what about the capital for the learning loop? Do you know Danielle Roose, who runs C10?

Danielle and I are really good friends. We’ve been talking to our governor, Maura Healey, who’s one of the best governors in the world.

I agree.

So, there’s a problem in our academic systems where the big companies have all the hardware because they have all the money, and the universities do not have the money for even reasonably sized data centers. I was with one university where, after lots of meetings, they agreed to spend $50 million on a data center, which generates less than 1,000 GPUs—

Right, for the entire campus and all the research.

Yeah. And that doesn’t even include the terabytes of storage and so forth. So, I and others are working on this as a philanthropic matter. The government is going to have to come in with more money for universities for this kind of stuff.

That is among the best investments.

When I was young, I was on a National Science Foundation scholarship, and, by the way, I made $15,000 a year. The return to the nation on my $15,000 has been very good, shall we say, based on the taxes that I pay and the jobs that we have created.

Creating an ecosystem for the next generation to have access to the systems is important. It’s not obvious to me that they need billions of dollars.

It’s pretty obvious to me that they need $1 million, $2 million. Yeah.

That’s the goal.

Yeah. I want to take us in a direction of wrapping up on superintelligence and the book. We didn’t finish the timeline on superintelligence, and I think it’s important to give people a sense of how quickly self-referential learning can get us there and how rapidly we can get to something that’s 1,000 times, 1 million, or 1 billion times more capable than a human.

On the flip side of that, Eric, when I look at my greatest concerns once we get through this 5- to 7-year period of, let’s just say, rogue actors, destabilization, and such, one of the biggest concerns I have is the diminishment of human purpose.

Mhm.

You wrote in the book—and I’ve listened to it; I haven’t read it physically, and my kids say, “You don’t read anymore.”

You listen to books; you don’t read.

But you said the real risk is not the Terminator; it’s drift. You argue that AI won’t destroy humanity violently, but might slowly erode human values, autonomy, and judgment if left unregulated and misunderstood. So, it’s really a WALL-E-like future versus a Star Trek, boldly-go-out-there future.

We’re very clear in the book, and my own personal view is it’s very important that human agency be protected.

Yeah.

Human agency means the ability to get up in the day and do what you want, subject to the law. It’s perfectly possible that these digital devices can create a form of virtual prison where you don’t feel that you, as a human, can do what you want. That is to be avoided.

I’m not worried about that case. I’m more worried about the case that, if you want to do something, it’s just so much easier to ask your robot or your AI to do it for you. The human spirit wants to overcome a challenge. I mean, the unchallenged life is going to be so critical.

But there will always be new challenges.

When I was a boy, one of the things that I did was repair my father’s car. I don’t do that anymore. When I was a boy, I used to mow the lawn. I don’t do that anymore.

Sure.

There are plenty of examples of things that we used to do that we don’t need to do anymore. But there will be plenty of things. Just remember, the complexity of the world that I’m describing is not a simple world. Managing the world around you is going to be a full-time and purposeful job.

Partly because there will be so many people fighting misinformation and for your attention, and there’s obviously lots of competition and so forth. There are lots of things to worry about. Plus, you have all of the people trying to get your money, create opportunities, deceive you, what have you. So, I think human purpose will remain because humans need purpose.

That’s the point. There’s lots of literature showing that people who have what we would consider to be low-paying, worthless jobs enjoy going to work. So, the challenge is not to get rid of their jobs; it’s to make their jobs more productive using AI tools. They’re still going to go to work.

And, to be very clear, this notion that we’re all going to be sitting around doing poetry is not happening. In the future, there will be lawyers. They’ll use tools to have even more complex lawsuits against each other. There will be evil people who will use these tools to create even more evil problems. There will be good people who will be trying to deter the evil people. The tools change, but the structure of humanity—the way we work together—is not going to change.

Peter and I were on Mike Saylor’s yacht a couple of months ago, and I was complaining that the curriculum is completely broken in all these schools.

But what I meant was, we should be teaching AI. And he said, “Yeah, they should be teaching aesthetics.” I looked at him and said, “What the hell are you talking about?” He said, “No, in the age of AI, which is imminent, look at everything around you. Whether it’s good or bad, enjoyable or not enjoyable, it’s all about designing aesthetics.”

When AI is such a force multiplier that you can create virtually anything, what are you creating and why? That becomes the challenge. If you look at Wittgenstein and the sort of theories of all of this stuff, it’s all fundamental. We’re having a conversation that America has about tasks and outcomes. It’s our culture, but there are other aspects of human life: meaning, thinking, reasoning. We’re not going to stop doing that.

So imagine if your purpose in life in the future is to figure out what’s going on and to be successful. Just figuring that out is sufficient, because once you’ve figured it out, it’s taken care of for you.

That’s beautiful.

Right? That provides purpose.

Yeah.

It’s pretty clear that robots will take over an awful lot of mechanical or manual work.

I like to repair the car. I don’t do it anymore. I miss it, but I have other things to do with my time.

Yeah.

Take me forward. When do you see what you define as digital superintelligence?

Within 10 years.

Within 10 years. And what do people need to know about that?

What do people need to understand and prepare themselves for, either as a parent, an employee, or a CEO?

One way to think about it is that when digital superintelligence finally arrives and is generally available and generally safe, you’re going to have your own polymath. So you’re going to have the sum of Einstein and Leonardo da Vinci in the equivalent of your pocket.

I think thinking about how you would use that gift is interesting. And, of course, evil people will become more evil, but the vast majority of people are good. Yes.

They’re well-meaning, right? So, going back to your abundance argument, there are people who’ve studied the notion of productivity increases, and they believe that you can get, we’ll see, 20% to 30% year-over-year economic growth through abundance and so forth. That’s a very wealthy world.

That’s a world of much less disease, many more choices, much more fun, if you will, right? Just taking all those poor people and lifting them out of the daily struggle they have—that is a great human goal. Let’s focus on that. That’s the goal we should have. Does GDP still have meaning in that world?

If you include services, it does. One of the things about manufacturing—and everyone’s focused on trade deficits, and they don’t understand—is that the vast majority of modern economies are service economies, not manufacturing economies.

And if you look at the percentage of farming, it went from roughly 98% to roughly 2% or 3% in America over 100 years. If you look at manufacturing, the heyday was in the ’30s, ’40s, and ’50s. Those percentages are now down well below 10%. It’s not because we don’t buy stuff; it’s because the stuff is automated. You need fewer people. There are plenty of people working in other jobs. So again, look at the totality of the society. Is it healthy?

If you look in China, it’s easy to complain about them. They now have deflation. They have a term for it called “lying flat,” where they stay at home. They don’t participate in the workforce, which is counter to their traditional culture.

If you look at reproduction rates, these countries are essentially having no children. That’s not a good thing.

Yeah.

Right. Those are problems that we’re going to face. Those are the new problems of the age.

I love that.

Eric, I’m so grateful for your time.

Thank you. Thank you both. I love your show.

Yeah. Thank you, buddy.

Thank you.

Okay. Thank you, guys.