与马丁·卡萨多和史蒂文·辛诺夫斯基一起探讨计算机的演变

Right now, if I give 20 people $1 billion, they can actually use it usefully. We've kind of moved the industry from an engineering-bound problem to a capital problem. That's fundamentally very different.

Math is very much a leading-edge indicator of what the market might be interested in. Why? Some people will walk in and say the foundations of AGI and reasoning are going to be math, but that doesn't tell you anything about reality. For me, it's still in the domain of: it's really good at playing a game. This is the best StarCraft player ever, which is cool and very powerful, but I have a hard time connecting that with the market.

1. Will AI Math Ever Map Onto Physical Reality?

The startups don't aim straight at the incumbents, and the incumbents just don't pay attention. Microsoft is worried far more about what Amazon and Google are doing than anyone in the startup space. Everybody who's from a big company in Silicon Valley always thinks, "Oh my God, we're just going to crush all of these little companies." And then you realize they never get crushed. I think this is why we're seeing such meteoric rises from Cursor, Anthropic, and OpenAI. Capital is scarce and hard to get, but once you get it...

2. Making Sense of AI & Math: The Riemann Hypothesis Moment

First off, thanks for both of you making time to be here. That's great. Jared Kaplan tweeted a few days ago something along the lines of how he told Claude to try to solve the Riemann hypothesis and to try harder. I don't know if there was actually any progress made, but it's part of the larger conversation around the accomplishments that seem to be happening. How do we make sense of this in terms of what is actually happening, and what does it mean for math?

I'm going to let Steve go.

I'm no mathematician at all, but I think it's an important moment because it sort of divides the world into 2 groups. There are the people who are very excited that these things are being solved; it doesn't matter if you understand them. Actually, the number of people who understand what these things are is very small. Then there are the people who are just like, "Oh, it's fake. It's going to put people out of jobs. No one's going to know the future of where these fields go."

The most interesting thing about it is that the group that's most excited is mostly mathematicians, and they're the ones who are most impacted by what this level of AI did. That confuses everybody, because if you're of the school that says it's going to put people out of work, we're all going to get dumber, and it's the dawn of idiocracy because computers are doing all of our work, you're confused that the people most impacted are the most excited.

Yeah. Yeah. And I think that is itself shining a light on this moment that we're in right now.

You're talking to 2 systems guys, 2 product guys. You're going to get the same caveat from both of us. I feel there are some things we're actually both very expert on. This is not one of them, so I'm going to speak from the peanut gallery. I've got 2 comments.

One of them is that I view economic utility as a very important measure when you're talking about AI. There have been a lot of hours spent trying to solve some math problem, but if you sum up the entire postdoc salaries of all the people who have been working over the years on these problems, it's probably not very much. Part of me is saying that it's great that there are these capabilities, but I'm not sure that the fact that these have been longstanding problems is that much of an indication, because there hasn't been a huge economic incentive to solve them. Now, that doesn't mean that it's not hard. It's just that I don't think we have that validation that this unlocks some deluge of economic value.

The second point is that it's not surprising to me that AI is very good at solving an almost purely axiomatic domain that requires knowing a whole bunch of different things and putting the solutions together from very disparate spaces. Often, when I read these solutions—I've been reading them obsessively, like everybody else—they're like, "Oh, I came up with the solution." It's like, "Yeah, the solution was pretty straightforward. It just borrowed from a bit of math that I didn't know."

I think if there's a meta-learning here, the meta-learning is that there is a set of problems that probably require you to be too broad for most humans or most educational systems, and AI is going to solve those. It's clearly very good at solving axiomatic systems, but I don't think it provides a strong indication that it's solving things the market hasn't been able to solve, because there really hasn't been a market around these problems. Those are the best questions for us to answer. It's very exciting, seems reasonable and understandable, but I'm not sure what the longer-term implications are.

I do think there's something interesting about math being very much a leading-edge indicator of what the market might be interested in. I remember when I was in school, there was some big thing where someone at AT&T invented a new algorithm—a new program for doing linear algebra, a new way to solve linear equations—which is super important right now in the AI world. But his big thing was, "Well, now we can just calculate the United Airlines flight map in 3 hours less time than we could last week," right?

But let's dig into this. It's just not clear to me that the problems being solved are roadblocks to existing economically useful tasks, right? And if they were, it's not clear to me that they wouldn't have been solved. A postdoc who's been ruminating on a problem, getting paid $30,000 a year for 5 years, is very different from the market deciding that this is the 1 thing to unlock.

Maybe these problems being solved are the key problems to unlocking some big, economically productive use case. I just haven't seen that yet. So, for me, that's the next thing I'm looking for.

I don't even know what 12-dimensional spaces are or what that means. I'm completely with you on that. I don't even know what problems are in 12-dimensional space. Are you very skinny? Are you very tiny? I'm really confused by that.

Maybe I'm wrong here, but for me, it's still in the domain of: it's really good at playing a game. This is the best StarCraft player ever, which is cool and very powerful, but I have a hard time connecting that with, first, maybe the reason we didn't have it before is that there just wasn't an economic need, and, second, how does that actually map?

Listen, there's a huge range of these things. We get pitches all the time. Some people will walk in and say, "The foundations of AGI and reasoning are going to be math. Once you do that, you'll be able to answer every question, because the universe is based on fundamental mathematical principles. Once you understand that, you understand everything."

Other people, candidly, walk in the door and say, "Listen, that's great, but that doesn't tell you anything about reality." So I think there's more work to do, and this isn't just about getting better at math.

I do think what's interesting is that part of the reason mathematicians are very excited about it is because they work a certain way. If you work in history, there's basically no abstraction in history. There's just a bunch of facts, and then people develop these models that you can think of almost as force diagrams that explain war or famine or whatever.

Whereas mathematics has this super-long historic arc of layering on abstractions after abstraction—and don't worry, we'll get to OSI in a minute. But this idea that all of a sudden a bunch of math becomes a new level of abstraction...

I've found that it's mixed. Some are very excited, and some are in an existential crisis. The ones who are excited basically say, "It solves 20% of my job—the 20% I didn't like anyway. This allows me to explore a new frontier that's very important," or whatever.

What I've always wondered is whether that's a function of the type of problem being solved. I just can't imagine that if AI came along and solved cancer, someone who works on cancer would say, "Oh, I'm so existentially depressed. This is amazing." On the other hand, if we solve this math problem, someone might say, "Oh, I'm so depressed. AI solved the math problem." Maybe literally the entire utility of that problem was keeping somebody employed to solve it.

Or just writing articles in the back: "Another attempt, and here's where I went wrong." So let me offer it this way.

There's nothing on the other side of the solution, and so we're depressed because now this useless activity is gone. Let me stop. No, I mean, that's too cynical. I love math.

I think we caught you being a little cynical, but not really. It's more that—let me take it from the side this way, looking at the history of computer science. I had to take this class, so I looked at all the course catalogs for a bunch of schools.

You don't have to take it anymore. That was like discrete math, basically.

Yeah. Yeah, and then algorithmic complexity theory, which was a required class for a very long time, and now—

Do you remember Concrete Mathematics from Donald Knuth?

I didn't. You're a Stanford guy; I'm not. My state school didn't have that. That's a Cornell joke for us Cornellians. My class was taught by one of the luminaries in the field of algorithms—ironically, a Stanford PhD—John Hopcroft, of course.

Who, for the people who are pragmatic, invented 2–3 trees and a bunch of stuff as his thesis at Stanford. That's a legend.

But John was our professor in all this crap, and we had to learn all this P = NP stuff. I remember thinking, “This is the four-color—this is the four-color—”

Proven by computers.

No, exactly, but that's where I'm going. You just buried the lead.

Yeah, but for those of you who don't know, we had to take a whole course in college that basically boiled down to this problem. The interesting thing was why: the theoreticians had postulated that if you could solve this problem in polynomial time, then you could solve all these other problems, like the traveling salesman problem, much faster. That mattered because all of our computers were so compute-bound.

If you were the AT&T people who had a node with 6,000 switches and wanted to know how to route optimally, you'd say, “Well, we don't have enough. That's 2 years of running the simulation to solve this.”

Yeah. Yeah.

And so it turns out that one of the interesting things was that they proved the four-color theorem.

Yeah.

They did it—and, by the way, the four-color theorem just says that for any 2D planar map, you can use only 4 colors, such that no 2 adjacent areas have the same color. Right. Exactly. You only need 4 colors. You'll never need 5 colors.

And we learned it just so you kids know. That's literally how we learned it, and we could all repeat it like that. It's this very weird imprint over this problem.

And so what sort of happened was that no one ever arrived at what you could think of as a proof that looked like calculus. Instead, they proved that the number of potential solutions was finite.

Have you actually seen the proof?

Yeah. Yeah. 200 pages of combinations—

But they basically proved that there was a finite number of them, and then they computed all of them and said, “Look, it's only 4 colors.” It's this sort of brute-force proof, but it was only possible because of compute.

And to your point, that was actually very useful in the practical applications.

Right, right. Certainly, as a topology person—

Like setting strong bounds and things like that. So actually, I see—

And I think that, to me, was just a really good lesson in when you have a new level of abstraction that says, “This is a whole class of problems that can be solved.”

Yeah.

You can then build tools working at that level of abstraction, and everybody doesn't have to start from, “Okay, what's the 2–3 tree representation of what we're doing?”

So listen, it's hard not to get philosophical when you're talking about AI. I'm going to get philosophical, and you can tell me to shut up, but I just can't. You kind of do. This math thing seems to me a little different because it begs the following question: Will math ever be representative of physical phenomena? Has anybody ever taken a bunch of equations and actually predicted something physical? I don't know the answer to that.

I worked in these large simulation codes, and these large simulation codes are actually trying to compute physical phenomena, like the explosion of a star or what would happen to an airplane in a wind simulator. But all of those, even though they're just calculating these large differential equations, were based on empirical results.

Yeah, literally the equations of state for the—

Well, they were models. They were just—we could measure temperature in these places—

That's exactly right. It was all based on empirical equations of state, and so I've always wondered—

Is simulation computationally irreducible? You actually have to run the simulation in that case. It's not clear to me to what extent AI helps. I know people are trying to solve this problem with AI, but I don't know if these math answers have any impact on that type of stuff.

Maybe there's some separate algorithmics domain, to your point, where they do—or maybe modeling or logistics. But when it comes to, “Will this star explode?” or “Will this building stand up?”—the actual simulation—I think these things are pretty disjoint.

Then I read a lot of these discussions on the math solutions, and there are claims that if it can solve all math, you can predict anything. I just think that's a huge logical leap, and it's not clear to me that it's obviously true.

Yeah.

Or that there's any indication that it's true at all.

So one way to think about that, I think I might—

Talk about that. Again, this is so out of my league on the actual math.

To any systems people—

I'm good, but I'm—

Compelled. I'm inherently a tools person, and so I get this part of it, which is—

What's happened is that AI might not be the next tool to solve math problems—

At some scale that matters, but it might—

But it might lead to the development of a new kind of model. I brought some props to show this off.

So, of course, this is the original—

Math tool. Before something like this, this is one of these real ones from a Beijing market.

Well, you know, it's the kind they sell to tourists in Beijing. But I'm very proud of that because I negotiated it down to 7 cents.

But that became a level of abstraction, and all of a sudden you just had this basic math thing. Then you fast-forward a whole bunch. I brought this because it's so freaking cool. It is. Everybody knows what slide rules are. Nobody knows how to use them. This is called a Curta, which is an Austrian, basically round slide rule.

Yeah.

And so it's like a coffee grinder or a pepper mill. You have all these ways to set the numbers on the side, and then you turn it one way to add and another way to subtract.

Whoa.

And this thing is—

Wait, is that used for multi-number arithmetic, or is it used for things like logarithms?

No, it's only arithmetic. Okay.

Well, I think it depends on how you use it, but—

It's from the mid-20th century, I think.

And my uncle brought this back from the war.

Wow.

Inside this are 600 pieces of machined metal. It would cost $50,000 to make one now.

Do you know how to use it?

I actually did, but I'm not going to try to do it. I went through the trouble of learning how to use it while preparing for this, so I wouldn't be completely useless. It's been sitting on my shelf for years.

But the interesting thing is that all of a sudden, a whole new level of problems gets solved.

Wait, so you're saying the new model is the new calculator or the new graphing calculator? I actually remember when the TI-85 came out.

Oh, of course. Yeah. Yeah.

I remember that came out, and all the math teachers had this crisis. They said, “We used to give you a piece of paper and plot the x-y equation. Now you can do it on the calculator, and you can solve equations. Our field is dead.”

But what's interesting is why it's so important to AI today. Those people didn't complain when calculus came out because calculus was a baseline to them. There's this notion that people react to change more than they react to the baseline of where they all started.

I lived through it. I literally got the TI-35. It was one of the first calculators in schools. The only advanced math it did was a percent key and factorial, which we didn't even know what it was. You could do 59 factorial, and that was the maximum it could display. I went to college, and the classes were no calculators allowed.

The whole time, I was on the cusp of what was allowed and what wasn't allowed for everybody. I was there for the graphing calculator.

Yeah.

You literally had a blue book just to show all of your work, to show that you weren't plugging it into the graphing calculator.

See, I missed the graphing calculator. Most of us were actually writing video games in the back and couldn't care less about its ability to do math.

But.

Absolutely. You just play that backward, and you realize that after these guys went through this march of algebra, linear algebra, calculus, Fourier transforms, fluid dynamics, and all of that, it was, to your earlier point, based on need. So much of this math—

Well, all computers are basically from difference engines, which were just trying to calculate integrals.

But, of course, to be really clear, they were calculating integrals so that we could shoot missiles and cannons at each other.

Okay, so that's what—

Yes, which I'm not judging. I'm just saying—

Well, I don't mean to be pedantic about this, but one of my favorite parts of history is that it actually started with tides, which also had massive economic value. People were trying to calculate the tides, and this is where you had the old architectures. Those architectures were co-opted, of course, into the war effort for the ballistics. That's where I came from. It was actually very interesting: it was about 5,000 times faster than a human being when it came to doing this. And then, of course—

And it didn't make mistakes, which was sort of the—

But it was very specifically math and very specific economic utility. The interesting question to me is whether these models are clearly good at a type of math. Is it one that has somehow unlocked some sort of economic utility?

Yeah.

I don't know the answer to that.

Oh, yeah. I think it's super interesting to keep going with that because, to me, what's so cool is that doing that basic calculus for the war, making those missile tables and things like that, then unlocked the space race, basically, along with jet engines, factory automation, and all of these things. People were cheering that on. To me, culturally, that's the most interesting thing. Not only were they cheering it on, every parent was looking at their kids and saying, “Go learn that in school. Go win the Westinghouse competition. Go win the General Electric math competition.”

3. The Cold War, IBM 1953 & the Cultural Roots of Computing

Was that because of the Cold War? Was it because—

Well, obviously, the Cold War was a big cultural part of it, for sure, but it was just a general sense of the future. I found this incredibly cool brochure from IBM from 1953.

Do you just have this stuff in your house?

I just stumbled across it. This one I just got. I can't even believe this exists. It's a brochure about the future of computing.

Wait, I want to see it.

But first, you've got to look at it. It's got nuclear— the whole thing. The future of computing is a guy with atoms racing around his head. Oh, wait, we're zooming in and doing the Carol Merrill thing.

The fascinating thing is that it's from 1953. You have ENIAC, and at that point, that's it. That's the computer at the time. This is pre-704, pre-370. It's a brochure from IBM explaining what a computer might be, not even what a computer is. The opening sentence is, “It took millions of years to invent and recognize the usefulness of the wheel.” People were eating this stuff up.

Here's the part that I want to get to: It talks about computers and the two families of computers.

Yeah.

You get the slide rule, which is explaining the history, and what this is really leading up to is that we could do this for text, too.

Yeah.

Imagine who was reading this in 1953. It has to explain hexadecimal, decimal, and binary, and compare them to Roman numerals.

That's amazing.

Nobody knew.

What's the name of that thing?

It's just called IBM Lights the Future. It has a rocket-, test-tube-like spotlight on it, and it's incredible. There are oscilloscope waves in the back. It is the most incredible thing. It has this dictionary in the back. Imagine the first time someone explains a computer, and the dictionary includes “arithmetic unit,” “binary digit,” “bit,” “cathode-ray tube,” “electrostatic storage tube,” and so on. But the reason I opened this is because there's one cool page that really matters: “What is the organization of digital computers?”

This gets to the point about abstraction for us and AI. For 75 years, this is how we thought computers were organized.

Yeah.

Input, storage, arithmetic, control, and output.

Yeah.

That's all—it's what we learned in school. You took courses basically in each one of those. Last night, we were going back and forth on the abstractions that will remain in computer science, and you tossed in networking, which is sort of control.

Yeah. Everybody forgets networking, by the way. Of course, that was—

Well, because most people stop worrying about networking—

They stop worrying about it as soon as the packet leaves the computer. I would say the late 90s was the end of basically a mandatory networking class.

Because it was solved. For me, it was the transistor. I was, like, the last time computer science majors had to know what a transistor was. Trust me, I actually don't get what one is now. It's like a triangle symbol.

But the interesting thing is that those abstractions led to these fields that each dealt with one of them. You spent 20 years of your career on storage, and you watched the march from tubes to drums to spinning disks to tapes and so on. If you did output, you watched the invention of going from a teletype to a line-oriented teletype, to a black-and-white terminal, to color, to vector, and the whole deal. All of those were fields, and they all rose in parallel. Any computer science department, which came out of the math department because of the missiles—

—ended up being departments made up of those things, and then it all collapsed and produced us—the systems group.

Sure. Yeah, yeah, yeah. Let me just push on one angle of this. I clearly love the framing that we move up an abstraction, and at every abstraction, there's still a set of problems. It's just a higher level of abstraction. But I still think this notion of economic meat is very important.

Oh, yeah. Right. For example, Bletchley Park was about cracking a code for a war, and that effort created innovation whose outcome was winning World War II. ENIAC was about doing nuclear research—not just research, but innovation in terms of a war effort. We needed to calculate integrals, and we were doing it by hand. At that point, these things were lauded as saving humanity. Everybody was super excited. All the physicists loved computers and used computers.

For me, the thing about the current solving of math is, I don't know what that thing on the other side is.

Oh, yeah. No, but on the other side, I do think we've had that in the past. We had the AlphaGo moment. We did a podcast—not in this room, but—

But even before AlphaGo, remember when computers beat humans at chess? We had the IBM chess thing. Frank Chen and I did this podcast about AlphaGo, and we had to try to make people understand why it was a good idea.

I think it's actually pretty reasonable for us to ask the question: There are things that these things solve, and there's a lot of utility and value in that. That's going to move things forward, and when that tends to happen, people tend to be excited and get behind it.

There are these things you solve where I think people don't have as positive a view, and I would submit that's because it's almost like solving the problem had become the end, as opposed to the actual end. But maybe we should all step back and say, if you're really sad about something being solved, maybe it wasn't worth working on to begin with.

Right. So this course—you’re doing the sand mandala and your inner peace or something—but that's not moving the economy forward.

Right. Well, we're both systems people, but I'm actually an apps person. I know you're not. I don't—system. Yeah, we're both people, but, like, systems—

I absolutely think that the waves that matter are apps and, of course, the internet.

This same problem happened in 1995 and 1996 with the internet. It was very exciting, but most people just sat around saying, “I don't know what that does for me.”

Look, there's a great book out now called Steve Jobs in Exile, which I absolutely think is required reading if you're listening to this podcast. Kahney wrote the book, but it's with Ed Catmull, who was at Pixar, and with Dan'l Lewin, who was Steve's super-good friend and was also at Microsoft. This book is fantastic because it encapsulates all of this notion of building things that people actually need and that solve problems.

But it pointed out very clearly that the NeXT was actually the machine that Tim Berners-Lee used to write the HTTP protocol.

Right. So he actually—

A NeXT machine.

And he used the NeXT machine. That's interesting.

It's super interesting because nobody knew what this machine was for or what it did.

But then he built that, and still nobody knew what the machine was for or what it did, because he was like, “Well, it’s to find the phone numbers of other researchers and to share papers.” And I’m like, “Huh?” I think there was a great example of a Seattle-based company called CyberSlice. This was a dot-com thing that didn’t even make it to 2000, I think, but the idea was basically an Uber or DoorDash for pizza—only pizza.

You would order, and then they would figure out a pizza place near you and send the pizza. That was the launch demo for the NeXT onstage. They did that, and they actually had pizzas in the back in case it didn’t work. I should say, for NeXTSTEP or OpenStep, the idea was that this was showing what you could do with it. Literally, the reaction was like, “Wow, that’s really cool, but have you heard of the telephone?”

Yeah, yeah. Your point is not everything we’ve known how to use. So I’m a little focused on the fact that there was a solution on the other side that people were going for. You’re making a point that there are a lot of platforms that get built where that’s not clear, but clearly they—

Well, the spreadsheet was like, “I will show.” Here’s probably one of my last visual aids for today. The word processor came out in 1982, and people were using it on Apple II computers and this new kind of computer called CP/M, which is the origin of DOS. People were like, “I don’t understand why you just type.” Once you used a computer, the idea of typing really just didn’t work anymore. At law school, you have to show it now.

Yeah, I will. I’m just building up. These people at Harvard Law School brought in the first laptop. So that’s the first laptop.

Weren’t those called luggables?

Well, no. This was literally just called an Osborne, and it was the only one. So, as a guess—Erik, you’re a kid—how was the battery life in this?

Not long.

There was no battery. This giant case weighed 25 pounds; there was no battery in it. It just plugged in. But that was a trick question, because every time I’ve ever pulled mine out—I have mine from college—people are like, “Well, how long does the battery last?” It was literally the size of a sewing machine. It was bigger than a legal carry-on ever was, and that was my college computer.

In my senior year of high school, it got banned from Harvard Law School. Someone showed up to do their exams. At Harvard, they used to bring your typewriter to exams so that the professor could read it. Two kids brought computers in: one brought an Apple II, and one brought the Osborne. Then the school banned them.

Wow.

They just said, “This is cheating.” For all the reasons you could read—and I have the Time magazine articles and the New York Times—every article you could read reads like, “Don’t use the graphing calculator.”

Don’t listen to rap music, or don’t read—don’t know—jazz.

Don’t play Dungeons and Dragons.

Those are the same arguments that are going on now. Three years ago, I tried to get Cornell to use AI in freshman writing, and they just stopped talking to me.

Wow.

Here’s the irony of that: my freshman year, when I had this computer, I was, of course, the only person in my 90-person dorm with a computer. I had to get permission from the dean to use it to write my papers for freshman English. This was the fall of 1983.

That’s exactly where we are now on all of this stuff. You could also think of it as a level of abstraction, because no one’s going to college now without a computer. Can I just say no? I agree with you, but let me—

Right. Right.

Every once in a while, I’m like, “Well, maybe it’s a little different.” Here would be the argument: I don’t think, in the history of computer science, that I can recall that we’ve ever abdicated actual reasoning or logic. It’s always been a resource, right? It’s been compute, network, and storage. That’s what you’re providing, and then the human is putting in the high-level thing and using the compute, network, and storage to calculate the answer.

But all of the initial setup we’re providing—wherein, I guess maybe it’s not true for the internet—now I feel like you’re actually abdicating thinking in a way where you’re like, “Tell me the answer.” You’re not even really sure what the question is. Again, you could say Google was kind of like that, too, but it was still very much a social thing.

It does feel like that’s a little different than just going up in abstractions. Going up in abstractions, you still tend to have a deterministic system that’s a higher level of abstraction. You have a computer, and it’s the human being who’s defining everything about the problem statement. This feels a little different.

Well, it definitely feels different. I also think, for me, graphing calculators felt different. To me, graphing calculators felt like cheating, because the test question was, “Make a graph.”

And so that’s what’s going on right now: the capabilities match the test question.

Getting us full circle to what we were talking about with computers and mathematicians, my freshman year, a new product—a new thing—came out. It was Maxima, which was the MIT symbolic math package. This was a way you could literally type an integral into a computer.

I remember the first time I saw Mathematica. I was like, “This stuff is black magic.”

Maxima is, you know, machine-aided computation—what was it?

Machine-aided computation, symbolic mathematics, I think, was the—

That was the lab at MIT that started in the late 1960s and early 1970s, and that had started to sweep through. In my freshman engineering class, we had a version of it that ran on an IBM PC. It was called muMATH. We got our calculus homework, marched over to the engineering library, checked out a PC disk, and just typed in the answers to—

So you don’t think that was cheating?

And that was cheating. Let me just push on a little bit, because I tend to agree, but every once in a while I have moments of doubt. I don’t remember writing programs where you actually abdicated logic. If I’m writing a program, I’ll use a cloud database, storage, networking—whatever it is—but correctness and logic for the program are under the programmer’s control.

Maybe I’ll use a third-party library, but again, I’m choosing the library. I know the inputs, and I know the outputs. I feel like we’re entering this realm where you’re actually abdicating logic to a third party. You’re like, “Tell me the answer.” So maybe that’s just a higher level of abstraction, but it feels a little different to me.

No, that’s the debate. I’m all in on the debate. Here’s a Stanford example. During one of the AI winters in the 1980s—

Stanford—the biggest—

One of the AI winters—

One of the biggest things at Stanford—and we have a podcast on that from 15 years ago—was to combine new AI with the medical school. There were all of these projects to do medical diagnosis, chemotherapy, and that kind of stuff. I worked on one that was doing organic synthesis with a team at Harvard.

All of those were among the earliest examples of, “Let me turn over the decision-making.” In fact, that’s the whole era of computers in the 1980s: the dawn of what they used to call expert systems.

I remember very well. I just—

Your classes were all about this—

It just didn’t work.

Your classes were mostly about this stuff. You had a bunch of classes on it—tons of expert systems. I’ve had to build expert systems. I’ve written a lot of Prolog.

Exactly. So I very much understand it. I just thought that never really worked, and—

Right. So the big difference is that stuff was working, but now—

It does work, and we’re abdicating logic using these—

But it’s interesting to compare and contrast—

Even in the case of Prolog, you’re coding it. It’s algorithmic. You’re still providing the end state, and it’s just finding a way to get to the end state. Whereas here, you’re almost asking it what the end state should be. It just feels a little different.

I agree. I love having this debate, because I think so much of it boils down to the concern and the willies you get thinking about it. It’s actually because of the context we’re in. Think about all the stuff going on where people don’t want to build data centers. Two years ago, people were beating each other up, governors were racing to have data centers built. Ten years ago, it was, “Build a car factory in our state”—the one that billows smoke, is really hard labor, and so on.

The context really matters to these discussions. You can’t separate them from—

Right, but I just want to go back to this layer, and I don’t mean to—I just think—

Your entire career has been moving up layers of the stack, but there’s always been a computer layer of the stack.

Yeah, yeah.

You could always map it down to the next layer in basically a deterministic way—higher levels of compute abstractions.

This is the first time it feels like a different layer of the stack. Maybe this really is the next abstraction, which is more of a human-level abstraction that doesn't map directly and is actually different. So, whatever it is, starting with transistor logic, then going to compute, then hardware, then OS, then applications, then platforms—you've been moving up the stack that way. It could be the case that we're at a layer where we have to rethink fundamentals, because it feels a lot different to me than just, “This is the next layer.”

The big difference is—and we can argue this, debate it, or label it either side—that we're actually making the leap from calculating to imperative programming.

Yeah, which is where we've been, and where everybody is up to now. Then, for a brief time, we were in this mode where the data really determined the program. That was the first recognition of all the inference and everything. Now we're at this point where it's arbitrary, random, and statistical.

Right. So the way that I think about it is the following: In imperative programming, you know all of the steps. You write the recipe and it follows the steps. Then there's declarative programming. Declarative programming is the end state—

—which is this Prolog kind of thing, for people—

—or Datalog, or SQL—you know, the end state—

—but then the computer does all the stuff to get to that end state, and you can't really bound the computer time. You're like, “This is like makefiles. Here's what the end state looks like,” and it does it.

And this is like this new thing where—

It's almost like you don't really know what the end state is specifically, and you just kind of pray to the model god in the right words, and then it produces the answer that ends up being useful.

Yeah.

But I look at it, and it is stochastic.

That's a factual statement. But it's also interesting to think about it going forward in terms of whether that itself is the next layer of abstraction in how we think of computing.

Yeah. And it may be like computing, maybe. This is where compute and natural phenomena actually intersect pretty heavily, because the answer is produced from human output, which is language, which is kind of different than—

4. Rethinking Fundamental Assumptions About Software

If we do need to rethink some fundamental assumptions, what may that look like?

Well, I just think that people like Steven and myself have built these deep intuitions on how systems function and how they hit the industry, based on 40–50 years of watching this stuff. I just don't know. Things like: Will value go to the model or to the app? How much capital can you apply to this stuff? What classes of problems can you solve versus not solve? What guarantees can you provide? How does this impact productivity?

There are a lot of things that we've got intuitions on, and for me the big question is: Do we have to reshape those assumptions or not, and to what extent do we have to? Because the laws of physics feel a little bit different. I'll just give you 1 example. I've said this many times, and I think it's so important: 20 years ago, if you're a startup of 10 people and I gave you $1 billion, what would you do with it?

You would have ended up spending a ton of money on building and buying your own computers and things. If that's where you're going—if that's what you did, hire people, buy computers—you blow up. You wouldn't know what to do with $1 billion.

Oh, I see what you're saying. Yeah, yeah.

Yeah. Ten years ago, if I gave you $1 billion, you'd hire engineers. Right, right, right. What do you do? You write code.

You know, the important part of that is it's $1 billion.

It's not that you got money. It's that it's a huge billion. It's a ton of money.

If I give you $1 billion 2 years ago—

Because with $10 million, you'd buy a bunch of stuff from Hewlett-Packard and the money would be gone. For sure. For sure. This one is $1 billion. I mean, in software, you hire people, and then it's all about the FTE scale. The mythical man-month is very real.

Yep.

And right now, if I give 20 people $1 billion, they can actually use it. It's like we've moved the industry from an engineering-bound problem to a capital problem that's fundamentally very different. We've never been like that before. This is a law of physics where our early intuition—that all problems are engineering problems—starts to change.

So I think there's this very open question we should be asking, especially people like us: To what extent do we have to reevaluate our priors on this stuff? It's not just 1 level of abstraction; it actually changes the nature of capital versus innovation versus competition versus defensibility, et cetera.

That's a great way to think about it, because it forces you to think about a new model. It's also interesting that computing was capital-bound for the first 30 or 40 years. If you wanted to do something with a computer—

Such an important point. If you wanted to do something with a computer, your first step was, “We have to get one,” and then you couldn't—

You were capital-bound, then you were engineering-bound, and now we're capital-bound again, which is crazy. It's almost like you have to hop back 40 years.

Yeah. Mad Men goes through the scenario where the computer shows up at the advertising agency, and they run around trying to figure out and explain what it does for people. They also got a copy machine; they did the same thing. But it was interesting because they couldn't figure out what to do, but they were excited that they had the capital to acquire one, and it made them look like they knew what they were doing.

Five years ago, Patrick Collison interviewed Sam Altman on a podcast, and Patrick was saying, “Hey, you know, we've been in this era of lean startup, but for your projects—OpenAI, this sort of energy-intensive project, and a few other things—you've raised colossal amounts of money right out of the gate. Is that underrated?” It's sort of speaking to what you're saying.

Yeah. You know, it's interesting. Prior to AI, there was always this battle between Eric Ries and Ben Horowitz, right?

Yeah, yeah. See, lean startup, and then Marc and Ben wrote, like, the art of the fat startup—

—which basically argued, “Raise the money and go for it.” But there's always been this natural limiter, actually, which is engineering.

Yeah.

Complexity. That's actually been the reality.

And so Patrick Collison is right: We now have a discipline for taking a lot of money with small teams and using it productively. That's a very, very big change. I don't think we've internalized it—

Which also is incredible. That is why there can be so much optimism now, because although capital is scarce and it's hard to get and all of these other things, once you get it, the—

As we know, building based on people was also hard—just scaling that and doing more. Nine people can't do anything faster, and—

I'm telling you, my 10-year job was recruiting and giving these early teams money, then helping them recruit, and then waiting for 2 years while the engine— I think engineering just doesn't scale.

It also has implications for venture capital, because for the last decade people have been saying, “Hey, there's way too much capital, way too much capital.”

I just think this is such a crazy view. There's been this view in venture, this zero-sum thinking, which is funny from the people that shouldn't be thinking in zero-sum terms. They'll go on about, “Too much capital is chasing too few deals,” and all this. You're a venture capitalist; don't you believe in positive-sum stuff?

But if you look at the numbers, the more capital that flows into private markets, the larger the market gets. There are a couple of reasons. One of them is the one we've talked about: technical waves that are actually able to consume capital, like AI. But another is that if there's more capital available in the private markets, companies will stay private longer, so more value accrues on the private side.

I think capital going to private markets grows the TAM. It's not a limited TAM. Early-stage venture investors, who should think in terms of positive-sum outcomes, need to stop thinking about zero-sum.

Well, one way to think about that is—I'll bring it back to what I think your foundation enables, what I think is the most exciting thing—which is that we're really on the cusp of a wave of apps—

And the fact that now you can apply capital without also being a recruiter for 10 years and have output. Now, all of the world that's unserved by software—which is literally all of it. Everybody who complains about whether it's medical records or scheduling to go to a doctor, or my favorite, AI lawyers.

Nobody has cheered more at, “Oh my God, we’re finally going to be able to automate lawyers with AI,” which is the weirdest thing in a world where everybody is against everything except having more lawyers. All of this means that the person who has the domain experience—we used to love the venture-capital thing: “It turns out it’s really, really hard to build for commercial real estate. Wouldn’t it be great if somebody who understands commercial real estate built a software company?” But then they don’t know how to build software, so they should get a co-founder who knows how to build software and teach them about 20 years of commercial reality. It’s really hard.

But now the path from that kind of idea is a capital problem, and that’s a new level of abstraction. I remember my very first customer visit as a professional product developer. I visited a doctor who happened to have gone to medical school after majoring in electrical engineering and computer science.

Oh, wow. And he wrote a DOS program to schedule a doctor’s office. That’s amazing.

You’d think it’s just scheduling. It’s a calendar with hours. But this was 20-year-old me hearing this guy explain, “No, you don’t understand. You call the doctor, and you’re talking to a scheduler. They’re listening for keywords to decide: Is this 5 minutes, 20 minutes? Do they need the X-ray machines? Do they need the EKG?”

So they’re actually scheduling a blood draw and all of this stuff in parallel, not just the 10 minutes you need with the doctor. That’s what his software did. It took him years to bang that out himself.

Yeah.

But that’s the kind of thing that’s going to be able to happen now. That problem can get solved by the person who knows—

Code. No-code is finally here.

Well, it could really be that you’re not just building throwaway code that’s hard to use, but that everybody else’s abstraction layer is rising. You don’t need to design that piece of code. If you’re doing it for a phone, the phone’s abstraction level has risen, so you’re not building a text control; you’re not building UI controls, whereas 20 years ago, step 1 of building a company was building all of those things.

There’s a lot to how important this is in terms of what you’re able to do.

5. Incumbents vs Startups: Why the Innovator's Dilemma Still Wins

I want to talk about any other fundamental assumptions that might be interesting to revisit. How about incumbents versus startups? We’ve talked a lot about the innovator’s dilemma. Does that mean that now these startups are either the incumbents or have the capital advantage? Are they able to do more? At the same time, we’re seeing startups that you would think incumbents would just destroy.

The crazy thing is, if you had told me 6 months ago that you would ask this question, I would have said, “What advantages do incumbents have?” They have the same advantages incumbents always have: they have the capital, they have the cash flow, and they have distribution.

What’s crazy is that AI, A, solves the distribution problem—it solves the demand problem—and B, these companies are able to raise so much money that they’re actually on competitive footing with the Microsofts and the Amazons. I think we’re in very new territory when it comes to these new challengers versus the incumbents, specifically for these 2 reasons.

I think the distribution point is often misunderstood—how impactful it is. In the past, if you had a company and you wanted to get people to use your stuff, it was hard. You’d hire marketing, but you had no idea how much to invest or where, and you didn’t know what you were getting out of the return on investment.

But the demand is so unlimited for tokens and GPUs. Literally, you can just decide how much money you’re putting into it in order to drive top-of-funnel growth. The things that have typically been very, very hard for startups are much easier now.

I think this is why we’re seeing such meteoric growth from Cursor, Anthropic, and OpenAI. That results in capital access, and it has put them on even footing.

I think it’s to your point about how hard it is. Look, my whole life was managing thousands of engineers to build things that couldn’t be built anywhere else. That was the moat. To build an operating system, it was infinite.

You have to have one culture.

Yeah, I know. No, I get it. No, but he’s brilliant. Read “Steve Jobs & the NeXT Big Thing,” because you can really get a sense of building things up. In fact, of course, NeXT famously took the code from Mach at Carnegie Mellon and started from there. We couldn’t have done it completely from scratch.

But this whole idea of just how important it is to think through the domain-specific issues and how you disrupt people—there was an old joke at Harvard Business School when Clay was still with us. It was weird that they taught disruption as a theory in the business school, when really it should just be a fact in the physics department.

I love that. I was there in ’98 when he was writing the book and the paper and everything. That’s when I was teaching.

That’s nice. Great. I used to have, of course, this lore with everybody who’s from a big company in Silicon Valley: when you arrive, like I did, the theory is always, “Oh my God, we’re just going to crush all of these little companies.” You always think that when you’re at the big company, and then you realize they never get crushed.

Ben always makes this point, and Mark does in his movie: AWS actually put them out of business.

Right, right, exactly.

Because the startups don’t aim straight at the incumbents, and the incumbents just don’t pay attention. The incumbents are only interested in what the other incumbents are doing. Microsoft is worried way more about what Amazon and Google are doing than anyone in the startup space.

The elements of disruption that matter are the cultural ones of being a big company, and those are constant. Those are the laws of physics, so you just can’t change them. You can’t change scorecards. You can’t change field sales, go-to-market, compensation, organizational structures, legacy, and customers, because of the way you behave when you have 500,000 customers you’re serving.

There’s a bunch of stuff you just can’t do. You’re stuck. That is really the essence of disruption.

That’s why we’re at a magic moment where it’s not just that the culture is there, like it always is, but the startup ecosystem is very much a reflection of what happened during cloud. It was a whole bunch of stuff that you needed. Again, it’s this abstraction layer. If you’re a startup, you don’t have to go build a data center and build your own egress and call AT&T and do all of that stuff. Now you’re up and running in the first hours of your first day.

But the thing with cloud—I actually think you articulated it very well, and I use this because it’s great. With cloud, nobody thought they could put AWS out of business.

Right. Right.

You just kind of accepted the oligopoly and built on top of it. The question was, “Will they kill us in our little pipsqueak corner?” The answer was—

Will they just add you for free, or for some price, or match you or whatever?

I actually think that’s always been the question: Will Microsoft eat the app? But now these companies are actually taking on the incumbents.

And you make this great point, which I hadn’t thought about this way, but compute has been defined by these very complex, large engineering efforts: building a chip, building a system. What’s that—the “The Soul of a New Machine”?

Yeah, yeah, yeah. “The Soul”—well—

The solution—

“The Soul of a New Machine,” yeah.

Beautiful book, right? It talked about how hard it was to build these giant systems, building an operating system, even in the cloud. Jeff Dean and the people in that era were building these distributed clusters, and they were the first people who could figure out how to do that. Once you had that, it was a massive advantage.

These were massive engineering efforts that no startup could do. Now, for these models, it really is just capital access. It’s a very different set of laws of physics where, if you can amass the capital, you can do something.

Google is Google. They have all the data and all the intelligence, and their models are getting trounced by OpenAI and Anthropic.

Because of the cultural element.

I think people on the outside underestimate it until you’ve lived the cultural element of trying to do it. I’ll bet it’s cultural. It’s not a typical engineering problem like the ones they’re very good at, because they’ve out-executed. GCP is fantastic. That’s engineering, after all.

And I bet it’s probably hard to free up that much capital for one of these companies, honestly.

Well, all the big companies—you can tell from their earnings calls how constrained they’ve been about capital. You have Google doing its bond deal to move it off balance sheet, basically, in some weird way. You had the rumors of—I don’t remember which company—the rumors of, “Well, they’re rationing the tokens so that they go to the enterprise customers and not to the internal products,” and so the internal products are AI-starved.

And of course, none of the competitors to those products are starved.

I’ve learned to really appreciate it. Look, I fought and fought and fought to not be disrupted by the mobile platforms, like by ARM, basically. And Intel just didn’t care.

Yeah.

You know, I came down here, sat across the table from all the Intel leadership, pulled out the first Surface, and said, “Here’s our new computer.” They got very excited, and then they were like, “But what’s in here?” I said, “Well, it’s an ARM chip.”

Oh, wow.

The fact that I even brought one into the building was very, very tough. They just never felt that it was going to be anything more than a chip used in a printer. And I’m like, “But it’s the power, the graphics, the always-connected—all of this stuff.” The culture was that they did Moore’s law at Intel, and just like with Google, they did hyperscale.

So, like, if AI moves on-device—

Yeah. Yeah. Yeah. Of course.

That’s not what they do.

Yeah, sure. And, you know, with Microsoft, they were squeezed. They’re squeezed now. And I think you raised super interesting points about the opportunity, though, for—

For startups, with this capital inversion kind of thing.

Go raise capital and go after the—

Well, and it’s not just that. You’re also saying, “We’re actually not going to question you if you’re trying to raise that capital. We’re not going to look at you like you’re crazy.”

6. The Limits of Current AI Architecture & What Comes Next

You just look at the raises that are happening right now, and these companies have been quite successful as a result. The last thing: when Vishal came in and we had him on the podcast, he thought LLMs were a great achievement, but he was bearish on their ability to invent new discoveries, particularly scientific breakthroughs or things like that. I’m curious if you think the math progress is consistent with that, or what your latest thinking is on the limitations of the current model architecture versus, “Well, we need more…”

So, here’s my new view: I think we know exactly how these things work. You put a bunch of data in them. They’re stuck to that data. They can only do in-distribution stuff, and they can move along that manifold in a perfectly Bayesian way.

Okay, so we can say these words, and then the question is: What are the implications of that? What problems can it solve? I think it’s just so hard for a human being to reason about a digital artifact—in this case, the model—that was built with $5 billion. In the history of humanity, we’ve never created a single digital artifact that had that many FLOPs and that much data in it.

On one hand, we know exactly how it works from a mechanics standpoint. On the other hand, that is so much data and so much compute. Maybe all of that stuff’s already in there, and it can solve anything that you want. The conversation has moved from, “But how do these things work?” We know. “Can it do out-of-distribution stuff?” No. “Is there transfer learning?” Probably not. If I RLVR one thing, it doesn’t do something else. Are we at the singularity here? Probably not. I think many people agree that we’re not in a fast takeoff. We’re stuck with it being in-distribution.

But what I don’t think anybody knows is: You’re still putting $10 billion into that thing—what’s it capable of now? And if you consider this meta-economic machinery, which means the ability of Anthropic to raise lots of money and then pour all of that money into this thing to create something super powerful, I don’t think any of us can predict what that means or where that goes.

It’s a different conversation, but the question is the same: Will that be able to cure cancer? Maybe. But if you put $20 billion into something, maybe it can cure cancer effectively. That’s where I think the discourse has evolved and where it is now. I honestly have decided that I cannot predict what an artifact created using $20 billion is capable of.

I think it’s just so important for people who are deep in watching everything that’s new to admit that they can’t predict. And I think that’s great, because I wrote 58 memos on what the Internet was going to be, and I was wrong in a lot of them, by far.

But even this one is a little different. This is like: I take $20 billion and put it into a model, and then you and I look at that model and we can do whatever we want. I don’t think we can comprehend what that even means. It’s so many FLOPs and so much data. I don’t know what that’s capable of.

I think that’s really true. But I will say, on biomedicine in particular, the other half of my household is a research doctor who uses AI. We have a Spark at home, and she’s loaded—

Like a ton?

Like a Sun SPARC—no, no, an NVIDIA DGX Spark. Oh, yeah. No, no, no—not with a C, with a K. Oh, yeah. Wow. We were in old times there. I was like, “Not a Scott McNealy SPARC.” No, no. It’s a relic.

It’s all AI. She does brain stuff and surgical brain stuff—all AI. It’s so interesting to see, because what it really can do is see patterns that you can’t, patterns that only experience could tell somebody. But if there are 10,000 papers on a topic that’s part of her model, it’s just finding the patterns that no one has. That’s a pretty basic AI capability at this point, but it’s actually opening up solutions, problems, research directions, and things like that.

I will say, just for the record, this is not magic for discovering drugs, because the hard part of drugs has always been clinical trials—not candidates. It’s always been efficacy and safety. The candidates, since the ’80s, have been able to be developed faster than we could test them. It’s human patients, and it’s very, very, very hard.

Can I tell you something that I got wrong on this? I love the question that you asked, which is how our thinking has evolved on whether these things have capabilities in generality. I was responding to this Bostrom notion of recursive self-improvement and fast takeoff: You create one of these things, step back, and it takes over the world. I dismissed that because that’s clearly not what’s happening, and I think many people agree that’s the case.

But here’s what I got wrong: I did not know that we could effectively just continue to pour money into this. The scaling laws are holding, and I don’t know what it means to do, let’s say, a $100 billion training run—to have this thing that you’re putting $100 billion into. That money comes from this meta-economic machinery. They may want to solve cancer, but they may also want to create a weapon. Who knows?

This concentration of this many resources in a useful way is very new. I don’t think we understand the implications. You could reasonably argue that it’s very dangerous if you apply that $100 billion in the wrong way. I think that’s where this conversation needs to evolve to: less fear and more, “What does it mean to be able to concentrate resources?”

Well, I mean, you’re basically talking about exponential growth, and this is just exponential in dollars. We all know none of us can model exponential very well.

Yeah, we’ve never been able to do that. With a complex engineering project, you’re not tackling 1 problem with a lot of money; you’re just kind of building this machine. You’re right. You’re 100% right, and I completely agree. But I remember sitting in meeting after meeting at Intel saying, “We have 5 gigahertz, we have this many gigahertz, this many transistors,” and literally nobody knew what we were going to do with them all.

No, no. You’re building the machinery. I’m saying, in this case, if you’re like, “I want to exhaustively explore every protein combination,” we can just turn that into a money problem.

Yes.

It’s a great way to say it: We can take previously infinite problems, apply capital, and they become finite.

It just makes it a capital problem and not an engineering problem. It’s just a very different set of laws of physics.