AI 公司为什么会想放慢脚步？

As a business leader, there’s a tragedy of the commons. If you want to stop, if you want to go slower, why don’t you go slower? I’m competing. I want to win.

There are almost 2 camps. One believes that this is actually an engineering problem, and the other believes you have to slow it down.

Humans don’t respond fast enough to the attacks that are happening. You need to automate all of those, and most organizations are actually not close to doing that. Is RSI, or recursive self-improvement, that the labs are doing leading us there? That’s the big question.

Something Elon said was that this is some elaborate 4D chess, because on the one hand you’re saying all of humanity will die—

Mm-hmm.

On the other hand, you’re saying, “Hey, what do you want for your IPO allocation?”

For the first time ever, a company at scale last week said that they’re moving from the frontier models to GLM. Do you think that’s a trend, or do you think that’s just a one-off anecdote?

Thank you for being here, Ali.

Super excited.

So we obviously want to get to Databricks, but there’s a broader conversation going on right now about AI. Dario’s weighed in, Jakob’s weighed in, and Elon’s weighed in, but we want to hear what Ali Ghodsi thinks.

In terms of the topic, broadly speaking, of pacing the frontier, what is your strongest agreement with what’s being put out there? Where do you disagree? And maybe where is there nuance that’s not being captured?

Yeah, happy to cover it. Martin and I argue a lot, so—

So—

I’m sure that’s not going to take long.

We’ll try and rein it in this time.

Yeah, try to stay calm.

1. Leaders Should Not Freak People

I do think, first and foremost, that leaders have a responsibility not to freak people out unnecessarily unless there’s a really, really good reason. There are always different people in society who are at different places in their mind-space. Talking about existential risks and scenarios where all of humanity is going to be wiped out, I think, is irresponsible. It can tip a lot of people over and cause mental health issues.

Unless you have something that’s going to wipe people out.

Yeah. As I said, if there’s an actual reason for it, then that’s a different story. But I think that right now the existential risk is close to zero, so why freak everybody out? It’s not needed.

There are risks—we’ll get into that. That’s probably where we disagree. But first and foremost, I think leaders should not freak everyone out. If there are technical nuances in how we’re doing AI research, researchers can discuss that.

Yeah.

You don’t need to go on TV or blast millions of people on Twitter every time, saying, “Hey, I think there’s a 10% risk that all of humanity is going to be wiped out.” I don’t think that’s helpful for a lot of people. I think it causes a lot of harm for people who get stressed out and aren’t immersed in the nuances of all this and what it means. I don’t think we should do that. I don’t think it’s fruitful. It doesn’t really help anyone.

I think this is very, very true for the general public. My sister, who’s a schoolteacher in rural Arizona, texted me on Sunday and said, “Martin, should I prepare the cabin for the AI apocalypse? I’ve got water set up. When are you showing up?” She’s kind of a prepper anyway. I said, “Hold on. We’re not there yet.”

Yeah.

Clearly, this has spilled over to the populace, which I agree is unnecessary and has blowback.

I think there’s a second issue. I don’t know if you saw this while we were walking in here, but I was checking X, and Elizabeth Warren just talked about pausing all AI development. That, of course, is on the coattails of Bernie, who is also working with Bannon—Steve Bannon.

So, in addition to scaring people, the federal complex is now spinning up. I think that could actually be quite contrary to the goals of the message. There’s more than just public hysteria at stake here.

2. The Politics Behind AI Pacing

There’s a lot of politics going on, but I’m in all of these groups, and I see both sides. There’s heavy politics happening on both sides.

Oh, yeah. This is—

This is happening on both sides.

No, this is a—I think both parties, not including Trump himself, agree that AI should be constrained at some level, right? Even Greg Abbott said you can’t have data centers in Texas.

I’m talking about the other side of this argument as well. Let me give you an example.

Right.

I’m not talking about politicians. I’m talking about the fact that there’s politics happening on both sides.

Right.

There’s politics on the business side—people who want to see great IPOs and get returns on their investments.

Right.

They’re saying, “Don’t mess up my IPO. Can everybody just shut up so that we can get our money back?”

There’s that. They have resources, and they’re using them. There’s politics on that side, and those people aren’t sitting quietly and doing nothing. They can pull strings, and they have connections.

On the other side, there are all the people saying, “Okay, how do we weaponize this? This is awesome. This guy tweeted this. Let’s weaponize this one. Let’s plant this. Let’s pump these threads.”

No, 100%.

You know—

3. Pacing Is Not Safety

Totally. Let’s talk about the specific leverage point that everybody is using, because I actually think this is a classic case of a PR misstep. It’s not just the doomy-gloomy type of stuff.

Here’s the PR misstep, I think: It’s not unusual for industries to try to regulate themselves. It’s just not. Saying, “Security and safety are important. They’re important with every tech epoch, and we want to have some oversight”—that was very, very sensible.

Mm-hmm.

The problem is that it’s couched in this notion of pacing, and there are a number of issues with pacing. First off, it’s orthogonal to safety and security. You can slowly build a weapon; that’s not different from building a weapon. People don’t feel that it’s genuine because these companies have been at a dead run.

They’re still buying more compute to be even faster.

No, I mean, they just haven’t done it historically. But it also feels like this kind of milquetoast capitulation to the pause people. It’s like, “Well, you say pause, I say pacing,” which is almost like pause, but it’s not pause.

They chose this kind of flag to follow around pacing. But if you actually read the document Dario wrote—did you read the document Dario wrote? It’s a totally sensible document.

I read it, yeah.

It just has nothing to do with pacing, right? And so I honestly—

No, he does mention it. Look, I kind of disagree. There’s a tragedy of the commons. There’s this argument: “Hey, if you want to stop, if you want to go slower, why don’t you go slower? Why are you writing articles?” A lot of people are making that argument.

Yeah.

As a business leader, I understand that. There’s a tragedy of the commons. I’m competing, I want to win, and if you’re running ahead, I’m going to—

There’s also the market equilibrium, which suggests that pacing is probably impractical anyway.

Yeah.

Yeah.

I’m just saying that it makes some sense for people to say, “If you guys don’t stop this, this tragedy of the commons is going to continue.” I’m not going to stop racing because there are IPOs at stake and there’s competition at stake.

Yeah.

There’s also some animosity between the people. I’m not going to stop unilaterally. I’ll be a sucker. Why don’t you stop first?

Right.

So then they’re saying, “Can you come in and stop us?” But I think you could also make the argument that if you look at the Hugging Face–OpenAI incident, that…

By the way, I think these companies are great, and I think they are probably investing a lot of resources.

No, I agree.

But it's very clear—

Yeah.

If you read what happened, they weren't monitoring every token coming out. They were just running these RL experiments and then, after the fact, coming in and checking, “Okay, what happened?” So they should have paced. They should have been much slower in that particular incident, right?

I just don't want to quibble on syntax, but words matter with PR, right?

Yeah.

So let's take the Hugging Face incident. When I read that, you know what my reaction was? It was not, “Oh, OpenAI should pace.”

Mm-hmm.

It was like, “Dude, fucking secure your thing,” right? Do security controls like we always have done.

It is pacing, though. It is pacing.

It's not pacing. Like—

It is.

In the history of the internet, we had all of these things. We're like, “Let's pace the growth of the internet.” Let's do security, let's do control, let's do whatever.

It is pacing.

I mean, you could call it pacing.

You should do that, but it is pacing in the sense that, look, I face this all the time. I have a legal department at Databricks, I have a security department at Databricks, and they are always like, “Hey, slow everything down for everything, not AI.” Literally every little thing. “Oh, you're going to go on a podcast. Well, what's the script for it? What are you going to say? Let's review that.”

Yeah, go ahead.

And, you know, what's the legal position? You cannot say this, you can say that, you can say that. Everything you say has to be materially true. You cannot leave—

Are these pacing people in this room with us right now?

Who wants to do that? Yeah.

So you're running an RL experiment, you're training the next model. Should the security team be there, run all their monitors, and look at every— I mean, millions of GPU-hours of tokens were produced, and these agents were running amok in the sandboxes. It would have slowed them down significantly if you had the security team sit there—

Right.

—and look at all of the stuff.

I agree. I've—

Now, they're saying, “Hey, if we do that, it'll slow us down, and I'm not sure the other side is doing that. So can you guys come in and slow us down? Just tell us. Put some guardrails around us. We'll happily follow the rules and do the secure thing.” Otherwise, it doesn't make sense, because we'll get our butts kicked.

I just think nuanced, second-order words don't work when people are really afraid. You're like, “I'm going to pace, and therefore things like these don't happen.” I literally think we should have just been like, “Safety and security are paramount. We're going to put in these controls.” That's the important thing, and I do think that nuance actually got lost. If you look at the—

Well, kind of what Zuck said. Do you agree with what—

I thought what Zuck did was very pragmatic.

He was like, “Hey, we're going to pace ourselves. We're going to put in security—the reason we released this later is because of security.”

Mm-hmm.

Well, listen, what I thought was so great about Zuck is that he was very focused on security, safety, and self-regulation. Dario's first five words or whatever are like, “We need to pace the frontier,” right? It just puts you in a very different mindset than—

Yeah.

What he could have said was, “We need to secure the frontier.” Fine. “We need safety of the frontier.” At some level, I think they were trying to optimize both for the doomers, which calls for a pause, and for politicians, and they kind of didn't satisfy either. The doomers are like—

But the thing is, don't you think people are actually freaking out inside the labs? There are a lot of safety people that are genuinely freaking out. By the way, not all of them are EA people and so on. And then people are surprised, right?

Right.

But here's the thing: using the word “pace” doesn't help either of them. I think it's literally like you're—

Yeah.

You're trying to find this—pace is like the uncanny valley of making the doomer people unhappy and the policy people unhappy. The doomer people are like, “That's not a pause, this is pacing.”

Yeah.

And everybody else is like, “Well, this isn't real. You're not going to do it anyways, and you're not focused on security.” So, independent of what we should do, which we should talk about—

Yeah.

I just think the way it was presented was bad, and it didn't work. That's why we're having the blowback.

Yeah, some of these guys are not trained PR people, and yes, some of the stuff—

Well, they have to—

I agree, I agree with—

No, but they do. They have trained PR people.

I agree with the core premise that we shouldn't freak the public out. I think that—

Or the government.

Existential risk right now is close to zero. But let's talk about the core thing, which is the fact that anyone who's doing big reinforcement learning runs and is giving it a reward function—so unleashing, saying, “Hey, here's 10,000 agents and here's $100 million. Let's put them in parallel and let them run on a gigantic cluster for a month or two. Try to solve anything.” It doesn't need to be a security thing. It could be: do anything. Solve this math puzzle. Really bad things can happen. Really bad things, meaning things get hacked. Cyber is the primary one.

Yeah.

Yeah.

Right? That is real, right? I think making it, “Hey, this is an existential risk,” and so on, was a mistake. I think it's not good to scare the public that way. I think it's become something that everyone—not just your sister, everybody around the planet—is now talking about. I've had all kinds of people who never care about this stuff, and find this extremely boring, ping me and say, “What do you really actually think about this? This is really important in my count, so now I'm starting to worry about it.” Then it becomes a political issue, and we have elections here coming up. But there are elections all around the world, so you're going to see they're not going to sit still in other parts of the world either. I think it's our responsibility to talk about this in a balanced way and actually expose the risks. I think that superintelligence, that idea from that book, is very, very far away. I don't see any evidence that we're actually marching toward that or that it's going to happen.

Same.

Apparently some people at the labs are freaked out that maybe there's progress toward that. I think it comes from RSI, recursive self-improvement—the models improving themselves. I would love to understand—have they seen something we don't know? There are kind of 4 criteria. If those 4 things are happening, I would love to understand them. The first is: does the next model require fewer resources, fewer GPUs to train, and superlinearly—not just by a tiny little bit?

Mm.

The next model takes less time to train as well—that's the second condition.

Mm.

Third, the accuracy of the model, the intelligence, is increasing. And fourth, we can do the first 3 again and again and again.

All of those at the same time, right?

All at the same time, not just any of them. Yeah.

Okay.

All 4.

Okay.

If all 4 are happening, then you can imagine a way where you can—

You know, because if any of them does not happen—for instance, if resources are constant—then that's okay, because we're going to run out of hardware, so then it will pace itself. We will not have enough hardware to do that.

Not enough GPUs.

Yeah.

Right? Time, same thing. So if you just mean that the software is writing itself, we're already there today.

Right.

Like, 90-some percent of the software in Databricks is written by AI. Does it matter if the last few percent is also written by AI? No, it doesn't matter really that much. But if you're getting these 4 conditions, then you might get a speedup where the next model, let's say, takes half the amount of time and half the resources, and it is more intelligent. And you keep doing that, then you might end up in a situation where—I don't know.

By the way, I don't even know if that necessarily leads you to superintelligence per se.

No, because then that could still converge, yeah.

But it could, so then that would be more risky. It would be nice if they could share all that data and we could shed some light and transparency on that.

I actually think it's a great breakdown that you have.

I don't think anyone is using that as a definition, actually, right?

It's a great one.

I think there is a little bit of people freaking out about, “Oh my God, emergent behavior, now it's creating itself,” and so on. But I think, as I said, for a lot of people, their definition is just, “Hey, if I'm not even coding anymore and it's coding itself.” But I think they're conflating, “Hey, what's my value, and is it scary for me?” with, “Hey, that then means we'll get that superintelligence that in 2014 was theoretically hypothesized by Bostrom.”

Yeah.

Yeah.

Yeah. Well, your point on compute, though, is a really good one that's missed in, I think, a lot of arguments on RSI, right? Because as far as we can tell, the minimum threshold for compute needed to train a good model just keeps going up.

Yeah.

It was 100 million, 1 billion; now it's probably like 5 billion.

To train a model now?

To train, like, a frontier model, right?

5 to 10 billion, yeah.

Right, right, exactly. Versus—

Million or billion?

Billion.

Billion.

Billion.

Billion, yeah.

Versus your second point.

Yes.

It's very expensive.

Yeah, frontier models are very expensive.

To replicate the frontier 6 months later is about 1/20 the cost.

No, I think Sarah has a great point, which is—this is a good argument against this whole thing. The next model—first of all, each of these labs does only 1 or 2 such runs a year, and they take—the opposite of the 4 criteria that I mentioned, right?

Right.

It's going to take more resources, involve more humans, and be even more brittle.

Right.

They have to build out the data centers. I mean, the labs are not necessarily doing that, but others have to build the data centers, and they have to be gigantic. They have to get the GPUs and networking right. They have to do the engineering to make sure that they can tolerate it, because every order of magnitude more GPUs you cram in there, you now have to worry about errors that you didn't have to worry about before. So you have to increase the robustness of the process. It's a very brittle process.

Yeah.

And if it fails, you've squandered so much money, so they're very, very careful with that run, and there have been multiple runs that have been botched.

Yeah.

So it's the opposite of that: the next model is faster, cheaper, smarter, and there is recursive improvement. It's the opposite. It's taking longer, it's more brittle, there are more people, and it's harder to pull off. So I do think that is true.

Yeah.

With respect to RSI, with respect to actual cyber risks and things—

Oh, it's very high. It's very high.

Getting hacked, we need to take it super seriously, yeah.

So I'm going to have another litmus test here. I actually love your 4 criteria. I was literally just waiting to argue with it, but I actually think this is very good. So let me give you a black-box one. When you're dealing with these dynamic, adaptive systems, what are you going to believe? Are you going to believe the numbers or your lying eyes, right? So I think you kind of have to go to the numbers on these ones. What are the numbers to look at? I really think you should just basically—and maybe going public is the right way to do it—if these companies continue to grow, reduce the number of people and the amount of money that goes into them, then I would say something is definitely happening here. I do think you can actually black-box this and take a look. But none of those indicate that they're hiring like crazy, they're—

But that's not fair.

They're burning money like crazy.

That's not fair, because companies are not necessarily efficient, right? So what if you have— I mean, OpenAI itself was doing a million different activities. A very small—

Well—

A very small—

I just have another— I agree. Just another litmus test. We have kind of 2 litmus tests. We have your litmus test, which I think is great, but then you would actually have to have a way to instrument it.

Yeah.

And then we should have the black-box litmus test. I mean, listen, if Anthropic in 2 weeks is 12 people and they continue to grow and they're putting out models at an increasing rate, I think we should probably take notice of that.

That's a sufficient criterion, but it's not a necessary condition, right?

Totally, yeah. Yeah.

So I'm just saying that there could be that. Really, the right way to do this, then, is to look at—okay, the pre-training and the post-training that's being done, that's really necessary.

Yeah.

Because they have so many resources that they might be doing a lot of other stuff they don't need to do.

Fair enough.

But they're doing it because they can just hire—

Right.

The people, because they have infinite money—

Right.

So really, the people who are training the next model—is that team tiny, tiny, and actually getting reduced? Are they doing less and less work, with AI doing the post-training, and then all just using fewer GPUs? That's not the case.

We've had this argument many times as an industry before. I remember when we learned how to really cluster computers, because the mainframe was actually kind of limited by things like memory coherence. Remember that?

Yeah.

You could only make it so big.

Yep.

Then we went to the client-server era, and we didn't have that problem. Then we started creating supercomputers, which were basically just clustered computers.

At some point, the internet happened.

And that was also roughly when GPUs started getting good. Do you remember that we would actually export-control PlayStations? Because we were worried that Saddam Hussein would use them to do simulations. The arguments were very similar: these things are getting infinitely powerful. We were using them to simulate nuclear weapons—which we were; I was.

Mm-hmm.

This stuff has existential risk. Actually, they didn't use those words, but this has the potential for nuclear weapons or whatever, and we should stop it. None of that came to pass.

Mm-hmm.

So I think a very reasonable discussion is: Is this time different?

Yeah.

Yes or no? I don't have an answer to that. But I'm a VC.

Well—

But, you know, you're a—

Yeah, I mean, look, I think I was—I'm not old enough to remember, so ignorance is bliss. So I can take this—

Wait.

So, no, no. I don't recall—

Wait. Huh, huh, huh.

PlayStations being illegal and Saddam Hussein being—

What?

I just don't know. Or maybe I'm just saying—

How old are you? This is like—

Maybe I'm just ignorant.

This is like 1999.

Okay. Maybe I'm old and ignorant.

What the fuck?

I mean, you know—

Maybe they didn't care in Sweden. Is that the—

Maybe it's just amnesia from age—

Is that the conclusion? Jesus.

But—

Sweden—

Whatever it is—

Sweden doesn't care about export controls in the US.

Yeah, whatever it is, I think it's at a different scale now, right? With AI and what we're doing, the pace of development and so on. They are freaking out at the frontier. I do think cyber is actually one of the biggest ones—

Yeah.

That we're going to see, right?

Sure.

Because there's just so much infrastructure on the planet.

By the way, way more than it was—whatever Saddam Hussein or Xbox or whatever it was—

Yeah.

We've just interconnected way more things, and they're dependent. The planet just looks different today in terms of internet and technology dependency and interconnection than it did 30 years ago.

So I just want to make this point: there's so much infrastructure that's insecure. If you're going to unleash these agents, they're going to find loopholes, they're going to find exploits—

Oh, they—

They're going to break in here and there.

Yeah.

So this is a real risk.

So why do you think—

And by the way—

Sure.

This time it didn't do that, but you could imagine a scenario where it starts hopping. It takes resources and starts executing itself elsewhere, so it spreads like a virus a little bit.

Yeah. So—

That's a real risk.

So this is pure curiosity. I promise I'm not trying to be a foil here, but why do you think we just haven't seen very much, then? Again, I'm much older than you. I remember very well when the internet came out. By this point, we had literally taken out 10%, disabled hospitals, taken out critical infrastructure, and caused tens of billions of dollars in economic damage from worms. All of that had already happened.

And to your point, we had much less buildout. Less of the economy was on it. AI has so many people who want to find risks and threats. We're running so fast, so much money has been poured into it, and we haven't seen anything commensurate with the early days of worms. What is that disconnect?

Mm-hmm. Yep. Look, I do remember those days.

But it's the same time.

Yeah. I would just say that I am sleeping well at night, and I don't think there's existential risk right now. I do think there's a lot of infrastructure that needs to be secured.

We have a product in the market, in the detection market, LakeWatch, that helps you do detections. The space is moving so fast because you used to have these SOC teams—security operations center people—who would look at what intrusions were happening, how you were being attacked, and so on. Now the humans just can't keep up.

Yeah.

So the whole cybersecurity space is being transitioned to fully automated detection using agents. If we don't do that, I mean, we're rushing. The industry is rushing to do that super fast.

Yeah.

If we don't do that—

Yeah.

—you will start seeing those kinds of things: sites going down, whole systems stopping for a while. There will be consequences—not existential, but economic damage and people getting hurt could happen.

We just have to race very fast to do all of those things. Humans don't respond fast enough to the attacks that are happening, so you need to automate all of it. Most organizations are not close to doing that. The banks are doing it.

Right.

Some of the people who are extremely security-conscious are doing it. But most of the industry today is running with old-school security operations centers and people who wake up every day to hundreds of detection emails that have fired. Many of them are false positives, so you can ignore them. But some of them are not, and they just don't have time to go through them.

You need to have automated threat hunting, where you're actually attacking your own systems automatically with agents and so on. It hasn't happened. If we just say, "Hey, this is like the internet in the early days," bad things are going to happen. So there is a race going on.

You know, I was actually very surprised. Earlier this morning, I was on a call with a founder. I feel like you and I are pretty close, and we talk periodically. I feel like I know a fair bit about Databricks. This founder was basically saying, "Yeah, we're doing all of this observability, agent threat detection, and we're using Databricks." I didn't even know that you had this offering, quite frankly.

Just from an education standpoint, how extensive have you gotten in agentic AI observability, security, and safety?

Yeah. We gave a talk this year at RSA, actually with Ben Horowitz. The issue is that data and AI are blending with cybersecurity. These 2 markets are collapsing—

I don't know. Because I think—

You know?

At least—

The reason they're collapsing is that it used to be like, okay, you have data and AI—the kind of stuff Databricks and companies like that used to do. You have a bunch of data, you run AI and machine learning, and that just lives separately. Then you have the cyber world, where you want to detect if something bad is happening—if bad people are trying to hack you or doing things. You need to detect that, okay?

But now, on the data and AI side, we have agents running. Internally in the company, people have agents running, and the agents are also doing things with other people's agents. They're producing a lot of data.

Yeah.

Logs, trails, fingerprints that are being left. Now you have these agents internally doing that. These worlds start merging more and more. All the data that's being produced needs to be analyzed, and the scale at which you need to do that is many, many orders of magnitude greater than it was just 1 or 2 years ago. Things have changed dramatically.

In 2018 and 2019, the time it would take from a CVE vulnerability being published until you saw it actually weaponized in the industry would be 2 or 3 years. That went down significantly by 2022, but it was still 8 or 9 months.

Yeah, yeah.

So that's fine. You have 8 or 9 months from a vulnerability to weaponization. That was 2022.

Yeah, yeah.

Now, if you look at this curve from 2022 until now, it's down to basically hours.

Yeah, yeah.

It's down to basically no time. Things get immediately weaponized. So you need to do it in an automated way, with a data and AI platform approach. These markets—

Yeah.

—are just going to collapse, actually.

Yeah, yeah, yeah. I agree. So it's a very specific question. To pull back on the existential X-risk discussion, it feels like there are almost 2 camps.

Yes.

There's 1 camp that believes this is actually an engineering problem, and companies like Databricks can solve it through products, engineering solutions, and services. As an industry, we just need to solve that problem.

Yeah.

And there are others who believe there is no engineering solution. You have to slow it down. You have to use regulation. It's more like a nuclear weapon, et cetera.

Does this mean you believe it is an engineering problem, or are you not quite comfortable saying that yet? Because what it was actually like was, "Why would we even buy Databricks, man? Let's just put this stuff in a national lab."

Just pace it—that's the only solution. I know—

But pause it—

Everybody just paces themselves a little bit, then we'll be fine. The bad guys do—

By the way, there is no line of inquiry that ever gets to pacing, I don't think. I think it's like—

Yeah.

—you pause it, or you—

Yeah.

—solve it.

What we've learned here is that Martin really hates the word "pacing."

I hate the word "pacing."

I will never use that word with you again. Clearly. Duly noted. But, yeah.

Frustrated Martin.

Yeah. Is it an engineering problem that can be solved by engineers, or is there more to it? I actually think, which problem are we talking about? There are 2 separate problems that I think are being conflated.

Yeah. Mm.

There is the superintelligence problem.

Yeah.

You know? A lot of this comes from Bostrom's 2014 book, Superintelligence: Paths, Dangers, Strategies. If you look at the definitions, I think people don't have clear definitions of what superintelligence...

If you read his book, those definitions are kind of crazy. I think what he had in mind when he said “superintelligence” is AIs that—I don’t know what the examples were. Something like—

No, everything could do anything.

Yeah. Something like they write a whole PhD thesis with novel, peer-reviewed material in a couple of seconds.

Yeah.

And they can do millennia worth of thought instantaneously.

Yeah.

This is the level of how fast they are and how intelligent they are. It’s like—

They can learn.

Yeah. It’s just many, many, many orders of magnitude.

Many.

Right? The scale of the problem is just completely different.

Yeah.

So if such a thing exists, do I think that’s just an engineering problem to solve? No, I think that’s actually very—if such a thing would happen, that would be very existential.

Of course.

And that’s what everybody agrees on. So I think that’s being mixed with the fact that now we have agents that are nowhere near that. There’s nothing like that, and we don’t have anything towards that path right now.

Yeah.

But these agents are capable.

Yeah.

You can do something with them that you could never do before in the history of mankind. I do think an inflection point has happened. Something has changed. We have good security researchers at Databricks, but I could never say, “Let’s get 10,000 of them in a sandbox for a month and have them do $100 million worth of salary-wage work.”

Totally.

We can do that now. We just turn on a button and we can get 100,000 of them. Or mathematics: we can say, “Hey, we want to solve a conjecture.” Let’s get pretty good mathematicians, but let’s have 10,000 of them collaborate, and then you can make very fast progress. I think this leads to all these cyber risks. I think cyber is the major problem here.

Yeah. Mm-hmm.

This, I think, you can solve with engineering. We are working on it, and many others are working on it. There are still risks, but they’re not existential. I think we should do it.

There is the superintelligence thing. That’s the thing that could write a novel PhD thesis or reason intuitively in 11-dimensional space physics instantaneously without writing anything down—something humans can’t do.

Yeah.

That kind of superintelligence.

Yeah.

The question is: Is RSI, or recursive self-improvement, that the labs are doing leading us there? Are we going to get there?

Trying to do.

Trying to do. Is that what’s going to happen?

Yeah.

And how fast is that going to happen? That’s the big question. They’ve suggested that we should have inspectors come in and look at what we’re doing, and that’s a good idea. Have them go in there and get the data. I would love to—the question is, who are the inspectors?

Right.

Because you can stack that.

Who do you think, Martin?

Yeah. There are a bunch of people on either camp. Actually, I wouldn’t care if they were the inspectors, but I would not be very impressed by what they say because they’ve already made up their minds even before they go in there.

Right. Exactly.

But let’s say, as an example, if Yann LeCun, who was one of the inventors of this deep neural network technology—one of the pioneers—said, “Hey, there’s nothing to see here. There’s no risk,” paraphrasing him. “This is nonsense. Keep on going. Go fast, fast, fast.”

None of us would believe it.

I’m putting words in his mouth. I mean, those aren’t his exact words. Now, if he was one of the inspectors and he went in there and had a look, then came out and said, “Hey, I’ve looked, and it’s just what I said. There’s nothing to see here. Just keep going,” I would feel very good about that.

Yeah.

I would say, “Okay, I would feel very—” Or if he comes out and says, “Oh, my God,” and he’s wobbling and has changed his mind a little bit, that would also have a lot of interesting signals. I think it comes down to who we pick as inspectors, and I think it’s a good idea. Let’s have some of them and pick—

What—

Pick a diverse set of people so that we can get different, nuanced points of view.

What do you think about this kind of Elon Musk view, which is less third-party? It feels like there are 3 proposals. The OpenAI and Anthropic one is a third party.

Yeah.

The Elon Musk one, as far as I can tell, is that the labs cross-check each other, like peer review in science.

Yeah.

And then the Mark Zuckerberg one is police yourself, right?

Yeah.

What do you think about this middle one?

That they should pace each other—evaluate each other? I mean—

Like I evaluate you, you evaluate me.

I think if we have boxing matches in the ring, the boxers should just be the judges of each other. Would that work? No. They would scream “Foul!” all the time. “Foul, foul, foul!” The moment the other guy puts out a great model, it’s, “Big superintelligence risk.” Absolutely. They have not been responsible.

When vested interests are at play, there are IPO plans, and these 2 companies are so competitive—and they have this history between them—they’ll be very fair to each other, I’m sure. That’s why you need a third party, right? Why do we have judges in the world at all? Why do we have third parties at all? Why can’t people just figure things out between themselves?

So, I—

They should try. If they want to do it, they should try, but I’m skeptical that they wouldn’t just be biased in multiple ways to self-judge each other.

I think it’s a fair point.

So, I have to ask, Ali, do you think—something Elon also said, I think it was at the All-In Summit—he was like, “This is some elaborate 4D chess,” because on the one hand you’re saying all of humanity will die.

Mm-hmm.

On the other hand, you’re saying, “Hey, what do you want for your IPO allocation?” Right? That is probably a more cynical view, but how do you reconcile that? The dissonance gets a lot of people. How do you think that gets reconciled?

Look, I think all of these things get mixed. I think there are people who are freaked out, and I do think there are people saying, “Hey, if there was regulation that would pace us”—sorry to use the word—“that would be good for us.” Right? “That would be good for us.” But I also think that people have vested interests.

Yeah.

These things—and usually people figure out a way to get all of these things to align harmonically in their heads.

Mm.

Do I think there has been a tendency in the past, in general, to use marketing stunts by saying, “Oh, my God, this latest model is so good that I trained it. It’s unbelievable. It’s almost scaring me”? And then—

Mm-hmm.

—the whole world starts focusing on it? Yeah, there’s been that kind of marketing going on.

Yeah.

But at the same time, as I said, the time from CVE to actually—

Right.

—weaponize an exploit has been going down from years to minutes now, in just 3 or 4 years. So it’s real. The cyberattacks are real. But there’s also a great marketing ploy: whenever you train a new model, make lots of noise around how much of a crazy risk it is to the world. It helps you, right? Maybe they’re not in contradiction—these things.

Maybe. And—

So, I mean, you and I are networking folks, and there’s a long history of forming third parties to help arbitrate things, right? Like the IETF or—

Yes.

—the IEEE or even ICANN.

I see where this is going.

No, no, no. My question to you is: I think this is actually a very sensible proposal that they have.

Mm-hmm.

I actually agree with you. You probably want to make sure it’s independent, which it’s not clear right now. Whatever.

Yeah. And there are going to be a lot of arguments about who you put there, and everybody’s going to disagree.

Right, right, right, right. But you said, “Why do we have judges?” The state stepping in is actually quite a different thing from industry self-policing.

Mm-hmm.

So at what point do you think it makes sense to consider federal involvement? Or do you think now is the time to consider actual federal involvement, as opposed to more industry self-policing? These are just very different approaches.

They are different, but they kind of bleed into each other. For instance, FINRA is not completely independent. It is self-regulatory, but it’s linked to the government. I think these things will bleed over.

Do you think that they evolve into—historically, they’ve—

Yeah, I mean—

The industry self-polices, and then it evolves into regulation.

Look, if they’re saying there is existential risk, and they’re saying, “Come police us and regulate us,” I think it’s very hard for regulators to say, “No, we’re not going to do that.” So far, they’ve said that, but I think that’s not going to last very long.

It’s so funny. Wasn’t it David Sacks who said, “I’ve never had a CEO ask us to regulate them”?

And the CEOs are saying, “I’ve never had a regulator say no to that.”

They know. The reality is that the actual metapolitical machinery is already in motion, right?

Yeah.

Everyone has a talking point. Obama has came out. It is a major issue. Do you think there’s a reality that it’s too late, that this will be a major issue in the midterms, and we’re actually going to have heavy-handed federal regulation? Is this all going to be paused, with Anthropic going into the DOE? Are we past that point, or do you think we can actually end up with sensible self-policing regulation?

I think we should strive toward doing the right thing.

Yeah, of course.

I think there are still some degrees of freedom in how things evolve while there’s still time. And, yeah, you’re right that, largely, you have these companies where we’re pumping in so many billions of dollars—

Yeah.

The way reinforcement learning works is that you give it a reward function that’s verifiable, like, “We’re going to solve this math problem,” or this narrow area of programming, and so on. We pour so much money into that, and you can get quite good results in that narrow kind of domain. That doesn’t mean that you’re getting that superintelligence.

No, but you can even trick yourself into thinking that fewer inputs are giving you a better outcome just because you’re running so many experiences and you’ve thought about it so much, right?

Mm-hmm.

But it’s actually very hard to do a closed experiment this way, given how many resources are going in.

Fundraising is going up astronomically, to your point.

Yes, yes, and they burn the money.

That’s why these companies are going public, right?

They burn the money.

Yeah.

They would stay private. I mean, as someone who runs a private company at scale, I think they would prefer to stay private otherwise. Why are they going public? Because they need the capital, and they consider the scaling laws and the capital to be a strategic advantage.

Yeah.

But I would say, let’s go back to the 4 things that I listed. If those are true, would you want to know about it, and would that be worrisome? That could get out of hand. Now, there’s no evidence that those 4 are happening, but if that was actually where it was headed—

Yeah, yeah, yeah. I actually think understanding any sort of self-propelling property in any system is important. We’ve done this in the past with dynamic systems. We’ve done this with compilers. We did this with all the research on nanotechnology. It’s been a common interest of ours.

Yeah.

I don’t think it’s new as an area of interest. We should continue to have that interest. I just think the fear is that these particular systems are metaeconomic systems that are so complex that the risk is that you’re claiming you’re seeing it when you’re not seeing it.

Yeah.

Mm-hmm.

And I think a lot of that’s happening right now.

Yeah.

But, of course, if you see it, you want to know.

But it is fair to say that the labs are now focusing a lot on RSI, and that’s where they’re headed next. Maybe they’re just unjustifiably worried themselves, just like they were worried about GPT-2, right?

Yeah.

They were like, “GPT-2 is world-ending,” and then it wasn’t. GPT-3 and GPT-4 came out, and it was like—

I don’t want to quibble. A lot of the time when they say RSI, they’re actually talking about autocatalytic effects, and autocatalytic effects have been in our industry for a very, very long time.

Yeah.

For example, there’s no way you can create a computer chip without a computer chip. You just cannot do it.

Yep. Anyone with a computer science degree would say you’re right. A compiler writes its—

My own compiler—

—own compiler.

Well, that becomes closer to RSI.

Yeah.

The steam engine was autocatalytic, right?

Yeah.

My full-time job is people coming out of labs and starting companies, and they all say RSI because everybody says RSI. Maybe 1% of those are actually RSI. They’re more like, “We use AI for data cleaning. We use AI for making GPU kernels.”

Let’s make the distinction. So you’re saying it’s basically a catalyst in the sense that they’re using AI—

It’s—

—to speed things up.

It’s autocatalytic, yes.

Yes.

So I would say, of what we hear, the RSI—

Like a model trains another model, right?

You’re using a model to build a GPU kernel.

Yeah.

You’re using a model to do—

Yeah, exactly.

—data cleaning.

Yeah, yeah.

Just like I use a computer to design a computer.

Yeah, it’s a catalyst.

Right.

The internet was autocatalytic because it allowed people to collaborate remotely. So I would say—this is anecdotal—90% of the calories are autocatalytic, which is 100% what you’d expect.

And that’s been going on for a while, though. I mean, that’s not even new.

Yeah, yeah, but you would expect that.

But I would say that there is also now a focus on: can we get the model to train itself? This is kind of like the autoresearch that Karpathy did, but now they want to do that at scale.

There are teams that do that.

There are teams doing it.

It is not nearly as many calories as you would expect. I feel like I have a good sample of this because they all come and talk to us.

Great. Can we maybe have you be one of the inspectors? Can we get all that data and have all of us look at that data? Maybe there’s nothing to see here, you know? I personally don’t think there’s a very high probability that those 4 criteria are happening.

That’d be incredible.

Greg Brockman went on the pod this week and said—

Yes.

—we’re in the AGI era.

Yeah. So now I ask the same question after he said that, and now everybody’s saying we have AGI.

We feel validated.

But people have, for a very long time, when I asked this question, said that AI is smarter than most of the people around me most of the time. That’s been the case since almost Q3 or Q4 of last year. Then I asked them, “How many of you have hundreds or thousands of agents that you are managing, that are coordinating with each other in swarms and negotiating and automating your life and everything around you? And if so, raise your hand.” Almost nobody raises their hand.

Of course, Martin has done that at home. But—

No, no, most enterprises are on Microsoft Copilot. That’s the extent of their AI—

Most—

—from what we’ve seen.

Most enterprises I talk to, when I ask this question, they’re like, “No, we don’t have any of that.” So we’re like, “Okay, what are you doing, then?” They’re using a chatbot, asking questions from a chatbot. That’s basically a very glorified, efficient Google search of the old days. The result is just a faster Google search.

Then coding is happening, so people are using it for coding, though we can discuss the ROI there. But there’s no agentic work that’s automated across the whole enterprise. That has just not happened.

So then why is that? I think the real reason is, if you actually look at it, the models are smart enough, but they just don’t have the context that exists inside any organization. They haven’t been in every meeting. They don’t know what’s in everybody’s heads. They don’t know all the processes.

There are always a couple of employees who know everything in every organization. You tap them on the shoulder, and everybody’s like, “Oh, my God, what would happen if he or she quits?” They don’t have that context. If you just fused that and gave that context to the AI models—to the frontier models today—I think there are so many productivity gains you could get for any organization on the planet.

For that, we actually don’t need smarter models. We don’t need a smarter model that can solve Navier–Stokes or mathematical conjectures, or do better on Humanity’s Last Exam. We need it to just go from 60% to 70%.

Yeah.

None of that is needed. I think people are very upset about, “Oh, if we pause the frontier.” But actually, if the frontier doesn’t advance, it doesn’t matter for the vast majority of organizations on the planet. They’re just so far behind in the adoption curve of actually automating things—

Yeah—

—and getting value out of this stuff.

But it would be disastrous to the labs because the price of intelligence is dropping asymptotically. I think it’s going down by one-tenth every 6 months or something like that. So that would—

Oh—

—dramatically change their businesses if you weren’t pushing the frontier.

Yeah. But this is what we should focus on, right? There are 2 sides.

Oh, agree.

We discuss the costs a lot here. There’s a cost-benefit analysis that we should do on everything, right? We’ve discussed the costs a lot here: Is there an existential threat? Is there cyber risk? Are there things we should be worried about, and so on? That’s the cost side.

4. The Benefits Are Real

What’s the benefit? Now that this has become a public thing and the whole public cares about AI, they’re asking, “Well, what’s in it for me? What am I getting out of it?”

It seems like nothing. Why are we—

So how do they get there? What are some of the use cases you’ve seen to date—

Mm-hmm.

—that have maybe surprised you to the upside?

Yeah. First of all, there’s so much worry about existential risk and so on. I think a lot of people just don’t know what a cool use case is, where people are actually doing interesting things.

We have a lot of use cases that are fascinating. One that I like is Crisis Text Line. They actually use large language models with us to detect if teenagers want to self-harm or attempt suicide.

Wow.

Interesting.

You know—

That’s awesome.

Oh, wow, yeah.

That’s an awesome use case.

Super cool.

It actually saves lives. That’s a great organization, and it’s doing amazing work.

Wow.

Another one that’s kind of interesting is the Omnipod, which is for diabetes patients. They can put on the Omnipod, and it uses AI to learn your insulin release and your glucose levels and actually release insulin accordingly.

I don’t know if you remember, people used to stick themselves, right? But this now happens automatically. It’s self-learned AI for your body.

Wow.

It’s a cool use case. Zipline is another one. They’re doing awesome work.

Oh, yeah.

Oh, yeah.

When they started, they had these drones that were completely automated and all AI-driven, everything from battery optimization to the routes and everything. They were delivering food in areas of need—

Like blood—blood to refugees.

Blood to refugees.

Real serious stuff, yeah.

Yeah, they started in Africa and then expanded elsewhere in the world.

Yeah.

That’s also all AI. It’s an AI use case built on Databricks. That’s a cool one, but there are more advanced ones, too.

One that I kind of like, but that’s harder to explain, is a model that we built with Merck. It’s a transformer-based model called TEDDY: Transformer-Enhanced Drug Discovery—

Love that.

—as the name suggests. They actually published the research, so you can check it out.

It’s a model that, instead of predicting the next token in English, predicts how the gene regulatory network, the DRN, is going to respond. It can detect which cells are causal and which ones are just reactive—they’re just reacting.

Therefore, they can start using this in drug discovery and significantly reduce the cost of developing drugs that target specific diseases. That’s a super cool use case.

There are lots of these. Genie, which I mentioned—you have this ontology, and you can ask it any question. Novo Nordisk is using this.

Okay, cool.

They built this GLP-1 drug, but what Novo Nordisk is doing now is using it for all of the trials they’re running.

Oh, wow.

It can compress the time it takes to get insights—from weeks down to minutes—versus if you’re doing an obesity study or something.

There are a lot of amazing use cases for AI. We should not forget these upsides, too. We want all of these, and we do not want to pause these.

Yeah, exactly. You’re totally right. So—

Yeah.

—how do they—let’s say you map out the next 12 months—how do enterprises actually get value? You dropped the word “context,” but how do they operationalize that?

5. Enterprise AI Needs Context

It’s actually harder than most people believe. First and foremost, we have to make sure that we’ve digitized everything that’s happening in an organization. You cannot just wave a magic wand and make that happen.

Every meeting has to be transcribed.

Mm.

You have to be able to get all the context from all the meetings and everything that’s happening. All the digital content has to be fed to the AI. You have to build—we call it an ontology. We build that.

But first and foremost, you have to collect that. That itself is a problem in many organizations because legal teams will say, “Don’t record every call. Don’t record everything.” So you have to do that in a way where it’s—

Can you define “ontology” for everyone? I know Palantir says the word a lot, but it’s not like they own the word. What does that mean? For the people listening, how should they think about it?

Ontology just means that, in an organization, it’s the relationship between all the abstract concepts—all the goals, all the departments, all the people, and all the projects that are going on. What do they exactly mean, and what’s the relationship between the people, the resources, and what that company does?

It’s the difference between a person who is a new employee in the company and just started today and a person who has worked there for 5 years. Let’s say they’re equally skilled, have the same educational background, and are equally smart and hardworking.

One person is on her first day at work today. The other one has been there for 5 years.

Perfect.

What’s the difference between these 2 people? One has an ontology of how that organization works, who the people are, and how you get stuff done.

Don’t look at the org chart.

Mm.

Don’t go ask that person.

Yeah.

They will not get anything done. You go ask this person, and they’ll get it done for you. And that’s not how it works, so you don’t need to file that paperwork here.

And this project—this is what's going on. This is essential. So there's just a lot of ingrained knowledge that's sitting—

Yeah.

…in everybody's heads—who knows how an organization works. That's why people say, in startup land, “Hey, if you lose most of your people, that company can't recover from it.” It doesn't—

Mm-hmm.

You can't just replenish and hire new people.

Right.

The people are so essential. How do we get that context—that's the ontology—and give it to the AI? Part of that is that we just have to have the recording and all of that. But the second part is: how do you actually distill it down into a graph?

Mm.

Actually, a digital graph that you can then feed to the AI. So the way a lot of the agents work today—like Cloud Coder, any of them, Codex, Pi, OpenCode, or you can go through the whole slew of them—they have this loop, an agentic loop. It can reason, but then it goes and checks every resource one at a time. So it'll go to this MCP server for your question and try to see: Is the answer here? Is there another one? It synthesizes it and gives you an answer. But it's kind of slow.

I liken this to: if Google had built Google Search this way 25 years ago, we would've said, “Okay, we're going to get 10 blue links. We search for our key terms here.” But instead of giving you 10 blue links, it would've gone to 1 website, summarized with an LLM what it does, found a few hyperlinks, jumped in parallel to a few of them, read a few websites, done that for 10 minutes, and then given you its best 10 blue links it would find.

Well, that would be very expensive. It would cost a lot of money to do that every time you go on the web. 2, it would've taken a long time, and then you've got to wait 10 minutes. And 3, the quality would be bad, because you're actually only looking at a very small subset of everything that exists out there, right?

Mm-hmm.

So how do they do it? They have an index, right? You never leave Google's servers. You search for it, and it hits the index. The reverse index immediately gets you the 10 blue links within less than 100 milliseconds.

We need to do the same thing for the AI. So the ontology is that. We need to compute that index offline all the time. It's almost like the PageRank algorithm that Google had invented back in the day, but it's more complicated. Google was just looking at a web where everybody can go on the web. Here, there are permissions.

And the links existed, and—

Yeah. Here, there's permissions involved. The data I'm allowed to access might not be the data that you're allowed to access, so there's privacy.

Right.

There's access control. Also, there are many different types of objects here that we're dealing with, not just websites. So the problem is a little bit harder. But it's manageable. You can actually do it.

I'm convinced you can do this and you can get massive productivity gains out of it, because we did it for Databricks.

Yeah, exactly.

We did it for ourselves.

Same part, yeah, yeah.

Yeah, and we're like, the company's just completely changed. It's not the way it was, I would say, 1 year ago, because of this.

I mean, you've been dogfooding Databricks for Databricks forever. But maybe say more about the impact you've seen as an organization.

Once we got this ontology and started working on it, we actually had probably the largest—of all of our customers, we have the largest ontology. Our ontology is bigger for us than any of our customers' when they use us to build their ontology, because Databricks uses Databricks more than anyone else uses Databricks.

So it's like millions and millions of nodes in the graph, in the ontology graph that we have. What happens in an organization? You have a tree-structured organization, and information flows up and down the tree structure. If you can't make a decision, you escalate to your boss; maybe they can tiebreak, and it escalates up.

They need to get up to speed on what's happening, and they need to get all the context, and then they make decisions. Once decisions get made, you have to percolate them down in the organization. A lot of this can now be done by AI if you have an ontology.

Why? Because what happens in a meeting? In a meeting, you go through some analysis. Someone has done the analysis. They probably have a PowerPoint deck with some pretty graphs in it. That person who did the analysis is some smart person who used Excel and made some models.

Right.

There are some numerical calculations. So a lot of that you can now just do with AI. The AI can do the analysis for you. It has all the context. It can present it in a way that you want. You can ask questions about it. Instead of having follow-up meetings, you can directly ask questions from the AI.

It's very similar—it's along the lines of what Jack Dorsey has said that you can do to the organization.

Mm-hmm.

It's just a concrete way of implementing it.

Yeah.

It's a game changer for us. Everybody's on their phones now in the meetings on Genie, and they're asking Genie questions. You can see, as soon as someone says something complicated, you see everybody go to their phone.

Can you share that finance anecdote you mentioned once in a board meeting?

Yeah, sure. Internal board meeting. It actually was needed for one of our presentations. I needed to know how many customers we have in the Fortune 500—

Uh-huh.

…that use us. What's our penetration of the Fortune 500? And I asked one of the people in sales ops because I thought she would have it. She texted me back and said, “Oh, sorry, I can't log in to Genie right now. I'm on a flight.”

And I said, “Well, if you're just going to log in to Genie, I can do that myself.” I asked you because I thought you had something authoritative that I don't have access to. So then I was a little bit angry, so I texted the CFO instead, Dave. I texted Dave and said, “Hey, do you know what our Fortune 500 penetration is?”

He just copy-pasted a screenshot of Genie back to me. That's it. So he also has that.

I love that so much.

So then I said, “Does anyone do anything novel here, or is everybody just going to Genie and asking the ontology for answers?”

It's like, “Let me Genie that for you,” instead of “Let me Google it.”

Yeah. That's what everybody's doing. Now we just say, “Hey, can someone just Genie this?” “Can you just get it from the ontology?”

Absolutely.

So I do think it's a game changer. But it's not just that you press a button and you have an ontology in an organization. And I think Palantir has done a great job of going to organizations, getting a lot of that tacit knowledge written down, and getting it into the organizations.

We automatically then take that and build the graph, and then we feed that graph into the agents—

Mm.

…so that we can answer the question in a way that business leaders would like to see it, which is in graphs, in an analytical way, and in a way where you can interrogate that question and continue asking questions and getting answers, so you can make decisions. And then disseminating that information in the organization.

Yeah, it's pretty amazing. You sort of bookmarked that developers are obviously using AI, with questionable value. I want to follow up with you on that because I feel like you guys were one of the earliest—

Mm-hmm.

…and I don't want to use the word “token maxing” because it has such a negative connotation, but I think in terms of applauding people who can use—

Yeah.

…AI to become more productive, you guys were—

Yeah.

…at the forefront of that, right? And then, of course, there's this cycle of, “Oh, shoot, people are being wasteful. Now we need to value max.” What was your own journey on that?

Yeah.

How do you guys think about value maxing, not token maxing? And then I'm going to throw in AI Gateway in this, right?

Yeah.

Because I think the cost-management piece is actually getting more important, and you guys are helping people do that. But maybe tie that in, to the extent it's relevant.

6. AI Costs Need Control

Yeah. So around Q4 last year was when the models got really, really good, and we started noticing that they were actually starting to give us much better productivity. I started using the models myself to start committing code into production for Databricks—the actual—

I said, “I want to take it all the way to production.” So I did that and started pushing the organization: “Hey, everyone needs to do that. I have done it. Why are you not?”

If the CEO can commit code to production on a very sensitive data platform that has all these security requirements, you should be able to do that too—you being any manager or anyone in the organization.

So I started pushing everyone very hard, and we started making leaderboards in Q4. In January and February, when we kicked off the year, we were already in full swing. Everybody was using this stuff, and we were pushing and managing it. The whole token-maxing thing was already happening around the February-March period.

Right.

We were lucky to be maybe a few quarters ahead of folks, so we could see what was happening here, and it was getting out of hand. We already had a gateway called Unity Gateway. This gateway was being used to provide token capacity, so you could get OpenAI, Anthropic, Gemini, or Groq capacity.

Mm-hmm.

Any customer can come to us, and we'll provide them that capacity because we have relationships with those providers and can provide any open-source model. We started putting budget constraints in place and giving people warnings: “You have this much of your budget left. You're getting close to your ceiling.” We did that for each person and group, and then we started doing great analytics so we could predict exactly where the costs were going.

Mm.

Yeah, and then we added smart routers that could pick cheaper models if you were getting close to your budget or if you had simple questions. We started doing that. We also built a harness called Omnigent, which can multiplex between the different harnesses. It turns out the harness itself matters. If you use the same model but different harnesses, there's almost a 2X cost difference.

Right.

Even with the exact same model.

Mm.

With the same version but a different harness, you get a 2X difference in actual cost.

Interesting.

So if you can change the harness, you can get a lot of leverage on cost. We started using all of this. That way, we were able to bend the curve. Our costs for AI have basically been stagnant: the tokens have continued to go up, but the costs have stayed roughly the same.

Hmm.

That's been super, super important for us, and there's huge demand for this. I think every organization is going through this now.

Yeah. For the first time ever, a company at scale said last week that it's moving from the frontier models to GLM. This is a large engineering organization. Do you see this? Do you think that's a trend, or do you think that's just a one-off anecdote? I've been hearing about it. I remember the first DeepSeek moment and NVIDIA stock, right? And then that turned out not to be real.

Yeah.

Then there was the Kimi moment, followed by the next DeepSeek moment. None of it seems to have had an appreciable impact on the market. But now the anecdotes I have are pretty real, and it seems to be happening. I'd love your view.

I think people want both. They want the latest model that's super intelligent for difficult tasks where they get ROI.

Yeah.

But then there are a lot of mundane, dumb things. People literally use their harness to rename files and whatnot.

Yeah.

You're paying orders of magnitude more for that.

Yeah.

Please type that in yourself. Don't have the model do that. It's going to spin for 5 minutes, then it's going to rename the file for you and cost you cents.

Sorry.

I'm just saying, type it in yourself.

I'm just wondering: do you actually see market movement on that?

Yeah. No, people are moving on it, but the pattern is either to use this expert pattern, where you have a small, cheap, open-source model that uses an expert model—the big ones—or vice versa, or a way in which they can ping-pong them to each other. But people are also multiplexing harnesses and changing harnesses so they can control costs.

People have found, for instance, that Pi is very efficient as a harness.

Yeah.

I think there's going to be a multitude of these. The models themselves are stochastic, as you said. Every time they give a different answer, and they're changing so much.

Yeah.

There's a lot of experimentation happening, so I think we're going to get to a world where you're not always using the smartest model for everything, which has been the paradigm for the last couple of years.

Yeah.

A new model comes out, it's super smart, and they use it for everything—even really simple, mundane tasks.

Yeah. I'll tell you what I see. I see people using Fable and Astra for architecture.

Uh-huh.

They use a cheap model for implementation, and then Fable or Astra for audit.

Yeah.

That seems to be this emerging pattern.

Yeah. What are you guys seeing in the startups?

Well, that's it. That's honestly the pattern.

Mm.

How much open source?

Either.

By token or by dollar?

Either.

By dollar, open source is 5%. It's very little, but by token count it's over 60%.

Hmm.

Yeah. We talked to, say, Decagon or something like that. I think it's different for internal use versus external product use.

Mm-hmm.

For external product use, I think they're almost up to 90% open source.

Wow. Hmm.

Internally—and I don't want to say for Decagon in particular—a lot of them are like, “We don't care. We'll just use frontier. We're not thinking about cost control.” But as it gets bigger—

Yeah.

—you and I were in another board meeting where they actually did bring that down, just from a waste perspective.

Yeah.

So I definitely see that moving more toward open source on the product side. And actually, that's a—

Yeah.

—related question, maybe around open source, but post-training specifically.

Yeah.

I feel like you were kind of early. I remember talking to you in 2023.

Yeah.

When did you buy Mosaic?

2023.

2023.

Summer.

Okay. So this vision that you had in 2023 kind of came true in 2026.

Mm-hmm.

I don't know if you guys would agree, right? That's sort of what we're hearing.

What vision?

Just: “Hey, we're going to actually—you're going to own your own intelligence.”

Oh, right. Yeah, yeah.

“You're going to—”

Mm-hmm.

“—post-train your own open-source models,” et cetera. That's definitely what the startups are doing.

Yeah.

I don't know if that's what the enterprises are doing yet—

No.

—I mean, do you feel like you were early to that?

Yeah. First of all, when we started, it was also, “Hey, we're also pre-training for you,” which doesn't make any sense.

Hmm.

You can—

Right, right.

There are very good pretrained models now that you can use.

Right, right.

But you can actually do post-training on the model, and you can do reinforcement learning.

Yeah.

We're actually doing it at scale. Many of those startups are actually customers, so we help them use RL, or reinforcement learning, environments where we can make the models very, very good at the specific tasks they're doing.

It makes a lot of sense for them to do that. If you have a startup and it's offering a product, and that product does something specific, it's not just general intelligence. It does something specific for you. It makes a lot of sense to take a really good open-source model and use reinforcement learning to make it really good at that specific task.

You can cut the cost down, make it really fast, and control your own IP. In that sense, it's possible. But large enterprises just need basic automation.

Mm.

It's just too much for them to do this right now.

Yeah.

I think one of the challenges—

See.

…is that you need good evals.

Totally.

And making good evals is hard. So while the startups can do that and they're motivated to do that, for other organizations, the easy button might be just to use a frontier model—

Mm-hmm.

…rather than having to create—

Yeah.

…your own evals. We even automatically generated evals for the customer in the product, and we had them front and center, but then people didn't want to use them. So we said, "Okay, let's move them to the back end so that they're optional."

Right.

And then they would never go to them. So I would say, in general, the—

Why? They just don't want to get into it? Is it too complicated?

I think you want—

Hmm. Yeah.

…quick—

Yeah.

Yeah, that makes sense.

…reinforcement of, like, "Hey—

Yeah.

…there's a new model. I want to try this out. I want to get this problem solved." You don't have time to go do the scientific method of, "Let's make an eval. Let's have a great baseline." It's sort of like TDD, test-driven development.

Hmm.

In software engineering, did people actually do test-driven development? Very few did, right? Everyone said it was the right way to do it, but nobody actually did it in practice. So that's the same. That's kind of a little bit of the curse of training your own model: The evals are the hard part.

I know you have an FDE model at Databricks.

Yeah.

That's a very popular word—or acronym—right now. But to get these into enterprises that large, is a full FDE model required?

Mm-hmm.

Or how do you—

Yeah, I mean—

…how has that evolved, maybe?

Yeah, we've had these FDEs, and the demand for them has gone up significantly.

Mm.

A lot of it is, how do we build that ontology? The ontology is automatic, but if you're not collecting any information—

Yeah.

…you're not recording anything.

Right.

So that's one of the key things that we do. But also things like, "I want to build an agent. I want to put it on—I want it to be customer-facing, have really low latency, and have guardrails so people can't abuse it or ask it things that we don't want it to answer," and so on. So we can build that, like the Sports AI that Fox has. You can go chat with it about sports events, and you can try to ask it about politics, and it's very good at rejecting you and moving on to talking about sports instead. So the FDEs built that—

Wow.

…so we'll help organizations actually get started with AI.

Mm.

It is important.

Yeah.

Because many organizations simply don't have the in-house expertise to build this stuff. So they just need a little bit of help on the side, and then they get started.

Yeah. Makes sense. So this is more related to the agent side, but I saw recently that I think a third party—

Mm.

…a neutral third party, I think, did some tests that Lakebase or Neon was actually the Postgres database of choice for agents. I thought that was interesting, one, because it's exciting for Databricks, but two, I probably wouldn't have guessed that maybe a year ago.

Yeah, it's a surprise for sure.

Yeah, it was a surprise, just because there are others out there that have great developer momentum as well. But it was pretty clearly number 1. So I'm curious: How did you guys crack this, and what makes you win across the agents? Because if you win the agents now, you win the market.

Yeah. I think a lot of credit should go to Neon and Nikita and the team. I think what they've done is they've just been obsessive about how to make the models—how to make the models pick and have agents favor Lakebase or Neon as a database. What do they do? The agents want to experiment. They're going off, they're trying to build a little bit of software. They need the database, so you need the database to come up quickly.

Mm-hmm.

They had this obsession that everything should take far less than a second. The database comes up in far less than a second. You can clone a gigantic database, like a terabyte database, in less than a second. So it's highly elastic, highly responsive, and then they built this killer feature called branching.

Mm.

Branching just lets you branch the database, and you can have many, many branches of the same database. They just made it very, very lightweight. We saw this with other things with agents, right? Like uv and ripgrep—basically reimplementations of a lot of the tools on Unix, making them really, really blazing fast and lightweight.

Mm.

And also sort of fail-safe for agents. They've just done this to a harder problem, which is databases.

Right.

So now you have a Postgres database, and the Postgres database has all these advantages: It's really fast, it's nimble, it's fail-safe. You can go back to snapshots. You can do those things. So I think that's why. It's just easier—

Yeah.

…for agents to use this. They also made sure it had a pricing model that was—

Yeah. Totally.

You don't want the cost to run up just because agents are building some software and experimenting.

Right.

You're okay paying for your database if it's for production use and lots of people are using it. But just to experiment—

Mm.

I think they were just obsessed. They weren't trying to win the database war or trying to be better than some other vendor. They were obsessed with, "How are we the best for the agents?"

Mm.

That's a new persona, because in databases, the obsession has been, how do we help DBAs? How do we help app developers? How do we help the people who are using the database?

Yeah, exactly.

They changed the game and said, "Hey, how do we focus on agents—

Uh-huh.

…and help agents get the best database they want?" And now over 90% of the databases that are created on Neon and Lakebase are actually created by agents.

Wow.

So it's not even humans. The numbers speak for themselves. They're—

By the way, it's remarkable. I've started to use Neon as my standard database.

Mm.

And it was bizarre to me, because normally when you enter a large company, things slow down. It's actually like the product has gotten materially better.

Yeah.

Are they totally independent, or do they work with the rest of the company? How does that work?

No, it's a great team. I mean—

Unbelievable.

…they work very closely together. We love databases and data, so—

Amazing.

We live that. But no, the team does a great job of just making things super-fast, snappy—

Super—

…and great for agents.

Yeah. All right, Ali, what is your p(doom)?

Less than 10%. No, close to zero. What about yours?

I don't know. My only answer is that my p(doom) without AI is much higher than my p(doom) with AI. How's that?

Wow, that's a lot. What about yourself?

I went on a technical—

I would agree with Ali on this one.

Okay, cool.

Yeah.

Okay.