Jonathan Ross: DeepSeek Special - How Should OpenAI and the US Government Respond | E1253

Everyone’s seen the news about DeepSeek today. Is it as big a deal as everyone is making of it?

1. Scraping OpenAI Models for Higher Quality Output

Yes, it is. It’s Sputnik 2.0. It’s true that they spent about $6 million, or whatever it was, on the training. They spent a lot more distilling or scraping the OpenAI model.

I can’t speak for Sam Altman or OpenAI, but if I were in that position, I would be gearing up to open-source my models in response, because it’s pretty clear you’re going to lose that. You might as well try to win all the users and the love from open-sourcing. Open always wins.

Always ready to go, Jonathan. I’m so excited for this. I’ve heard so many good things from so many different people, so thank you so much for doing this emergency show with me today.

No problem. Before we start, can I just say one thing? I think you have the most amazing, unique go-to-market strategy that I’ve ever seen in my life for a podcast. I’ve never seen this before.

I think your strategy is that you’re literally interviewing every single audience member, forcing them to watch videos and get addicted to you.

I thought you were going to say my accent, but I’m totally going to take that. That’s wonderful. Yes, you’re absolutely right: sometimes the biggest benefits of your business you don’t actually see until you do them at scale. It’s totally true.

2. Concerns About US Customer Data Going to China

I do want to start with DeepSeek. For a little bit of context, why are you so well placed to speak about DeepSeek? Let’s just start there.

My background is that I started the Google TPU, the AI chip that Google uses, in 2016. I started an AI chip startup called Groq, with a Q, not with a K, that builds AI accelerator chips, which we call LPUs.

3. Is DeepSeek News as Big a Deal as It Seems?

Fantastic. I wish everyone was as coherent as you in terms of their introductions.

Everyone’s seen the news about DeepSeek today. I want to start off by saying: is it as big a deal as everyone is making of it?

Yes, it’s Sputnik. It is Sputnik 2.0, and even more so. You know that story about how NASA spent $1 million designing a pen that could write in space, and the Russians brought a pencil? That just happened again. It’s a huge deal.

Why is it such a huge deal? Let’s unpack that.

Up until recently, the Chinese models had been behind Western models. I say “Western” including Mistral and some other companies. It was largely focused on how much compute you could get.

Most people don’t realize this, but most companies have access to roughly the same amount of data. They buy it from the same data providers, then churn through that data with a GPU, produce a model, and deploy it. They’ll have some of their own data, which will make them subtly better at one thing or another, but they’re largely all the same. The more GPUs, the better the model, because you can train on more tokens. That’s the scaling law.

This model was supposedly trained on a smaller number of GPUs and a much tighter budget. I think the way it’s been put is that it cost less than the salary of many of the executives at Meta.

That’s not true?

It’s actually an element of marketing involved in the DeepSeek release. It is true that they trained the model on approximately $6 million worth of GPUs. They claim that was GPU usage for, I think, 60 days, which, by the way, was also about the same amount of GPU time—4,000 GPUs for 30 days—as the original Llama 70B, I believe.

More recently, Meta has been training on more GPUs, but Meta hasn’t been using as much good data as DeepSeek, because DeepSeek was doing reinforcement learning using OpenAI’s model.

4. Distillation & DeepSeek's Use of OpenAI Data

What is distillation, just so I understand? Can you help me and the audience understand what distillation is in this regard, and how DeepSeek has been using distillation to get better-quality output through OpenAI data?

It’s a little bit like speaking to someone who’s smarter and getting tutored by someone who’s smarter. You actually do better than if you’re speaking to someone who’s not as knowledgeable about the area or is giving you wrong answers.

Before we get into any of this, I need to start with the scaling laws. These are like the physics of LLMs. There’s a particular curve, and the more tokens—which are sort of the syllables of an LLM, although they don’t match human syllables exactly—the more tokens that you train on, the better the model gets.

There are asymptotic returns where it starts trailing off. The thing about this scaling law that everyone forgets—and that’s why everyone was talking about how it’s the end of the scaling law because we’re out of data on the internet—is that it assumes the data quality is uniform. If the data quality is better, you can get away with training on fewer tokens.

Going back to my background, one of the fun things I got to witness, although I wasn’t directly involved, was AlphaGo, when Google beat the world champion Lee Sedol at Go. That model was trained on a bunch of existing games, but later they created a new one called AlphaGo Zero, which was trained on no existing games. It just played against itself.

How do you play against yourself and win?

You train a model on some terrible moves. It does okay, and then you have it play against itself. When it does better, you train on those better games, and then you keep leveling up like this. You get better data. The better your model is when it outputs something, the better the result and the better the data.

You train a model, use it to generate data, train a model, use it to generate data, and keep getting better and better. That lets you beat the scaling-law problem.

One quick hack for getting past all of that is, if there’s a really good model already right here, just have it generate the data and go straight up to where it is. That’s what they did.

It is true that they spent about $6 million, or whatever it was, on the training. They spent a lot more distilling or scraping the OpenAI model. They scraped the OpenAI model, got this higher-quality data from that and from refining it, and then got higher-quality output.

Correct?

Correct. All of that said, they did a lot of really innovative things. That’s what makes it so complicated. On the one hand, they kind of just scraped the OpenAI model. On the other hand, they came up with some unique reinforcement-learning techniques that were so simple and so impressive.

A lot of people wanted to say, “The Chinese copy and duplicate, as they always have done.” No, they came up with innovative stuff.

The best way to describe it is this: have you ever taken a test, gotten an answer right, and had your professor mark it wrong? Then you go back to the professor, argue with them, and everything, and it’s a pain, right?

If there’s only one answer, it’s a simple answer, and you say, “Write that answer in this box,” then there’s no arguing. You either get it right or you don’t.

What they did was, rather than having human beings check the output and say yes or no, they said, “Here’s the box. Output the answer here,” and then checked it. If it’s correct, we have the answer; if it’s not, we don’t. There’s no need to involve a human. It’s completely automated.

I read about reward-modeling stage, and that they innovated on this in a unique way. Did they not? Can you explain that area for me?

I’m not as familiar with that, so I’m probably not going to. You’ve been doing the research, so why don’t you tell me what you saw, and I can tell you if it tracks?

Essentially, they combined 2 different types of reward models to get higher, more accurate output. That was what I didn’t understand.

Yes, that’s not an area where I’ve dug too deeply into it.

Can OpenAI not just do distillation on DeepSeek’s model and then get better?

They don’t need to, because they’re actually still better. They’re a little bit better.

Could they buy the GPU usage, or is that questionable?

I don’t think you have to distill it because of the quality delta. However, why would they try to smuggle in GPUs when all they have to do is log into any cloud provider and rent GPUs?

This is the biggest gaping hole in the whole way export control is done. You can literally log into a cloud provider, swipe a credit card, and pay to use GPUs.

So are the export-control laws unnecessary, then?

They’re good, but the problem is that it’s like the Maginot Line: you just go around it. You need to seal it up a little more. There’s a little bit of room left to go here.

The other thing is, keep in mind that OpenAI was effectively subsidizing the training of this model accidentally, because DeepSeek was using OpenAI. Rumors are that OpenAI may not be completely profitable yet in terms of every token in the API—maybe on the subscriptions, but in the API. Each token they generated was effectively losing OpenAI a little bit of money while DeepSeek was getting training data.

OpenAI probably still has that data. In theory, they could just train on it.

George Krizan said in a tweet today that this would likely be a violation of U.S. export laws. Do you think that’s not true?

I’m not aware of where it would be an export issue. I do know that many people log into cloud providers and use them remotely.

One of the problems is that we actually block IP addresses from China, and I believe we might be unique in doing that. It’s also a little bit fruitless, because someone can just rent a server anywhere and log into us from there. There’s nothing we can check.

I don’t know that IP addresses are really the right way to do it. We need something more sophisticated.

You mentioned blocking IP addresses from China. There’s a lot of concern about U.S. customer data going back to China. Do you think that’s a legitimate and justified concern?

Yes. It’s probably the most significant concern. There are other concerns, but that’s probably the most significant.

People are so used to using these services that they might be shocked to hear this: when you use one of these other services and say “delete,” what they do is write “delete” next to your data. They don’t actually delete it. They just mark it as deleted. When you later come back and ask for your data, they give it to you with the word “delete” next to it. It’s still there.

These are well-meaning companies. Do you really think the CCP doesn’t have all your data and isn’t going to look it up later?

Some governments are more aggressive than others. If they have access to your data, it’s not even necessarily your data. It could be your next-door neighbor’s data. Your next-door neighbor might put something in there that accidentally gives information away and makes you more vulnerable.

Maybe they had a package delivered and put a complaint somewhere. You might not even do it yourself, but other people around you might. Think about the health data of a spouse.

5. DeepSeek and Its Potential Use by the CCP

Jonathan, I’m going to avoid the British indirectness: do you think DeepSeek is an instrument that will be used by the CCP to increase control on less democratic countries?

Yes, but I don’t think it’s DeepSeek that’s doing it.

You have to understand that any company operating in China and Hong Kong—the “one country, two systems” thing didn’t quite work out as anticipated, or maybe as anticipated but not as stated—has no choice.

When Groq started in 2016, we decided that we weren’t going to do business in China. This wasn’t a geopolitical decision; it was purely commercial.

We kept seeing companies like Google and Meta fail over and over again trying to win in China. The formula is actually pretty simple: you’re not allowed to make net money. You’re allowed to spend more money in China, but the moment you start to become profitable, or anywhere near profitable, all of a sudden there’s a thumb on the scale.

Companies that manufacture a lot in China and send more money to China can be successful there. They can sell things there. It’s a pretty simple formula: you must send more money to China than you take out.

At the same time, they require you to hand over all data. They also require that certain answers be in a form they find acceptable.

One of the more common things you see about DeepSeek right now is that, when you ask about Tiananmen Square, if the temperature is low on the model—and temperature is how creative it is; we don’t need to get into that—it will give you an answer that basically says, “I don’t want to talk about that. It’s a sensitive topic.”

You can ask it about other things that are sensitive topics elsewhere in the world, and it’ll just answer. But what happens if the CCP requires that they start to say, “What about TikTok? Should it be banned? Absolutely not. Here’s why,” and it gives you a cogent reason? That’s scary.

What do we do from here? I share your concerns completely. My challenge is that TikTok can be banned and shut off. It’s a closed-end product that we can ban tomorrow if we really want to. Here, it’s open source.

And worse. Until recently, we refused to run any Chinese models. We had to make a very difficult decision on DeepSeek. We now have it on our API at Groq.

Why did you decide to break the rule for DeepSeek?

When we saw DeepSeek become the No. 1 app on the App Store, the realization was that people were going to be putting their data in there. We want to make sure that there’s actually an option.

We store nothing. There’s no “delete” or anything like that. We store nothing. We don’t even have hard drives. We just have DRAM, and when the power goes off, everything goes away.

We wanted to make sure there was an alternative where, when you use DeepSeek’s model, your data isn’t going to the CCP.

Right now, the CCP is probably going to be taking the safeties off the weapons. They’re going to be asking, “Why are you making this model open source? Please direct your data toward us. Go win a bunch of customers this way.” But now they want the data.

They’re going to change the strategy. Remember, DeepSeek is a hedge fund. They’re doing this themselves, and they’re just influenced by the CCP. Now that the CCP has seen the success of this, it might see it as yet another TikTok.

They will see it as another TikTok. My question to you is: how long is it before the U.S. reacts to prevent this?

It should be. The first question to ask is whether we’re going to be talking about DeepSeek, or R1, for the next 6 months. The answer is absolutely not. We might be talking about R2, R3, and R4, but R1 was a one-shot.

The question is whether they’re going to keep coming up with interesting things, whether we’re going to play cat and mouse, and whether everyone is going to learn from this.

The biggest problem is that this has made it absolutely, nakedly clear that the models are commoditized. You’ve been asking the question: if there was any doubt before, that doubt is over.

What is the moat? I love Hamilton Helmer’s 7 Powers. It’s one of my favorites. I do it for every single investment we make. Every person at Groq has to fill it out.

Marketing is the art of decommoditizing your product, and the 7 Powers are 7 great ways to decommoditize your product: scale economies, network effects, brand, counter-positioning, cornered resource, switching costs, and process power.

The question is, who’s going to do what?

OpenAI—and you have to give Sam Altman and that team credit—has amazing brand power, like no one else in this space. That’s going to serve them for a really long time.

What you see Sam trying to do is scale. He’s trying to scale. That’s why we hear about Stargate and $500 billion. That’s the power he would like to have, but the power he has right now is brand. He’s trying to bridge that.

Doesn’t this news ridicule the $500 billion announcement, at a time when we’ve seen increasing efficiency on a scale like never before with DeepSeek today?

The $500 billion doesn’t ridicule it. Actually, I don’t think it’s enough spending.

We saw this happen at Google over and over again. We built the TPU, so why did we do it? The speech team trained a model that outperformed human beings at speech recognition. This was back in 2011 or 2012. It was the first time that had happened.

Jeff Dean, the most famous engineer at Google, gave a presentation to the leadership team. Slide No. 1: good news, machine learning finally works. Slide No. 2: bad news, we can’t afford it.

We’re Google, and we’re going to need to double or triple our global data-center footprint, probably at a cost of $20 billion to $40 billion, just to get speech recognition. Do you also want to do search and ads?

It turns out there’s always this giant “mission accomplished” banner every time someone trains a model. Then they start putting it into production, and they realize, “Oh, this is going to be expensive.”

This is why we’ve always focused on inference. Think about it this way: at Google, we always ended up spending 10 to 20 times as much on inference as on training.

Now the models are being given away for free. How much are we going to spend on inference? I guarantee it’s going to be enormous. With test-time compute, I’ve asked questions of DeepSeek where it took 18,000 intermediate tokens before giving me the answer.

I think Jensen Huang said that now half of NVIDIA’s revenue is from inference. What does that look like in the future?

I think it’s 95%. It just makes sense. You don’t train to become a cardiovascular surgeon and then do that for 95% of your life and perform for 5%. You train for a little while, and then you do it for the rest of your life.

Do you think the U.S. will put sanctions on DeepSeek to prevent the CCP from using it for data capture on U.S. citizens?

I don’t know what the solution is. There’s a carrot and there’s a stick. You can use a stick and block it. That might be effective, although I don’t know that the U.S. has really done that before. I’m not aware of a case, although it may be possible that it’s happened.

There’s also the carrot. It’s interesting how DeepSeek is being offered for free in China, and not just in China, but to anyone else. Others are doing that, too.

Is it possible that the CCP is underwriting that because it wants the data? They’re doing it with the car industry. The subsidization of Chinese cars by BYD is destroying the European car market.

Absolutely. The thing is, we have a lesson from the Cold War, which was mutually assured destruction.

The problem is that we do some sort of tariff, and then China does a tariff back. There needs to be some sort of automated response: if you do this, we will respond. If you subsidize this industry, we will automatically subsidize the equivalent industry. Make it automatic, so don’t do it, because there’s no benefit to you.

How does the fact that it’s open source change everything?

It’s the only reason people are using it. If it wasn’t open source, it wouldn’t have gotten the excitement. Open always wins.

Keep in mind that Linux won back when people didn’t trust open source. They thought it was less secure, that the features were worse, and that it was more buggy. It still won.

6. Is DeepSeek Diminishing OpenAI's Distribution Advantage?

Now people expect open source to be more secure, less buggy, and to have more features. How is proprietary software ever going to win?

Everyone always says that distribution is one of the major advantages that ChatGPT and, hence, OpenAI has, especially over the other providers. Every single day that DeepSeek is out and being used so pervasively, it diminishes the value of OpenAI’s distribution.

I agree, especially for pricing, because they’re losing their pricing power.

I can’t speak for Sam Altman or OpenAI, but if I were in that position, I would be gearing up to open-source my models in response. It’s pretty clear you’re going to lose that, so you might as well try to win all the users and the love from open-sourcing. Otherwise, you’re already at a point where you’re going to be using your other powers, like brand.

Would that be possible? Wouldn’t it cannibalize one core main line of revenue?

How would it cannibalize it? Remember, people like distribution.

How many people are going to buy something because they trust Dell? People trust Dell because Dell has earned its reputation over the course of decades. Supermicro builds interesting hardware, but look at what they’ve been going through recently. There are pros and cons. It’s cheaper, it’s trusted—you have to make a decision.

OpenAI has been around for a while. Most people think of them synonymously with AI. They could just switch to DeepSeek and people would still use them. That’s brand. It’s one of the 7 Powers.

If you were OpenAI, on day 1, would you switch to open and offer it for free?

I would. There’s probably more cleverness they could use. They could probably strike some deals before they do it, or whatever, but that would be the move I would make.

It would also be a position of strength. It would simply say, “Look, the only problem is the timing. If it happens right after DeepSeek, it looks like a response as opposed to an intentional thing.” I don’t know how you do that, but it is a response.

Why not just own that it’s a response?

Maybe that’s a good one. You just say, “Look, we had to respond. We’re better. Let’s see which model people choose.”

7. Perplexity in 3 Years

What do you think the internal discussion is within OpenAI today?

I would imagine it depends on where you are. If you’re senior, you’re going to have very different concerns than if you’re at the foot-soldier level.

At the foot-soldier level, you’re going to be worried: is my equity going to be worth anything? Is there any longevity here? How do I do my job? Am I going to have a job?

If you’re further up, it’s going to be more like: how do I keep everyone? How do I keep morale up? What is my response?

You’re going to have a lot of very difficult decisions in front of you. The No. 1 driver of bad decisions is fear. What they have to do is pick something, commit to it hard, and be brave about it.

So many different decisions work if you commit and align. It’s all about alignment.

How should we think about Meta? Meta shares the open-source values that DeepSeek espoused. Does this help or hurt Meta?

That’s a good question. One of the ways we’ve been looking at LLMs is a little bit like looking at an open-source software project, like Linux.

8. The $500BN Stargate Project

The thing is, Linux has switching costs, and I think what we’ve discovered is that LLMs have no switching costs whatsoever. That’s why the analogy to cloud doesn’t hold up at all. Everyone says, “There are going to be a couple of cloud vendors,” but you don’t switch your cloud very often.

Let’s map the 7 Powers to the top tech companies. I would say Microsoft’s biggest strength is switching costs. I love Microsoft as a company, but you go into a room full of people and ask, “Who uses Microsoft?” A bunch of hands go up. Then you ask, “Who likes using Microsoft?” The hands go down. It’s largely switching costs.

With Meta, it’s network effects. They could literally give every piece of technology away for free. I’m completely jealous of that, because if I had that right now, I would open-source everything. You don’t have to worry about it, and you get everyone helping you.

Meta is always in a position where open source is to its advantage because of the network effect. It almost doesn’t matter where it comes from.

I’m sure Meta would prefer to have the Linux of LLMs, but the more it goes open source, the more of an advantage Meta inherently has.

If you were Meta, would you do anything differently?

Meta is an amazing competitor. Normally, if this were something proprietary—a social mechanism, for example—they would try to replicate it and compete. They would say, “Come join us,” or not. I don’t think “come join us” works here.

The beautiful thing is that all the information for this model is available. Meta has already been doing this. It has way more compute. The question is whether it’s willing to scrape OpenAI like DeepSeek did. I don’t think it is.

Meta has been super careful about everything it’s been doing, and that’s the disadvantage.

I’m not being rude, but do you put morals aside to win? This is the AI arms race.

I think that’s going to happen. You cannot lose. What this has done is change the game.

Let’s talk about Europe for a minute. We almost forgot about Europe.

For me, watching everything, it feels like Europe lacks a willingness to take risk. There’s a black mark if you get it wrong. Everything is about downside protection, whereas in the U.S. it’s, “That was a great effort. You failed, but I’m going to fund you again.”

Then you look at China. China practices IP theft. It’s just part of the culture, and it’s not only against Western companies; it’s against each other, too.

The difference is that if you’re a Western company, the government steals from the Western company and provides it to Chinese companies, which is less fair. There are famous stories of turning on Huawei switches and seeing Cisco’s logo, with all the bugs.

Does the West have to adopt a more theft-on attitude?

I really hope not. For Europe to compete with the U.S., Europe has to adopt a more risk-on attitude. But adopting a more theft-on attitude is viscerally disgusting to me. I’m literally repulsed by the idea.

Are we not being idealistic? If you’re running in a race with someone who’s willing to take steroids, and you want to win, you’re going to have to take steroids, too. Then everyone is taking steroids, whereas if no one were taking them, everyone would be healthier and you’d have a real competition.

It’s a real problem. The question is whether governments can get involved.

I would love nothing more than to compete directly with Chinese companies on a fair footing. They have really smart people. DeepSeek has proven this. But when the government keeps putting its thumb on the scale, we’re going to try to avoid that competition wherever we can.

Now there’s no avoiding it, so maybe governments just have to get involved. I’m being blunt: Xi Jinping cares about 1 thing—power retention and growth. That’s the only thing that matters to him, and AI is central to that. He will do whatever it takes to win.

Having rational discourse about rules of play is, bluntly, unrealistic.

China has a lot of advantages, but the chief advantage is the number of people it has. Number of people is not sufficient, though. You also have India, and India has an advantage from the number of people. China has out-executed it. In fact, India was asking China for some time to help build out roads and infrastructure. China has really mastered that.

China has people, organization, discipline, and alignment. The concern with AI is: what if an LPU or GPU becomes the equivalent of a contributor to the workforce? You could literally add more to GDP by creating more chips and providing more power.

If that becomes the case, does China’s advantage erode? China is concerned that, in terms of workforce, the U.S. or the West could catch up. At the same time, China has a huge population advantage.

9. Advising the EU on Europe's Stance Today

This is why I want Europe to get into the fight on AI. If there are 500 million people who could be jumping into this.

If you were to advise the EU today on Europe’s AI response, what would you say?

Have you ever seen Station F?

Of course. I was there last week. We hosted there.

I would say that by the end of this year, Europe should have 100 Station Fs, and by the end of next year, it should have 1,000.

You’re basically collecting 3,000 people and surrounding them with other risk-taking entrepreneurs. They support each other, and they’re risk-on. When you surround yourself with other people who are risk-on, you’re going to be risk-on, and you’re going to take the entrepreneurial leap.

What does this space look like in 3 years’ time? How fearful should I be? I’m obviously a venture capitalist for a living, and all of my friends are saying, “Oh my God, we just lost hundreds of millions of dollars on these foundation-model companies.”

How many companies are you aware of that have become incredibly successful without pivoting?

Few. Most pivot.

Exactly. Pivot. Get over it.

Frankly, I’ve been talking to a lot of the LLM companies, and they have some good ideas. I really like the Suno founder. I think he saw it from the beginning: models are going to be commoditized, and that’s why he’s focused on the product.

He got it from the beginning. What is your product, not what is the model? The model is a piece of machinery. It’s an engine. What is the car? What is the experience?

What do you think Perplexity is in 3 years?

The question I used to get asked when we were raising money a little while ago was, “Is AI the next internet?” I said, “Absolutely not.”

The internet is an Information Age technology. It’s about duplicating data with high fidelity and distributing it. That’s what the telephone does, what the internet does, and what the printing press did. They’re all the same technology, just at a much different scale, speed, and capability.

Generative AI is different. It’s about coming up with something contextual, creative, and unique in the moment. The LLM is just the printing press of the generative age. It’s the start of it, and there are going to be all these other stages.

Imagine trying to start Uber before we had mobile. “Great, I’m going to book a trip over to here. How do I get home?” You couldn’t carry a desktop with you. You need to be at the right stage.

When I look at Perplexity, I see it as being perfectly positioned for the moment when the hallucination—or, really, confabulation—rate comes down. The moment these models get good enough that you don’t have to check the citations anymore, it will open up a whole set of industries.

All of a sudden, you’ll be able to do medical diagnosis from LLMs. You’ll be able to do legal work from LLMs. Until then, it’s like trying to create Uber before we had smartphones. It doesn’t make sense.

However, people are willing to use Perplexity today, even though you have to check the citations. It has an actual business that gets to ride the wave. The moment that tsunami of a lack of confabulation, or hallucination, comes along, Perplexity is perfectly positioned.

Does Mistral survive?

Each company has to find its own thing. I would look at Suno as a great example of how things are being done around the product as opposed to just the models.

Is it possible to pivot when you are OpenAI, Anthropic, or one of the very large providers? You’ve ingested billions of dollars. If disruption happens and you’re not able to pivot now, you’re not going to be able to pivot later when you get disrupted anyway.

10. Commoditization of Models & Big Tech's Stock Struggles

Wouldn’t one think that, with commoditization of models and cheaper inference, big tech actually wins? Have you seen the stock market today? NVIDIA and the others have been hit hard. How do you think about that?

What you see is a bunch of people who are concerned about training and the need for it, with everyone still thinking that most compute is training. They see someone training a model on 2,000 GPUs—the nerfed H800 version with slower memory, or whatever it is—and they say, “People aren’t going to need as many chips.”

But think about Jevons’s Paradox: the more you bring the cost down, the more people consume.

For the last 5 or 6 decades, like clockwork, once a decade, the cost of compute has gone down by a factor of 1,000. People buy 100,000 times as much compute while spending 100 times as much. Every decade, they spend 100 times as much.

You make it cheaper, and people want more. Every time one of these models gets cheaper, we see our developer count skyrocket. It goes up, comes back down a little bit, but the slope is higher than when it started.

Better models create more demand for inference. More demand for inference leads people to say, “I should train a better model,” and the cycle continues.

I just bought a whole lot of NVIDIA because the stock dropped 16%, on the thesis that increasing efficiency obviously means we won’t need as many NVIDIA chips. I thought exactly what you said: you’ll still need NVIDIA for inference, and you’ll just have much higher usage.

To me, it’s the most screaming buy of the century. Do you share my optimism on NVIDIA, given what you just said about Jevons’s Paradox?

Over the long term, I’d say the only thing I can say is what Warren Buffett and Charlie Munger said: in the short term, the market is a popularity contest; in the long term, it’s a weighing machine.

I can’t tell you about the popularity contest. But in terms of the weighing-machine part, there’s a misunderstanding. NVIDIA is actually more valuable thanks to DeepSeek, not less valuable.

Jevons’s Paradox was discovered by William Stanley Jevons and was recently made famous in Sacha’s tweet. However, I beat him to it by quite a bit.

Just as Sacha likes to say that he made Google dance, I’m going to say that I made Sacha dance. He might take exception to that, but less than 1 month before he posted that, I did a cute little tweet on it.

What was really happening in the 1860s was that Jevons wrote a treatise on steam engines, which I guess is what you did for fun back then in England. He realized that every time steam engines became more efficient, people would buy more coal. That’s the paradox.

But if you think about it from a business point of view, when the opex comes down, more activities come into the money. People do more things.

Every time we’ve seen the cost of tokens for a particular level of model quality come down, we’ve seen demand grow significantly. Price elasticity.

11. Nvidia's High Margins and the Strength of Their Moat

A lot of people suggest that NVIDIA’s incredible high-margin status—and I’m going to butcher this; I can’t remember what it was in the latest release, but it was 45% or whatever it was, and very, very high—relates to your margins as my opportunity.

Do you think your margin is my opportunity, or do you think that defensibility is that margin today?

There’s a wonderful business selling mainframes with a pretty juicy margin because no one seems to want to enter that business.

Training is a niche market with very high margins. When I say niche, it’s still going to be worth hundreds of billions of dollars a year long term. But inference is the larger market.

I don’t know that NVIDIA will ever see it this way, but I do think that those of us focusing on inference and building things specifically for it are probably the best thing that’s ever happened for NVIDIA stock, because we’ll take on the low-margin, high-volume inference so that NVIDIA can keep its margins nice and high.

Do you think the world sees this?

No. We raised some money in late 2024, and in that fundraise we still had to explain to people why inference was going to be a larger business than training.

Remember, this was our thesis when we started 8 years ago. I struggle to understand why people think training is going to be bigger. It just doesn’t make sense.

For anyone who doesn’t know, training is where you create the model, and inference is where you use the model. You want to become a heart surgeon, you spend years training, and then you spend more years practicing. Practicing is inference.

12. The Future of Efficiency After Nvidia's Success

I’m thrilled to hear you share your optimism around NVIDIA. Where does efficiency go from here? Everyone was shocked by how much more efficient R1 is and what we’ve seen from it. What’s next?

What you’re going to see is everyone else starting to use this mixture-of-experts approach.

Just so I understand, is that the segmentation of where information goes, so that it’s routed to the optimal part of the model?

Yes. It’s called MoE, which stands for mixture of experts.

When you use Llama 70B, you use every single parameter in that model. When you use Mixtral 8x7B, you use 2 of the roughly 8 experts, although there are some shared weights on top of that. It’s much smaller, and while it doesn’t correlate exactly, the number of parameters correlates very closely with how much compute you’re performing.

Take the R1 model. I believe it’s about 671 billion parameters, versus 70 billion for Llama. There’s also a 405 billion-parameter dense model, but let’s focus on 70 versus 671.

I believe there are roughly 250 experts, each of which is somewhere around 2 billion parameters. Then it picks a small number—maybe 8, 16, or 32 of them; I’m forgetting exactly which—and only needs to do the compute for those.

That means you get to skip most of it, sort of like your brain. Not every neuron in your brain fires when I say something to you about the stock market. The neurons about playing football don’t fire. That’s the intuition.

Previously, it was famously reported that GPT-4 had, I believe, something like 16 experts, and they got it down to 8. I forget the exact numbers, but it started off larger and they shrank it a little.

With the DeepSeek model, they’ve gone in the opposite direction. They’ve gone to a very large number of experts. The more parameters you have, it’s like having more neurons: it’s easier to retain the information that comes in.

By having more parameters, they’re able to get good results with a smaller amount of data. Because it’s sparse, because it’s a mixture of experts, they’re not doing as much computation.

Part of the cleverness was figuring out how they could have so many experts, how it could be so sparse, and how they could skip so many of the parameters.

If we take that as where we are—how DeepSeek became so efficient—what’s the next stage? All the experts can be routed so efficiently. What happens now?

Here’s a fun one: Meta recently released Llama 3.3 70B, and it outperformed its Llama 3.1 405B. Its new 70B outperformed its 405B.

What was surprising to me was that I thought they had retrained it from scratch. It turns out, when you read the paper, they just fine-tuned it. They used a relatively small amount of data to make it much better.

Again, this goes to the quality of the data. They had higher-quality data, took their old model, trained it, and made it much better. That new 70B outperforms their previous 405B.

What you’re going to see now is that everyone has seen the DeepSeek architecture and is going to say, “I have hundreds of thousands of GPUs. I’m now going to use a lot of them to create a lot of synthetic data, and then I’m going to train the hell out of this model.”

The other thing is that, while it’s sort of asymptotic, the question is where you stop on this curve. It depends on how many people you have doing inference.

You can either make the model bigger, which makes it more expensive and means you train it on less data, or you make it smaller and cheaper to run, but you have to train it more.

DeepSeek didn’t have a lot of users until recently, so it would never have made sense for them to train it a lot. They would much rather have a bigger model. Now what you’re going to see is all these other people either making smaller models or trying to make higher-quality models of the same size by training them more.

We’ve seen DeepSeek now say that only Chinese phone numbers can log in. That’s a new sign-up restriction. What has happened, and what’s the result?

They ran out of compute. This is another reason chip startups are going to do just fine. You train it once, but then you need inference compute.

You spend money to make the model, like designing a car, but then each car you build costs you money. Each query you serve requires hardware.

Training scales with the number of machine-learning researchers you have. Inference scales with the number of end users you have.

Do you think DeepSeek is truly astonished by the response it’s received from the global community, or did it know this would happen?

I think it marketed very well. You look at some of the publications, and they make it sound like it’s a philosophical thing. They talk about spending $6 million on the GPUs, and everyone zoomed in on that, neglecting the fact that Llama’s first model was trained on about $5 million worth of GPU time and set the world on fire in a good way.

They ignored the fact that DeepSeek spent a ton generating the data and doing all of this. They’re really good at marketing. I think they were probably surprised at how well it worked, but I think this is what they were going for.

Is there anything I haven’t asked, or that we haven’t spoken about, that we should?

Maybe ask what’s up with the $500 billion Stargate effort.

What’s up with the $500 billion Stargate effort? Do you buy those numbers?

I’ve gone back and forth on that. Gavin Baker tweeted some math, and before I saw that tweet, I came up with very similar math—spookily similar math.

However, talking to some people in the know, some of the comments are that they’ve got it. Then you keep pressing, and it’s like, maybe there’s some cutesiness to it.

What I think it is, is an acknowledgment that the models have been commoditized and infrastructure is what’s important in terms of maintaining elite scale. Scale is one of the 7 Powers.

What you’re seeing is an attempt to move from having a cornered resource or something like that into scale economies.

Do you think it will work?

I don’t think you get there in a short period of time with GPUs, because most of the compute is inference. If you’re talking about building out all the power and all the infrastructure, it’s going to take time. It’s infrastructure. It’s capex.

I think the real win here is brand. That’s what I would be doubling down on. I would hire the best brand firms I could and do a complete makeover.

Will OpenAI have a stronger or weaker brand in 3 years’ time?

Much stronger. I think they’re going to double down on that and focus on it.

Who will lose?

People who can’t adapt to disruption. Anyone who just wants to keep going in a straight line and do what they were doing before is going to lose.

The rate of disruption is probably going to increase. Think about it this way: going back to the analogy of LLMs being the printing press, imagine if there were a couple of smartphones left over from an ancient civilization.

All of a sudden, the printing press is invented, and you say, “Uber’s coming. I want to position for it.” You know where this is going. We are the smartphones. We know where generative AI technology goes.

Now everyone is saying, “We know how big this gets. Let’s put money into it. I can’t be the one who doesn’t spend money on this, because I know how big an advantage it’s going to be.” It’s like getting to add more workers to the workforce.

I think the generative age is going to be speed-run faster than whatever comes next, because we know what it looks like.

Is there any chance we see a plateau? We saw it in self-driving, where we went through this desert of slower progress and suddenly, all at once, it came. Will we see that, or will we just see this continue to accelerate?

With self-driving, the problem was that the threshold had to be way higher. If you look at the number of miles driven by these self-driving vehicles, it’s an enormous number, and the number of fatalities and incidents is lower per mile.

But we have no tolerance whatsoever for that when it’s a machine. When you’re writing poetry or code, it’s very different from doing surgery or driving a car.

How are you feeling, and do you feel better or worse post this?

I would probably feel both better and worse. I would feel better about my bet on building out more hardware. I would feel worse about trying to build out my own model. Why is Elon doing that? There’s plenty to choose from—just pick one up off the ground. Why are you making your own?

13. Excitement or Nerves in the AI Arms Race?

Are you excited when you look forward at the next few years, or are you quite nervous? You could say this is a time of heightened international warfare in terms of this new AI arms race: China stealing everything, the U.S. forced to steal back.

Long ago, I stopped having good days and bad days. It’s how many good things; it’s how many bad things. When you run an organization, I’m both excited and nervous, and I’m excited and nervous about different things at the same time.

The thing that I am most nervous about is that, unlike nuclear war, you can use AI tools to attack each other. Google just announced recently the first zero-day exploit found by an LLM that was previously unknown.

Yeah, that’s a scary one. So now, just for anyone who doesn’t know what zero-days are, how would you like me to have access to your phone?

Not ideal. How would you like the CCP to have access to your phone? Even less so. That’s a nation-state, and nation-states have a lot of resources. If they stand up a bunch of compute and start scanning for vulnerabilities in all the open source that’s out there—and not even the open source, just scanning ports on the internet and trying to figure out if they can break in—they can automate that now. They don’t need to hire people to do that.

Now the defense has to be automated because there’s no way to keep up with automated attackers. What happens if this gets out of control? But worse, it’s small enough—it’s not killing anyone—and it’s also deniable. That’s the hardest part about it, because is it really China? Is it Russia? Is it North Korea? Is it a friendly that’s making it seem like it’s one of them, or vice versa?

You go from where we had a Cold War, because having a war was unconscionable—it was unthinkable because of the consequences—to, “Yeah, I’m just hacking you,” and that could spiral out of control. I’m worried that we’re going to have more back-and-forth.

Think of it this way: If you are a nation-state and, let’s say, Harry, you’re a beacon to the venture community and you want to rally the European entrepreneurs to be risk-on, and I’m someone who doesn’t want that because I don’t want the competition, a country that doesn’t want that could sully your reputation. Maybe I make you persona non grata. How is that any worse than shooting someone? It could be worse in some ways, but you can get away with it. That has me nervous—really nervous.

But I’m also really excited, because we are seriously going to be able to innovate as fast as we can come up with ideas now. You’re not going to have to implement things; you’re going to be able to prompt-engineer your way through things. Just as we moved from hardware engineers to software engineers and sped up productivity, you’re now going to be able to have a prompt engineer who doesn’t even write software.

One of our engineers made this app where you can just describe what you want built, and it builds it. Because we’re so fast, you just iterate, and it’ll build an app for you—crazy things.

I just don’t understand where the value accrues then, because you mentioned that they created this tool, which allows you to prompt and build the app. I’m sure you’ve seen Bolt.new. I’m not sure if you’ve seen Lovable, where it’s basically ChatGPT but for website creation, in its bluntest terms. Is there value in that?

Everyone was like, “There’s no value in these wrapper apps.” Everyone’s like, “There’s no value in these foundation models.” Where the fuck is that value?

That’s part of the exciting part—it’s discovering that. I think people will always prefer to use the highest-quality, most polished product. I think there is an opportunity for artisanship and craftsmanship, and just perfecting it and getting to a certain number of nines in the details.

There’s the Eames quote: “The details aren’t the details. The details are the thing.” I used to be a little concerned with the quote, “If you’re not ashamed of the quality of your first release, you’ve waited too long,” because there’s a subtlety and nuance there. There’s soundness, and then there’s completeness.

What you want is an incomplete product—something that doesn’t do everything. That’s why you should be embarrassed. But it shouldn’t give you the blue screen of death. That’s not a good embarrassment. What you’re going to see now is that, because it’s so easy to come up with something that just kind of works, people are really going to value well-crafted, high-quality products.

Jonathan, I cannot thank you enough for breaking down so many different elements for me and putting up with my basic questions. You’ve been fantastic, and honestly, I so appreciate the short notice.

No problem. Good luck, and have fun out there. This is a brand-new age. It really is.