AI Token 的供需｜Dylan Patel 访谈

What used to matter a lot was that execution was very, very difficult, and ideas were cheap. Now ideas are cheap and plentiful, but execution is very easy. So really, only the good ideas are the ones that can justify the spend on super-cheap implementation.

You told me this incredible story about how your own team’s use of tokens has changed dramatically this year. Will you tell that story and what it is teaching you about what’s going on in the world?

Yeah. Last year, we thought we were heavy users of AI. Everyone’s using ChatGPT, everyone’s using Claude, everyone’s got whatever subscriptions they want—I’m providing whatever subscriptions anyone wants. It was on the order of tens of thousands of dollars in spend for our firm.

1. Surging AI Spend

This year, the spend has just skyrocketed. It really started in late December with Opus, and that included Doug O’Laughlin, who’s president. He’s very much leading the charge in the sense of nontechnical people using AI for coding, and so he’s basically pulled the whole firm slowly over time. I think he’s been the leader in doing that. Obviously, the engineers were using it anyway, but spend in January just started to inflect and rocket and rocket and rocket and rocket.

We signed an enterprise contract with Anthropic, and it’s gone to the point where now, I think when I last talked to you, it was a $5 million spend rate. It’s actually a $7 million spend rate now.

That was last week, by the way.

A lot of that is just the usage, right? People who have never coded before are using Claude Code and spending thousands of dollars, sometimes a day. But across the firm, we’re spending $7 million a year on Claude Code at the current rate, versus our salary expense being in the neighborhood of $25 million. So we’re north of 25% of spend on Claude Code as a percentage of salary.

If this trajectory continues, then we’ll spend more than 100% by the end of the year, which is a bit terrifying. Thankfully, I don’t have to decide between people and AI because our company’s growing so fast. It’s more like, okay, well, I don’t have to hire nearly as fast, and I can spend a lot more on AI, and it works, and we just grow faster.

But I think other folks will start to reckon with the fact that, huh, if this person can do the work of 5 to 10 to 15 people using Claude Code, then all of a sudden I should probably cut people. But right now, I think the use cases are so broad.

For example, one thing is that we have a reverse-engineering lab in Oregon that we’ve been building for a year and a half. We have a bunch of fancy micro subs and scanning electron microscopes. The whole purpose of this is that you reverse-engineer chips: you get the architecture out of them, and you get the materials that they’re using to manufacture. This is some of the data we sell. This is a very slow process of analyzing that data.

Instead, one person on the team has been able to spend a couple thousand dollars’ worth of Claude tokens to create this application that is GPU-accelerated, runs on a server that we have at CoreWeave, and, any time we send it an image, it’s able to take the picture of the chip and overlay where every single material is. Oh, this part is copper. Oh, this part of the gate is tantalum. This part of the gate is germanium. This part of the gate is cobalt.

And so you can do a finite-element analysis of the entire stack-up of the chip very, very quickly, visually, with a dashboard GUI—it’s everything. It took a few thousand dollars of Claude tokens. The person previously worked at Intel, and he said that was an entire team’s job to build that and maintain that. Now rack that up across the entire firm. It’s insane.

Another example that I think is super fun is Malcolm. He was an economist at a major bank before. Their economics department was like 100 or 200 people. What he built was the most incredible thing ever. He piped all of this different data—FRED data and all these other data sets, right? Employment reports and all these other things from various APIs. We signed a couple of contracts with folks to get API access to data.

He pulled it all in, started running regressions, started looking at the impact of various economic variables on the economy from a deflationary and inflationary perspective. The BLS—the Bureau of Labor Statistics—has this entire set of 2,000 tasks. And so he did that with AI: which ones can be done by AI, which ones cannot, and grading them across a rubric. About 3% are doable now with AI, and so he’s created this metric so that you can measure the things that can be done by AI, what the massive deflationary impact is, what the cost of being able to do those with AI is, and therefore the deflationary aspect of it.

Output can go up. It’s called Phantom GDP, is what he’s called it. Phantom GDP. Output can go up, but because cost falls so much, actually GDP theoretically shrinks. So he created this whole analysis and a brand-new benchmark of language models, a set of evals across 2,000 different evals, right?

He does it all by himself?

He does it all by himself, yeah. And he’s like, “Dude, this would have taken a team of 200 economists a year.” And he’s just—he’s completely cracked out on Claude. He’s like, “Everything has changed.”

How do you think about it as a business owner, going from close to zero to 25% and accelerating toward whatever percent of total spend? At what point are you like, “Whoa, I need to put the brakes on this and be careful how much we’re spending? Maybe we don’t need to spend on the most cutting-edge Opus 4.7, which came out today. Maybe I can throttle it back to something that’s a little bit cheaper”?

Ultimately, I’m in the information business, right? We sell analysis, we do consulting, we create data sets. I don’t see why this wouldn’t be completely commoditized on a pretty rapid basis if I’m not constantly improving. My first product that I was selling as a data set—actually, there are more people trying to do it now. We’ve made it constantly better and better and better and more detailed, and so therefore it sells in a market, but the way we were doing it in 2023 is not terribly different than what everyone else is doing now.

If I don’t move up the bar, then I will be commoditized. If I don’t move fast enough, I will also lose my edge. So the question is, yes, AI commoditizes things, just like it commoditizes software. Those who can move fast and keep control of their customers and keep providing them an awesome service and keep improving the service won’t shrink; they’ll grow. They’ll grow faster. Those who are incumbents and not doing anything, they’re going to lose.

And so it’s a bit of an existential—if I don’t adopt AI, someone else will, and they will beat me. Another easy example is the energy space. So we’ve had a few energy analysts for like a year now. We’ve been trying to build out this energy model. It’s very complex. The energy data-services market is something like $900 million.

So obviously, a huge market for me to try and break into, but we really hadn’t broken into the energy data-services business despite a year of having multiple people on the team. Then Claude Code psychosis hits. One of the people who leads the data-center energy and industrial business at SemiAnalysis is Jeremy. It hits him, and now all of a sudden, in 3 weeks, he spent a lot on it.

He was spending like $6,000 a day. It was an insane amount, but he scraped every single power plant in the U.S., every single transmission line above a certain voltage, and created this entire mapping of the entire U.S. grid, as well as a lot of demand sources, all from various public sources of data. We’ve built it, and it’s got this dashboard where you can view and check. You can see all the microregions of the U.S. where there are power deficits and surpluses—all of these details, built in a handful of weeks.

We started showing some of our customers who buy our data-center data set, but we also showed some of our energy traders. And they’re like, “Wow, how long did this take you? This is really good. This is better than XYZ company.” Then we dig deeper: XYZ company has 100 people and has been working on this for a decade.

Now, obviously, our thing is not fully as robust, but in some ways it is better. I’m going to commoditize these energy-services companies, these data-services companies. Who’s going to commoditize me if I don’t move faster? And so the question from a business owner’s perspective is, yeah, I’m spending a lot, but what is that spend getting me? Is it getting more revenue? Yeah.

Are you worried that, in the limit, the people who control and invest capital, who were often hiring you for what you do, will just say, “Well, we have analysts, too, who are really smart about this. We’ll just build this ourselves”? If it’s getting that easy, at what point does it all pool into the investment firms that stand to gain the most because they have the most leverage on top of the data or insights that they glean?

First of all, any information services business—obviously, I don’t generate as much value as my customer does from that information. If I sell you information for $1, you’re only buying it for $1 because you know that information helps you make a decision that lets you make more than $1. Therefore, you have made more money off of me than I did from the information myself.

2. Token Demand

These investment funds all have their own information services, especially the Jane Streets and Citadels of the world. They’re really detailed on their data. Yet these folks also purchase data from us, continue to do so, and continue to grow with us because I think there’s just some “it” factor, right?

We move faster, we’re more nimble, and we’re a smaller team that’s focused on just one specific thing: AI infrastructure and the huge revolution that causes in AI, tokenomics, and all these things. We really see where it’s headed, so we’re moving faster and building faster.

I think investment professionals would try to build some of the stuff we do. More likely, they’ll just buy the data from us. It’s cheaper for them to buy the data from us and then build on top of it than it is to build it themselves. But ultimately, some may try.

I feel like every conversation I have with you, what I’m always getting at is just the supply and demand of tokens. That’s the thing that’s interesting to me in the world right now. What has this experience taught you about the demand? Has it changed your view on the demand side of that equation, just feeling it viscerally yourself?

If we take a step back and look at the macro lens, right? Anthropic has gone from $9 billion in revenue to what? They’re at $35 billion or $40 billion now. Probably by the time this airs, $40 billion or $45 billion. Who knows? [Laughter]

Their compute has not grown to the same degree. If you do the calculations and assume they didn’t decrease their research and development compute, they clearly didn’t. They released Mythos. They have Opus 4.7. So they clearly didn’t decrease their research compute spend.

Ultimately, what they’ve done is, even if you assume all the incremental compute they’ve gotten has gone toward inference, their margins are at a floor of 72%. In reality, some of that incremental compute probably went to research and development. Their gross margins may be higher than 72%.

To be clear, at the start of the year, there was a leak by someone from their funding-round documents. Someone leaked it: 30-something percent gross margins. Where on earth does a business like this grow margins like that? It’s supply and demand, right? Their demand is so high, they’re able to cut back on usage limits, rate limits, and all these things.

What really matters is having an Anthropic representative and having an enterprise contract with them, getting the rate-limit increases that you need. Otherwise, tokens are ultimately super, super in demand. Whoever can pay for them gets them.

Anthropic has the same problem—or, not a problem. It’s just the reality of how capitalism works. People are sending them $40 billion in annual recurring revenue for tokens, but those tokens are generating way more than $40 billion in value. Various businesses will have different value generation per token.

As we get more and more intelligent, what really matters is access to these most intelligent tokens and leveraging them for things. You, as a person, have to decide what the best way is to leverage these tokens to grow a business and generate value. A lot of folks will want tokens and generate value, but the shitty SaaS startup in San Francisco that’s using Claude to generate its software product is not necessarily creating a ton of value. Therefore, they’re going to get priced out of tokens soon enough.

Are you at all surprised that I had this experience just today? On the flight here, I got rate-limited on something. I saw that 4.7 came out, and what I immediately wanted was to be on 4.7 that second. I just couldn’t think about using 4.6 anymore now that 4.7 was out. I was perfectly happy with 4.6 for the last many weeks. It’s amazing. Are you surprised that people are so insistent on going to the most expensive, leading-edge thing to the degree they are?

Without a doubt. One of my funniest memories in the past month and a half is myself and a buddy of mine, Leopold, being on our knees in front of an Anthropic co-founder, begging him for access to Mythos and then pretending it doesn’t exist. [Laughter] We knew it existed, and we were like, “Please give us access.” He’s like, “I don’t know what you’re talking about.”

What was your reaction to that rate card, or that eval card, coming out?

It was rumored in the Bay Area. Everyone knew it was supposed to be really good, but if you just look at the benchmarks—obviously, benchmarks change over time—Mythos is potentially the biggest step up in model capabilities in 2 years. I think that’s a really important detail.

It’s so good that they don’t want to release it, even though they already announced the price to the people to whom they did a selective release for Cyber 4, and it’s 5 or 10 times the token cost. They just don’t want to release it because they’re worried about the impact on the world.

They’re releasing a shitty, worse version, Opus 4.7, to us. They explicitly said in the model card, “Hey, we actually preferentially made it worse at cyber.” I don’t know if you read that.

Whoever you are, if you have enough capital, you should get a freaking enterprise Anthropic subscription where you pay per token, not with these subscriptions, because then you won’t get rate-limited as much. You need to figure out how to leverage those tokens for the highest-value tasks and make money off of it.

Ultimately, what you’re doing—maybe a year from now or 2 years from now—is arbitraging tokens. The tokens are amazing, but let’s figure out what direction to point them in. Then, 3 or 4 years from now, the model will know what to do with the tokens and how to make the most value.

You can look at this retroactively. Pick any benchmark. The cost to hit a certain capability tier used to cost X, and now it costs 1/100 or 1/1,000 of that. DeepSeek, for example, on GPT-4 was 1/600th the cost. Since then, the costs have fallen further for GPT-4-class models.

Of course, no one gives a crap about GPT-4-class models. They want the frontier because the frontier lets them create economically valuable things. But GPT-4-class models can still be used in stuff, so people are using them in some tiny use cases. It’s just that the costs have fallen so fast.

It’s not really what’s driving the demand. What’s driving the demand is all these new use cases. Current Opus 4.6- or Opus 4.7-tier models a year from now, my spend for the same exact quality of the model would probably be around $70,000. I bet it’ll be 100 times cheaper.

3. When Ideas Are Cheap and Execution is Easy

That’s irrelevant because I’m going to be using a way, way better model that can do way, way better things. Anthropic Mythos is more expensive as a model, but it spends a lot fewer tokens to do the thing. Therefore, it’s actually cheaper in most tasks than Opus 4.6 because it’s just way more efficient, even though each individual token is smarter.

When I last saw you, Mythos had just come out, maybe the day before, or the card had just come out. You said something like it actually made you feel a little scared. It was so good. What did you mean by that?

Anthropic’s whole goal in 2025—and even for a lot of 2024—was, “By the end of 2025, we need an L4 software engineer in our model.” By and large, they achieved that with Opus 4.6.

What they didn’t say is that, if you look at Mythos and compare the benchmarks, it’s like an L6 engineer. L4 is pretty new. L6 is quite well-experienced. I think Anthropic said that the model was internally available in February.

So, in 2 months, they’ve gone from an L4 engineer to an L6 engineer. What’s next? When you think about model progress, it’s only accelerated. Anthropic’s release cadence has compressed.

OpenAI’s release cadence has compressed. Why? Because, generally, to make a better model, you need a few things, right? You need amazing compute. Compute is very expensive, and it has a time scale that we track, and it’s growing, but it’s sort of set in stone for the short term.

There will be delays and shifts, and somehow you can find a little more, but it’s generally pretty set in stone based on what you’ve already signed. There are amazing researchers that people are paying tens of millions of dollars for. Lastly, there’s implementation.

Historically, implementation has been very difficult. If I have an idea, now I have to implement it. Implementing is hard. Now, ideas are there, and implementation is very easy. It’s expensive, but it’s very easy.

So how does one decide what ideas to implement? It turns out that if your implementation is just so much easier, now you can implement more ideas and move on the treadmill faster and faster and faster. Whether that is AI model research and scenario modeling—the model release cadence has shrunk to 2 months from where it was 6 months before—or, hey, I want to take every power plant in the U.S. and every transmission line and model it and run regressions and see the micro supply and demand, I can also do that.

The idea is cheap. Which idea makes sense? Which idea is worth the capital that you have to spend on the tokens, because the implementation is there? That’s, I think, the key learning. If implementation costs continue to tank, which they are, we don’t even have benchmarks yet. It’s only been a handful of hours since Opus 4.7 launched, but my team is pretty excited about it internally.

What now comes to the world? It’s a complete reordering of how economies work. What used to matter a lot was that execution was very, very difficult, and ideas were cheap. Now, ideas are cheap and plentiful, but execution is very easy.

Really, only the good ideas are worth it—the ones that can justify the spend on super-cheap implementation.

So, are you actually scared, or does it just introduce some uncertainty that’s hard to grapple with?

Uncertainty is there, but I do think that causes some fear in terms of how society reforms itself. How does one exist in a world where your ability to implement something is not actually that important? Your ability to choose the correct idea for AI to implement, and then your ability to sell that idea or sell what the AI has implemented, is what matters. Your ability to garner capital toward that is what matters.

Going back to the point that it’s very important to have the newest model always, who’s going to have access to the newest model? Anthropic has a project—I know it’s not called Earwig, but I troll Anthropic people by calling it Earwig. [likely Glasswing?], Anthropic Earwig, where they only release Mythos to certain companies for cybersecurity—that’s just going to be something that continues. Models will have less and less broad deployment.

I know OpenAI and Anthropic and all these people are like, “We want to have great AI for everyone.” AI is very expensive. Who’s going to pay for the trillion dollars of infrastructure? People who have money and can build useful things with AI.

You don’t want people to distill your model, so you don’t release them broadly. You release them to a smaller and smaller set of customers. Those customers are also now wrestling over the tokens. Unless Anthropic jacks them up, they could double their pricing on Opus and I would continue to pay, and I bet most users would continue to pay.

4. Model Hoarding

I bet that wouldn’t solve their humongous capacity problem. So then the question becomes, where does this cycle end, where token usage—and therefore the benefits of those tokens, the additional value generated on top of those tokens—aggregates among fewer and fewer companies?

I don’t have Mythos. You know who has Mythos? Top freaking banks. Now, they’re only using it for cybersecurity, but at some point I can envision a world where, hey, maybe because I have an enterprise Anthropic contract and because Anthropic people kind of like me, they’re willing to give us slightly earlier access or slightly higher rate limits or something for a model. I hope that’s what happens. Then my competitor, whoever that is, doesn’t have that, and I’m able to crush them.

There are people like Ken Griffin of Citadel who are super well-connected and super rich, and he just signs a deal with OpenAI or Anthropic that’s like, “Yeah, I’m going to get access to your models, and I’ll buy the first $10 billion worth of tokens each year. So whenever you release the model, I’ll spend the first 10 billion tokens, and then everyone else can get the model after that.”

It’s like, “Okay, well, now what does that do?” Well, now he’s going to crush everyone in the market. That’s just an example. It could be cyber, like Anthropic is worried about: “Oh, now I can hack people.” It could be an information-services business like myself, where I crush someone else.

I think it’s such a broad base. We don’t know what these models can do. Anthropic doesn’t know what these models can do. No one knows what these models can do. It’s up to the end user to figure out where they can leverage the tokens to see what they can build and imagine, which is tremendously productive and uplifting for humanity. But then what happens to the concentration of resources and usage of it?

Patrick O’Shaughnessy

Presumably, right now robotics, or robots, consume relatively zero tokens versus everything else. Do you see—what’s your view of that? If that’s like a second demand curve that could start to ratchet, there’s a new startup every single day, within a mile of here, trying to build something interesting in robotics.

5. Robotics

There’s this concept of a software-only singularity, which is that the world has an AI singularity but only in software. What about the rest of the world? The vast majority of the world is physical. You can see the world orient around hardware, not software.

That’s actually why I think a software-only singularity is just a blip and not, you know, something where we get everything else, because once software is super easy, what makes robots really hard? It’s programming microcontrollers and actuators and controlling all this stuff, which is very difficult.

Right now, the interesting thing about AI models is that they’re really inefficient at learning. It’s just that we’re able to give them so much data that they’re able to learn and surpass us in certain ways. Robots—currently, robot models, VLAs, or vision-language-action models, which are very popular right now—are probably not going to be what ultimately scales beyond that. They’re inefficient in terms of data, and we can’t scale the data for them fast enough.

There is going to be some way to large-scale pretrain robot models where, just like humans, they see all this data throughout their lives. What’s interesting is that humans—the reason why we’re so good is that we’re sample-efficient. With 1 example, we’re good.

Applying that to robotics, once you have this software-only singularity, implementation is super cheap. Anyone can start to build these models, and now robots are actually useful. I think in the next 6 to 18 months, we’ll start seeing real breakthroughs in robotics that enable few-shot learning.

That is, there’s a pre-trained robot model, and now there’s a robot that you have hired or bought or whatever. You show it a few examples, and it’s able to do it. You show it how to stack these 2 things, or you tell it, “Hey, this can actually balance perfectly,” and it starts doing these things.

Patrick O’Shaughnessy

Job. [laughter]

One shot. No, trust me, I’ve spilled many a time. So I think robots will get few-shot learning. Right now, there are a lot of companies doing robots for advertisement or robots for simple stuff like that, but it’ll be like, “Oh, folding clothes.” It’s going to get really niche, like robots just for cleaning chalkboards.

It’ll be a rental service, or it’ll be a model package that you download onto your standard robot that then does that, right? You pay for that. Anyways, there’ll be a huge explosion in physical-goods acceleration and deflationary effects there. That’s ultimately going to keep token demand going crazy, and I don’t think token demand slows down personally.

Patrick O’Shaughnessy

Did you learn anything else about the world based on Mythos’s results and how it was built? My way of asking is: if you break down the components of scaling laws, like pre-training—

It is a materially larger model than prior models. And so, yes, it is a much larger model. What chip it’s trained on is not really relevant. It’s the scale.

Obviously, 100,000 Blackwells is equivalent to hundreds of thousands of prior-generation chips. TPUs and Trainium have different release cadences, so it’s not exactly mirrored 1-to-1. Ultimately, yes, Mythos is a significantly larger model.

It’s proof that the scaling laws still work. Everything about it shows that the trend line continues: more compute into a model makes the model better. Along the whole way, it’s not just that more compute into a model makes the model better; we’re also getting these compute-efficiency wins.

All this research compute that the labs are spending is actually turning into compute efficiency. If I want X capability from your model, every 6 months—or every 2 months—that cost is dramatically decreasing.

But then if I scale it up massively, I get a humongous capability jump as well. And so, yes, it's proof that this is still happening. Google and Anthropic are not heavy users of GPUs on the training side, but OpenAI—they'll start having their new class of models. I think they're taking a more sensible, principled approach to scaling in small steps. Anthropic really went for a huge jump. We'll see better and better models throughout the year, and the release cadence is only going to get faster.

Patrick O’Shaughnessy

We've gone a long way in the conversation saying almost nothing about OpenAI, which would have been so strange.

So this is the interesting thing. Everyone's like, “So, okay, so Anthropic's just won, right?” They had Mythos in February. They never even released it because they didn't feel the need to. They're already sold out. Their revenue's already adding $10 million a month. And then you've got Opus 4.7 today, all before OpenAI's alleged [spud?] release, which media such as The Information and others have posted about.

Clearly, Anthropic is in the lead, right? And OpenAI's cooked. What's interesting is that because Anthropic has such bounds on compute, and they can only grow it so fast, Dario Amodei used to gloat about how OpenAI was being too aggressive on compute and Anthropic was more sensible in their scaling. Now Anthropic is like, “I wish we had a lot more compute.”

6. The Compute Bottleneck

OpenAI is able to pay the bills perfectly fine. In fact, they've raised a ton of money to get incremental compute in addition to the irresponsible levels of compute that they are buying from Oracle, CoreWeave, SoftBank, all these people, and Microsoft. Now they're getting Trainium as well from Amazon.

So they've done this insane thing on compute, and they know they also need more. But what's interesting is, if you were to say Opus 4.6, let's ignore models getting better over time. Let's just take diffusion of this technology. You and I may jump on the model immediately, day 1, but other businesses take time, and they take time for people to learn. The spark of an “Oh, Claude psychosis” moment doesn't hit everyone at the same time.

And so, by the end of the year, let's say an Opus 4.6-tier model, the economy would spend $100 billion on. I don't think that's unreasonable. It's spending $40 billion right now.

Patrick O’Shaughnessy

That's like a linear extrapolation.

It's a linear extrapolation, not an exponential. To get the exponential, you need the better models. Anthropic won't have enough compute to do that. Presumably, OpenAI and Google will hit that tier soon enough.

Whoever hits that tier next—sure, Anthropic may get to charge 70%-plus gross margins, but if OpenAI hits it next, they charge 50% gross margins. They still get all of this incremental demand, and probably they also won't have enough compute to serve all the users.

So, sure, maybe Mythos is a model where, if the world had enough compute, it'd be $500 billion of revenue or something crazy. There is such demand for these tokens and such limitations on compute. We see this with H100 prices skyrocketing, and the useful life of these GPUs continues to extend.

It's pretty clear even the tier-2 lab is going to be sold out of tokens, let alone the tier-1 lab. The tier-1 lab will have better margins, but the tier-2 lab will be sold out, and probably the tier-3 lab will also be close to sold out. The economic value that the best model can deliver is growing faster than our ability to actually serve those tokens to people via the infrastructure.

And so this gap will continue to grow, and the model labs will continue to have expanding margins until people in the hardware supply chain and infrastructure supply chain are like, “Wait, no, why don't I just jack up my margins?”

Patrick O’Shaughnessy

So, suffice to say, I think the assessment today—or your assessment of the demand side—is completely explosive in your own particular example here at SemiAnalysis, but just more broadly, as people fall into what we can call AI psychosis, as people fall into this experience of what they can do, the implementation difficulty is going completely away. I've certainly felt that. My own token spend is just through the absolute roof in a matter of weeks. So that feels like a pretty good assessment. Anything we're missing on the demand side?

If you don't use more tokens, you'll never escape the permanent underclass.

Patrick O’Shaughnessy

Just expand on that.

So either you use more tokens and generate economic value outside the economic value for the use of those tokens. A lot of people are doing it the boring, lazy way: “Oh, I guess I'll just work 1 hour a day instead of 8 hours a day, and I'll have AI do most of my job.” That's the boring way.

The cool way is, “I'll still work 8 hours a day, and I'll do 8x the work, and maybe I'll make 5x the money.” Maybe not. You can't do this with a job, obviously. There are people who have multiple jobs. There are people who start companies and start selling stuff. Get that economic value from this AI before everyone is using it and it's table stakes, because it's still not table stakes.

If you don't use more tokens and generate the value from them and capture that value, there are 3 different problems here: using more tokens, generating value from those tokens, and capturing value from the value that you created from the tokens. If you don't do these 3 things, you'll never escape the permanent underclass—that is, as models continue to skyrocket in capability and the concentration of resources potentially happens.

Patrick O’Shaughnessy

7. The AI Permanent Underclass

All right, let's talk about supply. What is going on? How would you describe what's changing at the frontier of supplying the entire stack that's required to serve all these tokens as the demand curve explodes?

As demand skyrockets, prices are going up for everything on the supply side, whether it be the NVIDIA GPUs—their prices are going up. In addition, their useful life is extending.

Patrick O’Shaughnessy

H100 prices look like this.

Yeah, exactly. There are people who have argued GPUs' useful lives are less than 5 years—complete nonsense. There are clusters now re-signing; 3- or 4-year-old Hopper clusters are re-signing for 3 or 4 more years. There are A100 clusters that are re-signing for another couple of years.

So the useful life is clearly not 5 years; it's maybe even 7 or 8 years, arguably. We don't know yet. We'll see when Hopper gets there, but it's clearly not 5 years. So the useful life is extending, and the prices are going up on that renewal.

In effect, the gross margin was not 35% on a cluster; it's beyond that. Margins are expanding in the cloud layer. Margins are extremely healthy on the hardware layer, with NVIDIA still charging 75% or whatever percent gross margin.

8. Supply Chain Reality

As we move down the stack, memory margins have obviously skyrocketed. In places like optics and logic, there are large prepayments, and margins are growing slowly. More importantly, the companies that are making chips like NVIDIA are paying huge prepayments.

So, in effect, the cost of capital, or timing of cash flow, or return on invested capital is going up even if the gross margin isn't. You see this across the whole supply chain. You see ASML is completely sold out, and they need Carl Zeiss to expand faster.

Everyone along the chain is either sold out and margins are going up, or they're getting prepayments, which increases the return on invested capital because the invested capital is lower. And so this is a consistent trend across any part.

It's even like a PCB. To make a PCB requires copper foil, and that copper foil is sold out, and people are making prepayments for it. It's like anything and everything that has a pulse and is sold out: people are jumping to get more incremental supply and fighting over the supply for the years after.

And it’s not like we haven’t seen 18-month-long lead times in other industries. It’s just that building incremental supply didn’t take years. This is the case with memory, right? Memory can only grow capacity by low double-digit percentages a year—20%, 30% a year—even less for NAND, a little bit higher for DRAM.

Even though the demand signal was very strong at the end of 2025, the memory companies immediately started reacting. None of that incremental capacity really gets here until the capacity they’ve decided to add beyond the typical 20% to 30% comes online. They can stretch a little bit, but the true incremental supply doesn’t come until 2028—early or late 2027 at best.

As a result, memory prices have gone through the roof. And guess what? They’re going to double and triple again, at least on DRAM especially. People are like, “Oh, the memory story is overplayed. Everyone gets it.” And it’s like, “No, no, no, you don’t get it. DRAM will double or triple from here still, because that’s how much capacity is required.”

They have to steal capacity from somewhere else, and the only way to steal capacity from somewhere else in a capitalist economy is demand destruction via higher pricing. We’re not rationing stuff here. Ultimately, that’s what’s going to happen, and margins continue to go up.

I think logic also has humongous capacity problems. TSMC just had its earnings, and it keeps upping CapEx. Ultimately, it takes them quite some time to build fabs. They’re trying to do everything they can to squeeze every little bit of output out of every fab that they have.

But ultimately, they’re not raising prices fast because they’re good people, it seems like. They’re doing single-digit price increases instead of triple-digit price increases, like the memory guys have had. So you ultimately have this market where TSMC is a great company, but are they actually going to extract all the value?

I mentioned things like copper foil, glass fibers for PCBs, and lasers. These are well-understood, niche supply chains, but they’re very, very tight. Upstream, the semiconductor wafer-fabrication-equipment supply chain is one that I still think has gone up a lot but is still very underappreciated.

TSMC’s CapEx this year, they say, is $56 billion. We’ve had $57.4 billion since January, and we may up it slightly more just because we see some ways that they can get incremental CapEx. But what people aren’t focusing on is what that means next year and what that means the year after.

Maybe 2 years from now—right?—it might be 2028. TSMC may spend $100 billion on CapEx in 2028. People just can’t fathom that. But what does that mean for their downstream supply chains?

Companies like Lam Research, Applied Materials, and ASML, or their further downstream supply chains like MKS Instruments and all these other companies—the tail of the whip just gets whipped harder and harder. Ultimately, that’s a shortage if TSMC wants to spend $100 billion in 2028, which is a real possibility. I think people would think that’s insane, but that’s a real, real possibility.

Patrick O’Shaughnessy

What about other parts of the chip ecosystem where GPUs have been completely dominant? What about CPUs or ASICs, or things that start to pop out as both opportunities and bottlenecks beyond just NVIDIA’s GPU dominance?

ASICs are obviously taking off, but I’ll pivot away from AI chips to talk about these other things. There’s a project we did on FPGAs, and it turns out there are 120 FPGAs per next-generation AI rack. What about all the FPGA names?

9. CPUs

CPU-wise, all these reinforcement-learning environments, plus all the slop code you and I are generating that’s now running on some likely Vercel instance or whatever it is, or some AWS instance or some bucket that we’ve spun up—all of that requires CPU. CPUs are completely sold out, and demand is skyrocketing there.

Patrick O’Shaughnessy

Yeah, help people understand the role that CPU plays in everything.

There are 2 main reasons why you need tons of CPU. One is when you’re doing reinforcement learning, the CPU is very critical to that. Before, you would throw all the internet’s data into the model, train it, and spit some stuff out. Now you train on all the world’s internet, put all the internet data into the model, and then put it in this environment.

This environment is like, “Hey, model, try this out,” and it tries stuff out—tries a bunch of different things. In the end, there’s an environment that scores whether or not what it tried out is successful, and it grades it. These environments can be anything.

It can be, “Hey, check if the text was outputted in the right way—structured output.” It could be very simple stuff, or it could be very complex stuff. People are starting to get into very complex things, right? Like, “Hey, I want you to open this file, change it, edit it, update it, and submit it to this website.” I want you to open up this physics simulation from Siemens and edit this CAD model.

The environments can get more and more complex, and those environments run on CPUs. They don’t run on GPUs. They don’t run on ASICs. The ASICs run the model that takes the input data from the environment and runs it through the model.

The model creates outputs of various different trajectories—ways that it thinks it could solve it in different instances. Those trajectories are graded and scored. The successful ones are used for training, and you update, reiterate, and iterate. So CPUs are very useful for that, for one.

Once you have these great models and you’re deploying them, those models are generating code. They’re generating useful output. That useful output doesn’t go from a GPU straight to the human brain. It goes from a GPU or an ASIC through to a deployed app that you’re deploying somewhere, and that actually just runs on CPUs. So that’s another area where there’s a lot of demand, and things are sold out in a large way.

Patrick O’Shaughnessy

As you continue to assess and try to be the world’s best-informed person on both the trajectory of supply and demand, what are the things you wish you knew to make that understanding better—things that you don’t know?

I think the hardest area for us and for everyone is understanding tokenomics—the economics of tokens. I think we have a tremendously good insight into how much it costs to run infrastructure, what the cost of tokens are, what the cost of models are, and what the margins of these labs are. But usage and adoption are what’s really difficult to model.

In January, we had crazy estimates for February, and Anthropic smashed them. How do we calibrate this model? What are the data sources for this? In February, we had crazy assumptions for March, and then they smashed them.

Everyone sees the number of $10 billion, and they’re like, “How do they add $10 billion in revenue? Who is using all these tokens? Why are they using them? What are they building with them?”

More importantly, how is what they’re building with these tokens actually diffusing into the economy, and what value is that generating? Because it’s not really something that you can capture in any GDP statistic, right?

All of the value of the tokens that I use gets transformed into better information, which I then sell at a discount to what people used to sell information for, relatively, because that information is now making its way throughout the economy. People are making better investment decisions or better competitive decisions, whether they’re a semiconductor company, a data center company, or a hyperscaler.

How much—what is the value of this, and what has that done to the economy? It’s clearly, by every subjective metric, amazing. But where is the phantom GDP? What is the phantom GDP? How do we track the real economic value? Because the GDP metrics are not accurate.

If you were to say, “What is the GDP that Dylan Patel is making?” it’s tiny compared to the value that I think is being created. Ultimately, what is the value being created by these tokens? Not on the basis of just a simple question: What is the knock-on effect? What is the knock-on effect of all the things that these things are doing?

I think that’s the real question and challenge that’s hard to measure. I think we’ve got a tremendous reading on the supply side of things. I think we’ve got a tremendous reading on even a lot of the demand-side signals, but what value these tokens are generating—that’s hard to quantify and measure.

Patrick O’Shaughnessy

I hope we get a chance to do this every 3 months because this changes so quickly. What do you think’s going to happen next? When I come back 3 months from now and we’re in San Francisco together again, what do you expect?

Large-scale protests.

Patrick O’Shaughnessy

Really?

Yeah, I think there’ll be a large-scale protest against Anthropic and OpenAI.

Patrick O’Shaughnessy

Expand on that a little more.

People hate AI. AI is less popular than ICE, less popular than politicians. I’m confused about how Pew surveyed this, but apparently AI is less popular than politicians.

With Anthropic adding so much revenue, that’s going to start causing business changes downstream. People are going to get more and more scared of AI. They’ll start blaming more and more of their own problems on AI, as well as things that are global, deep-seated problems that have existed for a long time.

10. Predictions: Public Backlash

Those will bubble up and be blamed on AI. Probably some politician or some social media people will start to be able to take influence, or will be able to start taking and weaponizing AI against people. You look at the comments of news articles where Sam Altman had a Molotov cocktail thrown at his house twice in 2 weeks. People are cheering it on. This is just the beginning, so I think we'll see large-scale protests against AI in 3 months.

Patrick O’Shaughnessy

What is the counterweight to that? How should the AI industry head that off?

First of all, Sam Altman and Dario have to stop getting on interviews. They're so uncharismatic. I don't know what they're doing. Every interview they do is like, “Normal people are going to hate you even more.” Sam being on Tucker Carlson probably made all Republicans hate OpenAI. And same with Dario—they just have no charisma.

I think that's first. Two, they need to start showing uplifting things that can be done with AI. Three, they need to stop talking about how the capabilities are going to change the whole world constantly, because then people are going to get fear of that capability.

Patrick O’Shaughnessy

To use it.

There's no connection to it, either. The average person doesn't know an Anthropic employee. The average person doesn't know an OpenAI employee. The average person doesn't know who these people are or what their goals are, and they just view them as this sneaky cabal of 5,000 people at these companies who are going to change the world, automate all the jobs, and destroy society. That's what they view it as.

And as people who are funding the building of all these data centers and power plants that are going to pollute the world, they don't quite understand what's happening. They have to stop talking about the future thing that's going to happen and only talk about the present—how uplifting AI is. I think a huge reorg and rebranding needs to be done.

Patrick O’Shaughnessy

I love doing this with you. Thanks for your time.

Awesome, thanks.