我与 Dylan Patel 和 Stacy Rasgon 的对谈
Austin LyonsStacy RasgonDylan Patel
- 融资正成为供给约束,但 Rasgon 认为未来1到2年大概率还好。 Patel 表示,neocloud 越来越需要强劲的承购协议或芯片供应商兜底;即便是有大型 AI 实验室合同支撑的项目,债务成本也可能高达13–15%。Rasgon 认为,这些制约或许会带来“更强劲、更持久”的行情,因为它们能避免产能过度建设。
- 到2028年,基板和存储可能比电力更限制 AI 部署。 Patel 称基板是“所有瓶颈中最大的瓶颈”,并估计明年全球数据中心建设规模接近50吉瓦;现场发电有助于缓解电力限制。Rasgon 认为2028年大概率仍会吃紧,到了2029年,关键将转为需求问题。
- 基础设施价格上涨,正在重分配 AI 供应链各环节的利润。 Patel 估计,每吉瓦建设成本已超过600亿美元,高于2024年的约400亿美元;等产能建成后再出售,价格可能远高于提前签约。Rasgon 表示“通胀在 COVID 期间救了这个行业”;眼下的角力在于,哪些供应商能守住毛利率,哪些客户要承担上涨的成本。
- Nvidia 能否守住毛利率,取决于性能提升能否快于价格上涨。 Rasgon 指出,Blackwell 到 Rubin 的参考成本从每吉瓦约250亿美元升至400亿美元,对应的性能提升据称达到10倍。如果假设中的 Feynman 只快20%,他就会担心:“你得持续给客户创造价值。”
- Patel 预计 Nvidia 的 CoWoS 和 HBM 份额会下滑,但明年盈利仍至少翻倍。 他的团队称,OpenAI 的首款芯片在 InferenceX 上运行开源中国模型时胜过 Blackwell,这让“芯片设计会不会像软件一样走向商品化”成为更尖锐的问题。Rasgon 强调年度新品发布、规模和融资能力是防线;两人都将估值低迷与竞争忧虑联系起来。
- 企业采用 AI,需求或主要来自生产率提升和购买 tokens。 Rasgon 说,Claude 用80分钟完成了一周的建模工作,不过他认为回报方面的证据虽令人鼓舞,但仍处于早期。Patel 预计,批处理效率和模型规模经济会推动企业使用云端算力:“大多数人最终都会直接买 tokens。”
1. 资本约束可能拉长建设周期
- 对 Rasgon 来说,融资是首要担忧,但“未来1到2年大概率还好”。CoreWeave 约15%的无担保债收益率反映的是二级市场交易,并非其当前融资成本;它所显示的是新增无担保融资可能要付出的代价。收入能否追上,取决于当下算力短缺能持续多久。
- Patel 表示,neocloud 占建设总量近一半,但融资越来越依赖高信用等级的承购协议。即便有 Anthropic/OpenAI 合同支撑的项目,债务定价也达到13–15%。他认为,超大规模云厂商明年不可能支出超过2万亿美元,而小型客户很难通过这些渠道之外的方式为产能融资。
- Nvidia 的兜底安排可以保证每小时3.50美元的租赁价格,也就是“云厂商一分钱都赚不到”;若其他租户支付更高价格,双方再分成。Patel 举例称,Sharon AI 和 Fermi 的分成比例为50%,Lambda 的 Anthropic 集群则为10%。Broadcom 把 Anthropic 的付款分摊到几年内,实质上是“先租后买”。
- Rasgon 看好“更强劲、更持久”的行情:约束可能推迟供给,而不是重演暗光纤过度建设。Patel 表示,Meta 承诺的算力和数据中心支出达1.4万亿美元;更广泛的市场可以基于这些承诺为产能融资。
2. 材料和洁净室可能先于电力成为瓶颈
- Patel 将基板列为首要瓶颈,其次是存储,再往后是显著的光学器件约束。他认为,保守的日本材料供应商投资不足,限制了加速器、CPU 和其他零部件的供应。就连一家大型公司也无法满足其2028年全部共封装光学器件需求,受限环节包括激光器、光纤连接及相关组件。
- 面对市场对电力的关注,Patel 估计明年数据中心建设规模接近50吉瓦:美国约35吉瓦,中国约7到8吉瓦。现场涡轮机、往复式发动机和燃料电池都能提供替代方案。Rasgon 认同这些方案可行,但承认自己不确定涡轮机的交付周期。
- 存储股目前的预期市盈率为2到4倍,Rasgon 因此追问:如果市场担心的价格崩跌并未在供给于2028年前后增加时出现,会怎样?存储约占 AI 机架硅片面积的70–80%;每吉字节 HBM 所需晶圆量是 DDR5 的3到4倍。需求继续增长,可能让供给持续紧张。
- 今年晶圆制造设备的估算起初约为1200亿美元,最终可能接近1550亿美元;一家供应商对明年的初步展望已超过2000亿美元。Rasgon 表示,安装受到洁净室空间限制。部分新空间将在2027年交付,更多要到2028–29年。他认为2028年“可能仍会吃紧”;至于2029年,“我不知道”。
3. 定价权正在重分利润池
- Rasgon 的判断是,“通胀在 COVID 期间救了这个行业”。2023–24年行业出货量与2019年持平,但综合平均售价(ASP)高出50%,即便不计存储也是如此。他预计今年半导体收入约为1.5万亿美元,甚至可能更高;存储 ASP 贡献了增长的70–80%,出货量则增长约10%。
- Patel 对比称,TSMC 的毛利率从45%稳步升至60%,而存储行业几年前的毛利率还为负。TSMC 维护长期合作关系;存储供应商则会根据客户的周期性议价作出反应。如果 Apple 在下行周期压价,“周期向上时他们就会狠狠报复 Apple”。
- Patel 估计,每吉瓦建设成本已超过600亿美元,高于2024年的400亿美元。等产能建成后再出售,Musk 就可以争取每吉瓦500亿美元,而不是150亿美元;Patel 称,Anthropic 每吉瓦推理产能能创造700亿至800亿美元收入。Nvidia 提价并不意味着毛利率上升:内存等投入成本上涨,毛利率大致持平或略有下滑。
4. Nvidia 必须持续扩大性能与价格的差距
- Lyons 问客户何时才会拒绝不断上涨的成本。Rasgon 的回答是看性能:Ampere 到 Hopper,ASP 大致翻倍,性能提升约10倍;Hopper 到 Blackwell,ASP 上涨约50%,性能提升3到30倍,优化后可能更高。Blackwell 到 Rubin 的参考成本从每吉瓦约250亿美元升至400亿美元,但 Rasgon 称性能提升达到10倍。客户也能从每吉瓦算力中获得更多产出。
- Rasgon 明确表示,只要“这个差距还在扩大”,更高的价格就仍然可以接受。如果假设中的 Feynman 只快20%,他就会担心定价和毛利率。创新放缓将迫使供应商自行吸收成本,最终演变为一场“谁来买单的恶斗”。
5. 定制芯片会先挤压毛利率,增长随后才受影响
- Patel 表示,OpenAI 的首款芯片在 InferenceX 上运行开源中国模型时胜过 Blackwell;他的团队在 OpenAI 实验室进行了测试。AI 参与了芯片设计和编程。Anthropic 希望用 OpenAI 一半的开发时间做出自己的芯片,目标“很激进”,能否实现取决于模型是否能加速研发流程。
- Patel 称,让 Nvidia 在 InferenceX 上跑起来花了6个月,OpenAI 只用了1个半月。他提出疑问:芯片设计会不会像软件一样走向商品化?其中的经济诱因很大:去掉芯片设计公司的利润,他估算每吉瓦约600亿美元的成本栈可降至200亿美元。固定的光罩和 EDA 成本仍约为5000万美元,此外还有数据和规模化方面的挑战。
- Lyons 问,Nvidia 份额下滑是否意味着芯片出货量减少。Patel 更倾向于看 CoWoS 晶圆、HBM 和相关支出,因为芯片尺寸各不相同。他预计 Nvidia 明年份额还会进一步下滑,尽管盈利至少翻倍。初创公司还面临另一道门槛:Cerebras 和 Groq 说明,只有“真正疯狂”、让现有厂商难以复制的方案才有机会。
- Rasgon 强调,年度新品发布、生态系统、已部署基础设施和融资能力都是防线;Nvidia 也在用 AI 辅助设计。他认为 CUDA 的护城河仍有讨论空间。9或10个月前的一份研究发现,过去10年里 Nvidia 相对 SOX 指数估值更低的交易日只有13天;如今估值更低了。“这是一场竞赛。”
6. 企业回报不一定需要企业自购硬件
- Rasgon 表示,Nvidia 的收入大约一半来自超大规模云厂商,另一半来自非超大规模客户,但实验室需求横跨这两类客户;实验室直接和间接贡献约20%。他说:“没有企业采用,就没有这个行业。”编程已经成为一个杀手级应用。
- Rasgon 的团队此前试过用 AI 建模,结果“一塌糊涂”。最近,Claude 用1小时20分钟建出一个模型,纳入了10年历史数据、业务重分组,以及参考 CEO 讲话得出的预测。一位同事原本需要一周。Rasgon 认为,生产率回报的证据令人鼓舞,但采用仍处于早期。
- Patel 预计,大多数企业会购买 tokens,再与自己的数据结合使用。批处理效率和模型规模经济更有利于这种方式,而不是企业自己买硬件、运行模型。他还补充说,即便是制药公司和对冲基金,大部分算力和 AI 支出可能也会投向模型,而不是购买算力来运行开源模型。因此,企业采用 AI 可以扩大,而无需同步出现一波企业自建加速器的采购潮。
完整逐字稿
All right. Hello, everyone. Y’all ready for this? This should be good. This should be fun. We’re going to talk about the industry, look ahead to 2027 and 2028, and hear these two gentlemen’s thoughts about where it’s headed. We’ll try to cover it all, from financing to GPUs and CPUs, down to foundry. Let’s get into it. I actually want to start with the finance side.
1. Financing the Buildout
Lately, a lot of the news has been about the cost of capital. We know hyperscalers are investing a ton, and free cash flow is going negative. Now we’ve got the 10-year Treasury over 5%. Even if you zoom out to CoreWeave, there’s talk about its bonds yielding 15%. My question is, should we be concerned? As companies invest more and more, is it becoming more and more expensive to finance this? I’ll start with Stacy.
Look, there are a lot of constraints in general that are moderating the pace at which this buildout can happen. These are power, supply, semiconductor tools, and now maybe financing. I’m a Wall Street analyst. If you look at the stocks, they’re still up a bunch on this, but they’re clearly way down from the peaks. The market is telling you something: it’s telling me they’re worried there’s a lot of uncertainty about the pace and sustainability of the buildouts, and maybe the peak is approaching. Clearly, financing is one of those concerns. It’s probably near the top.
You mentioned a number of different reasons. The hyperscalers were the most cash-generative businesses in the history of humankind. Now they’re burning cash and probably going to be raising equity. Meta just had a new share-class structure that, among other things, will enable it to do that. We saw Google raise money, and others did too. You mentioned CoreWeave’s bonds. That’s its unsecured debt. It’s not what the company is paying right now; that’s where the bonds are trading in the market. But that would suggest that if they go out to the market today to get new unsecured debt, that’s roughly what they’ll be spending. And it’s a lot of money.
I think these are all broad concerns right now. Though I think that, at least for the next year or two—which is as far out as I can look, frankly—we’re probably okay. But I do think it’s pretty clear that revenues need to grow to the point where this becomes less of a concern. Maybe that happens. I don’t know. SpaceX reported earnings this week—or last week; I can’t remember. He’s running out, clearly he’s running out in the spot market, and things are very, very constrained. The only thing we’re hearing is that nobody has enough compute.
He’s renting capacity out on short-term contracts at something like $50 billion a gigawatt, at those kinds of rates. I don’t think it’s actually that hard to imagine a scenario where this stuff gets covered and then some. But again, it comes down to how sustainable those rates are, and that raises broader questions. It really depends on what demand actually looks like relative to supply, and how long the supply-demand mismatch lasts. These are questions Dylan would probably have a better idea about than I do. But absolutely, it is a concern. It is one of many constraints out there that I think are moderating growth.
I’ll say one more thing, though. Nobody likes the idea that these constraints are moderating growth, but at the same time, it may be a good thing in some sense, too, because it stops things from going nuts. We saw this during the tech bubble. There’s dark fiber in the ground today that was overbuilt and never used.
Some of these things may actually give us stronger-for-longer behavior, because they almost force the demand that is there to push out supply even more and allow things to be better matched, rather than going completely nuts today and risking a crash down the line. So I don’t actually think constraints in general are necessarily a bad thing. But I do know how they’re perceived, and clearly they’re impacting the trajectory of the stocks in the near term.
Yeah, totally.
Echoing Stacy’s point, in any buildout, everyone’s going to want to build. When we look at any buildout over the last few decades, we’ve always had the industry double-order, triple-order, and build too much capacity. Supply chains are really bad at forecasting demand. As Stacy mentioned, there is a bit of self-moderation, especially on the finance side. If we go back a year or two, any cloud that wanted capital could get it. They would pay a pretty high interest rate, but they could get it.
Now, if you don’t have really high-grade offtake, you’re not able to build if you’re some random cloud. Ultimately, the hyperscalers can take their free cash flow down to zero, raise some debt, and their earnings are obviously growing. So their capacity to build out grows, but they can’t do north of $2 trillion in capex next year. They can’t do $2.5 trillion in capex next year. That moderates how much they can spend. And when you look at the neocloud industry, which accounts for non-hyperscaler demand, nearly half of the stuff being built out, how are they going to get it financed if they don’t have offtake from a hyperscaler?
The market’s already saying, “Hey, if you’ve got offtake from Anthropic or OpenAI, we’re going to price it at 13%, 14%, 15% debt.” We saw that recently with Volta. They’re making a cluster for Anthropic, and their debt is basically a 12% coupon, but they sold the bonds for 95 cents. So it ends up being like a 15% interest rate. If we look at everyone who isn’t getting offtake from these companies, hyperscalers, or the major AI labs, how do you build that capacity? How do you build capacity to serve smaller enterprises? How do you build capacity to serve new startups? You literally can’t, unless it’s coming from a hyperscaler.
That’s Elon’s whole gambit: he’s making all this capacity that he can sell at very high prices on short-term rental contracts to startups and AI labs. But ultimately, it is quite a dampener. We’ve seen Nvidia recognize this as a very big problem for them. They have five customers who can buy as much as they want, or as much as they have capital for, but what about the other part of the market? And how do they enable that?
They’ve gone and done really creative structures with Sharon AI and Fermi and a few others. They’ve taken revenue shares on the cluster in order to guarantee that it’s financed. They’ll say, “Hey, we’ll rent the cluster from you at a guaranteed price of $3.50, which means the cloud makes zero money, and then if you rent it to anyone else above that price, we get a revenue share.” We’re starting to see more and more of these structures. In fact, even to build a cluster for Anthropic, Lambda had to do a revenue share with Nvidia.
In this case, it wasn’t 50%, like Sharon AI and Fermi had, but it was still a 10% revenue share. So it’s really tough for these clouds to get the capital to build, get to the point where they build everything they want to build, and reach the oversupply scenario, just because of the financing. We’re seeing a shift from long-term loans and debt to something shorter-term. Broadcom has done the same thing with Anthropic, where they’re basically leasing the chips to them. It’s rent-to-own. They’re stretching out the payments over a few years.
The whole name of the game here is that if anyone wants to get capacity and they’re not one of the big companies, they’ve got to come with a deal, a structure that has Nvidia, Broadcom, or AMD backstopping these clusters. Ultimately, the financing is a big dampener. The hyperscalers also have very large off-balance-sheet and on-balance-sheet commitments. Meta has $1.4 trillion of commitments for compute and data centers, and so on and so forth.
That can then get financed by the whole market: Meta can get a loan and buy a data center, rent a data center from someone, or rent chips. Then those companies can turn around and get capital for it. But ultimately, there’s a big struggle for anyone to do more than that. A lot of people have worried about even Meta’s financial structure. Meta is completely fine, but even the capital markets are getting spooked by that. Financing is part of the holy trinity: land, power, shell; chips; and financing.
That’s good. That’s interesting. So it’s getting more difficult to finance. The hyperscalers are fine, but maybe it’s a little bit harder for neoclouds. We’re seeing these new creative structures come around to help with that. If we set finance aside and go back to the traditional bottlenecks that are ultimately limiting the pace of growth in the industry, talk to me about where you see things in 2027 and 2028. ASML says they’re sold out in 2028. TSMC always has more demand than supply, even though they’re doubling capacity for 2 nanometers. And then people talk about power as a bottleneck.
Dylan, I know your team recently—maybe this week—put out a post about up to 50 gigawatts coming online next year. Maybe I’ll start with you. How would you stack-rank these different parts of the semiconductor supply chain, and where should we be looking for constraints?
Yeah. On the semiconductor side, the biggest constraint, besides memory, is substrates. Substrates, PCB materials, substrate materials—all of these components were severely underinvested in. Many of the companies in Japan that create these materials are more conservative and unwilling to invest heavily in capacity. So we’ve seen tremendous bottlenecks in what can be made over the next two years, for accelerators as well as CPUs and every other component in the supply chain.
I think substrates are the biggest bottleneck of all the bottlenecks, including land, power, shell, semiconductors, and tools.
I’d say the next bottleneck would be memory. Following that, there are a lot of bottlenecks around the desire to ramp optics to a really, really large degree, with co-packaged optics or near-package optics. There are humongous constraints on ramping that. We simply cannot ramp enough, even if everyone wanted to do CPO in 2028, or even if one major company wanted to do CPO in 2028. They could not satisfy their volumes in terms of lasers, fiber attach, and all the different components that go into it.
As far as land, power, and shell, power is definitely a big constraint. But one thing our research shows, which I think is counterintuitive to much of the market, is that next year there are data center constraints, but a lot less than people realize. Nearly 50 gigawatts of data centers are being built globally: 7 or 8 gigawatts in China, about 35 in the U.S., and a bunch in the rest of the world. Australia, India, and Malaysia are some of the big places, but data centers are being built at a tremendous pace all over the world.
While power is a huge bottleneck, people can find creative ways around it, whether it’s turbines, reciprocating engines, or fuel cells on site. All of these things are helping alleviate the power bottleneck. So I think power is actually a lot less of a constraint than most people think when they say data centers and power are the biggest constraints. They’re constraints, but I think they’re a lot less significant than substrates, memory, lasers, and other things in the optics supply chain. People can ramp power capacity over a shorter time span. Those solutions are significantly less complex than the semiconductor supply chain.
Yeah, you bet. I think it’s really interesting on the power side. Jensen and Hock at Nvidia and Broadcom have given some pretty sizable guidance. Jensen said he’ll grow his revenues by at least 70%; Hock said 100%. Those are actually constrained numbers, and they’re looking primarily, I think, at land, power, and shell as their constraints. Maybe that’s because they’re semiconductor companies. They know what they can make in terms of semiconductors. But on the land, power, and shell side, I agree there are ways to get around this.
Although even for some of the things going on site, I don’t know what the lead time is on a turbine these days. Musk says he’s going to start building his own turbine blades to get around that. I think the memory side is really interesting. The memory stocks are all trading at 2, 3, or 4 times forward earnings, and pricing has gone through the roof. I don’t cover the memory stocks, by the way. A colleague of mine does, but take a look at SanDisk: they guided to quarterly EPS higher than the stock price was 24 months ago. It’s really amazing, because pricing has gone through the roof and they’re all trading at single-digit P/Es. That’s how you trade things at the peak of a cycle, and everybody’s worried that, especially as you get into 2028, supply will start to come online and pricing will collapse.
I sit here and always wonder what happens if it doesn’t. Look inside these racks. Look inside an AI rack: 70% or 80% of the silicon area inside that rack is memory. It’s mostly DRAM. There are other issues when you’re making high-bandwidth memory, the memory that goes into the AI chips. There’s something called a trade ratio, but it takes 3 or 4 times as many wafers to make a gigabyte of high-bandwidth memory as it does to make a gigabyte of standard DDR5 DRAM that would go into a PC or smartphone.
So you’re sucking up wafers from that. We need a ton of memory. And this feeds into one of the other bottlenecks, which is semicap: the actual tools to build this stuff. We had a company report this week. They gave a preliminary outlook for WFE, wafer fabrication equipment, for next year. They said more than $200 billion. I think a lot of investors thought it was going to be a lot higher than that, which I found really surprising because I just don’t think it’s possible. We don’t have the fabs.
There’s another constraint: we don’t have clean rooms. They’re all full. They’ve actually been squeezing as much as they can out of the existing footprints. That’s why the WFE estimates at the beginning of this year were maybe $120 billion. We’ll probably end the year at $155 billion. That’s all from finding better ways to use the existing footprints, but now they have to build, and it takes years to put the stuff into the ground. Clean rooms do start to come online a little in 2027, more in 2028 and 2029. At that point, I guess we’ll know if there’s a supply-demand mismatch or not.
But if demand profiles continue at the rate people seem to think they will, memory may still be a constraint. You talked about substrates, PCBs, and other things. Again, going back to the tool vendors, they’re all complaining about short-term supply constraints that are actually impacting their own ability to deliver upside even in the near term. It’s exactly that kind of thing: PCBs, semiconductor chips, and even other components, valves and mass flow controllers. Simple stuff is affecting other parts of the space as well.
We saw a little bit of this during COVID. I do think the semicap players, at least, and most of the folks in the supply chain learned some lessons from COVID. I think they’re managing that sort of thing better now than they used to. But at the same time, we’ve got all kinds of constraints in the spaces that need to get fixed before the other constraints can actually get fixed. So you have these layers to things. I don’t really see it changing. I think things are going to be tight for a while, frankly. I think this year is fine. I think next year is going to be the same thing. 2028 will probably still be tight. 2029, I don’t know. That turns into a demand question. But I suspect we’re going to be tight for a while.
Okay. So if we’re tight for the next couple of years, and we’ve talked about prices skyrocketing, with the memory industry making margins of 80%—
Higher, higher.
Yeah, 88%, 90%, right? So do you see all those players, everyone constrained up and down the whole stack, continuing to charge for their value?
Well, it’s funny. In this industry, pricing coming down was just taken as a given. The semiconductor industry as a whole was historically a massive value destroyer. It never covered its cost of capital. All of the value flowed to the consumer. I did a piece of work at McKinsey many, many years ago. It was called “Value Creation in the Semiconductor Industry.” We defined value very classically: economic profit minus WACC times your average invested capital.
We aggregated it across about 150 different companies, 12 different subsectors, over a full cycle. I think the initial work was from 1996 to 2004. Over that period, collectively, the industry destroyed $5 billion in aggregate economic profit. If you took Intel out of the mix, it was $50 billion destroyed, which was a big number back then. The industry didn’t really start covering its cost of capital, meaning that all the economic profit it had created before that was flowing directly to the consumer. It was a wonderful time to be a customer for semiconductors.
But after 2008, there was a lot of rationalization and consolidation, and at least we started to come in and meet the cost of capital. Then COVID happened, and what the industry discovered during COVID was that it has pricing power. I think that was something it didn’t feel in its bones before. We had input costs going up, and I cover a bunch of companies. I didn’t have a single one experience margin compression because of input costs going up. They all raised their prices to offset it, and they raised their prices at their margin structure.
So if their cost went up by 100, they raised prices by 160 if they had a 60% gross margin, and they had no problem passing it along. I’ve said that inflation saved the industry during COVID. If you look at industry unit shipments in 2023 or 2024, they were flat compared with 2019, and blended ASPs for the industry were 50% higher, even if you take out memory. We’re seeing that on steroids right now. Clearly, memory pricing has gone through the roof. The industry, by the way, is going to do $1.5 trillion this year, maybe more. We’re going to be up probably 100%.
Seventy or 80% of the growth we’re seeing this year is memory ASPs. A big chunk of the rest is also non-memory ASPs. Units are up 10% or something like that, whatever it is. But the industry is now flexing that pricing power a little more, and by and large, I think customers are accepting it. They don’t really have any choice. So, yeah, it may stay tight. If it stays tight, I think we’ll probably continue to see this.
Yeah, I would say the semiconductor industry is very interesting in that some people flex their pricing power in the good times and suffer really badly in the bad times, while others just sort of eat margin the entire way through. If you look at TSMC over the last decade, they’ve risen from 45% gross margin to 60% now. It’s been a very steady climb up. Yes, they’ve taken more when times were good, but they haven’t given much back when times were bad. And you look at memory: they’ve gone from basically negative margins just a few years ago to now, you know—
People forget we’re coming off the worst memory cycle since the tech bubble just 24 months ago.
Negative gross margins.
When you look across the supply chain, some of these folks are taking tons of pricing because of the structure of the market. TSMC, where they’re a near monopoly and a very long-time partner, has not really jacked up pricing even though they could. Whereas memory companies, if Apple’s going to screw them in the downturn, they’re going to screw Apple over in the upside of the cycle. That’s pretty clearly what’s happening.
To build a gigawatt in 2024, it was roughly $40 billion across chips, land, power, shell, everything. Now it’s north of $60 billion. The price to build a gigawatt of compute has skyrocketed. Of course, not many people were building a gigawatt—or really, no one was building a gigawatt—in 2024, and a lot of people were building gigawatts in 2026.
You’re getting more out of the gigawatt now, too, by the way.
Yeah. And that gigawatt’s profitability is huge. I think everyone is recognizing that profitability extends. A lot of the mechanics I mentioned earlier mean you can’t build a cluster as a cloud provider unless you have offtake from a hyperscaler, which means you’ve locked in pricing ahead of time. Elon is exploiting that. If chips, land, power, and shell are constrained, and I build these things but don’t sign a contract until they’re actually up, I can make $50 billion per gigawatt instead of $15 billion.
Anthropic, in its recent disclosures, is making on the order of $70 billion to $80 billion per gigawatt off of its inference capacity. The AI labs are kind of forced to eat the cost. Yes, their margins are 75% to 80% when they’re spending $15 billion per gigawatt for compute, but if they’re spending $50 billion, their margins are much smaller. So it’s not clear who’s going to eat all the cost. Nvidia has had to keep its margin roughly the same, slightly decreased, but it’s had to eat the memory pricing increase and other costs in the supply chain, like substrates, optics, and all these other things.
The prices they charge their customers have soared despite their margins being flat. We’re seeing something really interesting: who gets to keep their margin, who gets to expand it, and who’s going to eat margin. I think it’s a very fun time to be on the pricing side, because you’re discovering in real time that you can sign a deal at many different prices. The best salespeople are going to get really insane pricing, and hopefully they’re rewarded for it.
Hilarious. Okay, Stacy, I kind of like what you said about COVID. It was almost like a dress rehearsal for AI, for the industry to start realizing, “Hey, we can charge for our value here.” And Dylan, you make an interesting point about a system builder like Nvidia having to decide, as the prices of all the components go up: Do we pass it on, or do we keep our margins the same?
Now, I’m feeling a little tension. There’s still going to be buildout all the way through. We’re not sure what’s going to happen in 2029, but clearly supply is limited and demand is high for the next couple of years. And yet we started by saying it’s getting harder and harder to finance. I’m wondering if there comes a point where those people at the top of the stack—the Metas and the model lab companies—push back and say, “This is just getting too expensive,” or, “Beggars can’t be choosers.”
Remember, you have to look at more than the margins and the cost. You also have to look at what they’re getting out of it. I’ll take Nvidia as an example. Their pricing goes up with every generation, but what you’re getting out of it is going up more than the price, which I think is important. You went from Ampere to Hopper.
Yeah.
The ASP roughly doubled, and performance went up 10x or whatever. Dylan would have better numbers for exactly what the performance increase was, but it was a lot. It was a very NPV-positive sort of thing. No problem. You can pass that along.
You go from Hopper to Blackwell, and it’s a little harder to tell. The Blackwell architecture is much different, but call it roughly a 50% increase in ASP per GPU. Performance went up anywhere from 3x to 30x, and maybe more given some of the later optimizations and software. Again, no problem.
You go from Blackwell to Rubin, and it’s the same thing. The cost of the racks is probably doubling, and so is the cost per gigawatt. I think Nvidia said that, for its own reference, it was something like $25 billion for Blackwell and $40 billion for Rubin. But again, performance is up 10x. That’s what needs to happen. You need to keep giving your customer value, because ultimately that’s what’s important. It’s how much performance you’re getting out of that gigawatt of compute relative to how much it costs. As long as that gap is getting bigger, I think it’s still not easy, but it’s palatable to do this.
Hypothetically, if they were to roll out Feynman and it were only 20% better, then I’d start to worry a lot about pricing and margins. This is always a huge controversy for a company like Nvidia, because it does have margins up in the 70s on a massive revenue base. People wonder: You’ve got competition, ASICs, other GPU vendors, and other things—how can they hold those margins in the wake of all that? In my view, as long as they can keep improving performance at a significantly higher rate than their ASPs are going, I don’t worry. They have to raise ASPs because their own costs are going up.
Things are getting more complicated, so you have to keep that performance-to-cost equation working for the customer. As long as they can do that, I don’t really worry about their margins. But if the pace of innovation slows, then I would worry an awful lot. They’re aware of this, too. I went to the Hot Chips conference in 2023, and I didn’t understand anything there. I’m a process engineer by background, so all the computer science and chip design stuff goes over my head. But Bill Dally from Nvidia, their chief scientist, gave the keynote. It was the one thing there I understood.
What he was talking about was exactly this: GPU performance improvements. He made the point that over the last 10 years, they’ve improved GPU performance by 1,000x. It was a whole bunch of different levers that they pulled, but they got there. He said that over the next 10 years, they want to improve performance by another million times. So far, they’re on that trajectory, and if they can do that, then I don’t worry as much. If they or anybody else falls off, then I worry a lot, because ultimately somebody has to pay for it. In a perfect world, that somebody is the customer, and there’s a lot of value from it. If the costs go up at an unsustainable rate, the suppliers have to start absorbing some of it, and then it’s a dogfight to see who does and who doesn’t.
Yep. Gotcha, Dylan.
That’s certainly the trend: all of these companies are trying to make chips. OpenAI showed, I think for the first time ever in the industry, that you can make a first-generation chip that’s really good. They let us into the lab. We tested it on our open-source InferenceX benchmark, running all these open-source Chinese models, and the performance was better than Blackwell. That’s a first-generation chip, and they used AI heavily in the design and programming process. Anthropic is starting its chip program, and it wants to do it in half the time OpenAI did.
They want to get a chip. These are aggressive goals, but if models are able to help them really accelerate this process, then you’ve got Google selling TPUs externally. Amazon’s Trainium is really quite good now and fully usable in PyTorch. Meta has a very aggressive roadmap, and so does Microsoft.
If you look at how I said it went from $40 billion to build a gigawatt to $60 billion, and you remove all the margin from the chip design houses—Nvidia, Broadcom, Marvell, and so on—you’re actually at about $20 billion a gigawatt. If you were able to do that, then you could pay for 5 gigawatts with 1 gigawatt’s worth of compute running inference. The economics here are obviously very aligned. As Stacy said, these companies just have to run like crazy. You can’t afford to be only 2x better in performance per dollar per year, because you’re going to get completely smashed by these other companies.
If Nvidia can keep up this incredible pace, sure, it deserves to continue to earn margin. But if the other competitors—and their main major customers—are able to design chips and deploy them, then they can scale their manufacturing. Memory without a markup, TSMC wafers, substrates, optics, and so on are really, really cheap.
Those companies have a lot of room to run on pricing, especially if things get more disaggregated and Nvidia loses its position or Broadcom loses its position. That’s why those two stocks trade at basically their lowest multiples ever. A lot of people are worried about that. Is Meta going to disentangle Broadcom? Is Google going to disentangle Broadcom? Are both of these companies going to stop buying Nvidia and just insource completely? I think the component vendors are going to be able to jack prices up like crazy.
The question is whether Broadcom and Nvidia can continue to earn these incredible margins. At least for the next two years, it looks like yes. But beyond that, is it off the table now? At the end of the day, chip design is very, very complex, but models have solved software, and programming accelerators is actually really easy now. It took us 6 months to get Nvidia inference working for InferenceX, our open-source, largest open-source inference benchmark. It took us 6 or 9 months to get AMD working. We got Google working in a few months, and we’ve got Amazon working in a few months.
We got OpenAI working in a month and a half. The pace from chip works to being able to run software on it is rapidly compressing, and now these companies are all turning their eyes to chip design. Can that actually work? It’s similar in the sense that it’s very verifiable, like software. Obviously, there are a lot of challenges with data and scaling, but the big question for the semiconductor industry is whether design becomes more commodity-like, as software has, in which case everyone can design chips.
Obviously, there’s the pesky fact that you have to pay $50 million for all the fixed costs of masks and EDA tools and all this other stuff. But the number of people designing chips is going to explode just because programming them is much easier. If designing them becomes much easier, margins have to fall. We’ve seen SaaS companies get obliterated in the market, and we’ve seen Nvidia and Broadcom really trade sideways despite their earnings growing at 2x-plus year-on-year. We’ll see what happens, but that’s the big question.
So, Stacy, I liked your point that it’s not about the fact that it’s going from $25 billion to $40 billion to $60 billion. It’s just: are you squeezing out more tokens per dollar? That’s all that really matters. And then, Dylan, you brought up XPUs and OpenAI. Very interesting example. How should we think about the growth of GPUs and the explosion of XPUs over the next 2 years? There are all sorts of AI accelerator startups taking different architectural bets. Is it going to look any different in a couple of years?
For XPU companies, there are sort of 2 paths. If you’re a hyperscaler, you can take a more conservative design approach and do similar things to Google or Nvidia, and it works out. But if you’re a startup, you have to do really crazy things. That’s why, in the first generation of AI hardware startups, the ones that did the craziest things, Cerebras and Groq, were the most successful. As we look at the current crop of startups popping up, only the ones that do really crazy things that are very different and can’t immediately be copied by Nvidia, Google, or AMD are the ones that have a shot at succeeding.
The nice thing is that so much capital is flowing, and a lot of really crazy ideas are being tested by companies trying to build some really insane formats for XPUs. In 2023 and 2024, Nvidia swallowed up a huge percentage of total HBM, total CoWoS, and so on in the industry. As we looked at 2025, it started to moderate, but then in 2026, Nvidia lost share. They’re using a lower percentage of the total CoWoS wafers and a lower percentage of total HBM. And again, next year they lose share.
Ultimately, the number is growing so fast that their earnings are still doubling year-on-year next year, or more. But this is a big thing that’s already happening: Google’s taking share, Amazon’s taking share, and AMD’s taking share.
On a unit basis, is what you’re—
Sorry?
On a unit basis. Yeah.
Yeah. I mean, units are always difficult. Who makes a big chip, a small chip? But CoWoS wafers and HBM are kind of hard to measure. And revenue, right? Dollars spent on foundry wafers, on HBM, on these sorts of things. These guys are taking share. Ultimately, Nvidia’s base is growing so fast that they’re still going to double earnings.
As we go up to 2028 and 2029, maybe growth rates aren’t as high. Maybe revenue isn’t increasing 100% year-on-year. Then what happens? Does growth moderate to the point where Nvidia actually loses share entirely, or are they making some crazy bets for their next-generation architectures past Rubin? Maybe that’s to Stacy’s point: they just have to run really fast, and that’s how they defend themselves.
This is one of the reasons Nvidia has moved to an annual cadence for chips. It wasn’t just for them, of course; it was for everybody else to try to keep up. They’ve got a scale that makes them hard to catch. People always talk about moats. Nvidia’s got a lot of them. There’s the CUDA software, although we can have a debate about whether CUDA is actually a moat anymore or not. There’s also the ecosystem and install base of infrastructure, and even their balance sheet and financing are a moat.
They’re leveraging AI to speed up their own design. The industry isn’t doing anything there that they’re not aware of or using. But it’s a race. Jensen, at least, is paranoid. He took the old Intel lessons to heart. He’s looking over his shoulder every day.
Yeah, this is absolutely why these stocks are trading where they are, right? They’re trading cheaper than they ever have. I wrote a note—this was probably 9 or 10 months ago—but we were looking at Nvidia relative to the SOX index, which is a broad index of semiconductor stocks. As of when I wrote the note, I think there were only 13 days in the last 10 years when Nvidia had traded cheaper relative to the broader semiconductor industry. That was 9 months ago. It’s cheaper today.
These are the things that investors worry about. It’s beyond just worrying about overall demand sustainability. If the demand’s not there, we’re all screwed, regardless of whether it’s Nvidia or anybody else. Within the group itself, this competition is clearly their biggest threat, because they’re the forerunner by a mile and everybody’s trying to eat their lunch.
But at the end of the day, that’s semiconductors. There’s always somebody trying to eat your lunch. It’s up to you to make sure you can stay ahead, and that’s something that will probably get harder and harder as you go forward, just naturally.
Sure. So when we’re talking about GPUs and XPUs and unit share and all these competitors gaining share, we’re really focused on the data center and hyperscalers, pretty much. What do you think about enterprise on-prem as an opportunity for someone like Nvidia?
Well, actually, even today Nvidia splits its revenue out by hyperscale and non-hyperscale. It’s not exactly clean, because some of the labs are in non-hyperscale, but some of the hyperscale spending is going to support the labs, or vice versa. So it’s not clean, but they’re about 50/50 right now. I think the labs, directly and indirectly, account for roughly 20% of Nvidia’s revenue right now. It’s sizable, but I’d say Nvidia’s revenue base is maybe even a little more diverse than we had thought, which is why they’re giving that presentation now.
At the end of the day, though, you clearly need enterprise adoption. That’s where this is. If we don’t have enterprise adoption, there’s no industry. It’s still early days, but I think we’ve got at least one killer application for enterprise, which is coding. Agent coding revenues have gone vertical. Does anybody write software by hand, like artisanal code, anymore? I don’t know that they do. I think everybody uses it.
Yeah. What’s coding? Right.
I don’t know. I also think that on the enterprise side, people are looking at revenues, but some of the actual impact may not show up as revenue. It may show up as productivity. It’s harder to measure, but I’ll give you a dumb personal anecdote. I’m a financial analyst. We build models. One of the guys who works for me has been using Claude to build models for his own personal account. We tried this among the team a few months ago, and it gave us garbage. But he said, “I think it’s working.” I said, “Okay, I want you to—we’re going to get on a Teams call, and I want you to throw it up. We’re going to have it build a model for a company we already cover, from scratch.”
It one-shotted a model in an hour and 20 minutes. It used 10 years of history, went through all the different resegmentations, captured it all, and even forecasted it. It listened to what the CEO said on earnings calls and tried to use logic rather than randomness. It was something that would have taken one of my guys a week to build and would have taken me 3 months to build. It did it in an hour and 20 minutes, so I could do more with less.
There are other implications of that. Are we going to be in a world where AI adoption in the enterprise is rampant and we have 20% unemployment? I don’t know. There are other factors to think about here, but I think we’re still early. Most of the anecdotal evidence we’re hearing right now—because this is the overarching debate—is about whether there’s ultimately a return from using this stuff. I think anecdotally we’re coming down pretty solidly on the side of yes, there is a return, but it’s absolutely still early. A lot of companies are still in the experimentation phase. They don’t have the expertise to know what to do or what they need to do, so it takes time. But I’ve been encouraged by what I’ve seen.
The only thing I would say about enterprise adoption is that I don’t think enterprises are going to adopt that much on the hardware side and run their own models and such. I think the efficiencies of running batches and the scale economics of models mean that most people are just going to buy tokens and implement them, combining them with their own data. Ultimately, that’s the real way most people are going to consume AI.
So it still leaves this sort of stratification. I think the industry has gone more and more in that direction. If you go back 2 decades, everyone bought their own infrastructure and deployed it themselves, and now most of the spending is in the cloud.
Most of the spend is going to be on buying tokens. Even pharma companies and hedge funds—most of their compute and AI spending is going directly to models, not to buying compute and running open-source models. There will be providers who take that on, but I’d love to talk more about this, although we’re a bit over time. I think it’s very fun.
I see a big zero minutes left. Okay, I guess that means we’re done. Okay, guys, this was awesome. I learned a lot. This was great. Thank you so much.
You bet.