Mike Volpi谈AI基础设施为何能比炒作周期更持久

We live in a world where demand for AI capacity and tokens is virtually limitless, while supply is extremely limited. In other words, we all live in Jensen’s world.

At that time, you were involved in M&A of the most valuable companies on Earth. Looking back at Cisco’s successful M&A process, one finding is that they weren’t afraid of failure and executed deals frequently enough to allow for some exceptions. You purchased this for $500 million, and 2 years later, you’d made a profit of $1 billion. Looking at the development of AI, how do you reflect on your previous experiences?

Seeing the surge in Anthropic’s revenue, OpenAI is undoubtedly experiencing a much better growth rate than we saw back then. Do you think the current development is useful, or at least will be fully utilized?

100%.

1. The internet buildout vs. the AI buildout

Will it be owned by the same people who own it today?

2. Why this is the best time to be investing

Mike, thanks for joining us on the podcast. It’s a truly wonderful opportunity to have Mike, who was one of the key figures in building the internet during his time at Cisco, join us. Now, of course, they’re also working on building AI. Guido, myself, and all of you listeners would like to hear Mike’s thoughts on the past and present from all angles, including the actual construction, its capitalization, and the technological differences.

For those who don’t know him or don’t remember him, let me first tell you about Mike. Mike was a senior executive at Cisco and oversaw many of Cisco’s mergers and acquisitions and strategic moves in the late 1990s, around the time the internet was being established. In other words, Mike was at the center of it all.

Okay, let’s start there. When the internet first started to become widespread, the entire industry was scrambling to build a lot of bandwidth. For a while, it proved useless, and it turned out that there wasn’t actually enough demand, right? There are some similarities between what’s happening now and that situation. Seeing the progress being made in building AI, how do you reflect on your experiences so far?

First of all, it’s a real honor to be here with 2 legends like Raghu and Guido. Guido and I had the opportunity to work together at least once, so it’s wonderful to be here. As you said, it’s an interesting time. They say history doesn’t repeat itself, but it certainly rhymes sometimes.

Yes.

There are indeed some similarities. It might be helpful to provide some background on what we observed about construction in the late 1990s and whether any of it is still relevant today.

One similar theme was the technology known as the internet and the web, and how they built everything that’s written on the internet. We expected this to dramatically change the world, and envisioned a wide range of applications that would ultimately benefit greatly from that build. Applications used a variety of media types, right? Initially, there were only images and text, then audio, video, and high-quality video applications like commerce and social networking. These were all things imagined in the 1990s, but they didn’t yet exist.

At the same time, demand increased dramatically. At the very least, demand for connectivity surged, and demand for bandwidth also increased significantly. Simultaneously, a massive influx of capital occurred in 2 forms: equity and debt.

Looking back at what happened, first of all, in the world of bandwidth, especially fiber-optic bandwidth, the factors and amounts driving bandwidth increases were quite different from those in computing. That’s because they invented a technology called WDM. It essentially uses optical fibers to increase capacity from what was previously 1 or 10 gigabits to 40 or 80 times that amount by incorporating more colors of light into the same optical fiber. In other words, technological innovation dramatically increased capacity.

That’s actually different from computing, which will ultimately be constrained by something similar to Moore’s Law. So the pace of improvement is more gradual. It may be hard to believe, but computing is actually much less demanding now than it was back then.

Well, another factor I think is different is that the practical applicability of AI is higher today than the practical applicability of network bandwidth.

Yeah. That’s reflected in the increase in provider revenue. Looking at Anthropic’s and OpenAI’s revenue growth, I don’t know what the numbers look like internally at Google, but even open-source model providers like Fireworks, Together, and Modal are definitely experiencing growth. While the margins may not be the same, it’s certainly a much better growth rate than what we saw back then.

Yeah. So, what happens next?

At some point, there was a shock: some of these businesses built on the internet weren’t good enough. This rippled through the entire industry, ultimately leading to the bankruptcy of the player with the most debt.

What’s interesting is that those liabilities were used to purchase assets, namely fiber-optic cables. It turned out that those assets were indeed very useful in the future.

That’s right.

However, the bankrupt company didn’t own them; someone else was managing them.

Yeah.

The first thing to recognize is that the foundational asset built at that time was bandwidth. This compute cycle will certainly be useful. It is undoubtedly useful.

Whether or not those assets are owned by the same company is an interesting economic issue. The answer to whether the current buildout is useful, or at least will be utilized to its fullest potential, is a long one.

I completely agree. Will it be owned by the same people who own it today?

There are many intriguing dependencies, and many things that are actually beyond our control, such as macroeconomics and geopolitics. That’s outside the builders’ control.

In particular, when it comes to the builders, some of them have quite healthy balance sheets, don’t they? The last time I spoke with people at NVIDIA, there were 200 neoclouds, and their balance sheets were not good. So I expect there will be several failures. While some may be better than others, broadly speaking, the market expects an economic adjustment. However, the underlying assets will be very useful over the long term.

One of the biggest differences from back then is that everything is so much bigger now. At that time, only a tiny fraction of the budget was invested in internet infrastructure. Today, we’re investing about 2% of our GDP in AI infrastructure, right?

Yes, it’s gotten much bigger. But this is an interesting concept, isn’t it? What is being built at a 100,000-foot level? At that time, what was being built was communication. Today, we are building intelligence. Furthermore, the economic value of intelligence is probably higher than the economic value of communication.

Probably, yes. That’s right.

Yes, so I don’t think that’s surprising. You might not believe it, but even though it’s been almost 30 years, the market is more efficient now, isn’t it? That’s why, even back then, a much larger amount of risk capital began to flow into AI. In short, I’m actually more optimistic about this cycle, but I also believe that adjustments and streamlining of the industry structure will be necessary.

The role of IT has also increased significantly, hasn’t it? In the late 1990s, PCs were somewhat widespread, but they weren’t ubiquitous. Today, software has essentially taken the world by storm, and it has become indispensable for every business. Therefore, the cost of distributing intelligence—

Oh, that’s certainly true. And not just every company: as many as 7 billion people have a supercomputer in their pocket.

Yes, that’s exactly right. In other words, the consumption of the fundamental profits generated is more evenly distributed.

3. Financing the buildout, and why GPUs are outliving their depreciation

Yeah. I can’t completely recall all the puts and takes from the 1990s and early 2000s. From a financial perspective, one of the most recent systemic shocks has clearly been rising interest rates. The sheer scale of the capital is what causes people to say, “This will put pressure on everything else, and we might not even have enough capital.” That wasn’t the case when we were building bandwidth and the internet, was it? Interest rates didn’t rise sharply, did they?

No, interest rates didn’t rise significantly. There were several major differences, weren’t there? One reason is that, proportionally speaking, I think more capital came from equipment vendors’ balance sheets back then than it does today. Today, the structure is driven by capital markets: private debt, public equity, and so on. NVIDIA also has vendor financing, though I don’t think it’s appropriate to call it vendor financing. It’s very similar.

Yes, we do have some of that, and we do have some corporate finance, but the majority is equity. No, it’s not linked to the purchase of any specific equipment. Therefore, it could be said that the capital structure is now a little cleaner.

Interestingly, I think one of the advantages of the 1990s was that a lot of the money was invested underground in the form of fiber optics. When fiber-optic cables are laid underground, they can be used for a very long time. It’s like an asset that can be used for 20 to 30 years.

I believe one of the decisive factors is the actual lifespan of the computing infrastructure. This is an interesting phenomenon, isn’t it? Because the dominant part of that computing is the chip.

That’s a GPU. That’s the depreciating asset.

Yes, that accounts for a large part of it.

So, do you remember when people were excited about Hopper, the H100? They said it probably had a 5-year lifespan, and they were renting it out for $1.82 per GPU hour.

That was 4 years ago. That chip should practically be nonfunctional by now, but it isn’t malfunctioning at all.

Yeah. No. Furthermore, they’re being rented at a higher price. It’s about $2.50 or $3.

What’s interesting is that, thanks to AI itself, we thought an asset with a 5-year lifespan would remain useful for quite a long time. In that context, if your business is about financing assets, and the useful life of those assets becomes much longer, you benefit because your business model is built on certain assumptions.

I believe an unintended consequence of AI’s software superiority is that the lifespan of the underlying assets is extended, thereby making fundraising easier. That’s an interesting phenomenon I hadn’t anticipated at all when this started.

Investors aren’t there yet. While investors haven’t widely accepted the idea of extending the lifespan of GPUs, neoclouds clearly believe in it.

I’m not sure. I don’t really know either. In short, what’s happening right now is a significant shortage of assets. So even previous generations still have high value, right? That might change completely once we reach a point where GPUs are abundant. You might return to those numbers.

The key metric is how much token revenue can be extracted per watt, right? So that means Hopper is much worse than Blackwell, and Blackwell is much worse than Vera Rubin, right? You’re getting dollars from watts much more efficiently. Theoretically, if Vera Rubin is available in sufficient quantities, there’s no reason to run H100, because it’s simply a worse way to convert electricity into dollars.

That’s right.

Right? However, because demand is currently very high, the asset will continue to be used even if the conversion rate from watts to dollars is low. Scarcity is currently very high, and while we may eventually reach that point, ultimately the scarcity lies in the manufacturing of the semiconductors themselves. It doesn’t look like that’s going to change anytime soon.

That’s right. As scarcity continues, the lifespan of the asset will continue to lengthen.

Yeah, I agree.

Yeah. Yes, I think that’s correct.

Another significant difference is that there was a period when unused bandwidth was available. These days, there’s no such thing as an unused GPU, right? In other words, there’s a small amount of unused energy.

I’m not sure if we’ll talk about the network, but if the network is good, you should be able to extract more FLOPS from a particular GPU.

Yes, I was trying to say that there are some spot factors depending on the specific use case and how people built it, but generally speaking, that’s not the case.

4. Cisco’s M&A playbook, and advice for Nvidia

When you were at Cisco, was Cisco the most valuable company on the planet?

Yes, I was definitely there.

In other words, you were involved in the M&A of the most valuable company on the planet at the time, and I’d say it went quite well.

Looking back, I’m satisfied.

If I were to give advice to Nvidia in 2026, whoever is running M&A won’t be in the same position as you, but Vishal is in the same position as me.

Looking back at the M&A process, what worked well at Cisco was that we were basically content with making enough acquisitions to get statistically significant results. Some worked very well, some were okay, and some didn’t work at all.

When I drew one for each of the 3 sectors, one-third worked very well, one-third were okay, and one-third were failures. The benefit of what we did was that we weren’t afraid of failure. We failed enough times, and we got a lot of outliers, like Maria Mazzola’s company, and a lot of things that gave us venture-like returns. They acquired this company for $500 million and made a profit of $1 billion 2 years later.

That’s amazing.

When I joined the company in 1994, I was the only employee hired as a business development manager. During my time there, around 2000, 50% of Cisco’s revenue came from acquired businesses. This was extremely helpful, and we rarely spent more than 4–5% of our market capitalization on any particular acquisition.

Yeah. Most of the acquisitions were actually quite cheap compared to market capitalization. What does that mean for Nvidia?

I don’t even want to think about it.

Yeah.

My logic at the time was that if the stock price rose above the acquisition price in a single day, you could make as many acquisitions as you wanted, and nobody would care.

Were all of those transactions conducted using stocks?

Yeah. Almost everything was in stocks. Some of the smaller acquisitions were paid for in cash, but essentially, they became part of our main product lines. It wasn’t just a minor addition that added some functionality; in some cases, it was a completely new product line that became a leading product line in the industry.

Those of you involved in networking will understand this very well. What makes network-related acquisitions slightly different from other acquisitions is that networks operate on open protocols. If I acquire another company that makes switches, they’ll be able to communicate freely with my switches, routers, and other devices right away, because all the protocols are open.

Cisco used several protocols, including OSPF, BGP, IS-IS, Ethernet switching, and 802-something, and they all worked without any problems. The product could be integrated into the sales team immediately, and the sales team literally only needed to swap badges. We were able to expand our business at a very rapid pace.

How is that different from Jensen’s world today?

First of all, they’ve made some really smart acquisitions. For example, Mellanox is the world’s largest network company, right? That’s genius.

However, they sometimes make acquisitions that leave you scratching your head. For example, I don’t really understand what Hugging Face is. There might be a reason for it, but I find it difficult to understand. I don’t think that’s important, though. I think it’s fine for them to make these acquisitions and take risks, and some of them will probably be successful. If one becomes as successful as Mellanox, that would be amazing. Some things won’t work out.

The market is giving them an incredible number of options, right? Basically, they can do whatever they want. This is the perfect time for them to take these risks. I think they’re very clever, and there’s room for criticism of any acquisition. Believe me, Cisco was heavily criticized for acquiring StrataCom and Cascade, and for the fact that the optical network company it acquired from Vero never went anywhere.

But in the end, during my time there, our team made 70 acquisitions, which accounted for half of the company’s revenue, and our revenue dropped from $30 billion to $15 billion.

The strategy worked. I think the Nvidia team is doing the same thing.

That’s interesting, and I think it’s a clever idea. Let’s take a look at the extent to which they are funding the industry.

Yes, yes, yes. But that’s not right, is it? In that sense, I think Nvidia is fundamentally trying to reshape the industry.

Yeah. I think the industry is strong and will find its own unique form. Reconstructing it is difficult.

Please allow me to defend Nvidia just a little. I was at Intel and saw how major hyperscalers separated Intel from end users. Ultimately, because Intel lost track of what its customers were doing with their chips, hyperscalers were able to create chips that were superior to Intel’s. I can understand the desire to have 3 or even 4 or more customers.

I agree.

Yeah. We can discuss what an efficient allocation strategy would be to return profits to investors, or whether we should invest everything in future innovation during this perhaps once-in-a-lifetime period in history.

I think both of you have made very accurate points. What I want to say to Guido is that the industry tends to take on a certain form because it generates the greatest utility and the greatest economic value. Why are there only 3 hyperscalers? Because scale matters in this business. Software, right? Because scale is crucial in this business, the industry wants to concentrate its efforts. No matter how much funding is provided, small businesses will always be at a disadvantage.

If it’s clearly Nvidia, that’s not desirable. Having only 3 clients is a problem. We want 50 clients, so we’ll provide the funding, but eventually they’ll go bankrupt. That’s because industries tend to consolidate as they grow into large-scale businesses. Even if you funded 100 airlines in the United States, ultimately only 3 would survive.

What I’m trying to say isn’t that it’s a bad idea, but that it’s pointless. It’s a noble intention, and as you pointed out, there’s a possibility of getting a very good return, but I don’t know if they actually care about the rate of return on capital. They might be concerned, but if you withdraw at the right time, it’s not a matter of return on capital.

Certainly, to some extent, it can be said that it shapes the industry, that is, determines the form of the industry. However, because our startup and many other startups now have access to that capability, a lot of innovation is also possible.

Yes, yes, yes. No, not at all. Overall, I understand why they do it that way, and many of our companies benefit from it. Thank you, Jensen.

5. Making acquisitions actually integrate

That’s right. At VMware, we’ve made over 50 acquisitions throughout the company’s history, though probably not as many as you have.

At VMware, whether we acquired a 20% stake or a company with 2,000 employees, the post-acquisition work involved in integration, culture, business models, sales teams, and so on was always enormous. How did Cisco achieve that at the time?

Structurally, we had a very small team in charge of the acquisition. A small group means fewer than 10 people. We also had a fairly large integration project management team, with rules and guidelines for how the integration should be carried out. That team collaborated with colleagues from various business units, departments, and functional units within Cisco.

The key point is that companies excel at processes but struggle with one-off events. Carrying out numerous acquisitions required a repeatable process, and there was a central group that managed those processes. In Cisco’s early days, this was something people did frequently, but no one was officially assigned to it. It was like saying, “Raghu, this is your job. Enjoy the integration process, and let me know when you’re finished.” That makes it difficult to know where to begin. We thought, “This is not good,” so I created a process, and it worked.

The difficult part was that I used to enjoy buying small things, but the business grew so large that those small things became less important to the overall business. There was no longer any difference. A little after I left my M&A job in 2000 and switched to managing business units, they tried it. After me, they started acquiring bigger things and ran into the kind of problems you’re talking about. Integrating large companies is very difficult. They’ve existed for a long time and have a long-standing culture.

For example, we acquired Scientific Atlanta, which was a huge mistake. We acquired a lot of things that didn’t work well. We also fell into a mode of partial integration rather than complete integration.

6. When founders should sell, and how deals really happen

Yeah. That doesn’t work very well. M&A is about having a process, but when you stop respecting that process, big problems arise, as you mentioned. What about from the perspective of the portfolio companies? Some of our portfolio companies have been acquired and exited by other large corporations. I have 2 questions. If you were the CEO of a portfolio company, what would you pay attention to? And if you were sitting on the board of these companies, how would you advise your portfolio companies these days, specifically on general exit strategies?

If you’re on the board and you’re faced with a situation where you need to sell the business, there are 3 possible outcomes. First, when a company is doing well, trying to sell it is a really bad idea, so you try to persuade the CEO to stop. The second reason is that the company is in financial difficulty and doesn’t have much money. Selling is a good way to leave. Everyone is satisfied, and all that’s left is to help the deal go through. Then there are cases that fall somewhere in between.

I think there are several important indicators that can help you decide what to do. The first is whether the CEO, founders, and founding team are passionate about the company. Do they actually want to do this? Do they believe they can overcome this intermediate period and jump into the next stage? Do they want that? If the CEO or leader says, “Maybe we should consider this. I’m a little tired. We’ve been working hard for a long time,” then let’s sell this and move on.

The second element is that you have to be fairly calm and objective about whether this will actually be a sufficiently interesting business. The question isn’t whether it will double from here. If it does double, I’ll take the money today. But will it increase tenfold from here? I think that’s a really difficult decision, because it’s emotional. You’re not the founder, but you’ve worked at this company for 4, 5, or 6 years. You’re pouring your heart and soul into it and have encouraged close friends to work there. Nevertheless, stepping back and looking at it objectively, you may have to say, “This will not increase tenfold from here.” Then I help steer the ship. But I think the most important factor is where the founder’s mind is.

Yeah. For many founders of portfolio companies, there’s a mystery around how these things actually happen. People might think that one day someone will call another company and say, “We want to acquire your company.” But there’s also the fact that people have built relationships over a long period of time. How did Cisco approach this? And what advice would you give if a portfolio company is expected to grow tenfold?

Yeah. Is it just a matter of making a phone call and saying, “Hi, I’d like to acquire your company”? Is that all?

No.

Of course, we’ll do all we can to build the network and understand what’s happening. Previously, the basic idea was to catalog inventory. Since every company in the market is a potential acquisition target, we catalog them and find out where they are, what their growth rate is, who their founders are, who their backers are, what their aspirations are, who their customers are, and so on. It means obtaining information. The corporate development group had to do it, but I thought it was everyone’s job to always be aware of what was happening around them.

Basically, Cisco was trying to weave that inventory into a strategic statement. For example, we’re a router company, but we sell more switches than routers, so we need a switch. Which switching companies are available? Alternatively, fiber-optic networks are becoming increasingly important because of bandwidth limitations. Which companies are available?

Companies often come to us, say hello, and express their desire to be acquired. I think that’s generally a bad idea. If there was an auction going on, or if I was interested, I should have thought about it sooner. We pay very little attention to inbound approaches. We don’t pay attention to inbound interest. It’s the opposite.

You’re currently running a small company or startup. What are your thoughts on that?

I always encourage company founders to talk to and engage in dialogue with large corporations and competitors. Some people say they’re “selling it themselves,” but that’s not the case. It’s a reverse information-gathering mechanism. I even encourage our CEOs to talk to our direct competitors. Let’s go get some coffee. If you don’t know anything, you might leak information to them. I hope you’re not completely clueless, but they might teach you something interesting and shape your perspective on the industry.

I always encourage people to go out, talk, and see what’s happening in the market. Usually, they have a pretty good understanding of when their company is struggling a bit and when they want to sell it. We’ve been talking about who’s where, so we already know about them. I don’t think startup CEOs should spend all day talking, but having coffee with powerful people at NVIDIA, Amazon, and OpenAI once or twice a week is a good thing. When the right people are speaking, that’s what you should do. There’s not much to lose by exchanging information, and there’s much to be gained from having such a relationship.

I think this is really good advice. I always tell the CEOs of our portfolio the same thing. If you don’t know the cell phone number of your competitor’s CEO, that’s a problem. The day will come when you’ll need it.

7. Mapping today’s AI infrastructure

That’s absolutely right. It’s a network that needs to be built.

Yes. Yes. Now, let’s move on from the M&A talk and get back to the actual technology. You’ve been involved in infrastructure for over 30 years. Few people know as much about infrastructure as you do. Please summarize the current situation. What are the opportunities, the technological gaps you see, and what are the founders really focusing on?

That’s a very interesting topic. I believe we need to consider the current state of this industry from 2 perspectives. One is a vertical stack that starts with EUV equipment and continues through AI, applied AI, applications, and everything in between. We need to consider the infrastructure aspects in particular. What kind of value chain is that?

First and foremost, we’re currently living in a world where the demand for AI capacity, tokens, and similar technologies is virtually limitless, while the supply is extremely limited. We’re all living in Jensen’s world. He is the champion. There’s no room for doubt.

I think the industry will start to change depending on whether the characteristics of demand or supply change. In my view, the supply side is currently the biggest constraint. While there are certainly power constraints, the dominant constraint is silicon. I don’t think that will change unless the production capacity of the fabs increases. It’s unlikely that things will really change in the next 2 or 3 years, or even 5 years.

It’s conceivable that the production capacity of the fab will increase. This could involve processing more wafers with the same machine or increasing production of ASML’s machines.

However, that is probably the dominant factor in our industry. We live in a fairly rare world, and companies with fab production capacity are the kingmakers of the infrastructure world.

On the demand side, I think the world will become smarter about the types of tokens it consumes. The last few years have been a time when everyone wanted ultra-high-end frontier tokens. And today’s topic will be model routers and how to migrate data.

The economic difference between high-end human-centered tokens and high-end Kimi tokens is 5:1, 7:1, and so on. That’s a very large economic difference. I think people will start to become wiser about consumption, and at the very least, prices will start to fluctuate according to demand. This is far more likely to happen in the near future.

These are the dynamics that govern our industry. Everyone caught between the two poles of supply and demand—whether it’s a neocloud, inference cloud, or model builder—has to consider the infrastructure itself. I think we’re still living in a world where we need to think of the entire infrastructure as a single system. It’s quite different from the old network era.

In the old days of networking, traffic volume fluctuated greatly depending on the type of traffic and the time of day, making it statistically much easier to duplicate workloads. Nowadays, we basically have a large number of GPUs that are always running and always connected, resulting in a more constant flow. Because the interrelationships between various components still largely rely on proprietary standards, such as NVLink, it’s extremely difficult to swap one component for another.

In fact, I’d argue that today, perhaps the best architecture is the Google TPU architecture, because it’s being considered as a single system, literally. I think the NVIDIA-style architecture is moving toward a TPU architecture, but there are still some interchangeable parts.

Yeah. Many other companies, such as those using Arista switches with Cerebras chips, are inefficient because, at least in today’s world, those systems aren’t built as complete systems.

It’s like a patchwork.

Yes, I think the industry will evolve, but today, companies that can think of the whole as a single system and optimize its architecture have a competitive advantage.

Yeah. In other words, it seems that the smallest unit is shifting from cards to racks, and in some cases, to several racks.

A rack?

Yes, I think that’s true.

This is the part that was recently announced, right?

Yes, that might be the smallest unit.

But when you look at what Google is doing with toroidal pass-through optical MEMS switches and such, it’s literally like the entire data center is becoming the smallest unit. In other words, a column is the smallest unit.

Yes, that’s exactly right.

Yeah, yeah. But it seems NVIDIA also wants to go there.

8. Will we see more competitive AI silicon?

Yes, obviously. Both dominant providers are moving toward a systems approach. Even small startups are moving in that direction.

How do you see AI chips evolving? Jensen is the champion now, and he’s incredibly dominant, isn’t he? Google TPU has been very successful. Currently, most of them are sold by Google, but that’s expected to change. We don’t have enough TPUs.

Yes, we don’t have enough TPUs, exactly.

What will the future hold? Will we see more competitive silicon available?

I definitely think so. We all want that.

Yes, we all want that. Well, we all want to invest in it.

First, what I find interesting is the considerable diversity of model architectures. Some people use FP4 or FP8, while others use a combination of many experts. Some people have few experts. Some are increasing the number of parameters, while others are not.

I believe there’s a correlation between chip design and the model optimized for it. So, regarding your point about whether the diversity of chips will increase, in a sense, I think so, because it will increase the diversity of model designs. Depending on what model you want to use and what you want it to do, you can choose a different chip architecture that suits your needs.

Up until now, the strength of inference has determined the nature of the chip architecture. This is because stronger inference means an increase in the number of parameters and an increase in the KV cache. That means there’s a lot of communication needed between the memory and the chip.

Yeah. If the only way for chips to communicate is through their edges, there will be fewer connections than if you could use the top and bottom of the chips. Therefore, I’m a big fan of the 3D DRAM architecture. Among the new developments in hybrid copper bonding, there are some that are even more interesting because they allow for even higher density.

As long as we continue to see use cases that heavily utilize KV caches, I believe this architecture has great potential. It appears that the most value-added work, labor, or AI is reasoning-driven work. Architectures with very high memory bandwidth should have an advantage.

Yeah. We have invested in Cerebras. Cerebras has succeeded in growing rapidly. However, if a model isn’t compatible with SRAM, many models won’t be compatible.

Yeah. Yes, we need a lot. If you take it off the Cerebras board, it won’t be super fast. Furthermore, because the chip, or wafer, is so large, being outside the chip actually puts you at a significant disadvantage in terms of memory bandwidth.

Yeah. So, they’re currently running Sol [?] on Cerebras chips. This is a large model. They found a way to contain it. My guess is that OpenAI must have gone through a tremendous amount of work to fit that model onto the chip.

Yeah.

No, we’re also fans of 3D architecture, but manufacturability is far more complex due to thermal effects and other factors.

Yeah. Yeah. Yeah.

Yeah. No, I’m a big fan. I’m a mechanical engineer who studied thermodynamics, so heat is on my side.

Only a very small number of people say that. Now, let’s see if TSMC and other companies’ packaging departments are calling you.

Yeah. Yeah. I agree. The packaging of these chips is particularly complex due to their thermal properties. In a world where both logic chips and memory chips are in the same package, even though their coefficients of thermal expansion are the same, logic chips become far more difficult to work with. The ability to put all of them into the same package is, needless to say, not easy.

The chip is getting hot again, and even hotter. Of course, another possibility is to keep them separate and create an optical interconnect between them. Do you believe that?

Yeah. Yes, I think so. However, optics is a very finicky field and requires many experiments. If you heat up optics, they don’t like it. Photons don’t like to get hot.

Well, theoretically, any substrate with waveguides would suffice. But once the logic chip starts to heat up and goes outside the comfortable temperature range, various connection problems and other issues arise. I find light fascinating. My opinion is that if it can be done with copper, then it should be done with copper.

However, even with scale-up racks, it’s not just about interconnection. The situation is clearly different with scale-up racks. It simply simplifies things like cable routing.

We’re talking about modules within the server.

Yes, yes, yes. Yeah, yeah. No, if you move to the top of the rack, you definitely won’t have enough bandwidth.

9. What triggered Cisco’s fall

Well, at the time, Cisco was the most valuable company on the planet, an early hardware leader in networking, and supported the entire internet. At some point, it ceased to be the most valuable company on Earth. What triggered it? What can we learn from this?

A common occurrence during a demand bubble, or when demand is perceived as high, is that customers over-order because they know they’ll be cut back.

That’s right.

As a result, production capacity has been exceeded for all parts of our product line. We didn’t have enough stock, and therefore couldn’t implement a full switchover. Order after order, and then order after that, were piled on top of each other.

Of course, we were placing orders across the entire supply chain, but we were taking on financial risk with these orders because demand felt very strong. It was like saying, “The money is here, so please hold onto it and send the goods as soon as possible.”

So, what went wrong?

Yeah. When demand dries up, it doesn’t mean people stop buying and cancel all their orders. It seemed like they hadn’t ordered twice the amount they originally intended to order. What was needed was half of half of that. As a result, orders decreased to about a quarter of what they used to be.

Then, A, the supply chain took on risks, and B, demand plummeted. That’s why I think it happened in the December quarter. This happened in December, so it left a bit of an impression on me. In other words, it was off by a month.

November, December, January. You know what usually happens in business when things are delayed by a month, right? December is slow, but things start to recover in the first week of January and make up for lost time by the end of January. I thought December was a slow month and nothing special. December is always a slow month, and nothing happens in the first week of January.

Nothing happened during the second week of January either. Nothing would happen in the third week of January either. Oh no, I messed up. And so, the quarter began.

What do you think?

It might have been off by about 35% to 40%.

Amazing.

Amazing.

Yeah. Incidentally, for a company that has been publicly listed for 10 years and has never missed a quarterly financial report, this is quite an achievement.

Yeah. And that’s how everything was adjusted. That was difficult. I think the stock price fell from $80 to $9 per share. In other words, its value literally dropped to one-tenth of its original value in 6 months. The situation was extremely difficult.

10. Rethinking data center networking for AI

So whenever the workload at the top layer changes enough, the underlying infrastructure software and hardware also need to change accordingly, right? We’ve seen this in AI. We talked about computing, but what happens on the network? Significant changes have happened there. There’s no longer a single network. Suddenly, a second backend network appeared. Where did that come from? Did you say Ethernet is dead?

That’s right. Ethernet is dead. Ethernet is the framing mechanism for the internet.

It’s back. It was dead, wasn’t it?

I believe the core principle of the communications world began with circuit switching. When the telephone was invented, the idea was that there were 2 endpoints, A and B, with a certain bandwidth between them. Communication would begin once the line was connected. But it turned out to be quite unreliable, because communication would be interrupted if even one connection was lost. And in a world with uneven traffic patterns, it clearly doesn’t scale.

So we decided to use packets. Packets are very useful because they enable statistical multiplexing. That’s a major advantage. David and Andy strongly supported it at the time, didn’t they?

Yeah.

Variable-length packets, and IP packets in particular, were designed around the assumption that devices wouldn’t need bandwidth all the time. Their arrival and departure times are uneven, and workloads are distributed across the network globally. Not everyone communicates at the same time, and the output of received signals fluctuates considerably. That’s the internet, and that’s the workload we face every day. For a long time, that was almost a dogma, wasn’t it?

Now, let’s move on to today’s topic. What exactly is an AI data center?

It’s not like that. There are no random arrivals. Links are always fully utilized. I hope GPUs will be fully utilized, too. Traffic patterns are now far more predictable than before, especially during training. They’re highly predictable. Inference is less predictable, but we can observe its traffic patterns. With proper batching, it becomes highly predictable. Large batches in particular can make it even more predictable. The fundamental principles of workloads have changed.

But what’s the name of Broadcom’s mainstream network architecture today?

It’s Tomahawk, not Trident.

Tomahawk is a statistically multiplexed packet-switching architecture that Broadcom acquired from Dune.

Yes. I believe highly predictable, fully utilized architectures will become mainstream for data center networks, rather than unpredictable, probabilistic, statistically multiplexed ones. Basically, that means returning to circuit switching.

Yeah. Ultimately, it’ll become an optical circuit switch.

Yeah. Or time-division multiplexing would work, too, right? Or some combination of those. There are many ways to multiplex, but they aren’t probabilistic. The timing is constantly adjusted in some way. You can do TDM and other things with these architectures. The problem is that the switches you can buy today have adequate bandwidth, but their architecture isn’t adaptable enough.

I think future data center switches will eventually work that way. Using MEMS might make them even more similar. Basically, it means bringing ATM back to data centers.

But even ATM uses statistically multiplexed, stochastic cells, right?

It was a completely unreliable connection.

Yeah.

So this is bad news for network vendors, including my former employer. If the situation doesn’t change, they can’t charge a high price. The key point about switches is that they’re complex devices handling statistically multiplexed traffic flows. When a packet comes in, it has to be written to memory, a lookup has to be performed, then another lookup, and then it has to be sent out through a port. That’s difficult, so it’s expensive. In a relatively static world, though, you don’t need that much intelligence. You just send it as is.

Given this workload, I think there’s an opportunity to rethink network architecture and make it much cheaper. This is a classic example of the innovator’s dilemma, because it’s not a good trend for any existing vendor. They probably don’t want to do that. But a newcomer could say, “I don’t care about statistical multiplexing anymore. The world is different. I’ll do things my own way,” and create something interesting. It could be optical, electrical, or a combination of the two.

I think we’ll have to wait and see what comes next. We can get some clues from how Google is building its network.

Yeah.

I think Amin was a pioneer, especially in optical circuit switching. They were the first. And it was something like a toroidal shape.

Yes, yes, yes. By the way, I don’t understand why it’s so slow. The switchover takes about a minute.

I don’t fully understand it either. I think that’s partly because the use cases today involve fairly static configurations.

I don’t perceive it as static. Don’t you mind?

Oh dear. I understand, but give me a few milliseconds, not a second.

Yes, but to be honest, I think it’s probably possible. The technology exists. There just aren’t any use cases yet.

Yeah.

Yeah.

So whether it’s MEMS or Max Zander’s MZI, I’ve looked at a lot of these to see if there’s anything there. But someone needs to come up with the architecture.

The question then becomes: who will accept it? You can’t build the switch without putting together the rest of the system. It’s another thing that relates to user inertia.

That’s exactly right. Networks have returned, to some extent, to being infrastructure. Today they’re an essential part of the architecture. So this innovation needs to originate, to some extent, with GPU vendors.

Yeah. It’s interesting, isn’t it? NVIDIA is the only option. AMD may reach that point someday, but for now, only NVIDIA and Google can ship rack systems.

Yes, that’s right. So they have to adopt it today. As the number of chip vendors increases, it’ll be interesting to see whether there’s an alternative architecture they can all agree on. It’s something we used to do, a long time ago. It’s like having 2 dominant vendors.

ITF 2.0?

Well, there are 2 vendors. They’re dominant, and each has its own way of doing things. It’s unlikely they’ll change direction. Could there be a group where everyone else says, “No, this is a standardized network architecture. We all agree on it, and we’re going to build it”?

Rebel Alliance. I like it. The buyer has to agree. That’s the key.

Yeah.

So, Mike, you’ve seen a lot of cycles so far. Some are big, and some are even bigger. You’ve seen the last really big cycle. How does this compare?

As you may already know, I’ve launched a new venture fund called Anabi Capital. Strictly speaking, I was planning to retire in 2023, buy a yacht, and play golf.

Did it work out? Why?

I actually think now is the most exciting time to do our work and be in our industry. It’s not just because the changes are significant. Even though everyone knows how transformative they are, it takes a lot more intellectual effort to understand what will happen next. It’s like not having a crystal ball. It’s unclear what the next move will be. It’s difficult.

We have to keep thinking, and we have to think at the system level. For example, I don’t think I’ve ever been in an industry where we had to consider how light is focused on a Zeiss mirror to determine whether a chatbot would work properly. It’s all connected in amazing ways, with bottlenecks changing here, there, and everywhere.

I feel this is the most intellectually fascinating era we’ve lived in. The potential reward is so huge. By understanding how these different parts unlock, you can build a company here, build a company there, and one will be worth a trillion dollars, another $500 billion, and so on. Honestly, I never thought I’d encounter a time like this again in my career. We feel that now is the most dynamic and interesting time to do what we’re doing, and I think we should all feel very fortunate.

That’s absolutely right. I completely agree. We’re all working on the same business for pretty much the same reasons. This was a very interesting hour. Thank you so much. This conversation could go on forever, but let’s save it for another time.

I had a great time. Thank you, everyone.

Mike, thank you for coming.

I am grateful.