我们正逼近芯片的物理极限

1. The end of Moore's Law

Welcome to magic. The potential of photonics. The technology for connecting these GPUs. Global payroll and HR platform. 8 billion operational data points. Scalable companies are better than startups. It's better to have a million of one thing than one of everything else. Creating an intelligent layer for defense. There is a lack of electricity. Missing memory chips. We have reached the limits of physics in what we can still do. Okay, great. So, we have the last report for today. Steven and Adam are with us today. Stephen, you are from IMEC. Adam, you're from Stripe. So we'll talk a little bit about what comes after GPU. Let's start with you. Can you explain what IMEC is?

Of course. I think we are perhaps the most unknown hidden gem in the semiconductor industry, at least to the average person. Although we are certainly not strangers to the semiconductor world itself, we are a world leader in chip production in the early stages of their creation.

Hardware is not software. It's not like you give a command and get something 30 seconds later. It's a 5- to 10-year process, and we're at a very early stage where we have unique clean rooms that don't exist anywhere else in the world.

That's why all the biggest chip manufacturers in the world first come to us, in Belgium, to design the next-generation chip that you'll see maybe in 7, 8, or 10 years. After going through the full development phase, we hand it over to them for further commercialization. In this sense, there is almost no chip in the world that imec has not had a hand in, in one way or another.

We have been around for 40 years, and during that time we have, to some extent, defined the roadmap for chip development.

So I should have asked you a secret question, because you know all the secrets of chips. And I am allowed to answer some of them. What can you answer?

Well, I think we'll definitely talk about this more, but we're actually seeing a huge amount of transformation. I think we're now reaching a plateau in what's called Moore's Law.

What does this mean? In recent years, chips have become larger and faster. We have always achieved this through scaling. That means we have been making the transistor smaller and smaller.

We are at the level where the dimension of the transistor is about 1 nanometer. What is 1 nanometer? That's about 100,000 times smaller than a human hair.

This means that we have reached the limits of physical capability, and I believe we are entering an era where our hardware solutions need to be significantly more inventive than they have been in the last 10 to 20 years. That's the only way to stay on the roadmap and enable further increases in chip speed, as has been the case for the last 4 decades.

How do you assess the situation in the chip industry? How do you structure it?

Yes, I think GPUs are great at training right now, but we're running into a bottleneck at the output level. The bottlenecks I pay the most attention to—and look for companies that are overcoming them—are power and energy, memory, cooling, and data transfer.

When we look at the data movement side, the memory shortage in general, increasing the bandwidth, and reducing the latency when transferring the necessary data to the GPU are all very interesting and promising areas that need to be developed.

Regarding cooling, I believe that in the next 5 to 10 years, progress will be made in bringing the liquid that removes heat from the chip as close as possible to the GPU itself, and a significant part of the value will shift to the manufacturing sector. Everyone knows how serious this limitation is, so increasing capacity in this area is critically important.

2. Is hardware the new software?

I recently interviewed Tony Kim from BlackRock at the RAISE AI Summit in Paris. We talked about hardware. It's like an Iron Age renaissance. Everything is shifting toward hardware, while software is struggling right now.

People don't understand this, but the profitability of chips is fantastic, especially compared with software, which is being eaten up by artificial intelligence and token spending. How do you explain this shift in value, and the fact that the pyramid seems to have tipped over?

The demand for what is in short supply on the market is simply extremely high. We lack energy, and we don't have enough memory chips.

If you look at memory, there are only 3 memory companies in the world that account for 95% of the market: Micron, SK hynix, and Samsung. Two of them are in Korea, and a significant portion of their supply is being bought up by hyperscalers, taking away all the stock.

So you've seen prices increase by 700% this year just for the memory part of the chip. This will happen at all levels, because these are the components that are in greatest demand right now. Training and software are constantly improving, but the physical components are harder to improve overnight.

To be honest, I don't think things have changed dramatically. I think it's more about the immaturity of the software market right now.

It may seem strange that I'm talking about the immaturity of the software market, but what we've done in AI over the last 4 or 5 years has, of course, been colossal in terms of software. If you look at it from the perspective of the computational complexity of doing what large language models do, it's still a brute-force approach.

As a result, the value that can be gained from this is quite limited. I think that if we want to move forward, we're going to see new types of software evolution that are much more closely tied to hardware. Because of this, I believe the value will increase significantly.

I think we are already filling the base of this pyramid with iron, creating a lot of value there. I think software will follow suit in that sense.

3. Is there a limit to how many chip companies can exist?

Every year we see the creation of more and more chip companies, probably more than ever before, and venture capital definitely helps finance that. Is there a limit to how many chips can exist on the market?

I think there's a kind of endless demand for them right now. I think it's still a question of how this will continue. The bottleneck is more likely to be production—the production of the right type of components in different regions.

For example, what TSMC is doing there is extremely important. But having the right components to do this, I think, is the real bottleneck, not necessarily the number of companies doing it, because each company can offer a completely different, unique design.

In that sense, it could also revolutionize what we're going to have. We'll probably see many more different types of algorithms in the software world. As a result, we will also need greater diversification of hardware types—not just GPUs, but other things too.

As models get bigger and bigger, what fails first at the chip level?

4. Will AI models keep getting bigger?

Right now, memory is definitely the number 1 component that's already problematic. But I don't think models will necessarily get bigger and bigger.

If you think about it, the latest open-source models, like Kimi K3, already have more parameters than the human brain. On the other hand, the chip we need to operate the K3 model is about a million times less energy-efficient than the human brain. So I think we're going to see a different type of paradigm.

For the last 5 years, we have been in an era of creating ever-larger models. We went from 175 billion parameters to about 10 trillion parameters now. This will stop.

Many companies are now making this transition, abandoning the so-called scaling hypothesis, which involves building ever-larger models, and trying to solve problems with memory bottlenecks.

5. Photonics vs. Copper

The copper era, one might say, is coming to an end. I'm curious what you think about photonics and how it changes the landscape.

Yes, I think there are many advantages to using photonics instead of copper. Copper is convenient in that it is fully electrical and digital, but the bottleneck with photonics is the transition between the optical and electrical levels.

Faster and more efficient data transfer, without the latent heat generation inherent in electrons moving through copper wires, will help us further scale data-transfer efficiency. I think we'll see the architecture constantly changing depending on where photonics are placed in the data center.

Currently, this is happening between clusters using linear interconnect optics, but I think that in the next 5 years we will see these optical components become increasingly closer to the GPUs themselves, and we will move to a state of co-packaged optics.

For me, my daily work at imec is about thinking about what AI could be like in the future. How will AI evolve in 5 to 10 years? From a software perspective, we're seeing a significant diversification of paths and many different approaches, whether it's world models or reinforcement-learning-based approaches.

Whatever scenario ultimately becomes the dominant model of the future, the fastest possible movement of colossal volumes of data will always be a necessity. By analogy with the human brain, our brain is incredibly efficient precisely because of its unique 3-dimensional structure.

Therefore, I believe that photonics, especially in the coming years, will become a key technology that will allow us to create the chips of the future.

I think that, with regard to photonics, it solves many of the problems I mentioned at the beginning. It helps with power consumption because it transfers data faster, and with cooling because GPUs require less cooling if data is transferred via light.

It also solves the issue of data transmission at a fundamental level. Because it eliminates most of the bottlenecks we see in building AI infrastructure right now, it is the technology that is most likely to win.

6. We'll laugh at today's AI in 10 years

In closing, this is our last report today, so thank you all very much. What is your wildest guess at the moment?

I think in 5–10 years, we will laugh at how old-fashioned AI was today. This is my boldest assumption in the sense that, of course, we have experienced an explosion in AI capabilities, but I fundamentally do not believe that today’s technologies are what we will be talking about in 10 years. I think we are actually on the verge of a software revolution.

Those of us who are old enough may remember the days of dial-up internet connections, when we listened to all those signals to connect to the network, and now we smile as we remember how we did it back then. I think in 10 years we’ll be talking about large language models that are mostly “brute force,” just the same. I may be a little biased on this point, but I will say that it is a great idea to hire or invest in people who have received academic and technical education in Europe, as well as experience and business acumen in the US, combining these two worldviews. Many of the best founders I’ve supported and seen have this combination of these two sides. So I would say that both in hiring and in investing, these two are amazing traits.

Well, Stephen, Adam, thank you very much.

Thank you.

I think Louis will come up next and say something.

Hi, this is Molly. If you like our interviews, subscribe to our sourcery.vc newsletter, where we send out weekly emails with the best deals and top tech news, as well as go deeper in our podcasts. Subscribe to Sourcery today. And don't forget to subscribe to the podcast on YouTube, Spotify, Apple, or wherever you listen to us. Registration link in the description.