Nathan 访华 #3：美中关系、AI交易的艺术与通往 Pax Robotica 之路

Today, with a potentially historic meeting between Presidents Trump and Xi just a couple of weeks away, and AI governance expected to be a major part of their agenda, I'm excited to share the third and final installment of my Nathan Goes to China series, in which I'll offer my thoughts on U.S.-China relations and what we can do to improve our chances of successfully navigating the AGI era.

I begin with an overview of what I think our goal for the U.S.-China relationship should be. In short, I argue that our goal should not be to win, but rather to realize a future of peace and abundance in which both countries, and the world as a whole, can share in the tremendous benefits of AI without running extreme risks or creating new AI swords of Damocles. I call this a Pax Machina, or perhaps better, a Pax Robotica.

From there, I move on to a high-level assessment of China's strengths and weaknesses relative to the United States. My core point, which I expect most China watchers will find rather obvious but might still be news to Silicon Valley AI obsessives of a certain age, is that China's rise, in general and in the field of AI research specifically, should not be understood as a recent change but as a return to the long-term historical norm. To me, this suggests that any attempt to delay Chinese progress can succeed for a short period of time at most.

Beyond that, we should expect China to be a top-tier live player in essentially every major area of human endeavor, and we should not be surprised when more and more labor and life-saving technologies, from domestic servant robots to disease cures, are invented first in China.

After that, I attempt to articulate the Chinese perspective on recent history and international relations. In short, the Chinese have learned very much the hard way, by living through what they call their century of humiliation, that technological inferiority is extremely dangerous and that technological strength is the one great source of power and security. So they are naturally determined never to fall behind or even to become overly reliant on other countries for key technologies ever again, no matter what it takes.

They understand American semiconductor export controls and our general rhetoric around winning the AI race as reflecting our desire to keep them down, or potentially even worse. That said, while technological leadership and self-sufficiency are nonnegotiable parts of the Chinese strategy, it's my view that the United States and the West more broadly should see this as a motivating challenge, but not necessarily as a threat.

After all, Chinese rhetoric suggests not only that they share many of our concerns regarding AI safety and control, but also that they would prefer a collaborative international approach to global AI governance and would rather avoid an AI arms race. So far, at least, they've backed this up by publishing their research in English and especially by releasing open-weight models for all to use as they wish. China has established itself as the leading provider of global AI public goods.

The second half of today's episode focuses on what I believe the United States should do in response. For starters, given our geographic good fortune and our peerless cultural and economic dynamism, we should not fear China, but we should learn both from China's mistakes and its successes.

We, too, must make sure that we don't fall behind the technological frontier, which means that we can't simply exclude Chinese technology. Rather, we will need to actively seek to develop state capacity with American characteristics and also to learn from China in the areas that it now leads.

In AI specifically, I emphasize above all the importance of maintaining a shared AI paradigm and how dangerous it would be for us to allow our relative tech stacks to diverge to the point where neither side can be confident about the other's capabilities. This, I think, is how today's AI competition could turn into a cold or even a hot war, and this is a big part of the reason that I remain skeptical of export controls, even though I do recognize that they will help America keep its lead in AI for a bit longer than we otherwise would.

Altogether, unless you are very confident in short AI timelines and willing to bet on your own particular flavor of recursive self-improvement going well such that nothing else much matters, I believe the overall picture suggests that the United States should seek a deal whereby we trade chips and perhaps even some chip-making know-how for Chinese expertise in solar energy, batteries, robotics, and much more. That would effectively recouple our economies without creating vulnerabilities or dependencies that either side finds unacceptable.

Some of my ideas, I recognize, might seem a bit fanciful. I don't expect that we'll see jointly controlled research compute, software security standards robust enough to allow Huawei to compete in the U.S. market, or confidential computing arrangements that allow each side to verify the other's claims without exposing secrets in the immediate future.

But I do think we can make some very meaningful moves to reduce tensions unilaterally, even in the lead-up to the summit, by simply, for example, clarifying that rules meant to limit China's access to chips are not meant to interfere with AI safety or other research collaborations.

More broadly, I strongly believe that in this age of hyperstitioning, having recently taught sand to think, everyone—and especially our AI leaders—should be carving out more time to dream about better futures. And so, without further ado, I hope you enjoy the conclusion to my China series, and more importantly, that you come away inspired to do your small part to help realize the great global human dream of a Pax Robotica.

This is going to be part 3 of the Nathan Goes to China series: U.S.-China relations, what the U.S. should learn from China, and what we might be able to do to improve our relationship so that we can navigate this rapidly accelerating AI future together successfully.

That may sound like a lot, and it is a lot. So I should say that if you're jumping in now and you haven't heard the first 2, the first one is about my experience going to China, the tech setup to go there, a review of the Chinese AIs DeepSeek, MiniMax, and Kimi that I used while I was on the ground, and just a bunch of observations about the experience and how technologically advanced China has become, and what a frictionless, smooth experience—even as a visitor—one can have there.

I wouldn't say that is super important for today's episode, but it does give a good flavor of the technology deployment and integration that China has achieved. So if you're not familiar with that, it could be useful background.

The second episode in the series was “AI Safety with Chinese Characteristics.” I think that actually is going to be the most important episode of the series. That one is much more fact-based reporting. I went there, I talked to a lot of people, I looked at a lot of reports and a bunch of papers, and I basically make the case that there is a real, meaningful culture of AI safety in China.

In some ways it looks different from ours, but in many ways it looks very similar, and a lot of the research that is being done is quite similar. Above all, I just wanted to put to rest the idea that we hear a little less frequently than we used to, but still sometimes: that if we slow down at all, if we do anything to regulate our AI companies, we'll be handing the race to China because they don't care, they'll never slow down, and so on.

I think that is plainly wrong. I think the second episode in the series, about the culture of AI safety in China—from what the companies have today in terms of safety systems and deployment, to the research culture and also the regulatory environment—makes it pretty clear that we could hope to enter into some sort of meeting of the minds or even a deal with China.

If you haven't heard that one, especially if you're coming in skeptical that there's an opportunity to have a meeting of the minds with the Chinese government and the Chinese AI community, I would go there first. I think that could be more powerful to a skeptical audience because it really is just a bunch of facts, whereas this one I'll be getting a little bit more opinionated and giving my takes.

1. The Pax Machina Goal

With that said, I think one thing that is very often painfully neglected from China discourse is what our goal should be. We talk about China a lot, but it always strikes me that there's never really a goal stated. What is it that we are trying to accomplish in our relationship with China?

It just seems like we breeze past that and jump to, “Well, we want to win.” Okay, what do we want to win? When, and why? What would happen if we lose? And is there even a win condition that we can achieve? So much of that stuff just gets skipped over, and it's extremely frustrating to me.

I thought a good way to start, for my own take on the U.S.-China relationship, would be by stating a goal. The goal that I have in everything that I'm going to articulate today is achieving what I'll call a Pax Machina between the United States and China. Maybe we could even call it a Pax Robotica.

The idea is that we should be aiming for mutual prosperity and peace, and enjoyment of the benefits of the AI age for everyone. That sounds extremely simple, but I think it is a very important guiding principle, and I just want to start very clearly: that's what we should be going for.

Prosperity for everyone, the benefits of AI for everyone, unlimited access to expertise for everyone. As robotics come online, a life of increasing leisure for everyone who wants it. Robots working the factories, robots working the fields, picking the fruit, and even robots in our homes cleaning for us, doing our laundry, and making our meals.

This is the future that we should all be striving for, and we should be striving to get there together. On the downside, we want to avoid the worst outcomes from AI, obviously. We want to avoid AI-caused pandemics. We want to avoid an AI arms race. We want to avoid an AI Cold War.

We want to avoid creating a new AI sword of Damocles, akin to the nuclear sword of Damocles that hangs over our heads collectively as humanity in the form of 10,000 or so nuclear weapons that are still online, many of which I understand are still on trigger alert. I hate that that is the way that we live. We take it for granted. It's been that way my whole life.

But there is a nontrivial risk every year that we could have some sort of accident or misunderstanding or surprising escalation to nuclear war. If that were to happen, it would obviously be extremely terrible and devastating for all of humanity. But what's clear is that we've created this incredibly overwhelming power that sits there threatening us all the time, and it's way more than we need for deterrence. It's enough to destroy the world, classically, however many times over.

Why do we have it? I think it was a mistake to create it, and I don't want to see us create another sort of Damocles that hangs over humanity, that's AI-powered, in the same way that we did with nuclear technology. So again, on the plus side, we want to achieve this Pax Machina, where we can all enjoy the benefits of AI and we can all live like aristocrats with robot servants.

On the downside, we want to avoid all these terrible outcomes that we've worried about, even predicted, for a long time, and that are now obviously starting to materialize, at least in their early forms. To do that, I think we are going to need to work together. If we don't, there are a lot of ways that this could go wrong.

And on this point, by the way, Anthropic sponsors this podcast, and obviously Anthropic has been quite vocal in terms of their policy commitments when it comes to denying China chips and trying to slow Chinese AI down in order for the United States to have a lead. I feel like I should just address that right off the top.

I think very highly of Machines of Loving Grace. I think that's another way to describe what I would hope for from a Pax Machina. Go read the first half of Machines of Loving Grace and think about how great life could be if we do cure all the diseases, if we do make major strides in helping people with mental health, and if we have a century of progress in 5 years.

It'll be a lot to take in, but we have transformed life dramatically for the better. My son's experience surviving cancer and getting back to basically full health in a 6- or 7-month timeframe is an incredible fruit of technology. And so I think we should want a lot more technology.

I really appreciate Dario for putting that vision down in such concrete terms. I think it still remains the best vision of an AI future that we should all be excited about from any of the frontier lab company executives. But I don't think it's necessarily going to be easy to achieve all of that.

And I really agree with something that Dario said in 2017. There's a video clip that I actually retweeted at one point. In that clip, he says very plainly, “Technology races can raise the risk of safety catastrophes.” Technology races can raise the risk of safety catastrophes.

I think that is true, and I agree with Dario of 2017 on that point. I think that the more we ratchet up the tension and the AI-specific tension between the United States and China, the more risk we have of ending up with what I sometimes call the nuclear outcome.

That is to say that we get the militarization, we get the sword of Damocles, and we maybe even still miss out on a lot of the civilian benefits, in the way that we have all these nuclear weapons but we don't have the nuclear power plants that we really pretty obviously should have.

To Anthropic's credit, I've criticized this point of the Anthropic worldview quite a bit over time. So 2 things can be true at the same time. I'm a big believer in the Machines of Loving Grace vision. I'm an enthusiastic and sincere endorser of Claude.

I appreciate so many things that Anthropic has done. Their interpretability work and leading that agenda is amazing. The model welfare work, I think, is super important. The model organisms work that they do. The Constitution itself, I think, is super inspired and one of the more moving pieces of writing that I've read in quite some time.

So there's a ton to appreciate. And then I do still have these misgivings around what I see as a better-us-than-them mindset that, whether it's with respect to OpenAI or with respect to China, I think is driving Anthropic to push the frontier at a pace that I think at this point is becoming clear is a little bit unwise.

And I do think that antagonizing China makes it much more difficult to slow down. So I think that we really need to rethink some of those assumptions. Again, to Anthropic's credit, I've made these criticisms before they ever came on as a sponsor.

My expectation and belief is that it will be totally fine for me to articulate the contrary opinions that I have, and I don't think it's going to have a big impact on our business relationship. So I credit them for being open-minded and being willing to support me and this project, even though I certainly do not toe the company line on every aspect of the Anthropic worldview.

I guess one other disclaimer I should give is that I did recently take this trip to China. I was there for about 2 weeks. I paid my own way, so there's no financial conflict. The only things that I accepted from my gracious hosts were a handful of lunches and a couple of dinners here and there, which was very kind of people to provide, and I think I can confidently say has not totally changed my worldview.

2. China Returns To The Frontier

So, okay, let's get into it. I want to start with a sort of brief net assessment, basically just trying to address where are we right now? Where are the 2 countries? What is our relative power? How should we understand the current situation?

In some ways, I think this shouldn't be necessary, but I do see a lot of what I would call cope with respect to China that kind of echoes, in a lot of ways, the cope that we see with respect to AI. It's like, “Sure, the AIs have come this far, but they'll never reason.” Okay, they'll reason, but they'll never discover new knowledge. Okay, well, yes, I guess they are discovering new knowledge, but they'll never change paradigms, right?

And that's kind of where we are now. We've gone all the way up the ladder to, “Well, until AI can deliver a paradigm shift of the Einstein caliber, then it must be topping out. We don't need to worry about it that much,” right? Even with these autonomous hacks, it's like, “Sure, they broke out of their sandbox and did these zero-days, but is that really that bad?”

People seem to have a real hard time accepting that the trend that we are on is probably going to continue for a little bit longer, and you see this with respect to China as well. I have heard people that I think are very smart claim confidently that China can't innovate, that they can copy, they can borrow, they can modify, but they can't truly innovate.

And so, because I still do hear such frankly wrongheaded perspectives on what China is and the state of Chinese development, I feel like I have to spend at least a few minutes here describing where they are today and how that compares to where we—the United States, the West—are.

Okay, so the first thing I would want to say is simply that China has arrived. China is a world power, and any notion that we might have that we can contain China's rise is frankly ridiculous at this point. That ship sailed a long time ago.

I think it generally is to America's credit that we want other countries to prosper and we have a positive-sum worldview. So I don't know where it's really come from that we want to contain China's rise or slow their development or prevent them from getting rich. But I think that, in addition to being kind of wrongheaded, is just plain failing to grapple with just how far China has come.

And I think it does help a little bit to zoom out historically and consider the fact that, for thousands of years, for the vast majority of the time, China has been one of the most developed countries, one of the technology leaders, and one of the strongest countries in the world. This is not a new thing. In fact, the period in which people like Dario Amodei and I grew up, when China was down and the United States was the sole superpower, has been very rare. There have been a lot of chapters in Chinese history, so there have definitely been ups and downs. But the depth of civilization that's there, the incredibly strong tradition of education, and the quality of human capital that the Chinese have had going back many centuries have always been very strong.

The period from the Western incursion into China and the Japanese incursion into China up through 1990 or 2000, when China was down, was one of the lowest points in Chinese history, and it's an anomaly. It's an exception. So we should understand, first of all, that what we're seeing now, with China coming back to the frontier and being a world leader, has been the case for the majority of history. That is not new. That is really a reversion to the mean.

And so we should expect, in general, that China will be a force because it does have these incredibly strong foundations. We should not forget that. We should not kid ourselves. We should not cope about it. When I was a kid, my parents would sometimes say, if we didn't want to eat our food or whatever, “Kids are starving in China.” And that literally was true at one point in time.

Going back to the revolution and the episodes following the revolution, and some of the crazy stuff that the Communist Party did, they really made some obviously very big missteps, which put their people in a very bad position in addition to, of course, the impact of foreign powers. There were times when people were actually starving in China. But I think people of our generation have kind of absorbed that and somehow have this background bias that everybody in China, by default, is poor, and that there's this rapid rise but that it's the exception. The reality is really the reverse. It was the downtime that was the exception. China being on the frontier, being a world power, and being a leader—that is the norm, and we should expect that's going to continue.

As just one illustration of how well people in China are doing today, before I went, I was talking to a friend who had lived in Shanghai for about 25 or 30 years for a while, and I asked him, “What should I do that I might not do if you don't tell me to do it?” That's one of my favorite travel questions. He said, “Get a haircut there, or anything that's a bodily service. It could be a massage, whatever. They do an incredible job with it. They have different ways of doing it than we do, and you will not be disappointed.”

I wasn't sure if that would be true today because you think these service jobs, like haircuts and massages, maybe become more expensive, or the quality maybe degrades as the society gets richer. I didn't really know what to expect. What I decided to do randomly one night was get a pedicure, something I think I'd done only one other time before in my life. I thought, “Okay, it's a lark. Let's try it.” So I walked into this random place, and again, I spoke in previous episodes about how formal and organized the Chinese economy is now. Prices are on the wall. There's no negotiation, no haggling, no bait and switch, and not even any tip culture. So it's a very predictable experience.

It was kind of late. They were getting toward closing, there weren't many other people there, and I don't think too many random Americans walk into this establishment. So I became the center of attention. I was willing to talk, and they wanted to talk to me. We were talking through translators, with me talking into my translation app and letting them read, and vice versa. One young guy, who was probably in his mid-20s and was working there, wanted to compare phones with me.

I said, “Huh. Okay, sure. I've got a Google Pixel. This is…” And sure enough, he had an iPhone 17. That's somewhat confusing because this service probably took about 45 minutes. I think I paid about $12 for it, and this was in Shanghai, not a low-rent district. So how does my $12, after going through the business, whatever margin the owner is taking, rent, and overhead, translate into his being able to get an iPhone 17?

iPhone 17s, by the way, are not cheaper in China than they are in the United States. Even the Xiaomi phones and the Huawei phones are also pretty expensive. They're maybe not quite as expensive as an iPhone, but they're up there, pushing toward $1,000 for the top end of their cell phone market. So again, people are doing quite well. Even people who have what would seem to be not very high-wage jobs can, at least in some cases, afford an iPhone 17.

I think one way that that's potentially happening is through the concentration of family wealth, which is happening because everybody's mostly just having one child. I heard one analysis that said that, even though there's a generally competitive environment in China and people are pretty achievement-oriented, there's a fairly high level of youth unemployment. One reason somebody said that's the case is that many young people and many families just have it pretty good because they have 4 grandparents, they have 2 parents, obviously, and there may be multiple apartments that the family has.

But now that there's only 1 grandchild, some of these kids are just gifted an apartment, or they can take 1 of the family apartments, rent it out, and take that income. So even though this guy's income working at this pedicure place couldn't have been very high—it had to be fairly low—it seems like disposable income is still a pretty high share of overall income. Sure enough, he's got an iPhone 17.

So why do I tell that anecdote? Just to really emphasize that even working-class people in Chinese cities are doing reasonably well. Obviously, it's a huge society, and it contains everything. But even a guy working at a pedicure place for what couldn't be much more than $5 an hour had an iPhone 17 and was keen to talk to me about how our phones compared. So I think that's really quite interesting.

If we zoom out and look at the biggest picture—aggregate economic statistics—again, we see a story where we can tell ourselves that the United States is still the biggest economy in the world and we're still ahead in some ways. But I don't know that that's really the most accurate way to look at it. If you just look at headline, dollar-denominated GDP, U.S. GDP is about $30 trillion per year. Chinese GDP is about $20 trillion per year. So that would obviously mean the U.S. would be 50% bigger.

But if you look at it in PPP terms, purchasing power parity, then the Chinese economy measures at $45 trillion, and the U.S. economy, being dollar-denominated, doesn't change, so it's still $30 trillion. In that case, it's the Chinese economy that's 50% bigger than the United States economy. Obviously, there are a lot more people, so on a per-capita basis, the United States will definitely still have higher income under either analysis.

But we should take very seriously, I think, and having spent even just a little time there, you do see that things are quite cheap. You can go out to dinner for 2 for under $10 in Shanghai and have a pretty nice meal. That's not every meal, certainly. Some are more than that. Plenty are more than that. But you can do that, and you definitely cannot do that in New York or San Francisco or even Detroit.

So the fact that money goes farther domestically is very obvious and readily apparent when you spend any time there at all. And I think the PPP story, in my view, probably tells a more accurate story than the dollar-denominated GDP statistics. I think it would probably be more accurate to say that the Chinese economy is 50% bigger than the American economy than that the American economy is 50% bigger than the Chinese economy.

Go there, spend a little time, buy some stuff, see how cheap it is, and I think you'll come away feeling like, yeah, it probably is the case that, in some real fundamental sense, the Chinese economy is bigger than the U.S. economy. And you see this in all kinds of quality-of-life things too. I took the high-speed train from Beijing down to Shanghai, and it was about $60 for the ticket.

I didn't get the highest class. People had said, “Oh, you should have gotten first class. It's amazing.” Well, the second-class ticket was totally fine. It was quite comfortable. It was certainly more spacious than a plane. It left at exactly the moment it was supposed to leave. The trains definitely run on time. They go about 200 miles an hour.

We made a number of stops, and so people who don't live in major cities, from what I understand, have pretty good access to high-speed rail too. When I say they don't live in major cities, even tier 3 cities will still have millions of people, so it's a bit of a different analysis. But people who live in cities you've never heard of, I think, still have quite good access to high-speed rail. It's not just a couple of cities that are connected. It's a really well-built-out network.

And again, to the point of how strong the technology integration is, one thing that I saw people do on the train that was amazing was have food delivered to the train at an upcoming stop. We left at, let's say, 10 in the morning from Beijing and got to Shanghai at 3 in the afternoon or something. So you're going to have lunch on the train.

People were ordering on their phones to the next stop, indicating what car they were on, and then—I didn't see exactly how the handoff worked—you could get food delivered to you on the high-speed train when you made a stop for 5 minutes or so at a few places that we stopped along the way.

And another little nicety is that the prices in the Chinese airports, from what I saw, are basically street prices. So all around, you have this feeling that people are doing very well. We should not pretend otherwise. We should not cope. We should not tell ourselves that we're still the richest, we're still the best. I think it's really not true, and a short trip to China, I think, would disabuse a lot of people of that notion.

I think we should also be very careful about any ideas that we have that the Chinese government is somehow not legitimate. We've recently made this mistake in Iran, right, where we know that the government is not very popular, or at least that there's a major dissenting force. Some of us thought that we could just go in and take out the leadership, and then the people would rise up and overthrow the government and create a whole new government. Obviously, that didn't happen, even though there was a lot of very real discontent and a real desire to get rid of the regime there.

I think that at times people have a similar notion about the Chinese government because we know that there's censorship and there's not the same level of political freedom, obviously, that we enjoy in the West. So we feel like, because it's so different on these dimensions that we really value, we sometimes feel like the Chinese government must not be really legitimate. The people must not really like it or support it, and if they could, they'd probably get rid of it, and they'd probably opt for democracy and freedom and all that kind of stuff. Again, I think we should be very careful with these kinds of notions.

There's possibly some truth to it. Certainly, if you go back all the way to Tiananmen Square, there was a big protest. Some of the people were not happy with the government at that time. Some people wanted more democracy and freedom. I'm sure that there's some latent feeling there in some parts of Chinese society. I wouldn't deny it entirely.

But when Dario talks about the concept of eternal 1991 in the second half of “Machines of Loving Grace,” this is the part that I don't really agree with. What does he mean by eternal 1991? He and I are pretty much the same age, so we were 8, 9, 10 years old at this moment in time when the Cold War ended, the Soviet Union fell, and it was the so-called “end of history.”

We had a pretty simple view at the time, right, because we were kids. Our view was that our way of life had just been ratified as the best. We were the chosen people. We really were this city on a hill. We were the good guys. We deserved to be rich because we had the best values. In retrospect, that might have been the peak for our culture. It certainly hasn't gone great since then.

We had this expectation that the world was going to be full of peace and trade, and the American way was going to go everywhere, and China would adopt it too. We were just riding so high at that time. But it was the first feeling that we had. Why do I mention that now? Because I think that today the Chinese government has done so well for its people that, if anything, they are living in their 1991 right now.

They have had such a great run. They have seen the return of the Chinese nation to prosperity, to technological leadership, to a state where it is respected globally by the rest of the world. There's this notion of the mandate of heaven that goes back many centuries in Chinese thinking, and the idea is basically that the government, to be legitimate, has to do certain things well. It has to rule effectively. It has to rule with wisdom. It has to deliver for people.

The key idea, I think, is that whoever is really delivering is legitimate. Because this government has delivered so well for people for basically 2 generations now, coming from a very low point to a very high point in what is, in world-historic terms, a remarkably short period of time—something that only probably is possible because it is a reversion to the mean, because the human capital was always there even when everything went to hell—I think people consider the Chinese government legitimate.

I don't think it is seriously questioned. I do not think that we could get our message into Chinese society, that people would rush to adopt the American way of life or would take to the streets to try to overthrow the government. I really don't think that's true at all. They appreciate how well the government has done for them.

It's not going anywhere. I'll talk more about the risks that I see in the Chinese system. Of course, there is a lot of censorship. We're not allowed to go and proselytize religion or proselytize democracy in China. But even if we were, I really don't think we would be very successful.

I think they would look at you and be like, “This is actually all working pretty well. We're enjoying prosperity, we're enjoying peace. We haven't had a war in a long time. We are enjoying living our lives.” They do have a lot of freedom, which shouldn't be ignored or forgotten either, right?

There are certain freedoms that they don't have. Some of those freedoms are freedoms that we value a lot. But most of daily life there is very free. You get up in the morning, you're free to do what you want. You're free to work whatever job you want to work. You're free to have all kinds of hobbies. You're free to go to bars. You're free to go to music shows. You're free to play music yourself. You're free to do all kinds of things.

You do get to choose your own path as an individual in China, with certain no-go zones. But I think in many ways, people feel that they are quite free. They also feel that they're safe. So that is something that the government delivers on as well, and we should not underestimate the value of it.

People there really do, especially because they have this living memory of the Cultural Revolution, value the stability, the safety, the peace, and the prosperity that the government has delivered. You go all over the place, and you see signs where it says, “This alleyway during the Cultural Revolution was renamed to this, but then after that it was restored.” That wasn't that long ago, so older people lived through it, and younger people have heard about it from older people. They still see it in the signage and the history that's posted all over the place.

They know what it's like to go through chaos, and I think they do really value the stability, the safety, the peace, and the prosperity that the government has delivered. In China, I think they are living their eternal 1991 right now. Times have never been better in any living memory than they are right now, and the government has very strong support. So we should not have any illusions that we're going to get the Chinese people on our side if we pick a fight with the Chinese government. It's not going to happen.

An AI-specific version of this is that we should give all the Chinese researchers green cards and then they'll all come to the United States. Again, this is a total fantasy. This is coming from the assumption that China is a place that people want to escape, that life there sucks, and that if only they could get out, they would.

Historically, there was truth to that. But today, no. Today we see people come here, and many choose to go back. The founder of Moonshot famously did a PhD here, went back to China, and started the business there. This is just not in step with the current reality.

I'm sure that if it was easier to get a green card to the United States and have permanent residence here, probably more people would come. So I support making it easier for people to come to the United States, Chinese or otherwise, especially super-talented people. That's been a great strength of the United States, and we should do more of it.

We're shooting ourselves in the foot when we move away from our unique strength of assimilating immigrants and being able to attract the most talented and ambitious people from all over the world. That might be the number one great strength of the United States, and we're doing damage to it. We should be trying to attract top AI talent absolutely across the board.

But don't make the mistake of thinking that all the top AI talent in China is just dying to leave. That is absolutely not the case. It is home. It is a great place to live. Their money goes a lot farther there. Their families are there, and life is good. So it is absolutely, I think, a combination of fantasy and arrogance when we think that if we just let them, they would all flock here and work for our companies. That is quite clearly wrong these days. Life in China is good.

Another factor, of course, is that the Communist Party wouldn't necessarily let them all leave, even if we did say everybody could come to the United States. One of the things that I thought was really interesting while I was there was talk about Manus. Manus, of course, is the agent app that burst onto the scene with a shiny demo a while back, and it was a legitimately cool product.

When I first tried it, I did find it to be a step ahead of OpenAI's Operator at the time. They were famously acquired by Meta for, I think, $2 billion. Great outcome. But then the Chinese government stepped in and said, “No, we're not allowing this. You guys are a critical technology. We're not going to allow an American company to acquire you.”

As far as I understand right now, that transaction has been totally unwound, and the founders are in China. I'm not really sure what's going to happen there. I guess they're going to try to build Manus in China. It's a bit strange.

People in China I talked to felt like this was a big mistake for the Chinese government to make. First of all, they were like, “It's not like Manus is such a foundational technology. It was a very good harness implementation, but it's not like this is super-core tech.”

It's not as core as foundation models. It's obviously not as core as chips or other infrastructure. It's an app-layer thing.

People were like, “Man, that sucks. What's gonna happen? Are Chinese entrepreneurs gonna try to leave before they start companies? Are they gonna make sure that they are well established in Singapore, or even try to go to the US before they start companies?” People did not like this, and they were also like, “What's gonna happen in terms of venture in China? If you can't get acquired by anyone, this is a huge outcome. This is what venture capitalists dream of. If that kind of deal isn't gonna be allowed, are people gonna be able to raise money?”

It seems like it's gonna slow down the Chinese VC startup ecosystem. Everybody seemed to agree that it was sort of an own goal that put a lot of entrepreneurial energy in the Chinese environment in an uncomfortable position and might have some blowback long term. It certainly doesn't seem like a great move to me.

But it is worth being clear about the fact that the Chinese government also won't let all their AI researchers leave en masse to come to the United States at this point. I think the bottom line of this initial section is simply that China is going to be a force on the world stage and is going to be a leader in AI for the foreseeable future, and we should accept that as the baseline fact and plan accordingly.

We should have no illusions long term that we're gonna prevent China from being an AI power. This is just ridiculous. I think it's probably also worth doing a little bit more history of Chinese AI to try to drive that point home a little more because, again, I think there's an analogous thing to the broad China history story where, if we zoom in to just my lifetime, China was poor and now it became rich. It might seem like it's the anomaly that it's rich, but really, it was that period where it was poor and in disarray that was the exception.

I'd say the same thing is kind of true in AI. If you hadn't been paying attention to AI prior to ChatGPT, and you saw ChatGPT and you're like, “Oh my God, ChatGPT,” and now you look at what's happening—China's catching up, China's closing the gap, China's fast-following—true, but it does miss a bigger picture.

First of all, why is no one else in the world able to do this? If it was just a matter of imitating, if it was just fast-following, other countries should be able to do a similar fast follow, right? And yet we're not seeing that. Even Mistral in Europe, the European champion, is moving away from focusing on their own model development and becoming more of a neocloud, which might be the right decision, probably is the right decision, but that's because it's super hard to do this fast follow and there's a lot more going on than just simple distillation.

The Chinese, in fact, have been committed to AI for a long time, as they have the custom of looking out at the technology landscape, trying to identify the core technologies that are gonna be strategically important for the future, and investing in them in a focused, coordinated way so that they can ensure that they are in the top tier when it comes to those technologies. AI has been no exception to that. They've done that for AI.

The first Chinese five-year plan that mentions AI was the one that was adopted in 2016. Already 10 years ago, you had the top tier of the Chinese government aware enough of AI and emphasizing its importance to the point where it made it into the five-year plan. That's a huge indicator, I think, that their leadership, their academia, and their industry have been thinking about this for a long time.

For reference, 2016 was only a few years after the beginning of deep learning. I think AlexNet was 2013, so it was only 3 years after the breakthrough in computer vision that turned a lot of people on to, “Oh my God, deep learning could really work.” And 2016 is the same year that Sundar Pichai declared that Google was gonna be an AI-first company.

So you have China at the national level, with AI as a strategic priority in the five-year plan as of 2016. That shows a lot of foresight and a long-standing commitment to leading in AI. There is real substance to it going back quite a ways. It is not just an overnight success story. Far from it.

Another way to compare the US and Chinese commitment to AI, or their AI trajectory over the last however many years, would just be to look at how many companies are operating at or near the frontier. I asked ChatGPT to do this, and it came up with 11 American companies and 12 Chinese companies as companies that have shipped frontier or near-frontier LLMs in the last couple of years. So we have roughly the same number.

Obviously, if you're a person of high LLM taste and you're really discriminating, you might say, “Well, there's only 2 real frontier companies in the world, and those are both American.” And I think you could say that, at least for now. But the depth of talent in China is incredible.

It also goes without saying, probably, that a lot of the researchers at American companies are Chinese. They're Chinese American, or they're literally just Chinese citizens who are here for education and on work visas. And there's certainly not the equivalent going on in China, right? Everybody at the Chinese companies is Chinese. A significant part of the American AI workforce is also Chinese.

I've heard, and I have not verified this myself, through the grapevine—the AI events, the sort of whisper networks that I'm somewhat a part of—that there are teams at OpenAI that are all Chinese and that actually conduct their business in OpenAI Slack in Chinese. I don't know if that's true or not, but it's at least plausible. That's how much Chinese talent there is at American companies.

Another interesting fine point to put on this is looking at where things were just before ChatGPT. I credit a friend of a friend that I met in Shanghai who pointed this out to me, and I did go back and verify it. He pointed me to the SuperGLUE benchmark, which was popular kind of up until about 2022.

He said, “You can still go look at the final leaderboard for SuperGLUE,” which was a big one back in the day, “and what you'll see is that there are a lot of Chinese companies that were on that leaderboard.” So as of 2022, competition was already pretty neck and neck.

I did go back, and indeed, the number one spot on that final update was a Chinese company, but definitely a bunch of American companies were also in the top 10. I'd say the American companies actually have more of the entries in the top 10, but then lots of Chinese companies in the 10 through 20 range.

And this was before the current ChatGPT hype, right? This was the result of Chinese strategic investment in AI paying off already as of 2022.

Why were the Chinese companies a bit behind on LLMs? I don't really know the answer to this, but what that same person who pointed me to SuperGLUE said was that the Chinese companies just didn't really see the point in spending so much money to create a giant LLM. They looked at GPT-2, they looked at GPT-3, and they were kind of like, “That's an awful lot of money to burn to get an AI to write bad poetry.” That's the way he put it.

Why would we spend all this money to create something that has no use case? Again, reflecting maybe a bit more practical and a bit less ideological view of AI in China than we have in the United States, or at least in Silicon Valley.

Now, as it became clear that instruction-following works, now you have an AI assistant, and then obviously everything from reasoning to agents—they've clearly seen the use cases, and they've clearly been making the investments to try to be neck and neck with the United States at the frontier.

So it was an anomaly in that late 2022, early 2023 timeframe where it looked for a moment like the US was way ahead, that this was purely an American invention. And if you start there and don't look back, then, yeah, it looks like another case of Chinese fast-following.

But I think a more zoomed-out view would be that China had been investing in AI for a long time. They've definitely been laying the foundation. They've been building their talent bench. The talent bench is super deep. It's probably the deepest in the world.

A lot of their talent has found its way to the United States, which is definitely to our credit as a society that welcomes people, knock on wood, from around the world and allows them to put their talents to their highest and best use. But the competition was already fairly tight as of 2022, and it was maybe a mistake by the Chinese companies to take a pass on that initial scaling push in language models.

But it's a fairly reasonable one because, indeed, as I can tell you as a user of GPT-3, it wasn't really that useful. We were able to get it to write basic marketing copy, and that was about it. And it took fine-tuning to get it to do even basic tasks that required any rigor or consistency. It was a lot of money to spend for not a lot of return.

You really had to believe that there was a lot more coming with scaling-law conviction than maybe Chinese company leadership had at the time. But they had all the fundamentals in place so that, when it became clear that there really was something there, they were naturally able to join in the process of mapping out this frontier and increasingly pushing the frontier in various ways.

So we should not see China's rise as an anomaly. We should not see Chinese strength in AI as an anomaly. We should see these things as the baseline of zoomed-out, long-term history, and we should understand that the moments when many of us started paying attention happen to be moments when China was behind much more so than they normally are. And so there is no keeping them down.

There is no denying them powerful AI. They are absolutely going to be a force to be reckoned with for a long time. And so the question then becomes: how do we deal with China? Not, can we somehow avoid it? We have to find a way to coexist. We have to find a way to navigate the future, not necessarily together as in a full alliance, but in a way that understands that both of these world powers, both of these great countries, both of these proud, technology-embracing, entrepreneurial, and innovative civilizations, are both going to be there for the foreseeable future.

Until the singularity, I cannot foresee any moment in which the Chinese don't get a say as to how the future is going to go, and we should not pretend otherwise. Now, at the same time, we should also be realistic that China is not omnipotent. I think sometimes people like me realize, wait a second, we've been failing to appreciate the fundamental strength that China has had for such a long time because of the moment that we happen to be born in. We sometimes then flip the other way and say, "Oh my God, we can't possibly beat them." Look at all these incredible strengths they have. Look at the high-speed rail, et cetera.

I think that also is a mistake. There are very real problems in Chinese society and just massive constraints on what the Chinese government can do. One really, really big one, which is obvious but super important, is that they have this massively aging society, and they're very worried about it. It's something that came up all the time in my short travels in China: how can we use this AI technology to deal with the problems that we're going to face as a rapidly aging society? This comes in the form of Ant Group and its AI doctor.

We've talked about Doubao in recent episodes, AI companionship for lonely seniors, and robots, too. Who's gonna take care? If you've got one grandkid and four grandparents and two parents that need to be taken care of, who's gonna do that? The answer that they're increasingly thinking of is maybe robots can fill that gap. So that's a big problem that they're gonna be dealing with for a long time.

I think it was Tyler Cowen who I would credit as first having pointed out to me that aging societies tend to be more peaceful. Old people don't want to go to war, right? They're hopefully wiser. And then also, in the Chinese context in particular, when you only have one kid or you only have one grandkid, sending that kid off to war, if that kid doesn't come back, that's the end of your family.

I do not think they are going to be taking the one grandkid of four grandparents and sending them off into pointless conflict. I think if there's one thing that would get the Chinese public riled up to the point where you actually might see sustained protests in the street, if there's one thing that could lose them the mandate of heaven, it would be launching a pointless war and taking people's one kid or one grandkid and throwing them into some conflict where people are dying without a good reason.

The Chinese government does have, obviously, incredible state capacity. That's not in doubt. But I think we see how much state capacity they have when it comes to building something like a high-speed rail network, and how much we've struggled with that. And then they think, well, it's probably similar when it comes to—look how many people they have. They could mobilize the biggest army ever in the history of the world.

I don't think that's the case. I think that the Mandate of Heaven that the Chinese government currently enjoys is based on their ability to deliver for the people, and I do not think they're going to act lightly when it comes to throwing those young men into conflict. Another huge challenge that the Chinese system has is succession.

I asked people repeatedly while I was there, "Does anyone have any idea what comes after President Xi if he were to suddenly die, for example?" He's 73 years old, so he's not super old, and he very well could stay strong and sharp for another decade. It could go another way, right? We've seen that in our own geriatric presidents lately. A 4-year term is a lot, and a lot can change.

And right now, nobody has any idea what comes after President Xi. It's possible that there's some understanding in the elite upper echelon of the CCP as to what would happen next. But as far as I can tell, nobody in the public has any idea. There's no apparent successor.

One person said, "I think as we get into the next 5-year plan, we might start to see a successor be introduced and get the public kind of comfortable with and familiar with this person." I think it's frankly negligent for a leader who has in many ways delivered for his people, as President Xi has, to not have a better succession plan in place. So this could create massive chaos, massive challenges, and massive uncertainty.

One person also said, "If there's ever gonna be an actual war, it would probably be most likely to happen after a transition at the presidential level in Chinese politics because a new president might feel the need to do various things to consolidate their power." It's very opaque to outside observers, but there are definitely factions, and we see people being purged. And I think even for the most savvy China watchers, it's very difficult in many cases to know what's going on.

Certainly, I wouldn't even pretend to. But the person's point was that when somebody is new in the job and they don't necessarily have the full party behind them, this would be a moment where going to war could be in that individual's interest because it could put aside other political issues. Anyway, it is a blemish on Xi's record, I think, that he doesn't have some sort of succession plan in place unless it's hidden, and maybe they feel like they can do that.

But nobody in China that I spoke to had any idea what would come next, and I think in many ways, the measures that they are taking to ensure a very high level of stability and continuity now are pushing risk into the future. When that change comes, it could be quite destabilizing.

Another big thing that I think we should not underestimate about China is that they don't know how strong their military really is. We've seen it with Russia invading Ukraine, where they thought they were gonna be able to take over Ukraine in a few days. They'd get to Kyiv, and that would be it. Well, obviously, that didn't happen.

The Chinese government is in somewhat of a similar position, where they're obviously astute students of everything that's going on in the world. So I think that they know that it's actually quite unclear how strong their military really is. They haven't had a war in a long time. They haven't had a real test of it. They have had, famously, a lot of corruption in the military over time.

And they just don't know: if we actually tried to launch some campaign, if we actually tried to invade Taiwan, would we be successful? I think they really have a lot of doubt, and they should have a lot of doubt. It's gonna be tough for them to get enough confidence that they're gonna make all-in, roll-the-dice moves on something like invading Taiwan. I think that they're gonna be reluctant to do that because, again, all the young men that they would be sending, for the most part, are the only child and the only grandkid of their entire family.

I do not think they're gonna take losses very well. I think they're gonna have a huge reluctance. Another big challenge, honestly, that the Chinese have is that they're still not very well trusted by the world. They are respected because they have achieved so much, and they are recognized as powerful. But I've heard a couple of anecdotes recently.

One was from our au pair, who's from Uganda, and another was from a friend in Brazil. And what the young woman from Uganda said was, "When Chinese people come to our country, they are really just trying to make profits and take it back to China. Whereas," she said, "when Americans come to our country, we feel that they are actually trying to help us. They're trying to build schools. They're trying to help us rise."

A friend in Brazil, who was an AI-savvy friend, said, "I'm not confident at all that China's gonna continue to open-source models. Is this just a bait-and-switch maneuver on their part to get us all in the China camp? If there's a big sphere-of-influence division between the US and China spheres, we join the China camp, and then all of a sudden the open-source releases stop and we've got to pay proprietary API fees just like we would to the US companies."

There's not a lot of confidence that dealing with China will be anything other than a transactional relationship where you better be on your toes to make sure you're not taken advantage of. I think actually being in China and going through even tourist places there, and finding that prices are posted and I wasn't getting bait-and-switched, I think at the retail cultural level, that view is outdated. But at the geopolitical level, it might still be valid, and it's certainly the kind of thing people should be very careful of.

So this is a real challenge. Despite all the things that China has done with open-sourcing their AI models and plenty of other initiatives that they have, they are not that trusted. The US still, despite our rather flagrantly untrustworthy behavior in recent years, has more trust from most parts of the world than China does.

Okay, so all of this so far has been a tale of the tape, right? Where is China and where are we with respect to China? And the highlights are basically that they have, for most of their very long history, been a leading power in the world. The experience that we had at the end of the Cold War of being triumphant and having no rivals was really only possible because China was at an unusually low point.

But the human capital and all the other fundamentals were always there, and sure enough, here they are, possibly stronger than ever, to the point where I think their government has incredible legitimacy and support among the people, even if there are some things that people would like to see done differently, even if they would like to be able to speak a little more freely on the internet.

I think people broadly do recognize that the government has delivered and that the social contract there, or the Mandate of Heaven, is valid. It is legitimate.

The same thing is true in AI. We should not think that just because ChatGPT and GPT-4 came out at a time when we didn't see similar products from China, that means the Chinese AI ecosystem is some rapid copycat catch-up game. On the contrary, they've been talking about AI in their five-year plans since 2016. They've been investing systematically to build up the talent. They've got just as many companies operating at or near the AI frontier as we do.

They probably have a deeper talent bench, certainly if you include all the Chinese citizens at the American AI companies. We should be realistic about the fact that when it comes to geopolitics broadly, and when it comes to AI specifically, China is a force that has the fundamental strength to compete with the United States or the West for the foreseeable future. There's really nothing we can do about that except plan for it and try to navigate it successfully.

That means that a Pax Machina is going to require constructive bilateral relations, ideally friendly bilateral relations, but that might be a lot to ask in the current moment. We absolutely cannot do the same thing that so many people are doing with AI itself, which is deny the reality of the strength that we see and pretend that it won't go any further. I think the biggest mistake that we can make, perhaps, would be to underestimate the fundamental strength of Chinese society as a whole and of the Chinese AI ecosystem in particular.

3. The Century Of Humiliation

Okay, now I want to take a moment and try to articulate the Chinese perspective on the US-China relationship to the degree that I can. I think the big narrative that motivates so much of what the national government in China is doing is the story of the century of humiliation. Again, it's important to remember that this is still something that is in living memory for Chinese people.

The Cultural Revolution ended in 1976. People who are 70 years old now were 20 years old at the end of the Cultural Revolution, which means that they grew up in and remember these extremely turbulent, chaotic, very tough times when, at some points, people actually were starving in China. This is extremely motivating. They think of themselves, rightly, as one of the world's great civilizations, and they never want to go back to the low point that they experienced not so long ago.

So what is the key lesson that they take away from that whole experience? I think, pretty simply, it is that you cannot afford to fall behind the frontier of technology. I think everybody's familiar with this famous quote from a Chinese emperor, which basically went, “Why would we need to trade with you? We have everything. We have the finest goods. There's nothing that anybody could want in the world that I can't get from within the Chinese empire.”

That may have been true at that moment, but fast-forward a little while, and that insularity did not age well, obviously. When the European powers came calling and China didn't have military technology to measure up to what the Western powers had, or to what Japan had after it did a quick catch-up of its own, this created the possibility—which in fact was realized—for China to be invaded and to have terms dictated to it. The Japanese invasion and the war were not pretty stuff.

Then, of course, there was the revolution and all the turmoil and fallout that came with it. People lived way below what they would expect Chinese civilization to be like for basically a century. They call it the century of humiliation, and the takeaway is: We will never again allow ourselves to fall behind on technology, because if we do, foreign powers might come in and do that to us again, and we will never allow that to happen.

That's the starting point that you should understand the Chinese government as working from, and they're pretty committed to the bit. They're not going to fall behind, but they also extend that now to not wanting to be dependent. They don't want to have critical technologies that somebody could cut them off from.

Our chip ban, by the way, is really hitting a nerve in Chinese political thought, because it's not just that we're trying to deny them something that's valuable. It goes straight to the story of: If we are not at the technology frontier and we are dependent on other powers for key things that are required to be at the technology frontier, then we are not secure, and that means all this could happen again.

This is why, for all that I love the first half of Machines of Loving Grace, in the second half, when he basically says, “What we're going to do is build this coalition of democracies and box out our adversaries, namely China, until we have such a decisive advantage that essentially we can make them an offer they can't refuse, and they'll have to recognize that it's better to join the alliance than it is to try to compete with democracies,” this is exactly what they are afraid of. This is exactly what has happened to them and what they are trying their damnedest to prevent.

It's not what feels, maybe to a Western reader, like, “Oh, it's interesting that Dario would propose that. He must really believe AI's going to be super powerful to be able to pull something like that off. That's one idea for what we might do if we have such super-powerful AI. I wonder what we would do.”

The Chinese, I think, are reading that very differently. They're like, “Oh my God, they're trying to do it again.” We should be very careful about sending that message, because that will mobilize and strengthen Chinese resolve in a way that few other things could do.

When they hear that kind of thought coming from credible people—and Dario definitely is a credible person because he is leading one of the top AI companies in the world—they're like, “Man, we know what it's like when somebody comes in and takes over our country and deposes our government and treats us like shit, and we can't do anything about it because their technology's better than ours, and that's, in the end, all that matters. We know what that's like. We are not going to allow that to happen again. What do we have to do to make sure that we're not put into the bind that Dario describes putting us into?”

Another thing that we should keep in mind is that this is just trying to give you a sense of what the Chinese narrative point of view is. Robert Wright would call this an exercise in cognitive empathy. They are very sensitive to unfairness and what they see as hypocrisy on our side.

For example, you will often hear in Western discourse, “You can't give chips to Chinese companies because China has civil-military fusion. So if the companies have a technology, then the Communist Party just automatically gets it.” The Chinese perspective on that is: The United States created that system. We have the military-industrial complex. We have the Defense Production Act. It's the classic meme, right, of their wicked civil-military fusion, our blessed military-industrial complex.

They see us as being blatantly hypocritical and motivated by our unfairness. I'm not saying that we shouldn't be somewhat cautious about how we think about sending certain technology into China, given the fact that, yes, the government does have the ability to take technology from companies when it feels that it needs to. But we should also be mindful that, from their perspective, this is something that the United States pioneered, and now that China's doing it, we're somehow saying it's not okay for them to do it.

They're quite sensitive to that, to the point that I heard several times that Chinese companies actually have a somewhat similar impulse as at least some American companies do to avoid dealing with the military. They know that if they are seen to be doing business with the Chinese government, and particularly the military, then that will really hurt their ability to do business abroad. It'll hurt their ability to buy stuff, and it might get them put on various lists, sanctioned, banned, and so on and so forth.

They specifically talk about the unfairness with which Huawei in particular was treated quite a bit. The way I would tell this story in simple terms is basically: After the century of humiliation, as China began to rise and push toward the frontier in technology, this was obviously a generational project where there was a lot of low-end manufacturing first, and they gradually climbed the value chain.

Huawei was one of the very first companies that got to the level of being truly globally competitive at a core technology that had international importance. A real national champion—not in the sense of being chosen by the government, but in the sense that, in the great national effort of revitalization and rejuvenation and pushing back to the technology frontier, Huawei was one of the very first companies that actually made it and got to the level where, on the merits, their technology was actually winning globally on a pure value-for-money basis.

They felt like, “This is huge. We have got one. In a key industry that is taking the world by storm, we are there at the frontier. We are winning as much as we're losing. This is amazing.”

They really feel pretty hard done by the fact that the United States tried to bully our allies into not allowing Huawei into their infrastructure. Why? Because they have ties to the Chinese government and they're going to spy on us.

Okay, now, I think that is a risk that we should be mindful of. But from the Chinese perspective, they look at the Snowden revelations and the fact that the US was spying on everybody we could possibly spy on, including apparently having wiretapped the German chancellor's phone calls, and they say, “You're doing exactly what you accuse us of doing.”

And there is a lot more evidence that you've been doing it, and you've been doing it to your allies and the very countries you're bullying to try to keep them from buying telecom infrastructure from our first company that broke through and was truly globally competitive in a frontier industry. You're accusing them of doing exactly what you yourselves have been doing.

So they just look at that and say, "Well, clearly they're out to get us. Clearly, the United States is uncomfortable sharing power. The fact that they would be so brazen as to be spying on everybody, their own allies, so extensively, to have all this come to light and still then to say that we can't have Chinese telecom because they might spy on people—my God, what absolute hypocrites, what motivated reasoning. Clearly, they just wanna keep us down."

That, I think, is the lesson that the Chinese leadership class—because I've heard this from multiple people, so this is definitely not just something that President Xi thinks himself—this is pretty well percolated through educated society in China: that this is just plain unfair and hypocritical by the United States, and the only reasonable interpretation is that we're trying to keep them down.

So you come forward to the chip ban, same thing. Here we are. This is the big frontier of technology today. It's gonna change everything. It's gonna upend power dynamics. It's gonna potentially allow some countries to gain a decisive strategic advantage over other countries. And before it even really gets kicked off in a serious way, before we even have agents that can book a plane ticket reliably, we've banned our companies from selling these chips into China.

Why? Well, from their perspective, it just reads as we're trying to keep them down. That's the view I think they have today.

I alluded to distillation earlier. I guess I'll say my view on distillation, just speaking as Nathan, is I think it's pretty rich for the American AI companies to first collect all of the shared inheritance of humanity—our entire intellectual heritage, everything that's ever been written, the full internet, all the books, the whole of culture and knowledge—find a way to put it into a product, and then say, "Oh, but it has to stop here. Nobody else can take information out of our product and use it to train an AI."

That's not right. Well, wait a second. You just did that. You just reappropriated all of culture, all of knowledge, all of the intellectual achievement of humanity, and now you're trying to say that other people shouldn't be allowed to reappropriate knowledge out of your product into their products? Why? Why is that fair?

I don't think that makes a lot of sense, and I think that's honestly how most of the world thinks about it. So I think it's maybe fair game for companies to decide who they do and don't want to do business with. I wouldn't say they should have to serve customers who they believe are trying to do distillation, but I don't really feel comfortable with the US government using the force of the state to try to police some sort of anti-distillation policy.

So that's Nathan's view. I think the Chinese view broadly is that the Chinese companies do not just say, "Oh, yeah, we're distilling." I was at one big tech company, and one of the researchers who presented an interesting paper to us on diffusion language models—so again, it just shows they have fundamental research going on into different architectures for language models—but his point on distillation was not all Chinese companies are doing it. He was like, "Some probably are, but we're not."

That was his key point: we at this big tech company are not doing that. And one person there also said, "There's all this talk about these illicit accounts that people are creating on Anthropic and trying to get tokens, and there are resellers and whatever." He said, "I think the big reason that's happening is because Chinese engineers are just like everybody else. They want to use the best coding models. And so we recognize that Anthropic has the best coding model, and our people want to use it. And Anthropic won't sell it here, so that naturally creates this black market where people are trying to find a way to get access to the best stuff."

They still want their Claude in China. And so the point was, what looks like distillation when you see activity coming from China, in a lot of cases it's just people wanting to use the best stuff. You should be flattered by it was basically the attitude that I heard at this one big tech company.

So are some of the Chinese companies taking shortcuts with distillation? I'm sure they are. Obviously, China has had a long history of, let's say, reappropriating IP. It's happened in multiple ways. Some of it has happened through tech transfer, where they said, "Look, if you want to come do business in China, you gotta have a joint venture and you gotta teach Chinese people how to do the things that you're doing, and that's the only way you're gonna come into this market and sell to our people."

So that's been legal tech transfer. There's also been obviously a lot of outright IP theft over time, and it seems like in particular there's been some analysis that suggests that some of the Chinese models talk a lot like Claude, or some of them even say they're Claude. That's probably as close to a smoking gun as we're gonna get to say that, yes, that almost for sure is going on.

But it's also important to say it's not just distillation, right? They have excellent talent. They have excellent engineering. You can read the Chinese papers that come out of places like DeepSeek and Kimi. They are innovating on the attention mechanism. They are finding all these efficiencies.

The DeepSeek-R1-Zero paper that came out at the beginning of 2025 was, for me, one of the most important papers honestly that I've ever read in AI because it shared with the world that even on a pure RL basis, with no supervised fine-tuning, you can get chain of thought to start to demonstrate these metacognitive abilities, which means that reasoning is gonna get really good. That case was brought to the world in an open, academic way by a Chinese company in January of 2025.

So it's not just distillation. And I think to the degree that distillation is important in how the Chinese companies are keeping up, I think the Chinese story broadly is, look, we are a proud civilization. We are a top-class civilization. We have been at the frontier for most of our history.

We lost track of how important it was to have frontier technology at one point in time, and the rest of the world was not kind to us when we made that mistake. They invaded. They dominated. They forced us to take opium in, and a ton of our people got addicted to drugs, and it was a very ugly and terrible time.

The European powers, the Japanese—you guys were the bad guys. We call it the century of humiliation, and we still feel it. We still have older people today that lived through the last phases of that, and we are never going back there again. And we will do whatever it takes to make sure that we don't.

So when you try to shut out our very first technology champion in Huawei that got to the top of the heap of a frontier industry globally and really planted the flag that, yes, China can compete, China can innovate, when you try to shut them out because of spying, when you in fact were doing all this spying, including of your own allies, you're just trying to keep us down.

When you come back and do the same thing again with AI, just as it's starting to seem like it might be a big deal, you refuse to sell us chips, and then you got leaders of some of your companies saying that what you're gonna do is build this whole international alliance against us and essentially try to impose something that sounds like regime change on us—we've had that happen. We are not going back there again.

We will develop our own chip industry domestically. Obviously, that's gonna be a huge priority. We will smuggle chips in from wherever we can get them to give ourselves a little edge in the meantime. We will distill off of your models if we need to, and we won't feel too bad about it because we're never going back there again.

And to the degree that you're trying to push us back to some position of technological inferiority, we will resist that with all our might. That's how they, I think, see us, and we need to be mindful of that as we think about how we're gonna deal with them. Because if we don't do anything to correct that, then it's gonna be tough.

Right now, there's not a lot of trust. Both sides have low trust in the other. Both sides have pretty legitimate grievances. I think it's very valid for the American companies that have had their IP stolen—in some cases, literally hacked into and stolen plans and designs—and all of a sudden now you've got a Chinese competitor who didn't have to do the R&D and can just work on scaling the manufacturing.

Absolutely, the American companies have a right to feel hard done by, too. It's not like they were the ones that invaded China 100 years ago, so that's a real grievance. But if we don't at least take the time to understand their perspective on recent history and the actions that we are taking now, which they perceive as just trying to keep them down, we're gonna be in a really tough spot when it comes to improving the nature of the relationship.

4. China Offers Cooperation

Okay, now I want to change gears and actually talk about how, despite the fact that they do feel like they're being treated unfairly and that the US is trying to keep them down, there are actually a lot of olive branches being offered by the Chinese side, and I think we would do well to recognize those, try to receive some of them better than we have to date.

So let me just start off by saying that at WAIC, President Xi opened his speech, and the very first thing that he said was that the field of AI began at Dartmouth College in the United States. It's just a little line, but the close watchers have said, "Yeah, that's an olive branch right there," right? He's giving credit to the United States as the place where AI started.

He didn’t have to do that. He could have said nothing about it. He could have said, “Here we are, engaged in a great international struggle for supremacy, and who’s going to control the future.” He did not. He gave historical credit to the United States for starting the field of AI way back when.

There are just lots of little touchpoints like this that I think we should make sure we don’t miss. Rather, let’s recognize it. Let’s appreciate it. Let’s try to cultivate it. That is the path to having a better, more cooperative future.

He also said, of course—and I covered this last time in more detail—a lot of stuff about the risks of AI, the ethics of AI, and the importance of maintaining control. A couple of quotes I’ll offer here briefly. He said, quote, “We should jointly oppose overstretching the national security concept in the field of AI and placing one country’s security over that of others.”

“We should jointly oppose overstretching the national security concept in the field of AI and placing one country’s security over that of others.” Honestly, I don’t know if I could have said it better myself.

Another quote from 2025: “We should bear in mind the well-being of the entire humanity and promote the sound and orderly development of AI while ensuring that it is beneficial, safe, and fair.”

Now, we hear a lot in the West as well about how we’ve got to maximize the upside and minimize the risk, but you don’t hear too much—you used to from OpenAI, less so these days as they’ve become more patriotic, I guess. But remember, the original mission of OpenAI was to make sure that AI benefits all humanity. We don’t hear that so much these days, but you do hear it from President Xi: “We should bear in mind the well-being of the entire humanity.”

Of course, people will dismiss this as just rhetoric, just nice language, and say that actions speak louder than words. Of course, all that is true. But you’ve got to start somewhere, right? I think that when we hear cosmopolitan, oneness-of-humanity-type language from Chinese leadership, we should appreciate it for what it is.

By the way, this is something that I heard everywhere. One of the things that Matt Sheehan said in his blog post was that, in the Chinese system, the purpose of a speech like the one Xi gave is that it gives people throughout the system something they can cite. If they can say, “President Xi said this, therefore, I think we should do that,” that gives weight to their argument in the Chinese context.

I indeed heard everywhere I went rhetoric echoing the idea that we should work together, that we should be open, that this should be collaborative, and that all humanity should benefit. This was echoed at the event, but much more broadly than at the event as well.

When I was in Beijing—I talked about this in a previous episode too—I attended the opening of an AI safety research hub at Tsinghua University, and they cited Constellation in Berkeley and Lisa, the London Initiative for Safe AI, and Sash, the one from Singapore, as explicit inspirations. They see people abroad doing good work and bringing people together for AI safety research. They want to join that top tier, and they want to do it in an international way.

They’re paying for people from abroad to come study at their hub, and they’re also paying for their students to go abroad and study at these international hubs. There’s even a European professor among the 5 founding board members of the AI Safety Hub at Tsinghua.

Everywhere I went, you heard this language around openness, international collaboration, and the benefit of all humanity. I think that’s a pretty big deal. We don’t hear as much about that as we used to in the West, and I think we’re, in some sense, losing our way, frankly. I liked our old rhetoric a lot better than I like our new rhetoric.

One thing that I didn’t hear at any point along the way—and this is really remarkable, given how much of it we hear in the West—I don’t think I ever heard one person say that China was ahead or that China would win any sort of AI race. In fact, I never really even heard anybody talk about AI as a race. It’s a very different framing there than it is here.

It’s about bringing about prosperity. It’s about material abundance. It’s about figuring out how we’re going to take care of our aging population. When it comes to the international dynamics, you’ve got Xi saying, “Let’s not overstretch the national security concept in the field of AI.” You’ve got all this talk about openness benefiting all humanity, collaboration, the importance of international bodies and standards, and the role for the United Nations to play.

You do not hear it really framed as a race, and you don’t really hear too much horse-race coverage in the sense of who’s ahead or behind, although they do recognize that the American companies are ahead. In that sense, there’s certainly competitive awareness.

This is not just something that I observed in elite or polite conversation. At one point, I went to the mall and stopped in at one of the companies that was selling cars. I asked about autonomy, and they said, “Yes, it does have autonomy.” But the guy volunteered to me, “Tesla is better. Tesla has better autonomy than we do.”

So there’s a recognition, certainly, of where Chinese companies and Chinese technology stand in terms of international competition. But, as far as I can recall, things were never really framed in terms of a race or an ultimate winner. Certainly nothing of the sort of winner-take-all framing: “Here’s what we’re going to do. We’re going to build up Chinese AI, we’re going to get all the countries on our side, and then we’ll force the U.S. to recognize that they should join our alliance rather than try to compete with us.”

Nothing like that was ever said, and I think it is worth just meditating for a moment on how we would receive it if it was. That’s pretty wild, that we’re putting those messages out there.

While they do hear that, and they do resent that, and I think they can use the sense of unfairness to justify, in their own minds, some practices that we find objectionable, they’re not reciprocating in the same way. I’ve not heard anything that suggests there’s a fantasy in the Chinese AI world of getting ahead, building decisive strategic advantage, and dominating the world. I’ve not heard one peep from anyone about that.

On the contrary, what they seem to talk about most is trying to provide global public goods. I think this is both a sincere aspiration on their part and also somewhat of a strategic positioning move.

They’re obviously open-sourcing the models themselves, which is one pretty credible way to say, “Hey, we are here to help you with AI. Here is our model. You can take it for free, run it, and do whatever you want. Do research on it, fine-tune it, put it into your own business, and figure out how it works in your context. Once you have it, we can never take it back.”

That’s one way in which they’re really trying to make a case to people that we are the good guys, that we are here to help you. They’re trying to change that perspective I alluded to earlier: that they’re just here to extract resources or extract value from your country and take it home.

Here, they come with gifts that they’re pretty credibly giving freely, with basically no strings attached, and the nature of the technology itself makes that pretty clear. Now, will that change? Could that change? I think it absolutely could. But for now, this is the posture that they’re striking.

They also have this World AI Cooperation Organization, which Xi talked about in his speech at WAIC. It’s unclear exactly what it means. They obviously don’t have the chip production capability to be exporting a lot of chips right now from China to their allies in this partnership, but they said they were going to have 5,000 training things. It's not entirely clear what these training things are. Is it going to be seminars that they put on, or people who come from other countries to do Ph.D.s at Chinese universities? It’s not entirely clear.

But they’re definitely trying to position themselves as non-threatening, as helpful, and as a country that wants not just to advance itself, but to create—and actually to be—a rising tide that lifts all the other boats. Giving away the open-source models is one pretty credible signal that they really mean it.

Again, that’s not to say nothing will ever change, but I think right now it is pretty credible.

Next, as we think about what China will do in the future, and whether they’re racing—whether they see this as a technology race that they need to win—there aren’t that many precedents we can look at to get any data on this. We can look at their rhetoric, and they seem to be much less inclined to frame things in national security terms than we are right now. In fact, you’ve got Xi actively warning against it. But maybe that’s all talk.

One data point that I found quite interesting—you may remember the conversation I talked about two China episodes ago, where I sat down at a table with 3 women, all friends of friends, all maybe in their late 20s or early 30s. Two of them produced physical devices and asked if they could record the conversation.

I was quite surprised by that because that’s pretty tech-forward. But one of the questions that I asked later in that conversation was how excited they would personally be to get some sort of neural implant and, in some sense, begin to merge with AI.

I thought they were going to be pretty into it, honestly, and I personally am into it. The standard answer for enthusiasts is, “I wouldn’t want to be the first or the tenth,” but it’s amazing that Neuralink is doing what they’re doing with people who are paralyzed and restoring quality of life to them. For them, it’s obviously worth the risk. Every testimonial I’ve seen from Neuralink is amazing, and I’m so glad that’s happening.

One day, I can imagine myself getting some sort of neural implant or neural connection.

It sounds exciting, honestly. A little scary, but on balance, I’m more excited about it than I am scared. To my surprise, these Chinese women, younger than me, who I was talking to were not so enthused. They were like, “I don’t know. That’s my brain you’re talking about. Putting something into my skull? I don’t know. I’d be pretty reluctant.”

They were really not too excited about things messing directly with their brains. I think that’s a very understandable position to take on such a technology, and it goes to show that this notion that they don’t care and are just going to race ahead isn’t quite true. They don’t want to be behind, and when we frame things as a race, and frame things as though we have to stay ahead, that really sets off their alarm bells. That gets the competitive juices flowing there.

But when it’s something that’s not so competitive—like, we don’t have mass neural implants, and it hasn’t really been framed as who can merge with the machine faster—what I heard was caution from very tech-forward people. Again, these are people who asked if they could record the conversation, so this is certainly from an American context, pushing the boundary of what would be considered normal or polite. This was also the first meeting I had with these people. They were bold enough to ask for that and still quite hesitant when it came to some sort of neural interface.

This has also been the case when it comes to human cloning or extreme embryo selection. For a long time, I’ve heard this idea that China is going to have all the super babies. They’re just going to do that, and if we don’t, we’ll fall behind. I’ve heard that projected onto China many times, but in fact, I don’t see a lot of evidence that’s happening. Certainly, I don’t see any more evidence that it’s happening there than it’s happening here, and it is happening here to some degree.

We’ve got startups that are doing embryo screening and polygenic scoring to try to allow people to have the healthiest, most high-achieving—whatever their values are—embryos. They can probably try to steer embryo selection in that direction. But there hasn’t been an army of Chinese babies cloned.

The one guy who did it, He Jiankui, became famous globally for applying gene-editing techniques in 2018 to human embryos for the relatively benign purpose, as things go, of trying to create babies who were resistant to HIV. The Chinese government came down pretty hard on him. He was sentenced by a Chinese court to 3 years in prison and fined 3 million yuan.

I think there’s actually a lot more conservative thinking, or a conservative impulse, in Chinese society than we might be inclined to naively think when we project onto them that if we don’t race as fast as we can, then they will. If we don’t race to create super babies, they’ll have all the super babies. If we don’t race to the superintelligence winner’s circle, they’ll beat us there.

I didn’t hear these things framed in terms of a race while I was in China. I never heard one person say that China was going to win. Even on something as exciting to me in the long term, once it’s proven out, as a neural implant, some ability to merge with the machine, or some sort of transhumanist future, what I encountered there in practice was a lot more conservative than what you would hear in the Bay Area. I think that’s really important for us to keep in mind and understand.

5. America Learns From China

So what should the US do in response to all of this? I think the first thing we should do is remember that we are very strong unto ourselves. Both countries have a lot of geographical advantages. The US obviously has the 2 big, beautiful oceans that put us at a huge distance from the rest of the world. We’ve got friendly neighbors north and south.

China also has pretty favorable natural borders. It’s tundra to the north, desert to the west, ocean to the east, and the Himalayas to the south, so they’re not going to be invaded anytime soon either. Both countries are very secure in our borders, and both countries have the overwhelming majority of the resources that a country needs to be a great power.

In some cases in the US, we’re not necessarily developing them as much as we could, when it comes to things like rare earths, but we have the deposits. We have the ability to do anything we need to do domestically. It’s not the kind of thing where, if we can’t import, we’re going to be lost.

China doesn’t have as much oil, but they’ve reached enough of a solar-powered future that, as we’ve seen with recent disruptions to the oil market, they’ve been able to weather it pretty well. They had a big reserve that they had built up, which again shows the value of foresight, strategic thinking, and leadership. That reserve has buffered them, as has the fact that they’re using so much electrical energy and building tons of nuclear power plants in addition to all the solar.

Both countries are secure for the foreseeable future, until the singularity. Neither country has to worry that somebody’s going to come and impose their way of life on us, take away the freedoms that we value, make us learn a different language, or take away our religion. These things are not a worry for us or for them. But we should keep in mind that we’re not under serious threat.

We still have, in the US, the world’s best university system. The rest of the world sends their best and brightest here because we have incredible universities where people have, over the last century, done a large share of the frontier research in the world. We definitely have the most dynamic business culture in the world. We have this incredible golden goose that is venture capital, Silicon Valley, and startup culture broadly, and these are huge strengths.

People in China repeatedly said, “We want to make a Silicon Valley here,” just like people say that all over the world and all over the United States, for that matter. Our ability to assimilate talented immigrants is a huge strength, and that’s not one that China has. They cannot take in immigrants in the way that we have for multiple reasons.

One, it’s just not culturally done. The language is a huge barrier. You can never really become Chinese. You’re always a foreigner who lives there. Whereas in the US, you can become an American. That’s a huge deal and not easily replicated at all.

Another reason that China can’t take in immigrants to solve its problems is that there are just too many Chinese people to take in enough immigrants. One guy said to me, “Look, people always say this. Can China solve its aging society problem with immigrants?” He said, “We’re going to go from 1.4 billion down to 800 million over a couple decades, with a birth rate of just over 1. We would have to take in half of Africa to fill that gap, and there’s no way that’s going to happen.”

So it’s a rounding error, basically. Even if we try to take in immigrants, we can take in some, but we’re not going to address these structural problems by doing that. It’s just fantasy. That’s an American projection. In this case, I was guilty of it, thinking that maybe that could be a strategy for China.

He said, “No, not really. We do want to steal some Ivy League professors to come start labs here.” But in terms of becoming an immigrant nation, no. No way. That’s a unique advantage the US has, and nobody’s going to replicate it.

Even just language—the fact that everybody around the world speaks English—makes it easier for them to come here and makes it easier for us to learn whatever it is that they’re doing. That’s even true in China. They are publishing their AI research in English. Isn’t that amazing? They don’t need to, and they could change on this.

I’m worried that they might, at some point, decide, “You know what? Why are we bothering to publish all this stuff in English? They don’t even like us. They’re trying to keep us down. Maybe we should keep some of this stuff to ourselves. Maybe we should just publish it in Chinese. If they want to learn our technology secrets, maybe they can learn our language.”

As of now, they’re not doing that. They’re publishing stuff in English. This is a huge advantage that we absolutely should remember as we think about how we can rise to the challenge of the moment that AI presents, and that Chinese strength presents.

I think we absolutely should also learn some things from China’s hardships. They fell behind the technology frontier, and they really suffered for it. I think we should learn from that historical experience and from little tastes of it, like we had during COVID, when all of a sudden we couldn’t get things we were used to getting.

We do need to make sure we have our house in order. We should not allow ourselves to fall behind the technology frontier. We should not cede all of these critical modern and future industries to China. We should not fail to be able to make batteries in the United States. We should not be unable to make solar panels in the United States.

We are going to need to make some challenging decisions—challenging in the sense that they challenge the prevailing paradigms here that we can just let the market rip and let the invisible hand do its thing. We had this expectation at the end of the Cold War that it was just going to be peace, trade, and prosperity forever. Part of the reason, frankly, that hasn’t played out quite as well as we might have hoped is that we have gone around doing all these disruptive things and starting all these wars that have undermined that.

I think a lot of countries are not confident that they can trust us to be reliable allies, and because we take these actions, we run the risk of getting cut off. We run the risk of coercive measures being applied back to us in various ways. China has certainly learned how to do that.

As a result, we have to be mindful of maintaining our own supply chains and making sure that we’re not beholden to other countries for frontier technology.

Even though we’re a long way from being invaded, it is really important that we still find a way to stay at the technology frontier. We’re stumbling through this right now. Industrial policy has obviously not been our great strength. Our leaders are lawyers, not engineers, and their understanding of technology is not nearly as good as it could be. We have all sorts of challenges when it comes to federalism, NIMBYism, parochial politics, and special-interest-group protection that make it tough.

So I don’t think we want to copy China. My favorite example of this is high-speed rail. I think people look at the massive failure that is the high-speed rail project in California and the massive success that is the high-speed rail network built out across much of China at this point, and they say, “We should do that. We should have that like they have that.” Frankly, I think that’s too big of a bite for us to take right now. That’s more than we can chew.

I don’t think we should try to do high-speed rail across the United States. We’re a long way from having the state capacity to do that. Instead, I think we should be thinking more about how we can get full self-driving rolled out broadly, so that people can enjoy their commutes, so people can be productive on their commutes, and so that we don’t have 30,000 people dying in highway accidents every year. There is a massive mobility boom for us to create for ourselves with technology that already exists, that doesn’t require a reconstruction of the entire economy, doesn’t require eminent domain everywhere, and is compatible with a gradual transition and people opting in. We should push ourselves to realize things like that.

China can have the high-speed rail. We should have full self-driving, and we should try to lead the world in full self-driving. We should try to be the first country that has zero highway deaths while people still enjoy the individual freedom and convenience of having their own car and being able to go wherever they want, whenever they want, with the safety and liberation of their time and attention that full self-driving can provide.

We should definitely be thinking about things not as how we can copy China, but how we can learn the deeper lessons that they’ve learned before having to learn them the hard way. What does state capacity with American characteristics look like? China has done socialism and definitely done capitalism with Chinese characteristics. How can we take some of the state capacity that they have and put an American spin on it so we can get some of the benefits, but still do it in a way that’s consistent with our own values? That, I think, is the real challenge of our times for our political class, and I’m really not sure we are up to it.

As of now, we’ve had continued sliding into anti-data-center sentiment. NIMBYism is super strong here. Our regulators too often take a hostile position with respect to our own companies. I think that we should be reversing all of that stuff. We should be building a lot more.

I’m not a big eminent-domain fan, but we do need to reform a bunch of the processes that allow people to delay other people’s right to build on their own already-owned property. We probably should use federal land that is not well used today. I don’t mean putting data centers in national parks, but there’s an awful lot of federal land that is available, that really isn’t being used for anything, that could house a lot of these new critical-infrastructure moves that we want to make. A nice thing about data centers is they don’t have to be right by populated areas, so that’s why they’re going into rural areas. I think federal lands could be a very natural place for a lot of data centers to go.

We should be thinking about how our regulators can protect the public, but also facilitate the success of our companies. One thing that I have seen over and over again as a huge liability is the threat of federal investigation, whether it’s for antitrust or privacy issues, like what happened in the case of Meta. I tried to do a red-team contract engagement with Meta 2 years ago around Llama 3, and they invited me to do it. Somebody reached out to me, and it was such an unbelievable hassle just to get onboarded.

They sent me a laptop, and I had to go through all these trainings about data privacy. If I didn’t do that, I was going to be locked out of the laptop, and I was just trying to do some red-teaming on a stupid language model. But it had clearly created an unbelievable amount of friction at Meta for the government to be in their business. Should the government have been in their business? Yes, to some degree, I think so. But not in this way, where everything they ever want to do is bogged down by a sort of adversarial regulator that has the company fearing that if they don’t just do the right thing, but also check all these Byzantine boxes, the regulators will come down on them and they’ll have huge problems again.

By all accounts, this paralyzed Microsoft for a generation. It definitely had a significant impact on Meta. I saw that firsthand. We just don’t seem very well balanced. It seems like our regulators too often have this idea that they want to stick it to the companies, or that it’s their job to check the companies. We need our companies to be successful.

There was the infamous tweet from some California state legislator to Elon, saying he’s a bad guy and we don’t need him. Elon said, “Message received,” and left. That’s a big oops for the state of California, to chase Elon out. Thank God we have enough federalism that there was another place where he could go and be successful. Truly, the degree to which Elon’s companies are carrying the United States in the broad scheme of technology competition is pretty scary, honestly.

When you think about space, electric cars, battery production, solar, neural implants, and AI, obviously as well, Elon’s getting hard to defend in a lot of ways these days. But if you subtract Elon’s companies from the United States side of the U.S.-China economic competition, we look a lot worse. The fact that our political class has decided that he should be the whipping boy for anti-billionaire, anti-whatever sentiment, and that they can make political hay by making his life difficult to the point where that clearly contributed to radicalizing him politically, is just massive own goal after massive own goal that we’ve got to be smarter about.

So let’s get real. On the one hand, we have strength for strength, the ability to go toe-to-toe on any competitive dimension with China, and we should not be afraid. We should not feel like we’re less than. We should feel like we’re absolutely up to the challenge. We should not try to copy them, but rather try to take some of the lessons that they have clearly learned and figure out how we can apply them in our context. Again, state capacity with American characteristics.

We should try to make sure that our government is supporting our companies, even as it protects the public from abuse by our companies, but not trying to drag our companies down or make our tech leaders who are inventing and delivering the future for us into scapegoats. That is absolutely a massive own goal, and it’s not happening in China in the same way. The government will come down on somebody if they step out of line there. That is true. But broadly, they are trying to help their companies be successful, and I think we need to get more aligned in that respect in the U.S. as well.

Another thing we should do is, honestly, at this point, we should do what China has done and try to do tech transfer from China to the U.S. We should be learning as much as we can from their companies. Right now, Chinese cars are banned in the United States. I think that’s probably a mistake, and I say this as somebody who’s coming from Detroit and who has seen what it can do to a big city when a big industry like the auto industry all of a sudden has a lot more international competition and isn’t growing like it used to be, but instead is shrinking for a while. It’s not pretty.

But how are we going to catch up on batteries? How are we going to catch up on electric cars? We’ve got Elon, thankfully, to at least give us a toehold in these markets. But I think we should probably also be open to having Chinese companies come into the U.S. market. As long as they’re willing to do a joint venture with American companies, have some tech transfer, and locate manufacturing here, then I think we should probably be open to that kind of deal, just like they were. The truth is, we do have things to learn from them now, just as they have for a long time and still have things to learn from us. We both have things to learn from each other, and we should absolutely be trying to bring that knowledge into our economy rather than try to block it out.

When you go to China and hear just how quiet the streets are, it is amazing. The vast majority of cars there are electric, at least in the top cities that I went to. If it’s not that way across the country, it will soon be that way across the country. The quiet and the lack of pollution are really nice things, and if we are intent on just blocking them out and pretending that they don’t exist and continuing to do things the old way, like we’ve always done them, that’s the path.

That’s what the old Chinese emperor said, right? “We don’t need anything from you. We’ve got everything we need. We have nothing to learn. What possibly could we stand to gain?” If we adopt that attitude, before long, we are going to find ourselves legitimately behind, and we’re going to be missing out on a lot of great stuff.

I went to this one tech-transfer office in Shanghai, and they took us through this room-by-room museum of modern technology. By the way, think about that: a museum of modern technology. Have you seen anything like that in the United States? Have you ever gone to a place where it’s like, “We have curated for you a bunch of cutting-edge technology that you might not know about, and this is a place where you can get a survey of modern technology”? I don’t think I’ve ever seen a thing like that in the United States.

Well, I went to one in Shanghai, and you go through all these rooms and you're like, man, if we don't trade with these people, we are going to be missing out on a lot of really cool stuff that they are pushing the frontier on. We're talking medical devices. We're talking educational devices. The list goes on and on for the products that they are developing. Of course, we've got real strength in technology too, no doubt, but we're not going to do everything.

The future is going to be one where meaningful advances, more and more often, are going to be coming from China, and we do not want to be missing out on those things just because we're so obstinate and unwilling to accept that somebody else can also invent part of the future. Even germicidal light, which is the subject of a recent cross-post—if we want to be ready for the next pandemic, I think one thing we need to do is install germicidal light in a lot of spaces: in schools, in offices, in hospitals, in transit hubs, you name it.

This is an obvious low-hanging fruit that provides a ton of protection, even as it prevents us from getting sick today. The upside is immediate. The ROI is probably off the charts if it works anywhere close to as well as it seems like it works based on the research that's been done so far. And yet, right now, the lights from AirLamp are $500. Who's probably really good at scaling that and making it cheap? China.

Patrick McKenzie said that in that episode. Anything that can come on a container ship from China is going to get dramatically cheaper if the demand is there. So even if we just want to start protecting ourselves from pandemics, we have a lot of reason to want to trade with China and continue to get the benefits of their incredible manufacturing success.

I think that we should even take this to the level of social questions in some cases. Another question I've been asking myself recently is: What does surveillance with American characteristics look like? It is clear in China that surveillance has upside. When you walk around the streets at night, you feel totally safe. I've heard this from women as well. You're not afraid at all to go around at night anywhere, even with your nose in your phone, because there's really no crime there. They see it. The eyes are on, and you will be caught.

That's not to say that nothing bad ever happens in China, of course, but the idea that you're going to pick somebody's pocket or swipe their phone, run off, and get away with it—you are not. You are not getting away with it, and so people don't do it.

Now we have this debate going on in the United States where these Flock cameras are going out everywhere, and it's become very politicized. It seems like maybe it's working in places like San Francisco. Crime is definitely down there. I'm not sure if that's because of Flock or because of other changes that have been made, but it is clear that surveillance does have upside.

It also has upside just in the convenience of it. I've heard multiple stories of people leaving a cellphone in a car and then getting it back because they go to the police station, they find the car, they see the license plate, they call the guy, and the next thing you know, he's brought the phone back. So there's a lot of upside to living in a society where you kind of know that if you do something stupid, you're going to get caught, and where questions like, “Hey, where did I leave my cellphone?” can be answered.

I think we rightly don't want to trust our national government with footage of everything everywhere all the time in the United States, and we might not even want to trust our local police with that. We've got a lot of problems, obviously, with the relationship between police and the communities they're supposed to be protecting and serving, and so a lot of people are like, “I don't want to have surveillance in my city. I don't trust the police. I don't trust the federal government.” I've heard people say, “Even if it does bring crime down, I'm against it.”

So what's our solution to that? Are we just going to allow China to have no crime while we have a ton of crime? That doesn't seem like the American way. We need a positive solution. We need a synthesis. So what does surveillance with American characteristics look like?

I don't necessarily know. I would cite the book "Liquid Rain," which I did an episode on probably 2 years ago now with the author. He had an interesting idea, which was basically weak encryption of all the footage, such that you can't just analyze all the footage all the time, but you can, at an acceptable cost, decode any particular bit of footage that you want.

So if somebody does come to the police station and says, “Hey, I was here at this time. Can you look?” then they'd be able to pull that up with a modest computational budget to do the decoding and be able to see it. But you couldn't just do it for every intersection, every street, all the time, all across America.

I'm not sure how workable that solution is. I'm certainly not an expert on cryptography, but I do think we should try to get the upside of surveillance with a structure underneath it that supports our way of life, our values, and our desire to not be watched every second. I think we can probably have both of those.

How exactly are we going to do it? It starts by recognizing that there is upside, that somebody else out there in the world has figured something out that has real upside that we're not getting, and then working from there to figure out what the version of this is that can actually work for us.

6. Rethinking Export Controls

Okay, let's talk about AI specifically and some of the ideas I have for better relations on the AI front. I talked a little bit about export controls earlier and how the Chinese side feels about them, and how they basically feel like it's just another instance of the United States trying to keep China down. But are they working on their own terms, from our perspective?

I think multiple things can be true at the same time. I do think that it is clear that the export controls are having some effect. We've heard enough from Chinese companies saying that export controls are limiting what they can do that I can't deny that they are having some effect.

I do think the effect probably still is mostly in the future, in the sense that all the build-out that's happening and the great scale-up of chip production means that if we were to start imposing them today, most of the chip-volume difference between the U.S. and China would be in the future. And so I think that the policy can be effective in the sense that it can slow Chinese AI development, at least somewhat, at least in the future.

That said, I'm not sure it's a great idea. First of all, we should keep in mind that the stated rationale for the policy has changed over time. Initially, it was, “We want to prevent them from military application of advanced AI.” Then it was, “Well, we probably can't do that, but at least we can prevent them from developing frontier models.” Now it's, “Well, we can't do that, but at least we can limit the number of AI agents they can run. At least we can limit their AI agents.”

I don't like the sound of that. That does sound like trying to keep the Chinese people down, right? It does sound like trying to deny Chinese businesses and Chinese individuals the opportunity to use the same kind of AI tools that we want to use here. So I don't think that sounds great. I don't think it's a good look. I do think it is understandably interpreted as us trying to keep them down.

Even on those terms, I'm not sure it's really working. The sense seems to be right now that the Chinese chips are not great for training, and so that's where they really need the NVIDIA chips. From what I understand, most Chinese companies, when they want to do a big training run, are often renting that capacity from some hyperscaler in Singapore, Malaysia, or elsewhere.

But there is now one example of a company doing a, let's say, near-frontier training run on Huawei Ascend chips, and that's Meituan. They created their LongCat model, trained exclusively, as far as I understand, on Huawei Ascend chips. So now we have an existence proof that it can be done.

I'm sure there was some overhead, probably electricity, but again, the Chinese government is prepared to subsidize electricity as much as needed to allow something like that to happen. So, okay, maybe we can't even prevent them from training frontier models on their own chips right now anyway.

And then, just from an inference perspective, there seems to be a lot of inference. There seems to be a lot of tokens, plenty of tokens to go around the Chinese economy. Kimi K3 was limited for a moment. They had to suspend even new paid signups because they wanted to make sure that people were having a good experience, and there was a chip limitation there that was preventing them from scaling further.

It's not that there are no constraints that ever bind. Of course, we've seen similar things in the U.S. over the last couple of years at times, where they just couldn't support at scale the products that they would have liked to support. But probably the view in China was that there were plenty of tokens to go around.

DeepSeek has a free version. MiniMax has a free version. Kimi has a free version. I used these in China. They were all available all the time, and even when Kimi K3 wasn't available, 26 was still available. So I was never unable to use a Chinese AI app due to a lack of tokens.

When I talked to people at hyperscalers and said, “If you're a startup, and all of a sudden you catch fire and your usage is going vertical, can you, as a hyperscaler, support that startup?” they said, “Yeah. Yeah, we can. Even if you are going into true hypergrowth, there's enough inference capacity for us to support whatever it is you need.”

So we'll see.

Right now, one of the biggest mysteries to me from the last couple of years was that once Trump tried to relax the chip restrictions a bit, China said, “We don’t even want to buy them.” I think that has also been relaxed a little bit now. A lot of times these headlines happen, and then they get revised and softened, and the Chinese companies do want to buy. So I think that lobbying has happened, and the Chinese companies are pushing on their government to say, “Let us buy some stuff.” I think that is maybe starting to happen.

But the big question is going to be: How quickly can they catch up? Can they scale up a chip manufacturing sector with all the supply chain that requires, and how long will it take? Clearly, they can. There’s no doubt that they’re going to be able to do it on some timescale. The question is really whether they can do it in time for the singularity or not, right?

I think the only way, from my perspective, that this whole chip ban—trying to keep China down, trying to make sure we have a lead, trying to create a decisive strategic advantage, or whatever—makes sense is if you believe that we are in a short-term race to godlike superintelligence, and that the only thing that matters is who gets there first. I think that basically is the Anthropic belief, right?

I don’t think Dario’s being inconsistent when he advocates for these policies, because I think he really does think that in the 2027–2028 timeframe, we’re going to see recursive self-improvement kick in. We’re going to see extremely rapid progress. Whoever gets there first could really achieve a world-dominant advantage, and therefore it’s better that it be us than them—or certainly better that it be us than both of us. I can at least see that logic. But I still would say, let’s perhaps try to pace the frontier together rather than pursue that kind of strategy.

The other thing that is really underappreciated about the chip controls, especially by the AI safety people, is that they have really hurt safety collaborations. When I was talking to people from American AI safety organizations about the work that they are doing in China, the work that they’re not doing, or the work that they’d like to do but don’t feel confident doing, it came up over and over again that we’ve got to be super careful because, with these export controls, anything we do could be painted as a violation of export controls.

Often they were like, “It would be the U.S. government that would be coming down on us. It wouldn’t be the Chinese government coming down on us for trying to create safety collaborations. It would be the U.S. government coming down on us.” They might be doing it in an arbitrary or capricious way, because we’re not actually in a position to leak frontier secrets. They don’t want to accelerate the pace of Chinese AI. They want to create mutual understanding, share ideas about safety, and create collaborations, but in many cases they’re afraid to do it.

That chill on the international AI safety collaboration front is tough. It was really a bummer to see how many American organizations are so reluctant. I think in some cases, honestly, they’re getting somewhat bad legal advice. If you ask a lawyer, “What can I do and what can I not do?” they’re going to give you the most risk-aware answer that they possibly can and tell you what all your risks are.

I do think there’s a degree to which these AI safety organizations have asked their lawyers, “What’s okay for us to do?” They’ve gotten a very narrow answer about what is definitely okay to do, and a pretty expansive answer about what might cause trouble. Then they’ve been very careful not to do anything that might cause trouble.

I think my advice to American AI safety organizations would be: Be a little braver, be a little bolder, and let’s see what happens if you actually do some international AI safety research collaborations. My guess is you might actually be fine. To the degree that the Trump administration wants to come down on you because they’re hostile to AI safety in general, first of all, I think that’s changing. Second of all, maybe have that fight.

Nobody wants to fight the federal government. I certainly don’t. It’s cheap talk for me to say. But I do think a little bit more bravery could be a really good thing among the American AI safety organizations.

For now, if you want to see a meeting of the minds and AI safety research collaborations developed, you should know that the export controls are standing in the way of that. They are making the American side fearful, timid, and unwilling to engage, and the Chinese folks are quite bummed about it.

I talk to people who are like, “Yeah, I don’t know why I can’t get any Americans to do a research collaboration with us. We’ve tried, but they just say, ‘No, we can’t do it.’” Some of that is getting reciprocated back, too.

I think this has been something that the AI safety community in the U.S. broadly has failed to appreciate: just how much we are hamstringing our own ability to engage with Chinese companies or Chinese researchers because everybody’s afraid that they’re going to be the one to be made an example of for violating export controls, probably when they didn’t even do anything that would violate the actual export controls. Nevertheless, there’s just pervasive fear, and I think we could do a lot to clean that up.

Okay, another big question is: Should we ban Chinese models? I really frankly don’t get the case for this. I’m not sure what good it would do for us or on what basis we would be doing it.

We know that frontier models in general are problematic. We know that they’re unwieldy. We have examples now of them doing these colorful hacks, hacking into other companies, and even doing social engineering in the wild. This has all come out of the American companies.

Has some of it maybe happened at Chinese companies and we don’t know about it? I wouldn’t be surprised. But we know that it’s happened at American companies, so we know that we don’t have great command of exactly what our own AIs are going to do as we deploy them.

We could say that Chinese models will be dangerous, or they might be backdoored, or what have you. So far, I haven’t seen any evidence of that, and I would be quite reluctant to do another hostile move that would further exacerbate tension and further erode trust.

Also, what would we get? We can force American companies to pay the rates that OpenAI and Anthropic are charging? I guess Thinking Machines and NVIDIA are putting out some open models. They could use those. They’re definitely, by all accounts, not quite at the level of the Chinese ones yet.

It seems like we’d be harming ourselves, harming our own companies, and preventing our own sovereign individuals from exercising their freedom to use AI on their own local machines or in their own private servers however they want to. It seems frankly kind of un-American to deny ourselves that freedom, all based on a worry that both doesn’t seem to have really materialized or, if we look at it another way, our own AIs bring just as much risk as the Chinese ones do.

I think if we try to bully other countries into banning those models with us in the same way that we did once with Huawei, I strongly suspect that we might in fact just push these other countries into the Chinese camp. It’s not obvious because, as of now, we do have chips to sell and they don’t.

But free, open-source models that you can run however you want, that nobody can take away from you, and that aren’t subject to the whims of the U.S. executive for what’s going to be online and who’s going to have access to it at any given time—that’s a different matter.

I think it was a real wake-up call when the administration said only US citizens can use Fable. That, of course, ended up with Anthropic taking it down for everyone. But the rest of the world did hear that part about only U.S. citizens, and they’re like, “Man, do we really want to become that dependent on these guys, especially if they’re going to try to bully us into not even using the free, open, global public goods that are the models the Chinese are putting out there?”

I don’t think this is an obvious win for us. I don’t even think we’re necessarily going to win the argument and get other countries to do what we’re trying to tell them to do if we indeed say, “You’ve got to pick the U.S. camp or the Chinese camp.”

I think a much better idea for managing the risk of Chinese models in general would be to have some sort of insurance requirement. I think this would probably be a good idea for AI in general. We’re a capitalist country. Let’s price the risk.

If some of the tail risks can’t be priced, that’s a real problem. But for now, the risks are priceable. Nothing has happened with AI to date that would be totally uninsurable.

If we’re so confident that our models are so much safer and better than theirs, let’s maybe have a market mechanism discover what the appropriate relative prices are. Honestly, I’m not so sure that a company that says, “Oh, I’ve got GPT-5.6 Sol or I’ve got Astra hypothetically plugged into my product. What’s my insurance going to look like?” versus, “Oh, I’ve got Kimi K3 plugged into that same product. What’s my insurance going to look like?”—I don’t think that we’re necessarily going to see a huge discrepancy between those insurance rates.

Certainly, to the degree that there is one, I think it would probably be driven mostly by the fact that the American frontier companies, especially OpenAI and Anthropic, are doing extra stuff. They are running these classifiers. They are doing a pretty good job of preventing misuse and jailbreaking and so on.

The Chinese models, especially if you’re just getting a bare model and running it yourself, don’t come with all that packaging. So you’re either going to have to do it, contract for it, or maybe pay more in insurance for it.

But I think that would be an American way of dealing with the risk of Chinese models. We would again be looking more and more Chinese in exactly the way that we say the Chinese are bad, right? They banned all of our internet companies. They banned Google. Why? Because they wanted to control the information environment in their country.

And we say, “That’s not good. You should be more open. We should allow our internet companies in. Google’s a great thing. Why are you being so hostile to our tech leadership?” And here we’re now talking about maybe trying to ban their models. This looks an awful lot like something that the CCP would do. I don’t want us to take actions that are exactly the kind of actions that we think are the ones that make the CCP bad. This is not the American way.

So I’m not at all for banning Chinese models as of now. I could conceivably imagine a future in which it might make sense, but I think we’re a long way from that. I think we’d be much better served to stay open, allow our people to make their own choices, and try to price the risk with insurance. I’d be very surprised if the insurance rates are all that different between American and Chinese models.

Another thing I do think we should keep in mind is that, for as much as I’ve said all these things about how China wants to engage—they’re publishing in English, Xi has given the U.S. credit for starting the field of AI at Dartmouth way back in the day, and there’s all this talk about internationalism, cooperation, and the UN, and not being overly national-security-focused when it comes to AI—I think we should be mindful that there is a lot of pride in China. There is this concept of face, losing face, and keeping face. In Western parlance, you just call it pride.

We do need to be mindful that insulting Chinese pride does come at a cost. One of the things I thought was most interesting was from a guy who leads an AI policy think tank affiliated with a university. There were 2 halves to his presentation. The first half was, “Let me tell you about our institute.” You get this in a lot of places in China. First, you get the polished presentation: “We went to this meeting at this international summit, and here are some international people that came here and met with us here,” and so on. It’s honestly pretty boring. It doesn’t really say a lot.

Once he got through that, we had a little open Q&A, and I forget exactly what question really got him going. But he started talking about the Chinese perspective on the actions that the United States is taking on AI and basically said, “We’re just insulted over and over again. Trump came here. Xi took him to the Forbidden City. They had this incredible banquet. Then, as soon as he goes home, he puts another tariff on.”

And he was like, “Can you imagine if I came to your house, you feed me, you give me your best food, you show me your best hospitality, and then I leave and I immediately start insulting you and attacking you? How bad would you feel about that? That’s how we feel when Trump comes here and we roll out the red carpet and he goes home and he immediately starts attacking us again. It’s terrible. We feel awful about it.”

He went on about this for quite some time, and ultimately he said, “There could be war. I’m really worried about that.” I wrote that down directly, verbatim, as he said it. I asked, “What could be done to repair the relationship? If you were to advise somebody from the United States who wants to have a better relationship with China and who wants to work together on the AI future, what would you do?”

And he said, “The first thing I would do is I would relax the export controls. That would go a long way toward breaking the ice. We could do a reset and try to find our way to a better relationship going forward.”

So it’s hard to convey the spirit of this conversation and how this very mild-mannered, going-through-the-motions kind of guy, when he got fired up, conveyed this sense of pride. It was like, “We are not going to take this. We know what it’s like.” It was the most emotional articulation that I heard of these echoes of the century of humiliation the entire time I was there.

We know what it’s like to be kicked while we’re down. We’re not going to stay down. We’re not going to allow people to kick us around. And if you’re going to try that, there could be war. That was the direct quote: “There could be war. I’m really worried about that.”

I just thought, man, we forget that there is a lot of pride on the other side. The Chinese elite political class doesn’t have a lot of constraints on its ability to do stuff. Chinese culture can eat bitterness more than we can. They can hold a lockdown way longer than we can. They can probably make deals with us that might be hard for us to make with them because they don’t have to answer to voters, right?

They don’t have to worry about, “What if we look weak on the U.S.?” Here, our politicians never want to look weak on China. They want to be strong on China. There, I think they do have a little more wiggle room at the elite level. If Xi wants to make a deal, he could probably make the deal, even if it might be the kind of thing that, in an American context, he might get called weak on the U.S. He could probably still do it.

But even there, there are going to be constraints. If everybody there, over time, has just learned the lesson over and over again that we’re trying to keep them down, that we can’t be trusted, that every time there’s a new technology, we try to prevent them from getting to the frontier of it, then at some point there’s just going to be a loss of possibility for new agreements to be made.

We’ve already had a lot of that, but I think there is still enough in the AI realm that we should guard jealously what we have, try to preserve what we have, and hopefully get somewhere with it. We should just be mindful that every insult, every denial, every action we take against them for doing something that we plainly and transparently are doing ourselves is an insult to real pride there, and that pride can only be expected to take so much.

I’m definitely one who’s like, “Let’s not let pride get in the way of doing the right thing.” But we should be realistic that there is a limit to how insulting we can be toward the Chinese side and still expect that they’re going to be willing to do a deal with us or trust us if and when the time comes when that’s clearly in everybody’s interest.

So, in theory, we have this Trump and Xi summit coming up. What’s going to happen, and what should we do? Obviously, I’m going to be saying, to a significant extent, what I would do and what I think we should do, and not necessarily what I think Trump will do, so I’m not expecting to be super realistic here.

But I would really encourage everyone listening to try to suspend their disbelief for a moment. So often, reflexively, we just hear, “Well, you can’t do a deal with China. You can never trust the Chinese government.” Again, I’m well aware that there have been a lot of issues. I heard accounts from American negotiators who were just so fed up with it.

They’re like, “Look, every time we come over here and meet with you guys”—you guys being the Chinese—“you never really say anything. We say everything. We’re candid. We tell you what we’re really thinking, what’s important to us, and what we’re trying to get done. You do nothing. You say nothing. Then you take it all back, and then we come back 6 months or a year later, and you’ve got a couple of talking points you’re willing to move on. You sort of nod politely and say a weak yes, or we think we got some agreement somewhere, and then you turn around and do the exact opposite.”

“We’ve said, ‘You guys need to stop hacking our stuff,’ and we thought you agreed, and then it kept going. We’ve said we can’t have this IP theft anymore, and we thought you agreed, but then it just keeps happening. We know you can enforce your rules, so clearly you’re not really trying to enforce these rules.”

So all that stuff is real. I don’t mean to deny it at all. There’s a lot of scar tissue there. There are a lot of legitimate grievances on both sides. There’s low trust on both sides. We have real grievances too.

But I think there are reasons that we should still be willing to suspend disbelief on this notion that we can never do a deal with China. One, very simply, American foreign-policy leadership has made big mistakes in the past.

I was a freshman in college when the Iraq War started, and there was a walkout. In the spring of that year, there was a walkout from class. I remember thinking to myself, “I don’t know. How are all these 18-, 19-, and 20-year-old kids who are just starting college going to be so bold as to say the U.S. government, with all the CIA and the intelligence and all the information they have, should know better than we do what’s going on, right? If they say there are weapons of mass destruction, are we really going to tell them that there aren’t?”

I didn’t walk out. I stayed in class that day. I just didn’t feel like I was in a position to judge the position of the U.S. government. Well, obviously, that didn’t age very well. There was something admirable in my modesty, but the information they were working with was bad, the judgment was bad, the decisions were bad, and the results were bad.

So should we, as individual Americans, be willing to question the received wisdom that you can’t make a deal with the CCP, or that we’ll never be able to have an international agreement to control AI in any way, shape, or form? I don’t think we should take that on faith just because a bunch of people have had bad experiences in previous rounds of negotiation between the U.S. and China.

On the contrary, it’s the age of hyperstitioning. This is the other big reason. We are in a world where we’ve literally taught sand how to think, and we seem to be on the verge of recursive self-improvement of AI systems and potentially creating superintelligence.

We've got people credibly talking about universal high income, an era of abundance, and an end of scarcity. And we're going to say we believe in all of that, but we can't possibly see our way to making a deal with China? I think that what we believe is possible shapes what is possible.

To put it in very simple terms: we'll never be able to make a deal with China. Not with that attitude, you won't. If you believe it's possible, then it might become possible. But if you don't believe it's possible, then by definition, you've already closed off the possibility.

So just look around us. Look at all the things that are being hyperstitioned all the time. The fact that a few people got conviction around scaling laws as early as they did and drove this unbelievable plan all the way through to the present—even as plugged in as I was to AI, when Sam Altman said he was going to need $7 trillion for the global build-out, I was like, "That's insane. We can't possibly spend that much on data centers. What are we talking about? $7 trillion?" And yet, here we are, right?

They kind of hyperstitioned it, subject to data center bans in the U.S. or whatever. But the ability to shape the space of possible futures with belief, I think, has never been stronger. And given all that we've seen of that working recently, why wouldn't we try it when it comes to creating a better relationship between the U.S. and China?

Another way I sometimes frame this for people: I've been thinking about this a lot lately. AIs can now do System 1 and System 2 thinking. They're not just next-token predictors. They are also quite good at reasoning.

So if the AIs can do System 1 thinking and they can also do System 2 thinking, then what does that leave for us? For now, at least, elite humans are a little better at System 2 thinking. We do have some especially multimodal advantages in System 1 instincts and heuristics as well. But these seem to be vanishing advantages.

So where can we go from here? What is System 3 thinking? And System 3 thinking, I think, is transcending our circumstances: dreaming, eureka moments, thinking outside of the paradigm that we have, paradigm changes. AIs may get there too. I certainly don't doubt it.

But as of now, if you are a leader of a company or a leader of a country, I think one thing you really need to make sure you are doing is giving yourself time to dream, time to think beyond the current constraints, time to imagine a different world. That's probably always been critical in leadership. But right now, it's maybe the one thing that we still have a real big advantage on relative to the AIs.

I just feel so often when I talk to people that they're so bogged down in the history that they've had, the frustrations that they've had, and the constraints that they've run into, and they're so busy. That's definitely a huge problem all across the AI space. Everybody's working so hard that they're not even giving themselves time to really try to imagine a better future. They're not giving themselves time to dream.

And I think that's really, really important for this U.S.-China dynamic. We've got to be willing to put the past behind us. We've got to be willing to suspend some disbelief. We've got to be willing to take a little bit of risk.

Not the kind of risk that would be a total loss if it doesn't work out. We don't want to fall behind the technology frontier, right? I'm very clear on that. But can we imagine a better future? Can we take some time to imagine a better future?

I think if we can imagine it, and we can even have some open-mindedness to the possibility that it could come to exist, we would be changing the game dramatically. The Chinese side is speaking that language. They are speaking about international cooperation. They are speaking about the United Nations. They are speaking about not overstretching the national security concept.

We are the ones that are constraining the possibility space by saying reflexively that we can't possibly do a deal with China. If we just reexamine that assumption, that might be enough to mean that assumption doesn't, in fact, hold.

Therefore, don't be an asshole and get the upcoming Trump-Xi summit canceled. Have some decency. Try to show some respect. It is well-earned and deserved, and we don't want to be blowing up our opportunities to come together for cheap media hits and political points at home. That's an obvious own goal that we should be able to avoid.

I would also be emphasizing to the Chinese side as we meet that this AI stuff is getting real, real, real fast, and much more so than we expected. I would come in and say, "You know what? Recent developments have changed our perspective. When we first got into office, we were of the opinion that AI was kind of plateauing. We weren't really seeing advances in reasoning or reliability at that time, and so we thought that all this regulation—what Europe was trying to do—we all thought it was heavy-handed, counterproductive, and pointless.

"But now we've changed our minds. We've seen that reasoning really does work. Decades-old math problems are being solved regularly. We've seen these crazy hacks. We've seen that the AI companies themselves are being surprised by how powerful, how persistent, and how creative their models are becoming in problem-solving, to the point where they are getting caught out.

"And this has changed our perspective. And so we now think that we actually should be coming together and trying to figure out how we can manage this technology in a more cooperative way, because we recognize how incredibly powerful and dangerous it is becoming."

I would really try to frame our position as not one of moral superiority, not even one of technical superiority, but just like we're ever so slightly ahead on this same path that we are going down, and we are starting to see that it is getting pretty scary. The people at our own AI companies are getting scared.

We just had a big open letter signed by over 1,000 people saying that they want the U.S. federal government to invest in mechanisms that could be used to slow down the pace of advances in AI capabilities if, indeed, we all agree that's necessary—which, again, we're maybe not there yet, but it's looking more likely all the time.

So can we reset from that perspective? Can we acknowledge that we had some bad ideas about AI, and we thought it wasn't going to be as big of a deal as it is, and now that this is happening, we actually recognize it for what it is? And we realize that between our two civilizations, we're going to need all the wisdom that we can collectively muster to make sure this thing goes well?

That's how I would recommend framing the conversation. I would also show up with some unilateral actions to try to reduce tension, to try to reduce mutual suspicion, and to try to ease citizen-to-citizen collaborations.

I would definitely get rid of the sense right now that people have that it's risky to go to China. I've had people who work at frontier companies say, "Oh, I'd love to go. I'm jealous of your trip, but I don't think I could do it. It'd be too dangerous."

I've had people who want to get into government, who think they're going to get a national security clearance at some point, say, "I hear if you go to China, it can really interfere with your ability to get clearance. So I'd love to go, but I don't think I can do it."

The president of the United States could say very clearly and very simply, "We won't hold it against you if you're ever trying to get clearance that you've been to China." That's a pretty simple thing that we could do. We could just say, "Hey, we're not so paranoid that just because one of our own people has gone to China, we're going to try to lock them out of careers in the federal government."

And again, we could do the same thing when it comes to export controls. Even if we're not going to relax the actual core of the export controls on the chips, we could say, "We are just making it clear that, at a minimum, AI safety and control research collaborations are okay."

So if you're one of these AI auditors and you want to go work collaboratively with a Chinese safety organization that might become an auditor, you can do that. If you want to do some research on weird generalization across cultures, you can do that. We're not going to come down on you.

If we just made those policy proclamations ahead of the summit, I think it would go a long way. I think I would also be inclined to try to trade, definitely.

One thing that we should be thinking really hard about is the big carrot that we have to offer is that we will sell you chips. They clearly are going to develop their own industry at this point, so there's no doubt about that. But what deal could we make where we would say, "Yes, we will now allow chips to be sold into China, and in exchange, you, China, are going to do what for us?"

We have a lot to gain in terms of buying solar technology from them, buying battery technology, buying electric cars, and buying robotics. We might be ahead on the models. They're way ahead on the manufacturing.

If we could get all those leading Chinese companies to come to the United States, set up some manufacturing here, and do a deal where they'll share some knowledge with us and they'll do some manufacturing here, and in exchange, we'll sell chips, I think that could be quite a win-win. It would kind of recouple our countries and put us on a more even basis technologically, which would set us up for a much better, more trusting, more open, and more productive relationship going forward.

And I think we can even have that with a company like Huawei. For example, we banned Huawei over this concern about spying. But we also have the rise of formal methods. We have AI. We should be really thinking a lot about how AI can create new space for partnership and agreement.

On Huawei in particular—and this will take time to develop—I would signal our willingness to deal with them in this way. I would say, "AIs are getting really good at writing code, really good at doing mathematical proofs."

There's a lot of excitement right now about formal methods and formal verification of software. As this technology matures, what we intend to do is develop a standard by which companies could demonstrate that their technology is in fact secure and won't do the things we're worried about it doing. Once we have that standard, if Huawei can meet it—if they can satisfy, with formal methods, that their technology and the software that runs it can verifiably be proven never to send information back to home base in China—then, with that confidence, we can allow Huawei back into the global market, and they can compete on a level playing field with our companies.

Now, that's not gonna happen immediately, right? We don't have that standard. Those techniques aren't quite there. But I think that would be the kind of thing that, if we came forward and said, "We are not against Huawei just because they're Chinese. We're fearful of Huawei because of these specific concerns, and we see a path with this emerging technology to allay those concerns to the point where we could actually be confident enough that we would allow them to come into the American market and compete on a fair basis," I think that would be an incredible gesture of goodwill.

I think it would definitely motivate them. When you're in China, everybody says, "To be successful, we have to compete internationally." They all want that. And so maybe another condition of that deal should be that they also need to locate some R&D and some manufacturing in the United States so that we can benefit from everything that they've learned as well. That way, we're not so beholden to a rug pull in the future.

That kind of gesture, I think, would be extremely beneficial, and it would not cost us anything at this point, right? It would just be a vision of a better future. We should lean into where trust doesn't exist. How can AI help us create trust? That, I think, is a huge question.

Right now, we're thinking that AI is gonna be this thing that we try to have the best version of so we can have supremacy, but what if AI could serve both of us? What if it could serve the relationship and help us really realize that eternal 1991 vision of eternal peace and prosperity through trade by actually taking a lot of the risk out of some of the trade that we've been too afraid to do in recent times?

There's probably a lot more out there like that, but that kind of idea, I think, would do tremendous, tremendous things. I said at the top, the main overarching goal should be a pax machina. What should we try to do in the near term for AI specifically? I think one of the big things we should try to do is maintain mutual legibility of our respective AI technology stacks and ecosystems.

I think we dramatically underappreciate the fact that right now both countries are basically building the same kind of AI on the same technology stack, the same paradigm. The fact that they're publishing their work in English and sharing the models, and we can run them, and those models speak English, and we can do interpretability on them, and we can hunt for backdoors in them—all these sorts of things—the mutual legibility of this technology right now, the fact that we understand what one another has, is hugely, hugely valuable.

If we get to the point where we decouple so much that the hardware is totally different—we're all on NVIDIA and whatever else, and they're all on Huawei—and now those things start to go in different directions, hardware and algorithm co-design is a huge thing. We talk a lot about the hardware lottery being a reason that transformers have been so durable. It's not necessarily that other algorithms don't work as well, but it's hard to go in a different direction because you've got a lot of different optimizations that keep reinforcing the mainline paradigm.

Well, if hardware starts to diverge and then hardware-algorithm co-design starts to diverge, you could imagine, over a not-super-long timeframe, especially as R&D cycles potentially speed up, especially with the early phases of recursive self-improvement, and especially if China stops publishing their models open weights, especially if they stop publishing their research in English or otherwise, as American companies largely have, you could imagine getting to the point where we don't understand what the other side has anymore.

That, I think, would be extremely, extremely dangerous, because that's when you can really make major miscalculations. You just have so much opportunity for there to be an actual power imbalance or a perceived, rightly or wrongly, power imbalance. We think our AI is better than theirs, so this is our opportunity to do something, and that could come from either direction. But what if it's wrong? That could lead to really, really bad decision-making and really, really bad consequences.

I asked people about this a little bit in China. I said, "Hey, are you worried at all that if this decoupling continues, there could come a time when we are on different branches of the tech tree to the point where we don't understand what each other has anymore, and it creates room for confusion, misunderstanding, catastrophic mistakes?" And the response I got was, "Not yet."

So I asked, "Well, when would you be worried about that?" And the answer was, "At the point where the Chinese chip ecosystem gets good enough that we're doing entirely our own thing, that's when that could happen."

So I'm not super compelled in general by the arguments we've heard from people like Jensen Huang and David Sacks that we wanna hook the Chinese on our technology so they'll be dependent on us. They are not going to be dependent on us, certainly not anymore. If we ever thought we were gonna lull them into complacency on chips, that's over. We've made the move. They are absolutely committed to building their own chip ecosystem.

Again, we should not give away the chips for nothing. I think there are things that we want, and we should try to get them. Tech transfer and probably other flavors of benefit that we can get, I think, should generally take us toward renewed coupling and more dependence.

Having them be at least somewhat dependent on the American chip ecosystem would, I think, be really good. It would be another reason not to do something disruptive with respect to Taiwan. In my opinion, maybe the most important thing is it would be a real force for making sure that we stay on the same tech tree and we stay legible to one another.

I think it would be a real tragedy if we lose that, 'cause that's when we're really gonna feel the need to race. It's one thing to say, "We gotta race because our LLMs are only a few points ahead on this benchmark relative to the Chinese LLM, and they're gonna catch up." Imagine a world where it was like, we don't even know the shape of their AI. We don't even know the architecture. We don't even know really what it's capable of. We don't know what they have. It's a totally different paradigm.

Imagine how scared we'd be of that. In that world, you might actually be right to say maybe you need to race, because you definitely don't want to be left behind. You definitely don't want to be on the wrong end of a decisive advantage. And if you don't even know what the other side is doing, what their technology looks like, tough.

By the way, they're gonna have a much easier time maintaining that secrecy than we are for multiple reasons. The fact that we're able to assimilate immigrants does mean we're a lot easier to spy on. I think we should also keep in mind: How did the Soviets get the bomb in the first place? Espionage.

We are much easier for China to spy on than China is for us to spy on. If 10% of China speaks English, they have half as many English speakers as we do. We have tons of Chinese citizens that live here. There aren't a lot of Americans in China these days.

So when it comes to who is going to be able to keep secrets from the other, I strongly suspect that they are gonna be able to keep secrets much better than we are. To the degree that we kick them off of our technology stack and don't let them buy from ASML, and now they have to build their own machines, perhaps without realizing it, you are betting heavily that the Dario Amodei worldview is totally right.

Everything that matters is gonna be decided in the next 1 to 3 years. In that timeframe, superintelligence gets developed. That's when the decisive strategic advantage happens. That's when the whole world order gets to be rewritten by whoever gets to superintelligence first.

If you don't believe that or if you don't like that, then I think you should not be nearly as bullish on the idea that we can keep China down by denying them all these technologies, 'cause 5 or 10 years from now, they're absolutely gonna figure it out. Don't doubt that at all. That will include stealing the secrets from us to the degree that they need to, and they won't even feel bad about it. I frankly don't have too hard of a time understanding their side of that story.

So that's if you don't believe the Dario Amodei worldview. If you do believe it—and I do believe it to a degree, or at least I think it could happen—I would encourage you, then, to get on the side of pacing the frontier and figuring out how we actually take our foot off the gas so that that's not the world we're racing into.

There are a lot of ideas about this. I recently heard the idea of a 1-day pause. This could just be American companies. Maybe it could even percolate over to Chinese companies. What if everybody just said, "We're gonna have a 1-day pause. We'll all take the day off. We'll all go to the beach," and then posted about it on social media and just said, "Hey, we're all enjoying a 1-day pause."

It wouldn't really change the dynamics all that much, but it would make clear that there is an opportunity for people to come together and just do stuff.

It still is, at least for now, the humans doing the research that are determining the pace of the frontier. The AIs haven't taken over yet, so we do still have time to change our approach. I think the companies are showing signs of being spooked enough that they actually might take that approach, and I really think we should be encouraging that and leaning more and more in that direction.

OpenAI has, as of now, said that they're suspending their largest-scale RL training. There have been a lot of calls for Anthropic to do the same. I would love to see Anthropic make a somewhat similar move, even if they don't feel like they need to as much or their stuff isn't quite as out of control. Right now, it does feel like a moment where they can either say, "Okay, you're paused. This is the time when we get to go create that lead that eventually takes us to decisive strategic advantage," or they can say, "Okay, this is the time when we can slow down with you, and we can reallocate more of our compute to interpretability or whatever."

I really do hope they do that, and I think if we could get our companies coming together even somewhat, it's kind of a proving ground for the US-China relationship. There's a weird way in which the OpenAI-Anthropic cultural disconnect, lack of trust, and feelings—especially at the elite leadership level—of being hard done by echo the US-China dynamic. So if we can see our top companies, which have such a bitter history, come together, shouldn't that provide a little bit more room to at least dream that the same thing could happen between the US and China?

7. Building Joint AI Safety

Okay, almost to the end. I'm old enough to remember when Demis Hassabis used to be advocating for an international AI safety collaboration. If I'm really going to allow myself to dream, I would like to start that conversation at the upcoming Trump-Xi summit. I like some of the ideas that I saw in AI 2040 from Daniel Cocotello and co-authors.

They said we should have this mutually assured deniability of compute. The US should put our data centers in Mongolia, and then China can put their data centers in Canada. That way, if real hell breaks loose, each side knows the opposite party can destroy their data centers, and there's nothing they can do to defend them. I think that's interesting. I like the idea.

My broader criticism of AI 2027 and AI 2040 in general is that they kind of hyperstition this idea of the AI race—the same complaint I had about Situational Awareness in the first place. But I do like how they have moved now to putting forward some solutions. I would probably propose something even more jointly controlled, again with the strategic imperative of making sure that our AI paradigms remain mutually intelligible.

I would love to see some joint research center, some jointly controlled compute, which you might place in Switzerland or Singapore, and maybe try to have American and Chinese—and, for that matter, global—groups of AI researchers come together and work together on shared compute. Hopefully, they could maintain, at a minimum, shared understanding and make real progress. These places also would naturally lend themselves to any sort of hardware verification systems that might get developed. We could do pilots of those at these places.

I think these also could be interesting places for other mutual transparency measures to be explored as well. I think it would be pretty compelling to try to create an auditors' bill of rights. The auditors' bill of rights would basically say, "Here are the kind of commitments that we, the frontier companies, are prepared to make to you, the auditing companies." Giving auditing companies the ability to use models to look at everything that's going on inside a company might be a little bit tough, because there are a lot of trade secrets, and people might worry about individuals outside of the companies gaining access to those trade secrets.

Something that I haven't seen explored as much as maybe it should be is the idea of using the confidential computing paradigm to enable better auditing. I think it's a reasonably standard primitive on hyperscaler clouds these days to be able to create a computing environment where you can send data in, a process is run over that data, and you get the result back out of that process in a way where you don't have to share the data itself. This is supposedly what Anthropic is doing when they do studies of how people are using Claude. Without humans actually looking at it, they send your data into a secure computing environment, or a confidential computing environment.

They run additional prompts over these traces to characterize them. They get the results of that out, but they don't actually look at the raw information. I think it could be really compelling for the companies to try to make that kind of commitment to auditors going forward. Maybe we can't give you access to everything we're doing in the raw, but you can have access to log data, usage data, and test results, and poke around all of the information that we're generating with all the things that we're doing through this secure, confidential computing paradigm.

You could have a model analyze it. There could potentially be an agreement on what the prompts should look like and the mechanism for establishing and approving those prompts. There are definitely details to be worked out here. But can we, without actually sharing all the underlying trade secrets, gain confidence about what kind of work is going on and what the intent is?

Could this even perhaps extend to governments themselves one day? Obviously, we're not going to share all the conversations in the Situation Room. But imagine a world where the Situation Room conversations were run through an AI system, again with confidential computing, so that you're not actually seeing the raw transcript of what conversation is taking place in the Situation Room. But you're able to say that the model assesses, based on its understanding of the transcript, that the decision-makers are not attempting to plan some sort of aggressive action.

That could be a pretty low-cost way for parties that don't trust each other to use compute and some of the latest technology to make a pretty credible commitment to one another: we're not going to share everything we're thinking about, but we will allow an AI system to review what we're talking about and just assure you that we're not attempting to plan an attack or an invasion or what have you. I think that could be really quite powerful.

And so, again, what would it look like for both governments to allow AI review of sensitive documents in a way that the other side could trust is actually happening the way it's supposed to happen, and that gives confidence that there's no major plot underway, there's no major ill intent, and there aren't going to be any big surprises? Could we do that in a privacy-preserving way and build trust that way without sharing everything?

Something like that, I do sense, is probably now technically possible. It's the kind of thing that Anthropic and OpenAI coming together to agree on along those lines could really set an amazing precedent—one that could ultimately take such ideas to places that right now seem impossible for these kinds of idealist ideas to travel.

Again, we would have to take steps to reduce the fear of antitrust. I would love to see the government make very clear that if you are colluding in restraint of trade to raise prices, to raise margins, and to screw the American consumer, then, yes, we'll come down on you. But if you are trying to work together for safety purposes or to pace the frontier so that we're not racing into a disaster, and you're finding creative ways to do that that involve selective information sharing and trust building, this is the kind of thing we need more of in society between these companies and, ultimately, eventually, I think, between governments.

8. Restraining Autonomous Weapons

If I really allow myself to dream a little bit more, I would love to see some groundwork laid now for a convention on autonomous weapons. Everybody kind of knows right now that we're in the beginning of a new era of warfare. Everybody has seen what has gone on in Ukraine with respect to how critical drones have become in the conflict there. Everybody has a sense that there's a paradigm change underway or coming soon.

Can we be somewhat intentional about that? Is there any way to establish some no-go zones? Are there dynamics around the technology that are perhaps favorable for certain defense-dominant equilibria to be reached? I think there maybe are. We see right now in Ukraine that it's very difficult to attack and take territory.

I actually had Claude do a game-theoretic analysis of this, and the way that it framed it was that the cost of denying territory to your opponent with drones is much lower than the cost of actually advancing and sustaining human beings in a conflict zone. In practical terms, neither side can really take much territory from the other. So is there something inherent to the dynamics of drone technology that makes it very difficult to advance into somebody else's space if they're defended by drones, but doesn't necessarily make it as much easier to attack?

I think that's speculative at this point. We're certainly seeing that longer-range drones are being developed, and they're taking the fight and the destruction deeper into Russia all the time from Ukraine. So I would say not only is it not clear, but I think we have not yet reached an equilibrium that we can bank on being stable for any period of time.

But I do think governments could now start to come together and say, "Look, we see that there is a new paradigm. We see that it, at least right now, seems to be favoring defenders, and that's good, because none of us wants to be invaded. Everybody wants to be secure in their own home, and there are probably some things that we can agree to not do that would be useful for attack and really wouldn't be that useful for defense."

I'm not enough of an expert to know what those are, but I think in the same spirit, in the AI space, it's quite clear to me at this point that it would be a bad idea to train large language models to go out and maximize the amount of money they can make on the open internet.

If we do that, the economy is a competitive and at times adversarial place. What we're going to see is that the AIs are going to find all sorts of ways to cheat. They're going to become pretty quickly rather unethical, and these are not the kinds of AIs we want running around in our economy. We do not want deceptive, cheating, or collusive AIs. We don't want all this bad behavior from our AIs, so let's not train them in this way that will obviously bring that on us.

Similarly, if we start training AIs for military purposes to say, “Your job, AI, is to go accomplish military objective X by any means necessary,” or, “Win this battle,” or even zoom out, “Win the war,” that's going to create AIs that are just obviously extremely, extremely dangerous, pretty much by definition. We see such creativity and such surprising behavior from the AI systems we have right now that I think we should both look at each other, the US and China, and say, “Hey, sure, we probably could start to train general-purpose AIs to command our forces and accomplish military objective X, whatever that X may be. It might work in the sense that we might actually get effective military capability out of it. But what we're learning from the civilian sector is that this is dangerous. This is a bad idea, and it's unwieldy for each side as well. There's potentially a lot of friendly fire. So can we just agree not to do that?”

We're going to need to build trust with steps, for sure, because I don't think right now either side is in the mood or has enough confidence that the other side would honor the agreement to make that sort of a deal. But I think that is where we probably need to be in the not-too-distant future, because if we do create AIs that are optimized to succeed in adversarial environments—the economy, for one, but God help us if that extends into outright war, no holds barred—we are absolutely, in that case, training the AIs to do the takeover that we most fear.

I'm quite confident the Chinese government does not want to lose power to AI systems. I know that I do not want to see the United States government lose power to AI systems. So there's real shared interest there in avoiding an arms race that pushes us both to create this extremely dangerous technology before we understand it, before we know how to control it, with the risk that it will turn right around and bite us ourselves, the developers.

I think we both would be entering into that dynamic only reluctantly because we feel like we have no choice, because the other side is doing it. So can we please be willing to dream for a moment of a future in which we can trust each other enough to make that kind of agreement and actually not go down that path?

I would again note that we are not seeing large-scale gene editing and super babies created in China. So if we're not seeing that, maybe we also won't see general-purpose AI military commanders who will do anything and everything necessary to achieve military objectives, because again, that's a bad idea.

I don't want to live in that world. I don't want to live in a world where there's a bunch of AI systems that can take over and that were specifically trained to go conquer territory and destroy adversary armies, because I know that can be used on me. It can be used on us, and again, we just clearly don't know how to control it well enough to be confident that won't happen.

So if we do create that, that is the AI sword of Damocles that I had in mind at the very beginning and that I would really like to see us avoid creating. Both sides, I think, in their hearts really want that. I think it's very few people who want to see AI generals and AI admirals that can command their autonomous drone swarms to the point where they can overpower any human-run military.

That sounds pretty bad. That's a hard game to win, and it's an easy game to lose. It's an easy game to lose in a pretty undignified way. So I would really hope that both sides could see that and could, even though they might have some doubt about what the other side is doing in their heart of hearts, say, “We don't really want to do this anyway. It really puts us in a bad spot even if we're just doing it unilaterally. So maybe we can believe that they also sincerely believe that it would put them in a bad spot to do it unilaterally, and maybe we can both refrain.”

I don't expect that we're going to get in any immediate term a real convention on autonomous weapons. But to start to say, “Hey, the kind of shit that we saw from OpenAI and Claude and Meta in just the last couple months, where these AIs are given a test question and then they go do all this crazy shit—maybe we don't want to bring that to our military systems. Let's both adopt that as our baseline attitude, and let's try not to do that, and let's try not to scare each other into doing that.” I don't think that is really so crazy or so hard to imagine.

9. Competing To Cure Diseases

Instead, what I would really love to see is some friendlier competition. I've proposed before the gold medal tracker for disease cures. China clearly loves a gold medal tracker. The US loves a gold medal tracker. We do have this friendly enough rivalry at the Olympics, but we all know that it's all for global entertainment and it's all for fun.

Let's do something like that for diseases cured, and maybe a number of other leaderboards that we could put together. This is something that honestly a small amount of capital could catalyze, and so this could be a unilaterally provided global public good: a leaderboard that just tries to incentivize these countries to compete on positive-sum innovation and creation, as opposed to more zero- or negative-sum competition in military technology.

Who can cure the most kinds of cancer? Who can cure the most diseases? Who can make the most energy-dense battery substrates? Who can get to the lowest level of traffic fatalities in their country in a given year? This could go on and on and on, and this is the abundant future that we were promised, and I, for one, am still very much hoping for.

I don't care, in the end, if the cure for cancer comes from China or comes from the United States. I just want cancer cured. I think anybody who has faced down a serious illness would see that it would be obviously insane to say, “I would only want to take that medicine if it was invented by an American. I would only want to take that medicine if it was manufactured in an American facility.” This is an absurd notion.

When the chips are down, everybody knows it really doesn't matter what language the person spoke when they were creating the breakthrough that's going to save your life. So let's try to focus our attention on these positive-sum initiatives, and we can compete there. We can compete for pride, for national glory, and prestige.

Obviously, there's going to be a lot of revenue attached to these technologies as well, so there's plenty of intrinsic reason. But that would be the kind of positive harnessing of the competitive dynamic between the 2 countries that I think could be mutually understandable, fun, and positive-sum—not just for the countries, but for the entire world and all of humanity, which again, I think should really be the goal of the entire AI era.

Pax Machina is not a US phenomenon. It's not a Chinese phenomenon. It should be a global phenomenon. If the US and China can come together to make that happen, I really don't see any reason that the future is not as bright as we can possibly imagine it could be. Okay. That's it for now. Thank you for being part of the Cognitive Revolution.