[BidClub_]
Hard Fork · · 33 分钟

OpenAI 对决 Anthropic:五角大楼如何选定合作伙伴

Kevin RooseCasey Newton

YouTube
TL;DR
  • Anthropic 选择了2项不可谈判的安全底线——不得进行大规模国内监控,不得部署完全自主武器,即使代价是失去联邦业务。 2月26日,Dario Amodei 表示:“这些威胁不会改变我们的立场。我们无法问心无愧地接受他们的要求。”随后,Trump 下令联邦政府在6个月内停止使用 Claude,国防部长 Pete Hegseth 则指示将 Anthropic 列为供应链风险。
  • 这项供应链措施可能带来足以关乎生死存亡的惩罚,但截至录制时,其正式法律地位仍不确定。 Hegseth 表示,任何军方承包商、供应商或合作伙伴都不得与 Anthropic 开展商业活动,但主持人看到的只有他的社交媒体帖子,并非送达公司的正式程序文件。Anthropic 表示将通过诉讼抗辩。
  • OpenAI 宣布达成协议,将模型部署到五角大楼的机密网络,同时声称保留的基本底线,正是 Anthropic 因坚持这些底线而遭惩罚的内容。 合同未公开,眼下有2种解释:要么官员是在对 Anthropic 展开政治性报复,要么 OpenAI 通过法律措辞接受了实质性让步。Casey Newton 的判断是,除了国防部长之外,局中几乎没人知道究竟是哪种解释。
  • 这场争端的关键,在于“所有合法用途”能否在 AI 和隐私法让大规模监控行为保持合法的情况下保护美国人。 联邦机构可以合法地从数据经纪商处购买数百万美国人的数据,并可能将其输入 AI 系统进行大规模分析;Casey 称这种行为在“功能上等同于”国内监控。他的框架是:“这就是整场博弈的核心。”
  • OpenAI 提出的“安全栈”或许能让员工得到安慰,却未必能解决数据来源的根本问题。 模型不一定能判断个人数据是通过合法方式收集,还是从数据经纪商处购买;知情人士将这些防护措施称为“安全剧场”。这意味着员工审查、内部抵制以及最终吹哨,将承担非同寻常的分量。
  • 对投资者而言,这一事件将政治站队从声誉层面的噪音,提升为实质性的交易对手风险和平台风险。 Trump 的一名前官员将针对 Anthropic 的行动称为“谋杀企业未遂”,Kevin 则认为,这可能是本世纪联邦政府对一家美国大型企业采取的最严厉行动。早期消费者反应可能朝相反方向发展:一些用户正从 ChatGPT 转向 Claude,其中包括购买 Claude Pro 的 Katy Perry。
摘要 · 为研究而整理的核心内容

1. Anthropic 的安全底线触发非同寻常的联邦威胁

  • Kevin 将升级节点定在2月26日星期四,距离五角大楼最后期限只剩1天。Amodei 拒绝在大规模国内监控和完全自主武器问题上妥协:“我们无法问心无愧地接受他们的要求。”

  • Trump 周五深夜发表声明,下令所有联邦机构停止使用 Anthropic 技术,并允许在6个月内完成过渡。由于声明没有提到供应链认定,也没有援引《国防生产法》,Casey 起初认为,政府或许会“从整场闹剧中抽身,设法挽回一些颜面”。Casey 表示,援引《国防生产法》将实际把 Anthropic 国有化或部分国有化,并迫使其制造一个按政府意愿行事的 Claude 版本。

  • 1个多小时后,Hegseth 进一步升级行动:“立即生效,任何与美国军方开展业务的承包商、供应商或合作伙伴,都不得与 Anthropic 开展任何商业活动。”截至录制时,Casey 的限定至关重要:尚未出现正式程序文件,而 Anthropic 表示将提起诉讼。

2. OpenAI 获得一份保护条款仍无法核实的协议

  • 在周四一则表达对 Anthropic 声援的泄露消息之后,Sam Altman 周五宣布,OpenAI 将把模型部署到五角大楼的机密网络中。OpenAI 坚称,协议包含同样针对大规模国内监控和自主武器系统的限制。

  • Casey 认为,冲突源于五角大楼要求实现“所有合法用途”。在缺乏有实质意义的 AI 监管或全国性隐私法的情况下,Anthropic 认为,仅仅要求合法并不能构成足够的安全保障,反而会让危险做法继续存在。

  • Kevin 仔细解读 Altman 的声明后发现,其中存在解释空间:五角大楼“同意这些原则,将其体现到法律和政策中,而我们把它们写入协议”。Amodei 表示,Anthropic 也曾获得类似让步,但随附的“法律术语”会让这些承诺失去效力。

  • Casey 提出了2种尚未解决的解释:官员可能出于意识形态或个人原因对 Anthropic 展开报复,包括外界报道的 Emil Michael 与 Amodei 之间的积怨;也可能是 OpenAI 接受了 Anthropic 拒绝接受的条件。答案或许就在尚未公开的合同中。

3. 合法购买数据暴露技术护栏的弱点

  • Casey 举的具体例子,是联邦机构再次对潜在移民展开社交媒体全面抓取。官员可能否认这属于监控,但如果 AI 能够实时审查每一条帖子,二者在实际操作上的区别将越来越空洞。

  • 数据经纪商可以合法汇总数百万美国人的信息,联邦机构也可以合法购买这些数据。Casey 认为,这种安排不符合国内监控的法律定义,但在实践中“功能上等同于”国内监控。将这些数据输入模型,也无法让模型可靠判断数据究竟是如何取得的。

  • OpenAI 告诉员工,公司相信自己通过模型层面的“安全栈”赢得了更强的保护。Casey 回忆称,涉及 Bryan Cranston 的 Sora 防护措施此前也曾被承诺,却未能按时落地;Kevin 的消息人士则将这份新方案称为“安全剧场”。

  • 主持人认为,问题不只是合同纠纷:强大技术究竟由谁控制——开发它的人,还是部署它的军方和政府?

4. 政治忠诚正成为开展业务的前提

  • Trump 前官员 Dean Ball 将这场施压行动称为“谋杀企业未遂”。Casey 将其背后的治理逻辑比作中国对待科技公司的方式:与党保持一致,否则就可能被碾碎。

  • Kevin 判断,这项受威胁的行动“以相当大的幅度”成为本世纪以来、或许是有史以来,对一家美国大型企业最严厉的惩罚。他将这种寒蝉效应与硅谷的右转联系起来,并举例称 Tim Cook 为 Trump 制作金色奖杯,Greg Brockman 则向超级政治行动委员会捐款2500万美元。

  • OpenAI、Google DeepMind 以及其他实验室的员工签署联名信,支持 Anthropic 的立场。Casey 认为,这种行动具有实际意义,并希望员工能够拿到合同、仔细审查其中内容;如果实际部署接近国内监控,也应当吹哨。

  • Casey 将监管俘获的指控用于 OpenAI:它凭借“感觉、魅力”和政治关系,进入了一场涉及其最大竞争对手的争端,最终接受了这样一个事实——政府业务要求企业按照政府的条件运作。“这就是监管俘获。”

5. Altman 的信誉与 Anthropic 的道德立场正面临真正考验

  • Casey 强调了 Altman 爬上的那根“树枝”:他的前同事批评他会对不同的人说不同人想听的话,这一批评与他在2023年被解雇的经历有关。Casey 希望 Altman 确实守住了2条底线,但担心天真或欺骗可能让监控现在发生,并在或许3年或5年后出现“自主杀人机器人”。

  • Kevin 提出了3个未决问题:供应链认定是否会正式落地、会约束哪些对象;OpenAI 接受了哪些 Anthropic 不愿接受的条件;以及消费者会作何反应。一个早期信号是 Katy Perry 发布了自己刚购买的 Claude Pro 套餐,并配上1颗红心。

  • Amodei 长期以来都会给员工《原子弹的制造》,因为他相信,先进 AI 对国家安全和全球秩序的重要性,可能不亚于核武器。Kevin 的结论是,这正是 Amodei 预料中的道德冲突:政府希望以“他们的条件”获得强大技术。

Kevin Roose

We have a lot to talk about today because it has been a very crazy 48-hour period in the AI industry. This dispute between the Pentagon and Anthropic, with OpenAI now involved, sort of came out of nowhere at the 11th hour. It has been truly an insane day and a half in my life. How has it been for you?

Casey Newton

Well, let me put it this way, listeners: Kev, imagine you get engaged and then 1 week later your fiancé is declared a supply chain risk. So, yeah, it's been a really, really crazy few hours over here as well.

Kevin Roose

Does this mean you have to call off the engagement?

Casey Newton

Never. It's going to take a lot more than Pete Hegseth to break up this love.

Kevin Roose

And just because we're going to talk about Anthropic and OpenAI and all of this today, we should make our AI disclosures. Mine is that I work for The New York Times, which is suing OpenAI, Microsoft, and Perplexity over alleged copyright violations.

Casey Newton

Yes, and if you missed the other big breaking Anthropic story from over the past week, the man that I am now engaged to works there.

Kevin Roose

Where should we start, Casey?

1. The Pentagon Crisis Explodes

Casey Newton

Yeah. I think if you're tuning in, maybe you've heard the biggest headlines, but it's worth hitting you with a few key bullet points. One is that, in the story we've been covering over the past couple of episodes, it has come to the point of crisis. Anthropic said it had 2 red lines that it would not cross, the Pentagon said it was going to move to declare the company a supply chain risk, and then, somehow, within 24 hours of that happening, Sam Altman and OpenAI swooped in and signed a deal that they say will observe those safeguards. It was a truly chaotic 24 hours, and we should dig into it.

Kevin Roose

Yes, and none of this has been happening through normal diplomatic channels. Basically, as far as I can tell, the entirety of this conflict has been contained in a handful of posts on X, a handful of blog posts, and some information leaking out from either side. I've been making calls for the last 2 days to the people who are involved in this situation, trying to get some information, and I've gotten a little bit of it. I'll happily share that with you, but I would say confusion reigns. No one, even the people who are directly involved in this situation, is clear about the details here, so I think we should also say up front that there is still a lot that is unknown about what's going on right now.

Casey Newton

Absolutely. Maybe to start, Kevin, we could go back to a part of the story that I think is pretty well known, which is what happened between Anthropic and the Pentagon, particularly in those final hours when the Pentagon finally said, “Hey, this isn't going to work. We're not going to give you what you want.” Time ran out, and they did not come to an agreement.

Kevin Roose

Yeah. This escalation started on Thursday, February 26, when there was basically 1 day left until the deadline that the Pentagon had given Anthropic. Dario Amodei, the CEO of Anthropic, put out a statement on Anthropic's website basically saying, “We are not going to compromise, no matter what, on these 2 exceptions that we want: mass domestic surveillance and fully autonomous weapons.” He explained why they weren't going to compromise on those, and then he said, in the line that a lot of people have been quoting, “These threats do not change our position. We cannot in good conscience accede to their request.”

Basically, they had been trying to work out a deal while preserving these exceptions that were very important to them, but they had not been able to do so.

Casey Newton

It's probably worth saying, Kevin, that I think a reason that quote stood out so much was that I cannot remember any tech leader invoking conscience as a reason not to do something since Trump was reelected. It felt like a shift in tone for the whole discussion around tech and power, and it was something we had not seen from Silicon Valley in a while.

Kevin Roose

Yes. What I understand from talking with people close to the situation is that even after this post from Dario Amodei, there were discussions happening between the Pentagon and people from Anthropic. They were trying to work out the contours of a deal. There was some willingness to at least change some of the language around these exceptions.

2. Trump Escalates The Fight

Casey Newton

While these discussions were happening in the back channels between officials at the Pentagon and people at Anthropic, President Trump posted a statement on Truth Social late Friday afternoon, just before the deadline that the Pentagon had given Anthropic. He said, “The United States of America will never allow a radical left woke company to dictate how our great military fights and wins wars.”

He also said that he was directing every federal agency in the United States government to immediately cease all use of Anthropic's technology, with a 6-month phaseout period for federal agencies to switch from using Claude to using other models.

One thing the president did not mention is the idea of declaring Anthropic a supply chain risk, right? This is something that we talked about on the last show. Basically, this is a much stricter designation, something that we don't think has ever been applied to a major American company before. It's usually used for Chinese chip suppliers or things like Kaspersky Lab. But Trump did not say that he was going to designate the company a risk to the supply chain.

I think some people at Anthropic and elsewhere thought, “Okay, this is a deal that we can live with. We're going to lose our government contracts, but we're not going to be declared essentially an enemy of the state.”

And more than that, Kevin, he also did not invoke the Defense Production Act, which, to me, was the true worst-case scenario here. The United States government would effectively have nationalized or partly nationalized Anthropic and forced it to make a version of Claude that did its bidding. When I saw the Truth Social post, my initial thought was, “Okay, maybe they're just going to walk away from this whole debacle and try to save some face.”

Kevin Roose

Yes, it did look like that. Then, a little over 1 hour after Trump's Truth Social post, Pete Hegseth, the defense secretary, posted his own take on the matter on X, in which he directed the department to designate Anthropic a supply chain risk. He said, “Effective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic.”

This was a pretty severe escalation, and the people who thought, “Okay, maybe Anthropic is going to get away without being declared a supply chain risk,” thought, “Maybe they're not after all.”

Casey Newton

Yeah. At the moment of this recording, so far, the only evidence we have that the Pentagon plans to declare Anthropic a supply chain risk is this social media post, right? My understanding is that Anthropic has not been informed of any new proceeding against the company. Anthropic says it would fight the designation in court.

So while this may happen, and we'll talk about what it would mean if it does, for the moment, it also appears like it could just be a threat.

3. OpenAI Enters The Classified Network

Kevin Roose

Meanwhile, while all of this was going on between Anthropic and the Pentagon, OpenAI had been working on its own deal with the Pentagon to use its models inside the government's classified networks. There had been some reporting on a leaked message that Sam Altman sent to OpenAI employees on Thursday, basically indicating that they were standing in solidarity with Anthropic.

That's very unusual because these companies do not like each other, and their leaders have not had a long history together. But basically, he was saying to OpenAI's employees, “We're not going to cave on these exceptions. We're committed to not having our models used for mass domestic surveillance or fully autonomous weapons,” and he was saying some supportive things about Anthropic.

But 1 day later, on Friday night, after this whole deal between Anthropic and the Pentagon had blown up in spectacular fashion, Sam Altman went on X and posted that OpenAI had reached an agreement with the Pentagon to deploy “our models in their classified network.”

He was basically saying that OpenAI had confidence its models would not be used for domestic mass surveillance or autonomous weapon systems, that the Pentagon had agreed with those principles, and that they had put them into the deal.

So those were the events of the past couple of days. When I summarize them, it sounds insane because what we effectively have are 2 companies, OpenAI and Anthropic, that claim to have identical red lines when it comes to the use of their products by the military: mass domestic surveillance and fully autonomous weapons.

1 of them, Anthropic, has been declared a supply chain risk, which is a very punitive, harsh measure that basically requires the company to cut off all business with the U.S. military and the federal government. The other, OpenAI, just announced a deal with the Pentagon to use its systems in classified networks, with the same 2 red lines that Anthropic had objected to.

There's some nuance there and some details that I'm sure we'll get into, but if you zoom out and look at the facts of the case, it is a truly insane series of events.

4. The Contract Language Matters

Casey Newton

It is, and I think we should talk now, Kevin, about this nuance that you bring up. We said at the top of the show that there is some uncertainty here. Kevin and I have not been allowed to review the contracts that Anthropic and OpenAI have with the military, although we would love to. We're hard-working at nwitimes.com.

But I think what we can tell you is that it appears this conflict comes down to this “all lawful use” standard, right? Keep in mind, the Pentagon signed a deal with Anthropic that had the red lines now causing it to freak out.

It went back to its AI labs and said, “Hey, we want to change this. We want you to say we can use this for anything that is legal.” On paper, that sounds great. Here’s the problem: We don’t meaningfully regulate the use of AI in this country and, as we’ve talked about on the show in the past, we do not have a national privacy law.

These are among the reasons that Anthropic has become very concerned about what powerful AI systems might do if they were given to the military in a country where there are not actually laws around how this powerful new technology can be used. Domestic surveillance is a really interesting one, Kevin. The Pentagon has said, “Well, we’re not going to domestically surveil people. That’s illegal.”

At the same time, Kev, there are other federal agencies right now that have mounted what amounts to a social media dragnet, looking through the social media posts of people trying to immigrate to this country, trying to find posts that are critical of the administration, and then using that as a pretext not to allow them to immigrate, right? Maybe the Pentagon will say, “Well, that’s not surveillance. That’s just part of our immigration process.”

But I think folks at Anthropic would say, “Well, no, no, no. If we knew how powerful these tools are—tools that can go through every social media post in real time—that might be an area that we are uncomfortable getting into, right?” This is where I think we start to understand what is different between Anthropic and OpenAI here.

Anthropic has said, “We’re serious about this stuff,” and I’m sure it’s possible to write into a contract a little bit of legalese that gives them enough cover to go back to their employees and say, “Hey, don’t worry. We’re not going to do anything untoward,” while at the same time doing a wink-wink, nudge-nudge to the Pentagon. The Pentagon could use these tools to do exactly what they’re doing with the social media accounts of would-be immigrants, right?

To me, that is what I see happening here, and it seems like a significant part of the conflict. Kevin, I know you’ve been on the phone all weekend. What do you make of that analysis?

Kevin Roose

Yeah, I think that’s largely my understanding. When he announced the agreement that they had made with the Pentagon, Sam Altman did put out a statement that left some room for interpretation, I think, on what OpenAI had actually agreed to. I will be very curious to see the actual language of these contracts, if that ever makes it out into public.

What I can tell you from talking with folks on all sides of this over the past couple of days is that OpenAI is framing this as essentially an identical set of constraints, right? They don’t believe that they have agreed to anything that would require them to use their models for mass domestic surveillance or for autonomous weapons.

But in his statement, Altman said that the Pentagon “agrees with these principles, reflects them in law and policy, and we put them into our agreement.” So, if you parse that very carefully, he is just saying what the Pentagon has been saying, which is that they’re not going to do mass domestic surveillance because it is illegal.

What Anthropic has been insisting on this whole time is that there are forms of mass domestic surveillance that are not illegal as the law is currently written, and so they want to prohibit the use of their systems for that stuff, too. More than that, Amodei has also said that during their negotiations, Anthropic was offered similar concessions, but the Pentagon accompanied those proposed concessions with “legalese” that would have made them ineffective.

That is entirely consistent with what the undersecretaries of this agency are saying on X, which is that they were not going to let any private company dictate how they wage war, right? I think that’s very important to say. Anthropic is telling us, “Hey, we were offered a very similar deal, and it did not protect you as an American in the way that OpenAI is now telling you that you were being protected.”

5. Two Explanations Emerge

Casey Newton

Yeah, I think when you boil it all down, there are basically 2 options here. One is that the administration and the Pentagon just have a political vendetta against Anthropic, right? There’s a bunch of language in the statements coming out of Pentagon officials’ X accounts about how these are all a bunch of woke liberals who are unpatriotic.

I think there is some sense in which this is just about style and tone and personality. Emil Michael, one of the undersecretaries at the Pentagon who’s been negotiating this deal, just clearly does not like Dario Amodei at all. I’ve heard that from multiple people, actually, that there’s particularly bad blood between those 2.

So, option 1 is that this is purely a political vendetta. OpenAI has been chosen for this contract because the administration likes them more, and there’s no substantive difference between what these 2 companies have agreed to do.

The other option is that OpenAI has agreed to things that Anthropic did not agree to. There are substantive differences between these agreements, and OpenAI is using this legalese, as you put it, to frame this as a victory when really they have conceded to the thing that Anthropic objected to.

I’m not sure yet which of those 2 is more true, but I don’t think anyone in this situation, except maybe the secretary of defense, knows. You know, there are 2 really important things about what you just said, Kevin. One is the idea that the federal government is trying to commit what Dean Ball, who was a member of the Trump administration and helped to write its current AI policy, called “an attempted corporate murder” just based on ideology.

And if you lived through the bias and censorship debates on social media of the early 2020s, it’s really crazy to hear elected officials saying that because we have a different ideology than you, we are going to take your contract away, designate you a supply-chain risk, and try to prevent other people from working for you, right?

That is how the Chinese government regulates its tech companies. Either you get on board with the party or they crush you, right? I think that is really chilling. And again, not just to me—to former members of the Trump administration, too. That feels really important to say. Do you—I’m curious what you think about that.

Kevin Roose

No, I’ve been looking back through historical examples of the U.S. government taking punitive actions against American companies, and I think it’s safe to say that this fight with Anthropic and the Pentagon is, by a fairly wide margin, the most punitive action that the U.S. government has taken against a major American company, at least this century and possibly ever.

We have seen this administration bully, strong-arm, and jawbone companies in the tech sector before. We have even seen them try to block certain companies from doing business with the government, but we have not seen them try to kill a company for what, as far as I can tell, are contractual disputes and ideological differences.

Of course, this is why almost all of Silicon Valley has lurched to the right over the past 2 years. It’s why Tim Cook is giving golden trophies to President Trump. It’s why Greg Brockman at OpenAI is donating $25 million to Trump’s political action committee, right?

There is this sense that you have to be in line with these people or they’re going to try and crush you. Until now, though, we hadn’t actually seen the Trump administration try to crush a company. Now we have, and I just can’t imagine what kind of chilling effect that is going to have across Silicon Valley.

Casey, I want to get your take on the employee activism that we’ve seen over the last couple of days. There was an open-letter petition—whatever you want to call it—going around that was signed by some employees of OpenAI, Google DeepMind, and other leading AI companies, basically saying, “We stand with Anthropic. We also do not want to make tools for mass domestic surveillance and autonomous killing,” and expressing solidarity with the stance that Dario Amodei has taken.

Do you think that’s meaningful? Do you think that’s part of what is fueling some of the decisions that these companies are making? That has been true in the past. Employees at these companies have had a lot of leverage over things like military contracts.

Casey Newton

I do think it is very meaningful. There are a lot of very well-meaning people at OpenAI, at Google, at DeepMind, as well as at Anthropic, who truly do not want to see the most dystopian possible AI scenarios come to pass.

It matters that they’re going to their leadership and saying, “We are not going to participate in this.” I hope that those employees get a hold of the contracts that their employers are signing and really scrutinize them. I hope that if they find out that their technology is actually being used for something that looks pretty domestic-surveillance-like, they would blow the whistle, right?

We really are going to need to rely on these employees in the coming years as the technology improves and as the Pentagon potentially does the thing that it is telling us today that it is not going to do.

Kevin Roose

Yeah, I think one other important thing to note here is that Sam Altman and OpenAI are trying to very carefully explain this to their employees in a way that does not suggest that they are just capitulating to the demands of the Pentagon.

OpenAI is saying to its own employees that they believe they got a stronger deal than the one Anthropic had in terms of protecting against mass domestic surveillance and the use of their systems for autonomous weapons.

Several people pointed me to this line in Sam Altman's post about how they were going to create what he called a safety stack: basically, a set of protections built into the model itself that the Pentagon is going to be using in classified situations that would essentially prevent the use of ChatGPT, presumably, for the things that they're worried about.

Casey Newton

Yeah, but by the way, this is the same company that told us it was going to build safeguards to make sure that Sora couldn't be used to make images of Bryan Cranston, Kevin. So, I'm just going to suggest that sometimes when OpenAI tells you it's going to build guardrails, they don't actually show up on time.

6. Legal Safeguards Face Reality

Kevin Roose

Yeah, I have also talked to people who say that this is basically security theater. If you dump a bunch of data that you've collected on Americans or purchased from a data broker into an AI model, it is not going to be able to tell whether that information was legally gathered. It is not going to be able to tell where that information came from, and so this is not really a meaningful change.

Casey Newton

Yeah. Let me underscore that point, Kevin, because it is so important. It is legal for data broker companies to buy up data on millions of Americans, and it is also legal for federal agencies to buy that data. Now, that does not constitute domestic surveillance to a legal standard, but it is functionally equivalent, right? So, this is the whole ballgame here, right? The Pentagon already has all of the tools it needs to do what is practically domestic surveillance. It's just not called that because it's legal to buy data about Americans from data brokers.

I understand we are so deep in the weeds here, but the reason we wanted to do this episode today is to try to persuade you. This is very high-stakes stuff. It is being done in the shadows, and the nuances really, really matter.

Kevin Roose

Yeah, I think the details and nuances are where the whole story lies right now, and it's hugely high-stakes. And so, I think on the surface this might look like some kind of boring contractual debate between AI companies, but this is really about the fundamental question of who controls technology. Is it the people who build the technology, or is it the militaries and the governments of the countries where that technology is built? I think that is the high-level question under debate here, and it's one where the Pentagon and Anthropic did not see eye to eye.

Casey Newton

This story, Kevin, is the whole reason that you and I have never been on the side of “AI is all hype, and it's fake, and it's a bubble that's about to collapse,” right? We saw these systems improving in real time. We knew that very soon they would be in a position where they could do the instant analysis of things like social media data, geolocation data, and other data that could potentially create massive new systems of oppression. And we are now on the precipice of those systems being potentially rolled out under the guise of a policy called all lawful use because there is no law to regulate them. So, it really just could not be more serious, and I'm glad we're getting a chance to talk about it today.

7. Sam Altman Faces A Trust Test

I want to bring up one more thing, though, which is the limb that Sam Altman may have just crawled out on. As I'm reading through his statement, I'm trying to square it with what I know. You were talking earlier in the show. It's like, “Okay, so you're telling me that the same day the Pentagon tries to kick one company out for having 2 things that it will never do, it signs a deal with another company and makes an agreement that it will never do 2 things.” It's so hard to square that, right?

And yet, you and I have both covered Sam for a long time, and we know that a criticism he has gotten from his former co-workers is that he tells people what they want to hear, right? This was at the root of him being fired in 2023: his co-workers saying, “This guy is telling me what I want to hear. He's not being consistently candid, and he's just leaving me in this state of perpetual confusion.”

And so now we fast-forward to a moment that is so much higher-stakes than that, right? Because we have to take Sam Altman's word that he has signed a deal that will not enable mass domestic surveillance of Americans in the short term, and maybe autonomous murder bots in the medium term—which is what? I don't know, 3 years, 5 years, who knows. So, the reason that I note that, though, Kevin, is that in every case, it has always come out in the end what the truth was, right?

I hope the truth here is that Sam got his red lines. I hope the truth is that somehow he arm-wrestled Pete Hegseth down, and Pete Hegseth said, “Okay, you got me, Altman. We're not going to do any domestic surveillance, for real. And we're not going to do any autonomous murder bots, for real.” My fear is, though, that either through naivete or deception, he has misled us, and we are going to find out sooner or later that, in fact, those 2 use cases are not only legal, but they're happening.

Kevin Roose

Right. I think that's still a big TBD, and I would also like to know—Sam, if you're listening, please come on and talk to us about this—because I think there are still a lot of unknowns here.

Casey Newton

But I would also bring up another point, which is that one of the big criticisms of Anthropic over the years has been about this idea of regulatory capture, right? There are many people, including some very high up in the Trump administration, who believe that all of Anthropic's warnings and statements about the risks of powerful AI systems, the speed with which they're accelerating, and the things that they could potentially do have been a pretext, right? That they're not actually sincere about this, that they're just trying to get a bunch of onerous regulation passed so that they can enshrine their status as an incumbent and prevent smaller startups and others from competing with them.

So, we've heard that term a lot: regulatory capture. This, to me, is an example of regulatory capture, right? This is a company, OpenAI, coming into a very hot dispute between its biggest rival and the United States government and effectively using what seem to be vibes, charm, and possibly some better political instincts to get a deal done through its relationships with the government.

Call it what you want. Call it savvy politicking or negotiating. Call it hair-splitting over the details of this contract. But this is effectively a company realizing that if it wants to do business with the U.S. government, it has to essentially abide by the terms that the U.S. government has set. That is regulatory capture, as textbook an example as you're ever going to see.

Yeah. So, where do we go from here, Kev?

Kevin Roose

So, I think there are a bunch of unresolved questions that I'm going to be looking at over the next few weeks and months. One of them is: What actually happens to this supply chain risk designation? This is something that the Pentagon has said it's going to do to Anthropic, but we have not actually seen any formal language about that other than Pete Hegseth's posts. We also have not fully understood what that actually would mean for Anthropic or what kinds of relationships it would be forced to sever with various other government contractors.

So, that's one bucket of unknowns: all the legal and contractual details of this supply chain risk designation for Anthropic. We also still have a lot to learn about what the other AI companies are being asked to agree to that Anthropic wouldn't, and what companies like OpenAI may have done to get their deal through while Anthropic's was being rejected.

And then I think there's a third bucket, which is: What does this do to the popularity of these companies with consumers? I think we are starting to see very early signs that some consumers who are very upset about the Pentagon's demands here are switching from ChatGPT to Claude. One of those users appears to have been Katy Perry, the pop star, who posted a screenshot on X of her newly purchased Claude Pro plan, circled with a little red heart.

Casey Newton

Katy Perry really said, “The Anthropic employees, those are my California girls, and they're undeniable.”

Kevin Roose

I should also say that I have to underscore that this is exactly the kind of moral conflict that Dario Amodei has been preparing for his entire life. One of Dario's favorite books—a book that he used to buy for all Anthropic employees—is called The Making of the Atomic Bomb. It's a very long history of the Manhattan Project during World War II. And the reason that he wanted Anthropic employees to read this book is that he believed that eventually what they were building—the AI models, the chatbots—would become as important to national security, to the government, and to the future of the global order as nuclear weapons.

He wanted to instill in them the idea that they were doing something with profound moral and ethical consequences. He understood that it's not just building technology, that if you build something that is powerful enough, the government is going to want to use it, and they're going to want to use it on their terms. And so I think this is exactly the shape of conflict that he was envisioning when he was telling people to read this book about the Manhattan Project.

Casey Newton

I think you're exactly right. It has been so amazing, honestly, to watch how many predictions that were made by the rationalist and LessWrong community in the early 2010s have started to come true. These sorts of conflicts between the government and the big AI labs were not predicted with any degree of specificity, but there was still a thought that we were going to get here.

Kevin Roose

And now it seems like that moment has arrived. I'm sure it must feel extremely surreal to Dario, as well as many other people who have been working on this for a long time. I just hope that we can navigate out of it safely.

Casey Newton

Mhm.