[BidClub_]
Sharp Tech · · 31 分钟

(预告)Anthropic 传奇仍在继续、Fox 与流媒体的未来、ChatGPT 问答、智能体购物、自动驾驶

Andrew SharpBen Thompson

播客
TL;DR
  • 商务部指令暂停所有外国公民访问 Fable 5 和 Mythos,包括 Anthropic 内部的外国公民;Claude 则被彻底撤出市场。 Thompson 认为,政府可能在要求概率型软件做出技术上不可能的保证,但 Anthropic 自身的行为和言论制造了如今推动政策转向的不信任。“最终决定权归谁?”才是争议核心。

  • 封锁前沿模型可能只能争取“几周或几个月,也许1年”的时间,但也会推迟在类似能力扩散前完成必要防御工作的时点。 发现漏洞与协助修复漏洞可能是“完全相同的请求”,而防守方拥有自己的代码则占据优势。Thompson 更倾向于推进“我们需要的曼哈顿计划”:大规模检查并修复网络软件和开源项目。

  • Anthropic 的安全叙事制造了技术能够实现其无法兑现的可控性预期。 公司多年来将 AI 与核武器相提并论,并最初通过 Project Glasswing 限制 Mythos;但约2个月后又开放了访问。Sharp 的判断是:Anthropic 承诺接近完美的安全性,越狱不可避免后,政策制定者便陷入恐慌。

  • 这场冲突涉及的既是网络安全,也是机构权威。 Sharp 援引报道称,Anthropic迟迟未披露111家获得高级 Mythos 访问权限的机构,以及后来披露的约50家额外实体,其中包括一家韩国公司;Thompson则提到战争部争议、模型“削弱”,以及 Anthropic“使用方式由我们决定”的红线。政府的技术判断可能有误,指令也未必能经受法院审查,但 Anthropic 无法令人信服地把自己置于国家之上。

  • AI 泡沫调整可能惩罚那些只是在既有工作流上“贴一层 AI”的公司。 Thompson 预计,真正持久的价值将来自围绕概率计算重构产品和组织,就像媒体从在文章旁投放广告转向算法信息流一样。这个转型需要数年,因为用户首先必须重新理解软件能够做什么。

  • AI 很可能带来深刻颠覆,但对确切结果做出自信预测,仍应保持怀疑。 Thompson 既反对“什么都不会发生”的 complacency,也反对 Anthropic 式的规定主义:互联网用25年重塑了社会,1993年的任何人都不可能描绘出这条路径,而 AI 的影响可能更大。缺少的正是“认识论上的……谦逊”。

  • 世界杯补水暂停说明,明牌捞钱也可能改善产品。 Thompson 认为,新增的商业广告库存能让疲惫的球员重新调整,并以克罗地亚的反扑为例,尽管英格兰最终4–2获胜;Sharp 同意“更少疲惫的球员是好事”,但认为这些暂停就该直接称为商业广告时段。更广泛的媒体判断是,Thompson 热衷于“体育赛事事件化”——押注高端赛事,而不是例行的每周内容库存。

摘要 · 为研究而整理的核心内容

1. 补水暂停既能从足球中变现,也能提升观赛体验

  • Thompson 认为,美国通过别扭机制实现政策目标的习惯是“最愚蠢的做法”:急诊室就医变相实现全民医保,AI 数据中心支出变成间接把钱塞进民众口袋的手段,球员安全暂停则变成利润丰厚的世界杯商业广告时段。

  • 他的反直觉辩护是,暂停可能改善比赛本身。克罗地亚对阵英格兰时一度“被压得节节后退”,随后借暂停重新调整,连进2球,让英格兰最终4–2获胜的比赛更具戏剧性;疲惫更少,也意味着更多球员能“发挥出最佳水平”。

  • Sharp 接受这种产品层面的解释,但拒绝使用委婉说法:就叫商业广告时段,承认这是利润优化。Thompson 更广泛的观看偏好是“体育赛事事件化”(eventization of sports)——世界杯、网球四大满贯和高尔夫重大赛事都值得看,但不必承担每周追更的投入。

2. Fable 出口管制暴露了一个建立在信息不完整之上的冲突

  • Sharp 提供的事实锚点是:周五晚些时候,商务部下令暂停所有外国公民访问 Fable 5 和 Mythos,“无论人在美国境内还是境外”;Thompson 补充说,人在 Anthropic 内外也一概如此。Claude 则被彻底撤出市场,截至他们周四录制时仍不可用。

  • Thompson 曾短暂体验过 Fable,认为它确实实现了能力跃迁,还开玩笑说自己被迫退回到“只能用 Opus 的 5.5”。但他强调,非公开信息让这场对峙格外难以判断:“我们不知道什么?”

  • 据报道,导火索是 Amazon 越狱 Fable,并向白宫提出担忧。Thompson 提醒,公开说法来自一名看过 Anthropic 报告的研究员——“这全是 Anthropic 的包装”——但他认为“只要略微修改提示词就能发现漏洞”的说法并非没有可能。

  • 他的奥卡姆剃刀式解释是,双方都有责任:政府说停止这种行为,Anthropic 说“你阻止不了”,而官员听到的则是来自一家已经制造“信任缺失条件”的公司的又一次拒绝。因此 Thompson 怀疑自己那周是不是“对 Anthropic 有点太苛刻了”。

3. 概率式软件意味着,完美限制本身就是错误目标

  • Thompson 给出的关键例子是:“找出这段软件里的漏洞”和“帮我修复这段软件里的漏洞”,本质上是“完全相同的请求”。确定性系统可以禁止某个定义明确的功能,但通用模型无法在不牺牲有用能力的情况下,可靠区分恶意意图和防御意图。

  • 自动驾驶提供了类比。过去多年,人们试图把每一种道路场景逐一硬编码,但变量实际上近乎无穷;端到端模型则通过从海量场景中归纳学习,并为从未明确遇到的事件形成内部启发式规则,最终变得可行。

  • Thompson 认为,把 AI 当作“高级搜索引擎”,犯的是同一类错误。Sharp 早期用 ChatGPT 查询体育统计数据,恰恰是“完全错误的功能”:模型的优势正来自它并非为每个查询逐一确定性设计。

  • 既然 Mythos 级能力很可能会在其他地方出现,Thompson 认为白宫的做法像是在“用拇指堵住堤坝漏洞”,但“洪水终究会漫过整面堤坝”。访问受限的每一天,也意味着防守方少一天时间使用最领先的模型去发现并修复弱点。

4. 网络防御必须穿过危险,而不是绕开危险

  • Thompson 并没有低估威胁:“网络上的几乎所有软件都很可能不安全。”他的判断更具体:访问不可能永远被封锁,因此“解决安全问题的唯一办法,就是穿过它”,在这项能力广泛可用前积极完成修复。

  • 防守方拥有一个关键优势:它们掌握代码,可以直接检查和修复;外部攻击者则必须从外部推断一个被遮蔽的系统。所需的动员规模就是“我们需要的曼哈顿计划”,覆盖网络代码库和开源项目。

  • Anthropic 在 Project Glasswing 上开展的防御工作值得肯定,但公司的传播方式也帮助引爆了反弹。Anthropic 多年来将 AI 比作核武器,在为银行和大型科技公司提供安全保障的同时,仍称 Mythos 危险到不能发布;约2个月后,它又让 Mythos 变得广泛可用。

  • Sharp 的反驳进一步放大了其中的矛盾:Anthropic 制造了完美安全和独家托管的预期,在政策制定者发现安全措施可以被绕过、试图重新夺回控制权后,公司却表现得十分意外。“如果你谈的是超级武器”,政府权威就是不可避免的归宿。

5. Anthropic 的真正问题,是声称自己拥有最终决定权

  • Thompson 反对的不是 Anthropic 关注安全,而是公司反复暗示:“只有我们能处理。只有我们负有责任。”他的直白推演是:如果 Anthropic 应该管理 AI,而 AI 将管理世界,那么 Anthropic 实际上是在说自己应该管理世界。

  • Sharp 追问,这种指责是否过度依赖推断。Thompson 提到了战争部争议:Anthropic 划定的红线是“使用方式由我们决定”,同时还表现出在模型行为违背自身偏好时,愿意暗中削弱模型。

  • Sharp 又补充了相关报道:111家机构获得了高级 Mythos 访问权限,后来又披露约50家额外实体,其中包括一家韩国公司。Anthropic 对此反应迟缓,说明它仍未真正理解“政府最终掌握决定权”,即便这套出口管制机制在法律或技术上可能并不成立。

  • Thompson 对 Microsoft 的审查给出了组织层面的诊断:“你们做什么,做得很出色;怎么做,还有很多功课要补。”但他最终也承认,问题同样是实质性的:一群可能并不理解“现实世界究竟如何运转”的人,正在对客户、企业乃至联邦政府主张权威。

6. AI 的价值将来自重构工作流,也来自更谦逊的预测

  • Thompson 预计,AI 崩盘的一部分原因会是企业试图在既有工作流上“贴一层 AI”,却在概率系统不肯像传统软件那样运行时感到挫败。真正持久的公司会重新思考整个工作流,就像互联网媒体从在文章旁投放广告转向信息流一样。

  • 这种重构需要数年,因为人们必须先重置预期,新的组织形态才会变得清晰。互联网用了约25年改变社会;Thompson 指出,1993年的人绝不可能预测到,互联网对媒体的影响最终会帮助促成 Trump 政府的出现。

  • AI 的影响可能更大,但 Thompson 既反对否认,也反对自信的安全领域精英所持的“确定性和规定主义”。理解 AI 不等于理解社会的一切;他的正面主张是保留更多怀疑,改善沟通,并对任何机构都无法预先描绘的结果保持“谦逊”。

Andrew Sharp

Hello, and welcome to a free preview of Sharp Tech. Hello, and welcome back to another episode of Sharp Tech. I'm Andrew Sharp, and on the other line, Ben Thompson.

Ben, how are you doing?

Ben Thompson

Sad, Andrew.

Andrew Sharp

Why?

Ben Thompson

Sad because I've been forced over the last week to use these really crappy AI models. Only 5.5 in Opus. Where's Fable?

Andrew Sharp

Oh, boy.

Ben Thompson

I told you Fable was good.

Andrew Sharp

Somebody needs to jailbreak Fable back to life for us, man.

Ben Thompson

We did need the 48 hours of truly leap forward in terms of model capabilities just to be truly—

Andrew Sharp

Poor Ben. His wings have been clipped here. Yeah, now here we are, living in the Stone Age. But look, it's only a matter of days until the rule is reversed and Fable is brought back to life, would be my guess.

Ben Thompson

I was going to say—

Andrew Sharp

We're going to start with Anthropic.

Ben Thompson

Was that Andrew Sharp Reports, or—

Andrew Sharp

No, look. It occurred to me as we came on to record and talk about the export controls and everything else that by Friday night, I'm sure there will be a twist that renders half of this obsolete. We're not going to start with Anthropic, though, actually, because you informed me that you have a World Cup take related to the hydration breaks.

Ben Thompson

Oh, you want to bring—

Andrew Sharp

And—

Ben Thompson

No, I don't know if we can do this at the top of the show because—

Andrew Sharp

Okay.

Ben Thompson

I'm concerned I'm going to make people so mad that—

Andrew Sharp

They will hit stop and unsubscribe.

Ben Thompson

Right. Well, if it was at the end of the show, then they might go and unsubscribe, but at least they've listened to the whole show. But if we put it up front, they might not listen to the rest of the episode. Although maybe if we make the rest of the episode really good, they'll decide to continue to be subscribers.

Andrew Sharp

A bit of grace after 90 minutes. Fine.

Ben Thompson

Okay.

Andrew Sharp

We can save it to the end.

Ben Thompson

No, that's fine. I'll drop it. Okay, here you go. Here's my World Cup hydration take. You remember previously, people sort of got annoyed at this, but I still stand by it.

The U.S. implements these policies in the stupidest way possible, right? So, as I said, we have universal health care: everyone has to be served by an emergency room. That made a lot of people mad, but I stand by it. The fact of the matter is, if you're in an emergency, you're going to get health care. That is universal health care in the stupidest way possible. “Oh, it doesn't cover preventive care.” Yes, I know. It's dumb.

Andrew Sharp

Yeah, in a very narrowly defined way, it's universal health care.

Ben Thompson

Right. The point of making that analogy is that it's a really stupid way to accomplish policy. Or, if that was in the context of AI data centers, they're just going to end up paying people to build them, and that's how we're going to backdoor into AI putting money in people's pockets.

Andrew Sharp

UBI.

Ben Thompson

Right?

Andrew Sharp

Right.

Ben Thompson

Yes.

Andrew Sharp

Yeah.

Ben Thompson

So, this is the stupidest way possible. This is the context for my hydration-break take.

Soccer is—and it is called soccer, by the way. Do you know where the word “soccer” comes from?

Andrew Sharp

I have no idea. Where?

Ben Thompson

It comes from England, to be clear.

Andrew Sharp

Okay, great.

Ben Thompson

Football was a sport played on your feet, as opposed to on horses, and then there was association football. S-O-C-C was soccer, came from England. I'm sorry that we preserved English culture better than they did and used the word soccer.

Andrew Sharp

And we've been getting it wrong for 150 years. What the hell happened, England?

Ben Thompson

That's right. Okay, no, wait, that's not the take. Don't unsubscribe yet. Give me a couple more minutes and I'll really make you mad. Soccer has all these traditions, the way it's always been played.

One of them is that you have 45-minute halves, the clock doesn't stop, and that's just part of it. The whole thing with the hydration break is, well, the U.S. is really hot. Number one, come on. Get over yourselves.

Andrew Sharp

It's not that hot.

Ben Thompson

Seriously. This is tied into the whole thing. Go to Taiwan in the summer. That's hot. I would come back to Wisconsin every year, and people would say, “How do you go to Wisconsin in the summer? It's so humid.” I'm like, “It's not humid at all.”

But it's not that hot. Whatever. Everyone's like, “It's so hot. We need to have some games where it's going to be too hot. Players need to stop for their health and safety and get water. If we do it in some games, it won't be fair. We have to do it for all the games.” What it's actually doing is giving us an extra commercial break in the middle of the game.

Andrew Sharp

Right.

Ben Thompson

It's like a commercial break, but you're not going to change the channel because you're in the middle of the game. It's a huge moneymaker. Everyone's upset about it: “You're destroying soccer just to make a few bucks,” blah, blah, blah.

Here's my take: we're actually fixing soccer. One of the complaints about the hydration break is that the momentum changes. A team is on its heels—

Andrew Sharp

The purists hate it because it's actually changing some of the outcomes of these games. Players come out of it—I think Haaland scored right after a hydration break. That's Erling Haaland, the player.

Ben Thompson

Well, it happened in the England-Croatia game yesterday. Croatia was on its heels. They came out, reset, and scored a few goals. And guess what? England was definitely the better team than Croatia. England was incredibly impressive in that game.

That game was so much more entertaining because the hydration break allowed Croatia to reset and score a couple of goals, which made it very gripping. England ended up winning 4–2. It made the game better. Guess what? Corners are a good idea.

Andrew Sharp

Yeah.

Ben Thompson

Hydration breaks—a blatant sort of “grow up, it's not that hot out” combined with a pure money grab. We're backing into making soccer better, and everyone should thank us for it.

Andrew Sharp

Fewer exhausted players is a good thing, I think. That's the kernel of the take.

Ben Thompson

Exactly. I like to see people performing at their best, uh, Suby.

Andrew Sharp

Don't unsubscribe, but that's my take.

Ben Thompson

And don't sue Ben.

That's right.

Andrew Sharp

My take on hydration breaks is that they should just be called commercial breaks, so maybe we'll get there by the time we get to the next World Cup. I do understand the instinctive rage at branding this profit-optimization strategy as a player-safety initiative.

Ben Thompson

That's right. There's a lot to be said for just being honest, okay? And guess what? You know what's good for the game? If everyone's making more money.

Andrew Sharp

We're all adults here.

Ben Thompson

Yeah.

Andrew Sharp

If everybody else has commercial breaks, it's fine. I hadn't considered the idea that it's actually going to make the second halves more entertaining than they would be otherwise. So, as far as I'm concerned, not a bad hydration-break take, soccer fans—

Ben Thompson

You know what?

Andrew Sharp

—of the world, or football fans—

Ben Thompson

And if you're super excited for the game, imagine that the U.S. is playing Australia tomorrow. Maybe you've already watched it by the time you listen to this. Maybe you're pre-partying; you're getting ready. You know what you might need? A dehydration break. In the middle of a half, run to the bathroom.

Andrew Sharp

There we go.

Ben Thompson

All good from my perspective.

Andrew Sharp

Well, my number-one World Cup take—

Ben Thompson

Yeah.

Andrew Sharp

You created a giant World Cup group chat, and there are 15 different people in there. You added me without asking me, so I've been in there for the past week.

Ben Thompson

Because you were dropping World Cup takes in other group chats, so I thought you wanted in.

Andrew Sharp

I was dropping Champions League final takes a couple of weeks ago, not World Cup takes. But being in the group chat, I'm fairly busy. I've got 2 toddlers and a bunch of podcasts. You are busier than I am, though, and I am flabbergasted by how many World Cup games you've consumed over the past week or so. You continue to be the biggest sports fan I've ever met, so congratulations.

Ben Thompson

Well, thank you. I love the World Cup. It's great. Games are on all day. Did you see Xu Xi Jin, the Chinese propagandist?

Andrew Sharp

No.

Ben Thompson

Oh yeah, he put out a tweet complaining about the game times, that they should be catering to China as far as when the games were played. The games are being played from 11 in the morning until 1 in the morning.

Andrew Sharp

Okay.

Ben Thompson

Relax, okay?

Andrew Sharp

9:00 PM Pacific.

Ben Thompson

Yeah.

Andrew Sharp

Some of the games are clearly being catered to China, so China’s being taken care of here. And you’re watching almost every single one. I’m getting texts from you throughout the day about different World Cup games I didn’t even know were happening. So I look forward to joining you probably by the time we get to the knockout round.

Ben Thompson

Well, just in general, what’s great about the World Cup, I’m not actually a soccer watcher. Every World Cup, I think, “Oh, I should watch more soccer.” No, it doesn’t happen. The whole week, I’m sure it’s great. I appreciate the quality of play with teams that play together all year. I get it in theory.

Andrew Sharp

Whatever.

Ben Thompson

I am all in on the eventization of sports. I love big events. It’s like tennis: I watch the Big Four. I don’t watch your regular tennis match.

Andrew Sharp

Mm-hmm.

Ben Thompson

I will watch the big golf tournaments. I’m not watching it every week. I want events. I’m in on the events. I enjoy the atmosphere around the events and the social media around events, and I don’t want to deal with it the rest of the year, so it’s perfect.

Andrew Sharp

There you go. Well, I would say I’m halfway in on the World Cup but all the way in on America, and I had a great time watching USA’s opening match and will be watching on Friday night in Seattle. But for now, Ben, we have news to get to. We did talk Anthropic and Fable last week, but as I’m sure 98% of the audience knows by now, late Friday afternoon the Commerce Department issued an export control directive to suspend all access to Fable 5 and Mythos by any foreign national, whether inside or outside the United States. So Fable was—

Ben Thompson

Or inside or outside of Anthropic.

Andrew Sharp

Right, indeed. That was part of the problem as well. Claude was pulled from the market altogether and remains off the market, as you noted at the top. As we record this Thursday afternoon, that directive is in place. We’ll see how much longer it lasts. What do you think of where we are right now? It’s kind of a frustrating story to talk and write about because, on one hand, it’s the biggest story in technology. It also seems like there’s a good bit of nonpublic information that’s informing behavior here. So how do you wrap your arms around where we are?

Ben Thompson

It’s hard to wrap your arms around it, in part because of what you said there. What do we not know?

Andrew Sharp

Mm-hmm.

Ben Thompson

It’s also hard because everyone in this situation is incredibly frustrating to me on a personal level.

Andrew Sharp

Great.

Ben Thompson

So I actually feel like, almost this week, I took the opportunity to just write about Anthropic in general. We don’t need to rehash it because we talked about it on Sharp Tech last week.

Andrew Sharp

Mm-hmm.

Ben Thompson

And that was one of those times when I actually got a fair number of requests like, “You need to write this out”—this idea of the sort of fusion of the safety culture with lots of choices that they make that are great for their business.

Andrew Sharp

Business incentives.

Ben Thompson

Yeah, what a magical alchemy that is. So I wrote that on Monday. We already covered it last week. You can go listen to that. But Fable was a way into that, and I said it there. I’m not actually going to really talk about this because who knows what the heck is going on? But implicit in that—and anyone who’s been listening to these podcasts for a while knows this—is an overall frustration with Anthropic and a general concern about this company, not because they’re purposely being bad actors.

Andrew Sharp

Mm-hmm.

Ben Thompson

But my overarching concern about people who think they are the only ones who are trustworthy and ought to make decisions for the rest of humanity is that it’s just bad.

Andrew Sharp

The track record is not great.

Ben Thompson

Yeah, I don’t think that works. And that underlies so much about their posture in general. There was a write-up on this AI summit, or whatever, a day or two ago.

Andrew Sharp

Yeah, G7.

Ben Thompson

And you could just see in Altman’s comments that he clearly doesn’t trust OpenAI. He kind of trusts Google. But it’s like, “We should decide.”

Andrew Sharp

Mm-hmm.

Ben Thompson

And I think Altman took a much better approach, which is actually, “We shouldn’t be deciding. Someone else needs to be deciding.” That’s closer to my sensibilities, I would say, for sure. So in general, I’m frustrated with Anthropic, not just in this case.

Andrew Sharp

Yeah.

Ben Thompson

At the same time, from what we’ve been able to garner, there’s a very good chance that the administration is just being incredibly stupid here.

Andrew Sharp

Hmm.

Ben Thompson

And so I almost feel like I’ve been a little too hard on Anthropic this week because, from what I can gather—and again, who knows? We don’t have enough details about what’s happening or what’s not happening—there’s sort of an Occam’s razor explanation here, which is the US administration is like, “Stop it from doing this thing.”

Andrew Sharp

Yeah.

Ben Thompson

And Anthropic is like, “You can’t stop it from doing this thing.”

Andrew Sharp

That’s not how AI works.

Ben Thompson

That’s not how it works. The example here is, what’s the difference between “Find the vulnerability in this software” versus “Help me patch vulnerabilities in this software”? It’s the exact same request, and the reality is Mythos is clearly—if you used Fable, you could just feel it—it’s clearly ahead. It’s something new. No one has anything to match it. But if the history of the last few months and years continues, we’re going to have Mythos-level models in the near future.

Andrew Sharp

Yeah.

Ben Thompson

And so what is the best possible thing we can do knowing that’s coming? Everyone needs to be working overtime to patch their vulnerabilities.

Andrew Sharp

Mm-hmm.

Ben Thompson

And we’ve been talking about this for several months on Sharp Tech. It’s going to be a messy few years, and it feels like there’s this sense in the White House of trying to patch things, put your thumb in the dike against this wave that’s coming.

Andrew Sharp

Yeah.

Ben Thompson

And the reality is, the wave is going to go over the whole wall. It’s going to be a mess, and the best thing we can do is—offense is the best defense. I don’t know if that’s the right analogy here. But go out there and work feverishly for the next little bit to fix as many vulnerabilities as we can before this is truly broadly available. And every day that this conflict is going on is a day when people aren’t able to do this.

Andrew Sharp

Yeah.

Ben Thompson

And I think it comes back to this—and this is where I do bring Anthropic back into the conversation—how many people in the White House understand how AI works?

Andrew Sharp

There are some people in the White House, for sure, who understand how AI works.

Ben Thompson

Mm-hmm.

Andrew Sharp

But at the highest levels, this distinction between a probabilistic piece of software versus a deterministic one—

With a deterministic system, you could say, “You can’t make this available,” and, “This isn’t possible,” and you could hit some buttons to turn off different features that create unreasonable risks. You can’t really do that with AI.

Ben Thompson

Right. That’s like the magic of AI, right? The strengths and weaknesses apply to everything. The reason AI can do that—is it truly fully generalizable? I think not as much as the advocates claim, but it’s pretty darn generalizable—is precisely because it’s not deterministically designed to handle every sort of thing.

We see this in self-driving cars, right? The reason why self-driving cars took forever to actually be viable is because people spent years trying to hard-code every single scenario that a car might encounter.

Andrew Sharp

Yeah.

Ben Thompson

There are so many variables in a car that you can’t do that. What you need to do is have an end-to-end model that is basically—not exactly the same as an LM, but the same concept of a neural net—that generalizes from being exposed to tons and tons and tons and tons of scenarios and develops its own sort of internal heuristics and probability distributions, such that when something new happens, it kind of knows how to react—

Andrew Sharp

It can adapt.

Ben Thompson

—without being—

Andrew Sharp

Yep.

Ben Thompson

—told what to do. And you think about this: it’s not just that it’s not deterministic, it’s that putting deterministic rules into that functionality makes it worse in many respects because you’re actually limiting it artificially in its capability to understand different situations. We’re just moving into this fundamentally new paradigm that’s totally different, and it’s really easy for people to look at AI as a fancy search engine.

Andrew Sharp

Yeah.

Ben Thompson

We had this conversation a few years ago.

You used AI for the greatest-of-all-time talk and looked up some stats.

Andrew Sharp

Oh, yeah. Which, by the way, it’s gotten a lot better at pulling recent stats.

Ben Thompson

For sure.

Andrew Sharp

And everything else—it’s a lot more accurate than it was 2.5 years ago. But indeed, it’s a different sort of tool than traditional Google.

Ben Thompson

Right. And what I told you at the time was like, “No, that’s the exact wrong function that you’re trying to do.”

Andrew Sharp

Yeah.

Ben Thompson

Right?

Andrew Sharp

It’s the worst thing you could use ChatGPT for.

Ben Thompson

Well, then this gets into so many issues, or so many concepts that flow out of this. One point that I made again and again is: What is the shape of the crash going to look like, or the bubble popping?

Andrew Sharp

Mm-hmm.

Ben Thompson

What it’s going to look like is people attempting to plaster AI onto existing workflows that were built around a certain way of computing, being frustrated that it continually doesn’t work. And then what’s actually going to happen is people are going to completely rethink their approaches.

This idea of moving from putting advertising next to an article to a feed—like the—

Andrew Sharp

Yeah.

Ben Thompson

Like AI—

Andrew Sharp

Create new companies built atop this technology to take—

Ben Thompson

That’s right.

Andrew Sharp

—advantage of it.

Ben Thompson

That’s right. And so the challenge is, this takes years. It takes a long time for people to even reset their expectations of what’s possible and what this means. And in the meantime, what we do have is this ability to look at code bases, find vulnerabilities, and exploit them—basically autonomously.

Andrew Sharp

Hack the planet. Just like Hackers. Great movie. So, to anchor people in this take, I think a lot of this springs from the report that Amazon was able to jailbreak Fable and raise some concerns with the White House—

Ben Thompson

Right, and no one knows for sure what this jailbreak is, right? There’s this analyst-researcher who saw a report from Anthropic, so this is all Anthropic spin, to be clear—

Andrew Sharp

Mm-hmm.

Ben Thompson

—who characterized it as very basic: just slightly change your wording, and now it’s looking for vulnerabilities. Which I think, again, is an Occam’s razor explanation. You could believe that would happen.

Andrew Sharp

Mm-hmm.

Ben Thompson

And then Anthropic is saying, “Well, look, that’s very minor. You can’t stop that.” And the White House is saying, “You guys are doing it again. We’re just asking you to stop it, and you won’t stop it.” And that’s where Anthropic’s at fault. They’ve created the conditions of mistrust with the administration.

Andrew Sharp

And also, though, is the core issue for Anthropic that the way they discuss AI risk then creates unreasonable expectations for how it can be managed?

Ben Thompson

Of course. The whole Mythos rollout about this is like, they go on for years talking about AI being like nuclear weapons. Then they have this huge Project Glasswing: “This is way too dangerous to release. We’re going to secure the banks and secure the big tech companies.” And then 2 months later, “Okay, everyone can use it now.” And then you turn around and you’re creating the conditions for this reaction.

Andrew Sharp

They create the expectation of perfect security, given how dangerous this weapon is for the whole world, and we’re going to be very careful. And then when it’s revealed that you can’t possibly paper over every possible risk that arises from this technology, other people who have been paying attention freak out and start to try to take back control. It seems like that’s some of what’s happened here.

Ben Thompson

Yeah. Well, the problem is that—I’m not minimizing the risk—it’s actually quite scary. Almost all software on the web is likely insecure.

Andrew Sharp

Mm-hmm.

Ben Thompson

But the problem is that you’re not going to foreclose that forever.

Andrew Sharp

Entirely, no.

Ben Thompson

Yeah. What is the administration buying with this? Weeks or months, maybe a year.

Andrew Sharp

Okay.

Ben Thompson

The only way to solve the security problem is to go through it—

Andrew Sharp

Mm-hmm.

Ben Thompson

—is to actually work and patch stuff like crazy. And by the way, the defenders have an advantage. They have the code.

Andrew Sharp

Yeah.

Ben Thompson

They can examine their code bases. They can fix the problems. Even if you’re attacking something from the outside, it’s a little bit more of a challenge. It’s more obfuscated. You have to sort of figure it out.

The Manhattan Project we need is a whole-scale effort: let’s go through all this code on the web, all these open-source projects. Which, to Anthropic’s credit, they did do. That’s the sort of aspect of Project Glasswing. But we’re just not there. There’s just not an acceptance of this reality.

Andrew Sharp

Yeah. Well, Michael says, “If Ben’s going to get so outraged about Anthropic’s actions even when he believes the safety threat is real, I’d love to hear a positive vision for what they should be doing to address safety concerns that wouldn’t be self-serving. I agree with Ben’s analysis about this being a convenient source of cultural alignment with the business incentives, but I just don’t have a problem with it. What am I missing?”

So, if you were running policy at Anthropic, what would you advise them to do to handle some of the safety risks?

Ben Thompson

The problem I have with Anthropic is the implicit and sometimes explicit message: “Only we can handle this. Only we are responsible. Only we are properly concerned about the dangers.”

And what’s implicit in that is, only we should control AI, combined with the rhetoric that AI is going to run the world. People got mad at the sentence in my article, but it’s just a logical construction. If A equals B and B equals C, then A equals C. They want to run the world. That’s the implication of saying, “We should run AI, and AI is going to run the world.”

Andrew Sharp

Mm-hmm.

Andrew Sharp

I mean, how explicitly have they ever said, “We should run AI”? Is it more an inference from their behavior?

Ben Thompson

Yeah. Their behavior and, I think, their rhetoric. They clearly don’t think Sam Altman should run AI, that’s for sure—which, by the way, I completely agree with, to be clear.

Andrew Sharp

That’s true. Yeah.

Andrew Sharp

And it sounds like, to Sam’s credit, even Sam agrees on that point at this stage.

Ben Thompson

Well, he’s cleaned up his rhetoric a lot. I think there was a lot of bad rhetoric that came out. And this is the challenge, because people are like, “Look, why shouldn’t they talk about job upheaval?” No, there’s the opposite mistake, which is to say nothing is going to happen, when you’re basically lying through your teeth.

Andrew Sharp

Mm-hmm.

Ben Thompson

And I think there’s an aspect of that in Silicon Valley that says nothing is going to happen. No, it is going to be disruptive. Now, we might not know exactly how it’s going to be disruptive. You go back to the internet. The internet was incredibly disruptive.

Andrew Sharp

Right.

Ben Thompson

The entire structure of society—

Andrew Sharp

It took 25 years. We were the frogs in the boiling water for a while there.

Ben Thompson

This administration is the Trump administration, which is directly downstream from the structural changes wrought by the internet and what it did to media. Imagine going back to 1993 or whatever, when the World Wide Web comes out, and saying, “This is going to lead to Donald Trump being president.” It would have blown people’s minds. AI is going to be arguably even bigger.

But part of the problem is the certainty and prescriptiveness of what’s going to happen.

Andrew Sharp

Mm-hmm.

Ben Thompson

There’s an aspect of this: things are going to change, and I think everyone could do with a little more doubt. What’s the word I’m looking for? Epistemic doubt.

Andrew Sharp

Humility?

Ben Thompson

Humility about the fact that we don’t know what’s going to happen. This idea of just explaining why this is different—

Andrew Sharp

Mm-hmm.

Ben Thompson

—appreciating that just because you understand AI and other people don’t doesn’t necessarily, one, mean you’re smarter than them, although often you are, to be fair, or two, that you know everything about everything because of this factor, right?

Andrew Sharp

Yeah.

Ben Thompson

You’ve been in conversations with people where they’re explaining something to you, and their tone and approach is so patronizing, and it just is infuriating, and you’re like, “Screw this guy. I don’t care.”

Andrew Sharp

Are you describing conversations that you and I have had about various consumer technology that you demand that I adopt and then I stubbornly refuse to?

Ben Thompson

Right, and you want to punch me in the face.

Andrew Sharp

Yeah.

Ben Thompson

Maybe I can identify this attitude because I can personify it at times. That’s a fair pushback.

I think the emailer does have his finger on something. When I was at Microsoft, I remember I never got promoted, and I was very frustrated about it.

Andrew Sharp

Mm-hmm.

Ben Thompson

I ended up on this team where I was actually a level 61, and everyone else on the team was level 63. I was doing all the work, and I thought I should get promoted. But the one piece of feedback I remember getting at my year review or something like that was, “Your what is excellent. Your how needs a lot of work.”

There was this idea that it wasn’t just my manager that had to be involved—the other managers on the promotion committee, the manager above the manager, and all those people.

Andrew Sharp

That’s amazing.

Ben Thompson

There’s an aspect of corporate politics that is just putting on a happy face, talking to people, and selling yourself and your ideas. This is why I didn’t last long in corporate America. I’m much better off, I think, on my own, to say the least.

Andrew Sharp

Yeah.

Ben Thompson

There’s an aspect here to my feedback to Anthropic: your what is fine. It’s the how that needs work.

Andrew Sharp

Right.

Ben Thompson

You need to figure out how to communicate these things.

Andrew Sharp

And to describe it, there needs to be a capacity for risk tolerance over the next 4 or 5 years, and there’s not much of that coming from Anthropic in the course of these conversations. They create the illusion of perfect security as long as they’re the ones in charge of that security, and that’s going to be a problem long term.

I’m open to the possibility that members of the Trump administration simply don’t understand that you can’t keep these models from being jailbroken, and you can’t reduce the risks on that front. I will also note, though, that there was that report in The Washington Post about Anthropic giving the government a list of 111 organizations that had advanced access to Mythos, and then there were an additional 50 or so entities that were added and had already received access.

Anthropic was slow to get that list of companies to the government, as the government was asking, “Who actually has access to Mythos?” One of the entities was a South Korean company. What’s interesting in that report is it seems like Anthropic still doesn’t understand that the government is ultimately in charge, and it’s really not that complicated. Dealing with the government shouldn’t be as impossible as Anthropic is making it look.

Ben Thompson

Yeah.

Andrew Sharp

So again, blame belongs on both sides of the equation here.

Ben Thompson

This goes back to the Department of War debate, which ultimately is a philosophical question of who’s in charge. The fundamental problem with Anthropic’s how is actually not just the how; it’s the substance.

In that debate, their red line was, “We get to decide how this is used.” So, as you asked earlier for evidence of them saying they should be in charge of AI, there’s your example.

Andrew Sharp

That’s the evidence. Yeah.

Ben Thompson

They’ve already demonstrated it. And by the way, we talked about this last week: they’ve also demonstrated the capability and willingness to silently nerf their models to stop behavior that they don’t think is acceptable.

Andrew Sharp

Yeah.

Ben Thompson

This is the core issue. There’s an assertion of authority that is reserved solely for them, and it’s not just with their customers, it’s not just with other companies, it’s with the federal government. That is the fundamental issue.

Andrew Sharp

Mm-hmm.

Ben Thompson

It actually does go back to that episode.

Andrew Sharp

Yeah.

Ben Thompson

That’s the issue here: they’re trying to dictate terms, and the government is probably being dumb. The export-control thing—who knows if it’s even legal or whatever it might be? That’s for the courts to decide. My guess is probably not, right?

Andrew Sharp

Mm-hmm.

Ben Thompson

What’s actually happening is the exact same battle that was happening before.

Andrew Sharp

Yeah.

Ben Thompson

Who gets final say?

Andrew Sharp

And when the government says jump, Anthropic should say, “How high? You want the additional 50 entities? Here are the additional 50 entities. You have a problem with this South Korean entity?”

Ben Thompson

How’s that boot taste, Andrew?

Andrew Sharp

Well, no, that’s just the way it’s going to work.

Ben Thompson

Right.

Andrew Sharp

And so—

Ben Thompson

No, that’s the thing.

Andrew Sharp

It’s inevitable. If you’re talking about a superweapon that poses dangers to the entire world, this is the inevitable conclusion of that conversation. But they still have not fully grokked that—

Ben Thompson

And that’s what’s so frustrating: the people who are implicitly saying they should decide everything seem incapable of understanding the reality of the world as it actually is. If you can’t understand the reality of the world as it actually is, on what basis should I have faith that you get to decide how the world is going to be?

Andrew Sharp

Mm-hmm.

All right, and that is the end of the free preview. If you'd like to hear more from Ben and I, there are links to subscribe in the show notes, or you can also go to sharptech.fm. Either option will get you access to a personalized feed that has all the shows we do every week, plus lots more great content from Stratechery and the Stratechery Plus bundle. Check it out, and if you've got feedback, please email us at email@sharptech.fm.