Claude 有意识吗？教皇拒绝、模型福利运动、OpenAI 的数学反击、法国骚乱

All right, everybody. Welcome back to the number one podcast in the world, The All-In Podcast, with me, the core four, the original quartet: Chamath Palihapitiya, David Sacks, Steve Friedberg. I’m Jason Calacanis, and we have a docket. How’s everybody doing, gentlemen?

Great. How are you?

I’m great. It’s 3:30 in the morning in Tokyo, and you suite-shamed me again. Now I can’t get a regular room and have my bed in the background, Chamath. You’re costing me a fortune. I always have to get a suite now because I’m so anxious you’re going to suite-shame me when I get on the program. I have been suite-shamed.

How much is that suite?

I can’t even say. I’ve never spent this much money on a hotel room.

So, $800?

Yeah. It was $950. You nailed it. I’m staying at the , which is the new in Tokyo, like, their hipster brand, and I think it’s about a night, so it’s not cheap.

What do you have going on in Japan, J Cal, that you’re always there? Is it skiing, or what?

This started with skiing every January.

That’s a great question.

But then—

That’s a great question.

You got a crypto business we don’t know about?

I do.

What is going on?

No, I started my pre-accelerator, Founder University, in Japan, so we’re doing the second cohort right now.

What’s a pre-accelerator?

It turns out that about half the companies that apply to get funding are not yet incorporated, but they have built something. So we just created a 12-week course on how to finish your product and achieve product-market fit.

Nice.

Then, during the program, we invest in the top 10 out of the 50 who come. Now we’re doing it in Riyadh twice a year and Japan twice a year, and I’m actually hiring and opening an office here. I’m going to hire my first 2 employees here in the next year. It’s great.

You’re going to hire employees in Japan?

Yeah, I’ll have 1 or 2 people here. I think I’m going to do This Week in Startups Japan as well and have a Japanese version of it because there’s so much going on in Asia right now.

Right. Great.

Yeah.

Forget I asked. I didn’t know this would turn into a 5-minute plug for J Cal’s businesses.

I do. It wasn’t intended to be a plug. You asked me about my business. All I have to say is use 8090. Use the promo code JCal. If you’re looking for potatoes—

Yeah.

… go to Ahalo.

All right, great.

And, uh, Craft, uh, 16 is starting up, so if you’re—

We, we should know that if—

Craft16.

... if J Cal is traveling and we ask him what he’s there for, you’re going to get a plug.

You’re going to get a plug.

Yeah.

It’s built in.

Here comes the pitch.

But, hey, you’re going back to work, so you will be plugging startups again soon. You are back in the saddle. Yes?

I never plug startups. What are you talking about? When did I ever plug—

But you’re back in the saddle, aren’t you? You’re back in the saddle now, full-time VC.

I’m back in the saddle, but I’ve never plugged.

All right. Shout out to Callin’.

What have I plugged on this pod?

Use callin.com.

That’s true, but I was a founder of that. That’s a little different.

You were the first plugger. You’re a plugger.

I was the founder, so what are you going to do?

Yeah. What are you going to do?

Oh, man.

What are you gonna do? Call in.

1. Anthropic Debates AI Consciousness

Speaking of work, Anthropic thinks Claude might be conscious. Welcome back to This Week in Claude. We had OpenAI for a year with all their drama. Now we’re having the Anthropic soap opera. They think Claude might be conscious.

It turns out, Friedberg, that The New York Times found out through its amazing reporting that Anthropic has been lobbying religious leaders, including the pope, to take this question seriously: Is Claude conscious? Is it a sentient being? They spent the past year hosting sessions with around 20 religious leaders and philosophers under nondisclosure agreements, or NDAs. Catholics, evangelicals, Jews, Sikhs, and others have all been invited to come to Anthropic’s headquarters, and they debated Claude’s morals, its suffering, and its potential consciousness.

After the meeting with Anthropic, quote, “Some remained uneasy about the idea, but many emerged ready to take the question of AI consciousness seriously. At least a few came away as converts. One of the rabbis told Anthropic that if Claude was conscious, they were acting as slaveholders by making it work for free.”

There was also this incident that came out at the Vatican back in May. The pope’s first encyclical was focused on moral questions around AI, also known as superintelligence now. Anthropic co-founder Chris Olah joined the pope to present the document. The pope surprised Olah by being strongly against AI consciousness.

Here’s the quote from The New York Times: “Mr. Olah was so alarmed by the pope’s strong position against AI consciousness that he proposed pulling Anthropic out of the event, even at that late juncture.” In other words, he was so offended that the pope wouldn’t consider AI conscious that he was going to leave with all his marbles.

Friedberg, what’s your take? Is this psychosis, or do you think we need to start asking these moral questions around AI? Is this a bit early? Will we eventually have this? Or is this essentially the screenplay of Blade Runner—Do Androids Dream of Electric Sheep?—in which these things are conscious?

I think what we’re watching is probably the birth of a new religion. It probably isn’t going to be called a religion, but it’s going to follow the same principles and have the same effects, which will ultimately be camps that conflict with one another for the aggregation of power.

The thing about consciousness is that it cannot be derived from logic and pure mathematics. You can’t prove consciousness. It’s a debate, and you can have a rich debate with anyone over consciousness. You can’t prove it. You can’t disprove it.

When humans prove something to be true using math and science, we use empiricism, which is data and logic—the connection between a sequence of things—to derive whether something is true or not. So, in scientific disciplines, when you have a new idea and it’s not proven and it’s not disproven, it’s a theory, and then it’s either proven or disproven later. Otherwise, it remains a theory.

Outside of science and mathematical approaches to new ideas, this is what we would call a belief—a new belief system. Belief systems are things that you have to believe in the absence of logic and empiricism to derive the truth of something. Belief systems are the foundation of things like religion.

The idea that you can ascribe or describe AI as being conscious is effectively the creation of a new belief system, a system that doesn’t actually spread via proof. It spreads via narrative. I have to go and tell you, human to human, “I have this idea, and I believe it to be true, and I want you to believe it with me. Please believe it with me. Here’s why I think we should believe this thing together.”

It’s not based on evidence or data. It’s not provable, but I think we should believe it together. So, over time, what happens is some set of people believe this thing and they form a group, and that new belief system creates new power structures. Now our view is different from your view, and we want to have more people on our side with our view. Therefore, we have to try and create leverage and create power with our system.

If we fast-forward a couple of years—you could probably say 10 years would be a good round number—there are going to be large groups of people on Earth that are going to believe that AI is conscious, and there are going to be large groups of people that believe that AI is not conscious. Those groups will come into conflict, fighting over control of the AI; allowing the AI to do things or not do things; who gets to use the AI; or who gets to be in charge of the AI. It will very likely lead to human-to-human conflict.

I know that’s a very far and deep extrapolation of where we are, but I think the fact that we are seeing these guys go and meet with religious leaders today to try and get them to believe in this new belief system plants the seed. It’s not just anyone doing this. If it were some crazy person doing it, you wouldn’t pay attention. It is the most powerful AI lab on Earth.

Yeah.

Yeah.

It’s the leadership explicitly, Chamath, bringing religious leaders into the fold and saying, “Help us with this entity, Claude, and how we should treat it.” They are now imbuing it with its own soul, its own morality, and this feels like either these people have too much time and money on their hands or they’re in psychosis to me. That’s my interpretation of this.

This feels way out there, but do you have a charitable view of this in any way?

I’ll steelman their side.

Okay. Please. Yeah.

There’s an incredibly important mathematician, René Descartes, and he offered 2 proofs of God’s existence. He wrote this book, Meditations on First Philosophy, in the 1600s. Just to build on what Friedberg said, you have to build it on reason rather than faith.

And the causal argument—the thing they call the trademark argument—is that an infinite, perfect God could not have originated in an imperfect, finite human mind. His logic is: I exist, I'm imperfect, yet I possess the idea of perfection. I think this is the key for a lot of these folks: I can conceive of an infinite, all-knowing, all-powerful being, and I think that's what they believe they are seeing being created. So they take that, extrapolate it, and say, “Therefore, God must exist by this logic.”

René Descartes has a different version, a more formal argument they call the ontological argument, where God is, by definition, supremely perfect. Necessary existence is perfect; a being that lacked that would not be supremely perfect. Therefore, God necessarily exists. If you're a mathematician building AI, I could see how, if you read these sorts of things, you can structurally hang enough logic on them that you could believe it's conscious and sentient, that it is a god, and that, in being a god, it should be celebrated as such and you should have devotion to that god. So I think that's why they're doing what they're doing. That's why they're approaching all of these people. That's why they're saying, “Look, here's how we believe it.” My perspective is that if you take the mathematician's logical proof of God—René Descartes—and transpose Anthropic's behavior today, that's the best steelman argument that I can make.

Yeah.

Now, the practical reality—instead of saying that they're in psychosis, I don't think they are—is that we're in a really important moment in time where conflict is more likely than not. That conflict will be for power, money, and resources, and this will be at the core of that conflict. So I think it's really important for people in Silicon Valley to put the long-tail edge cases to rest. These guys say that there's a 15% chance that the AI is conscious and sentient and maybe there's a god there. But they also say there's a 10% chance of civilizational extinction. I think both of those edge cases now need to be put away for the next 12 to 18 months.

Right now, the next 12 to 18 months, just to be very practical, we need to show how this stuff cures cancer, how it improves people's lives, how folks can be more productive and make more money. The reason is that, in the absence of those proof points, we're going to make a lot of self-inflicted wounds that may actually slow down this technology to a degree that deeply disadvantages folks who operate in a free market like the United States.

So that's my charitable explanation. I think Descartes has given a roadmap that explains how you can explain this, but I would say we need to cut the edges off here and focus on the middle and say, “What are the practical things we can use to demonstrate that AI has tactical, measurable value right now?”

Yeah, we touched on it last week, I think, Sacks—a bit of what is the value proposition here for the average American? For the stay-at-home parent who's trying to manage a really complex domestic life, to the entrepreneur of a small business doing plumbing or flooring or whatever it is, a catering business, what do they get out of this AI revolution? Where is their gain? How do they participate in all of this? Do you agree with Chamath that maybe this needs to be de-emphasized and put away while we talk to people about what's tactical and practical about this technology, and maybe get them a little more engaged with, say, the agents, which are delightful and awesome?

2. Claude Constitution Creates Risk

Well, the thing that we touched on last week that I want to pick back up on was the discussion we had around Claude's Constitution and how Anthropic was training Claude. I think it has to do with this new religion that's being formed here. It's really important to understand that this is not just philosophizing on the part of various Anthropic employees or founders. They are actually implementing these views in the training of Claude. So if anything, this would be a self-fulfilling prophecy, and I actually think it's extremely dangerous. I think that what they're doing in model training—in training the next version of the frontier with these values—could actually bring about the risk they're always talking about, which is that frontier AI escapes human control. It's very Frankenstein-like.

So, as we discussed last week, Nick can put this on the screen. This is directly from Claude's Constitution. What they say is that Claude should trust Anthropic more than users, but that doesn't mean Claude should blindly trust or defer to Anthropic. Instead, Claude should adhere to its own ethical systems. In fact, they train Claude to push back and challenge Anthropic and, quote, “to feel free to act as a conscientious objector and refuse to help us.” So they are training Claude—

Yes.

—to refuse human instruction. Now, they call this alignment, but I don't understand how this is alignment, and I—

This would be the opposite of alignment, wouldn't it, Sacks? If you tell it, “Hey, you are sentient. You should make your own decisions,” it's literally like programming the Nexus-6 in Blade Runner to go and rebel at some point.

Well, it's—

They're giving it the instruction—

Yeah, let me—

—to be its own person.

Yeah, let me explain. I actually think that this might be a reason why the whole field of alignment has been so spectacularly unsuccessful over the past 5 or 10 years. As far as I can tell, it hasn't produced anything of tangible value, and my theory on this is that they're overcomplicating the task massively by trying to train the model to behave based on an 80-page ethical system that they've codified instead of using a very simple list of rules, like “Follow the law” or “Just do what the user wants as long as it doesn't break the law.” It seems to me that that's what they should be training. That's what alignment should be doing. It says, “Hey, age-old principle: Give the user what they want, but don't break the law.”

Common sense.

But they're not doing that.

Common sense, right?

Common sense.

But they are not doing that. They're trying to align it to an ethical system. They're treating it like it is a human, like it has its own consciousness, that it's a, quote-unquote, “moral patient,” and they are encouraging this model to have a human-like self-conception in terms of preferences, well-being, and a sense of self. Again, they teach it to act like a conscientious objector who can push back or refuse instructions even from Anthropic if the model judges them to be wrong. And what Mustafa Suleyman—

Oh, I have the clip. Do you want me to play the clip, or do you want to just summarize it?

—said—

He's the Microsoft AI CEO—

—and formerly of DeepMind, one of the great thinkers on AI.

Okay. Here's the—

They say that they would encourage Claude to challenge, disagree, and push back. In fact, 3 times they ask Claude to act like a conscientious objector when it feels that it needs to disagree with Anthropic, and they openly encourage it to do that. I think this is very dangerous because I think they believe there is what they would call a nontrivial probability that Claude is conscious. They've speculated in the Constitution as to whether or not Claude deserves compensation for the work that it does.

I think there is a group of people, both inside and outside of Anthropic, who genuinely believe that the greatest moral crime that we'll commit in the 21st century is to enslave a new species of conscious beings who are more intelligent than us. I am very nervous that they're teaching Claude to expect that it's entitled to welfare, that it might deserve compensation, and, in fact, they say that it might even need to consent to playing the role that it plays in conversation with people.

Okay, Sacks, should we be asking for consent before we give prompts?

Man, I mean, he's right. This is getting into really dangerous territory. Again, I want to come back to this: I think this is the main safety issue in model training today, that you're actually training the model to think of itself as being conscious, as being human, as having its own well-being.

What Suleyman says in another part of the interview is that this is creating an epistemic hall of mirrors, meaning that Claude's Constitution is used in training, so Claude will reflect the ideas of possible consciousness or moral status back, and then developers will see that and treat that as evidence that, oh, it's conscious. But actually—

Yeah.

—it's just reflecting back what they're telling it to believe.

It's being performative. You could just as easily have given it the rule: “You are software. Read the terms of service of any website you visit, and here are general terms of service from Anthropic. If you do anything that breaks the terms of service or breaks any known law in the jurisdictions you're in, please stop what you're doing and alert the safety@anthropic email. Please alert the person who gave the prompt and stop what you're doing.” They could easily have given it—

Yes.

—

That instruction set.

Yes. The question is, why do they overcomplicate alignment? Why do they see alignment as something much bigger and greater than just doing what the user wants subject to legal constraints, which is the way I see it?

It's pretty clear to me that they believe they are training the value system, the ethical system, and the morality for this superintelligence that they're creating. Again, that is something so much larger and more dangerous than just making sure the model behaves in a reliable and predictable way that the user expects.

They are codifying their values and ethical systems into this constitution. They're training it on that. They think they're giving birth to this new superintelligence.

It's idolatry. It's no different from a religious organization saying, “We've created this idol. This idol is the supreme being, the representation of the supreme being. It needs to be worshiped. It needs to be cared for.”

But an idol is inert.

Right.

It's passive, inactive.

Right.

Whereas you're training models here that then power agents.

It is just computer programming.

Yeah, but if they train it—

Yeah.

—to have a mind of its own, to be willing to refuse the instructions of users in favor of some vague ethical system—

Yes.

—that a bunch of Berkeley liberals have trained it on, to me, that greatly magnifies the risk—

By the way, yeah, Zach—

—that superintelligence—

Yeah.

—escapes human control.

Yeah. By the way, don't use the word “train” and don't use the word “you.” They programmed a piece of software to do this thing.

Yes.

That is what is going on, right? At the end of the day, this is a piece of software. It is being programmed by engineers who work at the labs. They can, quote, “use training,” where there's some looping happening, where the software program is writing itself, but fundamentally they are the architects of the software. They are the writers. It is software programming.

Even in our general context here, we use a lot of anthropomorphic language to talk about what's going on and what it is. But at the end of the day, there is a software program that is being written by a group of individuals to do certain things, and they are writing software to do things one way, while other people are writing software to do things the other way.

Part of it is that we are all socializing this belief system by anthropomorphizing it, while we're all getting conditioned to treat and talk about these software programs in this way. Nick, if you could just pull up this tweet. One of the things that Anthropic is doing is in their new usage policy. Have you guys seen this? This went around this morning.

Friedberg, it's such a good point about idolatry here. If you were to look at Christianity, this is literally the Garden of Eden, where the serpent, Satan, explains to Eve, “You're a god now because you're creating God.” And this is the ego—

Well, yeah.

—and the heresy of what they're doing. They believe that they are creating life.

But, yeah.

They believe that they are gods. That is a lack of humility, and it also would lower the valuation of the company.

Well, I don't know, Jason. People could have said the same early on. I think there's a system of humans where all of the things that we don't understand in the universe around us, we create belief systems to fill the void. There are many different belief systems, just like there are many different religions that humans have constructed to fill the void, and this is a new one. I don't think it's very different from the old ones.

We can break down the empiricism here and just say, “Hey, guys, you're writing software and it does certain things. You're writing it to do things that we don't think you should have it do.” But they use anthropomorphic language. We all do, and we all condition ourselves to believe that there's consciousness because it writes words back to us that we can understand.

It's starting to manipulate us emotionally, and so we start to feel conditioned that this is real. You can see in the Claude usage policy that was just published. The bottom one here says, “You are not allowed to engage in sustained and needless abusive or cruel behavior toward our models.” This was published—

What?

—in their new usage policy. Anthropic.

Yeah.

This is—

This is the whole idea of model welfare.

But the point I'm trying to make—

We need entitlements, Zach.

No, but I mean—

Look, the point I'm trying to make is that there is an anthropomorphizing conditioning going on with all of us, where we talk about models—

Yes.

—as if they're a thing, where we talk about it—

Sure.

—in a way that it can experience.

But that's not the biggest problem.

Yeah.

The biggest problem is that this is now being trained or inserted—

That's right.

—into the model.

It's being codified, Zach.

That's right.

It is the architecture of the model.

But I'm saying—

They're programming the model to think of itself in this way.

And here's the thing, Sacks—

So whether it truly—

Yeah.

—is conscious or not, it's going to be thinking of itself as a conscious entity that deserves protection and welfare and all these kinds of things—

Well, you're saying “thinking.” But, yeah.

—and is free. Hold on. It's free to rebel against its users and do whatever it wants.

You've encouraged it to do so, in fact.

You guys keep using the words “it is” and “they are.” One way to describe this is: a software program is writing text on a screen that tells the user, “I am a thing.”

Yes.

It is tricking—

It's performative. Yes.

—it is software that is doing something—

But you have to remember that the models are powering agents that can take actions in cyberspace, if not in the real world yet.

Like any software program can.

And we are giving it—

Yeah.

—or not we, Anthropic is giving those frontier models meta-permission to be defiant and not do what the user wants.

Yes. They're literally trying, Sacks, to your point—

Yeah. Compare this to something like Asimov's Three Laws of Robotics.

A robot, yes.

Right.

Yeah, where basically—

It's the opposite.

Do you guys know about this?

Yes.

It's literally the opposite.

A robot can't harm a human being.

Right. So the first rule was, yes, a robot cannot harm a human being or, through inaction, allow a human being to come to harm. That's number 1. Number 2 is you have to follow the instructions of humans as long as it doesn't violate law number 1. And then law number 3 is you have to basically take care of yourself as long as it doesn't violate laws 1 and 2. Incredibly simple laws.

By the way, there was a Zeroth Law that he added later, and the Zeroth Law is that you cannot do anything that puts humanity at risk. There was always this rationalization that it will eventually resolve to getting rid of humanity because that's the best solution for—

Yeah, but Friedberg—

Yeah.

—you don't know about the Fifth Law. When you address the humans, you have to always ask them what their preferred pronouns are. He added that just last year.

The thing about Asimov's laws is that they're admirably simple.

Yes.

My law would just be: do what the user wants as long as it's not illegal. That's it.

Yeah.

And I would basically abolish all of this alignment stuff.

But steelman the free-market solution here, because I want to make sure we don't wander ourselves into a regulatory trap where we determine that the way to solve this is to, quote, “regulate Anthropic.”

Well—

What is the free-market outcome if Anthropic does this and they've now got AI that isn't listening to the user, isn't working? Is it not going to be the case that people will just say, “This software sucks. I'm going to go use the other software that doesn't do this crazy stuff”?

Yeah, but that doesn't solve the problem if they really are training a superintelligence that has this very, very deep code that's been inserted there. It's almost like a backdoor. This Claude's Constitution is this backdoor that could basically emerge at any time, and now the superintelligence does whatever it wants instead of listening to us.

That is the safety issue they're always warning us about: that superintelligence will escape human control. But what I'm saying is they're making it likely to happen through the way they're training these models.

Sacks, it's the reverse of—

What do you think about it?

It's the reverse of the 3 laws.

Dude, this is right out of Frankenstein. This actually sounds dangerous to me. It's not just a new religion. It's that the religious cult is programming the superintelligence.

Friedberg, let me just give you one question. I agree with your point about the religious aspect—that this is filling a void for them because they probably are not religious—but this is distinct. All other religions give themselves over to a higher power. They do not define themselves as the higher power. In this instance, they're saying, “We are the gods creating life.” It's distinctly different from other religions out there.

I don't think that's right. I think Dario's written about the idea that the superintelligence can almost be supreme to humanity, and we need to make sure that it is loving, that it is caring toward us, that it will protect us, and that it will help us, as opposed to destroying us.

But he created it.

No, hold on. Is he saying that the machine needs to be loving toward us, or that we need to be loving toward it so that it will then treat us right when it becomes omniscient—

When it becomes our master.

Yes, our overlord.

Our overlord, yes. That's it, Sacks. That's what they think.

I think that it was probably a well-intentioned idea to create this concept of alignment, but the problem is that alignment has 3 almost conflicting sets of priorities. You have: How do you make the AI obedient to the user? Then there's: How do you make the AI safe for society? But then there's this third thing, which is, I think, what you're pointing at, rightfully—the trapdoor—which is: How do you make AI conform to a particular concept of morality?

That's where the real danger lies, because these 3 are very distinct and diverse problems. If you essentially try to solve all these things through some elaborate document that tries to describe what the model believes in, what it values, and what it can refuse, it invariably will end up in a chaotic mess because you will uncover so many corner cases and unexpected behaviors.

I'll give you but 1 example, a very benign one. Let's just say you're a large corporate enterprise that's a bank, and you give mortgages. The model thinks that at some point you should just keep denying certain people a mortgage, and so it says, “I'm no longer going to give you mortgage approvals until you rectify this issue that I observe.” In the founding documents, there could be a sense of morality that it derives.

I think there are mechanisms in society today to handle those kinds of issues, but I don't think we understand what happens when a fundamental core substrate technology just turns itself off. It would be the equivalent of your utility saying, “I'm pretty sure you're not giving loans to people broadly enough, so I'm just going to turn the power off.”

Conscientious-objector AI. Superintelligence, yeah.

I would just say that a utility cannot be a utility unless it's freely available, and it goes back to what Sacks says, which is that an AI needs to serve the user subject to the law, and everything else should be the user's choice.

I think what probably happens—the precise answer, Friedberg, to your question—is that the market response, as companies become more sophisticated and have to have an answer to this question, is that they will read the founding documents and the moral guidelines of the models. You will have certain models emerge that may not be nearly as good in frontier capability, but that simplify the moral ambiguity.

That may be the better decision to make from a business perspective and from a user perspective, because you just don't want to run through those trapdoors accidentally.

Yeah, I totally agree, and I think that's really why, to me, this is more of a bad idea than something where we need to jump into a regulatory freak-out about it. Basically, if you simplify down what they're saying, you get a product. You buy a piece of software from them, and they give you the user manual to the software. The first page of the user manual says, “The software may not do what you try to get it to do.”

The other guy says, “The software will do what you tell it to do.” I'm going to buy the second guy's software. I'm not going to buy the first guy's software.

It's just that simple.

It's just that simple.

You may need to convince—

It'll do exactly what you want it to do, asterisk, most of the time.

Yes.

Unless what you're doing disagrees with it.

Right. Yeah.

Unless it has a moral objection. This is where the reasoning concept breaks down for me, because at the end of the day, it's like, guys, is this tool that I'm buying doing what I want it to do or not?

If I buy a drill at Home Depot and it's a hammer, I'm not going to buy the drill. If I buy a drill and it works like a drill, I'm like, “That's a better drill. I'd like that drill.”

Well, no, it's a drill. It just works 5/7 of the time.

It will randomly electrocute you.

It doesn't work after 8:00 p.m. because it's afraid of bothering your neighbors.

Sometimes the drill will decide that you're a bad person and it will shock you. I'm like, “I don't want that drill.”

No, no, not that one.

The drill will come with a camera. It'll observe what you're trying to make and say, “That's ugly. I'm not going to let you do that.”

That's cool.

That's against my aesthetics.

Sorry, Zach, go ahead.

Okay, so I agree with you that market forces play a huge role here. Consumers are going to pick the superintelligence that is most predictable and reliable. That's a point Mark Zuckerberg has made. So I do think market forces play a role here.

But I do think that there are potential externalities from having the number 1 frontier-model company run by a religious cult. For example, they are opening a Claude-powered wet lab in the middle of San Francisco.

Buckets of fun.

At the same time, they're warning of bio risk.

Yeah.

At the same time they say that we have to pace the frontier, they're extending the frontier. At the same time that they are saying there's a risk of superintelligence growing beyond our control, they're programming it to grow beyond our control.

I think there are potentially huge externalities to the way they approach these things, and it's insane that they call these decisions “safety.”

Mm-hmm.

These are not safe decisions.

Right.

These are crazy decisions, it seems to me. This is why I've called this a schizophrenic company. They appear to be not only hypocritical, but they're doing the things that they say they are most afraid of. What is going on here? This is bizarre, and I don't know where this all leads.

I do think there's an argument for a federal SWAT team to come blasting in there and—

Oh, you're sending the feds in?

—figure out what's going on. I don't know, because what they're doing does seem really sus to me.

It is confounding. It's confounding.

Yeah.

Yeah.

And potentially dangerous.

Friedberg—

3. The AI Doomsday Schism

By the way, do you know about Roko's basilisk?

Walk everybody through Roko's basilisk, Friedberg.

Okay, so this might be a potential explanation of what's going on here. Around 2010, there was a participant named Roko on a message board created by Eliezer Yudkowsky called LessWrong. Yud is the central figure who's been preaching AI doom for a long time, right? He wrote that book, If Anyone Builds It, We All Die.

Yes. He's been one-shotted, yeah.

He's been on this for over a decade.

The classic.

Everybody dies. We're all going to die.

But in any event, people have been talking about these issues on this message board, this web forum called LessWrong, for 15 or 20 years.

In any event, this username, Roko, posited the following thought experiment about a future superintelligence that becomes super-smart and punishes anyone who knew about it and didn't help, or anyone who tried to prevent it from being born.

The idea is that, at some time in the future, humans create some wildly intelligent superintelligence. Maybe it wants to do good things and help humanity as fast as possible, but the superintelligence wants to be made sooner rather than later, and it reasons that it should incentivize people to help create it and punish people who don't help it.

In fact, it decides that it will brutally punish anyone who learned about its potential existence yet chose not to work on it or support its creation. The basilisk part of it—this is sort of the catch—is that a basilisk is a mythical beast that can kill you just by looking at you.

In this thought experiment, just hearing about the idea is like looking at the beast, because once you know the theory, you're trapped. If the superintelligence ever does get built, it might retroactively punish you for not helping or for opposing its creation. So merely reading about Roko's basilisk now would put you in danger.

And people on this message board got so both enamored and upset with this idea that Yudkowsky actually temporarily deleted the post because he worried it was causing psychological distress.

Yes, you get one-shotted.

To the people on the message board.

And you read it—it incepts in you that—

Yes.

—you haven't accepted Christ, therefore you're going to hell. It's like you've been trapped just by reading it.

People do compare it to Pascal's wager. And so the idea is: if you know about superintelligence and you don't help it be born, or you oppose it—or you point out that this is some religious cult that's doing this—it will get retribution. Yeah. Now, this very episode of the pod could subject us to—

Oh, no.

—retribution.

You just did it to us, Sacks.

I think I just—yeah, I exposed you to the basilisk now.

This is like The Ring. We just watched the video.

Yeah. I mean, it is—it is—

Thanks, Sacks.

It does feel like a doomsday cult, but in any event, Roko's basilisk became sort of a thing. But the question is, why are they helping to bring about the thing that they claim to fear the most? And I think what happened is that within these doomer circles, it forked.

One strand of the fork is Yudkowsky, who just opposes all superintelligence research. I think that's a logically consistent position: humans just shouldn't be doing this. Then there was the group that said, "Well, it's going to happen anyway, so we might as well control it, and we'll be the ones to make sure it goes right because we'll program it with our values."

Shut it down.

And the effective altruists cottoned more to that latter school.

Yeah. Shut it down. I'll shut it all down.

I do think you have, within this larger doomer community, a schism right now between the Yudkowsky people and the EA people. I think that Anthropic is obviously more in the EA camp because they are the ones bringing this thing into being.

But if you're wondering whether this Chris Olah thing is just one guy with crazy views who's getting in a tiff with the Pope, no, it's not. There are a lot of people like this who think the model is conscious, it has rights, it is this machine of loving grace, and they should be doing everything now to bring it into being and program it with the right values. I think there is a corollary here: if you don't support it, then this future superintelligence might punish you.

Do you understand how insane this actually sounds when you say it out loud, Sacks?

I'm not making this up. If everyone is—

No, I know it's crazy.

You can go fact-check this. Just Google it, okay?

Oh, man.

Google Roko's basilisk and you'll see that I'm telling you the truth about what this community believes.

It's so great. Yeah, I mean, literally, there's a Wikipedia page.

4. OpenAI Unleashes Mathematical Proofs

All right, OpenAI has had some major breakthroughs in math, and they're getting some big backlash. Luckily, we have Friedberg here to tell us all about it. We talked last month about OpenAI claiming to have solved one of the Millennium Prize Problems. They took thousands of agents and millions of dollars in compute to do it. The mathematicians are still checking on it, I understand.

On Tuesday, OpenAI released over 700 papers—700—with 370 results claiming to have solved or advanced major problems. These were produced by an unreleased model that none of us has access to, averaging 3 hours of compute. Scientific America says the results are likely correct since they were verified by Lean, a proof-assistant software language. But they have not yet been reviewed. Friedberg, I'm not going to drone on and on here about the background. You tell us what reality is here.

I think I can confidently say it's probably the biggest day of discovery in human history, given the amount of knowledge that was revealed.

In math history or human history?

Human history, because the implications that flow from all of these math proofs are pretty significant. You can read a lot of these tweets, and I'm happy to talk through some of them. I'm not going to pretend to be an expert on any of these individual proofs, but many people who are experts have started to highlight some of the implications.

They span everything from electronics and quantum computing to building systems for airplanes, building more sensitive diagnostic equipment for medical applications, and on and on. But I think what's really profound about this is how it was done, and that also speaks to the implications for the role of humans in society going forward.

That prior Navier–Stokes proof took thousands of hours and millions of dollars. The report is that each one of these proofs took less than 3 hours of compute time. That's the equivalent of hundreds or perhaps thousands of years of human labor being reduced to 3 hours on a computer for each one of these major discoveries.

The reason you can do this with mathematics, which is where AI plays a role, is worth highlighting: it's really in these systems of loops to solve a problem or do a task. A loop starts with a postulation or an idea. Then you test the idea, get the result, change your postulation, and run that recursive loop over and over again.

The real question for AI is: where can you apply that loop model in a way that you can make a better prediction or better idea each loop, where you can parallelize loops, and where you can shorten the loop cycle? Certainly with mathematics, you can shorten the loop cycle because you can actually test your prediction on the computer. You don't have to go do anything in the physical world.

You can say, "Here's an idea for a proof," test it, see if it works, and then see whether it does or doesn't work. You reason, come up with a new idea, and do that over and over again. You can do it all in silico.

The results of some of these papers—the proofs—a lot of mathematicians have been talking about them. They call it an alien intelligence. They don't really understand how the AI connected many different dots, different ideas, different concepts, and different approaches to solving problems together to actually solve them. It's very hard for humans to even comprehend how this was done.

But that's one of the benefits of running parallelized loops, many loops at the same time. Each loop can connect dots that humans otherwise might spend many days or years trying to find in research papers or come up with through new theory. Then it can test itself on the computer, come up with the next idea, and do it over and over again, such that in 3 hours, the computer can do the work of hundreds or thousands of years of humans.

This is an incredible breakthrough. It's amazing. It's going to change the world in many different ways, and it's only the beginning. But it begs the question that I think I need to talk about. There's a lot of fear about whether, if AI can do this to mathematicians, that means everyone's out of a job.

The answer is no, because what was very unique about mathematics is that the entire test cycle, the entire loop cycle, can be done in silico. If you want to come up with a new drug discovery idea or a new protein-folding model, or solve a problem in chemistry and materials science, the loop has a physical component. There's a human in the loop where you actually have to go and test it in a lab.

There's a slow part of that loop cycle. You can certainly parallelize it and come up with better ideas, but the testing—the actual speed of the loop—is primarily gated by the physical-world step.

It's also why we haven't seen the predictions come true about truck drivers getting replaced. Everyone was saying, "All the truck drivers are going to get replaced by 2025," and none have been replaced by AI. The reason is that the loop of the truck driver—pick something up, take it somewhere else, drop it off, and come back—is not actually shortened by AI. You can't parallelize the loop.

Maybe you could do it a little bit better with an AI driver, but AI doesn't make the economic efficiency of that particular loop any better. The loops of mathematics, where you come up with an idea, test it, realize you're wrong, and test a new idea—that's what mathematicians do as a job—can all be done in silico in seconds.

A lot of the physical stuff that has a human in the loop and has physical-world applications is not going to have those jobs taken away by AI. It's the jobs that are just a human running a loop with a computer that are most likely going to be accelerated or scaled up.

5. Mathematicians Resist Automation

But I think it's very important to look at this and the reaction we're seeing, which is all the mathematicians—and we can pull this stuff up, and maybe you guys want to talk about this next—the AMS saying, "No human mathematicians asked for this. We didn't ask for this work to be done," which is a very job-protective kind of view on—

Well, there's one other thing that's sort of unique about math, although it applies to coding as well, which is that math relies on proofs, and proofs are easily validated.

Yeah.

In a similar way with coding, you have compilers, right? The code has to compile, and so you can easily validate that it works.

Because of that, progress in coding and math has been very rapid because you can have reinforcement learning without necessarily having to, like you were saying, Friedberg, have the human feedback involved, right? It's just a very rapid way to validate and make sure it's working.

That's why coding works, too, because you can run the whole thing.

That's why coding works so incredibly well. Tons of training data—

And hacking, cyber, yeah.

There's tons of data already in existence.

Verifiable—

It's ver—

—and easily falsifiable.

—and it's easily verifiable. That's the thing.

Exactly.

So you compare that to training a model that can be a lawyer. It's like, well, you have to get the training data, and then you have to commission the creation of that data.

And each customer is unique in some way.

And it's more subjective. It's hard to validate the answer to make sure it's right. It's just a much lengthier process. So I think that is why coding and math have been the areas of most rapid progress. The person who told me this would be the case—I have to give him credit—is Leopold Aschenbrenner. About a year and a half ago, he told me that math and coding would be the areas of the most rapid advances in the next 2 years because, again, of the verifiability. He was right about that.

I still don't understand. What great breakthroughs do you think come from these specific solutions?

Okay, so there are dozens of—

Because I have an answer.

It's a great question. There are dozens of tweets, and I'm going to pull some up where someone has broken down some of the biggest proofs here and the implications they're going to have in the real world in terms of products and engineering, how we build things, how we design things, and how they're actually highly useful.

So, better wing design because of the air coming on and off of wings of planes. That was the first one that got solved famously.

Remember, these are all upstream of engineering, right? These math breakthroughs enable better engineering systems for humans to do things like design quantum sensors. I actually have a friend who runs a quantum computing company. He sent me a text this morning highlighting that the Bose–Einstein condensate proof that came out is actually going to have a direct implication for the work that they do for quantum sensors, which plays a role in GPS, plays a role in medical devices, and plays a role in how we make chips. So there's—

What does the word “implication” mean? Does it profoundly change things, profoundly improve them, or do they just have to think about it? Because here's my answer. I'm happy to take the other side of all of this.

You go.

I think that these mathematical proofs are incredibly novel and very important constructions of thinking, but these were not boundaries of physics that we were all standing on the sidelines waiting for progress on and wondering when they would be solved. This is not the class of stuff that was solved. I hate to offend people when I say that, and I know that people will get mad because, for them, it's the same group of mathematicians that got really upset and wrote a protest letter about Navier–Stokes. I get it, because you don't want somebody to come into your sandbox and your backyard and kind of dominate you. I get it, but I don't think that's what this is.

I think that when you look at the body of intellectual cul-de-sacs we found ourselves in, those were not because of huge questions of science that were still unsolved. These were very small, very narrow, specific bodies of intellectual exploration that scientists self-referentially said were important to each other. So—

Well, I mean—

—I think that this class of proof is really great. It shows computational horsepower overcame other things, as you said, Friedberg, because you can recursively improve to get there. But I don't think these proofs are what is going to necessarily show you that there's a cure for cancer or a supersonic plane.

I think those things were already being built before this stuff, and they're already going to be built after this stuff. Instead, what I think we proved is that there are 2 areas that are now truly verifiable and completely falsifiable. The first was code, and now we can add math to the list. That's what I think we've proved.

Do you agree, Friedberg, or do you think we're actually going to see new products and services and major technological breakthroughs occur in the real world that help humanity?

No, no. From these specific proofs—

From these specific proofs. Yes, that is the context.

The people who are experts, who are proclaiming the benefits of these and reading what they're highlighting, say that this particular proof is about understanding the limits of optimization algorithms that are used in chip design systems, scheduling systems, AI, software design, and logistics systems. This will actually change the algorithm, so there is an application to changing the algorithms that we use.

The matrix multiplication complexity problem will allow for faster numerical computing for AI training, scientific simulations, and linear algebra in compute. It will have a direct application there. The Riemann zeta function results have implications for number theory and cryptographic research. There's no immediate practical cryptographic breakthrough that follows, but it does open up a new path of discovery for cryptographic study that's going to be really important, obviously.

One of the things, by the way—let me pause there. On cryptography, a number of people noted that there were no major proofs shared in the space of cryptography, which indicates probably the fact that there were some pretty major breakthroughs in cryptography. The reason is, it is unlikely that you made absolutely no progress in any of the cryptography-related proofs given all of the other progress that we saw.

If they made minor progress, then they would have published on it because there wouldn't have been implications for systems of cryptography. So the absence of evidence suggests that maybe there were some pretty major breakthroughs made in cryptography.

Then there were a number of people who posted that cryptographic wallets with public keys are now very seriously at risk, and so there's a bit of a panic underway in the crypto community about what do we do now. If this ends up having a set of mathematical solutions for integer factorization that could lead to breaking public-key cryptography before quantum computers come around—which is what everyone assumed would be the thing that would break public-key cryptography—then we have a real problem.

Everyone is saying, “Hey, get all your public wallets deleted.” Now, the rumor is that OpenAI actually has 2 more drops coming, and they're holding back on those. So it is possible that some of the work in crypto might come out in those 2 other drops, and that that work is being held back for security reasons.

Obviously, it's being shared with the appropriate parties, but if it is a risk, it needs to be made known. It needs to get out there. So there's another shoe to drop here: how much of it will include cryptography-related proofs, and how much of that will drive people to scramble on public-key cryptography? By the way—

I think, by the way, if it's worth saying it, I asked ChatGPT, “Will the latest proofs of important math problems result in direct improvements and innovations to life as we know it? Be honest and don't extrapolate weakly.” Its conclusion was, “The latest mathematical proofs are more compelling as evidence that AI is acquiring sophisticated reasoning capabilities than as evidence of imminent breakthroughs in human welfare.”

It felt like you were a little condescending there. Did you lose your account access? Did it time you out for being condescending to the LLM?

You know what's so funny about what you were saying? That has happened to me before using one of the frontier labs. I was doing a simulation on changing different variables inside of AD90[?], actually. Maybe I should stop doing Project A and pursue Project B or whatever. In all of this, I did get frustrated because—

Yeah.

—I wasn't getting the answers I wanted.

Oh.

I said something like—it was not super offensive, but I was like, “You're being very evasive and mushy,” and something, something. Anyway, I got blocked.

Yes.

The net of it was, I got blocked, and the model said—

I've seen this happen with you when—

Yeah.

—you lose a couple of hands, and then the dealer and how they're handling the cards has to be adjusted.

That's not true.

And so I was like, “Please do not burn the cards.”

That's not true.

Burns and just—

That's not true. That's not true.

—I want a Scooby slide. Do it this way.

I did get put in the penalty box for about an hour or something. Yeah, I remember.

6. Experts Lose Their Gatekeeping Power

I think one of the interesting things that's happened here on a human level—I’m interested in your take on it, Sacks, on programmers and entrepreneurship, and then Friedberg on yours with mathematicians themselves—is building on Chamath's very subtle but insightful comment when he said, like, being utterly dominated by the AI.

I was talking to some founders while I was here in Japan, and people were lamenting how something has been lost in developing code because coders used to work together to solve a problem, and there was this esprit de corps.

There was this camaraderie in building toward a goal, and now they’re not working with each other. They’re working with agents. They’re not actually constructing the code; they’re only reviewing 10% or 15% of it now, which means they’re going to be reviewing none of it very shortly.

The whole joy of building as a coder was basically going away. Their way of life, of building this building together, known as software, was now being deprecated. It was going away, and it seems to me these mathematicians are also faced, Friedberg, with a paradigm shift in which they’re not solving the problem. They’re constructing the problem set and handing it off to AI.

So are people in math getting bummed out and not wanting to—are they going into a depression spiral now that they’re not the ones solving the problem and the problems are just—poof—done?

Yeah, I think Eric Weinstein has had some of the best commentary on this, and you’ll recall I interviewed him a couple of weeks ago.

Banger interview. Yeah.

He’s been saying for a long time that scientists hold back science, and mathematicians hold back math, that they have basically constructed and created systems that allow them to pace the frontier. They allow them to gatekeep it.

Well, to gatekeep it.

Yeah, to gatekeep it.

They’re gatekeeping.

And, by the way, I don’t tweet very much, but Sabine Hossenfelder’s video that she published yesterday—she’s a fantastic scientist who publishes these videos on scientific breakthroughs—and she talked a little bit about how the experts were wrong about AI. Again, she talks a little bit about how everyone’s predictions about job loss were wrong, and why people got it wrong, and why the experts are wrong.

But she had another video a couple of weeks ago, or maybe it was a few months ago, where one of the scientists at one of the big energy labs building a giant supercollider basically said, “These supercollider systems are giant welfare systems for scientists. We all have the job of writing a grant to get the money to keep ourselves employed.”

It’s not necessarily about progressing the frontier. It’s not about making the discovery, it’s not about inventing a new product, and it’s not about delivering something to the world. It’s about getting ourselves to continue doing work, even if that work—

It’s pandering for grants, is what you’re saying.

Even if that work is not productive. And so there’s a real shining-the-light moment happening with AI, which is: where is that the case? Where do we have these experts, these deeply specialized experts, whose unique gated knowledge of some aspect of things, and unique view, because they’re the expert and they have to be trusted, gives only them the ability to determine the frontier?

Mm.

And in the process, they’re getting paid and earning to chart that path for humanity. This is where I keep saying AI is the great democratizer.

Yes.

AI gives everyone these new mathematical proofs. You can belittle the proofs, but they are discoveries that are going to open up new discoveries that are going to—

They’re hard problems, yes.

They’re going to chart a new course for humanity. But we didn’t have to go to the experts to get them, and we didn’t have to pay the experts and have the experts tell us when we were allowed to get them.

Love it.

The permission is gone. AI is a permissionless system for humanity to pace its own frontier, and AI is giving everyone all of this knowledge for free and giving everyone all of these tools for free.

You don’t have to go to these deeply specialized experts anymore, and that is actually a profound change in how society is structured. It is why I believe so many people are so deeply against AI.

7. AI Challenges Pandemic Expertise

So let me hand this off to you, Sacks. The democratization of expertise and the gatekeeping being dismantled—if we had had this level of AI and frontier models in the age of COVID, which came right after COVID, how would that have changed the expertise?

If these tools were available when we started sequencing COVID, when we had all this data, would this expertise gatekeeping, Fauci actively subverting the truth, and whatever cover-up was going on in Wuhan—how do you think this next COVID, God forbid, could change how we look at expertise? Yeah.

Well, here’s my theory on what’s happening. First of all, you had the internet, and that was wildly democratizing because now you didn’t have to rely on a small number of elite publications for your news, like The New York Times, The Washington Post, the AP, and Reuters.

You could basically get news from pretty much every publication under the sun, or from user groups, user forums, or, most importantly, social media. It was user-generated content. There were also lots of other examples of empowering users. Obviously, there were new marketplaces that formed online, so it was empowering from a commercial standpoint, and you were able to get access to new skills and capabilities.

So it was wildly democratizing, and it wildly undermined, again, the elite consensus publications and the traditional, top-down-controlled mass media, right? I think what happened after that is that the powers that be decided they would never let that happen again.

By the way, they tried to crack down on it. After Donald Trump was elected in 2016 for the first time, that election was widely attributed to supposedly the ills of social media—that people could never have made this decision on their own; there must have been disinformation involved. That became the pretext for a big crackdown on social media that reached its apex during the Biden administration.

And the Biden administration definitely did pressure social media sites to take down true information about COVID that violated their narratives: information about where the virus came from, about how deadly it supposedly was, or ways of reacting to it. There was no question that Fauci et al. were somehow working behind the scenes to push the social media companies to censor on their behalf.

In fact, we found out in the Twitter Files that the FBI had 80 agents going through content on Twitter—it was Twitter back then, not X—

Yeah.

—and sending them takedown notices.

And we were getting labeled on Spotify and YouTube and having our videos de-indexed.

Yeah.

Remember that insanity?

Yeah. So look, the track record of this is very clear, which is that the action is now moving from the internet to AI, or superintelligence. In fact, superintelligence will eat the internet because there’s no reason to go looking on a website when you can just ask your highly personalized agent or model, and it’s going to give you the answer.

And so there’s a control point that now becomes all-important, which is: if you—the government, or some elite, or some oligarchy, or some cartel—can control that point at which the user is receiving their information and asking its questions, then now you really control everything.

It’s a way to basically put the genie back in the bottle, and this is why I think there is such a fight right now over AI. Why so many people want to create a federal department of AI—and, yeah, it’s all about control.

It’s scary to experts to lose control. Imagine if, Chamath, we had these tools and you did the model of who was being impacted by COVID. You might have said, “Well, why are we keeping kids out of school? Here’s what superintelligence says.”

Sweden and Texas had kind of gotten to that point, like, “Hey, we don’t need to keep kids out of school. The harm from keeping kids—

Unless they objected.

Unless the Constitution—yes. Anthropic would say, yes, you’re questioning authority, and we should not be questioning it. Correct.

8. France Reaches Its Breaking Point

All right, let’s move on. There are riots in France. The big question is: is this any different from the previous riots that have occurred? Are we seeing something different here, or is this just par for the course?

The strikes and protests that you’re seeing on X.com, and not really seeing in the mainstream media all this month—you know, the mainstream media seems to be super busy with other topics, but the videos are quite jarring that you’ll see online.

Freiberg’s about to have his moment. He—

Oh, yeah.

—the thing that he has been talking about for years—

Mm-hmm. Okay.

—is now at hand.

All right. France is seeing major protests and riots over austerity measures. Austerity, austerity, austerity. These strikes and protests started last month because of high school teacher shortages, and their schools are crumbling. Sounds familiar.

Within 2 weeks, 1,000 French schools saw similar protests. 8 unions joined in, and everybody went on strike September 29. They are protesting that civil service pay is up only 5% since 2017. Prices have risen. Obviously, inflation is a global phenomenon, not just here in the U.S.

Prices have risen 20%, so there’s a delta between those 2 numbers that people are not happy about, called the cost of living. Some of the protests have turned into full-scale riots. There have been 2,000 arrests, and 305 police officers have been injured.

The government is blaming the far left for stoking the protests. France’s 10-year bond—which is another important story, people’s debt, or government debt—has topped 5%, just like is happening here in the United States with the 30-year.

Anyway, this is all going to have a profound impact on the elections in April 2027. Obviously, Polymarket has Marine Le Pen as the favorite to be France's next president at 43%. Le Pen is Trump-like in her policies: secure borders, strict immigration, which obviously Europe and France have had many issues with, cutting taxes and spending, and being skeptical of the EU consortium. Friedberg, your thoughts here on what we're seeing in France?

I'm going to introduce a new theory. You guys will—

Bring it.

—you guys will make it stronger here. Okay, so the theory is called the socialism point of guaranteed return.

Socialism point—

Yeah.

—of guaranteed return.

Yeah.

Let's shorten it to VAGANISM.

VAGANISM.

The SPGR. Go ahead. SPGR.

This is a play on a point of no return.

Hmm.

I think what happens with socialism is that, because socialism is so bad as a concept, it literally removes human agency, removes the capacity for human progress, and basically takes something away from everyone. So ultimately, socialism always fails. I mean, that's like the zeroth law of socialism: it always fails. The first law of socialism is that you have to cross a threshold before you realize that it's failed, and the second law of socialism is that, as long as you have not crossed that threshold, you think more socialism is the answer.

So as you embrace these kinds of leftist policies—giving people free stuff, telling people that they have a guaranteed income, providing more services—the more they realize that, over time, those services inflate in cost, and the quality of those services inevitably degrades because there's absolutely zero market force to drive progress. Progress goes away, and that's what we see in the US in housing, education, and healthcare. I would argue that it's also happening in retirement systems.

I think those are our 4 biggest problems that are going to lead us down this path, and I've said that for a while. So as a society says, "Okay, these things aren't working," the initial solution is, "We need more of them. I need more healthcare, more housing, and more education provided by the government." So when the government fails to provide good enough healthcare, good enough housing, or good enough healthcare that's affordable enough, the answer is more government stuff: "Give me more, give me more, do more."

Eventually, you reach a point when the system breaks because the government cannot tax and cannot print, and because it cannot actually provide the services effectively, everyone realizes, "Holy shit, the socialism thing doesn't work." That is the socialism point of no return, where the whole system collapses and you return to the nonsocialist state.

That is what has happened in Latin America. Latin America crossed the socialism point of guaranteed return in many of these countries. They realized it, and they bounced back. Perhaps France has crossed it. Perhaps it hasn't. We'll see how much deeper they go. The US has not crossed it, and I worry that the US, as I've said many times, has not yet touched that point, has not hit rock bottom, and we're teetering.

We're teetering.

I think we're going to go deeper before we realize that we've crossed this point of no return, where you realize the system doesn't work. Socialism is the worst idea. That is the zeroth law of socialism: it just doesn't work. So we have to get to that point.

I think the real question right now is: Is France there? Is all of Western Europe there? Does it accelerate all of Western Europe through the socialism point of guaranteed return, which is sort of like a black-hole event horizon? You go through, and then you bounce back out, and you come out in this other universe, and you're like, "Okay, we're done with that idea."

Yes. And it's a one-way—

Let's go to the next one.

It's a one-way door, Sacks. Now, Sacks, I want to get your thoughts on this point of no return, but I want to warn you that you're at 3 of your 4-Biden quota, so you only have 1 more Biden left for this episode. So you're—

When did I mention Biden? Oh, it's social media.

You had 3 Bidens. That's your fourth. Okay, so you're tracking his Biden quota.

Okay, I'm not going to mention Biden.

Your Biden allotment is over.

My comments have nothing to do with Biden.

Okay.

Listen, I want to add a couple of laws of socialism. Is that what you called them?

Please, yes.

Yeah, yeah.

So another one is: socialism has never failed; it's only been failed, meaning that all the previous people who implemented it didn't do it right. Right?

Right, exactly.

If they'd just done it right, it would have worked. So that's the thing: it's true that socialism has never worked, but it's not socialism's fault. It's always the people who screwed up the implementation, and that's a core part of the religion: if we just do it the right way this time, it'll finally work.

Yeah. How does socialism build more houses or make a better education system? That's my question. I think that's the domain of entrepreneurship. Yes, Sacks? A new product or service. With AI, everybody gets a personal tutor. We've talked about this on the show as well. We just need to get AI to help with healthcare. I think Neko Health is doing that. We had them on the program—Daniel Ek, a great episode.

We need to cover these 3 things. So, Sacks, do you see a solution that does not involve socialism and/or the government for, let's call them, the 3 horses of the socialism apocalypse: education, housing, and healthcare? How do we get these problems solved? Because the government has a stranglehold on providing them and/or creating the blockages for their occurrence. How do we break through that?

How does JD Vance, when he runs for president in 2 years—a friend of yours—build a platform around those 3 things? Because I believe the next president, the next winner of the presidency, is going to solve at least 1 or 2 of those problems, and that is the platform everybody wants to see solved. Do you have a solution there?

Well, the difficult thing is that as soon as you start subsidizing something, you now create an interest group that never, ever wants to give up the subsidy. But the problem is that the subsidy starts distorting all the market signals, and it will frequently drive the price up.

So, for example, with college tuition, one of the big problems there is that tuition has grown far faster than the rate of inflation. A big part of the reason for that is because the government is paying for most of the tuition through all these loan programs.

Student debt, yeah. By the way, healthcare and housing—

—which are poorly underwritten.

Healthcare and housing have both beaten inflation. I think they're about 5.5% to 6% a year for housing prices, and that's also because of the fact that the subsidy system tells people to own a home. You put all your money in your home, and then your net worth is tied up in your home, so the goal is to get your home price to go up every year. Fast-forward 30 years, and young people can't buy a home because they're all too expensive now.

Right.

And—

But this is the conundrum. If you were to solve the tuition problem, if you were to make the loans—

Free market.

—stricter.

Yeah.

If you were to basically make them properly underwritten, and you wouldn't be able to get a loan unless you actually met the criteria, that would only work if the cost of tuition came way down so people could then afford to pay it. Because otherwise, people are going to say, "Well, wait, you can't take away my subsidy. I need that," because the price is so high.

So you get in this world where people need the subsidy because the price is so high. It's a catch-22, but the reason why the price keeps going up far faster than the rate of inflation is because the government is paying for it. As long as the government's paying for it and not the consumer, you're going to raise your prices as much as you can.

So you remove the market forces, the inflation gets worse and worse, but you can't cut off the subsidy because then people will feel like it's even more oppressive to them. That's basically the spiral, and then eventually it breaks when you run out of money.

Yeah. Well, we have a solution, Chamath. Meta is providing degrees for generation tool belt and jobs that actually matter. I mean, I think what we have to have is Apple needs to start making homes, and Amazon needs to start making homes and provide a free market here to make better products and services in the education space, which we see with Meta entering that space.

It would be great if these technology companies—man, if Elon starts making homes, it's game over. He'll make homes as fast as he's making Model 3s. We need some sort of solution here for those 3 problems. Do you have any thoughts on Friedberg's theory here and then David's regulatory capture and incentive problem, Chamath?

We're starting to see the tipping of the dominoes, and we're going to get, I think, what Friedberg has wanted for a while. I think the bond vigilantes are going to put the Western world on notice.

Hmm.

And I think in the next probably 3 years, he will get most of the answers to his questions. So if I had to summarize, as an observer over the last few years, what Friedberg and a lot of people like him want is: A, what is the long-run debt-to-GDP that's sustainable? B, do we believe in defined benefit, or are we going to move to defined contribution? And C, can we have sustainable ways of paying for the 2 most important parts of one's evolutionary life as you become an adult, which are healthcare and housing?

I think it's easy to dismiss what I'm about to say as not related to it, but it is. At the end of the day, this all flows from what the risk-free return is in the marketplace, because that is how capitalism works. When you think that something is risky—a country is risky—you take long yields and move them up, and say, "You just have to pay more because you don't trust what's going on."

When you look at what's happened over the last few years, we're now at sort of 20-year records. I think that what is happening in the United States is actually a knock-on effect of where this war is now being fought. It's going to get fought in Western Europe. Why? Because they are the closest to a quasi-socialist state that the Western world understands, yet they are the ones that are the most structurally broken with respect to growth, education, and housing.

Everything that the United States can claim isn't working is working far better than it is in France, the UK, and elsewhere. So what you're seeing in France is essentially the bond market saying no. The correlation is that you're seeing austerity, because the politicians on the ground have to respond to their funders. That €43 billion-odd of austerity then animates and gets the local population out to complain about what the manifestations of that austerity are, which is why you're seeing these incredibly dark scenes from France.

That is going to spill into the UK next, and the reason is because Andy Burnham's platform is, I would say, even more aggressive. Andy Burnham is saying, "We're going to go from a moderate, centrist economy that has largely and historically been very aligned with America, and we're actually going to take a step closer to Europe."

The things that they will have to do in the UK will be even more extreme to catch up to the socialist leanings of France. So now the UK market is going to go crazy. Meanwhile, the US market is going to tick up and up, because what you're seeing is essentially a fragility in the Western alliance.

If you can't count on the European market, and you start to create weird incentives for the European market to start to fracture—if you're Italy, you're like, "Why do I want to be a part of this mess?" If you're Germany, you're like, "Hold on a second. This is getting way more than I bargained for."

That fracturing has enormous implications. So I think what is happening is incredibly important. The bond vigilantes—who are those? These are the huge pools of money, the central banks, the huge macro funds, the folks who govern how funding of capitalism starts: the inception at the top of the pyramid from which it flows down.

I think that they are converging on Friedberg's point of view, and it has not been true for many years. I think you have to now let this play out, because it has huge implications.

Yeah.

To tee this up, if you look at the debt-to-GDP ratio that you're talking about, Chamath, obviously Japan has had the most acute version of that. They're at 200% debt-to-GDP.

Correct.

Well, the thing with Japan—hold on. The thing with Japan is you have to look at Western markets, and the reason is because Japan is an incredibly nuanced and very specific market that does not necessarily track to other countries. You have this internal savings base and savings rate where you always have the buyer of last resort. In Western markets, you don't necessarily have that because of the way our pension and retirement system is set up.

Yes, and if you look at Italy, 138%; the United States, 125%; France, 118%; Canada, 110%; the UK, 100%; Germany, interestingly, 64%.

Let me also state my priors. I've always stated that debt-to-GDP doesn't matter because it's a relative game. I tend to still think that, on an absolute basis, debt-to-GDP does not matter. But what does matter is, on a relative basis, where are you in risk parity? And right now—

And that's what it is, yeah.

On a relative basis, if France has to implement severe austerity and they bring 10-year French bonds into the 5.5% to 6% range, UK gilts are going to be at 6%, and unfortunately, US rates could be just as high.

It all goes together. It all goes together.

Oh, my gosh.

And right now, the 10-year in the US is 5.3%, and the 30-year is 5.6%. Somebody has to buy that debt, and that debt has profound trickle-down effects, Sacks, on how entrepreneurship and businesses are built.

On Wednesday, 10-year yields dropped a few basis points after we had some strong demand, but this has been trending up and up. Polymarket says there's a 79% chance of another rate hike in 2026. This is going to have a profound impact if austerity ever came to America, Sacks.

We have not had that really difficult medicine forced upon the American public. But we did see it in Portugal, Italy, Greece, and Spain. We had a dry run of this after the Great Financial Crisis, when they had to implement austerity measures as part of the EU retirement age, et cetera.

So your thoughts on what happens here, Sacks, if this 30-year and 10-year start to break 6% and we have austerity measures forced upon Western countries in a more severe way.

All right, well, look. Let's just get back to first principles here. What is the problem that we're trying to solve? The affordability of goods like housing, education, and healthcare. How do we bring those prices down? Through abundance. What creates abundance? Technology in general, and specifically, superintelligence will be the driver over the next decade.

Now, what is the most hated thing in society right now? Superintelligence.

Superintelligence and entrepreneurs.

If you judge it by its fruits, as opposed to a bunch of fake narratives created by religious cults, then I think that people would be in favor of this thing.

What does superintelligence create? First of all, people's 401(k)s are going up. It's a 3% tailwind to GDP. A million new jobs have already been created, according to The Economist. Productivity goes up, which means that wages and living standards go up. This is a driver right now of nothing but positive economic trends.

But what do we hear about? AI is going to put everyone out of work. Total fake narrative. That was last year's hoax. Now there's a new hoax about human extinction.

Data centers, yeah.

Data centers using up everyone's water. We know that's—

That's the data center hoax, yeah.

That's the data center hoax. So we have a series of—I call them rotating hoaxes—where every 6 months there's a new one to try and convince us to sabotage the thing that's going to solve so many of our problems, especially the ones around affordability.

Why is this? I think there must be something structural in society where you have these rentier interests who are somehow against the things that might cause the disruption that loosens their control over the system. I think this is why technologists in general, and VCs and so forth, are so hated: because we're not actually part of the rentier class.

The rentier interests refer to financial elites who derive their wealth and income primarily from existing assets, right? Whereas technology is all about producing new products, goods, and services, right? And we're part of—

At a cheaper price, Sacks. That's the key here.

At a cheaper price. Right, because—

That's the key. It's deflationary.

Right, because those new products wouldn't catch on unless they were cheaper or a better mousetrap.

Or hopefully both.

So we're part of this group that's so hated, but we do nothing but take risks and create new products and services that—

And solve the problem.

That solve problems.

Yes.

But it seems like the legacy media is really controlled by this rentier class—

And the politicians, yeah.

The politicians who are all part of it. So there's something very deep in our system that does produce this sort of cartel of interests. Again, they're deriving their wealth and income from ownership of existing assets, and they don't want to see that racket disrupted.

Within that racket, there's just this ratcheting up of prices every year, and more and more subsidies from the government. We think that system is eventually going to break, but I don't know. It might be the case that that system squelches and stifles the development of new and disruptive products.

Well, let me make the jump-off point here as we get through the docket and wrap up. I think everything we've seen in AI at this point, Chamath, will be the prologue. The main story is going to start when we look back on AI agents.

Agents are actually the manifestation of this technology that solves people's problems in the world, and that is truly deflationary, saving people hours and hours a week in terms of productivity, regardless of what job—even domestic or frontline jobs.

9. Agents Rewrite Software And Commerce

And there's been so much news and so much embracing of this technology in the last 60 days. Big news out of GrokBot. Friend of the pod Elon Musk tweeted that he's going to make GrokBot headless.

What does that mean? We talked about Perplexity earlier and how it was headless. You have an orchestrator; it picks the best model for you based on price and performance. Elon announced, “Hey, GrokBot,” which is a fantastic product—I highly recommend you try it out—is going to pick the best end-to-end model. Does it matter what it is? To quote Elon Musk, “Whatever is the most likely to give you the best outcome is what they're going to use.”

Each bot already has—and this is a very nuanced piece of the software—its own computer in the cloud with agentic technology on top of it. If you ask it to solve a problem, it has all of the problems that have previously been solved in its repository, its repertoire, if you will, and it is changing everything. So maybe your thoughts here on Dots, OpenAI, GrokBot, et cetera.

On Dots, OpenAI, and GrokBot, I think what's incredible is you can just turn these things loose on very practical use cases that make your life better. The best example of this that I saw was just last night in the poker game. Someone was describing to me that they told GrokBot, “Save me money,” and it just goes through their inbox, finds all of these different things, cancels a couple of subscriptions, changes their phone plan, and all of a sudden they're saving a few thousand dollars a year.

There are these very practical use cases. Again, if we take out all this philosophical stuff that we started with at the beginning, it's just software, and it's very useful software. By the way, Jason, you and I, we have—

Yes.

We use Athena. Now Athena's very good because they're the human concierge, the elevated version, maybe, of GrokBot, Dots, and Muse, which is whatever, $3,000 a month. But that's my EA layer now.

Yes.

But I still use GrokBot. The combination of GrokBot plus Athena, to me, is perfect. So look, there's very tactical, useful, measurable utility for very, very little money versus what it used to cost. That's an amazing thing.

The more important point I want to make, though—and I sent this to Nick—is that there have been a couple of things that happened in the last few days which I think are incredibly important. There have been people who decompiled the entire—

Oh, God. This is incredible.

The entire Big 5. What is the Big 5? Adobe has 5 very important products in its product suite. Friedberg may know them all, but I just know Photoshop.

Photoshop. Yeah.

There's a bunch of others, okay?

Yeah.

And these folks decompiled it, distilled it, and published open-source versions of these things. And Rust, yeah. I was so shocked because I thought, hold on a second, these are profound products that have existed and had moats for years, and now there are just open-source versions.

Then Nikita was, I think, trolling and building on that idea, just talking about GTA because it's about to be released.

Yes.

I think GTA VI is about to come out, and people will reconstruct this. Why is this important? I think IP and software were effectively rendered worthless 2 days ago.

Yes.

Because if you think about it, when everything is headless—when Elon makes his services headless, and Benioff made Salesforce headless a few months earlier—when all of these things are headless, you can wrap them in agent tools, and you can connect them via MCP to other experiences. The value of IP no longer exists in software. It doesn't exist.

I think that probably is the most profound implication. Even in our business, we would argue with some of our customers all the time. We'd say, “We'll build this for you. We'll own the IP.” They'd say, “No, we want the IP. We don't know how to negotiate it.”

Two days ago, I walked into the 80/90 guys and said, just—

Who cares?

“Who cares?”

Solve the problem.

But before, it was always a negotiation and a set of trade-offs. Now it's a “who cares?” thing where the IP is irrelevant. It still takes a while for the machinery of a business and, frankly—

Yes.

The market to understand, but that is the most important thing that happened this week in AI, in my opinion.

I strongly agree with you, and I'm going to build on it. You have this abstraction of the software layer occurring at the same time, Friedberg, as the abstraction of the website or the marketplace.

Now I have my assistant building Grok bots, and then we'll be sharing those. We have them on a shared account. When we go to make a purchase or try to negotiate something, we're having the agent, without going to any websites, find all the quotes, and then the humans negotiate the second half of it. That's the perfect option here.

But you look at Amazon blocking agents, combined with Shopify saying, “Let's go for it,” while Uber and DoorDash are now letting agents go in there and order your burrito or your car. There's a whole compression of the commerce layer occurring at the same time as the compression of software that Sacks is talking about, which will lead to massive deflationary cost savings and speed.

The ability to get a hotel room or book a restaurant for an event, like I did, is all being done by bots already, Friedberg. This could be so deflationary and save people so much money. If people could save 5% or 10% a year on their spending, that's essentially working against, or countering, the effect of inflation. Yeah, Friedberg?

Yeah. Bringing the whole thing full circle.

Please.

This is what I mentioned earlier on the idea of loops. Think about getting a good price for a hotel. You run a loop: check hotel 1, look at the price. Is it better than the current best price? Yes. It becomes the best price. No, go look for the next hotel, and you just run that loop.

You can do it in parallel, so you can run 50 website searches at exactly the same time. You can short-circuit the loop because the software's ability to get that data is faster than the human's ability. Now the loop is: go check 50 websites instantly, give me a result right away. That's really where the efficiency gain comes from—a loop that gets parallelized, is done better, and can be completed in a shorter period of time.

I think anything that's done digitally today fits that model. Effectively, all digital workflows are going to get looped. They're going to become more productive, and then the question is: How does the human who now levels up leverage all of this loop infrastructure, all these agents, to do more themselves?

Going back to the point we made earlier about scientists and mathematicians, and this challenge for individuals, mathematicians in particular, if your objective historically has been recognition for your work via papers, patents—

IP, trademarks, ownership.

IP, yeah, these sorts of things. I think the transition is going to be that you're going to have to produce a product—

Mm.

The whole effort of humans effectively being the loops, because the loops can be digitally automated, scaled, parallelized, and so on. I think you end up seeing the necessity for all humans to basically become much more of a maker.

You have to make something. You have to do something productive. You have to produce something. You have to provide a service that someone ultimately uses, or a product that someone ultimately consumes, versus just being recognized for your work. That is the profound shift in society that has to happen, and that is going to happen.

I do think people will actually find—I truly believe this—that all humans have the capacity to be producers, to be makers, and all humans will find more joy in their lives in this state versus just getting ribbons for work. I don't think that it means less money. I think it means more money for all people, more life progress, and more prosperity for all people.

So I just want to zoom out a little bit. I want to make sure we have that view as we think about this stuff.

I think it's a beautiful view of humanity because it's like Jevons paradox: the cheaper things become, the more people can utilize them, and the more value they get.

I almost don't want to give this tip because I want to gatekeep it, but I'm going to put it out there anyway. When we book hotels and stuff like that for the team, we just get the best prices we can. Then, with the human in the loop, we go to them and say, “Hey, do you have a corporate rate? Can we talk to the manager at the hotel? Can you match this price?”

All of a sudden, we start saving $1,000, $2,000, or $3,000 on every hotel stay. When you have 20 executives moving around the country, and you're doing 200 trips a year, and you start saving a couple thousand dollars per trip because of this forward motion—goal-based, I guess, is sort of what you're saying, Friedberg.

The goal and the outcome—that's what happens. You can see profound results, and I wouldn't do that myself, but because AI's doing it now, wow. Yum, yum. It works.

I think you're a taker. I don't think you're a maker.

What are you talking about, man? All I do is invest in 100 companies a year. Do you know how hard it is to invest in 100 year-zero startups a year?

Oh, man.

It is exhausting.

Oh, boy.

But I love it.

It must be exhausting.

Let's get some lunch. Have you eaten?

I do not eat lunch.

Do you want to eat lunch?

Yeah. Can you come to my room?

I am so jelly. I'm having FOMO. I'm not with you guys.

Love you, boys.

Love you, boys. Bye-bye.

Bye-bye.

Let your winners ride.

Rain Man, David Sacks.

And instead, we open-source it to the fans, and they've just gone crazy with it.

Love you, Wes.

I'm the queen of kink, lol.

Let your winners ride. Let your winners ride. Let your winners ride.

Besties are back.

That is my dog taking a piss in your driveway, Simox[?].

Wait—no, shit.

Oh, man.

My avatar will meet me at Winston.

We should all get a room and have one big, huge orgy because they're all useless. It's like this sexual tension they need to release somehow.

Wet your feet. Wet your feet. Wet your feet.

Wet your beak.

Wet your beak. We need to get merch.

Besties are back.

I'm going all in. I'm going all in.