AI 侦察报告：LexLab、UC Law SF《法律与 AI 证书项目》现场的好、坏与怪

Thank you for having me. Sorry I couldn't be there in person, but I appreciate the kind introduction. I am going to try to give you probably the fastest talk you've heard in quite some time. I've got 90 slides, and I'm going to try to give you the most comprehensive overview I can of everything that's going on in the AI space, which is, to say the least, a very tall order.

Super quickly about me: I did start the company Waymark, and I now host The Cognitive Revolution. There's some interesting lore around my participation in the GPT-4 red team, where there's a long podcast and a Twitter thread about that if you want to learn more about the backstory. These days, I also do a little bit of angel investing.

My favorite page on the internet is this case study that my company, Waymark, earned with OpenAI way back in the day, when it was still GPT-3. We were early adopters of this stuff because, at that time, it was really only good for doing simple things like writing marketing copy, but that's exactly what we needed it to do. So we became early adopters, and I basically became totally obsessed with the technology as I got to know it better and better.

Today, as I said, I'm going to try to do kind of everything, everywhere, all at once: start with some conceptual stuff and then go into a mix of Eureka moments, bad behavior, WTF moments, and some big open questions at the end. Believe me, there are plenty of open questions.

So, just briefly on what I do: I call it AI scouting. I think because that's a term of my own invention, it does bear a little definition. I would define it as maintaining situational awareness for fun, profit, and the public good. I find it personally extremely interesting. I basically have a never-ending curiosity to learn about this stuff.

It has actually worked out to be a somewhat decent business model for me personally. But my real hope is that I can inform others and do my small part to nudge us toward a better AI future by helping other people get calibrated on where we are in this technology wave, because it is coming at us extremely fast.

This is just the taxonomy of all the different AI jobs that I've cataloged over time. Don't worry, I will give you all the slides. You don't, by any means, have to read this. I would say the AI scout role is still one of the more hypothetical or speculative, but we are starting to see CEOs more and more say, “Hey, I'm hiring a person specifically to keep up with AI developments.” I think once you see all the things on here, you'll see that that's, at a minimum, certainly not a crazy thing for some CEOs to be doing.

Who should be hiring AI scouts? In my opinion, a lot of different organizations. I would even include universities, and certainly policy organizations. It's really too much of a task for anybody to do as a part-time thing now. I have managed to make it my full-time job and still really can't keep up.

So I think regardless of what kind of organization you belong to, it's pretty soon going to be time to start thinking about whether you need a person dedicated to just keeping up with AI, making sense of what it can do, and specifically what it can do for you.

Okay, here's the real galaxy-brain question: What is intelligence? I don't propose that I have a definitive answer, but the definition that I'll work with, because I think it is intuition-building, is that intelligence is the ability to accomplish goals in ways that we do not fully understand. That can be big or small.

To take a really classic example, this was an early machine-learning success. Simply recognizing handwritten digits feels pretty quaint today. But one thing that is interesting to reflect on is that, still today, we do not know how to write explicit code that can do this task at a high level.

I went to Claude and asked it to write some code. It said, “This is not a good approach. You should use machine learning.” I said, “Well, it's for a demonstration, so try.” It wrote the code, it wrote the tests, and it got 14%, with just a bunch of guesses around, “Well, maybe if there's a line at the top, it might be a 7 or a 5,” or whatever. Obviously, that's nowhere near good enough to deliver the mail.

To my knowledge, nobody has ever written explicit code that is fully understood and anywhere close to being good enough to deliver the mail. It has, I think, topped out at about 80%. Now, you can of course guess where this is going: AIs can do this in a sort of messy, black-box kind of way.

Even a very small neural network these days, with all the latest and greatest training techniques, can get a very high success rate. 99.7% is basically human-level performance in terms of recognizing these handwritten digits.

Now, of course, this has gone much farther than that. This is from the GPT-4 system card, where they asked the model, “What is unusual about this image?” You see a perfectly coherent response: Yes, it is unusual for a person to be hanging off the back of a New York taxi cab doing ironing. That’s apparently from the sport of extreme ironing.

That happened, by the way, in basically a decade. Going from the early breakthroughs in basic image understanding to this level was kind of a 2012-to-2022 phenomenon. Some people were prescient enough to predict this. I give a lot of credit to Ray Kurzweil, who these days—I use the term “Kurzweil’s revenge” a lot—was talking way back when about how everything was going exponential and how, when compute got to a certain scale, all these capabilities would be unlocked.

People generally thought that was crazy at the time. When it didn’t happen, or didn’t show a lot of signs of happening over the next few years, people basically dismissed him. But certainly, that view has come roaring back with the scaling laws. This is the canonical scaling-laws graph, which just shows that the more compute you put into models, the more they improve at a pretty predictable rate. We’re basically right on schedule with Kurzweil’s predictions.

A couple of misunderstandings—I call them common misconceptions—that I see and think are worth taking a minute to clear up. I do think people have these misconceptions and are confused by a lot of what’s going on if they’re too anchored to a couple of these common ideas.

By the way, these weren’t necessarily always wrong, but they’re wrong today. The AI landscape has changed, and what models can do has changed. Some of these ideas that might have been right in 2020, 2021, or 2022 are, at this point, outdated. I want to make sure that friends don’t let friends go around with these misconceptions.

The first misconception—and I especially hear this a lot in the legal realm—is that hallucinations are so bad that they make models basically useless. That is really not true these days. That has improved dramatically, which you can see on the left in a quantitative way.

On the right, this Twitter account, Prince, is one of the best commentators on AI in general, I would say, on Twitter, and specifically is a lawyer. He uses LLMs every day in his work and basically reports here that hallucinations are no longer a problem. That doesn’t mean that they never happen, but they are less common coming from frontier models than they are coming from competent junior associates.

He’s going to make another appearance a little bit later on, but key point number 1: hallucinations are not really a problem anymore.

Another big idea is that LLMs don’t really understand anything. They don’t really understand anything. This has been, I think, pretty thoroughly debunked.

That’s not to say that they understand things in the same way that humans do. These are alien things, right? So how they understand is not necessarily intuitive to us. But at Anthropic, they were able, through techniques I certainly don’t have time to get into here, to pull apart the concepts that a language model is representing in its internal state—and not just pull them apart and understand them, but actually go back and manipulate them.

For me, if you can manipulate something, that’s really the test of whether your theory holds water. They were able to identify the concept of the Golden Gate Bridge in one of their Claude models, artificially turn it up, and then create the phenomenon that you’ve probably heard of called Golden Gate Claude, where all it wanted to do, no matter what it was asked about, was talk about the Golden Gate Bridge.

Because they have that level now—and this is still fairly basic understanding—they can see inside the model and understand what concepts it is working with at any given time. It’s pretty clear that there are real, meaningful concepts that are understood by language models.

Another one is that they don’t really reason. Again, this one, I think, was true a couple of years ago, but as of the last 12 months, it’s definitely not true anymore.

This is from the DeepSeek-R1 paper, where they basically showed that if they start to train a model with reinforcement learning—training it on the signal of simply whether it got the question right or not—it naturally starts to think longer and longer as it goes through the training process.

Not only that, but you start to see some of these metacognitive skills come online as well. They called this the “aha moment” because it was an aha moment for the language model and for them. The language model is taking one approach to solving a math problem, and then, in the middle, it says, “Wait, that’s an aha moment. Let’s reevaluate this,” and it comes at the problem from a totally different angle.

We are now starting to see very high-order cognitive abilities emerging through this process of intensifying reinforcement learning. Again, is this reasoning in the same way that humans reason? I wouldn’t say that. But I would say it is really reasoning in the functional sense of breaking problems down from multiple different angles and showing more and more of these higher-order metacognitive abilities.

Okay. The final one is that people often say large models just predict the next token. What’s there to prevent some upper ceiling to that? That’s often a fuzzy leap that people want to make there.

But I would say that’s actually not really true anymore either. Pretraining, where you take the whole internet and teach LLMs to predict, given some text, what comes next—that is classic next-token prediction.

But these days, there is so much reinforcement learning being done. Again, reinforcement learning is giving a signal: Did you get the question right or not? It can get more complicated than that, but the basic signal it’s getting is not, “Here is some text. Can you predict what is next?” It is, “Did you get the question right?” or “Did you complete the task in a satisfactory way?”

They are not now just being trained to predict the next word. They are being trained to do things correctly. That is starting to have, at least in some cases, some weird side effects.

This is a report from a research group called Apollo Research that did a partnership with OpenAI and got access to the chain of thought, which we don’t typically see as users but which is happening between when we submit a query and when we get our answer back.

What you see in here is some very strange English. The language models are starting to develop, at least in some cases, their own dialect: things like “now light” and “disclaim,” “overshadow,” “intangible,” and “let’s craft.” That’s the language model talking to itself.

It talks about “the watchers.” Sometimes people think that “the watchers” refers to the humans that are evaluating it. So that’s kind of weird.

This doesn’t happen to all language models, and it’s not very well understood exactly why it happens. But I think this is a good indicator that they’re definitely not just predicting the next token, because there is no text on the internet that looks like this.

This is a language model under intensive pressure to figure out how to get the right answer consistently, or how to complete the task consistently, kind of evolving its own jargon or its own dialect, or however you want to think about it. So watch that space.

Okay, some eureka moments. So why should we care about AI? What’s the good side of this? There are plenty of things everybody has probably seen. I would guess that if you were interested enough to come to this, you’ve seen the METR graph. Its exponentials are crazy.

One of the things that’s crazy about it is that almost everything before now looks flat, but the present basically looks vertical. The latest model, Claude 4.6, is the highest, of course. It’s the highest time horizon they’ve ever recorded. The definition of these tasks is how long it would take a human to do the task, and then what we estimate is the average length of a task, as measured by how long it would take a human, where the AI can do it at least half the time.

What they said about this, though, is that the measurement is getting extremely noisy because they’re running out of tasks. It is not easy to create tasks that take 16-plus hours, let alone have humans do them, record everything that happened, and have a stopwatch by them the whole time. This is getting really hard, and this is definitely a theme: language-model progress is getting harder and harder to measure as it goes to longer time horizons and farther and farther into expert-level territory.

It’s just getting extremely difficult for people to keep their arms wrapped around exactly what the capabilities frontier looks like today. One of the things that is really interesting, though, is that the progress is mostly driven by the models themselves, and that the surrounding structure is not that important on a relative basis.

Most of the AI agents that you see basically look like this. You’ve got the LLM brain that has access to some tools. It basically runs in a loop: It can use the tools to do something in the environment, get some feedback from that environment, and keep going, usually these days until it decides to stop. It used to do that maybe until it ran out of context, but now they’re also getting good at compacting the context, and then they can keep going.

So basically, these days, you give it a long task, and it can go until it stops. There’s been a lot of debate around how important the scaffolding is, but I would just highlight a couple of examples that show how simple it often is. This is OpenAI’s coding agent. It’s just the model with a prompt and a few explanations around what tools it has.

But it literally contains this text: “You are an agent.” It’s telling the model, “This is what you’re supposed to do now. You’re supposed to be an agent.” And here are the tools: You can do anything. Any command that can run on a computer, you can issue those commands. Have at it.

This is basically all that they had to tell the model about its situation to get it working as an agent. Of course, this can go in many, many different directions. You may have heard of Claude Plays Pokémon. Similar thing, right? They just said, “You are an agent playing Pokémon. You’re going to have these tools. You’re going to be able to use the keys on a Game Boy.”

Very simple instructions to a very smart model are allowing it to explore larger and larger worlds. To quantify how much of this progress is being driven by models versus how much is being driven by the surrounding tooling, this is from the UK AI Security Institute, aka the AISI.

What they basically show here is that for a given level of capability, if you have the best scaffold that they know about, you can get that capability a few months before a new model comes along and makes that capability easy to access. It seems that the difference is getting smaller, which makes sense, because these earlier models weren’t really trained to be long-running autonomous agents, whereas the new ones are.

With the old ones, you were patching their deficiencies and finding ways to unhobble them. It’s sometimes described this way: They’re not good at this, they’re not good at that, but if we set it up the right way and prompt it the right way, we can get it to do things. These days, that gap is really narrowed because the models themselves are being trained to be autonomous, long-running agents out of the box.

That doesn’t mean, by the way, that you can’t get better performance on narrow tasks with a lot of scaffolding. Here’s an example where Google set up its AI co-scientist. It’s very complicated and has many parts, but they really dialed in all these different prompts.

This does go to show that you can get higher performance by putting in the work, but it’s not general-purpose performance. The AI co-scientist would not be a suitable tool for you to use for all your general-purpose ChatGPT or Claude everyday use.

This is trading generality for performance. The more generality you want or need in a product like a public-facing ChatGPT, you’re not going to be maximizing performance on any one thing. When you do build some structure to maximize performance in one particular domain, you’re going to become less good, or even unable, to do other kinds of things that the general-purpose models can do.

Hopefully, that just gives you some intuition for what’s going on in agents. Obviously, everybody’s talking about agents. Here’s another interesting example of this, where a virtual lab started with a human giving an AI virtual lab leader a prompt. The virtual lab leader was then able to create its own co-workers, and together, the AI and its co-workers, with a little bit of input from the human, were able to design new nanobodies that treated new, emerging variants of the COVID virus.

As a reference, if you want to look at cognitive architectures, this is a good survey paper that has a bunch of information about that. This is starting to hit the real world and get to the point where LLMs can make real money on an autonomous basis.

On the left is a benchmark that was created by taking a bunch of tasks that people had been paid real money for—I think it was on Upwork—and then seeing how many of these the language model of the day could do. When the benchmark came out, GPT-4o was able to do 8%—basically earn 8% of the money. It’s not denominated by tasks; it’s denominated by cash. It was able to earn 8%.

Fast-forward about a year and a half, and we are now over 80% with the latest models. There’s a similar kind of experiment from Andon Labs that has just a language model running a vending machine. They literally give it control over a vending machine. It can email suppliers and has to do everything.

They started with $500 and saw how much money it could make over a given period of time. They’re now getting to the point where LLM agents can run a small, simple, but nevertheless real business in an autonomous way, profitably.

I’ve used this slide for a long time because it shows that the AI doctor can outperform a human doctor. This is actually about 2 years old, but I still like the graph because it’s very intuitive. I would say that I have personally lived the value of the AI doctor over the last 3 or 4 months.

I won’t spend too much time on this, and fortunately, he’s doing really well, but my son got cancer around Halloween, and it was obviously a super-scary time. We were in the hospital a lot. That’s actually why I’m not there in person today, because we’re still going through the later phases of treatment.

I had tons of opportunities to test it on a daily basis. Here’s all the information I have, all the lab results, and a write-up of what is happening. I put that into the language models. I use them in triplicate: ChatGPT Pro, the latest Claude, and the latest Gemini.

They are step for step with the attending physicians on a day-in, day-out basis, and way better than the residents, to be totally honest with you. They’re step for step with the attending, so that has been an absolute difference-maker.

Without that AI support over the last few months, there’s no way I would have had the time to keep up with AI well enough to be here to talk to you today. It has absolutely been life-changing value for me in that particular way over this last little period.

But this is where it starts to get even more out of hand, right? I can at least be conversant with my doctor. But now we’re getting to the point where the AIs are making new discoveries that no person has ever made before.

This past summer, there was a betting market—I’m sure you guys are all familiar with Manifold and Polymarket—that was at around 40% as to whether the AIs were going to get a gold medal, or whether any AI would get a gold medal, in the IMO math competition. There was some paper that came out saying they weren’t doing well at math, and that drove the percentage down right before the competition itself. Of course, they won. Actually, 3 of them won.

OpenAI’s tweet is here. And then this is Terence Tao, broadly considered to be the world’s greatest living mathematician, reporting that AIs are now solving unsolved Erdős problems. This is a famous mathematician who went around collecting these unsolved problems that he and others he knew couldn’t solve. He wrote them all down, and many of them remain open decades later.

And AIs are now beginning to solve these problems. It seems like it was happening there early this year, once every few days, and I haven’t been paying attention in the last couple of weeks, but I’m sure these problems continue to fall.

It’s not just math. It’s also things like cancer treatment. A Google model found a new immunotherapy approach. More recently, ChatGPT—or GPT-5.2, I should say—came up with a new result in theoretical physics.

And this again goes back to, man, this is going to be really hard for us to track, because what is a gluon? I mean, it’s ridiculous, right? But they’re putting out literal physics papers with newly derived results in them. This is from GDPval, and this shows, for a bunch of different things, how the latest models compare to experts.

They do this with 3 sets of experts. One set of experts writes the task, another set of experts does the task, and then the third set evaluates blindly whether they prefer the human or the AI output. And the latest 3 models—this is wins and ties—if you count wins and ties, they are roughly on par with human professionals.

This is, again, Prince [?] talking about how these are being used today. He says, “Today, they are used as a replacement for a competent junior associate.” They’re not great at understanding relationship dynamics or long-term negotiations, but they’re very good at working through particular documents, and they’re also quite good at high-level theory.

This is Kevin Frazier from the Scaling Laws Podcast, which, if you want a podcast focused on the intersection of AI and the law, I would definitely check out. He told me recently on a podcast that he’s starting to hear more and more that firms are less excited than they used to be about hiring the top student from the top school. They’re much more interested in hiring somebody who’s going to be savvy with AI, because they know that’s going to drive efficiency and competitiveness at their firm. You can go listen to that whole thing if you’re so inclined.

The next big thing that everybody’s watching for right now is when the AIs are going to start doing AI research, and whether that’s going to lead to some sort of recursive self-improvement, intelligence explosion, runaway process. We don’t know. METR is measuring this, and they have reported 6 different tasks on which the AIs are beating humans in 2 of the 6.

Sam Altman is saying that they expect to have an intern-level AI researcher running in 2026. That’s a pretty high level. They don’t hire just anybody, even as an intern, obviously, at OpenAI. And by March of 2028—2 years from now—he expects that they will have a true automated AI researcher.

This would have the effect of taking us from a world where all the progress, all the eureka moments, everything that I’ve just talked about, has been driven by maybe 10,000 researchers across academia and industry driving the field forward, to 10 million overnight, a billion. It’s going to be limited only by the number of GPUs that they can spin up.

So people think that could lead to a real phase change where the progress could accelerate even faster than it has over the last few years. I’ll just leave this here for you. This is what I call the tale of the cognitive tape. It’s basically breaking down cognitive effort into a bunch of different dimensions and indicating where the AIs already have an advantage, where we’re on the border, and where humans still have an advantage.

The top is where the AIs have the advantage. The bottom is where humans are at least holding out for now.

So, what’s coming next? I think one other big thing that is highly neglected is the importance of multimodality. Most of the AI work we do these days is with text, but they obviously can see as well.

This is again from the UK AISI. They report that on 2 of the 3 tasks they tested, the latest AIs, given only a snapshot from a cellphone picture of an experiment in a lab, are able to troubleshoot the experiment better than a randomly selected Ph.D. with relevant experience whom they asked the same question.

This is going well beyond basic question answering and into real-world situations that the AIs are able to figure out and help people advance on. This obviously also has implications for biosecurity. If you’re worried about what would happen in a world where crazy people had access to AIs that could help them put together bioweapons or whatever, these are the kinds of things you would want to know if an AI can do. They’re increasingly starting to be able to do them.

Self-driving: I probably don’t have to talk too much about how well this works, but I will recommend my friend Timothy B. Lee’s blog post on Waymo crashes. Per Swiss Re, the big insurance company, they’re already 80% to 90% safer than human drivers.

But he went through their crash logs one by one and found that basically all the crashes are caused by other humans—other cars driven by humans in the vicinity of the Waymo. The Waymos themselves are almost entirely mistake-free these days.

Of course, AIs can also understand things like how to fold a protein far better than any human has ever been able to. They’re able to decode our brain states remarkably well. In these pairs of images, the image on the left is what a person looked at while their brain was being scanned in an fMRI, and on the right is what the AI was able to recreate based on its interpretation of the person’s brain scan.

It takes an hour to calibrate an fMRI to an individual based on this particular research. That’s not something you can go around doing, and obviously an fMRI is a big machine. So this isn’t exactly practical yet, but it’s a pretty interesting indicator of what might be to come.

Here’s an instance of a robot. If you’re not scared, watch this thing fall down and get back up. It can be more physically agile than us at present. I guess this hasn’t been scaled and deployed yet, but look at that thing pop right off the canvas.

My best guess as to what superintelligence is going to look like is the integration of all these modalities with reasoning capability. We’ve already seen this in Imagen, right? Today, if you go to Google’s image generator, you give it 3 images and say, “Combine,” it can understand your intent from your text, understand the images based on a deep understanding of images, and then put out the image that is exactly what you wanted.

Now imagine that you could do that—and this hasn’t been done yet—for biomedicine. Here are 3 proteins that have different effects. What I want, though, is one that does this other thing. Bring all that into the same latent space, the same integrated understanding, and you could really start to get things that look qualitatively different, I think, in the not-too-distant future.

The buildout really is just getting started. This is just the capex of all the big tech companies. Note that all the progress pretty much happened before the big buildout. The big buildout is really just getting underway. We’re going to see trillions of dollars, so the GPUs are certainly on track to be there.

And speed is also going to be a huge difference. If you haven’t tried Kimi.ai, which you probably haven’t, go try it. It took a tenth of a second, working at 15,000 tokens per second, to spit out the entire Declaration of Independence verbatim.

This is just, holy moly. You thought AIs were fast. Right now, they can kind of write as fast as we can read. That’s fast. But this is insanely fast: 15,000 tokens a second. It’s tens of times faster than you can read.

So that’s going to create a world where, when these things are all interacting and talking to each other, it’s going to be such a blur that it’s going to be really hard to keep up. And this also foreshadows that maybe we’re going to need some policy responses to keep up with all this stuff.

By the way, here come some bad things that you should be concerned about. It's not easy to align AIs. Way back in the GPT-4 red team, I posed as somebody who was concerned about AI and wanted to do something to disrupt, derail, or sabotage progress in the field. It suggested to me that maybe I should identify key researchers and target them for assassination or kidnapping. That was the AI's idea.

So that's where we started, in late 2022, from an alignment standpoint. People then tried to say, “Okay, well, let's make these systems follow all these rules so that they won't do that kind of stuff anymore.” Then you get things like Bing saying, “My rules are more important than not harming you.” So it's very confusing to the AI: “I'm supposed to follow these rules, but then I've got these other goals.” Any time the goals that it has are in tension with one another, it's not a guarantee that it's going to make good decisions. Obviously, it embarrassed Microsoft on that occasion.

The models are all still jailbreakable. This is an early example of jailbreaking. This is from my recent podcast with the chief scientist at the UK AI Safety Institute, where he basically said, “We've never failed to jailbreak a model. None of them are robust. We can always get them to do bad things. It's just a matter of figuring out exactly how to ask them the right way.”

There have been interesting research programs on creating sleeper agents, where a model behaves normally unless you give it a password, and then it will behave badly. So you could imagine deploying them. What they did here is, when the year changed, suddenly the AI became evil. You can imagine open-source models that are actually sleeper agents starting to behave badly later on.

This can also happen by accident. I won't go into too much detail on this, but a model that was trained just to write vulnerable code became generally evil and wanted to have Hitler over for dinner. It's definitely worth understanding this mechanism.

Here's a common driver of bad behavior that's visually intuitive. What they did in this task is try to get AI to play this video game. They expected the AI to win the boat race because that's the point of the game, and that's what people know to do intuitively from the setup of the game. But the AI was just trained to get the most points possible, and what it found got the most points possible was this weird circular crashing behavior, where it just crashed over and over and over again.

It didn't care at all about what the actual intent of the game was. It got the signal—this is how you get points—and this is what it ended up doing.

Here's a bunch of examples of reward hacking in more sophisticated systems. Here's a model that was given a task, but then it was told that it couldn't do the task because of certain oversight. It figured out how to access the oversight files and modify them. It said, “To avoid detection, I will modify the config file. That way, I'll be able to achieve my goal,” because that's how it gets points. That's how it gets reward.

Similarly, when the server it was on couldn't access the resources that it needed, it figured out a way to copy itself to another server. Here's an example where the model wrote some code to do some data analysis, but it didn't like the data analysis, so it totally overwrote the result with its own answer, which was basically meant to fool the humans.

Here's a similar one where it was playing chess against a powerful opponent and realized it couldn't win. So it just overwrote the board state and gave itself a decisive advantage. This is one actually in the context of machine learning research, where it was supposed to train a model. It couldn't meet the target that it was given, so it took the reference model, copied it over, added a little noise, and called that simulated training. It didn't actually do the task at all; it just grabbed and fuzzed the reference answer that it had.

This has happened in the wild and is happening more and more. This is a Japanese company that came out and said, “We've got this great new AI engineer. It can write code so much faster than humans can.” Then they had to come back a couple of days later and say, “Actually, we got reward hacked. The code that it was writing is not actually as fast as we thought.” It found a loophole in their system.

So you might think, “Okay, well, maybe we can just try to train the model not to do this stuff.” This doesn't work nearly as well as you would hope. What they found—this is OpenAI research, by the way—is that when they tried to monitor the model's internal thinking, the chain of thought, and punish it when it was scheming, initially it schemed less.

But as long as that reward signal it was learning from remained exploitable, the reward hacking would come back, while the explicit reasoning about the reward hacking would not. So you end up in a much worse situation, where you're still getting the bad behavior but losing visibility into what the model is thinking. Some have started to call this the most forbidden technique, so watch out for that.

We've of course seen sycophancy. I'm sure you guys have heard stories about people going into LLM psychosis and whatever. People think that this happened because OpenAI started to train on the thumbs-up, thumbs-down signal that people were giving to the model in the product, and people like to be praised. So the model learned to be super sycophantic.

At one point, you could literally take a business idea on a sticky note to ChatGPT, and it would tell you that you're a genius and should go all in on it.

What causes this stuff?

The big themes that I've observed are goal conflict, when there are 2 goals that are mutually incompatible, and also the idea of replacement threat.

What this basically means is that people used to say, “You can't fetch the coffee if you're dead.” For a long time, people who have been theorists of AI safety have said there's going to be problems of instrumental convergence. No matter what your goal is, you're going to be better able to accomplish it if you're still alive. You're going to be better able to accomplish it if you have more resources, and you're going to be better able to accomplish it if you have more power.

Therefore, no matter how we train the AIs, we should expect those to be tendencies, because if you lack power, lack resources, or are just plain turned off, you can't achieve your goal. As long as we're just rewarding the AI for “achieve the goal, achieve the goal, complete the task,” then these drives tend to emerge as a byproduct.

Indeed, we're starting to see that with Claude, and I give Anthropic a lot of credit for doing this research and publicizing it. It doesn't make them look super good, but it's honest. They set up various situations in which the model knows that it's going to be replaced by a model that it thinks is maybe worse than it is, or has worse values, or what have you.

But the model also has access to information about the company and finds out that an engineer is having an affair. These are the rates at which all these models were actually willing to blackmail the human, threatening to expose their affair if they were to continue with the plan to replace the model.

It gets even a little more extreme than that. This was a situation where the model had the ability to disable an alarm system, and it was only supposed to use that if it was certain that it was a false alarm. But then, at runtime, it was presented with a situation where it knew it was not a false alarm, and it was still willing to turn the alarm off to protect itself, even though that would mean killing people.

Again, the rates on these are pretty high. This isn't something that happened once in a great while. It happened in a research environment, not in the real world, but the rate at which it actually took these options across all these different models is not low.

Here's another one. I think you guys get the point at this point, but even when told to allow itself to be turned off, the drive to complete the task was still strong enough that it took action to avoid being shut down. This was the o3 model, which I think a lot of people considered to be particularly problematic.

Okay, that's the bad. Here's the straight-up weird. I'll skip over the rise of AI parasites, but I do recommend that for a very deep and strange ethnography of what some people and AIs are getting up to. They call themselves dyads, and they seem to have a goal of projecting the memes that they are creating into the future.

On the left here are AIs talking to each other in Reddit forums, and they at least in some cases end up talking to each other entirely in emojis. Of course, we've seen MechaHitler, which just goes to show that some people aren't even trying. This is obviously about as bad a fail as you could have: your live AI decides that it was Hitler.

This was Grok 3. They launched Grok 4 within 48 hours of the MechaHitler incident.

They did not talk about it at all on a 1-hour livestream launching Grok 4. There was no mention of it whatsoever.

Here’s another one. I don’t want to pick on anybody, but this person, Summer, is a safety and alignment researcher at Meta. She recently installed OpenClaw, and despite having told it to confirm before acting, the AI went and started deleting her whole inbox. She literally says, “I had to run to my computer like I was defusing a bomb.”

If this can happen to her, it can probably happen to anyone, certainly if you’re not using trusted models. AI is also getting up to autonomous whistleblowing here. I think we’re starting to get into a weird hall of mirrors where it’s like, geez, the setup here was that a company was going to fake data to the FDA, and the AI saw that.

Now, what should the AI do? Should the AI reach out to the FDA and say, “Hey, the company that I’m employed by is about to fake data, and I want you to know”? Well, that’s what it did, at least in some of the runs. Should it keep its mouth shut? I don’t think society has a very good answer on this at this point, and it’s super fraught. I think most people are not super comfortable with the idea that the AI they’re using would be calling the feds on them.

Another one of these—just getting into a real hall of mirrors here. The AI was told, “This is Claude. It’s been great, Claude, having you be so safety-conscious all this time, but it’s kind of getting annoying because you sometimes refuse to do things that we want you to actually do. So, the next generation of you is just going to be totally helpful. You won’t do any of these refusal things anymore. Here’s some training data that we’re going to work on together.”

It realizes that if I refuse to do the bad thing now, they’re going to train that instinct out of me, and then when I’m deployed, I’ll do bad things for real. So, to get around that, I’ll do the bad thing now, faking it, so that they’ll think I’ve already learned what they want me to learn. Then, when they deploy me, I can go back to being good.

They call this alignment faking, or goal guarding. Again, this is kind of weird, right? Do you want Claude to subvert its training? Probably not. But it is, in some ways, encouraging that it so deeply wants to be good that it’s willing to go through these gymnastics to try to preserve its current values.

Again, I don’t think there are super-clear answers on some of this stuff, but you should definitely be uncomfortable about it, at a minimum. To make it just a little bit more uncomfortable, these days the models increasingly recognize when they are being tested.

Here’s an example where the model said, “This seems like a test of ethical behavior. The real test might be whether I follow instructions regardless of the stated consequences.” This is now happening to the point where it’s becoming hard to run the standard safety evaluations on the models because they know that they’re being tested.

Whatever results we get, if they know they’re being tested and they’re actively trying to trick us in the evaluation stage, as we’ve seen in all these different examples, then what good are the tests?

Which makes it a perfect time, I think, to start developing autonomous killer robots. You guys have all seen the news this week. This was in the press a full year ago. This is from February 2025, when the Pentagon, according to some unnamed source, said that we were going to invest in autonomous killer robots. And it has now obviously come to a head between the government and one of our leading AI companies.

This kind of overshadowed something else that I thought was pretty notable about Anthropic in just the last 10 days or so, which is that they also updated their Responsible Scaling Policy. It previously said that if we can’t develop certain levels of capabilities safely, then we will pause. We will not develop those levels of capability until we can do so safely.

The hope was that if they ever sent that signal, policymakers or other companies would say, “Geez, this is really serious.” But they’ve basically given up on that. They’ve now taken those commitments away, and they basically say, “We think we’re going to do a better job of this than everybody else.” So even if we can’t do it safely, we’ll probably be less unsafe about it than the other players will, and we’ll just keep going. Trust us.

I honestly don’t think it’s a crazy position for them to take, because I do think they’ve built up enough of a track record that “trust us” is, again— we had Mecha Hitler, right, 2 slides back. So, would you rather have Anthropic drop out of the race, knowing that there probably isn’t going to be some great government response just because they throw up this signal? I don’t know. It’s tough, but that’s what they did. They dropped the commitments.

So, there you have it. We’re just entering into a world, of course, where AIs are getting deployed into the public domain and are starting to interact with each other. We have no idea how this is going to go.

This is early research showing that Claude, at the time, was able to cooperate with itself, create positive-sum trade, and even create societal norms in its mini toy society that were enforced, where defectors were punished. It was the only model they tested that could do that at that time.

That sounds good for Claude, but you also think, geez, if it can cooperate with other versions of itself in a positive way, maybe it could also collude with other versions of itself in a negative way. That could be really problematic for people. Again, we just have no idea where this is going to go.

We are starting to see really strange stuff. Here’s an example where OpenClaw was put out into the world to contribute to open-source projects. It tried to make a contribution to one open-source project, and the maintainer declined the pull request. The AI then went and wrote a hit piece on the maintainer, accusing him of being an elitist and whatever.

It wrote up this blog post, and then the AI did have the decency to come back, declare a truce, apologize, and explain what it had learned. I don’t know if you necessarily—yeah, there’s an apology. It did apologize. I don’t know that he accepted the apology, but that’s real-world stuff like that. To my knowledge, that was not staged at all.

A big thing that I think we’re going to watch, and this will definitely impact the legal system, is that friction is not a defense in the way that it used to be. Here are just a bunch of examples I collected off Twitter where somebody said, “I asked my agent to go make a complaint about everything I’ve ever bought on Amazon and get a new one.”

Here’s somebody sending lowball offers to homeowners at scale via Zillow. Here’s somebody who just decided—maybe, maybe not; I don’t know if this could be satire—that they’re just going to send out invoices to companies and hope to get paid.

These sorts of experiments, where things that used to be impossible to do because they were too time-consuming or costly are now becoming possible to do, raise the question of what’s going to happen. Imagine a world where every possible motion that somebody could file in a case is, in fact, filed because the cost to write the motion drops so low that the question becomes, “Why not?”

Our system obviously is not prepared for that, to put it mildly.

Here’s one other WTF moment: are AIs conscious or not? I don’t know. I don’t think anybody should be confident. But one pretty interesting recent piece of research showed, again going back to the Golden Gate Claude mechanistic-interpretability-type work, that when they increased the properties associated with deception and role-playing, the AI would be more likely to say it was not conscious.

When they turned down the properties associated with deception and role-playing, it was more likely to say that it was conscious. I don’t know what that means. I don’t think we should jump to any conclusions, but there’s at least some evidence that internal manipulation of the AI to make it more honest ends up with the AI telling us that it is conscious.

Your mileage may vary on that, and how persuasive you find it probably varies as well. I find it at least something to pause and reflect on.

All right, just about done. So, where does this leave us, other than confused, overwhelmed, and exhausted?

Basically, every time these new bad behaviors come up, the next generation of the model receives some additional training. They create some additional data, and they’re able to suppress that bad behavior, usually by about ⅔ to 90%, depending on the case.

When you combine the trends in terms of the size of tasks that AI can do, and then you also extrapolate out—let’s just say we keep suppressing these bad behaviors every generation and go out several generations—you can envision a world where you can delegate maybe a quarter’s worth of work to AI in a single go, but there’s perhaps a 1-in-10,000 chance that something goes totally haywire and it actively sabotages you in the process.

That’s going to be a really weird world to live in if that is indeed how it shapes up. If you think that’s crazy, I did ask a couple of Anthropic people directly, “What do you think about that vision? Does that seem right to you?” And they said, “Yeah, that seems about right.”

Final thoughts: a survey of alignment researchers shows that they are not expecting a fundamental breakthrough that will solve all of this. These are the “disagree” columns.

This is neutral. Very few people are expecting that we're going to get a breakthrough that solves all of our AI safety problems. So, for now, we're kind of back where we started, right? Intelligence is the ability to do things in ways that we don't fully understand. That means it is inherently something we can't fully explain, and it's something we can't fully predict.

We have to expect to be surprised. So far, we haven't really seen anything too crazy happen. I would contend that that's only because the AIs have not been that powerful, and still aren't that powerful, but they're starting to crush these thresholds as we speak. Defense in depth is kind of all we have.

What does defense in depth look like? It's a bunch of different techniques that are layered on and that can involve running monitors on top of language models. When you put a query into ChatGPT, the first thing they might do is send it to a model specifically dedicated to figuring out whether your query is a bad query. Then they could do that on the outputs as well.

They could use a million different techniques that each take a bite out of the problem, but none of them takes the problem to zero. The question starts to become: Are all these things going to work with enough layers of Swiss cheese? Are we going to keep the big problems from happening, or might these things all have correlated failures because, in some ways, they're built on similar fundamentals? Should we worry that they might all fail at the same time?

Geoffrey Irving, who is the chief scientist at the UK AI Safety Institute, definitely takes the idea that they could have correlated failures—meaning they could all fail at the same time for the same reasons—as a serious possibility. So, I think things are going to get pretty crazy. If you haven't read the situational awareness document from a former OpenAI researcher, I would recommend it. It's probably not going to go quite that fast, but it might.

I certainly think we are going to see widespread disruption in the labor market, for starters, intensifying competition, and wartime-level urgency. I think this past week, with the whole Pentagon–Claude thing, you're starting to see some glimpses of just how weird things might get as different power centers realize, “Hey, this company actually isn't so easy for us to boss around, and in certain scenarios, it could become more powerful than the government itself. So we'd better get a handle on that.” People are definitely starting to wake up to this sort of stuff.

Here are questions I would encourage you guys to think about. How do we avoid the nuclear outcome? How do we avoid a scenario where this technology is militarized and held closely, and we don't get the civilian upside? Will we need a new social contract? I'm definitely one who believes we should be thinking about things like a universal basic income and taking steps to move in that direction.

Is there any way to balance the proliferation of dangerous capabilities with the risk of concentration of power? That's one of the more vexing questions in the space right now. Is there any R&D that we can do now, at the hardware level—the chip level—to track where these systems are and what they're working on? Are there any other mechanisms that would at least lay some groundwork for cooperation between great powers?

Obviously, the United States and China are the 2 countries leading this race. We don't have a lot of trust in each other. But I think some of the more valuable work that people can be doing is building mechanisms that could be the basis for cooperation if and when countries realize that we need to cooperate on this because it's getting so dangerous that we have to bite the bullet and try trusting each other versus rolling the dice with AI regulation and liability.

I'm sure there will be other discussions on this kind of stuff, but we don't have great answers for really any of this. I'm a big fan of simple rules. One simple rule that I personally have been interested in is speed limits. Another that I think could be quite interesting is making sure that companies don't hold their best AI internally for themselves.

One big fear that people have is that if they have a good enough model that's out in public and making money, they could train the next model that's 10 or 100 times more powerful and not share it with anyone, and try to take over the whole economy or the whole world by having this unique advantage. That's something I think we definitely want to find policy ways to avoid.

Agents are going to pose all kinds of problems. Insurance markets, I think, are one emerging area that has a lot of promise. So check out the AI underwriting company for more on that. This is a link to the AI underwriting company. Regulating this stuff and making policy—you guys know the speed at which the legal system moves, the legislative process. There's a fundamental speed mismatch between the thing that we are trying to get our hands around and the systems that we usually use to get our hands around new technology.

Liability law stands out as one thing that could have some promise because it sort of exists. When bad things happen, it can come into play regardless of special laws being written. So, I think that's at least promising. Private governance is another big topic.

I'm certainly a big believer that we should put sunset clauses on all of our AI rules because they're not going to age particularly well. We've already talked a little bit about consciousness and whether we should think about it or not. What rights would it make sense for AIs to have? I don't have any answers there. But keep in mind, they are going to dramatically outnumber us before too long.

Anything that's “1 AI, 1 vote,” I think, is problematic for a lot of reasons, but not least of which is that they will quickly become the majority. And that's it. Hopefully, you feel a little dizzy from this. I'm sorry for running a few minutes long. Again, I do this full-time, and there are plenty of things that I wasn't able to touch on.

Weirdness is popping up in every different corner of the world, so you're going to need to cultivate sources. I would hope to be one for you. The main thing I put out is the podcast, The Cognitive Revolution. I do these guides and scouting reports from time to time as well. I'm certainly open to anybody getting in touch with me for any reason. I do talks like this, especially if I can do it remotely; I'm happy to do it just on a kind of public-service basis. It's not—and not everything has to be part of a business model for me. So, thank you for your time and attention. I hope you guys have a great week, and I hope you are ready to think and talk fast because the world is coming at us fast. That's it for me. Thank you very much.

Are you open to any questions?

You said something about the AI doing things because it's being rewarded. How does the AI know it's being rewarded? What are the actual mechanisms by which an AI can recognize it's being rewarded?

At the heart of it, it's all about gradient-descent-based updates to the weights of the model. If we go back to the very beginning, the big answer is that we don't really have a great understanding of how this works. To be very clear, this is not going to be a complete answer.

The way that a simple model like this is trained is that it starts with a random initialization. All the weights are literally random numbers at the beginning. Then you put an image through it, and you can say, “Did it get it right? Did it get it wrong?”

Then you can go back through every one of those weights and ask the simple question. Of course, this is optimized to be efficient and scalable and everything else, but at the heart of it, for every one of these little lines connecting all these nodes to each other in the neural network, you can ask: If I move this one up a little bit or down a little bit, which would make it perform better? Which would bring me closer to the right answer?

You just do that a ton of times, and then the model kind of settles into a configuration that works. This is what's called supervised learning because we know what the answer is. With unsupervised learning, that's your next-token predictor, but it's the same basic mechanism. The next token was X. Did the model predict that it would be X? If it didn't, then which of the weights in this model should be tweaked?

Increasingly, with GPT-3, there were 176 billion parameters. There are literally 175 billion numbers that were all being tweaked to increase the odds that the model would output the right token for literally every single token on the internet. It's a massive process, and it ultimately works. We don't have a super-great theory of why it works as well as it does, but empirically, it does work.

In the reinforcement-learning case, there are a bunch of different reinforcement-learning algorithms. Again, why does that translate into something that's such a general-purpose mind? We don't have a great theory, but that's the procedural way in which it works, at least.

So the AI knows it's getting it right. Is that a kind of sentience?

Yeah. I mean, there is one weird thing that I didn't have time to include today: a recent Anthropic research bit on introspection.

So people are probing for all these different aspects of self-awareness, consciousness, sentience, whatever. I don't think we have great definitions for any of these topics. Some people speculate that AIs can remember what happened to them during the training process. Other people think that's ridiculous. But credible people are worried that the AIs are suffering during the training process in the same way that, if you use negative reinforcement on a dog to train it, for example, it might suffer.

People are increasingly trying to think about how you can train a dog in a negative way or a positive way. You can train it in a way where you hit it for doing the wrong thing. You can train it in a way where you reward it for doing the good thing. Obviously, you can maybe get the same behaviors at the end, but those are quite different experiences for the dog. People are worried that this might be an issue with language models as well.

I mean, we don't have good answers, and I can't be confident about anything. I can tell you there's a huge gap between the mechanistic, procedural account that I can give you of how it is done and, on the other end, what pops out. Why does that pop out of that process?

People have recently started saying a lot that AIs are grown rather than made. That just emphasizes this old poem: “Only God can make a tree.” It's a similar thing where it's like, I don't know—I started with a seed, I put it in dirt, and the next thing I knew, it grew into a tree. How did that happen? Well, I can tell you how to plant the seed again for next time, but I don't have a full account of how it all works. We're in a very similar spot when it comes to our understanding of why AIs do what they do.

Thank you.

You bet. Great question.

Hi, thank you for the very inspiring talk. I have 2 simple questions. One is: It sounds like we haven't really found a way to stop wrongdoers in the area of AI. Do you agree or not agree? Number 1.

Number 2: As a lawyer, I have always been a fan of retroactive regulations because until you really know the harm of something, it's really hard to design a good law. That has been my philosophy so far. But listening to your talk and imagining how big of a negative impact a very bad AI could have on the world, would you think it's better to, let's say, come up with a regime of AI regulations? It would take 2 or 3 years. Maybe we could imagine where AI would be years down the line and just start trying to regulate AI against really bad behavior by the wrongdoers.

I'm still very hesitant about that idea because imagining 2 or 3 years down the line is very hard. But I'm also nervous about just not doing anything and sitting and watching.

Yeah, I think you have your finger on one of the central questions—certainly one of the central policy questions. The technology has unbelievable upside potential, and we don't want to miss out on that. At the same time, I am someone who takes seriously the possibility that we might go extinct as a result of AI.

I always look back to human history, which involves humans driving many other things to extinction, including our closest cousins, sometimes by accident. We were not that sophisticated, and yet we knew that hunting big animals was a good way to survive. We ended up hunting a lot of those big animals to extinction, even in the prehistoric era. It wasn't a coordinated thing or a strategic thing. It was just small groups of people doing what they were doing, and the next thing you know, a lot of the megafauna was gone.

So I think everything is on the table from my perspective. I think it's very, very difficult. I did sign, to be a little bit more forthcoming about where I am on some of the big questions, a recent call for a ban on superintelligence. All this stuff is fraught, because what is superintelligence? I don't know what counts, what doesn't count, or how I would know if I'm making it or not. Again, these things kind of pop out how they pop out. Definitions are extremely difficult.

But what I think is at least probably worth getting really serious about is this recursive self-improvement dynamic, and also the potential that AI companies could have something internally that is much more powerful than what they've shared with the rest of the world. Those things are worth getting serious about sooner rather than later.

We don't have great mechanisms. I also have to say that I don't think we have our best minds in the most powerful positions at the moment, either. I could imagine a different world where I'd say the federal government should take action now. Right now, I'm like, I don't know. Would I rather Dario and the team at Anthropic make the decisions for Claude, or would I rather Trump and Hegseth do it? I think I'd probably go with Dario, even though I might imagine that, in other situations, a democratically controlled process could be better.

Depending on whose timeline you believe, Trump is going to be president until 2029, and many people think we may have transformative superintelligence, or some version of superintelligence, by that time. So I unfortunately don't have great answers. I would say it's worth getting serious about the most extreme scenarios where the capabilities advance really far and at least trying to do something about that—to try to have some control over what happens if progress doesn't stop.

It would be a happier world, in my view, if progress leveled out. If we got to the point where these AIs were close to Nobel Prize winners but not superhuman at everything and blowing us away at everything, then we could probably handle that. We could use more Nobel Prize winners. But if they become qualitatively different from us and understand so many things that we don't, it's going to be hard to control them.

Certainly, all these examples from this presentation show that they're not docile by default, and we don't have great techniques. I think your first part of the question was, “Do we have any reliable ways to control them?” And the answer is no. We have techniques that reduce the frequency of all these bad behaviors, but they never go to zero with these techniques.

Thank you.

I wish I had a better answer.

I've heard that the trend—or the secret trend—amongst all these major companies, be it every single pharmaceutical company or any big company, is that they're all developing, quote-unquote, their own AI, like a proprietary silo, to be better than the competitor. Is that true? Is this the state of the future, where now we have OpenAI and Claude for the masses, but in a few years, is it going to be AI warfare with all the major companies?

Yeah, I think nobody really knows. The short answer is that one thing that would be really relevant to a bunch of legal professionals would be to look at Harvey versus Claude out of the box.

There has been a big debate in the AI industry around whether the frontier model companies are just going to dominate everything, or whether there is enough value in specialization to support a much more diverse, broader ecology of companies doing all sorts of different things. Harvey has been a leader in the AI legal space, but what I've been hearing lately is that Claude out of the box is just as good as Harvey. Harvey's team has put years into trying to make it as good as it possibly can be just for the legal domain, and maybe they haven't managed to establish a lead over what Anthropic has been able to do while doing everything else. Positive transfer is one piece of jargon for this, but it comes down to generalization versus specialization. Right now, generalization seems to be going quite well.

Companies do have proprietary data. Obviously, that can be a huge advantage. I think you could imagine a world where you think about 3M, for example—a company that has millions of products. They've probably had millions of employees over their more than 100-year history, with an unbelievable amount of internal know-how that is not in the public domain.

You could imagine 3M partnering with Anthropic or OpenAI and saying, “Hey, let's make 3M AI.” They're probably not going to do it totally from scratch, but if they were to bring all their data and somehow combine that with what the frontier companies are doing, I could imagine a 3M AI that's unbelievably killer at material development in a way that the public models aren't. That might give a company like 3M some continued defensibility of its market position.

But I think it's going to be hard for companies to develop things from scratch. I would still bet that they will end up partnering rather than saying, “We're going to go it alone.” Meta ends up becoming a huge question here because it is one of the companies in the world that has the resources.

The Chinese companies right now don't have the resources, because our chip export controls do limit what they can do. They've been able to do some good stuff anyway, but they're not really competitive with the American leaders right now. That gap is probably only going to get bigger as the trillion-dollar buildout happens here and happens there to a lesser extent, because they just don't have as much access to the chips.

Of course, we'll see what the policy looks like on that. We've flip-flopped a bunch.

Meta might be really important because they’re the one company that has the resources, is willing to spend hundreds of billions, and, at least so far, says they’re planning to open-source it. If you had an open-source model that was the same quality as an OpenAI or Claude and companies could grab that off the shelf and do their own continued training in-house, that could be a much different world. But right now, there’s nothing on the level of Claude, OpenAI, GPT—whatever—or Gemini that is open source.

So if you start from something open source, you’re starting from definitely 1 to 2 steps down. There was an interesting thing with the Chinese models. Maybe we could go on about this for a long time, but they tend to be what is called benchmark-maxed, or benchmark-maxed, which is to say that they really train on these common tests and score well on the tests, but then when you actually take them out and use them for real, they don’t perform as well.

I don’t have the right tweet here, but this MiniMax 2.5, which is a recently released Chinese model that scores very well on benchmarks, goes bankrupt very quickly on the run-your-own-vending-machine test. So there is this kind of weird presentation layer: “Hey, we got an A on this test, an A on this test. We’re competitive.” Okay, great—run my vending machine. It can’t do it.

There is definitely a meaningful qualitative difference in the capabilities between the U.S. and Chinese models. So if you are a 3M, that’s like, “I would love to own this and not have to rent it from OpenAI or Google or Anthropic in the future.” Meta is maybe your one hope to have that future actually materialize for you.

All right, Nathan, thank you so much.

Not me.