扩展“思考”：Gemini 2.5 Tech Lead Jack Rae 谈推理、长上下文与通往 AGI 的路径

Hello and welcome back to the cognitive revolution. Today I've got the honor of speaking with Jack Rae, principal research scientist at Google DeepMind and technical lead on Google's thinking and inference time scaling work. As one of the key contributors to Google's blockbuster Gemini 2.5 Pro release, Jack has tremendous insight into the technical drivers of large language model progress and a highly credible perspective on the path from here to AGI. Gemini 2.5 Pro, as I'm sure you know, marks a significant milestone on Google's AI journey. It's the first time that many observers, myself included, would rank a Google model as the number one top performing model across many important dimensions. And this is not just about topping leaderboards. In my initial testing of Gemini 2.5, which I conducted before Google's PR team reached out to schedule this interview, I experienced one of those rare moments where a model significantly exceeded my expectations, forcing me to re-evaluate my sense of what's possible today and inviting me to reimagine my workflows to take advantage of its unique strength in not just accepting, but actually demonstrating incredibly deep command of hundreds of thousands of tokens of input context. This is a practical step up that I could feel almost immediately. So, naturally, I jumped at the chance to talk to Jack about all the work that went into it and how he understands the current state of play along a bunch of critical conceptual dimensions. We begin by asking why techniques like reinforcement learning from correctness signals appear to have suddenly started to work so effectively across the industry. Does this represent a proper breakthrough? Or is this more a culmination of steady incremental progress that has finally crossed important thresholds of practical utility? We also unpack the reasons that nearly all frontier model developers are releasing similar reasoning or thinking models in such a short period of time. Is this simultaneous invention driven by obvious next steps? Or is there more cross-pollination somehow happening behind the scenes? We then consider the relationship between reasoning and agency. Will these reasoning advances translate to agentic capabilities or is something more still needed? From there, we look at the role of human data in shaping model behavior. How does Google think about collecting human reasoning and step-by-step task processing data? And how intentional has Google been in training models to follow recognizable cognitive behaviors versus letting them develop their own problemsolving approaches during the training process. We also exchange intuitions about the relationship between models internal feature representations and the patterns of behavior they use to leverage them. Consider whether reasoning in latent space should scare us or can be made safe via mechanistic interpretability. And discuss whether the application of reinforcement learning pressure to the chain of thought itself should be avoided as OpenAI recently argued in their obuscated reward hacking paper. Finally, we'll discuss the road map from our current capabilities to AGI. What are the remaining bottlenecks? Do we need a memory breakthrough or will continued scaling of context windows be enough to overcome all practical limitations? And should we expect deep integration of more and more modalities as we've recently seen with text and image? Throughout our conversation, Jack provides thoughtful, nuanced responses that absolutely should help us improve our understanding of today's AI systems, the work going on inside Frontier Labs, and the overall trajectory of AI development. Personally, I leave this conversation with the sense that for most developments we see from the Frontier Labs, the simple explanation is the best one. There's still a lot of lowhanging fruit left in large language model development. Researchers have internalized the bitter lesson and are trying to keep their approaches as simple and scalable as possible. And the rapid progress we observe is mostly the result of pursuing pretty obvious high-level conceptual directions and then methodically chipping away at the practical engineering challenges required to make them work at scale. The teams involved, as you'll hear, are seriously concerned with developing the technology safely, but are also feeling both a high level of genuine excitement and competitive pressure that keeps them moving forward as quickly as possible. As always, if you're finding value in the show, and I definitely think this is one of the higher alpha episodes we've done, we'd appreciate it if you'd share it with friends, write a review on Apple Podcast or Spotify, or just leave us a comment on YouTube. And considering that the future is radically uncertain and the stakes are crazy high with outcomes from a post scarcity disease-free utopia to an existential catastrophe or even outright human extinction all live possibilities in just the next 2 to 20 years. I take my responsibility in making this show extremely seriously and I earnestly invite your feedback and suggestions. You can reach us either via our website cognitive revolution.ai AI or by DMing me on your favorite social network. Now, I hope you enjoy this insider's perspective on scaling large language model thinking and the path from here to AGI with Jack Rae, principal research scientist at Google DeepMind. Jack Rae, principal research scientist at Google DeepMind and technical lead on Google's thinking and inference time scaling work. Welcome to the cognitive revolution.

Jack Rae, principal research scientist at Google DeepMind and technical lead on Google's thinking and inference-time scaling work, welcome to The Cognitive Revolution.

Cool. Thank you so much for having me. I'm excited for this conversation.

Congratulations on Gemini 2.5 Pro Experimental 03-25, I think it is. The long name doesn't reflect what a big release this is. Obviously, that's a common trope in the model wars these days, but it is a big deal.

In my estimation and in my testing, this has been the first time that I would say a Google DeepMind model has been the number-one model in many important respects. It has also given me one of those hair-raising moments that don't come along too often, although they happen remarkably often.

When I dumped a full research codebase into the thing—400,000 tokens—and said, "I want to extend this. I want to reuse as much as I can, but I want to make a really light touch and not mess with other people's code because this is a shared, collaborative space," I was really amazed by how much command the model had of the super-long context.

It was hair-raising because it did feel like a qualitative difference, a very immediately noticeable step up. We're all still adjusting to what it can do and calibrating ourselves, but I think, as the kids say these days, it is safe to say that you guys have cooked on this one. Great work, and I'm really looking forward to understanding a lot of the work that went into it.

We're super happy with this model. We're really happy with the trajectory of our models, and this one was a true Gemini team effort. I'll probably touch upon this, but this was a knockout performance from the pretraining team, from thinking, from post-training, and from many areas across Gemini, all really pulling this together.

We feel pretty good about it. We liked it internally, but we didn't know exactly how it would be received. It's great to see that people are really finding it useful. They're feeling the AGI with it, and they're seeing noticeable deltas on real-world tasks. That's been very cool to see.

I really appreciate the praise. I just want to say that this was a full Gemini team knockout, but I'm really happy to talk about some of the model development, especially things on the thinking side.

Let's get started with a question that I've been thinking about a lot recently, and I think a lot of other people have, too. Why didn't the simple approach—and, of course, I'm sure you guys used more complicated techniques, but here I'm really thinking about the DeepSeek-R1-Zero demonstration that a really simple RL setup with a correctness signal can work now—why didn't that work sooner?

I assume many people tried it in many contexts, and I'm not sure if they were missing something, the models were missing something, or what it was that kept that idea at bay for a while. Now, of course, it seems to be working everywhere.

I suppose, from my vantage point, we've basically been leaning more and more on RL to improve the models' reasoning ability for quite a while—for at least a year within our Gemini large language model. As we've been releasing models, there has been a greater and greater presence of using reinforcement learning for accuracy-based tasks.

We're getting a very discrete, verifiable reward signal and using that to improve the model's reasoning. We've actually been doing that before thinking even started, and we've been shipping models with that. It's been helping the model's reasoning process.

I think the way I see it is that this has been something that a lot of amazing reasoning researchers and RL experts have been improving for a while. Progress has hit a bit of an inflection point where it's really captured people's attention, and maybe it feels like there was a threshold moment for a lot of people around, say, the DeepSeek technical report.

I think it's been working for a while. There hasn't been one key thing that has discretely made it work. It's just crossed the capability threshold where people have really taken notice.

Interesting. So, fair to say you see what may seem to outsiders as an emergent phenomenon as more of a mirage? Under the hood, it's a pretty smooth curve.

That's how I see it.

A lot of these capabilities, when we internally track them, are going up with sometimes almost scarily predictable improvement—almost a Moore’s law-style improvement. What I’ve come to notice, and this also happened during my time in pre-training, is that we would see that phenomenon. With each piece of improvement to the reinforcement learning recipe or the model recipe, you don’t always know what will help, so there’s a bit of stochasticity there. But as you accumulate things, there is this almost consistent trend of improvement.

What I feel usually happens in the public domain is that it crosses these thresholds occasionally, where people really take notice and get very excited, and it captures people’s imagination. Crucially, the model just gets sufficiently good that it really feels like a step change, especially with these discrete releases that we make. That’s my perspective on it.

That juxtaposition between smooth progress on leading-indicator metrics and the threshold effects of downstream tasks is one of the most interesting dances in the entire field, I think, and probably will be for a while to come. On your personal production function, obviously everything is going exponential in the space right now, and the number of papers and different techniques being published is in keeping with that. How do you allocate your time—or how do you think about allocating your time—between reading and keeping up with research that the rest of the field is doing versus keeping your head down and pursuing your own ideas? Are there any AI tools that are making that more manageable for you right now?

In terms of reading research versus doing coding, running experiments, and things, on some level, I don’t know whether my own experience is just influenced by career progression and changing how I work. Earlier on in my career, I’d spend a lot of time reading research. There was so much to brush up on, and it felt like maybe at conferences and things, this was where all of the action happened. It was really about consuming a lot of different ideas and things.

Now, I feel like—and this could just be partly because I’ve switched from a more junior research role to something where we’re directing things a little bit more—there are a lot of very known problems for which there’s no research out there that has the solution. The solution is going to be discovered among the group of people that I’m working with day to day. The amount of time I spend reading research has definitely gone down a lot compared with 5 years ago or even 10 years ago.

But I still find it very inspiring and useful when people are publishing cool ideas. I still take the time. I use X, I follow people, and I use arXiv filters to try to filter out interesting papers or blog posts, as well as podcast interviews or YouTube videos. A lot of this stuff is coming through different formats now.

In terms of tools, I know this may sound predictable or clichéd, but right now I do use Gemini a lot for reading, summarizing, and asking questions about papers, especially because that has been its forte for a long time. I feel like I can trust its ability to ingest not only a whole paper but sometimes even a collection of papers if I want to add in a bunch of cited papers and then ask questions or ask for summaries. That’s pretty useful, especially because as you read research more and more, you start to get a bit more demanding about cutting straight through to the critical idea and the critical results. Sometimes it’s just a bit hard to do that if you don’t have the time to pass through the text by brute force and look for what you need to know. It’s very useful to have the model do this.

Yeah, Gemini’s long-context ability is really good. It’s been very good at question answering and summarizing a long span of technical text. I like it for that, and that’s my go-to tool.

Another striking observation about the field right now is that close to all—maybe not quite all—of the frontier model developers have pursued what, from the outside, appears to be a very similar trajectory over the last year. We basically see now a whole new class of reasoning models that follow a similar paradigm, where they have a chain of thought, where they’re thinking for a while, and then they give you a final answer.

That convergence is something that I’d like to understand better. I don’t know if it’s just simultaneous invention because the conditions were so overdetermined to make that the next logical step, or if it’s the other theory that you hear, where people are meeting up at these infamous San Francisco tech parties and sharing what they’re working on over drinks or whatever. How would you describe your understanding of why everybody is developing seemingly very similar ideas in parallel right now?

I think it’s just a phenomenon that existed even before the invention of San Francisco tech parties. People are always looking for avenues of progress. Even from very small bits of information that show a model is improving in a certain way, people notice very quickly—especially now, when we have an unprecedented number of smart people working in AI and an unprecedented amount of compute that allows us to react quickly. We’re seeing that follow through to an unprecedented level of speed and velocity.

When there is a new paradigm—let’s say test-time compute in this case—and there’s a bunch of performance and capability to explore in this domain, people will flood into it very fast. If I think about how this unfolded within Google, within Gemini, we assembled the reasoning groups to work on the specific topic of thinking and test-time compute around September or October. Within a month or so of focusing on this space, we were finding what we felt were modeling breakthroughs that were very exciting. That led us to ship a first model in December, an experimental model based on Flash with Thinking.

If I reflect on how that team’s progress went, there was just a very natural process of people exploring this space and really getting involved, with more and more people thinking about it, running experiments, and progress happening very fast. I would imagine that’s just a common phenomenon now within these very talented research groups, and that’s why you suddenly see a bunch of reasoning models within a short time span of each other. There’s just a very natural phenomenon of curiosity, exploration, and talent right now. People are always super motivated to find the next big breakthrough and explore it as fast as possible.

Can I summarize that as: the idea itself was a pretty obvious candidate, and the density of low-hanging fruit—the richness of that vein—was just so striking once you started to mine it? That accounts for all of the leading developers at least exploring it a bit and then all of them finding that, yes, this is really a way we clearly should be investing a lot.

Yeah, that’s at least how things have unfolded within Gemini. I think we’ve also been seeing a lot of initial signs of this making sense and had some initial results. Fortunately, this whole thing required a deep confidence in applying reinforcement learning to language models, which is something we within Google were very comfortable with, interested in, and working on. In that respect, there was a low barrier to entry to really explore this space and then find a bunch of really cool capability breakthroughs from thinking. It was a natural extension for us. I can’t really comment on the other labs, but I imagine similar things must be happening across the board.

One really small detail, but I wonder how you would contextualize this for me. In the DeepSeek-R1 paper, they said they tried reinforcement learning on smaller models and basically couldn’t get it to work, and they seem to be pretty cracked, as the kids say. That seems like they would have been trying something pretty smart. Later, though, it does seem like it’s working everywhere. Any light to shed on what would account for somebody in the recent past trying to apply reinforcement learning to somewhat less powerful base models and not getting it to work? Does that sound right or wrong to you?

Yeah, that’s completely valid. These things are way more difficult, I think, than people often realize. Even pre-training, people now consider to be completely solved and completely obvious. I was working on pre-training, let’s say, 6 years ago, when training a large language model with 100 billion parameters or more involved a million components that could go wrong or diverge. It was in an alchemy stage.

Training reinforcement learning on these powerful language models and getting them to reason and think more deeply is also in an alchemy stage. I imagine people have tried and failed many times because there are a lot of key, crucial details to get right.

So, I just think it's hard, and it requires a lot of things to be fixed. When you have 5 things broken, it can be very difficult: you may find 1 thing that is broken, fix it, nothing changes, and you get disheartened. At some point, maybe you feel like this just doesn't make sense, that this won't work. Then it just requires a few iterations of that until more and more things are lined up, and the whole thing starts to shine.

I feel like we saw some initial sparks that were very cool last year, where, just with reinforcement learning, the model was using thinking and we started to see really cool phenomena happening during the thoughts, like self-correction and exploring different ideas. That's exactly what we would have hoped would emerge from reinforcement learning, but we didn't really know if it was possible until we saw it for ourselves in our own experiments.

Yeah. So, how do you think about those sorts of cognitive behaviors, as they're increasingly commonly known? There are obviously multiple different ways that those can come to exist in a model. One possible explanation for why RL maybe doesn't work on smaller models is that you need a big enough scale of model and training to have those begin to take shape at all in a model, so that the reinforcement learning can bring them out. But you can maybe also get them to be learned during supervised fine-tuning, or maybe if you just do enough RL, they can sort of pop out semi-randomly. How much work do you guys do to sort of sculpt and really curate those cognitive behaviors, versus how much are you seeing arise at which stage of the training process?

Yeah, I think people have different opinions on this. We're a pretty outcome-driven team. At the end of the day, we'll do whatever recipe gives us the best results, the best model generalization, and the best final result.

Taking one step back from that, there are some priors and opinions in the space. One school of thought, which I'm quite in favor of, is to choose the simplest recipe that leads to a powerful model. There's a bit of Occam's razor in that. If you can impose fewer and fewer priors into what the cognitive faculties should be, and you can still get a really powerful model, so everything is more purely learned from data, that always feels like a better approach.

That said, we explore human data, we use model-based synthetic distillation data, and we try to have a lot of things arise from end-to-end reinforcement learning. So, we try everything, and then in terms of the final model and the final mixture, we just go with what works best, with some kind of preference for simplicity and generalization. So, yeah, I don't know if that's a satisfying enough answer. Obviously, we can't go deep into what our training recipe is, and it's also always evolving so fast, but those are the general principles we use.

Yeah, that makes sense. I don't expect you to spill all the secrets. Human data obviously has some nice upsides, in that we would expect models trained on it might be a little more humanlike. I obviously don't want to overstate how humanlike they become.

I guess I wonder: Have you seen one of the famous tidbits from the DeepSeek-R1 paper, where they reported this language-switching behavior in the context of the chain of thought? I've also personally seen that from Grok. I have not seen it from Gemini. Is that something that you guys observed? Did you take any action to try to select against those weird behaviors—or maybe not necessarily select against them, but set a proper prior so they didn't come online in the first place?

Yeah, I think ultimately one principle is that we want the model to use its thinking tokens to just be a smarter and better model. From that perspective, there may be some slightly weird phenomena happening in the thinking tokens: it might get quite cyclic, or it may appear to be emitting text that's not so useful all the time. But if it leads to the model then being much stronger at solving the problem, one philosophy is that you should just let it do that. This is supposed to be a scratch space for the model to figure out how to respond with the best accuracy, safety, factuality, et cetera.

That said, we did notice some things about the thoughts. One is that Gemini's thoughts are usually in English. They usually prefer to be in English. We actually found the model was quite strong at reasoning tasks— I18N, as we call it, basically non-English reasoning tasks. It would mostly perform its reasoning, though, in English.

That was one question: Is this a bad product experience, or should we allow it to do that if it allows the model to be quite strong at these reasoning tasks? So, that was one debate over this. It's not quite language-switching, you could say, for the thoughts; it's just sticking to one language.

Another was that some of the thoughts, especially in the original Flash Thinking launch, were quite templated. The model would often choose to use a formula-like structure for how to break down the problem and then formulate a request. That was another line of research: Do we want this to be very templated? Ideally not. It should be quite natural. It should be the model thinking through the problem, not necessarily always following a particular template. It feels like if it's always adopting a particular template, then maybe it's not getting the most benefit out of that thinking compute and other things.

There are other aspects of the thinking tokens. We obviously want it to be efficient and maximally benefit the capability of the model. So, those are some topics we're always thinking about.

Yeah, cool. Okay, that's interesting. Just to make sure I have a clear understanding of what I am looking at when I look at the chain of thought, is it fair to say that what is being shown—I actually mostly use AI Studio; maybe you could comment on that if it's at all different from the Gemini app itself—is the full, raw, unmodified chain of thought?

Yeah, that's right. We launched in December and then launched again in January. With 2.5 Pro, in all cases, you're seeing the raw chain-of-thought tokens from the model, both in AI Studio and on the Gemini app.

This is something we're always thinking about. It's not clear what the best thing to do, honestly, is. People do like to see the raw tokens. At the same time, they can be quite verbose. We might want to create summaries that are actually more useful. We might want to do other transformations.

There was a cool piece of work in NotebookLM where there's a thought explorer with a graph, and you can follow different ideas in a graph structure. It's still a pretty new space, and I think we haven't finalized the best way to surface thoughts right now. They're the raw thoughts.

Yeah, interesting. So, I was just wondering what, if any, debate went into the decision to share the full chain of thought, because obviously OpenAI initially chose not to and cited a mix of reasons. But I think most people interpreted it primarily as a competitive consideration: They didn't want to share the full chain of thought so everybody could just go and distill it, or do SFT or whatever, on their work.

That does not seem to have proven a durable moat for them. But I wonder what considerations or debates you guys had as you decided, “Yeah, let’s go ahead and share the whole thing.”

I feel like these kinds of decisions are often a mixture of input from the safety team, the researchers, and leadership, and it really is a complex decision. I couldn’t give you a very specific roadmap, but for each release, it’s carefully considered. Our leaders, like Koray and Demis, will often want to have a very good understanding of the pros and cons.

For me, I don’t weigh in on this, so I’m not really the best person to ask. I just try to make sure all the models are incredibly strong and that we have a lot of good options on the table. I think it’s an area of active exploration. We haven’t settled, and we’re not fixed on one particular way of surfacing these thoughts.

In fairness, also for OpenAI, I don’t know why they chose to show summaries. We could speculate. They did give us some reasons, but I’m sure there could be a mixture of reasons that go beyond just things like distillation to other aspects.

I think there was an initial worry from some group of people that maybe if we show thoughts, then we have to start RLHFing thoughts to make them look really nice to users, and maybe we don’t want to encourage models to have deceitful thoughts. There’s another school of thought, which is that once you have these thoughts, they’re great for interpretability, and you can understand how the model formed its output.

So I guess there’s just a whole debate going on about what’s the best way to ingest and communicate this content. From my perspective, I just want to make sure the thoughts are resulting in a way stronger answer, a way more capable model. That’s my main concern.

Is it fair to say, then, that you don’t concern yourself with how the chain of thought looks to the user? OpenAI recently also put out the “Obfuscated Reward Hacking” paper, where they showed that fears of reinforcement learning on the chain of thought are not entirely unfounded.

They showed that when they started off with a model that learned to reward hack and then put pressure on the chain of thought not to reason about reward hacking, that initially would tamp down the reward-hacking behavior. But later, you’d see the reward-hacking behavior come back without the reasoning showing up in the chain of thought—thus, the obfuscated reward hacking.

It seems like there is something quite concerning there. Do you see that as concerning? Do you endorse what I take to be the conclusion of that paper, which is, “Thou shalt not select intensively on the quality of the chain of thought”?

I think we show the chain of thought right now as part of these experimental model releases, and we’re trying to get feedback and learn from real user behavior. This is often an incredibly important aspect of releasing any technology.

We’re seriously taking in feedback, looking at how these things are used in practice, and making more educated decisions on how to surface information from chain of thought in the future. Safety is definitely one thing that plays a big part in that decision.

To put a little finer point on it, you could do RL on the chain of thought for any number of different objectives, right? To try to make it more readable, to try to avoid weird cyclic behaviors, or to try to tamp down reasoning about reward hacking, which may have this downstream negative effect.

But there’s definitely a strong school of thought out there that says, “Don’t do that.” Do you see that as a strong taboo because of the obfuscation that it can create, or do you think there’s some way to do it and not have such a big problem?

I think it’s a pretty safe angle to say that we want these thoughts to actually improve the factuality, safety, and capability of the model. We want it to have that scratch space. If we’re going to be showing thoughts, then we want them to be interpretable and faithful to the computation that the model is undertaking, and we probably don’t want to add training objectives that would encourage things like deceit. So I think that’s a very valid point.

Going back to the mix of different data types and human data for a second, I’ve tried in my own work to get people to record their chain of thought, even before all this reasoning stuff. I personally found that when fine-tuning a model by simply including example chain of thought in my fine-tuning dataset, I would usually get much better performance on 1 or a very small number of tasks.

As I’ve worked with other people to help them build their AI applications or automations, I very often say, “Okay, what I need you to do is staple your pants to the chair. I don’t really care how you do it. You could do it in text, or you could turn on your webcam and record yourself, whatever. But I need your live chain of thought as you, the expert whose work we’re going to try to automate, actually do the work.

“We need to know not just what your inputs and outputs are, but how you’re thinking about it and why you’re making these little incremental decisions along the way.”

I find that really hard to get out of people in a lot of situations. This may be a little bit outside of your specific responsibility set, but I wonder what you or the broader team have learned about how to coax that data out of people, if anything. Or maybe it’s just so hard for you guys as well that you’re sort of like, “Oh, God, we’ll go with synthetic.” What’s the state of actually eliciting human chain of thought out of humans?

Your question had 2 components. One was: How do you get that process data? So it’s not just prompt and then solution or response, but actually what was the process that led to the solution? Then there’s something like chain of thought, which I guess is one instance of that.

Funnily enough, I think it’s really hard to get people to transcribe actual chain of thought faithfully. It’s a pretty latent thing. Actually, I think part of the reason all of these models, especially Gemini, are able to click into this mode well is because people have already detailed their own thinking process.

Maybe it’s not when they’re put under the task of doing this explicitly, but even in essays, various pieces of work, or online discussion, people will often break down how they’re going to solve the problem and why they’re writing what they’re writing. So there are already a bunch of examples in the pretrained model of what it means to reason through a process.

That’s partly why, even before we were really trying to bring this out and make it really powerful with reinforcement learning, you could do things like prompt the model, “Let’s think about this step by step,” and it was basically doing this zero-shot.

What I’ve found, though, is that when you put people artificially in a situation and say, “Now you have to record all of your reasoning toward a problem,” when it’s not happening organically but under a directive, it seems to be quite hard to get a lot of value out of that kind of data.

But I think that is a bit separate from your other question, which was: How can you record process? I think that is very valuable. If we can get more and more examples and training of the processes that people naturally use to solve their tasks, that feels very valuable.

I’m just not so sure people are very good at describing their inner monologue, training on that, and having it be useful when asked to do that.

When you talk about recording process, are you imagining computer use—how people click around and interact with the environment—or what sort of recording are you envisioning there?

I think more in this kind of space where you’re going to solve a more open-ended task and have to do a lot of intermediate calculations, maybe actions, for example.

Yeah, I think that's kind of what I have in mind. But this is really part of the question of what's the best way of getting more agentic data and that kind of thing. That really isn't my area of expertise, so I wouldn't be the best person to chat to about that. Does that mean you see a significant distinction between reasoning and agentic behavior? Because I think a lot of people right now have the sense that reasoning is going to be the unlock for agentic behavior.

No, absolutely. I just feel like reasoning and agentic behavior, as a research thing, are very tightly coupled. But you can still segment which parts are the critical research questions for acting and creating environments for agents. That part, we have a really good group for; we compartmentalize it, and there is a group of people that work on that. The thinking area really collaborates when it comes down to the reasoning behind actions or behind responses.

Okay. So, you mentioned a minute ago that people struggle to write down their thoughts in part because it's a sort of latent thing. So I want to take a turn into the latent space with you, if you will.

First of all, I'd love to give you a sort of undoubtedly overly simplified understanding of what's going on in a model as it's reasoning, and have you critique, elaborate, or expand upon it. My general working model has been that the pre-training process determines what abstractions, representations, or features—whatever you want to call them—a model has to work with, what concepts it has, basically. And then post-training determines the patterns of behavior by which it deploys those concepts and puts them in juxtaposition against each other and tries to figure out a path through to a solution. My sense is—well, react to that.

Yeah. One way of maybe paraphrasing what we're saying—I largely agree—is that pre-training can learn this massive bag of function approximators that allows you to model the whole distribution of both good and bad behavior, strong reasoning behavior, and incorrect reasoning behavior. You get kind of everything. You can try and mold it a little bit with your selection of your pre-training data, but it's still really trying to reflect all types of behaviors and really just trying to understand. So the better you can predict the next token, the better you can compress this text, maybe even the better you can understand the whole distribution.

During post-training, you're going to drop a lot of modes. You're going to drop a lot of types of behavior and really try and fixate on a couple of types of ways of reasoning, ways of responding, or acting on various different tasks that are important. And then, hopefully, if we do reinforcement learning really well, you are also going to learn to compose some more primitive skills to build up your skill set toward this smaller set of important tasks. I don't know if I'm critiquing or exactly mirroring what you're saying, but that's how I think of it.

I guess the distinction—and maybe this will blur—is that part of the premise has been that the vast majority of the compute goes into pre-training, and then post-training is, by comparison, very small—maybe 2 orders of magnitude less. And I think now, obviously, the scale of reinforcement learning is going up as well, and maybe this dichotomy is ultimately going to become a spectrum, which certainly is a common theme in everything that I study.

Maybe one way to put it is: do models learn new, fundamental concepts about the world during post-training, or is that largely learned during pre-training? And is that going to change as we go from 1% to 10%, or whatever, of FLOPs being deployed in that post-training phase?

My sense is they have to. It's absolutely crucial, if we're going to build highly capable systems, that during the reinforcement learning stage we're not just reshaping known concepts, but learning new skills—especially if we want these models to eventually completely surpass us at very critical tasks. It can't then just be reshaping the knowledge that it's seen from behavioral cloning during the pre-training stage.

And I think that's one of the most exciting research directions we're all in right now: how do we get the composition of reinforcement learning to help scale up these models' capabilities to being incredibly powerful, general, and robust? And I would totally bet on it being during reinforcement learning.

So, another big—maybe the one frontier model developer that hasn't joined the reasoning party in full force at this point would be Meta. They did put out, though, what I thought was a very interesting paper—although kind of a scary paper from some points of view—about reasoning in latent space.

Instead of actually cashing out to a token at the end of a forward pass, they would just take the last latent state before that final decoding, pass that in as the embedding for the next token position, and just let the model chew on its own thoughts for however many forward passes in a row.

To me, there is something quite scary about that. I would like to be able to know what my AI is thinking as much as possible. There were also some nice features about it. There was an attractor state there, I think, where it required fewer forward passes to reach similar performance. And there was some evidence that they could do breadth-first search as opposed to having to go depth-first, which seems to be more the pattern that explicit chain of thought lends itself to.

So, what do you think about reasoning in latent space? Should we be scared of it? Should we taboo that, or are there some ways that we could embrace it safely?

Okay. I think tabooing a piece of technology before it's been researched and understood, I'm never in favor of, unless there are incredibly strong arguments to do so. In this case, I would say the reason that people could raise a question mark over it is this interpretability question: we need those latent vectors to be interpretable.

I actually want to draw an analogy. I'd say we should pursue it if it leads to better thinking and it can be interpretable and made safe—why not explore this direction? It seems very promising.

And actually, I want to draw one analogy to—I don't know if you know this, but MuZero was an extension. We had AlphaGo, then AlphaZero, then MuZero. So those were a series of algorithmic developments. Obviously, AlphaGo was the moment where we had a reinforcement learning model beat the world champion at Go.

The difference from AlphaZero, which essentially only used self-play—no SFT—and there were many other algorithm improvements, but that's the tagline—to MuZero was that instead of essentially unrolling over states, which is happening in AlphaZero, they unrolled in latent vectors. Those vectors could still be decoded into states, and there were a lot of advantages that they found with MuZero to being able to search in this latent space.

So I was pretty inspired by that, and often when I think about thinking in latent space, I think of MuZero. That was definitely the most powerful one of that series. It was the most powerful progression, and they still could make it interpretable because they could decode states from these latent vectors. So I think it's quite possible that this could be a very promising direction. I wouldn't rule it out at this stage.

Yeah, it seemed like a good idea. I guess the skeptic, or the safety hawk, might say, it's all well and good when you're talking about game states that you can decode to in a quite high-confidence way, right? I mean, ultimately, there is a game state that this thing has to operate in, and we know what that is, and it can't go off into far, far away places. But we don't have a similar sort of ground state that we can feel so confident in when it comes to what exactly is going on inside a general-purpose AI.

I've spent quite a few hours reading the outstanding work that Anthropic just put out about tracing language model thoughts. And I think the headlines of that have unfortunately maybe led a lot of people who are not in the field to a high level of overconfidence in our ability to really understand what's going on. As much as I think the work itself is awesome, I tend to also look at, well, geez, the replacement models that they create can only explain 50% of the behaviors, and there are a lot of error terms that are being added in to make sure all this is being explained.

So, big picture, my sense is that the field at large does not think we're going to get interpretability working well enough by the time we expect to have powerful or transformative—or whatever you want to call it—AI to really be confident in what the models are thinking or why they're doing what they're doing. What's your overall outlook for interpretability? Do you think it will get there faster, and we really will know what they're thinking as we get these powerful systems everybody's expecting?

Yeah. There's a rapid advancement in capability. What I usually believe is that these also transfer not only to models doing tasks like coding or agentic tasks that people find useful in the real world.

It also accelerates mechanistic interpretability. If we have more powerful models, we have more powerful tools to examine these questions. So it's not super clear to me that capability is going to improve exponentially while our ability to do mechanistic interpretability or safety work is going to improve linearly, creating a massive mismatch. I would imagine the two are going to track each other.

But actually, to your question about latent vectors versus thoughts in tokens, this is a really good point. In any case, you want some really good pieces of research and tools—eventually, artifacts—that can try to trace how close the actual content of the thoughts is to the underlying computation and thus to what the outcome of the model's answer will be.

I feel like that is just a very interesting research problem. That was a really cool piece of work from Anthropic. We have really cool people working on this within Gemini. It's a really important problem, and we should try to solve it in any case, whether it's latent vectors or continuous tokens. It seems like people both like and need this kind of interpretability from the model.

Yeah, I think it's huge, especially if we're going to have these things running large swaths of the economy—or, heaven forbid, the military, which seems to be more and more the kind of thing certain people are dreaming about. Knowing why they're doing what they're doing seems to me to be an imperative.

One of the big challenges with interpretability is that automated interpretability might be a huge unlock, or it might be sort of a spinning plate that we could see crash at any given time. The auto-labeling of features is one of the big challenges there. Again, the Anthropic work is just beautiful: the interface and the way they've published it, where these features appear inline in the post, let you expand and see the actual passages from the dataset that caused the feature to fire. Some of them, I have to say, I look at and think, “I would not have come up with that label.”

So this becomes quite philosophical. Maybe I'll ask it in a philosophical way. I'm sure you've seen the paper called “The Platonic Representation Hypothesis.” I wonder to what degree you buy that hypothesis. What that means to me is that there's sort of a convergence between models with growing scale, which seems to suggest that they may be converging on one true world model. Do you think that is actually what is happening, and, by extension, with further scale, should we be more confident in our reading of what the models are doing?

Maybe could you paraphrase the question a little bit? Are you saying that, across all the different models that are being trained, as they're growing in scale, they will start to converge more? I wasn't sure exactly.

Yeah. Maybe more deeply and philosophically, are they converging on some actual representation of reality that we can trust as being well grounded?

I would say that the only place I feel like I have a very strong theoretical conviction is what is happening with pre-training. As we're approaching that point by decreasing perplexity and improving the compression of the text that we see, if we could hit the noise floor, hit the entropy of the text, and have Bayes-optimal text compression, then we would have the model that best understands the world model that generated this text.

That is a thing I feel has a very clear mathematical grounding, from Ray Solomonoff's work and even Claude Shannon's work. It's always referenced in terms of how the optimal text compressor would have the best world model of the generation process that generates this text.

That does sometimes feel like a philosophical argument, though, because even that object is not what we really want for AI. It's not just something that has the optimal understanding of the dynamics that generated the text that exists today. We want the model to be trained to go and do something useful, to faithfully follow the instructions that we give, to do complex tasks that maybe have never been done before, and to generalize to completely new and unseen environments. All of those aspects, I feel, are not covered by that world-model description of what's happening in pre-training.

That's why, even though I've spent most of my career on pre-training, pre-training is not the only component to building AGI. At some level, I think it sounds like maybe I agree with the hypothesis that you said, but also its relevance—I don't think it's the full story of how we build AGI. So maybe it's something that's been downweighted in my mind as being the only story I should think about.

I do think that once you're starting to get into the realm of training these models with reinforcement learning at scale, they're definitely not all converging to 1 model. Actually, there's a lot of responsibility in doing this well, such that we really build the systems that are useful. I don't feel like you can even see it on the ground right now: the models are quite different already.

There are already a lot of different pros and cons across them, and a lot of capabilities that we work very deliberately on within Gemini to make them more useful in certain domains that I don't think just naturally arise across the board across all models. So it still feels very steerable. It doesn't feel like one eventual process toward one kind of world model of everything. It still feels very directable from the research side.

But I'm not a philosopher. I just try to make these things work really well. I feel like I would be very interested in hearing what a couple of philosophers who are keeping up to date on AI would think about this.

Yeah, it seems like, to summarize—and I think the empirical sciences definitely have a lot to inform philosophers about as well, especially these days—it seems like you're saying the world model itself is something that maybe everything is converging on, but how you navigate that world behaviorally is still a vast scope of information or a vast scope of possibility where there's not a single right answer. That's kind of where taste and safety and all these sorts of things have a lot of space to explore and diverge.

So maybe, for the last 10 minutes or so, how do you map the roadmap from here to AGI? Obviously, I don't mean in a detailed technical sense, but sort of one big thing that Gemini has is really long context. Do you think we can just scale it up and have some data where you have to actually have command of long context to succeed, and the model will learn from that?

I may be oversimplifying. Tell me if I am. But is just continuing to push on that going to be enough, or are we going to need some sort of more integrated, more holistic process of memory and forgetting to really have these long-running agents that people imagine? I guess, is memory something you think is already solved if we just push on our current levers, or do we need some sort of conceptual breakthrough?

Yeah, it's a good question. When I joined DeepMind in 2014, I started in an area called episodic memory. Memory is what Demis did his PhD on, looking into episodic memory and imagination and things, so I've always been very inspired by human memory, human episodic memory, and the hippocampus. My own PhD was on lifelong reasoning with sparse and compressive memories: how do we have a memory system and a neural network that is expressive and has this huge range of time spans, as we have in our own mind?

When I started that PhD, I would have never imagined how much progress we'd have made. We now have something that, say, with 1 million tokens or 10 million tokens—these kinds of context lengths, depending on how you represent your text or your video—are starting to verge on lifelong scales. But I still don't think memory is solved. I don't think it's all done yet. I think there are some really cool breakthroughs we'll have even in the memory space, and there were a lot of very cool ideas.

At DeepMind, we had this kind of Neural Turing Machine and Differentiable Neural Computer. These were a mix of large attention systems with a lot of different read/write mechanisms. My sense is that probably something in this space will prevail, and this will be a very cool way of having extremely long, infinite, lifelong memory. But it's still an active research area.

But to the roadmap toward AGI, I suppose, with each piece that we make, it does seem to compound very well. A year ago, we released what we felt was a breakthrough in long context, and that has ended up stacking really well with our current reasoning and thinking work because we found that there's just a really useful coupling of being able to think very long and deeply about a problem and also being able to use a ton of context, maybe 1 million or millions of tokens. That has ended up unblocking a bunch of extra problems that we now can solve that, if we didn't have both of them, we would have needed...

I think that the path remaining to AGI is obviously agents, and that's a super-high-priority area. Thinking and reasoning are still not at the endpoint. These models have a long way to go in terms of being so reliable and so general that you really feel like you can trust their responses on more and more open-ended tasks.

From our perspective, there's still a lot of just making the system better. There are a lot of known bottlenecks right now, and we'll continue doing that: making thinking better and, within agents, making agents better. But I feel like combinations of much better agentic capabilities, better reasoning, and ideally better memory systems—such that we can have almost a lifelong range of understanding and reasoning across time—will really feel like AGI to a lot of people.

The current systems, to me, feel like AGI. I feel like I'm using AGI when I use Gemini 2.5 Pro. It can now one-shot complex codebases, and that was something we felt was a futuristic piece of technology 3 years ago. Now it's just there, and it works.

We're always hungry for the next thing. But I think those combinations—much better memory systems, a much deeper thinking and reasoning system, and the capability to work with many different tools and an action space that's very open-ended—will really feel like AGI. When it's coming, I think it's hard to say, but it's all being actively developed right now, so I feel like it's coming quite fast. I feel that, too.

Yeah. Okay, 2 more quick questions, and then I'll give you the floor to share any final thoughts that you have.

One thing I didn't hear you mention in that description is the integration of more modalities. I've been inspired to think these last couple of weeks, as we've seen Gemini 2.0 Flash Image come out and also the GPT-4o image generation come out, that there's a lot of power in a deep integration of the text and image modalities, as opposed to a sort of arm's-length, tool-call-type integration.

Do you see that happening across many more modalities? Is there a world in the future where Gemini whatever Pro, instead of calling AlphaFold, is deeply integrated with AlphaFold, such that those latent spaces are actually merged and co-navigated in the way that we're now seeing with language and image?

Yeah, AlphaFold is a good question. I would say that a very good design decision for Gemini was that we made it multimodal first. It's been incredibly strong at image understanding and video understanding. It had native image generation trained within Gemini 1; it's in the technical report. It didn't end up getting released immediately in its first form, but I think that, to your world-model question, having everything deeply multimodal is super important.

Training everything and getting that world model not just over text but over multimodal video, images, and audio has been a cool aspect of Gemini. It's great to see these things launching. People really liked the native image generation. They love the fact that suddenly you can edit images and do a lot more interactions, instead of just calling what would be a pure text-to-image model as a tool, which is very static.

Anything that you can bring into the world model and train jointly, you're going to have a much deeper experience and understanding. I think that's very cool. Then it goes to the question of the dividing line: Where do you decide when to bring things into the pretraining mix and have them jointly understood? That's a really difficult question.

I think what you're seeing across the board right now is a pragmatic choice of starting with almost the most compressed and largest information sources, and then building out from there. Text was a very natural starting place for a lot of these large generative models because it's so compressed and knowledge-rich, and it's available at scale.

But the decision of how to grow this out to maybe smaller-scale sources of data or slightly less information-compressed sources is a difficult one. In biology, for example, in genomics, it's very cool to try to co-train genomic generative models with a large language model. People are looking into that, and I don't know where the dividing line is.

It's going to be something about how much you get from co-training versus just calling it as a tool, and how much positive transfer there is from all the world knowledge within your text, video, and image space to this new task. If there's not much positive transfer, maybe there's not much benefit in co-training it, and maybe you just want to learn to use it as a tool. Those are the main decision factors for whether you should bring it all into one world model or leave it as a separate expert system.

Well, I'm betting on the one-world-model approach, but we'll continue to watch the space. So, last question. I really appreciate your time and you coming to share so much alpha with the community here, but one question people would definitely be upset with me if I didn't ask is: Where is the system card for Gemini 2.5 Pro?

We sort of thought we were going to get one, and it seems like with the last couple of models, we haven't. I don't know if there's a policy that determines when a model actually gets the full technical-report treatment.

The approach with experimental releases is that we release these models because we really want to get them into the hands of consumers and developers, get real feedback, and understand their limitations. But they are released as experimental models. This experimental tag means we don't do the full provisioning of these models, and we don't necessarily have all the artifacts, like system cards.

We're moving as fast as we can to get these into a stable state where we feel like they're ready for general availability. There will be system cards when the model is made generally available.

Has all the safety testing been done at this point?

We do extensive—probably an industry-unprecedented level of—safety testing before we release models. But with experimental models, there may be a different level or tier of testing that we undertake. Part of the experimental release is getting real-world feedback, which is also a useful part of the testing process.

For these releases, it goes through a very standard process in terms of the policy team and safety team. There's a lot of red-teaming and other work. That is happening. But right now we're in this experimental stage, and we're racing to get toward general availability, which will have even better provisioning and things like system cards.

Yeah. Okay, cool. Thank you. A lot of the questions came up at a cloud event last week: When will it be made available on Vertex? And I'm like, “Oh, soon,” and then it ended up being the next day. In some of these cases, we kind of underpromise and overdeliver. These things are happening pretty fast, and the technology is also moving very fast.

Does that red-teaming process include third-party red-teamers? Do you work with people like Apollo, Haize Labs, or METR—the usual suspects?

We publish these Gemini technical reports, and we usually detail external red-teaming, but I can't comment on who our partners are at this stage. I think there are good reasons why we don't always discuss who our red-teaming partners are, but we do work with external red-teamers.

Gotcha. When the technical report comes out, will that have the roster of the external partners?

I think I'd have to check, but my understanding is that in our past technical reports, this is something we acknowledge.

Yeah. Okay, cool. Fantastic conversation. I really appreciate you working through all these questions with me. I guess, in closing, are there any other thoughts or notions that we didn't touch on that you'd like to leave people with?

I'm curious: You've played with 2.5 Pro a little bit so far. Are there any things that you found it was unlocking that you hadn't seen before, or any feedback you had?

The long context, for me, was the thing that felt different. I have a general complaint with almost all RAG apps, regardless of whether it's an IDE-integrated one or otherwise. I feel like they don't—and often this is more of a business problem than a technical problem—because I pay a flat monthly amount for whatever product they want to have some margin on.

They set the hyperparameters in a way that tries to give me the best performance they can while also not spending too much money and burning all the cash they have. That typically leads to not enough context being included in the model calls. Then I feel like, “Oh, God,” because so often there's just something that could have been there that wasn't there, which was leading me not to get as good an answer as I could have.

What I often do, if I can, is print my entire codebase to a single text file and then paste that into the model. I do a lot of small personal projects and proofs of concept.

Usually, I can get away with 100,000 tokens or whatever. I can put that into any of the leading models. But this recent one with the research codebase—I happen to be the least valuable author on the Emergent Misalignment paper. Long story, but I call myself the Forrest Gump of AI because I sometimes wander through these important scenes as an extra. And this happened again here.

I had this research codebase, and it's not production code. Folders are sort of named “Daniel folder,” “Nathan folder,” right? It's not best-practices software engineering, but we're all just exploring stuff. This was 400,000 tokens, so it was significantly too much for me to put into any other model.

The command that it had of it was just incredible. I really was like, “Boy, previous Gemini models obviously could handle that much, but I was never 100% sure if they were really in full command or only in partial command.” This felt to me like really incredibly strong command of that full context window, and that felt like a real game changer.

Without having strong benchmarks or anything to really ground myself, my feeling is that I can take dumps of information and have much higher confidence. I still don't want to be overly trusting, of course, but I feel like I can take dumps of information that I don't even necessarily know what's in there and be much more confident—though of course still not fully confident—that the 2.5 model will latch on to what's actually important and help me navigate this super-deep context, even if I myself don't have a good sense of what's in there at the start.

That feels like a huge difference to me, because it's one thing to be able to help you navigate long context if you know the long context yourself, but it's a very, very different thing if it can help you navigate long context that you don't have great command of. I think there's more work to do to really validate that for myself, and obviously for the community at large and for you guys all working together, but it feels different. I can say that for sure.

That's great to hear, because I know I worked with a lot of the long-context people last year, when we were in the run-up to the original breakthrough. I communicate a lot with them because I used to be in pre-training for a long time, with some of the people who had been particularly focused on making long context really good for 2.5 Pro.

There was a lot of work, not only in the initial phase to make 1 million, 2 million, and we'll see more happen, but also to make it really effective. With the 2.5 Pro release, I actually forget the name of the external leaderboard, but I think there is an external leaderboard. It's shared on X, where, at 128K context, Gemini 2.5 Pro is using it way more effectively than basically any other model out there right now.

It's not only that it can go to 1 million, but now, especially with 2.5 Pro, we're seeing that it feels like it's read everything. It's not dropping things or missing key details. It feels like it's read and studied all that information.

That kind of gives people a bit of an AGI feel. Within a second, you feel like you've studied a very large codebase and know every detail at quite a good level of understanding. That's quite a remarkable thing. But yeah, that's great to hear.

It's well-deserved praise. These step changes—I’ll never forget where I was when I first tried GPT-4. There aren't that many moments in the last 2.5 years where I felt like, “Oh, this is qualitatively different from everything I had used up until that particular moment.”

But this was one. It really did have that quality where it was like, “Okay, I can feel a new level of unlock. I'm going to have to recalibrate myself a little to what this makes possible.” Definitely an exciting time.

This has been fantastic. I really appreciate it. The final send-off, of course: Jack Rae, principal research scientist at Google DeepMind. Thank you for being part of The Cognitive Revolution.

Great. Thank you so much for having me. Cheers.