Asimov：与 ReflectionAI 的 Misha Laskin 共建全知 RL 预言机

Hi, listeners. Welcome back to No Priors. RL is back with a vengeance, and one of the most talent-dense new research labs has a product release, a new code comprehension agent. Reflection AI's co-founders, Misha Laskin and Ioannis Tsantéoglou, worked together as leaders at Google DeepMind on groundbreaking projects like AlphaGo, AlphaZero, and Gemini. I talked to Misha about building universal superhuman agents, the trickiness of reward modeling, bringing all knowledge work tasks under data distribution, how RL for language and robotics differs, the Windsurf non-acquisition, and the landscape from here. Misha, welcome. Thank you for doing this.

Yeah. Thanks, Sara, for having me.

So it’s been a wild year and a half since you guys started the company. Is that about right?

Roughly a year and a half, maybe a bit less, but I’d say that’s ballpark correct.

Well, can you just start by describing—you’ve said that the company’s mission is to build superintelligent autonomous systems, and we’ve talked before about why this is the moment in time when that’s possible. What is different about that from building just superintelligence, which is now a more popular ambitious goal?

1. Superintelligence Starts With Products

At a high level, it’s fairly synonymous, but maybe there are different ways of thinking about how to build superintelligence and what that might look like. I think on one spectrum, there’s an academic way to look at it, which is that, in some sense, superintelligence has already been achieved. AlphaGo was a superintelligent system, and there were other systems during that time that were built that were superintelligent in narrow domains.

I think you can go for the goal of building a very broad superintelligence by locking yourself up in an industrial lab that’s decoupled from products or customers, maxing out all the benchmarks that are out there, and building superintelligence that way. I think that is one approach.

I think the other approach is to think about what superintelligence is more concretely: How is it going to be deployed? What is it actually going to look like in people’s hands? Then you build backward from there. I would say that approach is more about co-designing product and research together.

The benefits of that approach are that you’re optimizing for real problems. The con is that you have to be a lot more focused, because your product defines the capabilities that you want to draw out of the system. You have to start out a lot more focused before expanding across other product categories and other capabilities.

I would say that on the spectrum of companies that are building superintelligence in just a research lab and then figuring out what the product is once it’s built, as opposed to co-designing product and research together to build very powerful systems in what I would call ASI-complete categories, you can pick something that’s maybe too small of a category to draw out a superintelligence. As long as you pick a category that I would say is big enough to be ASI-complete, I think—and this is our approach at Reflection—it makes a lot more sense to be focused and co-design those two things together: the product and the research.

I want to come back to the choice of initial problem in a minute. In terms of just having the intuition and the confidence to say, “We can go do this as a team. We’re going to recruit great people and go build Reflection,” you and your co-founder, Ioannis, were working at Gemini together in key roles before, and previously you had been part of Pieter Abbeel’s lab, who’s an amazing researcher as well.

You described to me as having, I believe, somewhat muscled your way into AI and deep learning from originally a physics background. How did you decide to go work on this and end up in Pieter’s lab?

2. Physics Leads To AI

Yeah. As a kid, I became really interested in physics—­theoretical physics. It was probably a byproduct of the fact that I’m Russian, kind of Israeli-American, and moved around. When I landed in the States, it was in a desert in Washington State, learning a new language. So I had a lot of time on my hands and bumped into—my parents had the Feynman Lectures in their library, and I spent a lot of time just reading what was on the shelf. I bumped into that and got really interested in physics.

How old were you?

When my interest in physics started, that was probably around middle school, and it really became the thing I wanted to do in high school. The reason physics was so interesting was because it seemed like the science that was at the root of many of the things that became impactful.

I was reading about the history of the transistor, and it was invented by a group of theoretical physicists. I was reading about how GPS works. It turns out you need special relativity in order to accurately account for spatial coordinates using GPS. So I felt that physics was the root science to pursue.

I went in and studied it and got my PhD in it. At the same time, I started seeing deep learning take off and really saw AlphaGo happen. My sense was that I wanted to pursue the root science, but there isn’t such a thing as the root science of our time.

I think physics as a field is very interesting, but it has crystallized a lot more than a new, dynamic field that was being born out of nothing. AI felt like it was going through the moment that physics went through maybe 100 years ago. When I did problem sets in physics, the most exciting stuff I was working on was basically the things that people were discovering 100 years ago. So I saw it happening in front of my eyes, and I decided that was the science to bet on.

In particular, AlphaGo inspired me because it was just unbelievable to me that you could train a neural network to have such immense reasoning capabilities. This thing was able to be superintelligent within the realm of Go. I decided that I needed to get myself into the best reinforcement learning lab I could. Peter’s lab was that lab for me.

And then you and Yannis were working specifically on RL at Gemini.

That’s right. Yannis, my co-founder, was the overall RL lead for Gemini at the time, for 1 and 1.5. I was working very closely with him on his team.

It was a really exciting time because we both went from being reinforcement learning researchers to training large language models at scale. At the end of that project, we saw what was to come. Gemini 1 and 1.5 landed, and it became pretty clear to us that the next paradigm—and effectively the final paradigm that we need to have in place before what people used to call AGI, or what I think the goalposts have shifted to now, ASI, is reached—is figuring out how to scale reinforcement learning on top of large language models.

The first instances of that have been happening over the last year. I think we’re still actually a lot earlier than people think, but there is a wedge in, and things have started to work.

Yeah, I definitely want to talk about what you think is solved and unsolved here. The entire field has clearly gotten more focused on deep reinforcement learning over the last 18 months. You have this huge product launch this week with Asimov. Can you just describe what it is?

3. Asimov Understands Complex Code

Asimov is the best code research agent in the world. It’s a comprehension agent, meaning that it’s really designed to feel almost like Deep Research for large codebases. The way a developer is supposed to feel interacting with it is effectively like they have a principal-level engineer who deeply understands their organization at their fingertips.

It’s very different from the existing set of tools that are focused primarily on code generation. Every coding tool has some code-generation and some comprehension aspect. But as we spent a lot of time with our customers trying to understand why coding tools—and this is enterprise-specific, so I think the world is different with startups—but within enterprises, when they’re adopting coding tools and you see the impact that this is having on their actual productivity, I think it’s much lower than people expect. In fact, it’s sometimes negative and sometimes negligible.

Did you see the recent METR report on that?

Yeah. The METR report was very close to what I’ve been hearing when talking to engineering leaders within larger organizations. It’s not just enterprises; I’d say it’s growth-stage startups as well. It’s any kind of engineering organization that has a sufficiently complex codebase and sufficiently large team that no one engineer can have the entire codebase in their head. Reflection is one of those places as well.

We use our product actively because training large language models is complex. There’s the large language model code base, and there’s the product code base. Knowledge is scattered across engineers. It’s not just in the code base; it exists in your chats, project management tools, and other places where knowledge lives.

What we’re effectively building toward is this kind of omniscient oracle for organizations. You can go in, ask any question at any level of complexity, and it’ll provide you with an answer at the level that a principal-level engineer would have given you—or, in the future, as the product expands to other categories, at the level of what the person who’s most embedded in the organization understands. Of course, once you have that solved, it begets much more reliable agents that act for you as well.

I think the world today is focused on, I would say, 80% action and 20% understanding: 80% code generation and 20% comprehension. The actual problem is exactly the opposite. When you look at what an engineer does in an organization, 80% of their time is spent trying to comprehend complex systems and collaborating with teammates. And what is collaboration? It’s usually someone asking someone else a question about a system they don’t know.

That, I think, is the problem at the heart of what would prevent a superintelligence from actually working within an organization. It’s really about understanding and being able to ingest information from a lot of sources and from the team. Once you have that, the action part becomes—I don’t want to say trivial—but a lot easier. To me, it seems like really 20% of the problem is teaching these agents how to act, and that’s more or less solved.

That definitely squares with both my understanding of engineering and my personal experience with coding agents. If you think about the context-load time required just to understand a new system, or code anyone else has written, or code your agent has written, in the end, it’s a very stupid implementation that, if you had reasoned through it with the context of the system, you never would’ve made such a mistake—or a “works in my environment” type problem. I think that very much mirrors my intuitive understanding of engineering here. That’s great problem formulation. What makes Asimov different in terms of its ability to understand better versus just generate code?

4. Long Context Drives Comprehension

There are a few things. This is why it’s so important to co-design research and product. As a researcher, you’d go in and say, “The answer is entirely in the agent design or the model,” or something like this. As a product person, you would say, “Well, it’s in these product differentiators”—being able to draw not just from your code base, but from knowledge that lives in other sources of information, or being able to learn from the engineering team to offload their tribal knowledge.

An engineer can go in and teach Asimov, like, “Hey, when we say environment jobs on our team, we mean this specific thing—kind of Google bat jobs.” Now, when another engineer asks a question about environment jobs in the future, the system just knows what they’re talking about. A lot of knowledge is stored in engineers’ heads, and I think you need both of these things. You need to understand your customer really closely and develop a differentiated product almost independently of the models that are powering it.

But then you also need to innovate on the research—in terms of agent design and model training—to actually drive the capabilities that you want to see out of the system. This becomes an evaluation problem, which is basically at the heart of any frontier lab as well. This is, I think, the least-spoken-about part of what frontier labs do, but possibly the most important: figuring out how they evaluate.

What makes Claude magically feel better at code than another model out there? They did something right in their evaluations. When you look at this problem specifically, there are different capabilities that you need to train. What we do is really post-train models; we focus on post-training today.

Some of these things are long-context reasoning. When I say long-context reasoning, I don’t mean—I actually mean small models with very long contexts that are able to go into giant code bases, suck up as much information as they can, reason over it, and output relevant stuff, basically. It’s almost like neural retrieval.

There are capabilities like tool use and multi-hop reasoning. This is more for an agent: you have your agent, and it’s designed with some tools. There are 2 ways of training agentic models. One is in this very general way, where you train it on thousands of environments and make it the most general agent possible.

That is almost like the pre-training of agents. That’s what a frontier lab does. There’s a new release from Kimi, Kimi 2, and that’s kind of what that model does. That’s definitely part of it. It gives you a nice general base to start from.

But then, to drive a capability depth-wise—if you really want this reasoner that has search tools, the ability to call long-context reasoning models, and other tools that it might want to interact with, like, “When do I read from Jira? When do I read from another tool?”—this is a reasoning problem. If you train with those specific tools in mind, that’s typically what people refer to when they say tool use. They actually train for a specific set of tools and really drive the capabilities for those tools.

These are the kinds of research problems that you need to solve in order to build the overall system that’s the best in the world. It’s not any one thing; it’s all these things combined. An example of a system being trained for a specific set of tools is Grok—the Grok 4 release. They showed a plot of their general model and then the model that was trained with a tool to basically perform on Humanity’s Last Exam. There was a big, noticeable difference between the 2.

That’s great, but I think the downside is: Does Humanity’s Last Exam actually matter in any meaningful way for an end user? I would argue that there’s a weak correlation, but the answer is most likely no. You have to build the tools and train for the things that users actually want. I think there’s no way around that.

What can you share about how you evaluate, either technically or philosophically, that makes Asimov’s performance great?

This is why it makes sense to do something like this as a startup. The only advantage that you’ll ever have as a startup over a big incumbent—especially when there are such talented teams out there—is focus and velocity against the thing that you’re focused on.

If you want to be playing in what is, arguably, I think the biggest category in AI, which is coding, then you need to have the talent as well to do it. But what do you do if you don’t have the billions of dollars to pre-train models? The only way we can win, I think, is by being very focused.

The way I would describe what it looks like to work on a big model within an incumbent lab is that you are 1 of hundreds of evals. When you look at the model card for, let’s say, the o1 paper that came out last year, if you look at the distribution of what most people worked on in that paper, it was evals. You’re 1 of many people doing all sorts of evals and spreading yourself in that sense. You get something that’s general, but it’s spread fairly thin.

As a startup with a very focused product—one that isn’t being too diffused and is pretty opinionated about what it’s building—your evals are basically what, in the startup lore, Paul Graham would tell you to do: go talk to customers. Spend half the time building the product and half the time talking to customers.

I think in the AI age, it’s developing your evals based on what customers are saying and what they’re doing. You have to work with your customers to look at what prompts they’re trying to solve and what general questions they’re trying to unlock. There are very specific pain points that we’ve identified, like onboarding being 1 of them.

In a big company, it takes months to onboard an engineer. How do you develop evals that accelerate the onboarding of an engineer from months to hopefully just a couple of weeks, now that all the questions they had can just be asked of Asimov, allowing them to onboard much faster?

I think there’s no silver bullet other than coupling to the information coming from customers, but then being very scientific in the evals that you develop across them.

So you have these customer needs, let’s say onboarding and a bunch of others. Then you have your system capabilities, which is: What do you need in order to provide a good experience there? This customer is being onboarded onto a giant code base. It might be a code base that, on its own, is 100 million tokens or something. Well, then you need to figure out some way to reason over that giant code base, so you have a kind of long-context reasoning capability.

Or you look at your agent and ask, “What’s preventing it from satisfying this query from a user?” You work backward and reverse-engineer from what a user is asking for to what capabilities you want to drive in your system. But the important part, I think, is to be able to tweak every part of the system, from the product features to the agent design to the model training, in order to build the best overall system. If you’re capped in which parts you can change—if you can only change the product and agent design—then you’re actually pretty limited in what you can do, because you’re at the mercy of what these general third-party models can do.

What I’m hearing from you is also that there’s some trade-off between serving all different kinds of users and optimizing across those different evals, because each one of the teams that is thinking about a particular use case or audience at a more general organization, for example, is less likely to have the ability to work through the entire pipeline, from training to product, to win their use case.

The thing that was extremely satisfying about working on Gemini is that you’re driving research at the frontier, and there’s something very gratifying about that. The downside was that you were so far removed from the product that it was kind of a broken-telephone game, talking to the 4 different people that information flowed through before the model got into a customer’s hands. That coupling was very loose, and I think it’s very true that just because a company might have the best model on some general set of academic benchmarks doesn’t actually mean they have the best product.

I think what we’re seeing is that when things really fit together, there’s usually a tight coupling between a product and a model. It’s a whole system; it’s not just the model alone. Obviously, the first big example of that was ChatGPT. ChatGPT is an incredible product that was coupled with the model, and the model was post-trained for the prompts that were coming in from users of ChatGPT.

When I saw the first coding blog post that ChatGPT produced for me, that was just insane. It was an insane, magical moment, and they post-trained specifically for that. I think there’s another example of that happening right now with Claude Code. That’s a tight model-to-product coupling. I really think that it’s important to be able to do both at a great degree of excellence.

What is an example, as you guys open up the waitlist, that you want users to try where it should just be obvious that the answers are better than those of other coding agents?

I think the kinds of queries that it tends to be better at are what we would call semantic queries. An example of a query where this is not the best system to use is a file-level query. If you’re looking at a file and there’s a specific thing in that file, and you’re just trying to get a quick answer to it, you don’t really need the hammer of a deep-research experience. You don’t need to wait tens of seconds or a minute or 2 to get that answer, because that should just be delivered snappily.

But if you don’t exactly know where you’re looking, and you don’t know the function name or something, then this is useful. These are the hard problems that engineers are usually in. There’s a flaky test. You know that this test is flaky, but that’s where your knowledge stops, right? That’s when you usually go to Slack and ask some engineers, “This test is flaky. What’s going on? Does anyone know?”

The way we’ve used it is when you’re training these models, there’s a lot of infrastructure work that goes into it, and it fails in interesting ways all the time. You might ask, “My jobs are running slowly, 5 times more slowly than usual. Why is that?” That’s a vague query that would be very hard to answer with existing systems, especially since the knowledge around that query might live not just in the code base.

In the example I just brought up, when our environment jobs were slowing down, it turned out that 2 different teams—the infrastructure team and the research team—submitted pull requests that passed tests. It wasn’t that they were wrong, but they conflicted in a way that caused what was effectively a race condition and slowed everyone’s jobs down. These are the kinds of bugs that engineers spend a few days trying to solve. You have 2 or 3 engineers working on one of these problems.

I think these kinds of semantic queries tend to be the place where a product like this shines, in the same way that when you think of what kind of query you would ask ChatGPT when it just needs to use the browser tool, it’s a quick factual thing. You wouldn’t invoke the deep-research experience. But when you want to compile a lot of information around some more nebulous query, I think that’s where people seem to find a lot of value with deep research. A similar kind of mindset holds here.

One thing I would do, working on a new system with a principal engineer next to me, is just have them explain the entire system, right?

Yeah.

Because I want to have that context, or I can’t even tell the agent what to do. I’m curious, from a product perspective, about the way you have memory for agents or even for teams. It’s an increasingly popular idea, and there are lots of ideas about how to do it. I don’t think there are many examples of collaborative memory in production in a useful way yet, but I’m sure it’s coming. Have you guys designed it in a form I can understand, too?

Yes. This is actually one of the more fun things to work on in product today, and I think it’s one of the more fun kinds of features to work on at the company: How do you design a team-wide memory?

Mm-hmm.

There are all sorts of details around who can edit the memory, who can view different parts of the memory, and how you maintain a repository of this memory for people to edit and view.

You have to have a concept of authority, right? People are going to say things that are wrong.

Right. The way it’s worked with the customers we’ve started working with is that they typically want to start off with a group of trusted, senior, staff-level-or-higher engineers who are the gatekeepers, which is a very common notion. You have permissions and ownership structures in code bases, and they’re basically the ones who populate the memory first and then expand the scope.

But I think it works—it’s actually a much more complex feature to build because it touches on organization-wide permissions. There are some parts of the code where a certain engineer should be able to edit the memory, but other engineers shouldn’t. It actually starts looking like the new way of versioning code, effectively. It’s kind of GitHub++, because you’re not versioning the code; you’re versioning the meta-knowledge around it that helps language models understand it better.

We’ve built that, but I think it’s something to iterate on a lot until you get the right design, because you’re effectively building a new Git from scratch.

Yeah, it’s interesting. You’re trying to design some sort of permissions into it, versus the dominant system today in actual version control, which is, at best, pull request review. You try, and somebody in the organization with the ability to review makes a determination as to whether or not Misha should be able to make this change, based on the content.

Right.

That’s quite different from traditional role-based, group-hierarchical access control, which is quite static. It makes sense to me that it would look perhaps a little bit more Git-like, in that the person who knows what part of the code base you are creating or editing knowledge about is going to evolve over time as the code base evolves and the team does as well.

Yeah, exactly. But I think this was also very common at Google and, I think, other places as well: different parts of the code base had owners, and there were ownership files. Basically, if you're on the ownership file, then the review has to go through you, or it has to be approved by at least 1 of the members of the ownership file. As people move around teams and so forth, the ownership files themselves get updated. So I think a pretty similar structure is probably going to hold here, but it's a lot more nuanced than building an individual memory, which is just personal to you and lives on your computer in your agents on D file or something.

Okay, if we zoom out and place Reflection overall in context a little bit and talk about the larger environment?

Sounds good, yeah.

Coding as a root problem in this era of AI research is a somewhat commonly held belief, right? One criticism of companies that went after pre-training focused on coding was that, in reality, you actually needed language. You needed a lot of the capabilities—who can say exactly which—but you needed the reasoning capabilities that could be elicited from large pre-trained models to do code anyway. And so you had to do all of the work without the general use.

Is it specifically the availability of pre-trained models that are more capable in open source that made you feel like we can go after superintelligent, autonomous systems in coding without spending the pre-training dollars up front as a new lab? Or help me think about that logic a little bit more.

5. Open Models Lower The Barrier

I think that's roughly correct for why you can get into the game in the short term. A bet that we made when we were starting a company 1.5 years ago was that there'd be pretty decent open-weight models out there, and that pre-training—we saw pre-training as starting to more or less converge on a known paradigm. There's a known big data set on the internet. Yes, there are going to be some algorithmic innovations, but you're basically extracting signal from an extremely noisy data set, and we felt like there's only so much signal that one would be able to extract without getting into just absurd dollars for scaling this in terms of what you're trying to get out of it.

So what we felt would happen is that there'd be decent open-weight models. I think the quality of the open-weight frontier has surprised me. The models are actually better than I thought they would be.

Mm.

And we thought that you could just focus on—you know, we're in this brief period in history right now where the RL FLOPs are still manageable. You can really have a best-in-class product if you're focused. And yes, you still need a decent amount of GPUs, but from a FLOPs perspective, it's nowhere near where pre-training is.

2 orders of magnitude off.

Exactly. So you can get into it and build out both a product and a research arm. Our thought was that this was the time when you could actually start a generational frontier lab that does not need to be coupled to a big cloud provider. Because if you do it right, you'll actually be able to generate sufficient revenues not to have to be acquired or find some strange deal where the cloud provider kind of owns you. And that was the model, I think, of a lot of what frontier labs looked like pre-LLMs.

I think we're already starting to see that this is more of a field-wide thing independently of Reflection, right? When you look at how fast Anthropic's revenue is growing, I think they're kind of in this spot where it's a massive revenue-generating business that's growing at an unprecedented rate.

But that was very much the ethos: We can come in, we don't need to pre-train. You can get by with 2 orders of magnitude less compute and really get something out there that's really good. I think that, roughly speaking, you won't need the amount of compute that I think a frontier lab needs today if you're focused, but you'll still need 1 order of magnitude less.

So I think the capitalization requirements are still high. There's no way of avoiding that. But asymptotically, they're probably the same. The idea is that at that point, you just have a generational business that can raise capital off of that.

I guess part of my read at this point in time is—and maybe it was always true, but especially now—that your actual capabilities in terms of understanding what evals to go after and how to design reward models are perhaps less understood, with more dispersion in the field, in post-training strategies versus, as you said, more maturity in pre-training right now. If it were a simple question of scaling RL on language models, people would be doing it more aggressively right now, right?

Maybe that's a good question for you: How would you describe the challenges in scaling here? Why are we only able, as a field, to put a much smaller amount of compute to work here and still get best-in-class results versus pre-training-scale GPUs right now?

6. Rewards Define The Frontier

I'd say there are 2 categories where one would think things fall. One is more about limitations from the problem structure, and the other is, well, maybe the structure is fine, but you need algorithmic advances to really drive the next frontier forward. There's some mixture of both, but the biggest weight I'd put is on the problem structure.

The thing that I led for Gemini was reward models. I built out the reward models that were used to post-train Gemini 1 and 1.5. And my thought is that if you have a reward that accurately describes the outcome of any arbitrary task that you throw at it, then that's it. At that point, it's just algorithmic advances. But even the very simple RL methods we have today will be able to get a lot out of this. They'll only be bound by their exploration abilities. So that's the only thing, right?

But today, we certainly are not in this world where we have clean rewards for every task we could imagine. And so, as a field, we have to make various shortcuts and compromises to that. You'll have things like LLM-as-judge with different rubrics, and that works to some extent, but a noisy or stochastic reward inevitably gets hacked. So you need a lot of these, and there's only so much you can extract out of them.

Then you have sources that do have ground-truth rewards, but there are not many of them, and so you have to hope that by optimizing against those, you'll get some generalization effects. And so I think the fundamental problem is the reward problem. You can either go in and say, “All I'm going to focus on is rewards,” or you can say, “I'm going to take things as they are and just be more creative in the methods that leverage the rewards I have today.” Basically, every synthetic generation pipeline is an example of this.

It's a messy problem, but I think fundamentally we're in a reward-bound world. I don't think there's going to be any breakthrough where, all of a sudden, we go from “We didn't have rewards for everything” to “We do,” because the reward problem in itself is, as I thought at the time, AGI-complete. Now I'd say it's ASI-complete. By the time you have a neural network that can accurately verify any outcome, that is probably a superintelligence.

And so then it goes back to evaluations. If you're training your reward models on something, what are you evaluating against? What are the tasks that you want it to be good at? So that's kind of how I think about it. I think it's fundamentally a reward-model- or rewards-bound field.

Then there's also algorithmic progress in terms of the RL methods we have today. They're quite bad, I would say, at exploration and credit assignment. They're sort of designed to take the things that work and make them happen more frequently, and the things that don't work and make them happen less frequently. But they don't discern at all along your reasoning chain which part of the reasoning was correct and which part was incorrect.

And so you get these reasoning chains that are kind of garden-path meandering. They'll explore all sorts of things that are completely unnecessary and don't look at all like the kind of structured thinking that a person would have. That's how the algorithm works. It doesn't actually look at—there's no credit-assignment step at any atomic level. And so that, I would say, falls into more algorithmic-progress bottlenecks.

Can I ask you for a few hot takes quickly?

Yeah, let's go for it.

What do you think of all these efforts—either in-house with labs and vendors or from young companies—to create software environments that look like popular software to train agents in, like copies of Airbnb, Amazon, Salesforce, or Excel?

Personally, maybe the take is not very hot. I'm very bullish on it. Maybe the hot take is that there's no such thing as generalization; there's just bringing the test distribution into train.

Okay, that is an aggressive take. Wow. Yeah.

So as long as your train distribution looks something like what you would actually want to evaluate for, then users will experience it as generalization. I think there is some generalization that happens in these models, but we probably, as users, overestimate it because we don't actually see how they were made. But then, if you saw, “Oh, this synthetic environment was actually very similar to the thing I was asking about,” it would make sense why the model would be good at that.

Maybe 6 months ago, I think you said, “I think it's possible we have my definition of ASI in a couple years.” Do you still believe that's true?

I still do believe that's true. I think that where we'll be a couple of years from now is that there'll be definitive superintelligence in some meaningful categories of work. When I say coding, I don't mean all of coding. There will be a superintelligence within some meaningful slivers of coding that are driving immense progress in the companies that can benefit from that.

Mm-hmm.

The reason why I would say that the problem of ASI would've been solved by then is because, at that point, it's just a matter of operationalizing what you know. It just so happened that these particular categories—you might have a superintelligent front-end developer because there's so much data for that distribution on the internet, and it's easier to make synthetic data for that. But at that point, you have the recipe, and it's just a matter of making economic decisions about whether it's worth sinking X amount of dollars into getting the data in this category, to get something close to superintelligence there.

An example of that is what happened with reinforcement learning before language models. Effectively, the blueprint for building superintelligent systems was developed. It happened with the Atari games and AlphaGo. Then Dota 5 and AlphaStar were near-superintelligent systems, and if OpenAI and DeepMind had sunk more compute into them, they would've definitely become superintelligent. It's just that, at that point, it didn't really make sense economically. Why would you do that?

Then this is a definitional issue, because I was going to ask: Help me understand your view. One of the big criticisms of RL overall has been lack of generalization. That's been a general question for this direction. I do have friends at every large research lab that, in a sense, believe the following. Tell me if you hear something of a different tenor or just believe differently: They believe we're going to have systems that are much more capable than humans in many types of knowledge work, but they believe less in generalization. In a resigned way, they're also, as you're saying, like, “I guess we're just going to bring all of it under distribution one way or another.”

Yeah.

But that means it's a little bit different from my view that, at some point, you just have enough capability that the rest you get for free—the rest of the useful capability you get for free, right?

I think I have a similar viewpoint to the people you describe. I think the generalization capabilities of these things have been weaker. First of all, it's mind-blowing that this exists. We went from having fundamental existential crises about generalization.

Before language models, the field of reinforcement learning was: We have these systems that we can make amazing at very narrow tasks. We have absolutely no answer for generalization—zero. We went from that to things that feel like they're generalizing. They're certainly generalizing much better than anything we had before. But it's likely because the training distributions are so broad.

At least the way I think about it is more about output as a user. Is the system superintelligent in some meaningful categories of work? Then, from a research perspective, is it obvious how to make it general for anything that you might care about? At that point, again, it's just a matter of economics. Maybe there are some categories where collecting the data is so expensive and the return on investment is low that, effectively, it's just better to have craftspeople than superintelligent AIs.

Yeah.

So I think we're moving into this kind of jagged superintelligence, where you have a handful of these superintelligences for categories that matter, maybe subsumed into one model at some point. But at first, it'll probably be a few companies that have product-model coupling that is superintelligent in different categories.

I think an example of starting to see the first glimpses of superintelligence, but in a way that hasn't really transferred to anything meaningful yet, is that we have these superintelligent test takers now. The AMIE benchmark is completely saturated. In Codeforces and other competitive coding environments, the models are almost the best in the world, and within the year will probably be the best in the world. And yet we have the best competitive coding agents.

Then you go into a company and ask them, “Have these things been helpful?” And they say—

It's uneven, yeah.

Yeah, right. So, in the parts of work that are really meaningful, that's where you want to see these things driving meaningful increases in GDP. I think the only way you will see that is if you go into a company and there's a universal understanding that, yeah, my engineers are, as a whole, every single one of them, more productive by double-digit percentage points. That's the kind of thing where, if that starts happening across every field, then you'll see double-digit increases in GDP.

So I think that kind of benchmark maxing is a bit different from what benchmark maxing used to be, because you have benchmark maxing that is weakly correlated to customer outcomes. But it still looks very similar to taking a board game, training an RL agent on it, getting a landmark result in superintelligence, and then making a claim that superintelligence is solved. I think the reality is that deployment of it is half the problem, which goes back to evaluating on customer problems and building the product together with the models.

So you must have seen the news of the Windsurf non-acquisition by OpenAI, but non-acquisition by Google DeepMind. What do you make of it?

7. Frontier Labs Go Vertical

We're seeing this verticalization basically happen across categories that are material to frontier intelligence. One could argue that the first verticalized category was actually search, through ChatGPT. That's sort of the place where OpenAI verticalized first. Coding has obviously emerged as another frontier-level category that all these companies have aspirations of—

ASI.

Yeah, ASI. But then I think, being basically trillion-dollar companies or more, I don't think economics is really the driving factor. It's more that, if you want to sustain frontier research, that's kind of what you have to become. Coding has clearly become one of these categories where verticalization is extremely important.

I think there are two sides of the story: one on the frontier lab side and the other on the product side, like a startup that builds product but does not have its intelligence in-house. On the frontier lab side, I think this is exactly what Yannis and I noticed when we were working in Gemini: Your model is so far away from the product that, oftentimes, even though you have the best model, that does not at all mean that you have the best product.

There's a reason why startups are the places where adoption of coding tools took off rather than the frontier labs. So there's a verticalization happening there, and some are going to do it successfully and some are not. I think we're already starting to see that with Claude Code really being an example of a successful verticalization.

I don't think it's guaranteed that a big lab can buy its way to the end user, because the fundamental problems of your research team being far away from your product team will still be true, and the company having 100 different focus areas will still be true. So I don't think that acquiring an asset will change that fundamentally. But it does underscore the importance of verticalization.

Then, from the startup side, I think it actually puts companies that are in these critical-path categories, like search and coding, in a pretty existential place if they can't build their own frontier models.

Not all frontier labs will be able to verticalize correctly, but some will. Maybe one will, and that's going to be enough, I think, to take the thunder out of a company that's built a great user experience on top of someone else's model. I think some of those dynamics are probably starting to play out as well. There are some question marks around if you're in this critical-path category and you don't have your own intelligence: How do you compete when your competitor can just subsidize their product a lot more than you can?

Because you're effectively, as a startup that's building on top of these things, to grow quickly, subsidizing the margin that an Anthropic or Gemini or whatever is making. Google and Anthropic and OpenAI can subsidize their products a lot more than you can. So I think that companies that don't own their intelligence or are not deeply integrated into a customer in some way that makes them hard to remove find themselves in this pretty existential place as it becomes clear to the frontier labs that this is a category they need to verticalize around.

I work with a few robotics companies, and so much of my lens on RL comes from that. I think it is far less clear in robotics that RL will be a dominant part of the training versus imitation learning. You'll actually appreciate this point about imitation from humans using tools, right? We run this grant program twice a year for amazing people using ML in different fields, and it's called Embed.

One of the ideas I had as a joke recently was: You should just record everything—not obviously just the codebase, but your Slack, all your documentation, and all your conversations—because you are a software engineering team. I'm 100% sure that I can take that dataset, if you ship something into production to an end customer that has real issues at any scale, and sell it to a friend who's a researcher at a lab working on this stuff. So you have some floor value that's millions of dollars for your couple-person company, plus bonuses. Maybe the software company works, right?

Mm.

There we go. See? It's not an idiot idea.

Yeah, especially if there's diversity.

I can sell the whole set.

Yeah. So is the question how you leverage alternative sources of data?

I don't want to over-analogize to robotics, right?

Mm.

Within robotics, you have learning from world models, learning from simulation, learning from embodied data of different types, imitation, and then you have RL. I think it's much less clear that you can use RL for a lot of robotics today, especially some of the harder manipulation problems. I'm curious, given that your team has this enormous strength in RL as a starting premise, how you look at other types of data to create the coding-agent experiences you want.

I was actually a robotics researcher in reinforcement learning. Pieter Abbeel's lab is a robotics lab, and it was a mixture. Pieter's lab was always around the intelligence problem, with robotics as a domain where you study it. One reason I came to lead reward models for Gemini was because that's the question I was studying with robotics.

We had these RL algorithms for getting robots to do some very narrow tasks, like moving blocks and various other narrow tasks in simulation, and the question was, “Well, how do we get generalized manipulators? How do we build this all into one system?” It seemed like the rewards were a bottleneck, so a lot of what I was studying before getting into language models was how we design reward functions or models for robotics or for 3D video games like Minecraft, which have, I think, similar challenges scientifically.

The challenge is that if you think language-model rewards are hackable, vision-language-model rewards or other sensory-signal rewards are infinitely more hackable. They're much more short-lived than language rewards. You can think of language as just a compressed representation of the world that we have, which we kind of magically get to start with, whereas if you're processing pixels or a sensorimotor signal, this is a raw signal that has a lot more noise in it.

If you train a neural network that's trying to detect whether this thing was manipulated correctly or this thing was moved correctly, then that thing is just infinitely more hackable than anything you have in language models. The same problems blow up and become much larger. That's actually why I changed to language models, because I felt that this was a fundamental problem, but we now have these confounding factors of these noisy signals coming in.

I think that, at least in a generalizable way, that's why it's really hard to get reinforcement learning to work with robotics. The one place where it really does work well is when you have a clean reward signal, which happens to be in these locomotion-like scenarios.

Mm-hmm.

There's a lot of work on building very robust sim-to-real locomotion pipelines, and it's because locomotion is just your body. You don't have to manipulate the world around you, and so you can actually build reward signals that are like, “Your quadruped is moving at this velocity without damaging its body,” kind of thing.

Maybe it's a bit of a roundabout answer to the question, but I think these 2 fields are very different in the data distributions they support. The kind of imitation-learning data for language models is, of course, the internet. It's all the data that people have gathered on how we write and so forth. Aside from that, when we're generating synthetic data, the only scalable path is really reinforcement learning.

The other thing that I'll say here is that when you're collecting data for robotics, you can do it in a teleop way. The things that we're trying to train robots to do are very intuitive for humans as well—actually, more intuitive for humans.

Mm-hmm.

People are master manipulators, so you can have a lot of teleop-like data collection. The things that we want language models to do are at the level of the chain-of-thought process that goes on in a human's head when they're trying to solve some task. It's really hard to collect data on that, and that's kind of the data that you need.

For that reason, I think language models favor this more synthetic-data, RL-like approach where it's easier for us to verify whether the thing was done or not than it is to actually generate all that data from a person specifically.

Maybe we just need a network interface to get the chain of thought.

Yeah. Maybe. Actually, when Yann and I were starting the company, we were thinking about maybe somehow having people speak into a microphone as they're doing tasks in order to capture that.

Just stream it.

Yeah. And it seemed logistically very hard to pull off.

Okay, one final question about Reflection AI's path from here. This is a decision you get to make in the future, but at what point do you try to look at other problems beyond engineering and coding? Do you feel like there's a level of sufficient depth where you should just go attack different domains?

The thing that makes coding as a category special is that it's not synonymous with software engineering. It's just how we think about the market today. The reason code is special is if you believe that the way a language model will interact with almost any piece of software is through function calls, and therefore code, then if you build very capable reasoners—coding reasoners that are purpose-built for organizations—so you've solved the long-context, how-do-I-reason-over-a-bunch-of-disparate-sources-of-information problem, and I can act on pieces of software through code, then you've built a system—the technology that will generalize, at least operationally, across other categories of work.

So the way I think about it is more: First, just build, not trying to get too ahead of yourself, the most depth-wise comprehension system for software engineers.

This will naturally induce more reliable coding agents, right? You can plug that in as an MCP to your favorite IDE or coding agent, or use one of our own. You can plug that into whatever surface area makes sense for the customer and then naturally start seeing where you're getting pulled from there. The reason I think this will work is because this is what we're already seeing, right? You can see this in how you make the system useful for product managers or technical support people, and then, I think, moving on to things like sales or something like this.

There are already places where customers are pulling us in different directions. It's just a matter of whether you engage on that today or not. I think the risk that a startup has is that you see a lot of shiny areas where you can go, and you start going diffuse before you've really nailed a category. I think it's really important to be focused and not diffuse in the short term. If you build the right thing, what we think of as a contextual core for an organization—in this case, an engineering organization—then you can naturally start expanding that into adjacent areas of work in that enterprise.

Okay, last question, Misha. Where would you characterize us as being on the path toward deployment of these capabilities in different fields?

I think we're a lot earlier than most people think. This is going to be one of those areas where the technological building blocks outpace their deployment. Within the next couple of years, the blueprint roughly for how to build ASIs will have been set, more or less. Maybe there are still some efficiency breakthroughs that need to happen, but more or less there'll be a blueprint for how to build a superintelligence in a particular category.

Actually going in and deploying it and building it for specific categories of work, there are going to be a lot of product and research innovation specific to those categories that will probably make this a multidecade thing. I don't think that it's a couple of years from now and GDP starts growing 10% year over year globally. I think we're actually going to get there, but it's going to be a multidecade endeavor.

I tend to see a lot of patterns now in real-world deployment with reinforcement learning research, as it worked before large language models. Before large language models, it used to be that you picked an environment—Go, StarCraft, or something else—and tried to solve it with some combination of imitation learning and reinforcement learning. When you look at all those projects, these were basically things that were called strikes within DeepMind.

Each strike within and outside of DeepMind was a bit of a snowflake. The reinforcement learning methods and environment setup for Go were, at a high level, conceptually similar to those for StarCraft, but at the detailed implementation level, very different from StarCraft and very different from Dota 5. I think that's what we're going into: every big category having a different environment, right? Different kinds of agents with different tools.

That means that you'll have general base models that you can start with, but you'll need to post-train things in specific ways for those categories. We're starting to see that already in the sense that the model that powers OpenAI's Codex is not the o-series of models. It's a model called Codex, which was post-trained for that environment. The Deep Research models—that's a specific environment—are also post-trained for that environment.

I think we'll basically see more and more of that. For any category that has a sufficiently large business around it and requires an intelligence core to power it, there'll be all sorts of interesting design decisions at the research and product level about how to actually gain the most performance out of that particular category. I think we'll see a lot more depth-first players emerge over the coming decade or so.

I'm making a bet on it. Part of your point about choosing the problem for the era is that we don't get to choose a problem at Conviction for 100 years. We do get to choose for this decade or so, right? If you actually believe it's going to be a very long-term endeavor to get to the sort of productivity and abundance you described, but we are going to get there, then the other thing you think about is the path to supporting the cost of bringing anything under distribution during a particular period.

We've already backed companies in some of these areas, but let's say in life sciences or materials science, it is more expensive to collect the types of data you might need, and that might be a longer endeavor or one where you have to figure out how to fund it, right? Or in robotics. I think it's a really interesting timing question for any of these really big categories. But I believe coding is this era.

I think coding is this era as well. This one will take longer than people thought because, again, enterprise organizational problems are much different from the benchmarks that we have today. But I think it will be one of the faster ones, so I don't think that's a decade out. That's within the next, say, dozen or so months. I think the next generational companies in coding are definitely being built today.

Well, congratulations on the release, Misha. Thanks.

Yeah. Thank you, Sarah.