[BidClub_]
No Priors · · 36 min

Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown

Sarah GuoNoam Brown

YouTube ↗
TL;DR
  • A benchmark score without a test-time-compute budget no longer measures model quality cleanly. Brown says 5.5 looked only modestly better than 5.4 on standard grids, while users quickly found a larger improvement. He attributes the understatement to unmeasured test-time compute and describes 5.5 as more efficient with its thinking; separately, he says controlling thinking time reveals a substantial o3-over-o1 jump. The replacement is a cost, token, or time curve: “There should be an x-axis.” In practice, he says users should iterate quickly when needed and allow longer thinking when the problem warrants it.
  • The industry may be releasing models before anyone discovers their capability ceiling. Modern systems can improve for weeks and remain on an upward slope beyond 100 million tokens, while new models arrive every two or three months. Brown proposes extrapolating expensive performance from smaller runs—for example, predicting a $10,000 inference result using experiments capped at $10 or $100.
  • Safety frameworks inherit the same measurement failure, with much higher stakes. Policies designed around GPT-3 largely treat capability as intrinsic, but today it is “a function of how much money you put into it”: a model given $10,000 may do far more than at $10, and $10 million may unlock more again. Existing policies do not clearly answer which budget should govern evaluations of cyber, bioweapon, or other dangerous capabilities.
  • Brown’s poker-solver test suggests model progress is much larger than benchmark deltas imply. With 5.2 he built a river solver roughly five times faster than working alone, although the models could be unreliable; 5.5 can nearly build a full solver with gentle steering. He would not be surprised if, within six months or a year, one model could reproduce “basically my entire PhD thesis.”
  • Existing models may contain valuable scientific capability that remains uneconomic to excavate only briefly. Brown says a scaffolded 5.5 could likely have found the Erdős unit-distance disproof before OpenAI’s internal model, at a ballpark cost of $1,000–$100,000 of inference. If each release cuts such costs by 10x or 100x, and sometimes more, the investment question becomes when to deploy compute versus wait for the next cost curve.
  • Recursive self-improvement looks gradual because research taste and elapsed time remain binding constraints. Models can optimize existing algorithms by 10–100x yet still fail to invent a better one, and their strongest results require long inference runs: “Time itself becomes a bottleneck.” The larger upside may come from persistent multi-agent knowledge accumulation, not an overnight intelligence explosion.
  • Routing businesses must prove they beat simply letting the strongest model think longer at the same cost. Consensus across models can raise scores, but Brown asks whether it still wins after equalizing test-time compute—and whether benchmark gains survive real-world use. That makes budget-normalized evaluation central to assessing routing, orchestration, and model-choice layers.
Digest · the substance, structured for research

1. Model quality is now a curve, not a grid cell

  • Brown traces the 5.5 launch skepticism to the familiar benchmark grid: it showed only a few percentage points over 5.4, while users quickly found a much larger experiential improvement. He attributes the understated grid results to failing to control test-time compute, describes 5.5 as more efficient with its thinking, and separately says that controlling thinking time reveals a substantial o3-over-o1 jump.

  • The old instruction—run each model until performance plateaus—worked in “GPT-3 land back in 2022.” Modern models, when reasonably scaffolded, can think productively for weeks, pushing the plateau beyond any practical release evaluation. Every comparison therefore needs a fixed budget or performance plotted against tokens, cost, or time. For actual use, Brown favors flexible thinking time: quick iteration when responsiveness matters and longer runs when the user wants deeper work.

  • Cyber evaluations Brown referenced show models continuing to improve at 100 million tokens and beyond. His proposed research problem is to estimate the far end cheaply: can runs capped at $10 or $100 predict performance at a $10,000 inference budget?

  • Benchmark maxing becomes easy without that control: run a model five times, select the best answer—or let another model judge—and the score rises. Brown’s objection is not that scaffolding is invalid, but that the comparison is misleading unless compute is equalized; private held-out sets can help guard against direct benchmark optimization.

2. Poker exposes both the capability jump and the remaining unreliability

  • Brown uses poker bots as a personal evaluation because little usable open-source code exists, despite abundant published papers. Building one requires sustained reasoning, iteration, and handling “a lot of small gotchas” he already understands well enough to diagnose.

  • With 5.2, Brown built a river solver—the final stage of poker—about five times faster than alone. It behaved like a graduate student needing gentle steering, especially around blockers, yet its code optimization was striking: he says it made the work 10x faster than he could, and later characterizes the gains over his algorithms as 10–100x.

  • The weakness was that 5.2 felt like it was “gaslighting me.” In a separate test with another model, a unit test said folding $100 already in the pot lost $92, then defended itself: “It’s close to 100. It’s fine.” With 5.5, Brown says the task is nearly zero-shot; he can now work toward a full-scale solver with gentle steering, although he has not yet simply given it a month-long /goal and received a completed solver.

3. Safety policy has no agreed price tag for capability

  • Responsible scaling policies and preparedness frameworks were largely designed when giving GPT-3 $10 million produced little beyond a $10 or $1 run. Today, Brown argues, $10, $10,000, and $10 million can represent materially different capability levels—leaving an unanswered question: “At what budget should you evaluate these models?”

  • Sarah Guo’s pushback—worth keeping—is that this is the mirror image of useful capability scaling: if large budgets keep improving desirable work, they may also improve tasks society does not want performed. Brown declines to settle whether releases should be blocked, but calls it an “inconvenient truth” that many policies do not account for the budget issue.

  • Longer horizons compound the mismatch. Models can now execute experiments for weeks or months, yet releases arrive every two or three months; fully testing a six-month task requires six months. Brown’s conclusion is stark: “Nobody actually knows what the ceiling of capabilities is,” while labs face competitive pressure not to delay launches.

  • Brown says the river solver is already within reach if he gives a model /goal, but a full poker solver is not yet at the point where a bare /goal and a month of runtime would reliably produce it. He expects that threshold soon.

4. Latent scientific capability creates a wait-versus-compute trade

  • Brown’s central example is OpenAI’s internal model disproving the Erdős unit-distance conjecture at what he calls a “dirt cheap” budget. The result mattered because mathematicians had spent substantial time on the problem and found the model’s method interesting and useful, not merely correct. Brown says the team had trained a new model, was curious what it could do, and had not invested heavily in this particular search.

  • Afterward, someone elicited the same disproof from 5.5 by asking for many approaches, identifying a promising path, and repeatedly extending it. A general-purpose scaffold could automate that search; Brown ballparks the expense at $1,000–$100,000, meaning the capability was publicly accessible but insufficiently explored.

  • The economic complication is that a new release might cut the same discovery’s cost by 10x or 100x, perhaps more in some cases. Brown says OpenAI does not encourage its mathematicians and physicists to spend all their time harvesting current models’ limits; the priority is building safer, more capable models that let scientists worldwide solve those problems more cheaply.

5. Test-time compute slows recursive improvement before it removes researchers

  • Brown rejects the claim that arbitrary inference spend already yields general superintelligence. Factual recall resembles asking someone for Lincoln’s birth date without Wikipedia: a week of thought cannot recover unknown information. Sudoku is the opposite extreme—randomly test enough assignments and, given sufficient time, any puzzle eventually yields.

  • Research sits between those poles, and today’s models lack “very good research taste.” Brown watched them radically optimize his existing poker algorithms, then fail—even with more time and published work—to invent a superior algorithm. Better scaffolding might help, and each release improves, but “please come up with a better algorithm” is not yet enough.

  • His recursive-self-improvement model is bottleneck migration: if one research step accelerates 100x, progress becomes limited by steps that did not. As those constraints shrink, he expects a gradual takeoff that transforms researchers’ work rather than immediately replacing the entire research cycle.

  • That dependence on lengthy inference argues against an overnight intelligence explosion. “Time itself becomes a bottleneck,” Brown says, because powerful outputs still require models to run. Frontier competition remains intense and AI already amplifies lab researchers, though Brown takes comfort that researchers across the labs understand both the upside and the severe risks.

6. Durable orchestration must outperform longer reasoning at equal cost

  • Brown sees more headroom in multi-agent systems through civilization’s example: human achievement compounded because billions of people preserved, shared, and extended knowledge. Today’s models are instead “born into a world,” live for a short context window, then “just disappear”; Moltbook and OpenClaw were overhyped when they appeared, but indicated where large-scale coordination could go.

  • For everyday users, his advice is to revisit assumptions formed around 2022 or 2023 models. He now uses AI for tax questions and condo paperwork and says he can arguably trust its output more than an expert human’s—while his poker experience still illustrates why verification matters when errors become confidently rationalized.

  • The publishing norm remains a “bad equilibrium”: everybody supplies a grid because everybody expects one. Routing and consensus layers face the same test—compare them against a single model thinking longer at equal cost, then probe fragility and real-world relevance. Otherwise, an apparent orchestration advantage may be benchmark-specific and may not transfer to real-world use.

Noam Brown

With GPT-3, you couldn't scale test-time compute. If you gave it a budget of $10 million and said, “Okay, let's see what GPT-3 can do,” it really couldn't do that much. The current frameworks and responsible scaling policies don't really account for the amount of test-time compute. They just say, “Okay, what's the capability of the model?”

The problem is that we're in a world now where the capability of the model is a function of how much money you put into it. Basically, if you give it a budget of $10,000, it can do a lot more than what it can do with a budget of $10. Give it a budget of $10 million, and it can do even more. At what budget should you evaluate these models? The policies that exist today don't really address that question.

Sarah Guo

Hi listeners, I'm Sarah Guo and welcome back to No Priors. Today I'm here with Noam Brown, one of our godfathers of AI reasoning. We talk about the broken state of evaluations, very large-scale test-time compute, how he thinks about recursive self-improvement, and what's next on the horizon for competition at the frontier. Welcome, Noam. I'm so excited to have you back.

Noam Brown

That's great to be back. Yeah.

Sarah Guo

You are our first guest. I'm very proud of my taste in friends and researchers for the pod. Given how important inference-time scaling has become to the industry, you should be proud too, having actually pioneered it.

Noam Brown

I played a part, yeah, among many others.

Sarah Guo

You just wrote this essay that really resonated about large-scale test-time compute and why the industry is not evaluating these models as robustly as it should be. What was the motivation for it?

Noam Brown

The motivation was that we released 5.5, and the initial reaction was skepticism that it was a substantially better model. To be fair, that only lasted for a few hours before people had some time to play around with it and try it out themselves, and they saw that it was actually substantially better.

But I think a lot of the skepticism came from the benchmark grid that was published. Basically, whenever a new model is released, there's this benchmark grid where they show all these different benchmarks on the x-axis and the performance of different models on the y-axis, and you can just compare different models. It's a single number for a model on a single benchmark.

If you look on paper at the difference between 5.5 and 5.4, or other models, it was an improvement, but it wasn't a huge improvement. It was only a few percentage points on some benchmarks. People looked at that and were skeptical that it was actually a better model.

Once they played around with it, the story changed. I think the reason why it doesn't show up as so much better on the benchmarks is because the benchmarks are being presented—the benchmark results are being presented—in the wrong way. They're not controlling for the amount of test-time compute that is being used on that benchmark question.

It turned out that 5.5 is just much more efficient with its thinking. If you run it at maximum settings, o1 is thinking for a lot longer. It takes longer to get back a response than 5.5. Once you control for the amount of thinking time, you can actually see that o3 is a substantial jump over o1. That is, I think, people's day-to-day experience with it.

When I mention this to people, the typical question I get is, “Okay, well, why not just have 5.5 think for as long as 5.4?” The question is, how long should they think for? Typically, the response I get is, “Well, until the performance plateaus.” There's some point where the performance on the benchmark is going to plateau, and you evaluate to that point.

The thing is, the point at which it plateaus is actually really far out these days. It was true in GPT-3 land back in 2022 that the models couldn't really think productively for that long, so you could just run them until they plateaued. It's not that far away.

But what we're seeing today with the modern models is that 5.5 and other models, if you scaffold them reasonably well, can think for weeks before performance plateaus on some of these benchmarks. The point at which they plateau is simply too far out to reasonably test. We all need to actually enforce either a patience limit or a budget limit from a token perspective now, and that wasn't true a few years ago.

Sarah Guo

Exactly. I think the proper way to evaluate the models now is that you either have some kind of budget for the benchmark, whether it's tokens, cost, time, or whatever, or you plot the performance as a function of the amount of test-time compute that's going into the model. Then it becomes much clearer how to compare the performance between these different models.

Given the model evaluation cycle and the fact that performance does not asymptote for many tasks over quite a long period of time, what do you do about that issue? Some of the evaluations that you would want to run are beyond the scope of a reasonable budget or timeline, given the current model release cycle.

Noam Brown

For things like cyber, we've seen—and actually, the AIs in their evaluations have shown—that the models continue to improve at 100 million tokens. If you run them for 100 million tokens, they're still improving beyond that point.

That can take a very long time to run. But you also see that the performance isn't just a discontinuous jump. You can actually see the slope of improvement over those 100 million tokens. So you could probably do some kind of evaluation up to a certain budget and then just say, “Okay, well, this is what we project the performance to look like.”

I think there hasn't been a lot of research on this yet. I actually think this would be a great paper to publish if there are any academics out there looking for something to research. Can you predict what the performance looks like at an inference budget of, let's say, $10,000, using only inference budgets up to $10 or $100?

Sarah Guo

So maybe an orthogonal question for you: Do you think users are systematically not thinking long enough with their models about problems?

Noam Brown

What do you mean by “not thinking long enough?”

Sarah Guo

If you can build an agent or control the amount of test-time compute being used, there's what's done by the model itself and there's what the user can do. Do you think the industry is using test-time compute optimally, way undershooting it, or is it a problem in the models where they just need to be able to do that thinking faster?

Noam Brown

I think it depends on the problem. This idea that you just let the models think for a week or whatever and then they respond—it sounds nice, and yes, the benchmarks look great, but it's not very practical when working with them, because you ask the model a question and then sit there for a week waiting for it to come back to you.

What people have found most effective is to iterate quickly with the models, and so the thinking time needs to be flexible. When it makes sense to respond quickly to the user, it should respond quickly. When it makes sense to think for a long time and the user wants it to think for a long time, then it makes sense to think for a long time. I think people have been striking the right balance given what they have to deal with right now.

Sarah Guo

How would you characterize the landscape of benchmarks today? There's a lot of talk about benchmark maxing and the ability to game different benchmarks. Do you have favorites that you think are more indicative of capability than others?

Noam Brown

The benchmark-maxing thing is also a motivation for writing the essay. I think it's really easy to show that you can do much better than previous benchmarks or previous models on benchmarks by, for example, scaffolding a bunch of models together.

If you say, “Okay, instead of just running this model once, we're going to run it 5 times and take the best of the 5 responses,” or ask a judge which one it thinks is best, then you can get much higher scores than that model. It's really easy to make something that looks a lot better on paper but is actually not better once you control for the amount of test-time compute.

That is one thing that I'm worried about when it comes to benchmark maxing. It's a little misleading; that's the only concern I have.

As far as the benchmarks themselves, I think there's always a risk of just optimizing for the benchmark. I've certainly encouraged my team, and I think at OpenAI we're pretty good about not trying to optimize for specific benchmarks. But once you put out a benchmark, it's always at risk of just being optimized for.

One way to address that is to keep a held-out private set that isn't publicly available.

Sarah Guo

The most popular fallback advice for figuring out whether a model is significantly better is to just play with it for a while. Do you have anything more sophisticated than that that you suggest people do? Do you create your own set of new evaluations each time, besides a private holdout at OpenAI?

Noam Brown

I think everybody has their own set of questions that they like to ask the model whenever it comes out.

For me lately, I've been using them to make poker bots and see how good they can make a poker bot. I think it's a nice evaluation because there's very little open-source code for making poker bots. There are a lot of published papers on it, but you really have to reason through everything, and it requires a lot of reasoning and iteration, with a lot of small gotchas that I've already worked through myself. So I can see where the models fail along the way.

Sarah Guo

They've gotten really good at it now. Can you describe, perhaps, with your poker bot creation, how reasoning might have progressed in model releases for you guys over a few releases?

Noam Brown

Yeah, when the early models were really bad at it, they basically couldn't do anything. And then 5.2, I was able to work with it to make a river solver.

Sarah Guo

So that's the final stage of poker.

Noam Brown

And that itself was, I thought, really impressive. I mean, I had to work with it a little bit, but I was actually really impressed because I was able to make the river solver probably about 5 times faster than I would have alone. There were a couple of things that I got tripped up on. Blockers were always a big, big issue.

But overall, with a bit of gentle steering, it kind of felt like a grad student. They would run into issues, but at least I would know what those issues were and know how to fix them. I could just make suggestions, and it would go off and do it. Pretty quickly, it would come back with something really good.

Sarah Guo

Mhm.

Noam Brown

And then especially the optimization, I thought, was very impressive. It was able to make it 10 times faster than what I was able to do because it was just able to optimize the code so well.

The downside with 5.2 was that I felt like it was gaslighting me a lot, and I always had to be very careful checking it and making sure: Is it actually doing what it said it did? Are there any glaring issues that it's not recognizing, or is it just pretending they aren't issues?

I remember there was one point where, for one of the models I was playing around with—not 5.2—I told it, as a unit test, “Okay, well, let's say I have $100 in the pot and I fold. How much am I losing?” The model said $92.

I was like, “That's crazy. I have $100 in the pot and I just folded. How do I not lose $100?” And it said, “Oh, you know, it's $92. It's close to $100. It's fine. It's no big deal.” And I was like, “Clearly, this is a problem, right?”

So the models did have this problem where they would gaslight you a lot. But once we got to 5.5, I actually thought it was way better. It was able to basically do it zero-shot.

In fact, I've been working on just doing a full-scale poker solver, and it's basically able to do the whole thing with some gentle steering from me. I wouldn't be surprised if, 6 months or a year from now, the model is able to do an entire poker solver zero-shot—basically my entire PhD thesis in one go.

Sarah Guo

Let's talk about the larger implications of needing to evaluate these models relative to, let's say, the speed of their reasoning or efficiency versus token volume, or dollar budget, or whatever your scalar is. Can you describe some of the larger implications in your essay, including around safety evaluations?

Noam Brown

Yeah, the safety evaluations thing. It's a bit of an inconvenient-truth thing. I guess, for background, all of the labs have these things called either responsible scaling policies or preparedness frameworks. They go by various names, but the idea is that whenever a model is released, they go through a series of evaluations to measure: Are there dangerous capabilities? Could these models do things that we wouldn't want a bad actor to do?

If the model isn't very capable, then it's no big deal. But if it is very capable—if it could be used, for example, to make bioweapons—then you want to put in mitigations against that.

But the question is, okay, well, how do you evaluate whether the model is capable of that? They have various protocols for how they do these evaluations. A lot of these frameworks were developed around the era of ChatGPT, before test-time compute scaling was really as much of a thing, and it made sense. With GPT-3, you couldn't scale test-time compute. If you gave it a budget of $10 million and said, “Okay, well, let's see what GPT-3 can do,” it really couldn't do that much more than what you could do with $10 or $1.

The preparedness frameworks and responsible scaling policies don't really account for the amount of test-time compute. They just say, “Okay, well, what's the capability of the model?” The problem is we're in a world now where the capability of the model is a function of how much money you put into it.

Basically, if you give it a budget of $10,000, it can do a lot more than what it can do with a budget of $10. If you give it a budget of $10 million, it can do even more. And so, at what budget should you evaluate these models? The policies that exist today don't really address that question.

Sarah Guo

Mhm.

Noam Brown

Some do better than others, but for the most part, this is not really a factor that's being heavily considered. Now, whether it should be released anyway, I don't want to wade into this question. I think there's arguments on both sides, but I think the important thing to recognize is that this is a question that is not being addressed. We're just pretending that this issue doesn't exist. And I think it's important to just, one way or the other, account for it.

Sarah Guo

Yeah, it's the mirror image of the capability question: If the models can continue to do more and more without asymptoting on some tasks at very large budgets, then they should also be able to do so for tasks we don't want them to do as a society, right?

And so, testing for that—and what budget is allocated—also seems out of sync with the model release cycle itself, right? There's been this acceleration where you get a new model every few days or weeks at this point, versus every 6 months. And you have a line in the essay where you say the only way to truly evaluate an agent on some very long-running task might be to run it for a year.

That's going to be true of both useful and negative tasks, right? And so, how do you think about that versus the model release cycle?

Noam Brown

Yeah, this is also an interesting dynamic where, basically, as the models have become stronger, they're better able to operate over longer horizons. So again, with GPT-3, if you wanted to run it for a week, there wasn't much you could do to scaffold it into something useful that could actually run for a week.

But we're seeing now with the most recent models that you can actually scaffold, for example, 5.5 into doing a series of experiments that can run for weeks or months.

Sarah Guo

Have you given your poker solver task an infinite budget yet?

Noam Brown

I haven't really scaffolded something together where I just tell it, “Okay, just run this for weeks.” I think I could probably give it `/goal` and tell it to go nuts.

But I think at this point it could 100% do the river solver. If I just give it `/goal`, I don't think it's at the level yet where it could do the full poker solver if I gave it just `/goal` and told it, “Yeah, go run for a month.”

But we're going to pretty soon be at that point where I probably could just tell it, “Yeah, go work on this for a month and then come back to me with a full, complete poker solver that's state-of-the-art.”

The problem is, if you want to evaluate the capabilities of a model—what it can do after running for a month—the only way to be fully sure is to actually run it for a month. And if you want to know what it can do after 6 months, the only way to know fully is to run it for 6 months.

I'll get to things we could do to address that a little bit later. But it's important to recognize the model release cycle: We're releasing new models every 2 or 3 months at this point. And so, a model comes out, it takes 2 or 3 months to push it to its limits, and then you have another model come out.

Nobody actually knows what the ceiling of capabilities is for these models because nobody's actually run them for long enough to really tell. When `/goal` came out, for example, people started running things that took over a week for it to finish. People actually didn't realize that this was a big deal until a week after it was released.

Sarah Guo

Mhm.

Noam Brown

I think that's going to be more and more true. The implications of that are pretty interesting because what do the labs do to fully evaluate their models before they're released? It's actually very difficult because the only way to really do the evaluations is to delay the model release cycle. And there's a lot of competitive pressure right now not to do that.

Sarah Guo

Do you think there's exciting latent capability in the models that are already released that people have not fully explored, given the timeline?

Noam Brown

I think absolutely. I think actually a really great example is the Erdős unit distance problem. For the viewers that don't know, we used an internal model at OpenAI a few weeks ago to disprove the Erdős unit distance conjecture.

Now, I'm not a mathematician, but this seems like it was a pretty big deal in the math community. It was the first problem that a lot of mathematicians had really spent a lot of time on, and the model was able to do something that they weren't able to do in a way that was actually interesting and useful for mathematicians.

Honestly, it did it at a budget that was dirt cheap. We didn't put a lot of effort into this. We just trained a new model, and we were curious what it could do. We ran it through some problems, and on this one, at a pretty low budget, it was like, “Oh, yeah, I think I have a disproof.” Then we were able to verify that this proof was correct.

After we announced the results, a bunch of people found that you could get the answer out of 5.5 as well. It's not as simple as just asking 5.5, “Hey, here's the unit distance conjecture. What's the disproof?” You had to scaffold it a bit; you had to steer it a bit.

Somebody found that you could ask GPT-5.5, “List a bunch of ways that you could tackle this problem,” and then it lists one of the paths that are actually promising for getting to the disproof. Then you tell it, “Okay, explore this some more.” If you do this enough times, it actually ends up arriving at the disproof.

Now what this means is that you could, in principle, ask 5.5, as a general-purpose scaffold, to list a bunch of different strategies. Then, for each strategy, you could tell it to investigate that strategy, and it would probably be able to arrive at the disproof with a general-purpose scaffold.

That scaffold would be very expensive. I would ballpark it at $1,000 to $100,000. But it would have been possible for somebody to disprove the Erdős unit distance conjecture before we did using a general-purpose model. Nobody had sufficiently explored what would happen if they put $100,000 worth of compute into 5.5. What could it do? The answer is, yeah, you probably could get stuff like that out of it.

Sarah Guo

So people should be experimenting more with the current generation in terms of—

Noam Brown

Well, this is, I think, an interesting question of whether it’s worth it to experiment with, because, again, the model release cycle is every couple of months. We put out a new model that’s even more powerful, and so the cost of disproving the Erdős unit distance conjecture drops by 10 or 100× with every model release cycle, probably in some cases more so.

Sarah Guo

You’ve seen the meme that’s like, “Why bother doing any engineering work when I should just wait for the next—

Noam Brown

model, go on vacation, come back 2 months later, and it’s 1,000 times cheaper.” So—

Sarah Guo

Do you agree with that?

Is that what you’re doing right now at OpenAI, just waiting for the next model release?

Noam Brown

I will say we’re in a period where progress is very fast, and the models are becoming more capable. At OpenAI, one of the things that we’re actively not doing—and look, we have a lot of mathematicians and physicists, and people are very excited about what these models can do right now, especially the internal models—is encouraging people to spend all their time going through mathematical open problems and physics problems and pushing the models to their limits to see what they can prove or disprove.

We really think the focus should be on how we make even more capable models, and how we get them out safely to the world as quickly as possible so that all the scientists in the world can use these models to solve the problems themselves. In some sense, we are thinking about this. It’s really tempting to put all of our efforts into scaling up these models and seeing what they can do at their limits right now, but the focus should be on how we use these models to make even more powerful, more capable models that can do everything much more cost-effectively.

Sarah Guo

What is changing about the direction or allocation of resources for research in your mind, given your beliefs about the impact of very large-scale test-time compute? How does this interact with the idea of recursive self-improvement, for example, where it’s a dominant idea for how any lab gets to the best-capability model?

Noam Brown

One thing I should clarify is that I don’t think we’re at the point where you just give a model an arbitrary, extremely high inference budget and it’s superintelligent across the board.

Sarah Guo

So, the goal—

Noam Brown

Yeah.

Sarah Guo

—to make GPT-7 or whatever and just go nuts. What’s between us and there, then?

Noam Brown

I think, having played around with the model, first of all, there are some benchmarks where the models will just not improve if they have more inference budget. A lot of factual retrieval questions fall into this category. If you ask a person, “When was Abraham Lincoln born?” and they don’t know the date, they could sit there and think about it for a week, but if they don’t have access to Wikipedia or something, they’re not going to be able to do better answering that question after thinking about it for a week compared to 5 seconds. It’s the same with the model.

Interestingly enough, if you give the model these kinds of factual retrieval questions and give it a little bit of time to think, it does better. But if you give it a week, it’s not suddenly going to be better at remembering dates.

On the other extreme, I think there are benchmarks where they will obviously keep improving without limit with more test-time compute. The example I like to point to is Sudoku. There’s a really simple strategy to solving Sudoku: try a bunch of different random numbers and see if they fit the criteria, if they match all the constraints. If they don’t, just try a different random combination of numbers. Clearly, with enough time, you’ll be able to solve any puzzle with this strategy. You can trivially see that any model could keep doing better and better if it were just given more test-time compute.

All the benchmarks exist somewhere between these 2 extremes. The models are not at the level where, if you just give them enough test-time compute, they’ll be able to do all of our jobs. There are some benchmarks where they will not improve, and there are some things where they will not improve.

One thing I see for research in particular is that they don’t have very good research taste right now, so I think they’re actually a very good complement to researchers. I’ve found that I’m much more effective by using these models, but they’re not able to fully replace the whole research cycle.

Sarah Guo

Now, does that change with time?

Noam Brown

Probably. I think the models are getting better across the board. Some things are getting better faster than others, but they’re not at the point where they’re fully replacing researchers with just enough test-time compute.

Sarah Guo

Can you give an example or 2 of asking the model to do a research task where you think, “This is a terrible idea”?

Noam Brown

Going back to my poker solver example, I was really impressed with the model’s ability to optimize the algorithms that I had developed in my PhD. It was shocking to see how inefficient I was in retrospect, and they were able to make it 10–100× faster.

Then I said, “Okay, can you come up with an algorithm that’s better than the algorithms that I came up with or that anybody else came up with? Go ahead and look at all the published work, synthesize that, and then try to come up with something novel.” It’s not able to do it. I can give it a lot of time, and it’s still not able to do it.

It’s possible that if I scaffolded something and constrained it a bit more, maybe it could eventually come up with something better. But it’s not as simple as saying, “Okay, please come up with a better algorithm.”

Sarah Guo

How do you think that gets improved?

Noam Brown

What I’ve seen is that with every model release cycle, it does get better at this sort of thing. It’s still bad, in my opinion, but it’s not as bad as it used to be. I wouldn’t be surprised if, at some point—same thing with coding, same thing with math—there’s just an inflection point where suddenly it’s good enough to be useful. I wouldn’t be surprised if we encounter that point for research taste as well.

Sarah Guo

Given that, what is your framing of RSI today? How should we think about it?

Noam Brown

The models are definitely accelerating what researchers can do inside the labs, but I think they’re accelerating some things and not other things. Currently, we’re at the point where, if something goes 100× faster, you get bottlenecked by the things that don’t go 100× faster. Over time, the things that we’re getting bottlenecked on are going to shrink, and there will be, I think, a kind of gradual takeoff in that respect.

But right now, it’s more about transforming what researchers do rather than fully replacing the researchers.

Sarah Guo

So that actually implies that you don’t think we’re close to a very fast takeoff right now.

Noam Brown

I think fast takeoff is relative. Things are moving very fast, but I think there’s this hypothesis that you could have basically an overnight intelligence explosion, where the models discover some kind of breakthrough to make themselves smarter, and then that leads to more breakthroughs that make themselves even smarter immediately. You have, basically, in an instant, the models becoming superhuman across the board.

I don’t think we’re headed to that world, largely because the models rely so much on large-scale test-time compute in order to achieve their greatest intelligence. If it requires so much test-time compute to unlock the full capabilities of the model, then that means you’re bottlenecked by time. Things can only go so fast because the models need to run for long enough to actually do something really powerful.

Time itself becomes a bottleneck to what we can do. I think that’s the case right now for a lot of the labs. Ultimately, I think the biggest bottleneck for all of us is time, and that’s why all the researchers are working so intensely right now. So many hours per week are being put into this because we all see what the overhang is. We see what the capabilities are, and we’re just bottlenecked by how quickly we can do things.

Sarah Guo

What do you think is on the frontier that is less explored now? We’ve talked about multi-agent before.

Noam Brown

I think multi-agent is quite explored.

Sarah Guo

At sufficient scale,

Noam Brown

I think there’s a lot more that could be done. But it’s also one of the things that’s hard to do—a lot of research is hard to do at small scale. Multi-agent, in particular, really requires, in order to fully unlock the capabilities, frontier models. We’ve seen some pretty interesting multi-agent scaffolds. They’re able to do a lot, but I think it’s really just scratching the surface of what it will be able to do.

One way that I think about it is, if you look at human civilization, it’s not that humans have evolved to become smarter over the past 50,000 years. It’s that humans are able to do a lot more today than they were back in caveman times because there have been billions of humans thinking for a long time and building off of each other’s accumulated knowledge.

We have very good retrieval and scaffolding versus 50,000 years ago. I wouldn’t even call it a scaffold. It’s a very organic, emerging property of humans being able to accumulate knowledge, share it, and build off of it. We’re not seeing that with AI models today. They’re born into a world, and they exist for a very short context window, and then they just disappear.

There are things that you can do to continue them, but it’s very limited. I do think eventually we will—and we’re starting to see signs that we’re entering a world where they can coordinate on a large scale. I think Moltbook and OpenClaw, when they first came out, were obviously a bit overhyped, but they were an indication of where things could go in the future. Eventually, I do think we get to that kind of world of some sort of coordinated, compounding state.

Sarah Guo

Yeah, the ability of the models to share knowledge on a more global level and be able to build on that knowledge productively.

Given this set of beliefs and your work, how would you characterize competition at the frontier between the three kingdoms if there is no overnight takeoff? It’s just researchers grinding away, trying to make good, high-taste algorithmic and investment decisions about where to go, and then compute allocation, policy decisions, and eval decisions. That feels slightly more grounded than racing toward some immediate hard takeoff that nobody can catch you on.

Noam Brown

I think the competition is very intense right now. I do think the models that exist today are accelerating what researchers at the frontier labs can do. There are obviously, as I said, limits to that right now, but the ability to use the models to improve model research is a real thing, and it is an amplifying force. I think that will continue to be true. I think it’ll become more true over time.

One thing that I am comforted by is that I think all the researchers at the frontier labs—all the frontier labs—recognize what’s at stake and what the risks are with these models. That’s something that I find comforting. I think everybody really understands that this is a pretty serious thing, and it can lead to really great things or it can lead to really bad things. Yes, there’s a competitive dynamic between the labs, but we can also try to figure out how we all get to the positive outcomes rather than the very negative outcomes.

Sarah Guo

I think I’d be remiss not to ask, just because you have been right very early for a long time on the importance of test-time compute and reasoning as a framework: are there ways in which you use the models that you would encourage others to? Is it just `/goal` everything?

Noam Brown

I think for a lot of people—and this is probably not even true for your audience necessarily—but there are a lot of people who experimented with AI back in 2022 and 2023, felt like they couldn’t trust the outputs, and then didn’t use it for really high-stakes decisions. Actually, I think the models have progressed to a point where they are very good for these kinds of things.

I asked them for tax advice. I bought a condo recently, and I was asking it for advice on, “What’s all the paperwork that I have to fill out? How do I—what does it all mean?” It’s actually really good for these kinds of questions.

So I use it day-to-day for a lot of this kind of stuff. I think they’ve been at a point for a while now where I feel like I can just trust the outputs, arguably more than I could trust the output from an expert human.

Sarah Guo

Okay, I have 2 final questions for you. One is: is there something you think that the rest of the research community doesn’t agree with you on, or doesn’t understand the importance of quite yet?

Noam Brown

Oh, these are such good questions. I wish I had time to think about this ahead of time.

Sarah Guo

You can just hang out with me and think about it. Is it weird to be at consensus now? You were a bit salty 3 years ago when you were like, “Why don’t people understand how important this is?”

Noam Brown

I still feel like it’s not consensus, though, because people still don’t publish the benchmarks this way.

Sarah Guo

Oh, that’s true. Yeah.

Noam Brown

That’s actually why—

Sarah Guo

I think that’s inertia.

Noam Brown

That’s kind of—yeah, but that’s kind of why I wrote the essay. I was just like, look, we can talk about this, but part of the motivation is that I would talk to researchers about how it makes sense to show the benchmarks with an x-axis, whether it’s tokens, cost, or time. There should be an x-axis, and everybody would say, “Yeah, that makes sense. We should do that.” But everybody—

Sarah Guo

They’re not acting with the importance of, like, “This is—we have to measure the correct thing.”

Noam Brown

Well, really, their response is, “People expect us to publish the grid.” And then, okay, why do people expect the grid to be published? Because everybody publishes the grid. So you kind of end up in this bad equilibrium where everybody kind of knows that it’s a bad equilibrium, but nobody wants to break out.

I felt like, okay, well, if I just hopefully come out and say, “Look, guys, let’s all recognize that we’re in a bad equilibrium and let’s move to this different equilibrium where we’re plotting things with an x-axis,” hopefully that can mean that, next time there’s a model release, a company can feel comfortable not publishing the grid, at least not at the very front—the top line—and we can have a more productive evaluation of these models.

Sarah Guo

Then, a last question for you. How do you think about companies across all of these specialized domains who feel the value that they have is essentially the routing layer, the choice layer? My goal is composed of a bunch of discrete tasks. Some require more intelligence and some less, and within my job as a vendor is to solve that problem or achieve the optimal outcome, taking into account the budget constraints.

I will manage the parallelization, how much inference you spend on it, and from what model. I think the frontier lab point of view is that that routing happens both behind the API, behind the application, and then some of it in the model itself. Pieces of that are clearly being externalized in all these applications.

Noam Brown

Yeah, I do think this is related to the fact that benchmarks should be evaluated with an x-axis of tokens or cost. I’ve seen some evals recently that show that, with a routing layer, you can achieve much better performance by basically doing consensus among the models.

Sarah Guo

Yeah.

Noam Brown

I definitely believe that if you do consensus among the models, you’re going to achieve better performance than any individual model. But it’s important to ask: are you going to do better than having that model basically think for longer? Once you control for the amount of test-time compute, is it actually still doing better? That’s the question that you want to figure out.

Sarah Guo

Okay, that’s a very principled view, which is, yes, routing is fine, but it’s all subject to the same budget question. Yeah, right. If you put it on the same scale, then you can make an optimal decision, and I think maybe I win.

Noam Brown

Mhm. I don’t even know necessarily that I would believe that the routing does better, but then there’s still a question of: is it going to do significantly better? Is it very fragile? Is it reflective of real-world use cases compared to benchmarks?

One issue you could run into is that you could optimize for certain benchmarks with the routing and then show, “Oh, yeah, we see this big improvement on these benchmarks.” But in real-world use cases, it actually ends up not being a significant improvement.

I would say, at the very least, you want to control for test-time compute, and then you also want to have all the same skepticism about benchmarks that you would normally have.

Sarah Guo

Awesome. Noam, thanks so much, and for being on the mission of breaking us out of this false equilibrium.

Noam Brown

Yeah, it’s great to be back.

Find us on Twitter at no prior pod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way, you get a new episode every week. And sign up for emails or find transcripts for every episode at no-briers.com.