从 Vibe Coding 到 Vibe Researching：OpenAI 的 Mark Chen 与 Jakub Pachocki
GPT-5 的战略目标，是让推理成为默认能力，而不是迫使用户在即时响应的 GPT 模型和更慢的 o 系列之间做选择。 OpenAI 正在研究每个提示词究竟应分配多少思考时间，目标是消除这种产品层面的摩擦，并让更具代理性的行为“默认”可用。对投资者而言，竞争正转向自适应思考预算、延迟、可靠性和可用的自主性。
OpenAI 认为许多熟悉的评测实际上已经接近饱和，正转向以真实发现和经济相关性为基础的基准。 强化学习可以造出狭窄领域的专家，因此从96%提升到98%，相比过去未必还能说明同等程度的泛化能力。AtCoder 和 IMO 仍是可信的标尺，因为许多顶尖研究者都从这些竞赛中成长起来，但下一个里程碑是“让真正具有经济意义的事情发生实质性进展”。
核心研究路线图，是打造能够在机器学习及其他科学领域发现新想法的“自动化研究员”。 Jakub Pachocki 估计，接近掌握高中竞赛水平，大致对应“1至5小时的推理”；接下来的进展需要更长时间跨度的规划、持久记忆、从失败路径中恢复，以及在不断拉长的时间尺度上进行自主运行。
强化学习之所以持续带来进步，是因为语言模型预训练提供了早期 RL 系统所缺少的丰富环境。 自然语言建模赋予模型对人类语言的细腻理解，此后研究者便能探索更多目标和领域。Mark Chen 预计奖励设计会变得更简单，而 Pachocki 认为学习方式应逐渐更接近人类，并警告企业“不要假设现在的状态会永远持续”。
GPT-5-Codex 表明，部署质量取决于是否正确分配智能和时间，而不只是把智能最大化。 上一代模型在最难的任务上投入时间不足，却在简单任务上耗时过多；新版本的目标是在简单工作上降低延迟，在复杂、混乱的编码环境中进行更深度的推理。Jakub Pachocki 说自己过去主要使用 Vim，如今一个30个文件的重构可以“基本完美地在15分钟内完成”，但这些工具仍处在“诡异谷”中，距离一名同事还有差距。
“Vibe researching”是人们期待的未来，但嘉宾认为品味、坚持和诚实的失败分析仍是不可替代的支柱。 研究意味着尝试“最可能失败”的事情，在不刻意“证明它有效”的前提下保持信念，并同时识别软件 Bug 和错误的概念框架。人类研究者仍需选择重要且困难的问题，并学会判断何时坚持、何时转向。
OpenAI 的组织方式结合了受保护的基础研究、审慎的优先级管理，以及仍然稀缺的算力。 实验室不会追逐每一次竞争对手发布，分别为算法进步和产品导向研究设定不同使命；当被问到如果多获得10%的资源会投向哪里时，Jakub 的回答被打断在“算力”。投资过于分散的风险，是最终“每件事都只能排第二”；长期约束也会从算力扩展到能源、机器人和物理世界。
1. GPT-5 让推理成为默认能力，而非独立模式
Mark Chen 将 GPT-5 定义为一次“让推理进入主流”的尝试。过去，用户必须在即时响应的 GPT-2/3/4 系列与思考时间长得多的 o 系列模型之间做选择；GPT-5 则试图为每个提示词推断合适的推理预算。
产品理念与原始能力同样重要：用户不应再为该选哪种模式而感到“困惑”。OpenAI 希望替用户拿走这个决定，同时让推理和代理式行为“默认”可用，尽管模型整体能力也优于 o3。
两位嘉宾描述了不同的个人使用门槛。Chen 说，o3 已经让数学公式和推导对日常工作“相当可信”；Pachocki 则表示，GPT-5 又进一步进入了硬科学领域，物理学家和数学家反复发现，它能够产出此前模型无法完成的非平凡数学成果。
2. 静态基准正让位于发现这一真正评测
Pachocki 对传统评测的批评是：在一个几乎饱和的测试中从96%提升到98%，已经不再具有决定性。在 GPT-2 到 GPT-4 时代，一套预训练方法可以通过广泛泛化能力来判断；如今强化学习能够有意把模型训练成某一领域的专家，因此从专业领域高分推断广泛泛化能力，依据已经变弱。
竞赛成绩仍然重要，因为它们不只是人为设计的测试。Pachocki 称 AtCoder 和 IMO 是“衡量未来研究成功与否的现实标志”，并指出许多杰出研究者都在这些竞赛中成长；OpenAI 在 AtCoder 竞赛中已经达到第2名。
更有分量的证据来自一线科学家。Pachocki 描述了一个“灯泡亮起的时刻”：GPT-5 自动化完成了一项原本可能需要学生数月才能完成的工作，但他也强调，这些数学并非没有难度，却未必非常复杂。
因此，OpenAI 认为自己正处于“评测赤字”之中。替代目标不再是另一套已经饱和的问题集，而是证明模型能够发现新事物，并让具有经济相关性的工作“真正取得进展”。
3. 自动化研究员首先是时间跨度与记忆问题
Pachocki 直截了当地说出了路线图：“我们研究的重大目标，就是打造一个自动化研究员。”让模型自动化 OpenAI 自身的机器学习研究是显而易见的测试，但团队也希望模型推动其他科学进步，而不是只通过自我指涉的循环来衡量自己。
Pachocki 估计，接近掌握高中竞赛水平，大致对应“1至5小时的推理”。下一次跃升需要在更长时间跨度上进行规划并保留记忆，因此可靠自主运行的持续时间会成为尤其重要的评测指标。
主持人的质疑值得保留：每增加一次工具调用或规划跳转，都可能带来质量回归，或增加系统偏离轨道的机会。Pachocki 的回答是，这本质上是同一个问题：保持深度意味着保持一致性，而推理模型已经能够延长其偏离轨道前的持续时间。
Chen 用数学工作说明了所需的稳健性：尝试一种方法，吸收尖锐反馈，诊断错误，选择另一条路径，然后重复这一过程。推理不是简单地产生更多中间步骤，而是这个持续恢复的循环。
4. 长时间跨度的研究正在抹平可验证工作与开放式工作的界线
Pachocki 不认同在任务持续数月或数年后，形式化领域与开放式领域之间仍存在清晰边界。一个持续1小时的约束问题可能拥有有限的想法空间；但要证明一道千禧年大奖难题，则需要选择相关领域、从物理学中寻找灵感，并设计完整的研究计划。
即便是降低建模损失这样看似明确的目标，在研究尺度上也会变得开放式：更难的问题是，团队是否“真的在提出正确的问题”。因此，时间跨度越长，问题定义、品味和探索就越会成为原本可验证工作的一部分。
Anjney Midha 提到，OpenAI 也在思考“开放式”的边界，并引用了 Sam 此前关于让模型进行更有创造力写作的推文。这里的主张不是每个领域如今都有客观奖励，而是研究最终必须同时覆盖强反馈和不那么明确的判断。
5. 语言预训练让 RL 变成不断复利的研究平台
Chen 将 RL 早期的瓶颈归因于环境：研究者拥有强大的学习方法，却难以让智能体扎根现实，甚至曾考虑构建模拟岛屿，让智能体在那里协作和竞争。大规模自然语言建模提供了一个对人类语言拥有“极其细腻理解”的环境。
当强化学习运行在这个预训练基础上，研究者便能在一个稳健、信息丰富的环境中追求多种目标。Chen 称这或许是 OpenAI 数年来“最令人兴奋的时期”，因为许多新方向同时显现出潜力，但也承认团队仍在摸索它们如何结合。
当被问到生物学家、物理学家或企业该如何设计奖励时，Chen 预计这一过程会迅速演化并变得更简单。Pachocki 表示，这个领域仍在摆脱两年前理解微调数据集的方式，转向更接近人类的学习；他的核心警告是：“不要假设现在的状态会永远持续。”
6. Codex 重新分配思考时间，让手动编码变得可有可无
Chen 将 Codex 团队的工作描述为：把原始的推理智能转化为现实世界中有用的编码能力。编码环境本身很混乱，而质量还包括风格、主动性以及模型是否“偷懒”等更柔性的行为，因此团队实际上既在定义行为规范，也在提升任务表现。
核心修正之一，是为每项任务重新分配时间和延迟。上一代模型在最难的问题上耗时不足，却在简单问题上耗时过多；GPT-5-Codex 的目标是在问题简单时降低延迟，在更深度的工作确实能改善结果时提高延迟。
Pachocki 说自己过去主要使用 Vim，并描述了自己的转变：“一个30个文件的重构，你可以基本完美地在15分钟内完成——你多少得用起来。”但他仍将这种体验置于“诡异谷”中：非常有用，却“还不如一个同事”。
竞赛编程的前沿尚未完全关闭：IMO 第6题和最难的编程题仍然“留有一点前进空间”。但 Anjney 与高中生的接触说明，文化转变已经到来——对他们而言，从头编写代码的机制显得奇怪，因为“Vibe Coding 就是默认的编码方式”。他期待的续集是：“希望未来会是 Vibe Researching。”
7. Vibe researching 仍取决于坚持、品味与失败
Pachocki 的定义从不确定性开始：研究是在创造或学习某个尚未被证明有效的东西，这意味着“你总是在尝试最可能失败的事情”。研究者需要清晰的假设、学习的准备，以及足够的诚实，不因想证明偏好的想法有效而扭曲证据。
主持人追问，信念与求真是否天然存在张力。Pachocki 的回答是否定的：一个人可以坚定地相信并坚持下去，即便结果持续为负，只要他准确追踪进展，并从失败中更新认知，而不是把承诺与确认偏误混为一谈。
他选择问题的启发式方法，是追逐那些既重要、又被普遍认为不可行的问题，然后追问究竟是什么阻挡了下一步。对结果的强烈在意，提供了支撑数年坚持所需的动力。
Chen 强调，经验会教会人判断合适的问题时间跨度，也会教会人管理反复失败带来的情绪。论文和同事有助于培养“有趣性”，但“捷径非常少”；有时正确的选择是坚持，有时则是换一个问题。
一个反复出现的突破机制，是发现 Bug。这些 Bug 可能是悄悄让数月实验失效的真实软件缺陷，也可能是导致错误假设的思维框架“Bug”；最初的推理模型和规模更大的预训练模型都遇到过这类问题，最终需要重新思考原有假设与框架。
8. OpenAI 保护前沿研究，而算力仍然稀缺
Chen 表示，招聘主张是基础发现，而不是追问“公司 X”或“公司 Y”刚刚发布了什么：“我们真的不喜欢复制。”招聘更看重候选人是否在物理学、计算机科学、金融或其他领域解决过难题，同时具备扎实的技术基础、雄心，以及坚持处理困难工作的意愿。Chen 还提到 OpenAI 的 resident 项目，通过让其他领域的人亲手实现核心成果，把他们带入 AI。
研究人才不必符合单一的工程师范式。有些研究者通过源源不断的想法产生“极高的 alpha”；另一些人则围绕一个想法严谨地绘制实验空间。组织的目标是把这两种风格结合起来，而不是要求每位贡献者以同一种方式体现影响力。
两位负责人都认为，必须把“基础研究”保护起来，避免其被持续不断的产品需求和竞争性发布牵着走。产品导向的研究者应承担明确责任，而另一组人则需要空间去思考模型在1至2年后可能是什么样；自动化研究员这一目标提供了粗略的方向，但不会规定每个项目的具体内容。
Nano Banana 体现了投资组合中的张力：有吸引力的媒体工具可以释放日常创造力，OpenAI 并不阻止对这些产品感兴趣的研究者。解决方案是保持一致的优先级管理，同时单独保护算法进步，并根据每月需求在核心研究与产品研究之间动态调配算力。
当被问到如果多获得10%的资源会投向哪里时，Pachocki 开始回答“算力”，随后话题转开。他还表示，并不认同 AI 很快会主要受数据约束的说法；任何怀疑资源稀缺的人，都应该“来替我干一周的工作”。如果缺乏优先级管理，危险就是变成“每件事都只能排第二”。
Pachocki 的长期判断并不止于算力，还包括能源和其他物理约束；机器人很可能在“不太远的未来”成为主要关注方向。至于智能本身，他不愿作出太多假设。
实验室的速度部分来自避免陷入学习平台期：Chen 说，新结果出现得足够快，跟上进展本身就是一份全职工作；Pachocki 则描述了团队如何围绕新的约束和可能性不断重组。他们的合作关系，也源于一次早期并不受欢迎的推理工作——Pachocki 深入攻克技术问题，Chen 则把分散的贡献者组织成一支统一的团队。
The big thing that we're targeting is producing an automated researcher: automating the discovery of new ideas. The next set of evals and milestones that we're looking at will involve actual movement on things that are economically relevant. Anjney Midha
I was talking to some high schoolers, and they're saying, “Oh, actually, the default way to code is vibe coding.” I do think the future, hopefully, will be vibe researching.
Thanks for coming, Jakub and Mark. Jakub, you're the chief scientist at OpenAI. Mark, you are the chief research officer at OpenAI, and you both have the privilege and the stress of running probably one of the most high-profile research teams in AI. We're really stoked to talk with you about a whole bunch of things we've been curious about, including GPT-5, which was one of the most exciting updates to come out of OpenAI in recent times.
Stepping back, how do you build a research team that can do not just GPT-5, but Codex and ChatGPT and an API business, and can weave all of the many different bets you guys have across modalities and product form factors into 1 coherent research culture and story?
To kick things off, why don't we start with GPT-5? Tell us a little bit about the GPT-5 launch from your perspective. How did it go?
I think GPT-5 was really our attempt to bring reasoning into the mainstream. Prior to GPT-5, we had 2 different series of models: the GPT-2, GPT-3, and GPT-4 series, which were instant-response models, and then the o-series, which essentially thought for a very long time and then gave you the best answer it could.
Tactically, we don't want our users to be puzzled by which mode they should use. It involves a lot of research to identify what the right amount of thinking for any particular prompt looks like and take that pain away from the user. We think the future is about reasoning more and more, about agents, and we think GPT-5 is a step toward delivering reasoning and more agentic behavior by default. There are also a number of improvements across the board in this model relative to o3 and our previous models, but our primary thesis for this launch was indeed bringing the reasoning mode to more people.
Can you say more about how you guys think about evals? I noticed even in that launch video there were a number of evals where you were inching up from 98% to 99%, and that's kind of how you know you've saturated the eval. What approach do you guys take to measuring progress, and how do you think about it?
One thing is that the evals we've been using for the last few years are pretty close to saturated. For a lot of them, inching from 96% to 98% isn't necessarily the most important thing in the world.
Another thing that's maybe even more important, but a little subtler, is that when we were in this GPT-2, GPT-3, and GPT-4 era, there was kind of 1 recipe: you just pre-trained a model on a lot of data and used these evals as a yardstick for how it generalizes to different tasks. Now we have different ways of training, in particular reinforcement learning on serious reasoning, where we can pick a domain and really train a model to become an expert in that domain and reason very hard about it.
That lets us target particular kinds of tasks, which means we can get extremely good performance on some evals, but it doesn't indicate as great generalization to other things. We think we're in a little bit of a deficit when it comes to great evaluations. For me, the most exciting thread and actual sign of progress this year has been our models' performance in math and programming competitions, although I think they're also becoming saturated in a sense. The next set of evals and milestones that we're looking at will involve actual discovery and actual movement on things that are economically relevant.
Totally. You guys already got number 2 in the AtCoder competition, so there's really only number 1 left.
Yeah. I mean, it is important to note that these evals—AtCoder and IMO—are actually real-world markers for success in future research. A lot of the best researchers in the world have gone through these competitions and gotten very good results, and we're preparing for this frontier where we're trying to get our models to discover new things.
Yeah, very exciting. Which capability from GPT-5 before the release surprised you the most when you were working through the eval bench or using it internally? Were there any moments where you felt like this was starting to get good enough to release because it was useful in your daily usage?
I think 1 big thing for me was just how much it moved the frontier in very hard sciences. We would try the models with some of our friends who are professional physicists or professional mathematicians, and you already saw some instances of this on Twitter. You can take a problem and have it discover maybe not very complicated new mathematics, but some nontrivial new mathematics.
We see physicists and mathematicians repeating this experience over and over: they're trying it and saying, “Wow, this is something that the previous version of the models couldn't do.” It is a bit of a light-bulb moment for them. It's able to automate maybe what could take 1 of their students months of time.
Well, GPT-5 is a definite improvement on o3. For me, o3 was definitely that moment when the reasoning models became actually very useful on a daily basis. Especially for working through a math formula or a derivation, it got to a level where it was fairly trustworthy, and I could use it as a tool for my work.
It is very exciting to get to that moment. But now, as we're seeing these models actually able to automate—yes, as we're saying, solve contest problems over longer time horizons—I expect that was quite small compared to what's coming over the next year.
What is coming in the next 1 to 5 years, at whatever level you're comfortable sharing? What does the research roadmap look like?
The big thing that we're targeting with our research is producing an automated researcher: automating the discovery of new ideas. Of course, a particular thing we think about a lot is automating our own work—automating ML research—but that can get a little self-referential. So we're also thinking about automating progress in other sciences.
One good way to measure progress there is to look at the time horizon over which these models can actually reason and make progress. As we get to a level of near mastery of high school competitions, let's say, I would say we get to maybe on the order of 1 to 5 hours of reasoning. We're focused on extending that horizon, both in terms of the model's capability to plan over very long horizons and its ability to retain memory.
And back to the evals question, that's why I think evals of the form “How long does this model autonomously operate?” are of particular interest to us.
Actually, maybe on that topic, there's been this huge move toward agency and model development. But I think, at least in its current state, users have observed a trade-off: too many tools or planning hops can result in quality regressions, versus something that has a little less agency, where the quality is at least observed today to be a bit higher.
How do you think about the trade-off between stability and depth? The more steps that the model is undertaking, maybe the less likely the 10th step is to be accurate, versus asking it to do 1 thing, which it can do very, very well, and having it keep doing that 1 thing better and better. For more complex things, there's sort of that trade-off, but of course, to get to full autonomy, you're taking multiple steps and using multiple tools.
I think, actually, the ability to maintain depth is a lot about being consistent over long horizons. So I think they are very related problems. In fact, with the reasoning models, we've seen the models greatly extend the length over which they're able to reason and work reliably without going off track. This is going to remain a big area of focus for us.
Yeah. I think reasoning is core to this ability to operate over a long horizon. Imagine yourself solving a math problem: you try an approach, it doesn't work, and you have to think about what the next approach you're going to take is and what the mistakes in the first approach were. Then you try another thing, and the world gives you some hard feedback. You keep trying different approaches, and the ability to do that over a long period of time is reasoning, and it gives agents that robustness.
We've talked a lot about math and science. I'm curious to get your take: do you think some of the progress that we've made can extend similarly to domains that are less verifiable?
They're less explicitly right or wrong.
Oh yeah, this is a question I really like. I think if you truly want to extend to research and find ideas that meaningfully advance technology on the scale of months and years, these questions stop being so different.
It is one thing to solve a very well-posed constraint problem on the scale of an hour, where there is a finite number of ideas you need to look through. That might feel extremely different from solving something very open-ended. But even if you want to solve a very well-defined problem on a much longer scale, like proving a Millennium Prize problem, that suddenly requires you to think about what fields of mathematics or other sciences might possibly be relevant.
Are there inspirations from physics that I must take? What is the entire program that I want to develop around this? Now these become very open-ended questions, and it is actually hard for our own research. If all we cared about was reducing the modeling loss on a given dataset, measuring progress on that—asking whether we are actually asking the right questions in research—becomes a fairly open-ended affair.
Yeah. I think it also makes sense to think about what the limits of “open-ended” mean. A while back, Sam tweeted about some of the improvements we were making in having our models write more creatively. We do consider the extremes here as well.
Right. Right. Let’s talk about RL, because it seems like since o1 came out, RL has been the gift that keeps giving. Every couple of months, OpenAI puts out a release and everyone goes, “That’s great, but this RL thing is going to plateau. We’re going to saturate the evals. The models won’t generalize, or there’s going to be mode collapse because of too much synthetic data,” or whatever.
Everybody has a laundry list of reasons to believe that the gains in performance from RL are going to tap out, and somehow they just don’t. You keep putting out continuous improvements. Why is RL working so well, and what, if anything, has surprised you about how well it works?
RL is a very versatile method, and there are a lot of ideas you can explore once you have an RL system working. A long time ago at OpenAI, before language models, we were thinking, “RL is this extremely powerful thing, of course, on top of deep learning, which is this incredible general learning method.”
But the thing we struggled with for a very long time was: What is the environment? How do we actually anchor these models to the real world? Or should we simulate some island where they all learn to collaborate and compete?
Then, of course, came the language-modeling breakthrough. We saw that if we scale deep learning on modeling natural language, we can create models with this incredibly nuanced understanding of human language. Since then, we have been seeking how to combine these paradigms and how to get RL to work on natural language.
Once you do, you have the ability to execute on these different ideas and objectives in this extremely robust, rich environment given by pretraining. I think it has been perhaps the most exciting period in our research over the last few years, where we have found so many new directions and promising ideas that all seem to be working out. We are trying to understand how to compare them.
One of the hardest things about RL for people who are not practitioners of RL is the idea of crafting the right reward model. Especially if you are a business or an enterprise that wants to harness all this amazing progress you are putting out but does not even know where to start, what do the next few years look like for a company like that?
What is the right mindset for somebody who is trying to make sense of RL and craft the right reward model? Is there anything you have learned about best practices, or an approach to thinking about using this latest family of reasoning techniques? What is the right way I should think about approaching reward modeling as a biologist or a physicist?
I expect this will evolve quite rapidly. I expect it will become simpler.
Maybe two years ago, we would have been talking about what the right way was to craft a fine-tuning dataset. I do not think we are at the end of that evolution yet, and I think we will be inching toward more humanlike learning, which RL still is not quite.
Maybe the most important part of the mindset is not to assume that what is now will be forever.
I want to bring the conversation back to coding. We would be remiss not to say congratulations on GPT-5-Codex, which just dropped today. Can you say a little more about what is different about it, how it is trained differently, and maybe why you are excited about it?
One of the big focuses of the Codex team is to take the raw intelligence that we have from our reasoning models and make it very useful for real-world coding. A lot of the work they have done is consistent with this. They are working on having the model handle more difficult environments, and we know that real-world coding is very messy, so they are trying to handle all the intricacies here.
A lot of coding has to do with style and softer things, like how proactive the model is, how lazy it is, and being able to define, in some sense, a spec for how a coding model should behave. They do a lot of strong work there. As you see, they are also working on much better presets.
Coders have some notion of how long they are willing to wait for a particular solution. We have done a lot of work to dial in on the right trade-off: for easy problems, much lower latency; for harder problems, the right thing is actually higher latency to get you the very best solution. Being able to find that preset sweet spot for easier problems versus harder problems is important.
What we found is that the previous generation of the Codex models was spending too little time solving the hardest problems and too much time solving the easy problems. I think—
That is probably what you might get out of o3 out of the box.
Maybe just on the topic of coding, since you guys were both competitive coders in prior lives. I know you have been at OpenAI for almost a decade now, but I was struck by the story of Lee Sedol, the Go player who famously quit Go after he lost to AlphaGo multiple times.
I think in a recent interview you were both saying that the coding models are now better than your capabilities, and that gets you excited. Say more about that. How much would you say you code now? If you are hands-on keyboard, you can talk about OpenAI generally, but how much code is written by AI now?
In terms of coding models being better, I think it is extremely exciting to see this progress. Programming competitions provide a nice, encapsulated test of the ability to come up with new ideas in a boxed environment and timeframe.
If you look at things like IMO Problem 6, or maybe some of the very hardest programming competition problems, I think there is still a little bit of headway to go for the models, but I would not expect that to last very long.
Being humble.
Historically, I have actually been extremely reluctant to use any sort of—
Tools. I just used Vim, pretty much.
Old school.
Yeah. Eventually, especially with the latest coding tools like GPT-5, I really felt, “Okay, this is no longer the way.” You can do a 30-file refactor pretty much perfectly in 15 minutes. You kind of have to use it.
I have been learning this new way of coding, which definitely feels a little bit different. I think it is a little bit of an uncanny valley right now, where you kind of have to use it because it is exciting so many things, but it is still not quite as good as a coworker. Our priority is getting out of that uncanny valley.
Yeah. But yeah, it is definitely an interesting time.
Yeah, definitely.
To speak to the Lee Sedol moment, I think AlphaGo was a very formative milestone in AI development for both of us. At least for me, it was the reason I started working on this in the first place.
Maybe partly because of our backgrounds in competitive programming, I had an affinity for building models that could do very, very well in these forms of contests. Going from solving eighth-grade math problems to, a year later, hitting our level of performance in these coding contests.
It’s crazy to see that progression, and you kind of imagine—or like to think—that you feel some of the feelings that we all felt too, right? It’s like, wow, this is really crazy. What are the possibilities? This is something that took decades to do, and it took a lot of hard work to get to the forefront of it. So you really do feel the implication of that: with these models, what can’t they do?
I do feel like it’s already kind of transformed the default for coding. This past weekend, I was talking to some high schoolers, and they were saying, “Actually, the default way to code is vibe coding.” I think they would consider that maybe, sometimes, for completeness, you would go and actually do all the mechanics of coding it from scratch yourself, but that’s just a strange concept to them. Why would you do that? Just vibe code by default.
And so, yeah, I do think the future will hopefully be vibe researching.
Yeah. I have a question about that, which is: What makes a great researcher? When you say “vibe researching,” there’s a big part of vibe coding that’s just having good taste and wanting to build something useful and interesting for the world. And I think what’s so awesome about tools like Codex is, if you’ve got a good intuition for what people want, it helps you articulate that and then basically actualize a prototype very fast.
With research, what’s the analog? What makes a great researcher?
Persistence is a very key trait, right? I think what is different about research, when you’re actually trying to—I think a special thing about research, right, is that you’re trying to create something or learn something that is just not known. It’s not known to work; you don’t know whether it will work. So you’re always trying something that will most likely fail.
I think getting to a place where you’re in a mindset of being ready to fail and being ready to learn from those failures is important. Of course, with that comes creating clear hypotheses and being extremely honest with yourself about how you’re doing on them, right?
I think a trap many people fall into is going out of their way to prove that it works, which is quite different from believing in your idea and its significance. That is extremely important, and you want to persist in that, but you have to be honest with yourself about when it’s working and when it’s not, so that you can learn and adjust.
Yeah, I think there are just very few shortcuts for experience. Through experience, you kind of learn what’s the right horizon to be thinking of a problem, right? You can’t pick something that’s too hard, or it’s not satisfying to do something that’s too easy.
A lot of research is managing your own emotions over a long period of time, too. There are going to be a lot of things you try, and they’re not going to work. Sometimes you need to know when to persevere through that, and sometimes when to switch to a different problem.
I think interestingness is something you find through reading good papers, talking to your colleagues, and distilling their experience into your own process.
When I was in grad school, I was a failed machine learning researcher. I was in grad school for bioinformatics. But a big part of my research advisor’s thrust was about picking the right problems to work on, such that you could then sustain and persist through the hard times.
You said something interesting, which was that there’s a difference between having conviction in an idea and then being maximally truth-seeking about when it’s not working. Though both those things are sometimes in tension, because you kind of go native on a topic or a problem that you have deep conviction in.
Have you found any heuristics that are useful at the taste step, at the problem-picking step, that help you arrive at the right set of problems where that conviction and truth-seeking are not as much in zero-sum tension as they are with other kinds of problems?
Yeah, to be clear, I don't think conviction and truth-seeking are really in a zero-sum tension. I think you can be convinced, or you can have a lot of belief in an idea, and you can be very persistent in it while it's not working. I think it's just important that you're honest with yourself about how much progress you're making and are in a mindset where you're able to learn from the failures along the way. I think it's important to look for problems that you really care about and really believe are important, right? And so I think one thing I've observed in many researchers that inspired me has been really going after the hard problems, looking at the questions that are widely known but not really considered tractable, and just asking: why are they not tractable? What about this approach? Why does this approach fail? You're always thinking about what is really the barrier for the next step. If you're going after problems that you truly believe are important, then that makes it so much easier to find the motivation to persist with them over years.
And in the development of GPT-5, during the retraining phase, for example, were there any moments where there was a hard problem, the original attempts that were being made to crack that problem weren't working, and yet you found somebody persisted through that? And what was it about any of those stories that comes to mind that worked well, that you wish other people and other researchers did more of?
I think on the path there, along the sequence of models—both the pre-trained models and the reasoning models—one very common theme is bugs.
There are silly bugs in software that can stay in your software for months and invalidate all your experiments, in a way that you don’t know. Identifying them can be a very meaningful breakthrough for your research program.
But there are also bugs in the sense that you have a particular way of thinking about something, and that way is a little bit skewed, which causes you to make the wrong assumptions. Identifying those wrong assumptions and rethinking your frames from scratch—I think both for getting the first reasoning models working and getting the larger pre-trained models working, we’ve had multiple issues like that that we’ve had to work through.
As leaders of the research org, how do you think about what it takes to keep the best talent on your team? And, on the flip side, how do you create a very resilient org that doesn’t crumble if a key person leaves?
The biggest things that OpenAI has going for it in terms of keeping the best people motivated and excited are that we are in the business of doing fundamental research, right? We aren’t the type of company that looks around and says, “What model did company X build, or what model did company Y build?”
We have a fairly clear and crisp definition of what it is we’re out to build. We like innovating at the frontier. We really don’t like copying, and I think people are inspired by that mission, right? You are really in the business of discovering new things about the deep learning stack, and I think we’re building something very exciting together.
Beyond that, a lot of it is creating a very good culture. We want a good pipeline for training up people to become very good researchers. I think historically we’ve hired the best talent and the most innovative talent, so we have a very deep bench as well.
I think most of our leaders are very inspired by the mission, and that’s what’s kept all of them there. When I look at my direct reports, they haven’t been affected by the talent wars.
I was chatting with a researcher recently, and he was talking about wanting to find the cave dwellers. These are often the people who are not posting on social media about their work. For whatever reason, they may not even be publishing; they’re sort of in the background doing the work.
I don’t know if you would agree with this concept, but how do you guys hire researchers? Are there any non-obvious ways that you look for talent, or attributes that you look for that are non-obvious?
I think one thing that we look for is having solved hard problems in any field. A lot of our most successful researchers have started their journey with deep learning at OpenAI and have worked in other fields, like physics or—
Computer science or finance in the past. Strong technical fundamentals, coupled with the intent to work on very ambitious problems and actually stick with them. We don’t purely look for, “Who did the most visible work or is the most visible on social media?”
Yeah. As you were talking, I was thinking back to when I was a founder and running my own company, and we would recruit for great engineering talent. Many of the attributes you described were ones that were on my mind then. And Elon recently tweeted that he thinks this whole researcher-versus-engineer distinction is silly.
Is that just semantic nitpicking, or do you think these two things are more similar than they actually look?
Yeah, I do think researchers don’t just fit one shape. We have certain researchers at OpenAI who are very productive and are just so good at idea generation. They don’t necessarily need to show great impact through implementing all of their ideas, right? I think there’s so much alpha they generate in just coming up with, “Let’s try this, or let’s try this, or maybe we’re thinking about that.”
There are other researchers who are very, very efficient at taking one idea and rigorously exploring the space of experiments around that idea. So I think researchers come in very different forms. Maybe that first type wouldn’t necessarily map into the same bucket as a great engineer, but we do try to have a fairly diverse set of research tastes and styles.
Yeah. Say a little bit about what it takes to create a frontier-winning culture that can attract all kinds of shapes of researchers and then actually grow them, help them thrive, and make them win together at scale. What do you think are the most critical ingredients of a winning culture?
I think the most important thing is to make sure you protect fundamental research, right? You can get into this world with so many different companies these days where you’re just thinking, “How do I compete on a chat product or some other kind of product surface?” You need to make sure that you leave space and recognize the research for what it is, and also give researchers the space to do that. You can’t have them being pulled in all of these different product directions.
So I think that’s one thing that we pay attention to within our culture, especially now that there’s so much spotlight on OpenAI, so much spotlight on AI in general, and on the competition between different labs. It would be easy to fall into a mindset of, “We’re racing to beat this latest release,” or something.
There’s definitely a risk that people start looking over their shoulder and thinking about what these other things are. I see it as a large part of our job to make sure that people have the comfort and space to think about what things are actually going to look like in a year or two, what the really big research questions are that we want to answer, and how we get to models that vastly outperform what we see currently, rather than just iteratively improving in the current paradigm.
Just to pull on that thread more around protecting fundamental research: you guys are obviously one of the best research organizations in the world, but you’re also one of the best product companies in the world. How do you balance that, especially since you’ve brought on some of the best product executives in the world? How do you balance that focus between the two while protecting fundamental research and continuing to move forward the great products that you have?
Yeah, I think it’s about delineating a set of researchers who really care about product and who really want to be accountable to the success of the product. They should, of course, coordinate very closely with the research work at large. But I think people understanding their mandates and what they are rewarded for—that’s a very important thing.
One thing that I think is also helpful is that our product team and broader company leadership are bought into this vision of where we are going with research. Nobody is assuming that the product we have now is the product we’ll have forever, and that we’ll just wait for new versions from research. We’re able to think jointly about what the future looks like.
One of the things that you guys have done is let such a diversity of different ideas and bets flourish inside OpenAI that you then have to figure out some way, as research leaders, to make it all make coherent sense as one part of a road map. You’ve got people over here investigating the future of diffusion models and visual media, and over here you’ve got folks investigating the future of reasoning when it comes to code.
How do you paint a coherent picture of all that? How does that all come together when there might be, at least naively, some tension between giving researchers the independence to do fundamental research and then somehow making that all fit into one coherent research program?
Our settled goal for our research program has been getting to an automated researcher for a couple of years now. We’ve been building most of our projects with this goal in mind. This still leaves a lot of room for bottom-up idea generation and fundamental research in various domains, but we are always thinking about how these ideas come together eventually.
We believe, for example, that reasoning models go much further, and we have a lot of explorations on things that are not directly reasoning models. But we think a lot about how they eventually combine. What does this innovation look like once you have something that is out there thinking for months about a very hard problem?
I think this clarity around our long-term objectives is important. But it doesn’t mean that we are prescriptive about all the little pieces. We definitely view this as a question of exploration and learning about these technologies.
Yeah. I think you want to be opinionated and prescriptive at a very coarse level, but a lot of ideas can bubble up at a finer level. Have there been any moments where those things have been in tension recently?
One provocative example could be the new image model Nano Banana from Google. It’s an extraordinary example of how lots of everyday people can unlock a lot of creativity when these models are good at understanding editing prompts. I could see how that would create some tension for a research program that may not be prioritizing that directly.
If somebody talented on your team came and said, “Guys, this thing is so clearly valuable in the world out there. We should be spending more effort and energy on this,” how do you reason about that question?
I think that’s definitely a question that we’ve been thinking about for quite a while at OpenAI. If you look at GPT-3, once we saw where language models were going, we had a lot of discussions about how there were clearly going to be so many magical things you could do with AI.
You would be able to get to extremely smart models that are out there pushing the frontiers of science, but you would also have incredible media generation and incredibly transformative entertainment applications. How we prioritize among all these directions has definitely been something we’ve been thinking about for quite a while.
Yeah, absolutely. The real answer is that we don’t discourage someone from being really excited by that. If we’re consistent in our prioritization and our product strategy, it will naturally fall into place.
We do encourage a lot of people to be excited about building AI products—whatever kind of products they’re excited by. But I think it’s important for us to also have a separate group of people whose goal we protect: creating algorithmic advances.
How does that translate—and, just to build on Anjney’s question, into a concrete framework around resourcing? Do you think about, “Okay, X% of compute resources will go to longer-term, very important, but maybe a bit more pie-in-the-sky exploration,” versus current product and inference work? There’s also this thing in the middle where it’s achievable in the short to medium term.
Yeah. I think that’s a big part of both of our jobs: this portfolio-management question of how much compute you give to which project. Historically, we’ve put a little bit more toward the core algorithmic advances versus the product research, but it’s something that you have to feel out over time, right? It’s dynamic.
Month to month, there could be different needs, so it’s important to stay fairly flexible on that.
And if you had 10% more resources, would you put it toward compute, or is it data curation or people? Where would you put that on the margin?
Good question. Honestly, I think compute—
Today, that’s a reasonable answer.
Yeah. I mean, honestly, I do think, to your question of prioritization, that in a vacuum, any of these things are things you would love to go and excel and win at. I think the danger is that you end up in second place at everything and not clearly leading at anything.
So I think prioritization is important, right? You need to make sure—
There are some things you’re clear-eyed on: this is the thing that we need to win.
Yeah.
Yeah. But I think it makes sense to talk about it for just a little bit more. Compute sets so much of compute is destiny, in a way, right, at a research organization like OpenAI. A couple of years ago, I think it became very fashionable to say, “Okay, we’re not going to be compute-constrained anytime soon because there are a bunch of gains that people are discovering, we’re going to get more efficient, and all the algorithms are going to get better. Eventually, we’ll just be in a data-constrained regime.” It seems like a couple of years have come and gone, and we’re still in this very compute-constrained environment.
Does that change anytime soon, do you think?
I think we’ve seen for long enough how much we can do with compute. I haven’t really bought that much into the “we’ll be data-constrained” claim, and I don’t expect that to change. Anyone who says that should just step into my job for a week. There’s no one who’s like, “I have all the compute that I need.”
You know, historically, the job of advancing fundamental research has largely been a mandate that universities have had, partly for the compute reasons you just described. That hasn’t been the case for frontier AI. You guys have done such an incredible job channeling the arc of frontier AI progress to help the sciences. When those worlds collide—the fundamental world of university research today and the world of frontier AI—what comes out?
I personally started as a resident at OpenAI. It was a program we had for people in different fields to come in, learn quickly about AI, and become productive as researchers. I think there are a lot of powerful elements in that program. The idea is, could we accelerate something that looks like a PhD in as little time as possible?
A lot of that looks like implementing many core results. Through doing that, you’re going to make mistakes. You’re going to think, “Wow, if I set this wrong, that’s going to blow up my network in this way.” You just need a lot of that hands-on experience.
Over time, curriculums have probably been developed at all the large labs in optimization, architecture, and reinforcement learning. There’s probably no better way than to try to implement a lot of those things, read about them, and think critically about them.
Yeah. I think maybe one other nice thing that you get to experience in academia is persistence. You have a few years, and you’re trying to solve a problem. It’s a hard problem, and you’ve never dealt with such a hard problem before.
The pace of progress is very fast. Maybe ideas tend to work out a little bit more often than they did in the past, because deep learning just wants to learn. Getting your hands on a more challenging problem for a little bit, perhaps being part of a team attacking an ambitious challenge, and getting that feeling of what it feels like to be stuck and what it feels like to finally be making progress is also very useful to learn.
How does external perception of a particular product launch impact how you prioritize something? Is it to the extent where perception and usage—in the case where they’re married, obviously there’s probably a clear directive there—but in a case where maybe they’re divorced a bit, does that impact how you think about the roadmap or where you emphasize resources?
We generally have some pretty strong convictions about the future, so we don’t tie them that closely to the short-term reception of our products. Of course, we learn based on what’s going on. We read other papers, and we look at what other labs are working on, but generally we act from a place of fairly strong belief in what we’re building.
That’s for our long-term research program. Of course, when it comes to product, the cycle of iteration is much, much faster.
With every launch, we’re trying to aim for something that’s wildly successful on the product side. From a fundamental research perspective, we’re trying to create models with all the core capabilities needed to build a very rich set of experiences and products.
There are going to be people who have a vision of one particular thing they could build. We’ll launch it, and everything we launch, we really hope goes wildly successfully. We get that feedback, and if it’s not successful, we’ll shape our product strategy a little bit. But we’re definitely also in the business of launching very useful, wildly successful products.
It feels like, because of the completely unbridled pace of progress that we’ve just spent a lot of time talking about, a lot is going to change over the next few years. It gets really hard to predict, I imagine, 10 years out, let alone 10 months out. Through all that change that the frontier of AI is going to bring, what are some priors that you think should stay constant?
One clearly is that we don’t have enough compute. Is there anything else that you think doesn’t change—something that would be a strong, reasonably held prior as a constant?
More broadly than compute, there are physical constraints, including energy. At some point, not too far from now, robotics will become a major focus. I think thinking about the physical constraints is going to remain important. But on the intelligence front, I wouldn’t make too many assumptions.
Very few startups can get to the scale that you have, both from an employee perspective and a revenue perspective, and maintain the breakneck speed that you probably had seven or eight years ago when you both joined. What’s the secret sauce to doing that? How do you continue to maintain this pressure to ship as quickly as possible, even though you’re on top now?
I think one of the clearest markers that we have a really good research culture, at least in my mind, is that I’ve worked at different companies before, and there’s a real thing called a learning plateau. You go to a company, you learn a lot for the first 1 or 2 years, and then you find yourself thinking, “I know how to be fairly efficient in this framework,” and your learning kind of stops.
I’ve really never felt that at OpenAI. There are all these really cool results bubbling up, and you’re learning so much week over week. It’s a full-time job to stay on top of all of it, and that’s been very fulfilling.
I think that’s a very accurate description. We just want to generate a lot of really high-quality research, and it’s almost a good thing if you’re generating enough that you’re barely able to keep on top of it.
Yeah, exactly.
I think the development of technology is a driving force here. Maybe we would become comfortable after a few years of working in a given paradigm, but we’re always on the cusp of that new thing, trying to reconfigure our thinking around the new constraints and new possibilities that we’re going to face.
I think that creates this feeling of constant change and the mindset of always learning the new thing.
One thing that came up in our research about things at OpenAI that have not changed through a lot of the change is the trust that the two of you have in each other. There was an article or profile of you guys recently in MIT Technology Review, and one of the highlighted themes was that your chemistry, your trust with each other, and your rapport are something that a lot of the people at OpenAI have come to treat as a constant.
What’s the backstory? How did you guys build trust there? How did that happen?
It’s like asking you—have you ever seen When Harry Met Sally?
I feel like you’re on the couch, and now you’ve got to—
Yeah, exactly.
I do think we started working together a little bit more closely when we had the first seeds of working on reasoning. At the time, that wasn’t a very popular research direction to work on. Both of us saw glimmers of hope there, and we were pushing in this direction, figuring out how to make our work better.
Over time, we grew a very small effort into an increasingly larger effort. That’s where I really got to work with Jakub in depth. He’s just a phenomenal researcher. On any of these lists, he should be number 1. His ability to take any very difficult technical challenge and personally think about it for 2 weeks and just crush it is remarkable.
It's incredible that he has the wide range that he does in terms of understanding, as well as the depth where he can just personally solve a lot of these technical challenges.
Now you get to say some nice stuff about him.
You don't have to say anything nice about me.
Thanks, Mark. I think the first big thing that we did together was start seeing that this algorithm was going to work. I was thinking, how do we direct people at this? We were talking with Mark, like, “Oh, we should establish a team that's actually going to make this work.” And then Mark went and actually did this—he got a group of people working on very different things, got them all together, and created a team with incredible chemistry out of this whole disparate group. That was such an impressive thing to me.
I'm really grateful and inspired to get to work with Mark and experience that. I think this incredible capacity to both understand and engage with and think about the technical matter of the research itself is coupled with this great ability to lead and inspire teams and create an organizational structure that, in this whole mess of chaotic directions, is actually coherent and able to gel together. That's very, very inspiring.
It's awesome.
Well, on that note—
Great note. Yeah, look, some of the greatest discoveries in science, especially in physics, have often come from a pair of collaborators, often across universities and across fields. It seems like you guys have now added to that tradition. We're just super grateful that you guys made the time to chat today. Thanks for coming by.
Thank you.