《后训练现状》：从 GPT-4.1 到 5.1：RLVR、Agent 与 Token 效率——Josh McGrath，OpenAI
后训练的经济吸引力在于杠杆：Josh McGrath 将追求 3% 的预训练算力效率提升，与改变模型 40% 的行为进行对比。 代价是运营复杂度：每个 RL 任务都可能带来独立的评测器和基础设施，放大一次运行失败的路径。真正稀缺的能力，是同时熟悉 ML 与分布式系统，而不是只精通其中一门。
从 RLHF 转向 RLVR，关键是奖励数据更干净，而不只是换了一个优化器。 McGrath 认为两者本质上都是策略梯度方法，只是处在信号质量不同的光谱上：人类偏好有用，但不如数学解答是否正确那么可信。他批评已发表研究把重点放在梯度方差、GRPO 等方法上，而真正的创新在于“数据从哪里来”。
从 GPT-5 到 5.1，战略上最重要的进步，是用少得多的 token 获得更好的评测结果。 McGrath 用 token 而不是小时衡量 Agent 的任务跨度，因为一个人需要 4小时完成的任务，Codex 可能 10分钟就能完成；token 消耗下降，就能在实际服务预算内留下更多工具调用和行动空间。主持人谈到质量与 token 曲线时说：“老哥，我就靠那些图活着。”
OpenAI 的购物模型是测试可中断、可操控 Agent 的试验场，这些能力未来可能汇入通用模型。 用户可以看到模型正在考虑哪些商品，并在运行过程中修正需求——“我想要的是 USB-C。”McGrath 已经更偏好高强度思考的 GPT-5 Thinking，而不是最初的 Deep Research 模型，因为公开评测“基本持平，甚至更好”，尽管有些用户确实更喜欢不同模型各自的脾气。
长上下文与压缩是互补押注，而上下文利用不充分更像暂时问题，而非根本缺陷。 McGrath 预计，扩大上下文窗口与发展长期保留有效信息的方法之间会持续“共舞”；Graphwalks 评测之所以重要，是因为它要求在整个上下文中进行多次变换，而不是从单一位置检索信息。尚未解决的问题是，超大窗口最终会成为直接存储，还是承载大量子搜索的基础设施。
预训练和后训练都没有过时，而当前的算力分配也无法让人确定最终均衡会是什么样。 McGrath 提到，早期工厂用一台电机带动整间厂房的设备：电力这项变革性技术直到数十年后布局改变，才真正显出威力。预期进展会时而沉寂、时而陡峭——“一切都完了。我们又杀回来了”——因此，路线图的灵活性和情绪稳定，比自信宣布未来已经定型更有价值。
1. 后训练以运营复杂度换取行为杠杆
McGrath 从预训练数据整理转向后训练，是因为两者的杠杆完全不在一个量级：“我是要追求大约 3% 的算力效率提升，还是要把行为改变 40%？”他并不认为预训练已经过时；只是后训练提供了更令人兴奋的前沿，这一点被“无数个深夜”进一步确认。
RL 运行涉及更多活动部件，因为每个任务都可能需要独立的评测配置和配套基础设施。凌晨 12:30，排查一次可疑运行意味着要迅速读懂多个组件、多个贡献者写的代码；代码来自内部还是外部合作方并不重要，关键在于研究者是否理解底层系统。
主持人说，Codex 改变了他的生产函数：写一份设计文档之类的东西，原本要花 30–40分钟，现在 15分钟就能触发比手工数小时完成的工作更多的后续产出。尚未解决的是人的时间安排——这类 15分钟的 Agent 工作间隔太短，接不上另一项深度任务，让他“还在适应”这种新的工作节奏。
2. 专用 Agent 正在验证最终会汇入通用系统的交互模式
购物模型源于 8月一次围绕 Black Friday 上线的建议——当“现在它归你了”变成现实后，McGrath 开玩笑说自己有些后悔。它最重要的特性是可中断：用户能看到模型正在考察哪些商品，也能在运行中修改提示词，比如补充一句：“我想要的是 USB-C。”
McGrath 认为，购物从根本上并不需要一个独立模型。推出专用版本，只是更方便测试“适用于购物的 Deep Research 风格模型”，让它在整个互联网范围内广泛搜索；他的判断是，专用系统与通用系统最终会“在能力上汇合”。
最初的 Deep Research 模型与高强度思考的 GPT-5 Thinking 之间，已经能看到这种汇合。McGrath 表示，OpenAI 的公开评测结果看起来“基本持平，甚至更好”，他自己使用 Thinking on High，同时也承认用户完全可能理性地偏好旧模型的独特风格。
个性仍然是产品的真实维度，但 McGrath 自己的偏好更偏工具属性：“我个人希望我的模型是一个工具。”主持人把选择概括为 Anton 与 Clippy 之争——安静而可靠的能力，还是热情而主动的协助；McGrath 表示，OpenAI 正通过开关和自定义指令同时提供两者。
3. RLVR 的突破在于可信反馈，而不是优化器的品牌效应
McGrath 的框架消解了一个流行区分：RLHF 与 RLVR 都是策略梯度方法，主要差别在输入数据。把 RLHF 称为“不可验证”略显奇怪，因为训练后的模型确实是在预测人类反馈；更深层的区别在于目标是偏好还是真实性，以及这种信号能够承受多大程度的优化。
他批评当前研究讨论常常把论文写成优化工作——聚焦梯度方差——而真正重要的轴是信号的干净程度。实际问题是：“我有多信任它？”以及在持续优化下会发生什么；人类偏好评分的可靠性，显然不如验证一道数学题的解答。
主持人提到 DeepSeek Math 论文中的 GRPO，认为这种优化方法的影响力似乎超过了最初的预期。McGrath 则把注意力重新引向奖励信号：“当你找到一道数学题的答案时，争议空间就小得多。”方法当然重要，但这种高度可信的新奖励信号，才是被行业低估的变化。
4. Agent 任务跨度的提升，来自模型少用 token，而不只是运行更久
McGrath 不接受用 30小时或1天来定义长跨度自治；他用 token 思考问题。Codex 可能把人类 4小时的工作压缩到 10分钟完成，因此经过时间掩盖了真正可优化的变量。从 GPT-5 到 5.1，综合评测得分提升，而质量与 token 的曲线显示 token 消耗大幅下降。
Token 效率会直接改变 Agent 的可行边界：它决定在真正可服务的合理 token 数量内，能够容纳多少次工具调用和行动。主持人说：“老哥，我就靠那些图活着。”因为这些曲线同时记录了基准能力，以及等待任务完成时的真实使用体验。
主持人的质疑是，显式调用 GPT-5 的路由与隐式控制思考力度可能发生冲突：上游路由器可能做出错误选择，即使 GPT-5 本身有能力判断任务难度。McGrath 预计，今天不断增加的旋钮最终会简化成“一个你总会使用的工具”，由它决定该思考多久。
对于上下文压缩，McGrath 没有透露产品细节，但预测会长期采取双向策略：一边增加上下文长度，一边开发让窗口更长时间保持有效的方法。他提醒不要把接口冻结，因为固定的抽象层可能“困住”某次未来的模型改进，而那次改进可能要求接口本身发生变化。
5. 模型与系统必须共同演进，因为两条扩展路径都尚未定型
McGrath 并不认为完美利用上下文是无法实现的目标。他以自己参与 GPT-4.1 评测的 Graphwalks 为例，称他们把上下文窗口效应放大到了 10×；这类评测要求在整个窗口中进行多次变换，因此分数仍在持续上升，不同于只检索某个孤立事实的测试。他认为上下文利用不充分“肯定是一个暂时问题，我们会随着时间推移不断爬坡”。
主持人描述了一个针对公司客服工单的 RAG 代码库：10万份文档，总计约80亿 tokens。这种规模需要检索系统，而不是单纯依赖一个 1000万 token 的窗口。McGrath 的回应刻意保留了不确定性：搭载 GPT 的 Agent “就是异常有效”，但巨大的上下文也可能服务于一个更大的计划，其中包含大量子搜索。模型扩展与系统工程仍然需要共同设计，而不是二选一。
最难招聘的人才画像，正是能同时驾驭分布式系统、核心工程、统计学和 ML 的研究者。项目瓶颈可能反复在基础设施与学习之间移动，但教育体系通常不会同时针对两者进行优化。McGrath 低置信度的理论猜测是，这两个领域大致同样困难；一些传统意义上的 ML 研究可以更多地当作黑箱处理，而构建训练环境则是复杂的数据工程问题。
主持人还引用了一张 Grok 4 图表：当预训练扩展到某个算力水平后，如今投入后训练的算力已经达到可比规模。McGrath 拒绝宣称预训练或后训练已经胜出。就像早期工厂用电力带动整间厂房的一台电机一样，行业可能正用旧技术时代继承下来的布局承载新能力，尚未找到它们的自然形态。某些想法可能沉寂一阵后突然回归，因此这个周期会反复让人觉得“一切都完了。我们又杀回来了”。
We're here with Josh from OpenAI. Welcome. How do you introduce yourself?
I work on a bunch of the thinking models at OpenAI. Recently, I've been focused on search-related work, but I'm primarily a post-training researcher at OpenAI.
You were on with us for GPT-4.1 when we were talking with Michelle, who's on maternity leave. I didn't know that. Now we're in GPT-5.1. It's been a whole generation.
Yeah, it's been wild. GPT-4.1 was a non-thinking model, and since then we sort of switched into doing—
Was that your last one?
No, we're still releasing non-thinking models. That was the one we did that was specifically for the API, so the focus has shifted a little.
How did you get into post-training?
Before OpenAI, I was doing pre-training data-curation work. Looking at the news and reading papers, I thought, “It seems like there's going to be a lot of interesting work in post-training.” I wanted to make some contributions there.
It wasn't necessarily that pre-training was dead, but it was definitely changing. Do I want to make compute-efficiency gains of 3%, or do I want to change the behavior by 40%? Honestly, it seemed more exciting to go into post-training, and many late nights later, that's definitely true. It's also a different kind of data and engineering discipline. The kind of work you need, especially in reinforcement learning and scaling it, is very strange.
Yeah, definitely. The number of moving parts in an RL run is much higher—an order of magnitude, or—
I don't know if we could say an order of magnitude, but if you think about pre-training, you're moving tokens to many machines, getting a scalar from them, and then backpropagating.
The issue with RL is that you're doing tasks, and each task could have a different grading setup. Each of those grading setups requires more infrastructure. When I'm staying up late trying to figure out what's going on with a run, there can be many more things involved than there generally are in a pre-training run.
Does it matter if you own the code for the task, or if it's an outsourced third-party person's code? My sense of it—and the external sense of it, obviously, since I don't see it up close—is that you work a lot with external partners, as well as on internal stuff. Which is better?
Honestly, I don't think I'll comment too much on how many external partners there are.
There are some, and there's some internal work.
Yeah, there are.
But what is the technical trade-off of, “I don't own this code”?
When it comes to not owning the code, when I'm babysitting a run, it doesn't really matter whether it's internal or external. The question is: Do I understand the underlying system?
You end up having to jump into a lot of code and say, “I actually don't know what this does.” I'll be watching a run and working on my pieces, while other people are working on theirs. Do I understand what their code is doing? At 12:30 in the morning, when something looks wrong and I'm looking at this code, can I get enough context quickly enough to understand it?
Throw a Codex at it.
Wrong.
I use Codex so much. It's really changed how I work. I feel like there's a degree to which I'm trapped by Codex, because if I spend 30 or 40 minutes writing something that looks like a design document, Codex can do more work than I could do in a few hours, in 15 minutes. But then what do I do during those 15 minutes afterward?
It's actually changed the flow of my day, because I now have to manage these 40-minute sessions with 15 minutes in between where I could do something, but it's not nearly as effective as this new flow of the day. I'm still getting used to that.
I think it should also be interesting for codebase understanding when you're encountering unfamiliar code.
You briefly mentioned the shopping model before we started. That's the latest, hottest thing, and we're recording this right after Black Friday and Cyber Monday. Did you find anything interesting from releasing shopping in ChatGPT right into that period?
The first thing is that I don't know why I would say in a meeting in August, “Black Friday is coming up. Maybe we could do a release by then.” In hindsight, I'm thinking, “Why would I say something like that?”
They're like, “Yes, now you own it.”
Yeah, exactly. The most interesting thing to me is the new interruptibility and the qualitative experience of using it. The same thing happens with Codex: You write a prompt, and you can press Escape and say, “I messed something up.”
We did the same thing in the shopping model. It shows you its chain of thought and the products it's looking at, and you can send it a new message saying, “I actually wanted this.”
Like, “I wanted USB-C on this,” or whatever it is.
I think that's a really new and interesting interaction paradigm that we have in a couple of different services. I'm excited to see how people use it and whether they enjoy it.
Why did it have to be its own model and not just a new tool?
Stay tuned. There's no reason we couldn't eventually do it in the same model, but if we want to try out new things, sometimes it makes sense to create a new model.
This time, it made sense to ask, “Can we do a Deep Research-style model for shopping, where it's going to look really hard across the internet for different things?” If you look at the original Deep Research model and GPT-5 Thinking on High today, I think you'll see that eventually the models all converge in their capabilities.
Would you say this is a discussion that's a little spicy, which I've kicked off in the community? Maybe 30% of the community is still using Deep Research, but a lot of people have moved over to just using GPT-5 Thinking as Deep Research.
Is that the spiritual successor, or are they direct replacements? Are there things that we lose from the original Deep Research model if we do that?
If you look at our published evaluations, they look basically on par, if not better. Personally, I use Thinking on High rather than the Deep Research model.
As we've learned over the past few months, though, people sometimes prefer the quirks of one model over another. If people like the Deep Research model, more power to them.
Was there anything special in the GPT-4o post-training? Are people really responding to personality? Is that a differentiator that people care about, and is it part of your job to care about personality?
People definitely care quite a bit about personality. Over the past few months, we've been working a lot on giving users more choice over what personality they want.
Right, through the toggles.
Yeah. Now we have those toggles.
What's your favorite toggle?
Honestly, custom instructions. I personally want my model to be a tool. I don't necessarily want warmth or anything; I just want answers, because I'm mostly using it at work.
I call this the Anton-versus-Clippy divide. Anton is the machine from HBO's Silicon Valley.
Okay.
It's a machine. It only does work. It doesn't try to be helpful or friendly. It tries to be helpful, but it doesn't try to be cheery.
So it sounds like you also come down on the side of using—
Anton. Yeah, I think a lot of developers want Anton.
Right. It just quietly does its work, and when it's done, it shuts up.
Yeah. We're doing a lot of work to provide both Antons and Clippies, and I hope people like them all.
Generally, I was thinking about what we can update people on in post-training. What do we know today, in 2025, that we didn't know in 2024?
At the time, there was still this whole PPO-versus-DPO discussion. That was a whole era.
Yeah.
Since then, we've moved on to RLVR and a lot of agent-specific RL training. Am I missing any large chunks of the post-training debates that are going on?
Yeah.
I mean, not necessarily internal debates, but my read personally, from looking at the different papers that are coming out, is that when you look at an RLVR paper or an RLHF paper, they read more like optimization papers. To me, the interesting thing that's going on is that we have this spectrum of how high-quality a signal is. At the end of the day, RLHF and RLVR are both policy-gradient methods. What's different is just the input data.
It's always interesting to me that we call RLHF nonverifiable because we've trained the model to be good at predicting human feedback. In some sense, that's verification, but obviously—
It's human preference rather than truth.
Yeah. Yeah. But if your value of truth is, “Does the user like this more?” there's something strange that I think we haven't looked at: how clean is this signal? How much do I trust it?
I totally agree that you don't necessarily trust the RLHF signal as much as, “Is this the solution to this problem?” But I think there's a whole spectrum of how high-quality the signal is and what's going to happen when you do a lot of optimization against it. That's very different from worrying about the variance of different gradients, which I think is what you end up seeing in a lot of the papers currently coming out. Rather than being very data-centric, they're pretty optimization-centric, even though I think the innovation really is where the data is coming from.
Yeah. Before I go broad, before I go deep: are there any other discussions that people are having in Europe, or sort of around this time, on post-training debates? What do you talk about with your peers at Anthropic and DeepMind?
At Anthropic and DeepMind, we're all saying, “I'm working on stuff and things.” I think it's more about talking broadly with my friends there. We're just talking about, “Man, the industry is so hard to keep up with.” We're not necessarily talking too much about methods directly, because on one level, it kind of doesn't matter.
Yeah. And I think there's something very different about academic work, where what really matters is how narrativizable it is. That's one of the reasons you see a lot of optimization papers come out: a lot of the data work has a less clear narrative around it.
I think the data and the scaling are actually more important than a specific method.
Yeah, but it doesn't necessarily have the same narrative that you get out of some of the papers that you see here. So there becomes more of a question of, given a specific vertical, how do I understand that? I wish there were actually more papers on it here, but I think it can sometimes be harder to wrap up into a clean story.
That's also something that we're having a lot of conversations about with other folks as well: what's next? Where do you go from here now that we have some kind of road map? I think what's interesting for me is that the innovations exposed by the Chinese models are maybe copies of, or discussions of, what's going on in the labs.
GRPO—you mentioned that a lot of these RL optimizations present themselves as optimizations. GRPO came out in the DeepSeek Math paper, which, when it came out, I read and thought, “Okay, this is kind of cool.” It's a little bit cheaper, but it does seem to have a broader impact on the industry as a whole than was initially appreciated. I don't feel like we've processed that enough.
Yeah, definitely. As you said, it came out in the DeepSeek Math paper, and it's an interesting optimization method. But the more interesting thing is that they have a new reward signal that we can really, really trust. When you find the answer to a math problem, it's a lot less debatable than, “Was this thing that the human preferred actually what we wanted to do?” You want to be right at math.
Yeah. Yeah.
And so I think, in some ways, that's underappreciated in what's getting published.
Yeah. Let's talk about long-horizon.
Yeah.
What do people consider very long horizon? Are we talking about 30 hours—more than a day—of autonomy? Is it just more of the same, or is there anything qualitatively different?
Okay. So first off, what I would say is that I tend to think more in terms of the actual number of tokens than time, because—
Yeah, the human in the loop can take a while.
Yeah, and it gives you a different measure to optimize against. As I was saying earlier, when I use Codex, it does something that would take me much longer. It would take me 4 hours in 10 minutes. What we can actually push on there is token efficiency.
That is a huge, huge research area.
Yeah. And so you can see that from GPT-5 to GPT-5.1, our overall eval went up some, but if you look at a 2D plot of how many tokens it takes for us to get that, it went way down.
Dude, I live by those charts. Not necessarily that one, but that shape of chart. I think that's something we think about a lot, just because it contributes so much to your experience: how long does it take to do this task?
Yeah. And I think the other thing is, as you're pushing token efficiency, it changes how many tool calls I can make and how many different things the agent can do in a reasonable number of tokens that we can actually serve.
Yeah. I think the interesting—or hard-to-understand—thing from the outside is having an explicit router in GPT-5, but then also basically having an implicit router in terms of the thinking-spending thing. That conflates things a little bit, right? At some point, you do kind of need to merge them, or else you're just going to get these weird bumps where sometimes the router at the top decides something and it's wrong, and actually, if you just handed it to GPT-5, it would have figured it out.
Yeah. And I think we'll figure out the correct abstractions over time.
Is the intention still to merge? Because that's what was said in the paper.
Yeah. I think eventually we'll have AGI and you're not going to have to worry too much about how hard to think directly. It'll just be one tool that you always go to, and it knows how long to think for and things like that. I think the abstractions and the way that we drive these things today will change. We've changed so much already—from having a non-thinking model to choosing between 2, and now we can sort of route and decide how hard you want to think. We're adding lots of knobs, and eventually it'll probably simplify.
Yeah. Another super-interesting knob that everyone is doing is context compaction, or memory compaction. What's going on there?
Nothing to share at the moment.
Let me share. Okay, clearly an important feature, clearly inspired by Codex usage as well, obviously. But from the engineer's point of view, it feels like I used to do that as part of my harness, and now the model's doing it for me. I don't know how to think about that in terms of—I guess I'm used to having more control, and now I have less.
Yeah. Is there a specific, like—
There's a specific question I'm just getting feedback on: Is this a trend that we need, where it's basically a permanent fact of life from here on out?
Oh, I see. I don't know. I worked on long context; that was what I was working on last, for GPT-4.1, where I think we 10×ed the context-window effect for GPT-4.1. And so there'll always be some dance of, well, if we want to push as much as we can do, not only should we increase the length of the context window, but we should also have strategies for keeping that context window available for as long as possible.
I'm guessing that both things will happen, just because we want to put as much power into the models as possible.
Yeah.
Yeah. I think we're still in a period where we should all be expecting changes in the interfaces that all the models give to us. That way, we can improve the models, because if we lock the interface, I think what would be sad from my perspective is if we lock the interface and discover something new about models, we might trap that improvement under an interface that needs to change.
Right? Talking about long context as well, there is some discussion about context rot, or the utilization of the context. Even if you gave us a million-token context, we probably wouldn't use all of it. What's the recommendation there? Where are things going? Are we going to have perfect context by next year? Is that an impossible dream? I don't know.
No, it's not an impossible dream. I'll give a shout-out to some of the evals that we did for GPT-4.1 called Graphwalks, where—
I love Graphwalks.
We covered this in the podcast.
Yeah, we did. I think if you look over time, all of those evals are still climbing. One of the interesting things about that is that you have to do complicated transformations across the entire context window.
But the problem is, if you only have to sample from 1 point in the context window, it's sort of easy. Whereas with those graph-walk problems, you're having to do multiple transformations across the entire context window. I think—keep watching those. They've been climbing, and they'll continue to climb. I would say that that's definitely a temporary issue that we're climbing over time.
So, is 10 million tokens realistic? Is 100 million? Where does it end? Is there a natural end, or is there no end and we just go as far as I can see?
Oh gosh, I don't know. What do you think?
I feel like there are use cases that require billions, and there are use cases that require many, many billions, maybe trillions.
Out of curiosity, what would be billions of tokens?
We just had a context-engineering discussion about a RAG codebase over support issues for a company, and it was 100,000 documents totaling about 8 billion tokens. You can't stick that in a context window for now.
That's fair. I would still say I don't know, but I've been really surprised. It reminds me of when I was doing more information retrieval work, and BM25 and these very simple n-gram indexes were just super hard to beat.
I think the agents with GPT feel really similar to me. They're just unreasonably effective. But then I will not use your 10-million-token context window, even if you gave it to me.
Maybe. But what if we're using that context window in service of some larger goal that just has a lot of sub-search calls? That's why I'm saying I just don't know, and I think that's what makes it so exciting.
Yeah. I would say that the other modalities, like video, would eat up a lot. Then, obviously, the hard sciences have proteins and all of that, where a lot of information is just encoded in physics.
I have mixed feelings about it, just because I'm like, "Well, this will never scale—not with full attention—and we probably just need to invest in systems anyway," which means we're good with what we have. I mean, get your graph walks up, but—
I don't know if we need 10–100x. Maybe we need to figure out ways to go 1,000x or 1 millionx.
Yeah.
Right. These are just different slopes. I'm definitely glad that you're happy with the current context windows. I think my dream would be to push it and see what happens anyway. But I think the engineers' incentive is always to say, "Well, the systems matter more than the models," and the researchers' incentive is to say—
"Screw your systems. We'll just scale the models."
Oh, no. So, differently. I think that's one of the most beautiful things about—
Post-training at OpenAI is that everyone—
Co-design.
Yeah, it's all co-design. I spend a lot of time just doing our systems work, and I also do lots of things where I'm making graph walks and doing a lot more on the learning side. I think it's a great culture to have a place where people just move seamlessly between the 2.
What are you guys hiring for? Presumably, you're hiring, but what are you hiring for that's hard to hire? What is the skill set where you're thinking, "We really need this. We can't find it. Please, everyone, go skill up on this"?
This is definitely my personal opinion, but I think we're still having trouble—not at OpenAI, but as a whole—producing lots of people who want to do both systems work and ML work. If you're trying to push the frontier, you don't know which place is currently bottlenecking the frontier, and it changes all the time. Even within 1 project, it might change multiple times where the current bottleneck is.
I think the education system we have right now isn't really optimized for that. I studied math, and then I was very lucky to have some great mentors after school who taught me to be a good software engineer. But it seems like, if we're going to be in this place for a while—and I think we will be—we should probably be producing more students who are great at doing both distributed systems and a lot of core engineering, as well as the statistics and other things required to be a good machine learning researcher.
If we were to throw Codex at it, obviously we can't use Codex for everything.
Let's say: which will progress faster? Which is more solvable by an LLM?
That is a spicy question.
You can't say they're both equally hard. Maybe they are. I don't know. They're differently hard. One is more hill-climbable than the other. Which is it? Because then we can go do it.
Okay. I think one thing that's slightly simpler about some of the ML research—or, you know, ML research is also distributed systems, to be clear—is that some of the things traditionally called ML research are things that you can treat a bit more as a black box. Whereas the environment to train on, building these different systems, is actually just a complicated data-engineering problem.
Theoretically, I would say that they're probably roughly equal. But I think there's some amount of effort that goes into making the environments for—
Yeah, but they require GPUs themselves as well.
Yeah. I guess they both would, but that's my guess. I don't have high confidence in it.
A lot of people are building these AI scientists that automate research. You guys have your own benchmark, PaperBench, and that's the one area that, for example, at Cognition, we've just decided not to do because it's so hard. Any other people on the post-training team you want to shout out who have done interesting work this year? They should get more attention, but they're not getting credit.
For sure, everyone on the shopping team I was just working with: Andrew Hoyle, Manuka Stratta, and John Hallman. They're all great people. Isa Fulford, obviously, is the manager for it.
And she was the original Deep Research person.
There were like 3 of them. Yeah, and so definitely that part of the team. But everyone is so great, and I think it's hard to give out a list. It's a really fun time on post-training right now. It's exciting every day. It feels like we're all enjoying our Diet Cokes together in the office late at night.
I did want to squeeze this in before we end. Nobody serious is saying that pre-training is dead. It's just a meme. There's a lot of work going on in pre-training. In fact, a lot of my researcher friends are saying that too much money is going to post-training. That's also spicy.
I don't know. One of the charts I hold in my memory from this year is the Grok 4 chart. I don't know if you've seen it, but it's basically saying, "We scaled pre-training to here, at about this level of compute, and now we're spending the same level of compute on post-training as well." That's very controversial, I guess, because we're all used to post-training taking orders of magnitude less data and compute, or whatever. Obviously, we're scaling that up now. Do we get to a point where they're equal? I don't know. But that's a topic for conversation: how much do we invest in this versus more of the different pre-training that you've done before?
Yeah. So, first off, neither one of those is dead. I think it's really interesting to be living through something when all of my other historic or technological revolutions are things that I read about in history books, and this one's live as it's happening.
Yeah. We don't know the end yet.
Yeah. There's this almost fog of war where I'm like, "Did people think that, when we got the steam engine, they would have the factories?" I don't know if you know this, but factories used to be very linear because you had to drive 1 motor across an entire room. When electricity was developed, they just tried to do the same thing, and they said, "Ah, this isn't all that useful."
It took a couple of decades before they realized what was most ergonomic. Then manufacturing was transformed by electricity. I think that really gives me no confidence in saying, "Oh, this thing is dead."
Yeah, our timelines are so short. But recently, the way good ideas get experimented with, funded, and propagated is still on a human timeline. That's not on an AI timeline.
Yeah. And so I think things will maybe be dormant, but they'll be spiky.
There will all of a sudden, you know—yeah, yeah—and then we’ll all feel different. It’s like—what’s the meme? It’s so over. We’re so back.
It’s going to be that many times. And I think—
Some emotional stabilizing to it is probably going to be good for everyone’s sanity.
Yeah. More sanity. Well, thank you so much for joining. Thanks for all the great post-training this year.
Yeah, thank you. And continue giving feedback. I love to hear what you think.
Yeah. Awesome.