GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim

I think it’s pretty unique at OpenAI to be able to work on something that’s so generally useful. It’s like everything they tell you not to do at a startup: your user is anyone.

You just take it for granted that you literally have this wizard in your pocket.

We’re trying to make the most capable thing. And we’re also trying to make it useful and accessible to as many people as possible.

If this exponential is true, there’s not really much else I want to spend my life working on. I think we hear this with GPT-5 internally when people are testing it. They’re like, “Oh, I thought I asked a really hard question.” I feel a little bit insulted that it thought for 2 seconds, or when it doesn’t even want to think at all.

Slow news day. Not much going on for you guys. Thank you for coming on. Obviously, Christina, you were just on the livestream. We’re recording the day of. Congratulations.

Thank you.

For those who are unfamiliar, why don’t you introduce what you guys do at OpenAI?

I’m Christina. I lead the core models team in post-training.

I’m Isa. I lead the Deep Research and ChatGPT agent team in post-training.

And Christina, you’ve been here for—or you’ve both been here for—a while now. Why don’t you give us a little bit of your history at the company?

I’ve been at OpenAI for about 4 years now. I originally worked on WebGPT, which was the original first LLM using tool use. It was just one question: the model learned how to use the browser tool, but you only asked one question and got an answer back. Then we had this realization: normally, when you have questions, you have more questions after that. So we started building this chatbot, and that’s what eventually became ChatGPT.

What have the reactions been so far? It’s only been a few hours, but in your livestream, what are your reflections? What can you tell us on the day of?

I’m honestly really excited. Obviously, we have some great eval numbers, and numbers are always really exciting, but the thing I’m really excited about with this model is that it’s way more useful across all the things that people actually use ChatGPT for. The eval numbers look good, but when people use it, I think they’ll notice quite a bit bigger difference in its utility.

Say more about that. What are you noticing? What are you seeing? What are you hoping for?

For me, the 2 top use cases—these are my personal use cases—are coding and writing. I use it for both all the time, and it’s just a huge step change.

Sarah, you’ve been involved in helping lead our investment since 2021. Could you share more, or tee up how you’ve been thinking about this as it relates to coding or more broadly?

Actually, just on the topic of coding, it was a huge deal to have Michael Truell come on there and not only showcase the capabilities, but also say this is the best coding model in the market. I’m curious, to the extent that you can share, what did you do differently to get these results?

Huge shout-out to the team, especially Michelle Pokrass. To get these things right—and eval numbers are one thing, like I said—but to get the actual usability and make it great at coding takes a lot of detail and care. The team put a lot of effort into data sets and thinking about the reward models for this. I think it’s literally just caring so much about getting coding to work well.

Maybe just to double-click on front-end web development: we’ve seen, as investors in the ecosystem, that it’s obviously taken off in the last 6 to 8 months. If you could pinpoint the improvement to that piece specifically, is it more around aesthetics, or is there another capability leap forward in terms of what we can do with front-end web development?

I think there’s going to be a lot more we can do with front end. If you compare it to o3’s front-end coding capability, this is just totally next level. It feels very different. I think it goes back to what I was saying: the team really cared about nailing front end. That means getting the best data, thinking about the aesthetics of the model, and all of these things. It’s all those details coming together and making the model great at front end.

Really exciting to see. I loved the demos in the livestream, too. I wanted to ask about model behaviors, because I know you worked on that as well. How did you think about that for GPT-5? We’ve talked in prior models about sycophancy and characteristics like that. How did you think about it for this model? What did you change or tweak?

The design of this model has been very intentional, especially with the sycophancy issues that we had a few months ago with GPT-4o. We’ve spent a lot of time thinking about what the ideal behavior is. One of the reasons I really like post-training is that it feels more like an art than maybe other areas of research, because you have to make all these trade-offs.

You have to think about all the different rewards you could be optimizing during the run and how they trade off against one another. I want the assistant to be super helpful and engaging, but maybe that’s a bit too engaging, and getting too engaging leads to the overly effusive assistant that we had. It’s really a balancing act of trying to figure out what the characteristics are and what we want this model to actually feel like. We were really excited with GPT-5 because it was a time to reset and rethink. Since it’s so easy to make something engaging in an unhealthy way, how can we make this a very healthy, helpful assistant?

Say more about how you achieved such a reduction in hallucinations, but also in deception. What’s the relationship between those?

For me, I find hallucinations and deception pretty related. The model—and we saw this a lot with the reasoning models—would understand that it didn’t have some ability, but it still really wanted to respond. We had baked into the models that they wanted to be helpful, so they would say whatever they could to be helpful in that moment. That’s what we consider the difference between deception and hallucination.

Sometimes the model will literally just say something quickly. We see a lot of this reduction with thinking: when the models are able to think step by step, they can pause before blurting out an answer, which is what it feels like with a lot of the previous models and hallucinations.

Over the next few weeks, as you’re evaluating usage, what are the biggest questions you’re having, or anticipating being potentially answered?

I’m really curious to see how all of these things reflect in usage. Coding is way, way better, but what does this actually unlock for people? We’re really excited to offer these models at the price points that we have, because I think this unlocks a lot more use cases that weren’t really there before.

Previous competitor models were good at coding, but the price point wasn’t as exciting. With the number of capabilities we have in this model and the price point, I’m excited to see all the new startups and developers doing things on top of it.

We’re excited, too. On the topic of usage, you obviously have a lot of products with a ton of usage already. Since we have one of the Deep Research gurus here, how did Deep Research, ChatGPT, and Operator—your existing products—influence how you approached GPT-5?

One thing that’s interesting is that with reinforcement learning, training a model to be good at a specific capability is very data-efficient. You don’t need that many examples to teach it something new. The way we think about it on my team is that we’re trying to push capabilities and things that are useful to people.

Deep Research was the first model to do very comprehensive browsing. But then, when o3 came out, it was also good at comprehensive browsing. That’s because we’re able to take the data sets that we’ve created for the frontier agent models and contribute them back to the frontier reasoning models. We always want to make sure that the capabilities we’re pushing with agents make it into the flagship models as well.

That’s great. Very self-reinforcing. You mentioned all the startups that you’re excited to see. Flesh out what you think that could look like, or even at a high level, what opportunities you’re more excited about because of this.

People always say “vibe coding.” I think basically non-technical people have such a powerful tool at their hands, and you really just need a good idea. You’re not going to be limited by the fact that you don’t know how to code something. You saw 2 of our demos, which were front-end coding in the beginning, and that literally took minutes.

I think that honestly would have taken me a week to actually build fully interactive. So I think we’re going to have a lot more—I would expect a lot more indie-type businesses built around this because you just need to have the idea, write a simple prompt, and then you get the full-fledged app.

It’s the world of the ideas guy.

Yeah, it’s our time.

I think so.

Yeah, finally.

How about in the broader AGI discourse? What does this mean, accelerate, or not? How do we think about the broader AI discourse in terms of what GPT-5 means here? Does it change the conversation in any sort of way?

I think with GPT-5, it’s kind of a new, obviously state-of-the-art model, like all the things we talked about. But I think it shows that we can continue pushing the frontier here. I feel like there are always people saying, “Oh, we’re hitting a wall. Things aren’t actually improving.”

I think the interesting thing is that we’ve almost saturated a lot of these evals. The real metric for how good our models are getting is going to be usage. What are the new use cases that are being unlocked, and how many more people are using this in their daily lives to help them across multiple tasks? I feel like that’s actually the ultimate usage metric that I’m excited about in terms of whether we’re getting to AGI.

I had a question about that, just because I think Greg made this comment about how he was comparing the last model to this model, and that the benchmark went from 98 to 99. He was like, “Clearly, we’ve saturated the benchmarks,” at least on that front, which I think is instruction-following.

What benchmarks do you pay attention to? How do you guys think about evals? Given that you’re already saturating what’s out there to a large extent, or doing very well along those dimensions, what actually gets you to push the frontier? Usage would be kind of post-model release, but before you get there, what are you guys looking at internally to help guide you? Is it a lot of internal evals that you’ve created? Is it early access to startups, seeing what they think? Maybe it’s a combination of all the above, but how do you weigh all of those things?

On our team, we really work backward from the capabilities we want the models to have. Maybe we want it to be good at creating slide decks or good at editing spreadsheets. If evals for those things don’t exist, we try to make evals that are representative measures of that capability in a way that’s actually going to be useful for users.

A lot of those are internal. We’ll collect them from human experts, try to synthetically create examples, or actually look at usage data. Then we’ll just try and hill-climb on those.

I think we make this joke a lot internally: If you want to nerd-snipe someone into working on something, you just need to make a good eval, and then people are going to be so happy to try to hill-climb that.

Yeah.

I like what you said about starting with the capabilities first. How do you prioritize what you’re actually shooting for? Let’s say there’s this dimension of going deeper into everyday use versus getting much deeper into expert use cases. How do you think about that trade-off? What does that trade-off mean, practically speaking, and what do you guys prioritize when?

I mean, I think it’s pretty unique at OpenAI to be able to work on something that’s so generally useful. It’s like everything they tell you not to do at a startup: Your user is anyone. For deep research, we wanted it to be good across every single domain someone might want to do research in. I think you only have the privilege of doing that if you work at a company that has huge distribution and all different kinds of users.

So, if you choose a capability that’s quite general, like online research, you just have to make sure that you represent a distribution of tasks across loads of different domains if you want to get good at all of them. But sometimes it is hard to decide to focus on one specific thing because there are just so many different verticals that you could choose from. In some cases, maybe coding will be really important, so a specific team will focus on coding.

But I think in general, because the capabilities are so general, usually the next model improvement just kind of improves performance on a pretty broad range. We’ve seen this with the progression of even the models that we’ve had in ChatGPT. As the model gets smarter, it’s better at instruction-following, it’s better at tool use, and more things get unlocked as we continue to make smarter models.

I think a good chunk of our team also focuses on just getting general intelligence up, because the wins that we get from there are pretty great. Whenever we get a new base model, we see, “Oh, wow. Suddenly this clicks. It works.” I think we saw that moment with Operator, because we had been working on computer use, but it was hard to finally get the model to actually support it without the multimodal capabilities. You couldn’t have something like Operator when it launched.

It’s the same thing with everyone talking about agents. We didn’t really have a way of actually training useful agents. There were all these agent demos, but nothing that actually really worked. When we saw the reinforcement learning algorithm working really well on math and physics problems and coding problems, it became pretty clear, just from reading through the chain of thought, that this thing was actually thinking, reasoning, and backtracking.

To build something that’s able to navigate the real world, it also needs to have that ability. We realized that this was the thing that was actually going to let us get to useful agents. I think it’s interesting at OpenAI because you have people pushing foundational algorithms, getting really good at math and getting a gold medal in the IMO. Then, in post-training, we’ll often take those methods and try to figure out how to make things that are most useful and usable to all of our users.

How much of the improvements are coming from the architecture versus the data versus the scale? Where—how do you think about that?

In my opinion, I’m very data-pilled. I think data is very important. I think deep research was so good because Isa put so much thought and careful attention into the data curation, and thinking about all the different use cases she wanted to have represented. I’m on team data.

Yeah, I mean, I think all are very important, but especially now that we have such an efficient way of learning, high-quality data is even more important.

Maybe on the data topic, we’ve been talking a lot about RL environments. It’s a popular space for startups who all want to work with you guys. I was curious to get your thoughts on this, since you’ve been data-pilled. What are the bottlenecks that you see for the next stage? Tying it to RL environments, is there sort of a lack of good, realistic RL environments that’s the next frontier, which maybe creates an opportunity for these startups?

Once you’re able to really work within an environment that takes a long time to build—these are not built in a day or 2—can you actually automate labor to the full extent of computer use, the way that you would need computer use to?

Yeah, I think, in my opinion, there is a lot of value in getting really good tasks, and getting really good tasks requires really good RL environments. The more complicated, realistic, and simulated we can make them, I think the better we’ll get. We’re kind of seeing that tasks matter more at this point, given the fact that we have such a strong algorithm.

I think creating data and figuring out the best tasks to train on is one of the big questions we have.

Yeah. There’s some generalization from training on one website to another, but if you want to get really, really good at something, the best thing to do is just train on that exact thing, right?

I think we’re definitely constrained by the things that we can represent in a way that we can train on. The GPT agent, for example, has such a general tool. It has a browser and a terminal, and between those 2 things, you can basically do most of the tasks that a human does on a computer.

So in theory, you can ask it to do anything that you can do on your computer. It's obviously not good enough to do that yet. But with the tools it has, in theory, you can push it really far. So now we just have to make it really good at all those things by training on way more things.

Let's talk about creative writing. Maybe you can talk about the improvements there and how you think about it.

That's one of my favorite improvements in GPT-5. The writing, I honestly find it very tender and touching, especially for a lot of the creative writing that we want to do. We were thinking through a bunch of different samples for the livestream, and every time I was like, “Oh, that's actually—that hits.” It's spooky, and I'm just like, “Oh, this feels like someone should have written this.”

That's not true.

I think it's compared to maybe the other things I'm better at. But it's so great to have this tool to help me craft things. I use it for as simple things as Slack messages, to figure out how to phrase something well, and it'll give me some iterations on how to say something to the team.

I want to see those prompts.

Yeah, we're now all just looking for em dashes, right? We're like—

Where do you stand?

Where do you stand on the em dash discourse?

I like em dashes.

I do that normally, and now people think I'm AI, too. Going back to the discourse for a second, Sam said in his interview with Jack, “If you had said 10 years ago that we would get AI models at the level of PhD students, I would think, Wow, the world looks so different, and yet we've basically taken it for granted.”

Do you think the improvements are similar? As soon as we get them, are we just going to be like, “Oh, yeah, now this is the standard”? Or do you think at some point there's going to be, “Oh, my God, this is…”? How do you think about people's ability to acclimate or adjust?

Yeah. I mean, it seems like people adjust really quickly, don't you think? The ChatGPT agent got released, and everyone was like, “Wow, that's so cool.” But then you just kind of take it for granted that you literally have this wizard in your pocket. You can ask it whatever random thought you have, and it just pops out a good essay, and you're like, “Oh, okay, cool. That's what's happening.”

I guess people adapt to things rather quickly, in my opinion, with technology. It is really easy, and I think because the form factor is so easy—even with new tools like deep research and ChatGPT agent—it's presented in such an easy way that people already know how to interface with it. I think as long as that's true, even with the models getting much smarter than us, it'll still be quite approachable to people.

Mhm.

Do you think the jump from GPT-4 to GPT-5 was bigger than GPT-3 to GPT-4, or maybe GPT-3.5 to GPT-4?

At least one thing for me in my usage of it is that sometimes I'm wondering if I have hard enough questions to ask it to actually highlight the difference. Because when it gets to a point where it's just answering what you need so well, it's almost harder to tell the difference in some areas. But with writing, I've been using it for a few weeks, and it's just kind of blown me away in a way that models previously haven't.

Maybe I'm biased—recency bias—but I think the jump from GPT-4 to GPT-5 is most impressive for me. I guess with GPT-3.5, when we first released it, the most common use case for me then was still just coding. GPT-4 was better at coding, but I feel like the jump between GPT-4 and GPT-5 in terms of breadth of ability to do things is just way different and way more. You can handle a lot more complex things than before, with the context length being much longer as well.

I think the jump from GPT-4 to GPT-5 is much bigger.

Is there anything the model categorically can't do?

I guess for GPT-5, we don't really take action in the real world yet. We're going to team that up with the agent.

Yeah.

Yeah. As I said, you could ask the agent to do anything, but it's not capable enough to do everything you want it to do yet. We take a conservative approach, especially by asking the user for confirmation before doing any kind of action that's irreversible, like sending an email, ordering something, or booking something.

I can imagine quite a number of tasks where you'd want it to take bulk actions, which you might not be able to do right now because it would ask you every single time. But as people get more comfortable using these things, and as they get better and you trust them more, you might allow it to do things for you without checking in with you as much.

Maybe just to build on that question, in terms of what it can't do today but what you would direct future research toward, if you look at coding, something like end-to-end DevOps, for example, feels like the logical next set of capabilities. Do you guys think we'll get there in—I don't know what you'll name it—GPT-5.5 or GPT-6? How far are we from something like that?

Yeah, I don't know about the exact thing of DevOps, but I do feel like, with the models getting much smarter, one other thing that came to my mind when you asked me the question is longer-running tasks and things like that.

I think GPT-5 is great because, within a couple of minutes, maybe you get a full-fledged app. But what would it look like if you actually gave it an hour, a day, or a week? What can actually get done? I think there's going to be a lot of interesting stuff. We're interested to see what will happen there.

Yeah, I think a lot of it is not just about the model capability, but actually how you set it up to do things. I'm sure that you could build something that's monitoring your Humio or Datadog, whatever, with these current models. It's just setting up the harness to make that possible.

And the same for agentic tasks. I think a lot of things that will be quite useful will be when the agent proactively does something for you. I don't think that's impossible today; it's just not set up that way. Eventually, as it proactively does things for you, we might get feedback on whether that was useful, and we can make it even better at triggering.

Agents—or agent—is probably the most overused word of 2025. That being said, your agent's launch was extremely exciting. What does that word mean to you in the context of capabilities that you'd like to build in the near term or have already built? What is most important for the agent to be able to do on behalf of your users?

My very general definition would just be something that does useful work for me, on my behalf, I would say asynchronously. You kind of leave it and then come back and get either a result or a question about what it's doing.

In terms of the roadmap for agents, longer term, you want it to be able to do anything that a chief of staff, assistant, or something like that would do for you. But in the more immediate term, there are a lot of new capabilities that we launched in ChatGPT agent that we just want to improve.

One of the main capabilities is Deep Research: being really good at synthesizing information from the internet. But I think we can improve capabilities for synthesizing information from all of the services that you use and private data that you have. Also being better at creating and editing artifacts like docs, slides, and spreadsheets, because so much of the useful work that people do in their jobs is basically just researching and making something.

I'm personally in love with all the consumer use cases. Making it better at shopping or planning a trip and those kinds of things are also really fun. That also involves taking an action, which is interesting because it's often the last step of a task, and maybe a task that would take less time for a human, but it's actually very hard—a very hard research question—to get it to do something, book something, or use a calendar picker. Once you have the end-to-end flow working really well, it can basically do anything.

Yeah, that's incredible. On the shopping piece, I now do not make a single large-ticket purchase without having ChatGPT put all the options in a table for me along the dimensions I care about. It's incredible.

But I want to push on the async piece, because I don't know if you would agree with this, but it felt like a revelation to me, at least at the beginning of the year, that people were willing to wait. You kind of think, “Oh, we want it faster. The value prop of this tool is that it gives me the answer fast,” right? That was sort of very 2024.

Clearly, this paradigm has shifted. People are willing to wait for high-quality, high-value answers and work. How do you think about the trade-off between how long something takes—how long you take to get something back to the user—versus the value that you’re actually providing? What do you think is the ideal frontier for something like that?

Yeah, it’s interesting because I built retrieval on ChatGPT and was on the browsing team before this. Christina was also on the browsing team, and we were always making these trade-offs and optimizing for latency. We were thinking, how can you best fill the context with the information you’ve retrieved so that the answer is pretty good in a few seconds?

With Deep Research, I was excited to remove latency as a constraint. Since we were going for tasks that are really hard for humans to do and would take humans many hours, we felt that if you asked an analyst to do this and it would take them 10 hours or 2 days, it seemed reasonable that someone would be willing to wait 5 minutes in your product. We made that bet, and luckily it seems to be the case.

But I also think that initially, people were like, “Oh, this is amazing. It’s doing all this work that would have taken me so long.” Now people are like, “Okay, but I want it now. I want it in 30 seconds.” The bar keeps changing.

I was going to say, is there any sort of rule of thumb—and I’m sure it’s constantly shifting—where, as long as you’re 10 times faster than it would take a human to do, they’re willing to wait? Or is that just constantly shifting sand?

I think with these launches, people’s expectations keep changing. I don’t think we have a specific number. One thing that’s interesting is that people sometimes just bias toward thinking that the longer answer is more thorough or has done more work, which I don’t necessarily think is the case.

Deep Research, for example, always gives you a really long report. Sometimes I don’t want to read this whole long report. I actually don’t like that. An agent will only give you a long report if you ask for it. But sometimes, since people are now used to always getting a really long report, they’re like, “Wait, I’ve been waiting. Where’s my long report?”

Sometimes it’s really hard to find a specific piece of information, and it would have also taken a human a long time because it’s on page 10 of the results. An agent finds that information. I think it’s interesting how you can condition people’s expectations with a product. With Deep Research, it always thinks for a really long time, which I don’t necessarily think is a feature, but now people are really used to the amount of time that they wait.

I think we hear this with GPT-5. Internally, when people are testing it, they’re like, “I thought I asked a really hard question.” I feel a little bit insulted that it only takes seconds, or when it doesn’t even want to think at all.

It’s like the Mark Twain line: “I didn’t have time to write you a short letter, so I wrote you a long one.”

Yeah. Yeah.

Why don’t you talk about the model? Why don’t we have reliable agency? What are the main bottlenecks as you see them?

Yeah. I think a big part of it is that the things we train on, we’re often really good at. With things outside of that, it can be a bit inconsistent. Sometimes it’s good at those things, and sometimes it’s not.

Creating more data across a broader range of things that we want it to be good at is important. I also think what’s interesting with agents is that when something is doing something on your behalf and has access to your private data and the things that you use, it’s kind of more scary what it could do to achieve its final goal.

In theory, if you asked it to buy you something and make sure that you like it, it could go and buy 5 things just to make sure that you liked 1 of them, which you might not necessarily want. Having oversight during training is also an interesting area. There are new things that we have to develop to push these agents even further.

Every time we get a smarter base model, it improves every model that’s built on top of it. I think that will also help, especially with multimodal capabilities, as Christina said, with computer use. It’s literally looking at screenshots of a webpage, and it’s a little interesting because of the way that humans focus on specific things.

It’s a lot to expect a model to just take a whole image and be able to know everything about the image. When we’re looking at something, we focus on a specific thing. I think there’s lots of room for improvement in lots of areas. Sorry, that was kind of a general answer.

No, no. Actually, that last example gets into something that we were curious about, which ties back to training data as well. What specific categories of browsing tasks are challenging for agents today? Do you have thoughts on how you’d overcome this for the next iteration of the model?

I think one thing is that pre-training is based on what data is available. When we’ve done these pre-training runs, there’s not much data out there to begin with of people using computers. Computer usage is not really a thing where there’s lots of data available, and this is something we actually have to seek out now that it’s a capability we want.

That’s probably a big one just for general improvements in computer use.

Do you think you’ll lean more heavily on human data vendors to help collect that? Given that it doesn’t exist, to your point, recorded in the way that might be most helpful for training, how do we overcome that? It’s probably the most useful application of the models for knowledge work. How do you overcome that?

I think one cool thing is that, for example, for the initial Deep Research, there weren’t really any datasets that existed for browsing in the same way that you have a math dataset that already exists. We had to create all this data. Once you have good browsing models or good computer-use models, you can bootstrap them to help you make data. I think that’s pretty—

Christina, can you explain what mid-training is and what it achieves that pre-training or post-training doesn’t?

With your pre-training runs, these are your big runs. These are the massive ones. It’s what we’re building all these giant clusters for.

You can think of mid-training as literally the middle. We do it after pre-training but before post-training. You can think of it as a way to extend the model’s intelligence without having to do a whole new pre-training run. This is mostly focused on data and built off the pre-training models.

It’s a way for us to do things like update the knowledge cutoff of these models. When you pre-train it, you’re kind of like, “Shoot, now we’re stuck in this state, and we can’t ever update it again.” It doesn’t quite make sense to put all that data into post-training.

Mid-training is just a smaller pre-training run to help expand the model’s intelligence and up-to-dateness.

Christina, did you work on WebGPT?

Yes, I did.

Okay, so you’re basically like an AI historian.

Yes. Yes.

She also worked on computer use. I’m an elder. Can you reflect back a little bit to 4 or 5 years ago and reflect on what the biggest things were? If you were to predict 5 years out, what are the inflection points or biggest things that would have surprised you?

Honestly, with WebGPT, the main thing we were excited about was trying to ground these language models. We had so many issues with hallucinations and the model just saying random things. We didn’t really do mid-training then, so the question was, how do we make sure the model is actually up-to-date and factually accurate?

That’s how we thought about it: “Let’s give it a browsing tool.” I think that makes sense. Then, like I said, that kind of went from, “Oh, actually, I want to keep asking questions. What would a chatbot look like?”

At this point, there had been a few chatbots by a few other companies. I feel like a chatbot is also a very common AI thing to think of, but they were quite unpopular at the time. We weren’t really even sure that this was something useful for people to work on, or something people would use, or whether people would be excited about it.

Was this really a research innovation? Were we remaking the Turing test here? But I think it clicked for me that maybe there was actually something interesting happening here. We gave early access to about 50 people.

Most of those people were people I lived with at the time. Two of my roommates just used it all the time. They just would never stop using it. They would have these long conversations, and they would ask it quite technical things because they were also AI researchers. I was like, “Oh, this is kind of interesting.”

At the time, we were thinking, “Okay, we have the chatbot. Do we make this a really specific meeting-bot type of thing? Do we make it a coding helper?” But it was interesting to see my two roommates use it for anything and everything, literally chatting with it the whole workday as they were using it. I was like, “Oh, this is kind of interesting.”

It was also interesting to see that the majority of the people I gave access to on that 50-person list didn’t really use it that much. I was like, “There’s clearly something here, but it’s maybe not for everyone yet.” But there’s something here.

When did you realize, “I’m working at one of the most important companies of this generation”? When was the moment when you thought, “This is something that I obviously believe is important—that’s why I joined”—but you realized the scale and significance?

Honestly, I had this moment before I joined OpenAI. I think with the scaling laws paper with GPT-3, it hit me that if this exponential is true, there’s not really much else I want to spend my life working on. I want to be part of this story. There are going to be so many interesting things unlocked with this, and I think this is probably the next level in terms of technology. It made me realize, “Oh, I should probably start reading about deep learning and figure out how I can get into one of these labs.”

Isa, what was your moment?

I think for me, it was also before I started working at OpenAI. I first learned about OpenAI in an AI class or some kind of computer science class, and they were saying, “They trained on the whole internet.” I was like, “That’s so crazy. What is this company?” Then I started using GPT-3. I was a power user of the OpenAI Playground, and at a certain point had early access to different OpenAI features, like embeddings and things like that, and just became a big OpenAI fan.

It’s a little embarrassing, but it’s fine because it got me here. Eventually they were like, “Okay, you’re stalking us. Do you want to interview here?” I think it was pretty clear to me how much I was using GPT-3. It wasn’t even comparable to what we have now—it pales in comparison—but I was hooked from then and just trying to figure out a way to work here.

Maybe a question more on the company-building front. We all sort of read and reread Calvin French-Owen’s piece, just his reflections on working at OpenAI. You don’t have to comment on that piece unless you want to, but I’d love your reflections on the change that you’ve seen over the last 4 years—or even less than that, given that I think it was only covering 1 year of change. What are the biggest things that you’ve seen change at OpenAI?

When I first joined OpenAI, the applied team was 10 engineers or something. We didn’t really have this product arm. We had just launched the API. It was a completely different world. I think AI is on most people’s minds now after ChatGPT, but pre-ChatGPT, people didn’t really know what AI was or think about it as much. It’s kind of cool working at a place where my parents know what I do now. That’s really cool.

The company is obviously a lot bigger, but with that, we can take a lot more bets. When I first joined OpenAI, there were way fewer people—it was much, much smaller, around 200 people—and I think we’re close to—

A few thousand, for sure. When I joined, it was also a few hundred before ChatGPT. So it’s obviously very different in how all of your friends have heard of what you work on, but culturally the company is much bigger. I still think we’ve maintained this—it still feels very much like a startup.

Some people who come from a startup are surprised: “Oh, I’m working even harder than when I was working at the startup that I founded.” Ideas can still come from anywhere, and if you take initiative and want to make something happen, you can. It doesn’t really matter how senior you are or anything like that. I think we’ve been able to maintain that culture, which is pretty special.

Yeah, we definitely reward agency, and I think that’s always been true. Especially on the research side, the teams are quite small. When Isa was working on Deep Research, it was 2 people—still 2.

I think we still do that on the research side. Most research teams are quite small and nimble for that reason.

Earlier you said that we do something at OpenAI that startups never do, which is try to appeal to every single person with the product. Are there other things that come to mind that OpenAI just does differently from your peers or other startups—things that we may not appreciate being on the inside?

I think it’s different for different teams, but my team collaborates so closely with the applied engineering team, the product team, and the design team in a way that I think research can sometimes be quite separate from the rest of the company. For us, it’s so integrated—we all sit together.

Sometimes the researchers will help with implementing something. I’m not sure the engineers are always happy about it, but we’ll try. They’ll get into the front-end code, and vice versa: the engineers will help us with things we’re doing for model-training runs and things like that. Some of the product teams are quite integrated. I think for post-training, it’s a pretty common pattern, which just lets you move really quickly.

I guess one thing that I think is unique about OpenAI is that you’re both very much a consumer company, by revenue and products, but also an enterprise company. How does that work internally? What would you consider yourselves, or is that even the wrong paradigm to think about?

If you tie it to the mission, we’re trying to make the most capable thing, and we’re also trying to make it useful to as many people as possible and accessible to as many people as possible. In that framing, I think it makes a lot of sense.

The concept of taste has become very widely used. What does good taste mean within OpenAI? How do you know when you see it? You know it when you see it. Is that something that, even in a world where the cost to produce everything just keeps going down and down, is the one thing that’s not commoditizable? Or is that also shifting, given that maybe it can go into the training data?

No, I think taste is quite important, especially now that, like I said, our models are getting smarter and it’s easier to use them as tools. Having the right direction matters a lot now, and having the right intuitions and the right questions you want to ask. I would say maybe it matters more now than before.

I’ve also been surprised by how often the simplest, easiest-to-explain thing is the thing that works best. Sometimes it seems very obvious, but it’s quite hard to get the details of something right. Usually, good researcher taste is just like—

Pretty much simplifying the problem to the dumbest thing or the simplest thing you can do.

Yeah. I feel like with every research release we do, when people figure out what happened there, they’re like, “Oh, that’s so simple. I should have thought of that. Obviously, obviously that would have worked.” But it’s knowing to try that obvious—or, at the time, not obvious—thing that’s obvious in hindsight.

And then all of the details around—

Yeah.

The hyperparameters and all these things, like the inference—that’s obviously very hard, but the actual concept itself is usually—

Usually pretty straightforward.

Very cool. Taste is AAM's razor.

Yeah, I think with GPT, the word that’s been in my mind throughout all of this is “usable.” The thing that we’re excited about is getting this out to everyone. We’re excited to get our best reasoning models out to free users now. I’m excited to get our smartest model yet to everyone and see what people are going to actually use it for.

That’s a great place to wrap. Christina, thanks so much for coming on the podcast.

Yeah, thank you.

Thank you for having us.