OpenAI 如何服务每周8亿用户：模型专业化与微调

We want ChatGPT as a first-party app. A first-party app is a really great way to get 800 million users or whatever now.

One-tenth of the globe, right? [laughter]

Yeah. Yeah, 10% of the globe uses it—

Every week. Every week.

Yeah. Even within OpenAI, the thinking was that there would be one model to rule them all. It’s definitely completely changed. It’s becoming increasingly clear that there will be room for a bunch of specialized models. There will likely be a proliferation of other types of models. Companies just have giant treasure troves of data that they’re sitting on. The big unlock that has happened recently is reinforcement fine-tuning. With that setup, we’re now letting you actually run RL, which allows you to leverage your data way more.

Sherwin, thanks very much for joining. We’re being joined by Sherwin Wu. It’d be great if you provided the long form of your background as we get into this, just for those who may not know you. I view Sherwin as one of the top AI thought leaders, so I’m really looking forward to this.

Yeah, thanks for having me. I’m really excited to be on the podcast. Maybe we can start from the present day and go backward. I currently lead the engineering team for OpenAI’s developer platform. The biggest product there, of course, is the API.

Is there more to the developer platform than the API? I just assumed they were synonymous.

I also think about other things that we put into our platform. Technically, our government work is also about offering deployments in different areas. We do have a local deployment at Los Alamos National Laboratory. It’s super cool. I went to visit it, and it’s very different from what I’m used to—a classified supercomputer with our model running there.

Did you go to Los Alamos?

We did. Yeah, I went to Los Alamos. It was great. They showed us around, including some of the historic sites. Real history.

I used to work at Livermore, man.

I work on the developer platform. I’ve been working on it for around 3 years now. I joined in 2022 and was basically hired to work on the API product, which at the time was the only product that OpenAI had. I’ve basically just worked on it the entire time. I’ve always been super interested in the developer side and the startup story of this technology, so it’s been really cool to see this evolve.

Before OpenAI, I was at Opendoor for around 6 years. I was working on the pricing side.

I think that’s such a dissonant transition—pricing at Opendoor to running an API.

It’s such a different experience. It’s been fascinating for me to see the differences between the companies; they’re run so differently. They both have “open” in the name, so there’s some overlap, but that’s basically it.

I was at Opendoor for around 6 years, working on the pricing team. Our team basically ran the machine-learning models. We were actually pricing the assets on Opendoor—the inventory.

Exactly.

Opendoor would buy and sell homes. Its main business was buying homes directly from people who were selling them, with all-cash offers. My team was responsible for determining how much we would pay for them. It was a really fun machine-learning challenge, and it had a huge operational element as well, because obviously not everything was automated.

Is there any sense of that on the API side, like GPU-capacity buying, or is it totally unrelated?

On the API side, there’s a small bit of how we price the models, but I don’t think we do anything as sophisticated as Opendoor. Opendoor is just such a hard problem. It’s such an expensive asset, and the holding costs are very expensive. You’re holding on to it for months at a time, and there’s variability in the holding time.

A massive long tail of potential things that could happen.

Long tail, yes. You try to think about it from a portfolio perspective, and if one of them means you’re holding on to it for 2 years, it blows everything up.

Everything goes negative.

Everything goes negative. So it’s a very, very different challenge. I spent 6 years there, with lots of ups and downs. I saw a lot of the booms, saw a lot of the struggles, and then we IPOed before I left.

In general, it was a great experience. For me, it also had a very business-operations-oriented, by-the-book type of culture, whereas OpenAI is very different.

What’s so interesting, and I was just thinking about it now, is that even for a company like that—you don’t think about it as a tech company—but if there is a deep technology problem, it’s actually the pricing, right? It’s actually a machine-learning problem. It’s not the website driving it, it’s not the platform, and it’s not the API. It’s literally that.

Yep. Yep. Yep. That’s what attracted me to it. I thought that was interesting. It’s also a much lower-margin business than OpenAI, because you’re making a tiny spread on these homes. They talk about basis points—“eating basis points for breakfast” and all that.

Anyway, I was at Opendoor for around 6 years. Before that, my first job out of college was at Quora, with Adam D’Angelo’s group there. I was working on the news feed.

Oh, yeah.

I worked on news-feed ranking for a bit and worked on the product side. That was actually my first exposure to machine learning in industry, and I learned a lot from the engineers at Quora. We hired a lot of the early feed engineers.

Was Charlie still there when you were there?

Charlie wasn’t there when I was there. It was a really legendary team, known as a super-iconic founding team. The early founding team was really solid. I’m still amazed by the quality of the talent we had. I think the company was only 50 to 100 people, but a bunch of the Perplexity team was there. Denis was on the feed team with me. Johnny Ho and Jerry Ma were there, and Alexander Wang, who’s at Scale now, was there between high school and college. It was an incredible team. I don’t think I appreciated it while I was there. It was a good group.

How did you get to Quora? What did you study in undergrad?

Before that, I was at MIT for undergrad. I studied computer science. I did one of those computer-science programs where you can cram in a master’s degree.

I ended up at Quora because I got what we called an externship there. At MIT, you actually get January off. There’s the fall semester, then January is off, and then you have the spring semester. It’s called Independent Activities Period. Some people take classes, some people do nothing, and some people do month-long internships. Some crazy companies will offer a month-long internship to a college student.

Did you come out here from Boston?

Yeah, it was crazy. You had to apply. This was, I think, January 2013, and I remember the Quora internship was the one that paid the most. They paid, I think, $8,000 or $9,000.

I can eat for a year.

Yeah. As a college student, that was great. It was for a month, and you were ramping up for half the time. They would fly you out here, so I did the interviews and luckily got an offer. I came out for January. That was right when they moved into their new Mountain View office, and I basically ramped up for 2 weeks and then had 2 weeks of good productivity working on the feed team.

Was that a user-facing product?

Yeah. I distinctly remember my externship project for those 2 weeks was to add a couple of features to our feature store, and those would make their way into the model. My mentor there was Tudor Achim, who’s now running, I think, Harmonic. Crazy team.

This is—I mean, by the way, I think it’s one of the untold stories of Silicon Valley, how good that original team was. A lot of them are still there and still good, but the diaspora from Quora is everywhere.

Yeah. That’s actually how I ended up at OpenAI, too. Fast-forwarding from there, OpenAI had kept a quiet profile, but I’d always kept tabs on them because a bunch of the Quora core people I knew had ended up there. I’d check in on it, and they’d say, “Yeah, something crazy is happening here. You should definitely check it out.”

So, yeah, I definitely owe a lot to Quora. Part of the reason I went there over other options as a new grad was that the team was just so incredible, and I felt like I could learn a ton from them. I didn’t think about everything that would happen afterward. I was just like, “Man, if I could absorb some knowledge from this group of people, it’d be great.”

Awesome. Yeah, one place I wanted to start is something that I find very unique about OpenAI. It’s both a pretty horizontal company—it’s got an API. I would say we have this massive portfolio of companies, and a good fraction of them use the API. But it’s also a vertical company, in that you have full-on apps. Everybody uses ChatGPT, for example, and you’re responsible for the API and the developer-tools side.

Is there an internal tension between the two? Is that a discussion, where the API may help a competitor to the vertical version? Or are things just growing so fast that it’s not an issue? I’d love to hear how you think about that. It’s very unusual for companies to have both of these things this early. It’s very unusual.

Yeah. I completely agree. I think there is some amount of tension. One thing that really helps here is that Sam and Greg, just from a founder perspective, have been very principled from day one in the way we approach this. They’ve always told us, “We want ChatGPT as a first-party app. We also want the API.”

The nice thing is that they’re able to do this because, at the end of the day, it comes back to the mission of OpenAI, which is to create AI and distribute the benefits as broadly as possible. If you interpret that broadly, you want it on as many surfaces as possible. The first-party app is a really great way to reach, I don’t know, 800 million WAUs or whatever it is now.

100 million, wow. I don’t think many people listening to this understand how big that is. That is crazy.

Yeah, it’s pretty mind-boggling to think about. That’s going to be historic, actually—the amount of time it’s taken to get to 800 million.

It’s historic. It’s also just the amount of time and how much we’ve had to scale up.

Like a tenth of the globe, right?

Yeah. Ten percent of the globe uses it.

Every week. Every week.

Yeah, and it’s growing. At some point, it’ll go even higher than that. Obviously, the reach there is unmatched. But there’s also the ability to have a platform where we can reach even more than that.

One thing we talk about internally sometimes is, “What is our end-user reach from the API?” It’s actually really broad. At some points, it was definitely larger than ChatGPT. It’s hard to say now because ChatGPT is growing so quickly, but the fact that we’re able to tap into all of this and get the reach that we want is really good.

There’s definitely some tension sometimes. I think it’s come up in a couple of places. One is on the product side. As you mentioned, sometimes there are competitors building on our platform who might not be happy if ChatGPT launches something that competes with them.

Platform.

Who might not be happy if ChatGPT launches something that competes with them.

Yeah. I mean, that’s the tale as old as the cloud or operating systems. Does ChatGPT worry about the competitor—about enabling a competitor?

Yeah. The interesting thing is that I would say, not particularly, mostly because we’ve been growing so quickly. It’s just such a force right now.

Yeah. Growth solves so many different things. The other way we think about it is that everyone is building around AGI and building toward AGI. Of course, there’s going to be some overlap here. But I would say, at least in my position, I feel more of this tension from the customers—the API customers themselves. They’re like, “Oh my gosh, are you going to build this thing that I’m working on?”

That story is as old as computer systems. There’s never not been a computer platform that didn’t have that problem.

I want to try one out on you. Historically, with offering core services and APIs, you can get disintermediated, right? I can build on top of it, but then the user doesn’t know what I built on top of the cloud. I get disintermediated from the cloud, and then I can switch to another cloud or whatever.

It occurs to me that it’s kind of hard to do that with these models because the models are so hard to abstract away. They’re unruly. If you try to have traditional software drive them, they don’t manage very well.

Part of me thinks that it’s almost like an anti-disintermediation technology that you have to expose directly to the user. Does that make sense? Even if I think ChatGPT is really just trying to expose the model to the user, and the API is trying to expose the model to the user, there’s almost an argument that if the real value is in the models, it doesn’t really matter how you get it to people. It’s going to be very tough for someone to abstract it away in the classic computer-science sense, where they don’t know they’re using the model. You always know you’re using GPT-5.

Yeah. The interesting thing is that the entire industry has slowly changed its mind around this, too. In the beginning, we thought these were all going to be interchangeable.

It’s just like software. You can swap out a piece of infrastructure.

We’re learning this on the product side with the GPT-5 launch, GPT-4o, and how so many people liked o3 and GPT-4o and all of that.

I felt that. I felt that when it changed. I was like, “You’re not as nice to me.” I like the validation.

It’s actually funny because I really loved GPT-5’s personality, but the way I used ChatGPT was very utilitarian. It was mostly for work or information.

Yeah, I’ve definitely come around. I actually felt a dissonance when it changed. There’s this emotional thing that goes on.

It’s a disintermediation technology. You kind of have to show this to the user.

Yeah. Then you see a lot of more successful products, like Cursor, doing this directly, especially coding products where users want more control. We’ve even seen some more general consumer products do this, so it’s definitely been true on the consumer side.

The interesting thing is that I think it’s also been true on the API side.

No, exactly. That’s exactly what I’m saying. The argument could be that I could use the API—

—to disintermediate you, but you don’t see that happening because it’s so hard to put a layer of software between a model and a person. You almost have to expose the model.

The models are almost diverging in terms of what they’re good at and their specific use cases. I think there’s going to be more and more of this. Basically, it’s been surprisingly hard—or rather, the retention of people building on our API has been surprisingly high, especially when people thought you could just swap things around.

You might even have tools that help you swap things around, but the stickiness of the model itself has been surprising.

Do you think that’s because of a relationship between the user and the model, or do you think it’s more of a technical thing? My evals work for OpenAI, and the correctness is maintained.

Yeah, I think it’s both. There’s definitely an end-user piece here, which is what we’ve heard from some of our customers: They just get familiar with the model itself.

But I also think there’s a technical piece. As a developer, especially with startups, you’re going really deep with these models and iterating on them, trying to get them really good within your particular harness.

You're iterating on your harness itself. You're giving it different tools here and there, and so you really do end up building a product around the model. There is a technical piece where, as you keep building with a particular product like GPT-5, you're actually building more around it so that your product works uniquely well with that model.

So I use Cursor a lot, just for a lot of stuff, like writing blogs. We're investors, and I use it sometimes for coding.

It's remarkable how many models I use in Cursor. Literally, my go-to model is GPT-5. I love GPT-5. I think it's a phenomenal model, and I use Max Mode with GPT-5 for planning. I like the tab-completion model that's in Cursor, and the new model they just dropped is good for some basic stuff. The Composer one is good.

And so I think that kind of reflects this, too, because there's a particular model for each particular use case. I've talked to a bunch of people who've used the new Composer model, and it's really good for a fast first pass, keeping you in flow, and then you bubble out to another model if you want deeper things. I literally sit down with GPT-5 to help me plan something out, and it's really good at that. When I'm coding, I'm doing the quick-chat thing, then I'll use Composer, and if there's some crazy bug or something like that—

So, do you remember in the early days of all of this, when we were like, “There's going to be 1 model”? Even investors were like, “We will never invest in a model company, because there will only be 1 model and it's going to be AGI.” But the reality is, it feels like there's this massive proliferation of models. Like you said before, they're doing many things.

So maybe 2 questions—maybe too blunt or too crass. The first one is: What does that mean for AGI? [laughter] And the second is: What does that mean for OpenAI? Does that mean you end up with a model portfolio? Do you select a subset? Do you think this all gets superseded by some god model in the future? How does that play out? Because it's against what most people thought. Most people thought this was all going toward 1 large model that does everything.

Yeah. I think the crazy thing about all this is just how everyone's thinking has changed over time. I distinctly remember this, and the crazy thing is that it wasn't that long ago—just 2 or 3 years ago. I remember that even within OpenAI, the thinking was that there would be 1 model that rules them all. It's like, why would you even have a fine-tuning API product? Why would you even want to iterate on it? There was going to be this 1 model that just subsumed everything, and that was also the most simplistic view of what AGI would look like.

Yeah, it's definitely completely changed since then. But the other thing to keep in mind is that it might continue to change, even from where we are today. It's becoming increasingly clear, I think, that there will be room for a bunch of specialized models. There will likely be a proliferation of other types of models. You see us do this with the Codex model itself; we have GPT-4, GPT-4o, GPT-5, and all of this. So I definitely think there's room for all of this.

I don't think that's bad, for what it's worth. If anything, I think as we've tried to move toward AGI, things have just been very unexpected, and the market evolved and the product portfolio evolved because of that. So I don't think it's a bad thing at all.

You could easily argue that it's very good for OpenAI and very good for the model companies, because you don't have these winner-take-all, consolidated dynamics. You just have a healthier ecosystem and a lot more solutions you can provide.

Yeah, as the ecosystem grows, it generally is helpful. This is one thing we actually think about a lot, too: As the general AI ecosystem grows, OpenAI stands to benefit a lot from this. This is also why, with some of our products, we've even started opening them up to other models. Our Evals product now allows you to bring in other models. We think any rising tide generally helps us here.

I want to talk about that in just a bit, because the open source is actually very interesting. I thought the open-source model was great, but clearly it's something that companies have to be careful with. Before that, I want to talk a little bit about the fine-tuning API.

I've noticed that you're moving toward more sophisticated uses of things like fine-tuning, which, in a way, you could read as a bit of a capitulation: There is product-specific data and product-specific use cases that a general model won't do, to your point. As opposed to a proliferation of models, you do that.

It seems like a lot of that data is actually very valuable, right? To what extent is there interest in almost a tit-for-tat, where you can expose the ability to get product data into fine-tuning and then you also benefit from that data because the vendors provide it to you, versus they keep their own data and there's no interest in that? Because it feels to me like the next level of scaling. This is kind of where we're at, so I'm just curious how—

Yeah. So maybe even taking a step back, the main reason why we invested in a fine-tuning API in the very beginning is, first, there's been huge demand from people to be able to customize the models a bit more. It kind of goes into prompt engineering, and I think the industry has changed its mind on that as well; it's evolved. But the second thing is exactly what you said: Companies just have giant treasure troves of data that they're sitting on, that they would like to utilize in some fashion in this AI wave.

The simple thing is to put it in some vector database, do RAG with it, or something, but if they have a more technical team, they do want to see how they can use it to customize the models. And so that is actually the main reason why we've invested in this.

The interesting thing was that, way back in 2022 or 2023, our fine-tuning offering was, I'd say, too limited, so it was very difficult for people to tap into and use this data. It was just an SFT, or supervised fine-tuning, API, and we were like, “You can kind of use it,” but in practice it really was only useful for instruction following plus changing the tone. You were really instructing it.

The big unlock that has happened recently is the reinforcement fine-tuning model, because with that setup, we're now letting you actually run RL, which is more finicky and harder, and you need to invest more in it, but it allows you to leverage your data way more.

This is just a naive question for me: It feels, from my understanding and from my own portfolio, like there are 2 modalities of use. One of them is: I've got a treasure trove of data that I've had for a long time, and I create my model on that treasure trove of data. All that happens offline, and then I deploy that.

Yeah.

There's another one, which is: I actually have the product being used in real time. I've got a bunch of users, and I can actually get much closer to the user. I can A/B test and decide which data to use, and it's more of a near-real-time thing. Is the focus more on product stuff or more on the treasure trove?

So the dream with the fine-tuning API was that we should be able to handle both. We actually had this dream, and we have this whole LoRA setup with fine-tuning inference, where we should just be able to scale to millions and millions of these fine-tuned models, which is usually what would happen if you have this online-learning thing.

In practice, it's mostly been offline data that they've already created, or that they're creating with experts, or something, while using their product. That's what they're able to use here. But the main thing I was trying to say around the reinforcement fine-tuning API is that it kind of changes the paradigm away from just small, incremental improvements—tone improvements, which is what SFT did—to actually improving the model to potentially SOTA-level performance on a particular use case that you know about.

That's where people have really started using the reinforcement fine-tuning API, and that's why it's gotten more uptake. If the discussion is less like, “Hey, I can make this model not speak in a certain way,” or improve less complex planning or something, and more like, “You can create the world's best model using your dataset with RFT,” then it becomes a lot more—

And will you ever find ways to get access to that data? If I had the data and I wanted cheap GPUs, I'd trade you for it. I don't know.

Yeah, we've talked about this, and we've actually been piloting some pricing here, too. This data is really helpful, and it's kind of hard to get. If you actually build with the reinforcement fine-tuning API, you can get discounted inference and potentially free training, too, if you're willing to share the data.

It’s always up to the customer, but if they do, it is helpful for us, and there will be benefits for the customer as well.

That’s awesome. You said that views on prompt engineering have changed.

Yeah, I wasn’t actually aware of that. All the other things I was aware of; this one I wasn’t. How?

I mean, I think the prevailing view—this is back in 2022; I remember I was talking to so many people—and they were basically—this is similar to the single-model AGI view as well—which is that prompt engineering is just not going to be a thing. You’re not going to have to think about what you’re putting in the context window in the future. The model will just be good enough, and it’ll know what you need to do. And that’s not a thing.

Yeah. But I don’t know, maybe people forget it, but that was a very common belief back then because of the scaling laws or whatever. You’d just mind-meld with the model, and prompting and instruction following would be so good that you wouldn’t really need to do it. If anything, it’s clearly been wrong.

But it is interesting because I think it’s a slightly different world that we’re in now, where the models have gotten really, really good at instruction following relative to GPT-3.5 or something. I think the name of the game now is less prompt engineering as we had thought about it 2 years ago. It’s more the context-engineering side: What are the tools you give it? What is the data that it pulls in? When does it pull in the right data?

Well, this is very interesting. To reduce it to an almost absurdly simplistic level, the weird thing about RAG, for example—the classic use of RAG—is that you’re using cosine similarity to choose something that you’re going to feed into a superintelligence. You’re like, “I’m going to randomly grab this thing based on embedding space,” and then you want the superintelligence to decide the thing to do. Pushing intelligence into that retrieval clearly makes a lot of sense—pushing the intelligence out, in a way.

Exactly. To be fair, I think RAG was introduced when the models were pre-reasoning models. You only had one shot to do this, and it wasn’t that smart. But now that we have reasoning models, one of my favorite models is actually o3 because it was one of the most diligent models, in a sense.

It would just do all these tool calls, and it’s really the intelligence itself trying to do the tool calls or RAG or anything like that, or write the code to execute. The paradigm has shifted there, but because of that, I think context engineering—prompt engineering, what you put in, what you give the model—is extra important now.

Yeah. Okay, so you have an API, which is horizontal. You’ve got ChatGPT and other products, which are vertical. We haven’t even talked about pixels. It’s all just language. Are agents a new modality? Is that something else, like Codex?

What do you mean by modality there?

They feel both vertical and horizontal to me. To me, ChatGPT is a product, right? It’s a product, and my mom uses it, right?

Right.

An API is a developer thing. You give it to a developer, and a CLI is somewhere in between to me. Is it a product? Is it horizontal? How is it handled internally? Is it a totally separate team that does agents, or—

No. So it’s interesting because the way that you framed it just now almost seemed like agents were this singular concept that might have their own particular—

Maybe a better question is: What is an agent to you?

Yeah. Even getting a language for this is important for the conversation.

Yeah.

I actually don’t even know if it’d be helpful for me to share, but my general take on agents is that it’s an AI that will take actions on your behalf and can work over long time horizons. I think that’s the pretty general utilitarian definition.

If you think about it that way, maybe this is what you mean by modality, but it is just a way of using AI. I guess it could be viewed as a modality, but we don’t view it as a separate thing from the API. Let me just try and give you a sense of where this question is coming from.

I know how to build a product, and we know how to do go-to-market for products. We know the implications of turning them into platforms. We’ve been doing this for a very long time, right? We know how to do the same thing for APIs, right? We know how to do billing. We know the tension of people building on top of it and all of that stuff.

What I’ve been trying to understand—and this is just maybe a personal inquiry—is whether an agent sits in one of those 2 camps. Is it more like the product camp? Is it more like the—

Yeah, yeah.

Because it’s kind of both. I could literally give you code, and as a user, you just talk to it. Or I could kind of embed it in my app. But then that means something to you as far as how you price it and what it means for the ecosystem. For example, would you be fine if I started a company and just built it around Codex? Is that a thing?

Starting a company and building it around Codex? I actually think that would be great. We released the Codex SDK, and we want people to be able to build it and hack on it. Actually, I think this might be what you’re getting at, which is—

This is a unique thing about OpenAI and kind of reflects how it’s run: At the end of the day, OpenAI is an AGI company. It’s an intelligence company. Agents are just one way in which this intelligence can be manifested.

The way that we actually think about it internally is that all of our different product lines—Sora, Codex, the API, ChatGPT—are just different interfaces and different ways of deploying this. You don’t really need—

So there’s no single team thinking about agents?

I would say the way that it manifests itself more is that each product area thinks about what this intelligence is actually turning into—a form where agentic behavior is more possible. What would that look like in a first-party product like ChatGPT? What would that look like? This is actually why Codex ended up becoming its own product. What would it look like in a coding-style product?

We explored it in ChatGPT, and it kind of worked there, but the CLI interface actually makes a lot more sense. That’s another interface to deploy it. If you look at the API itself, that’s another interface to deploy it. You’re thinking about it in a slightly different way because it’s a developer-first mindset. We’re helping other people build it. The pricing is slightly different, but it’s all these different manifestations of this core intelligence—that is the agentic behavior.

Yeah, it is so remarkable how much of this entire economy is basically just token laundering, in a sense, right? It’s literally like anything I can do to get English in, or natural language in, and then the intelligence out. It’s because things are so resistant to layering. It’s so hard to layer a language out.

I could even do it easily with Codex. I could use it as a component of a program and basically launder intelligence through it. Of course, I’d be charged to do that.

My view of this, having now seen so many launches of different products, is that I’ve seen agent launches and the definition that you have. I’ve definitely seen APIs, and I’ve seen products. These are actually quite different from what we’re used to. The COGS is different, the defensibility is different—everything—so we’re kind of rewriting it.

You came from a pricing background. I mean, you were working on a demand model for pricing. Now you have the API. I’d just love your thoughts on how you’ve evolved your thinking and how you price access to intelligence, where you don’t know how many people are going to use it. It’s almost certainly usage-based billing, not something else. Can you talk just a bit about the philosophy around pricing these things? Is it different for product versus API?

Yeah, I think the honest truth here is that it’s evolved over time as well. I actually think the simplest reason why we’ve used usage-based pricing on the API, honestly, is because it’s closest to how it’s actually being used.

And so that's kind of how we started. I actually think usage-based pricing on the API has surprisingly held strong, and I actually think this might be something that we'll keep doing for quite a long time, mostly because—

The thing is, I don't know how you don't do usage-based.

Yeah. And then there's also the strategy of how we price it. Internally, one thing we do is we always make sure that we actually price our usage-based pricing from a cost-plus perspective. We're trying to make sure that we're being responsible from a margin perspective. By the way, this is a huge shift in the industry in general, because I remember the shift from on-prem to recurring.

Yeah, that was a big deal. That created Zuora. It created whole companies. There are whole books and a bunch of consultants on how you do this change.

I think the shift to usage is even bigger, and it's also a really hard technical problem.

I can't even imagine 800 million. Wow. How do you build—

Yeah, well, 800 million is a little easier because it's not usage-based pricing; it's subscription, so it's that way. But I mean, there are still a lot of users on the API that we need to manage on the billing side.

Are there some overages or stuff you've got to deal with on that?

What do you mean by overages?

I don't know. I guess most people have quotas, and then we'll set a max—we don't let people go over. But in practice, these quotas are pretty massive.

That would literally be one of the most complex systems somebody's ever built if you did usage-based at that scale. I mean, these are very, very— you have to be correct. These are very hard systems to scale.

Yep, yep, yep. Yeah, yeah.

I mean, we have a whole team thinking about this now internally. Usage pricing is also interesting. We acquired this company called Rockset a while ago. One of the founders—his name is Venkat—is right here. Awesome. Venkat's incredible. He's one of the best.

Venkat, if you're listening, we're huge fans. I'm a huge fan.

He's going to love this.

No, he's great, man. He's a legend.

Anyways, I was talking to him about pricing as well. His take is that pricing is kind of like a one-way ratchet. Basically, once you get a taste of usage-based pricing, you're never going to go back to per-deployment pricing. I think that's definitely true, and I think it's just because it gets closer and closer to your true utility. You're getting all this value. The main pain point is that you have to maintain all the infrastructure to get it to work well. But if you do have it, he thinks it's a one-way ratchet where there's just no going back.

And I think the hot new thing now is that, with AI, you can kind of measure outcomes. So that's another step forward. If that works, maybe it's a one-way ratchet.

We thought about that. Is there some type of outcome-based pricing? This is more on the first-party side than on an API. It's kind of hard to measure—very hard.

I mean, that's hard because you end up having to price and value non-computer-science infrastructure, right? You're literally going into verticalization.

I mean, listen, if it's porting a codebase, maybe you'd have some expertise, but if it's whatever—increasing crop yield—at some level, you need to—

But there could be a world where the AI is good enough that it can actually make judgments about these things and do it accurately enough that we can tie it to billing. I think this is a problem with AI conversations because, at any point in time, you're like, “But it could get good enough.”

It's not a problem anymore.

Yeah. At some point it'll be solved.

It's so much like prompt engineering and the single-agent thing from before. Yeah, yeah. It's like when you reach that level—when you push it that far—everything's kind of solved on outcome-based pricing.

It sounds very appealing. If it can work, it can work. But one thing that we've started realizing is—

It actually ends up correlating quite a bit with usage-based pricing, especially with test-time compute. If the thing is just thinking quite a bit, actually, if you charge just by usage-based pricing and not outcome-based pricing, you're basically approximating outcome-based pricing at this point. If the thing is thinking for so long, it's highly correlated with what it's doing—

Just adding more value.

Yeah, exactly.

And so maybe, at the end of the day, usage-based pricing is all you need, and we're just going to live in this world forever. But it's constantly evolving. I think our thinking has evolved here as well. I'm personally keeping track of whether outcome-based pricing setups can actually work here, but at least on the API side, it's such a usage-based setup. We have to get infrastructure around this, so I think we'll probably stay with that for a while.

So how do you think about open source? I think you're the only big lab that's releasing open source.

No, Google has some of theirs, mostly smaller models on their side.

That's right. So how do you think about open source vis-à-vis competition and cannibalization? What's the strategic complexity?

I personally love open source. I think it's great that there's a—

All of us grew up with it, right?

Yeah, we grew up with it. The internet wouldn't exist without it. So much of the world is built on top of it.

Cloud wouldn't exist without it. Nothing would exist without it, except maybe Windows.

It was interesting, because I felt like, over the period before we launched the open-source model—I know Sam feels this way as well—there was this weird mindset where, because OpenAI hadn't launched anything, it just seemed like it was super anti-open, like super anti-open source. But I'd actually been having conversations with Sam ever since I joined about open-sourcing. We were trying to think about how we could sequence it. Compute is always a hard thing: Do we have the compute to train this thing? So we've always wanted to do this. I'm really glad that we were able to finally do it.

I think it was earlier this year? I lost sense of time.

AI time is so weird.

Yeah. Was it last year? No, it was this year, when gpt-oss came out.

I was just really glad that we did that. The way that I generally think about it is, one, I think—as a—this is also particularly true for OpenAI because, as you said, we are both a vertical and a horizontal company—we want to continue investing in the ecosystem. From a brand perspective, I think it's good, but then also—

I think from OpenAI's perspective—

If the AI ecosystem grows more and more, it's a rising tide, and this is all really helpful for us. If we can launch an open-source model and it helps unlock a whole bunch of other use cases in other industries, I think that's actually not good for us.

I'll say what people don't talk about a lot is how well these open-source AI business models actually work, because the cannibalization risk is actually very low.

Yeah.

You don't really enable competitors a lot because, when we say open source, you really mean open weights, right? It's not like they could recreate it, right? If I can distill your API as well as I can distill you giving me the weights in some way, it doesn't really change that dynamic a lot.

To be clear, we have not seen cannibalization at all from models, and it seems like a very different set of use cases. The customers tend to be slightly different. The use cases are very different. And, by the way, it turns out inference is super hard. To actually have scalable, fast, performant inference, that's a hard, hard problem.

Yeah.

I'd say the way that I personally think about open source in relation to the API business in particular is, one, it hasn't shown cannibalization risks, so I'm not particularly worried about that. But also, especially for all these major labs, there are usually 2 or 3 models where you're making all of your impact and all of your revenue. Those are the ones where we're throwing a bunch of resources into improving the model, and these tend to be the larger ones that are extremely hard to run inference on. We have a really cracked inference team at OpenAI, and my sense is that even if we just literally open-sourced GPT-5 or something, it would be really, really hard to run inference on it at the level that we were able to.

There's also, by the way, a feedback loop between the inference team and the training team, too. So we can kind of optimize.

Can you—can you, like, is it possible to verticalize models for products? Have you trained models specifically for products?

Yeah. I think we've kind of done this with GPT-5 Codex, right? Or do you mean even more verticalization?

Like deep, deep, deep verticalization, where the released model wouldn't—you know, it's actually part of a product.

I think we're basically starting to move in that direction. I think there's a question of how deeply you verticalize it. Most of what we've done is mostly at the post-training, tool-use level. Codex is particularly good at using the—sorry—GPT-5 Codex is particularly good at using the Codex harness. But there's even deeper verticalization you can do than that, and that one, I think, is more of an open question.

Yeah. A lot of my mental model comes from the pixel space, which is—you can LoRA a bunch of image models, right? And you can do a bunch of stuff to make them better and more suitable for some products. You would believe that you could verticalize a model for editing, or cut and paste, or this or that—that's actually part of a product. But you actually don't see that. Did that happen?

Yeah, it's almost always that you're just exposing a model, not something specific to a product.

Yeah, I think so. I think there's a distinction to be made between the image-model space and the text-model space, also, because the image models tend to be way smaller and you can iterate on them a lot faster. That's why you get that crazy, cool proliferation on the image-model side, whereas for the text models, there's always going to be this really big, fat pre-training step that you have to invest in here.

Even the post-training side isn't the easiest thing. From a compute perspective, obviously it's much smaller, but it's still pretty heavy to do a full mid-training or post-training run.

I've got to say, it is a bit of an antipattern to do both language-based models and diffusion, pixel models, in the same company. Most that have tried it have found it very clunky to do, but you and Google are the 2 counterexamples. So is it possible to even converge the infrastructure on these things? I mean, is it totally different orgs? Is it shared infrastructure? How do you operationalize it?

Yeah, I think you're totally right. It's an antipattern. It's pretty tough to pull off. Props to Mark on our research team for structuring things in a way we're able to do it.

From my perspective, the biggest thing is that our image—or, I think we call it the world-simulation team, or the team that builds Sora and all that, under Aditya—is just extremely solid. It's probably the highest concentration of talent that I've seen in a while.

But is it the same infrastructure? Do they use the same infra?

Yeah, so it's actually pretty separate, and I think that's part of the reason why we're able to do this. One, the team needs to be extremely strong, which they are. Two, they're run very separately. They're thinking about their own particular roadmap, and they think about productization very separately as well, right? That's how the Sora app came out of that as well.

Even the inference stacks are slightly different—they're kind of different. They own a lot more around their inference stack, and they optimize their inference stack pretty separately. I think that contributes to helping us run things in parallel, but it's pretty hard to pull off, for sure.

Maybe you can educate me on this. I think about APIs from OpenAI as mostly text-based. Do you do actual pixel-based stuff?

Yeah, we do. We have a bunch. DALL-E is in the API—the OG model. DALL-E 2 is in the API.

That was the first real text-to-image model, right?

Yeah. That was actually the model that got me to go to OpenAI, because it was that summer when I was looking for—I was thinking about something new. It was when DALL-E 2 came out, and it completely blew my mind. I distinctly remember I was asking it to do the simplest thing, like draw a picture of a duck or something. It was the simplest thing, and it generated a picture of a white duck. That was actually the thing that got me to OpenAI in the first place.

But yeah, we have a bunch in our API. The image-generation model is in our API as well, and then Sora 2 is in our API. We launched it at Dev Day. It's actually been a huge hit. I've been very, very surprised.

Wow.

From the API standpoint, a lot of the infrastructure is shared for those, but once you reach the inference level, they're separate, right? You have to inference them differently. That team has been really laser-focused on making that side particularly efficient and work well separately from the text models. We have image generation, we have video generation, and we'll continue adding more to the API there.

It feels like we've been evolving our thinking as an industry on a bunch of stuff, right? One of them, for sure, is the models, like we've talked about. The other one is context engineering. It seems to me that how you build agents and expose them has evolved, too. So maybe you can talk a bit about that.

Yeah. At Dev Day this year, when we launched Agent Builder, I got a bunch of questions around this because Agent Builder was a bunch of different nodes and was deterministic. People asked, “Is this really the future of agents?” We obviously put a lot of thought into this when we were thinking about building that product. The way I think about it is that there are 2 things at play here.

Do you think those questions came from a point of being constrained, by the way? They're like, “Oh, this is too constraining.”

Yeah, I think people are like, “It's too constraining. It's not AGI-forward. At the end of the day, AGI will do everything.” [laughter] So why have nodes in this node-builder thing?

Just tell it what to do.

Yeah. And so I think there's a practicality component. The other thing is that there are actually different types of work out there that could be automated into agents.

On the practicality side, the models today just—maybe in some future world, instruction-following would be so good that you could ask it to do this 4-step process and it would always do the 4-step process exactly. We're still not there yet. In the meantime, this entire industry is being born, and a lot of people still want to use these models. What can you build for them? There's a practicality component to it.

When did you launch that?

Dev Day. It feels like forever ago—it was earlier this month, October 6th or something. So, less than a month ago, actually. It's been crazy seeing the reception to it. The video where Christina on my team demos Agent Builder is one of the most-viewed videos on our YouTube channel now.

I will say, just anecdotally from my perspective, people love it, but I also saw the dissonance. When it came out, people were like, “Wait, what is this? Is this no-code, low-code?” [laughter]

Yeah, exactly. It's another low-code thing.

I know people love it.

There's a practicality piece. There's another piece, which is that when we were talking to our customers, we realized that, at the end of the day, a lot of this agent work is just trying to automate the work that people do in their day-to-day jobs.

There are actually 2 different types of work. There's the work that we think about, which is maybe what software engineers do, which is very undirected. There's a high-level goal, and then you have your cursor and you're just writing code, exploring things, and going toward an objective. That's more knowledge-based work, like data analysis or coding.

But then there's another type of work, which is actually what we realize is maybe even more prevalent in industry than software. We're just not aware of it. Work tends to be very procedural, very SOP-oriented. Customer support is a good example of this. There's a very clear policy that these agents and people have to follow, and it's actually not great for them to deviate from it and try something else. The team running these teams really wants the SOPs to be followed.

And this pattern actually generalizes to different types of work.

A standard operating procedure. Yeah, sorry. So it's the way in which you need to operate the support team. But this extends to marketing, sales, and a bunch of other areas—way more than it has any right to. What we realized on that side is that having determinism here, with an Agent Builder with nodes that helps enforce this, ends up being very helpful. I think a lot of us, especially in Silicon Valley, don't really appreciate that there's a ton of work that falls into this camp.

I have to say, there's a pattern that's similar to this. I'm wondering if you've seen it. I've seen it where some regulated industries actually can't let any generated content go to a user.

And so what they do is—and I think it's so interesting—they'll either pass in a conversation tree, and you can choose something from here.

Yeah, so there's some human element to it.

As part of the prompt, they're like, "Here are the viable things you can say. Choose which one to say." So the language reasoning is happening by the model, but nothing generated comes out.

Interesting.

Does that make sense?

Yeah.

Another one I've seen is actual pseudocode. I'll pass in a Python function.

And then it'll ask a human to use the pseudocode to write actual code that makes it in.

It actually has a response catalog as part of it, and it has the logic to apply. Interesting.

And so the model takes the language in from the human user.

And then the logic of how to respond is in Python code, because it just turns out that there's been a lot of code written for these types of things. Then it includes the responses that you would send out. Does that make sense? Actually, a lot of NPCs are done this way—actual video game NPCs. The way that I think about it is—

So that way, with the NPCs, the actual code being generated by the model is not what ends up making it to the end user.

That's it: the code is not being generated by the model. The prompt has the code. So let's say that I have an NPC.

Let's say you're the gamer. You're coming in and talking to my NPC, but my NPC has some logic that it needs to do: if you say a certain thing, I'll give you a key, or maybe I'll barter. Describing the game logic in English just doesn't work if you try to do it. Actually scripting the output doesn't work either if you need to use it in a game context; you would have to give a specific direction or a specific this or that. So how do you make these things behave in a more constrained way?

People will pass in functions. They'll actually describe the logic in Python. My prompt will be, "You're an NPC in a video game. The user just asked you a question. Here's the logic you should go through: If the user says this, then do this." It's pseudocode: if the user has this in their belt, do this—whatever, whatever, whatever. Then here are the set of valid responses. So you're almost constraining it.

Oh, I see. And then when it actually does give a response, you can validate that it's one of those responses.

I see. So it's highly structured. The NPC still only exists in the space it can act in, which is only within the space of the program that you gave.

Yeah. Well, the logic is in there. So it can have a normal conversation, but only insofar as you're trying to guide the logic for game design or game logic. You see this with NPCs, but you also see this with regulated industries.

Yeah, I was going to say, what you described sounds like giving the SOPs to a set of human operators and telling them to stick to it.

Yeah. You must say these 3 things.

You cannot give a refund if it's less than this amount.

Very interesting. I don't want to equate them to NPCs, but this is very similar. I'm just saying that if you want to really guarantee what happens, there's a set of techniques that you use.

There are some situations where you want to constrain what they do. It could be from a regulatory standpoint. It could be because you want it to run for a long time. It could also be because you actually have game logic, and your game logic is a traditional program. You have a monetary system, an item system, and a battle system. You can't describe that in English; you have to give it to them so it can behave within that.

Yes. And that is exactly the problem I think we're trying to solve here, right? If you do not give it any of this, it can just go off and do whatever. There are regulatory concerns around this, and that is the exact use case that I think we're trying to target with Agent Builder.

That's awesome. Well, listen, we're running out of time, and there's a million more things I want to ask you. I really appreciate you taking the time to come in. It was great surveying what's going on, and particularly teasing apart horizontal versus vertical in the space, which I really wanted to do. Thank you so much.

Yeah, thank you.