No Priors 第112期｜Isa Fulford 谈 OpenAI Deep Research

Hi, listeners, and welcome back to No Priors. Today I'm joined by Isa Fulford, one of the pioneering minds behind OpenAI's Deep Research. This is a new agentic product that OpenAI released in February of this year, which uses reasoning and tools like web browsing to complete multi-step research tasks for you. Today, they're making it free to all U.S. users.

Welcome, Isa. Thank you for doing this.

Thank you so much for having me.

You and your team have shipped one of the most exciting AI products lately. I use Deep Research a lot. Where did the idea come from? Tell me the origin story.

Around a year ago, we were very excited about the progress internally on this new reinforcement learning algorithm. We were seeing a lot of progress on math problems, science problems, and coding problems. At the same time, I was working with my friend Yash, who works at OpenAI, on a few side projects. We were very interested in agents generally and wondered if we could apply the same algorithms to tasks that were more in line with what the average user would do every day.

The first 2 things we were thinking about were online browsing tasks, because I think in a lot of different professions, people have to do a lot of research, synthesize a lot of information, and then come back with a report. We were also thinking about software engineering. We've been working on those things, and I've been focusing on browsing.

To start, with the math and coding problems that people were already training on, those data sets already exist. You can have a math problem with a ground-truth answer and train on those. But for browsing, it's more open-ended. You don't really have data sets like that.

We really started by grounding the research in the product use cases we actually wanted the final model to be good at. We would literally write out a list of things, like, “I hope the model could find this list of products for me and rank them by these reviews from Reddit,” or, “I wanted to be able to write a literature review on this topic.”

I feel like a lot of people, when they think about browsing and agents, land on the same 2 or 3 transactional use cases that I don't think are particularly inspiring. It tends to be, “Order a burger on DoorDash,” or something like that. I think ordering flowers is also a really common one. Why do you think you came up with such a different set of goals for the agent?

I think, before focusing on taking the right actions—which those are examples of—we wanted to get really good at synthesizing information from a large number of sources, mostly through read-only tasks. That was for a number of reasons. First, a huge number of knowledge-work professions mostly do that, so it would be quite useful for those groups of people.

Secondly, I think the overall goal for OpenAI is to create an AGI that can make new scientific discoveries. We felt that a prerequisite to that is being able to synthesize information. If you can't write a literature review, you're not going to be able to write a new scientific paper. It felt very much in line with the company's broader goals.

More practically, the safety question for a read-only task is a bit more constrained, so it was a good thing to start with as well.

It's also very meta, because you have helped make an AI that makes me better at learning, and it's learning.

Yeah. I hadn't thought of that. I love that.

It seems that, in the read-only space, people were also not nearly as ambitious as you—or you and Yash—were in thinking about what it might be able to understand for me.

You thought of these end-to-end evals and came up with a set of tasks that could be autogradable or fit a set of characteristics that made them a better fit for the algorithms. And then what?

That was actually a huge process in itself. Initially, we'd built a demo to pitch people on this idea, and it involved no model training. It was fully just prompted models with the UI, pitching the vision of what this product could look like.

After that, we were at the point where we had to start thinking about how we were going to do this, how we were going to create the data, how we were going to train the model, and what tools we needed to create to enable the model to browse the internet effectively. That was a lot of iteration.

I was working very closely with Edward Sun and a few other people on this, and we also collaborated a lot with the RL team. It was definitely a big undertaking. A good thing about it was that we were able to work uninterrupted for quite a few months, making the numbers on our evals go up.

It was nice not to have too much pressure to ship something really quickly. We were just able to iterate and get it to a good state.

Did you have a favorite, or most important, task?

We had a few tasks. People would just propose different tasks. One of them was to find all of the papers that Liam Fedus and Barrett Zoph had written together. I think there were 11. The model now can find most or all of them. We would always ask that question.

Another one, which the model actually can't answer anymore—probably for good reason—was finding the middle name of one of our coworkers.

Personally, I think I started using it pretty early on for finding information for product recommendations and travel. Quite a few people internally had found a Streamlit playground that people would just use. A lot of people had found it and were using it.

Sam told me he used it to buy a bunch of things. Every time it would go down, people would message us: “What happened? We need to use the model,” even when a previous version honestly wasn't that good. I think that was a good initial sign.

What can you say about the actual bulk of the work—the tool creation and the data creation?

For the data, we did a bunch of different things. We used human trainers for some of it. We had to come up with new ways and new kinds of data sets, I guess, and figure out how to design data sets to exercise the kinds of skills we wanted the model to learn.

Then you have to make a way to grade those data sets as you're training them. You also have to make good tools for the model to actually complete the task successfully.

Right now, we just have the browsing tool, which is a text-based browser, but it can see embedded images and open PDFs. It also has access to a Python tool, so it can do analysis and calculations and plot graphs.

You can imagine that, in future versions, we'll expand the tool set, and the model will become more capable. But we'll also need to make data sets that actually make the model exercise all of those different tools and figure out how to use them, backtrack, and do all these different things during training.

That way, it's actually able to flexibly answer new problems from users in the product.

It's clear that reinforcement fine-tuning on very powerful base models can do very useful things. That's super exciting. What advice would you have for startups or other companies that are thinking about doing RFT for a particular task, in terms of when it's worth doing or when they can just try traditional orchestration, where agents are a component?

In general, you will always get a model that's better at a specific task if you train on that task. But we also see a lot of generalization from training on one kind of task to other domains.

You can train a reasoning model on mostly math, coding, and other reasoning problems, and it will be good at writing. But if you trained it on that specific task, it would be better at it.

If you have a very specific task that you think is so different from anything the model was likely trained on, and you've tried it a bunch of times yourself, tried a lot of different prompts, and it's just really not good at it, then maybe it's some genetic-sequencing task or something that's so out of distribution for the model that it doesn't know how to figure it out. I think that's a good time to try reinforcement fine-tuning.

Or, if you have a task that's so critical to your business workflow that getting an extra 10–15% performance is really make-or-break, then you should probably try it.

But if it's something where you think, “The model's pretty good at this, but it gets things wrong some percentage of the time,” and then you see that with every next model that's released, it gets a little bit better, it might not be worth the effort if the model is naturally just going to get better at those things. That would be my recommendation.

Great advice. You've talked about needing to use human experts to create some of this data. I think of browsing as a somewhat universal task. I guess there are better and worse browsers. Where do you feel like you need expertise, or what do you know about browsing expertise that you didn't before—or information-gathering expertise?

I guess it's one of those things where basically every single profession involves having a question or wanting to do research in a domain, and then having to find information from many different sources to synthesize an answer.

While doing that, you have to have the expertise to reason about: Is this a useful source? Is this not? Should I include this? Is this completely off-topic? Whatever.

That is kind of universal to most jobs or most scientific domains. The cool thing with RL is that you don't necessarily need to know the whole process of how the person would do the research. You just have to know what the task is and what the outcome should be, and the model will learn during training how to get from the problem to a good answer.

So I think we just took a pretty broad approach. That's one thing that, if you work at a place like OpenAI, you can do—something they would tell most startups not to do—and just try to focus on a really broad set of users, get experts in loads of different domains, and try to get good at everything at once, which was the approach that we took. Then we also created a lot of synthetic data sets and things like that, but the human data was definitely a really key part for making this model successful.

Did any of the learned planning from the model across these domains surprise you, in terms of the path to find the perfect handbag or the restaurant in Japan or the set of papers that was relevant?

Sometimes it will use search terms that I wouldn't necessarily have used. We didn't teach it to plan up front, but sometimes we'll see that it does end up making a plan up front before starting its research. Sometimes the model will do smart things and try to get around restrictions you put on it.

You have to make sure that it's not hacking and trying to use a different search engine other than the search engine that you gave it, or something like that. It will do smart things that you have to make sure you're looking out for, in case you don't want to allow the model to do those things.

Maybe we can actually use this as a moment to talk about some of the failure modes. How do you think about some of the classic issues with agents, like compounding error or distraction or even safety?

With deep research, it can't actually take the kinds of actions involved in the typical agent safety problems you would think of. But I think the fact that the responses are much more comprehensive and take longer means that people will trust them more. So I think maybe hallucinations are a bigger problem.

While this model hallucinates less than any model that we've ever released, it's still possible for it to hallucinate sometimes because it will infer something incorrectly from one of its sources. That's part of the reason we have citations, because it's very important that the user is able to check where the information came from and, if it's not correct, can hopefully figure it out. That's definitely one of the biggest model limitations and something that we're always actively working on to improve.

In terms of future agents, I think the ideal agent will be able to do research and take actions on your behalf. That's a much harder question that we need to address. It's kind of at that point when capabilities and safety converge, where an agent is not useful if you can't trust it to do a task in a way that doesn't have unintended side effects that you don't want.

If you ask it to do a task for you and then, in the process, it sends an embarrassing email or something like this, that's not a successful completion of the task. I think that is going to be a much more interesting and difficult safety area that we're starting to tackle.

You can tell me if you just don't have a projection here, but do you think people are going to want explicit guardrails? Do you think you can learn a bunch of those characteristics in the model itself? If you've used Operator, I'm sure you have to confirm every write action.

I think to start with, that makes a lot of sense. You want to build trust with users, and as the models become more capable, maybe you've seen them successfully do things a few times and you start to trust them more. So maybe you allow them to do that every time: You don't have to ask me every time you send an email to these people; that's fine.

I do think that as these agents start to roll out, we will definitely want to have guardrails and confirmation. While they're not the end-state capability, we still want to make sure we have a good level of oversight. But I think that they will get so good that we'll just trust them to do things on our behalf.

What are some of the obvious ways you feel like deep research as a product is going to get better?

It's going to extend into actions, right? You just implied that at some point. I think maybe the ideal state would be to have a unified agent that can do all of these different things. Anything that you would delegate to a coworker, it should be able to do.

How are we going to make decisions about whether it's, like, “Sarah, you do this” versus “Agent, please do this”?

I guess. Or is it always just, “Try the agent first”? Probably. I would try the agent first if it was my work. It's kind of the pattern that every time the model becomes more capable, the level of abstraction of the human becomes higher, if that makes sense.

The task you're asking it to do is just higher and higher level, but you're still initiating the task. Maybe a year ago I was asking it to write a function for me, and now I'm asking it to write a whole file. Maybe next year it will make a whole PR for me or something like that. I still think we'll be in the driving seat.

As to deep research, I think obvious next steps for deep research would also be to have access to private data—to be able to do research over any internal documentation, GitHub, whatever it is.

There's a golden thread here, because when we first met, you were working on retrieval, and I was like, there cannot be only one person at this company working on retrieval. Everything—all roads lead back to retrieval.

Mhm. So I think that will be really cool, and then eventually taking the right actions or calling APIs. Obviously, there are just a lot of things that the model is not perfect at now that we need to improve.

I think we have a really cool working relationship with the reinforcement learning team. A lot of teams will contribute data sets to the big runs that they do. We contribute data sets, and then as they train models with a ton of compute, it just becomes a better base model for us to continue training from. I just think the capabilities are compounding.

This was not a low-key research preview, but a side project that turned into a very interesting, internally pitched project. How do you think about what is a product that OpenAI—or at least you yourself—want to work on independently versus what belongs in the core research path?

A cool thing about OpenAI is that even though the company is bigger, I think the culture of anyone being able to have an idea, prove it out, and then push it to completion has still been maintained as the company has grown.

For me personally, I'm always motivated to work on things that I will use myself. With deep research, for example, I do use it a lot for looking up various things and travel recommendations. I think I'm probably a daily active user.

It's fun when you get to dogfood now.

Oh, amazing.

Yeah, I'm burning a lot of GPUs.

Are there use cases where you're the domain expert? Are there ways that you or Yash have used it, or that you've seen the user base use it, that you encourage people to use deep research for?

I'm always interested to see people using it in domains where I have absolutely no expertise. For example, in medical research, I've seen a lot of different scientists posting about how they've used deep research and how it helped them do something.

To me, that's the most interesting, because when we were working on it, I obviously had no way of judging whether an output was good or not. Seeing experts actually ratify deep research responses is useful.

An area where I was surprised to see people using the model was code search and coding questions. For example, using the latest package or latest version of whatever repo to help write a file. Data analysis is also something the model is already pretty good at, and I think we'll just continue to get better at.

Uploading a file and having it do some analysis for you, or do some research and then create a report with numerical analysis, is pretty interesting.

I actually haven't tried this. It's not a browsing task. What makes the model particularly good at this, or what is it capable of? Is it really multistep, with the ability to do planning and understand the task and produce a report that's cohesive?

I think the base model, or the model that we started fine-tuning from, o3, is just a very capable model. It's trained on many different data sets, including a lot of coding, reasoning, and math tasks. That inherited capability is pretty strong, and then when you add browsing on top of that, it's still able to do that analysis. Those 2 things together can be quite powerful.

Before the podcast, we were just talking about the idea of learning taste or preferences from users. OpenAI just released a bunch of memory features. How do you think that deep research—or agents in general—could evolve to take into account how people want to learn or their information-ingestion preferences?

Agent memory will definitely be very important. It would be very annoying if every time you ask it to do a task, you have to repeat the same information, how you want it to do the task, everything about you—which currently, for deep research, you do have to do.

And I think as the tasks get more complex—and right now, they take 5 to 30 minutes—you can imagine that in the future, it might take hours or days to complete a task that you ask the model to do. You definitely want the model's research to be compounding. You don't want it to have to start fresh every time. So I don't necessarily have a good answer, but I think it's something that will be very important.

There is a common understanding among many people at some of the leading labs that the recipe to AGI is, I'd say, somewhat known, or there's confidence in this, and the return of RL is very exciting for everyone. The stance that I've heard from you and others is one of enthusiasm: This seems to work, we're going to get real capability out of it, and it's quite data-efficient. It's also going to be a lot of work. Tell me a little bit about the emotional experience of building Deep Research and whether that changes your view at all.

I agree with everything you said. I think it's so impressive to see how data-efficient the algorithm is. I guess the data you train on is much higher quality and smaller, so actually curating that is an undertaking. Then there's making sure that the model has access to all the tools that a human would have access to in order to do the work they need to do.

Making sure that you represent tasks that people will find useful or do in their jobs in a way that you can judge whether the model did a good job or not is also hard. There are so many other challenges for pre-training, where you have so much more data. You have to do all of these different things, which is, I think, just a different challenge. Both are compounding: You need a really good base model to be able to do RL, and then, for our team, we just do more RL. It's all very compounding, but I think everybody does see a pretty clear path to this broadly capable agent.

Do you think there are big blockers to progress? Maybe not exactly describing it as the next iteration of Deep Research, but just confidence that we're going to have these unified agent capabilities and that it will feel like a coworker. What stands between us and that?

There are a lot of really hard safety questions that we need to figure out. We would never ship anything that we don't have very high confidence is safe. The stakes are way higher when it has access to your GitHub repositories, your passwords, and your private data. So I think that's a really big challenge.

I guess also, if you want the model to be able to do tasks that take many, many hours, you need to find efficient ways to manage context. It's kind of similar to the memory thing, but if you're doing a task for a really long time, you're going to run out of context. So what's an efficient way of dealing with that, allowing the model to continue to do its thing? Then there's the task of making the data and making the tools. I've said this already a few times, but that's a lot of work.

I was just looking at my history of queries. My user request is, "I want to see what things I asked of Deep Research versus other models, in particular, in my memory."

It has ranged from, obviously, if I'm trying to get up to speed on a market for a company I'm looking at, or on a technical topic, to travel planning, which is a big one. I've also looked for things that are taste-related. I'll say, "I like this set of books for these reasons. I want you to give me a long-form summary of a bunch of other things you think I should read and explain why."

I realize I don't have a super-clear mental model of when Deep Research should be better than o3. What instinct can you give me here?

Deep Research is very good when you have a very specific or well-defined query. Maybe not a general overview of a topic, but if you're looking for some specific information and you think it would be supplemented by existing research online. Even if we also train the base model on that information, having live access to it is quite useful.

So if I have any instinct about directing it to retrieval or particular sources, that focus is useful, I think. We also trained it to have much longer outputs than normal models would. If you're looking for something very comprehensive—maybe sometimes too comprehensive for some tasks—I think Deep Research will be useful for those things.

Connect this for me to a Deep Research fashion task. I've used it to find new brands. I'll say, "These are the kinds of brands I like. Please find new brands where I can find this specific coat that looks like this one," or something like that.

It's very good at finding those, whereas I think the base model or the normal model will give you some brands, but it won't necessarily fit all of the constraints that I had given. I want it to sell this fake-fur coat that's this length, this season, or something like that. It's not going to be able to do that because it just won't have the up-to-date information, and it also won't necessarily be able to deal with all of the constraints in a query in one shot.

o1 isn't browsing as comprehensively. I'll use it to find things when I'm looking for a very specific item that would take me hours to find. I'm looking for a very specific item or sweater that's probably available on The RealReal or somewhere, but I can't find it, or I'm looking for an Airbnb with very specific constraints. So I think those kinds of things are good for Deep Research, and for more general, high-level things, you should use normal search.

Yes.

Well, I will admit I have had some multi-hour browsing and shopping tasks that I'm now making a cron job for Deep Research.

I suppose I want to ask just one more experience question, which is: Was there a particular win or failure that surprised you in the training of Deep Research?

It really was one of those things where we thought that training on browsing tasks would work. It felt like we had good conviction in it, but actually, seeing a model work the first time you train it on a new data set using this algorithm, and playing with the model, was pretty incredible, even though we thought it would work. Honestly, just that it worked so well was pretty surprising, even though we thought it would, if that makes sense.

It's the visceral experience of, "Oh, the path is paved with strawberries," or whatever.

Exactly. But sometimes, some of the things it fails at are also surprising. Sometimes it will do such smart things and then make a mistake where I'm just thinking, "Why are you doing that? Stop." So there's definitely a lot of room for improvement, but we've been impressed with the model so far.

I'm used to all my technology tools being instantaneous. Deep Research is not instantaneous; it's thinking and using tools. Can it be faster?

I do think there's a good middle ground in between, where sometimes you don't want it to do really deep research, but you want it to do more than a search. I think we'll release things soon that people will be happy about and that will fill that gap.

I don't know how to communicate this preference, but I want to toggle at some point and say—I would say this to a human—"I want you to do as good a job as you possibly can in the next 5 minutes."

That's something where I think it seems like bad UX to make the user make that decision. The model should just be better at knowing how much time to think. I think we made a decision when training the model that we were just going to go for maximum thinking time every time. I'm sure I've asked it a really simple query sometimes just to test it, and then gotten quite frustrated that it's still thinking.

I do think that's also an area for improvement: knowing how long to think for. I suspect that with Deep Research, we'll always be focusing on the tasks that take the maximum length of time, and then I think o3 or o-next will have a better in-between.

What is an example of a task you can imagine Deep Research taking a day to do in the future?

I mean, there's some GPU scaling. Right now, in 5 to 30 minutes, it can do what human experts rate as taking many hours. So I guess in an hour, it could do something that would take a human days. In a day, it could do something that would take a human weeks.

Obviously, there will be a lot of challenges to get it to scale like that, but I think you can imagine it doing a research project that would have taken weeks to complete, or writing a thesis or something like that.

I'm going to make our intern compete with it over the next couple of months, then.

Yeah, sounds good.

If you were to project forward a year, which is a really long time in AI land, what is something that you think will surprise people that agents can do and that will actually be released?

Taking the safety considerations into account, I think a general agent could help you do a lot of the tasks that you would do in a lot of different areas. For me, I do a lot of coding. I'm hoping that there will be an agent that's pretty proficient at coding that I will just trust. I'll give it a task, and it will hopefully make a PR or something. But maybe I can ask the same agent to help me book a trip to Korea or something.

I hope that we'll get to a more unified experience, but I also think that the rate at which these models are improving is going to be pretty surprising to most people.

Why do you think a unified experience is important, or why do you think that makes sense?

I think today it's quite different to think about. Obviously, ChatGPT is one experience that's very encompassing, but there are models that people use in different contexts, like next-line-completion-type models for coding, that just feel like a very different setting. I think that you'll probably want both. You'll probably want an experience where you can, at some point, override or interrupt the model and say, “Oh, no. I didn't mean that,” or take over and start typing something.

Especially in the short term, as the models are not as capable as humans in a lot of areas and are more capable in other areas, I think it will be a combination of you asking the model to do something, but then, maybe to go with the coding example, you're also in your VS Code or whatever it is—your Cursor—and it's been doing something for you, but you can also actually type and write some of it yourself. So I think it will be a combination of those things, but I kind of want it to be something that is just like—it's like having a coworker on Slack or a remote coworker you can just ask to do things for you, send them a Slack message, and then they'll start doing it.

Then you can review their work or help at some point. It seems like a pretty nice general interface, and you don't have to think about which agent you should ask to do which task. It should just be able to figure it out.

The mental model I have for this is that my general ethos is actually that I love the people I work with. I prefer to work with fewer people with less management overhead, all things considered, because each person has more context and I have more understanding of them. So the universally useful agent is attractive for that reason. You only have to tell it something once, and it will remember, and then it will have state on everything you're working on—things like that.

Awesome. Well, this has been a great conversation, Isa. Thanks for doing this, and thank you for the product release.

Thank you so much for having me, and thank you for using Deep Research.