Agent 推理界面：Claude、ChatGPT Canvas、Tasks、Operator——与 OpenAI 的 Karina Nguyen 对谈

I’m Alessio, partner and CTO at Decagon, and I’m joined by my usual co-host, swyx.

Hey.

Today we’re very blessed to have Karina Nguyen in the studio. Welcome.

Nice to meet you. We finally made it happen.

We finally made it happen. The first time we tried this, you were working at a different company, and now we’re here. Fortunately, you had some time, so thank you.

Thank you for inviting me.

Karina, your website says you lead a research team at OpenAI creating new interaction paradigms for reasoning interfaces and capabilities like ChatGPT Canvas and, most recently, ChatGPT Tasks. I don’t know—is that what we’re calling it? Streaming chain-of-thought for o1 models, and more, via novel synthetic model training. What is this research team?

I need to clarify this a little more. I think it has changed a lot since the last time we launched. We launched Canvas, and it was the first project that I attacked, basically. Over time, I was trying to refine what my team is, and I feel like it’s at the intersection of human-computer interaction—defining what the next interaction paradigms might look like—with some of the most recent reasoning models, as well as trying to come up with novel methods for improving those models for certain tasks.

For Canvas, for example, one of the most common use cases is writing and coding. We’re continually working on how to make Canvas coding go beyond what is possible right now, and that requires us to do our own training and come up with new methods of synthetic-data generation. The way I’m thinking about it is that my team is going from the very full stack—from training models all the way up to deployment—and making sure that we create novel product features that are coherent with what ChatGPT can become.

There are different types of Canvas features, but all those components go together to evolve ChatGPT into something completely new.

I like your tweet about that. It’s kind of modular: you can compose it with the Stocks feature, the creative-writing feature—I forgot what else we have. There’s a list of other use cases, but we don’t have to go into that yet.

Can we go back to when you first started working with LLMs? I know you had some early UX prototypes with GPT-3 as well. How did that inform the way you build products?

My background was mostly working on computer-vision applications for investigative journalism when I was at school at Berkeley. I worked a lot with the Human Rights Center and with investigative journalists from various media organizations. That’s how I learned more about AI, including vision transformers.

They have some Pulitzer Prize-winning professors at Berkeley, right?

No, it was mostly reporting for teams like The New York Times and the Associated Press. It was all in the context of the Human Rights Center.

That was computer vision. Then I saw Chris Olah’s work around interpretability at Google, and that’s how I found out about Anthropic. At that time, it was the year when the war in Ukraine happened, and I was trying to find a full-time job. I got distracted—it was springtime, and I was very focused on figuring out what to do.

My best option at that time was to continue my internship at The New York Times and convert to full-time. I was working mostly on product-engineering work around R&D prototypes and storytelling features for the mobile experience. We were thinking about how to employ NLP techniques to scrape some of the archives from The New York Times.

I always wanted to get into AI, and I had known about OpenAI for a while, since I was at Berkeley. I applied to Anthropic through the website and was rejected the first time. They weren’t hiring for product engineering or front-end engineering, which was what I was interested in at the time. Then there was a new opening for a front-end engineer, so I applied, and that’s how my journey began.

The earlier prototypes were mostly things like using CLIP for fashion-recommendation search. That was one of those projects. Before joining Anthropic, I was thinking that maybe I should start my own company, but I didn’t have enough confidence and conviction in myself that I could do that. Twitter is a good platform for trying out projects and helping me make something visual.

The Ukrainian crisis hit home for you more than it did for most people because you’re from Ukraine, and you moved here for school.

Yeah.

We’ll come back to that if it comes up. Then you joined Anthropic—not just as a front-end engineer. You were the first designer. Is that true?

Yes. I did both product design and front-end engineering together. At that time, it was pre-ChatGPT—I think it was August 2022—and Anthropic had really decided to do more product-related things. The vision was that we needed to fund research, and building products was the best way to fund safety research, which I found quite admirable.

The first product that Anthropic built was Claude in Slack. It was launched not long after, but it was one of the first products. I still come back to the idea of Claude operating inside an organization’s workplace, inside Slack. There was something magical in there.

I remember we built ideas like “summarize the thread,” but you could imagine having automated ways for Claude to summarize multiple channels every week, customized to what you like or what you want. We built some really cool features, like tag clouds, asking Claude to summarize what happened in a thread, and suggesting new ideas.

We didn’t quite double down because you could imagine Claude having access to files or Google Drive, with connectors and connections inside Slack. The UX was also constraining at that time. We wanted to build certain features, but the Slack interface constrained us, and we didn’t want to be dependent on a platform like Slack.

After ChatGPT came out, I remember that my manager gave me a challenge: could I reproduce a similar interface in 2 weeks? One of the early mistakes I made in engineering was saying yes. Instead, I should have said, “It will take 2 times as long.”

So you actually wrote Claude.ai as your first job?

I wrote the first 50,000 lines of code without any reviews at that time, because there was no one. It was a very small team—6 or 7 people—and we were called the deployment team.

I actually interviewed at Anthropic around that time. I was given Claude and Sheets, and Sheets was my other form factor. I thought, “This needs to be in a table so we can copy and paste it and bang it out,” which was kind of cool.

The other rumor we might as well mention is one that Raza Habib from Humanloop often talks about: there was some version of ChatGPT at Anthropic. You already had the chat interface in Slack. Why not launch a web UI? How did OpenAI beat Anthropic to ChatGPT?

At that time, the ChatGPT model itself had come out well before we decided to launch Claude 2. Claude 1.3 had a lot of hallucinations, so one concern was that the leadership didn’t have the conviction that it was the model they wanted to deploy.

There were a lot of discussions around that time. Claude 1.3 was extremely creative, though, and it was really cool.

You had a tweet recently saying that things like Canvas and Tasks could have happened 2 years ago, but they didn’t. Do you know why? Was it that there were too many researchers at the labs who weren’t focused on UX, or was it simply not a priority?

I come back to that question a lot. I was working on something similar to Canvas for Claude in 2023. It was the same basic idea: a Claude workspace where a human and Claude could have a shared workspace or document—not Projects.

I was on the product-engineering team at that time, and then I switched to the research team. The product-engineering team grew a lot, and they had their own ideas around Artifacts and Projects. They may have looked at my previous explorations, but when I was exploring Claude Documents or Claude Workspace, I don’t think many people were thinking about UX in the same way. Not many researchers understood that.

The inspiration actually came from the Harry Potter Tom Riddle diary. I still have all the sketches. The idea was to have Claude write into the document and communicate back. In the movie, you write a little bit, and then it answers you.

That was only in the context of writing, though. Canvas also serves coding, which is one of its most common use cases.

Yes. Those ideas could have happened 2 years ago. I just don’t think they were a priority at that time. The AI landscape was very nascent, and it was unclear. Even when I talked to some of the designers and product designers at that time, they weren’t thinking about it at all. They didn’t have AI in mind.

There was one exception: my designer friend Jason Yuan, who was thinking about it. Jason is now a computer scientist.

We’ll have him on at some point. I had him speak at my first summit, and you’re speaking at the second one, which will be really fun.

Let’s stay on topic for a bit and then move on to the more recent things. The other big project you were involved with was Claude 3. Tell us the story. What was it like to launch one of the biggest launches of the year?

Claude 3 was Haiku, Sonnet, and Opus all at once. It was the Claude 3 family. I was part of the post-training and fine-tuning team. We only had 10 or 12 people involved, and it was really fun to work together as friends.

I was mostly involved in Claude 3 Haiku on the post-training side, as well as evaluations—developing new evaluations and writing the entire model card. I had a lot of fun.

The way you train the model is very different, obviously, but what I learned is that you end up with maybe 70 models, and every model has its own brain damage, either in terms of personality or performance. Every model is very different.

One of the interesting research questions is how to understand the interactions between data and training. If you train a model on contradictory datasets, how can you make sure there won’t be weird side effects? Sometimes you do get side effects, and the lesson is that you have to iterate very rapidly, debug and detect them, and make interventions to address them.

Some techniques from software engineering are very useful here, including how you manage data.

I really empathize with this. If you put the wrong dataset in, you can essentially screw up the past month of training. The problem for me is the existence of YOLO runs. I can’t square that with what you’re saying. If you’re taking such care with datasets, then every day I should check in, run evaluations, and do all that stuff. But we also know that YOLO runs exist. How do you square that?

It depends on how much compute you have. There are a lot of research questions around how to use the compute you have most effectively. Maybe you can have 2 or 3 runs that are YOLO runs, but if you don’t have that luxury, you need to prioritize ruthlessly and decide which experiments are the most important to run.

That’s basically what research management is: how do you fund efforts, prioritize, take research bets, and make sure that you build conviction in those bets rapidly? If they work out, you double down on them.

You almost have to ablate datasets, too, do it on a side channel, and then merge it in. It’s super interesting.

Tell us more. I have the model card in front of me, and you said constructing this table was painful. Pick a benchmark. What’s an interesting story behind one of them?

GPQA was interesting. I think Anthropic was the first lab to publish GPQA numbers, because it was relatively new after NeurIPS.

One thing I learned is that some evaluations have very high variance. GPQA happened to be a very high-variance evaluation, so one thing we did was run the average of 5 and take the average.

The hardest thing about the model card is that none of the numbers are apples to apples. You need to go back to, for example, the GPT-4 model card and read the appendix to make sure that the settings were the same as the settings you used.

It’s never truly apples to apples. When you market models as products, customers don’t necessarily know that. They just say, “My MMLU is 99. What do you mean?”

Why isn’t there an industry-standard harness? There’s the EleutherAI harness, which it seems none of the model labs use. OpenAI put out SimpleEvals, and nobody uses that either. Why isn’t there one standard way everyone runs these evaluations?

The alternative is to rerun your evaluations on their models, and obviously your numbers will be lower. They’ll be unhappy, so you don’t do that.

That operates on the assumption that the models—the next generation of the model, or the model you produce next—will behave the same way. The way you prompt o1 can be very different from the way you prompt Claude 3.

There’s a lot of prompting required to get the evaluations to run correctly. Sometimes a model will output new lines, and the way you parse it will be incorrect. That happened with Stanford’s HELM benchmark. Claude wasn’t performing well, and that was because the way it was prompted was wrong.

Has that gone away with chat models instead of raw-completion models?

Each model can still be run in a very different way. Sometimes you can ask the model to output in XML tags, but some models aren’t good at XML tags. Do you change the formatting per model, or do you use the same format across all models?

Then there are the metrics themselves. Accuracy is one thing, but maybe you care about some other metric, like an app score. It’s hard.

Speaking of o1 prompting, we recently had an o1 prompting post in the newsletter. It apparently went viral.

I got pinged by people at OpenAI asking whether it was helpful to them. I said, “Okay.”

I think it’s now one of the top 3 most-read posts, and I didn’t write it.

What are your tips on prompting o1 versus Claude? What did you take away from that experience? I know that with GPT-4o for Canvas, you’ve done reinforcement learning on the model. What did you learn generally? Should we think about prompting these models differently now?

I don’t think I’ve even harnessed the magic of o1 prompting. One thing I found is that if you give o1 hard constraints about what you’re looking for, the model has a much easier time selecting candidates and matching the candidate that best fulfills the criteria.

There’s a class of problems where o1 excels. For example, if you have a biology question or a chemistry question with very specific criteria around a protein or chemical bindings, the model can be really good at determining the exact candidate that matches those criteria.

I’ve often thought that we need a new evaluation for this, because this is basically instruction-following, isn’t it?

Yes, but I don’t think IFEval has multistep IFEvals.

That’s what I use AI News for. I have a lot of prompts, steps, and criteria, and o1 checks through each one systematically. We don’t have an evaluation like that.

Does OpenAI know how to prompt o1? Sam Altman is always talking about incremental deployment and getting people used to a model. When you release a model, you obviously do all the safety testing, but do people internally know how to get 100% out of it? Or are you also learning from the outside how to prompt o1 better?

You learn so much from external feedback about how people use o1. A lot of people use o1 for really hardcore coding questions. I don’t fully know how to use o1, except that I use it for synthetic-data explorations.

Once the model comes out, do you get a company-wide memo saying, “This is how you should try to prompt this,” especially for people who weren’t close to it during development? I don’t know if you can share anything, but I’m curious how these things are shared internally.

I feel like I’m in my own little corner in research. I don’t look at some of the Slack channels because the company is very big, so I don’t know whether something like that exists. It probably does, because we need to share guides with customers about how to use the model.

I often say that the reason AI engineering can exist outside of the model labs is that the labs release models with capabilities they don’t even fully know about. You never train specifically for every capability; some of them are emergent. You can rely on crowdsourcing the search of that behavior space to the rest of us.

You don’t have to know everything. An interesting thing about o1 is that it’s really for the average human, but sometimes I don’t even know whether the model produced the correct output. It’s really hard for me to verify, even with hard STEM questions. If I’m not an expert, I usually don’t know.

The question of alignment is therefore even more important for complex reasoning models: how do we help humans verify the outputs of these models? Learning from external feedback is important.

One last thing on Claude 3: you had a section on behavioral design.

Behavioral design is a really cool area. I’m glad I made it a section.

Were you going to publish the model card without it, and then you insisted on adding it?

I put the section in, and Jared, one of my favorite people, said, “That’s cool. Let’s do that.”

Nobody really had this term of behavioral design for models. It’s a new little field of extending product design into model design: how do you create a behavior for the model in certain contexts?

For example, in Canvas, the model enters a more collaborative environment. What is the most appropriate behavior for it as a collaborator? Should it ask more follow-up questions? Should it change its tone? What is a collaborator’s tone? It’s different from a conversationalist’s tone. How do you shape the persona and personality around that?

There are philosophical questions too. Behavioral design is like creating a character in a video game. What are the core principles? What are the values? What are the charisma, intelligence, and wisdom of the character?

For Claude, this is much easier than it is for ChatGPT because it’s baked into the mission: honest, harmless, and helpful. The most complicated thing about model behavior, or behavioral design, is that sometimes 2 values contradict each other.

This happened with Claude 3. One of the main things we were thinking about was how to balance honesty versus harmlessness or helpfulness. We don’t want the model to always refuse, even for innocuous queries or creative-writing prompts, but we also don’t want it to be harmful. There’s always a balance between those things.

It’s more art than science. Dataset curation is also more of an art than a literal science. You can do empirical research on it, but synthetic data goes back to the Constitutional AI paper: how do you create completions such that the model agrees with certain principles?

If you create the model’s core values, how do you decompose those values into specific scenarios? How does the model need to express its honesty across a variety of scenarios? That’s where generalization happens when you craft the model’s persona.

It seems like what you describe as behavioral modification or shaping used to be a side job. Anthropic focused on it first and the most, but now every lab has some version of a “vibes officer.” At Anthropic it’s Amanda, at OpenAI it’s Roon, and at Google it’s Stephen Johnson and Risa. Do you think this is becoming a job? Does every company need a tastemaker?

I think the model’s personality is a reflection of the company and the people who create the model. At Anthropic, Amanda did a lot of Claude-character work. I worked with her at the time, but there wasn’t a team called the Claude Character team. There was a small team later, but before that there was none.

With Claude 3, we doubled down on feedback from people who used Claude 2. People said Claude 2 was much better at writing and had a certain personality, even though that was completely unintentional. We hadn’t paid much attention to it, and we didn’t know how to productionize that property of the model until Claude 3.

We knew we wanted to launch Claude in chat, and we wanted Claude’s honesty to be good for enterprise customers. We wanted to make sure factuality went up and hallucinations went down. We didn’t have a team until after Claude 3.

It’s happening now, and I think every lab is taking it seriously. OpenAI has a team called Model Design, led by a PM named Joanne, and I work closely with that team. We worked on the writing improvements we made to ChatGPT last year, and I worked on how to make ChatGPT act as a collaborator in Canvas.

Yes, we worked together on some of those projects.

I don’t think it’s publicly known, but his actual name is Roon. We’ll beep it, and people can guess.

Let’s move on to OpenAI and some of the recent work, especially Canvas. The first thing about Canvas is that it isn’t just a UX feature. There’s a different model behind it, post-trained on o1-preview distilled data, which is pretty interesting.

Can you walk people through the process? You come up with a feature idea; how do you decide what belongs in the model and what belongs in the product?

The most unique thing about ChatGPT Canvas was that the team formed out of thin air. It was around July 4, during the break. I remember I was taking a break, and I pitched the idea to Barrett Zoph, who was my manager at the time. I said, “I just want to create this Canvas.”

I didn’t know how to navigate OpenAI. It was my first month there, and I didn’t know how to get product to work with me or how to make ideas like this happen. I’m grateful to Barrett and Mira Murati, who helped staff the project.

It was about 5 or 6 engineers, and Karina was the researcher on the project. That’s how the team was formed. I didn’t know anyone there at that time except for Thomas Dimson, who built the initial engineering prototype of Canvas.

We learned a lot about how to work together across product and research. I think this was one of the first projects at OpenAI where research and product worked together from the very beginning. That made it successful, in my opinion, because the designers, engineers, PMs, and research team were all together. We pushed back on each other.

If something didn’t make sense to handle on the model side, we collaborated with the applied engineers to make sure it was handled on the applied side. You can only go so far with a prompted baseline. Prompted ChatGPT was the first thing we tried: Canvas as a tool, or defining the behavior of Canvas.

Then we found a lot of edge cases that we wanted to fix, and the only way to fix some of them was through post-training. We reinforcement-trained the entire GPT-4o model with our Canvas data.

There were 2 reasons for this. First, we wanted to ship it as a better model in the dropdown menu so we could rapidly iterate on user feedback after shipping it, rather than going through the entire integration process into the new o1 model. That process took time—it took about 3 months to go from beta to general availability.

We wanted to ship our own model with that feature, learn from user feedback quickly, and iterate rapidly. That was one of the decisions we made.

With Canvas itself, we had a lot of behavioral questions. When does Canvas need to write a comment? When does it need to update or edit the document? When does it need to edit the entire document versus a specific section the user asked about? When does it need to edit the Canvas itself?

Those were behavioral-engineering questions. I was also working on writing quality, so this was a perfect way to teach the model how to use Canvas and improve writing quality, which was one of the main use cases for ChatGPT.

There are so many questions. What does improving writing quality mean? What are the evaluations? How do you improve it?

I think about it in 2 directions. The first is improving the quality of writing for the existing use cases of ChatGPT. Most of those use cases are nonfiction writing: emails, blog posts, and cover letters.

The second direction is teaching the model to think more creatively and write in a more creative manner, so that it creates novel forms of writing. That is a much longer-term research question. The first is more straightforward: improve data quality for writing use cases.

For evaluating writing quality, I worked with Joanne’s team on model design. They had a team of model writers, and we worked together. It was a human evaluation, an internal human evaluation, on the prompt distribution that we cared about. We wanted to make sure that the models we trained were better.

So there was a test set of 100 prompts that you wanted to make sure were good?

I don’t know how large the prompt distribution needs to be, because you’re catering to everyone. It was a more opinionated way of improving writing quality. We worked with the model designers to come up with principles for what makes a particular piece of writing good. What makes an email good?

We created a rubric for what makes the writing good, and then, during the evaluation, we checked the models against that rubric.

That’s what schoolteachers do when they grade essays.

Exactly.

When do you work those improvements back into the model? If the Canvas model is better at writing, why not make the core model better too?

For example, I built a small podcasting tool using the GPT-4o API. I ask it to write up an episode based on the transcript, and then I do the same thing in Canvas. The Canvas output is much better than the raw GPT-4o output. Why not put those improvements back into the core model?

They did put them back into the core model. The GPT-4o Canvas model is now the same as the core model.

You must have missed that update.

What’s the difference, then? It still feels a little different. I’ve only tried it 3 times, but the Canvas output feels very different from the API output.

There’s always a difference in model quality. The original better model we released with Canvas was actually much more creative than the current one. When I use GPT-4o with Canvas now, I think it’s less creative.

It’s a complexity and versioning issue. Your version 11 can be very different from version 8, even if the data you put in is the same.

I’ve used it a lot more than 3 times, and I’m a huge fan of Canvas. It’s weird when I talk to my friends; they don’t really get it yet or use it yet. Maybe it was sold as writing help, when it’s really a scratchpad.

What are the core use cases?

Literally drafting anything. I’ll draft copy for the conference I’m running. I’ll put it there first, keep the Canvas open, and say what I don’t like about it. Then it changes it, and I might edit things there and paste them back in.

For example, I wanted to draft a brainstorm list of signs that you might be an NPC, just for fun. I wrote 10 of them and pasted them into ChatGPT, asking it to generate the next 10. They were all horrible.

It spun up the Canvas with the blog post, and I said, “Self-critique why your output sucks, and then try again.” It iterated on the blog post with me as a writing partner. That is so much better than having to manage all the intermediate steps.

The primary use case is drafting anything. It’s like having a writing partner.

Another way to think about it is that Canvas is an inversion of what Google Docs wants to do with Gemini. Google Docs starts with the main document and puts Gemini on the side. ChatGPT starts with chat and puts the document on the side.

It’s a reversal of what the main thing is. Google Docs starts with the Canvas, which you can edit, and then you call in AI assistance. ChatGPT is AI-first, with a Google Docs-like output on the side.

We definitely want to improve the writing use case, including making it easier for people to format and edit. There is still a lot of room for improvement.

The other major use case is coding. One thing we’re doubling down on is executing code inside Canvas. There are a lot of questions about how to evolve this. It’s becoming an IDE for both writing and coding.

The way I’m thinking about it is that ChatGPT evolves into a blank interface that can morph into whatever you’re trying to do. The model should derive your true intent and modify the interface based on that intent.

If you’re writing, it should become the most powerful writing IDE possible. If you’re coding, it should become a coding IDE.

It’s a strange decision to call those 2 things the same product, because they’re essentially 2 different UIs. One is Code Interpreter and the other is Canvas.

Yes.

I’m curious about some of the harder things. When I was reading about forcing the model to do targeted edits versus a full rewrite, it sounded like that was very hard. In the AI-engineering mindset, sometimes you might think that passing one sentence in the prompt will make the model rewrite that sentence, but obviously it’s harder than that.

What are some of the hard things that people on the outside don’t understand about building products like this?

It’s always hard with any new product feature—Canvas, Tasks, or anything else—because you don’t know how people will use it. How do you build evaluations that simulate how people will use the feature?

We therefore lean on iterative deployment to learn from user feedback as much as possible. We didn’t know, for example, that code diffs would be so difficult for the model.

You have to decide whether to fundamentally improve code-diffing as a model capability or use a workaround where the model rewrites the entire document, which can yield higher accuracy. Those are some of the decisions we have to make.

How do you improve the product-quality bar while making sure model quality is part of it? What kinds of cheats are you comfortable using?

This new way of product development is a combination of product research, model training, and product development, all going hand in hand. Defining the entire model behavior is one of the hardest things because there are so many edge cases.

It becomes even harder when you combine Canvas with other tools. For example, Canvas plus DALL-E or Canvas plus search: if you select a section and then ask for search, how do you build those evaluations? What features and behaviors do you care about most?

Do you test against every feature of ChatGPT?

No. I don’t think there are that many features that you can test exhaustively. It would take forever.

The decision boundary between Advanced Data Analysis, or Python, and Canvas is one of the trickiest behavioral boundaries we had to figure out. How do you derive intent from a human’s query?

When you say deriving intent, do you mean deciding whether the user expects Canvas or some other tool?

Exactly, and then making sure the result maximally matches the user’s intent. That is still one of the hardest problems.

It’s especially hard with agents. You don’t want an agent to spend 5 minutes doing something in the background and then return with a mediocre answer you could have gotten from a normal model, or something you didn’t even want because it didn’t have enough context or didn’t ask the right follow-up question.

You said the magic word. We have to take a shot every time you say “agents.”

Agents.

Let’s move to Tasks. You just launched Tasks. What was that like? It’s your baby.

Now that I have a team, Tasks was primarily my resident’s project. I was mostly a supervisor, so I delegated a lot of things to my resident, whose name is Vik. That was one of the projects where I learned management.

It was really cool. I tried to replicate the Canvas operating model: how do you work with product people, applied engineers, research, and design? I tried to replicate the operational process with Tasks.

Tasks was developed in less than 2 months. If Canvas took about 4 months, Tasks took about 2 months. Again, the process was similar: how do we build evaluations?

Some people asked for reminders in ChatGPT, even though they knew the feature didn’t work. There was still demand or desire from users to do this. I think Tasks is a simple feature that you would want from any model, but the magic is that the model is so general. It knows how to use search, Canvas, create science-fiction stories, and create Python puzzles.

When it’s coupled with Tasks, it becomes really powerful. Again, it came down to shaping the behavior of the model. We shipped it as a separate model in the model dropdown, and we’re working toward integrating that feature into the core model.

The principle is that everything should eventually be in one model. Because of operational difficulties, however, it’s much easier to deploy a separate model first, learn from user feedback, iterate quickly, and then improve the core model.

Tasks was also developed collaboratively from the very beginning. Designers, engineers, and researchers worked together, along with model designers, to create evaluations, test them, and do a lot of bug bashing.

I’d love to have a Canvas for this: distilling the ideal product-management or research-management process. Do you start with a PRD or some other document? From the PRD, do you get funding, staffing, and resources? Then do you prototype?

The prototype is a prompted baseline. Everything starts with a prompted baseline. Then we create the evaluations we want to capture and use to measure progress, and we make sure the prompted baseline fails on those evaluations. That gives us a place to climb from.

Once you start iterating on model training, it’s very iterative. Every time you train a model, you look at the benchmarks and your evaluations. If the evaluation goes up, that’s good, but you also need to make sure the model isn’t overfitting.

That’s where you run other evaluations, such as intelligence evaluations, and make sure there are no regressions elsewhere.

Is that your job, or is it the rest of the company’s job?

I think it’s mainly the job of the people doing the work. Regressions are going to happen, and you don’t necessarily own the data for all the other capabilities.

What’s happening right now is that you ablate your datasets. You compare against the baseline and compare regressions against the baseline model during training. Then you bug-bash, and that’s about it. Then you ship.

I took a course with Andrew Ng, and there was one little lesson about this.

I saw you tweet a picture with him, but it wasn’t clear whether you were working on a course.

I wasn’t working with him. I just took the course with him.

How do you think about Tasks? I’ve started creating a bunch of them. Do you see them becoming composable later? Could you schedule one task that does multiple tasks chained together?

Tasks is a foundational module. It should generalize to all sorts of behaviors that you want. Sometimes I see people put 3 tasks into one query, and right now I don’t think the model handles that very well.

Ideally, we learn from user behavior, and the model becomes more proactive in suggesting things. It might say, “I can do this for you every day because I’ve observed that you do it every day.”

Right now, you have to be explicit: “Every day, remind me of this.” Ideally, the model will think about you in the background and suggest, “I’ve noticed you’ve been reading these AI news articles. Maybe I can suggest them to you every day.”

It becomes more like a natural friend.

There is an actual startup called Friend that is trying to do that. We’ll interview Avi at some point.

It sounds like the guiding principle is simply: what is useful to you? That’s a little bit consumer-focused. Is there any B2B push, or do you not think about that?

I personally don’t think about it as much, but I definitely think B2B is interesting. Again, I come back to Claude in Slack. It was one of the first interfaces where a model operated inside an organization.

It would be very cool for a model to become a productive member of your organization and handle processes. I’m thinking about processing user feedback. It would be great if the model could start doing that for us, so we didn’t have to hire a new person just for that.

There are also simple use cases around data analysis and data analytics. How do you analyze a feature? Do you do the analysis yourself, or do you have a data-science team that tells you the insights?

I think there should be a startup that does automated data insights. You give it your data, and it tells you what’s going on.

Exactly. That’s what a data team at any company does: “Give us your data, and we’ll make PowerPoints.”

That would be very cool. I think that’s a really good vision.

You’ve had some thoughts on agents in general and more proactive systems. You tweeted a definition that’s interesting. I’ll read it to you, and you can explain it:

“Agents are a gradual progression of tasks, starting off with one-off actions, moving to collaboration, and ultimately fully trustworthy, long-horizon delegation in complex environments, like multiplayer, multi-agent tasks. Canvas affords the first 2.”

What does that mean?

I like writing long sentences. I need to learn how to write shorter ones.

I’m not sure it’s my definition of agents, but it’s how I think the progression makes sense. For me to trust an agent with my passwords or credit card, I need to build trust with that agent and know that it will handle my tasks correctly and reliably.

I would approach that the way I naturally collaborate with other people. When you first meet someone, you don’t know their working style—what they prefer, how they communicate, and so on. You spend the first 2 weeks learning how they work, and then you adapt to each other’s working styles.

That’s how you create collaboration. At the beginning, you don’t have much trust. It’s the same with a manager: how do you build trust with your manager? What do they need to know about you, and what do you need to know about them?

As you build trust, it comes through collaboration. That’s why I think building Canvas was one of the first steps toward more collaborative agents. With humans, you need to show consistent effort to each other—consistent effort that you care, that you work together well, and that you’re reliable.

Consistency and collaboration create trust. Then I would naturally delegate tasks to a model because I know the model will not fail me. It’s about building intuition for the form factor of new agents.

A lot of researchers and people in the community say, “Agents should delegate everything,” but on the way toward that, collaboration is one of the main roadblocks and milestones. Through collaboration, you learn the implicit preferences that will help you reach a model of full delegation.

I have an AGI working for me, and we’re still working on the trust issues.

We’re recording this just before the launch of Operator. The other side of agents that’s very topical is computer use. Anthropic launched computer use recently, and OpenAI is rumored to be working on similar things. A lot of labs are exploring how to drive a computer generally.

How important is computer use for agents?

It will be one of the core capabilities of agents. Computer use—agents using your desktop or computer—is the delegation part. You might want to delegate something like ordering a book, booking a flight, or searching for and ordering something.

This idea has been around for a long time, at least since 2022. Finally, we’re here. There was a lot between the idea and full execution—2 or 3 years of progress. Vision models had to get much better, especially at perception.

It has major implications for consumers, especially delegation. Latency is one of the most important factors. You want to make sure the model correctly understands what you want. If it doesn’t understand or doesn’t have the full context, it should ask a follow-up question and use that information to perform the task.

The agent should know whether it has enough information to complete the task with maximum success. That is still an open research question.

The second idea is that computer use enables a new class of research questions. Can we use computer-use agents in reinforcement learning? That is a very interesting and nascent area of research.

What’s one thing you think people will be using computer-use agents for a lot by the end of this year? I know it’s hard to predict.

Coding, perhaps. With Canvas, we’re thinking about the paradigm of moving from real-time collaboration to asynchronous collaboration. It would be cool if I could delegate a task to a model—“Figure out how to build this feature”—and the model could test that feature in its own virtual environment.

There will obviously be a lot of consumer use cases too, like shopping for me.

Everyone says booking plane tickets, but that’s a bad example because you only book plane tickets 2 or 3 times a year.

Concert tickets, maybe. Taylor Swift tickets.

I want a Facebook Marketplace bot that scrolls through Facebook Marketplace for free stuff and gets it.

I’ve been bearish on computer use. It’s slow, expensive, and imprecise, and the accuracy is still horrible, even with Anthropic’s new system. I’m waiting to see what OpenAI does to change my mind.

The way I think about it is that my opinions changed a lot between last January and last December. I’m trying to figure out what I’m wrong about today, and computer use is probably one of those things. I don’t know whether, by the end of the year, we’ll still be using these systems.

Will every ChatGPT instance have a virtual computer?

Maybe not. For coding, there are a lot of companies working on code sandboxes. There’s E2B, for example. But computer use is coding plus browsers plus everything else. It’s an entire operating system.

You need to be pixel-precise, understand what you’re operating, and know what you’re trying to do. I don’t know whether the models are there yet.

Do you think the progress of small models, like o3-mini or o1-mini, will help? Claude 3 Haiku and Claude 1.2 Instant were examples of small models becoming very powerful and fast. I’m sure computer-use agents could be coupled with small models to solve some of the latency issues.

Yes. In terms of operating systems, I think about this a lot. We’re entering a task-oriented operating system. In a few years, people will click on websites much less.

I want to see a plot of website clicks over time. My prediction is that clicks will go down, and people’s access to the internet will be through the model’s lens. Either you see what the model is doing, or you don’t see what it’s doing on the internet.

My personal benchmark for computer use this year is expense reports. I have to do an expense report every month.

For example, if I expense a lunch, I have to go back to the calendar to see who I had lunch with. Then I need to upload the receipt, tag the person, and fill in the expense report. It’s simple on a task-by-task basis, but you have to go to every app: Uber for the ride, the camera roll to get a photo of the receipt, and so on.

You can’t actually do it today, but it feels like a tractable problem. By the end of the year, we should probably be able to do it.

That reminds me of the idea that you could show computer-use agents how you want to book your flights. It’s like few-shot demonstrations. There may be a more efficient way that you do things, and the model should learn to do them in that way.

Again, it comes back to personalized Tasks. Right now, Tasks is rudimentary, but in the future it should become much more personalized to your preferences.

One takeaway I’ve gotten from this conversation is that ChatGPT will need to integrate much more with my life. It will need my calendar and email, for sure. Maybe it will use MCP.

I haven’t looked at MCP.

It has a lot of adoption.

Before we wrap up the OpenAI side, is there anything else we’re forgetting? Maybe something people should use more?

ChatGPT Search is interesting. The magic of ChatGPT when it first came out was that you could give it an instruction and it would follow it. “Write a poem,” and it would give you a poem.

The magic of the next generation of ChatGPT is that when you ask a question, the ideal output won’t just be text. It might be a React app generated on the fly. That’s already happening with Search. You ask for Apple’s stock, and it gives you a chart and a generative UI.

That’s what I mean by the evolution of ChatGPT into more of a generative operating system with task orientation. The UI will adapt to what you like. If you prefer 3D visualizations, the model should give you as much visualization as possible. Maybe you like rounded corners or a certain color scheme.

The UI becomes more dynamic and more customized—a personal model. We move from a personal computer to a personal model.

You’re one of the few people who has worked at both OpenAI and Anthropic.

Not anymore.

What are your general takes on the cultural differences?

I love both places. I learned so much at Anthropic, and I’m really grateful to the people there. I’m still friends with many of them. I was really sad when John left OpenAI, because I came to OpenAI partly because I wanted to work with him.

When I first joined Anthropic, there were maybe 60 or 70 people. When I left, there were around 700, so it went through massive growth.

OpenAI and Anthropic are different in terms of their product mindset. OpenAI is perhaps more willing to take product risks and explore different bets. Anthropic is more focused. They have to prioritize, and they’re definitely doubling down on enterprise more than consumers.

The research cultures are similar on a daily basis. It’s no surprise: the way you run experiments is largely the same.

You mentioned Barret, Mira, and Sam. What’s one thing you learned from each of them—or one lesson you would share with others?

I wish I had worked with them for longer. What I learned from Mira is her interdisciplinary mindset. She’s very good at connecting the dots between product and research, balancing them, and creating a comprehensive, coherent story.

Sometimes researchers hate doing product, while other researchers love doing product. There’s a dichotomy between the two. Safety is also part of this process. You want to think from a systems perspective and consider the bigger picture. I learned a lot from her about that.

I definitely feel that I have more creative freedom at OpenAI, and that comes from the environment the leaders created. If I have an idea and want to propose it, I can.

There’s also more creative freedom and resource reallocation, especially in research. Being adaptable to new technologies and changing your views based on empirical results—or changing research directions—is important.

I’ve seen researchers get stuck on the same direction for 2 or 3 years even though it never works out. They remain stubborn. Adaptability to new directions and paradigms is one of the most important things.

Is that a Barret thing, or is it part of the general culture?

It’s part of the general culture.

To wrap up, we usually have a call to action. Founders often want people to work at their companies. Do you want people to give you feedback, or do you want people to join your team?

I’m definitely hiring research engineers who are product-minded—people who know how to change models but are also interested in deploying them into products and developing new product features.

I’m looking for those archetypes of research engineers and research scientists. If you’re looking for a job or interested in joining my team, I’m happy for you to reach out.

More generally, what do you want people to do more of in the world, whether or not they work with you?

This is something I tell a lot of designers: people should spend more time playing around with models. The more you play with a model, the more creative ideas you’ll get about potential product features and new interaction paradigms.

We’re bottlenecked by human creativity when it comes to completely changing the way we think about the internet and software. AI is pushing us to rethink everything we’ve done before.

I don’t see enough people doubling down on those ideas, or enough human creativity in interface-design and product-design mindsets. It would be great for people to spend more time doing that.

As research becomes more product-oriented, you can train models for the things you want to do in the product.

You defined the process. This is now my go-to example of how to manage a research process. It’s common sense, but it’s nice to hear it from someone who actually did it.

Thank you for driving innovation in interface design and new models at OpenAI and Anthropic. We’re looking forward to your talk in New York.

Thank you so much for inviting me. I hope my job won’t be automated by the time I come to New York.

I hope you automate yourself and then do whatever else you want to do.