AI 架构师：Bret Taylor

Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO at DES Partners, and I’m joined by my co-host swyx, founder of smalli.

Hey.

Today we’re super excited to have Bret Taylor join us.

Thanks for having me. It’s a little unreal to have you in the studio. I’ve read about you so much over the years—even before OpenAI, effectively. I used Google Maps to get here, so thank you for everything that you’ve done.

Your story and history are extensive, and I think people can find out what your greatest hits have been. How do you usually like to introduce yourself when you summarize your career? How do you look at yourself?

That’s a great question. Before we went on the mics here, we were talking about the audience for this podcast being more engineering-focused. Depending on the audience, I’ll introduce myself differently because I’ve had a lot of corporate and board roles. I probably self-identify as an engineer more than anything else, though.

Even when I was co-CEO of Salesforce, I was coding on the weekends. I think of myself as an engineer, and all the roles that I do in my career start with that, because I do feel like engineering is a mindset and a way of approaching most of my life. I’m an engineer first, and that’s how I describe myself.

You majored in computer science around 1998.

High school, actually. My college degree was in 2002, and my master’s was in 2003. I’m not that old—come on.

I was thinking from 1998 to 2003. Engineering wasn’t as established back then. We didn’t have the title of senior engineer; you were just a programmer or a developer, maybe. What was it like at Stanford? Did it feel like you were on the cusp of a great computer revolution, or was it more of a niche interest at the time?

I was at Stanford from 1998 to 2002. 1998 was near the peak of the bubble. This was back in the day when most people did their coding in the computer lab because there were Sun Microsystems Unix boxes there that most of us had to use for our assignments. Every single day, there was a dot-com buying pizza for everybody. I didn’t have to do anything; I just got free food for my first 2 years of university.

Then the dot-com bubble burst in the middle of my college career. By the end, there was tumbleweed going through the job fair. It’s hard to describe unless you were there at the time, but the level of hype around being a computer science major at Stanford was incredible. There were a thousand opportunities, and then, when I left, it was Microsoft, IBM, and the 2 startups that I applied to: VMware and Google.

I ended up going to Google in large part because a woman named Marissa Mayer, who had been a section leader—a junior teaching assistant, essentially—for one of the big introductory computer science classes, had gone there and was recruiting me. I knew her, and it felt safe. I didn’t think about it much at the time, but it turned out to be a real blessing.

You always want to think that you’d pick Google if given the option, but no one knew what it would become at the time. I sometimes wonder: if I had graduated in 1999, would I have gone home and said, “I just got a job at Pets.com. It’s going to be huge”? In the end, I didn’t have many options. I had to decide whether I wanted to make kernel software at VMware or build search at Google, and I chose Google. It was a 50/50 ball, and I’m not really a 50/50-ball person, so I feel very fortunate in retrospect that the economy collapsed. In some ways, it forced me into one of the greatest companies of all time. I think I lucked into it.

The famous story about Google is that you rewrote Google Maps in 1 week after the Where 2 acquisition. What was the story there? Is it actually true, or has it been glorified? How did that come to be, and is there any detail that Paul hasn’t shared before?

It’s largely true, but I’ll give you the color commentary. It was actually the front end, not the back end. It turns out that for Google Maps, the front end was the hard part, because Google Maps was largely the first really interactive web application.

Gmail certainly was one too, although people who weren’t engineers probably didn’t appreciate its level of interactivity. It was just fast. Google Maps, because you could drag the map and it was graphical, really made that innovation mainstream. MapQuest at the time had up and down arrows. Each map was a single image, and you would click left and then wait a few seconds for the new map to load. The maps were small too, because generating a big image was expensive on the computers of the day.

Google Maps was truly innovative in that regard. There was a small company called Where 2 Technologies, started by 2 Danish brothers, Lars and Jens Rasmussen, who are now 2 of my closest friends. They had made a Windows application called Expedition, which had beautiful maps. Even in 2004, when we acquired—or sort of acquired—their company, Windows software wasn’t particularly fashionable. But they were passionate about mapping, and we had made a local search product that was middling in terms of popularity, sort of a Yellow Pages search product. We wanted to go seriously into mapping, and their small team seemed passionate about it, so we said, “Come join us. We can build this together.”

It turned out to be a great blessing that they had built the Windows application, because you’re less technically constrained when you’re doing native code than when you’re building in a web browser, particularly back then, when there weren’t really interactive web apps. It changed the level of quality that we wanted to hit with the app because we were shooting for something that felt like a native Windows application. Their unusual technical choices turned out to be a great blessing.

We spent a lot of time asking, “How can you make an interactive, draggable map in a web browser? How do you progressively load new map tiles as you’re dragging?” We got down into the weeds of the browser. Most browsers, including Internet Explorer, which was dominant at the time, would only load 2 images at a time from the same domain. We ended up making our map-tile servers have 40 different subdomains so that we could load maps in parallel.

There was just a maximum parallelism of 2?

Exactly. If you had a map set of 8 tiles, we wanted to load them all at once. We were down in the weeds of the browser, so there was a lot of plumbing.

I know a lot more about browsers than most people, but by the end, there was a lot of duct tape on that code. If you’ve ever done an engineering project where you’re not really sure of the path from point A to point B, it’s almost like building a house by building 1 room at a time. There isn’t a lot of architectural cohesion at the end.

Then we acquired a company called Keyhole, which became Google Earth. It was a native Windows app and a separate, great application, but with that acquisition we got licenses to all this satellite imagery. In August 2005, we added satellite imagery to Google Maps, which added even more complexity to the codebase.

Then we decided we wanted to support Safari. There weren’t mobile phones yet, so Safari was this niche browser on the Mac. It turned out there were a lot of decisions behind the scenes inspired by that Windows app—heavy use of XML, XSLT, and all these technologies that were briefly fashionable in the early 2000s and that everyone hates now, for good reason.

All of the XML functionality in Internet Explorer wasn’t supported in Safari, so people were reimplementing XML parsers. It was this pile of shit. It went from being a beautifully elegant application that everyone was proud of to something that probably had hundreds of kilobytes of JavaScript. That sounds like nothing now, but people had modems, so it was a big deal. It was slow to load, and it just wasn’t a great codebase. Everything was fragile.

I got frustrated with it, and one weekend I did rewrite all of it. At the time, the word JSON hadn’t been coined yet, just to give you a sense of the era. It was all XML.

We used what you would now call JSON, but I just said, “Let’s use eval so that we can parse the data quickly.” It was literally JSON, but there was no name for it at the time. We said, “Let’s pass it as JavaScript from the server and eval it.” Then I simply refactored the whole thing.

It wasn’t that I was some genius. It was just that if you knew everything you wished you had known at the beginning—and I knew all the functionality because I was one of the primary authors of the JavaScript—you could rewrite the whole thing. I drank a lot of coffee and stayed up all weekend.

I guess I developed a bit of a reputation, although no one knew about this for a long time. Paul Buchheit, who created Gmail and with whom I later started a company, told the story on a podcast. It’s largely true: I did rewrite it, and it’s probably the thing I’m proudest of.

I think JavaScript people appreciate this. The bundle size for all of Google Maps, when I rewrote it, was 20 kilobytes gzipped. It was much smaller for the entire application. It went down by about 10×.

What happened at Google after that?

Google was a pretty mainstream company, so our usage shot up. It turns out that simply being faster is worth a lot of percentage points of growth at Google’s scale.

How much modern tooling did you have? A test suite? Compilers?

No compilers. Actually, that’s not true—we did have 1 thing. Google had the Closure Compiler. I don’t know if anyone still uses it; it’s sort of gone out of fashion. Until recently, it was better than most JavaScript minifiers because it did much more variable renaming and things like that. Most people use esbuild now because it’s fast, whereas Closure Compiler is built on Java, so it’s slow.

We did have that. That was it.

It was a really interesting time at Google because there were a lot of teams working on fairly advanced JavaScript when no one else was. Google Suggest, for which Kevin Gibbs was the tech lead, was the first type-ahead autocomplete in a web browser, I believe. Now it’s pervasive in search boxes. ChatGPT just added it, essentially.

It was a 20% project for Kevin. Gmail was another example. Paul tells the story better than anyone, but he was basically scratching his own itch. What was neat about it was that email is such a productivity tool that it needed to be faster. He was making more things work on the client side.

Because Lars and Jens had set the bar with their Windows app, we felt that our maps needed to be draggable. We ended up innovating not only by having what would now be called a single-page application, but also with all the graphical stuff.

We were crashing Firefox like it was going out of style. When you make a document object model with the idea that it’s a document, then layer JavaScript on top of it and essentially abuse all of that, you run into code paths that haven’t been well tested. It was completely new territory.

It was super fun. There were a lot of great engineers figuring out new things. You had compiler people helping minify JavaScript in practical ways. The Closure Compiler was so good because it was built by someone who actually understood programming languages, not just by applying regular expressions.

The team that is now the Chrome team was there too. I believe Google was the main contributor to Firefox for a long time in terms of code, and a lot of browser people were at Google. Every time we crashed Firefox, we would walk up 2 floors and say, “What the hell is going on here?” They would load the browser in a debugger, and we could figure out exactly what was breaking.

We couldn’t change the browser because it was slow to update, but we could figure out exactly where the bug was and work around it in our JavaScript. It was a great engineering time—just a lot of people figuring out new things.

The term Ajax was also popular then. It stood for Asynchronous JavaScript and XML. I’m saying XML because that was the term at the time. The way you made HTTP requests from a client to a server was through an object called XMLHttpRequest. Microsoft created it for Outlook Web Access, and it turns out to have nothing to do with XML. It was just a way of making HTTP requests, but XML was fashionable.

JSON came out of that. A lot of the best practices around building JavaScript applications were developed before React. I think React was probably the big conceptual step forward that we needed.

Even my first social network after Google used a lot of HTML injection. Making real-time updates was still very hand-coded. It’s neat when you see conceptual breakthroughs like React because it’s obvious once you see it, but it’s not obvious until someone does it.

I feel like we’ll go through that evolution with AI agents as well. We’re missing a lot of the core abstractions that, 10 years from now, will make us say, “How did you build agents before that?”

There are a lot of contenders for the React of AI, but no clear winner yet.

One thing I’ve observed is that the early Google days had this interesting mix of product management and engineering. You didn’t wait for a product manager to tell you, “Here’s my PRD and requirements.” You were effectively the product manager.

I feel like these days, product is its own discipline, with its own lore and industry, and engineering is its own thing. There’s a process that happens, and the disciplines are separated. But you don’t produce as good a product when they’re separated as when they’re embodied in the same person.

Does that resonate with you when you compare early Google with modern startups?

I wear a lot of hats, so I’m biased, but I really agree that there’s a lot of power in combining product, design, and engineering into as few people as possible. Few great things have been created by committee.

If engineering is an order-taking organization for product, you can sometimes make meaningful things, but you rarely create extremely well-crafted, breakthrough products. Those tend to come from small teams that deeply understand the customer need they’re solving and have a maniacal focus on outcomes.

For some areas—if you look at software as a service 5 years ago—you could have a separation between product and engineering because most software created 5 years ago didn’t require many technological breakthroughs. If you’re making expense-reporting software, for example, you need to understand the requirements of the finance department and the individual filing an expense report. Then you implement it.

You understand how web applications are implemented, how databases work, and how to build auto-scaling with your AWS cluster. You’re applying best practices to yet another problem.

When you’re working in areas like the early days of mobile development, the early days of interactive web applications, or AI agents, you’re in a constant conversation between the requirements of your customers and stakeholders and the capabilities of the technology.

It’s almost impossible to specify the requirements of a product when you’re not sure of the limitations of the technology itself. That’s why I use the word “conversation.” You’re constantly saying, “Ideally, you could sprinkle some magic AI pixie dust and solve all the world’s problems,” but that’s not how it works.

Take coding copilots. Most of these tools—Cursor, Devin, Microsoft Copilot, and others—are remarkable. I couldn’t imagine development without them now, but they’re not autonomous yet. I wouldn’t let one write most code without interactively inspecting it. We’re somewhere between an amazing copilot and an autonomous software engineer.

As a product manager, your aspiration is obviously that the product should be autonomous. You should click a button and have a program come out the other side. But that requirement is meaningless. What matters is, based on the nuanced limitations of the technology, what is it capable of? How do you maximize the leverage it gives a software engineering team given those trade-offs?

Those trade-offs are changing more rapidly than any technology I can remember. Every few months, you have new models with new capabilities. How do you construct a product that can absorb those capabilities as rapidly as possible?

That requires a combination of technical depth and customer understanding. You need more integration between product, design, and engineering.

I think it’s why startups have a bit of an advantage relative to incumbents during big technology waves. They tend to be more self-actualized in bringing those disciplines together. In particular, I think proverbial full-stack engineers have an advantage because most breakthroughs happen when someone can understand extremely nuanced technical trade-offs, have a vision for a product, and then, in the process of building it, have that metaphorical conversation with the technology.

You might encounter a technical limitation you didn’t expect. It’s not just a matter of changing a feature; you might need to refactor the whole product based on that limitation.

I think that’s particularly important right now. If you’re building a big ERP system, there’s probably a good reason to have separate product and engineering disciplines. Those disciplines exist for a reason. But when you’re dealing with something as nuanced as large language models, there’s a lot of advantage in having individuals or organizations that integrate them more formally.

I’ve run a lot of engineering teams in the past, and I think the product-versus-engineering tension has usually been more about effort than whether a feature can be built. Today, you see much more of, “The models actually cannot do that.”

The most interesting thing is that, on the startup side, people don’t yet know where a lot of the AI value is going to accrue. You have this rush of people building frameworks and infrastructure-layer products, but we don’t really know the shape of the computer.

At Sierra, how did you think about building a lot of the tooling in-house for evaluations and agents, versus the startup opportunities that may still be out there?

We build most of our tooling in-house at Sierra. Not all of it. It’s not necessarily “not invented here” syndrome, although we may be slightly guilty of that in some ways. We’re trying to build an enduring platform, so we want to have control over our own destiny.

You made a comment earlier that we’re still trying to figure out what the React of agents is. I would argue it hasn’t been created yet. The jury isn’t still out; we’re in the jQuery era of agents, not the React era.

We shouldn’t rush it.

At Sierra, we help consumer brands build customer-facing AI agents. Our customers include Sonos, ADT Home Security, and SiriusXM. If you call them on the phone, an AI will pick up. If you chat with SiriusXM on its homepage, the AI agent is called Harmony, and it’s built on our platform.

We’re exploring the contours of what it means for someone to build a complete customer experience with conversational AI. We want to dive into all the trade-offs: where do you use fine-tuning? Where do you use models strung together? Where do you use reasoning? Where do you use generation? How do you use reasoning? How do you express the guardrails of an agentic process? How do you impose determinism on a fundamentally nondeterministic technology?

There’s a lot of important design space. I could sit here and tell you that we have the best approach—every entrepreneur will—but I hope that in 2 years we look back at our platform and laugh at how naïve we were, because that’s the pace of change.

As for startup opportunities, I’m not wholly skeptical of tools companies, but I’m fairly skeptical of them. There’s always an exception to every rule. I believe there’s a big market for frontier models, but largely for companies with enormous capital-expenditure budgets: OpenAI and Microsoft, Anthropic and Amazon Web Services, Google Cloud, and xAI, which is very well capitalized now.

The idea that a company can make money pre-training a foundation model is probably not true. It’s hard when you’re competing with unreasonably large capital budgets. Like the cloud infrastructure market, I think that will largely go to a handful of big companies.

I also really believe in AI applications. I don’t define that as building agents or tools. I define it as actually solving a problem for a business: what Harvey is doing in the legal profession, what Cursor is doing for software engineering, or what we’re doing for customer experience and customer service.

I believe in that because, in the age of AI, software can actually complete a task and do a job. That’s very different from the value proposition of software a couple of years ago. As a consequence, the way you build a solution for a domain is very different from how you would have built it before.

The incumbents don’t necessarily have a leg up. They have some advantages, but this is such a different form factor for providing a solution. It’s really valuable. Just think about how much money Cursor is saving software engineering teams, or how much revenue it can produce.

Tool-making is really challenging. If you look at the cloud market as an analog, there are many interesting tools companies: Confluent monetized Kafka, Snowflake, Hortonworks, and others. A lot of them had some mix of open source or open core.

I think developers are fickle. In the tools space, I probably default toward open source being the area that wins. It’s hard to build a company around that, although open-source companies can certainly work. The tools are changing so rapidly that I’m not totally skeptical of tool makers, but I think open source will broadly win.

The capital expenditure required to build frontier models is so large that it will go to a handful of big companies. I really believe in agents for specific domains, which I think will be analogous to software as a service in this new era.

If you think about the cloud, you can lease a server—a low-level primitive—or you can buy an application like Shopify. Most people building a storefront would prefer Shopify to hand-rolling an e-commerce storefront. I think the same will be true of AI.

If an entrepreneur asks me for advice, I say, “Move up the stack as far as you can toward a customer need.” That doesn’t reduce my excitement about people building agents and frameworks. It’s the right question to ask, but I think it will play out in open source more than anything else.

It’s not a priority for you, then. There’s a lot in there, and I’m curious about your idea maze. There are many customer needs, and you happened to identify customer experience as yours. It could equally have been a coding assistant or something else.

At a top-down level, how do you look at the potential problem space? There are many smart people who pick the wrong problem.

That’s a great question. I would also love to talk about the future of software, because even though I didn’t pick coding, I obviously care a lot about it.

When a technology is as exciting as large language models, you see a lot of people starting with the technology and searching for a problem to solve. That’s why you see so many tools companies. As a software engineer, you start building an app or a demo, encounter some pain point, and think, “A lot of people must be experiencing the same pain point. What if I make something to solve it?”

That’s incremental. I like to use a coffee metaphor. You can sell coffee beans, or you can sell roasted coffee beans. You added some value by taking coffee beans and roasting them, but roasted coffee beans are largely priced relative to the cost of the beans.

Or you can sell a latte. A latte is rarely priced directly as a percentage of coffee-bean prices. If you buy one at the airport, you have a captive audience, so it’s an expensive latte. There’s a lot that goes into how much that latte costs.

There’s a supply chain from growing coffee beans to roasting them. You could make a latte at home, or you could buy one at the airport. The margins of the company selling lattes at the airport are much higher than those of the people roasting coffee beans because it solves a much more acute human problem in that moment.

That’s one way to think about technology. You’re selling tools on top of a large language model. Your market may be large, but you’re probably going to experience price compression because you’re a piece of infrastructure, and you have open source and other things competing with you.

If you solve a genuinely important business problem for somebody—one that AI facilitates—they will value it according to the value of that business problem.

I don’t want to discourage people from trying things. Some of the greatest ideas have been things that no one believed in. If you’re passionate about something, go do it. Who am I to say otherwise?

Paul’s Gmail was his own email for a long time. I’m pretty sure he sent around a link, and the first comment was, “This is really neat; it would be great if it wasn’t your email but my own.”

If you want to build a venture-backed company, though, and you’re searching for an idea, solve an important business problem. If it’s a passion project, do it. Don’t listen to anybody. But if you’re trying to start an enduring company, solve an important business problem.

In the world of agents, software has shifted from helping more people be more productive to actually accomplishing tasks autonomously. As a consequence, I think the addressable market has expanded greatly because software can now do things and accomplish tasks.

How much is coding autocomplete worth? A fair amount. How much is the eventual software agent that writes the code and delivers it to you worth? A lot.

I would look up from the large language models and start thinking about the economy. Think from first principles. Which parts of the economy will benefit most from this intelligence? Which parts can absorb it most easily? What would an agent in this space look like? Who is the customer? Is the technology feasible?

I would start with the business problems. The best companies tend to have great engineers who happen to have great insight into a market. It’s that last part that some people lack. People start so much from the technology that they lose the forest for the trees.

How do you think about the model of still selling software versus selling more of a package of labor? When people are selling packaged labor, it feels more stateless and easier to swap out. If you’re just putting in an input and getting an output, it’s like coding: if there’s no IDE and you’re just putting in a prompt and getting back an app, it doesn’t really matter who generates the app.

You have less buy-in versus the platform you’re building. I’m sure that on the back end, customers have to put in their documentation and workflows. Where do you draw the line between managing customer support as a service—outsourcing it—and building on the Sierra platform? What was that decision?

I’ll decouple the question a bit. When you have an agent, who is the person using it, and what do they want to do with it?

Let’s take a coding agent. Who is the customer of an agent that produces software? Is it a software engineering manager? Is it a software engineer and their intern, so to speak? We’ll figure this out over the next few years.

Is it generating code that you review? Is it generating code with a set of unit tests that pass? What is the actual contract? How do you know it did what you wanted it to do?

The product and pricing model emerge from that. I don’t think the world has figured it out, and I think it will be different for every agent.

In our customer base, we use what’s called outcome-based pricing. Every time the AI agent solves a problem or saves a customer whatever it might be, there’s a pre-negotiated rate for that. We do that because we think it’s the correct way for agents to be packaged.

If you look back at the history of cloud software, particularly the introduction of the browser, it led to software being delivered through a browser. Salesforce famously invented software as a service, which is both a technical delivery model through the browser and a business model in which you subscribe rather than pay for a perpetual license.

Those 2 things are somewhat orthogonal, but not really. If software runs in a browser and is hosted in a data center that you don’t own, you need to change the business model because you can’t really buy a perpetual license to it. Otherwise, how would you afford to keep making changes? It only worked when you bought a new version every year or so.

To some degree, the business-model shift changed business as we know it. Adobe Photoshop is now something you subscribe to rather than purchase. The business-model shift turned out to be as significant as the technical shift.

With agents, because they actually accomplish a job, it doesn’t make sense to me that you would pay for the privilege of using the software. If a coding agent writes really bad code, fire it. I don’t know what the right metaphor is, but you should pay for a job well done.

That’s how you pay software engineers, although not really—we pay them a salary and give them options, and they invest over time. But you don’t pay them based on how many characters they write.

There’s a famous Apple story about asking for a report on how many lines of code an engineer had written. One engineer showed up with a negative number because he had done a big refactoring. It was a big “fuck you” to management that didn’t understand how software is written.

The traditional usage-based or seat-based model is going to look antiquated. It’s like asking a software engineer how many lines of code they wrote today. Who cares? There’s no correlation.

My view is that it won’t be the same in every category, but if an agent is doing a job, you should pay for the job being done well. It properly incentivizes both the maker of the agent and the customer.

It’s hard to measure engineering productivity, but you should do something other than count how many keys someone typed. Talk about perverse incentives for AI: I can write really long functions that do the same thing.

Broadly speaking, I think we’ll see a change in software’s business models toward outcomes, and a change in its delivery models too.

In our customer base, we empower our customers to keep their hands on the steering wheel of what the agent does. They want and need that, but the role is different. At many of our customers, the customer-experience operations staff have renamed themselves “AI architects,” which I think is really cool.

In the early days of the internet, there was a role called the webmaster. I don’t know whether “webmaster” is a fashionable term or even a job anymore. I don’t know whether “AI architect” will stand the test of time, but I do think the role is changing.

Everyone listening is probably a software engineer. What is the form factor of a coding agent?

I wrote a blog post right before Christmas about the future of software development. What’s interesting is that, if you look at how I use Cursor today, it’s inside a repackaged Visual Studio Code environment. I sometimes use the agentic parts of it, but it’s largely autocomplete, and I’ve developed a routine for tuning it properly.

When it can write code completely, I wonder what the future of development environments will look like. Have you driven around in a Waymo? There are these really nice Jaguar cars, but it’s funny because they still have a steering wheel, even though there’s no one sitting there. The steering wheel is turning and everything.

Clearly, in the future, once this becomes more ubiquitous, why have the steering wheel? Why have all the seats facing forward? Maybe it’s for car sickness, but you could rearrange the car. So much of the car is oriented around the driver.

It stands to reason that autonomous agents for software engineering wouldn’t run through Visual Studio Code. Having 1 source-code file open at a time is a strange form factor when the code isn’t being written primarily by you.

It begs the question of what your relationship with that agent is. I think the same is true in customer experience. Who are the people managing the agent, and what tools do they need? They definitely need tools, but they’re probably different from the tools we had before. It’s certainly different from training a contact-center team.

As software engineers, I’d like to see more innovation in programming languages, particularly on the passion-project or research side. We’re bringing the cost of writing code down to zero, so the fact that we’re still writing Python with AI cracks me up. Python was designed to be ergonomic to write, not safe or fast to run.

I’d love to see more innovation in how we verify program correctness. I studied formal verification in college. It isn’t very fashionable because it’s tedious, slow, and doesn’t work very well. But if a lot of code is being written by a machine, one of the primary values we can provide is verifying that it does what we intend.

There should be many interesting changes in the software development life cycle, including how we think about testing. If we have to manually read every line of code coming out of these machines, we’ll limit how much the machines can do. The alternative is totally unsafe.

I wouldn’t want to put code into production that hadn’t gone through proper code review and inspection. I don’t think coding agents work well enough to do this yet, but once they do, I think there’s an AI-native software development life cycle waiting to emerge. How do you enable software creators to produce the highest-quality, most robust, fastest software and know that it’s correct?

That’s an incredible opportunity. How much C code could we rewrite in Rust to make it safe and reduce security vulnerabilities? Can we have more efficient, safer code than ever before? Could you have someone like the operator in The Matrix, staring at the little green characters, operating a code-generating machine and being superhuman?

That’s a cool vision. Too many people are focused on autocomplete right now. I’m guilty of it too, but I’d like to see some bolder ideas. When you were asking about the React of AI, I think we’re clearly at a conceptual local maximum. It’s moving really fast, but I think we’re moving out of it.

At the end of 2023, I read a blog post called “From Syntax to Semantics.” If you think about Python, it took C and made it more semantic. Large language models are like the ultimate semantic program: you can talk to them, and they can generate any type of syntax in any language.

But the languages they have to use were made for us, not for them. As long as you need a human to intervene, you can’t change the language underneath. At what point will we reach a level of automation where we’re comfortable changing the underlying programming languages, rather than saying, “You have to write Python because I understand Python, and I’m more important than the model”?

I think it’s more nuanced. Some of the more interesting programming languages bring semantics into syntax. Rust is a good example. Rust is memory-safe and statically typed, which was an interesting conceptual break.

It’s why Rust is hard to write and why most people write Python instead. Rust programs are safer and faster than Python programs. Given the option, if you didn’t have to care about the labor that went into creating the program, you should prefer Rust over Python. It will run more efficiently and is almost certainly safer, depending on how you define safety.

Most people don’t write Rust because it’s a pain, and the audience of people who can write it is smaller. But it’s better in most ways. If you’re making a web service and you don’t have to care how hard it is to write, the Rust version will be cheaper to operate and probably more correct because of the static analysis implied by the language. It will probably have fewer runtime errors too.

Rust came out of the Mozilla team because there were a lot of security vulnerabilities in the browser, and browsers need to be fast. They decided to put more of the burden on the author at development time to have fewer issues at runtime, with the constraint that it had to be done statically because browsers need to be fast.

If you think about the needs of a programming language today, the role of the software engineer is to use AI to generate functionality and audit that it works as intended. How would you create a programming system that facilitated that?

I bring up Rust because it’s a good example. Given a choice between writing in C or Rust, you should choose Rust today. Most people would agree with that, even though C is efficient. C is generally less safe for similar trade-offs.

With AI, the game changes for writing these things. I wonder whether we’ll see a combination of programming languages that are more structurally oriented toward the values we need from an AI-generated program: verifiable correctness and so on. If it’s tedious for a person to produce, maybe that doesn’t matter anymore.

If I asked you whether a Rust program is memory-safe, you wouldn’t have to read it; you would just have to compile it. That’s a modest form of formal verification.

I think you can have AI inspect AI. You can have AI review AI code reviews. It would disappoint me if the best we could get was AI reviewing Python after scaling a few very large websites written in Python. Every team that has written a big web server in Python has experimented with PyPy and other tools to make it more efficient than it naturally is.

Python is convenient to write, but it doesn’t have true multithreading. I feel like we’re at a local maximum. I hope we create a programming system—a combination of programming languages, formal verification, testing, and automated code reviews—where you can use AI to generate software at high scale and trust it without being limited by your ability to read every line.

I don’t know exactly what form it would take, but it would be a pretty cool world to live in.

We had Chris Lattner on the podcast. He’s doing great work with Modular and Mojo. The idea is basically to merge Rust and Python, while keeping it compatible with Python so that Python developers can use it.

I wonder whether that still makes it a local maximum.

Exactly.

I’ll force you to make a prediction. Python is roughly 30 years old. In 30 years, will Rust be bigger than Python?

I don’t know. I don’t even know whether this is a prediction; I’m just saying something I hope is true. I’d like to see an AI-native programming language and programming system.

I use the word “language” because I’m not sure language is even the right concept. I hope that in 30 years there’s an AI-native way to make software that is wholly uncorrelated with the current set of programming languages—or at least not directly related to them.

Most programming languages were designed to be efficiently authored by people, with different trade-offs. Some were designed for abstractions for parallelism. Python and Perl were designed to be easy to write. Python has an interactive mode, which is one reason data scientists use it.

I’m a huge Python fan, despite all of my Python trash talk. At least 2 of my 3 companies were written exclusively in Python.

C came out of the birth of Unix. It wasn’t the first, but it was certainly one of the first major steps beyond assembly language, with higher-level abstractions such as for and while loops.

If writing code is no longer a meaningful human exercise, maybe the act of authoring code will go away. I don’t know. It may be one of those parts of history that disappears. But there’s still a role for the software engineer—the person actually building the system.

What does a programming system for that form factor look like? I hope it will be like React. I was at Facebook in the early days when what is now React was being created. When it was released as open source, I had already left, but I remember thinking, “This is so fucking cool.”

Modeling an app independently of the data flowing through it made everything easier. There’s a lot of chaos in front-end software today, and the abstraction soup can be overwhelming, but those core ideas felt ergonomic.

I’m looking forward to the day when someone creates a programming system that feels both like an aha moment and completely foreign to me. They will have created it from first principles, recognizing that authoring code in an editor may no longer be the primary reason a programming system exists.

That will be an exciting day for me.

Different versions of this discussion have happened before. At the end of the day, you still need to communicate precisely what you want. As someone who has managed people and worked on many legal contracts, you know how hard that is.

Now we have to communicate with machines. AI is interpreting what we mean and effectively reading our minds. I don’t know how to get across that barrier of translating human intent into instructions.

It can be more declarative, but I don’t know whether it will ever cross over from being a programming language into something more than that. I don’t know if it’s 2 years or 5 years.

I agree with you. If you look at a legal contract, the imprecision of the English language might feel like a flaw in the system. There are so many holes.

When you’re making a mission-critical software system, I don’t think it should be specified in English. That’s silly, because you want the precision of a programming language.

My point was less about that and more about the act of authoring. Some embedded systems use formal verification, and I know it’s common in security protocols because correctness is so important.

My intellectual exercise is: why not do that for all software? It’s probably silly to apply exactly what we do for low-level security protocols to everything. But the only reason we don’t is that it’s hard and tedious. Hard and tedious may no longer be factors.

If I could formally verify the most trivial app on your phone at zero cost, why wouldn’t I? The idea that the app should be formally verified feels laughable today because you wonder why anyone would spend the time. But if it costs nothing, then sure. It never crashes; that’s probably good.

I want to set our standards high. Software has been amazing. There was that Marc Andreessen blog post, “Why Software Is Eating the World,” and our whole life is mediated digitally. That’s increasing with AI. Our personal agents will talk to the agents on the Sierra platform, and it will be agents all the way down.

Our core infrastructure runs on these digital systems. We’ve had a shortage of software developers for my entire life. As a consequence, we’ve had things like the HealthCare.gov fiasco and security vulnerabilities that let state actors access critical infrastructure.

We’ve created this amazing system, and now we can fix it. I’m excited about the productivity gains in the economy, but as software engineers we should be bolder. We should aspire to fix these systems so that the digital infrastructure our lives depend on is more reliable.

I’m optimistic about it. Let’s go back to what you said about specifications, because I think it’s the most interesting part of AI agents broadly: most specifications are incomplete.

Go back to our product and engineering discussion. You have a detailed PRD with mockups and instructions about what happens when you click every button. You can still think of a missing requirement. Suppose you click a button and the internet goes out. What do you do? That probably isn’t in the PRD.

There’s always going to be something missing because humans are complicated. If you could measure it, what percentage of a product’s actual functionality is determined by its code versus its specification? For a traditional product, maybe 95% is determined by the code.

Code is the specification. That’s one reason open source has won out over specifications. For a long time, there was a W3C working group on the HTML specification. Once WebKit became prevalent, the internet evolved faster. It wasn’t at the expense of standards organizations; it just turned out that having a committee argue was less efficient than someone checking in code.

Suddenly you had vector graphics and a lot of other capabilities. Someone who worked on Google Maps could think, “That would have made my life easier.” SVG support would have made drawing a driving-directions line much easier.

In general, we’ve gone from protocols defined in a document to open-source code becoming an implicit standard. There is a specification for system calls and Linux—POSIX—but the kernel is what people write against. It includes both documented and undocumented behaviors, for better or worse.

That’s why Linus and others are so adamant about binary compatibility. This stuff matters.

One thing I think about with agents is how to express the guardrails. It’s not easy to specify them, but what about all the unspecified behaviors?

A lot of being a software engineer is reaching the point where the internet is down, you get an error code back from the call, and you have to do something with it. What percentage of the time do you decide, “I’m going to do this because it seems reasonable,” and what percentage of the time do you message your product manager asking what to do?

It’s probably more often the former than the latter, otherwise software development would be inefficient. But what happens when AI makes that decision for you? It isn’t necessarily a wrong decision; you just didn’t specify what to do in that case.

The agent is demonstrating agency and making a decision. Does it document that decision? That would probably be tedious too, because there are so many implicit decisions.

What happens when you click the button and the internet is down, and the agent does something you don’t like? How do you fix it?

I think we’re entering a world where what we express to an AI agent will always be an incomplete specification. That’s why agents are useful: they fill in the gaps with a reasonable amount of reasoning.

How you tune them over time is the hard part. Imagine building an app with an AI agent as your software engineering companion. There’s a long tail of functionality that isn’t specified. How do you tune that?

That’s what I mean about creating a programming system. I don’t think we know what that system is yet. For every agentic domain—customer service, legal work, software engineering—that’s essentially what the company building those agents is building: a system through which you express the behaviors you want, however esoteric or small.

It’s a really exciting area because that’s where the product insights will be. How do you encounter those moments? It has to be built into the user experience. The answer can’t simply be, “Write a better prompt.”

The prompt would be too long. If you had a PRD that specified the behavior of everything represented by the code, the answer would just be code.

Exactly. Prompts are useful, but they can never be a complete specification. That’s how you do the human-in-the-loop interaction: when and how do you involve people?

That’s why I believe in domain-specific agents. Answering the question in the abstract is an interesting intellectual exercise, but talking about agents in the abstract doesn’t mean much to me. It just means that software is making decisions.

In software engineering, it makes sense because you can describe the process: first you specify what you want in a product, then you use it, then you give feedback. You could imagine building a product that facilitates that closed loop.

The complete specification is the union of what you knew you wanted and what you discovered through usage. The rest is less important to the AI.

In the legal context, there’s certainly a way to know when an AI should ask questions and when it shouldn’t, and how to intervene when it’s wrong. In customer service, it’s very clear. Our customers review every conversation, and we help them find the conversations they should review when they have millions of them.

When something is wrong in one of those conversations, they can give feedback so that it’s fixed the next time, in a way that preserves the context of why the AI made that decision. But it isn’t up to us to decide what’s right; it’s up to our customers.

That’s why I think that, when you build an agent for a domain, how you interact with the people specifying its behavior is where much of the magic is.

Stop me if this is annoying, but I have trouble squaring domain-specific agents with the belief that AGI is real or coming. The point of general intelligence is that one way to view the bitter lesson is that we can always make progress by becoming more domain-specific, but then the next advance wipes that out.

You clearly don’t believe that. How do you square those things personally?

That’s a heavy question. I think a lot about AGI given my role at OpenAI. I spend time with OpenAI researchers and people in the broader community talking about the implications, because there are first-order effects and second- and third-order effects that are harder to predict.

First, it seems likely to me that AGI will initially be good in digital domains because it is software. If you think about AI discovering a new pharmaceutical therapy, the barrier may be less the discovery than the clinical trial. AI doesn’t necessarily help with the clinical trial; that’s a physical process.

Similarly, with climate change or carbon removal, there may be a lot of room for great ideas, but once you have the idea, there’s probably a large physical component. It’s not limited by intelligence. Intelligence may accelerate it somewhat, but it isn’t the only constraint.

There was an interesting conversation with the economist Tyler Cowen recently. He discussed how there are parts of the economy where intelligence is the limiting resource. Those parts will adopt AGI rapidly and see incredible productivity gains. There are other parts where intelligence isn’t the limiting factor, and those will interact with the first group.

Prices may rise in the domains that can absorb intelligence rapidly, which could slow things down. I don’t think the effects will be evenly distributed or felt as rapidly across the economy as people expect. I could be wrong, but I don’t think intelligence will generalize evenly throughout the physical world.

As for the practical question—why make software if AGI is coming?—my view is that AI, and AGI, are tools for humanity. When we were talking about whether your job as a software maker is to author code in an editor, I would say no.

A generation ago, your job wasn’t to punch cards. That wasn’t the job; it was the tool. Your job is to produce something digital, whatever it is. You might be producing a digital experience or firmware for a toaster.

Our job will change rapidly and meaningfully, but the idea that our job is to type into an editor is an artifact of the tools we have, not what we’re hired to do.

Software engineering will be one of the disciplines most affected. If you define yourself by the tools you use or by how many characters you type into them every day, that probably isn’t a stable long-term position because AI can do that better than you.

Your judgment about what to build and how to build it still applies, and that will always be true. One way to think about it, somewhat reductively, is to compare startups with larger companies. Google and Amazon have many more engineers than a startup, but startups still win. Why? They made better decisions. They didn’t type faster or produce more code; they did the right thing in the right market at the right time.

The existence of more intelligence will change a lot, including our jobs, perhaps more than any other industry. But it doesn’t change why the economy around digital technology exists. Entrepreneurs will still need good judgment about where to apply the technology to create economic value.

I’m bullish on the future of the software industry. Some things that are expensive today will become almost free. But the half-life of technology companies isn’t particularly long as it is.

When I started at Google, we were in 1 building in Mountain View. Eventually we moved to a campus that had previously been the Silicon Graphics campus. That was Google’s first campus, I’m pretty sure, and it still has that campus. Silicon Graphics was a company that was big enough to have a campus and then went out of business in my lifetime.

At Facebook, I joined the second office, an old HP building near Stanford. Then we got big enough to have a campus and bought the Sun Microsystems campus. Sun came out of Stanford, was once a darling of the industry, and was eventually bought for pennies on the dollar by Oracle.

All these companies were big enough to go public and have campuses, then go out of business or be acquired. A lot will change. I don’t mean that it will be easy or that no business model is under threat.

But will digital technology remain important? Will entrepreneurs still need good judgment about where to apply it to create economic value? Absolutely.

If you went back to 1980 and described many of the jobs we have today, it would be hard for people to conceptualize them. Imagine describing podcasting to someone in 1980. Imagine going back to 1776 and describing today’s economy to Benjamin Franklin.

It would be hard for him to conceptualize even the services economy, let alone the technology industry. How could so few people produce enough food for so many? It would take hours of explanation.

We have a view of how the world works that’s based on today’s constraints. There will be many other opportunities.

I don’t know whether writing code will remain valuable in its current form. It will probably change rapidly. The best thing we can all do—not just people in the software industry—is lean into the change.

Try the tools. Install the latest coding assistant. When o3-mini comes out, write code with it. You don’t want to be the last accountant to embrace Excel. You might not have a job anymore.

The analogy you made about food really resonated with me. I feel like we’re in an agrarian or barter economy for intelligence, and now we’re industrializing intelligence. That was an aha moment for me.

How do you think about a person being replaced by an agent, and how agents talk to each other? Even at Sierra, you’re building agents that people talk to, but in the future you’ll have agents that complain about the order they placed with customer support, all the way down.

You were the CTO of Facebook and built Open Graph. There were many positive things enabled by it, and perhaps many negative consequences too. How do you think agent protocols should be built, considering the implications for privacy, data discoverability, and everything else?

I think it’s too early for a protocol to emerge. I’ve read about a few attempts, and perhaps some of them will catch on.

One interesting thing about large language models is that, because they’re trained on language, they’re very capable of using interfaces built for people. My intuition is that if we can make an interface that works for us and works for AI, maybe that’s good enough.

That’s a little hand-wavy, but making a machine protocol for agents that is inaccessible to people has some upsides and quite a few downsides. I think agents will speak to agents using language for a while. I don’t know whether that’s true, but there are many reasons it might be.

When your personal agent speaks to a Sierra agent to figure out why your Sonos speaker has a flashing orange light, my intuition is that it will be in English for a while. There are many benefits to that.

We’re still in the early days of long-running agents. I don’t know whether you tried the Deep Research agent that just came out.

We have one for you on Deep Research.

That’s great.

It was interesting because it was probably the first time I’d been notified by OpenAI when something was finished. I brought up the interactive aspect earlier, and that’s the area I’m most interested in. Most agentic workflows are relatively short-running, but the workflows that are long-running, involve multiple stakeholders, and span multiple systems are interesting. We deal with many of those at Sierra.

Before the mobile phone, whenever you got a notification from an internet service, you received an email—not because email was the best way to notify you, but because it was the only way to notify you. You got tagged in a Facebook photo and received an email about it.

Once the internet was in everyone’s pocket, every app had equal access to buzzing your pocket. Now, for most apps I use, I don’t get email notifications; I get the notification directly from the app.

I wonder what the form factors for agents will be. How does an agent address and reach out to other agents? How does it bring the operator into the loop at the right time?

ChatGPT will be one of the major consumer surfaces, so there’s a lot of gravity there. But if I think about domain-specific workflows, there’s a lot to figure out.

I’m less interested in agent-to-agent protocols. I could be wrong, and I haven’t thought about them much. I’m more interested in how an agent engages with all the people involved. That’s one of the things I’m most interested to see play out.

To me, the core questions are about access control: can this agent access this thing? That’s less prominent in customer support, but it’s more interesting in enterprise settings.

There’s also language. If the human doesn’t have to read it, you can compress the language, save tokens, and make things faster.

You mentioned being notified about Deep Research. Does OpenAI have an internal notification that goes out to everybody when Deep Research is achieved? Does the board get summoned to see it? Can you give us any backstory on that process?

OpenAI is a mission-driven nonprofit that I think of primarily as a research lab. It’s obviously more than that. In some ways, ChatGPT is a culturally defining product, but at the end of the day, the mission is to ensure that artificial general intelligence benefits all of humanity.

A lot of our board discussions are about research and its implications for humanity, primarily safety. You can’t achieve AGI without thinking about safety as a primary responsibility of the mission. We also discuss access and other issues.

We definitely talk about things like Deep Research because they’re a big part of what it means to build AGI. But we talk about many different things. Sometimes we hear about things very early. If something isn’t related to the core mission or is far afield, the discussion may be more casual.

It’s fun to be part of because my favorite part of every board discussion is hearing from researchers about how they’re thinking about the future and the next milestone in creating AGI.

Let’s start at the beginning. There are very few people who have been in the rooms that you’ve been in. How did you get brought into OpenAI? Obviously, there’s some drama that you can go into if you want. Take us into the room. What happened? Was it a Thursday or Friday when Sam was fired?

I heard about it like everyone else—on social media. I remember where I was walking when I saw it. I was totally shocked. I messaged my co-founder, Clay, and said, “I wonder what happened?”

On Saturday, trying to protect people’s privacy, I ended up talking to both Adam D’Angelo and Sam Altman and getting a synopsis of what was going on. You would have to ask them for their perspectives, but my understanding was that both the board and Sam trusted me to some degree.

It was a complicated situation because the company reacted very negatively, and understandably so, to Sam’s firing. I don’t think people understood what was going on. The board needed to figure out a path forward, and they reached out to me.

I talked to Sam and ended up being a mediator, for lack of a better word. It wasn’t a formal role, but fundamentally that’s what happened. As the board tried to figure out a path forward, we had many discussions about reinstating Sam as CEO while also conducting a review of what happened so that the board’s concerns could be fully adjudicated. They obviously had concerns going into it.

I ended up there because I was known to many of the stakeholders. I’d like to think I have some integrity, so they were trying to find a way out of a very complex situation.

I formed a great relationship with Sam and Greg. It was a challenging time for the company. I didn’t plan to be on the board; I was pulled in because of the crisis, and I don’t think I’ll be on the board forever.

When I joined, I posted that I was going to do it temporarily. That was over a year ago. I really like focusing on Sierra, but I also care about OpenAI. It’s an amazing mission.

You’ve been in high-stakes situations like that before—perhaps twice, but obviously not at the same level. What principles do you follow when the stakes are as high as possible, whether that means the most money or the greatest consequences? You have a great reputation and network, but what are your must-dos and must-not-dos?

If there were a playbook for situations like this, they would be much simpler. I just go back to how I operate in general.

First is first-principles thinking. There are crisis playbooks, but there was nothing quite like this. You need to understand what’s going on and why.

Many moments of crisis are fundamentally human problems. You can strategize about people’s incentives, but it’s important to understand everyone involved, what motivates them, and why. That’s fundamentally an exercise in empathy. Do you really understand why people are doing what they’re doing?

The next thing is getting good advice. What’s interesting about a high-profile crisis is that everyone wants to give you advice, so there’s no shortage of it. The challenge is identifying the good advice.

That requires judgment. Based on a first-principles analysis of the situation and your assessment of the people involved, who has real expertise and good judgment? You want to validate your own intuition, or, if it’s an area of legal expertise where you aren’t an expert, you want the best people in the world to advise you.

I find that people often seek out the wrong people for advice. It’s important to get that right.

The Microsoft offer was real, right? At some point, Sam and the team could have moved over to Microsoft.

I’m not sure. I was involved from one vantage point, and I didn’t really have skin in the game. I still don’t own any equity in OpenAI. I was a meaningful bystander.

The reason I got involved was that I cared about OpenAI. I had left Salesforce, and by coincidence, ChatGPT came out the next month. I got nerd-sniped like everyone else. I thought, “I want to spend my life on this. This is amazing.”

I’m not sure I would have started another company if OpenAI hadn’t inspired the world with ChatGPT. Maybe I would have, but it had a significant impact on all of us.

The idea that OpenAI would dissolve over a weekend bothered me because I’m very grateful for its existence. It created the proverbial iPhone moment for AI and changed the world.

Microsoft is an investor in OpenAI and has a vested interest. Sam and Greg had their interests, and the employees had theirs. There was a lot of negotiation, but you can’t A/B-test decision-making. I don’t know what would have happened if things had fallen apart.

You also don’t know what was real and what wasn’t. You would have to talk to the people involved to know what was really happening.

I heard that Brian Armstrong was a surprisingly strong adviser during the whole process.

My understanding is that both Brian Armstrong and Ron Conway were close to Sam throughout it. I spoke to them, but I also tried to talk to the board because I was trying to mediate.

Obviously, you have a position on the situation. I felt that, from the outside looking in, I wanted to understand why it happened. The process seemed ham-fisted, to say the least.

But I was trying to remain dispassionate. If you want to put Humpty Dumpty back together again, you can’t be a single-issue voter. You have to understand the whole situation.

Brian is one of the great entrepreneurs and a true friend and ally to Sam. He has been through a lot as well.

Microsoft is obviously a huge backer. We spoke to David Luan, who pitched what I think was the first $1 billion investment in OpenAI. My understanding was that the best situation for OpenAI and Microsoft was for OpenAI to stay as it was. The second-best situation was for Microsoft to hire Sam, Greg, and whoever else from the team.

That relationship was very close and exclusive. Things have evolved, especially with Stargate, and there’s some uncertainty about the relationship between Microsoft and OpenAI. How would you characterize it now?

Microsoft is OpenAI’s most important partner. We have a very deep relationship with them on many fronts.

The relationship is always evolving because the scale of the market is evolving. In particular, the capital requirements for infrastructure are far beyond what anyone would have predicted 2 years ago, let alone when the Microsoft relationship started—perhaps 6 years ago. I don’t actually know off the top of my head, but it was a long time ago.

I don’t have anything specific to share. The relationship has evolved because the market has evolved, but the core tenets of the partnership have remained the same. It’s by far OpenAI’s most important partnership.

A lot of our listeners care about OpenAI’s priorities. I’ve heard them described as having 5 top-level priorities: always having frontier models, staying on the frontier of efficiency, and being first in multimodality, whether that’s video generation, real-time voice, or something else.

How would you characterize OpenAI’s top priorities apart from the highest-level goal of AGI?

They always come back to AGI. OpenAI is a mission-driven organization. A lot of companies talk about their mission, but at OpenAI, the mission defines everything we do.

If you’re trying to predict where OpenAI is going to go, you need to understand that. If something doesn’t serve the mission, it’s unlikely to be a priority. It’s a big organization, so occasionally there are side projects where people say, “I’m not sure this serves the mission as much as we thought,” and they stop doing them.

At the end of the day, people work at OpenAI because they believe in the benefits AGI can have for humanity. Some people want to build it, and the act of building it is intellectually rewarding. Others want to ensure that AGI is safe.

I think OpenAI has the best AGI safety team in the world. There are many interesting research problems as models become increasingly capable and gain access to the internet and tools.

Everyone is there because they care about the mission. If you want to predict where OpenAI is going, ask what you would do if you were running a company trying to build AGI and ensure that it benefits humanity.

Deep Research makes sense through that lens. If you’re thinking about what it means to create AGI, enabling AI to help further research is meaningful. That’s why many AI labs are working on software engineering and code generation. If you’re trying to create AGI, code is a huge part of that.

The same is true of tool use and agents. They’re central to what you need to do to achieve AGI.

The only reason the organization exists is to build AGI and ensure that AGI benefits all of humanity.

What I find remarkable is that, if I had described that mission to you 4 years ago, you might have imagined industrial applications, robots, and other things. ChatGPT has been a delightful and counterintuitive way to serve the mission.

The idea that you can go to chatgpt.com and access the most advanced intelligence in the world, with a free tier, is amazing. ChatGPT is one of the most important parts of the mission because it’s how many people will use this intelligence every day.

It isn’t limited to a few people or an inaccessible form factor. One of the contours of the mission is that everyone should benefit from AGI, and ChatGPT is a way to make that possible.

Cost is important. How can all of humanity access AI if it’s incredibly expensive and requires a $200 subscription? I pay for the $200 subscription because o1 Pro mode is mind-blowing, but you need both. You need advanced research capabilities and broad access.

That’s how we prioritize everything: what would help us build AGI and ensure that it benefits humanity?

I know we’re going to wrap up soon, but I’d love to ask some personal questions. What principles guide you in choosing what to do? You were co-CEO of Salesforce and CTO of Facebook. You could have done many other things. Do you have frameworks for making those choices?

I try to remain present and grounded in the moment. Meditation? I wish I did it more, but I don’t.

I try to focus on impact and on whether I enjoy what I’m doing. We talked about what an entrepreneur should work on when starting a business. I joked that some of the best businesses are passion projects, but I definitely take both things into account.

I want to have an impact on the world, and I want to enjoy building what I’m building. I wouldn’t work on something impactful if I didn’t enjoy doing it every day.

I also try to have balance in my life. I have a family. One of Sierra’s values is competitive intensity, but we also have a value called family. We say that intensity and balance are compatible.

You can be an intense person and still have balance. I don’t have many hobbies. I mostly work and spend time with my family, but I have balance there.

If you’re proverbially on your deathbed, what do you want? I want to be surrounded by people I love and to be proud of the impact I had.

I know you also love making handmade pasta. I’m Italian, so I’d love to hear about your favorite pasta shapes and sauces.

I don’t know where you found that. Was it Deep Research?

It was a deep cut.

I do love to cook. I started making pasta when my kids were little because I found that getting them involved in the kitchen made them eat their meals better. Participating in making the food made them appreciate it more.

We make a lot of spaghetti and linguine because they’re pretty easy. The kids could operate the crank, and I could put the pasta through. It was very interactive.

I make a lot of different kinds. A really simple marinara with good tomatoes and pasta is a classic, especially if you have excellent pasta. That’s probably the default because it’s easy.

I mean, you have to weigh in as the Italian here.

Yeah, I would say so. There’s one type of spaghetti called alla chitarra. It’s kind of like they’re almost square. Those are really good with a cherry-tomato sauce and olive oil.

Bret, you should also go to Italian restaurants with me. I highly recommend them.

I’ve heard you’re the head of the Italian tech mafia and a very good restaurant guide.

How do you keep up with AI? There’s so much going on. Do you have a special news resource that no one else has?

No. Most mornings, I’ll check what’s going on on social media, look for any buzz, and read papers.

The thing I really like is that we have a small research team at Sierra. We do sessions on interesting papers. Usually, someone goes deep on a paper and gives a readout over lunch.

I find that rewarding because I love research, but sometimes simple concepts are surrounded by ornate language. You add a few more Greek letters, and it seems like you did something smart. Talking through the concepts helps me get to the “so what?” more easily.

Conversations are useful too. If someone mentions something I’m unfamiliar with, I’ve gotten over feeling dumb. I just say, “I don’t know what that is. Explain it to me.” You can discover new techniques and papers that way.

It’s impossible to keep up, though, to be honest.

If you’re struggling, imagine the rest of us. Based on the conversations you’re having internally, what research directions should people pay attention to? What isn’t surprising to you?

The reasoning models. The chain-of-thought reasoning paper was important, and chain of thought has been meaningful from the beginning. It may have been a Google paper, if I remember correctly.

What’s interesting is the combination of distillation and reasoning. It’s making the relative performance—and the latency—of reasoning models more reasonable.

GPT-4 was a huge step change in intelligence, but it was slow and expensive for a long time, which limited its applications. Once you got to GPT-4o and GPT-4o mini, the cost and latency opened the door to many more applications.

When o1 came out, the quality was really interesting, but it was slow and expensive. That limited the applications. Now, I saw someone post that they had distilled one of the DeepSeek models and made it really small. It was doing chains of thought very quickly, with latency numbers similar to GPT-4 in its early days.

All of a sudden, you think, “This is really interesting.” For people working in applied AI, it’s basically a three-way trade-off between performance, quality, and latency.

For a long time, the market was so young that you had to choose which quadrant you wanted for your use case. The idea that we can get relatively sophisticated reasoning at o3-mini speed is amazing.

If you haven’t tried it, the speed makes me use it much more than o1. o1 was so slow that I would often craft prompts using GPT-4o and then put them into o1 because I didn’t want to wait for the turnaround.

We’re in the early days. Just as the rapid change from GPT-3 to GPT-3.5 to GPT-4 created an explosion of ideas and possibilities, the changes in reasoning models, inference-time compute, and the techniques around them are creating a similar explosion.

The use cases for these models are expanding. Coding is exactly the kind of domain where reasoning models should have better results. In our domain, too, some problems benefit from thinking more robustly.

These models are increasingly coming with more capabilities built in. I’m really excited about them.

Any final call to action? Are you hiring? Are you growing the team? Should more people use Sierra?

We are growing the team. We’re hiring software engineers and agent engineers, so send me a note at bret@sierra.ai.

We’re growing like weeds. Our engineering team is exclusively in person in San Francisco, but we do have some forward-deployed engineers in other offices, like London.

Awesome. Thank you so much for the time, Bret.

Thanks for having me.