[BidClub_]
20VC · · 45 min

Kevin Scott, CTO @ Microsoft: An Evaluation of Deepseek and How We Underestimate the Chinese

Harry StebbingsKevin Scott

YouTube
TL;DR
  • Where does value accrue? Scott's core call: "models aren't products" — models are "super valuable but they're only valuable to the extent that you can connect them to things that users need via product," and "in the limit I think product is the most important thing." Infrastructure and compute monetize along the way, but "we don't build infrastructure just for the sake of infrastructure." Harry's confusion is characteristic of the moment — every paradigm shift (internet, mobile) starts this way, and "very few of those ideas were actually the durable ones."
  • On scaling walls: "that's a ridiculous statement" — "I can very clearly see what we're doing now and what we're doing next and I don't see the limit to the scaling laws." His hedge is economic, not technical: there will eventually be a point of diminishing returns where "it's so expensive that we will decide it's not worth spending that next dollar to make this thing one unit smarter" — but "it's not in the viewfinder right now."
  • On DeepSeek R1: "we've had models more interesting than DeepSeek R1 that we chose not to even launch." It was "good solid technical work," but just "a point on a line of price performance improvement that maybe was invisible to everyone else" — inference price-performance improves much more through software than through the ~2x-per-generation hardware gain. And the surprise itself was the tell: "we should really really really respect the capability of Chinese entrepreneurs, scientists and engineers... that shouldn't have been surprising."
  • The agent thesis: memory is the piece "conspicuously missing" that makes agents "awfully transactional" — he expects it to get much better over the next year, and he hopes for more asynchronous dispatch ("you sort of dispatching your agent to go do something... while you are not paying attention"). The trajectory: 5-second tasks → 5-minute tasks → delegation "the same way that you would to a coworker." And it's many agents, not one — because product managers will probably have to be domain experts, helping set up agents' feedback loops.
  • 95% of net new code will be AI-generated in five years, "very little... line by line human written" — but "the more important and interesting part of authorship is still going to be entirely human." It's the assembly-to-high-level-language transition again ("there were some old farts who would say you're not a real programmer... nobody talks about that anymore), and he hopes small teams will be able to do big things more easily: "you can do a lot with 10 really great super motivated engineers."
  • Positioning tells from the quick-fire: the competitor he most respects is maybe Anthropic ("I think Dario's doing a good job"); "frontier models are probably better health diagnosticians than your average GP" already; and his biggest learning from the program's first year: "there's a bigger gap than there was even two years ago between what the most capable frontier models can do and what they're being used for."
Digest · the substance, structured for research

1. Models aren't products — value pools where product meets user

  • Harry's opening confession — for the first time in a long time he doesn't know where value lies — gets framed as characteristic: "that's exactly the thing that happens at the beginning of every big technological paradigm shift." In the early internet and early mobile, "everybody had these ideas about what was going to be valuable and very few of those ideas were actually the durable ones."
  • Scott's line of the past two years: "models aren't products." Pressed on whether that makes them not valuable (Andrew, likely from Cerebras, had told Harry compute wins), he's emphatic: "no no no, they're super valuable but they're only valuable to the extent that you can connect them to things that users need via product... in the limit I think product is the most important thing." Good models, infrastructure, and efficient compute all monetize — "as people build those products they will need to consume your platform" — but "we don't build infrastructure just for the sake of infrastructure."
  • The prescription for confusion: don't sit on your hands. "This is the best time to be alive if you have an entrepreneurial spirit." Launch, gather data, and "be super super brutal with your own self... you can't love your idea so much that you overlook what it is you're seeing." A tell of every early cycle: "technical people who get swept up in the technical bits and forget that the only thing that really matters is making good product."
  • Startups vs. incumbents: past cycles show "a pretty good mix of where value gets created" across both. Microsoft's approach is applying new capability to customers it already serves; but "it's impossible for any entity like Microsoft... to have enough imagination and enough perspective to know what every interesting thing is" — and the tools have never been "as cheap and accessible... as they are right now."

2. Scaling walls: "ridiculous" — the limit is economic, and not in view

  • On claims we're hitting scaling limits: "that's a ridiculous statement. I can very clearly see what we're doing now and what we're doing next and I don't see the limit to the scaling laws." Note the shape of his hedge: he intuitively feels there "must be" a limit eventually — unlike some who think intelligence scales "into weird territory" past the human 20-watt skull envelope — but the asymptote he expects is a cost decision: "it's so expensive that we will decide it's not worth spending that next dollar to make this thing one unit smarter." For now, "it's not in the viewfinder."
  • On data: synthetic mix is rising, and high-quality data plus expert human feedback is much more useful in post-training than low-quality data — you can "amplify that into the right set of tokens" worth far more than "the undifferentiated tokens floating around on the web."
  • The open question he'd most like answered: there's no science of data value. "It's very hard to know quantitatively what the incremental value of data is to the quality of a model... most of the assertions that people make are just unfounded," and measurements show "a pretty big disconnect" between what some people think their data is worth and what it contributes. The category error underneath: people treating models "like the world's worst and most expensive databases" — the point is reasoning over information, not recalling it, and "you just need different tokens" to train for that.

3. DeepSeek was a point on a line Microsoft already saw

  • Scott's frame for R1: years of "mind-boggling progress in optimizing the performance of models" mean models got bigger while API calls got cheaper — hardware gives "maybe a 2x benefit price performance every generation if you're lucky," but software gives "a much bigger improvement." R1 is "a point on a line of price performance improvement that maybe was invisible to everyone else" but not to people neck-deep in these systems — "and it's not the last point."
  • The internal reaction, delivered flat: "I was surprised at how interesting people thought that it was. We've had models more interesting than DeepSeek R1 that we chose not to even launch." He credits the work — "good solid technical work," and releasing it open source was "super cool" — the surprise was the public's, not his.
  • What he took from it: "developers want lots and lots of choice... we have to give people more 'how' than we have been." And on China, no hedging: "we should really really really respect the capability of Chinese entrepreneurs, scientists and engineers... how surprised everyone seemed to be — that shouldn't have been surprising."
  • A confessed change of mind on open vs. closed: "when I was a graduate student I was a complete open-source zealot... I've become a lot more pragmatic." His model for the future is search: open-source engines exist, search-as-a-service exists, Bing and Google exist — yet "all of the economics in search go to somebody who stood up a gigantic infrastructure" with its own feedback loop. He expects "lots of both" here too.

4. The 200-year interface breaks — and it's many agents, not one

  • His favorite framing: "we've had one paradigm for using computing devices for effectively 200 years, since likely Ada Lovelace wrote the first program" — you're either a programmer or dependent on one who anticipated your need. AI changes that: it "can understand a thing that you want your computing device to go do for you and figure out a way to make that thing happen." Timing hedged precisely: "I don't think this is next year but it's probably not going to be 10 years."
  • The need for anticipatory software shrinks — teams that guess granular user needs, write code, hang it on a UI, and grind on feedback. "You just aren't going to need as much of that anymore." Engineers remain builders of capability infrastructure; the interface surfacing those capabilities "will probably be agents."
  • The contrarian bit: "I don't believe in this one-agent-for-everything sort of theory. I think you'll have a lot of agents" — because product managers will probably have to become domain experts in medicine, drug discovery, "or early round venture investing," setting up the feedback loops by which PMs and users together teach agents their craft.

5. Memory and async dispatch turn agents into coworkers

  • Harry's pushback — we overestimate near-term adoption; will the world's largest companies really run agents in one to three years? Scott's answer: "usage always follows utility." Software-development agents already flipped developers from skeptics to "you will get this from my cold dying fingers."
  • On the no-lock-in worry: "there's no lock-in in search... you can send your next query to a different search engine — and yet you don't." Retention is earned by grinding "every day to make the agent better" — do that well and "they will continue to choose you."
  • The missing piece: "they are conspicuously missing memory, which makes them awfully transactional." Memory "is going to get a lot better over the next year or so," enabling abstraction and compositionality — solve a problem once, record it, never re-derive from first principles. He also hopes for "more asynchronous things happening over the next 12 months": dispatch the agent and let it work unattended. The plot to never lose: 5-second tasks → 5-minute tasks → "increasingly beefy work... the same way that you would to a coworker." His concrete bar: a 5 a.m. agent that digests overnight email and drafts urgent replies "while I drink my coffee."
  • On agent skeptics: "I don't know what prize you win for being skeptical about something if you're not going to go do something about it" — citing a colleague's book title, There's No Prize for Pessimism. Harry's addition, via likely one of the Collisons: "pessimists are right and optimists make money." Scott: "they are not wrong."

6. 95% AI-generated code — abstraction rises, authorship stays human

  • The number: in five years, "95% is going to be AI generated... very little is going to be line-by-line human written code. Now that doesn't mean that the AI is doing the software engineering job" — authorship "is still going to be entirely human." The analogy, as told: he wrote his first program at 12 (he's 52, "41 years already"), in the assembly-to-high-level transition "there were some old farts who would say you're not a real programmer if you don't know how to write in assembly language... nobody talks about that anymore." Same with GUI builders emitting "a crap ton of boilerplate."
  • The elite still go deep: "the extraordinarily good programmers... understand all the way down" and will "spelunk down into the lower levels of abstraction" when AI-emitted code breaks. Is everyone a programmer in a Bolt/Lovable world? "I think so" — but "the world's hardest computational problems" still need computer scientists, now insanely well-tooled.
  • Team structure: "I'm hoping it will get easier for small teams to go do big things... you can do a lot with 10 really great super motivated engineers with really powerful tools." His internal goal at Microsoft: "I don't want there to be any space between an engineer's ambition... and their ability to go try it."
  • His mortal enemy after 20 years leading engineering teams: tech debt — "just like financial debt, it carries interest." AI can turn that "very zero-sum problem" non-zero-sum; an MSR lab started about a year ago whose whole mission is "eliminate tech debt at scale." The year-one learning generalizes: "there's a bigger gap than there was even two years ago between what the most capable frontier models can do and what they're being used for."

7. Quick-fire: Anthropic respect, the histogram rule, and "are we going fast enough?"

  • Competitor he most respects — Google, Anthropic, or likely Meta: "maybe Anthropic... I think Dario's doing a good job." On physical constraints: since GPT-4 "we have been running a thousand miles an hour building infrastructure... literally going as fast as is possible" — the binding limits are "the rate at which concrete can be poured and power grids can be augmented."
  • Best advice he ever got: picture competence as a histogram from idiot to genius — with great effort you move something "one maybe two buckets to the right," so "all of the time you spend trying to get to mediocre you are not spending doing the things that you're a genius at." His own conceded weak spot: bureaucracy — "I could probably be a very mediocre bureaucrat if I wanted to be." From likely Satya Nadella, the two-part leadership principle: "create energy and produce clarity."
  • The science-fiction prediction he holds as fact: "frontier models are probably better health diagnosticians than your average GP" — already — citing his own family's inadequate care in rural Central Virginia; the world needs to "wake up to the fact that they're good so that we can go deploy this stuff."
  • The question he is not often asked: "are we going fast enough?" His answer — no. What faster looks like: invest "super heavily in education" so every child feels these tools are built for them, and point incentives at deploying AI wherever "we currently think there's scarcity" — healthcare, climate, education. "Let us go invest in that."
Kevin Scott

This is the best time to be alive if you have an entrepreneurial spirit. I can very clearly see what we’re doing now and what we’re doing next, and I don’t see the limit to the scaling laws. I don’t believe in this one-agent-for-everything sort of theory.

I think you’ll have a lot of agents. The reason I think you’re going to have a lot of agents is because your product managers are probably going to have to be domain experts. The agents will definitely be less transactional and less session-oriented going forward.

Harry Stebbings

Kevin, I am so excited for this. I was just telling you, I was listening to you and Dwarkesh on my run. I don’t think I’ve ever run quite as fast, which clearly means the conversation was brilliant.

1. Where is Enduring Value in a World of AI

I’ve never done a 10K so fast. I wanted to start with a super-easy question, which is: my job as a venture investor is to try and determine where value lies in different given moments. I look at the world today and, for the first time in quite a long time, Kevin, I don’t know. Where does value lie sustainably in this next generation of AI, do you think?

Kevin Scott

I think the thing that you just described, which is that all of a sudden things have gotten a little less clear than they have been, is exactly the thing that happens at the beginning of every big technological paradigm shift and every new cycle that’s driven by it.

It was super confusing in the early days of the internet, and I think it was super confusing in the early days of mobile, where everybody had these ideas about what was going to be valuable, and very few of those ideas were actually the durable ones that proved all the way through.

Harry Stebbings

In those moments of transition, where there is this confusion, what have you learned is the right action to take? Is it to be active, to iterate and learn, knowing that you’ll make so many mistakes that you regret, or should you sit on your hands and watch others make those mistakes?

Kevin Scott

You definitely shouldn’t do the latter. This is the best time to be alive if you have an entrepreneurial spirit.

The thing I think you have to do in these moments is not forget the things that you’ve learned from past moments about what works. It’s not, “Okay, well, here’s the specific thing to do.” It’s about how you go about doing the exploration that you just described.

Product matters. I’ve been saying this for the past couple of years: models aren’t products. Everybody was just so fascinated by the infrastructure itself, and this is also a characteristic of the beginning of these cycles: you have technical people who get swept up in the technical bits, and they kind of forget that the only thing that really matters is making good product.

That’s where we’re at right now. You have to make good product, and you have to have ideas and conviction. Then you have to go get stuff done really fast so that you can see whether you’re full of crap or not about the conviction that you have.

You have very few patterns at the beginning of a cycle to snap to. You’re not looking at someone else’s success and saying, “Okay, well, I’m going to do that, but just a little bit better.” You’re trying to figure out something completely new, and the only way to figure that out is to launch stuff, gather data and iterate. You have to be super, super brutal with yourself about what you’re seeing. You can’t love your idea so much that you overlook what you’re seeing in the data and the feedback that you’re getting.

Harry Stebbings

You said that models aren’t products. I just had Andrew—likely from Cerebras— on the show, and I asked him this question: if we think about compute, or hardware, then we think about models, and then we think about apps, where does the value lie? Naturally, he said compute.

When you think about that three-pronged tier of value, and you say models aren’t products, if they’re not products, does that mean they’re not valuable?

Kevin Scott

No, no. They’re super valuable, but they’re only valuable to the extent that you can connect them to things that users need via product. In the limit, I think product is the most important thing.

If you build good models, good infrastructure around models, efficient compute and all of these other things, you’re going to get lots of ability to monetize all of those things. As people build those products, they will need to consume your platform and your infrastructure, and all of that’s good.

But most of the value has to be in the products. We don’t build infrastructure just for the sake of infrastructure. We build infrastructure so people can make products.

Harry Stebbings

This is a leading question. If you think about those products, who benefits most? Is it startups that are able to integrate new technologies very easily from the bottom up, starting from nothing, or is it Microsoft integrating AI into incredible distribution already, or Google doing the same? Who benefits most in that respect?

Kevin Scott

Again, if you look to past cycles, you’ve got a pretty good mix of where value gets created across startups and new ventures and existing enterprises.

I think everybody’s kind of doing the same thing: you’re trying to discover the new. If you’re a big company like Microsoft, with a long tradition and a bunch of successful things already in the market, the thing you’re trying to do is figure out which of the things that you already know super well, and which of the customers that you’re already serving super well, you can serve with this new set of capabilities that you can provide.

Hopefully, you know that I run, among other things, Microsoft Research. I also have a charter to say, “Hey, can we go try to shine flashlights in places that no one else has shown them before?” We try to discover some super-disruptive, brand-new things.

That’s the job of the startup ecosystem as well. I’m also an angel investor, I advise startups, and I’ve worked at startups, so I think it’s really important that you’ve got lots of people hunting for those interesting new things.

I have super-high conviction on that in the AI platform transition that we’re going through right now, because it’s impossible for any entity, like Microsoft or any other big company, to have enough imagination and enough perspective to know what every interesting thing is. Having this vibrant ecosystem, with lots and lots of people exploring where value exists, is incredibly exciting and necessary.

I also think that there has never been a moment where the tools, infrastructure and platform have been as cheap, accessible, available and easy to use as they are right now. It’s super easy to pick stuff up and get cracking.

2. Why Scaling Laws are BS

Harry Stebbings

I was listening to your show, as I mentioned earlier, and you pushed back on the idea that we’re reaching scaling laws and this kind of asymptote of efficiency or effectiveness, when many people suggest that we are hitting a scaling asymptote soon. First, why do you think we’re not?

Kevin Scott

That’s a ridiculous statement. I can very clearly see what we’re doing now and what we’re doing next, and I don’t see the limit to the scaling laws.

If you’re just thinking about the raw capability of the models, and how well you can condition them to reason over increasingly complicated things, I’m sure that at some point there will be a limit. I intuitively feel like there must be.

There are some people who don’t believe that there’s a limit. The limit that human beings have on intelligence is that you’ve got so many neurons packed into your skull and about a 20-watt power envelope, and that’s the limit. Some people believe that, with AI, there is no such limit, and things will continue to scale into weird territory.

I don’t necessarily believe that. I believe we will get to some point where we’ll hit a scaling asymptote, and there will be diminishing marginal returns. It will be so expensive that we will decide it’s not worth spending that next dollar to make this thing one unit smarter, because we haven’t figured out how that translates into something useful for the people who are using the tool.

I think that point will come. I just don’t see it yet. It’s not in the viewfinder right now.

Harry Stebbings

When we think about the 3 core elements that make up efficiency in this respect—data, compute and algorithms—when we drill into data, what are your biggest observations on data efficiency? How do you think about the importance of quality of data versus quantity, and synthetic versus human data, today?

Kevin Scott

The mix of synthetic data is going up. High-quality data is becoming much more useful, especially in the post-training parts of the model-production pipeline, than low-quality data.

I think we’re clearly at the point now where, if you have the right infrastructure, super-high-quality data and super-high-quality expert human feedback, you can amplify that into the right set of tokens for training bigger and bigger models. That stuff has way more value than just the undifferentiated tokens floating around on the web.

Harry Stebbings

What questions about data and its usage do we not know the answers to that would be most helpful to know?

Kevin Scott

There’s a super-interesting thing right now that we don’t have around assessment. It’s very hard to know, quantitatively, what the incremental value of a token of data is to the quality of a model generated using that data in its training.

If you’re asking, “Okay, I think my data is super valuable, and if this gets used in a model, it’s going to make the model better,” most of those assertions are unfounded in any kind of science. The measurements we do have show that there’s a pretty big disconnect between what some people think valuable data is and how valuable it actually is to producing capabilities and models that are legitimately useful.

Most of what’s legitimately useful is models. People think of models as repositories of factual information, and they’re treating them like the world’s worst and most expensive databases. That’s not super useful. We’ve got search indexes and databases, and those things are plenty good enough for retrieving information.

What you want models to be able to do is reason over information. If you give them access to information, how well are they able to reason over a set of information to go do something that’s useful for you? You need different tokens for training a model to make it good at reasoning than you do for making it a recaller of facts.

3. In 10 Years Time: What % of Data Usage will be Synthetic

Harry Stebbings

It’s so funny you said that about reasoning, because it made me think about inference. I get really annoyed by the word “inference.” I wish we’d just delete it and call it “usage.” There’s training and there’s usage.

You’ve been very clear about the transition in emphasis and importance from training, which has had the focus over the last few years, to inference. What are we not talking about or seeing in inference that we need to spend more time thinking about?

Kevin Scott

The thing that most people miss, although the DeepSeek R1 launch a few weeks back clued everybody into it, is that we just have an incredible track record over the past handful of years—and it’s many years now—of repeated, year-over-year, mind-boggling progress in optimizing the performance of models.

The performance of inference is getting better and better and better. Over time, the models have gotten bigger and the API calls have gotten cheaper. A little bit of that is because you get maybe a 2x benefit in price performance from hardware every generation, if you’re lucky, but you get a much bigger improvement in price performance from all of the things that you’re doing in software.

There’s a ton of work happening there. The DeepSeek R1 stuff, which was good work, is a point on a line of price-performance improvement that maybe was invisible to everyone else, but not invisible to the people who are neck-deep in optimizing these systems. It’s not the last point. It just marches on.

Harry Stebbings

What was the internal sentiment when it came out? To what extent were you surprised by the public reaction?

Kevin Scott

We’ve had models more interesting than DeepSeek R1 that we didn’t even choose to launch. I was surprised at how interesting people thought it was.

They did good work, so don’t get me wrong. It was solid technical work, and it was super cool that they chose to release the thing and make it open source. It was really interesting seeing how the public reacted to what they did.

Harry Stebbings

Is there anything that you learned from how the public reacted to the release that you’ll take with you in your own releases?

Kevin Scott

Even when you’ve made it as easy and cheap as humanly possible for folks to go do something, they still have super-strong preferences about the how. We’re paying very close attention to that.

We have to give people more “how” than we have been, because I think developers want lots and lots of choice.

Harry Stebbings

What did you believe that you no longer believe, or what have you changed your mind on, in the last 12 to 24 months?

Kevin Scott

When I was a graduate student, I was a complete open-source zealot. As I’ve gotten older, I’ve become a lot more pragmatic. It’s probably more important for me to make a set of pragmatic decisions about how I’m going to build these things rather than singly optimizing for my curiosity.

Harry Stebbings

When you look forward to the next 3 to 5 years, or 3 to 10 years, how do you think about the pervasiveness of open versus closed? Which will be more dominant than the other?

Kevin Scott

I think there’s going to be lots of both. Part of it is that we should forget about AI, which is the controversial thing at the moment. It’s an area where industry structure hasn’t settled yet, and we don’t know exactly what it’s going to be.

Just pick a previous example, like search. There are a whole bunch of open-source search-engine projects out there. People who want to have a search feature in their application, or who want to build a search engine themselves, have lots of options. They can grab something open source as a starting point and stand up a product.

They can also load their data into something like Azure Cognitive Search, which is a search-as-a-service platform. Google has one, Amazon has one, and they’re readily available. Then you still have search engines like Bing and Google that are out there.

All of those things exist. All of the economics in search go to somebody who stood up gigantic infrastructure and is running a whole search business with its own feedback loop.

I think we’re probably going to have similar sorts of things happening here. For the infrastructure layer, you’re going to have lots of open-source infrastructure products, and people are going to use them in lots of different ways.

4. How Will AI Agents Evolve Over the Next Five Years

You’re also going to have a lot of people who don’t want to stand up their own infrastructure from scratch, or take an open-source project and build it out where it’s lacking for the things they need it to do. It’s good to live in a universe where you have both of those things.

Harry Stebbings

You mentioned earlier the centrality of product. Taking that into account with the current conversation, how should we think about whether chat is the right UI for the next paradigm of this product realm? OpenAI and ChatGPT have made it the default. To what extent do you think it is the right default, and how will we see that change?

Kevin Scott

I think it’s a reasonable step in the right direction. The thing I’ve been saying for a few years now is that one of the most interesting things happening with AI is that we’ve had one paradigm for using computing devices for effectively 200 years, since likely Ada Lovelace wrote the first program.

If you want a computing device to do something for you, you have to be a programmer yourself, which is a pretty high barrier to entry for a lot of people. Or you have to rely on the fact that a programmer has anticipated some need that you might have and packaged up a piece of software into an application that you’re able to run.

Those are the 2 ways you can get a computing device to do something for you—until now. The thing that changes with AI is that it can understand something you want your computing device to do for you, and it can figure out a way to make that thing happen. You don’t have to be a programmer.

It’s a profound change, because it basically means—and I don’t think this is next year, but it’s probably not going to be 10 years—that this whole notion of teams of people whose job is to anticipate a bunch of very granular user needs in some narrow space, write a bunch of code, figure out how to hang that code onto some user experience, and hope that they’ve done a good enough job anticipating the needs and designing the user interface in the right way will change.

They’ve gotten all of the code right, and they just grind away on figuring out what that feedback loop is. That’s going to change. You just aren’t going to need as much of that anymore.

What you’re going to need instead doesn’t mean that it completely goes away. You will still need all of the capabilities that these applications provide, but you’re probably going to want some kind of agent actuating those capabilities on your behalf, rather than having to do this weird impedance matching that we have right now between how a user has a set of expectations and how a product team has imagined what those expectations are.

Harry Stebbings

Is there a role in engineering or product teams today that you think, in 20 years’ time, people will look at and say, “What? You had secretaries who typed out voice-recorded notes from a doctor?”

Kevin Scott

You’re still going to have to have people who build capability infrastructure—make this thing happen in the real world, provide access to this weirdly situated repository of information, and build the capabilities that people will need.

The user interface that surfaces those capabilities will probably be agents. Product managers—I don’t believe in this one-agent-for-everything sort of theory. I think you’ll have a lot of agents.

The reason I think you’re going to have a lot of agents is because your product managers are probably going to have to be domain experts: people who deeply understand something like medicine, drug discovery, early-round venture investing, or just pick your thing.

They will have to deeply understand the idiosyncrasies of that domain, and they will have to help set up the feedback loops that help agents assisting people with those tasks do their jobs better and better.

It will be a combination of the product manager and the users of the agents teaching the agents how to be better and better at the things you’re trying to get them to assist you with.

Harry Stebbings

I often think that we overestimate adoption in the near term, or in a year or the short term, and underestimate it in the long term. When I look at the hype around agents, I share the excitement, but I question the immediate adoption—or the expectation that some of the world’s largest companies will be using agents in the next year or even 3 years.

To what extent do you think I’m right, or to what extent do you think this wave is different, given the distribution of a company like Microsoft?

Kevin Scott

I think usage always follows utility. You make useful things, and they get used a lot. Clearly, with software-development agents, we’re getting a lot of adoption right now.

We’ve gone very quickly from developers being skeptical about these tools to, “You’ll get this from my cold, dying fingers. This is one of the most essential tools in my toolkit, and I will never give it up.” The agents are becoming more and more powerful, and I even see—

Harry Stebbings

Can I ask you to what extent there’s lock-in there? When I look at them, and when I speak to people about them, you’re right: there’s user love, but everyone says, “There’s no lock-in. I’d happily switch to the next person tomorrow.” To what extent does that mean it’s valuable?

Kevin Scott

There’s no lock-in in search. You can send your next query to a different search engine than the one you’re using right now, and yet you don’t.

The reason that’s true is that it’s our job, building these agents, to grind and grind and grind, and every day try to make the agent better and better and better, and do more and more and more of value for our users. If you do that, and you do it well, they will continue to choose you.

Harry Stebbings

When you think about a 5-year time horizon, what will the interaction model look like between humans and agents?

Kevin Scott

The thing that’s missing right now with our agents is that they are conspicuously missing memory, which makes them awfully transactional. Even in the places where agents have memory, it’s a pretty limited form of memory.

5. What is the Bottleneck Today: Data, Compute or Algorithms

I think one of the things that’s going to happen, because I know lots of people are working on it right now, is that memory is going to get a lot better over the next year or so. As you’re using an agent, it will remember more and more about your past interactions with it, and it will be able to conform itself more and more to your preferences.

It will be able to do things that we do very naturally. You solve a problem once and record the solution to that problem, and then you don’t solve it from first principles over and over and over again.

Memory even gives you the ability, with these agents, to have some kind of abstraction and compositionality, where you can build up more and more powerful ways of doing things inside the agent over time because it’s remembering the past things that it’s done and learned.

I think the agents definitely will be less transactional and less session-oriented going forward. I hope we get more asynchronous things happening over the next 12 months. Right now, it’s very interactive: you go to your agent, send a prompt in, it goes and does something immediately, and gives you the response back. It’s, “Yep, I’ve done it.”

I think there’s going to be more of you dispatching your agent to go do something, and it goes and works while you’re not paying attention to it. The thing you want with agents, by the way, is that we should never lose the plot on where we’re going.

The first generation of agents are good at 5-second tasks, and the generation after that will be good at 5-minute tasks. What we’re going toward are things that you can delegate increasingly complicated tasks and increasingly substantial work to over time, the same way that you would to a coworker.

That’s how I think about the future. That’s what everybody’s going to want, and that’s where the capabilities are headed. How do you think about building product around where the future is almost certainly going to be, and what do you need to augment these systems with to allow them to do more of this thing that is ultimately what we want?

You don’t want a thing that’s just a good email summarizer. You want something that you can tell, “I get up every morning at 5:00 a.m.”

At 5:00 a.m. every morning, digest all of the email that came in overnight, draft responses to anything urgent, and show them to me while I drink my coffee. That ought to be an entirely possible, doable thing.

Harry Stebbings

Is there anything that many people think about agents that you often hear that you think is wrong?

Kevin Scott

I often think skeptics who say, “Oh, this is hard or impossible,” are probably wrong. But I’m not unique in thinking they’re wrong; there are plenty of optimists out there who think the technology is going to get more capable. It’s fine to have skeptics. I don’t know what skin in the game they have.

I have a colleague who wrote a book called There’s No Prize for Pessimism, and there really isn’t. I don’t know what prize you win for being skeptical about something if you’re not going to go do something about it.

6. The Future of Software Development

Harry Stebbings

Well, I think it’s John or Patrick Collison—I’m never quite sure which one—but likely one of the Collisons said, “Pessimists are right and optimists make money.” They’re not wrong.

We mentioned software development being one of the most widely adopted usage mechanisms that we’re seeing today. When we look forward 5 years, what percentage of net-new code do you think will be AI-created versus human-created?

Kevin Scott

95% is going to be AI-generated. I think very little of it is going to be line-by-line human-written code.

That doesn’t mean that the AI is doing the software engineering job. I think the more important and interesting part of authorship is still going to be entirely human. What does authorship mean in a world where you’re not the input master, you’re a prompt master?

It’s just raising the level of abstraction. Are you a programmer? No. We’ve accepted that for the past 35 or 40 years.

When I was a kid, I wrote my first program when I was 12. I’m 52 years old, so I’ve been doing this for 41 years already. By the time I was 12, which was in the ’80s, you were mostly writing your code in a high-level language. But the thing that runs on the machine is not a high-level language. It’s not even assembly language; it’s some machine encoding of assembly-language instructions that run on the hardware.

Nobody bemoans the fact that they’re not writing in machine code. There was a period when this was true, during the transition from assembly-language programming to high-level-language programming, when some old farts would say, “You’re not a real programmer if you don’t know how to write in assembly language. That’s the only real coding, and that’s the right way to do things.” Nobody talks about that anymore.

I think this is going to be a similar distinction. In the same way that GUI builders and things that have been around for 20 years have changed how we work—for example, when you’re designing an iPhone application in Xcode, you don’t write all of the code. You drag a whole bunch of user-experience elements around on the screen, and the system emits a huge amount of boilerplate code for you—this is, in my mind, just the same trend.

We’re raising the level of abstraction. We’re changing the interface that programmers use to communicate to the machine: “Here’s a problem that needs to be solved.”

One thing that is true is that the extraordinarily good programmers right now, even when they’re using tools at a very high level of abstraction, understand all the way down. If something’s broken, you can go into the machine code, look at the boilerplate that your development environment is generating, muck around, and figure out what’s going on.

The same will almost certainly be true when you’ve got mostly AI-generated code. The very best programmers are going to be able to say, “The thing emitted this, but something’s off. Let me spelunk down into the lower levels of abstraction.”

Harry Stebbings

Is everyone a programmer in a world where you have Bolt or Lovable, which allow you to create simple websites with pure prompts?

Kevin Scott

I think so. But it also doesn’t mean everybody is solving the same sorts of programming challenges.

Again, this is raising everyone’s level, so it makes everybody a programmer. You no longer have to get someone to make a website for you. But if you’re trying to solve the world’s hardest computational problems, I still think you’re going to need computer scientists. They’re going to use these tools insanely well to solve problems that were just harder than they could solve before.

Harry Stebbings

Will the structure of engineering teams be fundamentally different in the future?

7. The Thing That Most Excites Me in AI is Tech Debt

Kevin Scott

I think so, but maybe not in the way that people think. I’m guessing, and I’m hoping, that it will get easier for small teams to go do big things. The reason that’s important is that I think small teams are just faster than big teams. You can do a lot with 10 really great, super-motivated engineers and powerful tools.

Harry Stebbings

What would you most like to do but, because of scale, decision-making, or whatever it is, you’re not able to do? Scale is usually tough for 2 reasons in a technology company, but it does mean that sometimes you’re slower than you would like to be. Sometimes slow is necessary, but sometimes slow is a side effect of being big. Where have you been slow where you would like to be fast?

Kevin Scott

I want to be fast all the time. I want more product happening.

There are things that can’t go faster than they go because the laws of physics are attached to them. We’ve been running 1,000 miles an hour building infrastructure over the past 2.5 years since GPT-4, and we are literally going as fast as is possible to go. You just wish you could change the rate at which concrete can be poured, power grids can be augmented, and all of that sort of stuff. I wish it could go a little bit faster.

What I would love to be able to do in an ideal world, at Microsoft and everywhere else, is eliminate any space between an engineer’s ambition for what they want to do or a good idea they want to try and their ability to go try it. A lot of our internal use of AI right now is focused on figuring out how to enable that for all of our people at Microsoft.

There’s another thing, too. If you’ve ever managed any size engineering team, one of the nastiest problems you have, which has traditionally been very zero-sum, is the accumulation of tech debt.

At some point, you’re almost always confronted with a painful tradeoff: “I’ve got to get this thing out, which means I can’t quite get the technical bits of it in exactly the state that I want them to be. I’m going to launch now, and I’ll fix this thing later.” The minute you’ve done that, you’ve incurred technical debt.

Technical debt is just like financial debt: It carries interest, and you have to pay the interest payments. If you don’t pay down the tech debt, plus the interest, you’ll be in trouble at some point because it accumulates to a large extent. Then things start failing in your infrastructure.

One of the things I’m absolutely most excited about with AI is that I think we can turn this very zero-sum problem of tech-debt accumulation into something non-zero-sum, where you don’t have to make those trades in the same way that you have in the past.

We started a big research initiative at Microsoft Research about a year ago. The whole mission of the lab is to eliminate tech debt at scale using these new AI tools. It’s super exciting stuff. I’ve been leading engineering teams for 20 years now, and tech debt is just my mortal enemy.

Harry Stebbings

What have you learned from doing that program over the last year?

Kevin Scott

That the AI tools are more capable than people think they are. This is the thing in general: I think, honestly, right now there’s a bigger gap than there was even 2 years ago between what the most capable frontier models can do and what they’re being used for.

8. Quick-Fire Round

Harry Stebbings

Kevin, I could talk to you all day. I’d love to move into a quick-fire round, if that’s okay. Let’s start with a tough one: Which competitor do you most respect—Google, Anthropic, or likely Meta—and why?

Kevin Scott

If I had to pick one, maybe Anthropic.

Harry Stebbings

Out of interest, why?

Kevin Scott

I think Dario’s doing a good job.

Harry Stebbings

What was the best advice you’ve ever received?

Kevin Scott

I had a mentor one time who told me that you can imagine an individual’s or a team’s competencies on a histogram, where the bucket all the way on the left is “idiot” and the bucket all the way on the right is “genius,” with the middle buckets being mediocre or average.

Their assertion was that you could take everything that you do and everything that you’re trying to do and assign it to one of those buckets. The mistake people make is that, with great effort, you can take something and move it up 1, maybe 2, buckets to the right on the histogram.

The mistake people always make is focusing on trying to improve at the things they’re worst at. If you believe this theory, the best you’re ever going to do if you’re an idiot at something is get mediocre at it. All of the time you spend trying to get to mediocre is time you’re not spending doing the things that you’re a genius or very good at.

I think that’s very good advice because the thing about everything that’s worth doing is that you probably have to do it with a team. It’s super easy to construct a team where you complement people.

Harry Stebbings

What are you bad at that you’ve consciously decided not to get mediocre at?

Kevin Scott

I’m bad at so many things. I’m super impatient with bureaucratic things. I hate budgets and facilities and all of the mechanical parts of being an engineering leader. Bureaucratic things just bug me, and I could probably be a very mediocre bureaucrat if I wanted to be. I’m just terrible at it.

Harry Stebbings

I love that. I’m the same. Delegation is the secret to life.

9. Leadership Lessons from Satya Nadella

Likely Satya Nadella is one of the most incredible leaders of our generation. What have been your biggest lessons from working so closely with him and seeing him operate?

Kevin Scott

I think his core leadership principle is that you have to simultaneously create energy for people and produce clarity.

You really do have to make sure—and he’s very good at this—that the energy of conversations is positive and that people walk out of reviews and conversations, and anything that we’re doing, carrying energy with them that’s going to help them go do the hard thing ahead of us.

10. DeepSeek Evolution: Do We Underestimate China

At the same time, you can’t just produce a bunch of rah-rah and not clarify for folks what the most important things are.

Harry Stebbings

We mentioned DeepSeek earlier. Do we underestimate China’s ability in AI?

Kevin Scott

I don’t think we should. We should really, really, really respect the capabilities of Chinese entrepreneurs, scientists, and engineers. They are very good. If you are underestimating them, you shouldn’t.

Maybe some people did. That’s another interesting thing about the DeepSeek reaction: how surprised everyone seemed to be. “Oh my God, this is coming from China.” That shouldn’t have been surprising.

Harry Stebbings

What’s the crazy AI prediction that most people would call science fiction that you believe to be true?

Kevin Scott

It’s already the case that I think the frontier models are probably better health diagnosticians than your average GP. It’s a good thing to realize and act on as quickly as possible because we have a whole world of people who have inadequate access to high-quality health care, including my own family in rural Central Virginia, where it’s just not good.

There are a bunch of things like this where the models are already really good. We basically need the whole world to wake up to the fact that they’re good so that we can deploy this stuff. We should deploy it because the thing we really care about is the good of the public, not trying to sustain some status quo.

Harry Stebbings

Kevin, a lot of people ask you a lot of questions—team members, journalists, you name it. What question are you not often asked that you think is an important question that you should be asked?

Kevin Scott

I don’t know. Are we going fast enough?

Harry Stebbings

Do you think we’re going fast enough?

Kevin Scott

No.

Harry Stebbings

Is it possible to go much faster?

Kevin Scott

Yeah, I think so.

Harry Stebbings

How could we go faster?

Kevin Scott

I think we could go faster in a bunch of different ways. The thing that I would want in my ideal world is for us to invest super heavily in education. I would love to see every child feel as if these new tools that we’re building right now are for them, accessible to them, and expressly built for them to accomplish the things that they think are most important.

I want billions of human beings taking all of this creative energy that we all have and doing the most amazing thing with the best tools that they possibly have. I don’t want anybody feeling constrained by anything.

I would also love to make sure that, across the public and private sectors, we’re creating every incentive we possibly can to deploy these tools to produce good. Whether it’s health care, climate change, education, or whatever your thing is where we don’t think we have enough of what everybody thinks there ought to be, if I had a piece of technology that could create abundance in this thing where we currently think there’s scarcity, let us go invest in that.

Harry Stebbings

Kevin, I’ve so enjoyed talking to you. I really appreciate your tolerance with the wide range of questions and future pontifications. You’ve been fantastic, so thank you so much.

Kevin Scott

Thank you for keeping me company on my runs. This has been awesome.

Harry Stebbings

You’re very welcome.

Kevin Scott

Thank you for having me.

Kevin Scott, CTO @ Microsoft: An Evaluation of Deepseek and How We Underestimate the Chinese | BidClub