Former Microsoft Executive Explains Where We Are in the AI Cycle w/ Anish Acharya & Steven Sinofsky

Anish, Steven, we were having such a good conversation offline that I wanted to get this on the podcast. There are a few topics we wanted to discuss. First, we were all fascinated by Andrej Karpathy’s talk at Startup School. Steven, what did you find so interesting, or what were your takeaways or reactions from it?

Well, I totally loved the talk. He did an unbelievable, philosopher-king version of where we are, and I found his metaphors really compelling. In fact, what I might do is take it even further back and say that, since he used an analogy of where we are in computing—with lots of people talking about the Windows 3 era and stuff like that—and having lived through all of them, I tend to think we’re at the 64K IBM PC era of the microcomputer.

The reason I think that is actually a technical one: we’re at the point where people are still trying to figure out how everything works. All the coding and all the energy are working around these very basic, practical problems. With the PC, it was like, “Okay, we have 64K of memory, our programs are all too big, we have no display,” and all these problems. With AI, people are like, “It’s going to replace search, it’s going to replace Excel, it’s going to replace all these things,” but it doesn’t add up very well. It gives you a lot of errors. The thing that you say it’s going to do, it just doesn’t even do yet. So, I feel like we’re at a point that is so early, and he did a fantastic job of making that arc.

You know, the thing that struck me the most was that he talked a lot about our relationship with this new tool. In a sense, we want to use it in the same way that we’ve used all the other computing tools and technologies we’ve used in the past, but he really talked about this inversion of the relationship with LLMs—their being like people or spirits, with jagged intelligence. To me, that meta-point he made was one of the most interesting: we have to relearn how to use this type of tool before we know how to be productive with it.

And I think tools are a super interesting point, because the talk is anchored in tools, but the world itself is anchored in tools. The early stages of a platform are always about tools, and so you can get a little confused. Right now, of course, he was talking about vibe coding clearly, because he pioneered the term, invented the concept, and is living it. It’s very interesting because I actually think coding is one domain that always works best early in a platform, because all the customers of the platform are developers, and they’re going to make their tooling kind of work and come along.

But I really think that the most interesting thing for me, what’s being underestimated in the near term, is vibe writing. It seems weird to say anything with AI is underestimated, because Lord knows that’s not where we are. But vibe writing is so here: if you’re in college, you’re already vibe writing, and businesses are still working through, “Well, can we use this? This doesn’t seem appropriate.” That’s a thing I’ve definitely lived through with word processors. With the microcomputer, I had to get permission from the dean in college to use a computer to write papers. It is really, really no different than when calculators showed up and all of a sudden just doing math homework involved using a calculator, and people were like, “Well, you’re not going to know how to do math in the future,” and it’s like, “I won’t have to know how to do math. That’s like the whole point of the tool.” I have a power drill, so I do not know how to use one of those Amish drill things. The world moves up the stack, and that’s where we are. It’s just super exciting.

What I love about the vibe-writing concept, actually, is that it’s a place in which full autonomy can be fulfilled today. You can ask the model to vibe-write something really detailed and compelling, and it’ll do a great job. Whereas with vibe coding, I think there are a ton of constraints as to what the model can actually do versus what it can conceptually do. Understanding those boundaries and constraints is going to define a lot of the text-to-code stuff for the next 2 years.

Well, I push back a little bit on that. I of course agree on the coding side, and I think one of the things developers do early in a platform is they love to tell you that they’re doing something every day and it’s working, but it actually just isn’t. That’s just what happens early in a platform: they tell you all these things that they say are easy, and they’re actually not. They spend 18 hours struggling with something that didn’t work.

But on the vibe-writing side, it also hits a point that I think is so important: yeah, you can prompt it to spew out a bunch of stuff, but if you have a job and your salary depends on you submitting that, or you’re a student and your grade depends on you submitting that, it actually better be right. You can’t just say, “Look, I vibe-wrote this, and here you go.” I think people don’t get confused when it comes to math. Everybody knows you have to check the math: you ask it to do a table and then add a column that does math. But we’re going to see endless, endless human-wasn’t-in-the-loop vibe-writing things.

With programs, you can’t really see that right away, because in order to actually distribute it or get someone to use it, you had to at least fix the initial bugs. We’ll only see them later, when there are security bugs, authentication bugs, passwords stored in plaintext, or a zillion other problems that are going to happen from vibe coding.

I mean, in a sense, we’ve seen this already, right? We saw a bunch of lawsuits citing case precedent from cases that don’t exist. So maybe this is actually the operative point: there’s full autonomy, there’s partial autonomy. Maybe partial autonomy in writing is moving us from writer to editor, but you still have to be the editor, right?

We should also give him credit. Many people have talked about this, but he did a fantastic job using the Iron Man analogy of how we’re going to have autonomy, partial autonomy, and a slider to control what you want. I actually think that’s a fantastic analogy and a way of thinking about it that gives you a very clear picture from the movies. You live through the movies, and at the same time, people are very, very aggressive on their timeline for agents. There’s a very, very long history of trying to automate things that turn out to be very, very difficult to automate.

I think he did a fantastic job. He said people are talking about “the year of agents.” Yeah, that’s a good consultant phrase. We’re just—like he said—in the decade of agents, and it’s going to take a decade for things to be anywhere near living up to agentification as a meme.

You know, it’s an interesting point. I think a lot about agents as applied to financial services. I think there’s a set of problems in financial services that are high-friction, low-judgment. For example, when I want to refinance my personal loan, I don’t really feel attached to any specific brand of personal-loan provider. I just want the cheapest rate, so it’s actually a very low-judgment decision. But researching and applying for a personal loan is a high-friction process. That’s something I would love to delegate to an agent. I think it can do a nice job.

Whereas doing my taxes—wow. Steven, how much risk do you want to take on your taxes? How many things do you want to report or not report? That requires an enormous amount of judgment, and of course it’s also high friction. So when I think of the 2x2 of where automation is going to come first, I think a lot about high-friction, low-judgment.

I want to build on that, because I actually think it’s super important to also consider that, for anyone to offer alternatives to the market, there has to be an ability to differentiate, to explain, and so on. You end up with this kind of thing where, “I just want the cheapest flight.” Of course, for 20 years all the flight searches and stuff have worked on the cheapest, but it turns out that’s not actually what you want. Plus, a lot of people want to intervene in presenting your choices to you.

I don’t like this idea that all choice in life is going to be reduced to some headless API. I don’t understand—people have to go build that and make a living building those things. To your example of refinancing a home, the only reason that it can exist as a search problem today is because the different people who want to refinance you can target you with an ad and attract you as a customer and differentiate themselves on that offering. If you can’t do that, then your ability to actually automate that task isn’t going to exist, because there’s no economic incentive to just be, “Hi, I’m the headless, faceless, nameless, low-price mortgage lender.” That’s not really a business. There’s nothing there. Just like headless, faceless, nameless food isn’t a thing.

It doesn't show up in a white can labeled “food,” and then you consume it and you're okay: all good, I have food now.

Well, maybe Soylent, but yes.

No, in the future—in the dystopian future of “Repo Man.” That's where we end up. But that's not going to happen. I want the cheapest flight as long as it's not on Spirit Airlines, right? I want the cheapest flight, but I'm traveling with a family of 3. I don't want to leave at 5:00 a.m. No red-eye. I want miles on this airline. A lot of things don't add up to that.

And I have a—this is a real thing in business. It's a thing on the producer and the consumer side. Consumers really, really want much more choice than they often think they do. Anyone who's bought anything on Amazon knows they complain about the choice, but they really don't want just “phone case” to show up as the thing because it was $6.

I think this is a real throughline through the talk: partial autonomy, jagged intelligence. Karpathy is just talking a ton about the constraints of the technology, which I think is the right thing for us to be thinking through—trade-offs around—as builders. And he does a great job, very much as this philosopher that I love: his delivery, his tone. You really, really don't just go read the summaries. Don't read a post. Go just watch the video immersively.

Totally. Well, I want to get to automation and sort of employment, particularly on the entry-level side. But first, I just want to ask the broader question. There was this idea of AI plus human—I think it was in chess—that could beat AI for some period of time, and that was kind of the copilot view of the world: human plus AI will have a better product.

Then it turned out—I think it was chess, maybe it was Go, or maybe it was both—that that was a temporary thing, and AIs are just better. Then there's the question as to how much of the world is like chess, where a human plus AI is only better for a certain period of time and then the models get better, or how much the world is like something else, where human plus AI is always going to be better, or we're just always going to want humans to do it.

Look, my view is that in a domain in which you have a formal definition of correctness, the path will be no autonomy, partial autonomy, full autonomy. In domains where you don't have a formal definition of correctness, or where a ton of human judgment is necessary—human choice and human direction—the right product design is not to go all the way to full autonomy.

I would argue that chess and Go do have a formal definition of correctness, so it makes sense that those were fully automated over time. We're back to the early stages of where things are, which means that a bunch of programmers are defining what success looks like. Programmers are very good at “it works or it doesn't work,” or “I just want to automate this,” or “I'm going to reduce your job to a tiny shell script” kind of mentality.

I just look at the world as everything is gray, and everything is much harder than it looks when you don't actually have to do it. Ages and ages ago, I visited a really giant hospital in Minnesota to help them figure out how to use Excel within the medical profession. The doctor just looked at me and said, “I don't think you understand.” He was like, “My job is all uncertain. Every aspect of what I do is uncertain. So adding something that pretends to be certain, like a spreadsheet, to my uncertainty doesn't actually help me.”

And so, fast-forward: I've spent 25 years with a doctor, but that's a different story. There was a story this week about radiologists, and very early in—actually, if you go to ImageNet—everybody was immediately saying, “Radiology is doomed.” There was this idea that you never need to get a skin-cancer biopsy; you'll just take pictures of your mole and it will tell you. Then you find out, wow, there's judgment there—and there's even judgment in doing the biopsy, how to do the biopsy, and then what to biopsy, and all this.

But it turns out the radiologists have fully embraced AI, but they embraced it as no different from the latest MRI technology or the latest software update from GE for a CAT scan. I just think there are so many things like that, and so many jobs are either very, very uncertain, or most of the job is basically exception handling, right? People are like, “Oh, we're going to automate our taxes.” Okay. Taxes are literally a giant cascading set of if and switch statements of exceptions.

And so the idea that you will just automate that—well, you have to know the answer to all the exceptions. If you're going to prompt it with the answer to all the exceptions, then you're doing your taxes manually. It's a very strange thing. Everybody, once you reach a certain income, has to get help from an accountant to do their taxes. The first thing the accountant does is ask you for your tax organizer.

As a software person, I look at that and I'm like, “The tax organizer really, really looks like the input fields of the software you're using. So maybe I could just buy that software and then type it in.” I said that, and he's like, “Well, you're welcome to, but you will go to jail.” He explains, “Because every time I give him a number, it's a whole decision about where to apply it. Does it work?” And I'm like, “Well, you're not really a farmer, so don't fill anything in on that form,” and stuff like that.

People get automation wrong. Automation is extremely difficult. It's exception-bound, it's judgment-bound, and it's all uncertain. A field in which this question comes up a ton is product management. I've had so many conversations with product managers over the last 2 years about the death of product management. It's the end of the field. Why do we need PMs?

I think our developer generation has developed a real resentment toward product managers, which is a different conversation. With that said, I think the product management job is the job of addressing ambiguity—ambiguity that prevents progress from being made. Sometimes it's execution, decision-making, or product design. That will not change.

The nature of business, human interaction, and companies is that these are complex adaptive systems where there will always be ambiguity. I think you'll always need judgment, and you'll always need somebody who looks like a product manager.

Yeah, I think that really gets to the vibe-coding challenge we're dealing with, which is: how fast can we go from text to app? And I think what's so interesting here in the long arc of platform transitions is that we're also having this platform transition happen not just in the open.

We've had that before. Back in the earliest days of computing, these platform transitions happened in user-group meetings, like at the Cumberland Community Center down the street, or in magazines or newsletters, and then with newsgroups, then the internet, and so on. The whole internet was all IRC, and it was all in the open.

But now it's happening on CNN, on the nightly news. Everyone knows about the platform transition that's happening, in particular on social and in Discord. What's happening is you're getting a lot of vibe coding for clout. You're getting a lot of this: “I had an idea, I prompted it, and it worked, and here I am.” At some point I just go, “I'm calling BS on that. That's not a thing.”

And then I sound like an old person because some people think I am.

I don't.

But some people think I am.

I don't either.

It looks like, hey, you're just being old.

Yes.

But then you dig in and you find out, wow, you're prompting. Although it's English-like, it turns out you're just programming, and you're just programming in prose. People are like, “Oh, this is what we're going to do: we're just going to get the model to require a little bit more structure.” And I'm like, “You're writing a new programming language.”

We're just on this path of text to app, and vibe coding is just developing a new language, which is super cool. Lord knows the world is built on programming languages. In the '80s, if you drove slowly past a computer science department where people were trying to get a PhD, they would just invent a new programming language right then and there if you stood outside the building for too short a time.

But we can't lose sight of the fact that the arc of programming has been one of basically overpromising and underdelivering. When I was in college, the theory was that the market was going to need so many programmers that the whole workforce would just be software people, and that never happened. Now here we are: we're not going to need any; they're all just going to go away. I think it was extreme in 1990, and it's extreme today.

I think the big thing is this overpromising at each transition. Even most recently: low-code. Who even says that word anymore? We're not allowed to even mention it. It's always the same thing, which is, yes, if all you're doing is a very straightforward app that looks like all the other straightforward apps, but with a domain spin or branding, a logo, or something, it's possible. We see this with Wix and with website templates. It's possible, but you're not going to run a company on any of those.

I totally agree. Where I disagree, actually, is I think that the language—the language model in this case, but the language in your metaphor—is improving at a dramatic rate underneath these things.

So while I think almost all these products today are good at prototyping, they're trying to push into refinement, but they're not really usable as things that you can actually deploy to production at all. In fact, most of the cool demos you see on Twitter don't work 3 days later. So they're very much in the prototyping phase, but the programming language in the metaphor is improving dramatically. I think we'll get there, or at least make more progress than we think.

Versus a traditional programming language like object-oriented programming, it didn't feel like it 100x'd the number of programmers or reduced the amount of time to ship something to production to 1/100th. We just got new tools and new problems to solve.

Well, of course, you're benefiting from hindsight.

Yes, and that's a key thing. First, obviously, I agree: we're in an exponential improvement cycle with the models, so any predictive power goes out the window.

Correct. And anyone who says something negative is going to be the next person who says the internet is going to be a passing fad like a fax machine, and that's bad. You just don't want to be there. It turns out that having lived through these things also makes you very shy about making predictions, because you see how foolish people look for a long time.

Take something like object-oriented programming. I mean, this thing was hyped to the moon as the solution to everything. This was a wave of programming languages. Just to give you an idea of how quickly things move, they started in 1980, and by 1990 they finally reached peak hype. So it was 10 years of incremental improvement, and by then any programmer would have said, “It's not really doing much. It's sort of just changing the old programming-model paradigms of abstraction and polymorphism.”

Meanwhile, the magazines—which were the key measure of success at the time—were all over it. There was 1 magazine that had a drawing of a baby in diapers on the cover about how programming would get made easy. I remember seeing it at the newsstand, and I was working on the C++ compiler at the time. C++ was a brand-new language in 1990, and it didn't work yet. Here was a baby who was going to make programming possible for other babies at daycare or something.

Whether it was that or all the database programming languages, like Delphi or PowerBuilder, in an algorithmic sense they were all constant improvements. They added a constant factor, like plus 7, onto programming. None of them changed the mathematical order of magnitude. What I believe is that writing right now is changing the order of magnitude, so it's happening. Accuracy isn't there, but one of the things about writing is that when you read it, most of it in business is not really accurate already.

It's very much like autocorrect. Autocorrect fixed all the common typos, like “teh” to “the” in English, and just replaced them with these wild new autocorrects that replace what you typed with a word that has no meaning in the context of the sentence. That's what we face on phones all the time. So what we're going to see is a whole different set of errors in business writing or academic writing in schools that just replace other errors that have always crept in.

Totally. I remember Smalltalk was the hot language.

Right. Well, Smalltalk was the start of it, and it was called Smalltalk-80.

Yes.

And then it really didn't achieve any momentum outside of Palo Alto. But then C++ came along, and there were 50 languages in the middle that people don't talk about, like Objective-C being 1 of them. That was the iPhone language, which was really Steve Jobs's language. There was Object Pascal and Pascal with a relational database attached.

This was my master's degree, and then I quit grad school. I could go on about this 1 for far too long, so I'll just stop now.

Do you think there will be bestselling novels that are entirely AI-generated, or nearly entirely, in the next few years?

Absolutely. 100%. I don't think Stephen King is going to do that, but I think there'll be some new writer who will probably write it under a pseudonym. A year after the novel is written and has been made into a movie, they'll say, “Oh, by the way, I got the plot idea from a prompt, and then I just started having it write, and I was editing it along the way.”

And the copyright suit that follows from training models and stuff, that's a different issue.

Absolutely. And I think there are 2 things on this that are really interesting. These language models are averaging machines, and with art you almost definitely don't want the average of all the novels, all the writing, or all the authors. You want something that's at the edge. So how do we actually point them in a direction such that they can be at the edge of culture? I think that's important for making great art.

The other thing is that a lot of artists don't yet know how to use the new tools, and we're going to see artists who are native to the technology. Instead, what we're seeing a lot out there, what's called “slop,” has just been a lot of this low-barrier-to-entry art being created, which is great because it gives people the fulfillment of creative generation.

I think what we're talking less about is, “Hey, how is the ceiling being raised for artists because they have access to these technologies?” Without going all in on “What is art?” we all know that bad sitcoms are part of society, too. I think it's important. We tend to focus on the very, very best of things, but most everything isn't only the very best.

In business writing, it's all slop. I've written a lot of business writing, so I can say this confidently about what I've written and what gets written. Take something completely mundane that a lot of people in Silicon Valley spend a lot of time working on: the enterprise software case study. I'm telling you, GPT generates better enterprise case studies, faster, than the typical marketing associate does at a company, with 1-millionth the effort.

Does the content need to exist?

It actually does. It's just an important part of the selling process. Just like with something at the extreme, such as medical diagnosis, we tend to think about the most obscure diseases, the most difficult problems to understand, and the finest hospitals with the most resources. But you have to remember that 80% of the world has no access to anything.

So wherever you think medical LLMs are on the slop scale, most people don't have access to anything average. We have to make sure that the whole debate does not center around what Francis Ford Coppola is using as the book, who the actors are, or who the cinematographer is. That corporate case study often involves interviewing the person and filming it.

Yes.

All of a sudden, we see it today: those things are done over Zoom. Suddenly, flying in with a [?], or getting a satellite and booking [?]—we've changed our view of excellence because we wanted more access, and I think that's absolutely going to happen.

Should you get graded on slop in school?

That's a different problem. But most stuff is pretty average.

The world needs more slop, says Steven.

I feel like this isn't a press interview where you can put words in my mouth like that.

The world needs more slop. That'll be the title. Well, actually, Mark makes this point, and I think it's a really good 1: Is the bar for success perfection? Is the bar for success what people can do today, or is the bar for success just something that's better than the alternative?

In your case of 80% of the world having access to no medical knowledge, no medical services, and no medical opinion, of course this is dramatically better.

Yeah. When I had to get permission to use a word processor in college, 1 of the stumbling blocks was that my printer was an Epson MX-80 dot-matrix printer. It looked like a computer printer, but the rules for the papers were that they had to be written on a typewriter.

Then the Macintosh came out in the spring and only had an ImageWriter, which was another dot-matrix printer. All of a sudden, the standard changed because the value of being able to revise, edit, update, copy and paste, and use fonts was so much higher than the fidelity of the teacher reading it on bond paper with Courier. That's going to happen with content as well.

What I would love to talk to you about, Steven, is hearing your take on I/O. There was a lot of conversation around Google and how Google had sort of fallen behind and lost its ability to make new things. They released a ton of new software at every part of the stack at I/O. What do you think that says about Google? Do you think the demise of Google is overstated?

Of course I think the demise of Google is an absurd proposition. The demise of a giant company is a crazy thing to say. I was driving in and listening to CNBC, and someone—some investor, or whatever—was talking their book about IBM and whether it was the 1 to buy. I almost wanted to pull over to the side of the road and think, “What universe am I in where this company that has died 9 times in my career…”

The death of a company is just such a dumb thing.

Losing a position of influence, however, is a very real thing. In these platform transitions, big companies have an enormous asset, which is the shock-and-awe asset. They have the ability to tell the story: “We’re pivoting our whole company around this, and we’re a zillion-dollar company.” And here is a full assault across the board for every single asset we have and every single category the world is talking about.

That matters, and that’s what you could do. It was totally predictable that Google would show up with literally the B-2 bombers of software. But the question is really much deeper than that: Will they alter their context of how they build products and their go-to-market? Because that’s really what undermines the big technology companies.

What I’m looking at with Google is not, “Can they present all the technologies in the context of Google Search and Ads?” but, “Can they transform the way they think to something new?” Because that’s really where the disruption is going to happen.

I love that point. Awesome. Anish, Steven, thanks so much for this week.

Sure. Thank you.

Super fun.