The Future of Software Development - Vibe Coding, Prompt Engineering & AI Assistants

Infrastructure never goes away. It just gets layered.

A new piece of infrastructure changes the way that you program computers, and it changes the stack that's around it. We're building systems to build other systems.

Developers are making a lot of the decisions, and a lot of the marketing and sales to developers looks more like consumer these days than it used to.

Who is going to earn the distribution and earn the developer attention is going to be a different game.

From an application standpoint, we've abdicated logic. This is a pretty big deal. It's by far the biggest thing that I've seen happen in my life. Software was always the disruptor. One of the most exciting things about the AI wave is that software is being disrupted. We're being disrupted, right? And so we're like—

Disrupting software is eating software.

I know. I know. We're like—

Hey, guys. We're here today to discuss the state of infra. We've done a consumer podcast. We've done it with the consumer team. We've done it with the AI team. Now we're excited to do it with the infra team.

First, can we get a definition of infra? Where does infra differ from enterprise? How do we think about it internally?

I would say infra is basically what makes software work. We'll probably get pretty deep into a set of technical definitions. You mentioned networking, storage, and compute. Where does AI fit into that? But I think at the simplest possible level, if you want software, infra is what engineers are using behind the scenes to make all this possible.

Our formal definition internally is technical buyer, right? It's the stuff you use to build the stuff—the stuff you use to build apps. If it's used by a technical user, we consider it infrastructure. Whereas something like, let's say, vertical SaaS could be used by a flooring company, a marketer, or sales. That would not be considered infrastructure.

And technical user, for the record, is a developer, data scientist, analyst, or cybersecurity professional. Yeah, right. There's a wide range of people. These are our people—the kind of nerds behind the scenes.

In system terms, you can think about it as compute, networking, and storage, but also all the tooling that goes around a developer's day: what they're using to build software, and what tools and products are operating this ever-growing, more complex software as well. That goes all the way to semitechnical users who may want to either prototype or tinker with building applications.

We're very interested in anything in the technical domain that's used by technical people.

You mentioned compute, networking, and storage. How should we think about models? Is this the fourth layer of infra? How do they interface? How should we think about that?

I certainly think of it as a fourth layer of infrastructure. It certainly leverages and builds on top of all 3 pillars we're talking about. It has a lot of demand for compute and, of course, it's trained on and also producing large amounts of data.

To leverage and use these models for our purposes, latency and networking capabilities are also very important. But it's going to be as prevalent as any piece of infrastructure software. I don't know what the analogy these days is anymore. Is it a database? Is it a new form of compute?

Really, to me, it's a fourth pillar that incorporates everything but also provides intelligence for the software we're using and building today.

I think it's probably worth asking why a piece of infrastructure is a piece of infrastructure.

Generally, a new piece of infrastructure changes the way that you program computers, and it changes the stack that's around it. It's got different memory requirements and different latency requirements, so it requires rethinking how we build software and how we build infrastructure.

In addition to compute, networking, and storage, distributed systems would also be included, just because things like state consistency require you to think about proximity and guarantees. Databases probably did too, because they changed our programming model. You have different guarantees.

These models very much fit in that for a couple of reasons, and I think Jennifer's exactly right: you just build different data centers and different chips if you want to build these models. So it has that impact, but programming them is nonobvious. We're still trying to grapple with how you program with them. They don't really listen to you. Sometimes they do the coding themselves.

If I were to try to distill the single biggest difference that these models provide to infrastructure, it's the following: I don't remember ever in the history of computer science when, from an application standpoint, we've abdicated logic.

Actual application logic. In the past, we've abdicated resources. We're like, “Give me compute. Give me storage.” These are abstracted resources. But the logic—the yes or no, what it's doing—always came from the programmer.

With these, we're like, “Come up with the answer for me.” So it's requiring us to rethink what it means to be a programmer, what it means to be software, et cetera. It's clearly very fundamental to computer science. Again, to Jennifer's point, it's very much a new piece of infrastructure.

A lot of people are trying to reason by analogy. You alluded to this, Jennifer. Is it like a database because it can answer queries? Or is it like a network because it's sort of nondeterministic and we need to handle retries and weird edge cases?

I think people are really just trying to figure out how to program these things. You sort of said it, Martin, but we've got to start from a blank sheet of paper, which is what makes our jobs really exciting right now, because there's a lot of people trying to figure it out and coming up with new ideas.

I have to add—okay, listen. Many of us have been in computer science for a long time. We've been in our schools, in our operating lives, and in our investing lives.

Software was always the disruptor, right? We disrupt taxis, sales, the back office—everything. One of the most exciting things about the AI wave is that software is being disrupted. We're being disrupted, right? It's like—

Disrupting software is eating software.

I know. I know. We're like, “Oh, so we have to think about it as far as the self-eating.”

Yeah, that's right. Honestly, I think this is the first time I could honestly say that the profession I've dedicated my entire life to is being disrupted. It's very exciting because it's kind of eating itself in a way, and it's tempting to be a curmudgeon, right? We've all been doing this our whole lives. So really being open to and embracing the new stuff is the key thing.

What still applies is infrastructure never goes away; it just gets layered. In this case, all the infrastructure that we've been using and leveraging in the past is still very relevant, but it's definitely getting layered by having this fourth pillar, which is AI and models.

What's different from past supercycles, versus what can we learn as we enter this new one?

There are 2 things that happen. One of them is that often, when you bring the marginal cost of something down—with compute, we did it for computation, and with the internet, we did it with distribution—it increases the TAM a whole bunch. You almost always see this massive TAM expansion.

Part of that tends to be because the TAM is bigger and you've got new users. Because you have new users, there's normally a new behavior that happens, right? This is very much the case with the internet, where people weren't used to going to a computer and talking to everybody around the world.

Existing companies don't really know how to think about new behaviors. They've built their sales motions and operating systems around the old behavior. So you see TAM expansion, and you see new behaviors. Those new behaviors provide white space for challengers—new startup companies—to come and fill those models.

I think we're seeing exactly that happen with this one as well. Clearly, this market is massive if you look at how successful these model companies are. But you're also seeing use cases that computers just never really handled before. In that way, I think it rhymes very much with the internet, and it probably rhymes very much with the microchip.

Maybe I'll answer that question just from my personal experience. I'm always a tools person, and I like having tools fulfill a certain creativity, because I came to coding and computer science as a late bloomer, after my 20s. I really enjoyed all the tools available to me at that point for building software and learning computer science as well.

Now we just have massive leverage in trying to create anything, as long as you have a good idea. Martin laughs at me about this, but I was a big low-code/no-code champion for, let's say, the last 5 or 10 years.

Again, these are tools for people who have good ideas but may not be educated in computer science. The Retool of the world, Wix, and Squarespace—you can build applications and software easily with these tools.

But now you're given the next level of thought partners and tools to really, in any role in the company, prototype software interfaces for your end customer or end user. As long as you know what they need and what they want to see, you can really realize these ideas quickly, right at your fingertips.

So low-code is finally happening. It just takes a lot of coding.

It just turns out the code is natural language. I know, it's so funny because when Jennifer joined the team, she was very excited about low-code. But from my view, low-code is Python. It's a scripting language—you write interpreted languages—and PM is low-code, right?

So we kind of had to bridge that gap, and it was, in a way, a bit irreconcilable until AI came out. Now it's very, very clear what the promise of low-code was. So you're right: It really is disrupting software.

When I was a kid, the internet was sort of a new thing. I just remember really vividly there was this movie with Sandra Bullock called The Net, and she orders a pizza from her computer. This was completely mind-blowing, and now this is actually a common user behavior. What we're dealing with now is so much bigger than that.

I think some of these points about how the infrastructure will evolve, how the companies will adapt, and things like that are probably transferable. But this is a pretty big deal. It's by far the biggest thing that I've seen happen in my life.

Yeah. Let's zoom out and take this long view. And Martin, you're actually the perfect full circle because you were the first infrastructure investment ever as a portfolio founder. Were you the first infrastructure investment ever as a portfolio founder?

I think it was either me or Okta. But I will say that Todd and I were the infrastructure portfolio's early days, when they would trot us out in front of the LPs. It was me and Todd from Okta. So, for sure, I was one of the first 2.

And then you started it up, or were sort of one of the leaders of, the infrastructure fund. At what point did we develop a clear infrastructure practice at a16z?

We always had strong infrastructure people. Ben Horowitz is an infrastructure guy, and honestly, I would say Marc kind of masquerades as a consumer guy, but he's actually revolutionized the way we use computers in this deep infrastructure way. By the way, many things came from that: JavaScript, et cetera. We had Peter Levine from XenSource, and Scott Weiss, so there's always been deep infrastructure.

But when I joined the firm, we didn't think of it as infrastructure; we thought of it as enterprise.

Right?

Right. And so, you're either on the consumer team, the fintech team, or the enterprise team. The thing about classifying this stuff as enterprise is that the go-to-market motions for something that touches technology, and being able to reason about that, are so different from reasoning through a go-to-market motion that's purely through sales.

We've just learned over time that we do deep market diligence as a firm, and deep diligence on companies before we invest in them. The type of diligence we do for deep infrastructure required a different type of junior partner and a different type of analysis.

Over time, we realized that companies you can evaluate more by the business model, the market, the buyer, and the unit economics are sufficiently distinct. So we decided to pull them apart, and that's why we have the Apps Fund and, of course, the Infra Fund.

If you think about enterprise, there's of course the horizontal piece of it, but also a lot of vertical enterprise—applications, buyers, and sectors—where infrastructure is almost always horizontal. That's another thing we realized along the way: We still want to appeal to this technical audience and technical buyers, but we want to think about the space in a horizontal fashion, where this technology can be distributed to all these different sectors and verticals as well.

It impacts how we think about the stack. It impacts how we think about what is going to drive the form of how the stacks integrate with each other. It's just kind of different from how enterprise generally works.

Yeah, it's actually a very important distinction. I'm glad you brought it up. Technical buyers tend to be centralized buyers, in a way. IT will buy compute, network, and storage. Yes, you've got the compute person and the networking person, but it kind of rolls up to IT, and developers are kind of a centralized buyer.

So you can understand it, and you can understand the developers. But when I was looking at vertical SaaS apps early in my investing career, to understand how to sell into a flooring company, you have to understand the flooring market, and that's entirely different from the pet food market, and that's entirely different from the construction market.

So I felt like there was no real central software buyer for these kinds of verticals. As you get further away from core infrastructure, you get away from this notion of a centralized, educated buyer, and the level of analysis becomes very different. I think that's exactly right.

Well, there's even sort of the horseshoe theory of software buyers happening now, where you have consumers over here, and then you kind of work up to apps. You would think that infrastructure is way over here, but it's actually kind of bending back around. Developers are making a lot of the decisions, and a lot of the marketing and sales to developers looks more like consumer these days than it used to.

That's an incredibly important point. I was researching the stats the other day. When I first joined venture—or maybe even just started thinking about infrastructure—developers were still in the low tens of millions, and they were becoming the next generation of consumers.

Now they're definitely the next generation of consumers. It's going above 50 million, on the way to even bigger than that. Now we're talking about natural language being a programming language for everybody; they're making decisions just as a consumer.

So we're, of course, training ourselves to understand how developers, as individuals, adopt tools, but also how buying centers at customer and enterprise organizations evaluate tools. There's a lot to learn and figure out, given that these are the technical audience we really care about.

Yeah. I want to get to how we think about our investable universe and what subcategories are mature versus newer or ripe. But before that, I want to better trace the evolution a little bit.

Maybe, Martin, we can start with you. If you had to categorize, since you've been investing, the different waves of infrastructure—or maybe the different inflection points at which it changed how we even thought about it—how would you characterize the life cycle of the firm?

We started in 2009, and that was actually kind of pre-cloud.

A lot of the early investments were right on the cusp of cloud. For example, at my company, the cloud was out there, but it wasn't sufficiently deployed that you could use it as a core thesis. A lot of the early investments were like this.

We talk about it in terms of technology, but it rips through the entire business model. Early software was on-prem with a perpetual license. That's just different economics and different analysis.

The pre-cloud, installable-software era was one period, and then we saw the cloud transition. We went to recurring revenue, a totally different deployment model, and a totally different operating model. During that time, we saw things like net dollar retention, expansion, gross churn, and gross margin become more important. Everything changed.

Meanwhile, of course, we saw consumer being disrupted by mobile, with the rise of the Ubers and Lyfts and the Airbnbs. And now we're seeing it happen again with AI. I would say the AI transformation of the last 3 years has been the most dramatic I've seen in the last 30 years of being in this industry.

There's one interesting blip along that path, which was COVID. We all come from the realm of enterprise sales, being on the ground, deploying things, and all of that totally evaporated. That was a dramatic shift as well, but it was driven by a force majeure as opposed to a secular technical wave like the other 2 were.

I would say one of the benefits COVID brought was that the trend was already going toward developers buying and adopting tools—a bottom-up motion for infrastructure and dev tools. It got accelerated during COVID, when lots of people were tinkering and trying out products and tools, and more people were building dev tools during that time.

I feel like there was a flourishing ecosystem of these PLG, or product-led, dev-tool companies that is giving birth to what we're seeing now, with a lot of AI dev tools coming up.

There's nothing developers like better than being forced to stay home, not interact with other people, and write code.

That's amazing. Okay, now that's a good overview. Let's go deeper into the present a little bit. Can you guys share how we think about the different subcategories or landscape that makes up infra? We could also plug some examples of portfolio companies or spaces where we've made some bets.

Here are a few important categories. Developer tools mean anything developers use to make their lives better, easier, faster, or more efficient. Cursor is probably our top developer tool company right now.

And before that, GitHub, right? Yeah, GitHub before that. And Jennifer, you've actually backed a bunch of interesting dev tools companies in the last few years, too.

Yeah. From Lifi, Stainless—

And there was a long time when dev tools were kind of written off by VCs, right? The TAM was—

Oh, yeah. The TAM was too small.

Yeah, exactly. I mean, at the time of GitHub, would you ever have imagined a repository would be a huge company? It was almost a joke.

Right. And people worried about the business model, all these things. Small TAM is the classic red flag for infrastructure investing. If you're at home listening to this and someone tells you “small TAM,” run the other way.

They're not an infra investor. Yeah, infra creates TAM. If there's one takeaway from this thing, it's that TAM is created. So, like, I think dev tools have core infra, which is compute, network, and storage, right? This is like Tier 0. And then you tend to actually be quite a bit above the core infra stack. So maybe you talk through the areas you focus on.

Well, I think about both how developers are using tools to improve their efficiency, but also how customers are getting value out of that as well. A lot of the time, that's packaging dev tools into SaaS forms. I'm investing in a company called Pylon. It is a SaaS company that does customer support, but fundamentally it's building a data pipeline. That's core infrastructure that's very good at connecting systems and providing context to a lot of agents and AI models. To me, that's infrastructure. We're spending a lot of time on cutting-edge AI research, so a lot of foundation-model investing.

I will say the reason that we're a little bit skittish on this question is that, early in supercycles, it's very hard to distinguish between an infra company and application companies. The reason is that the TAM is so small and so new that the new technology becomes the app, right? Let me refer to the original supercycle that started the firm, which was the internet—Netscape. I remember when it came out; it was a consumer thing, or at least a student-in-school thing. Everybody was downloading Netscape from an FTP server and using it as individuals. The enterprise didn't know what to think about it and banned it or whatever. The same company that built JavaScript and was building this core technology was also doing the browser. Over time, it matures, and then, of course, you have all of these internet companies, and all the applications show up. We're seeing the same thing in the AI wave. Is Midjourney an infra company? They build a model. Or is that an app company? Well, it's kind of both in this sense. I do think that at this stage it's very hard to distinguish.

Right. It's exactly right. It's very hard to answer if OpenAI is an app company or an infra company. It literally is building infrastructure that's like a cloud running these models for different sectors and different use cases, but at the same time it's building a consumer app, ChatGPT. We think of foundation-model companies similarly, like ElevenLabs. They're a voice AI provider, and they have the creator application that can use a studio to create voices, but at the same time they're also supplying the voices to large-scale enterprise use cases that are fine-tuning, cloning your own voice, and distributing that through an API. So it's sort of both.

Yeah.

Another area we've done a lot of work in is data systems, and there have sort of been 2 branches. One is the back-end, data-engine-driven big-data systems—Spark, Hadoop, that sort of thing. The other is data analytics, more tabular, Snowflake kind of thing. As a firm, we've been super active and super aggressive. We've invested in companies like Databricks, Fivetran, dbt, which you guys invested in together, Hex, which is doing really well, and Tabular, which was acquired by Databricks. We're still really bullish on this. Unfortunately, AI has sucked the air out of the room for a lot of data companies from a bunch of different angles, but I think we'll continue to do a lot more of this as well.

How do we think about defensibility for AI companies, whether it's at the app layer or the model layer? Do they all have their respective areas of defensibility, or how has our notion of defensibility evolved?

We once wrote a blog post that there was no defensibility anywhere in the stack.

For anything?

For anything. And yet people make lots of money. The argument at the time was, “Okay, Nvidia has sort of a moat because chip designs are hard to copy, but if you go sort of up or down, they're all manufactured at the same place, right, at TSMC? If you go down, if you go up, the cloud providers provide effectively the same product. The models are trained on the same data and have similar capabilities. The apps are all kind of using the same model.” That was sort of the naive theory when we were just trying to understand this at the beginning, and I think maybe it was true at the time.

What Martin was alluding to a second ago is that, meanwhile, every company at every layer of the stack is doing fantastically well right now. During this initial phase of industry development, which we sometimes call the Brownian-motion phase, I actually think it's really hard to make pronouncements about what's going to work, what's not going to work, or where value is going to accrue. App companies are doing really well, and we're pretty clearly past the wrapper phase. I don't think there are any wrappers anymore. Building good products with AI is really hard, and the founders doing it now have really good intuition for how to do it.

You know, the models are clearly pushing the whole industry forward, and they've built huge companies—dramatically. Matt said it perfectly; I totally agree. It's kind of all working right now, and I think you could actually make a case for how defensibility will work. It's quite different from the way defensibility has worked before. Maybe you guys want to add on to that.

Yeah. On how defensibility worked for infrastructure companies before, largely it came from the fact that it's really hard to build, let's say, a new database or a new framework. It just takes a lot of expertise in the domain: understanding what has happened in the past, where the field came through, and what innovations need to happen to polish software into this new abstraction to provide to developers. One example I'm thinking of is DuckDB. It's such a high-performance, small, but really nimble database that it took 4 years for the team to write. To replicate that is really hard. That's generally what happened in the past infrastructure space: it takes these experts a lot of time to build a new piece of security software, whether it's YubiKeys or, again, Databricks on Spark. But now, as AI infrastructure comes through, I think a lot of those defensibilities still stay and are still true, because these are earned secrets about the downfalls and guarantees that other software runs into. But I do feel like the adoption phase is just really massive. Who is going to earn the distribution and earn the developer attention is going to be a different game.

The industry tends to go through these expansion and contraction phases. Think of it like the Big Bang or something: it expands, and then it contracts. So what happens when it expands? When it expands, zero-sum thinking is deadly because you just keep getting more market. We're clearly in an expansion phase, right? Everybody is like, “Oh, Nvidia can't sell more chips,” but they keep selling more chips. “Oh, the hosting platforms can't continue to get margins,” yet they keep continuing to get margins. Matt said it perfectly. I totally agree.

If you're in the expansion phase, then there's just more to sell. You should be aggressive investing. So what happens in the collapse phase? Look at any layer of the stack. In the collapse phase, things start to consolidate. Again, you have consolidation. But what is the end state of consolidation? The end state of consolidation will always be an oligopoly or a monopoly.

It’s not like layers ever go away. If you have an oligopoly, like the clouds, then you have what is effectively price fixing, but it’s tacit, right? Everybody’s like, “We’re going to price it this way, and we’re going to maintain our 30% margins.” So you still have value there.

You have margins, or in the case of a monopoly, you’ll end up with, say, Intel at the time, and then you can also maintain margins. So in none of this do you lose margins, right? And I just think this is why people think so sloppily about this. People use words like commoditization and no defensibility. That tends to be a battle between layers of the stack.

But the only way you can do that is to actually move down the stack and enter somebody else’s layer, which is incredibly hard to do. And you do see it. Of course, Google is going to build its own chips, and they start moving down the stack, but that’s a very, very different layer than somehow Google playing the different layers off against each other.

And so I would encourage anybody that does invest, at least in infrastructure, not to think zero-sum and to realize that historically every layer of the stack has maintained some level of value and margin. If not, it was because a layer above them managed to kind of verticalize themselves, but then it’s that 1 player against the rest of the world.

Yeah. It’s like faster-than-light-speed travel has just been invented, right? We’re sending all the spaceships out in all directions, and there are plenty of planets and stars to claim for everybody. We don’t need to—we’re not even close enough to each other to fight.

Yeah, for sure. This will slow down, and then the consolidation will happen, but I guarantee you’ll just end up with these great companies that maintain margin. AWS still has great margins. Google still has great margins.

Databricks, too, is growing at incredible speed for its scale. I think people underestimate how hard these problems are in many cases, right? You’re sort of applying consumer thinking because this is how we live most of our lives. It’s like, “Oh, wouldn’t it be relatively easy to move to a different part of the stack or take out your competitor, or a customer could just switch back and forth?” It’s just different laws of physics.

It just turns out, in general, the switching cost of infrastructure is so much higher. Even with API businesses, people tend to think you can just switch over to another API. There’s so much logic embedded in calling the API in the software itself. There’s a lot more switching cost compared to your regular SaaS software consumer.

Totally, because you’re actually integrating systems, right? It’s not necessarily a person who can just have a preference for one thing or another. You’re integrating. For sure.

Sam Altman once gave this advice to startups last year. He was like, “If you’re worried about us improving our models, you’re in a tough spot. But if you get more excited about your business by us improving our models, then you’re in a good spot.” Do you think that’s a helpful framework?

I think it’s helpful for OpenAI, for people.

No comments.

I want to say that, too. If you know a16z, if you think us investing in the next company isn’t good for you, then—but if you want to buy from our companies, that’s great.

That’s a very open question that’s actually a technical question. It isn’t a business question, which is: How much does general training generalize, right? So we know in the pre-training world, it generalized really well. You’d create 1 model, and that model was just as good at code as it was at writing a poem, right? So we know that it was very general.

In that world, sure, as the models get more powerful, then they can do all of the things, so they compete with all of the things, right? But it seems clear to me—and again, this is an observation, and it may not be correct—that as we get more into the RL world, you make some trade-offs. Let’s say I RL something for code; it’s not going to be as good as something else, and you’re making these trade-offs. In that world, then it’s not the case that the model is going to generally be good.

I think it’s great to compete at the model layer. Again, I think this is maybe a reasonable rubric, certainly for OpenAI to have people believe; maybe a reasonable rubric if you believe these models are going to be generally great. But I just don’t think it holds up to how things are going to play out.

This is a debate we had 2 years ago: whether the general model and the most capable model will rule, or whether a lot of small- and medium-sized models that are very good at specific tasks will. That’s going to be the future. It turns out both are true.

Both. Yeah.

When we’re talking about complex systems, you cannot just use 1 model that drives everything. At least not today. But you can compose very capable and powerful models to take certain tasks, and also chain together processes: from processing a document to feeding it into a model to having some reasoning and giving you back clean and structured data, to make decisions and put it into your application to serve end users. That’s a complex system that invokes many model calls instead of just 1 big model’s task.

Did you guys have any reactions, or is it worth talking at all about Karpathy’s talk? Did that framing resonate with you? Did you have any differences in how you would frame certain things? One thing he mentioned is that he thinks it’s not the year of agents but the decade of agents. Perhaps it’s not as immediately upcoming as we might have thought.

There’s this idea of prompt engineering that people have talked about, and somebody—it wasn’t Karpathy, but Karpathy piled on top of it—said, “It’s really not about prompt engineering; it’s context engineering.”

So what is context engineering? If you’re going to call a model, you kind of have to know what to put in the context, in that prompt. What tools do you have to do that? You could use other models, but at some point you’re probably going to use traditional computer science. You’re going to use things like indexes, and you’re going to have to do prioritization.

To really drive the best performance out of those models, you do want the context to be correct. And I do think it’s probably the right framing of this problem.

The next step is, inasmuch as we’re going to provide formalism to how you use these models, how you use existing tools, and how you improve performance, you should be thinking about what’s the right way to get the right context into those models. And I bring this up because, like we said before, new infrastructure pieces create new patterns and new methods of software and building systems.

This is a great example of that kind of emerging before our eyes and people reasoning about it. I truly believe in 5 years we’ll look back, come up with a whole new set of formal ways to build software, and they will have strong guarantees, and we’ll understand them, and there will be all the tools for it, et cetera.

There are a few different types of infra founders, right? We could probably come up with categories. There’s the sort of infra founder who loves solving really messy, long-tail, nasty problems. There’s a type that just gets fed up with a problem, and they’re like, “I’m going to solve this finally. I’m sick of this.”

Then there’s the type that just sees the world in a new way, right? It’s like, “This is actually how we should marshal these resources,” and it can change how we build things. React is a great example of this. We had all these progressions of front-end development frameworks, and finally React was the way that stuck and, for years now, has been sort of the default front end.

So I think what Karpathy’s talking about is trying to figure that out, right? His Software 2.0 thing was really interesting. We were investing in a bunch of traditional ML companies at the time. I think Software 3.0—he’s sort of right about that, too, directionally. My hope is it’ll inspire a lot of new infra founders to do this work: see the world in a new way and figure out how these primitives should really be arranged.

One of the difficulties of having any conversation around AI is that it just exploits this weakness in the human imagination to dump all of our fears and hopes and dreams into this anthropomorphic fallacy, right? It just—and this goes all the way back to the Promethean legend.

Let’s talk about even this context, right? We’re building systems to build other systems. Systems have constraints, right? You can fail on either side of this when it comes to this anthropomorphic fallacy.

On one side, you can say, “This stuff doesn’t work. You shouldn’t use it. You should only use traditional things,” which, okay, that’s clearly not the case. It seems very useful. But on the other side, you can believe they’ll solve all of our problems, and you don’t need formalism, and you just go to the beach and come back when AGI is done, and it’ll do it for you.

A part of our job—and what we spend a lot of time talking about—is trying to find that pragmatic, non-blinkered, non-pessimistic middle despite all of the rhetoric. You hear all the rhetoric, right? The stuff is going to—no, we’re not going to have to work, and we’re all going to be on the beach, right? Or it’ll kill us. I mean, the whole thing.

I think where we’ve landed is that this is a real disruption. It just is changing all of software. It’ll look something pretty different, but it is still going to require professionals. I do think that the statement “it’ll require professionals” is a very meaningful one. It means that you still need people who understand the specifications of the systems, and not everybody agrees.

Some people are out there like, “Listen, you will never need a programmer again, because people are beginning to just say some high-level thing and it’ll show up.” The only statement I’ll say to that is that formal systems came out of natural languages for a reason. Either you care about specifying what you’re designing or you don’t, and if you do, you need to be a professional. That’s why every professional discipline, even though they started with a normal, informal natural language, has ended up with a formal system.

What is your mental model on coding specifically? Will there be fewer engineers who are just higher-powered, or fewer junior engineers? How should we think about it?

I think the best way to think about this is simply that we’re going to have more developers. I think it’s very unlikely that we’re going to shrink development teams because we have amazing new tools. That’s just not how these markets have worked in the past. I think exactly the opposite is going to happen: we’re going to be creating so much great software, and it’s going to be so accessible to so many people who may work at a big company or may just be hacking on the weekend.

You can’t anthropomorphize these models, right? A model is a file on a hard drive in a computer somewhere. When you run a Python script, you can transform one piece of data into another piece of data like that. That’s what this is.

Programming is a fundamentally creative job, right? You are literally creating things in the strictest sense of the word: you’re creating software that didn’t exist before, and that’s something only a person can actually do at some level of abstraction. I personally think this is a huge boon for programmers. You have to change the way that you’re working, and it’s a huge productivity boost. I think that creates more, not less.

I cannot agree more. I feel like many of the developers and programmers I talk to today are going to Disneyland, just because of how many great tools there are to help them move faster and build things they’ve always wanted to work on, both as side projects and in their main jobs.

I do think it also changes the dynamic of how people are picking up new languages and new frameworks. It is, again, a next level of iteration speed, given what we’re seeing with AI agents and AI coding tools.

Here’s another useful mental model on this stuff. I think it’s worth asking the question: Why do people buy software? Why does someone buy some random SaaS tool? Is it because it’s so hard to build? No. Most SaaS tools are CRUD. They’re just these basic read-write databases. They’re all kind of the same.

So why do people buy them? Aaron Levie, who’s the CEO of Box, said this so beautifully: The reason people buy software is because somebody else made the decisions about what the workflow should be, what the operational logic should be, what data is important, and how you use that data.

Creating a product is a lot about understanding what is being used and guiding the user in that direction. Otherwise, I just give you a compiler, and you do whatever you want, or I give you a database, and you do whatever you want. There’s a reason that we have a proliferation of vertical SaaS, and it is this articulation—or this transfer—of domain understanding.

That just doesn’t go away, independent of how you create the software. We will still need to design products based on whatever problem is being solved, guide people so that they’re the most effective with them, and help them understand them as well as possible.

We did this with assembly before, and we did it with high-level languages, and then we did it with high-level frameworks, and then we’re going to do it with AI. But the fundamental process of that articulation will not go away.

It’s totally orthogonal to creating the software itself, right? It turns out to be a much harder problem to go out and collect requirements from an unknown set of users with an unknown set of needs, and figure out what to build. That turns out to be much harder than actually building.

What do you think is the average number of lines changed in a PR in the industry?

Two. Yeah, but it shows you that it literally is understanding the need from the business and the need from the user, and making some minor tweak. That is the long tail that goes into software.

And by the way, it turns out it’s 2.

Oh, really?

Well, I think it’s the median. The median is 2.

Yeah. So I’ve written a lot of PRs in my life. Just for the record, by the way, Martin said CRUD. CRUD is a technical term, right? Create, read, update, delete. We actually think applications are great, right? We don’t think they’re CRUD-y.

Yeah, CRUD is just a footnote.

Good, good to know.

Jennifer, you were mentioning earlier how, 2 years ago, we were having this debate on generalization, and we’ve learned from it. What are the debates we’re having now internally or with your peers? What are the main questions that we’re asking that we can’t wait to see how they’re going to reveal themselves in the next few months or next year, and that are going to impact our business?

Oh gosh, every week is different. What are some recent ones? Definitely, how realistic are agents today—really producing production-level software? That’s more on the coding-agent side, but also, in general, the agent evolution: Where are we in going from demoware to producing real, tangible value?

Synthetic data is one we talk about a lot. We’ve been talking about it for 10 years.

Yeah, I mean, this is what’s so great about infra: You can have the same debates for 10 years, so they never quite go away. You just change the background. It’s like everything else.

Consistency versus availability. Literally every system has this trade-off.

The synthetic data thing is almost an information theory question. Can you make models meaningfully better without introducing new information to the system? I think it’s now pretty clear you can do a little bit, but the question is whether this leads to a self-improving utopia of models or not. We have some pretty strong opinions on the not side of that.

Generalization, which Martin mentioned, is a pretty interesting one. If you train a model to be really good at math, does that mean it’s going to be really good at other things, or is it just really good at math? That gets me excited. Not everybody gets excited about this.

Another one that we talk a lot about is what these things are actually good for. The path we came from used a lot of AI. Pre-generative AI used a lot of AI, so there are a lot of AI-shaped holes in the enterprise, like chatbots and this and that. That’s very much on the brain, and it seems to sometimes confuse the discussions from the new use cases we’re seeing.

If you look at the most common use cases of something like ChatGPT, I think the top one is companionship and therapy, then managing my schedule. It’s the top of the pyramid of needs stuff. Number 5 is professional development, not low-code development or whatever.

We have this idea of what we thought AI was going to do, and the stilted attempts from previously. Jennifer actually ran product for a chat company before. What it really is good for clearly has some overlap and convergence, but it’s not nearly as big as people say.

We spend a lot of time just trying to be very honest with ourselves: What is the new behavior being created? Where is the stuff getting used? What is just being crammed into places it’s not actually quite good at?

To that point, how do we think about agents right now? What are they good for, or going to be good for soon enough? What is the state of them? How do we think about the broader conversation?

I mean, coding agents are awesome. They’re amazing.

It’s really amazing.

So I have a very simple way to think about this. I’m the anti-agent guy, by the way. I think it’s a marketing thing. This is the other thing about infra people: we’re allergic to marketing, which isn’t always a good thing. It’s a good thing you’re here.

What do you mean?

Yeah, exactly. If you take the simplest definition, an agent is an LLM running in a loop. A very simple way to think about this is that errors propagate throughout the loop, right? So if you have a small error, it gets worse and worse, and this is why a lot of agents doing, say, general web browsing don’t perform very well yet.

On the flip side, if you have a way to correct those errors in the loop—which is one thing that you have in code, right? You can lint, you can interpret, you can even try to compile and things like that—you actually do see good performance over time. So that’s a very simplistic and maybe not quite the right way to look at it, but if you can do this kind of error correction, I think you’re seeing a lot of improvement from this iterative approach.

It’s incredible. I’m actually on the GitHub mailing lists of a lot of the companies that I work with, mostly for interest, and even in the last week I’ve seen a bunch of Cursor-like agent commits. Even in the last 24 hours, I’ve seen the Slack integration coming from Slack.

And so I do think that, for bite-size, well-articulated tasks—tasks you can articulate very well—we’re starting to see them really work. So in the coding space, I would say I’m a convert. But to Matt’s point, if you go wander out in the woods and bring back a bear, I think we’re going to run into that constraint.

Will we see more vertical integration or more horizontal specialization?

You know, historically, we’ve seen both. And what’s interesting is we’re already seeing both now, right? Apple, of course, has historically been vertically integrated. Microsoft and Intel have historically been horizontal. Often, companies will start horizontal and then go vertical.

Google was horizontal. It was built on top of normal servers, but then they built their own servers, their own chips, and their own networking gear. So I think you always get a mix of the two. What’s interesting about now is we’re actually really seeing both.

I would say that OpenAI is very much a vertically integrated company now, with ChatGPT driving a lot of it. I would say Anthropic’s usage is more horizontal, and they’re doing a great job of that. I think we’re seeing this on the model layer, too.

A very interesting discussion we haven’t had—but it’s a very interesting one—is that “open source,” quote-unquote, really seems to work with these models just because you can’t, as a user, recreate it. If you look at Black Forest Labs, they’ve done a great job building a horizontal layer for these models. But then you’ve got companies like Ideogram, which have built a great vertical experience as well.

So I would say that, for AI, we’ve already got great examples of both, even this early on. And I don’t see any reason that that will change.

I think from the business front, it just poses new, interesting questions and challenges, too. How do you capture the value? Of course, horizontally, you can capture the value by being able to address every single use case, by providing that to developers or enterprises.

But if you work vertically, you kind of have to pick a lane: Do I want to focus on image models? Do I want to focus on graphic designers? Do I want to focus on people who are generating photographs? You kind of have to understand the market and the user personas and use cases pretty well to capture the maximum value, whereas you probably can take an easier path just to provide an API so everybody can use it.