Why AI Can Write Code But Can’t Keep It Running (Yet)
- AI coding has crossed from autocomplete into industrial-scale leverage, but the gains accrue disproportionately to engineers who already understand complex systems. Martin Casado describes shipping production code for a data-lake “auto-promotion” capability alone in 10 days versus roughly six months historically. Jad Naous calls that a “10- to 20-fold improvement.” His caveat is sharp: his team is small, not a 100-person sample, and “it’s not like some bad engineers suddenly became good engineers.”
- The bottleneck has moved from writing syntax to specifying architecture and verifying behavior. Jad runs three or four Claude Code sessions, has agents review design and PR documents, parallelizes implementation, then personally recomposes the work “point by point.” He says he uses Windows and WSL, names Opus 4.5, and later describes a technique with Opus 5. Tests, benchmarks, algorithms, responsibility boundaries, and human review matter more as large PRs become cheap to generate.
- Code and operations may occupy fundamentally different learnable distributions. Martin argues that language and code reduce a high-dimensional world into manageable representations, while production systems face rare states, changing workloads, hidden context, and “heavy tails, exceptions, power laws.” Systems history reflects that difficulty: instead of perfect prediction, operators repeatedly choose to “overprovision” resources.
- AI for operations needs signals and investigation, not just raw logs or lagging alerts. Martin describes looking for changes and early warning signals rather than starting with a metric spike and trying to infer root cause. Jad then describes scanning as much as a petabyte of logs per day for new patterns, feeding anomaly signals to an AI agent that inspects code, searches history, forms causal hypotheses, and runs experiments.
- Early operational evidence is promising but remains far short of autonomous cluster management. Martin says the system works internally and in a few customer environments; there were cases where signals the team ignored later became production incidents. Jad believes AI is “very close” to replacing human query-optimization work, yet answers “not yet” when asked whether it can independently run the distributed cluster whose failure takes down the company.
- The clearest commercial wedge is measurable cost reduction, not an abstract promise of AI reliability. Companies were spending 5–20% of infrastructure cost on observability while receiving diminishing marginal reliability; Jad says Grepr cuts telemetry costs by 90% before using extracted signals for troubleshooting. Customers hear the reliability pitch, then return to the immediate budget line: “You said you could save 90% of our Datadog bill? … That’s what I want to buy.”
- AI’s expansion beyond digital domains will depend on constraining environments and finding an economic equilibrium. Jad argues that models can run experiments and work through real-world complexity; Martin remains cautious because self-driving took 20 years and $170 billion of investment to reach “worse unit economics than Uber.” Martin sees decades of value creation, but operational AI must prove that its workflows can tame reality cheaply enough to deploy.
1. Observability’s cost curve broke before AI offered an alternative
Jad’s company began with a 2023 cost problem: engineering leaders wanted to cut spending, and observability was repeatedly raised after infrastructure as a cost target. Companies spent 5–20% of infrastructure cost on telemetry because what they were really purchasing was reliability and cost predictability.
That bargain deteriorated as architectures fragmented into smaller, vertically integrated microservices. Telemetry penetrated more layers and grew faster than the functional units being monitored, while “the marginal increase in reliability for every additional dollar spent on observability just leveled off.” Observability increasingly became expensive noise and insurance.
Jad’s response was to extract signal at ingestion rather than retain everything downstream: place the system in the data stream, reduce telemetry costs by 90%, then feed the surviving signals into an AI layer for diagnosis and remediation. The strategic asset is not merely cheaper storage; it is a real-time stream of operationally meaningful events.
2. AI multiplies systems judgment more than typing speed
Jad watched coding AI move rapidly from Copilot autocomplete—“look, it autocompleted these tests”—into a workflow that taught capable engineers how to communicate with it. Different models and tools retain different context, select agents differently, and require different prompting habits; without constraints, Claude Code might “launch 20 agents” for an ordinary investigation.
The performance split widened rather than narrowed. In Jad’s limited experience, good engineers became much better, while weak engineers did not suddenly become strong ones: “The more complexity your brain can grasp, the better you can divide tasks and distribute them.” He explicitly limits the claim to a small team building distributed systems and algorithms, not a universal 100-person test.
Jad uses Windows, WSL, and Claude Code. He says he is using Opus 4.5, then later refers to a technique with Opus 5. He usually works across three or four concurrent session windows; more sessions become counterproductive because outputs arrive faster than he can preserve context. Persistent command-line agents can serve as implementers and technical leads, question one another, review PRs, and iterate against shared design documents.
Human work moves upstream and downstream. Jad reviews the PRD, design, algorithms, responsibility boundaries, and dependency graph before telling agents to begin; afterward, he reconstructs the result “like a human, point by point.” The validation cycle now emphasizes tests, execution, and benchmark changes because enormous PRs are easy to produce.
3. A 10-day build does not erase the gap between code and behavior
Martin’s strongest productivity example is production code for data-lake “auto-promotion”: turning arbitrary fields in semi-structured logs into independently filterable columns, with formats such as Parquet or Iceberg. The work involved metadata handling, commit semantics, and coordinating different authors. Martin says he completed it alone in about 10 days; previously, he estimates, it would have taken about six months. Jad calls that a 10- to 20-fold improvement.
Martin’s challenge is that generated syntax does not determine runtime behavior. Availability, correctness, consistency, caching, locking, and performance depend on loads, access patterns, schemas, configuration, and live data. Jad’s example is his Gaussian-splat distributed renderer, which he calls “complete crap” under AI generation because millions of splats must be loaded, unloaded, and streamed in a system-dependent way.
Martin finds models effective when behavior can be turned into measurable feedback. During query optimization, the model initially saw nothing wrong with duplicated conjunctions and produced poor code; once given concrete query data, user workloads, and performance metrics, it generated hypotheses and improved the implementation. Ask instead, “How much memory should I allocate?” and “it has no idea” without real experiments.
4. Effective AI-SRE separates anomaly detection from causal reasoning
Martin describes an approach different from AI-SRE products that begin with lagging indicators—a metric spike or an alert—and then try to infer root cause through a complex system. He says that is why those systems do not work well: tracing root cause after the fact is problematic. Instead, he describes looking for changes in the environment and early signals, then working backward to see whether they indicate a real problem.
Jad later describes scanning a petabyte of logs per day for new patterns. A predictive model detects anomalies and sends those signals to an AI agent, which can inspect the code producing a message, look for prior occurrences, test hypotheses, and decide what to do. The goal is not for the agent to predict what will happen, but to give it a signal that something changed.
Martin reports that the system works internally and in a few customer environments; in some cases, signals the team ignored later became production incidents.
5. The state space may be enormous while the debugging workflow stays tractable
Martin doubts that operations share code’s manageable distribution. Adding one bit doubles a program’s possible operational states; the count rapidly exceeds “atoms in the universe.” Rare failures, power-law tails, changing workloads, and hidden state—perhaps known only because “a colleague at the cooler” mentioned a deployment—make long-running clusters dependent on human intervention.
Systems history reinforces his caution. Designers repeatedly sought smart virtual-memory management, congestion control, and adaptive routing, yet production practice defaulted to surplus capacity: “We overprovision all the time.” Swap-to-disk meant the computer was effectively dead; operators bought more RAM rather than perfectly predicting demand.
Jad agrees that operations is “a much harder problem,” involving external loads, unannounced deployments, sequential failure states, multiple people, and distributed knowledge. Yet he argues that most incidents follow a change, and sufficiently complete telemetry can make that change observable even if the resulting failure state was never present in training.
Their partial synthesis is that the failure universe may be effectively infinite, but the investigative procedure is lower-dimensional. AI need not memorize every bad state; it can formulate hypotheses, request evidence, write experiments, and simulate irreducible steps. Martin allows that models may reach an “event horizon” where they solve everything short of what must actually be executed.
6. Real-world AI is ultimately an economics problem
Security raises the difficulty again because the rare state is not merely accidental: an adversary actively searches for behavior the defender cannot anticipate. Jad still applies the same loop—detect a statistical or behavioral perturbation, ask AI for plausible causal structures, then test which structure fits the observed data—rather than expecting one model to intuit the attack from raw logs.
Martin expects the fastest progress where solution and training distributions overlap: code, images, and other bounded digital domains. Real-world systems will take longer and cost more. His cautionary benchmark is autonomous driving: after Stanford’s DARPA Grand Challenge optimism, deployment took 20 years and $170 billion while producing “worse unit economics than Uber.”
Jad’s counter is that people can change the environment to fit automation, as with warehouses, rather than forcing AI to master an unconstrained world. Martin agrees that constraining the system changes the economic equilibrium: almost any problem may be solvable with enough computation and money, but only some solutions will support attractive unit economics.
The buyer already reveals that equilibrium. Jad pitches proactive reliability, yet customers interrupt at the concrete saving: “You said you could save 90% of our Datadog bill?” Coding sells easily because developer productivity maps directly to a major cost center; operational AI remains early, harder to value, and commercially strongest when attached to an immediate budget reduction.
Full transcript
The world that artificial intelligence lives in is entirely digital, and there’s also the real world. These things aren’t necessarily the same. When we started using AI for programming, it was autocomplete. No one really knew how much AI could do to create complex software, but it has actually exceeded expectations as we’ve been able to model increasingly complex systems. I’m just not sure if operations are restricted in the same way.
Where can you get the data to learn how to build a system that understands all these operations?
I assume we have all the data. The question, I think, is complexity, and the extent to which you can take the operational things you describe and turn them into something AI can understand. That’s the main secret to the long-term management of a distributed cluster with only human effort.
Do you think we’re at the point where we can use AI to launch one of these services?
I think we have enough of everything to—
1. From Stanford to founding Grepr
So, Jad, thank you for joining me on the podcast. For those who don’t know, Jad is the CEO of Grepr, and we were in the same graduate school group together. When we first met, you were a new student. I went into the office, and there was a motorcycle helmet and a pack of Marlboros. I thought, “This guy must be incredibly cool.” So my first question is: Where did all your coolness go?
2. How AI is changing engineering
Maybe, for context, give us a brief overview of your experience. For those listening, Jad is one of the best systems thinkers I know. I love talking to him. We’ll delve into engineering from a systems perspective and discuss how AI impacts us, particularly when it comes to operational issues. But first, let’s talk a little about your background.
Of course. You met me at one of the peaks of my coolness.
Yes, you were very cool. Motorcycle and Marlboro cigarettes. I remember thinking, “Wow, are cigarettes even legal?” It was so unusual.
I remember Nick McKeown seeing me one day smoking outside the Gates building and looking at me. Nick McKeown was our scientific director; Gates is the building.
Yes. In fact, one of my favorite things—do you remember Professor Balaji Prabhakar?
Of course.
The thing I liked most about him was that he smoked. He was like the only professor who did, and I thought, “That’s cool.”
Just to be clear, I don’t smoke anymore. I’m too old, too middle-aged for that. I went to Stanford with you guys, and you were always the one I looked up to and thought, “Oh, this is who I want to be when I grow up.” I guess I started four years before you. So I was ahead and you were four years ahead of me. You graduated a little earlier than me. You went and founded Nicira. I stayed and lingered, working more on OpenFlow.
I helped build a bunch of things on the hardware side from a security perspective. One of the most interesting projects I worked on was creating the first set of SHA-3 implementations that people were considering for use. I built them at the hardware level to show whether they worked, for one of my thesis projects. It was fun.
Yes. Class.
Then I decided after a while that I didn’t really like networks, because they were just moving bits from one place to another. How difficult can it be?
I think you either have to philosophically agree that it’s profound, or you don’t. Most people think not.
I think that’s very profound. I mean, there’s something interesting about it, but ultimately, it’s not for me.
It doesn’t bother you?
Yes. Then I went to do a postdoc at MIT with Nikolai Zeldovich, who is also incredibly deep and very smart. He’s phenomenal. At the time, I was working in application security, starting a company based on research I was doing with Ramesh Chandra, who was a graduate student at the time. But the technology turned out to be too early.
What was that?
A lot of the work was actually Ramesh’s, and his thesis was about this: If you were attacked and hacked, could you recover by rolling back the state of the entire world associated with that application, and then rolling it forward with an updated version of the application to remove the exploit?
We started doing it in the real world and thought, “Oh, that sounds really complicated.” This is really about what we’re talking about: to what extent can you constrain state for operating systems, which I think is very difficult. I’ve almost made it work, almost. I feel like, for all niche disciplines, this is the reason people fall into the smart-ass trap: you see a little bit of progress, but… Anyway, I digress. Let’s continue with your—
Then I joined AppDynamics and was there for about 3 and a half years. The company grew from about 200 to 2,000 employees and was acquired by Cisco. I was a Cisco employee for a day.
And then you’re like, “Jad, I know. I love you. Come work with me. Come do this venture business with me.” I’m like, “What the…? What is a venture? I’m an architect. I do engineering stuff. What are you—”
He’s like, “Great.” The less you know, the better. That’s literally what you said.
This is true. The first day I started here, you said, “Slow down. People who move from operations to venture capital need to do a lot of brain rewiring.”
Yes. By the way, it’s just like the transition from a PhD to operational activities. You have to sort of repair the “damage” to your brain.
This is my experience. This is what the real world looks like.
I know.
I stayed here for a few years and did a lot of work with you. It was a truly wonderful time. I worked with many great people like Jennifer, Mark, Ben, and Satish.
Satish. Matt was here when I first—well, almost toward the end when I was leaving.
Yes. Then I joined Apptio as a product director. I helped build their engineering and product teams, scaling the team from approximately 15 engineers to over 90 across product, engineering, information security, and design. After about 3 years, I had a manager for each of the areas I was managing, and I was just sitting there coordinating and having meetings all day, so I wanted to do something else.
I started talking to a lot of engineering leaders about the challenges they were facing and what they were trying to do. Back in 2023, everyone was saying, “Hey, we’re trying to cut costs.” You’d ask them, “What do you spend your money on?” Everyone answered, “Observability.”
3. Why observability got so expensive
First it was infrastructure, and then observability. I started looking into this problem, trying to understand why people thought it was too expensive. It turned out there were several reasons. First, people spent 5% to 20% of what they spent on infrastructure on observability. But what they were really trying to spend that money on was reliability and predictability of costs, and the reliability of their infrastructure and applications.
Of course. Of course. Of course.
They invested in observability as an indicator of reliability. For a long time, the more money you spent, the more reliability you got. But architectures have been changing and evolving into microservices, which are much smaller and much more vertically integrated. The amount of data collected at each level has increased. Observability has penetrated deeper into the market, and the amount of data being collected, along with the cost of observability, has started to grow faster than the functional units you get.
The marginal increase in reliability for every additional dollar spent on observability just leveled off. That’s why people found it expensive, but they were still willing to pay 5% to 20% for reliability. So the question was: Can we actually solve this problem and get that marginal reliability gain back, not necessarily by spending more money on observability, but in some other way?
The thought was: If we can figure out what’s signal and what’s noise, can we transform the observability data you collect, which is currently mostly noise and insurance, and do this information extraction early on? Can we do everything that’s needed on the input, and ultimately significantly reduce the cost of observability on the output?
If you can extract this signal in real time at the input, then you can reduce costs and feed it into other systems to use it to ensure reliability down the road. By that time, something called AI had come along and was starting to come into play. We thought, “Great, if we’re in the data stream and we’re extracting a signal, then we’re in a strategic position to feed it into whatever comes up in the future, because it’s going to be needed.”
Yes.
That’s exactly what we built. Grepr helps companies massively reduce their telemetry costs. You put us in the data stream, connect, run the data through us, and cut costs by 90%. Now we’re building this second layer where we feed those signals into the AI layer, which does the troubleshooting: figuring out when something’s going wrong and fixing it.
I’d like to talk about your experience with how AI is changing development in general, because this is a question a lot of people are asked, and a lot of people are answering. But I feel like you, as an engineer for engineers, have been an architect, managed product, and managed engineering teams. How big is your company now? How many people?
4. Jad's AI stack and workflow
You build a lot of software with a relatively efficient team, so maybe give us a general outline. If you were a senior engineer with an architectural background, what kind of deliverables do you think people would expect? Maybe tell us how you use AI, what benefits you actually see, what your stack and process are, and how you think about it.
I can give you a historical perspective on what was going on with team management. When we first started, AI was still kind of autocomplete. I remember we were using Copilot in IntelliJ, and we were like, “Oh my God, look, it autocompleted these tests.” We were also like, “Yeah, but it’s just autocomplete.” Very quickly, things started to evolve, and I noticed that really good engineers were able to work AI into their workflows in a way that let the AI teach them.
Oh, interesting. In terms of how to talk to it and get the best out of it, do you think it depends on the model? Do you learn to use AI broadly, or do you become effective with a specific model?
I think there are certain nuances inherent to different models. There’s a way to work with AI, and I think there’s also a way to work with different tools. We’ve tried, for example, using OpenCode or different environments, or Codex versus Claude Code. Different tools have different ways of managing context, thinking about tasks, deciding when to run agents, and determining what to tell them and what queries to use.
With Claude Code, I’ve noticed that if I don’t tell it not to run 20 agents for a normal task, it’ll launch 20 agents to explore and burn all your tokens. You start to learn the environment and the models, how they interact with each other, and how they interact with the rest of the world. But I digress. I started to notice that some people adapted and learned to work this way, while others weren’t as flexible. When we started hiring more people on the team, the difference between good engineers and really good engineers multiplied tenfold.
Was it possible to predict who would be a good specialist? Did good engineers become better and bad ones not, or what was it really like?
If you’re a good engineer, you become a better good engineer. It’s not like some bad engineers suddenly became good engineers. In the end, it always comes down to really understanding the setup. The more complexity your brain can grasp, the better you can divide up tasks and distribute them.
I think that might be a non-consensus opinion. I hear it sometimes, but the idea that better engineers become much better is less obvious. Do you think that’s really a universal truth?
My team is still small; we haven’t tested it on 100 people. But in my experience, for what we do, that’s exactly what happens, because we’re building distributed systems and algorithms, and your brain has to deal with complexity. AI doesn’t do that, right?
Yes.
For the tasks we solve, no model can really think through what we do. For example, we have a query translation layer. We just talked about query optimization. You need to really understand the semantics of the query, how to get to those semantics, and then explain that to the model so it can create, say, a translation layer.
For those of you listening who may not be very computer science savvy, what do you mean by semantics?
Semantics is the meaning of things. For example, the semantics of the word “dog” is an animal that walks on four legs and barks. Syntax is that it’s just a three-letter word, made up of three different letters. The form of a word is different, and words can vary, but different words can mean the same thing. They can have the same semantics but different syntax. “Dog” and “puppy” may have slightly different semantics; they point to the same thing, but they’re still both dogs.
In computer science, I describe it very simply: syntax is the characters you type, and semantics is what the machine actually does. Those are the actual properties of the system at runtime.
So you studied computer science, I studied electrical engineering, you became a programmer, and I became a venture capitalist. That’s the problem. Got it. Anyway, I just wanted people listening to understand that syntax is writing code—the code itself. When you talk about semantics, you mean that a machine interprets the code, whether it’s compiled or not, and does something. Semantics are what the machine actually does.
Those things are clearly very different. Some semantics can’t even be deduced by looking at the syntax. You don’t know what the machine will do until it’s tested, and during testing things can go completely differently. A classic example is SQL: you can write a query, but how it actually executes and how long it runs depends on things like the schema and what data is in the database. I wouldn’t call that semantics; I’d call those performance characteristics.
Oh, of course. I’d say the semantic part is that, depending on the form of the data or the configuration, you can get different semantics. As you said, you might want null to be treated as false.
Okay. For the purposes of this conversation, let’s just say there’s the actual code you write, and there’s how the system behaves. We’ll call the second one semantics so we can have a conversation.
Well, what do you want to call them? We need to delve into the semantics of this and make sure she understands.
That’s right. Because I know a lot here. Okay, let’s stick to the general features. I apologize. Let’s go back. Before we started the podcast, we were chatting, and you were talking about how you can now implement projects in a week that used to take the entire team months of work. Can you tell us a little about how you personally carry out projects and what results you get? Tell us a little about your stack and process.
I use Claude Code. I’m going to say something pretty funny: I use Windows.
What? Was this created by Microsoft?
I use WSL, so I use Linux for everything—development and so on. There’s no X or graphical interface; I just use the command line.
How do you use Windows? I never really wanted to learn keyboard shortcuts. Are you a gamer?
It’s like switching tabs in Counter-Strike or something.
Well, why else?
I don’t hate Windows. I used it in the ’90s. That’s really the only reason: I never wanted to learn these abbreviations. I hate the one-button mouse. I hate these one-button things. I hate the number of modifier keys for shortcuts. I just never… I’m like, why waste time?
Okay.
So yes, I use Windows and WSL. I use Claude Code.
What is WSL?
It’s Windows Subsystem for Linux. They actually run the Linux kernel on Windows through a hypervisor and give you access to it.
This is great. They did a great job.
Windows, WSL, and Claude Code. The last thing I’m doing is using Opus 4.5. I’ll tell you, it’s a very good model. It’s an amazing model.
In the last 2 weeks, I’ve released these huge 20-point PRs that I just dump for my team to review. They’re like, “What?” and I’m like, “Don’t worry about it.” It’s like a new reporting structure where subordinates check PRs for the CEO.
Have we gotten to this point?
No, everyone writes the same PRs too.
I check my people’s PRs, and everyone’s just—PRs are flying everywhere.
What’s changed a lot in the validation cycle is that we’ve started to focus much more on tests and execution: benchmark expectations, whether they change in different ways, and so on. With Opus 5, a trick I recently learned is that instead of using subagents, you tell Claude to run command-line subagents, because they maintain their persistent sessions over time. You can run them separately from other sessions, and that lets you be much more effective with context.
How many agents are you running?
I usually work in 3 or 4 session windows at a time, because I’ve noticed that with more, I start to lose context between different things. They start to complete faster, and then they wait.
And they interact through PRs? Does that work?
Together? No, they interact directly through the command line.
Oh.
Of course. For example, I ask the main agent doing the implementation to have a technical lead reviewer, and they actually ask it questions: “Does this make sense? Does that make sense? Check out my PR,” and so on. They iterate on these PRs together, and they also have a PR document they’re working on, with a design doc and everything.
The first thing I do is ask the agent to do that. I ask the tech leads to review the PR document. I review the PRD. I review the design doc. I make sure everything makes sense. I go through each part in detail and make sure all the algorithms work, all the parts work, the division of responsibilities works, and the division between PRs and their relationships works. Then I tell it, “Just start doing it. Figure out if something can be parallelized, and then launch other agents to do it.”
5. A six-month project in ten days
But eventually you have to put everything back together and go through it like a human, point by point. Otherwise, it gets confusing and you just can’t handle it. How long does a project you’re doing now take compared with, say, 6 years ago? What’s the difference in time?
The last project I released—just to be clear, this is industrial code. These aren’t demos or anything like that.
So you’re talking about the benefits you get when you release it into production?
Yes. I’ll give you an example: the last PR I worked on, or the last set of PRs. This is what we call auto-promotion in DataFusion. In a data lake, you store all the data, usually in columnar files like Parquet or Iceberg. When you get data from logs, it’s semi-structured and has arbitrary columns.
To get the best data structure and query it efficiently, ideally each field in a log event becomes a separate column that can be filtered independently of the other rows. We call this auto-promotion, or splitting. Doing something like that is incredibly difficult. You need to figure out how the metadata system and commits work, how to coordinate the work of different authors, and so on. This took about 10 days. Previously, it would have taken about 6 months.
Wow. For one person. Was it just you, or—
It was just me.
Madness. That’s a 10- to 20-fold improvement.
Yeah, for production code in new products.
6. Can AI reason about operations?
One question that always comes to mind: I can understand that these models are trained on all the code that’s ever been written, so I understand why they can generate code syntactically. But when you work on the underlying infrastructure, say, distributed algorithms, the properties of how the system behaves in operation are very different. Often, you can’t just look at the syntax to understand what’s going to happen.
If you’re writing a distributed system, how it works—its availability and even its correctness—depends on what the load looks like. What are the access patterns? What’s the traffic like? What trade-offs are needed if the model doesn’t know what the access patterns will be? Where does it get the data to learn how to build a system that understands all the operations? Today, we rely on people to manage clusters and optimize databases. How do you apply AI to that problem? Have we made progress, or is it still an open question?
Are you talking about the performance aspects, or do you mean—
I mean availability, for example. Maybe you have to block certain things, so they’re unavailable for a certain amount of time. I’m also talking about correctness: maybe you decide to use eventual consistency, so when you read the data it won’t be consistent because there’s no locking. And then there’s performance: maybe you use a cache, maybe you don’t. Those are all trade-offs in the design space, and until you know the access patterns, you won’t know how it’s all going to work.
I can understand how AI would write code for a typical distributed data store. What I don’t understand is how it will know what trade-offs are needed if it doesn’t know the access patterns. These things can’t just be inferred from the code. How do you look at this? I understand writing code, but there are complex operational systems. How do you apply AI to this problem, and have we made progress, or is it still an open question?
I guess the question might be about complexity: to what extent can you take the things you described that are operational and happen in production and turn them into something AI can infer? How can AI draw conclusions about such a system?
Well, first of all, to what extent do you see that, when writing code, it can actually predict what will happen? How well does it handle that? And second, how do we get it to draw conclusions about this? So let’s ask the first question: if I use your method—the Jad method for writing code—and build a large distributed system, how often does it hit the target in terms of performance? How often is it relevant from an operational point of view?
By the way, I’m working on this Gaussian splat distributed renderer, and I’d say it’s complete crap. It’s terrible because you have millions of Gaussian splats that need to be constantly loaded, unloaded, and streamed, and it’s completely system-dependent. No one model does a good job.
In our world, we have very well-defined performance tests, how the system should work, and the build constraints within our stream processor to get proper performance. My last project was all about query optimization and query performance. I said, “Here’s a classic example: we’re generating code that conforms to SQL, and we want to run it in a pipeline. But you’re generating crappy code. Fix it.”
It started checking, and what it didn’t realize was that it wasn’t doing any code optimization in the first place. It was like, “Oh, it’s fine. What seems to be the problem?” I said, “Look, there are these conjunctions here. You use them, and this particular term is repeated. It should have been removed.”
It said, “Oh, okay, let me figure it out,” and started trying to work through the options. It built a bunch of hypotheses about what could be optimized to make it better. Because it was able to justify the changes with performance metrics, I said, “Here’s the query dataset and what users are asking for. Work on it.” It can handle that pretty well.
But when it comes to questions like, “How much memory should I allocate to this system?” it has no idea. That has to be based on real experiments in a working environment. I think these systems are doing pretty well with operational tasks right now because they understand, “Hey, I probably can’t answer this question. Let me do an experiment and get that information.”
7. How Grepr catches incidents early
But production systems don’t behave the way you expect them to, and that’s ultimately why incidents happen. Have you found—do you do something like AI SRE, where you have a running system with logs that you feed into the AI, and it helps you tune it?
Yes.
How well does that work? Anecdotally, my experience with other people doing this hasn’t been that great. The reports I’ve been getting haven’t been very encouraging.
A couple of points. We do this for both operations and security, but our approach is different from what others do from an AI SRE perspective. A lot of these systems are based on lagging indicators, like, “Hey, this metric jumped,” or, “An alert went off.” They look back through a very complex system and try to figure out why it happened. Of course, that’s why they don’t work: tracing the root cause is very problematic.
Instead, we do the opposite. Can we start by figuring out if something has changed in your environment? For example, can we do scale-out clustering and find a needle in a haystack—something that’s different? You assume everything was working fine before and now it isn’t, so something happened. But it could just be a change in normal traffic patterns, which is also fine and normal.
What we’re looking at is that when something goes wrong, the system usually tells you something bad has happened. If everything is working fine and scaling as usual, nothing will happen. But sometimes you see a message like, “Error, connection timed out,” or an out-of-memory error, or that a resource buffer is empty. Some indication, some early warning that something has happened. We want to intercept that message, then work backward and see if it’s a real problem we need to investigate and solve.
How’s it working so far?
It’s working pretty well. We use this internally in our systems, and we’ve run it in a few of our customers’ environments. There have been cases where we ignored it, and those turned into production incidents.
Really? Wow.
Yeah.
From an intuitive perspective, I can understand how these models understand syntax because, as I said, they’re trained on all the code that’s ever been written, and it turns out that covers a lot of useful cases. It’s hard for me to even imagine a limit to the actual operations of software. It sounds like a crazy statement, but it’s partially true. Every time you add a bit to a program, the state space of what it can do operationally doubles. Very quickly, you get to the point where there are more states than atoms in the universe.
That’s probably why product-based approaches to operations have never really worked. Chaos engineering has never worked properly, and we’ve always relied on humans, because the secret to running a distributed cluster for a long time has been human intervention. To train an AI to do that, you need enough data and configuration. I wonder if we’ve reached the point where you can use AI to run a distributed cluster or services, or if that’s still a future prospect.
I think there’s enough data to start that cycle. Ultimately, I’d probably agree with you: there’s a lot of space and a lot of nuance, but people ultimately follow instructions too. Of course, there are a lot of things that happen in production that require human intervention: “Hey, this is too complicated.” There’s hidden state that I know about because I was talking to a colleague at the cooler, and he told me about something I recently deployed.
There are a lot of things that may not be reflected in the state of the system.
Yes, but I’d argue that our goal is to get enough telemetry and insight to make everything visible: what the system looks like, how it works, and everything else. Most problems that arise during operation, if the system was working fine, are related to some kind of change. If you focus on detecting exactly what changed, then—
Excuse me, can I ask a stupid question? I’ve never thought of generative models as being good at predicting, but you seem to be talking about predictive modeling: predicting change or defining a baseline to detect anomalies. Am I misunderstanding that?
No, that’s right. The goal isn’t for the agent to try to predict, but to get a signal that something has changed.
Oh, deterministic. Now I see. Got it. Wait, so there’s a separate model for predictions that then notifies the AI?
Yeah. What we do is this: you give us a petabyte of logs a day.
Yeah.
We scan them all, and every time we see a new pattern in the logs that we haven’t seen before, we feed it to the AI.
Sure.
It’s a predictive model that does what these models are good at: it detects anomalies, and that data goes to the AI, which decides what to do next.
That’s right. So the AI can reason and direct it.
Sure. It can say, “Oh, this message—let me look at the code it’s coming from and figure out what the code is doing. Let me see if this has happened before.” It tests all these hypotheses and heuristics to see if they make sense, whether they’re meaningful in any way, and then determines what to do with it.
How close are we—let’s go back to query optimization, not logs—to AI doing the work humans used to do in query optimization?
I think we’re very close.
Really?
Yeah.
Wow. I think about running a distributed cluster. Imagine this: James Cowling, a PhD from MIT, runs a distributed cluster. If the cluster is down, the company is down; if it’s up, everything’s fine. I don’t know if we’re at the point where we can say AI can do that.
No, not yet. Look, it’s very simple: when we started using AI for coding, it was autocomplete.
Yeah.
Nobody knew how complex software could be built with AI.
Yeah.
When we looked at it, we thought, “AI will never be able to build X, Y, Z, or whatever.” Then it surpassed expectations, because we’ve been able to model more and more complexity in systems. I think about it quite simply: these models work on the principle of “distribution in, distribution out.” It turns out that you can learn from all the code that’s ever been written, and this manifold is actually quite manageable.
I just don’t know if operations are constrained in the same way, or if they’re more like the universe, with heavy tails, exceptions, power laws, and all the things that are so hard for these models to learn. But I don’t know. You’re in this field, so maybe they’re manageable. I’ll tell you, though: the history of systems is that you always want to build a smart algorithm that works efficiently—smart virtual memory management, smart congestion control, smart routing. And what do we do instead? Every time we build, we overprovision resources. Literally, every time.
Remember swap memory, when data gets flushed to disk? If you did that, you were done for. Your computer was dead. So we overprovision RAM. And routing—how many adaptive routing algorithms are there? We react to traffic, but there are fluctuations, and we just overprovision. The history of systems is that when you don’t predict, you overprovision because you can’t. In operations, I feel like we either build systems that are overprovisioned from the start to avoid complex operations, or maybe AI can do it like a human. But I think it’s very different from code because you’re dealing with exceptions, data is scarce, and so on. It just feels like a different realm. But I don’t know. I’m a venture capitalist.
I’m more optimistic here because I live in a world where we’re dealing with the real world, which is observable.
I was going to say, you’re a real expert on this, so it’s pointless for me to even ask you questions like that.
No, not at all. In my world, I assume we have all the data, and we know exactly when something new happens, an exception is thrown, the workload changes, or whatever. It’s all already been collected and streamed.
But a model that isn’t trained on a solid source for things like this—
No, it isn’t. That would be really hard, wouldn’t it? But the point isn’t to say, “Hey, Mr. Claude, here’s a bunch of logs. Is something happening?” It should be much more capable of running experiments, digging into the data, and seeing what possible causal relationships fit the given environment.
I see. That’s what you were talking about earlier. You’re saying the universe has a heavy tail: the number of bad things that can happen is probably exceptional, but the set of steps to set them up is manageable. That’s why you can train one of these models. If you give it the right signals, it can go through a series of steps to constrain the problem.
That’s right.
Interesting. So it really takes on the role of a human. Remember that whole wave of chaos engineering? The reason it didn’t get anywhere is that you tried to shrink the state space—something like the atoms in the universe—really quickly and make it manageable. A human was going through the steps to debug, and that’s what you’re saying the AI will do.
Yeah. The goal isn’t for the model just to say, “Here’s what happened.” It’s to say, “Run this workflow to determine what happened.” We try this in security as well. You might see something change in your statistics, like a behavioral difference in the entities or the cardinality of different things. That becomes a perturbation in the entire universe. We take that signal, feed it to the AI, and say, “Come up with a bunch of possible causal relationships, then see if any of them fit the structure of the data we’re actually seeing.” It becomes more like fitting a square peg into a hole, or finding a hole that fits the peg.
8. Why security is the harder problem
That seems even more complicated to me. With operations, the argument is that the system can go into bad states that are quite rare, and there are a lot of such rare states. You can’t train a model on them because there are too many bad, rare states it can go into. Maybe you can reduce that by taking certain steps. With security, it seems even more complicated, because it’s not just the natural order of rare states. You have an adversary intentionally trying to find rare states and things you can’t do. The code can be broken. Operations can take a little longer, and sometimes a lot longer.
Yeah, but think about this too: it seems to me that the whole world is quantized to some extent. The Planck length—is that what physicists say? I don’t know. I’m not a physicist, and I don’t really know what I’m talking about, but I think spacetime is discretized by the Planck length. I agree to some extent with what you just said, but for operations and systems in general, there’s a discretization of possible states and paths through them.
Yeah. I just think that when you’re dealing with them, there could be a huge number of states. I want to tell you—remember Sundar, who was in our group in graduate school?
Yeah.
He came to me one day looking really sad. He said, “Martin, I’ve just come to this sad conclusion. Any MP3 has a finite size—say, 4 minutes, about 4 megabytes—and that means there’s a limited number of songs in the universe. It might not, because it’s all made up of bits, right? And it’s 4 megabytes, which means you could just list every single one of these concepts.” He was so upset about the idea of music being limited. We were walking around the Dish when we had this conversation, and I said, “Oh my God, dude. Do you know how fast you surpass the number of atoms in the universe? You don’t even get to 1,000. That’s like 2 to the power of 2 to the power of 2 to the power of 2. What are you even talking about, man? Four megabytes—what are you talking about? That’s so many states. It’s effectively infinity. What are we even talking about here?”
So I really think that at some level we’re underestimating the number of states systems can go into. Maybe I’ll tell you, from the perspective of a venture capitalist looking at these companies: those trying to do operations have a hard time. It seems much more difficult than working with code, because I think it’s a much harder problem.
I agree 100%. It’s a much harder problem, and not just because the systems themselves face random external loads, or someone releases a new thing, puts it in production, and doesn’t tell anyone.
Or there’s a certain state that occurs where one thing happens, then another, and you’re stuck. It’s also because it involves a lot more people, not just one programmer. It also involves distributed knowledge.
That’s right. And the interactions involved in connecting the pieces and putting them together. But ultimately, I think about where AI is and the possible futures, the depth and foresight of all these things. To me, code is just a microcosm. Maybe I’m too fatalistic, maybe I’m too pessimistic about how the world works. I think everything happens in some probabilistic way, and if we can make this work in code, we can keep adding layer upon layer of complexity until we can solve a lot of people’s problems.
That’s not fatalism. It’s the opposite. People would say that’s very optimistic.
I don’t think the universe is giving up its secrets that easily, but we’ll see. It’s a big question, and I’d say there’s no answer to it yet.
No.
Yes.
No. But I don’t know—a lot of things that we humans think are random or impossible to solve…
Yes. And that’s what we’ve seen with coding. Initially, people thought the world of possibilities for what could be implemented in code was limited, and that you couldn’t build a system that could capture the full complexity of the human brain. But I think that, over time, that’s all—
It’s so funny that I’m telling you this, and you’re a venture capitalist looking into the future and saying, “No, well, I’ll give you my take.” My take is that when you train these models on data, they represent a certain distribution. Is that distribution useful or not? When you train on language, the fact that it’s useful tells me that people are incredible. They’ve looked at the universe, which is this multidimensional system, and reduced it to a manifold that’s actually useful for a model.
Yeah.
Right. But it’s different from the universe. It’s a reduction from this high dimensionality to a language that’s much less dimensional. I think code is a reduction of all the operating states of a machine to the instructions that make it work. So, for me, there’s still a gap between the instructions or the description and the actual functioning—the actual operation of the system. I think we can make some progress in that direction. But I will say this: you are kind of getting closer to simulation. There are things that are computationally irreducible. You have to run the process to get the answer. You can’t just predict it. Maybe we’re at the event horizon where models will solve everything that doesn’t require a simulation to get the answer.
But why should you only think about it in terms of having to answer the question right now, rather than being able to model and plan its own execution to get the information it needs?
No, maybe that’s where we’re headed.
Yeah. That’s kind of what I mean.
Yeah. And I think there’s a lower-dimensional space—the steps you would take—and AI can reduce those steps a lot. For the steps it can’t do, it can code to do them or simulate them.
But your idea is that the model can’t really go and figure out the extra dimensions of the world that we need to find in order to model the universe in its world.
That’s probably true. But can it model a scientist well enough to get to that point? Maybe.
Can I ask you a question? How much of the job of a venture capitalist do you think AI can do?
My model of venture capital is based on relationships.
I think so.
Okay. I just believe that markets are actually quite efficient. It’s not about finding a company that nobody else is finding. I think everyone finds the right company. I also have a model of venture capital where technology is positive. I’m optimistic about all technologies, and I’m not concerned about the market size, or TAM. I think the key is access. It’s like fundraising: access, the ability to win deals, and the ability to add value after that. A lot of that is stuff startups just can’t do naturally, like hiring great executives or helping with that. To me, VC requires being human, because that’s what real work is. If you gave AI a simulated universe where it could break the reward system, like building trust, then fine. But I’ve seen analysis done by AI that’s as good as or better than any analyst I’ve ever worked with. On the analytical side, they’re great. But I think that’s more a testament to the limitations of VCs’ ability to analyze than to how powerful these models are. I just think it’s all a show and a bunch of nonsense. The right VCs understand that technology is a net positive. Technology waves always add value, so let’s find the best company and put money in it. Markets are very efficient, so I feel like the answer to that question is almost a technical formality.
So, one of the things that got me here was wanting to ask you this question. We started talking specifically about operations.
Yes.
But to me, there’s the world of AI writing code, which is kind of an abstract simulation world that it lives in.
Yes.
And then there’s the rest of the world.
Yes.
Exactly. I think that’s what the whole discussion boils down to. There’s this world that AI lives in, which is completely digital, with training data and an internal distribution, and there’s the real world. Those things don’t necessarily match. Quite often, they don’t. So the question is to what extent AI needs to be trained to survive in that world, and what things actually… I don’t want to answer exactly what needs to be trained, but I’m interested in open spaces. We talked a little bit about operations. You said, “I don’t know if it’s going to happen.” And I say, “Yes, Martin, it’s going to happen.” There are so many other things, like self-driving cars. For a long time, we thought it wasn’t going to happen, but it’s clear that it’s going to happen. It’s just going to take time because of a lot of different factors. So what do you think are the next steps and areas that are coming up right after code?
9. What comes after code
My feeling is that AI is going to continue to accelerate in limited areas, where the solution space is also limited, right? Where the distribution of the solution space matches the distribution of the training set, where the syntax of the code matches the training set, where the generated images match the training set—those are the areas I think are going to accelerate. In areas where the distribution of solutions depends on the real world, I think it’s going to be much more difficult, take longer, and be much harder to make cost-effective. Autopilot is a great example. I remember Sebastian Thrun won the DARPA Grand Challenge when I was at Stanford. Remember that? We were like, “Yay! Self-driving cars are just around the corner.” Seriously. And it took 20 years and $170 billion in investment to get worse unit economics than Uber, right? Solving real-world problems where you have to deal with the distribution of the entire real universe is just much harder. I think of AI as a universal tool. It’s a very effective tool: you have a distribution of data and hope to apply it to a specific problem. We’re going to apply it to a lot of things, and I think the economic equilibrium is going to determine what works and what doesn’t. It’s a very general tool, like a universal function approximator. Whatever that leads to, one of the things I was talking about earlier is how AI teaches people to use it.
Yeah. I often think about the fact that some problems seem insurmountable because that’s the way the world is, and we’re trying to adapt AI or the tools we build to the way the world is. But in some situations, we tend to change the world to match what we can actually do.
One hundred percent. Of course, I think that’s the history of robotics.
Yeah. Automated warehouses. The question that always comes to mind when we look at these warehouses or things like that is: the only thing that’s worked is this approach—just automate the hell out of it. I’m wondering to what extent, as we improve or change how we work, or expand the scope of what we want to solve with AI tools, we actually start to model the world or change the world to match what it should look like for AI.
Yeah. I mean, listen, I think a lot of AI right now is just working in the digital world. For it, literally, the solution set has the same distribution as the training set, in a sense. And that’s fine. In the context of equilibrium, when you’re dealing with the real world, you can solve any problem you want. It’s just a matter of computation and money, and the equilibrium point is going to be economic. One way to affect the function you’re looking for in that equilibrium is to constrain the environment, to constrain the system. I think we’re in this metaeconomic flywheel of equilibrium right now, where we’re optimizing this super-general but very inefficient process with a solution set that people value. We’ve got 30 years ahead of us, and there’s going to be a lot of value created. Want to hear a funny story?
Yeah.
We built this AI thing, and we’re talking to people. We go to the customers and say, “Hey, we built this AI. We help customers become more reliable. That’s what we do.” They say, “Great. We want to talk to you.” Then we get on the call and say, “Hey, here’s what we do.”
You know, we built this system. It helps you optimize your data and reduce your costs. It also helps you become more reliable because now we can find these problems and fix them before they become an incident. They say, “Wait, wait, wait. Back up. You said you could save 90% of our Datadog bill?” We say, “Yeah.” They say, “That’s what I want to buy.” We say, “But what about AI?” And they’re like, “No, no, no. That’s what we want to buy right now.”
Yeah. And it’s interesting: for code, I always think about it. People just get it. They’re like, “Oh, shit, it just works.”
Yes, yes, yes.
And from the operations side, or anything else, it’s very hard to understand the real value, to put a dollar value on it and to pay for it.
One hundred percent.
So I also think, from a coding perspective, it’s literally developer productivity, which is one of the highest cost items. So it’s very straightforward.
Exactly. But you can also think of it as SRE productivity or, for example, business failure. Maybe you can prevent an incident and figure out what the cost is. I don’t know. It’s weird how people think about it outside of coding.
I also just think that code was so magical because it was so good, and nobody thought that was possible. And I think the operations stuff is still pretty early on.
It’s still pretty early.
So, yeah. Okay. Well, listen, it’s been great chatting. Thanks, Jad.
Thanks.