Why Big Tech Buys GPUs From CoreWeave | Corey Sanders

1. Product Development and Customer Feedback

I don't care if the APIs are consistent and commoditized. The level of quality, performance, and capability and experience that we deliver today will not win workloads in 2 years. For anyone who's deployed on a public cloud, especially with GPUs and you suddenly have a burst of capacity, like you're always sweating. And so what I like to think about for us is how do we go make all of that complexity then go away? I think Core's already got that. CoreWeave is the best place to run training AI workloads [Music] and it's why people use us.

2. Introduction

You're listening to Gradient Descent, a show about making machine learning work in the real world. And I'm your host, Lukas Biewald.

All right, today I’m talking with Corey Sanders, who is currently the SVP of Product at CoreWeave and, prior to that, was a longtime executive at Azure, where he worked on compute projects and a whole bunch of other things. It might seem like maybe this is sponsored somehow, but this is actually a conversation I would be excited to have regardless of where I was working. You should be aware that Corey is one of my colleagues now that Weights & Biases has been acquired by CoreWeave. But we try to stay as objective as possible and keep it interesting. I hope you enjoy it.

All right, Corey, do you want to sing us in? Should we do this?

The—I can be on your podcast? This is a little beat that I created, begging Lukas to allow me to join this session.

Yeah, and I guess we’ll put a lot of disclaimers in here, but we’re now coworkers. This certainly is not an unbiased interview. And I guess I haven’t really—

Biases, Lukas.

We’re all about biases here. I haven’t mentioned this in the past, but since Weights & Biases was acquired by CoreWeave, we actually run all these podcasts through CoreWeave Compliance. I’m a little disappointed that we’ve never triggered anything, including the episode with Martin Shkreli, who’s a little controversial. We try to make this interesting, but we’ve never had Compliance complain about anything. So maybe this could be our first time.

Let’s go for the win here, Lukas.

There are going to be big, quiet blocks. We’ll just chop it up.

Or really, really jumpy chops. [Laughter]

Yeah. Look out for those.

I won’t answer that stupid question.

3. The Evolution of AI Workloads

It’s also funny researching you. As I tried to Google you and find out about your past, I noticed that you were doing this amazing series of videos for Azure where you were taking what I would consider boring topics—even for a technical guy. I’m just like, “Man, this is…” The video was on backup.

And it wasn’t just backup. It was low-cost backup alternatives for a particular situation. I’m an Azure customer, and I’m not really that interested in this, but then you sell it so hard. You make it actually interesting. It’s amazing. I’m like, “Why am I watching this?” Then something happened. I feel like maybe you got a promotion or something, and then you were flying around the world, going from terrible production value—it looked like you had just gotten your camera out—to unbelievable production value, and also to fancy places. Did the marketing team notice, “We’ve got a real talent here”?

It was probably more that they were trying to stop me from doing my show. [Laughter] They said they would be willing to do this other thing if I promised to stop making that POS show. “Please, it’s embarrassing the company.”

But no, those trips were amazing for Cloud Cultures. That was fun, man.

That’s great. You were doing all this wild stuff. I don’t know—how did you even set this up?

We were having a conversation about a data center expansion, I think in London, actually, and we were facing this first wave of public sentiment saying, “We don’t want these Western companies building data centers right in our backyard without a lot of value to us.”

The realization was that the services running in those data centers were from the guy down the street and the big business down the road. They were all local businesses. So this was the concept: How do we translate that local culture into an interesting storyline? Basically, the culture feeds the technology needs, which then feeds the demand.

4. Core Weave's Technological Innovations

Some of the interesting things we did with the series—which I think were very foreign for a series at Microsoft at that time—were that we refused to say “Microsoft” or mention Microsoft products. It was all about focusing on those end customers and what they were doing. That made it a lot of fun because it was just talking with people about their stuff, which I guess is kind of what you do on this show.

The good news is you don’t need to budget for it. It was basically Corey walking around with the phone.

That’s 100%. There was zero budget. I think it was my cell phone that I was recording it on. The ROI was through the roof because—

And no studio. You were in the cafeteria. Did anyone complain? Like, “Why are you filming while we’re trying to eat?”

I was bringing guests in, and we were just sitting in the cafeteria. [Laughter] Yeah, no, man. Why would we not?

Pretty cool. Pretty cool.

We really upgraded when we brought microphones in. That was the big thing. It was just like, “Wow, we have microphones now.” [Laughter] That was versus the production value of 16 cameras everywhere I went, just tracking me. So that was fun.

All right, so you were at Microsoft, working on Azure most of the time for about 20 years, and then you came to CoreWeave. One question I wanted to ask you—and this is also of personal interest—is what did you notice? It must have been a jarring experience to go from 20 years at one of the biggest companies in the world to a startup.

I always worry about executives when I hire them and they’ve spent the last 5 years at a big company. How are they going to feel? Are they going to feel like it’s total chaos?

When you’ve spent 20 years at any company, you get to experience a lot of different parts of the company, a lot of different groups of people, and so on. In the early days at Microsoft, I had the opportunity to work on the very early days of infrastructure as a service. We wrote the first spec and worked through the beginnings of containers and the Kubernetes partnership. I was also part of the first Red Hat partnership with Microsoft.

Those early days actually remind me a lot of CoreWeave’s early days. It’s basically, to your point, a little bit of beauty in the haphazard. There’s a sort of beauty to the haphazard. If you take a far enough step back, it makes perfect sense, but if you’re really close, it looks like insanity.

That can exist inside big companies as well as in startups. Obviously, I think it would be present in pretty much all startups. The fun part of the job is finding that beauty and that harmony in everything the company is doing. The company is doing amazing, groundbreaking work in almost every facet in which it’s involved, so it’s an exciting and thrilling place to be. But it is fast-paced, that’s for sure.

So here’s really the question I want to ask you. I have one core question that I’m really curious about: What do you think gives CoreWeave the right to exist?

There are many years of these clouds taking off, and it’s sort of chaos. We’re back in IaaS, and now there are really kind of 3 clouds. Then you see these new neoclouds, and it’s like, what’s going on? Why are there new options here?

I think there are a few things at play. The first thing is that this AI revolution—this pivot toward massive amounts of new innovation and new ideation surrounding AI—has created this new tier of business-critical workload.

I compare it to the analytics wave of maybe 10 years ago, right? There was this realization with analytics that it could drive and fundamentally change your business. We’re seeing that again now with AI, and the reason why that’s important is because it sets the framework for what I think customers need in best-in-class solutions.

When something is business-critical and has a relatively high cost associated with it, that’s when customers are going to be willing to look for the very best there is and be willing to separate from existing contracts and existing environments that they’re comfortable with to be able to get the best-in-class. We saw it with analytics. I would argue that companies like Snowflake and Databricks, as examples, ended up existing outside of this jack-of-all-trades-type public cloud environment because they enabled a best-in-class offering versus just a best-in-suite-type solution.

I think now the AI revolution is seeing that same demand and that same opportunity for a company like CoreWeave to deliver best-in-class services and capabilities for this business-critical and very high-cost requirement. I’m excited to get to be a part of that momentum.

Although I guess one difference, maybe—I mean, it’s an interesting analogy with Snowflake and Databricks.

You could just say you agree with me, Lukas. That’s fine.

Oh, yeah. It is very exciting to be a part of. (laughter)

No, you know, forget it. Go on with your question.

All right. No, no. I think it’s an interesting—

Gotcha journalism edition.

Yeah, hard-hitting, hard-hitting. So Snowflake and Databricks run on top of the clouds, and they actually pay money to the clouds. I guess one difference here that’s maybe super surprising is that Microsoft and Google are actually customers of CoreWeave, right? How does that happen?

Yeah, it’s interesting. Although I will say Microsoft and Google—I suspect many of them use components from Snowflake and Databricks as well. But, yeah, they’re both customers and, in some ways, partners in some situations and circumstances.

I think part of that is also that there are elements of what makes an AI cloud special, or a neocloud special, in decisions, designs, and implementations that make them not fungible. One of my favorite examples from the software side of what makes CoreWeave super unique is our object storage—our implementation of object storage—which, again, for many people is a snooze fest. You’re talking about object storage.

That, coupled with a caching solution that we call Lotta Cache, is important because you need a lot of storage to deliver AI workloads. It’s focused on basically bringing as much data and throughput into the GPU as you can, right? That’s a very unique requirement that AI workloads—and particularly training workloads, but certainly inferencing as well—have to be able to feed the GPU so that the GPU is not idle, because the GPU is the most expensive asset across all of the componentry.

Would I say, “Gosh, you should use that to run your e-commerce website”? No, probably not. I don’t think our Lota Cache with our object storage would be the right thing to run an e-commerce website. But we don’t need to design for an e-commerce website, right? We can design for what we’re built for, which is AI.

That allows us to make these assumptions that are simplifying for us and daunting for the public clouds, because how do they make those types of assumptions based on workload? It goes all the way to the physical side, where you think of something like liquid cooling. Liquid cooling is a hard thing for fungible data centers to take advantage of. You’ve got to plan ahead, deliver all the componentry, and build the right piping, et cetera.

For us, we can make those assumptions because we know that we’re going to need that amount of liquid-cooled capacity. I think the public clouds have a harder time setting that space aside. Again, making that simplifying assumption that we’re delivering AI exclusively allows us to build better services, solutions, and experiences, such that even the public clouds will come in and talk to us about enabling us for some of their workloads or customers.

5. Customer Engagement and Future Prospects

So, okay, liquid cooling—obviously badass. I think we’re about the same age, clearly based on the fact that I understand your obscure musical references. I always wanted a liquid-cooled desktop in my youth, but—

What, like green liquid? You see it going through it, right? (laughter)

To actually cool things. You just need to see the liquid going through it. Anyway, go.

I just want to trade off reliability for something slightly faster. Absolutely, I want that.

That’s right. That trade-off always wins.

Yeah. (laughter) So, if I’m a customer of a liquid-cooled data center, if I’m actually using it, how does that manifest for me? I can actually buy these GPUs in lots of different data centers, so why would I care if it’s liquid-cooled?

Well, some of the GPUs require liquid cooling because of their nature. Some of the latest, greatest, largest ones actually require liquid cooling, so you can’t get them unless you have a liquid-cooling solution, right? That’s at least one reason.

For others, I think, transparently, the verdict is still out on the full extent of the value. It definitely improves efficiency, right? On our side, it improves our ability to serve those GPUs with lower HVAC costs and lower air-conditioning costs because we’re able to use liquid cooling to enable it. The hope is that would translate into cost savings over time.

I do think the stronger case is for GPUs that are so powerful and so strong that they’re actually going to require it. We’re then able to deliver those much more effectively and efficiently, and at larger scale, than others out there.

That makes sense. What about storage? I’ve used object stores my whole career. It’s a super-simple API. I haven’t really thought deeply about the different ways it might work, but are you saying the CoreWeave object store is faster at serving files to a computer? Particularly if I’m an e-commerce company, I also want fast downloads for my customers. It’s not totally different—what’s going on?

I think a key component of it is around the caching tier and how the caching tier is designed across the GPU framework. It takes assumptions around a multi-GPU deployment and delivers and serves the cache from the GPU or from surrounding GPUs, right? It enables you to optimize a scaled GPU deployment and leverage the cache effectively that way.

The e-commerce example is probably the reason why I like to use it. With any sort of cache and cache architecture, you’ve got to make assumptions around your read-to-write percentages, right? Sometimes, if your cache makes the wrong assumptions—if you expect to have this many reads and this many writes—you can actually hinder performance with a cache.

If you end up having more writes than reads, then, outside of a checkpointing process, which probably requires some back-end storage writes, these types of training workloads don’t have as much write-out. E-commerce sites may be exposing content, but I’m not an expert on designing an e-commerce site. Certainly, the management of the basket and the management of actually making an order—these sorts of things—take consistent writes to be executed.

I think this type of cache is not as optimized for that type of frequent need for consistency and high consistency, if that makes sense.

Totally. Are there other things that you expect to change when you look forward at the workloads that are coming? People want to run agents all over the place, and inference is constantly changing. Are there other places where you expect to have purpose-built stuff?

Yeah, I make different decisions.

One of the areas that I think is super interesting with inferencing, and that is also potentially questionable, is whether it’s a point-in-time thing or sort of a forever thing. The amount of time that an overall inferencing call takes—a huge percentage of it is inside the GPU, right? To do the actual inferencing call, the response, et cetera—

Whether it be an agent or a chatbot, the call is spent inside the GPU, and it dramatically reduces the percentage of overall time to serve that sits on the network. This creates some interesting opportunities around network flexibility. One of the things that I like to think about a lot as we think about designing that—and that I also think puts us in a different position from others—is that flexibility in network positioning and network deployment gives a lot of opportunity for changes in the way we think about workload requirements.

Whether that be availability: suddenly going from 5 different data centers to make your calls and basically being able to use whichever one may be available or not, you can dramatically improve the availability of a given serving app. I would argue the ability to react to bursts of capacity need suddenly becomes a lot easier. For anyone who's deployed on a public cloud, especially with GPUs, and you suddenly have a burst of capacity, you're always sweating that you're going to go ask for capacity and it's not going to be there in that location, in that region, in that zone.

Suddenly, with these types of workloads, I think you have a lot more flexibility around your original deployment. What I like to think about for us is, how do we make all of that complexity go away? How do we say, "You may want to run a given model off the shelf, or you may want to run a deeply customized model with a bunch of custom code that you're going to write to set it up. Regardless, you shouldn't have to care about how you're going to get your capacity"?

You should be able to say, in loose terms, "I kind of want it here. I kind of want it there," but you send that to the platform, and the platform will take care of it for you. I feel like that's a pretty novel approach, because these workloads are different from an e-commerce site, where the network time and your time to the actual VM are meaningful because so much of the time is spent in the GPU. Suddenly, that becomes less meaningful.

Does this change over time? I don't know. This is one of the fun things about working in this business: you have to be fast, and you have to listen, sometimes more than talk, which I'm not doing well on this podcast, to be clear.

Well, that's your role. I'm listening, and you're talking.

But here's—let me rewind just for a minute, because you started out being like, "Hey, you do these things. You talk about these really boring things, and you make them interesting." And now, on this podcast so far, I've talked about object storage. I've talked about liquid cooling. I've talked about global capacity availability. These are super boring topics. Is there anything?

But you're making them good. Really? I feel like you're selling these topics. I am actually more interested in these topics than the Tuesdays with Corey topics.

But I don't know. I'm enjoying it. There's no reason to be mean to your host—but to your guest, excuse me—but that's fine.

Okay, this is going to sound like a real softball, but it's a really genuine question. Hard so far.

We're both coming from that side. One thing that's actually pretty remarkable about CoreWeave is how much real customer love there is for what I think, before I joined, I really thought was a commodity product. Honestly, I think most people who aren't deep in this and trying to do big data center work feel like this is a pretty commodity product, but it does seem like people are getting wildly different goodput and TFLOPs out of the same machines because of technical details in how they're set up.

Or maybe—I suspect that it's actually the way that CoreWeave is engaging with customers. It feels friendlier than the other clouds. Having been on the other side of it for a long time, I do think there's a lot of room for customer service to improve, especially technical customer service. What's your take? Do you notice a difference in the way Azure versus CoreWeave engages with customers from the inside? Does that feel right?

No, absolutely. Look, I think there are 2 things to say. The product is real. In the examples that I give, there are numerous additional examples. One that I didn't actually spend time on, but is a key one for customer love, is our observability: the awareness and depth that we have into the infrastructure—what's happening and why it's happening—and then the automated integration of that into Mission Control, into the operations that the end customer is running.

This simplifies the ability to run these jobs. This is part of the magic of something like SUNK, or Slurm on CKS. With the observability integration, customers are able to manage their workloads and their training jobs in a much simpler way. To me, the product line is a big part of the love: from, again, object storage to CKS orchestration, to Slurm on top of CKS, to observability.

I also believe, to your point, that customer engagement is a big part of it. We've got this really amazing customer success and customer enablement team that's so focused on helping customers figure it out. We start with the assumption that this stuff is complex. It's hard to get things working, especially with the fast pace of the infrastructure.

For many of our customers, getting down to the infrastructure and using it effectively is critical to success. Enabling them and working closely with them—even the deep engagement of our CTO with the day-to-day activities of our customers—is stunning. He is out there on the channel. I think he has roughly twice as many Slack messages as anyone else in the company because he's just out there working with customers, helping them solve their problems, and making sure that they're using the platform in the best way possible.

That’s pretty unique. That’s a pretty unique outcome. I would argue that the big clouds probably have that, but with their top 9 customers or whatever it may be. I think, just from an overall percentage of our customer base, we’re able to deliver upon that with a much greater percentage.

I think that delivers a lot of love for the platform. That alone wouldn’t work, but coupled with all the product capabilities that we talked about, and that customer support to get people to use those products in the most effective and efficient ways, that’s really where that connecting glue becomes a lot of love.

6. Comparing Cloud Approaches

Was it the same with the other new things that you worked on at Azure, where you were making a small number of customers successful at big scale and then disseminating that to lots of people? I think what I’m used to from my Silicon Valley SaaS background is that typically the journey is more like, “We make a lot of little customers happy, and we gradually move upmarket.” I think the CoreWeave approach—and most of the neocloud approach—has actually been totally the opposite of that, and sort of different from the model I’m used to.

That’s right. Again, I would say the very early days of infrastructure as a service were closer to that. It was a small number of customers, and you made them happy. But today, if you look at all the clouds, it’s very much the way you were talking about it: you’ve got a scale of customers, and you’re building things to meet the needs across that scale of customers.

I would argue that CoreWeave ideates new products by sitting with a customer and hearing them say, “Well, I’m stuck.” In some ways, a lot of SUNK came out of this, right? Some big customers were stuck trying to feed their GPUs with the fullest amount of data possible, and the design and implementation came out of that. It was delivered to that customer first as, “Let’s work together on it,” and then became a product—something that we can now enable for everybody. It’s growing like crazy, so I think that approach is very different and pretty unique.

A lot of that is also the shape of the AI market, right? The shape of the AI market is that you’ve got the big elephants and then a whole lot of smaller animals, like little birds.

Do you think the natural life cycle here is that this stuff all becomes a commodity, and then you innovate on another dimension? Or do you think there’s something fundamental about these AI workloads, and that they kind of stay nonfungible?

I believe that while there’s value being added to a service—in operations, quality, performance, and experience—something will never be truly a commodity. I’ve made this argument about the public cloud for years. For a long time, public cloud infrastructure as a service was going to be this commodity thing that anyone could move to anywhere.

I made the argument, “Look, I don’t care if the APIs are consistent and commoditized. I don’t care if we all use the same APIs.” The service we offer, the quality of the experience, the quality of the uptime, and the performance that you get—those continue to differentiate forever and continue to make something not a commodity.

I would argue a very similar position here about the AI world. So many of the things I talked about were about enabling TCO, enabling better throughput, enabling a better ability to serve workloads, enabling better usage of your GPUs, and enabling betterput. These things in the set of services and capabilities that we talked about are all delivering upon that end goal for a customer, and that end goal is quite valuable.

So, no, I think there will always be value. Will the bar continue to go up? Yes, which is an amazing part of how these types of markets work. The bar will go up. The level of quality, performance, capability, and experience that we deliver today will not win workloads in 2 years. We’d better be moving the bar up. But there will always be differentiation, especially when the cost is so high and the business value is so high.

So, you would actually argue that Azure, GCP, and AWS are not equivalent?

I definitely would argue that they’re not equivalent.

I feel like I’m always talking to them now. Maybe I can say this: I kind of pretend, like, “Oh yeah, you’re special.” But as a startup guy, you’re kind of like, “Okay, who’s going to give me the most credits? I’ll use that one first.” Then, oh man, now I’m stuck on the thing that I tried first.

I think that’s the startup experience of these things. I have this sense that Azure is a little more enterprise-oriented. I have the experience that GCP is a little harder to get the attention of, including its support team. Amazon is kind of the default. I don’t know. What would you say is the advantage internally at Azure?

7. Balancing Executive Roles and Hands-On Projects

Look, I obviously loved my time at Microsoft, so this is a very biased perspective. But I thought that Azure was, to your point, the strongest at delivering upon the end-to-end enterprise requirements.

One of the things that I felt made Microsoft, and then Azure as a component of it, was the concept of the Microsoft cloud. The Microsoft cloud was and is super unique. The ability to come in and say, “Look, I’m an Office user. I use components of Dynamics for my business practices. So, if that’s the case, of course I’m going to use Azure, which is integrated deeply into all of those components, and bring it all together.”

You layer on the security layer, and now you’ve got a consistent security story that spans all the work happening at your company. You’ve got a consistent billing and quota story that spans all the work happening at your company. To me, that enterprise story—even a step up, with the Microsoft cloud story—always resonated really well. Customers in those groups really appreciated the fact that things hung together and felt like they were all unified.

There are also a bunch of one-off things. For the longest time, Azure had the strongest SAP offering, and so that was a big value add for a bunch of customers who wanted to deploy SAP. Cosmos DB is one of the best global databases in the world. There are a bunch of specific services and capabilities that I think are deeply differentiated.

Are you trying to create a CoreWeave story like that, where you have differentiated services and capabilities?

Yeah, basically, the story is that we have differentiated services and capabilities. CoreWeave’s already got that right. I think CoreWeave has got that on the training front. When you look at the platform we’ve got today, with all the things we’ve already talked about—the observability, SUNK, and the chaos layer—CoreWeave is, I believe, the best place to run training AI workloads in the market, period.

That class of workload, with all those capabilities and features, sets it apart from the other clouds, whether they’re neoclouds or public clouds. It’s why people use us at scale: because of that functionality and capability. I look forward to a bunch of other places where we’re on the path to being differentiated in a similar way, but the biggest one I’d probably say is scaling training workloads.

One thing that’s also different from anything I’ve seen in my career is that we look totally supply-constrained. All my friends ask me, “Hey, you’re working at CoreWeave now, Lukas. Can you get me some GPUs?” I’m like, “Probably not.”

Also, if you try to buy from the sales team, you should realize that they’re kind of at capacity. It’s a little more like you’re going to be selling yourself to try to get them to sell you GPUs. Doesn’t that create a really different dynamic here?

Yeah, it does. I think it’s interesting to be differentiated even when many of the customers are using multiple training providers, maybe just because they’re trying to get capacity anywhere.

Yeah.

Yeah. Well, this is where I think one of the challenges of the market is. First of all, the demand is insatiable. It’s stunning, the demand that we have in the market, which is a great place to work, for sure. I believe we have the best services, and I believe we have some of the best customer experiences, but the demand is really, really high.

That helps, certainly. From a supply perspective, it’s also a little bit tricky to give broad strokes to supply because it’s like, “Okay, great—supply of what GPU flavor?” Are you talking about GB200s, GB300s, B200s, H100s, or H200s? What supply are you looking for, and what quantity? Are you looking for 10, 100, 1,000, or 10,000? There aren’t a lot of places that can run 10,000 GB200s in the world.

So, yes, there are definitely some supply constraints in this range, and some of the requirements and demands are hard to deliver. But I would say there is a growing amount of both research and supply to deliver different approaches to those same workloads. Can you partition the workloads into more chunks, to your point, and go to more clouds? Can you do this in one cloud and that in another? Can you use a different generation of SKU for some of your workloads?

That’s crucial to continue serving those workloads, and we do have capacity when it comes to various types of workloads. We actually have a growing amount of capacity as we look at the increased demand for things like on-demand or spot-type pricing, where I think more and more workloads will have those expectations. Did I answer your question, Lukas?

I think so. That was a good question. I don’t know. I just keep dodging everything you’re asking, so I’ll just say what I want to: “The essential AI cloud, Corey. That’s the answer.” You’ve actually had PR training. I appreciate it.

I don’t use it, though. By the way, I look really dark here because it’s nighttime. You scheduled this when the sun’s down over here.

Yeah, trying to tire you out and then hit you with the hard-hitting questions. Thanks for being here.

Yeah, exactly. End of the day, man. [Laughter]

Okay, I’ll just keep asking the questions that I personally want to ask here. I think one of the things that I really admire about your process—and this is why I really relate to you—is that I feel like you’re really trying a lot of tools. You have coreyweave.com, where you put your projects. Maybe we should censor that one out. Maybe too many people will go to coreyweave.com and learn about what you’re up to.

They have to guess the subdomain. That’s the beauty of it. coreyweave.com doesn’t actually resolve anything, but the subdomains do. Yeah, go ahead.

It’s funny—I always feel guilty for not spending more time building projects and trying to stay on top of AI. I feel like you’re clearly doing this big executive function. How are you actually able to prioritize, and how do you switch your mind into being able to think clearly enough to actually build something?

As I get older, I find that switching from a bunch of meetings to trying to actually write some code—and especially use a new library that might or might not be buggy and that I don’t understand—is incredibly challenging. Are you blocking out weeks to do this, or what’s happening?

Yeah, no, it’s a good point. It’s so funny because I actually just finished a project last night. I worked on it until midnight, and I was so mad because I was like, “Oh, man, I’m going to be so tired tomorrow, and I have to be on for this interview. I have to be on.” [Laughter]

The other thing that really frustrates me is that my first 5 years at Microsoft, I was actually an engineer. I coded in Windows. I joke about how I was a professional developer for 5 years, but it was all assembly and C. I never professionally coded in any managed language. When I was in college—dating myself a little bit—I don’t think there was any course that was in a managed language.

No, I’ve worked with guys like you. I have a feeling I’d find some flaws in your coding style.

Oh, 100%. Man, this is why my production coding is not—

I’m not saying I’m the master. I’m just thinking about colleagues I’ve worked with whose background is completely in C and assembly, and now they’re writing Python, and it’s just particularly crazy.

Right. That’s right. The concept of functions is foreign to me.

Yeah. Why bother?

You know what? If you’re smart enough, Lukas, you can just write it all with jumps. I don’t understand.

Absolutely.

Anyhow, look, I think one of the things that’s both really good and bad is that when I get into a project, it does consume me. I don’t allow myself to do projects that often because I know that once I start one, it really consumes me. I get super frustrated because it inevitably isn’t going to work a bunch of the time.

Because it’s not production-level coding, part of it is also setting boundaries for what’s good enough. A good example is that I’ve been working on—and I just deployed it on a bunch of CoreWeave infrastructure—using Anthropic, actually. My brother works at Anthropic, so I thought I’d give it a go. I used Anthropic to basically build a new CoreWeave portal.

I wired it up to connect to our backend APIs. I tried to wire it up to Grafana, but there are points where you get to good enough. I couldn’t figure out how to get the Grafana token to connect to the API. I don’t know if that’s a CoreWeave thing or something else, so I have to dig into it. In the meantime, I just put mock data in. I thought, “Forget it. I just want to see if I can get something to work,” rather than spending forever figuring out how to get to our Grafana API.

That, to me, is a big part of it: setting boundaries on what I’m trying to do and making sure I don’t get too caught up in the things I’m not trying to do as part of it. It’s also being willing to take passes.

But then, yeah, it’s about carving out the time. I’ll admit, I think one of the meetings that you and I were in, I wasn’t really paying attention. I was coding.

Hey, wait, that was a meeting. I was thinking last night we were—

I think, actually—actually, yeah, 2 nights ago. There were moments 2 nights ago.

There were moments.

But look, I think it keeps me sharp. This thing that I built with the portal was a stupid thing to go build. I don’t know what the outcome is going to be. I’m going to write a spec around the outcome because I think there are some interesting things we could learn from it.

In some ways, I just learned a lot about the challenges of writing an application like that using an API like Anthropic. I wanted to be able to put stats and buttons in it, but to do that, I needed— The API calling was working great, but doing 2 things wasn’t working. You could only do 1 thing because you could only take 1 pass. Suddenly, I was designing around the question, “How do I make it 2-pass? Do I think about forking it off and doing 2 calls?”

It got me into the mindset of, “Okay, I feel like it was a huge waste of time, but now I understand how hard these types of applications are going to be and how to think about where we can deliver services and capabilities that make some of those things easier.” Some of them we can’t help with, but some of them we can make easier and simpler to build.

For me, the best way to be a product leader is to play with things out there in the mindset of what your customer is going to be doing, then talk with your customers about it and evaluate it. You have to make sure that what I was doing wasn’t totally stupid. The worst thing is that I do this thing and the customers say, “Well, I would never do that.” Then I think, “Okay, so it definitely was a waste of time.”

But for the most part, I think it keeps me sharp about where the product actually is. Using our own product is a key part of that. I’ve used your product a lot, Lukas.

No, I know. I appreciate it.

Oh, by the way, I used Weave in this product.

I had tracked all of the commands that I was doing through this portal thing through Weave. Yeah, it’s nice.

Do you have any advice for me from a longtime executive at a cloud company, as a founder who’s trying to be an executive? What should I be doing differently?

Look, it’s hard, man, because, for me at least, the unfortunate reality was that a lot of it was circumstance. I got tapped to go write the spec on infrastructure as a service in 2009, when there was 1 player in town. It was like, “Cool,” you know? So I think I got some things right.

The things that I got right are the things that I think you do naturally: iterate frequently, ship something, and learn. Realize that the first thing you’re going to do is make a bunch of mistakes, but listen to your customers.

One of the strongest arguments I can make—and I think you do a really good job—but it always takes refreshing is: Who are your customers, and are they satisfied and happy with what you’re building and what you’re planning? I wouldn’t say let your customers design what you’re building, right? There’s a fine line between that.

So what do you ask? Actually, I feel like this is something I talk about a lot.

I always come with, “Hey, here’s what we’re building. Now tell me what you think,” or, “Here’s the direction we’re going.” I seed it, because then you get actual, actionable feedback.

If it’s just, “Tell me what’s missing,” you’re going to get, “Oh my gosh, this morning I was really annoyed by this,” and that thing may never have happened before and may never happen again, but it’s top of mind that morning. This open-ended nonsense of “Tell me what you want” always fails with customers.

A customer’s job is not typically to be a product owner. Their job is to be a product user, so they’re not going to give it to you in terms that are actually actionable. But if you go and pitch, “I’ve got this idea. Let me show you what I’ve got,” and go as far as you can with screenshots or flows or whatever, then you build that confidence with the customer: “I’m sharing this with you ahead of time because I want you to tell me.”

I love the feedback from those sessions because you get really actionable responses, like, “Oh, this is cool, but I like this instead of that.” Or sometimes you get that heart-wrenching, “I don’t think I’d ever use this,” and you’re just like, “Oh, really?” But that’s better than building something and then saying, “What do you want?” They didn’t say they wanted that, but you think they probably do anyway. That’s a useless approach.

I love that approach, and I love getting the chance to do it. I haven’t done enough, frankly. I did a bunch of customer conversations at the beginning of my time at CoreWeave, and now I’m learning everything that’s in the pipeline. To me, my next step is, “Okay, great. This is what we’re launching in February. I want to go sit down with customers in December and test it with them and say, ‘Would you use this? How do you feel about that?’” To me, that’s the best way to get feedback.

Is this just something that you learned intuitively? I totally agree with it, but have there been books you’ve read or something you’ve consumed that has helped you along the way?

I don’t like to read. Sorry. No, actually, look, I think there were some books that I read. I’ll be super transparent: I feel like a lot of what I learned about how to think about product and how to approach a product in those early stages was from my boss at the time, Scott Guthrie, who’s still at Microsoft.

He had done a bunch of work with the original .NET and was a leader in some of the original design there. Then he took over Azure and led a lot of the Azure strategy along with Jason Zander. I feel like the approach to product that Scott had was probably based on a bunch of books he had read, so maybe you should have him on the podcast.

It was really this focus on going with customers, learning from customers, building, and iterating. So there definitely were a bunch of books. The Toyota Way, I think, was something like—I remember that working as a way to think about it.

Don’t start throwing out random books. Nobody wants that. [Laughter]

The Toyota Way.

He caught me. That was one of the books. I didn’t read it, though. It’s still—

What is it, like 2005? Jeez.

But yeah, that’s right. Dude, this was in 2009. This is the—

Okay, you know what? You know what?

Well, I appreciate it. I feel like your process is actually about building, which probably helps you authentically connect with the customers. The first thing I would say about customer interviews—which I’m not saying I’m the world’s expert on, but after years of doing it—is that you can’t ask open-ended questions. You have to show something.

I always try to make it look as built as possible. Showing someone a website, like what you do with CoreWeave, or putting a URL where someone can see a mockup that feels more like a product—I feel like you get more honest feedback. The more you can mock it up so it looks like, “Hey, I built this thing. What do you think about it?” the more feedback you get versus text ideation.

I will send you the link to my Anthropic-based CoreWeave portal experience. Actually, I’m not going to broadcast it here because it’s got my real data on it. It’s got my real cluster working. I had to make it work, man. Had to make it work.

This is some good IP. I think it constantly automatically configures based on what you might want. Sounds pretty awesome, if—

I mean, that’s the idea, man. But it works terribly. I’m sure real smart people could do a much better job. But it works, so it’s fine.

All right. Well, it seems like a good place to stop. A lot of fun. Thanks so much for listening to this episode of Gradient Descent. Please stay tuned for future episodes.