(Preview) AWS History and Trainium’s AI Future, OpenAI Makes a Deal With Microsoft, Meta and the Future of Wearable Devices

Hello, and welcome to a free preview of Sharp Tech.

Hello, and welcome back to another episode of Sharp Tech. I’m Andrew Sharp, and on the other line, Ben Thompson. Ben, how are you doing?

Perturbed, Andrew. I have a new computer.

Okay.

It’s complicated, but all the recording is happening on my main computer, but I need a computer in front of me for the rundown, which may or may not be accurate. You sometimes just suddenly pull out things that are not there, and I’m like, “What the heck is this?”

I’ve got to keep you on your toes, Ben.

And you’re like—

You know? Make you sweat a little bit.

Yeah. You’re like, “Uh-oh, I made my draft in Gmail, which I’m reading out of.” I’m using Microsoft Word for your sake. It’s unbelievable.

Mm-hmm.

So what happened was, I have had—I think I’ve talked about this—I have 2 computers. I have my MacBook Pro that mostly stays on my desk. If I’m traveling and working, I will bring it with me, but it is basically a desktop computer.

Mm-hmm.

And it’s great. The MacBook Air is amazing for carrying around, having with you. I had an M2 MacBook Air for a long time. Love it. Amazing computer.

Unfortunately, because I carry it around all the time, it has been to baseball games and practices. I’m working in the car, it’s gotten dropped. One time it was left in the backseat of the car. There were a bunch of boys in there, and it got stomped on.

Yeah.

It’s got—you know. But that’s fine. I mean, it’s not fine, but it’s okay. There’s nothing important on there. The problem is, it started kernel panicking. So last week, I think we talked about this. My computer crashed.

Mm-hmm.

The crashes are accelerating in frequency.

Oh, interesting. Okay.

Well, what happens is, especially if you drop it, stuff inside gets loose, and it starts shorting out. That’s an unrecoverable error. The whole thing is just going to go out.

Yesterday it got really bad. I’m like, “I need a computer for the podcast recording tomorrow.” So I went to the Apple Store and just bought a new MacBook Air.

Okay.

The problem is, the MacBook Air runs Tahoe, the new macOS.

Yeah.

And everyone’s been complaining about Tahoe, and I validate most of those complaints.

You’ve been resisting.

Yeah. But there’s a complaint no one’s mentioned, which is that I kept feeling like it was weirdly dim.

I checked the specs. Did they change the screen from the M2 MacBook? Of course not; they didn’t make a dimmer screen. My Tahoe complaint—I just want to add it to the list of everyone else’s—is that the interface is so white. It’s white on white on white on white everywhere that it makes the whole thing seem dim.

It’s a really bizarre effect. Yes, I could do dark mode. Unfortunately, I’m 45 years old. We’re going to circle back to me being 45 years old and my eyes at the end.

Great.

No dark mode for me. So anyhow, I just want to register this addition to the litany of Tahoe complaints: too much white. Contrast, please.

Too much white, indeed.

That’s right.

Well, I have a reveal live on the podcast. I, too, got a new computer this week because my computer—I have a MacBook Pro that I keep on my desk at all times.

Yep.

That’s my podcasting machine. And I use a MacBook Air all over my house. It’s what I write on. It’s what I prep for shows on. I’m using it on every floor of my house.

When you have 2 kids running around, they tend to pick it up and drop it and do all kinds of unhelpful things with your MacBook Air. So I had to hide my computer by my bed one night, and then I wound up stepping on that computer as I got out of bed. It just completely shorted out unexpectedly. Again, some screws probably broke loose, and then a day or 2 later, it just stopped working entirely.

So I got a MacBook Air, and I will say I feel the software pain with my new MacBook Air. There are so many things that I’ve had to resort to ChatGPT for to try to fix on my MacBook, which should not be how these machines work.

Right. They should be getting nicer to use over time, and they’re going in the opposite direction.

More complicated, more frustrating. It is what it is. Still a great machine, and hopefully it won’t be a broken machine over the next couple of years, because I’m excited about the new chips, and I do love the Air.

Did you consider getting a MacBook Neo? I’m sure someone is going to ask.

No, I did not consider getting a MacBook Neo, because I’m working on this every single day of my life and will be for years to come. So I feel like it’s worth paying the premium.

I didn’t go for 24 GB of unified memory. I was content with 16 GB of unified memory, although I felt kind of lame. Did you go 24 there?

No. I got just the base model: 512 mem—the absolute base model. Cheapest chip. Least memory.

Okay.

Again, the good thing—and this is actually very important—is that there was nothing important on my old computer.

Yeah.

That was good, because it was getting so bad by the end. It really accelerated in the last day. It took me 3 tries to erase it because it kept kernel panicking before I could erase everything.

So if I lost some data on there, it was rough. But no, this is like a netbook. Everything’s online. It’s totally disposable.

I like the Air. I like the little stuff, like the light-up keys. One of my biggest use cases is that my son is at baseball practice, which is a long way away. I sit in the car and work. Having light-up keys is actually useful.

It’s a delight. Yeah.

And the ambient light adjustment—I love that feature. Neo doesn’t have it. But I actually had to turn that off because the whiteness of the interface meant every change in brightness of the screen felt like it was putting a black pane of glass across the whole thing.

It was just so dramatic, the shift.

Mm.

Maybe I could’ve gotten Neo. Epiphany.

The whiteness, man. Fix the whiteness. Bring Forrestal back, and he can fix the whiteness for you.

Yeah.

A couple of new computer buddies here on the show today.

Yeah. Well—

Very exciting stuff. Yes. Well, we are not going to be talking about new laptops. We are going to be talking about Amazon and OpenAI on today’s episode.

But before we get to Amazon and OpenAI, Amazon did release its earnings later in the week along with the rest of Big Tech. We’re going to table some of that.

Oh, terrible.

So you have Amazon, Google, Meta, and Microsoft—

All on the same day, along with several other companies that I mentioned. This is the first time I’ve had one of these days where everyone’s there.

I thought, “Oh, this interview running on Tuesday is going to be great, because then I can at least hit one of the earnings Wednesday night for Thursday.” And I forgot that, being in the US, the transcripts of these don’t come out until later.

So I wrote about Amazon in my update on Thursday, in part because I’ve been talking about Amazon for the last couple of weeks, so that made sense. Their transcript dropped at 9:01 PM, so it was the first one to drop.

Right.

I think Google came out at around 11:00. And then, I don’t know—I haven’t even read the Meta or Microsoft ones. I glanced at the Google one, but yeah, we’re going to be focusing mostly on Amazon, both for topical reasons and also because I can’t stay up until 4:00 in the morning waiting for transcripts.

Indeed.

Well, yes, I look forward to immersing ourselves in the Meta earnings, the Google earnings—

Yeah, in Taiwan—

Microsoft—

They were always there. It was great. Another time-zone advantage of being in Asia, but what are you going to do?

And by the way, is it normal for all of those companies to release earnings on the same day? I remember it being more staggered, but maybe I’m misremembering.

Yeah. No, they’re generally all bunched together. Maybe someone can email us. I don’t actually know how that works.

How and when they announce them, I don’t actually know. Usually, the announcement of the date is 7 to 10 days before, it seems.

Okay.

Obviously, earnings has always been a core thing for Stratechery. It's something I'm always aware of and thinking about as far as scheduling. And you see this: not only were they all on one day, but lately they've always been on Wednesdays.

Mm-hmm.

It's very frustrating for my publishing schedule.

Well, it's not bad, though. From a Stratechery editorial standpoint, it gives you a couple of days to mull over what you're hearing and then—

Yeah, but I don't know—

—come back on Monday.

Every time there's a Meta 10% drop, I'm like, “Oh, this is my sweet spot. I want to get on it right away.”

Well, good news: the sweet spot will be waiting for you on Monday because Meta's taken some hits. For now, though, we will talk about Amazon, and I'll read from The Wall Street Journal.

“Amazon said Wednesday that its edge in cloud computing and aggressive investment in new data centers is translating into a surge in its artificial intelligence business. Chief Executive Andy Jassy said that revenue from the company's Amazon Web Services grew 28%, the fastest pace since 2022, in part because many customers building new AI agents want them stored in the same spot where they maintain their other cloud services and data. Revenue for the period rose 17% to $181.5 billion, while net profit increased a sharp 77% to $30.3 billion, which Amazon attributed to pretax income from its investment in Anthropic. Both figures beat analyst estimates. Shares were up more than 4% in after-hours trading.”

So, Ben, big picture, what is the story with these earnings? Has Trainium risen from the dead? Does the AWS AI story look better today than it did 12 months ago?

Yeah. Interestingly, Amazon actually ended up down so far today, so who knows what's going on there. But I think Amazon is certainly one of the many participants in the ongoing AI soap opera: who's up—

Mm-hmm.

—who's down. Everyone has their time.

I think there was a lot of concern, really crystallized in a SemiAnalysis article a couple of years ago, talking about how Amazon is screwed for AI in the long run because they won so hard, as it were, in the data center. I wrote about this a bit at the time, about Amazon and Apple together—the two big winners from the cloud—and whether they can actually adjust for the AI era if it requires different approaches.

Mm-hmm.

The framing in that SemiAnalysis article was, well, Amazon has invested heavily—not just because they were first, but in really maximizing their position in a commodity market. And what I mean by that is, we've talked about this on this podcast, there are two ways to make money.

What everyone in tech thinks about is making something that's differentiated, that has a moat, and selling it for a premium. That is the Apple model. I was at some sort of startup event a couple of weeks ago, and they were just talking about moats. That's what everyone in Silicon Valley thinks about. It's the most intuitive, I think.

Mm-hmm. It’s also what Nvidia’s doing

But in the real world—yeah, exactly—the real world where a lot of products are commodities, they are highly substitutable. If you don't get something there, you can get it somewhere else. How do you make money in that world?

Mm-hmm.

In that world, you make money by having a superior cost structure. You can deliver the commodity at a lower price than everyone else, and the price for your commodity is not based on your cost structure. You can always outcompete everyone by offering a lower price, but assuming there's sufficient demand, the market-clearing price is going to be the marginal cost to produce the commodity for the highest-cost provider.

If you have a perfect balance, assuming demand and supply can scale perfectly, the market-clearing price is going to be the marginal cost to produce the commodity for the highest-cost provider. If making a widget, and widgets are widely available, the price will be whatever it costs Company XYZ $10 to make the widget; then all the widgets in the world are sold for $10.

Now, again, there are lots of variables here. Supply and demand vary. There's elasticity in the price—how many people want to buy. But if you're looking at that segment generally, for a commodity where it's totally substitutable across companies, the market-clearing price is going to be the marginal cost of the most expensive provider.

Yeah.

Right. So if you have 4 providers, it costs provider 1 $10 to make it, provider 2 $9 to make it, provider 3 $8 to make it, and provider 4 $7 to make it, the company that has sustainable profits in the long run is—

The $7 company.

That's right.

Yeah.

They're making $3 of profit on every widget. And that's how you could make a lot of money that way, right?

Mm-hmm.

Because everyone needs the commodity, you're also in a strong competitive position. If the price goes down, Company 1 will go out of business, and then suddenly supply will go down and the price will go back up. The lower your cost structure in the industry, the better you are. And if you're at the lowest end, you have a lot of power and can be very sustainably profitable.

Mm-hmm.

So that was Amazon in the cloud-computing age. It's not just that they were first to build out the cloud; they were first to really invest in a few different things. One was, obviously, their own processors.

They make Graviton processors. The number-one use case for Graviton processors—which you could, as a customer, go and get an instance of—particularly in the early years, was that they sucked.

Yeah.

Amazon didn't just offer infrastructure. You could go buy a processor. They offered platforms, so you can get Redshift, the Amazon database service, right? In this case, you don't actually know what the processor is. You're just getting database as a service from Amazon, and guess what Amazon's probably powering Redshift with? Graviton processors.

Hmm.

Their cost to serve it is diminished because they're using much cheaper processors, and Graviton has gotten better and better over time. Because they can abstract that away, especially with their platform stuff, they can have a sustainable cost advantage there.

They also did a lot with networking, and they built this entire—actually, their first, probably most important chip was this coprocessor. With a server, you have a virtual machine. You have to actually run the actual server. On top of that, you have all these virtual machines that appear as a computer to the client, but actually one chip—one computer—is servicing hundreds or thousands of clients that are all sitting on top of it on their own little virtual machine.

You need to actually run the computer, though, and you need what's called a hypervisor to manage all those virtual machines. Amazon took all that off the main chip and had a side chip that basically handled all the networking and all the management of the system, so that the big Intel chips that power—

Mm-hmm.

That's why you had these Intel chips with tons and tons of cores, because you could—

Mm-hmm.

—because all those—one core could be dedicated to a particular virtual machine. You could keep all that expensive Intel chip capacity to run more hypervisors and more virtual machines because you offloaded the janitorial aspects of the server to—

Okay.

—this other chip, and then it handled all the networking and all that sort of thing. It's called Nitro.

This gave them a sustainable cost advantage. Microsoft is offering an instance that runs on Intel chips. Amazon is offering an instance that runs on Intel chips. But because Amazon can fit 20% more virtual machines on one chip, that means their cost to serve is structurally lower than Microsoft's was because they have this whole coprocessor sort of thing.

This is a great example of Amazon in a nutshell and why they are the anti-Apple, and I mean that in a very positive sense. Apple is all the way at the extreme of—

Yeah.

We're going to highly differentiate our products. We're going to keep our OS exclusive to our hardware. We're going to have a developer ecosystem, so we get network effects. We're going to have brand. We're going to have all those things that make Apple Apple, which gives us structurally sustainable—

And we're going to maintain—

—differentiation. They'll just charge a premium price—

—profit margins for 20 years.

That's right.

Despite all the competition.

Amazon is all the way on the opposite side. They're going to invest a ton of money over years to build structural cost advantages in commodity markets, so they can do things that—

Which they did in retail, too, yeah.

They did in retail.

It's the same playbook in cloud.

That's why they're launching satellites, right?

One question, though, as far as that history is concerned: As Amazon optimized for cost structure and served it more efficiently than some of its competition, did they then charge lower rates and take market share that way? Is that how AWS took over the world, or was it something else?

Well, they were just first in general, number 1. Number 2, they've always had way more features than everyone else because they keep building features.

And so it's an 80/20 thing where everyone complains about AWS and how all this stuff is hard to use, and they're like, "Oh, they should cut everything else but keep this one thing that I need."

But the one thing that I use.

The one thing everyone needs is great with everybody else, right?

Yeah.

And everyone goes out and they're like, "I'm not going to get locked into a cloud. I'm just going to use commodity hardware, a basic compute instance, and a basic storage instance so I can take it from Amazon and go to Azure if I want to, or go to GCP."

Mm-hmm.

And then you're developing and you're like, "Oh, I could spend a few months building this, or I could just use this API that's helpfully there from Amazon that will solve this one problem. We're just doing this one thing. Don't do it too often. We don't want to get locked in."

Yeah.

And then you fast-forward and you're totally locked in. You're using all their services. You're not going anywhere.

This is actually a super important point. If you're not going anywhere, Amazon has pricing power over you. They have features and pricing power over you. And if there's not enough demand or if there's not enough supply in the market, they're going to have a lot of pricing power.

But this is actually an important point. The big shock when AWS was revealed—and I call it the AWS IPO, like, a decade ago—was everyone sort of got the cloud, but assumed it was going to be super low-margin.

Mm-hmm.

And it turned out that, actually, no, the margins were great, especially from an Amazon perspective. I think at the time, I want to say it was something like 17% to 19%. People thought it'd be 1% to 2%. Now it's in the 30s—

Wow.

—or something like that.

Yeah.

Or maybe even higher. So it turned out it was—

And much higher margins than the retail business for Amazon.

Right. It turns out you could get—not... So, Amazon, yes, I'm talking about this very compelling total commodity market. People could switch wherever. The reality is everything's a mix, and there are moats. You do lock people in.

And so, yes, you can also offer stuff super cheaply if need be in a competitive bake-off. They can go to companies. That's why they'll give startups hundreds of thousands of dollars in credits. You're not even paying anything to Amazon unless you're actually a substantial business.

They will make long-term deals with companies: "Okay, sign up for 3 years. We'll give you an 80% discount just because you're locked in with us." And they could do that because they have this capacity to do so.

Right.

So—

That's one of the great advantages. Okay, so AI—I'm going to repeat my question. Has Trainium risen from the dead?

Okay, so it's interesting.

And does the AWS AI story look better?

So, there's a very—we're wandering here, and we're going to make it back. The concern that the SemiAnalysis article raised—and I'll put a link in the show notes to this article because it was really interesting at the time—is that Amazon is so committed to and locked into their proprietary networking in particular that they can get NVIDIA stuff and plug it into their HGX racks or whatever. So you can access an NVIDIA instance.

But actually, the future is these huge—it isn't just a chip, and it isn't just a rack or an HGX, which I think was 8 GPUs. It's entire racks. And it isn't just entire racks; it's entire data centers that are linked together.

This is what Jensen Huang goes on and on about with NVIDIA: It's like the entire data center as a GPU. And that is totally where Amazon's whole strategy doesn't work.

Let's not do that.

You have to do all NVIDIA's networking. The concern raised in that article is that Amazon is so committed to their strategy, particularly in terms of networking, that they're going to fall further and further behind in AI as networking becomes more and more important, and this systems aspect becomes more and more important.

And it's not that the article was wrong. That is all true in terms of training. Training needs this horizontal scaling, this super-low latency between chips and between systems, and Amazon has never been a big player in terms of training.

Mm-hmm.

It has been Microsoft, Oracle, Elon Musk and xAI building their own data center dedicated to doing this. What Amazon got right is that training dominated the amount of compute that was used for a long time, even post-ChatGPT, for several years. It was more like 60% of the chips in the world were being used for training, not for inference.

Mm-hmm. Mm-hmm.

But when and if inference came along, the needs would be different. In an inference world, you're mostly keeping everything within one chip. You want to get everything on one chip, and this is where the model-distillation stuff comes in. You're not even worried about too much of the horizontal networking.

It's all about batch size and getting stuff in. All these people are coming in, and this is where the CPU aspect gets more important because there's more orchestration: This job goes there; this goes there. You're trying to just keep these GPUs filled.

What you're not doing is running these huge horizontal clusters that go across. It's just a different data center setup for serving inference than it is for doing training.

Sure.

But if AI is—

And it makes intuitive sense that, as far as inference is concerned, that would be a more commoditized space where efficiency matters more than performance, where performance wouldn't be the only factor that matters in training.

Right. Well, that's where you're actually selling, right? That's right.

Yeah.

You're actually selling. And in theory, if all this training is going to be worth it, it has to shrink as a proportion of compute because the training manifests in a model that is used for inference.

All right, and that is the end of the free preview. If you'd like to hear more from Ben and I, there are links to subscribe in the show notes, or you can also go to sharptech.fm. Either option will get you access to a personalized feed that has all the shows we do every week, plus lots more great content from Stratechery and the Stratechery Plus bundle. Check it out, and if you've got feedback, please email us at email@sharptech.fm.