Muse Wonder：Zuckerberg 与 Wang 的1000亿美元 AI

At the end of last year, we were like, “All right, we’re quite excited about the progress. We haven’t seen the models yet. We’re going to write this check for, I don’t know, $100 billion, $200 billion.” I’m like, “Alex, are you sure this is going to work?” It’s like—

Cheers.

Cheers.

Yeah.

Cheers.

Cheers.

Cheers.

1. Muse Finds Its Use Cases

So, give me the update on Muse. If you were looking at the third-party reports, you would see that this might be the fastest-growing agent or AI app that has existed yet.

What are your guys’ personal favorite Muse use cases, either yours or other people’s?

I was actually talking about this at dinner yesterday. My favorite thing is that I have my Muse, whose name is Oiler, constantly source increasingly obscure teas from increasingly obscure tea vendors. At this point, on a weekly basis, my Muse is buying some random tea from some farm in Taiwan or some farm in Japan, shipping it over, and bringing it back.

What’s your favorite Muse use case?

I don’t know. I use it for a ton of stuff. My favorite use case, and probably eval, for the product is having it coach me in MMA. I put cameras in my gym, and it watches the cameras when I’m in there and gives me feedback between the rounds, which is a very hard problem, it turns out, because it has to process all the video quickly enough. You only get 1 minute between the rounds.

Oh, it coaches you between the rounds?

Yeah. I want feedback.

I want feedback on how to win. You know what I mean?

It’s like, come on.

Did you have to instruct it? I’ve wondered about this, too, because looking at videos is a really good AI use case. It feels like you can’t just give it a multigigabyte video file. So do you tell it how to slice stills out of it?

No, it pulls stills. I think it figured that out by itself. It turns out that Muse has pretty good native understanding of MMA, so that’s good.

The pre-training.

Yeah, someone on the team knew. It was just kind of out there in the corpus. I think it’s a good test of the kind of coding ability because, really, what it’s doing is, within the agent, I’m not seeing any code, but it’s basically writing this whole video-processing, image-processing pipeline.

Yes.

I’m constantly telling it, “All right, look, you took 30 seconds to give me that feedback. I only have a 60-second break. You’ve got to go faster. Just optimize yourself, optimize yourself.” It’s a good example of how things are going to go in the future, where basically everyone is going to be writing tons of code. They’re just not going to know that they’re writing code.

What kind of MMA advice does a model give you?

You want to see?

Yeah.

Basically, let me see if I can pull some stuff out. It gives me a lot of stuff about balance and hip position. When you’re kind of in side control here, you didn’t get this underhook in place, so you’re vulnerable to getting rolled over here. I’ll pull up some stuff. You guys should try this.

It was definitely one of his first use cases, even with the initial prototypes.

Yeah. Yeah.

I did this with OpenClaw before we even had Muse working. That’s why I was saying it’s kind of become an eval for this, too.

It’s good. I actually didn’t know that it was an eval for the product, so noted.

Yeah, there you go.

How is all this represented in the model’s training data? The fine-grained stuff, like this is what an MMA fighter looks like when they’re appropriately balanced, and this is when they’re too much on the back foot. I’m wondering, how has it consumed so many images of appropriate versus inappropriate MMA posture?

Here you can see it. It just pulls out frames from the video, and it’s like, “Too much on my back leg there.”

Yeah, but is this the emergent intelligence? Does it know from seeing lots of images of MMA, or does it know because it understands the concept of where you’re meant to be balancing? Has it read books about MMA and then sees that you’re too much on the back foot, or—

2. Multimodality Powers Personal Agents

Well, we did train the models with a lot of visual-perception data.

Yeah.

I think that’s what goes into making good multimodal models. This gets into a core approach that we have, which is that we’re training the models end-to-end to be good personal agents, right? So the question is, why would Meta want to build these models?

First of all, I think superintelligence overall is going to allow us to run the company better and build all kinds of different things. But for the products that we’re building, there are certain capabilities and certain mixes of skills that are just much more applicable to these cases. This is one of the things that Alex focused on from the beginning: we really want to make sure that our multimodal program is good because that’s disproportionately important for the types of things that people use all these services for.

How do you train a good multimodal model? Is this where Meta just has an advantage because you guys have so much image and video data?

It’s very helpful.

Yeah. Ultimately, a lot of this comes down to how focused you are on it. Because we wanted to build personal superintelligence, and fundamentally, I think most people in the world experience the world in a highly multimodal way. They like talking to other people and listening to things and seeing videos and filming things of their own. Most consumers have this highly multimodal experience.

Especially earlier this year, there was actually a point of tension in the industry because I think so much of the consensus view within the AI industry was that it was all about coding and coding was all that mattered. For the success of a product like Muse and the success of a lot of these consumer use cases, it really depends on the ability to be strong and multimodal.

Yeah. I think we’re just scratching the surface of that. A lot of the initial theory that we had was that most people don’t actually like reading that much, right? Even the text-based interaction model is something that people use. Everyone texts. But when you think about things like the glasses that we’re working on, or even our other products that are very visual, the glasses will eventually just be a pure multimodal product, right?

They can let it hear what you are listening to, talk to you, and see what you see if you let it. Eventually, I don’t even know if we’re going to have an app store on the glasses. It’ll just be a multimodal generation that creates whatever you need right in your vision. I imagine something like that will probably end up powering a lot of the apps that we build, too. But we’re still in the early days of that. For now, the perception work has been really impressive.

3. AI Products Hide Their Complexity

I want to ask you guys about the product space here in AI and try something out on you. If I were to describe to you, say, with Stripe, what I think a winning product looks like, it would take me pages and pages to talk about because we have the developer experience, and we have to talk about all the international heterogeneity of payment methods. Then we have to talk about stablecoins. It would take me a long time.

Whereas I notice a phenomenon in AI where it feels like the winning products collapse very quickly onto an agreed-upon product space that everyone is pursuing. Transformers worked, and then the chat-agent UI really worked. Now we’re seeing—you mentioned OpenClaw earlier—the kind of persistent agent that has a computer and is able to do things. Computer use out there in the world. We’re collapsing onto that product space.

There’s something about AI where the winning products tend to be very simple, pursued by all the companies that are taking it seriously. You could describe them on the back of a postcard. You could describe them very concisely. Obviously, implementing them is massively complicated.

We’ll get into the new model. There are huge amounts of frontier work that go into making a good version. But the winning brand—now you’re talking about the glasses or the future versions. Somehow, there’s a very small number of clearly correct products in AI, as opposed to it being this very broad and complicated product space.

I kind of disagree.

Sure.

First of all, I think part of the theory from the beginning was that this kind of agent, or whatever form it took, was going to be quite a different thing from this text-based assistant, right? Every company has one of those. Ours is Meta AI. It reaches a ton of people, but it’s the same thing that all these companies have.

We specifically think that it’s used for very informational purposes, but people don’t think about it as a thing that gets stuff done for them.

It’s retrieval, not delegation, or something.

Yeah, and even when you say it has a computer...

Yeah, but there are lots of different ways to do that, right? I think that it's only this most recent wave that gives the agent a dedicated VM, which I think is actually incredibly important for making you feel confident that the security and privacy of your information is not just, “Okay, it's connecting to some elastic compute that everyone uses.”

But my claim isn't that it's easy to implement, nor that it's obvious—

I think there are going to be a lot of things.

Yeah. Okay. You think there's going to be a lot of different things? Because again, my claim is that now that people see Muse, everyone looks at that and says, “Oh yeah, that's clearly how you want the AI agent product to work,” and that the future AI agent products that we'll see will be fairly Muse-shaped.

So I think my take on this is that we're just so early in the realm of what AI products will look like that a lot of the interesting differentiation between products gets collapsed in how we view them today. I think the traditional way in which we wrote down and thought about product specs—those aren't the relevant dimensions, levers, and axes to consider when making important design decisions for AI-native products.

Muse, for example, had so much attention put into the design of the model for the product. There were a lot of very intentional decisions around, in all these situations, how exactly should the model behave? What's the exact personality of the model in these situations? How should it handle these sorts of tool-use scenarios? These are very intentional decisions, and the full spec of how we wanted the model to behave in all these scenarios would probably be an extremely long document. But because of just how we think about products today, it's just AI.

It's kind of like how, when mobile first started happening, it was like, “Oh, it's just an app, and everything is just an app.” Well, it turns out there's a lot of nuance in what exactly an app is. We're in this phase where it's, “Oh, it's just AI,” or “It's just an agent,” or “It's just an agent with a VM,” or something like that. But we have to tease out the interesting differentiations.

Okay. So you guys think I'm making the classic mistake, which is that my domain is complex and this domain is, again, simple from a product point of view.

All payment processors seem the same. Yeah, no, I really agree with what you're saying. One thing that we talked about early on is that there is basically this pattern where I think the products are getting thinner and the model is becoming more of the product. One of the things that we say internally is, “The model is the product.”

Yes.

So when you're talking about an agent, a lot of the complex product work that went into Muse is basically around the privacy and security model for handling your data and the things that get connected to it. That's something that we're very proud of and that we really focused on. But that's a lot of the product surface that we need to innovate on.

The capabilities, the personality, and a lot of the other stuff are things that you're training into the model. Whether it's the people in the lab who are working on that or the people on Nat's team who are working on the product, when Nat is thinking about what we need to make the product feel good, he's talking about post-training the model. Of course, the UI of the product matters too, but the most important part of the UI is the product.

Yeah, exactly. I think there's still, very viscerally, a feeling from a user of using Muse over time. Maybe some meaningful portion of that is just Muse's reliability in handling certain tasks. But another meaningful portion is: how proactive is it, and in what context is it proactive? When it talks to you, how do you feel about how it talks to you?

There's clearly a lot of that, and I'm starting to see more and more of it show up on the X feed. People are starting to tease out these differences. They're all proactive agents, but what are the actual meaningful differences between them? These, I think, are the interesting product decisions. The funny thing is, how do you specify these product decisions? Ultimately, they're in model design decisions, which boil down to training methods, data, evals, and all these things.

Yes. As we talk about product space, one notable Muse detail is that there is no model selector and no effort selector. Do you guys think that's just obviously correct in a consumer AI product, or—

I just want to make it simple.

Yeah. I think that, somewhat infamously, a lot of AI products have been designed by a highly technical group that does not have a randomly sampled American in its mental model. The products—I mean, the naming of some of the models is so egregious.

The numbers go backwards?

Sometimes the numbers go backwards, and sometimes it's just a totally different word and you have another set of words. How is somebody who's not carefully reading API design specs supposed to know what they're supposed to pick? It's far too confusing.

Okay, we're talking about models a lot. Tell me about the new model. It sounds awesome.

4. Muse Wonder Reaches The Frontier

Yeah, I think we're quite proud of this. The new model, Muse Wonder, is a new class of model at the frontier. We've basically been working on this since the beginning.

When you're doing these pre-trains, you don't just go to whatever the biggest cluster is that you have. You want to work your way up the ladder, training models that are maybe 4 or 5 times bigger, and then you keep repeating that process. The first set of models that we had, Muse Spark, was all based on this initial pre-training that we did in December.

Yeah, at the end of last year.

Yeah. We've all just been quite surprised and impressed with how much intelligence and capability the team has been able to build into a model which is, frankly, not that big. After that, we were working on this next model. The pre-training had been done for a bit, and we'd been working on the post-training. Now it is here.

It's the next class of model. It is significantly bigger, so I think you will feel that in terms of the nuance with which it can think. A lot of the areas that we focused on were making it the best personal and professional agentic model. Some of the different things that you can optimize for—that's it for us—and then, yeah, a lot of the basic capabilities that we focused on building in were very good memory and a very good ability to persist through complicated tasks. A ton of work went into this across the lab and the product teams.

I think it's also a testament to the team being able to methodically figure out the process to scale these models. This is a milestone on the way toward the time between each subsequently bigger class of models compressing. That's very exciting for the type of stuff that we can build.

At Meta or in the industry generally?

At Meta. I think you're seeing something like this everywhere else, but—

Is this an RSI effect or something else?

Well, at Meta, part of this is that we needed to—we were famously not happy with the trajectory that we were on with Llama 4. We rebuilt the whole team. That's been this crazy thing over the last year and a half that we should probably spend a bunch of time on.

Oh, we'll get to that. Yeah.

Some war stories. But the team basically had to go build up different tiers of models to get to this one. Now that we're at the frontier, everything that we do going forward will just be pushing the frontier, as opposed to the background work that we needed to do before that. I don't know, what would you say?

Yeah. Well, we're really excited about it. We will have 2 models in this series that are going to be available. There's Muse Wonder and Muse Wonder Full.

Yeah, very good.

Which is basically a much higher—

Studio band version.

Yeah, no, a much higher-end model.

It's the full version of Muse 1.

I know, I got it.

Yeah, yeah, it's not—

It's 2 words: Wonder Full version.

You're clearly a father. This is an excellent dad joke.

What's up?

Yeah, there you go. The performance—we've seen incredible performance across a slate of evaluations on personal and professional agent tasks. It's state-of-the-art based on our measurement.

There are a number of specific highlights that we're really excited about, which we've designed into the model over time. Active memory, like Mark mentioned, is one area where we think that, to build personal and professional agents, memory is just one of the clear requirements for building reliable agents. It's one of the areas where we've invested disproportionately, actually, compared to many of the other efforts.

And we have our own internal evals of MemoryBench, which demonstrate a lot of the capabilities there.

What does memory mean in a model to a layperson? Is it just how much stuff it can keep in its working memory if you're working on a task?

Yeah.

How's it different from context?

I think it's in the experience of using the model. How well does it remember things that you mentioned to it over days, weeks, or months, and then leverage those in the process of doing future tasks?

I see. This is the stuff that's buggy AF in OpenClaw. I think we're all extremely fond of OpenClaw. I'm a very happy OpenClaw user, but it ultimately does this compaction where it has long-term memory and working short-term memory or whatever. I think once people use OpenClaw for a while, they find the memory pretty buggy.

Yeah. So we've designed both the model and a system for the model to use to develop a much stronger form of memory. We call it active memory.

Yeah.

In addition, as Mark mentioned on the persistence—

But this is outside the model? The model exists and it's—

No, but some of the capabilities that you're talking about, like the compaction, are things that the model does, right? So basically, being able to do this well, both with—

It's not a harness; it's in the model itself.

Yes, it's in the model itself. Exactly. So it's aware of its memory as a core system. It's about how it's able to leverage various sources to conduct tasks in the future, as well as understand its ability to do compaction and how it should do that compaction most effectively.

Okay. But this is a big deal because I think a lot of people's early experience of AI agents has been buggy memory.

And to be clear, there's still plenty of room to improve. Our evaluations show this, but this is something that we're quite proud of, and we think this is a meaningful contribution.

Okay. So active memory is one thing.

Yeah.

Active memory. Active focus is another. It's effectively the ability for the model, when conducting tasks, to carry them through, especially as we've seen Muse users, as well as Muse Code users, give it increasingly complex and ambiguous tasks. We want the model to be able to conduct those very effectively. This is one of the things that, in early dogfooding and testing, a lot of people have noticed about the model that is special and stands out from other models they've tried or used. The third one is domain expertise.

What is focus in AI terms? Is it like putting a lot of compute on the task? Is it not getting distracted with other—

Persistence?

Yeah, like persistence. It's basically the ability to conduct complete tasks over very long horizons. Then, when faced with various setbacks along the way, it's the ability to identify other solutions to those problems in an aligned way.

Okay.

Then there's a big step up in coding capabilities—frontier coding capabilities. It's state-of-the-art on DeepSWE and SWE-MARATHON, and it does very well on other benchmarks like Terminal-Bench 4 and many others. We think that users who use it for coding, which will be a minority of all of our users, but the ones who do, will see a big improvement and a big step up.

Okay, so we should also just be using Muse Wonder Full for coding at Stripe?

Potentially, yeah.

But if it's state-of-the-art on SWE-bench, we'd be missing out.

Yeah, no, it's a big improvement on coding. Obviously, we want people to try it and see how they experience it, but we've been very pleased in our own internal testing.

Yeah, I think people will like it for a lot of things.

Okay, so this is both just your latest frontier model? Developers can come directly to you, use it on OpenRouter, and consume it as a model, and it's also a big upgrade inside Muse that people will feel as active users of Muse.

Yeah. What we're going to do in Muse, because it's a lot bigger and more expensive to serve, is start off primarily in paid subscriptions. What we want to do with Muse as a product is for it to be free for everyone to use. We have this huge amount of free tokens that people can use with Muse Spark as the model. But if you want to use Wonder for most of the hard tasks, then, at least for now, you'll use the paid version.

Eventually—you know how quickly the field moves—we hope to, at some point not too far from now, have a model that is as good as Wonder but in the Spark form factor. That will be a great thing to be able to serve hundreds of millions of people around the world.

Okay, so you go to Muse, you subscribe to the—what's it called—the premium or pro plan, or... pay money.

Yeah.

And just don't stop until you've taken out your credit card.

Yeah.

Oh, it just gets better.

It just gets better for you.

Yeah.

Then it'll also be available via API, on the Meta Model API and OpenRouter. It'll also be available for Muse Code users. And then Muse Wonder Full will be available to subscribers of Muse Code. It's pretty exciting. As Mark mentioned, it is a much larger model, so we're working through how to serve that most effectively to as many people as possible. But we're excited to get it in the hands of as many people as possible.

It's a big step forward. With Muse Spark, we were very happy with the performance that we were able to deliver as we were getting up to this point. But this is the beginning of MSL being at the frontier. We feel like the models are quite competitive with the best models that are out there.

Okay, so you think it's like Astra, Fable, and Wonder altogether at the frontier? I mean, you can look at it empirically by the benchmarks.

Empirically by the benchmarks, one would conclude that.

Yes.

I see.

Yes.

I think one of the things that we're especially proud of is that we've taken the time, as we've mentioned, to optimize models for the product form factor of Muse. This is a state-of-the-art model. We think it really is the best model in the world for the use case of personal professional agents.

People inside companies complain about Goodhart's Law: a metric that becomes a target ceases to be a good metric. You hear complaints that models are benchmaxed and that, at a certain point, overfitting to the benchmarks actually makes the models slightly worse. As people who actually have to train AI models, is that true? Are the benchmarks pretty good at grading models? Do you start to see weird edge failure cases from the benchmarks?

The benchmarks have to keep moving, right? It happens very quickly, but they just get saturated so quickly. I do think that—

But you think non-saturated benchmarks are fine?

As we've gone on, we steer the models more toward our internal benchmarks for the use cases that we're focused on. Early on, when we were just doing the science of trying to climb the ladder, we cared more about what the external benchmarks were, just to make sure we were on the right track. I'd say that, with each release, the larger share of what we're looking at are internal ones.

One of the things that we quantified in our research blog post as part of Muse Wonder is this metric for Big Model Smell. Big Model Smell and the feeling of benchmaxing are kind of inverses of one another.

This idea that you could just tell when you're using a really big model—you can tell.

It just has nuance.

It's got that je ne sais quoi.

The way we quantify that is almost exactly what you're mentioning: the gap between in-distribution performance and out-of-distribution performance. We measure how well the model performs on tasks that are in distribution of our RL and post-training versus tasks that are out of distribution. We look at what that gap is between the Muse Spark line and the Muse Wonder line, and we actually see this huge step up from the Muse Spark to the Muse Wonder line because, as Mark mentioned, it's just a much better base model powering this new line.

There's an entire je ne sais quoi to that, but I think we've been able to quantify this effect. It's an obvious dimension that we're excited to continue improving on.

But if you're saying that external benchmarks have ceased to become as important, and you care much more about your internal benchmarks.

Is that kind of an argument against benchmaxing, that you think that just trying to get the top score on the external benchmarks that everyone talks about is maybe not going to lead you to develop a great model?

Well, yeah, but also we're focused on specific things. Part of the whole strategy is personal superintelligence and just trying to build different types of products than other people are trying to build. So even if we did have the exact same evals as everyone else, it just—

Yes. And though I would say a personal superintelligence is not a niche interest. It's probably something that other models want to be able to do.

I think that there are things that are very different about it. I think the personality probably matters more. One of the things that we talk about is this idea of discretion, right? If you want to use the model to get something done, it needs to be able to know a lot about you and get things done in the world while disclosing as little as possible.

So if you have an allergy or you're pregnant or something, and you're trying to get a table at a restaurant, you don't want to have to tell them your medical condition. You want to make sure that it's going to be a good place for you. How do you handle that? Humans have common sense, and the models so far—

If you're building a coding agent that works in an environment where everyone has access to the same context, discretion isn't particularly important. So there are all these things that I think are just more important for us.

Yes.

Aspects of coding where the code needs to work, but you don't necessarily need to expose it to the consumers who are using it. They may not even ever see the code. That probably matters more in a personal superintelligence context rather than serving software developers. So I think these things are just a bit different.

Ultimately, we care about making useful models. We want models that, when you put them into people's hands, give them a great experience. I think that is the most important metric for any model that we or other people produce.

I think that really focusing on the external benchmarks, if you dig into the supply chain a little bit, means it's some PhD student who made a benchmark as part of doing their PhD. So you're outsourcing all of your product thinking to that PhD student who happened to make a viral benchmark.

Right.

So clearly that's wrong.

Yeah.

That's why we care a lot about crafting amazing evals. A huge part of what a lot of people at Meta and MSL are focused on is crafting amazing evals.

And now that we have them, we can work on our own. But if you're building a lab, you're ramping it up, those public ones aren't a crazy place to start. It's just not where you want to end up.

At that point, you're just looking for signs of life. But when you're really in this mode, as we are now, where we want to be very intentional about how we're improving our model over time to build better and better products, a lot of the cases that Mark just mentioned are detailed very rigorously in the spreadsheet of all the little capabilities and nuances of the product that we think really matter to the end user.

Maybe each of them matters 2% to the user, but in aggregate, that's the obsession that will ultimately result in these models that feel magical.

5. Frontier Models Start With Data

Can you guys describe for me just what goes into training a leading-edge frontier model? I'm curious because I think a lot of people listening to this have a rough idea that there's a pre-training and a post-training step.

Pre-training is when you ingest this huge amount of data. You look at a lot of pictures, and these are cats and these are dogs. After a while, you start to develop a sense for what is catness and what is dogness and what is fur and everything like that.

That's the pre-training step, where you just hoover up data. Maybe you can talk about the data acquisition and just what goes into that. Then there's this RL stage, which is the newish advance over the past few years of AI. It sounds like that is where a lot of the specific nuance of this model has come in, where you guys have chosen better domains, better hills to climb for the model.

At a high level, describe how many people, how long it takes, what the steps are, and how you think about those in—

How much time do you have?

At a high level.

This happens to be what we spend all of our time on.

Exactly. And so summarizing—

YouTube courses.

Yeah, I think at a very high level, these are the stages. It's pre-training and post-training. Ironically, training is the superset of all of it, but that's—

How many people are on this whole effort when you develop these models?

So this, I think, gets into what is interesting about MSL and the approach that we've taken. From the very beginning, I think when I first reached out to you, we were just looking at this because we were like, "Wow, okay, how far we've come." I pulled up the screenshot of when I first reached out to Alex. My first message was, "Hey, do you have time to catch up and talk?" Then we talked for—

That's how they get you.

Forty-four minutes or whatever. Then, 2 days later, Alex sends me this long thing, which is, "Here's what I think I would do if I were in your position."

The very first thing is that you don't need a lot of people, and talent density is the thing that matters. I'd say from the beginning of the plan, this was the core of your strategy, right? You don't need a massive number of people, but what you do need is for each of the people to be as good as possible.

That informed this crazy recruiting push and very intense filtering. We had to take some of the best people from inside Meta and use a very intense process to get people from outside Meta. But I actually think a lot of what has made MSL special, and why a lot of people have wanted to join, is because in some ways it's more fun to work on a talent-dense, smaller effort.

It feels like a startup that's operating. In some ways, it has all the benefits of a startup with the stability and resources of a larger company. This was point 1 in your plan for what Alex was like: this is what we need to go do.

I don't know the answer to how big a pre-training team is. It doesn't need to be that big. But if you look at some of the other labs, it's probably quite big in some other places. I think part of the reason why these models are doing well is because you have people who we're betting on, and they have a lot of resources because we're taking our compute. We're not peanut-buttering it across a lot of competing efforts. We're basically just investing in the key people, a lot of compute per researcher, and then they can do their work.

Was Wonder the one big training effort? Was that where most of the training was?

It was the next pre-training. It was climbing up this ladder. We have this tiering system, and then Spark. Internally, Avocado was the codename for the Muse Spark line, and Watermelon was the codename for this one. We're out of fruits now. We're out of fruits. We're going to have to switch to a new convention.

But I think the only fruit that is technically larger is the durian, and you don't want to go to the durian.

No, no, no, no. Yeah, yeah.

No, you want to stay far away.

A durian is bigger than a watermelon?

I think it's definitely heavier.

That's horrifying.

Definitely heavier.

Spikier.

Spikier, yeah.

But no, I think, as Mark mentioned, each of the stages I'll describe involves tens of people working on it at Meta versus hundreds at—

So Meta is a huge company, but this is a really tiny effort inside of Meta.

Yeah. The research team is less than 200 people.

Yeah, it's very small. It's intentionally very, very tight.

It's closer to 100, right? And then there's a bunch of infra folks.

There's a lot of infra work. At a very high level, to go back to the "what is it" question, how does it happen? How does a model happen?

That's basically what I'm asking.

Where do models come from? You take a huge amount of data for pre-training, and here you're—

And roughly, it's all the public internet, all the books you can get your hands on, and just as much data as possible.

But different types.

Plus the proprietary Meta data.

But also different modalities, right? If you want it to be good at perception, then you want to make sure it has a lot of images and audio and video and this kind of stuff. If you want it to be good at certain domains, you don't just want to RL that in at the end. You want it to have exposure to that from the beginning.

Yep.

Yeah. In this stage, you're much less concerned with exactly what the data looks like, how it's formatted, or anything about it.

But it’s just lots and lots of data.

When you talk about this being the best personal agent model, was there data you had to get for the pre-training where it’s like huge amounts of restaurant reservations, call booking—I don’t know what it is, but—

You just kind of want to dump in large volumes of the modalities that people are going to care about. There were a bunch of things. I don’t actually think we talked about this when we were talking about where Muse Wonder excels. But some of the areas where people care a lot are shopping, personal advice, and personal legal, health, and tax questions.

Muse Wonder is quite good at those. I don’t know if there’s any quantification that we want to give, but let’s say it’s—

It’s very good on those personal advice questions.

Yeah. You can RL those to some degree, but you also want to put in a large amount of data. You can imagine health is a very data-rich area. There’s a lot of data out there about health, and in pre-training, you just want it to see all these different things. So you put these different corpora in.

As Mark was mentioning, one of the highlights of Muse Wonder is its ability across various domains of expertise.

Yes.

I think our philosophy here is that the best personal agent should feel like a team of experts in your corner. It should feel like it is your legal team, your finance team, your medical team, and so on.

Across a bunch of these domains, Muse Wonder is actually state-of-the-art on a lot of the finance evaluations, as well as a lot of the legal evaluations. It is the state-of-the-art model. These domain areas start in pre-training. We look for a lot of domain data to put into pre-training.

Yes.

A lot of the art of pre-training is—there’s this one Roon tweet where it’s like, “Pre-training is full of researchers who chew glass and stare out into the distance and into the abyss,” or whatnot. But I think there’s truth to that, because it is both the highest-stakes research area—we invested a huge percentage of our compute into training Muse Wonder, into pre-training Muse Wonder—and it was very high-stakes.

I think probably both of us, at various points in that process, went through—

Troughs of sadness.

Yeah, troughs of uncertainty. All 7 stages of grief happen while you’re pre-training a model. It is also the most fundamentally scientific in an interesting way, because you just don’t have very many samples. Your ability to infer exactly what will happen scientifically really depends on your ability to reason deeply about everything that’s happening in this process.

Okay. So in the pre-training stage, everyone who’s training frontier models is basically trying to get as much data as possible for the pre-training, with perhaps a bent toward specific domains. If you want Muse Wonder to excel at legal, finance, and tax questions, then you want to have as much of that specific corpus as possible in the pre-training.

Yeah, you still want as much intentionality about where you ultimately want your model to be, but again, it benefits from a lot more data. This is the massive-data, massive-compute stage.

Do you hit some saturation in the data you put into the pre-training, or ultimately is it just that more data is better?

There’s a relationship between the size of the model and the amount of data. If you want the model to be bigger, you need more data; otherwise, it overfits.

Yes, but my point is: is there data where it’s like, “We can’t distract the model with that. We can’t clutter up its brain attic with knowing that stuff”? Or is it just that more data is better?

There are some of these effects, I think.

Yeah, because this is part of the science.

Okay, we’re getting into the secret sauce. Sorry.

No, it’s not even that. I think there are all these questions about whether we put a lot of these tokens in. Is there any zero-sumness? In a bunch of places, there isn’t; it’s actually positive transfer. But depending on how it’s set up, you can have some negative transfer.

Yes.

There are all these interesting questions. But in general, one of the bigger challenges of scaling bigger models is that it’s not just about how many FLOPs and how much compute you have. You want to be able to scale the data with that.

This is part of what came out of the original scaling laws for pre-training. If the internet isn’t growing that quickly and you want to scale bigger models that can be more intelligent, one of the interesting problems is: where is the data coming from?

Yes.

I think that probably gets to the end of what we’re going to talk about on that. But we could go to the other—

Excellent CEO energy over here.

So, post-training. I think post-training is a very exciting field now, and it’s an area where, on a continual basis, the entire field is learning about the optimal recipes for post-training these models.

For people, post-training is where you give the model a task that you expect it to do, grade it on how it does, and improve it. If it’s a chess-playing model, it plays chess games and learns that—

There are actually 2 stages. There’s SFT, or supervised fine-tuning. That’s when you take this raw thing and give it both the question and the answer that you would want it to have. If you’re having a chat conversation, how would you want it to respond in this case?

When you’re training things like personality, SFT is a very important part of molding it. But I think most of the advances in recent years have been in RL.

Yeah. One mental model that can be helpful is that SFT is where you initialize a lot of the behaviors that you want the model to be able to exhibit later on. At the end of SFT, it may have those behaviors baked in, but it’ll only do them 10% of the time.

Then, in reinforcement learning, you give the model reinforcement in various situations to bring the elicitation of those behaviors dramatically up. There are a bunch of interesting things happening in RL and post-training.

One is that scaling RL and post-training has been one of the clear axes: spending more compute and gathering more data.

Just doing much more of it before you release a model.

Yes, doing much more of it, especially as the pre-training scales up.

It’s like anything in scaling laws: it has to go with the rest of the factors, or it doesn’t work.

Yeah, in general. If I were to make one broad statement about AI, it’s just: scale everything.

But in particular, you have to scale everything. Scaling one thing in isolation does not work.

You technically can.

You can scale one dimension. You’ll just get much more optimal performance if you scale many things altogether.

One way to think about Muse Wonder is: we did the pre-training, and we’ve done some RL. When you look at Muse Spark—version 1, 1.1, 1.2, 1.3—we didn’t change anything about pre-training. We just continued doing more RL on top of it.

When you think about the early versions of a model of a new class, they’re relatively less mature. The RL-to-pre-training ratio is going to be lower for RL than in the later versions.

That is kind of what the point releases in models are.

I think every company has its own convention. For us, that’s what it’s meant to be.

But for a lot of companies, the big number is the pre-training, and then the—

I think it has kind of lost that semantic value. I think now you actually can’t do that.

Okay, but that is true at Meta, at least.

But we haven’t even done it. We kind of decided not to do it.

It’s true at Llama 4 Maverick, at least.

It is true at Llama 4 Maverick and Llama 4 Behemoth. It’s true that Llama 4 Maverick and Llama 4 Behemoth are different pre-training runs.

We decided to kick the can down the road on this question for Llama 4 Behemoth by not giving it a number yet. But when we have our first dot release, we’ll have to decide that.

The RL stage is very interesting. First, there’s producing the environments and producing the tasks. That’s one of the areas where there’s a lot of intentionality in training these models. It’s also one of the areas where you notice very funny things.

In a lot of cases, less is actually more when it comes to data in RL. I think the feel and the science of it are generally pretty flipped from pre-training. The other thing is that it can give very magnified instruction to the model, if that makes sense.

So I think that a lot of safety or alignment cases that we care about are where, when you do large, high-compute RL runs on frontier models, you apply a lot of RL or optimization pressure in scenarios where the only way for the model to succeed is to do something misaligned or unsafe. That’s another thing that we had to be very careful and thoughtful about in training Llama 4 Behemoth: ensuring that we curated the RL environments very thoughtfully to ensure that all of them were eliciting and driving toward more aligned behavior from the model.

The point that you’re making about quality is an important one, because a lot of the previous questions back and forth were about quantity and had the flavor of, “Oh, you need to scale the quantity.” But I do think a huge part of this is the point that you just made: even a smaller amount of high-quality data.

Yeah, I think in RL, as we’ve seen pretty big jumps in our new Spark line, these have been accompanied by real, meaningful improvements to the underlying recipe. Each of these improvements in the recipe have been—I think the improvement between Muse Spark 1 and Muse Spark 1.3 feels night and day. I think, probably to most people, if you ask them, “Do these feel like they came from different pretrains?” they would probably say yes, because I think you can—

I think it’s happening industry-wide, for what it’s worth. We are learning to squeeze so much capability and elicit so much agentic capability from these existing base models. Now, that’s not to say that we believe scaling is unimportant. That’s why we have the Muse Wonder line in the first place. But these two things are happening at the same time, which is also why, for us, we’re seeing so much progress in general. I think it’s the fact that we’re scaling both pretraining and our recipes in RL concurrently, at a nice clip.

And then there are these other scaling axes too. Making sure that your agent harness is good and that you can co-optimize these things is probably just another dimension for scaling.

But if I’m understanding you guys correctly, are you saying that alignment is not about coming up with some alignment-specific RL steps, but instead about making sure the entire RL process is steering the model—

You have to be thoughtful about it throughout the entire process of RL. So I think this is a holistic effort.

But also, I think different labs have very different theories. Alignment is a very philosophical concept: who do you want it to be aligned to? A lot of our theory is that we care more about it being aligned to—you know, we’re trying to give people agents, right? If you ask it to do something, you need to have high confidence that it’s going to do what you ask it to do. You want it to do that with high reliability. You don’t want a moralizing lecture about why you’re wrong for asking it to do the thing.

Obviously, if you’re trying to do something catastrophic, it needs to gracefully—not piss you off—not do it. But in general, we view alignment as more of a question of instruction and values alignment with the person it’s intending to serve. I think that’s a different mode of thinking about it. It’s very different from how some of the others think about it.

One of the more interesting things, which is maybe related to your original question, is also just how long the process of training one of these models takes. In general, we’ve planned Muse Wonder since last August.

Yeah, I remember the offsite when they first presented this.

Yeah, all of the stages that we just talked about are things you really work on for quite a long time. It’s not that we started working on Muse Wonder when we finished Muse Spark. We’ve actually just been planning and working on this for quite a long time. We really view Llama 4 Maverick, if anything—as I think we mentioned in the first blog post where we wrote about it—as a stepping stone to get to Llama 4 Behemoth. That really was how we thought about it.

Even at the time, you thought about it that way?

Yeah, and in terms of how long it takes, we’re also talking just about the research part. There’s also the compute part, where we’ve been investing in building up a huge amount of compute overall.

Part of our philosophy on this is that you can measure the compute strength of a lab by how big its largest single computer, or supercomputer, is for being able to do these large training runs. We built this Prometheus cluster in Ohio, which is scaling to larger than a gigawatt. Now we’re building this Hyperion cluster in Louisiana, which is going to scale to 5 gigawatts.

Those actually predate Meta Superintelligence Labs, because building a whole data center site from scratch—the plans for those are coming online now, but that was—

You guys thought it might be the feed, right? That you could just use it for a lot of compute generally?

No, no, no. This predates even going back to the LLaMA program.

I see.

It was probably in 2023, or maybe 2023 or 2024, when we started making the plans to build those clusters. They’re now fully coming online. I guess the Louisiana one will come online over a couple of years.

Yeah.

But if you want to go really end-to-end on doing this stuff, then obviously the full stack—and Meta is a full-stack tech company. We build our own data centers, we build our own chips, and we build our own low-level infrastructure. Now this is the next component, which is arguably probably the most important component in all of this for the next wave of technology: building really good models and then delivering them to people.

If you want to go fully end-to-end, this is a long process.

I do think this is—maybe it’s well appreciated by folks in the tech industry, but it really is—you know, it feels like models are coming out constantly. So maybe you’d have the mental model that a company launches a model and then figures out how it’s going to make the next model, or something like that.

But it’s actually these very long planning processes and these very long workstreams that all come together at the very end and then result in these beautiful models.

Yeah.

6. Frontier AI Demands Big Bets

How do you build good products when the feedback cycle is so long? Ultimately, you start planning the model in August, and you don’t know until more than a year later if it’s good. Silicon Valley famously likes to run on metrics, but ultimately, do you just need someone who is the aesthete and says, “I’m sorry, I don’t like it”?

It’s nice when you have metrics, but I think the state of being an entrepreneur is how well you make decisions in an information-poor environment, right? So—

But this is that on steroids, right?

That on mega steroids. Yeah, I think that this is—and I think Mark gets a huge amount of credit for this—because of how long everything takes, there were many points over the past year since I’ve been here, but also, as you mentioned, long before that, where you have to make very large investments or make very big decisions about how you’re going to invest in or develop the technology, or what decisions you’re going to make, long before there are proof points around whether or not the thing is even working.

In some sense, it’s entrepreneurship on hard mode. It’s the opposite of Lean Startup. Lean Startup is like, you see if you get something working, then you double down, and then you see if that works, and then you double down. You cannot do that in AI. You have to weave— I mean, we’ve been investing in these data centers for years and years and years. We’ve had to—

And you’ll be radically behind if you wait to see if things are working.

Yeah. We have these conversations where, at the end of last year, we were like, “All right, we’re quite excited about the progress. We haven’t seen the models yet. We’re going to write this check for—I don’t know—$100 billion, $200 billion.” I’m like, “Alex, are you sure this is going to work?”

It was so extreme that we had the whole executive team—

I think Susan scared you a bit.

Yeah. The joke every time we talked about it would just be, “Yeah, and we hope it’s all going to work and be good.” But at the end of last year, other than me giving a vibe report on how our research was going—and I could show some plots and whatnot—I was feeling good about it, but it was certainly not clear at all that it was going to work.

Yeah.

You guys are saying that the essence of training a frontier model is that you have to push your chips all in before you’ve been dealt the cards.

You just have to decide that you're going to develop a frontier model, and you don't get to meter the inputs.

At this scale. If we go back a few years, the clusters didn't have to be this big, but now they do. So I think this is part of the excitement of booting up a frontier lab now, right? Basically, the ante is extremely high.

And I think this—I kind of viscerally remember this. We were in Hawaii at the end of last year. It was kind of like the whole executive team got together, and Susan pulled me aside many times and was like, “I just want to tell you, if this doesn't work, this is going to be really, really bad. It's going to be very, very bad in financial terms.”

I really need to impress on you that this model has to work.

It has to be good. If it's good, it's all good. We're great. But if it's bad, it's really, really, really bad. And Susan, who you know—

And that was useful motivation?

It was.

Well, it was one of these very funny moments where I was like, obviously, the amounts of money are so obscenely large.

What are the amounts of money in training? Order of magnitude, whatever you can share, but just how much are they costing?

Well, you can do the classic math. A gigawatt cluster—I think what you'd see out there would be, to operate a gigawatt for a year, call it on the order of $15 billion.

Okay, so it's roughly a $15 billion effort to train a bunch of models.

To operate a gigawatt cluster for a year. If you're talking about building the CapEx, it's probably closer to $50 or $60 billion. You pay for it upfront, but that's just one cluster for one part of training.

Yeah.

Then you have all the other parts. Research isn't just doing the run, right? That's probably an equal amount of it, because for each change that you make, you're testing all the ladders all the way up to the big scale. Then they're serving the stuff.

So tens of billions of dollars to train a frontier model now.

Yeah, definitely. I've seen the financial rollups. Tens of billions of dollars is a good estimate for—

Yeah.

And then you want to do something with it, right? I actually think some of the more complicated questions that we had were, “All right, so we're training this model, and we know we want to train the model because this is key, but how much product demand do we think we're going to have?”

Right.

Because now the product doesn't exist.

And as in, you're trying to build data centers to serve users for a product that doesn't exist yet.

Yeah.

And we have to make it years in advance, right? So this is probably where, when we were talking about it last year, there was a moment where I think a bunch of people were like, “Yeah, maybe we should just be a little more conservative.” And it's like, no, we're doing this. We're going to make it work. We're going to make it work.

We're going to burn the boats.

Yeah, this is going to work.

But it was like—even though I too am an entrepreneur and founder and have, at various points in my life, taken risk, I remember the visceral feeling at the time was, “This is a fuck ton of risk. I don't know if I really—”

Oh, it's only riskier.

Well, I do think this is one of the things that I think, Mark, is very impressive about you, which is that you have really internalized this concept of, you know, it's about how big it is when you win, not failure is okay. It's just how big the win is when you do win. And this was, I think—

Mark has fully internalized power laws in a way that many people have not.

I think quite deeply, yeah.

Well, I think the other piece is, I don't actually think it's quite as risky as this. The numbers are big, but I think—

Meta has a backstop in terms of how it can use compute.

Yeah, so I think there are 2 things. One is, if AI works overall, plenty of other people want all our compute. They're willing to pay, like, if the cost is whatever, $15 billion a gigawatt, we could get a lot more for that. So in order for it not to work out, it's not just that we have to fail; AI overall has to fail.

Yes.

And then even if that happens, we still have all these other uses to productively use compute. So in the grand scheme of things, we were definitely building out this compute for inference ahead of knowing that the product was going to be ready.

But you were AI-pilled enough that you said GPUs online and powered will be useful to someone.

Yeah, but also we believed in what we were building. You can look at the trajectory where it's like, all right, first there was ChatGPT as the text-based assistant. Then Anthropic got coding agents to work. No one had built the personal consumer version of that, and it just seemed like this is the thing that Meta should build in the world. I think that was pretty clear to you and me and Nat from the beginning, right? That's kind of what we're here to do.

I don't know. There's always the real question for me on the Muse launch: you never know when you launch something how it's going to be received, right? There's always the case where you get a home run, which in this case it was, right?

Yes.

And it's just extremely well received right off the bat. But the normal case would be that it's fine, and you have to iterate a few times to get product-market fit before you really grow. In some ways, I actually think we maybe could have grown the product faster, but we didn't have all the growth stuff really ready to go when we shipped it because our median expectation was, “All right, we're probably going to have to iterate on this a bit, and that'll be fine.”

But I don't know. If it takes a few iterations, it's not the end of the world. That's normal. You can have launches where it gets kind of catastrophic and then it takes a while to recover, but that's not good. Even if it took a few iterations, it would have been fine. So I guess it just seemed to me like this was going to be—yeah, the numbers were big, but it seemed like it was probably—

Was it bonding for the executive team to be holding hands, taking this risk together? You're talking about Susan taking you aside?

Bonding or bonding moment?

Is that a bonding moment?

It's stressful.

I think it was—

Both.

Both. Yeah. Being in a stressful situation together is generally bonding.

One thing that I think is quite notable about Meta is the degree to which, first of all, most of the executive team has been there quite a long time. Nat and I were really newbies. Most people have been through crazy situations many times in the past as well.

Mm-hmm.

And I think Meta as a company is an extremely resilient company that has been through many situations that, at the time, would've seemed incredibly daunting. So everyone knows the program, so to speak. This obviously, on paper, and especially when Susan Li, our CFO, is pulling me aside in the meetings, does seem like a particularly scary situation. But I think most people were like, “Yeah, yeah, no, but we're Meta. We're going to get through it.” I think there was real camaraderie around that and quite a nice energy around that, which I did appreciate.

I think it would be very easy for the other executives at the company who are operating and running—

Who are paying the bills.

Yeah, one of the greatest businesses of all time.

Yes.

And it'd be very easy for them to be quite annoyed by this whole endeavor. But I think everyone is also very excited about—

They are.

The opportunities for the company with AI. Taking a step back, I think one of Mark's jobs as the founder and CEO of the company is to ensure that the company is always pushing towards the most ambitious and expansive version of what it can be. One of the great things about this entire MSL and AI push is that this is one of the rawest expressions of that.

We knew that AI certainly is a very important technology and it's one of the bigger technologies right now. But if we were able to build it effectively and properly integrate it into everything that we do, as well as, as a full-stack technology company, build it into these incredible products that we know will exist in the future, this could be immensely expansive for Meta.

And I think this is true. Muse is an incredibly exciting product. It can go in 1,000 different directions. I think it's probably one of the most exciting products to just think about—when you're lying awake late at night, thinking about all the ways it can go.

I think Nat, Mark, and I just shoot messages back and forth late at night: “Oh, we could do this, and we could do this. We can make this move, and we can build this product.” I think it's very—

It's a very exciting time in technology right now.

Yeah, it's very blue ocean in that way.

Yeah.

Yeah.

I'm struck by, as we talk about Muse, how it clearly feels like the next platform—an agent operating on your behalf. You've talked for a long time about not wanting to be reliant on other platforms, and how the experience of building on Android and iOS burned you on being dependent on others. Often when you would talk about, say, Reality Labs, you'd emphasize the importance of owning the platform of the future, and how you've spent many years working on Reality Labs with the intent of owning that platform. Now this new modality comes along that feels pretty clearly like how you do a lot of productive work, and it is the platform of the future. Meta is really well set up.

Yeah, no, I mean, it's less about wanting to own a platform for its own sake. It's about being able to build the best things. At some level, if you look at other social apps, for example, I think part of the reason why Meta is so much better is because we don't just build the app, right? A lot of people think about us as an app company. We're not an app company. We're a deep technology company, and we do this because we're world-class at machine learning. All the recommendation systems are operating at a scale that basically few others are at in the world. I think it's basically just us and ByteDance on that, and then we just go all the way to the metal. To me, being able to also deliver—to go the last mile through the device—does come from having these experiences where we couldn't build the experiences that we wanted on phones. But I just think for the future, the models are going to be by far the most important part of delivering the technology experience.

This is something that I think we bonded over and agreed on very early: in every generation of technology companies, there's usually not one company that wins everything. I don't think that's how this is going to go either. With the internet, there were 5 or 6 very big companies, and then a whole ecosystem of other companies that were great. The companies that did the best had the core technology to serve people the best. I think in this next era, there are probably going to be a small handful of companies that have the ability to build frontier superintelligence. Those companies are going to be the ones that effectively drive the agenda and can build the most interesting experiences for consumers, enterprises, hardware, and these different kinds of things.

I just look at it as, is it even really a choice? Meta will probably exist into the future regardless of whether we do this. But if we want to be an interesting company that's inventing new things that matter to people in the future, I don't see how you can do that if you don't have the ability to build your own superintelligence end-to-end and build it into the products that people use. This is something that you just believed from very early on: there are going to be a handful of these. There's a personal and consumer-oriented version of this, and that lane is open because everyone else seems to be so obsessed with automating knowledge work—which isn't even that charismatic a version of the future, right? A big part of our belief is that the highest use of superintelligence is invention and creation, not automation. It just seemed like there's a big place in there, but you need to own the stack.

How is the Muse-mediated future going to shake out? Businesses variously need to build support for Muse, where it might try to use a computer to work its way in, but that might be janky in certain ways. You kind of want everyone to have first-class support for agents talking to things.

Yeah.

Then the bigger question is whether they might actively try to block agents from interacting with them. This might be because they have a misguided sense that it's actually in their interest to support agents, but they don't want to. It might actually not be in their interest to support agents. You can imagine that maybe they have a very thin moat, and it all comes from a business that only succeeds by consumers spending time on the site or only succeeds by overcharging consumers. They should rationally not want to allow agents, because the agents are like, why?

But those are going to get disrupted.

Yeah. So talk to me through the second-order effects of how the economy works.

Well, I think that it starts off with—we trained the models to have very good browser use, right? We want to make it so that no one has to do anything to integrate with Muse. Then, of course, we're going down the list of the most important partners, and for people who want to work with us, we'll just get direct API access, right? Then it ends up being way more efficient and faster for people to use Muse. It's more efficient because now you don't have an agent crawling your website; it just goes. Anyone who wants to do that, we eventually want to get there.

In the fullness of time, just like today every business has a website, social media accounts, email addresses, and all these things, every business is going to have agents too. We're also building that on the other side of it. Of course Muse agents will talk to other types of agents, but they'll work really well with our agents, right?

Because we'll be able to integrate that. So Muse using websites is just a moment-in-time thing.

It's just to make it easy for everyone. You want to have it as a backstop so it's reliable for people using it, and easy for businesses to get bootstrapped. But we have way more businesses calling us asking to integrate deeply, and very few who have tried to block it. My guess is that the ones that try to block it will end up finding that it's not the right strategic thing for them. But we'll see. This will play out over time.

It saves them a lot of time.

Yeah.

It saves them a lot of money. I think these are clearly good things for the consumer. In the long arc, you'd clearly expect that demand will shift from current avenues more and more toward agentic demand. There always will be demand that lives elsewhere, just in the same way that there's still brick-and-mortar demand in commerce. But there's a lot of e-commerce demand, and in the same way, I think agentic commerce demand will grow as a portion of the pie. The others will still exist, but agentic commerce will grow as a portion of the pie.

I think there are questions for businesses in the long run, like, do they want to participate in that slice or that portion of the economy over time? I think it's an incredibly exciting area. It's very, very exciting to imagine what will happen in this agentic commerce future.

I actually think Patrick wrote a great tweeticle. Is that what we call it these days?

I don't know. Patrick has his own thing going on with tweets. They're buried among—

I noticed that The Brothers Karamazov was quite a good literary analysis. But yeah, no, I think clearly this is a discontinuous event for how commerce will happen, broadly speaking—or just generally speaking, how all the business pipes will work, maybe even in a broader sense. But you just go back to this being clearly a lot better for consumers. It's a lot better for customers. So there's a natural gravity toward that reality.

One of the exciting things—and I'm curious what you guys see too—is that I think Muse is one of the first products where people are fully delegating purchases to the agents. This is a step change, right? We've been talking a lot about agentic shopping and the possibility of it, but now it's really happening. A lot of people are trusting the agents to shop for them. I think the agents are finding them—in my case, great teas, or in whomever's case, great products for them to purchase—which is quite exciting. There's clearly an exciting thing happening here. Part of the value proposition for the user is reduced friction.

Hey, I don't feel like clicking on all these websites and doing all this research to try to find the exact products. That step is taken care of for me. Also, criteria can just be automatically applied. Hey, I need something by Wednesday, so the agent will only look for things that could get to me by Wednesday.

Yeah, exactly. The old way of searching for stuff just feels so obviously primitive now that you can do this. Then there's the back part of the funnel, so to speak, but the ease of friction reduction—just filling out the form and getting it ordered—is quite nice. I think the long-term thing that we're quite excited about is greater personalization.

Ultimately, one of the exciting things about Muse is that it has this holistic context into you as a person and all of the aspects of your life. It can holistically understand things about your family, coworkers, or friends, and then help you co-optimize across all of those to get you the exact right gift for your best friend in the context of your friend group, based on what it knows about your friend group. I think there's a lot of exciting opportunities that are somewhat new and novel when it comes to these personalizations and recommendations.

Historically, a lot of these recommendation systems are quite siloed and isolated. They don't see a holistic view of you as a person, and Muse has an opportunity to offer much greater levels of personalization.

Yeah, I think people are already seeing that. If they ask Muse to buy something for them, quite frequently it'll add in some bit of context that it pulled in from a different conversation.

You had a good example of this.

My 3-year-old daughter loves baking. She loves cooking. I'm normally into smoking meats, but I'll do some baking in order to make Aurelia happy. We've gotten into this really cute routine where I've been playing with Muse voice mode, which isn't fully rolled out yet, but we're playing with it. It does this great thing: it segments the voices, so it knows which one is me and which one is not me based on that.

Throughout the week, we're talking about what we'll want to bake that weekend. She'll give it ideas or come up with crazy things that are not real recipes and bounce them off Muse. Then Muse makes sure that every Saturday, when it's time to bake, I get the ingredients from Instacart, and they show up on time.

The biggest thing is that if I were just starting the activity on Saturday, you'd have to get everything and it would take some time. Now everything's ready whenever we're going to sit down and do it. This weekend, it knew that watermelon was launching, so it threw a watermelon in the bag. It was very sweet and very heartwarming.

It deduced that the recipe didn't have enough expense to get the delivery, so it needed something else.

And it topped it up with a watermelon.

It was like, “Do you want this watermelon?” I was like, “Yes.” That's very cute.

Did you eat that watermelon?

Yeah, that was the one I ate on the livestream.

Muse just got it for me.

You just got it for me.

That was a very memeable moment of that very memeable situation. So we have Muse to thank for it.

Here's another interesting thing about the whole thing: I didn't really use Instacart that much before. Every once in a while, maybe twice a year, I'd use Priscilla's account or something.

Yes.

I went and got an Instacart account to give Muse, and now I use it once or twice a week because I'm talking with Aurelia through Muse, we're coming up with these recipes, and it's delivering them. Now it's a core part of my workflow.

I actually think that the opportunity here isn't just that it's going to be 20% more efficient on the conversion rate. I'm a new Instacart subscriber, and I just wouldn't have done this before.

Yes.

I do think that there's this question, which is maybe the more interesting economic question: is that Instacart account really for me, or is it for Muse? Over time, is the total addressable market for all of commerce just increasing dramatically?

Yes.

It's not just people anymore. Now agents are effectively going to have accounts. I think that's where this starts to become a really exciting opportunity, where anyone who sits this out is going to end up feeling sad over time.

And why did you have to sign up for an Instacart account? We're solving this at Stripe Projects for developer stuff where your coding agent—

The Stripe integration, by the way, is great. You guys did a great job with us on Muse. I think we're very happy.

We have been very excited about agent-to-commerce for a long time, obviously, because it felt to us like it's clearly how people are going to buy things in the future. That was apparent because no one needs all that friction in their life from the actual buying stage, and because early product research with AI had product-market fit, even from the earliest AIs.

It really feels like computer use was the last big unlock that we needed to make it happen, because now you can ask your Muse to buy anything, more or less. It'll just figure it out with computer use, right? It's the ultimate backwards compatibility layer.

Meta is building increasingly capable models, including Muse Wonder, and their models are available on OpenRouter, Alongside hundreds of others. Stripe has agreed to acquire OpenRouter, who's the leading model gateway. OpenRouter already routes more than 10 trillion tokens a day and handles much of the complexity of working with different models through a single API, making it easier to experiment, find competitive prices, and automatically route around provider outages. Try it for yourself at openrouter.ai.

Hey, could you order a watermelon to Stripe's office in South San Francisco? I'm doing a podcast here. This would be funny.

Funny podcast idea.

I'll check that watermelon delivery now.

Great. We'll see how we get on. Well, okay. This actually raises an interesting question. People have been trying to make HAL 9000.

It's working.

I really like this.

I think this is an awesome touch, where it's clacking away at its laptop. I really like that in the app. We'll just leave them there and see what we get. Yeah, it's tough to prop up. Yeah, there we go.

People have been trying to make the speak-to-a-computer AI vision happen for a really long time. Apple launched Siri in 2014 or 2015, something like that. Obviously, there's been Voice Mode and various AI products for a few years now. Why is it so hard? It's still not the case that if you were driving your car and wanted to check on the baking or move a meeting or something like that, you would actually do it with your voice. Maybe almost now with the latest things.

Yeah, well, there's a lot.

But you wouldn't develop Siri like you're doing.

Siri right now is actually a good example, as we're talking over each other. I don't know, when you're using different voices—

So is it just technical limitations? That's why we haven't done it?

Well, I think that there are a few things. One is, you actually want it to go do stuff. So there's a question of, when you speak with it, even if it understands, how's that going to work?

Then I think that there's a lot of stuff about the way that people have conversations. It isn't typically, “I will give a prompt, and now I will wait, and you will speak,” and then it just goes back and forth. How many of your sessions, when you're talking to an agent in voice mode, are basically, “Oh, it starts saying something, but then I feel like if I pause for 3 seconds, it's going to think I'm done and want to speak”? And then if it starts speaking but I cut back in, is it going to think that it's a whole new question?

I think there's a lot of stuff about the conversational dynamics that you want to get right. But then there's this other issue, which is that for voice overall, it's quite high-bandwidth for you to communicate with voice, but in terms of consuming information, having someone speak to you is very inefficient compared to having it be visual.

Yeah.

So I think that's why, partially, when we do these interfaces, it's kind of working on its task. I fired it off now. I think the ideal interface is probably not just strictly voice-to-voice, although I think voice output from you is pretty useful.

Yeah, I think there very clearly already is an explosion of voice input. I think you even see this in coding, where—

Yeah, people are speaking to the—

Yeah, yeah, yeah.

Most of the researchers just kind of whisper into their keyboard because, as Mark's mentioning, it's the highest-bandwidth way to get your thoughts out, or usually the lowest-friction way to get your thoughts out.

Yeah, yeah, yeah.

I think the most extreme version of this, by the way, is how college students now use AI. That may be the most interesting. I think a very common modality of using AI is that they just completely word-vomit for 10 or 15 minutes, in a full stream of consciousness, chain of thought, all the things.

And then have AI clean it up and infer.

Yeah, and have AI effectively do all of this organization and pull things together and structure everything. It's quite interesting. It's a very interesting modality of tech usage.

Yeah.

But yeah, I think voice input has been big. I do think agents are a big step change, because I think the mental model is just very different. If it's just an informational thing that you're talking to, you will clearly get into situations where it just feels useless. Whereas now that it can do things for you, it's much closer to the experience of talking to a human.

Totally.

Or to a person. I think there are all these interesting uncanny valleys in interacting with AIs that, until they've been able to take actions, have felt kind of strange. Now that we have agents, I think one of the cool things about Muse is this confluence of a number of things.

The fact that Muse is embodied, and it's this little guy that you feel like is working on your behalf, is only possible and a coherent representation and embodiment of the product because it has persistent memory, the ability to do things for you, the ability to take actions, and a little bit of a personality. It's the confluence of all these things coming together that makes this embodiment concept even work in the first place.

Then it turns out, actually, this is the real glasses, right? A lot of people want it even with no display or a small display.

Yes. The camera and audio are enough of a useful product.

Yeah. Then we started shipping the display glasses that aren't the kind of wide-field-of-view, full augmented reality, which I thought would be awesome too. But even the small display is pretty powerful. I think you'll start getting some interesting interactions there that are fully multimodal, right? So it's voice in and out, it'll be able to see what you see, and it'll be able to paint the pixels.

And one of the other exciting things that we've done here is Muse Gadgets, which is really unleashing the ecosystem to try cool device formats and form factors that could be really exciting.

Since releasing that, we've gotten a number of phone manufacturers talking to us, excited: “We're just going to integrate Muse Gadgets into our phone and see how well it works.” We're really excited about that. I think part of what is cool about the current moment of technology is that it's very clear agents are really exciting. It's really clear that this should unlock lots of things.

Will you guys come back to household devices? This is obviously a personal device, but we actually had, during COVID, the Portal—

The Portal. Portal.

There we go.

Yeah, he's finding the Portal inside.

One of the gadgets that we just shipped was the little HomeLink. It's the size of a little nickel or something, but with a USB, right? You can basically connect that, and then your Muse can connect to everything in your home.

This is actually one of the most fun things, too: having it connect to your Sonos and connect to all your lights. I have it set so that when it's time for me to start winding down in the evening, I just have it dim all the lights across the house, and it's nice.

How are we going to fund the AI buildout? Everyone's worried about Google going free-cash-flow negative. You were talking about 1 gigawatt and then 5 gigawatts and everything like this. Meta's not free-cash-flow negative yet, but there's always more CapEx to fund. So what's the answer?

Where is Susan Li?

Exactly.

No, I mean, look, I think that there are going to be a lot of ways to create value from this. What we're trying to do with Muse specifically is make it so that it can be widely used, because I think that there are going to be some valuable network effects at scale as people start being able to interact with each other, and that's going to be great.

I think there are all these reasons why you want it to get to large scale. We were talking about partners. Partners are going to want to integrate more the more people are using it, so it will get more valuable if we can get it to scale. The best way to do that is basically to have a model where it is affordable and free for people to use.

But there are going to be a lot of ways that I think we can scale revenue. Subscriptions are going to be a big one. Just because the base version of it is free and we want the free version to be excellent doesn't mean that there aren't things that naturally aren't going to be economical to include in that.

So, if you want to start using Muse Wonder in Muse tomorrow, the way that you'll do that is you'll pay for a subscription. I think that will be a big business. There are a bunch of other things around commerce that we're quite excited about, too, where we believe that Muse is actually going to make people money and save people money by using it.

Over time, there are lots of different ways to effectively take a small cut of what you are saving or making for people, and that will end up being a very large revenue stream. And then there's all the other things, right?

So you think Muse will generally just cost negative money for people because it helps them out?

Well, there are already all these sites that I think are interesting where people are like, “Go look through my emails, find a bunch of subscriptions I don't actually use, cancel them,” whatever. I don't know if that's the thing that we would monetize, but you get the point: it helps you make money.

One of the ideas that we talk about—and this is something that I think you've been kind of obsessed with—is making it like a business in a box for small businesses.

So someone can just— it can help them administer all these parts of what they're doing. You can imagine that being very valuable and very profitable, both for the business owner and for Meta. But I think one of the things that's interesting is, okay, if we're building the models to build the personal agents and the business agent, there is this large surplus of value because now the model is very general, right? It doesn't just do those things. We'll also have the API that'll be profitable.

A big part of why we want to start the Meta Enterprise Platform and have CJ Desai join us to build this is that we have this view: traditionally, Meta hasn't been an enterprise company, but we believe that a huge part of the challenge is building the leading models in the first place. They're very general. Once we make a significant investment in building an enterprise group that can provide that in the way enterprises need to consume it, I think a lot of enterprises are going to want to use Muse Wonder. It's going to be an excellent and affordable model.

I think that is going to end up being a very big business too. All of these things put together, coupled with the fact that we're already starting off as a very profitable business that has good credit and can underwrite these large investments, puts us in a pretty good position as a lab to build out the compute we need to build leading products, build up the network around this, and position the company to really lead in the area for decades to come.

Okay, so you think ultimately there'll just be a lot of revenue from these products, and so they'll be self-funding in fairly short order?

Yeah. I mean, that's a much shorter answer than mine.

But you're saying people are underestimating—

The question was, are we going to be able to make money from this? The answer better be yes.

I think the history—yes.

The answer is yes.

But the history is very front-loaded expense, right?

Well, the history of AI as an industry, I think, to some degree, is that people have always been hand-wringing about how these businesses are going to make money. Every year, people are generally surprised at the upside.

That superintelligence is a useful, incredibly valuable—

Yeah, exactly. This is what—

This is kind of Meta's whole playbook, right? We build something, get it to a billion people, then scale the revenue while growing our revenue primarily in the meantime by continuing to scale the things that are already kind of at scale and growing.

Yeah.

I think this is also one of the dynamics. Dario's talked about this as well, but in general, because it requires so much advanced planning to build these models, if you look at the R&D expense at any given time, that is an overlapping R&D expense of multiple model generations at once.

Yes.

These require a long time to work on, and then ultimately you see immense payoff when you deliver them. This is what we saw with the Muse Spark family of models. Again, we started working on these a long time ago and are seeing incredible consumer adoption through the Muse product.

The other thing is that we use the models to accelerate our core business, right? You have this core business that's hundreds of billions of dollars and still growing extremely quickly. If you can grow it a few percent faster by using these models to accelerate these things, then that ends up funding a huge amount of this too as you're scaling it out.

The Meta core business is an eighth wonder of the world.

It's an extremely good business.

It really is quite incredible and is growing at astonishing rates.

Yeah, you guys really cooked with that one. I think you're also getting at something, Alex, which is that there's an interesting accounting artifact where, if you go build a data center, you have CapEx and you get to depreciate that over a useful life of many years. People debate what that useful life is, but the point is you do get to depreciate it, whereas a lot of the AI costs are also just in researcher salaries. Those salaries are being spent on model training that also has a useful life of many years, but you do not generally get to capitalize them.

That's also a financial advantage for us building out the data centers, as opposed to a bunch of the other companies that are paying cloud providers. They're paying the annual rate, but then at some point the GPUs will be fully depreciated, while we're probably still going to be using them for a while because the value of even an H100 at this point is higher than it was last year, right? That's just because compute continues to be constrained.

Any advice for us with OpenRouter? We've agreed to acquire them. It has not closed yet.

Part of what we're starting to see in terms of the models, especially the frontier models that are being developed, like Muse Wonder versus, let's say, a Fable or an Astra, is that we're starting to focus in quite different directions from the other models. I think there's a lot of intentionality that goes into the model development process that we've discussed as part of this conversation, and that is genuinely different from how other models are focusing.

When these models get collapsed to the same set of benchmarks, that elides all of the interesting texture that differs between the various models. One exciting opportunity for OpenRouter, generally speaking, is helping developers understand what these subtle differences are so that they can make the most informed decisions about which models they ultimately select.

You can imagine the flip side of that, which is that you have all these developers using these models. Those developers are great people to ask about the interesting textural differences between these models that are more than just tweet length. It's more than just, “This model's good, this model's bad.” Getting to the wine-sommelier version of understanding the models and their relative strengths and weaknesses would be quite exciting.

Okay, so based on your experience, the OpenRouter view is that there will be a diversity of models in the future, and they're optimized for different tasks and—

I agree with that.

Exactly. That's just your guys' experience based on—

Yeah, we are focusing on different things. We care a lot more about personal and professional agents, in my observation, than the other major labs, which is good. There should be differentiation in the economy. This is the point of capitalism to some extent.

Yeah.

So, you're going to say something?

No, I just really agree with that point. Part of what I love about you guys working on OpenRouter and being able to scale it is that the more infrastructure that can get developed to support a diversity of players in the ecosystem, the better overall. For the same reason, I'm quite a fan of some of NVIDIA's recent moves to support the open-source ecosystem. This just goes with our whole theory and philosophy about what a good future looks like.

We think that a good future is one where you basically get prosperity by empowering people. That's kind of the personal superintelligence thing that we're doing, but there's the whole business version.

Yeah.

For a lot of the important safety topics, our view is that the way to get there is by having diversity and checks and balances, with a lot of different things out there and accessible to people—not just a couple of good models that are only in a couple of companies' hands. From my perspective, that is probably a riskier configuration for the future than lots of people having access to different things.

Do you guys have a theory as to why that is? I feel like maybe people predicted more of a winner-takes-all dynamic in AI. One story that people told back in 2023 was that one AI lab would start to develop an advantage, and that would allow them to get recursive self-improvement going. Then they'd just pull further and further away from everyone else.

Whereas it has remained extremely competitive, with multiple people at the frontier. You guys with Wonder today are coming along with your frontier-class model. Open-source models have done an impressive job of keeping up, all told. We had this story a few years ago that someone was just going to run away with the prize. Why hasn't that happened?

I think there are a lot of underlying reasons and mechanics that drive this. One certainly is just execution. For any given frontier lab, there are periods of strong execution and weak execution.

And I think that is true across all the labs. So that sort of starts to average itself out over time. I think another—

Very diplomatic.

Sorry.

Yeah.

Is it that some of the players in the lead screwed up?

Well, I think if all players, when they were in the lead, executed perfectly, they would probably be able to stay in the lead. So I think that's one factor. I think the power and strength of distillation is certainly another factor that's benefiting the open-source ecosystem quite a bit. And I think this is just like a—

There's an anti-hegemonic force in the phenomenon of distilling a model.

Yeah. And this is just a quality of the technology. It's not like anyone planned for distillation to work super-duper well, but it just turns out that you can get a lot of capabilities naturally through some form of distillation.

Broadly speaking, distillation is quite a good force for the entire ecosystem because it means that you can pack very strong intelligence capabilities into smaller and smaller models, which enables us to serve them at much wider scale. And that's certainly one of the very exciting stories: really small models today are much more powerful than the biggest models a year ago or 2 years ago.

And then I think the last is that there's just sort of this—I actually think a lot of people in the ecosystem want this future where it is not just this definitely one final company in all of capitalism kind of end outcome. I think this is one of the reasons—

Not just individuals, but companies putting dollars against things.

Yeah. And I think this is actually one of the reasons why a lot of people were excited to join Meta and Meta Superintelligence Labs in the first place: they wanted to work at a place that is focused on giving this technology to billions of people. That is not the stated goal necessarily of all the other frontier labs.

So I think there's a lot of natural forces where many people in the industry are grappling with, “Okay, what is the actual world state that I want to live in in the future?” And there are lots of decisions that people make at all levels—in terms of where they work, who they do business with, or what business deals they do—that drive towards a more equalized outcome.

7. Meta Rebuilds Its AI Lab

You get into some of the MSL setup, which I also wanted to ask about. It just seems fundamentally very hard still to—again, we talked about what goes into developing a frontier model, but also from a standing start. When was the standing start for MSL?

Well, we launched Llama 4 on April 4, 2025, I think it was. And then I think it was April 20 or 21 when I started calling people for advice on what to do.

I remember I was in a staff meeting and I got the WhatsApp notification: “Have time for a call?” I was like, “Huh, interesting.”

Okay, so MSL is a little over 2 years old.

No, no, no, 1 year. 1 year.

Is it 2025? Oh, it's 2026. Oh, sorry, I'm living in the future.

Yeah, you are in the future.

MSL is a little over 1 year old. What are your reflections a little over a year in on what was load-bearing, as the AIs would say, in how it came together?

Well, I think that there was this whole thing early on: first, we had to pull together this core group. Nat and Alex and I were talking a bit up front. At first, we were brainstorming. I was talking to Nat about whether there was a way we could structure this so that it was a win for Scale's shareholders and we could get him over here.

Then we were talking about what the plan would be. Basically, I was in this mode where I was asking a lot of different people what we should do. Every time we talked, I'd come away with 3 more things: “Oh yeah, this is what we should do.” It was very high-signal, with a lot of strategic clarity.

We were brainstorming and working on this for a bit. Then Nat and Daniel joined, and we pulled together a few of the initial researchers. In the beginning, it was sort of like the Supernova thing, where we just jammed a bunch of people together. I don't think either of us slept that much for the first couple of months.

No.

I don't even—I kind of have this mental block almost where I cleared my calendar for a month or 2 and was just meeting researchers and talking to them. I remember, even before we announced the deal where you were going to come over, we were recruiting people and people were like, “What is going on? What is this?”

But then there was the whole question, which was a really hands-on data labeler.

But that was this whole question too: if we're not happy with the trajectory of the current program, who were the right set of people to lead the program going forward?

Alex and I had spent a bunch of time talking over the years. Maybe once every few months or something we'd talk, and I was always very impressed by how he thought about things strategically. I had this view of, “All right, currently he's doing this very operational data business.” But it always felt to me like he was the sharpest entrepreneur of his generation, of the ones who I had had a chance to meet.

It always struck me: this is a guy who is running this very grindy, operational business, but—

Lovely business still, by the way.

What's up? We're big investors. It's not a commentary on the difficulty of running the business.

When you run the 8th wonder of the world, Alex, a lot of businesses look like a grindy operational business.

It's just my view of things. It's grindy, grindy. No, no, no. I just had this view. Even though this was sort of—I think when we got started, a lot of people were like, “Okay, how is this going to work? The social media guy is working with the data guy.”

Are we going to build a frontier lab here or what?

But my really strong conviction, because I talked to all the leaders of all the labs, all the heads of research, all this stuff, was: “No, Alex is the guy. He's the fucking guy.” We're going to be able to pull together a team and recruit the people that we need to go do this.

We needed the strategic clarity that he has around the different ways that we want to approach this, the extreme focus on talent density, and the clarity of communicating the vision, which we fashioned together over this period.

There were a bunch of things that I think were extreme but have basically worked. Having this very flat culture was something that you were really obsessed with, right? You have almost 200 direct reports. That's—I mean, we have team leads and all this stuff.

My Muse helps me manage that.

Yeah, well, that's—there were these series of things that you articulated quite early, and I was like, “Okay, this is kind of extreme, but it seems directionally right, and we should go do this.”

I went and talked to other people who were more the people you'd conventionally think of, and it was very much what you'd expect a big company to operate like. I was like, “No, we want to do something that's—”

As you describe this, I'll note 4 things that I think even other opinionated, effective founder CEOs would not have done.

One, you just called the recent prior effort by a bunch of people at the company a failure and not good enough. You were more diplomatic than that, but the act of standing up off the trajectory—

Yeah, sometimes you have to reboot a team.

That's one. Thing 2 is that you are willing to pay ungodly amounts of money for effective talent. You're paying a lot of money to bring Alex and some other people, but a lot of money for Alex in particular.

Thing 3—

And 49% of this very good grindy business.

Exactly. But let's keep the story straight. So there's that.

Thing 3 is, I'll note, Alex had never trained a frontier model before. You were taking some amount of a bet on Alex's potential to do something where he had done stuff in AI, but no offense, he just had not done the specific thing that you were being hired to do.

And then fourth, you were saying you cleared your calendar and only did this for 2 months. I think a lot of CEOs would have had the business review, the board thing, the customer stuff, and things like that. Those 4 things seem pretty interesting to me, because I think a lot of CEOs would not have done them.

Yeah, that's interesting.

I definitely think—one thing you may not appreciate about yourself, but I think one thing that is quite special about Mark is the degree to which, once he concludes that something is the right thing or makes sense, he fully shifts everything to be consistent with that belief set. To your point, I think the inertia of most, even very successful founders who are opinionated and edgy or whatnot, would be, “Oh, I still have my whole calendar, so I'll just talk about doing this thing and hope that somebody does it.” Whereas I think when Mark concludes that something is the thing he has to do or makes a lot of sense, then he really does it. He has a very high learning rate in that sense. He's very willing to pivot entire things to be consistent with it.

I think one very underappreciated thing is the degree to which so much of the company had to pivot to support MSL as well. While it was a more startup-y effort within a larger company, the needs of MSL were still quite large. We needed a huge amount of help on infrastructure. We needed these large-scale data centers to be built. We needed lots of support from many other parts of the company. And especially as Muse has been successful, we've also needed lots of deep collaboration with the rest of the company.

This is one thing that I think is only possible because Mark is the founder and CEO, and he can ensure that we have a global optimization toward this sort of AI-maximalist version of Meta, versus the natural inertia of an organization toward local optimization ad infinitum.

Yeah.

And one thing that I think is quite important is that—

A lot of how we work together is that it's important to me to be in the details of this work, so that we can get everything around the company to go do what we need to do. I'll have opinions about this stuff, but I'm not the lead researcher.

Right.

I think that's a lot of how we work together well. We're quite in the details of all of this stuff, but running the lab is a big thing. Running the rest of the company in order to support this is a big effort. You almost can't really get this done without both of those pieces.

And one thing that I think was quite important about when we set the whole thing up is that Meta, the company, invested a huge amount to stand up MSL. Then we very clearly told everyone, “Hey, there's no deadlines. We are going to take things at the right pace,” which, on the one hand, could be absolutely terrifying. You've just stood up this extremely expensive unit, and you're telling everyone, “Hey—”

Yeah, and you're worried inside a larger company that maybe 7 years from now something will ship, right?

Yeah, totally. But I think that, first, one of the cool things about focusing on talent density and hiring all these very brilliant people who have done so much already is that everybody had a lot of motivation to ship. Everybody wanted to leave their mark, so to speak, on the field. Everybody was highly self-motivated to move as quickly as possible.

One of the things that's been funny, and that I've had to continue reminding the team of, is that it is a team of a lot of insecure overachievers. At any one point, if I ask them, “Hey, how good do you think—where do you think we'll be in 3 months?” I always get the most—

Everyone always says it's awful. It's awful.

We're screwed.

Yeah.

These very pessimistic answers. Then I record it, and 3 months later I'm like, “Hey, we just shipped this model. Let me show you what all of you said, where all of you said we would be 3 months ago.” And this is—

Has that caused them to get any more optimistic, or no?

No, they're still—

They're very self-critical. I think it's not that they're pessimistic about MSL. I think the average person seems quite optimistic about MSL and very self-critical about their own project, which I think is actually quite a healthy dynamic.

But the trend continues. I think Muse Wanderer is, in many ways, an incredible milestone for everyone because it is a—

The team was very, very critical of it.

Yeah.

They were working on it along the way.

When did you guys know that Muse Wanderer was good?

That's a good question. There are all these intermediate signals that you can get along the way that things are good or bad.

You're peeking in the oven, and it's like, “I don't know. It seems kind of gooey still.”

Yeah. Going back to it, there are all these intermediate signals that you can get along the way that things are good or bad.

And what are the intermediate signals?

In pre-training, you have a lot of perplexity-based evaluation. You have a different regime of evaluations that can help you in pre-training. Then you have the more traditional benchmarks that can help you toward reinforcement learning.

This model, as you mentioned, has been cooking for quite a while. One of the funniest things about AI research is that a lot of the time people are just sort of squinting at a bunch of dots and saying, “Yeah, looks like it's good,” or, “Looks a little bit off.” One of the funniest things is that people have protractors—analog protractors—that they put against a set of lines on their monitor to measure an angle.

He still hasn't come for this job yet. I think there's a huge amount of vibes driving whether something is good.

But when did we have this conversation? When did you know it was good?

We had this debate after it was done pre-training. The question was, how long will it take in reinforcement learning before it is better than Muse Spark? I remember the folks on the pre-training team were just trying to set expectations. They were like, “Look, sometimes this takes a while.” And then it was immediate. So we were like, “Okay, I think we're on a good path.” That, to me, was sort of—

Yeah.

I think that's when people got—

We've been excited. I think we've been pretty excited about it for a few months now.

But there's a lot to do there. Going back to the beginning, I think one of the things that was very interesting and challenging in setting the thing up was that you really wanted this very high-talent-density effort. We were pulling some of the best people from Meta in and recruiting a lot of the best people.

You were asking, “All right, for people who—if you were asking before, Alex hadn't necessarily trained models like this.” But to me, one of the big things was that all of the best people were very excited to work with him. I'm sure you see this in a lot of the things that you do, which is that most people are not one-dimensional. Maybe you do a specific thing, but your ability to grow and expand is largely based on how effectively people who are excellent in different domains want to come work with you on different things.

Yeah, and the great people have followership.

Yeah.

I think there was a little bit of that, but it's also about the people who don't know you yet or don't know what you're about. Are you an impressive and compelling person that people want to come work with? Do people feel comfortable that you're going to create this environment?

I think you were very thoughtful about this early on. Some of the things, I think, you had fully formed as beliefs about how you thought we should do this. But I also remember when we were having the initial recruiting conversations, we were shaping it in real time. It was like, “Okay, this construct that we're talking about with this person isn't quite going to work, but they seem good, and they have this thought.” We were really coalescing this vision about what the shape of it should be in real time.

I'm sure you remember this. Some of the initial concepts—I think probably the most extreme thing was the concept of talent density and all that.

But I think we weren't sure upfront that it was going to have to be a full reboot, right? That basically started when we were out talking to a lot of the best people.

And we realized that in order to make this as attractive as possible, it should feel like we were starting this thing again from scratch. We would pull in a lot of the great people from the Llama effort and FAIR and all of that too, but that was a very intentional thing that we figured out as we were talking to the initial set of people about what was going to make this compelling.

It ended up being really smart and made a huge difference in the culture of the lab and how it got started. Being able to be that dynamic about figuring out what shape this needed to take—I think that’s a very uncommon trait. It’s a very entrepreneurial trait, right?

So, when I was thinking about what’s going to be important for someone who’s going to run a lab well, it’s not whether they’re an expert in whatever the latest modality or training technique is for a lab. It’s whether the person is a clear thinker, able to dynamically update and move as an entrepreneur would. I think this is just an inherently very entrepreneurial space.

Has this experience changed your view on the value of going external? Both your reboot and a bunch of the people you brought in were external, like Alex, Nat, Daniel, and others. But you’re also describing deciding whether you needed a reboot based on going to the outside intelligence of the aggregate views of researchers in the industry and getting a sense for that.

Do you feel like you’ll carry forward that approach for future unrelated projects? Because obviously the risk of a large company is that everyone says, “We’ve got a great approach. Let me show you our 100-page document on it.”

Obviously, you can’t do too many of those at the same time. A lot of what we had to do here was rotate a lot of the rest of the company. The infrastructure team now has to build these huge clusters to support this, which is obviously a huge thing. Huge parts of the company had to pivot to do this, and only one thing can be the top priority.

Yes.

But on balance, it has made me somewhat more optimistic. We brought in this guy Kunal Shah, who I think is one of the top entrepreneurs in India, building this great company, CRED. Now he’s running WhatsApp, which I always thought should do more in commerce. We were underserved there, and India is the biggest country that we’re in.

That’s a bit of a different situation, because we’re not pivoting the whole company toward WhatsApp. We just wanted to bring more entrepreneurial energy there. This was a level 10 out of 10 difficulty thing. I don’t want to talk about it in the past tense, as if it’s done, because it’s still a level 10 out of 10 difficulty thing.

Yeah, yeah, sure.

The number of pieces that had to work was quite all-encompassing. I think it’s both difficult for the people doing it, but I also think that puts some level of strain on the rest of the company and on the financial model of the company. I wouldn’t want to conflate those two pieces.

It’s good to have new blood, and it’s good to have people who understand the context. You were talking about people who’ve been around Meta for a while and understand the ups and downs. If you’ve worked at Meta for 10 years, you’ve seen some shit. We’ve gone through a lot of hard times, and I think that’s important context too.

A lot of this is just about trying to build a team. Alex had a theory for what the team shape needed to be, and I guess I had a theory for what the shape of the leaders needed to be. A year-plus in, it seems okay. First of all, it was just good to try something different, and different from what everyone else was doing.

Mm-hmm.

Yeah, I mean, that’s—

I think there are some other dimensions here that are worth mentioning too. When deciding that we had to do the full reboot, a lot of it was based on external input, but there was another important input.

One of the first meetings I had when I joined was with some of the leads of the existing effort. I said, “Okay, I think the first thing we need to do is train a really good reasoning model by some timeframe. I don’t know when, but this is clearly the most important milestone.”

One of the key leads in the group, who no longer works at Meta, said, “We don’t need a reasoning model. Do any of our consumers need a reasoning model? I feel like we don’t need to do a reasoning model. We could just do a normal model.” I had this heart-sinking moment.

Oh no, I’ve joined the wrong company.

I was like, “Oh, I’ve got to take 10 steps back.”

I remember calling Nat after that meeting. I said, “Nat, this is what they just said in the meeting.” I think it was specifically an indicator of something that Mark started to realize when he talked to people too: we needed leadership in the company to be much more AI-pilled and much more forward-leaning on where the technology was moving, how quickly it was moving, where it was going to go, and what the possibilities were.

As Mark started pivoting and shifting the whole company in this direction, it became very clear that this was where the company was going. Then a lot of the existing leaders very quickly picked it up and changed. Adam Mosseri, for example, who runs Instagram, has one of the most aesthetic but also effective personal vibe-coding and personal mini-app setups that I’ve seen.

I think he’s gone quite far in that direction. A lot of the leaders at Meta who have been there a long time are all very adaptive. They’ve been through a highly turbulent period of technology. Now AI is maybe 5 or 10 times more turbulent and faster-moving than those past waves, but they’ve adapted to new trends many times.

I’ve been very impressed by seeing how quickly a lot of Meta’s leadership has changed. If I compare the conversations I would have with them when I first joined to the conversations I have now—

They’re now super AI-pilled.

Yeah. It’s very impressive for that to happen at the scale of Meta, which is one of the largest companies in the world.

Yeah, we got them there pretty quickly. If you start coding, then it’s helpful. But a big part of the culture that we were trying to create is that when you have all these direct reports, the point isn’t that you have the direct reports; it’s the absence of non-technical managers.

I don’t think you would say that you’re providing a ton of managerial support to 200 people at once. That’s not a feature of the thing. But it’s well known among researchers that there’s this common pattern: you go from being very technical to a role where you’re no longer in the details, and then you very quickly become non-technical or stop understanding it much faster than you probably realize. It’s quite dangerous.

Alex is well known for career chats and emotional “How are you doing?” check-ins.

This whole thing of having to stay technical, having people use the agents, and having people write code has been a really important part of this whole thing.

Have you had any very funny disagreements between the two of you?

We have a lot of disagreements.

Yeah, we were working on naming up until very recently.

Naming is a good one.

It’s one of the things I appreciate about Alex. It goes back to your philosophy about recruiting back at Scale: hire people who really care. I think there are some expletives in it, but it’s fine.

But Alex really cares, it turns out.

I think that’s really important. When you have this supernova of talented but very strong-willed people together, you’re going to have fun fights and not-fun fights. To some degree, I’d rather have people push for things they believe in and argue over things than have people blindly execute something.

It's like people need to feel ownership. So yeah.

How is Mark a different boss than you expected?

I don't have a huge comparison set.

I think I can leave now.

Yeah, exactly.

No, I think that the— I think it has been very— one of the very cool things about working with Mark over the past year has just been watching the true degree of his boldness and this trait that we were talking about before: once he internalizes certain beliefs, the speed and magnitude with which he's willing to change prior actions and pivot towards the worldview that is consistent with those beliefs. I think these are pretty unusual traits, generally speaking, even among this base of founders.

Mark actually ingests new information and fully updates his worldview and then acts in accordance with the newly installed worldview.

Very Bayesian.

And then willing to take— Very Bayesian. Very Bayesian. I've never gotten such a nice compliment.

What lovely priors you have.

Yeah.

Posterior.

You're not allowed to talk about posteriors anymore after the whole incident.

Yeah. No, but then I think also just the degree of boldness. I think this was actually quite puzzling to me at first, the degree to which you were willing to make these massive bets. It was something Nat and I would often talk about. We were like, “Wow, we're really going big here.” And both of us would be like, “Would you go this big?”

Would you go this big?

And so I think that is quite a notable trait and something I really admire. If I look back at the history of Meta and imagine what the scenario would've been in all these pivotal moments, I can see how this is kind of the movie, so to speak.

Yeah.

Or this is kind of how a lot of those situations go. There clearly are moments where something very, very big is happening, and it may not even really be showing in the metrics. It may not be impacting user counts or revenue or whatever it might be. It's not really impacting things. But clearly, if you believe that thing is going to happen, then it has massive implications for the company and the product.

I think my post hoc retelling of Meta's history is that Meta is a company where every time that happens, Meta takes it extremely, extremely seriously. That has enabled the company to innovate and continue evolving with a very fast-moving and fast-changing tech landscape. Mark is very Bayesian. Yeah, very Bayesian.

So Bayesian. Life is short, just do the right thing.

Yeah. And I think AI is maybe that times 100 or whatnot. A lot of CEOs would've told you last summer, “Oh yeah, I think AI is the most important thing for my business.” Maybe every CEO would say that. But then you look at what they're actually doing and how consistent their actions are with that belief. I think this is quite a special thing about Meta and Mark.

Last question, because we're past all of our bedtimes.

I'm so flattered.

Yeah.

What do you make of all of Alex's deranged posting on Twitter?

I think most of it is great, but I guess my main view—

It doesn't matter how much you fail; it matters how big you win when you win.

With communication, I think it's most important to just be authentic. When you're trying to be authentic, some of the things don't come out the way you want. I think it is better to make the trade in that direction than it is to be overly scripted and not be authentic.

Most of it, I just feel, is hilarious. So yeah, what do you think about my deranged tweets? I'm not a heavy X user, but—

But you are back on X now.

I'm back to posting. A bunch of the researchers are there and we want to be over there, but everything I post there, I also post on Threads, and it's all good. Every time I load X, I have four Alex posts and it just cracks me up. It's like some of these— it's like, where do you— we're just talking about this.

Well, you're now a clearinghouse, I presume, where people send you good stuff.

I was like, “Where do you come up with this stuff?” It's like, yeah, this is—

Yeah. How does it work?

Most of the— we were talking about this with my partner yesterday, and he was— I think one observation was just, there was actually probably still, to this moment, my thought pattern oftentimes is, “Oh, that's funny. I wonder what Jolly would think about that.” And then you get good memes out of that.

That being said, I have experienced the brutal metabolism of the internet and how quickly it metabolizes memes. You really have to stay on the cutting edge of memeing to—

It's easy to fall behind.

Yeah.

Which is maybe a good summary of this conversation. It's easy to fall behind. You gotta stay at the frontier.

Stay in the details. Yeah, stay technical and stay at the frontier. Yeah. Yeah.

Alrighty, we'll leave it there.

All right.

Awesome.

Thanks, guys. Thank you.

Okay, wait, so what is everyone drinking?

Well, that.

What is this?

That's a keg of Guinness that I brought you.

This is Guinness from?

Dublin.

Oh, wow.

You were just there?

Yeah, I was just there this week.

Oh, awesome.

Yeah.

Well, thank you. We got the local stuff.

Yeah, there you go. Yeah, no, I figured this would be a good occasion to get the Guinness going. Yeah.