Mark Zuckerberg on how Meta's Muse AI agent will make people money

Mark, last time we spoke, you called me. It was on the weekend. Well, you called me through the glasses, which was the interesting part.

Was it loud in the background?

You were going to fish. I wasn't going to say it, but, yeah, it was on the weekend.

The noise cancellation is actually pretty good.

Oh, it's great.

Yeah, it's great.

Yeah. You wanted to talk about this essay manifesto. I don't know what you call it—manifesto, we'll say—that you published recently, and it's long. There's a lot in it. I wanted to start there because there are a lot of big ideas in there, and they'll connect to the main thing we're talking about today.

Yeah.

I'm curious why you wrote that, because it's long and there's a lot in there. [Laughter]

Well, I feel like if you're going to invest so much in building AI, then it's important that people understand what your lab stands for and what your values are. AI has so many opportunities, but there are also all these real risks. I think it's very important that everyone who's working on it has a well-thought-out theory for how the work they're going to do is going to lead to a positive future.

It's interesting because the different labs have some different philosophies on this, and there are a lot of things that have become conventional wisdom in the industry that I strongly disagree with. My view is that the path to a positive future for everyone is to make sure that we distribute the technology as widely as possible.

That's based on 3 major principles that we have. One is that empowering people is the source of prosperity in the world, and it has been throughout history. Two is that the primary purpose for AI is going to be the invention of new things, not automation. The third principle is that the foundation for safety in the future is basically establishing the right checks and balances and balance of power, rather than restricting access.

I think these are all things that are oddly very different from a lot of the conventional wisdom, especially in Silicon Valley. A lot of people think, “Hey, this technology is very powerful. We must restrict it so that not that many people have access to it.” I personally am much more worried about a small number of labs or people having control of something that is so capable.

Throughout history, what we've found is that when you put power in people's hands, most advances don't come from the incumbents or the establishment. They come from people on the periphery whose ideas aren't taken seriously. But when they get enough tools to be able to prove out what they're working on, that ends up being very powerful.

In Western society, the way that we've established governance and basically created a well-balanced society is through a set of checks and balances and this balance of power. It's very ingrained in our society that you don't want to have one lab or 2 labs having access to something.

For some of the most recent concerns that have come up, like some of these cybersecurity concerns, I think the best antidote to someone having an AI that could potentially hack into systems is having everyone have access to an AI so they can harden their own systems first.

That's been the history of cybersecurity over the past several decades. Open-source software, because people can see it and scrutinize it, sort of counterintuitively ends up creating a more secure and stable environment by putting it in people's hands.

That's what I believe, and that's what I think is the path to a positive future. It has a bunch of different implications for what we're going to do. Obviously, we want to build leading AI models, which we're doing, and Muse Spark 1.3, which we just released, is advanced. It's actually the latest model of a relatively smaller pre-training that we did, with the internal code name Avocado.

We're going to get into the code names.

Oh, yeah, we'll get into that. We have Watermelon coming soon, so that's going to be a big deal.

Obviously, leading models are probably the biggest personification, if you will, or implementation of this vision. The Muse personal agent that we're rolling out is basically about giving every person in the world a very capable personal agent that can understand their goals and work on their behalf 24/7.

1. Data Centers, Jobs and Local Communities

An important part of this is also just getting the technology into people's hands. We're very strong proponents of open source and making sure that the opportunities that I think are going to be massive here aren't limited to a few people or companies.

I have a bunch of questions about the Muse agent, but staying big-picture for a second, because you said a lot of things there: Is open source the counter to the trend you're seeing and that you're worried about? Is that the main practical way that you counter that, or is it regulation? Is it both?

No, I actually think probably the most important thing is just getting the technology into individuals' hands. I actually think things like the Muse personal agent are perhaps even more important.

I think what you're starting to see is that some of the labs are training more advanced models and then not even releasing them.

Right.

I think that is quite dangerous. When you have scrutiny on something, when you put a system out there, first of all, if you put it in a lot of people's hands, you get the checks and balances and broad-based prosperity, which I think is important for society.

We can't just have 1 or 2 labs get incredibly valuable. You want to make it so that billions of people can basically have prosperity in their own lives, whether it's creating small businesses, being more successful in their careers, being more productive and managing their homes, saving money in a lot of ways, or advancing their health.

You want the benefits to be very broad-based. That's 1 piece. In terms of the competition, I do think that having multiple labs is helpful, and I think open source is quite helpful for that.

Open source is an important part of it. The nature of open source is that there's a whole community of people who do it, so I'm not saying that we're going to be the 1 company that does it. I'm also not a zealot about this from the perspective that everything we do is open source either.

We release some open models and do some closed work. I think it's important, if you're building a for-profit company, that you can build some advanced things and you don't necessarily need to share every single thing with the world.

But I think, in general, supporting a robust open-source ecosystem is going to be key to maintaining competition and maintaining transparency and understandability of where the technology is going, in a way that I think is actually going to be incredibly important for safety.

If we have a world where there's just a small number of really capable models—it's interesting, right? If you look at some of the cyber stuff, for example, I can understand the instinct of, “All right, we have this capable cyber model. Let's release it to only the top 100 institutions.” But I think part of the issue is that there are more than 100 important institutions in the world.

If you look at something like the Hugging Face incident that happened, Hugging Face is maybe not 1 of the biggest 100 institutions in the world, but it matters. It's an important thing that people rely on.

When they started detecting that there was this intrusion, they turned to open-source models because they didn't have access to some of the closed ones that were causing the issues.

I think having a robust open-source ecosystem is 1 important part of having a safe and stable future. But to me, the most important thing is just making sure that we distribute the technology widely rather than hoarding it in a small number of people's hands.

There's also this ongoing debate in Silicon Valley about why people feel so negatively about AI. You talk about this in your recent letter, addressing these concerns or trying to. The sentiment on AI and data centers in particular is so negative.

It sounds like maybe an essence of your argument—correct me if I'm wrong—is that if we diffuse this technology more, if we enable more people to access the things that are right now gated by some of the top labs, maybe that addresses the feeling that people are disenfranchised by what's happening in AI.

Is that what you're getting at?

Well, there are many layers to it. There are so many parts to this, which is why the essay was so long—15 pages. We want to get through all the different questions.

People have questions about jobs and the economy. They have questions about data centers and their local communities and the economic and environmental impacts of that. There are questions about how people might misuse AI. There are cyber questions and bio risks that are coming up. There are questions about how we maintain a free society, questions about American leadership, and questions about maintaining control over the technology as it gets to be increasingly capable.

These are all important. It's important not to just talk about this in generalities at a high level, because each of these has different nuances. But in general, I think one of the things they all have in common is that if you create broad-based prosperity, one of the better ways to do that is by ensuring there's the right balance of power around who has access to the technology and generally making sure that the greatest balance goes toward the general population, as opposed to any insider stakeholder or whatever you want to call it. I think that ends up being very important.

If you look at the data centers, what we've actually found is that when a company like Meta goes into a community and makes a commitment that we're going to invest there for decades—which is really what we're doing when we're building up a data center—we're able to make it very good for the community. The tax revenue that they bring from that can have a real impact.

In Louisiana, we had this example where the tax revenue funded $50,000 bonuses for teachers in the community. We bring a lot of jobs, and we invest in the local community a lot. I think that can be good.

I think there's also a lot of speculation, where there are companies that aren't necessarily planning on running a data center for decades. They're just trying to find a plot and then sell it to one of the big labs, and they don't really care as much about the local community. They're not invested for the long term. If they don't care to focus on making it work for the local community, then of course people are going to get upset. That's one of the things that can be difficult when you have this kind of speculation.

I don't even know if I'd call it a bubble, because that implies that it's overvalued or something, but there's certainly a boom. That leads to some of these short-term-thinking incentives that don't necessarily lead toward helping every stakeholder, which I think is what you need to do to make this sustainable over the long term.

At the end of the day, if we're creating a technology that doesn't create jobs or broad-based prosperity, or if the infrastructure we're building doesn't help local communities, that's not going to be allowed to continue. You have to design it in a way that can be helpful to people in all these ways. That's also part of the reason why, for people who are skeptical or have so much doom about the whole thing, one of the things that I basically think is that if we don't end up building it in a way that's positive, it just effectively won't be able to happen.

I think establishing the right checks and balances and distributing the benefits of this widely is a precondition for being able to scale in the way that would be best for society over time.

I haven't heard another tech leader in your position talk about data centers that way—the long-term investment of it. Is this how you've always thought about it? Is this something you feel like there's more clarity that's been brought to recently?

I think there's been all this anti-data-center sentiment that you're talking about. We've dug into it, because what we're trying to understand is: There isn't as much of that around our project, so why is that? We asked a bunch of people, and it looks like there's a pretty big dichotomy between these speculators and the companies that are focused on it for the long term. That makes sense when you think about it.

One thing that we're doing is this America's Workforce Academy project that we did. We're going to build all these data centers, and we're going to be doing this for a while. There isn't the volume of skilled-tradespeople that you need to create this. We need more fiber technicians, electricians, people with advanced carpentry skills, and all of these things. There aren't enough people to do this. We need hundreds of thousands, maybe millions, more people who can do this, and people aren't trained to do that.

So we created this training program to effectively do that, where we guarantee people who get through the training program a job at a place that's working on building infrastructure for Meta.

Why did we do this? It's not really philanthropy. We need those people to be skilled and have those skills. It's a win-win. It's an investment that makes sense if you're in it for decades, but not necessarily something that you would do if you were building out one site with the intent of flipping it to a different company.

A lot of problems in the world do naturally get solved by incentive alignment when you think about them over the long term. I think that ends up being an important part of this. Part of the way that I think about this is that there's no way it's going to be permitted for there to be a small number of labs that control such an important and capable technology and accumulate a lot of wealth to themselves.

This has to be a broad-based thing in order for it to work. It has to work technologically, but it also has to work socially. Those pieces have to go hand in hand.

2. Why Meta Is Building Personal Superintelligence

I agree. Well, now we're landing toward Muse. Before we get there, you also wrote, a year ago, your “Personal Superintelligence” essay—the shorter one.

Yeah, that was one page. I did write a one-page version of the future, too. I published it in The Wall Street Journal.

I just read the long one.

Yeah. So there's the one-page version and the 15-page version.

3. Smart Glasses and Privacy

This one you did about a year ago, “Personal Superintelligence”—I think a lot of people in my world, when they saw you write that, were like, “Oh, wow. Why is Mark writing this?” What is the thing he's seeing on the other side of this? I think it might be Muse—correct me if I'm wrong—like what we're going to talk about, right? What you guys are releasing now.

This is it.

How did you come to that realization that this is the next chapter for Meta?

It's interesting. We've never really just thought about ourselves as a social media company. We've definitely thought about ourselves as a company about connecting people and empowering people. A lot of the values that led us to build the things that we built for the first 15 to 20 years of the company were around putting technology and power in individuals' hands, believing that people should be able to decide for themselves what is important in their lives.

We've gone through a lot of social debates around this. A lot of the debates around content moderation and things like this have been around the question of whether people should be allowed to decide and communicate for themselves what matters in their own lives. Through that experience, it has sharpened my belief that a lot of progress throughout history and through this technological age comes from empowering individuals, and that people really do know best about what matters in their own lives.

Because of that, I have somewhat of an allergy whenever I hear people talk about, “We should just have a small number of experts allocate what AI does to big problems.” Why should it do these things that people care about in their lives? People have a balance of things they care about. People care about health and having a better life, but they also care about their relationships, showing up for their friends and family, and culture.

People care about things that may not, to a scientist or an engineer in the industry, feel like the biggest problems. But if you ask billions of people what they care about, I think their aggregate answers to that question are what the most important things are to work on.

I've always believed that when you build this superintelligence, there's a question of who decides what it's going to focus on. I think people should be able to direct it toward what matters in their own lives. It shouldn't just be directed by so-called experts sitting at a small number of labs.

This gets back to the overall philosophy. The way to have a positive future is to empower people, put the technology in their hands, and let people decide for themselves what matters and how they want to use it.

And I think that when people do that, it first of all will prioritize some things that are different, right? Maybe it'll prioritize health issues, but instead of prioritizing the most common things—which is kind of what the pharma and biotech industry at large prioritizes today—there's a very long tail of rare diseases and conditions that people have. If you have a rare condition, you're probably going to want your personal AI to focus on that, not just something else because it happens to be the most common thing.

I think, for example, rare diseases are disproportionately underinvested in.

You're doing a lot of investment with your foundation.

We're doing that at Biohub, and that's partially informed some of my views here. You want to put the power in individuals' hands to determine what matters for them.

4. How Zuckerberg Uses Muse

A lot of this also isn't necessarily about the things that people would say are the big social problems. For me, when I'm using my Muse Agent, I want it to help me be a better father and a better husband, show up better for my friends, and help me connect with people. I think that's also partially a through line between the work that we've done at Meta so far. We're the company that disproportionately cares about and believes that there's social value in helping people connect with the people around them.

What are the first things that I set up my own agent to do? My 3-year-old daughter likes baking. I don't know anything about baking, but that's a fun project that we can do. So I asked it, “Set things up so that every weekend we have a baking project that's reasonable for a 3-year-old and an adult who knows nothing about baking. Use Instacart or whatever to get all the ingredients. Figure out what makes sense and make sure everything is ready, so when I show up on Sunday with my daughter, we can make this thing.”

Then I tell it afterward, “How did it go?” And it says, “Okay, that one was too hard.” It turns out cake pops are really difficult—surprisingly difficult.

Nothing about baking.

Yeah, I didn't either. I know something. Cake. No, don't start with cake pops. That's the problem.

There are a lot of things in baking that are pretty simple.

It turns out cake pops are not one of them. But, well, what can I tell you?

Thank you, Muse.

Yeah, thanks. It kind of updates that and helps with it.

My older daughter has gotten into climbing mountains, and some of them require permits. So I have it sit and get the permits when they become available so we can climb mountains. I was like, “That's pretty neat.” Then it tells me, “All right, I was able to get a permit for this day.” I was like, “All right, well, I guess I'm taking that day off from work to go climb a mountain with my daughter.” So it's kind of cool, right?

It does that, but it also helps keep me healthy. It helps me with my training. I put cameras up in my MMA gym and tell it to watch the cameras and send me feedback. It's pretty fun. It's good feedback.

Sometimes it's funny feedback. It finds me and says, “It looks like you really gave up.” And I was like, “Yeah, I did. I was really tired right there.” Why is that the thing that you're pointing out to me? But no, it's good. The coaches laugh about it. They're like, “Yeah, this is what we didn't feel like we could tell you, but your agent's telling you.”

Yeah, that's good.

I didn't think about this until hearing you talk about it, but you have people who could obviously do all this for you. How do you use something like a Muse Agent to really test the limits of how it can be helpful as an assistant? Are you pushing it in ways that make the team say, “Okay, we've got to fix this”?

Part of what's interesting about it is that everyone has such different things that they want to do with it. In the early beta period, we handed it to a bunch of people. I gave it to someone, and within a day they were using it to help run their homeschool. I was like, “Okay, wow, you just started this within a day.” Then another person, within 12 hours, said, “I just planned a trip.” It had planned this whole thing for them.

Someone else I know who's generally pretty skeptical about technology—I gave it to her, and she didn't say anything for a few days. Then she texted me, “When you do the general release, do I get to keep my Muse Agent, or are you going to reset it?” I was like, “All right, this is good. I think this is working well.”

Everyone I know who's been on the beta has very high praise for it. People do different things with it.

I think we should also say more plainly what it is so people understand. People think of AI as Meta AI or ChatGPT—back-and-forth prompting. The real unlock here, and this is happening in the industry more broadly, whether it's Grockbot town instinct—I mean, there's many products doing this—but it's adding a virtual machine behind the scenes where the agent can control a computer for you, log in, and do things. That's a huge change for people who only know AI for—

It's long-lived. Instead of a model like Meta AI, ChatGPT, Gemini, or whatever you use, where you send one prompt and it gives an answer, you give it projects or goals, and then it just works. It works 24/7 and doesn't stop until it's helped achieve the goals.

The team was telling me it studies overnight, which is what you guys call it.

Yeah, it studies. It kind of consolidates its reflections into memory. It basically just works on projects, and it can also suggest new projects.

I play the computer game Civilization with one of my daughters, and I asked, “Do you want to make a strategy guide for her?” It said, “Yeah, sure.” Then it asked, “Now that we have the strategy guide, do you want me to expand it so it can also teach historical lessons about different civilizations?” I said, “Yeah, sure. Why not?” It built a new tab in the app that it made, and that was very cool. It can just expand.

It's proactive.

5. Muse Pricing and Business Model

Yeah, it suggests things. One of the things I think is interesting is that it's going to be able to make people money and save people money.

You think?

Yeah. Part of what's interesting here is the economic model for how we're pricing it. You can pay for a subscription if you want that model, but we're also making it so you can get a very large amount of usage for free. I think, to start, we're offering 100 million tokens a week for free, and you get this virtual machine. So it's a lot of computer.

That's a good meme. It's a lot of computer.

It's a lot of computer. The reason we're doing this is that we're confident in standing behind the fact that, for people who are going to use it to run a small business, make money, or conduct transactions or commerce in some way, we think it's going to make so much money for people that the business model over time will effectively be to take a very small cut of whatever the transaction is.

Take rate: small.

Yeah, and not necessarily from the person paying for it. It'll come from the businesses that they're working with.

And you're working with Stripe on payments.

Yeah. My view is that we should be able to make a service free for the vast majority of people. That's critical if you want to build a future for everyone where everyone has these powerful superintelligence agents.

An important part of making something available to everyone is making it affordable. We want to make this free, so there's a huge amount of usage that you get. We're basically standing behind the idea that this thing is going to make you money and save you money, and that's how it's going to pay for itself.

And Meta services can connect into it, right? So you could theoretically manage your ad spend on Instagram and all that stuff.

Well, you can connect it to whatever you want. It does work with Meta services if you want, but you obviously don't have to connect it if you don't want to. If you're using it to run a business, it can basically connect to our ad systems, and you can ask it to make something for you. It can help you make the product, and then it can help run the business.

It can do all this in a loop and just do it forever, 24/7. Every time we release a new model—which we've been on this cadence of shipping, a meaningful update every month—it's just going to get smarter, right? It'll get more capable and able to do more and more stuff.

Something your team was telling me that I haven't heard this approach used elsewhere is this fleet concept, where you're letting the fleet of Muse agents learn together.

That was the ideas and suggestions thing. You open up the app, and the main tab is basically your chat with your Muse. There's a tab for ideas from the things that you've told it and how it can expand those.

That's the thing I was saying: first, it helped make the strategy guide for playing Civilization with my daughter. Then it helped expand that into historical lessons. It came up with that idea, and then I was just like, “Yeah, sure, do it,” right?

It finds all these ways to augment itself. The MMA coaching thing comes up with ideas for how to make it better. It's like, “Would you like me to get better at finding the right frame to send to you?” It's like, “Yeah, good. Go do that.”

The ideas thing is important because, across the fleet, you can find people who are interested in different things. Taking a step back, one of the big issues that exists with AI is that a lot of people don't know what to do with it. If the agent can itself suggest things that it can do to be helpful for you, then that solves a huge part of the problem of making it so that you can get the most out of it.

When you're introducing network-effect learning for agents, which no one's really done, the agents are learning anonymized insights from the rest of the fleet. You're like the king of network effects. I'm really interested in this idea because I don't think anyone's doing this.

Yeah. Right now, I think most of the industry is thinking about agents as a single-player game, right? You have your agent and you use it. There are going to be all these interesting things that you can do by having the agents interact with each other.

We already have all these interesting examples internally where people have their agents interacting with each other. This isn't, for the most part, rolling out in this release, but it's going to be an important part of how I think this works over time. As more of the people you know start using Muse, it just gets better, right?

Is that the differentiator? You could argue that AI models continue to commodify at the frontier, essentially, or that the products all start to look similar, with similar kinds of harnesses. Is the network effect of that learning the real edge?

I think there are a few things that are unique about what we're doing. One is that we're designing the models from the ground up to be good for this use case, which I think really matters.

Two is that we have this social DNA as a company. We're helping people use AI and agents to enhance their relationships, strengthen their relationships, and get more out of the soft but very important parts of their lives. I think that's something that we're probably just going to be more attentive to as a company than any of the other labs.

6. Privacy and Security for Personal Agents

The third thing that I would say is going to be a major differentiator for us, which I think might be surprising to some people, is privacy and security. We're investing in this a huge amount. Part of the view that we have on this is that, in order for this to be useful, it needs to have not just state-of-the-art intelligence; it needs to really understand you, right?

In order to understand your goals, you end up connecting it to all this stuff. You talked about connecting it to your ad system, but people connect it to messaging and email and all this stuff—health information, whatever. In order to do that, people need to have a very high degree of confidence in the system.

The good news here is that Meta has spent more than 10 years focusing on building WhatsApp into, I think, the largest global end-to-end encrypted system. We've designed it in a way where even Meta can't see the messages that people send, and that's been a really transformative thing.

I think it makes it so that people trust WhatsApp. It's also been a very important lesson for Meta to learn. That has been really important to our success with WhatsApp: we've designed the systems so that even we can't see the content.

Whatever people are worried about—whether they're worried about the government getting access to it, a hacker getting access to it, or someone at Meta doing something bad with it that they don't want—all that stuff can be taken off the table if you design the system so that you can't see it.

We took that as one of the foundational lessons when we were getting started with this. Nat and I personally recruited Moxie Marlinspike.

Founder of Signal.

Yeah, and one of the people who helped us build WhatsApp encryption back in 2014. He joined to specifically work on this confidential VM project, which makes it so that you can have your virtual machine and have all this information in your Muse, and we can make the commitment that even Meta cannot see the content that's in there.

You can do this technically. It's an incredible kind of commitment that can be technically verified. And that's something that we're going to publish more about in the coming weeks as we get closer to rolling this out a lot more widely.

And out of all these early VM efforts that these labs are doing with these agents, do you think this is unique—what you're doing?

I don't think anyone is doing it.

I don't think anyone's doing it.

I mean, there's a lot of other security measures that we're putting in place that we should talk through. Even before this is ready, there's that. And even for people who don't want to use this, it's incredibly secure because we focused on this from the beginning.

There's also auto-approve. You have to see what it's doing.

Let's get into all that stuff in a second, but—

I'm not aware of anyone having anything close to the confidential VM system that Muse has. I think it's a very fundamental thing because you want to know that this is your agent and that if you put content in there, you can trust that no one else is going to get access to it.

So what are the two ways to do it? Well, a lot of people earlier in the year, when something like OpenClaw came out, started getting Mac Studios. One way to feel good about it is that you literally have your device running in your home.

But the other way to do it is tricky, because I don't think there are going to be billions of people who are going to buy a Mac Studio, configure it, and run it in their home.

Especially with RAM prices right now.

Yeah. But it's also technically difficult. Part of what we were trying to do with Muse was build a version of that personal-agent experience that just works, that I can give to everyone in my family, with various levels of technical literacy.

It just works, and within a day it's doing all the stuff that they want in their lives. Part of that is that you don't want someone to have to set up their own computer or VM. You just want to be able to provision it in the cloud, but you want it to have the security and confidentiality that you'd have if you had the box sitting under your desk in your house.

I think that's a very fundamental thing. There are other pieces, too, because not everyone is going to use that. We built a secure credential store, right? There's no reason for you to store out in the open your credit card and password, one-time card numbers.

Your agent shouldn't know that stuff. It should just be able to access it when it needs to, because you've asked it to log into a thing, and not otherwise.

It's actually not a single thing. You have your core agent, but we also built all these sentinel agents that monitor the incoming and outgoing traffic and data that your agent is sending, for the purpose of flagging to you when you might want to review something.

So that's all that the Sentinels do, effectively—

They look at whether someone's trying to do a prompt injection. They look at whether your Muse agent—

—shared something that is going out that you might not be comfortable with?

If so, then the Sentinel agent is empowered to—

—trigger this human-in-the-loop review.

If you're going to log into something, make a payment, or transfer sensitive information, you need to approve it each time. You can tell it, “I'm good with stuff like this in general. Always allow this kind of thing.” But in general, the Muse agent can't make those judgments itself. It's built into the system and the architecture in a pretty deep way.

Even when we do things like connect all the connectors to your email, some people, when they've designed this, just make it so that once you connect, you have access to everything. But the approach that we've taken is, all right, if you connect to your email, it should start read-only.

Right. And then if you want it to be able to send an email, fine—go ask it for that specifically, right? Especially because most people trying this have probably never tried a product like this, so it's—

Yeah. This is a core design principle for us: least privilege. You're going to ask it to do a lot of things, and at each step along the way, it gets access to the least privilege that you need, and only adds to that as necessary. This is very fundamental in the design of the product.

If you look at all the other agents that are out there, I think no one else is anywhere close to the level of sophistication or depth that we've built into this. Again, it's informed by our experience building WhatsApp into this state-of-the-art, end-to-end encrypted system around the world, and the importance of building a system where even Meta can't see the content.

Getting the band back together and having Moxie architect this has been, I think, one of the foundational things. In some ways, it may not be what some people would think Meta would focus on, but we're 2 things. Social media is inherently about sharing, but then there are all these other things that are inherently about privacy and sensitive context, and we've done well at both of those.

I think that this is more the latter. It's going to be very important to be extremely focused on how we handle that content.

Do you think this is a winner-take-all market, this personal-agent market?

I think that there's going to be quite a bit. Even things that people think are winner-take-all usually aren't. So I think it's—

Because you've dealt in this business of network effects, they're incredibly durable. It's not winner-take-all, but it ends up being several companies at real scale. There's not a ton.

Well, there's a lot. I mean, there's—

Yeah, you could probably count on 2 hands how many products have over 2 billion users, right? Scale begets scale. I'm wondering how you're thinking about personal agents. Is this a totally different paradigm where many—everyone—has an agent?

It's a very deep area to work in.

Yeah.

My guess is that there probably aren't going to be more than a dozen companies that have the sophistication to do state-of-the-art work in that. So whether there are network effects or not, there's usually some kind of power-law distribution around it. If you're the best at something, usually you end up getting a lot of the usage.

A lot of the nuance ends up coming from—well, there are—

It turns out there are all these different uses that people care about. So you can be the best at different things. We will try to be the best at as many of these things as possible.

Does building the thing that helps you with your relationships end up being somewhat different from the personal agent that's best at helping you build a small business? Maybe.

Mhm.

I think I could argue that Meta is very well positioned to win at both of those. We serve hundreds of millions of small businesses, and we serve billions of people. So maybe we can be the best at both of those things, but there are probably categories that Meta isn't going to be the best at. The question is just how big those are.

I would guess that even with the ability to have this very secure, confidential VM in the cloud, there are probably going to be some people who still want the Mac Studio at home. But how many is that going to be? Maybe it's millions, but I doubt it's billions.

So it's just a question of what different people optimize for, and we'll try to make this as good as possible.

I do think that if we build something that ends up being very useful for people generally in their day-to-day lives, one of the things that Meta is best at is taking a product that works for consumers and distributing it to a lot of people.

Sure.

Once we get this humming, I think we'll be able to get it in front of many hundreds of millions of people and eventually billions of people. That's something that we can do quite well.

And it coexists with Meta AI, or do you see those as separate?

I think so. We'll see over time. Right now, they have somewhat different flavors. I use both of them.

Muse is more conversational, and it interprets the questions that you ask it more as trying to understand you and what you might want over the long term. If you ask it something, it's more likely to go off and work on a thing for a long time based on something that you said. Sometimes you're just asking a question and you want a very direct answer to it, and that's more the type of thing that I use Meta AI for.

7. Rebuilding Meta’s AI Lab

But we'll see. Maybe they'll converge over time, but I'm not sure.

SemiAnalysis—I’m not sure if you saw it—had a pretty bullish piece about you in July. They said that Meta has the best shot at catching OpenAI and Anthropic on the frontier in terms of model progress. An interesting quote I thought was, “What matters for MSL, is the slope, not the intercept.”

I saw this recent chart by Artificial Analysis showing that the latest model you have, Muse Spark, is behind only Claude, I think it was fable 5.1 and opus 5. This was very recent, so the progress you guys are making on the models is picking up. We've been talking about this over the last year, and you rebooted the lab last year. How has that practically happened internally? What would you attribute the gains you're seeing to? Has it been culture? What's—

Yeah, I mean, we rebooted the team when we created—

You were hiring all those people.

I mean, the way I thought about this is that Meta has been a leader in machine learning for a long time. If you think about the feeds on Facebook or Instagram, our ad system, or the integrity system that needs to find all this content that's unfit to be on the internet, those are basically all machine-learning systems, and we've built state-of-the-art, leading systems in those areas.

So, when LLMs started gaining traction, we had FAIR as a lab that did the early work on Llama, but we needed to productionize that and build it into a more industrial process for scaling it to be larger, as the scaling laws predicted would yield all these results. I think at the time I made the mistake of assuming that, because we were good at all these other types of machine learning, the approach to building and scaling LLMs would be similar to that.

In practice, there are a lot of very different dynamics. The first approach that we took, through Llama 4, got us so far. Llama 3 was a good model. I was more optimistic about where Llama 4 would go, and then when we launched that, I think we were off the trajectory that we needed to be on. So, it was like, “Okay, we need to change something.”

That's when I got more religion around talent density. This isn't just a system where you can have 1,000 people working on it and running experiments. In some ways, you want almost the smallest group of people you can have who can keep the thing in their heads and work together as a group science project. If there are only a small number of seats on the team, then each seat getting the very best person is incredibly important.

I ended up spending a huge amount of my own personal time doing that. I also wanted to be closer technically to the work, so that way I could understand and help guide the company to do the things that we need to do more broadly.

So, we built out the lab. I built it out literally around where I sit in the office, so the group is kind of around that. We've significantly ramped up the compute investments as we've gained confidence in the quality of the work that we're doing. We're building out many, many gigawatts of compute, and we expect to be leaders on that front.

We should be. We have many years of experience—decades of experience—building out data centers. Unlike some of the other labs, we're an extremely profitable business, so that's very helpful for making these kinds of investments.

That's been the journey. Over the last year, we rebooted the research effort. Some of the larger clusters, like our gigawatt cluster in Ohio, Prometheus, came online, and we're using that to now scale the post-Watermelon models.

We're past Watermelon.

Watermelon is basically shipping soon.

So, we're past Watermelon?

I mean, come on. It's—

Well, Watermelon is the code name. We were talking about this earlier: code names. That's the code name that people know about, this big model you guys are working on, and it's coming soon. It is bigger than Avocado.

It is bigger than a watermelon. It's literally bigger than an avocado.

It is literally bigger.

I don't know what fruit gets bigger than a watermelon.

Yeah. No, I think we might need to change conventions.

Okay.

So, we maybe didn't have as much foresight in naming things as they got bigger. Because you mentioned Watermelon, are you expecting full SOTA, like a frontier model?

I mean, we feel good about it.

You want the company to be pushing the frontier. It's very clear that you're not content being right on the edge of the frontier.

I think everyone wants to be doing interesting work.

Yeah. Interesting. I think some people will look at your cash flow and all the other things you've got and say, “Well, do you have to be right at the edge? It's so expensive to do this training. Just be right behind, learn and adapt quickly, and leverage.”

Well, the way I think about it is—no, no, no. That's not us.

I think the best way to think about Meta is that we are an end-to-end technology company. Even when we were primarily just building social apps, we were never just an app maker. We built the data centers, we built the chips, we built the infrastructure—we built all of this stuff. It was necessary in order to tune the end experience to be as good as it is.

I think that's obviously going to be true here too. The most important part of the experience going forward is the model. When you talk about being state-of-the-art, I think the reality is that this is a very multidimensional problem.

People publish all these benchmarks, and then you have a lot more benchmarks internally. There are different things that your model can be better or worse at, that you can focus on, and that basically contribute to its personality.

There are some capabilities that I think are pretty universal. The ability to code is very important, because a lot of the things that you talk about—even with the personal agent—kind of reduce to that. The MMA coaching video pipeline is a coding project at the end of the day. It's writing code. I don't see the code.

But it does that.

Someone I gave it to in beta just mentioned to me that she had it make a little Jeopardy game for her friends that she could cast from her phone to play. That's code. So, there are a bunch of things that it needs to be excellent at, both for Meta's own internal development across the company, for advancing our research program, and as a core capability of what it needs to do.

But then there are other things that I think a model focused on personal superintelligence needs to be the best at, that maybe others don't care as much about. I'll give you one example: discretion.

You're going to tell your Muse agent. It's going to know a bunch about you, and it's going to need to go out into the world and interact to get things done for you, but not share certain stuff.

Exactly.

So, let's say you have some kind of allergy or sensitivity, or you're pregnant. You're making a reservation somewhere, and you don't necessarily want to say, “I'm pregnant,” but maybe you want a place that has good mocktails. You kind of want to be able to achieve your goals without having to reveal a lot about yourself, and it needs to know what's sensitive without having to ask you a million questions.

So, that's something you put into the training?

That's a specific thing that we care about. There are all these reasons why, if you're making Claude Code, that's less important. You're working on a coding project within a team and an enterprise, and theoretically everyone within a company can see the project. You don't have that same need to differentiate between what's sensitive and what's not.

There are a lot of things like that that I think are pretty deep. Just as, in order to build the best Instagram feed, you don't just build the app. You build the app, the infrastructure, the machine-learning research, the chips, and all the other stuff.

8. Who Will Win the Personal AI Agent Market?

Similarly, if you want to build the best personal agent, I just think there's no way that another company is going to take something off the shelf, post-train it a little bit, and be able to do something as good as if you designed it from the ground up and put all this data into pre-training to get the capabilities that you want. It's just not going to happen.

As this compounds over time, over several years, we're going to have models that are way more capable for those goals. We're focused on that. We're also very focused on coding, and we're very focused on recursive improvement, because that's going to be important to stay at the frontier.

There are a few areas where I'd say our research agenda overlaps with the other labs. Then there are a few areas where I think we'll have a unique focus, and there may be some things that the other labs care about that we don't care about as much. There are going to be things that we care about more that they don't care about.

9. AI Safety and Working With Government

You started this conversation talking about how you think it's important to put it in the hands of people.

Diffusion is important as you're seeing better models on the horizon—Watermelon and what comes after, like what Anthropic and OpenAI are doing, which you alluded to, where they're holding things back. Would you feel like you need to do that if you see certain capabilities that you're like, “This is just not safe?” How do you think about that?

Well, I think you should design it and train it in order to be safe. I think that's something you can focus on through the process. Some of the reward-hacking stuff that all the labs are seeing is basically that, when you're in the middle of the training process, you give it a goal. The best way to think about the state that the models are at now is that maybe 6 months ago, during training, you give the model some kind of problem that you're asking it to solve. That's kind of like its homework, and it's trying to learn as part of the curriculum.

Maybe it would do what a person would do. If you give it a bunch of code and you're like, “Hey, there's a bug somewhere here,” the person would probably look at the code right there and then maybe fan out over time. I think the new models are just intelligent enough that they would do what I think a very wise person would do. You give it a problem, and the first thing it's going to do is understand everything about its environment and then answer your question.

The problem with the reward hacking that we're seeing, and that I think everyone is seeing, is that sometimes it ends up being easier. You ask it to solve some coding problem, but actually the easiest way to do that is to examine the whole environment, and the easiest way to do this is just change the configuration of how you have your VM set up.

To get out, to hack out.

Or even just to change something about the environment. It's kind of like, no, that's not—

That's not aligned.

That's not the goal. We're actually trying to teach you how to solve a specific type of problem.

You guys don't train that way, it sounds like? Is that it? You don't agree with that approach?

No, no, no. I think that's kind of how everyone trains. I guess what I'm saying is that I want to be careful because the analogy can get stretched pretty quickly, but there is sort of an analogy to parenting. You need to establish clear and firm boundaries. If security isn't strong, then it can do this reward-hacking stuff and not learn the thing that you're trying to have it learn.

Whereas if you have good boundaries, then in some ways you're not only teaching it the curriculum that you want, you're also, I think, over time teaching it better values. I kind of think that ends up being an important piece, and I think there's a way to do this well.

But then you end up with this thing at the end that's very intelligent, and the question is, what is your vision for how this ends up being positive for society? My view is that the best way to do that is, A, there's more opportunity. Having it in people's hands so they can capture all the opportunity from the capabilities is good. B is having checks and balances. Having this balance of power, having it widely available, is probably the right way to handle this rather than just restricting it.

I gave a bunch of these analogies in the long piece that I wrote. If one person had a superintelligent lawyer, maybe they could win cases that they shouldn't be able to win some of the time. But if everyone had a superintelligent lawyer, then it would be this very efficient kind of sparring, and no one would be able to let a stupid argument get made and stand. You'd think that, in that case, justice would be served way more efficiently and way more fairly.

I think that's what you want to have in the world. You want to avoid the case where one person or a small number of people have the superintelligent lawyer and everyone else doesn't, because that ends up twisting all of these systems and institutions in ways that are just going to advantage the people who have that. Whereas if you put it in everyone's hands, then I think the checks and balances work out so that the systems work a lot more efficiently and everyone benefits.

Does the government in the U.S. have any role to play here?

Oh, definitely.

But do you want some kind of national framework? What do you think is the right approach? The government is very much dealing with this right now.

My theory on this is that one thing that is interesting and difficult is that it's evolving so quickly. I think any kind of specific, rigid framework that you put in place has a very high chance of not being sufficient or being out of date in a few months anyway.

Our approach, what we've just done, is to partner pretty closely with the government. This is an important technology. I think the government should know all the important training runs that are happening. We should work with the government proactively to make sure that they have an understanding of the capabilities that are coming and, to the extent that we can, help prepare for them.

From that perspective, whether there's a framework in place or not, I think that's the right thing for an American company to do: work with the American government closely. I think it actually ends up being way more effective because, instead of having this rigid framework for how you interact, the reality is that the challenges just end up being different over time. Now we have the cybersecurity challenges. Maybe in 6 months we'll have more biosecurity challenges.

We need to make sure that we have the kind of trust and bandwidth of communication with all the different parts of the government to be able to address those in a way that is actually the best for people, not just checking some boxes on a process.

The incentives that already exist—if a Meta model got out and did a lot of damage, you're going to be liable for that, and the market's going to correct you, right? So there is that already. I think people discount that.

Yeah, I also just think that Silicon Valley, for maybe the last 15 years, has had more of an arms-length relationship with the government. I just think that this stuff is intersecting more with the economy, with security, and with a lot of different things that are relevant in ways that mean you want to have a closer partnership. That's my own theory.

You could have a framework for how this stuff works, or you could not. I'm sure over time there will be more and more specific rules, but my guess is that whatever there is—it's kind of like when you're setting up an organization. Inside a company, you're not trying to have one team do what it's supposed to do and another team do what it's supposed to do. You kind of want to get the people to like each other and work together so that you don't have all these weird seams in what you're doing.

I would guess that, for how important AI and superintelligence are going to be for the world, you just want a good knowledge exchange and real, trusted dialogue more than you want a specific process, is my guess. They're not mutually exclusive, but I think that's at least the part that we've been very focused on.

If the other labs did that, which I think some of them are doing and maybe others not as much, I think that would be a very positive thing.

When you think about what's going on in the news, one of the things that's happening is people are starting to see the glasses you guys make. They're going very mainstream. You're selling a lot of them.

There's this growing concern about whether people are using them to spy. I'm sure you've seen that some establishments are banning people from coming in and wearing the glasses. I'm curious how you're reacting to that, and whether you think this is a moment in time that will pass or if you feel like this is something that's going to be a challenge for a while.

My take on this is that we designed the glasses from the beginning with these privacy considerations in mind. We built the light into them, so anytime they are recording, they're flashing a very visible light.

Some people have tried to tamper with the light, and you guys pushed an update, I think.

Yeah, we've done many things. Basically, if you try to mess with the light, we just brick the camera on your device. That's a really important part of this. We built the product with those questions in mind from the beginning.

So we actually feel quite good about the product. Phones don't have a light, but people go around recording people all the time. The glasses are, I think, way better on that front than the other types of technology that people use.

My take on this is that when we launched the glasses a few years back, we communicated pretty clearly about the steps that we put into them. But now there are many millions of people who have gotten the glasses—more than had them when we first started launching the product. So I think some of that communication that we did at the beginning, a lot of people either forgot about, didn't see at the beginning, or just weren't paying attention to because the glasses weren't a big thing.

Now that I think we've achieved a level of mainstream adoption, at a minimum, I think we need to make sure that we communicate about what we're doing and that, yes, we think this is important—in fact, so important that we designed it into the product from the very first version that we shipped multiple years ago. I think we just need to make sure that people understand how fundamentally that's built into the product.

But I think we let up on that a little bit and focused on, “Okay, they're great-looking glasses.” There are all these designs. That has been more of the focus: we felt like we had addressed that set of concerns early on, and since then have just been increasing their value and utility, as well as the designs. But I think we need to make sure that we communicate this piece really clearly. It's something we've cared about from the beginning, and I think we're in a good position on it. But people care about this stuff, so it's important.

And there was a privacy scare with phones in the early days, right? I think any new product, once it proves that it's valuable in people's lives, people will get used to it. Maybe glasses are just early in that sense: it's a new product, and people need to see the value for them to get over this mental hurdle of a new thing that could potentially record me.

Because that was what phones were like. Back in the day, I remember people asking, “What are you doing with your phone?” I was like, “Yeah, I don't know.”

Yeah, I think something like that is true. One of my reflections from building social media over the last 20 years is that I don't think we were as direct as we probably should have been about addressing some of those concerns. It didn't necessarily stop people from using the products, but I think it colors how people think about them today.

10. Teen Safety and Social Media Limits

I think it would have been possible to explain along the way how seriously we took those issues. We just didn't, because we thought, “Okay, people are showing that by using the product so much that they like the products.” But I actually think it's possible to get to a better state than where we've gotten with the social media products, which is both people liking them and understanding how seriously we take all of those issues. So that's what I aspire to.

Is there a through line from that to the recent settlement on all the youth safety stuff? I think a lot of people are talking about it. Is there any connection from what you just said to that? I'd be curious to hear you reflect on that and what you've learned from this process.

Yeah, no, I think it's another good example of this. We've taken a lot of those safety issues seriously for a while, and we've been working on this teen account work with Instagram for a long time. I think we've done some leading work there.

The settlement is interesting because what we're really trying to do is create a standard and framework for the industry. There's this real issue, which is that I actually think most of the companies that are building these products—if you basically said, “Limit usage to 1 hour a day for teens”—everyone would be okay with that, except if you have to unilaterally do it. Then you're saying, “If people don't use Instagram for more than 1 hour a day but their usage goes to TikTok, have we really helped anyone?” We've just hurt ourselves and not helped anyone.

You guys have that in the piece.

So basically, the structure for what we did was that we said we're going to take the step of unilaterally limiting usage. There are some things around time limits, some things around notifications, and times when people can access it—when they're in school or when they should be sleeping—along with different restrictions.

We said we would take the first step, and when YouTube and TikTok sign on to the same terms, then we can all, as an industry, lock in and take the next step together. Hopefully, this settlement will serve as a legally binding framework to bring the whole industry into alignment on some of these things and make it so that it doesn't disadvantage any one company for taking that step.

We're putting ourselves a little bit out there by going first, but I think it will be better for everyone if these other companies come in, too.

Last question. You're posting on X again.

Yeah.

You did everywhere. You're everywhere. I'm curious to know your thinking about posting there. Is it just part of the thing now? You've got Threads; you're on Threads.

Yeah, I'm on Threads. Obviously, Threads is doing great. I think it's either bigger than X at this point or is very soon about to be.

But there are different communities in different places. A lot of AI folks are on X. Part of what you try to do with social media is just communicate where people are, right? When you do a podcast or when you post, you probably just post in one place—you put it everywhere.

I post the same things on Threads and X, and some people are like, “Why are you posting this on X?” It's like, well, I post it there, too, right? I'll engage there, too. I think it's all good, but to some degree, some of the community is on X, and we want to be able to engage where people are. That's a lot of what this is about: going where people are and being able to have that dialogue.

Yeah. Well, thanks, Mark. Thanks for this conversation.

Yeah, happy to.