Moonlake: Interactive, Multimodal World Models — with Chris Manning and Fan-yun Sun

I think this whole space is extremely difficult as things are emerging now. And it's not only for world models; I think it's for everything, including text-based models, right? In the early days, it seemed very easy to have good benchmarks because we could do things like question-answering benchmarks. But these days, so much of what people want to do is nothing like that, right?

You're wanting to get some recommendations about which backpack would be best for you for your trip to Europe next month. It's not so easy to come up with a benchmark. And it's the same problem with these world models.

Okay, we're back in the studio with Moonlake's two leads. I guess there are other founders as well, but Sun and Chris Manning, welcome to the studio.

Sun

Thanks for having us, Moon Lakes.

Thanks for having us.

You guys have come bursting onto the scene with a really refreshing new take on world models. I just want to ask how the two of you came together. Chris, you're a legend in NLP and AI in general. Sun, you're his grad student, I guess.

Sun Yu

Actually, my co-founder is Sharon. She was working with Professor Fei-Fei Li and Andrej Karpathy, and then she ended up working with Ron and Chris Manning here. Then I got connected to Chris initially through my co-founder.

What is Moonlake? I'm also very curious about the name, but why go into world models?

Sun

I was working a lot with NVIDIA Research during my PhD years on essentially generating interactive worlds to train reinforcement learning agents, or embodied AI agents. There were 2 observations, one in academia and one in industry. In industry, folks at NVIDIA are actually paying a lot of dollars to purchase these types of interactive worlds, whether it's for evaluation or training the robots, policies, or models. And in academia, the same thing's happening.

More specifically, when I was working with NVIDIA on the synthetic data foundation model training project, we were generating a lot of synthetic data and showing that these synthetic data are as useful as real-world data when it comes to multimodal pre-training. But, like I said, there's a lot of dollars being paid out to external vendors or other folks to manually curate these types of data.

It was very clear to us that, on our way to, let's call it, embodied general intelligence, models need to learn the consequences behind their actions, which means they need interactive data. The demand for those types of data is growing exponentially, but everybody's thinking about it from a pure, say, video-generation perspective or something else.

But we feel like the true opportunity is building reasoning models that can do these things like humans do today. So that's a little bit on the genesis of Moonlake. And I think the reason I got into world models was partly a philosophical take on the world, where I believe in simulation theory and stuff like that. But on the other hand, it's really just, "Oh, there's an opportunity there that I feel like nobody's doing the way I think it should be done."

I can say a little bit about that. The overall goal is the pursuit of artificial intelligence, and most of my career has been doing that in the language space. That's been extremely productive, as we all know from the story of the last few years. I don't have to tell you about how much we've achieved with large language models, but although they're being extremely effective for language and general intelligence, it's clearly not the whole world.

There's this multimodal world of vision, sound, and taste that you'd like to be dealing with, more than just language. And then the question is how to do it. Despite a huge investment in computer vision, it's a research field—computer vision has been for decades far, far larger than the language space, actually. I think it's fair to say that vision understanding sort of stalled out, right? You got to object recognition, and then progress just wasn't being made.

If you look at any of these vision-language models, it's the language that's doing 90% of the work and the vision barely works. So there's really an interesting research question as to why that is. And at heart, the ideas behind Moonlake are an attempt to answer that, believing that there can be a really rich connection between a more symbolic layer of abstracted understanding of visual domains, which aren't in the mainstream vision models, which are still trying to operate on the surface level of pixels.

One of your blog posts put it as "structure, not scale." Is that a general thesis?

Yeah, well, scale is good, too. Lots of data is good as well. But nevertheless, you want the structure to be able to learn much more efficiently.

The other thing I really liked was that you put out an example of what your reasoning traces look like, right? "Distill" is the word that comes to mind, although I don't even think that's a good description. It would involve, for example, geometry, physics, affordances, symbolic logic, perceptual mappings, and what have you.

That is the kind of example that involves, let's call it, spatial reasoning—world model reasoning—as compared to normal LLM reasoning. But also, taking it a step back, how do you guys define world models? A lot of people see, okay, you can do diffusion, you can do video generation, but you guys put out quite a few blog posts. You put out an essay recently—we can even pull it up—about efficient world models.

You have a pretty structural definition here, but for the general audience that don't super follow the space, what's the difference between what we see from a video-generation model and a world-generation simulator? How do you paint that landscape?

I think this is actually a little bit subtle because people look at these amazing generative AI video models or Veo 3 or these things, and they think Genie. They think, "Oh, this is amazing. We've solved understanding the world because you can produce these generative AI videos."

But the reality is that although the visuals do look fantastic, those visuals actually aren't accompanied by an understanding of the 3D world—understanding how objects can move, what the consequences of different actions are—and that's what's really needed for spatial intelligence.

A term we sometimes use is that you need action-conditioned world models: you only actually have a world model if you can predict, given that some action is taken, what is going to change in the world because of it. In particular, that becomes hard over longer time scales.

If you're simply trying to predict the next video frame, that's not so difficult. But what you actually want to do is understand the likely consequences of actions minutes into the future. To do that, you need much more of an abstracted semantic model of the world.

The question becomes: you want to have more structure than is available in just predicting the next token. Typically, let's call it, the experience of the last 5 years has been that that is just washed away by scale, right? So what is the right middle ground here? You don't ignore the bitter lesson, but you can also be more efficient than what we're doing today.

One possibility is, look, if we just collect masses and masses and masses and masses of video data, this problem will be solved. Under certain assumptions, that could be true. But there are multiple avenues in which it could not be true.

The first is that what's really essential is understanding the consequences of actions, producing an action-conditioned world model. If you're simply collecting observational video data, which is the easy stuff to collect when you're mining online videos, you don't actually know the actions that are being taken to see how the video is changing.

If you're never collecting actions directly and you're having to try and infer them from what happened in the observed video, that's not impossible, but it's very hard. And it's not really established that you can get that to work at any scale yet.

There's a lot of premium on collecting action-conditioned video data, which is part of why there's been a lot of interest in using simulation, so that you can be collecting data where you do know the actions, which is in quite limited supply.

Sun

But there's also the limit that, with as much data as you could possibly have, maybe the problem is eventually solvable. But even though we collect huge amounts of text data, text data is always at a great level of abstraction, right? Language is a human-designed, abstracted representation, where there's meaning in each token, and it's representing an abstraction of the world, right? As soon as you're describing someone as a professor, and as soon as you're saying that they're condescending, these are very abstracted descriptions of the world. It's not at what you're observing at the pixel level.

And so, to get to that kind of degree of abstraction starting from pixels is orders of magnitude of extra data and processing. And so, although we absolutely want to exploit and get as much data as possible and use the Bitter Lesson, nevertheless, if there are ways in which you can work with 5 orders of magnitude less data than people working purely from pixels, you're going to be able to make a lot more progress a lot more quickly. And that's the bit here.

You could just say that's only about wanting to be able to do it more efficiently, do it more quickly, do it more cheaply. But I think it's actually more than that. I think one should be making the analogy to how human beings work. At one level, yes, we have these high-resolution eyes and we can look and see a scene like a video. But all of the evidence from neuroscience and psychology is that most of what comes into people's eyes is never processed, right? You're doing fairly fine-grained processing of exactly what you're focusing on.

But as soon as it's away from that, there's another guy over there, and you're only processing, top-down, this very abstracted semantic description of the world around you. And so, that's what human beings are doing. They're working with semantic abstractions. And so, I think it is just the right representation because we also have other goals. We want to be able to do real-time worlds. That means there's a limit to how much processing you can do. And we want to do long-term planning and consistency. And again, that favors abstraction.

I mean, I guess there was actually a recent blog post that came out from our friends at Physical Intelligence. They were sort of heading in the same direction. They were saying, “Oh—”

The π0 model.

Yeah. To maintain a long-term memory of what's happening in the world so we can do things longer-term, we're actually storing text of what has been happening in the world, right? It's not such a successful strategy to try to keep it all at a pixel level.

And yeah, you can see it in video models, like that temporal consistency. Even at the scale of training on all the video data we have, we have it for maybe 30 seconds, a few minutes. It's not the same as a game state played for half an hour, right? I thought you guys broke it down pretty well. You have a blog post about “Building Multimodal Worlds with an Agent.” I don't know if you guys want to talk about this. This is one of the things I read. I thought—

Sun

Yeah, so this is the thing I talked about with the reasoning chain.

So, there's different phases to this. It seems like it's more of an agent scaffold, a very different approach than just typing in a prompt, where you don't have the same consistency. For people who are listening, I would highly recommend reading it. It breaks down the problem in a different light, right? What do you need to consider when you're talking about video-like world models or game models? What do you need to consider? What are the factors? What are the elements? What's the state? I don't know if you guys have stuff to talk about for this one.

Sun

I wanted to add a little bit to a previous point, just to say something quickly. I do feel like sometimes people confuse taking an approach with abstraction with not believing in the Bitter Lesson. That's just false, right? We are believers in the Bitter Lesson, but I feel like the question that we always discuss is: what is the right abstraction level today?

The analogy I like to make is: let's just say we can encode and decode all images, videos, and audio in bytes. Then the most Bitter Lesson approach is to train a next-byte-prediction model, as opposed to a next-token-prediction model. It's just, okay, it's natively multimodal. But to Chris's point, it's the scale and compute you need to achieve that. That's why we always come back to: what is the most efficient way to do it? And reasoning models, to the point of this blog post, are a showcase of how we're actually just reasoning about the world and reasoning about the aspects of the world that matter for me to learn what I want to learn from this world model.

Yeah, it's like you're improving the encoder of whatever you're trying to model, and a better representation will just represent the important things in less space.

Chris Olah

Yeah, and that would just be more efficient.

Shawn Wang

Yeah. So, I fully agree that it is not antagonistic to the Bitter Lesson. Do you want to mention one more thing? Are there any philosophical differences with the JEPA stuff that Yann LeCun is working on?

I've got to go there. You're mentioning some latent abstraction, and I'm like, okay, fine, let's talk about it, right? It's the elephant in the room.

Yeah, there are philosophical differences. Yann LeCun is a dear friend of mine, but he has never appreciated the power of language in particular, or symbolic representations in general. Yann is a very visual thinker. He always wants to claim that he thinks visually and there are no word symbols or math in his head. Maybe that's true of Yann, and it's certainly not the way I think.

But anyway, the world according to Yann is that the basic stuff of the world and of intelligence is visual, and language is just this low-bit-rate communication mechanism between humans. It doesn't have much other utility, and it's far inferior to the high-bit-rate video that comes into your eyes. And I think he's fundamentally missing a number of important things there, right?

Think of this evolutionary argument, looking at animals, right? The closest analogy is chimpanzees. Chimpanzees have fairly similar brains to human beings. They have great vision systems. They have great memory systems. They've got better memory than we do for short-term memories. They can plan. They can build primitive tools. But humans are massively ahead in what we understand about the world, what we can plan, and what we can build.

And essentially, what took off for us was that humans managed to develop language. And that gave a symbolic knowledge representation and reasoning level, which just gave this sort of vaulting of what could be done with the intelligence in brains.

The philosopher Daniel Dennett refers to language as a cognitive tool and argues that humans, unique among the creatures in the world, have managed to build their own cognitive tools. Language is the famous first example, but other things like mathematics and programming languages are also cognitive tools. They give you an ability to think in abstractions, in extended causal reasoning chains. And that allows you to do much more. We use that for spatial representation and intelligence and planning and gameplay as well.

So, we believe—and this is underlying the specific technologies that Moonlake is making—that symbolic representations are powerful, and you want to use them in your understanding of the visual world when you want a causal understanding, when you want to maintain long-term consistency and prediction. And, as I understand it, that's just not in Yann LeCun's worldview. So, I think that's a fundamental philosophical difference.

Then there's the specific model he's been advancing, JEPA. I mean, that's a reasonable research bet as a direction to head for building out a model of the visual world. To my mind, it's one reasonable research bet; it's not really established that it's the best one that everyone should be following.

At least it's been developed at scale with Meta, but it's not just vision, right? I mean, JEPA is just trying to build a prediction model, and it can be applied to anything, really. People have done it. If the argument is that there is a latent representation that is probably more suited to the task, then why not let machines do it for us instead of predefining it at all? And isn't something like a JEPA-shaped thing the right answer? If not, why not?

So, I think there's a part of JEPA that's right, which is you do want to have a joint embedding that gives you a consistent model of the world. And Yann's argument is you can never get that from autoregressive language models because they're sort of left-to-right, churning out one token at a time.

I guess this is where we're at in the research arguments of the field. I'm not actually convinced that's right because although the token production is this autoregressive process that's heading left to right, I guess it doesn't have to be left to right. Anyway, in a sequence of tokens, we could have right-to-left Arabic.

But although that's true, all of the weights of the model that are internal to the transformer are a joint model of the model's understanding of the world. And so, I think you can think of the weights of the model as a form of joint representation, and therefore, it is plausible to think that that could be the basis of a world model which avoids Yann's objections.

I think I follow, and obviously that would touch on what Moonlake eventually ends up doing as well, right? It's hard to tell because you put out the end results, but we don't know the inputs that go into it. So that's something that we have to figure out over time.

Sunil Vemulapalli

Yeah.

I mean, I guess this kind of breaks down some of the outputs. Do you want to walk us through it?

Sun

Yeah, so this really just walks us through the reasoning traces of, okay, let's just say we want to build a world. In this context, it's really just a game demo that shows the variety of interactions that this world model can build. And yeah, it's really just reasoning traces of, okay, if you're prompted to create a bowling game, how did it achieve what you saw—that level of causal interaction and consistency? So, yeah, this is almost just an example of the reasoning traces.

Very detailed.

Sun

You don't even realize, right? When a video is generated, what happens when a ball strikes a pin? So first, there's audio in that—audio triggers happen, the score increments, the world changes, pins have to start dropping, and there's a timer that goes on. It's very similar to how we're now used to reasoning for language models. There's a whole state of what happens: geometry, physics, all this stuff, and then—

Shawn Wang

Yeah.

Sunil Vemulapalli

There's kind of that single prompt, so asset, physics, all this stuff. It's a nice view to see what's going on.

I think Sun is also too polite to point out that both Google's Genie demos and World Labs' Marble do not have interactive worlds.

Sun

That's the benefit of having a reasoning model, right? Because you can say, "Maybe in this particular context, I want to learn how to bowl." And then you can say, "Okay, then what is important when it comes to learning how to bowl?" Maybe it's, "I need to understand the basics of physics, and I want to throw it at them." I want to know that when it resets, it's a new game, so I know that—yeah, basically, you know to pick up the ball, the ball's going to cause the pins to fall down. You know that what's important to this particular bowling game is to score, and then you know that the score corresponds to the number of pins that fell down.

So, if it's a model that sort of knows what it looks like, knows what a bowling game looks like, but doesn't actually allow you to practice over and over again and understand what it takes to get a high score, then it doesn't actually allow you to learn what you set out to learn within the world model, right? And I think this is really just one example of showing the advantages of the approach that we're taking over most of the—let's call it—the zeitgeist today, when people talk about classical world models.

Right. So it sort of seems like the question to ask when there's a world model is: can I not only wander around the world and look at the beautiful graphics, but can I interact with the objects in the world and see the right consequences of actions?

Sun

And you also understand what the consequences would be if you do something, right? So it's not just like, okay, there's one thing: if I pick it up, something will happen. There are 50 options, and I can infer what would happen if I do any of them, right? It's very different when you can actually see it and play around with it.

There are 2 cheeky elements of that. I mean, the sort of—I guess the less ambitious one is, let's really establish it for listeners: why is this fundamentally different from writing Unity code? Just creating a model to translate a prompt into Unity code.

Sun

So there is an underlying physics engine. In that sense, there are some overlapping things with Unity, but the way we think about it is that a physics engine, tools, or code are cognitive tools—borrowing Chris's term—tools that the model can employ as means to an end. So today, maybe you say, "Okay, in this particular context, we care about physics, we care about the long-term causal consequences." Then, yes, we employ a physics engine. Maybe tomorrow we say, "Okay, we're training, let's just say, drones, where we only care about fluid dynamics and the visual aspect of the world." Then, yeah, maybe the model doesn't actually have to use a physics engine, or maybe it employs other types of representations or physics engines to achieve the task. So, yes, writing code for Unity is sort of similar to a tool that our model can employ, but our goal is for the model to take a representation-conditioned reasoning approach or process.

Shawn Wang

Yeah.

Sunil Pai

Internally.

Yeah. Using these things is just like general tool calls, right? Which I think is very interesting. The other, more ambitious one is some kind of recursive element where it becomes multiplayer, right? Here there's a single-player element. You're not modeling any other people involved, and that is a whole other thing.

Sun

But in fact, we can already do multiplayer.

Oh yeah? Okay, I haven't seen any—

Sunil Pai

So if you just prompt our model to say, "Hey, configure it to multiplayer," then it'll do this. You'll be able to configure multiplayer and a persistence database for you.

Shawn Wang

Easy.

Sunil Pai

Yeah.

Shawn Wang

So what are some of the current limitations, and where are we at? There's one approach of, okay, scale up video predictors. Obviously, there are data issues. With approaches like this, is it data constraints? What are the next steps? Is it real-time? There's one side of writing an agent to write Unity code, but, okay, I want to be streaming a game in real time. I want to have characters also be agentic, but where do we see this scaling up?

Sun

Yeah, there's definitely a data constraint. The more data, the better. This reasoning model can basically act as humans to operate a variety of tools and software to build whatever is necessary. And then there's a sort of fidelity constraint, which we're actually solving with another model, Reverie, which we can talk about later. It's not as easy to get to photorealism with the approach that we're taking, but we think there are better solutions to that, which we can dive into later.

One thing here is that it's a diffusion model, right? So there are a few approaches: diffusion, Gaussian splatting. Reverie is a diffusion model. Do you guys want to introduce it?

Sun

Yeah, totally. So within our world-modeling framework, we think there are 2 models that we train. There's the multimodal reasoning model that we just talked about, which essentially handles mainly the causality, the persistency, and logical determinism of the world. And then Reverie is our bet on saying, "Okay, while that model can take care of all these things that we just talked about, its limitation compared to existing, say, video models is that it doesn't have as high a pixel fidelity right out of the gate." Reverie is to say, "Hey, we can actually take whatever persistent representation that we generate with our multimodal reasoning model and learn to restyle it into photorealistic styles or arbitrary styles you want." This model is almost saying, "Hey, I'm going to respect the persistency and interactivity of the world that you created, but my only job is to make sure that its pixel distribution is close to what we want."

Yeah, great example right there. You kept the KL divergence.

Sunil Pai

Oh, where?

Shawn Wang

No, I mean, this is a classic example of how you don't stray too far from the source material: you kept the KL, which is—

Sunil Pai

Oh, yeah. Cool.

Shawn Wang

I mean, the difference is—and Sun was pointing at this—it's in one way a more difficult path, but a better path. Typically, diffusion models produce the whole scene and it looks lovely, but there isn't spatial understanding behind it that allows for real-time graphics gameplay, spatial intelligence, or understanding the consequences of actions in worlds. This is taking a path where it assumes an abstracted semantic model of the world—the world state—and then the diffusion model is used on top of that to produce the high-quality graphics.

Sunil Pai

We actually believe that this is going to be the next paradigm of rendering. It's going to replace rasterizers. It's going to replace DLSS as we know it today because it not only has these pixel priors learned from the world, such that you can literally play any game in photorealistic styles, which is a lot of people's desire when they play GTA, right?

And all the mods, all the people adding perfect lighting and all this.

Sun

So it's skins for worlds, let's call it.

Skins. Let's call it skins for worlds.

Sun

Call it skins; you can call it customization. You can play it how you want, right?

Yeah, exactly.

Sun

And I think another thing that we really pointed out, specifically in this blog, is the programmability of it, right? So what this means is that this renderer—well, historically, a renderer is always a derivative of the game state, right? You're saying, "Okay, here's the game state.

But here, I'm saying, actually, this renderer can be part of the gameplay loop. I can say something along the lines of: if, upon getting 10 apples, I'm going to have my weapon of choice—my bullets—turn into apples. That's possible because we can basically dynamically have certain game state trigger the preconditions to the renderer, such that the rendering is now part of the game loop, too.

One thing is to just say, “Okay, it's appearance.” But the second thing is also to say there are these novel interactions that are possible because this renderer now actually has priors of the world.

And it's up to the artist to figure out what to do with it.

Sun

It is up to the creators, yes.

Shawn Wang

Yeah.

Sunil Pai

I also think that's another big argument we're making, and the reason that we're taking the bet we're taking, is that a lot of the time, whether it's for embodied AI or gaming, you want a layer where a human can inject their intentions, right? For example, in the context of gaming, it's obviously my creative intent. But maybe in the context of embodied AI, it's, “I take this foundational policy and I want to actually fine-tune it to deploy in my house.”

You want to almost have a layer where a human can say, “Here's the distribution of things I want to create to achieve my goal.” I think 3D graphics as it is today is basically the layer for people to say, “Hey, what do I care about in this world?” It allows human intent to be expressed in these worlds much more explicitly and distributionally, as opposed to just saying, “Hey, I'm going to generate something arbitrary,” and it's just prompts.

Shawn Wang

It's one of those things where I think you're going to build up a series of models, right? This is just one of them. This is probably the highest-utility or highest-frequency one. I don't know what to call this, where you can immediately drop this in on any game and you don't need anything else that you guys do, but I could see that.

I think human intent is something that people aren't even used to, because we're so used to static worlds or worlds that just don't react. You're kind of blowing my mind right now. I'm wondering if you've talked to people at GDC and what they're going to do with it.

Sun

Yeah. The stance that we take on this front is that we're not going to be more creative than our users. But we want to make sure that we're building things in a way that really allows them to express their intent.

The thing that you said about “here's the distribution that I want”—I think text may be too low of a bandwidth to really demonstrate, because I'm probably just going to want to drop in a bunch of reference assets. And then you can figure it out from there.

Sun

You want to do a mixture of both, right? You throw in a few images: “I wanted this style. I wanted it to look like this.” It's a mixture, right?

I think it's a mixture. There's clearly a visual component of this, and it's not that everything can be text, because of course you want to give a visual look. But there's also a massive amount of giving the overall picture of the look of the world and the behavior of things that you can express in a few words of text, and it would be very time-consuming and difficult to do via visual means. So, you want a combination of both.

So, one question I have is: how do we go about evaluating world models? There are many axes, right? One is, okay, I have preferences—how well do we adhere to prompts? One is the simulation: is there core logic that's broken? Coming from what we know about evaluating diffusion, there's fidelity and things like that. What are some of the challenges that most people probably aren't thinking about?

Sun

Yeah, I think this is a great question and probably one of the hardest questions in world models, because I think it always comes back to what you're building this world model for. Depending on your end goal and purpose, the evaluations should differ. In the context of games, the most direct way of measuring is how much time people are actually spending in this world that you create. If your goal is, for example, deploying an embodied agent, then your end metric is: after training in these worlds that you generate, how robust is it when you actually deploy it to the target environment? But it's hard to measure these end metrics.

Today, people have proxy metrics that try to measure what we really care about, which is the end metrics. Frankly, it's different for every use case.

Which seems like quite a challenge, right? In language models or video models and image models, your benchmarks are proxies, right? People aren't actually asking instruction-following or tool-use questions. They're proxies for how well it will do downstream.

But for this, should teams and companies have their own individual benchmarks outside of games? If you think of things like video production, movies, and stuff like that that also want to use world models, should they internalize their own proxies? Is this something you guys do? Where does that connect?

Yeah, I think this whole space is extremely difficult. Things are emerging now, and it's not only for world models; I think it's for everything, including text-based models. In the early days, it seemed very easy to have good benchmarks because we could do things like question-answering benchmarks: could you answer the question based on these documents? There were various other kinds of tasks, like logical reasoning or math. But again, these are sort of the visual equivalents of things like object recognition, small component tasks.

These days, so much of what people want to do with language models is nothing like that. You want to have an interaction with a language model and get recommendations about which backpack would be best for you for your trip in Europe next month. It's not the same kind of thing, and it's not so easy to come up with a benchmark for whether a large language model gives you an effective interaction for guiding you in a good way for shopping. It's the same problem with these world models.

If we take the game design case, success is that a game designer can produce what they are imagining in a reasonable amount of time. That's really the macro task, but it's a very hard thing to turn into a benchmark. I think a lot of this is going to turn into people voting with their feet. I guess that's what's happening at the large language model level, right?

When people are choosing to use GPT-5, Gemini, or Claude, individuals are trying out these different models and deciding, “I like the kind of answers that GPT-5 gives me,” or, “No, I feel like I get more accurate detail from Claude.” It's a lot of people just using it.

People feel it's giving them utility in what they want, right? And the interesting thing there is that a lot of people prefer the visual, right? This looks pretty, which is not the objective of what this is for. If a game designer is working on something, they care about the game engine, the state. It can look like whatever; you can fix that up later. Or you can have a really good game state and quickly edit it to 20 different versions that keep state.

Right. So, that's a really important distinction, and it speaks to Moonlake's strength, right? Great visuals are lovely to look at for a few seconds, but games are really all about the concept and the gameplay, and a lot of the time that doesn't even require great visuals. There are lots of very successful games with relatively primitive visuals, and there are other games where people have spent millions producing photorealistic visuals and the game sucks. Keeping those 2 axes apart is really important in thinking about what's important in a world model for different uses.

This conversation is reminding me of some game-review and fiction discussions I've had in my non-AI-related life. Some people might know Brandon Sanderson, who is a very famous fiction author. He's also a big game reviewer, and he's a big fan of video games where you change one thing about what you might normally assume about the world.

For example, Baba Is You—I don't know if you might have come across that—where the rules change as you play the game. There are also games where you can reverse time selectively or change gravity selectively. I think this also reminds me of other kinds of world models created by authors, where Ted Chiang is my typical example: he'll take the world that you know today, change one thing about it, and then create a consistent world based on that.

That's a long-winded way for me to ask: is it easy to create alternative worlds that don't exist, change one thing, and then run a whole bunch of people through them to see if it works?

My first answer is that this seems a lot easier and more conceivable to do using technology like Moonlake than with some of the other world models out there. Whether Moonlake can actually make it happen, I'll let him give the second answer.

I guess, for you, you're constrained by the game engine tool, right? At the end of the day, that's the thought partner that you have. If I ask for something where it's never allowed to reverse time, or if gravity only ever works one way, then that's it. But sometimes gravity might change. It's a lot easier to change with code as opposed to a model that is learned primarily on data of real-world and virtual worlds that are—I guess, for example, Genie, right?

It's actually trained on a lot of real-world data and a lot of virtual gaming data, and it's hard to say—well, maybe it's easy to say—okay, I want to change the visuals and the time period of the world, but you can't change gravity, for example.

I feel like you can, to limited bounds, right? Everything comes down to code being a better way to execute it, but the models aren't that diverse and creative, right? You can say, okay, make gravity slower; it can do that, but it's limited to your representation of how you test it out, right? They're only going to do a few iterations, whereas programmatically, if there's a game engine under the hood, you can kind of go wild, right?

One of the limitations of most models is that they're very overtrained to one style, right? Extracting diversity is pretty difficult, at least. That's something we've seen.

I mean, are there examples you have in mind where it would be easier to do that without using code? Like certain types of creative intent or state transitions?

Other world models are very good at clipping through things.

Clipping?

Like my legs clipping through a rock because it's just bad. You would have to struggle very hard with your stuff to actually make that happen. I actually think that's maybe a topic that you prepared on: Gaussian splatting versus the other stuff.

Yeah, yeah. Just for those not super familiar, there's Gaussian splatting, there's diffusion—what works, what scales up? I feel like in February, when Sora 1 came out, the blog post was literally titled—

Bring it up. Bring it up. Bring it up.

You know, “Video generation models as world simulators.” It's super Bitter Lesson-pilled. A lot of it is emergence, right? Not to go through their blog post, basically, their whole thing was, as you scale up, all this consistency, all this stuff just kind of solves. It's a very simple premise, right? They just scaled up diffusion, and from there—you know, this is February 2024. It's already been 2 years, which is basically 5 years, you know. How much more in AI time do we need to just scale up? Or do we hit a data cap?

But I think we already talked about this a lot, right? This is back to the beginning discussion of what's appropriate for the time, and that seems like your approach, right?

The point I'm trying to make is that there are many, many different types of world simulators. Having a world simulator that can produce pixel coherency is very, very useful for games, marketing, and all these things. But it's not as useful as people think when it comes to causal reasoning, when it comes to embodied AI.

And, yeah, this title is true. We're not saying that it's not a great world simulator, but actually, in the blog that we wrote, the bet is more so that there's going to be a disproportionately large share of value in real-world tasks and virtual tasks where high-resolution pixel fidelity is not needed. And yes, video models have their value.

This is at the absolute limit of my physics understanding, but one example that comes to mind is basically the equivalent of having to solve a three-body problem in a deterministic world, whereas the video models would just approximate it good enough. Right? There's some point at which your approach kind of runs into, well, you now have to simulate the world. Please, thank you very much. You're trying to do that, but only to the extent that the game engine lets you, and the game engine cannot do some things.

No, I mean, I think the interesting or more technical question here actually is: where do you draw the boundary between what's handled with, let's say, a diffusion prior and what's handled with symbolic priors?

Yes, okay. Okay.

Right? Because this boundary can actually be fluid. I think maybe what you're trying to get at is, okay, people are saying pixel prior everything. But what we're saying is, okay, there's a boundary that we draw where this is where we think provides the most economical value for the domains and things that we care about today.

And I actually do think—and it's something that we do internally all the time—that given new equations that we learn, or new elements of the world that we learn, or maybe some other knowledge that we acquire in the process of developing models, should we still be maintaining this line exactly as it is today, or should we move it a little bit left or a little bit right, right?

Sometimes we realize that maybe customers or folks want certain things that are better handled with a pixel prior as opposed to a symbolic prior.

Yeah, your skin thing is an example of moving it right. Or left side—I don't know what the direction is.

Yeah, exactly. Left and right is—yeah, yeah, yeah. No, the Reverie model—

Yes.

Sun

Actually, we have a few iterations of them. They're actually slightly different—

I know. Oh, you should do that. That's a cool dimension to show.

Hao Sun

Yeah.

Is quantum mechanics the diffusion prior of our world?

Sun

Right? It's like that's the boundary of classical mechanics versus quantum, right? That's it, right? At one point God plays dice and the other point doesn't.

I don't know what Chris would say, but I think generally physics is better with symbolic priors.

Even quantum physics?

Sun

Even quantum physics. Yeah.

This is getting to the MLST territory, as I call it, where he likes to get philosophical.

I mean, we need to get to the singularity. I heard some of that.

No, no, no. I think that is actually really helpful. I just want you to productize this. As a product guy, I'm just like, well, okay, like—as a researcher, you know, it's cool. This is theoretical; you have a very good way of thinking about these things, but I just want to see you express it.

I do think that you're fundamentally thinking about things when you leave open new tools: okay, use human intent to incorporate it into how you render. Artists are going to have to take 2–3 years to figure out what to do with this, and you just don't know.

Sun

But I think this gives a much more approachable and controllable world for—

The beauty of NLP.

Sun

—that will enable it to be adopted and used, and we're very hopeful about that.

Shawn Wang

Yeah, yeah.

Sun

Yeah. I mean, we are very focused on commercialization, actually, in the sense that we do really believe in the data flywheel approach: we put this in the hands of the creators and the users, and then they will teach us which capabilities our model should improve. That's why we actually have products in beta.

Yeah, focusing on gaming. What's the adjacent thing to gaming?

Sun

Embodied AI, basically. Maybe we can start with where we see the platform in 3 years, which is, okay, the user would tell us what they want to achieve. The end goal could be, hey, I just want to make something to teach my kids the value of humility. Or it could be, hey, I want to fine-tune my drones to be really good at rescue situations.

It could be vacuum robots. I want to train my manipulation, or my vacuum robot, to be very robust in my office—navigate very robustly within my office. Whatever the end goal that you want, our world model will say, “Okay, given what you want to achieve, let me generate a distribution of environments such that I can train and evaluate whatever it is that you want.”

Maybe for the purpose of games, it's just the end simulation and that's the end product. For certain policies, I can train them within these environments and then help you see where your policy is failing or not.

So in that case, much more of a training tool than in other applications.

Sun

Training, evaluation, both, right?

Shawn Wang

Sure. Same thing. Yeah.

Hao Sun

I think it's just this world model that allows people to train any policy that can act across any multi-world environments.

Would it be harder to reward-hack? Is there an angle here where it is harder to reward-hack? I'll just put it generally, because that's obviously a key problem that a lot of people face when training agents in these environments.

I don't know. Can you solve it?

Sun

I think not necessarily. To the extent that there's a misspecified reward, it seems like it could be hacked in a more symbolic world or in a more pixel-based world. I don't know if Sun has any thoughts, but I don't think that's really being solved.

The only thing that comes to mind is that you could just build a better Sora as a video-generation model, right? Because then you would move the diffusion side a bit further to the right, I think, if I got the directionality correct. Maybe it's better in some domains, right? In terms of consistency over an hour, for sure. It exists versus something not existing, right?

Yeah. I'm just riffing on what you can build with the stuff that you have. I do think that academia goes immediately to training and evaluation, but art tends to take unusual directions.

Okay, yeah, but the question is: can you use this piece of software to develop compelling gameplay? I don't think you can take Sora and produce compelling gameplay, right? If you want to have a world that you can wander around in a bit, you're good, but what are your abilities to have play mechanics implemented the way you'd like them to be, and to have things stay with a long-term history of your gameplay that influences future actions? I think there's just nothing there for that.

Yeah, I do tend to agree. I'm just trying to test the boundaries. I would also make the observation that, as the AAA games industry has developed, the line between what is a movie and what is a game has blurred. You do end up basically producing a 2-hour movie.

Sun

Honestly, there are so many actual applications in adjacent markets that our world model can go into. It's fun to riff on, although on our execution side, we need to stay focused on what capabilities we want to unlock over time. There's a roadmap for that. But if we're just riffing on the possibilities, I feel like it's endless.

The embedding for possibility and endlessness in my mind is very close.

Kyunghyun Cho

Yeah.

I do want to focus on one weird choice. I don't know if it's weird, and maybe I'm missing something here: audio, right? You could have just said no audio. Audio, in my mind, has a lot of reverb, whereas in video you can just do ray casting, and that's computationally much simpler. Audio just seems way harder. I don't know if you want to comment on the spatial 3D audio problem. Did you really have to do it? I guess you do to be immersive, but a lot of people do treat it as, well, it's just a TTS model on top of—

Sun

Well, there's a lot more to game audio than just speech, right? It's not just TTS.

SFX, BGM, spatial audio, echoes.

Kyunghyun Cho

Yeah.

Shawn Wang

And reflections. I don't even know what else. I don't know what other problems are in the space.

Sun

Yeah, I think this point is more pointing to the benefits of using a game engine as a tool that's available to the model, right? Part of the spatial audio comes from the code underlying the simulation. While we do give our model access to other types of audio models as tools, none of them would be spatial, I think. That's exactly the point: we're giving our model an abstraction, or a suite of tools, such that it's able to achieve that.

You can argue that spatial audio is an emergence from the tools and abstractions that we provide to the agents. I think that's the beauty of this approach. There are a lot of things, similar to how humanity has built technology, that are like LEGO blocks building on top of each other. It's the same thing here. There are going to be things that emerge from being able to put these things together in combinatorially interesting ways.

Right, so this integrated audio model exploits the understanding and semantics of the Moonlake world, right? Whereas, in general, for generative AI video models, there's no actual integration with audio at all. Someone might stick some music or a soundscape on top of their videos so it's not a silent video, but they're in no way connected into a consistent world model. There's nothing that says, “Okay, an action is happening in the video; therefore, there should be a sound coming from this part of the visual field.”

Kyunghyun Cho

Yeah.

Shawn Wang

Is that different from Sora 2? Does it not have audio?

Sun

There's no spatial audio.

It doesn't?

Sun

No.

Shawn Wang

I've played around with it enough. It just sounds like someone put an ElevenLabs voice on top of it and tried to do the lip sync. I've seen examples like, “Generate a dog at the beach reacting to a big wave,” and it moves around. With Sora 2, have the dog move away from the camera and see if the sound goes down, right? It doesn't, because they don't have spatial audio.

Sun

We do want our world model—the one we're training—to be aimed at having a combined representation across all these different modalities, such that it can reason across them. For example, if I close my eyes and you play a sound of a car skidding away from me, I can almost visually extrapolate that trajectory in my mind. We want our model to be able to reason in that way, and that's the reason we're taking this multimodal-reasoning approach. We want this combined latent space that can do that.

Oh, you said “latent space.” We like that here. We have to play the bell every time someone says “latent space.” You’ve got to train Daredevil 1, where it’s only audio but you have to work out where everything is.

Cool. I think that was about it for our Moonlake coverage. I do think that we have a couple of interesting questions on information retrieval and any other attention topics or NLP topics.

Okay.

Shawn Wang

No, no. I mean, it’s just fun. We talked a bit about how you guys met, but you were basically the godfather of NLP, right? You spent your whole career working on early embeddings and early attention. You did the 2015 attention work for machine translation, everything. You worked on information retrieval—RAG before RAG. We just want to shout that out and admire a lot of that. What prompted the switch over to world models? How did all that come about?

To some extent, it is the enthusiasm and creativity of students. But there’s a bit of a history there, right? Clearly, most of my career has been doing stuff with language. How I got into research was thinking, “This is just so amazing how humans can produce speech and understand each other in real time. Somehow they manage to learn languages when they’re kids. How could this possibly happen?”

Starting off, I was very focused on language. But as it got into the 2010s, I’d been working on question answering, and then I started to get interested in visual question answering. That was an area where it was very noticeable that the visual understanding was bad. These were the days when it seemed like there was almost no visual understanding. You were just getting answers that came from priors.

If you asked how many people were sitting at the table, it would always answer two, regardless of how many people you could see in the picture. It seemed like these models actually weren’t able to get semantic information out of images. I was interested in that problem and tried to work more on it, which required knowing more about what’s happening in vision and how you can represent visual information.

Then there started to be this revolution in generative AI images, and I had students who started looking at that. Before Moonlake, I was also working with Demi Guo, who founded Pika.

And Ian Goodfellow, obviously, with GANs.

Yeah, but Ian was never my student. I was very aware of Ian and his work with GANs throughout that whole decade. Ian was a Stanford undergrad.

There were links across that group at that stage as well. There were several papers in that era doing this kind of work. Andrej Karpathy was a PhD student at the same time as Richard Socher, and there was some joint language-vision work in that era as well. It seems kind of ancient by modern standards, but we were trying to go from textual dependency graphs to visual scenes.

At the time, GloVe embeddings really took over from a lot of TF-IDF and one-hot encoding, all that. The early vision-language models we saw were LLaVA-style adapters, right? It’s technically still just an embedding latent space that’s tied to the image. That’s what makes the modalities work, and that’s one of the things you put out there too, right?

Jiajun Wu

Yeah.

Yeah, well, thank you for all of that. Thank you for advancing the world’s work on world modeling. Honestly, I do think that if people deeply understand everything we just covered, they will see what’s coming. I think you guys have made some really significant contributions here.

What are you hiring for? We agreed that the CTA was a hiring call. I mean, don't we have AGI? You don't need engineers anymore, right?

Sun

Yeah. On the model side, we are actually striving toward basically a self-improving system, but what that means is that we need people to set up the self-improving system. More specifically, people who have the intersection of knowledge within code generation, computer vision, and graphics. That's sort of the core research background that we look for within our team, and the majority of the team today does have both backgrounds.

When you say computer vision and graphics, are they the same thing, or is computer vision one thing and graphics another thing? How intertwined are they?

Sun

They're intertwined, but different.

Shawn Wang

Yeah.

And I think this relates to some of the themes that we've been talking about, that the more explicit underlying world models being constructed inside Moonlake really draw on the computer graphics tradition. It's then combining that with the visual understanding from vision.

Shawn Wang

Got it.

Jiajun Wu

Yeah, all right.

So, if you've written a game engine, you should come talk to us, right?

Sun

Oh, yeah. Yeah, definitely. But I do think that the line is increasingly blurred these days, where if you have a general understanding of computer vision and graphics—

I think for your standards it is. For me, it feels like vision is—I’ll leave that to the big labs. Graphics, I can get that. You would want to do that from more first principles, but vision—there are so many vision models off the shelf that I can take, but they're probably not good enough for your needs.

Sun

I see. I see. If you're making that distinction, then maybe we care a little bit more about having graphics knowledge.

Yeah. It could be like, sometimes a hiring call can be as simple as, “If you know the answer to blah, you should talk to me,” you know? The sort of core known hard problem in your world.

Sun

Ah, I see. Yeah. In that case, if you've written a game engine before, if you've RL'd a variety of coding models on different objectives, like—

Shawn Wang

Easy.

Sun

Many of those, yeah.

If you've done multimodal latent-space alignment, I intentionally include that again. Our poor editor has to edit that every time. Yeah, latent-space alignment. Honestly, is it that hard?

Sun

Well, there are some scripts out there that I saved for the day. Someday, someday I have to do it, but I don't have to do it. It's done.

I think, yeah, there are versions of that that are done.

Sun

But I think we are aligning audio, text, image, and video, right?

Shawn Wang

Yeah.

Sun

And basically, we have these world models that are able to act as agents in these worlds and generate long-horizon videos, encoding that back into the models to self-improve. So it's an insanely exciting, but also technically challenging, problem. So people who want to do their life's best work—that'll be the place.

How big are you guys? Where are you guys based?

Sun

We're currently based in San Mateo, although we're moving up to SF. We're about 18 folks right now.

My ending question was going to be: What is the name? What's behind the name?

Jiajun Wu

Oh.

Very cool graphics and design, by the way.

Sun

Actually, at the time when we started the company, we were thinking a lot about how to make a company name that gives people a vibe of OpenAI, but with almost Industrial Light & Magic vibes. Because we care about creativity and using that as a funnel to solve AGI, we brainstormed a lot around DreamWorks, Industrial Light & Magic, and so on. There's a whole space of things that we feel are very, very semantically close to the company's identity.

Shawn Wang

Yeah.

Sun

And then it ended up being Moonlake, partly because of the DreamWorks vibe—the DreamWorks—

Cool. Exactly.

Sun

So that was a little bit of that inspiration. And then the moon was basically about reflection. The reflection part also implies the self-improvement loop that we really believe in, and that's the path toward multimodal general intelligence. So that's that. I'll leave it at that.

I love a good name.

Sun

This is a very, very good name.

Good lore. I'm glad I asked the question. I will also say, one of my favorite storybooks—biographies, really—is Creativity, Inc., with Ed Catmull's story about Pixar and how he was rejected as a Disney animation artist. So then he went into computing and bootstrapped his way back into Disney.

Sun

Yeah. And Walt Disney is also one of my favorite founders. His story is like, at the time, you're like, “Okay, I'm going to create this immersive park.” People don't even have the technology to create it virtually, but, you know what? Let's just build it physically so that people can—

So he's the first world modeler.

Sun

No, I'll tell people that theme parks are world models, too.

Yeah, yeah, yeah. I mean, “It's a Small World,” or Epcot Center with all the little replicas of the countries—those are very interesting. Okay, well, thank you. We've covered a huge amount. Thank you for your time and for inspiring us.

Sun

Thank you for having us. It's fun chatting.

Yeah, it's been a good time.