AI 正在错误的抽象层级上学习——Matthieu Wyart

I am Matthieu Wyart. I'm a full professor at Johns Hopkins University in the US and at EPFL in Switzerland. You know those machines that can build new images we've never seen before, or say new sentences that were never heard before? Our brain seems to learn languages with 100,000 times fewer words than machines. Why is it so? Are we doing the wrong thing?

I'm very interested in whether we should predict in token space at a very low level, or whether we should train machines to predict abstractions. What we've been doing over the years is trying to build a framework based on physics that's really tried to answer those different questions in a unified manner. Chomsky gave this poverty-of-the-stimulus argument, arguing that it was actually impossible to learn to become creative from examples. But if you have a deep architecture, there's a huge implicit bias to build those coarse-grained variables.

If you think about LLMs or diffusion models, the way they represent concepts, those concepts emerge from statistics alone. Those abstractions emerge; they are there in the data. Those concepts emerge if you group together configurations that predict similar contexts around them.

This is very pertinent because you've got a paper out basically saying that we should predict in the latent space, not the token space. So, again, in those models, what we found is that those algorithms that are introspective, that learn from their own latent space, are much more powerful in terms of sample complexity. Eventually, they will learn the same abstraction, but much faster.

If you never make mistakes, maybe it's a sign that you're staying a bit on the beaten path in science. Some of us want to explore the jungle, and in the jungle you can be wrong. Yes.

1. Notion agentic workspace

Quick pause. Agents are getting smarter every day, but even the smartest agents get stuck without the right context and the right tools. That is where Notion comes in. With the recent launch of custom agents, Notion became the collaborative AI workspace where teams and agents work side by side. And now their new development platform is turning that workspace into infrastructure developers can build on. Now, this is exactly how I run MLST. The whole show lives in notion. My guests, the publishing calendar, the commercial side, everything is in there. But what's changed is that it's now agentic. I just talk to my agent. It can be clawed or any agentic harness. And then it then talks to notion via the MCP or the CLI and it's just done. And then I can access it on my phone. It's an absolute game changer.

2. From statistical physics to machine learning

So, yeah, I'm a physicist, actually. I really liked learning physics because you have to deal with nature at all possible scales, and I focused on 1 specific field in physics called statistical physics. Statistical physics is essentially the field where you try to understand how many entities—particles—interact together to produce collective phenomena. A classical example is that you take water, cool down the system, and at some point, boom, it freezes, completely changing its organization.

I started to work on that initially on the stock market, where you have interacting agents that influence the evolution of the price, which is a very interesting sort of random walk. Then I went to study complex systems. Complex systems are physical systems with a rough energy landscape. It means that if you're flying above the Alps, like you just did, and you throw a ball in those mountains, it could stop at many different points. So the energy landscape has many metastable states, and those systems are really intriguing: physical systems with memory.

I worked on several of those—for example, sand. What's beautiful about sand is that it's a complex system: if you prepare 10,000 piles of sand, each of them is different, but it also has intriguing phase transitions. As you know, if you tilt a layer of sand, at some point it's going to flow. It means that the energy landscape was rough and you were in a metastable state, but you tilted this energy landscape, you had a phase transition, and then the entire system flowed. Although it's very dense, the particles managed to avoid each other. I've been very interested in understanding those questions geometrically.

But then, 9 years ago, I'm a Go player—a poor Go player, but I enjoy playing—and I was mesmerized by AlphaGo. So I started to think about machine learning, and I started to think of it as a complex system. When you train a machine, you build a function that is low if you fit your data well. It's called the loss function, or cost function. We were very intrigued by the geometry of this landscape, and what we discovered is that this landscape has exactly the same phase transition as sand.

It means that when you're underparameterized, when you don't have enough parameters, you have a rough landscape with many metastable states. If you train your machine many times, it will end up in different positions where it's actually stuck. But if you have enough parameters, then suddenly the system can flow, and your landscape has many flat valleys that have essentially zero energy. There's a really close analogy. We discovered that 9 years ago, and at the same time, others found a very similar—I mean, the same—phenomenon and called it double descent. So now that name has stuck, but the peak of double descent is really, for physicists, a jamming transition.

3. What physics can explain about learning

To finish with that, in the last 4 years we've been very much interested in another landscape that I think is even more interesting: the landscape of data. If you think about an image—let's call it X—it's a vector. You could ask, what is the density of those images, ρ(X)? This question relates to what the structure of the world is, and we think it's key to actually understanding how a machine works.

Does it make sense to talk about that? Obviously, you're a physicist, and you're applying this lens of analysis to large language models. Naively, I'm looking at this and saying, well, it doesn't feel like a material substrate; it doesn't feel like it has the same type of dynamics as things do in the real world. But indeed, when we look at the training dynamics of LLMs and the types of representations they learn, we could adopt a physics lens and say there are coarse-grainings, phase changes, and whatnot. Do you think it's coherent to make that analogy?

Yes, I think it's a sort of mesmerizing fact of science that some concepts can be applied to such diverse phenomena. Science is essentially built on those kinds of analogies. If you think about Huygens, who was one of the first to propose that light was a wave, how did he propose that? He noticed that waves on the ocean could cross each other without interacting, and he noticed it was the same for light, so he made this analogy. I think it's even hard for me to talk about because I think it's so fundamental that we're always building our understanding in terms of analogies.

For the specific example I gave you about sand and the loss landscape of machines, I think the analogy is very direct in this case because in both cases what you have are essentially degrees of freedom. In 1 case, those are the particles of sand; in the other case, they are the parameters of your large model, and in both cases the systems are trying to satisfy constraints. For sand, essentially, the particles are just trying to avoid each other, but for the parameters, what they are trying to collectively do is fit data. The more data you have, the more constraints you have. The universality that we've argued for in physics applies there: if you have a constraint-satisfaction problem and continuous degrees of freedom that can change continuously, then, boom, you have a universality class.

In this sense, yes, there's something universal about those kinds of problems. But that's a very specific example; I don't want to say that everything is always the same. This specific problem of jamming in sand and the one involving the loss landscape of machines is very much the same, yes.

It's such a tantalizing idea because I think it's constraints all the way down. In evolution, we have naturally convergent patterns, recurring patterns, like carcinization. I guess the only critique of this is that it feels like in neural networks, or just in computers, we don't have the same kinds of physical constraints. We don't have, you know, the physical constraint that 2 objects can't touch each other at the same time because of the laws of physics, and so on. So the constraints are there by dint of statistical patterns in the data, but they still apply pressure on the training process. Are those still valid constraints?

Yes. Here I was really not talking about any sort of constraint of the computer itself. I was thinking in some abstract way about the algorithm—what the algorithm is doing. The algorithm is doing some sort of gradient descent, flowing down an energy landscape in both cases, and that's where the analogy is. The analogy isn't related to the material aspect. It's true that in 1 case it's a material, and in the other case it's an algorithm, but at some level it's the same if you think about it correctly. Yeah.

It's so interesting that you're thinking of the constraints as being the algorithm rather than the energy landscape itself.

And I get it. I think you're leaning toward there being some kind of universal learning algorithm. But the way I intuit it, it's almost like the data and the world are more meaningful as constraints. Is that legible?

Okay. So I think we will be talking about creativity, and we will also be discussing a lot about constraint, but there will be, in my way of thinking, another space. I started to tell you we're discussing the loss landscape, and the constraint here was just to feed in data. Later on, something I'm really interested to discuss is: if you think about the world itself—sentences, the data itself—forget about the algorithm that's actually learning it, the data itself is very constrained.

All possible sentences are not valid in terms of syntax. So I think thinking of constraint is very useful in both cases, but I think of them as very different kinds of constraints.

And do you still think of yourself as a physicist first? I mean, because now we're talking about the physics of learning; we're talking about machine learning. How does your instinct as a physicist come across into this world?

That's a very interesting question. We have to debate it all the time in physics departments because we need to hire people, and what is physics? We always ask this question. It's true that initially, when I started to think about machine learning, it was more closely related to complex systems, disordered solids, and things like that. Now I'm rethinking it. I'm organizing conferences with linguists, neuroscientists, and so on, but I still think, deep down, it's physics.

I think we need to bring physics to this field and do the physics of linguistics. That's a long discussion, but maybe what is very special about physics—I mean, other fields of natural science do it too—is that we're really trying to build a dialogue between theory and experiments. When there is a new technology, it's asking you a huge number of novel questions. We can start thinking about it, making theories, and then we make simple models. The theories have to be predictive of what's going on.

First of all, I think we're very good at building this dialogue and building some empirical science. The second aspect is modeling. The world is super complicated. If you try to make a map where 1 mile is 1 mile, it will never help you, so you need to build a caricature of the world.

It's an art that physicists have done. To paraphrase Einstein, a model should be the simplest, but not simpler than that. It means there's tension. It's actually difficult to describe the problem at a good level of complexity, and it also depends specifically on what question you're asking. So I think physicists are also good at developing those sorts of models.

Just to give you an example, I told you about phase transitions. A century ago, Pierre Curie was thinking about magnetism and the fact that when you change the temperature, suddenly those materials become magnets and stick to your fridge, but at higher temperatures they don't. So what's going on? If you think of it at the microscopic level, it's awfully complicated quantum mechanics.

But the description that stuck, that made huge headway in terms of phase transitions and led to Nobel Prizes, many fields in mathematics, and so on, is a very simple model: the Ising model. You have essentially arrows on a lattice that are interacting with their neighbors to try to align. So a very crude description of the phenomenon was a good one to essentially think about this problem. I think that's also what we can try to bring to those questions.

Lastly, I think what you were describing—analogies like RG—is that we're physicists who try to think about problems at all scales, and to do it fast, we had to build analogies between different fields. So I think that's what we can bring.

When we spoke with Noam Chomsky, he was quite disparaging about the enterprise of physics. He was talking about the original hard problem of motion, and he said that Newton exorcised the machine—the mechanical-universe view—but he left the ghost intact. We still don't know how mind and consciousness work and all of that kind of stuff.

He was pointing to the notion that a lot of physics is idealization. Then there's an interesting question about whether our theories are really intended to understand how the universe works, or whether they're more to help us understand in terms that we can understand. Are we intentionally leaving something out?

Yes, I think both. Certainly, they are interacting with the world because theories in physics are super useful for building technology. Think about the laser. A big fraction of technologies are coming from theory, but another big portion is coming the other way around: technology is asking immense questions.

4. From Carnot to Chomsky bulldozer

So yes, I think we need theory also to have a sort of highway of thinking super fast about problems. The difficult question is, at which level of precision do you need your theory to be? That depends on the question you're asking.

Who were your mentors who inspired you? What books did you read? How did you land on your current trajectory as a physicist?

That's a complex question for me because both my parents are physicists. When I started to do my PhD, I tried to escape them by going into econophysics, doing more finance and economics. Already during my PhD, I started to be fascinated again by physics, how sand flows, and things like that.

When I was a postdoc, I was always mesmerized by our brain and how we think. I tried at that time to spend 1 year in Janelia Farm, a neuroscience institute, and I met lots of fantastic people and learned a lot. But I felt at that stage that a lot of the theory—how you have many connected neurons and what their dynamics are—was a bit of applied mathematics and detached from real function.

Then there was the question of how you learn intelligence, or rules, constraints, language, and so on. It wasn't at the level at which I really wanted to operate, so I gave up and went back to physics.

I think it's essentially the development of technology that's facing us with these questions. The analogy I like is the Industrial Revolution, when the heat engine emerged—another case where technology was first. Then you had to understand what was behind them, how efficient they could be, the limit to their efficiency, and so on.

Then Carnot, actually a French physicist, came up and wrote a beautiful text. It reads like philosophy; there's essentially no math. It introduced concepts like entropy, and it was the beginning of thermodynamics. Very deep ideas came from a technological fact.

Here, I think it's the same. With these machines, it's amazing. You look at them, and they're creative. You give them a bunch of images, and these diffusion models suddenly, like a painter, build new faces. They compose new faces. How can it be? Or they create sentences that they have never heard before.

Noam Chomsky and others said that it would be extremely hard to do. They do it. So how? Why? Being fascinated by questions—that's what drives me.

Sorry to bring Chomsky back again, but he said that large language models are like bulldozers. He says, “I love bulldozers. They're great for clearing the snow, but they're not a contribution to science.” He said, “I've got a theory: anything goes.” It explores all the laws of nature—anything that can be.

He says that when you've got a scientific theory, you have to explain why things are this way and why things are not that way. But you were just saying that when we discovered the steam engine, I think you believe that actually was a stepping stone to building theory.

For Chomsky, there's a huge difference between competence and performance. He had this wonderful expression about Deep Blue, the chess thing, and he said that's a little bit like a bulldozer winning a weightlifting competition. It's almost inconsequential. It's incoherent.

Yes. I respect him a lot. A lot of our work is actually inspired by him. The way we model data is sort of part of the classification he built.

But I think what he's saying is not wrong. It is not a theory, but it's obviously not wrong. That's not the point. To me, the point is that it's an amazing observation, and it's raising a bunch of questions.

You have a machine facing you that can learn to be creative. You can open it up, look at its artificial neurons, and ask how it's encoding syntax, semantics, and so on. How is it doing it? Even if ultimately the brain doesn't work like that, I would argue that, in some sense, I would love to think about the brain and have something to say about the brain, but I find intelligence in machines extremely interesting in itself.

So it's just like at the time of Carnot. We're faced with new technology, and it's asking many questions. I think physicists should be asking about the physics of it, and we're pushing physics departments to invest in those directions as well.

5. How deep networks recover hidden hierarchies

So, unfortunately, it is possible to make a system perform a function in many, many different ways, right? For example, LLMs apparently have linguistic competence, but it's different. You might just say, “It doesn't matter. If it does the thing I want it to do, it doesn't matter.” But I think we can be a little bit more rigid here.

One thing that seems to be missing is the acquisition of abstractions. Now, you've done some amazing work on this, but certainly when I use language models, what's abundantly clear to me—even when we do hill climbing and solve these mathematical problems—is that they traverse the spaghetti monster and get the right answer for the wrong reasons. They seem to be low down the abstraction mountain, whereas we have the ability to do this coarse-graining, or to use metaphor, and work higher up the abstraction mountain. It just feels to me that when we use stochastic gradient descent and learn from data, the models aren't acquiring these high-level abstractions.

That's the opposite. Essentially, I think those questions are super interesting, super deep, and we are approaching them as physicists. The way we're thinking about this question is that the world is very complicated. So let's stick to language, maybe. I think to understand how machines work, we first need to understand very well what we're talking about—what is the data that needs to be learned.

On that, I would say linguists have been one of the most impressive fields in terms of characterizing the data they're dealing with. In particular, Chomsky and others—I mean, he has argued for decades that underlying texts are trees, with different levels of abstraction. It's also been argued for images. It's called pattern theory, and we're very used to that in physics.

In physics, let's say you take a liquid. I told you about liquid before. You can describe it at the level of atoms, but if you have billions of billions of billions of atoms in your glass, that's not going to help you describe the glass very well. What we did as physicists was build coarse-grained variables—pressure, velocity, density, things like that—for this system.

It's sort of simpler because there is a single—I mean, there are 2 levels of description, if you want. Essentially, it's very microscopic and macroscopic, while real data have layered, multiscale levels of description. If you think about an image, you can think of it at a pixel level, at a very low level, and at a very high level, as the caption describing what's on the image.

You have many intermediary steps. At a low level, you could start to make edges and little geometrical figures from pixels, and at some point you could make eyes, a nose, and ears, and understand that they make a head. So you have many different levels of description.

The question you are asking—that's the one we wanted to understand. The first thing was: how do we model this? If you look at the most complicated context-free grammars, those are the sort of models based on trees that Chomsky introduced. Essentially, the idea behind those models is that if you want to describe the sort of linear objects that language texts are, you can describe them by some underlying tree. You have hidden variables living on those trees, and you describe the way those hidden variables can give rise to strings of symbols. Essentially, you're describing a generative way to make sentences in a recursive fashion.

But again, if you want to fit those models to English, it's very complicated. So what is a good model? Again, it depends on the question you're asking. There is no absolute answer to that. For the sort of question we're asking—the one you're asking, namely, in what sense do we build these coarse-grained variables?—we build context-free grammar models of data where you have these underlying trees.

But, like you as a physicist, we build a synthetic world. Once we build it, the game we're playing is that we have to believe it's rich enough. In our case, we want it to capture the fact that there is some hierarchical hidden structure to the world. We capture that, but we want to make it tractable.

Maybe it's a bit technical. In this case, we started the first models with a tree that was frozen in geometry, and production rules—which is how a latent variable gives rise to a string—were randomly chosen. Finally, randomness, although it's counterintuitive, often makes things simpler in physics. It allowed us to compute any correlation in this model, and from that we could understand how these models are learned by machines.

Indeed, if you have a poor machine, like a very shallow network, even these models would essentially be unlearnable. There would be a lot to say, and maybe I can come back to that. In high dimensions, it's very hard. But if you have a deep architecture, what we find is precisely the opposite of what you're saying.

The reason deep architectures can solve this task is precisely because they understand, just as physicists understood pressure, velocity, and fields, this hidden hierarchy from the statistics of the data. Otherwise, they would never be able to do this job. They understand that there is this hidden hierarchy, and from it they can perform the task that you want.

Just to play that back so that everyone understands: the idea is that there is—I mean, we're talking about grammar here, but more broadly we think that there are—

Structured generative processes in the world. Yes.

So we can think of those as being some kind of constrained generative model. We're talking about syntax here, and when we do machine learning, we look at the output of that generative model. The learning process ideally should be not to memorize the raw output, but to understand abstractly the model that generated it, because creativity is about respecting the deep structure and the constraints. If you have the structure, you can go on and generate many, many more things, and you obey the rules. You're creative, and all the rest of it.

So you're saying that you've done experiments. You've created a mathematical generative model that is parameterized. You can have as much depth as you want, and you found that on shallow networks it basically wasn't really learning any of the abstract structure. But when you have deep networks, it was—

Yeah, exactly. So maybe I can put it into the context of creativity and this discussion we just had with Chomsky. There's a question of creativity. I will use this term in a very narrow sense: being able to generate new sentences that satisfy hard-constraint syntactic rules, which the child would never have heard before.

Chomsky gave this poverty-of-the-stimulus argument, arguing that it was actually impossible to learn to become creative from examples. This would be a very crude way of summarizing his argument, but I described the fact that you have this sort of generative, tree-like, rich context-free grammar, really capturing the fact that the world has a hierarchy of abstract concepts.

But you have other possible generative grammars. Some are much simpler. One is called regular grammar. This will be a caricature, but essentially the idea is that maybe a group of words will fix the probability of the next word.

Chomsky's argument is to say that even if you give me 1 million sentences, I can fit those sentences with a context-free grammar, but I can also fit them with a much simpler—simpler in its classification—regular grammar. To fit those sentences, however, it would have to be awfully complicated, with many, many, many rules.

That's the debate between nativism and empiricism, and there are big debates on this question. Again, we felt like we wanted to address those questions as physicists. In our idealized world, where the true world is known, can a machine learn to be creative or not?

What we find is that if you have a shallow network, what Chomsky worried about is completely true. You learn some things, but you don't learn this sort of interesting generative grammar. You essentially memorize, and you can't do anything. But if you have a deep architecture, there's a huge implicit bias to build these coarse-grained variables.

This hierarchical architecture leads very easily to some iterative calculation. What we found is that, indeed, you can learn to be creative by being exposed to a very small number of sentences. Let's say in our model that if d is the size of the sentence, the number of sentences is exponential in d, but the number of sentences you need to see to be creative is only polynomial in d.

So these models are really a counterexample to his argument. Ultimately, it comes from the fact that machines have strong implicit biases. They are not comparing all hypotheses equally, in an equal fashion. If you're deep, you learn.

In some sense, I would argue that in terms of what needs to be innate, if you have a deep architecture, that does a lot of that. This is not to say that what he inferred is incorrect, right? It's not because I think an argument is incorrect that the statement is incorrect.

I don't want to imply that our brain is just a deep net and that there aren't much smarter mechanisms to learn much better. Actually, the brain can learn with 100,000 times less exposure to words than those machines. So I think there are lots of questions about how the brain works, and they are fascinating.

6. Where machine creativity still falls short

I think it's possible that you're both correct. Chomsky, even back in the 1950s—and he wasn't the first to do this—came up with these very basic transformational rules that could be composed together. That went on for quite a while, but they realized there were lots of problems and edge cases. Eventually, the Minimalist Program came out, and it was even more parsimonious: it was like “Move” and “Merge.”

That's very abstract, because I agree with you that these networks clearly have syntactic competence, which means they absolutely can create novel grammatical sentences. But in a broader setting, they're not creative because they don't understand many other abstractions in the world that we do, which is why we need to prompt them to be creative. They can render an image of a dog, but they're not creative enough to know what an interesting and worldly coherent image of a dog is.

Another thing I wanted to get to is the idea that networks, as we train them for longer and they get bigger and deeper, start to factorize. We have this notion that they have these fractured, entangled representations: they understand things at a shallow level; they understand some things and not other things.

I spoke with Tom McGrath at Goodfire. It's a big mechanistic interpretability company, and he's been studying networks for years. He says that as they get bigger and bigger, they become more factorized, and he actually believes they're converging toward some kind of natural factorization. At the moment, it's a bit weird that they have some factorization and then some fractionation in other areas.

You kind of see where I'm going with this, because we know that a neural network is a finite-state automaton, and language, in Chomsky's sense, is somewhere between context-free and context-sensitive. He wasn't super-specific about that, but we know as a mathematical fact that it's not a generative grammar in the abstract way Chomsky was describing. It is still a coherent generative grammar in some lower sense in those models.

First of all, what we find is that as you train the machine more and more, those factorizations or abstractions are created progressively. If you have an immense machine and you give it more and more data, then you start to play with more and more abstract concepts. Those are the hardest to learn from this viewpoint. That's my take on the first part of your question.

The second question was about the fact that transformers have finite depth. If you're thinking about sentences where you have 50 complements of sentences that are looping like that, it may be very hard to reproduce them and so on. But I think those are more academic problems that you never encounter in practice, because sentences that loop 50 times are extremely rare.

I'm not sure it's really relevant. I know that some people put a lot of attention on that, but I'm more interested in empirical studies. I don't know if those worries are actually relevant in practice.

Would you agree, though, that there is a spectrum of abstraction? It's a different type of syntactic competence, but maybe it doesn't matter—the performance-competence distinction.

An interesting example is that I feed my entire codebase into Claude Code. Isn't it interesting that it doesn't really understand what my intentions were? If I put it in a loop, put it in an agent, and say, “Just fix the bugs and keep evolving this software,” it doesn't respect the deep constraints. My mental constraints—what was I trying to achieve with this? What would I have done?

It has the syntactic competence. It knows how to write the code. Are you saying this is just a matter of the networks not being good enough yet? When they do understand, when they have a theory of mind and understand how the world works even more abstractly, eventually we could just autonomously create coding agents that would make Microsoft Word or something. There is a path to that level of competence.

Composition. Essentially, what we're saying is that imagine you have this diffusion model that learns to compose new faces. When it has seen enough low-level features, like noses, eyes, and mouths, it understands the rules of the game and composes them together.

What we also like about this description is that we can make nonlocal predictions. We test them with real images or real text, and maybe we'll come back to that, because I think it's a really important part of physics.

Creativity is not just putting pieces together that satisfy constraints. Although when you have a new idea, it's often putting existing ideas together into a new whole, I think creativity can be much more than that. If we think about what we discussed regarding what it means to be a physicist and how science proceeds, that's an example of creativity.

Think about Newton's understanding of the motion of planets and things like that. We talked about the dialogue between experiments and theory. We talked about building models at a good level of description. We talked about analogies, and I don't agree with you that all of that is in the machine.

I think there's no reason why we would not be able one day to build machines that can do that. I'm not sure if just scaling things up will lead to that. Maybe we need to do more introspection of how we function as scientists to come up with a good dataset and good procedures to teach machines to be good scientists.

That's just one example. Creativity in science requires a lot of abilities to create something really new and to interact with the world around us—abilities that I don't think machines have yet. So I think I agree with you that just scaling up will not lead to total success. We need to develop other abilities in those machines.

I think I'm just trying to understand what the gap is, because it would be consistent with your argument. If you're saying that we can learn the abstract structure of the world and be generatively competent in one domain, why would we not?

For me, creativity is not just about coherence and respecting the constraints. Transformative creativity, in my mind, is about discovering interesting new subspaces. We can traverse these constraints collectively and serendipitously; sometimes we happen upon fascinating new subspaces, and we go on to explore them. When we look back after discovering them, we think, “That was a very transformative, creative stepping stone.”

Yes, I think if you look at the history of science, it was very creative. Think about a mathematician inventing imaginary numbers, or Newton describing the motion of planets, or Huygens and something about waves and diffraction.

Humans have forever seen waves entering a port and being diffracted, starting to make more circle-like patterns. But if you're just showing those patterns to a machine, it would stupidly predict the next frame, because velocity will propagate.

What do we do as physicists? First, some people are really good at observing that something is intriguing. Then you have to simplify the geometry. Maybe you put it in a very simple geometry, and then you have to model it—that aspect of modeling that I was describing. How do we model this, and so on?

I think all that is very much needed. I don't see how you could learn it just by looking at everything that was ever written without enforcing those interactions. It's your interaction with the world, and simplifying the world requires an interaction.

7. How deep nets escape the curse of dimensionality

To recap, we were contrasting earlier the fact that Chomsky has this poverty-of-stimulus argument. He was essentially saying that it isn't really possible, realistically, with the amount of data—the sense data—that children have, for them to learn this grammar. Your paper demonstrated that it actually is possible, because you created this generative function and deep networks could learn it.

But I want to understand how. You said, for example, that the networks encounter ambiguity, and when there's a sufficient amount of data that has a bias in it, the network can suddenly grok it and learn this invariance. Tell me about that.

This is about how the machine actually builds those coarse-grained variables or abstractions. We looked at it in various cases: supervised learning, where you're trying to classify cats and dogs; generative models, like next-token prediction or diffusion models; and, very recently, perhaps smarter algorithms that are trying to predict in more abstract spaces.

Maybe I can start this discussion in the middle. Think about models like diffusion models or LLMs that are trying to predict very low-level tokens, pixels, or low-level features. Essentially, what we argue is that maybe there's an analogy again with a simpler algorithm, which is Word2Vec, introduced 10 years ago. It's a beautiful idea.

And so the idea there was: how can we build an interesting vectorial representation of the world? For each word, I want to associate a vector. There was a beautiful idea. What you can do is take this word and make a little machine with 1 hidden layer. You have neurons in 1 layer, and train this machine to predict the words nearby.

Essentially, it's a machine that's trained on co-occurrence—how often 2 words co-occur in the same sentence. Let's say it simply: if 2 words are synonyms, they will have a similar context, and this machine will represent those 2 words with the same vectors. Instead of the incarnation of those different words, you will just have the meaning. This is, for me, an example of a coarse-grained variable.

It's key because we talked about the fact that learning in a large dimension should be extremely hard. So it's really important that this machine manage, in some sense, to get rid of a lot of things it doesn't care about. To do that, it has to build those coarse-grained variables.

8. Diffusion, scaling laws and text entropy

Word2Vec is doing that at 1 level of abstraction, at a low level of abstraction. Essentially, what we're saying is that deep architectures—diffusion models or large language models—do exactly that, but in a recursive fashion. Once they have understood the meaning, they will group this meaning into supra-meanings.

Maybe let me give an example. Think about streets—the concept of streets. You have passersby, cars, sidewalks, and an immense number of possible streets, but it would be very useful to have a concept that groups all those different configurations together. That's the concept of a street.

If you think about LLMs or diffusion models, the way they build concepts is that they emerge from statistics alone. Those abstractions are there in the data, and they emerge if you group together configurations that predict similar contexts around them. If you have a street, typically you have houses nearby, and maybe the houses have colors or edges, so you would predict colors and edges.

In those models, at least, you find that if you have enough data, you can learn all the abstractions. But as you get more and more abstract, you have a problem, because you're always trying to build those abstractions by saying how predictive they are at a very low level. When you're very abstract, how you predict pixels or colors and so on is a super-noisy signal. That's why, in those models—we have empirical evidence, and I'm happy to talk about the empirical evidence—the more abstract concepts are the toughest to learn. Essentially, as you get more and more abstract, your signal gets diluted.

Yeah.

So, yes. This would be the mechanism whereby we think you build those latent variables or abstractions.

It's so tantalizing to think of them as being the 1 true abstractions. But we know, as we said earlier, that it's not like the Merge operator Chomsky was talking about. When we do mechanistic interpretability and look at how networks do addition, for example, it's super weird that they're composing trigonometric functions together. They're not doing it the way we can.

Maybe that's just a limitation of the architecture. Maybe if we had proper learnable Turing machines, they would converge higher up the abstraction tree. But I suppose a tangential thing that we need to talk about is this curse of dimensionality.

There has always been this statistical law, essentially, that when we have high dimensions, the amount of data that you need to make it tractable increases exponentially. There are all of these theories about why that's not the case. There's the manifold hypothesis, so the intrinsic dimension is lower. We spoke with Randall Balestriero about this. He's got this spline theory of neural networks, and he said that in high dimensions, all data is extrapolation. There's no manifold. It's actually doing this spline decomposition in an input-sensitive way. Lots of people have different ideas about this.

But you're saying that this kind of emerging behavior is actually how it becomes tractable?

Exactly. Before we started thinking about creativity, our first work was really trying to understand what sort of data structure allows deep nets to perform. So it's exactly as you said. Maybe I can say it again.

In physics, we know that a volume goes like a length to the exponent of the dimension. In 3D, it's L³; in 2D, it's L². L is a length. Think about a large dimension. If you think about an image, D may be the number of pixels, at least naively. If you think about text, it may be the number of words in your sentence, again naively.

Those volumes are huge. They're exponential in the dimension. What it means is that even if you give me 1 trillion points, because the volume is so huge, they're extremely far away from each other—extremely far away. If you have a machine that's just interpolating, and you ask a question about a new test point, you can prove mathematically that if the data has little structure—for example, you're trying to learn to regress some smooth function—it's hopeless.

The only way you will extrapolate and have the power to generalize is if you bring those points together. It means you have an exponentially large amount of data—more data than atoms in the universe. It's just impossible.

To me, this is a completely fundamental question. Sometimes in the literature, it's tossed aside by saying, "It's super-naive to say that the dimension is the number of pixels in an image. In fact, it should really be that the data lies on a lower-dimensional manifold." If you try to measure it, it's true that it lies on a lower-dimensional manifold, but this dimension is still large.

To me, the big problem is that if this was the answer to this question, it would mean that very simple algorithms, like kernel methods—they're ancestors of deep nets—or even shallow networks could do it perfectly. If you give them a low-dimensional manifold, you don't need to have any interesting architecture. But if you use those methods on text, I can tell you they fail lamentably. They do nothing.

The question is: why do you need deep architectures? Some of the things you said don't answer that question. That's really the question we're looking after. Essentially, the answer is that if the world is hierarchical, if it has those hidden coarse-grained variables, those machines are super-good at discovering them. They can discover them with a number of data points that's not huge—polynomial in the dimension, once again.

Once they discover them, it's a sort of summary of what the data is. Instead of describing things pixel by pixel, you can say, "There is a nose, there are ears," and so on. So you're reducing the dimension of the problem, essentially, and you can solve the curse of dimensionality.

I think this explanation has the advantage that, whatever explanation you come up with, it has to explain why you need deep networks.

9. Why predict latents instead of tokens

Yes. When I was speaking with Tom from Goodfire the other day, he was saying that so much of interpretability is going from essentially neural representations to text, trying to introspect about them. He thinks we could have a new type of training method where we go from text back to neural representations.

We discover these emerging modular structures, and during training we encourage them to become even more pristine and more evolved. But there are other people talking about similar ideas as well. Yann LeCun, for example, has this idea called a Joint Embedding Predictive Architecture, or JEPA.

This is very pertinent because you've got a paper out basically saying that we should predict in latent space, not token space. His idea, essentially, is that if we actually predict in latent space, then we can be significantly more sample-efficient than if we predict in the ambient space.

Tell me about that.

Yes. That's a question we've been fascinated by in the last 1 or 2 years. As we just discussed, the brain learns languages with much less data than machines. Machines are amazing; they speak better English than me, for sure. But in some definition of intelligence, they need much more data than us to do those tasks. So why are we so different?

There are many hypotheses, but 1 thing that's discussed in the field is the fact that those large language models, at the end, do something that seems a bit trivial. It's like you mask a token and try to discover it. Even to do that well, you need to understand the full hierarchical abstraction of the world.

Actually, we started to work on next-token prediction because I always had, for at least 1 decade, this sort of question related to the curse of dimensionality: How come, when we produce speech—think about the end of a sentence—maybe I said 30 words before, and the number of possible sentences is huge? How do I do that? Do I need to memorize those 30 words to do that? I mean, how is it possible?

Actually, those models gave a sort of elegant answer to that. What happens when you try to predict the next token is that, if you say a long sentence, maybe you have a coarse-grained variable that describes a coarse meaning of the first half of the sentence. As you approach what you're going to say, you have a finer and finer, more precise description. So, at least to me, this sort of way of thinking led to a possible solution for my paradox.

In any event, even if you try to learn the next token, you need to build those abstractions. But I told you that 1 problem with doing this is that, if you're very abstract, it needs a lot of data, because you build those abstractions by bringing together configurations in the data that predict a similar surrounding, but at a low level, like similar pixels around.

Going back, what has been proposed in the literature—actually, it's interesting also in neuroscience—is the notion that maybe the brain is doing some sort of very interesting self-supervised learning where, instead of just predicting what is going to be the next frame seen by its eyes, it's trying to predict the next activity of its cortex. It's predicting in some sort of abstract space.

These ideas also emerged in machine learning, and you talked about Yann LeCun. There are also other models, and they're extremely interesting. Again, the idea is: instead of predicting at the level of the token, can I predict in a more abstract space? This field has developed very interesting machines to do that.

You can think about Barlow Twins. You have 1 machine, duplicate it, and 1 machine is shown the entire data. It's a teacher, and 1 machine is shown some occluded version of the data. Your student has to predict not the tokens that were occluded, but how those tokens were represented by the teacher that could see them. It's beautiful, right? Those networks are doing some kind of introspection.

There has been a big debate about whether it's better or not, because after all, those LLMs are doing fantastic things. We felt that, since there was essentially no theory on sample complexity, we needed to think quantitatively about this question.

Again, with the same kind of model, the game we're playing is to develop a framework where, with a single viewpoint, you try to engage with many different problems: the curse of dimensionality, creativity, and now learning from your own latent. Also, scaling laws—maybe we'll talk about those.

In those models, what we found is that those algorithms that are introspective, that learn from their own latent, are much more powerful in terms of sample complexity. Eventually, they will learn the same abstraction, but much faster.

To build abstractions, you need to bring configurations together. Think again about the street. All those configurations—you need to understand that they're 1 entity: a street. What diffusion or next-token prediction does is that the signal they have to bring those together has to do with pixels around, very low-level features. I told you that this correlation between abstract things and things that are not abstract is there, it exists, but it's very noisy.

Imagine instead that, when you start—when you understand the concept of cars and passersby, and you also understand the concept of houses—those methods can build the concept of street by predicting that those configurations have not the pixels, the painting of the house, but just the concept “houses” nearby. Then the signal is much larger, and so you need much less data to extract the signal from noise.

So, yes, we do find that, to understand the hierarchical structure of the world, at least in those simple models, those methods are much more efficient.

Yeah. Many folks will know Yann LeCun's work in the vision space—Barlow Twins and all of these joint-embedding prediction architectures, roughly speaking—where you have something like a Siamese network and then you might do some kind of masked prediction. You might occlude tiles from 1 side, and you're learning this prediction function over the embeddings, the latents, rather than the ambient space.

But this also goes into his broader philosophy about energy-based models. The rough idea is that you can imbue domain-specific knowledge into a prediction architecture, and energies are composable. So you could be ridiculously specific and actually have variables that represent things in the domain.

What we're talking about here is something that's quite generic. It's a little bit like an inductive prior, which is not really domain-specific. So it could work for any type of vision, or it could work for any type of language model, and it's significantly more sample-efficient, as you just said.

But do we still have this issue that it is learning really good general abstractions? It's more efficient, but is there still something missing? We have these galaxy-brain abstractions. We can just select these meta-relations between things from a seemingly infinite set of possible relations. Is this just 1 step in that direction, but not all the way?

Maybe I should indicate a word of caution as a physicist. What I've been discussing so far was the same sort of theoretical approach, but we could test it empirically, make natural predictions, and test them. This last part, which is a paper that's 1 month old, is a theory that we are now testing. When I talk about it, I talk about it with caution.

I think it's nice. What we want to do as theorists is to have theories that are predictive—they make predictions. Being rigorous for us doesn't mean having a theorem. It's going back and testing those predictions. So we are in the process of doing that.

There is a very deep question there. What we find in those simple models is that indeed, you learn this abstraction with much less data. But now there is a question of whether those abstractions are represented in your machine and what you do with them.

From those representations, you can do tasks like segmentation in images or classification, and now you can be competitive with supervised methods on classification. So there is evidence that it's doing a very, very good job.

But, for example, if you want to compare them to large language models, we like them also because they are generative. We can talk to them. That's a question we're working on, and I still don't know the answer: Once I have discovered those variables, essentially I have created some sort of encoder of the world. Can I create, with not so many data, a decoder and build a generative model from them?

Can I really go back and say I can compete with those next-token-prediction models and build something generative? I don't know. So this is completely open to me.

You can build these sorts of very interesting representations of the world. In our model, we know what should be there, and we can check that it's there. But if you don't know where things are, how do you use this information to do specific tasks most efficiently?

10. The sample-efficiency case for latent prediction

To me, it's a fascinating field of study for the years to come.

In this recent paper that you just released about 1 month ago on predicting latents, not tokens, you should explain Figure 1. We'll put it on the screen now, but you actually visualize and have a kind of analytical explanation for why it's more efficient using latents and not tokens.

Exactly. What we like to do as physicists is also to have models where we can vary parameters, and then we make scaling predictions. It's very easy to test those predictions. You plot it in a log-log plot and see.

In terms of the conceptual picture, this figure has, I think, 3 networks. The first is supervised learning. In our models, the data are tree-like. There's a top root; maybe, if you think about images, this is saying whether your image is a cat or a dog or whatever. You don't see anything—those are hidden variables—and then you see just the data, which is the input.

One question would be: How much data do you need, from the input, to be able to classify the root of your tree? That's supervised learning.

And then maybe I will skip that for reasons of time. The central figure is more like diffusion models or next-token prediction, and what this figure is showing is really the concept I was trying to tell you. It's this concept of streets, and below streets you would have passersby, cars, and sidewalks. Think of the nodes below as passersby, and the node above as the concept of a street. What matters in those models is how you correlate this concept with very low-level aspects of your data.

As you go away along this tree, the correlation decreases, because every time you have to make several choices, and that leads to decreasing correlation. The middle panel would show that when you try to approach the top root, when you're trying to build an abstract concept, you're correlating with a very low-level aspect, and it's a long distance along the tree. We know that every time we move along this tree, we have to pay a multiplicative cost in the number of data we need. That's why the number of data you need to learn your hidden hierarchy is exponential in the depth of the tree, which is still good because, if you think about it, the dimension of the input is also exponential in the depth of the tree. It means you can learn that polynomially in the dimension of your problem, which is much better than exponential, which would mean impossible. Next-token prediction works, but it's still exponential in the depth of the tree.

Now, if you think about the last panel, what you would really see is that you can do something very different. Again, when you try to build the concept of a street, you will just predict what's nearby—houses—and you've already understood this concept of a house. The correlation is much closer on your graph, so it's much more correlated; the correlation is much stronger. You always have a signal-to-noise ratio: you need enough data to measure correlations, but if the signal is strong, you need much less data to measure it. Once you measure it, boom, you can build those abstractions. This is essentially what this figure shows. It's a summary of what we've been discussing in this example of streets and houses.

It's so frustrating that we know so many things that could advance the frontier, but OpenAI and Anthropic are still training old-school transformers. And I spoke to Llion Jones at Sakana, who was one of the inventors of the transformer, and he said any new method has to be crushingly better because we've invested so much time in hardware, optimizers, and compilers. There's an entire ecosystem around this, and it's actually very difficult to steer the ship.

Yann does have a couple of new startups, but the read I'm getting is that he's focusing on vertical domains. We haven't yet done the moonshot where we try these new models en masse.

Yes. So I agree. I will still say a word of caution: those LLMs are generative models. That's very important for them to be, because you can interact with them and they produce reasoning and so on. I still don't know, even theoretically, whether, even though I understood all this hierarchy hidden in the world, I can use it efficiently to go back to a prediction at the token level. But if we could know how to do that, it would mean, at least conceptually, that we could build a much better generative model.

You see, it's subtle, but there's a distinction between understanding the structure of the world, which is like building an encoder, and then decoding it for a very low-level aspect of the data.

You mentioned diffusion models, and you had a great paper out about that, but just conceptually, how do you think they are different from something like a transformer?

Well, often they are based on transformer architecture. So it's more the objective you mean? In one case, you are masking the future—you're predicting the next token—whereas in diffusion models you're masking randomly, at random positions. I think it's very similar, and in our theory, essentially, it's the same sample complexity for both. It's not that one has a huge advantage over the other; in some sense, the only difference is the order in which you're filling up what is being masked.

Oh, that's interesting. Intuitively, I think of it as: first of all, you have an arbitrary number of diffusion steps, and maybe you would say that's analogous to just doing more backward passes during training on a normal network. There might be some difference in that the diffusion model seems to learn global relationships first and then tightens up, whereas transformers and CNNs seem to have a locality bias.

I like to simplify the discussion by focusing on sample complexity—how much data do you need to learn? That's where we find analogous quantities, because you're incurring a sort of computational cost from going back and forth with this forward-and-backward process. I'm really talking about sample complexity, and what we find in both cases is that as you increase the amount of data, you learn those constraints, those grammatical rules, bottom-up: first low-level and then higher-level, in both cases.

Actually, we have some confidence in those statements because they make nontrivial predictions. For example, you would predict that as you train a diffusion model more and more to generate text, initially it would be random. If you don't have data, it's generating crap. As you increase the number of data, it should start to form coherent words. Then, later on, with more data, coherent groups of words, and then coherent full sentences. This is a prediction of those models: the sort of coherence of the context should steadily increase as you increase the number of data. We could actually check that for diffusion models, and also check the theory of scaling laws of next-token prediction.

About a year ago, you had a paper about scaling laws as well.

Yes.

As you increase the amount of data, increase how much compute you put in, or increase the number of parameters, your performance steadily improves. This observation by Kaplan and others had a huge impact for all of us because it drove the tech companies to just invest more and build maybe nuclear plants. It has a huge technological impact, but it's a bit embarrassing for us theorists that essentially it's not understood at all. Those scaling laws quantitatively have exponents in them—for example, describing how much better you perform if you multiply the number of data by 10—and there was very limited understanding of that question.

Just a few months back, with Francesco Cagnetta, Alon Ravid, and Surya Ganguli, we proposed a theory for this problem, inspired by those synthetic worlds I told you about, but detaching the essence of the lesson we learned from those models to really make quantitative predictions for natural languages. Essentially, the theory predicts that there is a simple recipe to extract those exponents. What's underlying it is the fact that if you give me more data, I can learn more abstract concepts, and more abstract concepts correspond to longer-range correlations.

At the end of the day, the 2 quantities you need to measure are, first, the fact that words or tokens are correlated, and that this correlation decreases as a power law of the distance between those 2 words. From that, you can measure exponents, and they depend on the language you look at as your dataset. You can measure them, and then there is another key quantity we argue is related to the entropy of text.

Entropy of text has been discussed already by Shannon in the 1950s. It's a beautiful question. Essentially, entropy is the logarithm of the number of possible words that you would have at 1 location, on average. What we argue is very important to look at, and that we could finally measure with LLMs or other architectures—and we find consistent results—is the entropy left after a sentence of n tokens. If you see n tokens, the more tokens you see, the fewer possibilities you have there. What is the entropy of that? In the toy model, it's a power law, and in real life, a power law is also found.

Essentially, what we argue is that with those 2 exponents, you can combine them in a way that we specify to get the training-curve exponent of LLMs acting on those natural languages, and it works very well. So we got very excited with that. Also, I have to say that, in addition, it's making nontrivial predictions in terms of how the loss should depend on the context that you give it, and also the number of data. So it's a function of 2 variables, and we predict it should bend in a very specific way, and we could test it, and we also observe it.

Can you give me some intuition on that? Often when we speak about Shannon, there's a graph people often see where, during a sentence, every single word reduces entropy. We're now talking almost at the population scale: when we have a huge corpus of text, entropy is just going down and down. What does that mean? Does that mean the problem is getting easier over time? Does it mean the models will just continue to get better, or maybe there'll be some phase change? What's going to happen?

So, I should be extremely careful, first of all, because we could test our theory at the sort of academic range, so it means 1 billion parameters, 1 billion tokens.

Okay. I told you that as you increase the number of tokens, this machine starts to use a context that’s larger and larger. We can visualize this context. So I can tell you the context scale for which our theory was tested, and it’s about 50 tokens, or 2 or 3 sentences. I think it’s great because there’s all the syntax and so on there, so there’s a lot of stuff.

But that’s where I’m confident—I think we have a very robust story that will hold true. The field has to investigate further and decide, but what we really have not done, because it’s not possible with our means, is test this theory for much beyond 3 or 4 sentences. So I don’t know if the mechanism we put forward still applies there, or if it’s something completely different.

11. The scientists we learn from and the mistakes we make

If I could explain deep learning to any scientist, dead or alive, who would it be?

That’s a question I’ve never thought about. I had a very strong bond with my dad, who was a physicist, and with whom we discussed a lot about science. At the end of his life, he was actually very interested in neuroscience. He thought about olfaction and things like that. So it would be him, yes.

Are there any things in your career that you’ve been completely wrong about and changed your mind?

I think it’s completely fine to make mistakes in science, but I think what’s very important is that once you’re convinced that you made a mistake, you state it, and it’s obvious for everybody. You don’t cling to it, even if you don’t believe in it anymore.

It happens, for sure. The game we’re playing as physicists is to propose models of the world and then make predictions. Already, we feel that doing that is our job. Then it needs to be tested, either by us or by others. Sometimes your prediction doesn’t hold because it wasn’t the right model, but that’s how we build understanding, hypotheses, and so on.

In this sense, yes, it often happens, but it should. That’s how science should work, in some sense: you’re making hypotheses, and then you really test them. I think it’s important that if you’re creative and take risks, you should make mistakes. If you never make mistakes, maybe it’s a sign that you’re staying a bit on the beaten path in science. Some of us want to explore the jungle. In the jungle, you can be wrong.

It’s been an honor having you on the show. Thank you so much for joining us.

Thank you so much. It was fun. Thanks.