大多数人对进化的误解｜Akarsh Kumar

I would just want to clarify: evolution is anything but random. Artificial life is really about understanding, as you said, life as it could be rather than life as it is. We need to understand the space of all possible intelligences, the space of all possible life forms that can be.

Conway’s Game of Life is a famous simulation by John Conway in which you have a 2D grid of cells, and they can either be on or off—alive or dead. That’s it. From this super-simple world, possibly the simplest thing you can think of, you get insane amounts of complexity that no one would have thought it could produce until you actually run it.

You can do counterfactual analysis in other universes. In what worlds do we get this open-ended creation of complexity and emergence, and in what worlds do we not get that? Why does this physics, when you run it, produce this amazing phenomenon, while this other physics produces just random noise?

Instead of hard-coding a single simulation, you parameterize a space of simulations, which is much easier. You just simulate it, see what happens, plug it into a foundation model, and see what the foundation model thinks about what happened. There are 260,000 rules, and we plotted all of them. We found that there’s literally a big island of solutions that are not open-ended, and a small island of solutions where all of the coolest simulations lie.

Some of them actually end up creating little patterns that move around and look like cells. This cell-like object is exploiting the physics in a way that creates a persisting object in this world.

Right? If you train them on calculus, they’ll learn calculus. You don’t have to train them on arithmetic first. It’s such a weird, alien way that we train these models. I really like to call them statistical intelligences, and they’re basically perfect statistically.

But this FE idea, in this Picbreeder line of work, is basically a different paradigm of intelligence that we’re trying to study. It’s a difference between a statistical intelligence and a regularity-based intelligence. You can see this in Picbreeder’s solutions, like the neural circuits that we find: the regularities are baked into the circuits.

We wanted to see what would happen if you used LLMs to evolve small assembly programs that compete against each other. There’s this programming game, invented in the 1980s, that people used to play. It’s called Core War.

What my program needs to do in order to win is basically inject an invalid operation in front of your program’s execution thread. It’s going to halt your program by crashing it. Can we create this open-ended evolutionary arms race of programs from this?

It’s like Game of Thrones, Turing edition.

Yeah, exactly. That’s a good way to put it. The LLM is not good at Redcode. If you just try to zero-shot it in this language, it’s terrible. But if you do this evolution and have a verifier, it turns out to be really good.

Evolution is probably the most powerful algorithm that we know for solving discrete combinatorial problems. If you solve even one part of a puzzle and hang on to that while you search for the others, you can turn your exponential search problem into a linear search problem.

We’re basically trying to search for cellular automata that go from self-replicating molecules—cells floating in primordial soup—to predator-prey dynamics to alien animals roaming around. You can imagine that if you did this with a big particle simulation containing quadrillions of particles, and you had the right foundation model and searched for this sequence of events, what would happen?

Maybe you end up with some physics rule for this big particle system that really does give you this entire sequence. You get animals out of it, just like what happened in our universe. The field of artificial life, I really think, is a very long-term bet on intelligence.

1. Life as it could be, not just as it is

The paper is ASAL: Automating the Search for Artificial Life with Foundation Models. Akarsh and his co-authors use vision-language models to find interesting simulations across Game of Life, Lenia, Boids, and more. Early on in the paper, you quote, “Artificial life is not just life as we know it, but also life as it could be.” What do you mean by that?

Artificial life is really about understanding, as you said, life as it could be rather than life as it is. If we really just wanted to understand life on Earth and how it is right now, we’d be studying biology, right? It’s the same reason that we don’t just study neuroscience to understand intelligence. We create AIs because we want a more mechanistic understanding.

Personally, I’m going to borrow a quote from Jeff Clune that I like a lot. If we want to claim that we understand intelligence or that we understand life, we can’t just talk about this specific form of life. We need to understand the space of all possible intelligences, the space of all possible life forms that can be. Only then can you talk about general principles for what governs intelligence or what governs life, right?

You can’t just look at one car and say, “This is how all cars are.” In the same way, you can’t just look at GPT-5 and say, “All intelligences are like this.” You can’t just look at the human brain and say, “All intelligences are like this.” There’s a space of things, and we need to study the whole space.

2. Why the order you learn things in matters

If you look at how humans and animals learn, they have a natural curriculum. They’re like, “I’m going to learn this, then this, and then this.” There’s no other way. But if you look at how our models are trained, you can train them on anything and it’ll just work.

A lot of people think that’s fine, and that maybe it’s better than how humans learn. That’s a possibility. It could just be better, right? But there are reasons to believe that there is something fundamentally wrong with that, or that the universe doesn’t work like that. The universe doesn’t work as if you can just learn anything at any given time.

Thus, I think the fact that humans learn in this sequential order and evolution works in this path-dependent way is the reason it creates such nice, regular structures in the representations that it uses. That’s the reason it’s so robust, adaptable, and generalizable.

We see this cashed out in the form of curriculum learning.

Which is this idea that we learn step by step and with increasing complexity.

Right. You were just pointing out that when you learn step by step, increasing the complexity, the representations you learn are better. But how would you describe that in a no-nonsense way? What’s the intuition?

The curriculum is not the only thing that matters, because people have been doing curriculum learning in RL, and I was obsessed with curriculum learning in RL. I think the key thing is that—and this goes back to the “Why Greatness Cannot Be Planned” lesson—you need a serendipitous curriculum, which is not preplanned, like, “I’m going to do this, then this, and then this.”

It has to be much more serendipitous and embrace the chaotic nature of the universe. You have to embrace the idea that it’s not just about going down this one curriculum, but about any possible curriculum that might lead you somewhere interesting. That’s just a shout-out that you need serendipitous curricula.

To answer the question of why curricula actually end up mattering, I think that’s a fundamental question that I’m really interested in answering. I don’t know the short-form answer, but it’s a fundamental property of our universe. If you want to capture, as you like to say, the universe by carving it at its joints, I think you need this process of complexification, where you build regularities on top of other regularities in a sequential way.

I’m not exactly sure why it has to be like that, but I know that natural evolution works like that. I know that a lot of theories, like assembly theory, work like that. That’s an evolutionary theory in biochemistry and related fields. Human learning also works like that; it works in a sequential way.

I’m not exactly sure why it works like that, but it definitely seems like something fundamental to our universe, and we really need to understand it better. If someone could answer why that is important, or prove that it’s not generally necessary, that would be a huge discovery.

3. No shortcuts: Wolfram, novelty search and regularisation

It would. The only analogy I can think of is Stephen Wolfram. What we’re talking about here, mapped onto his ideas, is that we need to have a constructivist approach to representations, which means that we build them step by step.

He noted that there’s this phenomenon called computational irreducibility: basically, there are no shortcuts. I guess the way that cashes out in my mind is that if you want to acquire a complex representation, you can’t actually jump ahead. You need to synthesize these building blocks, and you need to take the path.

Yeah. One of the things I like whenever Ken explains his philosophy is that whenever he says, “Oh, we need novelty search. We need this thing,” people say, “Oh, but that’s just searching in the dark. That’s random.”

It’s actually anything but random, because randomness is just doing random actions.

With novelty, you know what you did before, and you want to avoid that. So I guess what you're saying is that if you want to build—if you want to respect—the path of where you previously came from, whether it's by avoiding it via novelty or because you're respecting the symmetries that you learned along the way in a representation, you do need to respect it. That gives you an idea that constrains where you can go. The question is, how much does that constrain you? There's a good chance it constrains you a lot, and it constrains you in the right way, towards the right path.

That's why whenever I think about the FE hypothesis and this line of work, I really like looking at it through the lens of regularization. People talk about regularization in deep learning with weight decay or some other regularization mechanism, and I love that. I think everything is just regularization. The data is a form of regularization. It regularizes what you need to model the world. That's basically molding the clay in a way that you need to model the world.

L2 regularization, like weight decay, is another form of that. Basically, it's saying you want to find solutions with low weight norm, right? And then I think that this curriculum learning—you want to respect the symmetries—is just another form of regularization, but I think it's extremely, extremely, extremely strong regularization that you're subjecting this model to. And maybe it's the right regularization that we really, really need.

4. Kenneth Stanley: the path matters, not just the destination

And here's a quick refresher. Our Fractured Entangled Representations documentary that we published last year explained why the path is more important than the destination when it comes to representations in neural networks. This is Kenneth Stanley.

5. Statistical intelligence vs regularity-based intelligence

This hierarchical locking tells us something really important about how representations emerge. We think it's about finding the right building blocks now, but weirdly, it's about making future discoveries more likely. It matters not just where you get to, but how you got there, you know. And that's something missing right now, because we tend to just care where you get to. We look at the benchmark score in the field, and it's like, that's the result. It's doing really well; it just passed the math Olympiad. But what if it matters how you got there?

But that kind of structured evolution, I think, is a completely different paradigm from doing it the statistical way.

Exactly, exactly. And I think you said the right word: it's a paradigm to understand intelligence. But this FE idea and this Picbreeder line of work, it's basically a different paradigm of intelligence that we're trying to study. We're not studying it from the standpoint of statistics. We're studying it from the standpoint of regularities, right? So it's the difference between statistical intelligence and regularity-based intelligence.

I really like the idea of statistical intelligence. I think we did a good job going down this path. We should still go down this path, but we should also not put all our eggs in one basket and really try to understand this other thing, because there's lots of evidence that this other thing has lots of potential here, right?

And it really ties into all the criticisms of deep learning. They're basically talking about regularities. Whenever people talk about how we need symbolic processing, we need geometric deep learning, we need more inductive biases—and people have all these different fields, whole different fields on what we need, right?—they're all forms of adding regularities and symmetries to the system.

What I really like about this FE approach is that I like to think about it as an automatic symmetry-learning algorithm, right? People like CNNs because they're translation-invariant, right? People like transformers because they're permutation-invariant, and they encode certain symmetries, and that's great. I mean, we need permutation invariance for set operations, right?

But can you code lighting invariance into your model? Lighting invariance—if you want your CNN or image model to be invariant to whether it's nighttime or daytime. There's no equation in the world that'll probably get you there. Maybe there is in the future, but it's impossible for me to imagine that you could come up with a mathematical inductive bias that's going to guarantee that, right?

So what you really need is some sort of data-driven symmetry-learning thing that automatically respects it, and it forms neural circuits that naturally have that inductive bias built in, right? You can see this in, like, Picbreeder's results. It's not that the inductive bias is in the hypothesis space of things we're searching for. It's in the solutions, like the neural circuits that we find. The regularities are baked into the circuits.

6. Artificial chemistry and other possible universes

Well, I suppose another way to look at it is with human intelligence: our knowledge acquisition process is grounded in the world, whereas when we build these artificial life simulations, they're not grounded in the same way, right? But they do have these properties that we're just talking about. So they're actually building these representations, and interesting things happen when you look at the phenomena of how these things evolve, and you get certain things that just fall out of the fact that you're doing this iterative computation, even in an undirected way.

Yeah. So I want to touch on one key point that you said: you said it's not grounded in our world, right? And that's one of my favorite things about artificial life: you can ground it however you want to, in whatever artificial world you want to, right?

People often think about artificial life as if it's only talking about life. The reason I like it is because there's a subfield in artificial life called artificial chemistry. Instead of using the normal hydrogen and helium, you can change how they interact and see what happens, right? So that's the power of artificial life. You can do counterfactual analysis in other universes, right?

And I really like this idea because artificial life is not only about life, but it's also about studying what happens in general in artificial physics and artificial chemistries. Can we study the emergent properties in each of these different universes? In what universes? These are the questions that I really want to focus my research agenda around: in what worlds do we get this open-ended creation of complexity and emergence, and in what worlds do we not get that, and why is it?

And this is the key question: why is this parameter—why does this physics, when you run it, produce this amazing phenomenon, and when you run this physics, it doesn't do that? It produces just random noise or something, right? And that was the key reason I actually worked on Automated Search for Artificial Life, or ASAL, because I wanted to answer that exact question directly.

But ASAL was trying to basically parameterize a space of intelligences, find the good ones, find the bad ones, and do simple stuff like interpolate between them and see what happens, right? Interpolate the parameter space between them, run the simulation for each intermediate point, and just see what happens. Just simple scientific analysis like that.

7. Convergent patterns: shadows of the substrate?

Yeah. And another thing that jumps out to me is that you get these convergent patterns in rule space. We use this term emergence, and maybe we should talk about that as well. We don't necessarily mean emergence in the strong David Krakauer sense, but weak emergence is just that when you sequentially apply these rules many, many times, you see these very surprising, unpredictable phenomena.

These phenomena have characteristics which seem to be shared in rule space, like certain types of gliders, objects, and patterns. And of course, Wolfram believes that the universe itself is computational, but it's almost like we're seeing shadows.

What do you mean by that? You're seeing shadows of, I guess, a substrate that the universe is running on.

Well, I mean, it's only a hypothesis. We see these convergent patterns in different parts of rule space. Now, the skeptical person might just say, okay, well, those are just design biases. They are just features of the way that you are doing this computation. They are features of the structure of the Game of Life.

Maybe because you're using a grid or you're using a certain type of program interpreter to do it, those things are influencing these patterns. But they might also be hints of just something else. They could be shadows.

I kind of agree with you because in my ASAL work, I spanned the space of all possible 2D cellular automata, for example, right? And in my case, I had a lot of diversity. In one world, a glider looked like a Conway's Game of Life glider; in another world, the minimal glider looked like a train going down the tracks, right? And it left a long path.

8. How Conway's Game of Life works

So in different worlds, it can be different. But I guess what you're talking about is that when they're all the same in all the different worlds, that points to something fundamental—not only about the world that you're in, not only about the rule that you're operating on, but about the entire substrate that you're working in. Maybe all possible rules in that substrate have this type of thing.

Maybe we should just talk a little bit about some of these simulators, just so that folks in the audience understand what we're talking about, just to bring this to life.

Can you go through some of these simulators and explain how they work?

Yeah, I can go through some of the simulators. I’ll start with the simplest one: Conway’s Game of Life. Conway’s Game of Life is a famous simulation by John Conway where you have a grid of cells—a 2D grid of cells—and they can either be on or off, alive or dead. That’s it.

It has super-simple rules. If I’m dead and 3 of my 8 neighbors are alive, then I come to life. There are 2 or 3 rules like that, and that’s all you need to know. Then you start with a grid of random dead or alive cells, or you can start with some pattern that humans have created, and you run this simulation over many time steps.

From this super-simple world—it’s possibly the simplest thing you can think of—you get insane amounts of complexity that no one would have thought it could produce until you actually run it. Now there are whole Discord servers where people share what they’ve found. They’ll say, “I found this cool spaceship that oscillates, a repeating pattern that repeats every 64 time steps and moves at this speed.”

I really love this community because they talk about how fast the glider moves. They realized there’s a speed of causality in this system, which is 1 cell per time step. That’s the speed at which information can propagate, so they call it the speed of light, C. They use their gliders and say, “This goes at C/16,” just like in real physics.

It really is an artificial universe. A lot of people seem to think it’s a toy system, but the point is that it’s toy-like enough for us to study. If it were super-complex or the real world, we wouldn’t be able to study it.

I do think it’s a very interesting quest. One of my quests is to create more compelling universes that capture some properties of our universe, like emergence—the idea that you can build up global patterns from local interactions. But there are other things you want to capture too, like self-organization and open-ended evolution. A lot of my research is about trying to find simulations that have all those properties we care about, and that was the point of ASAL as well.

9. Change one cell, change everything?

So how structured is a space? Many of us have seen visualizations of representation spaces in neural networks using t-SNE and things like that, where we see islands and clusters. Is it the same for something like Conway’s Game of Life? If you change 1 neighboring cell, is it like a wormhole where you get transported into a completely different part of the space, or does it make a proportionately small change in the emergent high-level domain?

10. Lenia (with Bert Chan) and neural cellular automata

Yeah, exactly. That’s a great question, and it’s one of the things I really wanted to study. In the initial state of Conway’s Game of Life, it’s very chaotic. I’ve had examples where you change 1 cell—1 pixel—in a 64 × 64 grid. Beforehand, I used this in one of my talks: it basically runs forever. It doesn’t run forever—I shouldn’t say forever—but for tens of thousands of steps, it keeps moving and doing interesting things. You change this 1 thing, and it stops. It basically halts into an oscillating pattern in less than 100 steps.

In the initial state, it’s very chaotic. But the interesting thing is that this isn’t necessarily true for all simulations. In ASAL, we were really interested in searching over the physics—the actual rules that dictate the dynamical system. What I call the physics is different from the initial state. There are 2 things: the initial state and the physics.

If you change the physics, Game of Life seems to be a little more robust. There are basically 18 possible rules that you can toggle on or off, and if you change 1 of them, it doesn’t impact the system too much, in my experience. But there are some exceptions to that.

Very interesting. Talk about some of the other simulators as well.

One of the simulators we used in our paper is Lenia by Bert Chan. He created this continuous generalization of Game of Life that generalizes to 3 channels, which you can map to RGB. They’re basically continuous differential-equation update rules that dictate how the system operates.

Here is Bert Chan, the creator of Lenia, on what happens when he made the rules continuous.

So, by making everything continuous, what will happen when I run this system on my computer? Something creepy happened: a pattern is creeping across my screen.

Another cool one is neural cellular automata.

Yeah, by Mordvintsev. Are you going to interview him?

Hopefully.

He famously released the 2020 paper “Growing Neural Cellular Automata.” My God, we’ve talked about this on MLST millions of times—the thing with the gecko. It was showing emergent tissue organization and morphogenetic engineering. Essentially, you train a CNN, and because you can think of it a bit like a convolution operator, you learn the parameters via stochastic gradient descent.

It was trained emergently to reproduce an image, like a gecko.

Yeah. You’re training the local rules so that they reorganize and self-organize in order to form the lizard.

Yeah, and then you can delete parts of the lizard. Because this thing is running continuously—it’s just convolution, convolution, convolution—it reconstructs the lizard. Again, this is what we want in AI systems. We don’t want impostors. We don’t want to learn the lizard; we want to learn the ability to reconstruct the lizard from the building blocks—the path-dependency thing. It keeps coming up again and again. That’s so important, rather than just learning what the lizard looks like.

I guess, to build on top of that, you don’t really want to learn the intelligence. You want to learn the process that grows the intelligence, in a sense.

Yes. It’s similar to that. That’s one of the reasons deep learning took off in the first place: we’re not hard-coding all this knowledge about the world that we know. We’re letting it learn from the data itself. There are other problems with that, maybe.

11. Boids, Particle Life and cell-like creatures

Going back to neural cellular automata, for the audience, it’s basically that you have cellular automata, but the dynamics rule is a mapping from a 3 × 3 patch to 1 cell. That’s what the dynamics rule tells you: how to go from 1 time step to the next. What they do is replace this with a small neural network that dictates it. That gives you a whole space of possible physics rules that you can search over. I really like neural cellular automata.

I can briefly go over the next 2: Boids and Particle Life. Boids is a seemingly simple algorithm inspired by how fish and birds flock in nature. I actually just went to the Osaka Aquarium and was watching these fish. I’d seen these fish in person, and they were always together. There was a whale shark coming out behind them, and they all steered to the left a little bit. I thought, “How did they know how to do that?”

Only 1 of the fish knew about the whale shark, and it pulled the other ones to the left. The others all realized, “We should go there.” It’s an example of collective intelligence in nature.

Boids is basically inspired by that. You have these bird-like objects that flow through 2D or 3D space, and they have super-simple rules. They look at their neighbors: “I have a neighbor to the left and to the right.” Let’s say I have 4 neighbors I can see. I’m going to make a decision about whether to go left or right based on their positions. That’s it. If you make the right decision and you’re all making the same decision, you get the emergent flocking behavior that we see in nature.

We parameterized this with a small neural network and searched over this decision network. If the Boids make different decision rules, how does the global behavior of the collective change according to the local decision rules they’re making?

Particle Life is super cool because, in grid-based simulations, it feels like this spatial thing. Particle Life is basically saying that we’re going to have a 3D Euclidean space or a 2D Euclidean space, and we’re just going to throw particles at it—just vanilla particles.

But their physics rule isn't going to be Newtonian mechanics or anything like that. It's just going to be whatever we want it to be, right? In this case, in Particle Life, I was inspired by Tom Mohr. He has these great YouTube videos about this kind of thing.

Basically, you have this set of particles, and each particle can take on 1 of 6 types. These are dictated by their colors, and then you can have this 6 × 6 matrix, which tells you how different types of particles interact with each other. It could be that red and red repel, but red and orange attract, and orange and green are neutral. You have this 6 × 6 matrix, and if you change the values of this physics matrix, this is the physics of the world, right?

You can call it a periodic table, or whatever you want to call it. As you change this physics, how does the actual simulation change? Some of them are super bland and boring, and I don't want to look at them, but some of them are evolving, moving, and doing interesting things. Some of them actually end up creating little patterns that move around and look like cells.

You can literally see them. They have a cell membrane, which is like this red coating, and inside they'll have purple and yellow in a triangle, and that's like their insides. Basically, this is a simulation: This cell-like object is exploiting the physics in a way to create a persisting object in this world.

12. Persistence, entropy and what life is

And you just interviewed Blaise. This idea that life is just persistence, right? It's just the idea that life is basically about persistence. These organisms are trying to figure out how to persist into the next time frame, and they've exploited the laws of physics in order to do so.

Yeah. Blaise was saying that this persistence is kind of a natural phenomenon. It's what I was talking about with these natural convergent phenomena. That might be an example of one, because even though these systems are basically ungrounded—we're just running this computational simulation—isn't it fascinating that we see the emergence of persistence, that things just want to keep their form over time?

Yeah, the emergence of persistence is very fascinating. It doesn't happen in all simulations; it only happens in some. But I think this points to the broader point about what life is and what the second law of thermodynamics tells us.

13. ASAL: a foundation model as the critic

There's a generalization to persistence, which is this. At least, in my mental model of how I think about it, the second law of thermodynamics, life, persistence, and evolution, I should say, are all basically saying the same thing. They're talking about what exists right now and what is likely to exist in the future.

The second law of thermodynamics is saying that you're more likely to go toward higher-entropy states. That's just how it operates if you look at the macrostates and microstates of any simulation. But if you look at it from the standpoint of evolution, what is more likely to exist in the future? Self-replicators, because they'll naturally exist more often in the future versus non-self-replicators.

So they're really—people make it out to be like, “Oh, life is fighting entropy.” They're really 2 sides of the same idea of persistence, or what is likely to exist in the future.

Yeah, fascinating. So, in your paper, essentially, you are using foundation models as a critic, right? You're saying, “I want to have a principled way for searching this space.”

Yeah. We used it in 3 different ways. At a high level, I guess the field of artificial life is a very cool field, but often people only simulate single simulations and try to draw conclusions from them. I feel like it's cool, but you really need to be studying the space of simulations, right?

I think what our algorithm does is that, instead of hard-coding a single simulation, you parameterize a space of simulations, which is much, much easier. It's the difference between coding up the periodic table—how does each element interact with each element?—versus just saying that there are going to be 100 elements, and I don't know how they interact; you figure it out, telling the algorithm to search for it automatically. It's a much easier problem to just say there are 100 elements.

Once you have this space of simulations that you've described, you can search for what you want in an automatic way using a foundation model. The reason I like ASAL is because it captures this principle of computational irreducibility. You don't need to know what this physics rule is going to do and try to predict which one is the best one. Just simulate it. See what happens. Plug it into a foundation model and see what the foundation model thinks about what happened, and then go back.

The key thing is that here you're letting the simulation unroll, and you're not trying to predict what's going to happen, right? You're embracing this idea of chaos and computational irreducibility.

Yeah. And in a sense, it's a step past Mordvintsev's idea with neural cellular automata, because he was doing this emergence-based optimization where you're saying, “When the macroscopic thing emerges, I want it to look like this specific thing.” And you're now plugging in a foundation model and saying, “I want to ask the foundation model whether it resembles something I'm interested in.”

Yeah, exactly. And there are 2 reasons it's better. One, it's easier to specify what you want. You can just be like, “I want a cat.” I don't need to have a picture of a cat and supervise it in a pixel-wise manner. I can just say I want a cat in a general way. That's one way.

But importantly, it also makes the optimization and the search easier, because it's really dang hard to find a self-organizing rule that creates this particular image of this particular cat. If you do that in Conway's Game of Life, that's not going to happen. But if you just say, “I don't want that specific cat. I just want a cat. Give me any cat,” that's a much easier space of things that satisfy that problem, so it's much easier to search for.

We experienced this firsthand. In the open-ended simulation, we're not searching for particular configurations of states that we want. We're just saying we want some open-ended simulation. It can be open-ended however it wants to be. As long as the foundation model thinks it's open-ended, it'll work. It'll do.

The cool thing about why we use foundation models is not just because they're the newest and greatest thing that you can use. It's because we really care about what a human thinks about these simulations. It's really hard to describe what complexity is, what emergence is, or what it means to have a very interesting system. You can't really mathematize that easily.

And if you did and optimized for those simulations, you'd find Goodhart's law. You'd find something that you asked for, but not something that you wanted, right? So we really want what a human thinks about this in this subjective, vague way. We can't put a human in the loop because that's too expensive.

So what's the next best thing? You try to find a representation space that matches that of a human. There's a lot of evidence, statistically at least, that these foundation models are matching human representations in a statistical manner, right?

14. Impostors inside the simulation

Yes. So I'm going to show a graphic on the screen now. This is the first experiment you did, when you had a supervised target, and we've got a multicellular organism, a network of neurons, a virus, and a fungal colony.

In every single row, we can see what the emergent phenomenon was for Lenia, for Boids, and for Particle Life. But it also leads to the question: There must be some degree of ambiguity, because there are so many possible places in computational space where the foundation model would say that this was a match, and it's represented differently in the different simulators.

We had this concept—we were talking about this impostor. There are many different paths to produce the phenomenon. Would you call these things impostors, or do you think that they're different?

So these are—that's a good question. You're basically asking, let's talk about the Boids one. You're basically saying: Is the decision-making rule of each Boid an impostor, or is it the real thing that gives you the real thing?

Yeah. And what would an impostor mean in this simulator?

Yeah, that's a great question. I think the idea of an impostor—I like to ground this idea in its performance on future tasks and how it adapts to things we want it to adapt to. That's how I like to coin that. I don't really like to ground it in modularity or something specific like that. I like to ground it in a set of future time—

When you continue to run the simulation, does it decohere?

Yeah, in this case, they're pretty robust to that sort of thing. But in this case, I mean, it could be that you can imagine that if these Boids were really robust or, like, good, nicely factored, then small changes to their network would change the global behavior of the collective in nice ways, where it goes from a spiral to triangles or something—maybe not triangles, but some other cool shape.

But in this case, they don't do that. They actually degrade if you sweep the parameters; then they become basically random.

So, in that way, it kind of is an impostor version of this spiral.

Exactly. Exactly. And the thing is, in this case, they’re evaluated on whether they persist into the future of this simulation. In that way, they’re very, very well structured because they’re able to persist indefinitely. You could run this simulation for probably millions of time steps, and they’d still be doing what they’re doing.

But if you wanted them to do something else—make a triangle by sweeping some weights—then they would fail in that way. To be fair, they’re not trained to do that in the first place. Their evolutionary lineage doesn’t select for it; there’s no reason for them to want to do that, right?

15. From primordial soup to alien animals

I want to point out, too, that we had this figure in our paper that showed what the future of this direction of searching for simulations via foundation models could look like. We had this fun example, and I have it pulled up right here. The bottom one is particularly fascinating. We’re basically trying to search for cellular automata that go from self-replicating molecules—cells floating in primordial soup—to predator–prey dynamics, to alien animals roaming around.

This is just a fun toy example that we used. In this case, a cellular automaton just memorized that sequence of frames. But you can imagine that if you did this with a big particle simulation, with quadrillions of particles, and you had the right foundation model and searched for this sequence of events, what would happen? Maybe you’d end up with some physics rule for this big particle system that really does give you this entire sequence: you get animals at the end, just like what happened in our universe, right?

Yeah. But isn’t it fascinating that you’ve taken something which you previously argued was a fractured, entangled representation? The foundation model has fractured, entangled representations, but they produce impostors. You only need the impostor because all you’re doing is acting as a critic on the impostor, and then using that to bootstrap the creation of something which is a factored representation.

Yeah, yeah. It’s a good point, because the performance of these foundation models for this type of task will be flawless, right? This is exactly the type of thing where you only care about the output behavior. You don’t care about its internal representation, and you don’t care about creativity, in a sense. The humans are the ones injecting those prompts, and you’re using this statistical intelligence to bootstrap a real, natural, organic particle simulation which should have all the properties of our real world, right?

16. 262,144 rules and the island of open-endedness

Yeah. Exactly. And then the other experiment you did was creating a diversity search. You said some parts of the computational space are open-ended. So there’s this Goldilocks zone you were speaking to before. Maybe we’re in such a region of the computational space where all of this lifelike stuff happened and this open-endedness happened. But did you see similar things in the space, where these properties we’re talking about existed in a sliver of this computational space?

Yeah, definitely. If you mean that, then in the computational space, I guess we can talk about the laws of physics. That’s your space, right? In that way, definitely. I have it pulled up right here. I found simulations that look more open-ended than Conway’s Game of Life.

Yeah, that one. That exact one.

Yeah. Basically, we found simulations that are more open-ended than the Game of Life, right? But we wanted to go further than that and ask: What does the space of possible rules look like? What rules are open-ended, and what rules are not?

If you scroll down in the paper, we had this graph where we basically tried to plot all possible rules. There are 260,000 rules, and we plotted all of them. We colored them—we ran the rules, put this image into CLIP, and plotted this CLIP space in 2D—but we colored each of the points based on how open-ended they are.

We found that there’s literally a big island of solutions which are not open-ended, and there’s a small island of solutions where all of the coolest simulations lie. In that way, this graph outlines exactly what we’re talking about: the space of possible simulations, and what part of this space is like that. You can see that there are different islands: one big island, one smaller island, and one tiny island with a single simulation that’s very different from all the rest in its behavioral properties.

17. From artificial life to AGI

But then we have the problem that there seems to be a disconnect between building these artificial-life simulations and wanting to build a general-purpose intelligence which has utility. So how can we leverage this type of technology to build AGI?

Yeah, yeah. So that’s the grand question, right? That’s how I like to go about it. The field of artificial life is a very long-term bet on artificial intelligence, but there are 2 caveats.

One, it’s still interesting to study because it gives you lots of insights about our universe, emergence, and all these other scientific questions that we still care about even if we don’t create intelligence. But, second, I think Ken talks about this a lot: we don’t really need to simulate every possible particle at a lower level and get everything to bootstrap.

18. Core War: Game of Thrones, Turing edition

What we really want to do is create an abstraction of natural evolution and an abstraction of these simulations, and find an abstraction which is, one, ultra-efficient—something we can run 10 years from now—and, two, gives us an abstraction in the right way, where you get what you want, which is general intelligence. That’s an open problem, and that’s one of my research agendas: finding the principles that we can extract from natural evolution and artificial life in order to create better AI systems.

We wanted to see what would happen if you used LLMs to evolve small assembly programs that compete against each other. I can give you some background. There’s a programming game invented in the 1980s that people used to play called Core War.

The idea is that there’s an assembly-like language called Redcode. I create an assembly program, you create an assembly program, and everyone creates an assembly program. We put them into a virtual machine, place them at random memory addresses in this virtual machine, and just let the machine run. It’s going to run the programs, and they can do whatever they want. It’s a Turing-complete language, and the goal is for my program to be the last one running.

What my program needs to do in order to accomplish that is inject an invalid operation in front of your program’s execution thread. When it does that, it halts your program by crashing it. Your program crashes, and then my program is the last one running, so I win the game.

In order to do this, your program needs to defend itself against other people attacking it and attack other people. It’s a really cool game that you can just run. We saw that people had been making these human-written warriors for such a long time, so we asked what would happen if we used LLMs to evolve these programs and have them compete against each other. Could we create an open-ended evolutionary arms race of programs from this?

That’s amazing. It’s like Game of Thrones: Turing Edition.

Yeah, exactly. That’s a good way to put it.

19. LLMs as the mutation step: evolving warriors

Well, tell me more. How many generations were there? What was the population size? How did you set this thing up?

The actual algorithm is really simple. We start off with a program. This can be a human program, or it can just be an LLM-generated program. Then you optimize a program with an LLM as the mutation operator in an evolutionary loop to beat this program.

We actually ended up using MAP-Elites because the space that you’re optimizing is very deceptive and very nonlocal. If you just use a vanilla, greedy genetic algorithm, it won’t work. You need to use MAP-Elites.

Then we end up with a warrior. These programs are called warriors in Core War. You end up with a warrior that beats your initial warrior. Then you can optimize a third warrior to beat both of these two, and then optimize a fourth warrior to beat those three. You just keep doing this, and you end up with a long sequence of warriors.

The question is: Where do these adversarial rounds of evolution take you? One of the cool results—I guess we have 2 main results—is that in the first round of training, we can already beat 96% of a dataset of human warriors. There’s a dataset of around 300 human warriors, and we can beat 96% of them just by using an LLM to evolve the programs.

Importantly, the LLM is not good at Redcode. If you just try to zero-shot this language, it’s terrible. If you do best-of-N sampling, it’s terrible. But if you do this evolution and have a verifier in the loop, it turns out to be really good.

This is the same lesson from AlphaEvolve, right? These LLMs are not good at zero-shotting certain kinds of proofs and things like that. But if you let them evolve with the verifier in the loop, they can be really, really powerful.

That speaks to the power of evolution and the priors from the model. But the second result that we have, and that I’m the most excited about, is that if you do these rounds of evolution, the warriors that you end up with further down the line become more and more general. They’re more likely to beat human warriors that they haven’t seen before, so I think that’s a pretty cool result.

But even more than that, the cooler part is that if we have this vector describing the behavior of these warriors—their behavioral characteristics—we can get into the details of how we measure the behavior, but we have this vector. The variance in the behavior of these warriors goes down as you increase the number of rounds. That means they’re getting more general and trending toward reducing variance. They’re behaviorally becoming like a single generalist warrior. At least, that’s what the trend is toward.

20. Evolution is anything but random

Exactly. I think people in AI, I’ve noticed, tend to think of evolution in general as this very dumb, naive algorithm that you use when you can’t do gradient descent. They basically call it random search. I would just want to clarify that evolution is anything but random. It’s anything but random.

Evolution is probably the most powerful algorithm that we know for solving discrete combinatorial problems where you need to get pieces of the puzzle right before you get the whole thing right. The selection mechanism in evolution is really, really underrated because if you solve even one part of a puzzle and hang on to that while you search for the others, you can turn your exponential search problem into a linear search problem. In that way, evolution is a really, really powerful algorithm.

21. AutoML-Zero (Real et al.)

But all it needs is for your mutations to be good 1% of the time, right? If you just do this naively in program space and evolve—I think there was a cool paper from before the LLM era of AI, AutoML-Zero, where they did program search with random mutations—and that’s really hard to get working properly because there’s literally around a 0% chance that any mutation will give you some useful code.

But LLMs, sure, they can’t give you 100% success, but they can give you way more than 1%. So whenever you are successful, you can hang on to that achievement and build on top of it. That’s what gives this combination of evolution plus LLMs significantly more power than if you just use LLMs by themselves. It’s not even close.

Akarsh Kumar, it’s been an absolute honor having you on the show.

Awesome. Thanks for having me.

Yeah, it’s been amazing. This was a great conversation.

Yeah.