Latent Program Networks 能解决抽象推理吗？[Clement Bonnet]

My real name is Clement Bonnet. I’ll go by Clem, so Clement, Clem Bonnet—whatever is easier for you.

Amazing. Clem, welcome to MLST. It’s so nice to have you here.

Thank you so much for having me.

I’ve just been reading your paper, and Chollet told me yesterday that it’s one of his favorite approaches to the ARC challenge. Can you tell the audience about the approach?

1. Search Moves Into Latent Space

The ARC benchmark—the Abstraction and Reasoning Corpus—is a program-synthesis benchmark whose goal is to assess how AI systems can adapt to novelty at test time. It’s a very interesting benchmark because pretrained LLMs perform very poorly on it. The tasks at test time are very different from the training set.

The whole assumption behind the architecture I’m going to introduce is that we should build search into the architecture, so that it can perform test-time search. The way we do this is by embedding programs into a latent space that we train to be easy to search.

TABS.ai is a new AI research lab I’m starting. It’s funded from past ventures involving AI, and we’re a Swiss version of DeepMind: a small group of very motivated, hardworking people trying to do research, starting with LLMs and ARC-style models. We’re looking for a chief scientist and research engineers. You can check out the positions at tabs.ai.

There have been many approaches to the ARC challenge, going back to the days of DreamCoder. Kevin Ellis had a domain-specific language, and perhaps even before that people were searching a DSL, which is exponential time. Ellis had a neurally guided search, where you train a neural network, and these days everyone is embracing this kind of Greenblatt approach, where you just make an LLM generate programs.

Yours is completely different from all of the other types of test-time adaptation. You’re embedding the program in a latent space and then searching that latent space. You’re not actually creating programs; you’re generating the solutions directly.

Yes, exactly. The assumption we had was that we want to do search at test time, and we want to do search in a program space. To do that efficiently, we try to embed computationally hard programs into a continuous latent space that we can then search using any search method we want.

By learning the manifold of programs in a single, coherent latent space, we allow efficient test-time adaptation. There have been a lot of methods for this. The top-performing methods use test-time training, which is parameter-efficient fine-tuning given a new test input. You search in your parameter space for parameters that would perform better on your test input.

That’s a very effective way to do this kind of recombination synthesis at test time. We argue that it’s a very inefficient way to do so because the parameter space is huge, and it’s not obvious that you can recombine low-level primitives in a compositional way.

It remains to be seen whether this approach can solve ARC, but we believe that by embedding programs into a more coherent and compressed representation, we can perform easier and more efficient search.

2. ARC Demands Extreme Generalization

It might be worth meditating for a second on why the ARC challenge is so impervious to neural networks. Chollet wanted it to be that way. Why is that?

Chollet designed it to be robust to memorization. The reason it is robust to memorization is that the input-output tasks are so different from anything seen in the training distribution. It’s quite guaranteed that they don’t exist anywhere on the internet. It’s a private test set that is kept completely hidden, and therefore the task similarity isn’t high enough for LLMs to generalize zero-shot, or at least we don’t get zero-shot generalization.

I want to push on that just a tiny bit. Many of the sentences we’ve spoken in this conversation don’t exist on the internet either. What’s the difference between that and an ARC program?

I believe our discussion can be embedded into a rather small latent space that the LLMs have learned to emulate and compose. The ARC tasks are truly novel. They’re based only on core human-knowledge priors, but they compose these priors in arbitrary ways that I believe are not anywhere on the internet. The pretrained language models cannot make sense of them; otherwise, we would get zero-shot generalization, which we don’t.

Are you saying that the reason neural networks can’t learn ARC is simply because there isn’t a lot of ARC-like data on the internet? It’s not because there’s something intrinsic about ARC—that it’s a discrete program distribution, or something like that—that neural networks can’t learn?

You’re saying they can learn it; it’s just that there’s no data out there. If we knew the kind of distribution of tasks that are in the test set and trained a neural network on it, we would solve it.

If you trained any neural-network architecture on a distribution that included the test-set data, you would solve it. It’s machine-learning training. The reason we can’t solve these test tasks is that the test distribution is quite far away from the training distribution. The system isn’t able to perform extreme generalization or come up with a novel way of recombining knowledge to perform the task.

3. Induction Meets Transduction

MLST is sponsored by CentML, which is the compute platform specifically optimized for AI workloads. They support all of the latest open-source language models out of the box, like Llama, for example. You can pay on consumption, essentially, where you can have a model which is always working or it can be freeze-dried when you’re not using it. All of the models that they deploy support the OpenAI API specification out of the box, which means it’s just a one-line change in your application to switch over to CentML and start saving money and make your application go faster.

This gets to the next interesting point. A few years ago, if you had asked me whether it was possible in principle to learn ARC, I might have said no. Many people, including Kevin Ellis, love the idea of generating programs, and that’s what Greenblatt did. We intuitively feel that we should generate a program because when it runs on a computer, programs can perform this kind of compositionality and various forms of generalization.

Intuitively, we think that if we generate the actual answer without going through an intermediate program, it won’t be as good. But that isn’t the case, is it?

I assume you’re referring to transduction versus induction—going through a compressed representation, which would be induction-based, versus directly trying to predict the answer.

Maybe we should define induction and transduction, because even before you get to transduction, you could train a neural network just to generate the solution directly, without using transduction.

That would still be transduction. If you predict the output data directly from the inputs—reading out the inputs to me—I define transduction as generating a prediction directly from the data, rather than as a statement about using the solution space.

For example, Awni Hannun’s work is transductive because, given some input examples, he performs retrieval, fine-tunes the model, and then makes a prediction. He’s building a new model for every datum that is received.

But we could build a massive neural network that directly predicts the solution space. That would still be an inductive model.

That’s where the fine line between induction and transduction comes in. I don’t know whether the distinction between induction and transduction actually matters. What matters to me is how much you compress the representation.

When you have an inductive method that comes up with a Python program—or perhaps some kind of latent program—the small representation is supposed to emulate your output. I would call that an induction method because there’s a representation-compression bottleneck.

If you’re not trying to do that, and you’re trying to perform a shallow recombination through test-time training, I would put that in the group of transductive methods. For me, it’s all about the compression, because compression allows more efficient search in the architecture.

We build a very compressed representation of programs in which we can effectively search. I believe that methods that generate Python programs are very similar methods: they generate a very compressed, low-bit program in Python that can generate an output.

I was going to ask you why a kernel method isn’t induction.

The goal of linear regression is to come up with a linear function that best explains your data. Once you have this function, you can use it for inference on a new input. When you solve a linear regression of the form \(y = Xa\), you have a solution of the form \((X^\top X)^{-1}X^\top y\).

You can write the output as a pure function of the training data and your new input by just multiplying matrices. I think that’s one representation of the kernel trick. Although you have this very low representation—a linear function—you can rewrite the whole thing as one computation. You could unroll whatever computational graph you have, and the whole thing becomes a single function.

But isn’t the kernel matrix the training data? It’s the inner product of the data, essentially giving you this positive-semidefinite matrix. Isn’t that the inductive function?

Yes. If you write it out—if you unroll whatever computational graph you have—the whole thing is a single computation. That’s where the equivalence between transduction and induction comes in, at least as I see it. That’s why I don’t think it’s a particularly useful distinction here.

I really like the distinction. To me, it’s about creating a model from the data rather than reusing an existing model. For example, we could make your model transductive if we fine-tuned the encoder based on some augmentations of the data.

I would argue that your system is not inductive if we’re doing some kind of gradient optimization through the model but aren’t changing the model. Therefore, it’s inductive.

We’re not changing the model; that’s the whole point. Although I do think that our latent program network, or LPN, search actually fits on the spectrum of test-time training methods.

If you see the latent space as an input-conditioned parameter space, in a way, we’re changing this space. We’re searching through it using zero-order or first-order optimization methods to find a better explanation for your data. That’s quite similar to searching through your parameter space and doing parameter fine-tuning at test time.

I do think they fit in the same spectrum. I’m curious to explore methods where this adaptation at test time would be as efficient as possible. If we want to create systems that adapt online very efficiently, I think real-time training is probably a very suboptimal method, because it involves searching in this vast parameter space.

Can you talk me through the architecture?

We want to embed programs into a latent space. To do that, we first have an encoder that takes input-output pairs independently and embeds them into a program, similar to a variational autoencoder architecture.

If you think about it, an input-output pair can be explained by an infinite number of programs of varying description length or complexity. We would want to find a minimum-description-length program, but because you can explain an input-output pair with an infinite number of programs, we use a variational framework.

We encode these programs into a distribution of programs. We embed the input-output pair into a distribution of programs, and then we have this search component in the middle, which I’ll explain in a moment. That component refines the latent representation to make it better. Once we have a better latent representation, we try to decode it with the decoder.

The decoder takes a new input and the latent program, and generates an output for it. It’s literally applying a program. I’m saying “programs,” but I really mean latent vectors. We’re working in a geometric space.

The true novelty of this architecture is the middle stage and how it works. We expect the encoder to output a first guess of what the program looks like. It’s similar to human intuition when solving an ARC task: “I believe there’s something about shapes here, and we should move them around.” That’s a first guess.

We then use optimization methods to refine this latent representation, finding a different point in the latent program space that would better explain the data. You have this inner loop that uses the decoder and the input-output pairs to refine the latent representation and find a better point in latent space.

For each input-output pair, that point should generate the right output. Once you find a point in latent space that better explains the input-output pairs, you’re confident that you can apply it to your new test inputs.

That’s where the induction part comes in. We’re trying to find a small representation—a small latent program—that explains the whole task.

ARC typically comes with about 3 specifications. By “specification,” we mean that you get a few input examples and place all of them through the encoder. The encoder gives you 3 different points in the latent space, you average those together, and then you perform gradient steps to improve the latent representation to a point where you get a good solution for all 3 examples. Is that roughly right?

Yes, exactly. We need to find a way to recombine these different latent distributions for the different input-output pairs. Ideally, each input-output pair should generate a roughly similar latent distribution through the encoder because they correspond to the same underlying task.

The straightforward way to do this is to compute a mean in latent space and use that as the starting point for the search. I believe there are better ways to do this. We could view the whole aggregation as a mixture of distributions and perhaps sample with a small temperature. There are probably better ways to aggregate these distributions.

Our first proof of concept used mean aggregation, which ends up working pretty well. The nice thing about this architecture is that it’s trained end to end using a variational autoencoder loss, the ELBO loss.

The ELBO loss is decomposed into 2 terms: the reconstruction loss and the prior loss. The reconstruction loss asks the model, given a refined latent program, to reproduce an input-output pair given all the other input-output pairs in the specification. The prior loss makes sure that the latent space looks roughly Gaussian. This is the classic prior loss from the variational autoencoder, and it encourages the latent space to be structured.

We did try training without the variational framework—a pure autoencoding framework, with no prior on what the latent space should look like. We ended up with very unstructured, spiky spaces where most programs tended to be as far away from one another as possible. That made search impossible and rendered the latent space rather useless.

The variational aspect is very important for keeping a Gaussian, compressed representation of the program space.

Let’s double-click on that. We want to have a well-formed space that doesn’t degenerate and doesn’t memorize solutions. I think you also use the KL divergence. In your paper, you said that you want to prevent encoding the output directly in the latent space.

You encode an input-output pair into the latent space, but train the representation to decode the output of a different input-output pair, to prevent memorization. That was one of the tricks you used.

It’s not a trick. With most of these latent-space ideas, when they’re applied to program-synthesis benchmarks such as ARC, there’s usually a problem: you want to learn a compressed representation of whatever your input-output space is and then try to decode it, but the obvious issue is that you may end up compressing your output into the latent space.

You would leak your output into the latent space, making it easier to reconstruct. That would be a shortcut that machine-learning training would find, and it would be useless because you wouldn’t learn anything that connects the input to the output. You would simply learn to map the output into the latent space.

We deal with this by using a setup during training that is very similar to the setup we have at test time. We have access to N input-output pairs, and we want to predict an N + 1 output.

During training, if we assume N input-output pairs, then for each input-output pair we embed the other N − 1 pairs into the latent space to predict that pair. We do this for all the pairs in parallel.

The other great trick you use is gradient steps—search steps—during training, because that makes the final latent space more amenable to search at inference time. Was that something you thought of intuitively, or did you add it because the system wasn’t working?

We’ve always wanted to train using search. The reason is that during training, and even at the end of training, the encoder isn’t perfect. The assumption is that it provides a best guess of what the task is.

If you’re not allowed to refine that guess, the decoder can only be as good as the encoder’s guess. You end up with a latent space that cannot encode many programs; it just encodes something like the regression of the encoder’s best-guess distribution.

We activate the search method during training. It brings significant overhead at training time, so we propose some kind of pretraining without it, followed by fine-tuning with it. But the main point is that by switching it on during training—using random local search or first-order gradient optimization—we train the latent space to be good at being searched.

The best guess ends up being quite bad, but very close to a very good guess. That’s the whole idea behind meta-learning methods such as MAML and similar approaches.

Qualitatively, tell me about the latent space. You’ve looked at it. Do you find that different directions correspond to different types of ARC problems, or whatever? What do you see?

We haven’t analyzed the latent space that much yet by comparing different ARC tasks, although I think it would be very interesting to do so.

In the paper, we show a t-SNE visualization of the different programs that are learned. During training, we see a kind of blob containing the input-output pairs that aren’t well understood either by the decoder or by the encoder. They end up in this bag of unlearned programs.

As programs start being learned, they begin to cluster and occupy the space. We scaled this architecture for ARC very recently and got some nonzero, yet non-SOTA, results.

Although we don’t get very high performance, we get it without any priors: without using pretrained LLMs and without looking at the evaluation set. We trained only on the 400 training-task distributions, yet obtained around 10% on the evaluation set, which is itself a harder distribution of tasks.

That was quite impressive from the first scaling curves we obtained.

4. Training Transformers From Scratch

Can you explain that in a little more detail? You’ve got an encoder and a decoder, and they’re Transformer models. You’ve got the re-ARC tasks from Michael Hodel. What kind of Transformers were they, and what kind of training did you do?

The LPN architecture assumes encoder and decoder modules. To train this architecture on ARC, we used vanilla Transformers from scratch to represent the input-output grids.

You could use any generative model or sequence model. We decided to use self-attention Transformers that flatten the input-output grids into sequences of 900 values, because these are 30 × 30 grids. We used relatively smart 2D positional encodings to make sense of the spatial structure, but otherwise it was a very vanilla Transformer architecture.

The models were rather small—about 20 million parameters each, or roughly 40 million in total for the encoder and decoder. We didn’t tune the Transformer architecture very much. We had to do some work on the positional encoding to make sense of the grids, but otherwise it was very standard.

We focused on learning a good latent space of programs and on how to search it efficiently during training and at test time. We experimented with the Transformer architectures at first, decided we were good to go, and then focused on what we thought was interesting.

This blows my mind because most people who use Transformers fine-tune them. They take models that have been trained on internet data and fine-tune them. The intuition is that it’s very difficult to train Transformers, and they only start doing useful things when you train them with a lot of data for a long time.

You started from scratch. I think you said you didn’t even train it to convergence, using just Michael Hodel’s re-ARC dataset. That’s hardly anything, and it still worked quite well.

The goal of our training was to learn to embed these 400 training tasks into the latent space. If we do that in a structured way, then by interpolating through the latent space during search at test time—given a new input-output pair, or rather a new task—we hope to find programs that weren’t seen during training but represent interpolations or superpositions of programs seen during training.

Training a Transformer on single input-output pairs from a single program would be a very bad idea. There isn’t enough data for it to make sense of positional relationships and so on.

We use Michael Hodel’s very useful library of re-ARC transformations. The dataset he created is a distribution of input-output pairs for each of the 400 tasks, following that task’s program. The program is constant, so there isn’t much more information to extract. The program is constant; only the input distribution is broad rather than consisting of a single input.

This makes it possible to train Transformers to make sense of 2D grids in a way that they couldn’t when initialized from scratch on a single input.

We show some training curves on ARC. We didn’t have enough compute to train to convergence, but we showed an increase in accuracy on the training set. We believe there are bottlenecks in the architecture that make learning very slow or prevent it from converging to 100%. We’re trying to figure that out.

In this re-ARC dataset generator, with the 400 tasks expanded using Michael Hodel’s data generation, how much data do you have? How many program examples?

It’s on the order of 100 million data points.

That’s a lot.

It is quite a lot. We sample these programs extensively. Transformers are very bad at making sense of random sequences of pixels when they’re trained from scratch, so we need this intensive training to make sense of the input-output grids.

It’s a bit unusual, but we believe ARC is a developer-aware generalization benchmark. We wanted to have no priors from anywhere on the internet, so we didn’t use pretrained LLMs. We trained the architecture only on the data that matters to the task.

Isn’t the re-ARC dataset generator itself a prior, though? To me, that would invalidate the developer-aware generalization. The prior contained in re-ARC seems like a pretty big one.

The priors contained in re-ARC are about situations such as having 2 objects in an input-output pair when you could have the same transformation with 3 objects. It includes priors such as counting, objectness, and so on.

The priors used in re-ARC are primarily there to teach the Transformers how to make sense of 2D grids, which I believe is in line with the developer-aware generalization assumption.

We can debate that. How did it generalize to different grid sizes? You’re using a Transformer, so you turn the grid into a sequence. Did you notice any weird overfitting to sequence sizes?

We handle the shapes in a very naive way. Let’s say you have a 10 × 10 input grid and a 10 × 11 output grid. We predict that it will have 10 rows and 11 columns, and then we use the 900 tokens corresponding to padded 30 × 30 grids, most of which are padding that the Transformer ignores.

We end up with sequences of 902 tokens per grid, including the shape information. That’s how we predict shapes. There are many things to say about how to optimize this, and we don’t think it’s optimal, but it’s probably sufficient for ARC, given that we have a lot of data to train on.

5. Symbolic Programs Bring Composition

I suppose this might be an example of why generating an explicit Python program could be better, because it could generalize to different grid sizes.

Definitely. Python programs, or grounded programs executed by an interpreter, would make sense of all these priors that aren’t contained in a randomly initialized Transformer.

I do believe real programs are also a very smart way to perform induction and compress representations. One of the problems with program search, of course, is the combinatorial explosion of searching through the program space.

We try to compress that by learning the manifold of programs, which we hope will be easier to search. But it comes with some caveats. Potentially, you can’t represent all programs in this latent space.

There are interesting ways to combine both ideas. There are pros and cons to program search and latent-program search.

I’m thinking about when I first interviewed Chollet years ago. He always said there were Type 1 and Type 2 problems, and Type 2 problems were not interpolative. But now we’re taking it for granted that this is actually interpolative after all.

We have this latent space with all of these modes, and we perform test-time computation. The more test-time computation we do, the better the results. At the moment, if I understand correctly, you’re using a first-order gradient-search method. First of all, talk me through that. You do more search at inference time and get better results?

Before that, I do believe Type 2 problems would not work with our method. One of the big limitations of learning a continuous latent space is that you can’t fit arbitrarily complex compositional problems—or programs—into it.

I think you can do some things such as a superposition of programs, but I don’t think you can do composition of programs, which is what we’re trying to investigate in follow-up work. A single-threaded search in this single latent space would not bring you composition.

I think you can get some composition, perhaps a couple of steps of composition, but at the end of the day it’s a fixed, continuous latent space. It’s unclear whether you can really compose objects with it.

You’ve touched on an interesting point. This gets to the core of it. People such as Gary Marcus and Chollet were intuiting the lack of compositionality.

You’re saying there is no compositionality in the latent space, but we could develop some kind of compositional inference-time method that gives us compositionality. We could have multiple search threads through the latent space and a System 2 program that composes the things together, while still using your method. Is that what you’re saying?

I think that’s a promising way to think about it. Single-threaded search in a single latent space would never bring you composition, but having multiple threads or multiple things happening at the same time and synthesizing them afterward could be a way to obtain composition.

We wanted to do a proof of concept of how to fit as many programs as we can into the latent space. I think this opens up many possibilities for extending the idea.

With symbolic programs, it feels as though we could compose them together. For example, we could learn iterative steps of functions that are composed together. In this case, we have direct solution predictions.

Let’s say I’ve found 3 solutions that represent 3 different types of transformations that need to be composed together. Now we need a function that can somehow take those 3 solutions and combine them.

I don’t think you can easily look at those 3 solutions and synthesize them in one shot. It would be equivalent to having a single, better search thread.

If you want to explore different regions of the space and recombine them, you might need to perform multiple forward passes. One thread could give you a primitive program, and then you apply the second thread to the output of the first primitive, and so on.

That would give you a recursive, unrolled computational graph. It’s a research question how to think about these things.

Or perhaps the decoder could take multiple inputs and learn to compose them to a fixed depth. Language isn’t actually that compositional. We use only limited depth; some languages don’t use much depth at all. In English, we have a depth of perhaps 3, 4, or 5.

You might find that most of the ARC problems we ever want to solve have a fixed depth, which we could somehow encode in a neural network.

ARC solutions using brute-force program search have depths varying from 1 to 12 or 15, and sometimes up to 50. But for most tasks, you can solve them using a rather short program.

We’re pragmatic people. I love the theoretical arguments about compositionality, symbolic AI, and so on, but in the real world we don’t need an infinite conversation.

Going back to the search methods, the assumption is that there exists a different latent program in your latent space that is better than your first guess. The question is how to find that program.

It’s simply a question of search. We tried 2 things. First, we tried zero-order random search: a local search around the first program. Then we realized that the latent space itself, learned during training, ended up being very smooth.

By smooth, I mean smooth with respect to the decoder likelihood of decoding the correct input-output pair. You want to maximize the likelihood of decoding the correct output, and this likelihood ended up being very smooth.

That allowed us to use first-order methods. I do believe that, with different tasks or if we trained for longer, we could have more discrepancies and local optima in this space. That might bring methods such as evolutionary strategies or other zero-order methods to the table, perhaps in combination with gradient methods.

For our use case, gradient descent worked very well. I don’t think it’s the best method, but it worked.

At the moment, the space is smooth and amenable to a simple gradient-ascent method. What about its resilience to noise? Have you tried putting noisy inputs in?

Another possibility is that you could undertrain the architecture but perform more and more search. Have you explored that relationship?

We haven’t tried adding noise and deliberately shooting ourselves in the foot, although perhaps we should. We have tried scanning the architecture size.

If the Transformers are too small, they can’t represent or decode the output very well. There’s a trade-off because architecture size and latent-space size go somewhat hand in hand.

When the architecture is too small, it can’t make sense of the sequences, so it can’t decode the program very well, regardless of how large the search space is. When we train the architecture, there isn’t much tuning to do, but we need to make sure that the decoder has enough capacity and that the latent space is large enough.

There needs to be a point in latent space corresponding to the right program, and the decoder needs enough capacity to execute that program. That isn’t the case when you use Python programs, because the interpreter will execute the program correctly; you only need to find it.

There’s a second constraint here: making sure the decoder can learn the program. We have preliminary results—not included in the paper—showing that it isn’t obvious that the decoder can learn all of the tasks. There’s also a paper mentioning this issue.

It comes down to what priors you build into the architecture in terms of 2D spatial representations and so on. It’s a trade-off between how much time you want to spend optimizing the architecture to solve a specific benchmark and how much you want to improve the method itself, namely the search for latent programs.

Ellis and Welling recently released a paper comparing induction and transduction. They had an ensemble approach that did both at the same time. They have a beautiful Venn diagram showing that sometimes induction with Python programs works really well, while sometimes direct solution-space prediction works really well.

Have you analyzed what types of ARC tasks work well or poorly on your system?

We don’t have a strong analysis yet, but from what we’ve seen, most of the unlearned programs correspond to transductive tasks. I think that’s very much aligned with their paper. It was a great paper highlighting this distinction.

When it came out, about a week before the ARC Prize deadline, we realized that this was probably what was happening: we were struggling to learn these transductive tasks.

There are obvious ways to use an ensemble technique with our architecture and with other architectures. The problem with transductive tasks is that there is no point in the latent space corresponding to a transductive task. By definition, no induction is necessary there.

Everything happens in the decoder. I think there are ways to help the decoder with that. We’re trying to remain as benchmark-agnostic as possible and not overfit to ARC, but the obvious ways to improve are there.

6. Deep Learning Needs Program Synthesis

Is there a story that you’re not a symbolist? Have you always been a connectionist, and are you relatively new to trying to improve connectionist architectures to tackle the 1% or 10% of tasks that people are interested in these days—reasoning and search?

I believe in a future where most tasks can probably be tackled with one-shot, deep-learning-based architectures such as LLMs. But some of the hard, last-mile tasks—the final 1% or 10%—need something different.

I’m a strong believer in deep planning and program synthesis being merged to achieve that last percent of tasks.

So you’re not amenable to the school of thought that these systems are simply smarter than us and that we should let them do what they do, even if we can’t understand it?

I don’t think we can comprehend arbitrary combinatorial sequences of symbols. That’s where connectionism is useful: we learn manifolds of complexity. It can bring you very far, but it can only bring you so far.

Deep architectures can solve perhaps 90% or 99% of tasks, but there will be tasks that are inherently discrete, or tasks that require long-horizon planning and reasoning—things we’re trying to tackle with LLMs and other deep architectures today.

More recently, I’ve started to think that all the architectures could do this much more efficiently. I also believe it’s connected to creativity.

Deep-planning models are not highly creative. I won’t say they aren’t creative, because if you go back to Greenblatt’s solution—sampling a lot of programs—you can see logarithmic scaling curves as you sample more and more programs.

If you plot the distribution of possible programs sampled by an LLM that outputs code, you have nonzero mass, with long tails, on very creative and good solutions. You only get those solutions if you sample a million times.

In that sense, I can say that these systems aren’t creative. If you sample enough, they will create these things, but they’re exponentially less creative. You would need exponentially more samples to make them creative.

I think symbolic programming and program synthesis could help with this creativity issue. What we call programs may just be sequences of symbols. We could fight this creativity issue by doing symbolic program synthesis, moving into a combinatorial space that is completely novel and cannot be tracked or amortized.

That gets us closer to things such as computational irreducibility. I do think program synthesis and symbols could help deep architectures become more creative.

That makes sense. I’m very interested in creativity. I’ve taken an almost human-chauvinist approach to creativity, believing there’s a creativity gap in LLMs—that we have access to some source of creativity that LLMs don’t.

Many people I’ve interviewed in Europe have told me the opposite. Saba Karamcheti said the opposite at ICML in June. He said these systems are very creative, partly because creativity isn’t just about novelty. It’s also about interestingness—things we find interesting given our cultural biases and interests.

These systems seem to capture a lot of that. You can say that Greenblatt’s approach requires sampling 30,000 times to get something interesting, but is that really so different from us? We’re a huge collective intelligence. We all come up with ideas, most of which are rubbish, and every now and then we have a flash of inspiration.

Perhaps we’re embedded and situated, with access to all this entropy and everything else, but is it really so different?

There’s something to be said about collective intelligence. I haven’t thought that much about collective intelligence in general, so I can’t comment very well on it.

If you focus for a moment on individual intelligence, it seems obvious that you don’t assemble a million programs in your mind when trying to solve an ARC task or come up with something creative. Some kind of synthesis happens in your brain much more efficiently.

I’m interested in trying to get closer to that kind of synthesis. I do believe there’s something very interesting about collective intelligence, which brings much more entropy and much more of the sampling we talked about.

Perhaps both schools could come up with interesting breakthroughs.

When you solve an ARC challenge, are you being creative?

When I try to solve an ARC task, I have some intuition about what the task might look like within a tenth of a second. I see the structures and regularities very quickly. Then I realize, “No, this isn’t as easy as I thought.”

I propose a few hypotheses—perhaps 2, 3, or 4—and refine them. The fifth one is correct. I check that it matches all the specifications, apply it, and get the answer.

That’s how I solve perhaps half of the tasks. The other half takes more time, but the basic idea is to come up with a few hypotheses, test them, and not come up with a million hypotheses.

Those hypotheses seem quite creative in the sense that they can be very different from one another and recombine priors in a way that is combinatorial and compositional. I need to remain humble about how I solve the tasks, because I struggle with many of them, but that’s the idea: propose a few hypotheses and test them without generating a million.

How do you cut down the search space efficiently? There’s probably much more to human cognition, but getting this would at least be a good approximation.

Let’s say we scaled your solution up ridiculously: much more training data, a much larger model, and longer training. What do you think would happen?

I think it could scale quite a bit, especially if we scale the latent space. One limitation is how much you can actually search in that space.

I would expect the space to become less linear, less interpolative, and less smooth. If you end up scaling the tasks and problems, the space would become harder to search.

It will depend on how much search you can perform during training. You would probably need to scale the encoder, decoder, latent space, and search through the latent space. That would be quite costly.

There are better ways to iterate on this idea. I believe that coming up with a small representation of whatever task or query you have can be a very useful way to perform synthesis, adapt, and deal with epistemic uncertainty.

I don’t believe we can currently estimate epistemic uncertainty, but searching through this space could help resolve some of that uncertainty.

That’s very insightful. You’re saying that when we scale this up, the latent space might become less interpretable. Could we do meta-optimization, use a different type of latent space, or perhaps use a topological space or a graph representation? I’m just spitballing, but what else could we do?

I don’t think it’s only about scaling the latent dimension of the space, which has been the approach with large models. From my intuition, what’s happening in large language models is that they have intricate, highly sparse, high-dimensional vector functions that can be recombined in some narrow way.

The space is very rich, but also very local. You’re trying to find an abstraction locally that is slightly better. I don’t think that gives you all the benefits of composition and out-of-distribution generalization.

I’m a big believer in trying to find small representations that can explain your outputs. For natural language, that would be a query and the answer to the query.

How you build these spaces and how you search through them are open questions. I’ve seen a lot of interest in this direction lately, so it’s very exciting.

Congratulations on the work and on the recognition from Chollet. It’s a completely different approach from the others—a very interesting one indeed. Keep up the good work, and thank you for coming on.

Thank you so much. Thank you for having me.