机器如何学会忽略噪声（Kevin Ellis + Zenna Tavares）

You're building a model on the fly from very small amounts of data. But you are not passively receiving the data. You have to go out there and poke things, push things, and try things out.

Compositionality is a kind of double-edged sword. We saw this in the first wave of symbolic AI, when people tried to build these kinds of compositional production systems to solve problems. So the issue is, you immediately encounter this combinatorial explosion in the number of things that you could represent. You don't necessarily have any way of steering yourself through that space toward the kinds of concepts that are probable or that make sense in your current situation. You can think infinitely many things, and those things might be very out of distribution. You're both immensely powerful and immensely overwhelmed by possibilities.

The question is, can we build things from first principles? I think it's hard to say what is thinking and what is not thinking. It does seem that some core part of thinking is a kind of step-by-step process.

If you ask people to think carefully and verbalize a solution, they actually get worse.

There's something obviously right about scale, right? And there's something obviously right about learning. I think it's obviously true that programming languages are compositional, right? We build more and more complex programs by understanding the parts and then combining them together to build more and more complex parts. Most of the hypothesis development that we do as humans is of this form, both as an individual and as science as a whole. Science as a whole collectively finds hypotheses, and then we revise them communally in conferences like this.

So it seems natural to me to say that we should build systems that have this kind of refinement process. One of the hard things to do in trying to mechanize this is, how do you guide or reinforce good refinement paths? Another axis I think is important is this kind of tower of abstractions that Kevin mentioned. I think a key thing that is somewhat overlooked in the current discourse on world models is that there isn't a single world model, right? You can understand things at multiple different levels.

There are multiple different models you can build of pretty much anything. There's a camera right here. I've got a model of how this camera works, at the level of, “I press this button and an image is taken.” But I can also understand the internal structure of the wires and circuits, or I can go down to the sensor in the camera. All of these are different models that are useful for different things that you need to do with that model.

We want to avoid building these Frankenstein systems where we are hard-coding a whole bunch of different knowledge representations and heuristics.

But in those cases, as you're alluding to, it has required smart people to look at the world and say, “Okay, here are the inductive biases. Let me encode these into the system.” We think that the principles underlying this everyday science—how do we form hypotheses, how do we revise those beliefs, how do we take actions to learn about how the world works—are the same principles in everyday science that apply to real science.

And so we want systems that can do that, that can learn how to interact with new toys, devices, and interfaces in a way which isn't large-scale imitation learning, which is like thinking.

Benjamin Crouzier

I'm Benjamin Crouzier. I'm starting an AI research lab called Tufa Labs. It is funded from past ventures involving machine learning. So we're a small group of highly motivated and hardworking people, and the main thread that we are going to do is trying to make models that reason effectively and long-term, trying to do AGI research. So one of the big advantages is because we're early, there's going to be high freedom and high impact as someone new at Tufa Labs. You can check out positions at tufalabs.ai.

I want to make machines that learn in more human-like ways, especially learning more abstract knowledge from fewer examples. We've been working a lot recently on world models and on things that try to actively discover symbolic knowledge.

I'm not interested in everything people do, but I'm interested especially in the kinds of things that people are especially good at, but that the kind of AI we're building today tends not to be so good at. That includes learning from a few examples, generalizing to situations that are very different, learning knowledge you can communicate as opposed to being just embedded inside of a weight matrix, and also learning things that can cooperate between weight matrices and symbolic kinds of knowledge.

Yeah, hi, I'm Zenna. I'm a co-founder and creative director of Basis. Kevin and I went to grad school together. We share a lot of the same interests broadly in trying to understand and build intelligence. Within Basis, we have that as a goal, but we also care about scientific and societal problems. I've built an organization so I can do all the things that I find enjoyable.

I think there are a lot of good lessons from the scaling round. We do actually want algorithms that can make use of the new hardware that we have. At the same time, it's not the full story. I think our personal belief is that that's not going to carry us all the way.

Also, pragmatically, there are other people who are clearly trying to carry that torch, and if it does get us all the way, then maybe we should hedge and do things that draw on the amazing GPUs and amazing pre-trained models we have, but which also bring to bear ideas from cognitive science, classic AI, and so on. Zenna, how do humans learn from examples?

I think there are many possible answers to that. I somewhat believe in the Bayesian paradigm, right? You have beliefs about how the world works, and when you observe examples, you incorporate that knowledge into your current hypothesis about how the world works. But that's a very abstract theory, and the question is, how is that actually implemented?

The reality is it's probably approximate. It's probably not exact; we're not doing exact Bayesian inference. So there's a nice foundation for thinking about how an ideal agent should incorporate knowledge. And then there are practical questions about how to build systems that do that, right?

As Kevin said, we're very much not anti-large-scale deep machine learning. We're building these tools. We're using these tools. I would say the main distinction between how I think of the work that we do and what we want to do is, can we build things from first principles?

A lot of current mainstream machine learning is basically large-scale imitation learning, and that's because it's so effective. It's producing all of these amazing tools. But both from a scientific perspective and also from an engineering perspective, can we understand some of the principles about how to build intelligent machines from the ground up?

I think there are some pieces: uncertainty, causality, and reasoning in general. But there are many things that we don't know, right? Part of having a research program is trying to figure out those missing parts.

Kevin, what is the importance of compositionality, and what is compositionality?

Compositionality has a lot of different meanings in different situations. The way that I think we're using it in the work that we're doing right now is that you have atomic knowledge that you maybe learned in one situation, but then you can build bigger structures out of those little pieces of knowledge in order to extrapolate to new situations that might even be out of distribution.

I think that's really important for being able to learn in environments that are not totally new. It's not like you can plop yourself in a totally new world and immediately be fully competent. But if you're in a world that's changing, where the building blocks are the same and the causal mechanisms are being reused, then compositionality—and ways of having knowledge that can be broken into pieces and then recombined for new situations—is really crucial.

Very cool. Just as an extension to that, Kevin, I watched your amazing YouTube talk, which you published about 3 months ago, and you quoted Elizabeth Spelke from Harvard.

Yeah.

She's a psychologist, and she said, “The possession of infinitely many concepts that were expressible in an innate language would lead to a curse of a compositional mind.” Can you explain what she meant by that?

Yeah. Spelke's wonderful. What she's pointing out is that compositionality is a kind of double-edged sword. We saw this in the first wave of symbolic AI, when people tried to build these kinds of compositional production systems to solve problems.

So the issue is you immediately encounter this combinatorial explosion in the number of things that you could represent, and you don’t necessarily have any way of steering yourself through that space toward the kinds of concepts that are probable or that make sense in your current situation. You can think infinitely many things, and those things might be very out of distribution. So you’re both immensely powerful and immensely overwhelmed by possibilities.

I think what we’re seeing now is that you can address this curse of compositionality through ways of learning to guide searches over program spaces, and also by learning the basic atoms of a compositional language and treating them as neural networks.

When we design AI algorithms that navigate compositional spaces, how does that work? And maybe how does that compare to how humans do it?

One of the most interesting examples of compositional languages, in my view, is just programming languages, right? I think it’s obviously true that programming languages are compositional. We build more and more complex programs by understanding the parts and then combining them together to build more and more complex parts.

One of the differences between programming languages and natural language is that programming languages are strictly compositional, whereas in natural language, there are all these heuristics and phrases where you just have to learn the thing. But how do we build systems that build compositional structures? There are many ways, including a lot of old-fashioned ways: start with a grammar, expand that grammar, search through it, and try to find a program by searching through a grammar space.

Now people are obviously using language models to generate programs and compositional structures. So there’s this interesting spectrum of approaches where there’s a question of how much semantic knowledge about the compositional structure you incorporate into your model versus whether it’s purely data-driven from examples. We’ve both worked on both and continue to work on both. But as Kevin said, the challenge is that if you’ve got a compositional structure, it’s often of unbounded dimension—a very hard space to search through. So you need smart methods to try and find the structure or the program, or whatever it may be, within this vast space.

On that, then, Kevin, is there a principled way to find a set of primitive abstractions which can be composed together to build these higher-level abstractions?

I think that some kinds of basis sets of primitives might be strictly better than others. As an easy example, as Zenna said, one of the best examples of compositionality is programming languages. I think programming languages have gotten better over time, and it’s really easy to make a language worse, right? I could break Python in a million ways. So obviously there must be a kind of partial order between compositional systems where some are better than others.

But I think in the fully general case, it does depend on the kinds of problems and environments with which you’re going to be confronted. I think you see this reflected in the fact that we have tons of different programming languages. We don’t actually have a single best programming language; we have a kind of Pareto frontier.

A particular neural network is an architecture which implements some algorithm. You can implement an algorithm in Python, right? The Bayesian paradigm is a normative model, so it says what an ideal system should do, essentially without consideration of compute. You could also say, what would an ideal system do if you had bounds on compute, or if you had to reason about how much compute you should use? So I don’t think of it necessarily as bottom-up versus top-down, at least on this axis.

But you can think about composite methods which combine different kinds of systems. The paper that Kevin started, and that we joined forces to work on and complete, used a combination of different methods. Kevin will talk more, I’m sure, but we have an induction model and a transduction model. The induction model is then a program, in the case of ARC—a program that transforms the input to the output, like a Python program. The transduction model is directly outputting the output grid given the input.

I think it’s important to consider different axes. One is, in some sense, what is the representation of the function? Is it a Python program or is it a neural network? The other is, in some sense, what is the type signature of the thing that you’re finding? Are you directly outputting the final solution, or are you outputting a function which you can then apply to the input? We’ve explored a little bit of this grid of possibilities, but I think there’s more we could think about.

I like how you described it as the type signature. In a sense, the inputs and outputs of the symbolic and neural ways of solving problems are often very different. If you try to solve something which is a pure transformer mapping inputs to outputs, like we did within the transduction model, you’re not constructing these intermediate hypotheses. When you have these symbolic compositional languages, like when you’re doing program synthesis, the actual type of the thing that you’re trying to learn is different.

When we think about comparing neural and symbolic methods, they often confound these different factors. When we did this work, we were partly comparing neural and symbolic methods, but also partly comparing these 2 different styles of problem-solving: one where you look at a problem and think hard about an explicit way of solving it, a way you could maybe verbalize in a symbolic language, either in code or in language, and then contrast that with this more intuitive, implicit, more transductive way of making predictions.

It just so happened that it was convenient to map that onto this neural-symbolic divide. But you can totally imagine inductive methods that search for a vector that describes how to solve a problem. In fact, there were ARC teams that did this, and it actually kind of works, which is really cool. So as Zenna was saying, there’s really a grid of possibilities, and we’re trying to do a very careful study of these different ways of solving these inductive learning problems.

That was an amazing paper, by the way. As I understand it, there are 2 Llama 8B models: one’s an induction model and one’s a transduction model. In one case, you produce an explicit function. You do test-time inference—you Greenblatt it. That’s my verb; I use Ryan Greenblatt. I generate loads of example programs and take the good ones.

Then you’ve got the transduction approach, where you directly compute the solution, but still with some transductive active fine-tuning by augmenting the test example. The ensemble approach is that you try to see if one of the induction functions works. If not, you fail over to the transduction approach.

There’s this beautiful Venn diagram in the paper where you show that transduction works for some types of problems and induction works for other types of problems. Help me understand.

Interestingly, this does relate to some classic findings in cognitive science, where we know that there are certain kinds of problems where, if you ask people to think carefully and verbalize a solution, they actually get worse. For instance, if you try to have people infer a rule with exceptions, or if you have people try to do more of a statistical learning task, like learn associations between different symbols, having them think a bit and explicitly verbalize a solution can degrade performance.

We know empirically that this is true, and there’s a really nice paper from Tom Griffiths showing that LLMs also have similar kinds of splits. I think we were kind of rediscovering this within the Abstraction and Reasoning Corpus. We’re finding that there are some kinds of problems where, if you have the system think hard and churn through many thousands of different possibilities and test them systematically, that’s actually worse than having the system just blurt out an answer.

Because induction involves systematically sweeping through possibilities while transduction is just blurting out the answer, and because you can check the correctness of induction, you can think systematically for a while. If you fail to find anything that seems like a good explanation to you, you can just fall back on your intuition. So it’s a very natural way of ensembling or combining these 2 methods. We don’t really need a way of looking at a problem and deciding whether you should think explicitly in symbols or use your intuition, because we can validate the correctness of symbolic hypotheses.

Is one thinking and is the other not thinking? For example, if I do lots of transductive active fine-tuning and augmentation, and I give a subsymbolic, intuitive solution versus doing thinking by generating lots of explicit functions, is one thinking and the other not thinking?

I guess it depends on what you mean by thinking.

What do you mean by thinking?

Well, now people say thinking is what o1 does, so maybe that’s thinking. I think it’s hard to say what is thinking and what is not thinking. It does seem that some core part of thinking is a kind of step-by-step process where you go through some kind of internal mental computations, revising your beliefs. I think it’s hard to put a strong kind of circle around it.

Again, I do think there’s a distinction between representations of knowledge. In this case, you can have a symbolic program, or you can have a purely neural, connectionist system output an answer, versus the kinds of procedures that you produce. Again, you can combine and compose all the different possibilities.

We just talked about the fact that when you have an inductive model—in our terminology, a system that outputs a Python program—we can then use this Python program to check whether it’s consistent with the training examples. This is a very strong signal as to whether it’s going to be a valid solution for the test example. That is not currently true for our transduction model, but you could imagine a system where that is true. You could have the messiness of a neural network, but still get the ability to check whether it’s consistent with the examples.

On the question of whether that’s thinking, I think all of these are different versions of thinking. I think there is a kind of slow, deliberative, hypothesis-forming notion of thinking, which probably isn’t fully captured within any of the methods that we used in ARC. It’s probably something more like what o1 is doing, where you’ve got variable-time computation, and you’re not blurting out things in one step, but you’re understanding how much you have to do to answer a question. That’s something like thinking.

Maybe a more narrowly constrained version of thinking is reasoning. You have the purely classical forms of reasoning, where you’ve got some axioms and you go from that to some conclusion, and then you’ve got the common-sense reasoning that we do every day as humans to figure out how to carry out our everyday lives. I imagine that there’s some way to think about reasoning in general, broadly construed as forming beliefs given our knowledge, but allowing for the messiness of the world as we see it in a way that classical formalisms don’t quite allow.

In the ensemble approach, you biased the induction. You wanted to find an inductive solution first and then failed over to the transduction. Does that hint that you think there’s something special about the functional version?

I think that the functional version is more regularized. It’s harder for it to overfit. It’s not impossible, but it’s pretty tough to overfit with these modern, high-level programming languages. They’re designed to express general-purpose computations.

Neural networks can do that, particularly, as you were saying, in the large-data limit. They tend to eventually learn representations that start to really capture the kind of stuff you want. But that’s not always true, and often they interpolate around their data points.

I think if you can come up with a clear, explicitly verbalizable description that actually works for the problem you’re trying to solve, then that’s likely to generalize a lot better than just going with an intuitive, vector-based interpolation.

Zen, is there a better way that we could, maybe in a principled way, combine transduction and induction?

Several different ways. Again, I think one thing you could think about is whether you could have a transductive-inductive model where the underlying transformation representation is something like a neural network, but you can apply it pointwise to each ARC instance as an input. This would allow you to get perhaps a stronger inductive bias about whether this is the correct example by testing it on the training set.

A more interesting space of things to explore, although I don’t know if this is more principled, is on the representation side. What is it that a normal program gives you? Fundamentally, what is the benefit of using a Python program in some cases over a neural network? And what is the fundamental benefit of a neural network in terms of encoding transformations?

We talked a little bit about how maybe the neural network can capture the messy parts of the transformation. In the case of ARC, there are some things that are hard to describe with Python programs. They’re hard to actually describe if you sit down and try to write a program, because of just weird, complicated things that happen.

There’s a whole paradigm of neurosymbolic programming: let’s take some neural components and some classical programmatic components and combine them together. But you might try to think from the bottom up: is there a way to restructure programming languages in general?

If you think of them both as different programming languages in a sea of all possible programming languages, we’ve touched on a very small number of languages in the space. What other things exist? I don’t know. I think we really haven’t scratched much of the surface of different representations of computation and programs in general that we could explore.

Part of what we’re thinking about now is whether we can find not just deeper integrations, but more fundamental things about what, in principle, we want from a programmatic representation versus what we don’t want.

That’s fascinating to me, and maybe you could expand on that, Kevin. In the DreamCoder paper, it was using a DSL. Of course, we could use something like Python, and maybe you could argue that there’s some kind of computational equivalence between high-level programming languages because they’re Turing complete. What is the difference between these different expressions?

In the first generation of AI, there’s a saying: “You can’t learn what you can’t represent.” To tack on that spooky quote from earlier, if your computational language explodes too badly, maybe you could represent it, but it’s not really, in a practical sense, learnable.

In DreamCoder and in a lot of other languages that use domain-specific languages, often you’re just not able to represent all the stuff you could do in Python. Now there’s a simple fix to that: you can introduce a kind of escape hatch, where you have some primitives that upgrade your DSL, your domain-specific language, to actually be Turing complete.

Then, in some formal sense, you do cover all the stuff that you could do in Python, but the curse of computationality bites you. You couldn’t actually learn it in a practical sense.

In my opinion, having worked with these DSLs and with lambda calculus, Python is just a lot more practical for the kinds of problems that we really care about. This is true in ARC. I think it’s true when we’re building LLM agents, and I think it’s true when we’re having visual question-answering systems.

There’s an interesting kind of convergent evolution between the design of software engineering languages, which has caused them to converge on something that’s actually really great for a lot of problems we care about in AI. They’re not perfect, and I think that in this transduction-induction paper, we do see that Python is not actually covering all the stuff that we care about, but it’s miles better than lambda calculus.

Yeah. I think it is really important to recognize that there’s an evolution of programming languages, and Kevin said this at the start, right? The original programming languages were basically just long streams of instructions. Then we built in more and more forms of structure. There was a whole structured programming movement.

Then we have a variety of different families of languages that we use in modern software engineering today: very mainstream languages like Python, and maybe slightly less mainstream languages that are popular in other circles, like Haskell and functional languages.

And so I guess the key thing for me is that beyond just expressing computation, which is not that hard to do—we can express computation in many different systems—we've started to build more and more kinds of systems to help us build better programs. Class structures, type structures, different modular structures: they allow us to encode more about the world within our programming languages and make them more useful.

And so if you think of this as an evolution, on one hand, as Kevin said, we've got to a point where they're pretty good, but there's also room for development. We probably wouldn't expect that in 100 years everything would look exactly how things are now. I think an interesting question is: How do we expect the evolution of programming languages to develop?

And then when you add AI in general into the picture, it becomes a lot harder to predict. Now we're seeing these interesting compositions of ChatGPT calling out to Python and Python calling out to ChatGPT. It seems to be a moment of flux in the design space, and for me, it's interesting to see how this will evolve over time.

Yeah. I guess, should we have some kind of metaprogramming built into the algorithm? Start with maybe an iterative process of rewriting rules and improving hypotheses, rather than trying to immediately happen upon the final answer.

The answer is yes. I think it's hard to provide an argument for why that has to be true, but intuitively, it seems to me it would be quite hard to write exactly the right program in one shot all the time. And so there's some value in constructing a model, a hypothesis, or a program, depending on the context, and revising that with more information.

That more information could be that it doesn't work in some particular use cases, or that it doesn't support the evidence. Most of the hypothesis development that we do as humans is of this form, both as an individual and in science as a whole. Science as a whole collectively finds hypotheses, and then we revise them communally in conferences like this.

So it seems natural to me to say that we should build systems that have this kind of refinement process. One of the hard things to do in trying to mechanize this is: How do you guide or reinforce good refinement paths? How do you say, “Okay, this mode of reasoning, this mode of refinement, or this path of refinement is the good path, versus this other path, which is bad”?

Right now, in modern machine learning, people are doing this mostly based on either reinforcement from human examples, or, if you've got a well-defined objective function, you can backpropagate the signal from the correct solution, if you find it, through the reasoning paths. But that's not always the case. That's not always possible.

And so I think one of the big open questions is: How can we build systems that reason, where part of reasoning is refining hypotheses, when we don't have a well-defined, easy-to-compute objective function that we can evaluate at the end? I think if you can solve that, you can do pretty well.

Very cool. And any thoughts on that, Kevin?

Yeah, I just want to reinforce exactly what Zenna said right there, which is that a lot of this really hinges on being able to check that you're heading in the right direction.

We see this in our own work when we're trying to learn world models or programs for ARC tasks, because we have some data we're trying to fit, and we can check our fit to the data. I think you also see this in things like o1, where, to the best of my knowledge, they're mostly training it on math problems where you can check if you got the answer right.

And it is a big open puzzle, and I'm not sure I have great ideas on how to do this when you can't really check if you got it right. So I'm not really sure how to do that, but it does seem very important. One interesting fact is that when you're trying to learn a model of the world, you actually can check if you're getting it right. You can look at your data, what you've seen earlier, and say, “Okay, does my model correctly predict that data?”

Kevin, in the DreamCoder paper, you helped pioneer this wake/sleep fine-tuning strategy, essentially where a model can dream. It can expand what it knows, and then in the waking phase, it can reincorporate those dreams as hypotheses. It's absolutely amazing.

And of course, you're using it in a slightly different way in this new induction/transduction paper. Can you just tell us about the philosophy there?

Yeah. The philosophy there is tightly connected to what you see in a lot of machine learning, where you train a bottom-up model based on synthetic data produced by a top-down process.

Often, we're trying to solve some kind of inverse problem in AI. In vision, we're trying to look at an image and then infer the 3D structure. But the forward process of inferring the image from the 3D structure is a lot simpler. That's a rendering function.

So in wake/sleep, you're taking some forward process, imagining or dreaming possible ways it could run forward, and then learning how to go backward. For learning a function or a program, this means you're imagining programs, running them, and then saying retrospectively, “Well, I just imagined this thing and did this thing, so when I see this kind of behavior, I should infer this kind of program.”

What wake/sleep brings to the table, on top of just training on synthetic data, is that it's a kind of back-and-forth between learning from your own synthetic data and then learning how to make better synthetic data. It's not just this big batch-mode thing of lots of dreams and then some learning from dreams. You actually wake up, go into the world, try to solve problems, and then adjust your distribution of synthetic data based on the problems that you're solving.

This allows you to adjust to distribution shifts. You might think the world works a certain way and have certain kinds of dreams. You learn from those dreams, but then you wake up, see that the world is different, and during the next sleep cycle, your dreams shift to better match the world. So there's really a kind of cooperation between the wake and sleep cycles, and they're interleaved instead of being this kind of big batch of synthetic data.

I also think there's an abstract component here in intelligent systems. Certainly in all AI models, there's this iterative expansion and compression, expansion and compression. We see it even in neural networks.

You actually want to be able to cover a little bit more of the distribution, and I think this is where compositionality comes in and why it's good to do wake/sleep with programs. When you have a compositional language, you can take 2 atomic pieces of knowledge you learned earlier and glue them together in a way that you never actually saw but is plausible and which you might see. You should equip your neural network to be prepared for that composition.

So I don't think this is really a thing you want to guard against. You don't want to say, “Oh, I need to tightly fit my dreams to match my waking experience.” You actually want them to go a little bit beyond that.

And of course, it's interesting that in DreamCoder, it was an explicit process, and I suppose you could actually think of the induction/transduction paper as being the same thing. But it was using remixing and retrieval-augmented generation, because in DreamCoder, there was this notion of a library and expanding knowledge.

And now the process is becoming slightly more diffused. Can you sketch that out?

Right. Right. So we're moving more and more toward ways of implicitly doing the kinds of things DreamCoder was doing—

Yes.

—but building them on top of large foundation models.

Hmm.

Instead of having an explicit library of symbolic concepts that the system is learning over time, it has code that it knows was good in the past, and it uses a neural network to produce similar code. So this is effectively what library learning was doing. It was always saying, “I've written some code in the past. I'm going to learn some functions, and that will allow me to write similar code in the future,” except now it's done in a softer, more probabilistic way.

So I think actually you want to do both. There's something really valuable about how software engineers write reusable libraries. But it's tough to really write the correct library right now with AI systems. It's tough to automatically debug not just a program with AI, but a full library of programs.

And so as a kind of halfway measure, you can use in-context learning to approximate the kind of abstraction learning you would get from a library.

But in the end, I think they are complementary. We're not ditching libraries. We're just saying there's a kind of middle ground that's easy to implement and works reasonably well.

We want to build a library of knowledge, and there's an exploration–exploitation dilemma. How much entropy, for want of a better word—how much of that library do we keep hanging around?

Yeah. I think one way to think about it is, as a programmer, why ever build a library function? I think there are a few different reasons. One is that having a function allows you to express your current program more compactly, right? I've got some kind of shared structure, and I can reuse that in a variety of different ways within my current task.

Another related reason is more future-looking, right? People build libraries not for their current program, but in the expectation that they or other people are going to use that functionality in the future. So I think you might be able to think about, and perhaps even formalize, when and why you want a library. It's something like: in the expectation of current and future uses, it's going to make my life easier, right? I'm going to be able to use this thing that I'm caching now, perhaps sometime in the future.

We do this informally, again, as software engineers. We build things that we think other people will eventually use, and I think the mind probably does something like that in a very hand-wavy way. We're building structures that we expect we're going to be able to use and that we'd like to use sometime in the future, and we have this kind of cached computation that we can reuse at a future point. So that's broadly how I would think of it, and you can perhaps even try and formalize that and cash that out in some kind of rational decision theory.

Kevin, how can we test that these abstractions that we infer kind of represent the causal relationships in the real world?

You need to actually go out into the world. You need to have an agent that's in an environment, and it needs to be testing that its model faithfully describes real causal mechanisms. It needs to have an action space, be able to do interventions, and so on.

When we're just trying to learn functions, it's kind of hard to tell that your abstractions are the right ones, right? In a sense, there might be lots of equivalent ways of describing the same function space. But as soon as you put your agents in a world where they have to achieve a range of goals, where they have to plan and intervene on things, then you can actually falsify these hypotheses. In contrast, in program synthesis, you don't falsify a library. You just say, “This is not very useful right now. Maybe it'll be useful later, but it's not very useful.”

In contrast, if your program is making hard claims about how the world works, that's where you can actually say, “This is not faithful to the true causal mechanisms.”

Very interesting. We should bring in your Autumn paper as well, but I suppose you could say that something like ARC is non-agential.

ARC is not agential in the sense that there's no interactivity; it's a regression problem. You can try and solve it in a kind of internal agential way, right? I intuitively feel, when I'm solving ARC, that I'm doing little mental experiments to try and look at that, test that, and form this hypothesis. But the actual domain itself doesn't require me to take any kind of sequence of actions, right?

In contrast, most of the things in the real world are not like that. Most of the things in the real world require you to actually interact with the system through time. The canonical example is games, right? If I'm playing some video game, I'm taking some sequence of actions to try and explore it.

The Autumn paper, whose first author was a graduate student called Ria Das, was a joint paper with Ria, myself, and Josh. The goal there was to build a system that could synthesize essentially the source code of a video game after observing interactions with that video game. The background idea, or the background proposition, is that doing so is a kind of science, right?

If you can observe some dynamics and infer the source code of the underlying world behind those dynamics, that's something like understanding. That's something like building a model of how that world works. So the work that we did there was to build a little DSL for a certain class of interactive games, if you will. The way in which they're not games is that there's no external reward function; it's just an environment that a system can interact with.

From traces provided by humans of these games, or these environments, the Autumn synthesis method infers the underlying code. One of the key things we wanted to explore there was what you might call latent state: things that are true about the world but that you can't directly observe. The real world is full of things like this, right? Pretty much everything has some hidden state that we can't observe, and this hidden state is often very complicated and dynamic. It's probably best described by some kind of program evolving over time.

A key contribution there is: can we infer this hidden latent state in addition to the entire full program? But one thing, maybe going back to the previous point about abstraction, is that in that work, the programs or the models that we inferred are not abstract. They're ground-truth models of the world, and we don't really think this is how human thinking works or even could work, right?

There has to be some abstraction. There have to be some parts of reality that we omit or discard from our models. So a big open question is: how can you infer abstract models? How can you infer models that omit the right parts of the world in order to be practical and useful?

We didn't really explore that within Autumn, but this is very high on our minds as something we want to explore—and not just us, but other people in the community, too.

So we think in abstractions. She's on top of the world, and there must be some kind of hierarchy of abstractions. When we're dealing with perceptual input, for example, how do we navigate that abstraction hierarchy?

One thing that's interesting about how people think about problems at different levels of abstraction is that the abstractions are often defined on the fly for each problem. There's not one ground-truth abstraction. The world gives you data, and you could do the kind of Sora- or Genie-type world model where you truly model the full data and capture all the pixels, or you could do the kind of thing that we're advocating for, where you deliberately discard pieces of information.

When you do that, I think the problem just becomes underconstrained. In order to introduce the extra constraints that are needed to tell you what abstractions are valuable, the easiest thing to do—and something that certainly works—is to introduce reward. If you give a reward signal in the environment, then you can say that a good tower of abstractions, even if it's on multiple levels, is one that allows you to plan to achieve reward.

You see this in MuZero, where they're learning an abstract world model that isn't fully generative. It just predicts reward, policy, value, and so on. In some of the work we've done recently, we had these simulated robot environments where a robot is interacting with environments to use a tool or a mechanism. It sees pixels, but then it tries to define some abstractions on top of that.

The abstractions deliberately ignore a bunch of details. This was work led by Yichao Liang. It's called Visual Predicator, and it's taking this kind of MuZero perspective that someone gives you a reward signal. What I think is really interesting is that even if you don't tell someone what the reward is, someone can still play with a new object, a new web app, or a new appliance and form an abstract model.

We're still thinking about exactly how that would work in a program-synthesis context. A lot of the collaborations that we're planning right now are trying to answer that question. But I think it's very open. If you don't have reward, clearly humans can still figure out abstract models that, as Zenna was saying, omit a bunch of details.

It's trickier, and it might connect to certain normative theories of intrinsic motivation, or it might be something like wanting to be robust to a wide range of possible reward functions. There are a bunch of possibilities here.

Is there a principled way, though, of detecting, in a situation, which is the best level of abstraction?

Maybe.

The framework that I find quite compelling is the framework of resource rationality. This is basically saying that you should try to do the best you can with the resources you have, right? Maybe that’s a slightly convoluted statement, but the idea is that you have some kind of belief—a distribution of possible tasks or uses of a model—and you have computational constraints. You can’t run things forever.

I think a way in which you can cash out this question of when you should choose the right abstraction, or what kind of abstraction you should construct, is to say, well, I’ve got some beliefs about how I’m going to use this model: the questions it needs to allow me to answer and the tasks it allows me to do. These incur computational costs, and so I should consider all of those things and do the optimal thing.

Another axis I think is important is that Kevin said this kind of tower of abstractions. A key thing that’s somewhat overlooked in the current discourse on world models is that there isn’t a single world model, right? You can understand things at multiple different levels. There are multiple different models you can build of pretty much anything.

There’s a camera right here. I’ve got a model of how this camera works, at the level of, “I press this button and an image is taken.” But I can also understand the internal structure of the wires and circuits, or I can go down to the sensor in the camera. All of these are different models that are useful for different things that you need to do with that model. There’s no single correct answer except physics.

I think a key goal for us is to say, “Well, let’s embrace that plurality and try to find representations of models which incorporate a plurality of different models within them.” I’ve been using this term “polystructural” to capture this idea. We’ll see if that term sticks. But whatever you call it, we need to encode multiple different models of reality and the relationship between those models, right?

As a human modeler—let’s say, a formal scientific modeler—if I’m modeling COVID, I can say, “Well, hair color doesn’t matter in my COVID model,” right? But this relationship between the model and reality is encoded in my head as the scientist. We want that relationship to be within the computational formalism itself, and that is, in my sense, a hard scientific computer science question which I think hasn’t fully been explored. Maybe it will just emerge from scale and data. I don’t know. It’s kind of a question of whether we have to build these things in or whether they’ll emerge.

How can we automate this process of epistemic foraging?

I think what we want to avoid is building these Frankenstein systems where we hard-code a whole bunch of different knowledge representations and heuristics for reasoning with those representations. Instead, we want something which looks more like rational analysis from first principles. When you do that, you immediately run into hard computational problems. You get a big search space. It might even be hard, in the inner loop, to evaluate how good a model or abstraction is, because you need to retrospectively say, “Would this be good for the kinds of reasoning tasks I expect?”

That computational problem is a place where I think it would be good to insert learned neural networks that have good intuitions about everyday common-sense abstractions and so on, which can propose them. They can say, “This code would probably be valuable in this situation.” But we can use them to propose a bunch of alternatives in, as you were saying, the kind of Greenblatt-style way, more as a heuristic, so that we can still have this first-principles way of saying, “This would be a good collection of abstractions for the kinds of stuff I expect.” But then we can still take advantage of the kind of prior we get from pretrained neural networks.

I suppose the broad question is how much human engineering and seeding is required?

Yeah. If you look at the history of, let’s say, Bayesian computational models of cognition, a lot of which was done by psychologists and cognitive scientists like Josh Tenenbaum and others, there’s a really compelling and strong history of expressing some prior knowledge and showing that humans do something like approximate posterior Bayesian inference conditional on the data. But often in those cases, as you’re alluding to, it required smart people to look at the world and say, “Okay, here are the inductive biases. Let me encode these into the system.”

The bitter lesson of AI is that when you can, you shouldn’t encode explicit inductive biases. This will lose out relative to learning these from the data if you’ve got lots and lots of data and lots and lots of compute. So in my mind, there’s something obviously right about the fact that you need priors, and you can incorporate data to revise your beliefs. Again, I think Bayesian theory is a good normative theory for that.

But that doesn’t mean you have to adhere to the classical tradition of explicitly encoding these inductive biases. I think there’s a potential paradigm of saying, “Well, let’s encode priors, but let’s try and learn these implicitly, as much as possible.” Where do you learn these from? It could be something like the standard paradigm here in modern machine learning, where you’re learning these biases from large corpora of data.

They could be richer corpora, or corpuses, of data than just internet data, right? There are all of the things that you do as a human. I can observe you and infer some of your beliefs. So you could imagine richer sources of data than just internet data that would allow us to get closer to the inductive biases that humans have.

It’s tricky, as Kevin said, to actually implement these systems. You face hard computational problems. But I do think, broadly speaking, doing inductive inference over large corpora of data to learn implicit inductive biases, as opposed to explicit hand-coded ones, is a promising path to pursue.

And Kevin, maybe we should have started with this. What is an abstraction?

Well, it means different things in different contexts, for sure. But there’s always an element of hiding details.

In programming languages, abstraction is often synonymous with a lambda expression. So it’s a function. It has variables, and it’s ignoring what values those variables take on. That is a sense in which a lambda abstraction is an abstraction.

In the kind of stuff that’s sometimes called causal abstraction in causality, there’s also a kind of analogy or relation between 2 different causal models. The more abstract one is the one that’s ignoring details but still preserving some kind of essence of what the underlying causal model is. So it’s a word that means different things in different situations, but the analogy between all of them is that there’s some hiding of details but some retention of the essence.

What if we had a richer ontology to start with? We’re using symmetries, rotations, translations, and so on. What if we started doing some galaxy-brain stuff like causality and time and just put some different basis functions in there? Do you think that could have an uplift?

I guess one thing that’s true at the moment about the primitives that we put in is that they correspond directly to transformations to the ARC grid. Conceptually, I think it’d be cool to add, as you said, these galaxy-brain principles, but what do they correspond to in terms of the actual transformation that we’re trying to construct?

I think one actual promising area for new ARC approaches is precisely abstraction in the model—let’s say, the program, the transformation program itself. What do I mean by this? Right now, our approach and many other approaches synthesize something like a Python program, or literally a Python program, and then we apply this Python program to the input to get the output.

But that Python program is fully formed, right? It’s not abstract. It has all of its details there. You can run it, and that’s great because you can run it. You can see if it works. Intuitively, when you solve an ARC problem—at least for myself; I can’t speak for other people—you first find some abstract part of the rule, right? You’re like, “Well, I know that this object translates into this other object, but I’m not quite sure what the actual color transformation is,” for example.

And then I can go from there and say, “Okay, what could be the actual color transformation?” Conceptually, you can think of it almost like you write a sketch of a program in your mind with some holes, and then this gives you a direction to try to fill in these holes. You might fill them in one way, evaluate, and then go back and say, “That’s not quite right.” We don’t quite have, at the moment, in terms of the actual methods that we’re producing and other people are producing, these abstract program representations.

And I think that’s something we could actually build: a representation of ARC transformations that doesn’t have all of its details filled in, but is still useful as a partial solution on the way to a full solution. I think there are a lot of potential approaches of that form, where we’ve enriched the knowledge representation. And this isn’t quite what you’re saying about building a new kind of ontological idea. It’s like saying, “Well, we could actually abstract our current representations,” and that could be a powerful thing to do.

Kevin, when you solve ARC puzzles, can you talk through your conscious strategy?

Yeah. Sometimes it’s very intuitive, and I can’t quite describe in words exactly what I’m doing. It might be something more like, “I just denoise the input and imagine what it should be” in the parts that I can’t really see. Some things are definitely just perceptual and difficult to describe precisely, except by saying, “Well, you just denoise it.”

Other times, it is a very systematic thought process. I jumble up different ways of seeing it in my head. I see if it looks like it’s on the right track. I have half-formed hypotheses. It’s a much more perceptual and dynamic process than just the Greenblatt-style spamming out of thousands of programs, which, to be clear, we also do.

I do think it’s a little bit dangerous at times to introspect too heavily, and a little misleading. But I think even if you look at the kinds of mistakes that people make, they don’t exactly make the kinds of mistakes that these AI systems make. And that means that maybe there’s something about the dynamics of how we’re constructing the solutions that we’re not really capturing with any of these approaches.

Well, just comparing transduction and induction, one thing I think is good about induction is that it’s more compositional. I could mix and match programs together. It doesn’t intuitively make sense to me what would happen if you composed the transduction model. It feels like it wouldn’t compose very well.

But we could take this composed program and instead just think of it as functional programming, like a data-flow graph or something like that. Having those data-flow filters as first-class elements in the algorithm seems like a good step to me.

Yeah. You could imagine almost iteratively applying the induction model and the transduction model in sequence, all parts of it, assuming that the types match, essentially. You can also imagine almost a REPL-style approach to trying to solve ARC.

Suppose that you were given an ARC problem and you had a REPL, an interpreter environment, and then you could write code, evaluate it, check, and write more code. So instead of creating one big transformation that you run, you do it in a more step-by-step process where you’re continuously analyzing your current solution, writing some more code, and checking.

I imagine there are some approaches to ARC, and we’re pursuing some of these, which look a little bit more like that. You’re doing a step-by-step process where each step is producing code, and that code could be normal Python code, or it could be a transduction-like transformation, ultimately to get to a solution at the end.

Yeah, I love that idea. I think there’s something powerful about iteration, something really magical about refining a solution over time. What do you think about that, Kevin?

I strongly agree. In some sense, you don’t need it as much in ARC because you’re solving just 1 problem at a time. But if you think about an agent in a world that’s learning how to interact with many different causal mechanisms, then your agent needs to accumulate knowledge over time. It needs to revise its beliefs.

If you had something that was more factored, like the DAG that you were saying, or just anything that breaks up the knowledge even more compositionally, so it’s not just 1 program but lots of little programs that are all cooperating, then I think it would probably be better at ARC, and it would also be closer to what you need for something that can grow its knowledge over time.

So, Zen, with some of these solutions to ARC, we had Greenblatt’s and, of course, we had another paper with Wen-Ding and Kevin. Essentially, there’s this expansion where we do loads and loads of test-time computation, and I think in your paper, Kevin, you justified it as amortized dreaming. Is it in the spirit of ARC to be doing this massive expansion and all of this computation? Is that what Chollet wanted?

Well, I think we have to ask Chollet what he wanted.

Maybe we should. But what would you think he would say?

I think Chollet has said several times that ARC is an imperfect benchmark. There are ways to try and solve ARC that don’t necessarily lead to the fundamental insight that you might want, or that Chollet might want. And there are ways that are more fundamental, and I think the approach that we’ve taken is a mixture of both.

We’ve certainly got some ARC hacks in there to try and make it work, and there are also certainly some fundamental ideas in there that we’re trying to pursue. In terms of this kind of expanding horizon, I think the more that you’re trying to specialize to the particularities of ARC and build a DSL by going through and saying, “Okay, this is a useful element of ARC problems. Let’s include that. This is another useful element,” I think the more you’re diverging a little bit from the essence of the intention of ARC.

But there’s an open question of how much of that is necessary. You need inductive biases to solve ARC. They have to come from somewhere. I asked François Chollet this several months ago: “Do you think that a tabula rasa approach could solve ARC just from the examples within ARC? Is there enough knowledge or information in the ARC dataset as a whole?”

He thinks there is. He thinks that if you condition on all of the ARC problems, then that’s sufficient to solve ARC. You don’t need to pretrain on internet data. But it’s kind of empirically true that the best solutions are at least partially pretrained on internet data.

So, yeah, I think the ideal solution to ARC would be simple and elegant and wouldn’t require lots of ARC-related hacks and tricks. I think a robust way to try and get there is to introduce other problems that are related, or in the spirit of ARC, or capture some of the same things, but are not quite ARC, and kind of force solutions that work on ARC and this and that, to push against the ARC-specific domain hacks that humans might encode.

But Kevin, if you could design a better ARC, what would you do?

Well, that’s what we’re trying to do, in some sense. Yeah. This is part of this new Project Mara that Zen and I are doing. We want to have something that is, in a lot of ways, in the spirit of ARC, so we’re trying to learn something from very few examples and then generalize to new situations, but where you get to interact with something.

It’s not quite an MDP. It’s not like reinforcement learning. It’s more like a model-building exercise. I think this makes it a lot harder in some ways to just generate synthetic problems, which we did, because it’s a little trickier to generate lots of synthetic interactive environments.

You could do it, and I’m sure that we will try to do it. But it’s at least one way of introducing a forcing function that causes you not to overfit so much to ARC.

Zen, if philosophically, transduction and induction seem like duals to me. Certainly, from an expressibility point of view, the function space is the same. So why do we see empirical differences between the 2 classes?

A lot of it just comes down to representations of models or transformations. Again, I think programming languages are a good way to think about it. A neural network is a program in a class of programs, which is the class of neural network programs. And a Python program is obviously a program.

For any particular representation, there are some things that are going to be easier to encode and some things that are going to be harder to encode. If the languages are universal, then everything is possible, right? But some things are easier and some things are harder. It turns out to be the case that, in some class of ARC problems, some are easier to encode as Python programs and some are easier to encode as direct neural transformations.

But I think it is important to separate 2 different distinctions within transduction and induction. One is what I'm talking about now: the programmatic representation. The other is the type of object—the type signature of the object—that you're constructing. Is your system producing a function which takes as input an ARC instance and outputs an ARC instance, or is it a function that takes as input the entire ARC problem and then directly produces the solution? I think we have somewhat confounded those 2 things within the submission, but you could separate them and explore the different combinations. I think that would lead to different trade-offs in different ways.

To be a little more concrete, I think Python programs are obviously good at expressing loops, deterministic computations, and things where you've got maybe even an unbounded or variable-bounded number of computations. Whereas a transformer model is a finite model, right? It has one pass, and it produces an output. There's a lot of interesting work showing that this corresponds to a particular class of computations, and there are things that you can express within that and things that you can't. A lot of it comes down to the representation of programs.

Yeah. Interesting. Because I suppose, Kevin, philosophically they're duals, but, as Zenna was just saying, from a computational point of view, one is a transformer, so it's a finite-state automaton in the class of automata, and a Python program is Turing-complete. They're fundamentally different, but there's this weird kicker: the types of things that this limited form of computation—a neural network—can do can express programs that no human knows how to write.

Yeah. I don't really have a theoretical handle on that. I can't justify theoretically why a neural network should be able to do computations that are really tough to do in Python. It's an empirical fact. Obviously, Python can do things that a neural net is going to really struggle with.

As Zenna was saying, in the paper we found things like counting, or when we need to systematically do the same thing to every single object, Python is great for that. The other stuff is hard to really theoretically justify. I can just say that empirically, it's definitely true. We ran the induction-transduction model with different random seeds because we wanted to make sure this was not just an artifact of randomness. It really is just empirically the case that there are certain kinds of ARC problems, and I think many other kinds of problems, where Python in principle could do it, but a neural network is just much better for the job.

I think it was both of you. I'm not sure, but there's this Program Induction by Example paper, which is at NeurIPS. I spoke to Wen-Ding Li about it. Are you both on that, or is that just you, Kevin?

It was me and Wen-Ding Li.

Oh, amazing. Tell us about that.

That paper, in some sense, was our first attempt at trying to do something that was like wake-sleep in the DreamCoder style, but in a modern LLM setting. It starts with some human-written programs, on the order of 10 to 100, and this implicitly defines a generative model over code, because you can prompt an LLM with these example programs, and then it will make up similar programs.

This means we have essentially a forward model. We can imagine programs, and this gives us the dreaming phase of sleep because we can imagine these programs, run them, see what they do, and then train a program synthesizer based on the programs that we generate. We found that just doing this was pretty good at program synthesis problems. It was frankly a lot more effective than most of the lambda calculus stuff that I did in my PhD, and it substitutes a lot of symbolic machinery with neural machinery. In doing so, it was able to take advantage of a lot of the advances in scale that we've had in recent years.

The thing that I've described so far, where we make synthetic data and then train a synthesizer on it, was what gave us the induction model for the ARC paper. In the paper with Wen-Ding Li, we also introduced a full wake-sleep cycle where it then tried to solve problems, remembered those solutions, and dreamed about variations of those solutions.

This means that if your prior is mismatched to what you really care about—so imagine you only think you need to write short programs, but the real world has long programs—then you can pre-train on short dream data, go out and encounter the harder problems, solve a few of them, and then, during the next sleep cycle, fine-tune your model to this new data distribution. It doesn't have the library-learning component of DreamCoder. It has this softer, neural, in-context-learning-type generative model. We're trying to bring this wake-sleep cycle into program synthesis, but in a more modern, more scalable setting.

Zenna, one thing I'm seeing here is pragmatism prevailing. The connectionists are embracing hybrid models and neurosymbolic approaches and so on, and maybe even folks from your camp are embracing connectionism as well. What are your reflections on that?

As Kevin said at the start, there's something obviously right about scale, and something obviously right about learning. A connectionist architecture allows you to learn from large amounts of data. There's also something obviously right about symbolic architectures, in the sense that most of modern society is built on top of them. I think there's also something right about general normative principles of intelligence.

Given that, the question is: What should you do? Can you take the things that we know to be right and compose them in a way that makes sense? I think you're seeing that in a variety of different ways.

From the large labs, let's say OpenAI, they have large models that they've trained on huge amounts of data. But they also have these scaffolds around them, where the models call out to Python to do some computation. Why is that a smart thing to do? Because Python is an effective language for doing a certain class of computations, much more effective than a neural network for a large class of things. They've come to a kind of business decision that it's useful to have this kind of hybrid system.

From the other perspective, there's also a history of symbolic systems, and I've realized from a different perspective that you can't model the entire world through a list of propositional formulas. The real world is a lot messier and more complex than that. We want to do nice reasoning, but we have to handle the complexity of reality. This immediately leads to architectures which can do both of these things.

I think both the complexity of the world and what we want from reasoning and learning systems are leading to a convergence of ideas. Right now, I think that convergence is composition: taking these systems and plugging them together. The real question is, can we do better than that? Can we go from the ground up and re-engineer systems that have the functional things that we want but may not look architecturally like the systems that we have today?

Yeah. I'm very excited about building systems with LLMs, with the best components of everything. Zenna, you're building Basis. Can you tell me about the Everyday Science project?

Yeah. Project Mara is a project within Basis. We have a few projects within Basis, but Project Mara is a new project, and it's led by myself and Kevin. This is a 3-year program where we're trying to build upon many of the ideas that we've been talking about today, really focused on 4 components, which is what Mara stands for: modeling, abstraction, reasoning, and agency, or acting.

As a first approximation, you could think of it as active ARC. Can we build systems that model the world, abstract the world, reason with these models or reason to find these models, but do it in an interactive way? They have to be part of the environment and take actions to learn about how that environment works.

And so the way we've structured it is to do 2 major classes of things. One is to develop new benchmarks and new kinds of problems to solve, and the other is to develop new algorithms to solve those problems. The first thing we've done is take an existing benchmark, ARC, and make our best effort to solve it. We'll continue to do that, most likely. But we're also developing new benchmarks.

Within this high-level goal of building what you might call a general MARA system—a system where you can plug it into the world and it takes actions to learn about that world and build an internal model—we're focused on a narrower subset, which is what we call everyday science.

We've discussed this today in various contexts, but the intuition is that there's real science. That's what we do as chemists, biologists, and computer scientists, learning about the physical and artificial world. But then there's also everyday science: what we do as normal humans, adults and children. We learn about a new AC system in my hotel, a new microwave, or a new toy.

We think that the principles underlying this everyday science—how we form hypotheses, how we revise those beliefs, and how we take actions to learn about how the world works—are the same principles in everyday science that apply to real science. And so we want systems that can do that: learn how to interact with new toys, devices, and interfaces in a way that isn't large-scale imitation learning.

Going back to your earlier question, I would say this is part of a real approach to AI and to science, even if it's not actually discovering new useful science. So, joint projects, and ARC was our original—or our initial—output: this ARC solution, this induction-transduction model. We're just at the start of planning a whole program and building a whole team. Maybe Kevin can give his flavor and interpretation of what we're doing.

This is very ARC-like in the sense that you're building a model on the fly from very small amounts of data, but you are not passively receiving the data. You have to go out there and poke things, push things, and try things out.

Zenna and I both did graduate work in cognitive science departments, and I think, as a cognitive scientist—well, cognitive-science-adjacent—it's very exciting from that perspective. This is the kind of thing that's science-like, that humans do in everyday life. But I think it's also practically important because increasingly we're building these AI agents, both in digital and physical worlds.

They work well as long as their prior is well aligned with the kind of environment they're already in. But when they're faced with a new kind of webpage, or if you imagine a robot that has to figure out how to use a new kind of dishwasher, it has to experiment with different buttons. That's actually quite hard. The more abstract kind of knowledge you need to learn, I think that's a really exciting challenge.

Before we go, are you looking for investors? Are you looking for researchers?

We're certainly looking for researchers. If you're a research scientist or research engineer and you want to work on hard and interesting problems that are a little bit outside of the mainstream of what some of the larger labs are doing, then get in touch.

Investors, that's a little bit more complex. Certainly, BASE is a nonprofit, and so if you want to donate, feel free. But the project is funded, and we have an ambitious 3-year program. We can obviously make it more ambitious, but I think that would already be quite hard.

We're just excited to get things started and move quickly and find the best possible people to work with. Also, other collaborators: if you're doing adjacent things, that'll be a cool area to connect with.

It was such an honor to meet you guys. Thank you so much, and keep doing the great work.

This was fun. Thank you for having us.