Randall Balestriero 教授——无需预训练的 LLM 与 SSL

Prof. Randall Balestriero

We just launched this experiment, and we were very surprised to see that the hugely over-parameterized model not only trained out of the box—you have very nice training curves—but also didn't overfit aggressively at all. What we found empirically is that we can just use typical supervised training out of the box. We don't have to play with hyperparameter optimization, and you have very, very stable training.

So this also brings up the question: is it worth spending so much money to gather a gigantic pre-training dataset and spend months on many GPUs to produce those models? At least for some applications, it seems not to be much better than random.

Tim Scarfe

MLST is sponsored by Two for AI Labs. Now, they are the DeepSeek based in Switzerland. They have an amazing team. You've seen many of the folks on the team. They acquired Minds Eye, of course. They did a lot of great work on Arc. They're now working on o1 style models and reasoning and thinking and test time computation. The reason you wanna work for them is you get loads of autonomy, you get visibility, you can publish your research, and also they are hiring as well as ML engineers. They're hiring a chief scientist. They really, really want to find the best possible person for this role, and they're prepared to pay top dollar as, as a joining bonus. So if you're interested in working for them as an ML engineer or their chief scientist, get in touch with Benjamin Cruzie. Go to twoforlabs.ai and, uh, see what happens.

Prof. Randall Balestriero

1. Pretraining Loses Its Edge

Originally, the main motivation was to see how much information you gain by doing pre-training, right? Is this next-token prediction really making your network learn something about language and reasoning? One way to compare this, at least empirically, is to take a randomly initialized model and train it from scratch on a supervised task like sentiment prediction or sentiment analysis.

In theory, because we have a very, very small training dataset—let's say 20,000 samples—and because those models have 7 billion parameters, the pre-trained one will perform very nicely with a little bit of LoRA fine-tuning because it already knows how to reason about the world, right? Maybe you just adjust it a little bit to the specific task that you want. Since you have so much prior knowledge, you will solve the task very easily. But the random one will either overfit completely because you have 7 billion parameters and only 20,000 training samples, or maybe it will not learn at all because the training dynamics will be completely chaotic.

We just launched this experiment, and we were very surprised to see that the 7-billion-parameter, hugely over-parameterized model not only trained out of the box—you have very nice training curves, almost like you're training MNIST—but also didn't overfit aggressively at all. It overfit less than if you just trained an MLP on MNIST, basically. This is very surprising.

From this, we said, “Okay, actually, maybe there is a deeper question we could ask: how much implicit bias do you have in these language models?” We already knew from computer vision that, for example, with ImageNet, you can have a 50-million-parameter model on a 1-million-example dataset, so you have this 50-to-1 ratio and you have the implicit bias that prevents you from overfitting and just solving the task, right? But still, it's 50 to 1. This may sound like a lot to a statistician, but now it's 7 billion to 20,000. The ratio is gigantic, right?

To me, it was very surprising that the size of this ratio still allows you to learn something that does not overfit. This is very surprising because, in vision, for example, transformers are known to overfit more easily than ResNet. They seem, at least in vision, to have less implicit bias or implicit regularization. But with this type of next-token causal architecture, or LLM, you don't seem to overfit easily to your data.

Tim Scarfe

Yeah. We should bring in the name. This was your workshop paper at the Self-Supervised Learning Workshop here at NeurIPS, and it's called “For Perception Tasks, Is LLM Pre-training by Next Token Prediction Worth the Cost?”

Prof. Randall Balestriero

Yeah.

Tim Scarfe

This is absolutely fascinating, right? We've been given this belief that we need to have these huge pre-trained models. They're trained on all the data on the internet, and it turns out that, certainly for discrimination tasks—things like classification rather than generation—you can just start from scratch with a fairly small model and sometimes get even better results.

Prof. Randall Balestriero

Yeah, even with a small or a large model, you can just start from scratch. You do this very simple supervised classification task: given this prompt, is it a good or a bad sentiment, or what type of job is the prompt describing? This type of semantic classification—and I will not call it reasoning—turns out to work well from a random initialization. Even if you have a small training dataset, you will have performance that is sometimes as good as that of a pre-trained model.

This also brings up the question: is it worth spending so much money to gather a gigantic pre-training dataset and spend months on many GPUs to produce those models? For generation, there is no question that this is what you need to do. You have your next-token prediction, and you learn how to generate samples. But at least for some applications, it seems not to be much better than random. So it's quite interesting.

Tim Scarfe

So what are the differences in the learned representations?

2. The Representation Question

Prof. Randall Balestriero

That's something we do not really look at, such as the low-dimensional representations of what you learn. It's possible. Some work tries to look at attention entropy and the like—those mechanistic interpretability viewpoints of LLMs.

It would be interesting to see if you have this sort of neural-collapse phenomenon that happens. Even if you have a 7-billion-parameter model, maybe you end up learning a very, very simple subnetwork that does the task, a bit like the lottery ticket hypothesis, and that naturally emerges from the training dynamics. Or is it really exploiting all the parameters?

I think that's one thing. To extend the workshop paper to a conference paper, we want to probe more into what the useful parameters are and what they learn. Is each layer actually learning something, or maybe the first layers don't really learn anything and only the last few are learning something? There are lots of open questions here.

Tim Scarfe

What does it tell us about the nature of understanding and maybe even intelligence? We think that the reason these things understand is that they just have all of these representations of all of these different things in their experience.

Prof. Randall Balestriero

Yeah.

Tim Scarfe

And, and now we can shortcut to, to, you know, to want of a better word. What does that tell us?

Prof. Randall Balestriero

Yeah, I think that's a good question. In this case, we must look at very specific classification tasks.

Tim Scarfe

Yes.

Prof. Randall Balestriero

For example, you have a description of a job: what job is it? Is it a good or bad sentiment? You are able to solve this well, but you are not able to go out of distribution to solve a new type of question. For this job description, you cannot answer whether this job pays more than another job because that was not present in the training data, right?

I think you get very good models cheaply and quickly from random initialization, but they will be very specialized. The benefit of having pre-training may come if you want to do more open-ended classification or reasoning. It really depends on the type of application you want to solve, what your downstream task is, and how much you want to generalize to new scenarios.

At least now, it shows that pre-training with next-token prediction is not just better for everything.

Tim Scarfe

Going back 5 years, data scientists used to build specific classification models for doing everything.

Prof. Randall Balestriero

Mm-hmm.

Tim Scarfe

Now we're in this regime where we need these really big models—

Prof. Randall Balestriero

Mm-hmm.

Tim Scarfe

—and we do in-context learning and maybe—

Prof. Randall Balestriero

Mm-hmm.

Tim Scarfe

—even some fine-tuning and—

Prof. Randall Balestriero

Yeah.

Tim Scarfe

—we get them to do fairly specific discriminative tasks. But now you're saying—

Prof. Randall Balestriero

Yeah.

Tim Scarfe

—we should almost go back to where we were 5 years ago and start building specialized models again. Only now, rather than building classification models, we're actually—

Prof. Randall Balestriero

Mm-hmm.

Tim Scarfe

—we're still using the transformers and the LLMs, but we're making them do specific tasks.

Prof. Randall Balestriero

Yeah, exactly. I think if you only want to solve a few specific tasks, use this prior knowledge to have a nice architecture and a supervised dataset for that, and just do that from scratch. This is something that's probably going to work much better.

But again, you need to make sure that the downstream application will never go too far out of distribution. That's why it really depends on the application and the type of use cases that you have.

But I think, at least here, it shows that there exists some task where next-token prediction is not the answer. In fact, it’s not just not the answer; it’s not better than random initialization, which is really the worst-case scenario.

Tim Scarfe

Interesting. From a fairness and bias point of view, a lot of people say that large language models are bad in a way because there’s a dominance of North American cultures and so on.

Prof. Randall Balestriero

Mm-hmm.

Tim Scarfe

But you could also argue the converse, which is that the good thing about them is that they do have some awareness of value. We can fine-tune them to have guardrails and to sort of say the right thing and so on. Is that harder to do with this approach?

Prof. Randall Balestriero

Yeah. Here, because you’re in a fully supervised setting, you don’t have as much flexibility to change the behavior of your model, or it will have to take the form of supervised fine-tuning. But because you don’t have a generative capability, it certainly restricts the type of interaction you have with the model and how you can improve it.

The output is just, “Okay, is it a good or bad sentiment?” It’s not something that gives you a full answer that you can then try to argue against and generate a fine-tuning dataset from. It’s just, “Okay, good, bad,” and that’s it.

Tim Scarfe

Another thing is training strategies. The big players building these LLMs have lots of internalized knowledge around even the order in which you train the language models. Everything is important.

Prof. Randall Balestriero

Mm-hmm.

Tim Scarfe

Certainly, in the old days of basic models, you just stuck a load of data in there.

Yeah.

No one really cares.

Yeah.

Now, do people need to be thinking about specialized knowledge, maybe thinking about curriculum learning and all of this kind of stuff?

Yeah, this is a good point. We did a paper recently called “The FAIR Language Model Paradox,” where we show that when you do next-token prediction, because you have some tokens that are very low frequency, it’s very hard to train on them, and it takes a very long time to train, so it’s very wasteful.

The problem is that because you do this next-token prediction, you need to really capture the entire distribution of tokens, and so you spend a lot of time. But in this case, if the low-frequency tokens are not useful to solve your task, you actually don’t need to capture them at all. In terms of training dynamics, this is actually a much simpler problem in many cases.

What we found empirically is that we just use typical supervised training out of the box. We don’t have to play with the hyperparameter optimizer, and you have very, very stable training. That’s one thing that could also be interesting for future work: is this something that is easier to optimize? Maybe that’s why those 7-billion-parameter models can learn and not overfit on 10,000 samples.

It also brings up other things. Maybe this, on its own, could be a better initialization for next-token prediction as well. This is very open, but maybe you could think of a simpler supervised objective that would be a better pre-training solution, which you could then use for next-token prediction if you wanted to. At least this would be a better starting point than random, so you almost reverse the trend.

3. The Multitask Middle Ground

Tim Scarfe

We’ve spoken about two extremes. On one extreme, we have pre-training, and you can use it for any downstream task. On the other extreme, you start from scratch with just one task. Is there an intermediate solution?

What if I did this new approach but for multitask learning—for, let’s say, 5 tasks?

Yeah, that’s a great question. If you really think about it, in the limit, you could formulate next-token prediction as a multitask problem where each task is predicting whether the next token is a particular one or not.

In the extreme case, you could recover next-token prediction on one end, and on the other end you have what we have here: just one very coarse, high-level task, such as predicting whether it’s good or bad sentiment or whatever. In between, you have a huge spectrum that you can exploit.

If you can find, as you said, maybe 5 very different, representative tasks, this should be enough—or could be enough—to learn a representation that is as general as possible. Then you can use this for new tasks that come along the way.

I think the research question is how to design the minimum number of tasks so that you have as diverse a representation as possible. Of course, you don’t want to go to the extreme of just doing next-token prediction again. But this is a very nice research question, because if you have this spectrum and you can control where you want to be, then you can really have a per-use-case choice.

It’s not, “Okay, you’re always here or always here.” Tell me what you want to do and how many new tasks you expect your model to be exposed to, and I’ll tell you where you need to be in this spectrum. This could be very interesting as well.

Tim Scarfe

Very cool. It does make me think, though, that these models understand through naive statistical alignment. Is it possible that the benchmarks we use just don’t capture the gap in understanding that we’ve lost from moving away from the pre-trained models?

Yeah. Especially in recent years, we’ve focused a lot on generative, decoder-only methods. All the evaluation and the types of objectives we put on ourselves are really about good generation.

Even if you want to answer a question, you need to generate a good explanation, and you need to understand what the intermediate steps are. I think the fact that we focus on generative models means that we completely bias the evaluation and the way we approach this problem.

Maybe you could still have knowledge that is learned without being able to generate anything. I think this is also something that could be interesting to look at, or at least to keep in mind, when we explore these models.

Tim Scarfe

But philosophically, isn’t generation analogous to thinking in some sense? Don’t models that generate become smarter in some deep way?

4. Concept Tokens Beat Next Tokens

Probably what you want to do is imagine what could be, but I don’t think you want to do generation with very granular details, like next-token generation.

If you think about it, even in terms of a classification task, you have a lot of different uncertainty depending on the token. If I start the sentence, “Okay, I saw this movie for minutes,” there’s no way you can tell what the next token after “for” should be, right? You know a priori that it will be a time component. Maybe it’s 1 hour, 10 minutes, or 2 hours.

But do you really need to be able to generate, I don’t know, “52 minutes,” or whatever the answer was, to actually understand that I was seeing a movie and therefore staying in a place for at least more than 5 seconds? I think the token is way too granular.

If you had something like a concept token, that’s where you could start saying, “Okay, this is meaningful,” because that’s closer to what we do. But right now, we’re very, very low-level, because tokenization is a lossless compression. This is too close to the raw data.

Yet we have it easy compared to computer vision, because we already work in language, which is a very compressed representation of knowledge. Still, the token is probably too low-level.

Tim Scarfe

Well, that was a fascinating paper.

Thank you.

Tim Scarfe

Let’s move on to your next one: “The Birth of Self-Supervised Learning: A Supervised Learning Perspective,” which was with Yann Le Cun.

Yes.

5. Self Supervision Reframed

Tim Scarfe

Basically, you said that the observed differences between self-supervised learning and supervised learning are not due to the loss functions themselves, but rather to the labeling of the dataset used in training. Give us the elevator pitch.

Yeah. What we show in this paper is that you can have a supervised objective, such as least squares, to make it simple. You have the inputs, your network’s prediction, and the labels. You can turn this objective, which tries to predict sample xₙ to prediction yₙ, into a self-supervised learning objective that tries to compare samples with each other.

Basically, you go from saying, “Okay, this image is a car or a dog,” to saying, “Are these 2 images the same or not?” That’s the self-supervised type of joint-embedding world.

You can show that if you have labels, or knowledge of this pairwise relationship, they’re actually learning the same representation, up to some symmetry that is irrelevant if you do linear probing. The loss function itself—the self-supervised one or the supervised one—tries to do the same thing.

They just operate on a different view of the labeling: whether this image is that, or whether those two images or samples represent the same thing.

Given that, the next question is: how come self-supervised learning is able to generalize better than supervised learning? From this perspective, what you can say is that it is as if they were solving a supervised task where the labels are not about predicting all the cars as cars, but are very, very, very fine-grained labels where, in the limit, each image is its own class, basically. So if you think about supervised learning in this extreme setting, you also do not overfit to the task because you do not collapse any image into another one. Theoretically speaking, you can solve as many downstream tasks as you want. So this equivalence of losses at least brings a slight new perspective on the fact that it is not really about the objective; it is more about how you design the SSL pipeline. You say, “Okay, this sample is related to this sample,” but it is not the objective that makes you learn a better representation.

Tim Scarfe

Okay. And in the paper, you were talking about how SSL can maximize the worst-case downstream task performance. Can you sketch that?

Yeah.

How does the class balance affect the difference in the losses?

Oh, yeah. So this is a very good point, actually. In a follow-up paper we are doing right now, we show that current SSL objectives assume class balancedness. This is something we already highlighted quickly in our paper on self-supervised learning as a uniform cluster prior, which we did a couple of years ago. We show that current SSL objectives assume balanced representations of classes or concepts.

This means that if you train on ImageNet, things work out very well because concepts are sort of equally represented. But then if you go to another dataset like iNaturalist, which is very heavy-tailed, you have a huge bias in your representation. Until now, people did not really know how to solve this. One way people approached this was through data curation, saying, “Okay, I’m just going to remove the oversampled concepts to try to make it more uniform, and then I’ll do self-supervised learning on this.”

But because now we have this theoretical formulation and this equivalence of losses, we can use the exact same setting that people use in supervised learning to reweight based on the frequency of classes. We can use that to come up with a new self-supervised learning loss that takes this imbalance into account. This type of thing is enabled by this mathematical formulation and its principle. The way we do this weighting, you can prove that it is the right way to do it from this supervised theory.

This is really nice because suddenly, from this seemingly naive connection, you can now come up with a new generation of self-supervised learning models where you can actually match the real-world data distribution. So, for a nonuniform distribution of classes, or maybe even if you have some samples that are noisier than others, you can include that information as part of the SSL objective as well. Suddenly, you have a whole new world of possibilities, and because there is this connection, you can actually prove that this is the right way to do it, at least from the supervised theory viewpoint.

Tim Scarfe

You also pointed out a connection to VCREG.

Prof. Randall Balestriero

Exactly. So basically, what we do in the paper is show that if you have a least-squares supervised-type objective and you turn it into an SSL one, what you obtain is basically VCREG. You then have a few variations. It could be VICReg or W-MSE, depending on how you do this conversion from supervised to SSL. You can choose that depending on the type of supervised loss, and you recover different types of SSL losses.

If you look more at cross-entropy supervised learning, it is going to be more like a SimCLR type of loss. But you have this one-to-one correspondence, and this is also very nice because in supervised learning, at least, you know when one loss may be preferred compared to another one. This has been studied for a long time, because supervised learning has been around forever, and now we can reuse those insights for self-supervised learning.

To me, this is also a very, very strong benefit of this: suddenly, all the theory and the thousands of papers that have been done in supervised learning can be taken and applied to SSL. Another example is neural collapse, which has been proven in the supervised setting. Now it applies in five lines in an SSL setting as well.

This connection is really beyond just trying to say, “Okay, it is not the objectives that make SSL better.” It is really tying those 2 huge communities together toward a goal where you have a single unified objective to learn representations. This is nice, too, because if you speak to people, they will think, “Okay, you have supervised learning on one side and SSL on the other side, and basically, you are either in one camp or the other.”

But now what we show is that SSL is pretty much everything in representation learning, and supervised learning is just one realization of SSL. Then VCREG without labels is another one, and this one is another one. So you really have a better understanding of this relationship and what representation learning is trying to do.

Tim Scarfe

Galaxy-brain question incoming. Could you combine SSL and supervised objectives in some way to improve generalization?

Prof. Randall Balestriero

Yes, yes. There is one paper on supervised contrastive learning. The way they do it is that they use the labels within a SimCLR framework to basically do fully supervised learning, but with a SimCLR objective.

First of all, we can show that this indeed makes sense and that we can explain the empirical results that they obtain. But actually, we can do a little bit more than that. If you are in a semi-supervised setting, for example, it may not be clear how to combine those 2 losses anymore. Or maybe you could say, “Okay, I have the 2, and I have a coefficient to weight them,” but then you need to do cross-validation and so on.

From this perspective, you can combine them in a very principled way and understand which weighting makes sense depending on how many samples you have in one setting or the other. You can use all the literature from supervised learning for this setting as well. This is something you can do very easily with this formulation.

Tim Scarfe

Okay. So if SSL and supervised learning are 2 sides of the same coin, of course we can use this theoretical framework to design new forms of SSL frameworks. But does it—you know, is the distinction relevant if they are the same thing?

Prof. Randall Balestriero

I think it is not just 2 sides of the same coin. SSL is more generalized than supervised learning.

Tim Scarfe

Right. Right.

Prof. Randall Balestriero

So really, SSL could be the more general objective for learning representations. The more prior knowledge you have, the more you know about your downstream task, and the more you know about your labels, the more SSL slowly becomes supervised learning through the labels that you use for the SSL objective.

But then, because, as you said, you have this hierarchy now, it does not really make sense to say you have either supervised learning or SSL. Rather, what makes sense is to say, “Okay, what is this relation matrix? What is this pairwise matrix?” If you build it from labels, it is supervised learning. If you build it from other a priori knowledge—for example, 2 consecutive frames in a video are basically of the same class—then you are more in an unsupervised SSL setting.

But it is all about how you build this pairwise relation matrix. That is the main question.

Tim Scarfe

Very cool. Right, let’s move on to your next paper, “No Location Left Behind: Measuring and Improving the Fairness of Implicit Representations for Earth Data.”

Prof. Randall Balestriero

Yeah.

6. Fairness in Earth Models

Tim Scarfe

There are loads and loads of modeling frameworks now that use these implicit neural representations of geospatial Earth data—things like climate modeling, resource allocation—

Prof. Randall Balestriero

Exactly.

Tim Scarfe

—environmental modeling. I was actually interviewing Johannes from NXAI yesterday.

Prof. Randall Balestriero

Okay.

Tim Scarfe

I do not know if you know him, but he is working on similar stuff.

Prof. Randall Balestriero

Okay.

Tim Scarfe

The problem is, you have studied this and found that there are loads of biases and fairness problems.

Prof. Randall Balestriero

Yeah, exactly. So basically, what we show is that when you want to model, for example, temperature or precipitation, to make it simple, and you want to learn an implicit neural representation, it means that you want a model such that if you give it a location and a date, for example, it can predict what the temperature was there.

So if you have this type of implicit neural representation, it's very good because if you learn a nice model, then you can actually interpolate those values. Maybe you can estimate what the temperature was in this part of the globe where you did not have a sensor, but you can also do extrapolation as well. If you assume you really learned the true physical model of the world, you could start saying, “Okay, what will the temperature be two years from now?” So this is very nice to have this type of model for all sorts of applications.

The thing is that when you do this nowadays, depending on the architecture and the different design choices that you make, you may have very good predictions on average, so when you look at the average performance around the whole globe. But actually, if you look, for example, around islands or coastal areas, your prediction is going to be very bad, almost random. So this is something that can be very concerning because if you use this type of model to decide about a policy that will affect a specific island, using this model’s prediction is as good as using random guesses. It can be very detrimental, and people need to be aware of those biases.

What we found is that, for example, for this type of climate data, islands are often disregarded, as are coastal areas—basically, regions where you have a big gradient in the type of data that you try to model.

Tim Scarfe

How much responsibility do modelers have to detect these kinds of biases in the data?

Prof. Randall Balestriero

I think there are 2 components, as you said. One could be that just the dynamics of the data you are trying to model are harder near an island, or maybe it's even unpredictable because you don't have enough observations to do that. So you have some uncertainty that you probably can never recover from good design.

But still, what we found here is that a lot of the bias now comes from the architecture and how you encode those positions—the type of basis you use to do the prediction. So right now, it seems that a big chunk of the bias comes from the architecture, but I totally agree that I don't think we can remove the bias entirely, because there may just be different types of uncertainty in different parts of the planet as well.

Tim Scarfe

The world is a very, very complicated place. Realistically, to what extent can we mathematically model it?

Prof. Randall Balestriero

That's a good question. I think it depends on the type of horizon that you have and the type of data that you want to model. If you have a system that is much more chaotic or can vary very quickly without much change in past observations, that's something that current models are having a very hard time with.

If you want to predict something else, for example, temperature in North America—not near the coastal area, so really inland—maybe you have less gradient dynamics. Things are a bit more stationary, especially through time, so then it can become much better. But I think at this point we don't have an architecture that is really able to understand that you have different physics and different dynamics models in different parts of the globe. Because of this, you just see what's best on average, and it means you miss out on a lot of details.

Tim Scarfe

Can you tell us about some of the technical framework?

Prof. Randall Balestriero

One thing we showed, for example, at least for this type of global data representation, is that people use a Fourier basis to model the prediction. This is better than not using any basis at all. But what it means is that you imply the type of signal you're predicting is very stationary and not localized at all. This is a very strong prior, right?

So this may be true for some things, but for other things like precipitation or temperature, where you have localized, very high gradients, then it's a strong bias. If you come from the signal-processing community, you know very well that to have better localization, you go from Fourier to wavelets. That's one thing we did in this paper, and we showed that using a wavelet basis to encode that data allows you to have better localization, and this removes some of the biases.

Here, it's more of a proof of concept that different design choices give you different types of bias trade-offs. Wavelets are not the answer to everything, right? But I think the next step is to really be able to encode less and less a priori which basis to use and let the model learn from the data on its own. We are not yet at this point, at least for this type of climate data.

Tim Scarfe

How could it handle noisy or missing data?

Prof. Randall Balestriero

This depends really on the type of model you use. For example, if you have an INR, then you will not use the missing data as part of your training pipeline, and that's one of the benefits of them. So if one of your sensors stopped recording for some years, you just don't use that as part of your training data, because you really control where you have the data, when you have it, and what the prediction should be.

Tim Scarfe

These Earth models are now informing policy around the world. Who should we hold accountable? Is it the technology? Is it the scientists who design the models? Is it the policymakers who interpret the results?

Prof. Randall Balestriero

I think it's very hard for the person who designs the model to know a priori what it's going to be used for. So I think it's more downstream, when you know clearly what you want to do with it. You should first set up a nice evaluation pipeline to make sure that it's something you can actually use to make those decisions, and then you can report any type of failure modes you observe for people to improve on the design.

A priori, it's very hard to imagine what this model will be used for. In the ideal setting, you wish that there would be no bias at all. But in practice, the world of possibilities is so large that it needs to be more of a feedback loop, and then you iterate until you have something that you can really trust and act on.

Tim Scarfe

Earth modeling data is very anthropocentric, right? We focus on human populations and so on. Should we also focus on ecosystems and places that have nothing to do with humans?

Prof. Randall Balestriero

That's a great question. In fact, that's one of the big issues with a lot of the datasets that are crowdsourced. By definition, the amount of data that you get is proportional to the number of users you have, depending on the location. This means you have a huge bias in what your model is learning and what your model is focusing on, which means you miss out on a lot of things.

Crowdsourcing can give you a lot of data quickly, but it's very biased data. So then the question is, how much of this biased data versus maybe paying a lot more and capturing other parts of the globe should you have? Maybe you could show that under some specific conditions, just having 10% of the data—which is high-quality and uniformly sampled—and then 90% that is crowdsourced, you can try to use that 10% to anchor your representation and then use all that data together. But there is a huge amount of research in that, because that's a very big source of bias.

Tim Scarfe

This is a bit of a policy question, but we are using these things to do resource allocation, right? Giving more resources to some populations might be taking them away from others. Then there's the fairness-over-time thing as well, which is that what is fair now might not be fair in 100 years' time. So how should we think about this?

Prof. Randall Balestriero

That's a good question. I think this is also very application-specific. For example, if you want to predict where to build a house to solve some specific problem, maybe you don't really mind having bad predictions where there is no population anyway, because you're not going to build a house there. In this case, maybe the crowdsourced type of data is actually good, but this could really be dependent on the type of application.

One thing I will say regarding the point you made before is that this type of bias is something that you have in computer vision. There is a very nice paper done by Mark Ibrahim. Basically, they showed that most of the data we have, like from ImageNet, is from North America. So maybe you reach 90% state-of-the-art performance when predicting, for example, types of chairs and cars, but only for North American models. When you start looking at types of cars or chairs in Central Africa or East Asia, suddenly the model performance is extremely bad. This type of problem is something you have across modalities, and that's a very big issue.

Tim Scarfe

Randall, it's always a pleasure and an honor to have you on the show. Thank you so much.

Prof. Randall Balestriero

Thanks. Likewise. Thank you so much.