探索统一文本到文本 Transformer 的迁移学习边界

Tim Scarfe

Attention is just a way to route information between parts of the sequence, and that’s the genius behind a transformer. Layer by layer, it routes information to the correct place in the sequence such that it understands the entire thing at once.

Do you remember when GPT-2 was released—or, indeed, wasn’t released because it was considered too dangerous? It had about 1.5 billion parameters. It was a transformer language model trained on a corpus of about 40 gigabytes of highly curated text. Fast-forward 1 year, and Microsoft had released a new version of pretty much the same model that had 17 billion parameters. Eighteen billion—it doesn’t matter; it’s a lot.

In the olden days, we had these static, brittle knowledge graphs. They were incredibly onerous to create, but if we wanted to do something like question answering, they were the only way we could do it. If we wanted to ask, “What year did we win the Second World War?” then, without some kind of ontology of information, we couldn’t capture it.

With these transformer-based language models, you can pretrain them on a large corpus of text, let’s say Wikipedia, using a self-supervised task, and then fine-tune them on a downstream task. The incredible thing is that these models accrue information. They accrue knowledge about the world. You can ask one how old Obama is, and it will know the answer. It’s incredible.

The current revolution in neural representations for natural language originated with Mikolov’s word2vec skip-gram model in 2013. It was such a simple architecture. It used a contrastive loss, which we spoke about last week, and asked the simple question: does this word belong in the context of this other word? Using a fully connected architecture and a negative-sampling strategy with a contrastive loss, you could project these words into an embedding space that reflected their linguistic context.

The problem was that the words were atomic. They didn’t really take into account the structure of language, and language has this incredible structure. Words can modify each other from a long distance, and those relationships can be multiscale and complex.

The first evolution along the road was that we started to have language models like ELMo, which started to use bidirectional LSTMs to model the sequential dependencies between adjacent tokens. This was beginning to capture some of the meaning encoded in language.

Then ULMFiT came along, by Jeremy Howard and Sebastian Ruder, and this was when things really started to get interesting. It was the first model that essentially did transfer learning. You could build a language model that was just predicting the next word using a bidirectional LSTM, but then, exactly as happens in computer vision, you could do transfer learning. You could take off the last few layers, put a classifier on, and get the model to do something completely different.

You could pretrain a language model on Wikipedia, take the last layer off, turn it into a classifier, and do sentiment analysis. You had this transfer of knowledge between the self-supervised pretraining task and the downstream task. That essentially meant we had more training data, and the language model already knew loads and loads of concepts that would be useful for our downstream tasks.

In 2017, a chap called Ashish Vaswani came along from Google with a paper called “Attention Is All You Need.” This introduced transformers, and transformers are an entirely new architecture in neural-network design. We have densely connected networks, recurrent neural networks, and convolutional neural networks, but transformers are an entirely new paradigm—an entirely new type of network architecture.

RNNs and CNNs had been the flavor of the day for building language models. They had some interesting properties. They could support variable-length sequences, CNNs could easily be parallelized, and they had linear or logarithmic interaction lengths between tokens. The main issue was that they only modeled local dependencies between tokens. In CNNs, long-term dependencies required many layers and were lossy. Even GANs are now using self-attention because of its ability to model long-range, multiscale, complex dependencies, allowing us to capture more symmetries and patterns in the data within the input-size limit.

Myth 6 in “The Seven Myths of Machine Learning Research,” from early 2019, tried to dispel the idea that transformers were more parameter-efficient by discussing lightweight and dynamic convolutions as an alternative with similar results on machine-translation tasks.

Transformers have their own architectural challenges. They can only work within a bounded input length, they have nonlinear interaction distances between tokens, and they’re memory-intensive as hell.

People used to process sequences of text using recurrent neural networks, but that had a problem. If you have a long sentence such as “Dave shared beer with his friend,” then, to understand what “his friend” means, we need to know that Dave was at the beginning of the sentence. A recurrent neural network needs to store the fact that Dave was there until it processes the end of the sentence, and that can be very difficult to learn.

A transformer can do this in 1 step. It can go from anywhere to anywhere using an attention mechanism.

Fast-forward to November 2018, and Jacob Devlin released a paper called “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” This was a game changer—not only because there had been a tsunami of NLP papers modifying or modulating certain aspects of it, but also because many of the paper titles punned on the word “BERT.” I think that’s testament, if nothing else, to the impact this paper has had.

BERT was a bidirectional encoder-only transformer with 12 encoder layers. It had a pretraining task consisting of a masked language model and next-sentence prediction, and it was then fine-tuned on downstream tasks. The larger version of the model had about 340 million parameters, which was quite large at the time.

Because of the bidirectional context, it did particularly well on certain downstream tasks. It was state-of-the-art on question answering.

Since BERT, there’s been a tsunami of papers modulating some of the key findings in it. These papers even mention BERT or pun on the name BERT in the title, so we had RoBERTa, ALBERT, “Still BERT,” and “BERT on Stilts.”

The problem for the average consumer, like ourselves, is that we want to understand the relative contribution of all these different papers. We want to categorize what their changes to the pretraining objective are doing, what performance gains come from increasing the dataset size, what comes from having more in-domain data, and what comes from increasing the size of the model itself.

That’s where this paper is so important. It takes apart all these different factors that can lead to performance gains and isolates them in a large-scale study of transfer learning in the pretraining and fine-tuning pipeline.

Today, we’re talking about “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer” by Colin Raffel and his colleagues from Google. This paper is basically a huge ablation study. It asks what happens if we change all these different variables, and how those changes affect the performance of language models across a suite of tasks. It introduces a new type of language model, the T5 model, which is a text-to-text model. Its text-to-text framework provides a simple way to train a single model on a wide variety of text tasks using the same loss function and decoding procedure. It can be applied to generative tasks like summarization, classification tasks like natural-language inference, and regression tasks like STS-B. Despite its simplicity, the text-to-text framework obtained comparable performance to task-specific architectures and ultimately produced state-of-the-art results when combined with scale.

One of the cool innovations in this paper is that they created a text-to-text architecture. That means they didn’t need to change anything, even though they were testing against many different types of downstream tasks.

You might think this is a form of multitask learning, and it kind of is, but it isn’t. They still fine-tune the model on each of the downstream tasks, but they also did a version in which they trained it uniformly across all the tasks. After some tweaking, I was surprised to see that the performance wasn’t actually that bad.

By “text-to-text,” I mean that, in the input text, you actually define what the task is. You could say, “Translate this from English to German,” and then say, “Hello, my name is Tim.” The language model picks up which task you want to do and gives you the result. Incredibly, there’s no leakage between the tasks. It doesn’t seem to degrade performance in any way to have this text-to-text architecture.

Yannic Kilcher

One thing I think is so interesting about it, with respect to transfer learning, is that when you’re a developer trying to take this model for your application, it makes it much easier to figure out how to fit your input and output into this kind of pretraining and fine-tuning pipeline.

I’m curious about what you think about the text-to-text input and output for a specific application compared with BERT or these other approaches, where you have to be more careful about how you define your input and output. Also, the generality of the pretrained representation is decreased when you have to be more specific about the input and output.

Tim Scarfe

For that reason, I think the key benefit of the text-to-text framework is that it makes transfer learning easier.

Yannic Kilcher

They say that as well. Depending on how you do the pretraining, even in this text-to-text framework, it may be more or less effective. Maybe we should explain it for people who haven’t read the paper.

In this framework, if you want to translate from English to French, you don’t train a model specifically for that. You simply say, “Translate from English to French,” and then you put the text after that. The instruction itself is also text.

They define various pretraining tasks where the model is supposed to fill in parts of the text. I think performance really depends on how you do this pretraining, and that’s one of the things they show in the paper. What I love about it is that everything is text now.

Connor Shorten

I can also see this being useful in multimodal systems, such as an image-text model. Having this representation of text is super interesting to me because it means you don’t have to be so specific about how you define even the pretraining.

BERT has this masked-language-modeling objective, which fits nicely into sentiment classification because you do next-sentence prediction throughout pretraining. With this approach, you don’t have to be as careful about how the pretraining fits into the downstream task.

It’s interesting. We’ve gone through the takeaways, and 1 or 2 of them say, “We can do this and this and this, and then we combine it with scale, and it becomes state of the art.” It’s unclear to me how much scale influences this and how much the actual objective influences it. There’s a lot of uncertainty with respect to all these transformer-family models.

For example, in the BERT paper, the next-sentence-prediction task was specifically presented as something that helped the model understand the relationship between sentences. But in subsequent papers, some people have claimed that it’s necessary, while others say that you should just do masked-language modeling and not worry about next-sentence prediction.

To me, it’s still somewhat unclear where the magic is in these things. I don’t know how you see that.

I definitely think the ELECTRA objective, with the generator, is an interesting way of doing this. I hadn’t read SpanBERT before reading this paper, and I thought the small modification it makes provided more benefits.

There’s a lot of opportunity for exploration in self-supervised objectives. I don’t think the space seems particularly explored. They’ve tried shuffling and XLNet’s sentence scrambling, but it seems like they’ve tried almost everything with the pretraining objective. It was interesting to see all these little things combined and compared on an apples-to-apples basis in this framework. I don’t think anything else has done that at this level.

Tim Scarfe

One thing that interests me is that, every time I look at transformers, there’s always a small part of me that worries I don’t completely grasp what’s going on.

Vaswani’s original paper on transformers was an encoder-decoder model. BERT is apparently an encoder, and the GPT variants—and the NVIDIA model, what was that large NVIDIA one called?

Megatron.

Tim Scarfe

Megatron. That’s a good name. Microsoft, of course, also brought out Turing-NLG, which had an 18-billion-parameter model. Those models are causal, or autoregressive, language models, so they only have decoders.

Can you put a bit of detail on the difference between them?

In an encoder-decoder framework, you do what people originally did with LSTMs. You have an input, and you encode it fully into a hidden representation. Then you decode from that hidden representation. You can have attention and so on going on there, but basically you have 2 separate modules: one is the encoder and one is the decoder.

With BERT, as you said, you only have the encoder. You put in a sequence and get out another sequence of exactly the same length. With pure BERT, you can’t technically do something like produce language. People do it, but technically it’s kind of shaky. What you can do is classify each location, do span prediction, and so on.

If you modify the BERT architecture such that you make the encoder wider, so that you can put the input and part of the output in as the input, and then you do masked attention such that each output token can only look at its predecessors, then all of a sudden you have a decoder. You can do language modeling or sequence-to-sequence tasks using that.

Tim Scarfe

Aside from translation models, when do you think it’s interesting to encode only the inputs and then decode only from the encoder’s representation? For summarization, I suppose you could take the original document, encode it, and then decode only from what the encoder produces. But I guess I’m wondering why you would have explicit encoders and decoders for most of these tasks.

Even the original transformer in “Attention Is All You Need” was autoregressive in the sense that it had attention on what the decoder produced. It was just that there were 2 separate modules for that. Now there isn’t; it’s basically 1 set of parameters.

The main benefit might come from parameter sharing between all of these things. Generally, it seems to help to make your matrices larger but share them across the modules.

Tim Scarfe

What do you think about parameter sharing? It sounds like such an odd thing to me. Do they both get gradient updates? Is it like when you have a quantized codebook and take all the updates that came from every parameter sharing that weight, or is it just copying it?

Parameter sharing definitely seems like a way to save memory when storing the model parameters, but it seems odd to share the encoder and decoder parameters. I don’t quite understand the idea behind that.

Tim Scarfe

It’s strange, isn’t it? It’s somewhat similar to the way Siamese networks work, but rather than happening in parallel, it’s happening in series. Why would doing the same thing many times be beneficial? Why couldn’t you shortcut it?

I suppose a neural network is like a computer program. The idea is that you’re successively making transformations that transform data from one manifold to another manifold and then to another manifold. But why would you need to do that the same way at many intermediate steps?

It doesn’t make sense. It’s the same exact transformation applied again. But I guess it has something to do with the fact that, if you have such a complex series of transformations and copy and paste it across 8 layers, it will still make a significant difference.

Parameter sharing definitely seems like an efficiency thing. I don’t know if I’ve read anything that does a large study on parameter sharing other than neural architecture search.

What I meant when I said parameter sharing was more like this: why should I have 10 weights in my encoder and 10 weights in my decoder when I can have 20 weights in a module that does both at the same time? That seems to benefit more than simply sharing parameters.

You can make sense of it if you think of these layers as computing subroutines of a program—common subroutines, such as aggregating features in a certain way or sorting them. If you think of it like that, a subroutine in a program can be called all over the place. It’s kind of the same thing, but that is speculation.

Tim Scarfe

I like the concept that there’s overparameterization in these models as well. Several architectures have come out recently, and in particular I’m looking at DistilBERT, which has a teacher-and-student concept. It’s all about knowledge distillation.

The idea is that you train a normal-sized model, then create a student model that is 40% smaller and 60% faster at inference. You train the student to clone the teacher. The student model retains about 97% of the performance, but it’s a little bit like the lottery-ticket hypothesis: why do we need all this overparameterization during the training phase?

This idea has been in production in many places. It’s common in industry that you train large models and then push them down because you want to serve them. The idea isn’t new, but it’s cool to see it applied to transformers, because they’re so large and demanding.

It’s still a mystery why any of this works. What’s surprising is that we can get a small model with such high performance, but if we train the small model from the beginning with the training data, we can never get that performance. We need to train the large model with the same data and then distill it into the small model.

There’s a strange interaction here. You have to argue that the large model’s overparameterization makes the optimization procedure happy to accept the training data into the model. Once the information is in the model, it seems that you can transfer the features it learned by distilling it. This is still one of the stranger aspects of all of this: the fact that distillation works.

Tim Scarfe

I think you just gave a plug for my video on the lottery-ticket hypothesis.

You said it perfectly. In a way, it’s quite depressing that this is how it works. It’s a bit like having a block of clay and chipping away at it with stochastic gradient descent. Because most of these network architectures are so overparameterized, they contain n choose p subnetworks. There’s a combinatorial explosion of different subnetworks, and, probabilistically at least, one of those subnetworks is likely to provide a good inductive prior for the problem.

But this is a horrible way to go about it. How does pruning scale to transformers? Does it involve taking out the heads? How does transformer pruning work? How do you find a lottery ticket in a transformer?

One comment we can make is that Colin Raffel, in this paper, tried overparameterization as one of the tweaks we’ll come to later. He increased the model size, but his comment was that he did it only in the feed-forward section. He didn’t touch the attention.

Tim Scarfe

That’s really interesting, because the transformer thing isn’t obvious to prune. It’s an operation on a set, and you can’t just eliminate the attention in an obvious way as you could with a feed-forward layer.

Connor Shorten

I think distillation is more of a testimony to the power of label smoothing and label regularization. Even with naive label smoothing, where you put, say, 90% probability on the correct class and distribute the rest uniformly everywhere else, I don’t think distillation is really transferring the features from the teacher model. It’s a more clever way of doing label regularization.

Meta Pseudo Labels is a recent paper that shows something similar. It’s a more clever way of dynamically adjusting the labels, and that seems to be a really good way to train these neural networks.

That makes sense, but is it known how this relationship works across architectures? By outputting not just the labels but the probabilities of the labels, you could implicitly transfer the features because the student has the same architectural inductive bias. It’s a transformer and the teacher is a transformer, so, given the same outputs, the same architecture should learn the same features.

Is it known how distillation works across architectures? For example, if I distill this into an LSTM or something?

Connor Shorten

I know there’s a paper that takes transformer labels and puts them into an LSTM, but I don’t remember the details.

The feature transformation is more about transferring the embedding distances. It’s almost like a metric-learning thing. The teacher is showing you that one thing is kind of like a dog but very far from a ship, whereas another cat is closer to a ship. It’s more a transfer of the distances between the examples in the dataset than a transfer of the features themselves.

Yannic Kilcher

That’s an interesting way to think about it. Could there be a distillation procedure that uses this distance-metric idea and does some sort of contrastive learning, where you say that these 2 things should be closer together than this other thing?

Connor Shorten

It seems like some combination of those approaches should be possible. It reminds me of how neural style transfer uses intermediate features. It’s definitely a tricky idea, but distillation with distances seems interesting.

Tim Scarfe

Let’s carry on reading a little bit of the paper. The second part starts to talk about the encoder-decoder architecture, which I didn’t completely understand, so let’s read this section.

There’s a real contribution around architectures. It says: “While some work on transfer learning for natural-language processing has considered architectural variants of the transformer, we found the original encoder-decoder form worked best in our text-to-text framework. Although an encoder-decoder model uses twice as many parameters as an encoder-only model, such as BERT, or a decoder-only language model, such as GPT, it has a similar computational cost. We also showed that sharing the parameters in the encoder and the decoder did not result in a substantial performance drop, while of course halving the total parameter count.”

Just to confirm, was the original Vaswani model an encoder-decoder model, or is it referring, as Yannic was saying earlier, to the original recurrent neural-network sequence-to-sequence models?

Yannic Kilcher

I think the original model was an encoder-decoder model. I made the connection because the original sequence-to-sequence frameworks also always talked about encoders and decoders.

Tim Scarfe

I think that’s because it makes intuitive sense. The RNNs were learning a sequential representation, and clearly there was an encoding phase where the model went from a sequence into a flat internal representation, followed by a decoding phase.

But transformers are a little bit different, aren’t they? They don’t flatten the data into a representation in the same way. The data still seems to remain in the original time domain.

Yannic Kilcher

You can’t completely wash out the differences once you introduce attention into LSTMs and things like that. I think it’s mainly about how you program the systems, rather than about actual ideological or mathematical differences.

Connor Shorten

What do you think about the positional embeddings? When I first read “Attention Is All You Need,” the way it handled positional embeddings looked bizarre. Now people are using relative embeddings, as in Transformer-XL, so I guess that’s how they give the model a notion of time.

Yannic Kilcher

It’s a very strange way to do it. A transformer by itself has no notion of position in the sequence. It doesn’t know where information is coming from. It can query information and address content by its content, but it can’t ask where that information came from.

I suppose the researchers realized that information about whether something is at the beginning or end of the sequence, or 2 positions away from another token, is useful. It seems strange to put positional information in, but the original paper wouldn’t have included it if it wasn’t needed.

Tim Scarfe

I find the use of sine waves for positional encoding quite smart. You overlay sine waves, where one is very slow and the next is faster. By indexing a point across those waves, you can measure distance at different scales.

The large sine wave gives you a large-scale distance. It encodes your position without requiring the model to do arithmetic. The model can ask what is in its close neighborhood and use the fastest sine wave, or it can ask what is around it in a larger sense and use the slower sine wave.

Connor Shorten

Transformer-XL introduced recurrence into the model, and then you need a relative positional encoding for all the chunks. Coming back to transformers, in contrast to an RNN, where you’re learning a sequential representation and therefore have some concept of time built in, it seems somewhat similar to the way reinforcement-learning algorithms work.

Everything about the past is compressed into the present, but transformers have this wonderful capability to model long-term representations. You can attend from any token to any other token, so you can learn these long-term relationships. Does the model use the positional encodings as a ruler to measure how far away those things are?

Yannic Kilcher

I would guess that 95% of why these things help has a much simpler explanation. It’s just that the model knows that a token is at the start of the sentence. That gives it a hard attention to the beginning of the sentence. If the beginning says “what” or “who,” that is an extreme bias that helps the model understand what the sentence means.

A lot of what positional encodings provide is very simple. Some of it does have to do with which words are my neighbors, which words are far away, and which words come before or after me, because that is very important.

Tim Scarfe

One thing this brings to mind is that I’m interested in the concept of symmetries. A CNN can capture translational symmetry, but can a transformer capture the same symmetry? I don’t like the notion that positional embeddings hard-code the pattern so that a particular pattern of text has to start at the beginning of the sentence or at the beginning of the input.

Wouldn’t it be better if the model could learn the pattern in such a way that it was translationally invariant, or even invariant to different scales? It might be a pattern that occurs at different scales of language.

Yannic Kilcher

In some sense, it already can. If it pays attention to the correct frequency of the sine wave, it can learn that a phrase such as “My Little Pony” occurs at different positions. At one point on the sine wave, you have one value, then another value, and then another. Elsewhere in the sentence, the sine wave repeats, so the model can recognize the same pattern at both locations.

It can learn that if “Pony” follows “My” by 2 words, then it assigns a certain meaning. At a higher level, it could learn that if there is a relative clause after a person’s name, it assigns a particular meaning. You can learn these larger patterns at higher scales in the layers.

I think this encoding of overlapping waves is very cool for that reason.

Tim Scarfe

It’s beautiful. I don’t know whether they did an ablation study in which they removed the positional encodings, but if you think about it, the model has no concept of the relationships between the tokens without them. It can only be using the positional encoding.

Does that mean it’s only looking at relative distances between the tokens? Is it not looking at the immediate context of the words surrounding the token it’s attending to?

Yannic Kilcher

Who knows? If you introduce relative encodings, you have direct access to the immediate context. But I can almost guarantee that the model doesn’t work without positional encodings, because that’s why they’re there in the first place.

In T5, I think they use a learned scalar encoding, so they get away from the sine encoding.

Connor Shorten

The sine encoding seems to say that the word “eat” is more similar to word position 15 than to word position 1. It’s arbitrarily putting words near one another in that sense.

Yannic Kilcher

I think of it like a system I saw during my bachelor’s degree. A group was working on 3D face modeling. They had a machine that 3D-scanned your face by projecting patterns onto your skin.

First it projected black and white. Then it projected black, white, black, white. Then it projected black, white, black, white, black, white, and so on. In the end, every pixel on your face had a unique encoding of black and white across those different scales. Each pixel was unique, but neighboring pixels didn’t necessarily share the largest part of their encoding.

You could capture the fact that the 2 cheeks are equal in their bit encodings, except that one is black where the other is white in the first position. Beyond that, there’s a symmetry in the higher layers.

Tim Scarfe

I think that, of course, it does mean that some words are more similar than others, but if you do this across the layers, the lower layer can say that its neighbors are important, while the higher layers can say that things farther away are important. That might do the trick.

Let’s read a little more of the takeaways. The next section is about unsupervised objectives. In these self-supervised language models, the general approach is that there’s a pretraining task, which might involve next-sentence prediction, a masked-language model, or something else, and then you fine-tune the model afterward.

The paper says: “Overall, we found that most denoising objectives, which train the model to reconstruct randomly corrupted text, performed similarly in the text-to-text setup. As a result, we suggest using objectives that produce short target sequences so that unsupervised pretraining is more computationally efficient.”

Connor Shorten

I thought that was really cool. Instead of reconstructing the entire target, you only reconstruct what has been masked out. I think that speeds up training enormously. Rather than reproducing the entire original sequence, you only ask the model to fill in the missing spans.

Tim Scarfe

They’re saying it’s more efficient because the output space is smaller. I find it fascinating that it works at all. You give the model an input such as “Thank you me to your party week,” and the target is “for inviting last,” and somehow it manages to do that.

Yannic Kilcher

Especially because there are so many different ways to fill in the missing text that would make sense.

Tim Scarfe

It’s fascinating to look at the evolution of the different pretraining objectives in self-supervised tasks. In the BERT-style model, you had a random number of tokens masked out and the target was the original text, so that’s quite a large output space.

Then there was word shuffling, which didn’t get particularly good results. There’s also masking tokens. Is that the same as BERT? What’s the difference?

Yannic Kilcher

I think the key difference is the span. In BERT, there were also 2 objectives. Some of the input tokens were randomly swapped, which is why you might have something like “Thank you to your party Apple week.” The BERT model was asked to correct those tokens as well.

Here, with span masking, the model is just asked to fill in the masked tokens. I think this is the objective introduced in the paper called MASS.

Connor Shorten

Do you think switching “last” for “Apple” is useful? It looks like it doesn’t help much.

Yannic Kilcher

As they say, most of these tasks perform similarly, but it still seems important to do something like this. If you take a BERT model and train it directly on question answering or sentiment classification, sentiment might work, but question answering may not. Even though these objectives perform similarly, it’s still important to do some kind of denoising task.

I was impressed by the variety of tasks they came up with. It even looks like a pretty big gain to go from 83 to 84.4 on the GLUE tasks.

Tim Scarfe

We’ve been thinking about the ELECTRA pretraining objective, where there’s a generator in a different configuration. What do you predict would happen if they had put that into this pipeline? Would it be as good, or would it also perform similarly to the others?

Connor Shorten

You have a generator that corrupts the text. It’s a discriminator objective rather than a masked-token-prediction objective. You have a low-parameter generator that fills in the masked tokens, and then the larger model sees what the smaller generator thought should fill in those tokens.

The larger model isn’t reproducing the sequence. It’s making a binary real-or-fake prediction for each token. It’s somewhat like the GAN framework.

Yannic Kilcher

I don’t know what to think of that. GANs for text are still pretty far away. It might work, but a GAN objective seems like a much lower-information task than the actual language-production task.

Tim Scarfe

You have to weigh that against the fact that, for any given gap in a sentence, you could fill it with 20 different words and they could all make perfect sense. Here you have a training corpus telling the transformer, “No, this is the one I want.” That’s where a discriminator that assigns high probability to all those 20 words might benefit you.

Yannic Kilcher

I have no guess about whether it’s ultimately good or bad. I mostly don’t believe numbers in papers, because researchers come up with a new idea and then throw more resources at it, use a larger corpus, or apply some other trick. It’s very difficult to compare these things, even when they go to extreme lengths to perform ablations.

Connor Shorten

What do you think about competitions as a way to evaluate these models? Do they do a better job of decoupling the easy things to add? In a Kaggle competition, everyone augments the data, ensembles the models, and gives the problem as much compute as they have.

Maybe competitions are a better way to tease apart algorithms, but the benchmarks don’t seem to do that.

Tim Scarfe

That’s one of the key points François Chollet makes in his paper “On the Measure of Intelligence.” We’ve centered on the idea that benchmarking and competitions are the best way to get the best result, but competitions can produce perverse outcomes because we become so fixated on the benchmark that the winning solution doesn’t generalize to the real-world version of the problem.

Yannic Kilcher

There are 2 arguments here, or 2 modes. One is that a benchmark or competition is how we evaluate things. The other is that someone reads all the papers, implements them carefully, uses the same hyperparameters, and gives every model the same amount of compute.

Both are flawed in their own ways. If I had to choose, I’m more on the side of competitions. If you say, “Whatever the highest number you can get is, do it,” then you may not be able to interpret exactly where the gains come from, but at least you get an absolute number. You learn how far you can push the system with everything you have.

If someone implements every approach we’ve seen, however, you can run into problems. There was a famous paper called “All GANs Are Created Equal,” where the authors implemented all these GANs. Then the authors of the Wasserstein GAN came along and said, “We stated clearly in our paper that our algorithm only works with RMSProp, not Adam.” But in that paper, the hyperparameter grid included those algorithms, even though the original paper said it didn’t work that way.

All the options are bad, but if push comes to shove, I’m more on the side of competitions. Ultimately, as a practitioner, the best thing you can do is look at all these papers and try to crystallize what they have in common. Don’t necessarily believe any individual paper, but if you see the same tricks and architecture types over and over again, those are probably the things that work.

Tim Scarfe

It makes me think of some of Chollet’s other comments. He used the example that, in the 1980s, people such as Marvin Minsky had quite narrow definitions of AI, centered on how well a system could perform a task.

That’s a different metric for humans and algorithms. If an algorithm can beat the world chess champion, that doesn’t seem particularly impressive, whereas if a human can do it, it’s very impressive. That’s because the human is learning how to become highly skilled.

It reminds me of when we talked about Uber’s POET, because that was all about a system that learns and produces its own curriculum. These competitions and benchmarks are so narrow-minded that they stop us from making progress toward what artificial intelligence should be: the ability to learn things dynamically.

Yannic Kilcher

Is that even the question we need to ask to make progress in artificial intelligence? For a company such as Google, if its translation model is better than everyone else’s, then it makes money. That’s the practical objective.

With respect to intelligence, though, if you give a human the same amount of raw compute time—say, neuron firings versus GPU cycles—and the same amount of memory, where it can write down everything, I wouldn’t be surprised if the human became much better at chess. We have the impression that humans are so good because we have limited time and develop a feeling for how hard the task is.

Tim Scarfe

I think we’ll be doing a “Measure of Intelligence” talk very soon.

Connor Shorten

What do you think about the DAWNBench idea of limiting the computational resources for a task or competition? It seems that you can’t have a fair computational budget for everyone and still do something interesting with models on the same scale as when the budget is unbounded.

For machine translation and similar tasks, the best effort may not be to develop a cleverer algorithm. It may be to figure out how to train a 100-billion-parameter transformer, or even a trillion-parameter transformer. The differences between TPUs and GPUs, and the way they enable larger-scale matrix multiplications, seem like a more promising direction for these kinds of problems.

Yannic Kilcher

I had an interesting conversation with a friend about this. I was arguing that systems should receive equal resources, but his point was that some effects only emerge at scale. There are effects these models capture only because they have 1 billion parameters and 50 terabytes of training data.

The only real objective number you have is how well you can perform with unbounded computation and unbounded resources. It’s an entirely different task to say that you get a fixed number of CPU cycles. If I’m engineering something for a mobile phone, I care about that, but it’s a narrower field.

Tim Scarfe

That’s a nice segue into the dataset section of the paper. Some papers in this area, such as RoBERTa, discovered that BERT simply wasn’t trained for long enough. It was undertrained. As we’ll find out in this T5 paper, training for longer is one of the most effective strategies.

The paper says: “We introduced the Colossal Clean Crawled Corpus, or C4, which comprises heuristically cleaned text from the Common Crawl web dump. When comparing C4 to datasets that use additional filtering, we found that training on in-domain unlabeled data could boost performance on a few downstream tasks. However, constraining training to a single domain typically results in a smaller dataset. We separately showed that performance can degrade when an unlabeled dataset is small enough that it is repeated many times over the course of training. This motivates using a large and diverse dataset like C4 for generic language-understanding tasks.”

They started with this huge dataset. It was 6.1 terabytes, and after filtering it was reduced to 745 gigabytes. They removed JavaScript, kept sentences that ended in a period, and did some deduplication. It’s surprising how much they could filter away.

Connor Shorten

I thought another interesting point was that, although the current pipeline has an enormous amount of unlabeled text available, natural-language processing seems to benefit particularly well from unsupervised pretraining followed by fine-tuning.

They were able to get about 750 gigabytes of text from the entire internet, while they had roughly 20 gigabytes of Wikipedia, plus the BooksCorpus. But if you look at Table 12, the supervised multitask pretraining result is already only about 3 points behind the unsupervised result on GLUE.

That makes me wonder whether supervised multitask pretraining will be more scalable in the future. People are producing supervised natural-language-processing datasets every day. I would actually predict that supervised multitask pretraining may become more scalable, because you can’t get an unlimited amount of unsupervised data.

In images, it might be the other way around. You can drive a self-driving car around and collect more images than you could ever store in a labeled dataset. Maybe images will be the domain where we think of unsupervised pretraining as the most scalable approach.

Tim Scarfe

The takeaway from this paper was that unsupervised pretraining is still better, but it does seem to indicate that the magic of unsupervised training is simply that you get more data. You train for longer with more data, and that allows you to learn better.

They did some interesting experiments. They tried not doing unsupervised pretraining at all, and then they did something similar to Microsoft’s MT-DNN. They used multitask learning, training uniformly across all the tasks, and then separately fine-tuned the model on each downstream task. That significantly improved performance.

Yannic Kilcher

I definitely think multistage training is useful. You can do unsupervised masked-language modeling and then, even if you use the same self-supervised task, continue training on in-domain data.

That’s another major characteristic of the ablation. If you pretrain on Wikipedia, you’ll do better on something like SQuAD and worse on something like CoLA, because the model has never seen an unacceptable sentence before in Wikipedia. It has no sense of that distinction.

Tim Scarfe

Maybe a large part of the effect comes from the fact that the model has never seen the kind of text it will encounter in these tasks.

Yannic Kilcher

With images, the inputs are natural images all the way through. It isn’t as though a visual-question-answering task suddenly has a large arrow on the image saying, “What is this thing called?” If the model has never seen that during training, it can’t make sense of it.

That may be the real distinction here. Language has structures such as unacceptable sentences that the model may never have seen during training. In-domain pretraining may provide the model with a sense of the type of data it will receive during the actual fine-tuning stage.

That also somewhat undercuts the idea of huge general pretraining. If the explanation is ultimately that you still need to see the type of data your task uses, and once you do, you aren’t much worse than the huge pretrained model, then how much are you really gaining from the massive pretraining?

Connor Shorten

How interested are you in meta-learning, where the definition is domain adaptation or rapidly changing to a new environment? MAML, for example, tries to find a good set of initial parameters so that, when you shift domains, the model quickly adapts.

If you moved from Wikipedia to one of the book corpora, could meta-learning give you an unsupervised representation that works for every domain, or do you necessarily need another mechanism that performs domain adaptation?

Yannic Kilcher

One way of describing it is that there are more similarities between language corpora than there are dissimilarities. By doing unsupervised training, you capture as much useful, generalizable information as possible.

But there are words that mean different things in different domains. In the financial world, “default” is a very bad word.

With meta-learning, I’m always a bit skeptical of these papers. We don’t have that many tasks, even in NLP. Although people are producing new datasets all the time, there’s still a limited number of tasks and a limited number of ways in which they’re diverse.

To claim that you’ve found an initialization that works for general language seems like an overclaim. I’ve seen the same thing with Google’s Visual Task Adaptation Benchmark. You pretrain on something like ImageNet, but then you only have 1,000 samples for all these other tasks. The tasks are too similar to establish that you’ve found something truly general.

It’s a very cool field, and if it works, it will be game-changing for practitioners. But I’m skeptical about the claims.

Tim Scarfe

With respect to the dataset, it’s called the Colossal Clean Crawled Corpus, and I think the community may be running out of words to describe large things. There’s Megatron, XLNet, and Transformer-XL. I wonder how high they can push this word game.

They released C4, which means that someone at OpenAI must have been having a heart attack. How can you release something so dangerous? It’s 10 times as dangerous as the OpenAI corpus would have been.

Yannic Kilcher

One comment I wanted to make is that the dataset isn’t actually that big. The dataset they called WebText—their equivalent of what they did with GPT-2—was filtered to include high-quality news articles. I think that meant articles linked from Reddit that had been upvoted at least 3 times.

That dataset is only about 35 gigabytes, but its score is significantly higher than the filtered C4 score. How can that be? C4 is only 20 times larger than the filtered internet text. Isn’t that interesting?

Connor Leahy

It also raises a privacy issue. If you went through everyone’s Facebook messages, Twitter direct messages, and private conversations, you could get much better text data. Of course, you can’t do that because of privacy concerns, but it’s an enticing possibility.

Yannic Kilcher

These systems are especially worrying because the transformers are getting so large that, in essence, they may be remembering the training data and interpolating between it.

I’ve always had the impression that, to make the next step in NLP, you need to ground the knowledge. I think of Chollet’s example from the Abstraction and Reasoning Corpus, where you have an image with black pixels, 3 blue pixels, and 2 orange pixels going away from them. It’s clearly something bouncing off a wall, but we understand that because we live in the physical world, where things bounce off other things.

The next step needs to involve grounding. These models don’t have that. All they can do is interpolate patterns that they see in the training data. They’re becoming so large that they may simply remember the entire training corpus and figure out how to mix and match it.

They see a noun, observe that a sentence starts with certain words, and finish it with an example from the training data, or with a mixture of 2 examples. I’m not convinced they learn very sophisticated things; they learn how to mix and match the training data.

Tim Scarfe

There was an XKCD comic where the model starts writing an email with “The next meeting of the rebellion leaders…”

Connor Leahy

How would you ground language? You could have text-based adventure games, where you receive a text prompt such as, “You’re in a dungeon. You see this thing on the table. What do you want to do?” You navigate the world through text.

Chatbots are grounded in a similar way, because you receive a reaction to what you say. There’s also PointNav, where you navigate a visual world based on language cues, or the Allen Institute’s RoboTHOR. There’s another benchmark called “Read to Fight Monsters,” where you receive a description of a game in language and then play the game.

To ground language, you need the context around it. It’s unclear how you would define that context.

Yannic Kilcher

These text-adventure games and related environments are a nice idea, but if you want to make them useful for machine learning, you need a lot of data. You can only get that if someone writes a procedural generator for the text.

Then you’re no longer really dealing with language. You’re dealing with a procedurally generated grammar, where you look for the same patterns of words that the simulator writer included.

By grounding, I mean that if I say, “There is a car around the corner,” and I live in the physical world, I know that cars are objects, that they’re on roads, and that “a corner” in this case means the corner of a house. There’s a road here and a road there, with a house in between. I can’t see the car, but it’s around the corner.

Tim Scarfe

I love Chollet’s concept that intelligence is closely linked to the world and the environment in which we live. That’s his major argument for why there can be no explosion of intelligence.

But I also like what you were saying about neural networks being memorization machines. Philosophically speaking, you could argue that we’re the same. If we could remember everything, perhaps this conversation would simply be us regurgitating things we’d picked up from random places, without thinking fluidly or intelligently.

Of course, that isn’t true. What Chollet would say is that deep learning has a problem with minimal generalization. To learn anything, we have to memorize it.

There was a wonderful paper a few years ago saying that deep-learning algorithms don’t extrapolate; they only interpolate. The example was the identity function, f(x) = x, which is one of the simplest functions in the world to learn.

The researchers trained a neural network between –5 and 5 and then tested it outside the training range using several different activation functions. None of them could extrapolate. Isn’t that tragic?

Connor Leahy

How important is extrapolation? Going back to language grounding and the idea of looking at a car around the corner, Google recently released a Street View dataset where you navigate through street-view images based on language.

With enough data, you can probably interpolate between vision and language and navigate very well using language. Is extrapolation really necessary? Can we even define it?

You might say that a system can combine vision and language, but it’s still not intelligence. What else do you want to add? Audio? Touch?

Yannic Kilcher

Embodied cognition is essentially what I’m talking about. We live in the world through our senses. There are our senses and then our brains, which have been optimized to make sense of the inputs.

We interact with the world through input signals. You could argue that, if you give a neural network enough different modalities, it might be able to make the same connections.

Connor Leahy

It could do that in a superhuman way, because, as Tim said, it could instantly recall everything it had ever experienced.

Tim Scarfe

That would be a curse. It wouldn’t necessarily be a good thing.

Yannick Kilcher

I also don’t know how to think about this idea of generalization. If you consider the identity function, who’s to say that we don’t want the network to learn a function that rises and then returns to the mean?

It’s only because we tell the human, or the human somehow does this extrapolation, that we assume the pattern should continue. Based on what? Do we have an intrinsic notion that the pattern must continue as an identity function? How do we do this?

That’s the big question to answer.

Tim Scarfe

It’s interesting because, as you say, in the real world we have some useful inductive bias that allows us to reason about how things relate to one another. But in mathematics, if it’s such a simple function, why would it be so difficult for a neural network to extrapolate it?

Connor Shorten

One reason I became interested in generative adversarial networks is that studying a generator, which produces a high-dimensional output rather than a classification label, is a more interesting way to look at generalization.

You can give a generator a dataset of cats and ask it to design a new cat. Then you can ask whether it designed that cat according to your sense of what a cat is. I think it’s more interesting to think about generalization in terms of models that create things rather than simply label them.

Yannick Kilcher

That’s interesting, but is it generalization?

Connor Shorten

Disentangled representations are another thing I want to understand. If you can learn how to take the hat off a person, put sunglasses on them, or manipulate the separate factors of variation, that’s extremely interesting.

Tim Scarfe

Let’s continue with the paper. We keep digressing, but that’s no bad thing. This is what the channel is about. It’s called Machine Learning Street Talk for a reason.

The paper talks about training strategies. What struck me is that, if it truly were a multitask model, that would be wonderful. They got the best results by fine-tuning on the respective tasks, but imagine if you had a language model that was genuinely multitask.

You could say to the model, “Translate something from English to German,” and then give it the text. The model would give you the result without leakage between the tasks. They got pretty good results with that, and it blows my mind.

Yannick Kilcher

You have to remember that the model probably isn’t understanding the meaning of the sentence “I want you to translate this.” They probably prepend that exact sentence to all the translation tasks. The model simply learns that the particular combination of tokens indicates the language. It isn’t understanding the instruction.

Tim Scarfe

I agree, but a neural network is a computer program. It’s partitioning the model and delegating responsibility. Some parts of the model become responsible for translation, but the interesting part is the interconnected and shared nature.

It isn’t as though the top part of the model is responsible for translation. It’s reusing representations.

The MT-DNN paper from Microsoft does multitask learning as part of pretraining, before fine-tuning. The takeaway seems to be that training on multiple tasks at the same time gives you a kind of regularization, because you’re learning representations that are useful across all the tasks.

Yannick Kilcher

That’s very believable. It would be extremely interesting to have better introspection into these text models to see what’s really going on.

How far up are the representations shared? Is it only the word embeddings that are shared, with the model then dividing itself? Do the attention heads learn that, whenever there’s a translation task, they should do one thing while the others do something else?

It would be interesting to perform introspective analyses on these models to see how much the representations are shared and where the subroutines are reused across tasks. Maybe we’d find something very interesting.

There was a paper a while ago that did zero-shot translation. I don’t remember the exact procedure, but you pretrained on English and then fine-tuned the word embeddings for French. You could input English and output French, without having trained on that particular language pair.

Tim Scarfe

Recent neural-translation models do something similar. If they’re trained multilingual, they translate different languages into a common intermediate representation. In that sense, they can translate between almost anything.

Yannick Kilcher

It wasn’t exactly the same thing, but I’m going to find the paper because it was interesting. It suggested that there was something like an abstract language understanding—not grounding or intelligence, but an abstract notion of language.

It would be fascinating to see how multiple tasks and objectives are handled in the representations throughout the transformer layers.

Tim Scarfe

When you started explaining the possibility that the model is partitioned across attention heads, that was convincing. Maybe it does work that way. If this were simply a horrible fully connected neural network, it would be very difficult to understand what was happening. But clean partitioning across attention heads seems more likely.

Yannick Kilcher

It’s somewhat like a conditional-computation model, because of how the information is routed to the attention heads.

Connor Shorten

I had misunderstood the text-to-text setup. I thought the model actually understood “summarize” and then summarized the text. I thought that was one of the cool things about the approach: if you had a new task, such as “which pronoun does this refer to?”, you could prefix the example with that instruction and the model would learn it faster than if you used an arbitrary code such as “0 0 0 1 0 0,” where the code indicated pronoun disambiguation.

Yannick Kilcher

It’s possible, but the model represents tokens using word vectors, so you aren’t really giving it a one-hot vector. It’s in a much higher-dimensional space.

If you replaced “Translate from English to French” with meaningless text, such as “Sandwiches are nice,” and always used that phrase for the same task, it would still work. The model might have seen the phrase “translate” in that context somewhere in the training data, but if you always use the same arbitrary phrase, the model can learn that the phrase indicates the task.

Connor Shorten

It might make the training curve a little faster because the model still takes the word “summarize,” looks it up in the embedding table, and gets some sense of what it means. Hopefully the embedding has some representation of “summarize.”

Yannick Kilcher

That’s less exciting to me. Unfortunately, these things are never as exciting as they initially sound.

Tim Scarfe

Let’s talk about the loss, because the paper is almost depressing in showing that scale and the amount of training made the largest difference.

The baseline on GLUE was 83.28. When they used 4 times as many training steps, it went up to about 85.33, so that’s roughly a 2-point improvement.

The biggest improvement came from increasing the size of the network. They doubled the number of parameters, not by changing the attention heads but by increasing the number of parameters in the feed-forward layers. They also used twice as many training steps, and that produced about a 3-point improvement, which is huge.

Connor Shorten

I think what will happen is that we’ll train a very large model and then make only one pass through the available text. We’ll figure out how to train a 100-billion-parameter transformer before we figure out how to obtain the corresponding amount of data.

The model size is going to scale better than the number of training steps, because the 2 need to go together in these scaling relationships.

Tim Scarfe

It’s also interesting that, when they did multitask learning with a uniform prior over the tasks, some tasks performed quite badly. That’s a balancing issue. You’re undertraining some tasks and overtraining others.

They came up with a temperature parameter to weight the relevant tasks according to their contribution. But it isn’t only the differences between tasks. There’s also the problem of the unsupervised objective dominating the downstream tasks.

There’s quite a delicate balance. I think the most important thing is the transfer rate between all the different training objectives.

Connor Shorten

It’s similar to class imbalance. You might have many more examples of dogs than cats. In this framework, you also have task imbalance. It’s interesting to see the same phenomenon at the task level.

The text prefix might help with task imbalance, but conditional computation is interesting too. If you had sparse routing based on the task embedding, that might be the best way to overcome task imbalance.

It could also be related to catastrophic forgetting and continual learning. The latest solution from the Neuromodulation paper seems to involve another kind of sparse routing. I’m not exactly sure what happens in that paper, but it’s interesting to consider the composition and size of the multitask datasets.

Tim Scarfe

That’s pretty much the paper. It was really useful to explore what happens when you change each of these variables. Clearly, when you look at the history of papers on question answering over the last few years, BERT gave us a significant boost.

Since then, we’ve probably gained another 10 points, but the gains are beginning to level off. Over a 2-year time horizon, though, transfer learning—and perhaps transfer learning simply means training for longer on more data—gave us the biggest improvement.

Another point worth making is how democratizing this is. It’s now easy to download a pretrained model. You can download the unified T5 model we’ve been discussing, or you can train it yourself. You can perform sophisticated translation with it.

A few years ago, if you wanted to do high-quality speech recognition or translation, you couldn’t do it yourself. You had to use Microsoft or Google’s cloud platform—Cognitive Services on Azure—because you didn’t have enough data or a good enough model.

You had to pay for those services. Now it’s becoming quite democratized.

Yannic Kilcher

You can also form your own dataset, ask the model questions about it, and use that as a starting point.

The MT-DNN multistage approach is important too. If you want to collect all the deep-learning papers into a dataset, before labeling them you can perform self-supervised learning on the text. That gives you another useful stage in the pipeline.

Tim Scarfe

Why is language different? A high-quality language dataset can apparently be 35 gigabytes, which is nothing compared with vision. Google’s FaceNet paper trained on about 26 million faces, and that’s why it works so well. A facial-recognition dataset of that size would be incredibly valuable, and I wouldn’t be able to download it from Google.

It’s almost as if language is becoming more democratized than vision.

Yannic Kilcher

Language is a more compressed representation. We simply haven’t figured out how to compress images in a way that allows us to reconstruct the useful information.

Ultimately, you care about what ends up in your brain. For language, we found a good representation that encapsulates sound waves as words. For images, we haven’t found the equivalent representation yet. Maybe, once we figure that out, image datasets will become smaller.

Tim Scarfe

Another way to look at it is that language and vision are both unstructured data, but vision is much more unstructured than language. Language is actually very structured. You have atomic tokens, clear relationships between where they appear, and a 1-dimensional sequence.

Maybe that’s why you need much less pretraining data for language than for vision.

Connor Shorten

Possibly. You only have to learn the grammar of a language to make sentences, whereas, with natural images, you have to learn correlations between all the pixels. Nobody tells you which images are good and which aren’t.

Yannic Kilcher

It may also be that vision is almost a 2D image of the actual 3D world, with objects and spatial relationships. It’s a representation of something with position.

Tim Scarfe

I have another question. How far do you think this approach of encoding everything as text will go? If I were producing a chatbot or an assistant like Alexa, I would still hard-code all the individual commands. For now, that would work much better.

Connor Shorten

That’s an interesting point. If we had a model pretrained on chatbot conversations, could we fine-tune it somehow on all the deep-learning papers and then ask it questions about them? That would be interesting.

Yannic Kilcher

I find the notion of formulating the task as text interesting. What this paper doesn’t address is whether you can throw a task at the model that it has never seen before, given only textual instructions.

I saw a presentation by Richard Socher from Salesforce where he proposed that all NLP should be formulated as question answering. You always have a question and a context, and the model outputs text to answer the question.

You could frame next-sentence prediction or sentiment analysis as text generation. He also discussed pointer networks, where the model can either output text or point to something in the context or in the question as the answer.

Pointer networks seem to have disappeared. I don’t know whether they don’t work or simply aren’t fashionable, but they were used to formulate NLP as one general task structure. It’s a very similar idea to this, and perhaps we’re ready for a resurrection of pointer networks.

Connor Shorten

It reminds me of multihop question answering. You ask a question, but to answer it you need to ask 2 more questions first, and then use those answers to answer the original question.

Yannic Kilcher

Closed-book question answering is one of the things people claim T5 does well. You can ask, “When was the Second World War?” and it knows the answer because it has memorized it.

Tim Scarfe

GPT-2 did that too. You didn’t fine-tune it; you simply pretrained the language model, posed the question, and the answer magically came out because it was somewhere in the training data.

It’s a parallel with the old knowledge graphs. We used to explicitly compute knowledge graphs and say, for example, that Barack Obama was 45 years old. The problem was that building the knowledge graphs was extremely onerous, and the information was often conflicting.

People used to write unit tests to check for logical equivalence. That’s happening in language models too. My friend Marco Ribeiro has worked on the idea of creating semantically equivalent adversarial examples or logical tests.

For example, you might ask, “How many boots are there in this image?” and the answer is 3. Then you ask the logically consistent question, “Are there any boots in this image?” and you expect the answer to be yes.

You could use these examples to generate adversarial data for pretraining or fine-tuning, or as a way of deciding that a model isn’t reliable enough to put into production. I’m interested in this engineering reality, because when we have unstructured models, it’s useful to have a noninteractive way to reason about their behavior.

Yannic Kilcher

Humans would fail these unit tests too. There’s a personality test called the Big Five, which is statistically derived and self-reported. It asks the same question in different ways.

It might ask, “I often feel sad,” and then later ask the inverse in a slightly different form. The number of times the answers don’t correspond is astounding. It’s as though the test says, “You said you always feel sad, but here you said you’re mostly happy. What’s going on?”

Tim Scarfe

What’s even worse is that, when I applied for my current job, they gave me one of those silly psychometric profiles. They said I was only at a particular level on one dimension, and I asked why. They told me I had taken 10 milliseconds too long to answer a question.

Yannic Kilcher

Maybe these tests are going in the right direction, but at some point it becomes more a question of the tester and how well the tester has constructed the logically equivalent questions.

Language is subtle. The boundaries of sarcasm and hyperbole can be extremely narrow. It’s very difficult to write these tests procedurally. If we could write them procedurally, we could solve the problem.

Tim Scarfe

This is another area where we have potential. WordNet, for example, was also a knowledge graph. It was brittle, wasn’t updated very often, and didn’t understand colloquialisms or the way words change meaning with context.

But the composability and distributed nature of deep-learning architectures might allow us to start with a pretrained base model and then have a high-velocity model that learns the colloquialisms relevant to us.

Connor Shorten

Definitely. If we’re talking about Lyft customer service, and the map has made an error, you and I might describe our problems very differently in natural language. That’s the difficulty.

The system needs to adapt quickly to how Connor describes his frustration with the map, and then quickly adapt to how Yannic describes his frustration with customer service. Domain adaptation and rapid adaptation to a new domain are going to be key to making this applicable.

Tim Scarfe

There’s something interesting here. People think chatbots are all about AI, but they really aren’t. They’re a user interface, and they’re best constructed with static rules.

The natural-language processing you need is things such as intent recognition: what does the user want to do? If you had an unstructured output, imagine plugging a GPT model into your chatbot and asking, “How are you feeling today?” The GPT model might say, “I’m feeling great,” but what value does that have? It’s completely useless and unquantifiable.

If I wanted to test that system before releasing it into production, I’d need to write tests and understand what it was doing. With an unconstrained generative model, it’s completely useless. I have no idea what it’s doing.

Yannic Kilcher

That’s why I find this task formulation interesting. Could you pretrain a model on chatbot interactions and then fine-tune it? Could you give it a textual task and have it rapidly adapt to the new domain?

It would be interesting to know whether a textual task instruction actually helps it learn a new task faster than an arbitrary task identifier.

Tim Scarfe

That’s the end of the show. We were trying different formats. Last week we had an author on, so that was more of an interview style. The week before, we had several people in a field none of us was really well versed in. Someone would simply have something to say at some point.

We don’t yet know what’s best or most interesting, so we’re looking for feedback about what people like and what they find interesting. Should we be highly prepared and very technical for people who already know everything, or should we introduce the concepts first for people who don’t understand them?