为什么人类仍在驱动 AI [赞助内容] - Phelim Bradley

Tim Scarfe

There's a dirty secret in Silicon Valley and in the tech world, isn't there? I don't think people realize the extent of this. There's an absolutely huge importance placed on human data, human expertise, and human understanding, and that is completely glossed over.

Pádraig Ó Céileachair

Yeah. Fundamentally, artificial intelligence is founded in human intelligence. In the stack of data, algorithms, and compute, I think the human data element is often the least spoken about, maybe the least glamorous. People want to imagine that there's a simple kind of input-output equation, but ultimately there's a messy layer in the stack of human beings who are providing their data to label data, provide RLHF post-training data, and then ultimately evaluate and assess model performance. All of that ultimately has an element of human data in it.

I can't tell the difference between a doctor and someone who pretends to be a doctor. The only way that you can actually tell the difference is if you have this deep, abstract understanding and you know that they're breaking the rules.

It's increasingly clear that frontier models as a platform are going to be centralized and controlled by a relatively small number of players, which at the moment is almost exclusively these U.S. tech companies. So I think there is a bit of a wake-up call. What the future will hold is basically a marketplace of intelligence. In the past, we had a marketplace of oil, for example, or electricity. Intelligence is going to be the new traded thing.

I'm the co-founder and CEO of Prolific, and Prolific is a human data infrastructure company. So we make it easy for people developing frontier AI models and running research to get access to trustworthy, high-quality participants for high-quality online data collection. Prior to the ChatGPT moment, the primary modes of data collection, the people were fairly fungible, right? So you were optimizing for cost and scale, maybe offshore, lower-cost labor, which I think created this dynamic of human data for AI being a bit of a dirty secret.

1. Routing Expertise To Tasks

Tim Scarfe

Let's talk about your core technology. You've solved an interesting problem that I've tried to solve in the past. I started a company called Merge, and it was a code-review platform. It was exactly the same thing: I realized that code review has to be done by humans, and people talk about automation in the software engineering life cycle. It's mostly—it's actually orchestration. You need humans involved in every single step, most importantly code review.

We had a skill matrix, and we could learn their skills. A pull request would come in from one of our customers, and we would dynamically assign it to an expert in that field. They would do it because you don't want a superficial, rubber-stamping pull request. You need someone who actually understands the code and goes into it. I think this is one of the biggest problems in business in general: we have all of this expertise over here and these problems over there. How do we match them together? How have you done that?

Pádraig Ó Céileachair

2. Vetting And Incentivizing Participants

Firstly, I appreciate that you understand the challenges and the complexity of dealing with human data, as well as the orchestration and routing of tasks to the right human. Often, people want or expect this data to be as simple as calling an API and getting a CI/CD-style response where it's fully automated, easy, clean, and simple. But the reality is that humans are messy, and dealing with humans, especially at the scale that we're dealing with them—where we have hundreds of thousands of active participants or raters on the platform—is challenging.

Fundamentally, the value that we add is the deep verification and vetting of these participants. We build up a deep profile of the nuances of each of our participants and the behavior that they provide in data-collection tasks, so that we're able to route the most appropriate task or the right project to the right humans.

I think increasingly understanding how to incentivize these participants and making sure that we're incentivizing the right behavior is a win-win-win dynamic across the 3 different relationships on the platform: the data collectors, us, and the participants. We're not treating this as a strict supply chain where we're trying to commoditize or aggregate the participants and their data and get it for the lowest cost. Ultimately, we believe that the highest data quality is produced by people who are properly incentivized, understand the impact of their work, and are going beyond just the financial incentives.

Tim Scarfe

I mean, obviously, we could use the Uber analogy, but it doesn't quite work because we're talking about very deep expertise here. We're talking about very, very specific things. You need to find people and verify that they have that expertise. You need to check that they're not gaming the system. So I'm not sure you must have some kind of operational analytics where this is what a normal behavior profile would look like, and you need to incentivize them.

Certainly, when you employ people in the real world, you incentivize them in terms of things like autonomy and cultural fit, and lots of human-like factors. How do you do all of that?

Pádraig Ó Céileachair

The onboarding of the participants starts with everyone being ID-verified, to make sure they are who they say they are and are where they say they are in the world. Secondly, there's a feedback loop from the researcher. This involves analyzing and assessing the QA of the data, feeding that back into the model, and ultimately using that information to rank participants.

You're not just selecting for an audience; we're able to preferentially provide the participants who are going to provide the highest data quality for that task context. The third element is network analysis: looking at our participants as a network, understanding the interconnections between them, and finding pockets of participants who are behaving well, or people who are maybe trying to game the system. We use that information to filter the pool or downrank those participants so that they provide less data over time, or ultimately remove them from the platform if required.

The other thing is the incentive piece, which I think is very interesting—a kind of game theory around behavioral research. You mentioned Daniel Kahneman and co.; we know that when the opportunity arises, people tend to cheat a bit, especially when you have single-shot relationships. There's the classic game-theory idea of “share or steal.” I don't know if you're familiar with that experiment.

Tim Scarfe

No, it's a bit like the prisoner's dilemma. The prisoner's dilemma—that's what I'm thinking about.

Pádraig Ó Céileachair

There's the game-theory idea of the prisoner's dilemma, where ultimately, if you see that as a single-shot relationship, a single point in time, the incentive is for both participants to steal. The thing that changes that into a dynamic where both participants are incentivized to share—or, in our context, not cheat or game the system—is if you treat it as a relationship and have multiple touchpoints over a long period of time.

You have high communication between the 2 sides of the platform, and ultimately you drive toward this kind of win-win-win dynamic rather than a single-shot experiment or data collection.

Tim Scarfe

You might be measuring how long they take to do tasks. So how do you bring the human component into it?

Pádraig Ó Céileachair

3. The Human Layer Behind APIs

That's a great question. You're probably familiar with the analogy of the Mechanical Turk.

Tim Scarfe

Oh, yes. Tell the audience about that. There was a chap who was, I think, a robot playing chess. That was a classic example, and it was perceived by the audience as a fully autonomous process. Then, behind the scenes, ultimately, it was a human controlling the robot.

Pádraig Ó Céileachair

I think the analogy of the Mechanical Turk is super interesting because people want this process to be simple and to be able to fully abstract the humans behind an API and call human intelligence on demand via an API. That is the value that we ultimately want to provide, and we try to abstract away as much messiness and complexity as we can.

I think how we bring the human element back in is by trying to get out of the way as a middleman. Our philosophy is to try to build a direct connection between the people collecting the data and the participants providing the data. They're able to communicate, provide feedback, and ultimately understand the impact that their work is having on the data collections, whether that's research or model development, et cetera.

Getting this mutual feedback across the platform, this peer-to-peer messaging, is crucial to trust and to making sure that there's a sufficient amount of empathy for the people who are providing this super-valuable data.

4. Solving The Specification Problem

Tim Scarfe

Now, I've got a lot of experience with Upwork, and it's good and bad in a way. I don't like it because, as we were just saying, there's this huge epistemic history.

Even if someone is a creative professional and they've been doing it for years, I still find that quite often they're just uncalibrated, and the onus is on me to specify what I want to an insane level of detail. The onus of doing that is often greater than the cost; I might as well just do it myself.

How do you overcome the specification problem? Do the tasks that you do on Prolific tend to be quite close-ended? By that, I mean, do you have specific outputs, or are they sometimes quite ambiguous and open-ended in terms of the output?

Pádraig Ó Céileachair

That's a great question. I think it ties back to our obsession with data quality, and I think there are 2 aspects to data quality. There's the quality of the audience: what expertise, specialism, and training do the participants have? Then there's the quality of the task design, or even the specification of the people you're looking for.

We'll often get requests for PhDs in biology. There is quite a lot of nuance within biology: are you looking for genetics expertise, bioinformatics, healthcare, and so on? Supporting the researchers in specifying the audience with sufficient detail is therefore important.

We have a mix of tasks. The platform is fairly use-case agnostic. Some of it is fairly self-contained, while other tasks have a game-theory dynamic with multiple touchpoints involving the same participants. That tends to lead to a relationship between the researcher and the participant, which leads to better data quality.

We have many projects that are long-running, where the first part of the project is training or providing context to the audience, so that they're able to learn over time what good data quality means for this project. That can be very long-running—sometimes weeks or months of longitudinal or multistep data collection.

5. Human Expertise On Demand

Tim Scarfe

Very cool. So, essentially, you've built this platform that gives you human expertise on demand. What does this mean for the future of work?

Pádraig Ó Céileachair

We've optimized the platform for breadth of audience choice. Although you can earn a great side hustle on Prolific, we don't necessarily want to optimize for very professionalized raters or research participants. We want to tap into real-world users.

Let's say, for example, you're looking at healthcare workers to evaluate your medical chatbot. We want to tap into people who are actively working in the field, not people who've left the field and are now professional annotators. We're trying to reflect the real world and real-world users as much as possible.

We absolutely see our platform as an augmentation to work rather than a replacement, although increasingly this work of human data for AI is being professionalized.

Tim Scarfe

What I like about it is that it increases market efficiency. The market is all about having these economic tasks that are valuable and having people over here with the relevant skills.

Pádraig Ó Céileachair

Absolutely. That's something we're explicitly thinking about: how do we train people with the skills that are useful for these frontier-model providers?

For example, how do we take a general participant and turn them into a high-taste evaluator? It's not necessarily just domain expertise or domain knowledge that's valuable, but also a general audience that is skilled at overcoming the typical biases of preference evaluation.

Tim Scarfe

With Uber, for example, there is a critical mass. Dating websites are another example; you have to bootstrap the platform. Uber wouldn't work if there were no density of cars in my area.

Where did you have to bootstrap the platform and then find that it got easier? Or was it easier at the beginning? Even when you have loads of participants, you might have problems where there just isn't enough work to go around because it's very stratified. For example, if you're doing demographic research and you want 6% of people from a particular demographic, you might have this disparity problem.

By the same token, when you actually have this scale, do you find that it works better?

Pádraig Ó Céileachair

We have the chicken-and-egg problem of all marketplaces. You can think of this as analogous to the Uber cities problem. In the early stages, the analogy of a city in the Uber context for us would be a segment of the audience, or people with a particular skill set, where we might have bootstrapped from an atomic network to a scaled network.

For example, we might bootstrap a US general-audience population or a UK-audience population. But for each new expert or segment that's in demand, we need to go through that process of scaling the network up to a point where we're able to provide instant data collection. We optimize for data collection in a matter of hours rather than days or weeks.

Ultimately, we look at market liquidity as a user-segmentation problem. We might have scaled networks for particular audiences, and then we think about which next-marginal user could incrementally add the most value to that network. We drive the growth of the network in that way.

Tim Scarfe

The matching algorithm itself—could you tell me about that?

Pádraig Ó Céileachair

That's secret sauce, for sure.

Tim Scarfe

I'd be fascinated to know because I'm thinking that maybe there's an analogy to Google Search. They have the PageRank algorithm, which is an example of this kind of social-graph metadata: you put a hyperlink to another page if you actually like that page, and then you can build up a ranking from that.

Then there's intrinsic content-type metadata, which you guys have presumably found a way to mix together.

Pádraig Ó Céileachair

Exactly. I think another analogy is to what are called two-tower algorithms. Think about TikTok, Instagram Reels, or YouTube, where you have the context of the user who's filtering for the content, and then you have all of the available content.

The analogy here is that you have the task context and the human context, and you want to rank the humans so that the optimal people float to the top. In the same way that when you open YouTube, Instagram Reels, or TikTok, you get the content that's most relevant for you.

6. AI Power Is Centralizing

Tim Scarfe

Whoever controls these AI platforms controls quite a lot.

Pádraig Ó Céileachair

The internet, as a fully decentralized platform, isn't particularly owned by anyone. It's increasingly clear that AI infrastructure and frontier models as a platform are going to be centralized and controlled by a relatively small number of players.

They are predominantly US players at the moment. China may be playing a role, but unfortunately, there isn't a very significant role being played by the UK and Europe as a whole right now.

I think we're lucky that many of the people who work for these global labs, even if the capital is coming from the US, are international and have a global perspective. When we work with our customers in the frontier labs, we find that they have extremely positive intentions. They want these models to be globally useful and to reflect input from a wide variety of opinions and subjectivities.

There is obviously a risk that if these models become superintelligent and there is a massive labor impact, the cost is going to be felt internationally. It will be felt by us here in the UK and Europe, while the value of that efficiency is going to flow to the owners of the platforms, which at the moment are almost exclusively US tech companies.

I think there's a bit of a wake-up call for the UK and Europe. Even though we're maybe late to play a more significant role in the life cycle of these models, we could own more of the training, have more locally produced models, build data centers, and develop energy abundance in order to power these energy-hungry models.

I definitely think there's room for more UK and EU dynamism and accelerationism in this space. We are still hiring software engineers aggressively, including junior software engineers.

Some people have a philosophy that junior software engineers won't be hired anymore because a senior engineer plus AI agents will be 10 times more efficient. I think that neglects the fact that these models are extremely powerful teachers and coaches, and that junior engineers can become as competent as senior engineers much more rapidly with this copilot training to become better software developers.

When I started Prolific, one of my motivations was to learn about web development and building a product. This was a relatively slow learning process compared with what it would be now. I would have killed for a near-superintelligent copilot in order to build more products faster.

I think there's very elastic demand for many of these things, such as software, that models are going to improve our efficiency in building.

7. Humans Stay In The Loop

Tim Scarfe

There is something special about local, situated human expertise, and what I see is that this could make the pie bigger. Assuming that these models will be limited in how they understand different domains, we might in the future need to have an operational loop on top of language models, just so that people can actually verify the information. It might be consequential health advice or something like that, and the user doesn't know whether it's correct or not. So we could have a Prolific-type plug-in system where a user could press a button and say, “Is this legit?”

Pádraig Ó Céileachair

This is definitely where we see the direction of travel also: AI-human interaction, or agent-human interaction, analogous maybe to human-computer interaction. I think that's the next phase of research.

You could imagine, for example, a deep-research-style agent going off and doing a long-running task, and one of the steps along that workflow is to get a review from a human expert that's routed to the most appropriate person for that task. Again, I think something that we're looking at at Prolific is: How do we build these systems where agents and humans can collaborate, and where we can move beyond this kind of relatively simplistic preference-based evaluation in order to build out systems which better simulate the ultimate objective that we're trying to aim for?

Right? We know that models, when they have a goal, because of Goodhart's law, are very, very good at optimizing for that goal. Therefore, choosing the right goals is increasingly important, and developing the tools to effectively simulate those goals and get as close to the true objective as possible.

Whatever—I think you mean by that objective—but ultimately, I think it's real-world performance for real-world users, and real-world context is ultimately the thing that we're trying to simulate with all of these evaluations and benchmarks. We are still building a marketplace of intelligence by creating this link between all of these human-situated experts and the system, which matches and learns. Maybe there'd be a middle way. Maybe we can create some automation against that. But fundamentally, I believe that we're going to need human expertise more than ever.

Tim Scarfe

I think the pie has gotten bigger because so many people are just getting their appetites whetted. They're generating videos, writing code, building applications, and doing all these things they couldn't do before. Then they hit a brick wall because they realize they don't actually understand it deeply enough, so they need to bring the experts in. All of these experts are in more gainful employment, in my opinion, than they ever were before, and this could create a virtuous cycle. I think that would be a positive outlook of the future. What do you think?

Pádraig Ó Céileachair

I tend to agree. I think this reminds me of the dynamic between synthetic data versus human data, and people often frame that as an either-or proposition, whereas we're very bullish on both synthetic data and human data. Ultimately, augmenting human data and human expertise is expensive, and models are able to make that cheaper and maybe more effective.

But when you reduce the cost of something, you tend to increase the demand to compensate for that effect. That effect often comes up with these AI models as infrastructure. I think you'll end up with a similar effect even if we're accelerating human data in the context of post-training or model evaluation with LLM-as-a-judge or synthetic data, because of the explosive demand for these models in the first place. Even if the proportion of human data decreases over time, the actual scale and importance of that data is likely to increase.

The Financial Times, The New York Times, YouTube, Reddit, et cetera, get a recurring payment for their data, and it's licensed. I think you could also imagine human expertise being licensed in a similar way, where you get an ongoing passive incentive for providing data to improve the model—maybe analogous to Spotify, where the proceeds of all of your subscriptions go out to all of the people who are providing music.

You could imagine a similar approach to incentivizing human data in the future as well, where we get this kind of more ongoing incentive to continually improve either a very personalized model, as you mentioned—a digital twin of your expertise. I think that's maybe the more obvious example, or even to improve the infrastructure of centralized models.

Tim Scarfe

Yeah. It's almost a bit like you buy a solar panel and then you can give energy back to the grid.

Brendan Foody

Yeah.

Tim Scarfe

Yeah. Similar thing to that. Awesome.