为前沿 AI 输送数据的隐秘供应商

What do agents want? Access. And getting access is becoming difficult. The internet is changing, and AI is turning the situation upside down. More and more of the internet is becoming a black box.

This is where Grass comes in. It almost seems like magic. At a high level, it’s simple: users access the internet as they normally would. But the AI sector needs substantial data infrastructure. We’re already operating at the exabyte level, and Grass provides AI labs with exabytes of data.

1. Why AI Labs Won't Buy Slop

An exabyte is a lot. For those who don’t know, we flew to Ashburn, stayed up all night, slept on the floor, and assembled servers. Grass is a way to provide access to that information. That’s the goal.

2. From Nuclear Physics to Web Scraping

Before I say more, Delphi Ventures is an investor in Grass, and we own some of the token. Andrej, thanks for being here. Could you tell us a little about your background and how you got into this?

Thank you for having me. This is great. I’ve never been on a podcast in person before. I’ve done a lot of Zoom calls and things like that, but this is great.

I’m the co-founder and CEO of Wynd Labs. We’re the team behind Grass, and we’re actively building it. Grass is a network with tens of millions of nodes. People choose to use it voluntarily, share their bandwidth, and get rewarded for it. On the other side, we’re building data infrastructure that uses the bandwidth of millions of people to access web data at scale.

I don’t know exactly how I got into this field; it happened pretty randomly. I studied engineering physics in Canada, specializing in nuclear engineering and photonics. When I graduated, there weren’t many jobs in nuclear engineering in Canada, and the ones that were available were in remote areas. I was in my early 20s, and I didn’t think I wanted to go there. I’d learned skills in physics that could be applied to financial work, so I did that for a few years.

After that, I was planning to go back to academia for a higher degree in computational physics. Then the pandemic started. I applied for a PhD in applied math, working on complex physics equations and the mathematical modeling behind them. As a PhD student during the pandemic, I found myself with some free time and turned to a few side projects and hobbies. I started building things on a range of random subjects.

3. The Pandemic Signal Everyone Missed

One of them was an arbitrage bot. It looked at prices across different websites and found arbitrage opportunities. That required a lot of scraping, so I got into web scraping. A few years later, I met my two co-founders. We talked about our experiences and what we wanted to work on. It was very clear, and we’ve been doing the same thing ever since.

So it started pretty randomly. Public access to the internet was beginning to decline, and you noticed something others weren’t seeing. What did you notice that led you to pursue this?

The biggest sign was that the pandemic pushed a lot of businesses online. At the time, many businesses relied on data for intelligence—for example, price intelligence. How much are your competitors charging for a product? Governments were also trying to calculate the Consumer Price Index. They would send people to stores with clipboards to check prices, write them down, and then calculate averages and do analytics based on the data.

When the pandemic pushed North America into lockdown, many stores closed, and collecting that information in person became difficult. As a result, demand for web intelligence increased. People started monitoring websites to find products, use services, or make purchases. They expected information about those products to be publicly available. But we were also seeing a kind of wall being built around data, limiting access. The public could visit a website, but data centers and other automated sources were being blocked. That created a need for automated web data infrastructure.

I wouldn’t say it exploded then, because it really exploded later. But the need for this infrastructure started when everything moved online during the pandemic. We saw no reason for that trend to slow down. It started with quantitative data, but it was clear that it would go much further. E-commerce websites, prices, and the information displayed on websites were all part of it.

Six or seven years ago, websites were already blocking data centers. They were trying to control access to their sites and show different content based on browsers, fingerprints, and other factors. We thought that trend would spread across the rest of the internet, especially as machine learning algorithms became more powerful.

Before ChatGPT became popular, at the end of 2022, before the GPT-3 AI revolution really began, we were already working on this. AI was affecting many people’s lives, but they didn’t notice it. Much of that was happening through social media recommendation algorithms. We could see it was already underway, and we were watching those incentives. This was becoming one of the biggest problems to solve.

At the same time, hundreds or thousands of legitimate businesses around the world needed residential IP addresses to access websites that permitted it. To get the infrastructure they needed, companies were secretly installing software on people’s devices without permission. They were doing this and selling the resulting access to large organizations. We saw that as a big problem and felt it couldn’t continue. Grass’s main principle is consent: all nodes on the Grass network participate with consent, and they’re rewarded for it.

4. The Smart TV Proxy Scandal

That’s a really good point. Can you explain how an opt-in network like Grass differs from networks that use VPNs, smart TVs, or other devices as backdoor proxies?

This sort of thing gets shut down very often. Doing it without the user’s permission is not only completely immoral; it’s also unsustainable as a business. For example, a company could make it clear in a pop-up that, in exchange for having no ads, a customer agrees to share their bandwidth and earn income from it. The customer knows what they’re agreeing to.

With Grass, you download an app, and you’re directly rewarded for participating. It’s much scarier when software is inserted without your knowledge through an update, whether that’s a software or hardware update. These practices won’t last forever. If you search for the smart TV residential proxy scandal, you’ll find a lot of examples. Unfortunately, some companies continue to do similar things. But I love that Grass is designed around full consent and a strong ethical system.

5. Why the Internet Is Blocking AI

Sure. As artificial intelligence collects huge amounts of data, access to that information is gradually becoming more limited. What are agents unable to do right now, and what are they having difficulty accessing?

6. The Defense Tech of AI Data

Honestly, most of the internet was designed for humans, and it’s very difficult for agents to access. There are many reasons that will continue to be the case for a while. Websites are trying to prevent AI crawlers, or competitors, from scraping their information. They’ve made a lot of changes to identify who is accessing their sites.

Usually, that means blocking data center IP addresses entirely. If your agent is running on a data center instance, such as an AWS instance, websites recognize it as an AWS IP address. An agent scraping prices for competitors doesn’t look any different from other agents doing the same thing, so websites block it by default.

We’re seeing this more and more, and it’s not just an issue for agents. It also affects general LLM usage and the process of creating LLMs. A lot of Grass’s business is focused on this. When you crawl the web to collect data for model training, you encounter these restrictions frequently. To get the best training data, you want to access as much information as possible, so you can filter it and add the right information to your dataset.

7. Mapping the Entire Internet

It all comes down to the same principle: whether you’re using an agent, an LLM, or a crawler, if it’s accessing public information, the more access it has to the web, the better it works. The best way to get that access is to route traffic through residential IPs belonging to millions of people around the world. Sometimes you also need to run on custom browser infrastructure. Grass does that, for example. That’s necessary for tasks such as loading JavaScript on websites or presenting different browser fingerprints.

8. Training Data vs Live Data

Yes, there’s a lot there. We’re talking about both training data and inference data now. I think there’s a bit of a divide when you look at your business: you started by providing training data, but you’re also offering high-quality, live inference data. Training depends on snapshots of the current internet, while there’s a need for a lot of live data. How do you think the dynamics of these 2 markets will develop?

Of course. The way modern AI models are trained involves a training cycle, and it’s often a long process. Much of the data in that cycle is transformed into a format that’s useful for training. After that, you create more data based on the model’s outputs. That’s called reinforcement learning. Based on that data, you can make small changes to the model to make it useful for certain tasks in the real world.

9. The Metric That Matters Most

Once you’ve trained the model and released it into the world, it immediately runs into the limits of its knowledge. It hasn’t learned anything that happened after its training. That’s still an unresolved issue. There’s research happening on continuous learning, where the model is constantly learning and being updated. That’s very exciting, and I feel it will be resolved soon. But with the existing system, the model is trained up to a certain point, and after that, it’s in the dark about information in the world that isn’t available to it.

10. How Robots Learn From Web Video

So, a large part of the process where Grass is useful is data access. As you said, there are 2 different processes, right? There’s the training process, where you collect a huge amount of data from the internet to create useful pre-training datasets. Often, that means targeting a very broad scope—the entire internet. Other times, you target specific types of files, content, or content in certain languages. Once the model is trained, it needs to be able to see things on the internet. That relates to the point you mentioned earlier, right?

You can run a model on an AWS instance or any VM infrastructure, but the data center may not have residential IP addresses. In some cases, you won’t be able to see content that’s only visible in certain countries. Unless you can route through a network like Grass, you can’t access that content at scale.

We’re also really excited about a big emerging sector: inference data. Models need to be able to perceive things in real time. That means you’re using your model, or another model is using a model, for example. This is an ongoing development. For a model to be useful, it should be able to receive the latest real-time information from the world. On the internet, or in robotics, it may also collect information from outside the internet and use it to make decisions.

Maybe in a few years, continuous-learning frameworks that work at huge scale will be available to us. What will happen eventually is that these 2 things will merge. Models will constantly receive real-time information from the internet and update from time to time. I personally find that very exciting. In the current framework, the more you ask of a model, the more it can ultimately give you wrong information, or hallucinate. That’s just the nature of LLMs. Do you think the importance of training data will decrease, or will a lot of information still need to be collected? Will the 2 develop in parallel?

11. The $100 Billion Data Market

I wouldn’t say training data will lose its importance. I think the 2 will converge over time. What I see today is that Grass can collect billions of URLs within hours, across small clusters. That’s an unimaginable scale. As you crawl a substantial part of the internet for training data, you’re actually building a huge web index. A web index is very important for searching and finding the latest information.

So, in a way, Grass’s work on training data is mostly useful for building an inference stack, right? You could call it “treasure hunting,” maybe. I don’t like using that term, but I can’t think of a better one. You’re sifting through the entire internet. Crawlers are deployed to find specific types of data, but in the process, you’re kind of mapping the whole internet. Once you’ve mapped the entire internet, you have the foundation to run search on top of it.

You can search for things, and one of the best things about Grass is that once you find a URL through a search of the whole internet, you can summarize all the content on that URL and provide it in a format that’s useful for the model. And, importantly, you can collect the content on that URL without being blacklisted or getting caught in honeypots. It’s data from websites designed for humans to view, but here the consumer is an agent.

That’s very interesting. I’d like to understand more about the infrastructure needed to do this. You have nearly 8 million nodes, which I think is the most recent number you’ve shared publicly. But you’ve also expanded on the data center side. I know a lot of basic hardware is needed to secure and distribute this data. I’d like to learn more about that experience.

Grass’s data center footprint is constantly growing. We joke about it because every time we order new servers, by the time we set them up, we’re back at full capacity. It doesn’t take weeks or even days; usually it happens within hours, because there’s so much work to do.

Over the last 1.5 years, we’ve invested a lot in infrastructure. We currently have thousands of compute servers and many petabytes of storage servers deployed. We had to buy tons of RAM. Fortunately, we anticipated that before it happened. Our co-founder and CTO, Chris, once went to Ashburn when we were short on compute as we prepared to provide more data to a leading AI lab. Servers were down in the warehouse, so he stayed up all night, slept on the floor, and assembled them.

As I said, we have thousands of compute servers. For now, we’re operating at exabyte scale. Grass has provided exabytes of data to AI labs. What does an exabyte mean? For a lot of people, that’s not a familiar unit. The thing worth understanding is what it means to work with exabytes of data. Think of it this way: a terabyte is 1,000 gigabytes, a petabyte is 1,000 terabytes, and an exabyte is 1,000 petabytes.

Grass is operating at exabyte scale. We can say that because we’ve provided exabytes of content to leading AI labs. You need compute to send web requests. We have millions of people around the world, each with nodes responsible for providing a network fingerprint for those requests.

Storage is important too, because you might ask, “Who needs storage?” AI labs buy data directly, so if you can send them the data, why store it? We’ve recently started allocating more storage because many labs are facing gaps. They’re taking huge amounts of existing raw data, filtering and processing it, and transforming it into something more useful. The training data market needs this more and more every day. So, yes, you’ll need to store a lot of your data.

12. What AI Labs Really Pay For

Let’s get into this in a little more detail. From the customer’s perspective, what’s the use case? You can use this data and send it to frontier labs, xAI, and some other organizations. What’s most important to them about that data: its size, freshness, breadth, or quality? What value are you providing, and how do you think about it? How will the business around it be built?

I’d say it’s changing. Different labs will buy different things depending on their training policies and philosophies. The easiest way to understand it is this: if a lab is making a video model, video data is very valuable to them. If a lab is making a coding model, code data is very valuable to them, and the same goes for other areas.

You can expand that idea. If an AI lab wants to improve a specific aspect of its models—for example, a rare dialect or searching difficult websites—it will put a premium on that data. Size, quality, and speed are all important. Ideally, you’d have all of them, but in practice it will vary from lab to lab. Some labs and many companies in the field are under serious pressure.

Every few months, a new model comes along and beats the benchmarks. If you stop working, you’ll quickly be left behind. We’re seeing a major trend: the need for speed. The more data a model has, the better it works.

If you take the best parts of a huge dataset and weight them more heavily during training, the model can be trained more effectively. But collecting huge amounts of data and feeding it into model training is a massive job, and it takes time. In AI right now, time is the most valuable resource in the field.

That’s why we’re watching this shift: the search for AI training data is becoming inevitable. AI labs have their own theories about how to get data and train their models, but we’re trying to make their lives easier and provide more value. We can help identify the signal in some of that data. This is work we’re doing now, and it’s still in progress.

A large part of this is meaningful. Robotics data and video data are worth discussing, and we’re very interested in that sector. Robotics naturally requires a huge amount of training data.

You said you’d collected almost a billion public web videos, right? How are robotics companies using them, and how useful are they?

Sure. That number may be a little old. We’ve already indexed more than 100 billion.

Why? I mean, you’re not going to blindly download every video, right? You don’t scrape the content of every video. Usually, you have specific targets, because video is a very dense medium. Storage is expensive, and processing is expensive.

Robotics and robotics data are really interesting to me. I want to talk about something you mentioned a moment ago, because it’s fascinating. How does machine learning research usually work? Some listeners may not know. When you’re optimizing for something known, and your data is already in the right format, testing is easy, right? Then you get a lot of satisfaction from it.

If you have data available and feed it into a known algorithm, that algorithm can start learning from the data quickly, and you get a loss curve. The loss curve shows how many mistakes the model is making. As the curve decreases, the model makes fewer mistakes and eventually approaches zero. Seeing a good loss curve is usually a good sign.

When you’re trying to scale a model and improve its performance on a benchmark, giving it more of the relevant data is good. The curve comes down, so you want as much data as possible. A simple example is teaching a model Spanish. You need Spanish data, and you can see clearly what you want the model to learn. You might search Spanish forums or use Common Crawl to find Spanish-language training data. Give that data to the model, and it makes fewer mistakes and gets better results.

But when you get to the real world, it becomes a little more difficult. Ideally, robotics researchers want a model to watch billions of videos, listen to millions of podcasts, and read trillions of pieces of internet data, then learn how to behave from all of it.

For example, say I saw a person using a lawn mower. As a robot, I could learn how that person walks and moves, how they use their hands, and how they pick up and handle objects. If the model sees millions of examples, and you have the right setup and algorithms, it can learn from what it sees and do the task itself.

The exciting thing is that a model can watch a video of someone doing something, like pouring water into a cup, and learn how to do it. I think that was one of the first examples.

Yes. I remember they showed that with a specific task. They made it real: they showed a person doing something, the model watched, and then the robot imitated it in exactly the same way.

Since then, Roa AI, another robotics company, has released a research paper showing that behavior can scale using web videos. There’s a growing need for web videos in robotics.

Other forms of AI don’t have this same challenge. If you’re training a video model, like Sora, you can give it all kinds of videos: cartoons, people having conversations, nature, space, whatever you want. The model learns from all those different kinds of content and combines them. Then you can ask it for a cartoon cat in space, and it knows what to do.

More diversity gives you more combinations, and the model can produce more different outputs. With robotics, it’s almost the opposite. You need diverse data, but it also has to be very specific. You don’t need music videos or cartoons for most robotics tasks. That could change, but today those videos don’t provide much value. Video game gameplay might be useful, because characters move and interact with their environment. Watching people do things can also be useful for learning how to do those things.

We’re still figuring out how to make that data useful. Filtering videos at web scale is very difficult, and it’s work we’re doing now. I hope I can share more about it soon, because it’s personally very exciting to me.

Is this a new network you’re building, or do you need other core infrastructure? Are you facing the same limitations?

The problems are almost the same. Grass is one of my favorite things because it can expand in so many different directions. The same underlying infrastructure can solve another web data problem.

Ultimately, some data is public on the web, and if models need capabilities that can emerge from that data, Grass can help. That’s a very exciting area. The work now is to take that underlying infrastructure and apply it across different types of data and media. You can offer different kinds of data at scale and provide them to labs.

Once you provide the data to labs, who has the rights to it? Is there a risk in how they use it, or is there still a lot of untouched data?

13. Why Data Is the Alpha

Yes, there’s still a lot of untouched data. The data vendor industry may change, but right now many vendors are building specific reinforcement learning environments. They’ve recognized a problem in the models: Claude, ChatGPT, Grok, and Mistral can all fail at certain tasks, and those failures can be measured with benchmarks.

14. Delivering Exabytes to AI Labs

So vendors build an environment where a model can practice, almost like a little jungle gym. The model practices there and gets better on a specific benchmark. Then that improvement is sold as a capability. Not everyone provides a reinforcement learning environment, but many people are doing something similar. There will always be demand for it, but sellers need to understand new demands and new research directions from different labs very quickly.

A lab may train a model to develop specific skills, and demand for those skills can continue for a while. At some point, the lab may be satisfied and want to move in another direction. They may decide that improving the model further isn’t worth the resources and cost.

With Grass, if a lab changes its research direction, you can still give it access to the information on the internet. That information may be useful for its new direction, too. We’ve seen Grass customers change their focus. They’ll concentrate on one thing, become satisfied with it, and then their needs change completely.

The training data space is constantly developing and growing.

You recently released a new product, Live Context Retrieval. What’s the purpose of that?

15. One API Call for Any Website

Agents need live data at inference time. They need to go back to the current situation and access information, and not being able to do that is a problem. A lot of people have identified this.

You want your agent to be able to do anything, but getting it to do that can be a challenge. This is happening in a lot of ways, but who is the customer here? Where is the demand? I’d love to hear your perspective on the product.

Yeah, sure. I think we recently released the Content API. We call this suite of products Live Context Retrieval. It’s for agents that need to access things on the internet beyond what an AI model can access on its own.

We’re starting with the Content API. After that, there will be Search, and then we think we’ll add more capabilities. Many of these depend on multimodal grounding—for example, listening to audio or reading images and understanding exactly what’s in them. Those things are all in development, but for now we’ve released the Content API.

We originally built this API for our own needs, to collect huge amounts of training data. Our team collects different types of training data, and when we started building crawlers, requests through residential IP networks often weren’t enough. Some tasks require a complex TLS fingerprint or a browser running effectively on the server. Websites can also change how they respond. Sometimes, after you send one request, the website blocks the next one. That can happen for various reasons.

What I want to emphasize is that, for heavy web data collection, our team created an excellent Web Content API. Engineers can use it without having to build everything from scratch. The product has been developed internally to a very high standard, and we believe it’s the best available.

The first point about Live Context Retrieval is that we’ve simplified complex processes like managing web requests through worldwide residential proxies, loading browsers, and implementing anti-bot methods. We’ve turned all of that into a single API call. All you have to do is give us the URL, and we provide the website’s content very quickly. You can also specify which country you want to see the site from, and get the content in Markdown or plain text.

Websites often aren’t accessible to LLMs effectively, and people want to reduce the number of tokens they use. We’re also working on more features internally that we’ll make available soon. I’m personally very excited about this. What excites me most is that, over time, we’ll have the ability to serve businesses around the world.

And in terms of use cases, right? Let’s talk about businesses around the world.

This is something all kinds of businesses need more and more. In many industries, it’s very difficult to compete if you can’t get web data without being blocked. E-commerce and advertising are obvious examples, but there are also ordinary agentic use cases. This could change things across all kinds of fields.

For people, the ultimate goal is to have their agents access information that isn’t available in a centralized data center. For example, imagine an agent that tests a stock-trading strategy based on information from around the world. You might want to collect product and stock details from various websites, some of which are only available in certain countries, and calculate fundamentals before a company releases its earnings report. Without a facility like Live Context Retrieval, your agent couldn’t do that.

When you say “businesses around the world,” that gives me a sense of what you mean. Most Fortune 500 companies are already in this category, and as agents become more advanced, they’ll be in it too.

16. AI Flipped Search Upside Down

The Content API is the first product in the suite. Let’s talk about the rest in more detail, and how the product and its use cases might evolve. Search seems like the next obvious item. What’s the reception to LCR been like? Are you looking to speed things up?

We are looking to make searching the web much faster. The difficult part is getting access to the web. Once you can access it, collect the information, and search it, the remaining challenge is changing the layer.

If you know a website’s URL, being able to access its content is already a great thing. A simple agentic call might access 5 to 50 web pages. If you’re doing a deep analysis, you might need to collect even more. But how do you find the URLs to access?

The Content API is useful for businesses that want to monitor the same websites every day, or that already know what they’re looking for. Search normalizes that process, so your model doesn’t need to know where to look. The Search API lets you search for content and access it, and then you can call the Content API to summarize it.

Search is interesting because, in the past, competing with the world’s largest companies to build a search engine was very difficult. Those companies invested heavily in the expertise required to build large-scale data systems, connect concepts, and create advanced knowledge graphs.

There are now many related tools, including AI-related tools like ANNs, embeddings, and information retrieval. I wouldn’t say these have been commoditized, because there’s still a lot of innovation happening, but they’re much more available now. There are many open-source LLMs and machine-learning models that turn text into vectors, making it easier to search. Once information is in numerical form, you can search and rank it quickly.

A lot of what stops people from building a real search engine is figuring out how often to crawl each website and how to collect its content. In the past, when the internet was less restrictive, a few companies had the infrastructure to crawl the whole internet. Building the basic infrastructure, figuring out the logic, and making the investment was difficult.

The situation has changed. Websites are blocking you, so if you want to enter this field, you need the ability to collect data without being blocked. But much of what used to be difficult is now possible with open-source tools. For less than $100 a month, a startup can get started searching 50 million documents in an AWS S3 bucket.

I’m simplifying things, but we’re in a position to offer an excellent search engine using open-source tools. We haven’t reached that point yet, and we still have a lot of work to do. However, the competitive moat has definitely decreased.

17. Can the Network Keep Up?

I’m really excited about the Grass network. You were able to use tokens to create a network of nodes, collect data, and sell it to frontier labs and businesses. You’ve also been able to build all these products on top of it. Do you think the network can pay for everything the products need, or does it need to expand further?

That’s a good question. Expanding the Grass network and continually adding more residential nodes is important, especially for latency-sensitive inference tasks. Latency isn’t as important when you’re collecting training data. If a certain area has enough nodes, you can wait a little longer to collect the data. Speed matters, but inference data has other constraints.

With agents, models, or systems, you don’t have that flexibility. If you need data from a particular area, that area has to fill up quickly, and then you need more nodes. We expect the number of nodes required to grow.

I used to say that the number of nodes on Grass would have to increase tenfold before we could launch LCR. We’re launching it much faster than I thought, with fewer nodes than expected. We optimized the infrastructure, and found we could do more with fewer nodes than we’d anticipated.

That’s excellent. It lets you do this earlier than you thought. When we talked about the timeline before, we were saying this would happen by 2027, so it’s great to see. Do you think the network will keep scaling, and that you’ll be able to get more out of every node?

Sure, it seems that way.

Yes. There will clearly be limits to how far we can scale. We’ll eventually hit diminishing returns, but we’ll keep trying to optimize and capture as much benefit as possible.

You’re in a special position, because the frontier labs are your customers. What’s it like working with them? What are their demands? I’m curious.

It’s very exciting, man. AI will definitely be remembered as a major moment in human history—maybe even as something that changed an era. Seeing these models improve and help bring human civilization to a new level is really powerful.

It’s exciting for our team, too. For many people here, this is incredibly satisfying work, and I think it’s one of the things everyone is most excited about. A lot of the resources and talent are concentrated in these labs. The people there are exceptionally strong, so every conversation is intellectually inspiring. There are some really fascinating topics to explore.

One of the hardest things must be that you can’t always tell the world about the work you’re doing. I think I mentioned this a few months ago on the podcast. We’ve worked together to some extent, and I’m curious about this: when AI data helps a defense organization or a government catch terrorists or stop an operation, what kind of work are you doing with them?

They don’t talk about it publicly, because someone could use that information against their customers. In AI right now, access to the best capabilities is almost like a race. If a model company’s focus is on data, it makes sense that they wouldn’t want everyone to know what they’re doing. If everybody knows, they could build the same capabilities and compete with them before the model is released.

Where do you think the industry will be in 6 to 12 months? It seems like a lot of models have improved faster than people expected. As a data provider, are you seeing the requirements change as the training process changes?

It depends completely on the circumstances. Some labs have ongoing needs, while others are more affected by how they allocate internal resources and how much time they have available. Raw data and specialized datasets change from one situation to another. Often, you’ll see a process develop around collecting a huge amount of data from a particular domain.

Sometimes labs want to do their own verification. AI models need more data, so they’re exploring ways to build a model with a specific capability in a new domain. They may need a huge amount of data from that area, and we work with them to evaluate what’s useful. We filter the data, and they might come back and say, “We found that about 0.16% of the dataset is useful for changing the results.” Then they’ll ask if we can get more. We prepare data to meet those needs and provide it.

So that process is continuing, but the need for training data has increased significantly.

What are the biggest obstacles or limitations when you’re providing data to frontier labs? Do they have the capacity to do this themselves?

It may be a little different for us, but the labs are used to working with vendors that provide a particular type of data. They may assume that the data they need is already available from an existing vendor. Sometimes they want to improve their model in a particular way, so we consult with them about data we’ve collected that might be relevant.

They might contact us and say, “We’re looking for this category of data. Can you do it?” But Grass isn’t just a data vendor; it’s infrastructure for collecting data across the internet. Some labs have data-collection teams organized by topic. Some have teams focused on tokens and text, looking to help their models work better with text or learn different languages. Other teams focus on audio or vision. Procurement teams often operate separately.

These labs are growing, and the way they’re organized can be hard to see from the outside. They may already be using a particular vendor for a specific task, but we’re happy to help if what we’re doing could be useful.

That sounds like a big challenge. There’s less time for everyone, and you have to organize all these resources effectively. Do customers optimize for cost, or for quality? Do different datasets give you a financial advantage?

Both are important. Sometimes they’ll pay more for speed. If they need something in 3 hours instead of tomorrow, they’ll pay whatever it takes to get it. But generally, the labs aren’t interested in buying useless data. Sometimes that happens, and they’ll blacklist a seller or simply not contact them again for a while.

The more advanced labs are thinking about how to push beyond what they’ve been doing for years. Many have built internal benchmarks. They’ll say, “We have a specific benchmark, and we need to improve our model on it.” Then they’ll estimate the financial benefit of improving the model on that benchmark. They can calculate how many more tokens might be used if the model performs better, how much the data costs per unit, and what that data is worth. That’s a common approach.

You mentioned collecting data at the exabyte scale. What metrics are you proud of, and what’s the goal you’re trying to reach?

There are a few. The number of nodes on the network was especially important when we started. It’s still important. The number of people who download Grass and join the network matters, because it’s easy for people to download the app and passively contribute. A lot of people had to do that, and the early referrals helped Grass grow. Without them, this infrastructure wouldn’t exist.

Another thing to consider is the number of pages we crawl. But anyone can install a crawler, have enough computing power, and crawl an enormous number of pages. The question is whether the information on the internet is useful. Knowing what’s useful and deciding how to use it matters more than simply counting pages.

Latency is a big one, too. I’m really proud of the success rate of the Grass Content API. Many vendors claim numbers close to 100%, and the Grass Content API is very close. A 100% success rate means that, given any URL or public website, the system can find a way to collect its content. I’m very happy with where we are.

Latency is still a major problem, though. When traffic is routed across dozens of IPs around the world, every web request takes time. The physical laws themselves are the main constraint: the speed of light, and how long it takes for a request to travel from a person’s device through a third party to a web server. We want the system’s latency to be as close to the original as possible. We haven’t reached that yet, and that’s what we’re working on.

Yes, and there’s more. A lot of what we do internally is based on data. We often look at different statistics and ask, “How do we improve this? What do we need to do to achieve that?”

Got it. So, beyond data delivery and network quality metrics, I wanted to ask you about the data market. The market is estimated to be worth $100 billion by 2030. You’re building with that in mind—how much of it do you think you can capture?

Grass can expand into many different sectors. A network that provides access to information on the web is useful not only for that, but also for supplying pre-training data at scale. Inference is now becoming part of the data landscape, too. Preference data, synthetic traces, and all kinds of other data are also within scope. Ultimately, we hope we can address all of these areas.

What’s special about Grass is how much of the world’s information it can help people access. The capabilities it has can also provide huge benefits in other fields. For example, if you’re working with synthetic data and have a lot of raw data, you’re in the best position to create high-quality synthetic traces. Or, if you can find and access any part of the web, you can put better web access into the reinforcement-learning loop and improve a model tremendously. In my opinion, people aren’t talking about this enough. It applies to real-world data, too.

18. When Labs Change Direction

In terms of consumer distribution, millions of people have downloaded the Grass app. We shouldn’t forget the rewards for downloading the app, but there’s still a lot to do, and the product can be further developed for specific verticals. I don’t think there’s any part of the data space that Grass can’t address. Of course, there are areas we haven’t prioritized. Given the current situation, I don’t think we’d see that much value in them yet.

19. Who Actually Owns Grass

Yes, I think of the AI data market as one of the markets for Grass. You recently released a revenue attestation, and the market is very excited about it. You also get a lot of questions about its structure and performance. I’d like to hear your thoughts on that.

Grass has a very special structure. That may seem unusual to some customers, but it’s designed that way for important reasons, all related to aligning incentives. The attestation document was published by an accounting firm. They reviewed the purchase orders from major AI labs and other customers, then compared those orders with receipts for cash received. There may be small differences—for example, some invoices are paid 60 days later—but, generally, they published a reconciliation of the two.

A lot of well-known companies are involved. The entity is specifically the Grass Foundation, not a separate equity vehicle. What many people don’t know is that the Grass Foundation is an ownerless organization. It’s responsible for all of Grass’s customer relationships. Whether customers are purchasing pre-training data, fine-tuning data, or inference products, they ultimately deal with the Grass Foundation. No one owns it.

The foundation is overseen by independent directors, who approve everything that happens. Wynd Labs is a service provider to the Grass Foundation. You still need to hire people and provide benefits and pay for things like insurance, so Wynd Labs can operate as an equity-based vehicle. But it doesn’t have any intellectual property. When the token launched, all of that was transferred to the Grass Foundation. Wynd Labs doesn’t work directly with customers.

Any interaction between Wynd Labs and the Grass Foundation, including financial arrangements, has to be approved by the foundation’s directors. The Wynd Labs team are token holders, too. It’s a very special structure, but everything is coordinated and documented.

20. What a Real Moat Looks Like

I understand. Historically, businesses reinvest revenue back into the business. At your stage, that’s what you’d expect a company to do; a lot of startups grow that way. Historically, foundations have raised capital through token sales, and we participated in one of those. With that capital, will you pursue growth, or will you use equity? It’s hard to know.

Predictably, Grass will need more capital.

Sure. Predictably, of course.

Yes.

A few months ago, I had lunch with the founder of a leading AI lab. I’m fine saying his name here, but I don’t know if he likes being mentioned on the podcast. So I’ll take credit for this.

No, he was the one who said it.

I remember it, that’s for sure. He said that a lot of VCs ask him, “What’s your business’s moat? Other companies are building models, so what’s your specialization?” And he said the first thing you have to ask is what you mean by a moat. In his view—and I agree with him—a great moat is your ability to put capital behind your business and get more out of every dollar than your competitors do.

That’s an excellent moat.

Very good.

Yes. Getting more out of your investment than your competitors is an interesting idea.