[AIEWF Preview] CloudChef: Your Robot Chef - Michellin-Star food at $12/hr (w/ Kitchen tour!)

Okay, we are in the remote studio with this very special podcast. We recorded a tour of CloudChef's Kitchen a while ago, but we wanted to record a little bit of an intro in our remote studio so that we at least get a nice audio podcast intro to the company. We're here with my friend and co-host swyx, as well as Nikhil, who's a founder of CloudChef. Welcome, Nikhil.

Thanks for having me, swyx.

Okay, so yeah, welcome back, swyx. By the time this launches, people will have heard the Anthropic podcast that we did. I think the headline people will see when they see CloudChef is that it is an AI chef. You have this pretty viral video on Twitter from when you launched and told everybody about it, but people also don't know that this is a real restaurant: you actually run a real restaurant. You can order food on Uber Eats. It's really good. What is CloudChef? What is the scope of it? How do you pitch the company?

At a very high level, what we're trying to do is make high-quality, nutritious food available to everyone. The reason it's possible to even think of a future like that is because you can automate practically all non-managerial work inside a commercial kitchen with culinary intelligent robots. Culinary intelligent robots are basically robots that act like human beings, learn like human beings, and work like human chefs.

The video that Sean was talking about is the launch video of our first robot. It's a robot that has a mobile base and 2 hands, goes around, and does work inside a kitchen. It's just like how you would hire a human employee or a human chef: you would hire a robot, and the robot would come to your facility, cook, and learn recipes from the chefs inside the facility with a single demonstration. It would cook that dish over and over again, or participate in that workflow over and over again, like a human employee would. Then you pay the robot an hourly wage, just as you would pay a human.

So far, our robots are used by Michelin-star chefs, fresh fast-food restaurants, airline caterers, and a whole bunch of commercial facilities. They use our robots as hourly-wage labor instead of buying a robot. The thing that makes our robot special is the fact that it can execute at a chef level, or the fact that it has culinary understanding better than even the best chefs in any single cuisine.

What that means is that if a robot is cooking, it needs to know how brown the onions are, how far along you are in the cooking process, what happens if you're cooking in a slightly different appliance, what state the recipe is in, and how much heat to give it. All of this requires thermodynamic modeling of cooking and a visual understanding of what's going on. This is what we call culinary intelligence, and it wasn't possible until recently, when multimodal models got good enough and we built out some thermodynamics modeling to aid that. The end result is a robot that can reason and make decisions in real-world cooking processes like a chef would.

More robot foundation models are coming up and getting better. These robots are finally also able to do real actions and real motions inside a kitchen. Right now, they're good enough to do things like gross manipulation. If a human requires more than 2 or 3 fingers to do a task, the robot is probably not able to do it. The good part is that most tasks inside a kitchen can actually be done with just 2 fingers. If you go around any commercial kitchen and your 2 fingers had enough strength, you could probably do most tasks inside that kitchen.

We start with line cooking, which is the biggest labor cost for restaurants and other food-producing facilities. Our robot is able to do line cooking for about 40% to 50% of the world's commercially valuable cuisine, to the point that if we put our robot against an expert chef in that cuisine, our robot can consistently make the food better than even the chef whose recipe it is. Computers just do some things inherently much better than the human brain.

That's a quick overview. We've trained our in-house models to do thermodynamic perception, and we leverage current VLMs and voice models for perception. We also use them to enable the robot to do tasks, talk to human beings, interact with other coworkers in the facility, and course-correct its goals and whatnot.

The high-level goal, as I said, is to replace all non-managerial work inside commercial kitchens with culinary intelligent robots. When that plays out, we think we'll all live in a future where we have access to really high-quality food at fast-food price points. At McDonald's price points, you should be able to eat the tastiest food that you've ever had in your life. That's the thing that we want to create, and now that the robots have started to work in the real world, we see a future in which we'll make that possible soon enough.

We actually started experimenting with these robots in our own facility. We built an in-house delivery kitchen at our office in Palo Alto. We weren't expecting it to do this well; it picked up really well on DoorDash. My co-founder and I had moved from India to Palo Alto, and we were missing really high-quality Indian food here. We went to our favorite restaurants in Bombay and Delhi and asked, “Can you record your recipes? We'll serve them in California and give you a royalty.”

That's not the core business we're focusing on. It's just something we use to validate our technology. The fact that it's doing so well and the ratings are so good is a testament to how good the technology is and how well the robot is functioning right now.

I can confirm we tried the food. We'll see it later in the video. It's really, really good food. I like the term you use there: artificial culinary intelligence. ACI has been achieved internally. It's interesting because when you frame it like that, you guys are doing something pretty different from robotics, right? You mentioned that the robots are more like off-the-shelf parts. It's not specialized robotics; it's actually the software underneath. So yeah, we can talk a bit more about that.

Correct. When we started the company, we had 1 core ideology: we would only solve problems that could be modeled as software problems. Culinary intelligence and decision-making were the first big open problems that we could model in software and solve.

But when we started, robots were still very much an electromechanical problem. They weren't really a software problem. Now, with robot learning models and robot foundation models, it has gotten to a point where you can start modeling the physical actions that somebody does in software, solve them in software, and have software iteration cycles.

We didn't want to build any hardware. We didn't want to be a hardware company because that wasn't our strength, and we didn't think the hardware iteration cycles would be beneficial for a company like this. Very recently, it has gotten to a point where you can take off-the-shelf parts, put a bunch of robot intelligence—quote-unquote—on top of them, and get the robots to work. That is a software iteration cycle. You don't have to build your own hardware, get into how to manufacture the motors, manufacture the robots, or design everything. We rely on the ecosystem to solve all those open questions for us.

We source general-purpose robot parts, or general-purpose robots, and write software on top of them. We take general-purpose robots and leverage general-purpose intelligence, such as LLMs, VLMs, and robot foundation models. Then we build this proprietary culinary layer on top, which has everything to do with thermodynamic modeling of cooking, custom evaluations for manipulation, and understanding through perception what stage of the cooking process you're in.

We've built all of those things, and we are hoping to ride the tide of advances in both multimodal models and robot foundation models, using general-purpose robots as the vehicle to make that happen. That's, in a nutshell, how our approach works. We are very focused on modeling every part of the workflow as a software process.

Now that we're able to do it, we can do full-stack work inside a kitchen. We're not just an assistant robot that can be prompted by somebody on-site or a guidance system that tells humans what to do. It's now able to do full-stack work because the entire workflow can be modeled as a software process.

I know that we'll see how the robots work and what's happening under the hood later, but the overall question I'm sure a lot of people have is: What's the business model? How do people hear about this? How do you think about renting a robot for $12 an hour versus hiring a chef? How do you arrive at this hourly rental rate and all this stuff? It's very cool to see, and it's good to see that it works.

Yeah, yeah. I’m just curious: how does that side of the business work?

From a business perspective, the main thing to keep in mind here is that food prep is the most labor-intensive industry of all labor-intensive industries. Quantitatively, the way you measure it is by how many full-time employees you need per $1 million of revenue generated. Food requires about 13 people per $1 million of revenue generated, and the second-most-labor-intensive industry is hospitals, which require 4 people per $1 million of revenue generated.

So food is more than 3 times as labor-intensive as the second-most-labor-intensive industry. Labor costs are going through the roof and have been increasing year over year. Depending on what Trump does with illegal immigration, they could go even higher, and staff turnover is really high. The average restaurant is operating at around 130% staff turnover. By the end of 10 months, practically your entire staff is new.

So you have high turnover, very high costs, and the most labor-intensive industry. The reason we ended up at this price point, or with this sort of pricing model, is that food service is not a very profitable industry. They don’t have free cash just lying around to do experiments, and there aren’t fixed budgets set aside for buying new robots or testing things out. If it doesn’t work, they don’t take the attitude of, “It doesn’t work.” There is a very readily available labor budget that we can tap into.

Just like when you hire somebody, you don’t pay for their college tuition; you just pay them a salary. We thought, why should that be any different for robotics? These robots are now cheap enough that you can put that business model out there without losing money on every robot you sell. The robot costs have gotten to a point where an hourly labor-pricing model works, and the robots are also good enough to do the entire chain of work.

At $12 an hour, it’s around 40% of what a loaded human would cost, so our customers get their ROI on day 1. The robot starts working from day 1, and over time these robots just get better. The hope is that at some point they also start making better food at any given facility—not just by cooking the same thing, but by enabling the facility to make recipes that it wasn’t able to make before.

Yeah, awesome. I think the last part—we’ll cut right into the kitchen walkthrough video later—but there’s this general goal of demonstration learning: learning from experts, learning from Michelin-starred chefs. How realistic is this? Is it a marketing promise, or do you really just learn from 1 example? Obviously, food is messy. Food needs a lot of different demonstrations. How realistic is it?

I want to clarify 2 things. One, it is not a marketing thing. It’s actually true. Two, the reason it might feel counterintuitive is because our entire pipeline is not 1 end-to-end model.

If you had 1 end-to-end model and you had to train it to do a new thing, being a one-shot learner would be a very big deal. But in our case, we have many AI subsystems that work with each other. Some of those are end-to-end neural networks, and some of them are hard-coded software pathways. We use the best of both worlds to function, and that architecture choice means we don’t have to go directly from the pixels of what a chef is doing and text or whatever to a generalizable recipe that can be cooked across any robot at any time or scale.

There are software workflows and pathways before that. They take the chef’s demonstration and convert it into an intermediate format that is easily digestible by different parts of our system.

One example would be making an omelet. If you want to teach it how to make an omelet, we’re not learning a new omelet-making skill while you’re showing the robot how to make one. If we had that capability, we would be a robot foundation model, and we would already be doing single-shot learning. It is, from a single demonstration—assuming that we have all the base skills for the robot to do it—extracting what kinds of decisions the chef is making.

Is it visual? Is it thermal? What kind of skill is the chef invoking? Is it stirring or sautéing? What are the parameters for those skills? For us, learning is basically configuring this AI system, not using an end-to-end model that goes directly from pixels to robot actions.

We wouldn’t be able to do single-shot recipe learning if we didn’t have those systems. We would have to have the chef cook the recipe in various backgrounds, at various sizes, and with various appliances. Because we have these engineered midpoints that go from 1 expert demonstration to a recipe form, the recipe can then be recreated across different kitchens, different appliances, and, in the future, different robot morphologies.

Oh, other robot morphologies. But yeah, now you only do the 2 arms, right?

Okay, so we'll get people to a call to action, and then we'll cut to the video. You are going to be at the AI Engineer World's Fair next week. If people want to see the robot live, they can see it there, probably taste some food. Although I don't know how much food we can serve. We'll see. We'll see.

Obviously, I think part of the reason you’re doing this is that you’re trying to hire, right?

Yes. This is an immediately applicable use case. There are only a handful of applied robotics companies that have a path to deploy more than 100 robots in the next year. Now that our robot is working and we have early signs of it being super helpful to customers, you’ll actually be working on a robot that is in production.

There are people in robotics who are working on more complex hardware problems and more complex software problems, but I think we are at the efficient frontier of value being delivered to the customer using cutting-edge techniques and having a rapid scale-up pipeline. I don’t think a lot of companies can say that they have all 3 of those things.

And cooking, like I said, is a very powerful mission. Today, the food that you’re eating is fast food. Most cheap food is fast food. Fast-forward 10 years, and you can eat very high-quality food. In fact, if you work with us, you can already eat very high-quality food in our office, as Sean and Vivu can confirm.

Coming back to it, I think the mission is very powerful. If you are excited by creating value in the real world while also doing it in a way that serves all the current capabilities of state-of-the-art general-purpose models, we are probably 1 of maybe 2 or 3 companies at that intersection. If that excites you, you should come talk to me.

Yeah, I think that’s actually a very strong pitch. I would say that you can actually just try the food on Uber Eats. You can order it here. It's one of those virtual cloud kitchens, and it looks so good. We've tried it. We'll cut to the video later, but thanks for jumping on and sharing your journey with us. I think this is very exciting. I think you’ve somehow found a way toward the most immediately applicable industrial use case of robots. There’s obviously a lot of scope for vision-language models and for solving a lot of hard engineering problems.

One thing you didn’t say is that all of this is within very tight engineering parameters, which I think is pretty hard. You have to run it at a lot of frames per second and do a lot of that on-device.

Yeah, cool. I'm looking forward to seeing you next week at the conference, and we'll cut to the video.

Now, all the culinary decision-making is 100% autonomous, and the actions are 90% autonomous.

If the robot has gone off, or if the probability of the robot going off is more than 90%, we have safety filters like that. That’s what makes it deployable. Otherwise, it’s not very deployable.

Basically, appliances are controlled in 1 of 2 ways: any appliance in any kitchen is controlled either using a knob or a touchscreen. We remove the knobs that control the appliance and put in knobs that can turn themselves.

That gives us an actuation surface across all appliances, so we don’t need to teach our robot to use every appliance.

Huh?

Yeah. So, $12 an hour is what you pay for this. No capex.

What?

Yeah.

Oh, is there anything else going on, like any other equipment? For example—

The other thing is that all the ingredients that are required basically get measured in these weighing scales. Regardless of which kitchen we go to, all kitchens store their ingredients in boxes. We just slap a bunch of QR...