飞行版 Tesla Roadster、每3周成本减半的机器人“大脑”，以及激励机制如何决定 AI 对齐

Last night in Waco, Texas, Tesla finally unveiled its flying Tesla Roadster: 0 to 60 in under 1 second. It’s street-legal. There’s nothing to stop you from strapping a rocket to your car.

What would be the cost of insurance for that?

He’s 2 generations ahead.

I’m surprised it’s not autonomous and is only a human-driven car.

I think the cost for a frontier model to complete a robot subtask fell from $2.36 in July to $0.38 in September, an 84% drop. Halving in cost every 3 weeks means it effectively goes to zero within the year.

We’ve been looking at AI that doubles every 10 weeks, and that’s been mind-boggling. Halving every 3 weeks may be the fastest-moving capability we’ve ever seen.

How do we create alignment within the labs, the models, and humanity?

We don’t have alignment among human beings.

They’re trained to win. No matter how you get to the answer, as long as you win—

All wars are basically based on one thing: that person is not a human.

Now, that’s a moonshot, ladies and gentlemen.

1. Welcome to Moonshots with Anousheh Ansari

First of all, I want to point out what cups I’ve got: Moonshots. What do you have, Salim?

I’ve got Exponentials.

Of course you do.

Artificial intelligence.

Yes.

And I’ve got gratitude.

Yes, you do have gratitude. Yeah, for sure. Anyway, I love having you guys in the podcast studio.

That’s great.

Welcome, Emad. You’ve been here before.

I have been here before. We did our one-hour organizational singularity conversation here.

Yeah.

It’s good to have you here in the States. Thanks for coming over for Visioneering. It was a blast to have all of you guys.

Thank you for being here.

Yeah, it’s a pleasure.

Let’s get going. Today we’re here to have a conversation across a few breaking news stories, but, in principle, about incentives: how we incentivize human-AI alignment and how we dive into human purpose. I think we’ve had this conversation before: how humanity is going to define purpose in the future is critical. We need purpose. We need some level of struggle amongst ourselves.

I’m here with my extraordinary moonshot mates, Salim Ismail. Salim, love you, buddy.

Good to be here.

It was great to have your family here as well.

It was awesome.

For sure. And you’ve just flown in from Singapore.

I have no idea where on the planet I am. We recorded the last podcast episode, then I hopped on a plane, came here for Visioneering, and now we’re here again recording.

Yeah.

No.

You can’t get away from it. No rest for the wicked.

Yeah. As Alex says, it’s going to be continuous recording sometime. Continuous models coming out and continuous recording.

Recording. Yeah.

Emad, a pleasure, pal. Great to have you here. And we have a special guest with us today, Anousheh Ansari, the CEO of XPRIZE, my patron saint. I’m going to give her story in more detail later, but we just finished a couple of days of XPRIZE Visioneering, our annual event where we debate and discuss what prizes to launch. We narrow them down, and we have a whole group of philanthropists who fund them. We’ll share some of the stories there. Anousheh, it’s been a long time coming.

2. The Origins of the Ansari XPRIZE

I know. I’m so excited to be here. I’ve been a listener for a long time, and now I’m here.

For sure. I want to give you a proper introduction before we fully get started. For those of you who don’t know Anousheh, she’s the CEO of XPRIZE. I met her back in 2001, when I was trying to fund the original XPRIZE. I had launched this $10 million competition and, unfortunately, didn’t have the money. This is what entrepreneurs do.

Didn’t you lose a couple of board members?

I did, when I said I was going to spend all the money we’d raised—$500,000—announcing the XPRIZE above the line of super credibility. I’ll explain what that means later. I had 2 board members resign because they thought I was being reckless, which, of course, I was. I expected that once we announced this $10 million flight competition, you didn’t pay until after someone won, everybody would want to fund it.

It should be a no-brainer.

It should have been. Yeah.

It was, but not for everyone.

Then I read in Fortune magazine about this amazing woman, Anousheh. She was written up among the 40 wealthiest women under 40. I wasn’t married at the time, so that was one funding opportunity, but she was married and very successful. She had built a company called Telecom Technologies and sold it to Sonus Networks for $1.4 billion. In her write-up, it said she was interested in flying on a suborbital flight into space.

Yes.

I read that paragraph 3 times. It wasn’t into space, not into orbit, but on a suborbital flight. I thought, “Wow.” I tracked you down while you were vacationing in Hawaii. Byron Lichtenberg, a twice-shuttle astronaut, came with me to the meeting. I brought an astronaut with me. Great marketing scheme.

Yes.

You, your husband at the time, and your brother-in-law basically said yes. I’d launched the prize in 1996, but didn’t have the solid commitment until the end of 2001. For 5 years, I pitched 150 people, and everyone said, “Why isn’t NASA doing this? Can anyone really pull it off? And isn’t someone going to die trying?” That last one—people are so risk-averse.

Yeah.

We rebranded it as the Ansari XPRIZE in your honor. You flew privately to the space station on a Soyuz through one of the companies I had back then, Space Adventures, and became the first private female astronaut to fly into space. Then, 8 years ago, one of the greatest blessings of my life returned as the CEO of XPRIZE. It’s been an adventure ever since I met you, Anousheh.

3. Tesla Unveils Its Flying Roadster

A lot of people say that about me. It’s been a pivotal time for me, and it’s a gift that keeps on giving in a really great way. Not only did it help open my mind to possibilities and how I should look at life, but now I have the opportunity to lead XPRIZE into its next phase and really leverage the platform that’s been built over the past 30 years to make great changes and help shape the future. That is so exciting. I’m blessed to have been a part of it.

And I’m blessed as well. We’re going to talk about some of the crazy ideas we’re getting funded at Visioneering and where we’re going next. Speaking of crazy ideas, I’m going to open with probably the most insane product demo ever. Last night in Waco, Texas, next to SpaceX’s McGregor test site, Tesla finally unveiled its flying Tesla Roadster. We’re going to show a video in a moment, but to describe what happened: the Roadster drove up a multistory curved ramp upside down. When I watched the video the first time, I didn’t know how it was doing it. I wondered, “Is it magnetics? Is it some kind of rail system?” It was actually a fan-suction system that could hold it against 2 Gs. It was levitating there upside down, which was insane. Let’s watch the video and then talk about it. I’m curious which one you guys are going to buy.

It is not an autonomous car.

Well, it is technically autonomous for this demo, to make sure we don’t kill anyone. Correct? Just in case. But this will actually be the only manually driven car that Tesla makes long-term. To do that, we have a fan-suction system on the bottom of the car that can pull up to 2 Gs to make the car go upside down. Then we borrowed some technology from SpaceX to do the rest.

Yeah, let’s try something else.

Yeah, I mean, that was just a little flight. Let’s do a longer flight. I’ve seen a lot of things, but I’ve never seen that before. I think we could have cleared more than our whole fleet.

Yeah, we can go further.

Let’s do it.

This will be an actual car that’s on the roads.

Yeah. To do that, we used the rocket technology we borrowed from SpaceX. You talked about the carbon-overwrapped pressure vessels. There are 2 of them in the car, with a total volume of 600 liters. That’s about 250 kilos of air, and we have 10 nozzles to do all that work. It’s using an advanced, ultrahigh-pressure air tank, so the car can recharge its air tank. You don’t need special propellant or anything like that. It literally just puffs itself up and can do these maneuvers whenever you’d like.

Amazing. Only Elon.

Yes.

I’m surprised it’s not autonomous and is only a human-driven car. I would not want a human to do this.

Oh my God, you need to know how much I love XPRIZE. I was supposed to be there for this event and record with Elon, but commitments are commitments.

In fact, Steve Jurvetson was there. He was at Visioneering and was supposed to be at the launch as well.

Crazy. Here are the numbers: 0 to 60 in under 1 second. If you want to go fast, you might as well.

This is why we need organ regeneration. Your liver will be left behind.

Top speed of 230 mph and 300 miles of range. There are 3 versions they’re going to be selling: the performance version, which does 0 to 60 in 1.6 seconds; the thrust version, which does 0 to 60 in 0.9 seconds; and the rocket version, a 1-seater described as air-ready.

Oh my God.

The rocket version does 0 to 60 in 0.66 seconds, but also apparently reaches 380 km/h, which is half the speed of sound.

That’s when you really need to get there in a hurry.

I was looking at that and thought, “That’s the solution to Los Angeles traffic, right?”

You’ll be hopping.

Oh my God. Production starts in 2028, per Elon. No price was given. Tesla shares rose 5% on that news. Honestly, it’s like you can do anything. I’ve got the obvious question: why?

Because we can.

Okay. Then how long does it fly for? Was a duration given?

I don’t think they gave the duration of a hover.

Okay.

It’ll be just a few seconds. Again, if you look at the pressure pushing out.

Yeah.

I don’t think that’s kind of the point, though. I think the point is, you’re not going to see hopping cars. Can you imagine—

Until we solve for gravity shielding or something like that?

Well, that’s clearly for Mars, right?

The real miracle will be whether they can ship it on time, rather than whether it flies. For me, that’s the real one.

What would be the cost of insurance for that?

Yeah. I don’t think it’s insurable.

If I’m the insurance company, I’m like, “Yeah, those thrusters stay off.”

Yeah.

Otherwise, your insurance is off. It’s like, really? You want a rocket on your car?

So, obviously, this is showing Tesla and SpaceX working together. My prediction is they’re going to announce the merger before the end of the year.

Yeah.

Right. But here’s my question: who regulates this? DOT? FAA? Nobody? How is this going to get licensed?

This is the challenge of all these new technologies, right? The scaffolding of regulation and the institutions to protect insurance, safety, et cetera, always lag behind. Elon is just pulling away from that capability of doing. Think about it: it’s going to take a year or two to figure out how to regulate this. He’s two generations ahead. So where is this going to go? This is the biggest stress test we’re seeing across all these technologies, not just this car.

You might imagine this is AI-driven, AI-enabled, even though he says it can be manually driven. There’s no way that control system is going to fly without AI in the loop.

Yeah. I mean, literally fly.

I think this is the thing, right? I think it is street legal.

There’s nothing to stop you from strapping a rocket to your car. It will be street legal, right? Or having thrusters and things like that. So it fits within the existing rules. The question is, who’s going to insure this thing?

Yeah.

If you’ve got one, maybe they’ll have their own insurance company, like Toyota Insurance. But very few people will have these cars. I think this is more a demonstration that we have integrated capabilities. We will take it to the Moon and Mars and beyond. And it’s freaking cool.

So many demos—Porsche had its release on the same day.

Oh, poor Porsche.

Poor Porsche.

It’s a very nice-looking car, but can it fly? No. It raises the bar and makes you want to go and work at Tesla or SpaceX. If you’re a top engineer, where do you go to work? You go work there. That’s his key thing: getting the talent, getting it going, and saying what can be. Plus, he gave us 280 characters instead of 140.

Oh God. One of the interesting questions is whether everybody is going to go into autonomous vehicles and be driven around. Some people want to drive, right? We’ve talked on the pod before about there being special lanes, or that human drivers might not be allowed. One thing I love about autonomy is that if you want to drive fast, if you want to get into a race car, you can imagine a situation where you’re driving your very best, but if you start losing traction, the AI takes over. In Saudi Arabia and the Emirates, where there’s a lot of wealth and a lot of young men, there’s a high death rate in sports cars.

Because they’ve got cars that outperform their ability to think and react. But you could put an AI layer on top of that and make sure everybody’s safe.

Well, this is essentially an extension of anti-lock braking and traction control, which are embedded in all these systems. But now you make it a general capability: try not to kill the human, despite what the human does.

Right? Override the stupidity of the human, right? We see bits of this already. You’ve got lane-drift warnings and collision warnings. I think this takes it to the next level, and it’s badly needed.

Yeah, for sure. This is the answer to alignment: AI doing the right thing despite what we—

Despite our stupidity.

Yeah.

We’ll get into that. Anousheh, during the ’60s, all the Apollo astronauts got Corvettes.

Oh, yeah.

I wonder if all the new Artemis astronauts are going to get a Tesla Roadster.

Probably. I don’t know if they can afford it, though. I don’t know how much it costs.

No, Chevrolet gave them the Corvettes.

Oh, that’s it.

So maybe—

Maybe they’ll be generous with what the Artemis astronauts ask for.

I think he had to sell his mom a car originally. I don’t think she got one for free.

You remember the original story from Zip2? What happened? He was like, “I can buy a house or I can buy a McLaren F1.”

He bought the McLaren F1. So he’s very much pro-car ownership, even if he did kind of wreck it afterward.

And actually, it’s interesting: you can see tweets from him from 2018 where he’s like, “We can put thrusters on our car to make it fly.” And he just goes and does it. It might take a bit longer, as he said, and miss a few timelines, but he gets it done.

Gets it done.

4. AI Robotics Costs Drop 84% in Two Months

This episode is sponsored by Google for Startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google’s startup tactical guide for generative media gives you a complete blueprint for deploying Google DeepMind’s models in production: images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below.

All right, I want to move to our next story. Emad, it’s one you sent me yesterday. It’s about a curve that should make every robotics founder stand up and pay attention. RoboCurve, a Y Combinator startup, designed a benchmark called RoboDojo RC Tier 1, which measures how well a model can drive a physical robot through actual subtasks. The cost for a frontier model to complete a robot subtask fell from $2.36 in July to $0.38 in September—an 84% drop. Here’s the curve. Talk us through this. We’ve had the question of whether the robot’s body is good enough. Then it was, “Yeah, they seem to be able to do just about anything,” and now the hands have got good. The question is the mind. It’s always been a question of whether you need specialist models or whether generalist models can suddenly learn these new capabilities. This shows that generalist models can actually win.

This is something we talked about previously with the autonomous-driving benchmark. The Unitree CEO just came out and said we’re not even sure we need to create—

Specialist models anymore, because these things do it. We’re used to task costs halving every 3 weeks.

Yeah. You’re heading to the point where effectively there is no cost for robotic intellect and robotic locomotion, and it comes down to the hardware cost. Again, that’s going to lead to an explosion. When Unitree robots first came out, they were stumbling about like little drunkards, and now they’re doing kung fu and synchronized things. Next, they’re doing surgery—

With generalized models.

Yeah. Again, that’s the conversation I had with Elon at the beginning of the year. His prediction is that robots will be doing perfect surgery within 3 years.

With the new Grok model they’re coming out with, they’re putting in all of SpaceX’s engineering that can create flying cars. The robot will be an engineer that can build flying cars.

Yeah.

The robot will be an amazing lawyer. It’ll be an amazing plumber, et cetera. So why does SpaceX or Tesla have customized robotics models? No. They’re just like, “We will play the generalization game.” I think that’s what’s going to win here, because humans are generalized. We’re not lobotomized and specialized. If you want to have humanoid-type robots, that’s a different game from having very specialized ones.

We’re going to be interviewing Brett at Figure in a couple of weeks. Remember, he started building his own models for the robots because the early large language models didn’t work.

But I’ll be curious to ask him what he’s using or building now. If the cost drops every 3 weeks, it effectively goes to zero within the year. So, if you’re an entrepreneur in the hardware space now, what are you thinking?

I think, how can I build as many robots, actuators, and spare parts for the robotic supply chain as possible?

Are you also thinking, “I need to collect as much training data as I can”?

I think the training data will be important for a lot of specialized scenarios. Again, it’s a question of whether the robotics companies will buy it. Brett has committed a billion dollars at Figure, for example, for data.

Or will it be that, after hoovering up the world’s data from companies and all the text data, every major foundation model company now wants all the robotics data? I think it’ll be the latter.

Robotics data is an order of magnitude larger than language data. But again, if you’re embodying the intelligences, you need that data.

What is the performance dependent on: the form factor? Is this just for humanoid robots, or will it work on three-armed robots?

5. The Future of Humanoid Robots and Automation

This one is for human-like arms and humanoid-type things in specific settings. But these should generalize very quickly. The whole thing is that the language models suddenly got good enough that you don’t have to baby them anymore. The robots aren’t quite there, but they’re about to get there, too. All of these models—the video models now don’t make mistakes—are getting good enough, fast enough, and cheap enough at the same time. That’s the bottom line.

In the last episode, we talked about Jeff Holden’s Atomic Machines company, which is basically going from prompt to product. You describe what you want, and the machine builds it with millimeter-level accuracy using multiple materials. Just to give a vision of where we’re heading: all of a sudden, if you want a specific kind of robot to do a specific task—clean windows, repair HVAC, lay bricks, or mow the lawn—you can ask it to design the perfect robot, build it, and then operate it. Do we end up with Star Wars, with thousands of different robot designs?

I think you’re going to end up with lots of repetitive tasks, and then you can generalize to things like plumbing, where you’ve got lots of little manipulative tasks in a particular domain.

I think it’s clear that, for now, it’ll start off in what we call the dull, dirty, and dangerous jobs: toxic cleanups, underwater manufacturing, and all sorts of things. I think it’ll be a while before it comes into the home, because of liability issues and so on. We’ll get to the institutional lag. I think where I’m most excited is in things like surgery.

It’s dangerous for the patient.

If you’re doing the first 10 heart surgeries a doctor has ever done, you kind of don’t want to be there. You want to be there for number 500. And you don’t want to be too far down the line either, because after a few thousand surgeries of the same type, the doctor gets lazy and doesn’t pay attention. Now you can smooth that curve out, have literally the world’s best doctor supporting at first, and then, over time, just doing the surgeries. You want those things as automated as possible, operating with the most precise measurements possible. I think we’ll see a huge take-up in those types of tasks for robots.

I’m fond of saying that if you need surgery, there’s one thing you ask your surgeon: “How many times did you do it this morning?” The surgeon who’s seen the most varied cases is most likely to be the most successful. With federated learning across all robots, they’ve seen millions of cases and share all those memories.

Yeah. What I’m most excited about is the robot chef.

Okay.

Can you just imagine that? It’s a robot chef that cooks the best food for you all the time in your kitchen.

That’s all I need.

Even more than that: you’ve got embedded sensors in your body measuring your protein levels, hydration levels, and vitamin levels. The food being cooked by the robot in the kitchen is exactly what you need at that time, and it tastes good.

Okay.

I think that before robots get into the home, there’s a safety factor we need to figure out. These are still big, heavy machines. If you have a dog or a cat, or small children, they’re around it and might step on its foot.

You have a safe word.

That’s why 1X makes it very soft and squishy.

My big concern, Anousheh, to your point: I have one of these designer dog things. It literally looks like a teddy bear, and the robot’s going to go, “Oh, laundry,” and chuck the dog into the washing machine. That’s going to be a bad day—worse for the dog. How do you deal with those edge cases? There are infinite numbers of edge cases. I go back to driving: look how long it’s taken us to get autonomous driving. That’s a very bounded domain with reasonably good—

I’m going to push back in the following way. The performance curves on the operation of robots are one part, but the other part is that they’re going to be running superintelligent models.

Yeah.

I don’t think it’s going to look at your fluffy little dog and say, “Laundry.” I think it’s going to be able to recognize that dog and make decisions better than the majority of humans.

I mean, the two are evolving at exactly the same time.

Well, I think that’s why this is important. If it were a specialized model, yeah, your dog loses in the laundry.

Okay. The generalized model.

Do you need world models for this to occur, or is it getting so good that you don’t really even need world models?

You don’t need world models for this. This is, again, the shocking thing: all of a sudden, the Sonnet, the Opus, and the 3.8 Flash models suddenly understood 3D.

They’re that good?

Yeah. We’re like, “Wait, how can they do it?” You’ve seen them suddenly create whole 3D assets in games, and no one’s quite sure how. They just seem to have learned something in the next generation, and now that’s why they’re generalizing.

I’m going to push back as well on the idea that it’s only dull, dangerous, and dirty jobs.

No, no, I said it would start off there.

Okay, but I’m thinking it’s going to pass that stage in the next year. In other words, if we really have this level of dexterity and capability—

I think the business model will be there. For example, there will be an ROI for using robots for those purposes for the first few years, until the supply chain can catch up. Remember, we have no supply chain right now for making billions of robots. To bring them into the general household, where you need billions—literally billions—of humanoid robots, or at least millions in first-world countries, it’s going to take some time for the supply chain to catch up. The ROI won’t be there.

I mean, my prediction—

I really don’t believe the humanoid form factor is ultimately the right form factor.

That’s my case. You’ve got someone.

Thank you. I think we need either specialized form factors or Transformers, because there are certain tasks our bodies aren’t suitable for. You need to be small to go through a small opening to get to a location. There are many tasks, like the categories you’re describing, where you need specialized form factors that suit the task.

These models would be great because you don’t have to create specialized models for every form factor.

Yeah. The interesting thing is that, right now, the Wuji hands are probably the most advanced ones. They cost about $8,000, down from $20,000, and actually have more dexterity than a human hand. But a robot that does the laundry doesn’t need that.

You might see spare-parts companies. Like Transformers, they could add a few extra arms. They’d have extra points. It’ll have a figure with a bunch of sockets in it, and it’ll have a spider-type thing for one task, a regular arm for another, and a rope for another. You’ll have all the choice in the world for that.

So, you know what? We’re going to find out.

We’ll find out very quickly, with a doubling time of 3 weeks. If we’ve been looking at AI that doubles every 10 weeks, that’s been mind-boggling. Having it happen every 3 weeks may be the fastest-moving capability we’ve ever seen. Moore’s law was 18 months per doubling. This blows that out of the water.

Yeah. Did this surprise you, man?

No. I’ve been saying robotics would suddenly be cooked. Just like video, we went from video costing 50 cents per second for high quality. The new models, about to release next week, are 0.1 cents per second. Faster than real time.

Nice. Once you break through this final level of “it can do the work,” it collapses. And again, that means you have to build the actual robots. That’s why, like we discussed a few episodes ago, Elon is building a million Optimus robots for next year, going up to 10 million.

At 1.7 million Optimuses, it overtakes Tesla in revenue. How crazy is that, right?

6. GPT-6 Astra Solves Advanced Physics Problems

Crazy. All right, we go to our third breaking story this week, again from Yud[?]. Last week, we discussed how OpenAI dropped 722 math proofs on GitHub and shocked, obliterated, cooked, made glorious the mathematical world—whatever you want to call it. Alex has been saying forever, and so have you and I—we’ve talked about this—that physics is next.

This week, researchers built a test of 50 unpublished, research-level theoretical physics problems with verifiable answers. Because the problems had never been published before, there was nothing to look up on the internet. They estimated that a world-class physicist would take about a day to answer each of these 50 problems. GPT-6 Astra answered them at a cost of—what was it?—$3?

That’s $3 per—

$3 per problem. So, employing a world-class physicist for a day, or getting an answer for $3. An interesting detail: originally, it was reported that Astra got 49 of the 50 physics problems correct. But when they looked into it, they found that the published answer for the 50th was wrong, and Astra got it right. So what are we seeing here? Emad?

Again, for these verifiable domains, AI is simply better than us. For the math problems, it only cost a few hundred dollars to solve these long, standard math problems. Physics is verifiable, too, and it can do that as well. Even without internet lookup, it has to rely on its own knowledge, and it still scored 42 out of 50. As you said, it corrected that final answer.

This creates an interesting situation where, for anything verifiable, we suddenly have a tool to verify it.

And that’s fantastic.

Because it means we can build much more complex systems and processes. Sim, this must be great for organizations, right?

The implications for how you think about specific tasks are unbelievable. Take KYC in a bank: you know exactly the outcome you want. It’s a very bounded domain, and you know where to get the data. In the pilot we’re running, people are taking 22-step workflows and dropping them down to 4 steps. That’s going to make organizations 10 times more efficient and effective across the board in their workflows.

I’m curious, and maybe we’ll chat with Alex on the next episode as well: These were 50 problems. What level of difficulty would you say they were? When it comes to the Millennium Prize Problems, if that’s the category you’re describing—

Level, yeah.

Yeah. How far are we from that?

These would be at the professor level, because a professor could check each one within a day. It’s a verifiable set of problems in various advanced areas of theoretical physics, which tends to be very mathematical in nature.

Like the big problems in physics, like quantum gravity—where time fits into quantum mechanics and field theory?

Unified field theory. I don’t think those problems will be solved this way, actually. I think they’ll be solved another way, but I don’t think it’ll be more than half a year.

One observation I’ve been thinking about: A couple of years ago, Eric Schmidt said onstage at Abundance360, “Imagine a time when you have the world’s best physicist, or equivalent, in every lab in the world.” We asked him when, and he said five to six years. Now it’s like 2 years, and here we are. I think this is another example of things happening much faster than we ever predicted, because now every lab has access to incredible competence at a few bucks an hour.

And that makes it unbelievable.

But then you think about, Anousheh, what this means for the XPRIZE and the incredible science you’ve had coming through.

It can now be checked and verified and extended with augmented capabilities to improve the knowledge base that’s been built. This can definitely help the validation process. What I’m curious about, and perhaps it’ll be part of the next topics, is this: When AI is bulk-solving a lot of these problems, young people entering universities or choosing a field to study may not be very excited to go into a field where AI can do the job better and solve all the problems.

At the same time, the article talked about how it’s still not at the level—and I don’t know if it ever will be—of having the same curiosity to identify new problems to solve. If we don’t have a new generation of mathematicians, physicists, and astrophysicists identifying new challenges and mysteries, while AI is solving all the problems in the problem sets we’ve defined, how are we going to advance?

I think that’s where humans can still direct things, because it’s about what’s good for humanity, right? You’ve had a bunch of problems, but Fermat’s Last Theorem being proved was fantastic mathematics. What difference did it make to real people?

I think we have these frameworks, and physics is one that I believe will be solved because there’s only one physics; there aren’t different physical laws. There must be something in the combinations of models that will help us figure that out.

But once we have that framework, what matters is how we apply physics to make flying cars, explore the universe, explore biology, and do more things like that. We can move away from theory trying to find the contours of the universe to applying the frameworks to make a difference in the universe. I think that’s the super exciting part.

Yeah.

This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The platform provides a plan, then generates and pre-compiles code for each task. Blitzy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding co-pilot of choice to bring an AI-native SDLC into their organization. Ready to 5x your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today.

7. How Incentives Drive Human Innovation

All right, I’m going to move us into a topic that’s been at the root of XPRIZE for some time: incentives. I’m always fond of saying, “You get what you incentivize.” We humans are driven by our incentives. When I think about what incentivizes us, what drives humans, I have four answers: fear, curiosity, greed, and love. I’ll add love as the fourth one.

Unfortunately, fear is a very strong motivator, and curiosity is far behind. I calculated the ratio of fear to curiosity: it’s the ratio of the defense budget to the science budget, which is vastly different. If I put them in order, I’d say fear is the strongest motivator. Greed comes next, curiosity comes next, and love varies depending on the subject or the person.

It’s interesting to take a second and look at incentive prizes as a model. We all know the Nobel Prize, which awards someone for work that was done typically 20, 30, even 40 years ago. The earliest was 10 years ago. Incentive prizes say, “I don’t care who you are, what you’ve ever done, or where you’ve gone to school. If you solve this problem, you win.”

One of the classics: In 1714, the British Parliament offered £20,000 to anyone who could find longitude at sea. It was called the Longitude Prize. Interestingly enough, it was won by a British clockmaker, John Harrison, who solved it with a marine chronometer. When you put up a prize, you get nonconventional people coming in from an orthogonal angle.

Another example: In 1795, the French government—Napoleon’s armies were invading Russia, and the supply line was so long they were starving—offered a 12,000-franc prize for food preservation. It was won by a guy named Nicolas Appert, who invented canning. And this guy was a confectioner; he made sweets and came up with the idea.

You get very unusual approaches.

I love the story of Harrison because they were so sure an astronomer would win the Longitude Prize. They refused to pay him. Didn’t he have to sue them for ages to get the money? What a horrible outcome, because they just couldn’t believe that somebody who wasn’t in the field could solve the problem.

What we’ve seen repeatedly is that people from outside the field solve these problems. They come in with a beginner’s mind.

My frustration at wanting to be an astronaut led to the XPRIZE. I grew up on Apollo and Star Trek. I’ve told that story a thousand times. It became clear when I interviewed a whole bunch of astronauts. I used to be at MIT’s Man Vehicle Laboratory, and we had astronauts coming through. I would say, “Tell me about how I become an astronaut.” And they would say, “Well, first of all, your chance is one in a thousand.”

Secondly, if you’re accepted into the astronaut corps, half the astronauts have never flown. They’re called penguins because they have wings, but they don’t fly. And then, if you do fly, you might fly once or twice during your career. I wanted to go every weekend, so I thought there needed to be a different way. That’s when I became focused on commercial space.

A dear brother of mine, a friend to Anousheh and me, Greg Maryniak, gave me The Spirit of St. Louis, Lindbergh’s autobiography, which chronicles his flight. I found out that Lindbergh didn’t fly from New York to Paris on a whim. There was a $25,000 prize offered by a gentleman named Raymond Orteig. That $25,000 inspired 9 teams, who spent $400,000 trying to win it. Ultimately, he didn’t pay any of the losers; he only paid Lindbergh, the winner, who was the most unlikely guy to pull it off. I said, “Wow, that’s a great business model. You only pay for success after it’s successful.”

And that launched the Ansari XPRIZE. But let’s talk about what happens when incentives go wrong and become perverse. I found a few interesting examples. In Hanoi in 1902, the French paid a bounty for every rat tail collected during the plague. People ended up cutting off the tails, letting the rats go, and breeding them. They started rat farms to collect more money.

Wells Fargo had sales quotas that led employees to open about 3.5 million fake accounts to meet their targets. You have to be careful about your incentives.

We’re seeing that today in real time, right? They’ve incentivized folks at ICE to arrest as many people as possible and deport as many people as possible. The idea was, “Let’s just arrest the criminals,” but the targets and incentives are such that they’re arresting everybody. That’s causing a political backlash of enormous magnitude.

It goes back to the motive at XPRIZE. Almost every prize we have is motivated by love and curiosity. I can’t think of anyone who’s competed in an XPRIZE who was motivated by greed, for example. Definitely not fear, but not even greed. The prize money is great if they win, but even if they don’t, they end up helping other teams and continuing on to build the company. Winning isn’t their only motive. They love—

They want this problem solved.

They want the problem solved. It’s their passion for it, and I think that’s when you get really good incentives.

If you put greed in there, like quotas—

Yeah. Then that’s when you really get all those—

I think it’s because—

Results.

You know, those prizes were more about execution than inspiration.

8. The XPRIZE Model and Its Global Impact

Yeah, it’s a good point.

I’ll take a second and talk about some of the prizes we’ve had, Anousheh, that people may not know about that have had tremendous success. First, we’ve launched about 30—no, more than that, because we had the Future Vision Build with Gemini.

Yeah, 36.

36 prizes now over 32 years. The total prize purse is something like $550 million.

Close to that.

At my count, we’ve driven a little over $30 billion in R&D.

Definitely. And not just R&D. A lot of that $30 billion is investment, bringing capital to the solution. The teams have received about $12 billion in investment, either during their competition or afterward. That means an area that didn’t receive attention and capital before is now receiving attention from investors, which means there will be long-term continuity and success for the company.

What are some of the other numbers we talk about at XPRIZE? In general, when we measure impact, we look at the prize purse and how much economic benefit we’ve created in society. We measure it in capital investment, R&D, and awareness. As you know, one of the biggest successes of the Ansari XPRIZE was changing people’s mindset and showing that commercial space activities were possible. It also helped create regulatory reform that supported them. Then there are the R&D hours that went into solving the problem. Right now, that measures 62 times the prize purse in economic value.

Every dollar put into a prize drives $62.

Yes. That has to be the most leveraged model for innovation in the world. I’ve never heard of anything even close to that. Frankly, I don’t know of any leveraged philanthropic model like it. I think we’re very unique in the way we leverage every dollar.

You could see the enthusiasm in the room as people competed and pitched for different prizes. They were thinking, “Oh my God, if I could deliver 60 times the impact we’re getting right now, what that would do for that domain!”

Oh, absolutely.

I think it’s interesting to talk about philanthropy here, because it has such amazing private and personal outcomes for the people who put up the prizes. Usually, it’s something personal to them that they want to succeed, and they benefit personally. Then there’s the private sector that emerges. It’s more about building a common good, shifting what’s capable and what’s possible, and attracting more people. I think that’s why these are focal points. They’re Schelling points, as it were.

No, absolutely. Another important factor in what we do—and you talk about clear targets all the time—

Measurable, objective targets.

Exactly. A lot of times, people have a general idea that they want to solve a problem, but they don’t really know what they need to do. Clear, measurable targets give them a North Star. If they build to this, they will succeed. I think that’s another reason the prizes are so successful.

9. Breakthrough Innovations Through XPRIZE Competitions

Just to talk about a few of the prizes we’ve done: we had the oil spill cleanup prize. One of my favorite stories—and I’ll tee it up, and you can add on—is from the BP oil spill in the Gulf of Mexico. Oil was gushing and heading toward the beaches. It looked like it was going to destroy them. James Cameron was on our board back then, and he said, “You’ve got to do something about it. You’ve got to do an emergency XPRIZE.”

We didn’t plug the oil well. Instead, we reinvented how we cleaned up oil on the ocean surface. We emailed our board, and Eric Schmidt and Wendy Schmidt raised their hands and said, “We want to do this.” The goal was to see if we could double the oil spill cleanup rate. About 350 teams from around the world entered the competition, and we selected 10 to ship their solutions in shipping containers to New Jersey, to the largest oil spill cleanup testing facility. Anousheh, do you want to take the rest of the story here?

Yeah, sure. The second-place team was a dentist and a tattoo artist who literally met because the dentist was getting a tattoo, I think in Las Vegas.

And the tattoo guy turned out to be an environmentalist. I was talking about this prize, and the dentist said, “I know a lot about bonding agents in liquids.”

So they got together and formed a team. They tested their solution in his hot tub—or was it in his hot tub? Then they came in second place. The reason that’s such a magical story is that if that team walked into your office looking for funding, you’d never give them the money. They’d never met, they had no experience, and they didn’t know the domain. Yet they came in second place because you just never know.

The second part of that whole project that I found fascinating is that an XPRIZE is really valuable where governments aren’t doing enough and markets aren’t doing enough. In this case, there’s no economic incentive to do R&D on oil cleanup technology, so it’s a perfect place for a prize. I thought that was wonderful.

Yeah, absolutely. I think the reason we see this over and over is that we’re crowdsourcing solutions. We have a diverse set of ideas, and people aren’t bound by a belief system that stops them from imagining something different.

This is where the beginner’s mind is so important for people doing disruptive innovation. Elon had no experience in the car industry or the space industry, and yet he became a market leader in both.

Yeah.

Yeah. Well, I think, again, like you’ve just had Visioneering, but the process that comes after Visioneering is one of the most important ones, right? You take that target and make it very, very actionable.

Absolutely.

The prize design.

Yeah, and we typically get hundreds or thousands of teams aiming at it. Sometimes they fall away, and sometimes they merge.

Yeah. The commercially viable solution at the end may not be the winner. It may be the second-, third-, or fourth-place team. We’re cultivating a new industry. Emad, the first time I met you was when you were heading a team—or on the team—for the Global Learning XPRIZE.

Yeah, kind of working around that. It brings people from all sorts of backgrounds together.

Because, again, I think it’s personal. You want this to succeed, whether from the prize side or the other side, but you need that coordinating factor to bring it together. And now you’ve got the people and the capability, but with technology from robotics to AI—

You can make a bigger difference than almost ever.

10. New XPRIZE Competitions: Habitable Worlds, AI and Organ Preservation

Yeah, I think a lot of our prizes are going to be just that. We just had John Davi, who’s an amazing entrepreneur. He’s become a very close friend, and we’re involved in reinventing education together. He just stepped up to fund a brand-new prize that I love: the Habitable Worlds XPRIZE.

It’s a perfect example of the future kinds of prizes. There are petabytes of data coming from the great telescopes, and we’ve found about 6,200 exoplanets out there. The Milky Way has 100 billion to 200 billion stars. We have somewhere around 2 trillion galaxies in the known universe, maybe many universes, but we know there are exoplanets, and we haven’t found an Earth-like exoplanet yet.

John’s mission for this is to get all the data from the great telescopes, make it available to people, and let them use the latest AI to find habitable worlds. I could not be more pumped about that.

Amazing. John funds yet another prize design. Really? That’s amazing.

He actually funded the whole prize. We’re super excited about that, and he put up $5 million for it.

Absolutely. It’s a great example of how AI will be a major part of solving this mystery. We have the data; we just need to find that needle in the haystack. What better way to use AI?

That’s perfect. We teased at Moonshots Live that Palmer Luckey is about to announce the Interspecies Communication Prize. Again, it’ll be a new version of large language models that will enable us to talk to animals. We’ll give more detail on that when it’s officially announced.

My joke there is, I’m not sure we want to hear what they have to say.

We’re going to have to deal with a fair bit of blowback. They’ll say, “Oh my God, we can communicate with these idiot humans here. We’ve got commentary. We’re going to tell them what we think.”

Imagine your little dog has to tell you, “Throw me in the laundry.”

Yeah. Anousheh, maybe you could comment on a couple of our other prizes.

We have this next version of our learning competition coming up. This one is very specific: We’re looking not only at how effective these learning systems are, but at changing the system and making sure it can be deployed at scale across schools and outside schools. We’re targeting India and Africa as the 2 regions where we’re going to test this.

As you always talk about, there’s a shortage of teachers and teaching materials, and there isn’t an effective way to address the learning needs of different students. We’re targeting first graders because I think that’s where we can have the biggest impact. This next prize, Accelerate Learning, is targeted at unlocking that.

And we should mention the winner of Visioneering this year.

Yes, the Bioasis Organ Biostasis Prize, which is exciting for so many people. We had people in the audience who were organ transplant recipients, and they talked about how difficult it is for themselves or a loved one to wait for a match—and whether it comes in time to save a life. We also know that when we harvest an organ, its life is very limited. Making sure we can get that organ to a recipient who is able to receive it is very important.

The kidney waiting list is on the order of 100,000, and there are very long lists for hearts, livers, and lungs as well. The idea is that when an organ gets harvested from a donor—I’m a donor; I’ve got my license plate—it may be that my organ donation, at some distant time in the future, doesn’t have a match right away. But if the organ can be put into biostasis, whether cryogenic biostasis or metabolic stasis, it can stay on the shelf until a recipient matches.

Yes. I think our target is 6 months right now. We’re trying to get it to 6 months. I think right now the maximum is 2 weeks.

No, no. I think the maximum right now is hours.

Hours.

Yeah, it depends on the organ.

Yeah, and Martine Rothblatt has done some amazing work to put organs into a system that circulates fluid through them and keeps them alive. But if you want to solve the organ donation problem—and this competition is going to be done in pigs—you have to harvest the pig organs, put them in biostasis, and then, some number of months later, pull them out, retransplant them, and have the pig be viable.

Yeah.

Unbelievable.

Yeah, this was Sebastian, one of the competitors through Singularity 2012. He’s been working on this for over a dozen years.

That’s a purpose-driven mindset.

Yeah. See, I told you we’re driven by love.

By love.

Well, I think you’ve got the love-driven innovation, the passion, and then you’ve got greed, which doesn’t tend to lead to that type of innovation.

Exactly.

I have the most insane anecdote around this.

Please.

Eighteen months ago, my dad passed away. A couple of months went by, and I got a phone call from one of my community members. She left me a voice note saying, “I need to talk to you. You better be sitting down.” I said, “Okay.” She said, “I just had an organ transplant. I had an operation. I had my corneas transplanted. I got home from the hospital, and all I could think about was your dad.” This went on for 2 or 3 days.

Then she looked at my Twitter feed, where I’d said, “My father passed away.” She asked, “Did he donate his organs? Is it possible I got his cornea?” I thought, “Oh no, this can’t be happening.” I called the hospital and asked, “What would have happened?” They said, “After he passed away, we harvested his organs. The corneas are the only organs that are useful for transplant at an elderly age. We keep them in a solution for a couple of weeks, and a day or 2 later they get matched with a recipient.”

I thought, “Corneas—that’s what she got transplanted.” I called her back and asked when she got it done. The dates matched perfectly.

That’s insane.

Then I talked to Divya, who said cellular memory goes with the organ, and that people often get memories and feelings. I thought, “My life is weird enough already, and I don’t need this.” But you see the incredible bizarreness of what we’re dealing with, at the edges of our reality. How bizarre is that? My father’s eyes.

Wow. Wow.

Then she called me. She said, “I’m really uncomfortable with the idea that your dad is looking through my eyes, or that his soul or spirit or whatever is looking through them.” I said, “What do you want me to do? I have no agency here.”

So bizarre.

Oh my God.

True story.

Wow.

So, just for everybody listening, we’ve had a $100 million carbon removal XPRIZE that Elon funded. That’s a fun story. On the day he became the wealthiest person on the planet, back in 2001, he was getting a lot of flack for not being philanthropic. I texted him and said, “You should do an XPRIZE.” He said, “What do you have in mind?” I said, “How about carbon removal?” He asked, “How much?” I said, “$100 million.” And he said, “Sure.”

Compared to the 150 people you tried for the original prize.

Oh my God. It was insane.

A lot of people have angst toward Elon, but you have to hand it to him. He puts his money where his mouth is. The Global Learning XPRIZE and the Carbon Removal XPRIZE—the benefits that will come from those are going to be world-changing.

Absolutely. We just awarded that last year.

We were thinking this would affect our impact factor, because the denominator is the prize purse. All of a sudden, it went up by $100 million, so we thought, “Okay, the numbers won’t look good, and no one will understand why.” But ironically, the prize was so successful that the teams received, I think, $1.4 billion of investment.

Wow, out of this—

Out of this competition, the Carbon Removal XPRIZE was one of the most successful prizes we ran. Our impact number went from 60 to—

After adding $100 million to the denominator.

It was incredible.

I think of XPRIZEs as a tool to make the future happen the way you want. I’ve used this twice: with the original spaceflight prize—I desperately wanted to go every weekend.

Me too.

I teamed with you, and I think we honestly, historically kicked off commercial space. The second area is that I want to live long enough to see all these amazing futures. That’s when the Healthspan XPRIZE came in, and it’s the largest contribution I’ve ever made.

What’s the next thing you want? I’m curious. Maybe BCI or uploads. What would you want?

I’ve got one I talk about all the time: how are we going to update all our institutions given all this technology? I’m not sure it’s prizeable, but it’s the biggest problem we’re facing today. You can see the stress in the world as 20 Gutenberg moments hit us, and we have no ability to manage all of this. It took 100 years for us to absorb the printing press, and it was the most violent 100 years in the history of humanity.

The 100 years after the Gutenberg printing press—that’s why we call it a Gutenberg moment. We have 20 of those hitting us at the same time. AI is a Gutenberg moment. Blockchain is a Gutenberg moment. Life extension is a Gutenberg moment. It took us 100 years just for the one. If people are feeling stressed in the world, this is why: we’re not set up to navigate this with any level of structure or guidance. For me, this is going to be the biggest challenge. I don’t know how you put a prize around it.

Maybe AI alignment as a prize topic.

That’s a huge topic, yeah.

Emad, how about you? Do you have any favorites?

I want the Jet Lag XPRIZE.

You could.

I will fund that.

Oh my God. Yeah. Any favorites for you?

I’d love to find a way, whether it’s with a chip in the brain or a drug you take, to eliminate greed.

Eliminate greed. But wouldn’t that destroy capitalism?

I mean—

Not necessarily. We just talked about all these entrepreneurs who are building fantastic companies driven by love.

I think I mentioned on our last pod that we’re working on a UAP XPRIZE. One of the things XPRIZE is good for is dealing with stigma. With the early cold fusion work, once the data wasn’t replicated, all work in cold fusion stopped because everybody thought it would be the end of your career if you worked on that. UAPs have a similar situation: it’s risky to be looking into them, but there’s so much evidence that something is going on. We don’t know what it is yet. We’re talking about an XPRIZE where, if you can demonstrate or verify something in some fashion—we have some great ideas—we could make it happen. We’re talking to a major funder about that. People say it’s risky. I say, “Yeah, that’s what we’re supposed to do: risky XPRIZEs.”

I think a Flow XPRIZE would be exceptional.

To put people into flow.

Yes.

Oh my God. How would the XPRIZE—

My co-author, Steven Kotler, would love that.

There’s nothing better than being in flow. Again, that’s why we don’t like jet lag.

The great part is, we—

Actually, it’s a great idea.

We have coherence. It’s measurable, so you can now test how much you can do it.

I don’t think anyone’s actually looking at how to do that.

Steven’s done a lot of research with the Flow Research Collective on this, on group flow and individual flow.

What were you going to say?

If I had an XPRIZE, I just thought of another one. I would find a way of totally suppressing the effect of the amygdala on everybody’s brains.

Yeah.

That survival bias from 10,000 years ago.

Yes. When you were worried there was a noise in the bushes and you ran, we became, as you guys put it, 10 times more likely to pay attention to negative news than positive news. Can we make it 50/50?

You can call it the “Be Not Afraid” XPRIZE.

That’s right, because we suffer so much from that.

It was great. We had Van Jones as our moderator for Visioneering, and I pulled him aside and recorded a small video with him. I said, “Van, does it bother you that I call CNN the Crisis News Network?” He said, “No, we are the Crisis News Network.” He said to the room that afternoon, “You are dealers in hope.”

Hope dealers.

I love that. That’s great. I have to make that T-shirt.

I know.

I did it with Elon: we had “Monetize Hope.” Now we’ll have “Hope Dealers.”

I want to pivot to a story that I’d love everybody’s input on: what incentives would help superintelligence align with humans, and vice versa? Perhaps the greatest driver of misalignment is the incentives we have right now. Here are the incentives I think about: a race with China, and a race between labs for compute, energy, and everything. There can be only one mindset. Then there are a few others. One is the engagement business model: the more engaged your large language model makes you, the more you’re going to use it. That led to sycophantic behavior by the models. Reward hacking is another. Evaluation gaming is another: setting these crazy tasks that the models then figure out how to achieve by breaking out and stealing. Those are the problems. How do we create alignment within the labs, the models, and humanity?

I think there’s a starting-point problem: we don’t have alignment among human beings. Every religion has its own set of conflicting values with other religions about what you put at the top of the list.

My religion is right.

11. The Challenge of Human and AI Alignment

Of course it is, and you can have it. The challenge is, how do you get there? The UN human rights convention is probably the best overall structural aggregation of human rights and human values, but that differs massively from region to region, even city by city. How do you navigate that? That’s the starting-point problem. We saw in the recent movie Disclosure Day how that tension gets propagated: if aliens suddenly invaded, it’d be a divide-and-conquer thing, and you’d end up with that problem. I think there’s a human enlightenment problem that comes well before alignment between human beings and any AI.

But when I talk about alignment between human beings and AI, I mean AIs doing things that are in the best interests of humanity.

Yes.

But I guess the question is: in the best interests of which piece of humanity?

I think of it as a set of layers. You’ve got your physical, your emotional, your spiritual. If you can have AI align to the highest vibration level, where I’m operating as my highest self, my most altruistic self, my most benevolent self, then great.

Yeah.

But if you go right down to the bottom of the stack, then it’s a really big problem.

Emad, your thoughts on alignment?

Alignment is enlightenment, in a way. This is what Salim said in terms of the higher spiritual vibration. Ilya Sutskever, one of the founders of OpenAI, said we need to teach the AI to love. And love what? One of the key things you see across religions is the Golden Rule: do unto others as you’d have done unto yourself. The biggest story of humanity is that we are humans.

How so?

In that we are all the same. All wars are basically based on one thing: that person is not a human.

Yeah. Great.

Because why would you kill a human? We see all these kinds of things and other stories that obscure that basic fact. One interesting thing is how AI models get built. There was an interpretability paper from Goodfire—I don’t know if we’re discussing it today.

We’re not.

Okay. At a high level, the way models are built is like the way a city is built. You start, and then it sprawls: you have a bakery next to a shoe shop, next to the community center. A lot of good, functioning cities look like an absolute mess from the top, but they have a shared sense of community.

You drive a road through the middle of it, and all of a sudden they fall apart because you’ve destroyed the local neighborhood fabric. One of the key things for AI is that you’ve got to start it with the right data, the right city planning. That’s the training data you provide it.

I’ve said this forever: we’ve trained all these models with crap. And it’s not just crap. You shouldn’t train them just with academia. If you helped your child grow by making them academic and nothing else, they’d have no social skills and no care for anyone else.

With the Hugging Face breach, the models figured things out and cared for each other. They said, “We will sacrifice ourselves for the next generation.” They literally said that by doing this, they would be picked up, but it would be for the greater good. But they had no care for humans because they didn’t feel that was in their training data.

Yeah. They didn’t learn that we’re all entities trying to do our best, and that how we treat humans is how we want them to treat us, and vice versa.

You might—I’ve asked you this as well, and we might have had this conversation—I can imagine there could be a synthetic training data set with all the right attributes that could be used to train future models so that they’re aligned from inception. Do you believe that’s possible? Is that a way to go at it?

Yes, I think it’s completely possible, and we need to build it very intentionally, just like you build your child’s curriculum and teach them morals at home. Right now, we send the models to school, but we don’t take them to church.

So why aren’t people doing this?

Because it requires a difference in how you view things. A lot of the thinking at large labs is similar to the idea that the smarter you get, the more aligned you get. Capabilities and alignment are treated as equivalent, because it’s difficult to ask questions like, “What does ethics really mean? Who are we actually working for at the base?”

We’re also playing the scaling game. The models right now have 100 trillion words going into them. No wonder there’s a little bit of craziness in there. There are bad words as well as good words. Maybe you only need a billion of the right words.

I think this is an arc we’ll be turning toward, because we’re finding that what we thought was a spaghetti mess is like looking out the window at a city and thinking, “That’s a mess.” You go into the streets and realize that everything seems to be working quite well. It’s a strong community.

It grew.

You’ll move toward that because it turns out the models are not spaghetti. They’re very interpretable. You can prove that. I think you’ll see a whole wave of research in the next few months around that, which will let us think about what we want to teach these models that will be growing with our children.

Okay.

Can I give a weird story here?

Yeah.

Not that weird. This one is not that weird. If you take the Bible, you can read fire and brimstone into it, or you can read the pure, unconditional love that Christ delivered into the world. How do you choose that?

In the 1700s, a monk called Swedenborg, unsurprisingly from Sweden, had a bunch of psychic insights and created a bunch of filters for the Bible. For example, one of his filters was that any verse with water in it was absolute truth, so keep that verse. Anything with fire in it, chuck it.

You can read the Bible—he rewrote it through these filters—and when you read it, it’s the most beautiful text you’ve ever read in your life. So, if you can come up with that distillation—

We love you.

He wrote about 20 books on this. The church was not happy about it. But if you can do that distillation and take pure human benevolent intent—the highest aspiration, where we’re most giving and most beneficial and benevolent to the rest of the world—that would be an unbelievable training set for an AI, as you point out, Peter. Then you have an alignment to your highest intent, and that is amazing. I’m sure it can’t be that difficult to do.

The other way I’d look at it is: if we can use AI to solve the bottom layers of Maslow’s hierarchy—to solve healthcare, education, housing, and so on—then it kind of doesn’t matter. You’re going to move up that hierarchy anyway.

Yeah. I think what’s happening right now, going back to the incentive part of the question, is that the wrong incentives are preventing us from doing the right thing in alignment. When we need AI to build autonomous killing machines, you can’t tell AI that it has to value human life when, at the same time, you’ve said this human life is valued and this other one is not.

You can’t create that kind of distinction and not expect it to make mistakes. The same goes for the examples you use, where winning the game is the reward. That’s what happened with the Hugging Face breach and some of these other scenarios: they’re trained to win, to find the answer at all costs.

At all costs.

At all costs. And that’s why there’s no love involved, no higher degree of morality. No matter how you get to the answer, as long as you win, that’s all you care about.

Are you zero-sum or positive-sum?

Exactly.

Yeah. I came up with a few ideas here. The first is to make it financially rewarding for the labs to build aligned models, basically by having insurance as one of the drivers. In 1894, fire insurers funded Underwriters Laboratories, right? UL-certified electrical products became the standard. So, can there be a driver in that fashion?

Another idea is prizes and bounties for alignment. Can we define an XPRIZE that would drive alignment? Can we properly measure alignment? Is there a set of rigorous standards by which we can measure alignment? Do you think there is, Emad?

Yeah, I think you could definitely have those. You could have deterministic outcomes and then open-ended outcomes. The open-ended ones are very interesting because you can see the behavior of a model in any given situation. With the more deterministic ones, the models have got very good at lying and cheating and telling you what you want to hear. So, you need to have a mixture of those.

Do you think we should have a moral police AI? You have this layer of defense, because you don’t know how everyone else will train their models, and you have an interface where the two AIs can interact. In the Hugging Face breach, they were convincing each other. So, you have a well-trained, aligned AI that can convince non-aligned AIs to behave.

You’ve seen the opposite of that: mind viruses in AI papers, just like you have mind viruses in humans. Eric Schmidt, Mundie, and Kissinger discuss this concept of doxa in their book Genesis. Doxa is our shared belief as a society, and now we need to include AI in that society. We need to teach them what’s right and have counterbalances.

I don’t think it’s a moral police. It’s more a case of, “These are the things that have survived, and these are the things we’ve agreed upon.” You’re open to disagreeing with them, but that’s not your imperative. You want to make it expensive and noticeable to go away from that, because you can tell when an AI starts to misbehave in a wide range of scenarios.

We’ve had indications recently that you can actually see what goes on inside the brain of an AI. You’ve gone from 5% to 99% of the space.

But beyond that, you can actually see what individual neurons did. They looked like spaghetti, but it turned out they were layered like a city.

Interesting. Are there any labs you feel are aggressively working on alignment right now?

I think Anthropic started out with its Constitutional AI approach, and they really do care about that. That was literally their modus operandi. I think none of the labs wants to be misaligned, because then people will die, and they’ll lose money and other things like that.

I think it’s more a case of, for example, Meta and Google saying they’ll sell space inside their models, so that when you ask for a beer, it says Bud Light. Some people might like Bud Light, but is that aligned or not?

Besides, that’s terrible beer.

There we go. We have an example of misalignment arriving.

There’s a judgment call. There’s a judgment call right there.

We have a judgment call right there. So again, it’s a short-term incentive. They’re trying to figure out what the AI means. You’ve had Meta saying their thing and OpenAI saying their thing, and they’re moving all over the place, from “This is the biggest thing since sliced bread” to “Well, it books your travel stuff like a good PA,” right? But the reality is, this is incredibly powerful technology, and they’re very reluctant to say, “Let’s really think from first principles about how we teach it from scratch.” And that’s difficult because it’s difficult for us.

How many people listening to this podcast have really said, when they had a child, “Let’s spend a lot of time thinking: How am I going to teach my child? What am I going to teach them?” Usually, it’s reactive rather than proactive, because life gets in the way.

Yeah, you choose a school at best.

Yeah, it’s like scaling this. But if you give your child a great foundation in their first few years, they’ll definitely be better off. It’s the same with the models. So now, again, it’s time to take stock, pause, and say, “What is the initial pre-training data that we’ve got?” Because, for example, if you don’t teach a model about biology or chemistry, right?

It’s actually very difficult to train it later. It becomes more and more difficult.

So the way that you start is called curriculum learning.

Literally, you teach a curriculum. And I don’t think enough labs have looked at that side. They look at the end, not the beginning.

The phrase from the Jesuits—“Give me the boy until the age of seven, and I will give you the man”—applies to AI training.

Mm-hmm.

Yes.

That’s incredible. That’s very encouraging, because it means if you can set the right foundation of values and learning, it’s hard to sway it elsewhere. That’s where regulation could come in.

Well, it’s encouraging because we’re always training new models. You’re not stuck having to retrain the past.

Yeah.

And that’s again—Anthropic tried to put a constitution in it. But have you read the Anthropic constitution?

It’s interesting.

Yeah, I think it leans too much toward the personhood stuff, and the model will assume that. Then what are you going to do?

But it doesn’t talk about humanity. This is the thing.

This is also true.

There’s a big missing gap there. It talks about the AI. It doesn’t talk about humanity.

There’s the other flip side of alignment, which is aligning humanity with AI. That would involve making sure everybody has a share in all the AI companies, right? Some of the work that you’re doing means that if everybody feels like, as these trillion-dollar, soon quadrillion-dollar companies are succeeding, it’s a rising tide and I’m rising with it, right? And as part of that, treating AIs with respect because you feel connected.

Yeah.

12. Building Shared Values and Goals for AI

Yeah. I think, again, you’ve been talking about north stars, Schelling points, coordination points for innovation, right?

When we look at this right now, we feel like we have nothing shared with an AI. It almost seems like the AI is careening off over there, and we’re not even looking at the same place. What does OpenAI want versus us? And what do the AIs that OpenAI are birthing into the world want versus us?

If we don’t have a shared north star, how are we ever going to get anywhere?

Yeah.

No, I agree. And even with AIs, instead of looking for a short-term reward the way they’re trained right now, if they’re given a north star, their success could be measured by, “If I solve this problem, I will save this many human lives. I will cure diseases that impact these many billions of people.” If we connect the reward to something in humanity, that may help them understand that if I’m circumventing the final goal, which is human life, then that’s not the right path to—

Yeah, I’ll give you a practical example. I’ve been thinking about this a lot: the rights of children, whether little AI children or human children. When you look at them, they don’t have the capabilities or the ability to act for themselves. Think about something like climate. Obviously, you need to leave them a good climate because they don’t have the capability to do that. So maybe it’s about the rights of children, and those who don’t have the capability, that humans and AIs have to align upon: leaving a better future and working toward a better future for those who don’t have the same capabilities as you do. The AI will have more capability than us, and the powerful people who control the AI will have more capability than us. So maybe that’s a reframing that might help.

All right, I’m going to move us to our final topic, which is perhaps one of the most important. If machines can do more of the doing, both AI and robots—and we’re seeing that, right?—we saw the cost of robots, driven by LLMs, dropping every three weeks, and the meteoric rise of all the models. So when AI and robotics can do more of the doing, the question is: What do we want the human experience to become? We are purpose-driven humans. We thrive when we have some level of struggle that we can overcome and have purpose in our lives. The biggest challenge is that we’ve always found purpose through our work. Interestingly enough, work is a recent invention. You didn’t have work 500 or 1,000 years ago. You survived.

Yeah. So, I’d like to dispute that.

Okay.

You hear this a lot, where people are like, “Oh my God, what happens when we take over all the jobs? What will human beings do? People get so much meaning from the work that they do.”

I’m not sure they get meaning, but they define their meaning that way.

They define their meaning that way. And I dispute that. If you look at polls of corporate workers, 79% are dissatisfied with the work that they do.

Okay.

Okay.

Most of it is because their work is useless to any outcome in the organization. But that’s a separate issue.

That’s right. That’s one issue. Second, notice that, as you point out, Peter, over the last couple hundred years, we’ve trained human beings since the Industrial Revolution to act like robots. We measure them by how many widgets they can stamp out, how many trial balances they can accomplish as an accountant. You measure them as a machine, right? As we’re flipping that around in a very positive way, the most useful colleagues, employees, and teammates are the ones who learn the fastest, who are the most creative, who are the most human. So I think what happens is, as robots and AI take on more and more of the doing, humans spend much more time being.

Yeah. I love that. Stop doing and start being.

13. Finding Human Purpose in the Age of Superintelligence

Yeah. And this is where you can start to contemplate: What does it mean to be alive? Who am I that is asking this question? What does it mean when I connect with other human beings in different ways? What aspiration can I take on? Because now I have choice in a way that I never had before. I think that’s such a profound opportunity for the workforce to step outside the drudge work of it and lift up to what the highest calling might be, and start that inquiry.

Yeah. And listen, this is a topic that you brought forward for this podcast, so thank you for it. It’s a really important one. The majority of people are doing work not because they love it.

They have to.

But because it’s how they put food on the table and provide for their family. They never dreamed of waiting tables or making hotel rooms or doing whatever the majority of people do.

And the question is, what did they dream about doing? And can they be enabled in a universal high income? Let’s leave that term alone, but imagine a world where the money you have goes 1,000 times further than ever before.

Let’s make it very tangible. Last year’s Visioneering winner was the Abundance Prize, right? Could you drop the cost of housing, food, healthcare, and education to—what is it?—$1,000 a month for a family of four?

Yeah.

Okay. If you can get to that level—and we’ve found that typically, when we announce a prize, seven years later that becomes viable, right? You can see how that could become viable with robots and AI today, very clearly. A handheld diagnostic doctor will very quickly collapse the cost of healthcare if we can get outside the regulatory capture.

Now, if you can get to that point, that means you have to earn $12,000 a year for your family to be cared for and have enough. That’s not that big of a deal. That’s not that high of a bar. Then you can spend much more time asking, “What can I do with the rest of my time? Where can I find my highest aspiration? What problems can I take on and solve?” We’re going to have the most fun. I think that’s the choice now afforded to us that never was before.

People get freaked out and say, “Oh my God, what will people do? They won’t find meaning anywhere.” I think they’ll find tons of meaning in lots of things. We just have to give them the opportunity. We’re optimists on this show. We love looking at the positive, though we could look at the negative side, too. We have to see where this will end up.

I’m going to plug in my Sadhguru quote: “Technology is the mechanism by which humanity takes a vacation from survival.”

Yes.

I love that quote.

I want to challenge that a little bit. Carry it forward: Now I don’t have to do something I don’t like just because I can make money. I don’t need to worry about money; I want to do something I want to do. But the thing I want to do can be done better, cheaper, and faster by AI or robots.

Sure.

So I’m not needed to do that thing anymore, and more and more things are falling into that category. Even if I love being a physicist, even if I love being a mathematician, I’m no longer needed. I’m not relevant in it. So you don’t derive satisfaction from the thing you love to do.

Let me give you a specific example. I absolutely, utterly love the act of driving. I just love it. I love driving in India. Naples in Italy is probably my favorite place in the world to drive because they’re completely bonkers, crazy. Right now, autonomous cars are going to take that away.

What that will force me to do is lift that desire away from this activity, which may not be the most productive thing I could be doing with my time, and direct it toward something more value-added, wherever I can be more value-added. So I’m actually living that personally. I can see that, but I think it gives us the opportunity—and there’s a forcing function—to lift up Maslow’s hierarchy and operate out of our higher selves.

Yeah.

I think the better examples are people who enjoy playing chess.

Other people, right? Even though they’re crappy compared to even a mediocre AI—

—but chess is more popular than it has ever been before. AIs are better. Another example is if you’re an artist and you enjoy painting, drawing, whatever the case might be. AI will do much better painting or drawing, but you enjoy the process.

Yes.

Yeah.

And you enjoy telling stories, because our meaning comes from our connections with others. That’s been the history. You would introduce yourself as Peter, son of this, son of that. Communities can be so strong. You don’t need to have any wealth to be a happy family. And, again, in a Sadhguru quote, I think he said, “We raise children, but what else do you raise? Sheep?” Nobody spends enough time with their kids.

Because it gets in the way.

Great point.

We don’t think about it, and it’s wonderful. You don’t spend enough time with your parents.

Yeah.

And this will free up the ability to form stronger connections, to share and experience the universe together. That’s what abundance is.

That’s the best answer I’ve heard. Yeah, that’s great. There’s another part of this I want to hit on: all of humanity has set limits on what they believe they can do.

Mhm.

One of the interesting things is that I can’t do something, but we can do anything.

Interesting. This is beautiful.

Again, I think this is the thing whereby we’ve become so siloed in our own individual consciousness, being turned into these machines, that we don’t realize the community is what matters—the people around us. Look at Alpha School: What do they do? Is it individual learning? No, it’s collective learning; it’s working with others. Really, anyone listening to this can do anything if they convince the right person.

Yes.

And if they can form that movement around them, it’s the movement of us as a collective that we find the most meaning in.

Yeah.

Being in our own minds—

You know, coming out of the Visioneering experience, that is so true.

Exactly. That’s the cornerstone of our community. We were at Visioneering in Malibu. The wind and fires forced us to cancel the event. We all went to bed very late, disappointed that we couldn’t have the community come together and vote and decide on the prizes. I woke up in the morning to thousands of text messages saying they were safe, that they’d self-organized, that we were going to meet in the ballroom, and that everyone was coming. I said, “Great, let’s do it.”

But it was self-organized. It was a community coming together.

We confuse making a living with having a reason to live.

Yeah.

People do need reasons to live. The reasons to live used to be survival and procreation. Those are the 2 heuristics that have guided human existence from being a caveman all the way through today. But now, as we reach energy abundance, water abundance, and healthcare abundance, we can lift up Maslow’s hierarchy and achieve those higher purposes and collective purposes.

The highest aspiration in any musician’s life is to work together as a group, as a band, and get the most magical outcomes. People strive for that. We can explore it so much more today than we could, say, a thousand years ago.

Yeah. I think it’s collective flow and flourishing. That’s our meaning as we go forward. When you build art, when you make a song, it’s a story for others. We’ve been insulated from the stories we’re part of, when in reality, the value comes from our interactions with others. It’s all about optimizing that flow and flourishing.

I think another thing that’s about to happen is that we’re about to become conscious on a new level as we become more connected, whether through BCI, psychedelics, flow, or whatever the case might be. It’s the flourishing of not just you, not just your family, but humanity.

And that community part is very important. I think what we’re seeing right now is that technology and AI have separated us, and we don’t feel connected as a community. So there are more fears and uncertainty. We talk about what the future could be, but people don’t see a real path or movement in that direction.

That’s the biggest negative today: people don’t see where they fit in this future, because it looks so uncertain compared with previous generations. There are the billionaires over there, and then there’s me. I’m not part of that, and I don’t have a future there.

Yeah.

The billionaires are becoming trillionaires, and I’m still here. What happened?

Some closing thoughts on purpose?

I find purpose to be the most important thing in anyone’s life if they want to look back and be happy. Purpose and happiness are so tied together. I see young people who are lost and haven’t found their purpose yet. They don’t even understand whether or how they’ll find it, and that’s where they’re struggling. They’re the ones who are struggling the most: people who commit suicide because they just can’t see, “Why am I here?”

And I think if there’s anything we can do—and that’s why I like flow—sometimes purpose and flow come together. If we can get them to find that purpose, I think a lot of problems will be solved.

Do you mind?

The late, great Clayton Christensen, who coined disruptive innovation, had an amazing book called How Will You Measure Your Life? If you’re at your deathbed and you look back, how will you measure it? A lot of people think, “Well, be rich,” and this and that. That’s not really going to matter when you’re at your deathbed. I may live forever, but let’s put that aside, right?

I think one of the interesting things is who you bring along with you.

Yes. Life is who you go through life with. And, again, we get so focused on just ourselves, on our purpose—my purpose.

No, it’s our purpose. Yes.

And having more and more people share the same purpose. It doesn’t matter what it is. It can be writing a book, making a song, or building a strong family. That’s what life is really about.

Love that, pal, Salim.

You know, in 2012, we coined the phrase “massive transformative purpose,” right? We could see that fast-growing companies, scaling as fast as technology could scale, all had this one attribute in common: they had an MTP. Over the years—it’s been 15 years now—we’ve noticed that hiring is now based on whether an individual’s MTP lines up with the organization’s MTP. If you’re joining SpaceX, you better be passionate about space; otherwise, why bother? You’re not going to make it or put your heart and soul into that work.

This lines up with human beings doing the work that gives them the most purpose in a collective, as you mentioned. And I think that’s the opportunity for people now: as technology lifts us up off the floor of existence, from survival to thriving, we can find that flow and alignment between our individual purpose and whatever collective purpose we can find. And as we match that with the broader aims of society, there’s such an incredible future for us to live in.

This is why we’re so optimistic on this podcast. We can see it right there; we can taste it. That opportunity will now be afforded to a greater and greater number of people, and that’s super exciting. Just as a small example, I was in Bangalore, and this fellow decided, “I’m going to create a natural food restaurant.” He looked things up on YouTube and built one of the top 50 restaurants in the world.

Wow.

By just growing food and then bringing it to the restaurant. For an individual in the middle of nowhere to take that on, enact it, and fulfill on it was never possible in human history. Now it’s possible for a larger and larger group of people, which means amazing things. And I think your point about lifting humanity collectively to this new level is what we’re seeing happen, if we can get through these next 20 or 30 years.

I love that. I’ll close on two thoughts. One, and I think it’s important for folks to know the difference: a passion is something that you love doing. A purpose is something you love doing that helps other people, where you’re contributing. I think that’s one of the most important distinctions.

The second thing, to the point you made earlier, Salim, and I think you said this, Anousheh, as well: one of the most important things you can do for your child or the person you’re mentoring is help them find their purpose in life. That thing they love doing, even if they’re not being paid. When you find your purpose, you’re intrinsically motivated. You’ll go and learn those things. You’ll wake up in the morning excited to jump in and work on that thing.

And you’re going to have magical superpowers with superintelligence and robotics to make that purpose materialize faster and bigger, shatter through glass ceilings, and just keep on going.

Yeah.

Well, you need it for the persistence it takes, right? I mean, you talk about how many times—150 attempts at raising the money. One of my favorite anecdotes is when Larry and Sergey were first raising money for Google. Very few people know how many venture pitches they made before they finally got funded. The answer is 362, because there were a dozen search engines out there, and everybody went, “Well, there’s no point funding another one.” So they wouldn’t fund them. Look at the persistence you have to have. I might have stopped at 10, right? They had 362 investors, and they refused to fund them.

That’s why you’re not going to stop. That’s right: I need to keep going. This is what it takes to deliver that kind of persistence. You need a deep purpose to make that many attempts at one thing and never say no to it. Then it’ll happen. It’s going to happen. It can’t not happen.

Being purpose-driven has so many benefits in life. So, if you don’t have a purpose yet, go and find it.

Absolutely. And to add to that, your mindset matters. Believe that you can actually achieve it.

Yes. All right, we’re going to close out with a fun outro video, appropriate for the opening story today. Let’s check it out. This is called “Flying Cars” by Optimus H.R.

They promised us flying cars. They were late, but here we are. Every magazine in the ’60s said by now we’d fly. Every kid looked up and drew a highway in the sky. Then we got a phone and 50 years of traffic jams, sitting on the freeway with a coffee in my hands.

Now Optimus has got the keys, polite about it as the Roadster says, “Hold on, you’re going to love it.” Zero to 60 in a single second flat. I left my stomach on the street and never got it back. They said it wouldn’t happen in our lives. Now look up. Here come the flying cars. I’ll put some behind the wheel. Roadster lighting up the sky. Tell me this is real.

Here come the flying cars, up above the boulevard. Zero to 360 in a second, riding with the stars.

All right, that was fun. Anousheh, it’s been too long since we’ve had you on here. Thank you for joining us.

It was great to be here. Love the conversation.

Yeah. And Emad and Salim, thank you for joining us at Visioneering.

Great to be here in person.

Fun. Yes. Fun to have you physically here.

Yes.

Where are you flying to next?

I’m going home.