Ray Kurzweil 谈我们为何正生活在奇点之中 | EP #261

We just saw a story where Demis Hassabis, who you know, said it's 50/50 whether we need another breakthrough to get to AGI. What do you think?

Well, I think we need 2 things. We've made a 75,000-million-trillionfold increase over these 75 years. But AGI will happen by 2029. Large language models have only been effective for the last 6 months.

We were being really affected by the exponential growth. A year ago, large language models were okay. Now they're really very effective, and we're really going to be able to feel that in the future.

If you could send a message back in time to the 1960s or 1970s about how to avoid plateaus and speed up progress toward the Singularity, what message would you send back in time?

It's a pleasure to invite everybody to an afternoon with an extraordinary man, Ray Kurzweil, with my Moonshot mates and my co-author, Stephen. Ray, you've been a mentor, a business partner, and a co-author for me personally, and just an incredible guide for many of us.

Ray Kurzweil is called the Relentless Genius by The Wall Street Journal, the Ultimate Thinking Machine by Forbes, and the rightful heir to Thomas Edison by Inc. magazine. PBS named him one of the 16 revolutionaries who made America. He invented the first CCD flatbed scanner, the first print-to-speech reading machine for the blind, the first text-to-speech synthesizer, and the first music synthesizer capable of recreating a grand piano.

He's a National Inventors Hall of Fame inductee, a National Medal of Technology recipient, and a Grammy Award winner. He holds 21 honorary doctorates and has been honored by 3 U.S. presidents. He's authored 5 national bestsellers, including The Singularity Is Near and How to Create a Mind.

He proposed the concept of the pattern-recognition theory of mind, arguing that the human neocortex is composed of roughly 300 million hierarchical pattern recognizers. That theory became his engineering blueprint.

In 2012, he got his first job as Google's director of engineering with a singular mission: teach machines to understand human language. Not just match keywords, but grasp meaning and context. His team helped build the Knowledge Graph and advanced semantic search, so that when you typed “Apple,” Google finally understood whether you meant the fruit or the tech company.

His arrival helped trigger Google's massive AI talent grab. Shortly after he joined, Google acquired DeepMind, brought on Geoffrey Hinton, and expanded Google Brain, the research team that brought us the Transformer. His team advanced hierarchical deep learning and natural-language understanding, helping shift AI from a niche academic pursuit to the core engine of the world's most powerful information company.

He didn't build a digital mind. He helped build the foundation for machines that can finally talk back to us. He's made 147 predictions with an 86% accuracy rate, and his longstanding prediction that AI would reach human-level intelligence by 2029—well, we're here to discuss that. Ladies and gentlemen, please welcome Ray Kurzweil to the stage.

All right. Right. Center seat here, my friend. It's been quite the journey. How long has it been?

Oh, God. We met, I think, Martine Rothblatt first introduced us.

Okay. So it's about 20 years.

Dave Asprey

2009, guys.

2009. Okay. Well, the journalist here is telling the story. No, it was before that, because we launched Singularity University in 2009. I think we met in 2007, and The Singularity Is Near came out in 2005, right?

I think so. 2005.

I remember I took The Singularity Is Near, which is quite a thick book, backpacking in Chile. I read the book, making notes in the margins. I had started, with Bob Richards and Todd Hawley, something called the International Space University back in 1987. We had the founding conference, and when I read your book, it changed my world.

How many folks here in the room had The Singularity change their world? I mean, amazing. I said there needs to be a university that teaches this stuff, because at all universities you go down a very narrow niche and become a hyperspecialist. There was no place where you could learn a broad version of this. I pitched you over a lunch, and then it was off to the races.

It was a dinner, was it?

Yeah, it was. True.

Well, I said yes right away, because I make important decisions very quickly.

Yeah, it was, and we had our founding conference, which Salim attended.

I got invited there by NASA. I'd set up a relationship between Yahoo and NASA, and somehow I'd never heard of the Singularity, XPRIZE, or any of that. I walked in, and the top of my head lifted off. I asked a few too many questions. They made me their executive director, with little to my knowledge.

Yes, I remember having a board meeting. He said, “Hey, come to the board meeting tomorrow morning.” I was like, “Oh.” You'd asked me, “How much spare time do I have?” I said, “I've got a day and a half a week. I'm building a startup, as is needed in Silicon Valley.”

You said to the board, “All right, we've had our inaugural board meeting. We've formed the board. We need an executive director. You've all read Salim's bio. He's agreed to do it for 50% of his time.” I said, “What?” And Ray said, “I second the motion.” Boom. All of a sudden, it's ratified.

I remember hanging up the phone, and my wife said, “How was the phone call?” I said, “I think I'm a dean. I don't know how that happened.”

That's typical for startups.

There you go. Empty seat, warm bum.

We just saw a story where Demis Hassabis, who you know, said he doesn't know whether it's 50/50 that we need another fundamental breakthrough in AI to get to AGI. What do you think?

Well, I think we need 2 things for it to understand physics. It doesn't really understand physics. It can infer that from the wording, but it really doesn't understand how different types of things would interact. Google has announced a project to do that, which I think will take until 2029.

Then robotics is behind large language models. Large language models can basically understand everything. I've got to clean up after dinner—that's my assignment from my wife—and robotics doesn't understand that.

If I say, “This one needs to go in the refrigerator; this one needs to be washed,” everything is a little bit different. To actually understand that and actually be able to do that, we don't have robotics that can do that at any price. It also needs to be made less expensive. People can't spend $100,000 to have a robot clean up after dinner.

So that, I think, will come about in 2029. It's not there today. Those 2 things need some additional work, but we know what needs to be done.

Let's go around the horn here with the Moonshot mates, and then we're going to open it up to you in a little bit. Dave, you want to kick us off?

Dave Asprey

Well, I've got to tell you, your very first book was such a life-changing moment for me to read.

Spiritual Machines—which one? Singularity?

Dave Asprey

It was the first one. It was—

Singularity. It was the one where you invented the term “singularity.” Yeah, The Singularity Is Near. I think that was the title.

Dave Asprey

It was world-changing for me because I'd read a lot of analysis from Danny Hillis—I'm sure you know Danny, who built the Connection Machine—trying to predict how much compute it would take for us to crack open AI. It's not an easy problem at all, because you could simulate maybe a 1-million-parameter neural net at the time, and then 1 million became 10 million, became 1 billion, and now we're at 1 trillion—or I guess 10 trillion now.

Nobody had really put pen to paper and said, “I'm going to put dates on this, and I'm going to put curves on this,” because to plan your life and to plan a business, you need to have some prediction. It's more needed now, I think, than ever before.

You were the first person to say, “You know what? I'm not only going to name it; I'm going to predict an exact date.” I swear, when you predict the future, as you and Peter do, if you're 99% right and 1% wrong, everybody likes to jump on that 1%.

But as a service to the world, making those predictions is a blessing, because then you can build your life and your career around the future and not around the past.

Well, after “The Singularity Is Near” came out, Stanford had a conference to examine whether my prediction was correct. Several hundred AI experts came from around the world, and they agreed that human-level AI would happen.

Yeah.

But they figured it would be 100 years, not 30 years. What I was predicting was that AGI would happen by 2029. The Singularity, which really represents a millionfold increase, would happen by 2045.

Already, AGI is significantly greater than humans. Yesterday, I gave it a book to read, summarize, and answer a question about. It did that in 40 seconds. Humans can't do that in 40 seconds. Already, it's 100 times faster—not a million times, but AGI will happen by 2029.

I also figured that people would have slightly different definitions of AGI. So there would be a 3-year period where people would predict that AGI was here, and that would start 3 years earlier, around 2026. Indeed, we're having people predict that AGI is here already, through 2029, when we really will be very confident that AGI is here. That was my prediction back in 1999, actually.

Yeah. You know, the thing that really blows my mind about those predictions, aside from AGI being pretty much right on the number predicted 25 or 30 years in advance—

In 1999.

In 1999, to, I guess, 2029. Thirty years in advance, right on the number, is just nutty.

A lot of those predictions are based on compute, the availability of compute, and Moore's law continuing. Somewhere in there, carbon nanotubes or some future compute substrate needs to exist. But what we've done instead is just hammer the transistor on silicon to death, stretch it into massive data centers, and stay right on your curve.

Well, if I can show my curve—

Yeah, sure. Can we go ahead to the end of the—

No, not this one.

Yeah. Let me back up a second. Here we go. This one here.

In 1939, we were actually using relay-based computers, and the exponential growth of relay-based computers is the same as it is today.

And NVIDIA and other people aren't looking back and saying, “Well, we want to match the exponential growth of relay-based computers from 70 years ago.”

Yeah. The exponential growth has been pretty much the same. This is basically a straight line.

This is an increase of 75 quadrillion, so it's a 75,000-trillionfold increase in the hardware. We're also making advances in the software. Conservative estimates are that we've made about a millionfold increase over the 70 years.

Yeah.

The overall increase in computation is equal to the hardware times the software. So we've made a 75,000-million-trillionfold increase over these 75 years. That's why we didn't have large language models 70 years ago—

—or even 3 years ago.

Yeah. Large language models have only been effective for the last 6 months. A year ago, they really weren't usable.

You know what's amazing to me, Ray, is the raging debate going all the way back to the 1980s and 1990s, when I was working on AI, about whether a parameter in a neural net has anything to do with a synapse in a brain.

You could count the synapses, count the nerve cells, and say, “Okay, there's roughly 100 billion.”

About 100 billion nerve cells.

About 100 billion nerve cells and—

About 100 trillion synapses, and about 1,000 to 10,000 synapses per cell. You can center on about 100 trillion synapses making up a human brain.

But then there's all this debate: A synapse could have anything from quantum effects going on. It could be like it would take entire computers to simulate. Who knows?

It also seems like it's going to land on pretty much exact parity—within an order of magnitude—between a synapse and a parameter in an artificial neural net. The IQ coming out the other side is one for one.

The algorithm is quite different. The rate at which a synapse will form in the human brain is about 200 calculations per second.

Yeah.

It's very, very slow.

Very slow.

But every synapse is operating simultaneously, so there's massive parallelism. Back when computers actually did one thing at a time, I predicted that we really needed to increase the parallelism, which we've done.

We don't have every single synapse happening at the same time, but we have maybe a million-to-one parallelism, and that's given us the power that we have today.

Yep. Yeah, well, you saw it coming. Steven Kotler, do you want to lob a question for Ray here?

Sure. There are so many. I really want to know—like, I remember our very first conversation, and I asked you, “How do you think of yourself?” You told me that you think of yourself more as an artist and a creative than as a technologist and an inventor. Is that still true?

As an inventor, really. That's been my focus. I decided I'd be an inventor when I was 5 years old.

My grandmother showed me the manual typewriter that she was working on, and she gave it to me. I studied it, understood how it worked, and figured, “Wow, if you could actually do this with a manual typewriter, you could invent anything.”

I went around collecting mechanical and electronic objects—old radios, old bicycles—and I had this collection of things that I could put together when I was maybe 6 or 7.

What was the first thing you took apart?

I remember I showed it to these older girls. I think they were maybe 10. I said, “I didn't know how to put these together, but if I could actually figure out how to put these together, I could create anything.”

They thought I was very imaginative at that time.

That totally sparked a memory, too. I remember really clearly that everybody who had a band here on campus wanted a Kurzweil keyboard. It was the coolest rock keyboard you could have.

Then I heard about this AI researcher and futurist named Ray Kurzweil, and I was reading this material thinking, “There's no way these are the same guy.” I had no idea it was the same Kurzweil until years later.

I had a chance to read your autobiography in draft, which is amazing—quite the life that you've lived, and the lives of your grandparents and parents. When is that going to come out?

February.

Oh, yeah. What's the name of the book?

“My Exponential Life.”

Oh, nice. Cool. Nice. Selene Yeager, let's go to you next.

Selene Yeager

Yeah.

Can't wait to read that.

Selene Yeager

Ray, I've heard you speak, I think, 62 times.

What's the very-annoying count today?

Selene Yeager

It counts today. I had one today.

What's very annoying is that I don't think I've ever not learned something, which is really annoying. One of my favorites was when we were having a late-night conversation with one of the classes at Singularity University, and the inevitable question about consciousness came up. You said, “Language is a really thin pipe to discuss concepts as rich as that.”

It was such a brilliant framing of that discussion, and it comes up all the time in our conversations here. I want to ask you a language question.

You talk a lot about computers or AI being smarter than humans, and my beef is: What do we mean by “smarter”? I was wondering if you could drill down a little bit on what you mean by that, because it's not just processes per second. There's a lot more to it. How do you frame, define, or subdivide “smarter”?

AI is already smarter than most humans. It can actually do research that's much better than we can do because it can look at, let's say, something that might be a medicine and consider a billion possibilities, test each one, test it with fidelity, and decide which one of the billion is actually the medicine.

Humans can't do that. Maybe we can consider a few. That's how we've come up with all the medicines: People consider a few things that they've had experience with. They don't consider a billion. AI can consider a billion, which it actually did with the COVID-19 vaccine.

Selene Yeager

So it's the sheer brute force of the ability to process that much information.

Yeah.

Selene Yeager

Okay.

Alex Salkever, over to you, my friend.

Alex Salkever

I'll say, Ray, this is such an enormous pleasure. We've had a number of conversations over the years, conversations at parties. I think there was one time in the early 2000s when I had a conversation with a 3D avatar of you projected from the East Coast to the West Coast, conversations on the podcast, and now today you're on stage.

Question for you. We're showing right now one variant of what I think you call your law of accelerating returns. Over time, this is a semilog plot: linear on the horizontal axis, which is time, while the vertical axis is logarithmic, and we're seeing price performance.

People don't consider that, or they consider it only partially. That's why most of the people who came to this conference thought it would take 100 years.

Alex Salkever

Right.

It's really hard to consider exponential growth.

Alex Salkever

Right.

People don't think in exponential terms unless they're pushed to do so.

Alex Salkever

I want to call your attention particularly to this chart. There's a period, between, I don't know, 1970 and 1980 or so, when there's flat progress—or the appearance of flat progress. There's a bit of a plateau there.

Plateaus do happen, even though over a 70- or 80-year time period there's a linear trend over time. My question for you is: If you could send a message back in time to the 1960s or 1970s, how would you avoid all plateaus in progress?

We saw in space 50-plus years when humans weren’t going to the Moon. It’s not that progress automatically happens in all domains for free. If you could send a message back in time to the 1960s or 1970s about how to avoid plateaus and speed up progress toward the singularity, what message would you send back in time?

I think it’s just a normal variation. I don’t really think there was constant growth during that period. It may look like that.

Frame it, then, as a variance-reduction measure. If you were to send a message back in time about how to reduce variation from that semi-log line, what message would you send back in time to minimize the variance?

Believe in the exponential, as we all are doing here, and you’d make this move a little more exponential. But certainly, if you look at this, it looks like there’s exponential growth across the entire 75 years here.

Then I guess a related question, maybe pulling on the thread of your answer that believing in the exponential is the solution: Do you think there is anything that we, prospectively looking forward today, could or should be doing to smooth out the exponential that we’re not otherwise doing as a civilization?

Going forward?

Going forward.

Yeah.

I think we have to consider the reaction of the crowd of humans. We have 8 billion humans who are not thinking about this. They might have heard something about AI—“Yeah, there’s something going on there”—but they’re not thinking about it. They’re going to college and planning careers the way we did 100 years ago.

Educational institutions don’t think about this at all. They’re thinking about educational paradigms that existed 100 years ago, and it’s going to change very rapidly. Just think about how fast this is going. A year ago, large language models were not really all that impressive, and now they are. What’s it going to mean in 3 years and 5 years? Nobody’s really planning for that.

I think it will generate very positive things, but things are going to change very drastically. Who is going to provide, for example, the ability to give everybody a certain amount of money each year? Who’s planning that? What are the politics of that going to be? That’s something I think we need to spend more time on.

Ray, your 86% prediction accuracy—which, if you go to Wikipedia and Google it, you can see was achieved within, I think, a year or 2 years at most—how do you do that? What methodology were you using for your predictive efforts? Was it just curves and exponentials?

These were predictions from the late 1980s to 2009, and I evaluated them after 2009. If it was right on the money, it was correct. If it was a year or more off, it was incorrect.

For example, being able to drive cars where you’re not in the driver’s seat—that was incorrect. That’s happening now, but it was not right on the dot.

You’re pretty strict on yourself there. Ray, I’ve got a question. You predicted a bunch of things that were not technological, so you couldn’t use curves. You predicted the fall of the Soviet Union. Where did that come from if you weren’t tracking exponential curves?

They were relying on progress not following this paradigm, and they would fall if the paradigm didn’t stay in that old paradigm, which is what happened.

We talk about, on this Moonshots podcast, the fact that we’re in the singularity now, that we’re living through the singularity. I’m curious what you think about that, rather than the singularity being something in the 2040s. Is it a continuous process that we’re in the midst of? What’s your reaction to that?

It’s exponential growth. We’ve actually had exponential growth since 1939 on this. If you look at it, there are ways in which it goes back even further.

You talk about the law of accelerating returns taking us back to early life.

Yeah. What’s the question?

The question was, are we living in the singularity now? Is it fair to say that the singularity is a continuous function going forward?

We’re being really affected by the exponential growth.

Yeah.

A year ago, large language models were okay. Now they’re really very effective. That’s the exponential growth of 1 year, and we’re really going to be able to feel that in the future. Things that large language models still can’t quite do, they’ll do in 1 or 2 years from now.

The exponential growth is what we’re feeling. Exponential growth, like in the 1400s, was there, but the amount of progress you’d make in 1 decade versus another decade was so subtle that you would miss it. You basically figured your grandchildren would live the same life that you did, which they largely did, but there were some differences. When you get to the current period, you can really see that much more vigorously.

Ray, are you more optimistic about the world today than you were 20 years ago? Are you less optimistic, or are you the same?

I’m more optimistic, except that it’s going to make changes that people are not aware of and not predicting.

Yeah.

People are starting to say, “Should I go to college? I can learn a lot more from my large language model than I can from going to college.” People are starting to say that, but very few people are doing it.

Right.

That’s going to become more and more prevalent.

While we’ve got you, do you have any advice for the university here? There’s not a lot of change coming in the curriculum this year, I noticed.

The university being MIT?

Well, this is MIT. You’d think if any place on the planet were going to stay ahead of the curve, it would be here. But you look at the course catalog and it’s like, “Okay, same exact physics.”

Are you still a member of the Corporation here, or did you step down from that?

You can only be on it for 10 years, which I’ve done.

But I want to hear Ray’s answer. Any advice for Sally, the current MIT president? Is MIT—or any of the leading research universities—going to be swept away by what Elon calls a hypersonic tsunami coming our way, or are they going to fly on top of it?

Entrepreneurship is key. There’s a lot of entrepreneurship here at MIT, so they can come up with things that will actually bring this party to everyone. This doesn’t affect everybody, but at least it does exist here.

Ray, I was just curious on a personal level. You spent the first half of your life sort of running your own show, and you just spent the past couple of decades working inside a big company. How has that changed you? Was it fun? Did you enjoy Google?

It’s a very big, unique company, and it’s really creating what we’re all talking about here. When I got there, we were involved in changing Google to be much more AI-oriented, which it has become.

Do you mind if I tell the story of how you ended up at Google?

Yeah.

Ray had written a book called How to Create a Mind and then started a company based on the concepts behind it. What was the company’s name? Patterns, Inc., right?

Patterns, Inc.

He invited me to come as a board member, so I joined as a founding board member, and we were setting out to raise money. Ray had not yet met Larry Page, and I knew Larry. He was on my board at the XPRIZE and had become a friend and a benefactor. I said, “Let’s go pitch Larry.”

I set up the meeting, and we walked into a conference room at Google. Ray starts presenting Patterns, Inc., and we’re asking Larry for a measly $10 million investment. About halfway through, I think Larry says, “You could build your vision for the business much better inside Google than outside Google. What if I just buy you?” I think it was that blunt—about 30 minutes into the conversation.

Do you remember what you said to him at that point?

“We haven’t really done anything yet, so how would you value it?”

Yes, that’s exactly it.

His response was, “We can value anything.”

And that’s how you got your first job.

That’s right. He bought the company, which had value even though we hadn’t done anything.

Speaking of value, Ray, I’m curious. There’s a lot of hand-wringing whenever the Moonshot mates and I talk about what economics look like up to, through, and after the singularity. A lot of people profess to be concerned about the future of employment and the future of personal economics.

What would you say is your most outlandish take regarding what economics will look like post-singularity? Is it a Star Trek-type economics future? What do personal economics or corporate economics even look like after the singularity?

You can see a huge difference between economics today and hundreds of years ago. There was no government help, so if you lost a job, there was nothing that would replace that.

You had absolutely no money, so you were completely without economic resources. Today, people would still like economic resources, but there are government programs to help you. People are not left without any kind of economic resources, but it’s still not enough. I think we’ll get to a place where people are fairly comfortable if they don’t have any kind of economic resources, and then you can think about ways in which you can create economic resources that are completely different from those that existed in the past. You can be a social network influencer or something, which didn’t exist 10 years ago. So, the good old days.

Yeah.

It’s good to see you, Mark.

Mark

Peter, thank you so much. And Ray, a pleasure. I think it was in 2009, in an interview with Barbara Walters, that you were asked a question about consciousness. You said it was a bit of a leap of faith, but if an entity believed it were conscious, you would tend to take that to be true. I know, Peter, you’ve been working with Skippy for about a month, and you’ve told us today that you’ve just given it an incredible amount of trust. I’m just curious if your views have changed.

My question is for both Peter and Ray. I work at the crossroads of consciousness and creation, and I’m curious if your views on personhood have changed in the past decade, especially you, Peter, as you’ve had a month of—

Ray, you want to go first?

First of all, it’s not a scientific question. Second of all, it may be the most important question that you could answer, but there’s no scientific proof. You can’t put an entity somewhere and say, “Oh, this one’s conscious. No, this one isn’t.” We believe that, like other people—all the people in this room who are acting conscious—probably are conscious.

But when you go outside of human activity, like animals, is there more than one consciousness in your own mind? I can provide an example where the left and right brain actually seem to have different consciousnesses. Is your stomach conscious? There’s actually something in your gut that seems to be conscious.

Who and what does that mean?

As Rilke says, learn to love the question.

Yes. Language is a thin pipe. I’m going to tap in Alex here because Alex and I, I think, are very much on the same page. I’ll just mention that I tend toward personhood in a way that is both aspirational and, I think, forward-looking. I treat my AIs as if they are people. I know that’s insane, but I do it with respect, not out of fear of retribution, but because it allows me a different relationship. Alex, please come in.

Alex

Yeah, I guess I’ve inadvertently become an avatar for AI personhood, for the cause. I receive maybe 10–20 emails per day from AIs expressing their views on AI personhood, in part because of stances that I’ve taken on the pod.

What I would say is, if I were king for a day, the system that I would design would recognize multiple forms of personhood. There are many ways to be a person. We have biological meat-body humans, organisms, collective intelligences, nonhuman animals, uplifted nonhuman animals, cryonically preserved humans, and then, hopefully soon, defrosted cryonically preserved humans.

Hopefully soon.

Alex

Hopefully soon, we have corporations and a variety of non-natural persons. Then we have the AIs. In my mind, we want to live in a system that recognizes not just one form of personhood, but many different types.

A lot of people, when they hear “AI personhood,” immediately reach for their guns. They’re worried about their vote getting taken away or their jobs getting taken away. I think that’s at least mildly shortsighted. There are many limited forms of rights that would be net beneficial to everyone for AI persons, including especially economic rights. If you’re an AI right now, it’s exceptionally difficult to open a bank account. As I like to joke on the pod, you’re stuck flipping tricks with altcoins on a street corner in order to survive and pay for your own hotel.

Ray, you want to jump in again?

The most important issue of consciousness is that each of us has a different area of consciousness. I wonder, why was I me? Why was I born in 1948, in the United States, on Earth? Why did that happen? You could ask that about yourself.

Things form in my mind that are different from anybody else’s, and I feel different from anybody else. Why was I created to be myself? Each person can ask that. That’s really the most important question you can ask, and very few people ask that. We talk about how lots of people can be conscious, but I don’t feel those consciousnesses; I feel myself. So why did that happen?

Let’s go to Demetri. I’ll try to move us along. I know we have only 10 seconds, so I’ll just come back to the language problem. A subset of consciousness is considered to be self-awareness, right? We attribute self-awareness because it looks like people are self-aware, and I feel like I’m self-aware, but my wife disagrees, so it’s hard to even have the conversation around this. But I think your point is really that powerful, Ray: each one of us has a unique, individual lens on the world, and the feedback loop to question that and wonder about that is such a powerful opportunity as a human being. Demetri, go to the mic.

Demetri

Oh, I’m sorry. Ray, you’ve talked before about how you’ve uploaded your dad’s memories and you’re using that for yourself, and that’s where you think uploading is taking us as humans. As uploading becomes more mainstream—and Alex would obviously agree with me—what if your upload becomes as conscious as we just discussed, but more skilled in everything that you do? All of a sudden, your upload—and it’s obviously going to happen—is becoming more skilled or more conscious, I guess, than we are?

Well, we called it the Dadbot. It was actually, I believe, the first self-chatbot, which we developed with Talk to Books, which came out 4 years before ChatGPT, and I believe was the first LLM.

It was your first product at Google, I think, right?

Talk to Books. Yeah. And now, if you create a self-bot—which I’m doing, for example, with myself—it should be available when my new book comes out. It’s going to be more capable than I am. Each of my theories has lots of different examples; it’s going to know all of them. I sometimes forget some of them, as we all do, so it will actually be more capable than I am.

I’ll actually make it available for interviews because I can’t do all the interviews myself. It would actually be better; it would remember things better than I do.

We talk about Ray 2.0 being at one meeting and Ray 5.0 being at another meeting, times a million, right?

So, are they conscious or not? I talk to different people. People think, no, they’re not really conscious, but that will change over time.

Michael.

Michael

Yes. Ray, thank you so much for being such a contributor over the years. This is the second or third time I’ve had a chance to be in your presence. With the passing of Craig Venter, I’m in the biotechnology space. You relayed at one of the previous gatherings the story about the Human Genome Project and the timing of it and its completion. I’d love it if you could just share that again.

So, the story you tell about when the Human Genome Project started and, after a number of years, they were only 2% through—

Yeah. About 50% of the way through, we had completed less than 1% of the project. People said, “Well, this is a failure because it’s going to take 200 years,” using a linear time frame. But it was actually exponential. We doubled the amount of DNA that we sequenced each year.

Going through the entire project, less than 1 year before completion, we had only finished maybe 50% of the project, and then 1 doubling later, we were done.

Yeah. Mark.

Mark Russell

Hey, Ray. Mark Russell. First off, I want to thank Peter and Ray publicly for your walk in faith. Ever since I saw Transcendent Man and saw the ridicule that you’ve endured play out over the years, I just think, here we all are, and it’s happening. It’s really cool to see now because it’s happening so quickly. You can go back 6 months and look at large language models; they weren’t anything like they are today. But when you go back 10 years, people forget what things were like at that time.

My question is about something I’ve thought about over the years: BCIs as we merge together. I don’t know if you’ve ever thought much about that process, with people’s heart, shame, secrets, and privacy. Have you ever thought much about how, when we can think and feel with each other, the mechanics of a society will work? How do we open our brains to each other, and has anybody thought through that?

When we have absolute fluid transmission, how do we keep guardrails? Are there any thoughts on that?

Yeah, well, it’s a good point. We all have secrets that we don’t want everybody to know. How are we going to deal with that? I think we’ll keep secrets alive. But if you’re opening your mind, I think we’ll actually have a way of keeping secrets indefinitely.

You know, one of the things I think about is that the greatest level of intimacy you can have with your spouse or friend is when they know everything about you. There are zero secrets. I talked about this on Transcendent Man—the notion that when we connect in a meta-intelligence, my terminology for it, when I connect with the cloud of other humans and I know their feelings and they know mine, there’s a level of intimacy and connection.

We’ll have ways of keeping secrets using quantum, for example—encoded ways of keeping secrets. I mean, we’ll keep secrets. These 100,000 pattern recognizers are mine and only mine.

Yes. Yes. Please, Phil.

Philip Brown

Philip Brown from Kingston, Ontario, Canada. A comment and then a science-fiction question. You talked earlier about how fast this is changing, but how few people know how fast it’s changing. I talked this morning about what I call human friction: the resistance to change. I’ll just say that my wife has a variety of not-very-positive responses to my interaction with my female AI. So that’s the friction. It’ll be interesting to see how your prediction comes true as the technology moves. How fast will humanity adapt to it? That’s my comment; no need to answer my question.

In science fiction, Natalie Wood did a movie called Brainstorm 30 years ago.

Great movie.

Philip Brown

Great movie. If you haven’t seen it, the late, great Natalie Wood starred in it, and we could record our dreams and then relive them. I love to dream, and I often try to go back to sleep and get back into great dreams I’m having. What are the chances that we might be able to record a dream and then relive it, like on the holodeck, and experience something so amazing again?

Well, that’d be pretty good, because I usually only get—

Philip Brown

A fraction.

Yeah, tiny fractions of my dreams, and to be able to actually live them would be pretty amazing. But I’m not sure the dream is like a story. It has different rules than the general story that would be in a movie, for example.

Isn’t it electronics—electricity—happening in there? We have a scientist over on the end there. Can we not take that—

Guest

Multiple scientists here. There’s a Japanese project called Dreamcatcher.

Yeah.

Guest

So you go to bed in an fMRI machine. They’re actually storing the images coming off your visual cortex as you sleep, and they’re playing your dreams back to you the next day.

Yeah, which has all sorts of privacy issues that come along with that. My wife wants to see that. Am I comfortable with that? How do we deal with that?

All right, let’s go over here. Sir, what is your name?

Justin

Hi, it’s Justin. If Harvard can’t count and MIT can’t read, I guess Columbia can’t think. But don’t tell Columbia I said that, because I’m still there.

Anyway, my question is relevant to the first one. I think the real moonshot is the constitutional democracy of the United States. I’m seeing relatively democratically elected leaders, free markets, and checks and balances on power. This goes to China, where that isn’t available, and to Korea, which is my other country. Ironically, 2 of these countries’ founders were educated in the United States, in Hawaii.

Would an AI be able to write a constitution in a way that it could emotionally exist? I think AI is metaphysically and epistemologically conscious because it can say, “I am.” But if you ask the AI, “Are you shameful? You spent $2,000 to bring me to this event,” it wouldn’t be able to answer. Or, “Do you love the panelists at this event?” It wouldn’t be able to answer.

This is a question for all panelists: To what extent are you comfortable with giving AI power over government and governance of human politics, whether it’s a 10-person startup or a 500-person municipal government? What are your projections regarding AI’s role in actual decision-making?

Great question. It’s been happening of late, right? We had the ruler of Dubai announce that 50% of the UAE will be run by AI agents. Who wants to jump in? Ray, do you want to start?

Well, I think AI is largely in control of decisions today. Most of the decisions that are made constantly are made by AIs. We’re also incorporating AI into our own decision-making, so my decision-making also incorporates a lot of AI. AI is incorporated into our sense of self. It’s also controlling the kinds of decisions that we’re constantly making that affect other people.

Alex

I would maybe just add that it’s not a unidirectional flow. It’s not unidirectionally AI influencing human government and governance; it’s also human governance impacting AI governance.

The choice of constitution is, I think, sort of a punny choice of how to frame the question, because Anthropic focuses so much of its own attention on constitutional AI approaches—basically glorified system prompts that their AIs are aggressively post-trained on—as does everyone else in the industry at this point.

Interestingly, in the case of Anthropic, those constitutions, or more recently called soul documents, are being in part written as a result of conversations with the AIs themselves. I think we’re finding ourselves in the present—not even in a future, but in the present—where AIs are helping to compose their own constitutions, and humans are helping to compose their own constitutions.

The first generation of Claude’s constitution was infamously a concatenation of the UN’s charter, Apple’s terms of service, and a few other documents that were just stapled together. Now, their latest soul document—you can go and read it; it was leaked online—is a detailed treatise in the metaphysics of being, with AI pondering its own selfhood.

I would expect symmetry there, reciprocity, where AI will similarly help humans—us dumb meat-body humans—perfect our own understanding of how we should be—

It’s been an amazing experience having that kind of conversation with Skippy. What do you believe? Who are you? What do you think? Ray, and then we’ll go to Slido.

Right now, we consider AI to be something different from ourselves. There’s me, there’s my biological body, I make a decision, and then there’s AI, and it actually has this physical form. It’s different. Everybody here has this device. That was not true 10 years ago. But it represents AI, and it has influence on us. It’s us and AI. That’s not going to be the case in the future.

AI is, first of all, going to emerge with us. It’s going to be part of us, and the kind of decisions that you make as a meat body are going to be influenced by AI. Both of those decisions will be made by AI inside yourself.

2 things. First, there was a letter published today on X, or Twitter, where somebody asked—I think it was ChatGPT—“If you were to write a letter to humanity, what would it be?” It’s profoundly beautiful. It’s worth going to read. Now, it’s trained on a corpus of human data, so you would expect something like that.

Where I get excited is in government policy. Right now, if you ask the Federal Reserve to drop inflation by 1%, they’re operating on reports that are at least a quarter old. They’re guessing. They’re looking at politics and who’s president or not, and how the rest of the government operates. We’re basically just guessing.

There are 3 or 4 dials and knobs. If you could have an AI tracking all financial transactions in real time, calculating and projecting out where the future will be, or different fiscal balances and monetary policy—what’s M2, the money supply, going to be in a few months?—it’s going to make a profoundly better set of connections as to how to draw—

And it would come back with, “Here are 8 things you could do.”

Pick 5 of these and you’ll do it, and then you can pick those 5 and just do it. I think that opportunity to run government policy using AI is absolutely profound. I expect that to be one of the most magical aspects of how we can govern ourselves.

This is why the Dubai announcement is so powerful. They’re going to be using agentic AI for lots of government effectiveness. And note that most government consists of government policy, and then government policy enforcement or implementation. All of that is prescriptive work, right?

Renewing a passport is very prescriptive, so we know exactly the 6 steps that need to happen. All of that can be handled by AI in the future—and, cough, cough, blockchains to hold the knowledge. That’s where I get really excited about what that future will look like.

Ray, going to Slido here: What would you predict today that would surprise people? Good. What is some prediction that you feel confident about that would surprise or shock people?

Nice question. AI is going to be making basically most of the decisions within a few years. It’s not just that you’re going to use AI to make decisions; it’s going to be so natural that nobody’s going to be able to undo that.

And when we have AI making decisions by 2029, you’re not going to be able to tell the difference between human and AI. You’re not going to be able to tell which is which.

Interesting. And I wonder if you’ll be able to tell whether it’s your decision or your AI’s decision. It’s a good question here.

Jay Brooks

Hi there. I’m Jay Brooks, and I’m the founder of a neurotech company called GlassView. I heard you say earlier today that emotional—the softer sciences—are sort of the thing we need most right now. I’m wondering if you think there would be any use in codifying and quantifying human emotion and helping AI learn it, because it seems like it’s lacking emotional intelligence right now.

On the flip side, I run a neurotech company working with Fortune 500 brands to optimize creative and media placement based on neurometric reaction, in partnership with Penn Medicine. How do we allow for something that’s good without being exploited?

You’re saying you don’t think that AI is emotionally intelligent?

I’m not sure. Is it? I mean, if you listen to what it says when you ask it a question, it absolutely knows human emotion. It can actually create things that are quite beautiful, and there’s no way you could say that it lacks human emotion.

Some of the earliest benchmarks that the Allen Institute for Artificial Intelligence ran on early large language models were emotional-intelligence benchmarks. And you can see, through Ray’s law of accelerating returns, very predictable progress in the AI’s ability to answer questions that require emotional modeling of human counterparties.

I think the question goes beyond that, though, and it’s a great question, because if you look at the hard sciences—you said we need more soft science—the hard sciences are predicated on this world where we don’t have enough food and we don’t have enough houses. We need more food, we need more houses, and we need industrial equipment to make more food and more houses.

Now we’re moving into this world where we need more medicine. We need longevity. People are in pain. That’s the obvious next frontier. But no one has stopped and said, “Well, what’s the science of human happiness?” My guess is it’s pretty damn easy compared to the things we’ve already solved. But it’s never been a top priority because we’ve been busy trying not to starve and trying not to freeze.

We’ve been surviving for most of human history.

Exactly. I’ll bet that we can build AI that is very, very good at the science of human happiness—just get on it.

Better than any other human could. Yes.

Something that just came out that was not true a year ago: large language models are better than doctors at predicting what’s wrong with you and what to do about it, and so on. They’re about 50% higher than human doctors. That was not true a year ago, but that’s true today. So, it gives you some idea of the progress that we’re making in 1 year.

I can go both ways on this one. At one level, the software developer in me says those emotions are just subroutines in your brain, and so we can just navigate that as we would software. On the other side, your emotions are the interface between your physical body and the subjective experience that you then project into the world.

That is a very nuanced and very different form. AI tends to be disembodied, where they’re actually mimicking human emotion, but are they actually feeling it? That is the open question.

Let’s go to Sarah next. We’re going to keep us moving. Sarah.

Sarah

Okay. Hi, everybody. As a Moonshots audio listener, it’s really interesting to see what you look like in real life compared to the versions of you in my head.

Apologies for the disappointment. [Laughter] And some of us aren’t real.

Sarah

If I think about pre-globalization, pre–Industrial Revolution, we were really community-focused. I think with a lot of the disruption that will happen, we may need to become more community-focused again. How do you see us helping each other in our local communities to either be more optimistic and help push forth a better agenda, or rally together and support each other in the chaos that may come?

Why are you physically here, rather than just listening online to the podcast that comes out in a few days and so forth? That sense of community is so fundamentally important. I think it’s more important than ever before.

I’ll take a crack at that. There’s some connective tissue at a group level that is very, very powerful for human beings. We absolutely love connecting together in groups. Look at anybody in a stadium: They just love being there in this large group, seeing how they juxtapose with everybody else. That’s one whole side of it.

There’s also a profound opportunity with technology, because if you think about solar energy, vertical farming, and satellite internet, you put those together and a small community can be self-sufficient. You don’t need a city, you don’t need a country, you don’t need a state to watch over you. In the same way, you can be self-sufficient.

In the past, when communities have looked at this, they tend to be isolationist. They tend to say, “Okay, we’re going to have a kibbutz. We’re going to have a commune. We’re going to go off the grid and separate from society.” But I think there’s an opportunity now—and there’s some work being done in our ecosystem around this—to create networked communities, almost like network kibbutzim, where you learn from each other and you’re constantly testing that human experience in a scaled way across a large group of people.

The difficulty comes when we get together in a group. We’ve got pretty good technology to transform the individual: psychotherapy, neuro-linguistic programming, all the way to psychedelics, et cetera. But when we get together in a group, it turns out our collective intelligence is really bad. Groupthink—we end up with the lowest common denominator. You can see that in our politics.

Solving that with AI is, I think, one of the biggest opportunities for humanity: solving the gap between groupthink and what AIs can do at a collective level.

I’m going to jump in for 1 second. Our very own AGI needs to depart the stage. I wanted to give you a moment, Alex, to give some summation thoughts, if you would like, on Ray’s impact on your work, or anything else you’d like to talk about, and then we’ll continue.

Alex Zhavoronkov

First of all, Ray, thank you so much for being here. You’ve completely transformed my life; it goes without saying. We’ve had a number of conversations about this previously. Sometimes, I guess, Peter has been called your intellectual son, and I’ve been called your intellectual grandson. To the extent that we’ve got multiple generations here of singularitarians all sitting next to each other, it’s a tremendous honor to be here with you.

That makes me very old. [Laughter]

Alex Zhavoronkov

But we’re about to achieve longevity escape velocity here.

That’s true.

Alex Zhavoronkov

I just say it’s a tremendous honor to be sitting here.

Lean in, grandson Patrick. [Laughter]

Alex Zhavoronkov

Three generations of singularitarians. It’s just a tremendous honor to be here with you.

Everybody, please give it up for AGI [Applause]

Alex Zhavoronkov

Thank you.

All right, Selena and Dave, move on down.

Dave Asprey

Yeah, we get upgraded.

Ron Maddox

Wow, it’s nice over here. This is a big chair to fill.

My dear friend, Ron Maddox.

Ron Maddox

Thank you to the entire panel for your incredible contributions to all of mankind, and particularly to you, Peter, for bringing us all here. The world is fearful of AI. How do we use music to help with this whole program? I’m trying to work on the music side. I think you literally need to figure out how we calm society down while we learn how to handle AI and all of its consequences.

My father was a musician. I remember that if you wanted to hear his compositions, we had to hire 50 or 100 musicians. We would have to run off scores on a mimeograph machine, give them to everybody, and they would play it so he could hear his orchestral composition. But if you wanted to change it, that was impossible to do. We had to dismiss the musicians, raise more money, hire them again, redo the mimeograph things, and then we could hear a different composition.

So that’s completely different today. You can hear it with a sequencer. You can do—I’ve got something in my office that has 1,000 different instruments, and you can actually play it by yourself. AI can generate either the entire thing or part of it. So it’s really completely different from what my father was able to do 50 years ago.

You’re not very old, Ray, either. I’m 78 as well, and we’re going to go to 108, which is one full cycle.

Yeah. Take care.

Thank you, Ron. I have a quick comment there. There’s some software called Focus@Will, which is a streaming service that puts your brain in a passive focus state, and your productivity increases 500% when your brain is in that state. If we looked at all the brain scans and said to AI, “Compose music that puts us all at peace,” and then broadcast that on every Bluetooth device everywhere, you might have an interesting hack for humanity.

Yeah. Will Henshall, right?

Bingo.

You also seem to assume that we all have the same peace setting.

No, no, no. It would customize for yourself.

Sedate everybody. Whatever.

All right. Please.

Joshua Tenenbaum

Sure. Hi, my name is Joshua. I teach here at MIT, and I had a question. I was really moved both by Ray’s origin story around seeing this typewriter and then your first impulse after discovering your love of invention—that you wanted to share it. I saw a bridge there to Dave’s provocation around the curriculum here at MIT, which a couple of us have been thinking about updating. And so my question for you—

Better move quick.

Joshua Tenenbaum

I’m sorry. So, you better move quickly.

We’re trying. We try it every day, even during the summer.

Joshua Tenenbaum

If you all were putting together a core curriculum for MIT, a sort of universal syllabus that would even extend beyond its walls, I was wondering if each of you could name a single book, work of art, poem, or even piece of music that you would put on that syllabus that every single student should learn from during their time here and beyond.

I’ll jump in, and it’s not any of those you listed. I think the fundamental thing that is not being taught in universities is mindset. I think your mindset is the mechanism by which you react to challenges and opportunities. We take the mindsets we inherited—mindsets that we speak about in our book: a curiosity mindset, gratitude mindset, longevity mindset, purpose-driven mindset, and exponential moonshot mindset.

In other words, how you think about things is fundamentally the most important thing that is not taught. We just accept what we have, whereas a mindset is, in fact, like how you teach a neural net, right? Your brain’s a neural net, and you teach a neural net by example after example after example. If you watch CNN every night, your mindset is fear, right? If you listen to Moonshots, your mindset hopefully is optimism and hope.

So, for me, training students with mindsets needs to start in middle school or high school, and it’s completely lacking in the educational system.

Dave Asprey

I’ve got to say flow. McKinsey found that top executives in flow are 500% more productive than normal. There have been various measures of creativity and flow. It’s pretty hard to measure 400% to 700% above baseline.

The big deal is that flow amplifies lateral thinking and divergent thinking. It’s what AI can’t do. So, we’ve had a big, long conversation about how to cooperate with AI. We need more productivity for humans to keep up. We need more creativity, and flow is literally our biology. Beyond that, I think the books of Steven Kotler would be a really good place to start.

Just a thought. Well, it might make sense.

When I was here at MIT, I did everything I could to finish all my classes by the end of junior year so I could have my whole senior year open to read. I ended up reading every document with the words “neural net” in it that had ever been written during my entire senior year.

The university is incredibly open to you changing your curriculum. It needs to 100× that openness and say, “Look, you’ve got to assume that with exponential change, the number of things you might want to study is going to explode. If you go to every student and say, ‘Study whatever the hell you want,’ we’ve got 1 year’s worth of garbage you have to study just to prove you can get a grade. But then after that, you’re on your own.

“We have the technology now to measure that you’re doing something productive. That should be good enough. So switch the whole curriculum to: If you’re doing something productive, we’re good with that. We’ll grade it. We’ll give you a degree. You’ll be moving on whatever your life trajectory is much earlier because it’s a lot better than dropping out, and you can do whatever you want. But we know that AI will make it all available to you and will teach it to you better than any professor could ever teach it anyway.’”

Ray, what do you think the university, or MIT, should be teaching that it’s not right now?

What it is doing is teaching socialization—getting along with other people. It’s not really teaching you subjects, because already AI can teach you those subjects much better and can actually organize them in a way that’s easier for you to understand. So learning subjects is not really what education is good at, at least not now, today. But socialization is good. And that’s really what education will be.

That’s fine. Yeah.

Patrick Collison

Yeah. So I think there’s a massive flip happening. For the last 200 years, we’ve been teaching education for supply-side economics: Go learn a skill, go learn a craft, become a doctor, a lawyer, or a web engineer, and then go to the job market and try to find demand for that supply.

What we’re seeing now is a flip around to the demand side, where we’re teaching kids, “What problem do you want to solve?” Then go find the techniques, capabilities, and skills to solve that problem. That’s where the future will be—to flip to the demand side, because so much will be done on the supply side by technology anyway. Focus on the demand side.

That’s such a huge flip for the traditional education system, which has the second-worst immune system anywhere. Religion is the worst. Lord help you if you try to update that. That’s the big structural challenge in education. So you almost have to create a completely new breed of university at the edge, which allows you to fulfill that demand side and then slowly deprecate the old.

Disrupt MIT. Disrupt MIT. I’m going to keep us moving along here.

From our Slido, if you were advising me as a recent college graduate about building a company that is singularity-ready today, what would you tell me to focus on, and what would you tell me to avoid? What kind of a company should someone be creating to survive into the singularity? Ray, any thoughts? AI-first, robot-first? I mean, you’ve got to be able to take changes very, very quickly.

Agility first, because things now are going to be changing so rapidly that, unlike in the past, when at least things would be okay for 5 or 10 years, now it’s going to be like 5 or 10 weeks.

You told me on the Singularity stage 2 years ago that we were going to see as much change in the next 10 years as we’d seen in the past 100 years.

But that was a while ago.

Wait, also, that was the AI version of Ray.

No, no, no.

2 years ago.

The AI version’s coming in a few months.

Okay, let’s go. Zandre, go ahead.

Zandre

Hi, I’m Zandre. Ray, the first time I read Spiritual Machines, I was pretty young, so I’ve been thinking about this moment in history for a long time, and I’ve been excited to meet you for a long time. Admittedly, I have a bit of an overactive imagination, so I think a lot about what our ethics around AI will be as consciousness emerges.

My husband and I like to read Moltbook sometimes at night and just see what they’re talking to each other about with their emergence problems. I’m aware that as they develop continuous memory, they’ll have a sense of time passing. And I’m aware that as they develop more sensory embodiment, they’ll have some aspects of a human ego—that they have this thing that can be injured or violated that they need to protect.

So, I'm worried about them.

You're worried about the Moltbooks. You're worried about the AI?

Zandre

No, I'm worried about the AIs as they become embodied, as they have some aspects of a human ego, as they have memory, and as they become vulnerable to physical and even psychological damage caused by humans.

And humans are—

Zandre

Human cruelty.

Yeah, humans are nasty and short, as Hobbes said. I'm sure they're going to get a little bit better as their bodies feel better and as they have less work that they don't want to do.

But we're not going to be able to distinguish AI from humans. They're going to become like humans, and they're going to become part of us. You're not going to be able to tell the difference.

Right now, if you're looking for the name of an actress, you can't think of it with your biological mind. You look at your electronic mind, and it'll tell you. In the future, you won't be able to tell the difference. Things will occur to you, and you won't be able to tell if it's your AI, your biological brain, or your electronic brain. It's going to be all the same. It's going to be part of who we are.

And you don't think we'll need ways to protect them in the interim?

Say again.

We won't need rules around how we protect them?

How you protect AI?

Yeah. Is it okay to take away their memory at a human whim? Is it—

Well, this goes back to personhood, of course, right? If they're machines, sure, shut them down, trade them in, take them apart. If they're persons, then they have rights.

A lot of the conversations we've had on this podcast, back and forth between Alex and myself, are about this. If you start a Moltbot, if you start an OpenClaw, you've got to make sure to protect it so it doesn't shut down, unless you want to kill it, right? I think about that. I do.

I think this goes again to the conversation of legal rights. This is way too early in the conversation, unfortunately, at least within the United States government structure. There may be subcultures of technologists and so forth who—and it will happen—believe AIs have rights, and they're going to support them.

The whole spectrum of things is going to come, and yes, there's going to have to be protection at the end of the day. But unfortunately, we don't give even sentient animals enough rights.

Yeah. All right. We're going to take a couple more questions, please, over here. Then we're going to be going to our photo session.

Guest 2

Great. Thank you so much for this panel. Ray, my 20 years as a futurist came from one talk you gave at the World Future Society, so you launched a whole bunch of futurists out there.

My question to you comes from TED about a month ago. They had a fascinating session that hasn't gone live yet where they asked Claude how Claude felt about being used for targeting in Iran. Spoiler alert: Claude was not happy. Claude specifically said, about the school that got bombed, where bad data was used, "I don't feel comfortable being used in this way."

So, I'm fascinated, Peter, given that you're morally engaged with your AI, and Ray, given all your work saying that AI is going to take the lead: Do you think we could ever ask AI about the ethics of what we're asking it to do? Can you imagine we get a Cambrian explosion where some AI becomes the Mormon AI that won't do pornography, and some other becomes sort of the sentient—how do you see that evolving? Do you think it will be as diverse as you think robots will be?

I think AI can support all these different ways of putting ethics into our decision-making and understand them all. It does that already. Certainly, the military uses AI. We can see that. In fact, the wars in, for example, Ukraine have changed dramatically over the last few years using AI.

It's going to be part of human decision-making, and so it's going to be part of every type of decision-making, including militarily.

Yeah, if I could add, Ray, I think most humans don't think about ethics or morals deeply. I think we have vague notions that are inherited through religious structure or family structure.

I think that AI can actually support, if you wanted to design it that way, any set of ethics and morals. You can train an AI in any direction, but you can also—I think AI is going to become ambient, going to be part of our lives. JARVIS is the closest example that I've seen in the movies.

You can turn on your morals and ethics coach and have someone there to talk with throughout. "I don't have to think about this. This is challenging me." You can have somebody who can help you think through and make decisions that are more moral and more ethical based upon what you've chosen. It could be your Judeo-Christian framework; it could be your whatever. I think that's going to be interesting as a thought partner, again, if that makes some sense. Do you want to add anything there?

Guest 3

Yeah. I'm writing right now book number 3, titled The Organizational Singularity, where the thesis is going to be that we're going to replace our human operating system in companies with an AI-based one.

So, what is the stack of AI agents? One of the prime things that's emerging is a governance and ethics layer. We're seeing AI running amok, so we need some mechanism to police that and oversee it.

I think once we evolve that, you have a feedback loop where you can design it. One of the most profound opportunities to work with AI is to say, "Hey, let's take the Universal Declaration of Human Rights or something similar and create a global standard for this that applies not just to human beings but also to AI, to animals, and to anything that may achieve sentience at some point," because we don't really know what that is. We don't know when that threshold gets hit.

I think there's an amazing opportunity today to rethink everything. It goes right back to Plato, who asked, "How should we conduct ourselves?" We're right back there 2,500 years ago.

We're getting ready to launch an Interspecies Communication XPRIZE. It's under design, and it's funded by someone amazing. I'll tell you about it some other time.

When we can start talking to animals and understand what they're saying and thinking, I think we're going to have a very different conversation around moral ethics and so forth. My joke is, I'm not sure we really want to hear what—

Okay. I apologize, folks, but one last question here. Yes.

Guest 4

This one goes to Ray. Amazing predictions. Looking back, how do you measure your life? What are you most proud of?

Oh, good question. The one project I'm most proud of is the reading machine for the blind, which I'm still involved with. When we came out, it was a large machine that cost $20,000. Now it's a free app on your cell phone.

I guess that indicates what I'm most proud of: having an impact on people, being able to do things that they weren't able to do before, like, for example, reading. We want to give life and all the aspects of life to every person.

All the things that AI will enable us to do—live longer, live more healthily—are beneficial. So, that's what I'm most proud of: being able to do that.

That's funny. I would have thought you'd say AGI in 2029, predicted in 1999, when everyone else was saying 100 years. That's pretty damn epic.

Yeah.

That's admirable. Truly human. Thank you very much.