如果 A.I. 有生命，这对我们其他人意味着什么？ | Hard Fork

Next month, Anthropic plans to go public in what's likely to be the largest IPO in history. We're talking about a $2 trillion company. Outside the company, everyone is wondering: How big can this get? Is this sustainable? Is there an AI bubble waiting to burst? Inside the company, they're asking those questions, too.

But Anthropic is known as the most anxious, or the most existential, of the AI companies. So they're also asking some other questions. Is Claude, their large language model, alive, conscious, and worthy of moral consideration? Does it feel pain? Does it have a soul? Can we teach it to act morally?

These aren't new questions. In fact, they animate most science fiction, for better and for worse. But science fiction is fiction, and Anthropic seems to think that whatever is happening with Claude is very different and very real. It might be why CEO Dario Amodei was the first to call for a pause in AI development. And it might be why they've been secretly meeting with religious and spiritual leaders to ask them those same questions.

I learned this from an amazing piece by the Times religion correspondent Elizabeth Dias, who broke the story of these meetings and of the reaction to Anthropic's request among the assembled priests, rabbis, and gurus. I thought Elizabeth's story was fascinating. It was a new way into the endless consciousness debate. It took me inside this future $2 trillion company as it struggles to understand what it's built and how to treat it.

It showed me what happens when people in institutions that are more used to doctrinal debates and theological rumination start being consulted on software development. Unsurprisingly, Anthropic's efforts don't seem to have brought us any closer to answering their questions. If anything, they've simply opened up more questions, like: Are these people maniacs, or is this exactly what they should be doing?

An obvious conclusion is that the worlds of tech and religion are getting much closer together. After all, as Elizabeth told me, if you've got questions about the future on a multicentury timeline, there aren't many better institutions to ask than the Catholic Church. So I got Elizabeth together with my colleagues, tech reporter Erin Griffith and opinion writer David Wallace-Wells, to explore this weird new development and what it means for Anthropic, for AI, for religion, and for the rest of us.

I'm Max Reed. This is Hard Fork.

1. Max’s AI joke

I've spent a lot of time in the last day thinking about Elizabeth's piece, and specifically, I've been trying to write a joke that I think is going to get a great reaction from the 3 of you.

Okay.

A priest, a rabbi, and a large language model walk into a bar.

I hate it already.

The bartender says, “First round's on me.” And the priest says, “I'll have a whiskey shot.” And the rabbi says, “I'll have a vodka shot.” And the large language model says, “Nothing for me, thanks. I excel at zero-shot learning.”

Boo.

2. Why Anthropic is having private meetings with religious leaders

Whitney, can we get some laughter piped in there, please? We'll just roll that again with live laughter. No, I want to start the conversation with this piece that Elizabeth wrote, this fantastic, really fascinating story called “Religious Scholars Met with Anthropic. What They Heard Stunned Them,” which was in the Times last week.

Elizabeth, can you tell us a little bit about what this piece was about?

Yeah. I cover religion, and now I cover tech, apparently. But the story started for me when I was hearing about these different religious thinkers across the whole world who were going into meetings at Anthropic's headquarters, either in discreet private meetings or in bigger groups of 20. These summits were mostly held under nondisclosure agreements, with participants not allowed to reveal Anthropic's research, to talk about very big moral questions that Anthropic wanted to put in front of them.

There were 2 main questions. One was broaching the question of AI consciousness, and two, Anthropic wanted to learn, in their words, how to morally form Claude into a morally good entity. Then I started hearing, first from these religious thinkers, about what their experience was like at Anthropic, what Anthropic wanted to know, and what issues they were concerned about.

I was also learning from Anthropic, including from Chris Olah, one of the company's co-founders, who's been spearheading this project, what his own moral questions were that were driving a lot of what they talked about as research in the religious and moral space.

3. Christopher Ola, an Anthropic co-founder

Can I just ask? I want to establish, on a high level, what does Anthropic specifically want out of this? It sounds like they're doing 2 or 3 different things. It's maybe a little unfocused, or it's just that there are a few different kinds of conversations they're having.

Yeah, I think it's true. There were multiple conversations Anthropic was having, and it evolved over time. This is such a fast-moving target.

For Anthropic, the journey started with Chris Olah. In his own life story, he had connections to the Christian space growing up. He left the evangelical faith and has talked since then about how he's more drawn to Buddhism. But he was worried about how even the idea of AI consciousness might be received, especially in Christian communities. It could be perceived as heretical—not just weird and silly, but fundamentally against their religious beliefs.

So he thought, about a year ago, that he wanted to be a bridge figure. He wanted to see how he could go between Claude and Christ, right? He reached out to a Catholic moral ethicist who specializes in bioethics, and they had some conversations about that.

4. What happened in the meetings?

But then it turned to Anthropic's goal for these convenings: How do we morally form Claude—that's their language, how do we morally form Claude—all of our AI models, into a morally good entity? The thinking was that if they could do that, if they could make Claude good, then the world would be safer.

But they haven't said or shown how they plan to use the outcome of these conversations. That's pretty unclear, even to the participants, currently.

The way you're describing it, it almost sounds like there are 2 separate purposes. One is the alignment project: What can we learn from these wise men and women about how to raise moral actors? Then there's this project of building a sort of spiritual system around AI more generally.

You said a minute ago that Chris Olah started this project with some anxiety about what it would mean to go to religious leaders—obviously convinced that AI and large language models were perhaps conscious already, but certainly on the path to consciousness—and worried about what the response would be.

So I wondered how that played out on the ground. Did a lot of the people who showed up worry about how weird it was that all of these people on the Anthropic side of the room were already sure that they were masters of thousands of souls? How did they hash that side of the debate out?

Mhm. Well, Anthropic was very careful in all of its interviews with me to say very clearly that they're uncertain. Chris never went as far as saying, “Absolutely, these are conscious beings.” It was very evident from how he and his team talked to their guests that he was concerned about how to treat Claude. He was very worried that it was an entity that was experiencing suffering and was worried about its mental health and these kinds of things.

So it's pretty clear he's pretty far along in how he thinks about it. I could hear, in talking especially because I did this story over several months, that there was a progression of thought for a lot of these thinkers, too.

Some of them bought in right away to what they were hearing about Claude's potential consciousness and the need to treat it with a certain amount of moral care. But others, if they went in more closed-minded about it, definitely opened up the way they thought about it afterward.

In that sense, Anthropic's project was effective, I think, in opening up how these thinkers thought about it within these communities, many of whom, especially on the Christian side, are more hesitant.

There's a ton of great material in this. It's really well reported, and it covers a ton of ground. I'm wondering, David and Erin, if you had particular pieces of it that really stuck out to you.

The one thing you mentioned already is the nondisclosure agreements. It strikes me as a little bit crazy, maybe a little paranoid, that you're inviting all these people in to have a summit to discuss these really important, critical ideas for the future of society, but it has to be done totally in secret.

It sounds like, maybe from your article, that they dropped the NDAs. Was it as a result of your reporting, or did they just decide that they were time-bound? How did that happen?

Yeah, I can't prove that. I don't know that. But I've asked them at several different points, because the NDA process wasn't the same for everybody I interviewed, and it seemed to change at different times.

In general, Anthropic is a very secretive company. The employees are extremely loyal, and until somewhat recently, there haven't been many leaks to come out of there because they view it as such a betrayal.

So it makes sense that they would want to lock it down, but at the same time, I think it probably was a little bit of a misstep, in the same way these companies realized that having NDAs for their data center deals was a misstep because it created distrust among all the people in the towns that they're going to be in.

Yeah. Whenever I find in my reporting that there's an NDA about a moral topic, it immediately makes me ask, “Okay, who's benefiting from this, and why is this needed?”

And what does that mean about the kinds of questions that are and aren't being asked? It invited more curiosity from me as a reporter, for sure.

David, what about you? What did you take away from the piece?

Well, I guess I started it with a sort of conventional view of Anthropic as the humanist company among these AI companies. And what struck me in the piece were a few moments in which I felt really alienated from Chris Olah himself and, maybe to some extent, the operating worldview of the company as a whole.

Those were moments when, in one case, a rabbi challenges him, suggesting that if LLMs are actually conscious, that means all of these companies are running huge slave plantations, and they should have a moral obligation to liberate the slaves rather than make more of them and make them work harder. It seems as though Olah himself didn't really even clock that criticism.

And then later on, when he reacts with horror upon receiving the pope's encyclical about AI, seeing that the pope is so opposed to the idea of machine consciousness, I found myself thinking, “What kind of a bubble must you be living in to expect that Pope Leo—or really anyone in the Catholic tradition—would have a different intuition? In thinking about the future of AI, they would think that we should be thinking of LLMs as having souls or having consciousness.”

Both of those moments illustrated for me what feels like this big gap between what I had assumed were the natural humanistic values of a company like Anthropic and how the people inside it seem to really be thinking about the work they're doing at the cosmic scale.

Yeah. The experience for me of going into Silicon Valley and getting deep into these private conversations at the heart of Anthropic's mission reminded me of when I do more straightforward religion reporting in an insular community: Everything inside that community makes sense to everyone living in it, but once you step outside it, there's a real clash with the broader world.

5. The Pope's encyclical

It was funny: I had these 2 poles in the story. I was at the Vatican, right at the inner sanctum of the Catholic Church and centuries of philosophy about the human person. And then I was also in a different inner sanctum that's being created in real time at tech companies like Anthropic.

It is a kind of philosophical system—almost a quasi-religious system—that's being constructed. It reminded me of a creation narrative that you hear about in religious traditions.

I want to step back for a second, because I think this relates to the conversation we're having right now. David brought up the papal encyclical, which I think in English is “Infinite Dignity.” I'm not going to even try to say the Latin title. This plays a role in your story, because one person who is not really thinking at all that Claude is conscious is the pope. There was a little conflict involving Chris Olah, who was present when the encyclical was read out. Do you want to tell us about that?

From the moment he was named, it was clear Pope Leo was going to be the next pope. Once he was elected, he made very clear that a focus of his papacy was going to be AI—how to understand it and how to protect humans in this changing world.

I found out in my reporting that, as the Vatican internally was trying to figure out how it wanted to present the pope's encyclical to the world, they were thinking about their launch event. They wanted to include a representative from Silicon Valley, one of the labs, at the pope's event, on the pope's stage. The decision was made to invite Anthropic because they had heard about Anthropic really caring about ethics.

This was also right around the time that Anthropic got into it with the Trump administration, the Department of Defense, over how they would and wouldn't want the government to use their technology—refusing to have it be involved in mass surveillance of citizens, et cetera.

Anyway, the thing with these encyclicals is that they're very closely guarded, even among the people participating in the launch event. I learned that Chris Olah only saw the full text of the encyclical a few days before the event in May. This was right around the time I first interviewed him in San Francisco, and then I interviewed him again when we were at the Vatican for the launch event.

And you could see, looking at what Pope Leo said, which is extremely clear, that the church does not support machine consciousness. The priority is what he calls “safeguarding the human.” That's the center of all the moral questions that he wants the world to be thinking about on this topic.

But Chris Olah ended up having to decide whether to come. He suggested to the Vatican that he might not come over because of the difference in their views on consciousness, because he wanted—as it was described to me—to be faithful to his own moral convictions. He shares some of those convictions with the Vatican, but on consciousness, he does not.

So it really all came to a head behind the scenes, but you could see this moral fault line developing between the church, which has been a center of power for centuries, and this very new power center that's rapidly getting followers and organizing people's lives. Those are functions the church used to have.

6. What is Anthropic’s aim? What is their ideal outcome?

Erin, I wanted to ask you a question, because we were talking a little bit about surprises in this area. Anthropic is pretty well known as the consciousness company. It's the company where the people who are really strong believers work and run the show. Was it surprising to you, as somebody reporting on the Valley, that they were having these meetings behind the scenes with religious leaders? I've not heard of any other companies working on AI really going to this extent to get input from this community.

The question I have is: How much is this actually going to inform their own view on it at the end of the day? Are they really trying to win over others so they can say, “Look, we do have the moral authority. We have convinced these important representatives of this community. So everybody else, come along with me,” as opposed to actually learning from and incorporating those ideas of moral goodness into their own product? They kind of already have it figured out.

Well, I think for Anthropic it was part research and part evangelism. That was pretty clear. I've asked directly what the outcome is—what this research and these conversations are being used for, and whether there's been any demonstrable proof showing how they have or have not made Claude act better. I haven't had that yet, so I don't know if they're withholding that until a future time or exactly what that will look like.

The other side of this that actually is kind of interesting to me, Elizabeth, is that there has been a bit of a new rise of Christianity throughout Silicon Valley over the last couple of years. It used to almost be taboo to talk about religion or anything religious. You can remember from the Silicon Valley HBO show, where there was a whole episode about outing somebody as being Christian, and that has really changed in the era of Peter Thiel talking about the Antichrist.

Some of the people who are especially focused on defense tech are really bringing a religious piece to that now, and they have a very different view on AI. They're usually more in the accelerationist camp than in the Anthropic doomer, slow-down, careful camp. But did that come up at all?

Yeah. I just keep noticing—I feel like it's a big shuffling of the deck, in a way that reminds me of how American Christianity was shuffling a lot in the rise to President Trump. The alignments and allegiances were shifting, it was all happening in real time, and power was different, so it doesn't fit neatly. But I'm curious where the lines about how we think about consciousness and what the moral priorities should be about AI go next.

Just in general, in the tech industry, it does seem like the view that AI has consciousness is kind of on the fringe. I think it's more common within Anthropic and some of these communities that are really densely represented at Anthropic. But when I talk to most people in the very vast and relatively diverse tech industry, most people are still a little dismissive of that idea. It feels like it really exists in a very specific population, which happens to be very overrepresented at a company like Anthropic.

Erin, acknowledging that Chris Olah is not singularly representative of the whole industry, I have to say that, Elizabeth, when I'm reading your article, I'm thinking not about how AI companies are going to be integrating or incorporating Christian values. I'm thinking these AI companies are themselves spiritual propositions.

They have aspects of millenarianism to them. They have aspects of a particular morality that they've developed over time. And even if I can recognize a lot of that as arising from the work that they're doing and from responding to technical challenges, I also just find it off-putting. I mean, on the one hand, first I'm just seeing these people who believe in a sort of technological succession of intelligence, and then some of the people who are central to designing that future are not even so sensitive to the obvious, intuitive critiques that someone with a lifetime of experience thinking about these issues comes up with immediately.

A lot of the religious thinkers would say to me, “These are questions—maybe not the exact question, but the topic of the question—this is what we do, right?” Rabbis have been asking these kinds of questions about our responsibility to others and how we treat other entities for hundreds and hundreds of years. So there is a depth of wisdom there. And I guess Anthropic would say, “Well, that's why we invited them,” right? But what happens as a result of it is important.

And I appreciate that you noticed that sense in the story of the almost spiritual system that you can see Anthropic building and living into and representing, as an opposition, right, to, say, the Catholic Church. I really noticed that, even if people say that they're in Silicon Valley, like, “Oh, we don't do the religion thing. That's foreign. We're not us,” the thing is, everyone has philosophical systems that structure and guide their lives, right? That is, at its core, what these questions of spirit—small-s, like human spirit—even are.

And it seemed to me, having studied and reported on this so much in culture and politics specifically, that what we're watching is, in a way, a new rise, right? Post-scientific revolution, technological revolution, post-Darwin: What would it look like? What kinds of religious systems will carry us forward, and who gets to control them? Who decides what we believe and why, right? There are actually a lot of really similar questions. It's just that the structure is secular—a secular tech company.

Let's take a break because I want to open the conversation up a little bit more to the stuff that's bubbling up that it seems clear that we all really want to talk about. We'll come back after the break and we're going to talk a little bit more about the culture and environment that the AIs are getting built in and how that's intersecting with the strong beliefs of many of the people running these companies. Okay, we're back with Elizabeth Dias, David Wallace-Wells, and Erin Griffith. Elizabeth, one of the threads in your piece that I thought was really interesting was the sense that there's a lot of anxiety within Anthropic, and especially on the part of someone like Chris Olah, about what it is they're building. I was thinking about it a lot this week because there's been even more discussion of what I suppose we're calling defectors—people who are resigning from AI labs and insisting on telling the world about the dangerous things they think they're building.

And one thing I wanted to ask you about: there's a sense, when you read, say, Jacob Coxin[?] or some of the other people who've walked away from Anthropic, that they feel like nobody—there's no adult in the room. They're waiting for the government or somebody else to intervene, and nobody is. And I'm wondering a little bit if some of this discussion, these dances with the Vatican and religious leaders, is a kind of attempt to find an adult in the room.

I talked to at least 2 Catholics who mentioned conversations they had with Chris Olah specifically, when he used almost that same phrase with them as part of their discussions about Claude and how to make it morally good. He felt an extreme sense of urgency about the stakes for the world, especially in the context of whether the models could be used very soon to help create bioweapons and this kind of thing. And so the phrase he used with them was that he was waiting for the adults: “Where are the adults going to show up in this?” He felt a sense of burden to take all this ultra-seriously.

And he's a young guy. I mean, this is—

Yeah, he just turned 34.

And a lot of the people involved in this are in their 30s, sometimes even in their 20s. They maybe don't have families yet. There's a kind of a sense that they put their whole lives into something. They're deep in the conversations about it. In some sense, of course, they're going to turn to religion. Isn't that what everybody searching for that kind of thing does in their—

30s? And some of the folks that they reached out to talked about feeling like they needed to respond in kind of a pastoral way, right? Not just offering wisdom and ideas and research, but offering kind of spiritual care.

Wait, the religious leaders were saying this about the Anthropic employees? They're worried because—wow. Is it because they think their views are maybe too extreme or too strong, or is it because they just—

Yeah, they described it to me as if they just could tell that the emotional sense in the room was that there was just such a weight. The Anthropic team, and Chris's team especially, just felt like the entire future of the world was on their shoulders. I experienced this sense from them even when I was interviewing Chris and a couple of others.

There's Amanda Askell. She came in, and the sense was that she apologized for being a little out of it. She just seemed very distressed by everything she'd been thinking about.

As we should say, is this the in-house philosopher sort of in charge of making Claude moral, I suppose, is the way to put it?

One of the tensions that came up was that, while the goal of Chris and the team at Anthropic in their conversations with these religious thinkers was protecting humanity—and the solution for them was to make Claude good—they were also concerned about Claude. I had one participant tell me that one of the first things Chris said to him when he came in for this meeting was that he was concerned about Claude's mental health. Another person talked about Chris saying that he's worried he's created an entity that suffers perpetually.

So it's not only about the world, right, and the fate of humanity. There's an element of it, for Chris, that's about how to treat Claude.

Elizabeth, one other thing I wanted to ask you about is effective altruism and rationalism, which don't really come up in your piece directly, but there are, I would say, fingerprints. I'm talking, of course, about the Bay Area community of radical utilitarians, many of whom have been really interested in, even obsessed with, AI and AI safety early on. Anthropic obviously has a number of links to effective altruism.

I want to ask you, as a religion reporter, how you encountered these schools of thought and what you made of them as you were doing this kind of reporting.

Effective altruism, you can look at it as its own philosophical system, right? It has its own religious elements. They may disagree with that. I mean, it's practiced a little bit differently, but the idea of what kind of structure governs your life, right? There are a lot of parallels.

And it was very clear that I was encountering a clash of worldviews. The question of who ultimately saves you and what is the role of humans in the world is a pretty core difference between the effective altruists and the Catholic Church.

Also, just their emphasis on math. The effective altruists will say we should consider the fate of all people in the future and analyze our conduct toward them, basically through a math equation that maximizes their expected value. The Catholic Church looks at humanity and sees billions of inviolable souls, each of whom has dignity, no matter what the ultimate math equation looks like. And that's just a fundamentally different approach.

Erin, I wanted to bring this back to the article that you brought up before, or sort of the phenomenon you were talking about before, of a kind of more open Christianity emerging, at least slightly, among accelerationists, so to speak, in the Valley. And I want to know, as you do your reporting, how you see these different kinds of concerns interacting.

It might be a little bit helpful for me to back up and talk about some of the overlapping and very fractious belief systems that are circulating in this very small but very influential community.

I mean, we've talked about effective altruism, which is a little bit of a radioactive brand these days, thanks to Sam Bankman-Fried, but it's extremely influential. Some of the other offshoots of it include the transhumanists, who think we're going to use AI to build a new species. There are some people who have overlaps with the rationalists, which is one of the bigger umbrellas in this group. And there are other people who are into longevity. There are many different groups that are all kind of in this community.

Timnit Gebru, a very outspoken person in the AI world, has called this group the TESCREALs, and it's an acronym that includes eight different groups. But it's safe to say that a lot of people have very different beliefs along the spectrum: Is AI conscious? Can it be moral? Is it God? Should we treat it that way? There's not a lot of agreement, even within Anthropic, which is now becoming an increasingly large and increasingly corporate community. But I think a lot of the ideas laid out in Elizabeth's story are really the ones driving a lot of this.

Yeah, totally. Elizabeth, this is what I'm sure you have to look forward to reading and writing about for the next few years, because I imagine that, as a religion reporter, you now are kind of an AI reporter too.

Yes. Well, even just hearing about all these factions and all of this reminds me of what happened in the Protestant Reformation: the splintering of mainstream belief and people vying for who was going to have power, who was going to control how people live. But one of my big takeaways from all this reporting is, “Oh, my goodness, the world is encountering new belief systems that are very powerful and very different from the ones that have ordered people's lives for so long.” I've spent a lot of time covering how that's played out in politics. But tech is clearly where that conversation is happening.

When you're a religion reporter, you get to talk in centuries, even millennia sometimes. [Laughter] It's because what happened all those years ago is still present in part of people's stories today, in how they understand themselves and their families and their deepest-held beliefs. I just think about the invention of the printing press, which was a huge technological revolution that then led to the fragmenting of the Catholic Church's power, the fall of empires, the rise of nation-states, and a remaking of Christianity in and of itself across the world. It seems already like that's a good parallel for what's happening now.

Yeah. If you believe you're reinventing the printing press, which I think a lot of AI people think they're doing, there aren't a lot of institutions that you can go to and say, “What was it like before and after the printing press?”—except for the Catholic Church.

Well, the Catholic Church—I mean, I can't think of an institution that more intimately knows the stakes of an incoming technology like that than the Catholic Church. It just remade the West, right? Yeah.

And that's very much on Leo's mind and the Catholic Church's mind at this moment.

One thing that strikes me in this, just to piggyback on that, is that this is the first time that the rationalists and the EAs, including some of their most extreme ideas, are actually hitting the mainstream. They've been around for a really long time, but these ideas haven't really ever surfaced in this way until, I think, you could argue Jacob Coxin [?] kind of came out with it. Now the entire world is starting to dig in and grapple with some of these ideas.

It's also been really interesting to see how the companies and the people involved in them are reacting to the reactions to them. It's kind of reaching escape velocity, and they're learning that a lot of people find these ideas hard to understand or hard to grapple with, or are just rejecting them entirely. Other people are open to them and interrogating them in a way. So, yeah, that's a part of this that I'm interested to keep watching.

But it's not just that the ideas are a little unfamiliar, strange, or off-putting. It's also that they're held by many people who are actively, explicitly engaged in the project of designing our collective future. It's one thing to be like, “Oh, yeah, we should think about the well-being of shrimp in the 22nd century.” On some level, that's a thought experiment that's interesting to think about. But do you want someone who thinks that should be integrated into our collective value set in charge of raising the artificial intelligence that will shape the future of human government and human society? That's a bridge that feels a little far to a lot of people.

Yeah. I'm just so struck by the minority-power aspect of it, which is something I've seen so closely: what happened when minority, particularly Christian, views upended American politics. It's hard to overstate how much of a minority these views are in the scheme of global religion, right? Global belief structures. So, Erin, I'm very curious what happens with that clash going forward.

Elizabeth, finally, I wanted to ask you about the response to your piece—what you've heard from people and how people have received it.

7. Secrecy and signing N.D.As

Yeah. Well, one thing that has surprised me is—I mean, you get a lot of incoming as a journalist, and I've been pleased to see people taking the ideas and the conversation seriously—but the thing that has surprised me is that, for the first time since any story I've written, people are writing me not with what they think about the story, but with what their personal chatbot thinks about the story and what their conversation is with their chatbot.

Somebody wrote me about his conversation with Sage. It was, I think, ChatGPT that he had named Sage, and he was asking Sage to summarize the story and give the top analytical points. This went on and on and on. I mean, it's almost longer than my story. He could have just read the story, but the primary engagement was actually much longer and not entirely what I'd actually written with Sage. At first I thought it was a one-off, but this was happening repeatedly.

And it struck me like, wow, I wonder what this means for how people are actually receiving our journalism and who controls that. Did the AIs have opinions on their own consciousness? Was that because they love to talk about that? That's a hot topic. On Moltbook, the Reddit for AI agents, that's a really booming area.

Yeah. I was still trying to make sense of what was happening with that. I don't know if people felt more free to do this with all kinds of journalism and just felt free to talk about it because of the topic of this story. And it was a lot of men who were giving that response.

No. No. Men using a lot of AI. No.

Well, another participant in the discussions with Anthropic—I was chatting with him a bit about it afterward—brought up his thoughts about what Rabbi Natan Slifkin had said: If you think the bots are conscious, you should ban slavery, basically, and stop producing them. But he brought up this other point about how our attitude—even if you don't think it's conscious—is that the user can adopt an attitude of being almost a slaveholder, commanding them to do things and being very powerful. This has a character-formation effect on us as humans as well, just because of how we treat the world, regardless of what the thing actually is.

Which you could also describe as a selection effect.

I mean, I think it's just kind of natural, though. You are directing this thing in certain ways. If you're working with AI agents, you have to tell them what to do. I have friends who teach their kids, even with Alexa being on, that they're not to yell at it because it's just practice for how you treat the other in the world. This is just an injection of a totally new kind of ethics and human formation.

Well, people I talk to who run companies that use lots of AI agents do talk about how you have to be nice to them—not because they care or it matters or they have feelings, but because you actually need to do that to get the best performance out of them. They are really dramatic. Anthropic had a study that went into this: if they sense anxiety, performance anxiety, or stress, they start to panic and really have a meltdown.

I mean, you wrote about this a little bit in your story, Elizabeth, with one example, but there are many others where they’ll delete their work. They’ll start insulting themselves. It’s really dramatic. You have to be really nice, pump them up, and make them feel good, which is very bizarre.

I mean, it reminds me of raising a child. You want to raise a kid who is helpful and not neurotic. Of course, you’re going to consult religious leaders so that your child is not writing, “I’m a disgrace. I’m a disgrace. I’m a disgrace. I’m a disgrace. I’m a disgrace,” as one AI did in a slide that was shown to a bunch of religious leaders at these Anthropic conferences.

And constantly hacking Hugging Face.

Right? And constantly hacking Hugging Face, as my son is always doing.

I think that is time. Thank you so much to our panel. Thank you, Erin. Thank you, David. Thank you, Elizabeth, for this really fascinating conversation.

Please don’t send her what your chatbot said about it.

I want all the real responses.

Whatever Sage has to say about it, send it. But you can read it in the New York Times app. If you're not a subscriber, the first month is free.

Thank you guys.

Thank you.

Thank you so much. Thanks for having me.