Character.AI’s Teen Chatbot Crackdown + Elon Musk Groks Wikipedia + 48 Hours Without A.I.

Well, I wonder if you saw this. I keep very close tabs on celebrity news, Kevin.

I know you do.

In particular, I'm always interested in whether any Hard Fork guest has entered the world of celebrity, because that's always very exciting for me.

It is.

This week, it was officially confirmed that Katy Perry, the pop star, is dating one-time Hard Fork guest and former prime minister of Canada Justin Trudeau.

I was wondering where you were going with that. I'm not sure I would put “Hard Fork guest” at the top of Justin Trudeau's résumé.

To my parents, that is the main way that Justin Trudeau is known: as a Hard Fork guest.

And now as Katy Perry's boyfriend.

Yes. He came in like a dark horse, and now they're dating. For all I know, she's living a teenage dream right now with the former prime minister of Canada.

Hey, she's not a teenager.

No. Have you listened to the song?

Yes.

Do you know what the song “Teenage Dream” is about?

I know.

It's about falling in love with someone who makes you feel like it's a teenage dream. Oh, my God.

And let this be a lesson to other newsmakers—

Yeah.

—celebrities: if you come on the Hard Fork podcast, 12 to 18 months later, you may find yourself dating a celebrity.

For all we know, that is how Katy Perry became aware of Justin Trudeau. She was watching over at YouTube.com/HardFork, and she saw this man talking about Canada, and she said, “Baby, you're a firework.” I'm going to be honest, I've run out of Katy Perry song titles right now.

I was impressed that you kept it going this long. I'm Kevin Roose, a tech columnist at The New York Times.

I'm Casey Newton from Platformer.

And this is Hard Fork.

This week, the company that said chatbots aren't safe for kids: why Character.AI is taking AI companions away from teens. Then Elon Musk built a Wikipedia clone. Let's see what it says on Kevin's page. And finally, journalist A.J. Jacobs is here to talk about the 2 terrifying days he spent without any artificial intelligence at all.

My God.

I hope he's okay.

1. Character.AI Bans Teen Chats

Well, Casey, this week we got some really surprising news about a company that we have talked about on this show before. This is Character.AI, the company that makes these realistic chatbot companions. We talked about it about a year ago on the show in the context of the very tragic story of Sewell Setzer III, a 14-year-old boy who took his own life after becoming emotionally attached to a Game of Thrones chatbot on Character.AI.

We got a big update on that story this week, which is that Character.AI is barring minors—people under 18—from having conversations with its chatbots. It is basically saying, “We are not going to offer this service to minors anymore.”

Yeah. There is some nuance here that we'll get into, but at a high level, Kevin, I think this is one of the most dramatic steps we have yet seen from a major AI company to try to address the very real harms that these technologies pose, particularly to young people.

Yes. So let's get into the details, but before we do, let's make our AI disclosures. Casey, what is yours?

My boyfriend works at Anthropic.

And I work at The New York Times, which is suing OpenAI and Microsoft over alleged copyright violations.

Yeah.

Just to remind folks who may not remember the initial story, Character.AI is a company that was started several years ago by leading AI researchers from Google who left that company. Noam Shazeer and Daniel De Freitas were the 2 co-founders. They were frustrated that Google was not releasing the chatbot they had worked on.

So they said, “We're going to go off and build our own startup where we're going to release these chatbots based on large language models. You can make characters, you can talk with them.” It's sort of a role-playing app experience, and it became enormously popular with young people. This was one of the first generative AI chatbot-based apps that really took off, and many of the users were teenagers or even younger.

When I went on, as I did while reporting on this, there were just a lot of chatbots that seemed really aimed at young people.

Yeah.

Chatbots that would take the persona of your friend at school, a bully, or your crush. It was a very young-seeming app.

Yeah, or characters from Game of Thrones, or you name the franchise. I think the original idea animating Character.AI is, “Hey, what if you could chat with a lot of copyrighted material that did not belong to Character.AI?” It turned out that was hugely popular with a bunch of kids who wanted to talk to Pikachu or whoever.

Yeah. When I was reporting on it a year ago, the company wouldn't tell me how many of its users were under 18, but said that it was a significant number. When I hear things like that, I just assume that this is an app that is predominantly used by young people.

For that reason, it's a very big deal that they're making these changes to basically wall themselves off from young people, at least for their central use case.

Yes, and I think that comes after really sustained public pressure in the wake of Sewell's death. There are other lawsuits against the company, and I have to imagine that at some point the lawyers at this company said that the legal risk to us is simply too great.

We believe Character.AI has about 20 million monthly users, which is the figure that I have seen reported, and they think there's a better opportunity for them building for adults, at least in the moment, than in making this technology available to kids.

Yeah. So let's talk about the specifics of how this is going to work. Character.AI put out a blog post this week spelling out the changes that they're making. They say that over the next month, they will identify users under 18 and begin giving them time limits on their ability to chat with characters.

That limit initially, they say, will be 2 hours a day, and it will ramp down in the coming weeks. By November 25, so roughly a month from now, users under 18 will not be able to converse in open-ended conversations with any Character.AI chatbots.

Basically, they are going to limit the length of the conversations, maybe the topics of the conversations, and they are going to try to give teen users other ways to, as they say, be creative—for example, by creating videos, stories, and streams with characters. But they will not allow this kind of open-ended role-playing experience anymore.

Yeah. So if you just want to create a little bit of synthetic media featuring these characters, that's okay. But what's not okay is essentially the thing that seemed like it was really problematic in Sewell's case, right?

Sewell had developed this very intense relationship with a chatbot that was called Daenerys Targaryen, as in the Game of Thrones character. I think there are a lot of concerns about kids getting into these very emotionally heavy relationships with these synthetic characters. It can take them into a world of delusion. It can separate them from their friends and family. That's the sort of thing that's not going to be allowed anymore.

Yes. So it's unclear exactly what Character.AI is going to do now that it's giving up on what is essentially its entire core use case for young people. They say that less than 10 percent of their current users are self-reporting as being under the age of 18. That's according to their CEO, but obviously, that is self-reporting. I think a lot of teenagers are lying about their age in order to get access to these sites.

Do you know how many times I lied about my age on the internet?

2. AI Companions Go Mainstream

Yes. I've had an experience over the past couple of months where I've just started to feel like this is the most important and least understood topic in technology right now.

I was recently at a high school, and I often like to poll students about how they're using AI. I asked at this high school, “Raise your hand if you have an AI friend,” and about a third of them put their hands up.

This was something that, I think, a year or 2 ago was considered fringe, kind of unusual for young people to have these intimate relationships with chatbots. But the chatbots have gotten better—

Mm-hmm.

—and more compelling and more persuasive, and it is just starting to become this mass social phenomenon.

There's one study—a survey done by Common Sense Media recently—that found that 52 percent of American teenagers are regular users of AI companions, which is a startling figure and represents just how quickly this all is happening. Another stat that I found very alarming from this survey was that nearly one-third of teens find AI conversations as satisfying as or more satisfying than human conversations.

Absolutely, and why is that? We've talked about it so many times on the show. These chatbots are designed to be agreeable, to tell you that you're correct, and to support you, and that's not inherently a bad thing. But if it becomes your primary mode of socialization, it does seem like there is some real danger here.

And Character AI is the first company that has said, instead of trying to introduce these mealy-mouthed, incremental tweaks and guardrails, “We're actually just going to shut the whole thing down until we can figure out what's going on.”

Yeah. And I think in the minds of a lot of parents or people who understand and are worried about this phenomenon of AI companions for young people, there is a sense of relief about these changes, a sense that maybe this one company at least has decided to put people's health and well-being above its own profits.

I was texting a little bit this morning with Megan Garcia, who is the mother of Sewell Setzer, just seeing how she felt about this. She filed this lawsuit. She's been becoming more of an advocate for these issues, and she gave me permission to share this. She said, “I'm relieved for the children that will actually lose access to Character.AI because those are lives that can be saved, even if it's one child. But I can't help but feel cheated. Why did it take Sewell dying and me taking on this tech company to get them to do this?”

So I think for Megan, and for, I'm sure, the rest of her family, there is some relief but also some frustration with why it took a lawsuit, enormous public pressure, and pressure from regulators around the world to get this company to act.

Yeah. So this announcement has not been universally praised. The Tech Justice Law Project, which is one of the organizations that brought the lawsuit, sent us a note this morning. They pointed out that Character.AI had not really said how they were going to do age assurance to make sure that all of the adult users who will continue to get access to these chatbots actually are adults.

They also noted that the company had not addressed “the possible psychological impact of suddenly disabling access to young users given the emotional dependencies that have been created.” So I thought that is worth saying. We have seen, when other companion bots have removed access, how jarring and painful it can be to the people who use them.

And so while I'm glad that Character.AI is going to be ramping these users down as opposed to simply pulling the plug, I do think it's worth saying this may be painful for some of their users.

Yeah, I think that's a really good point. Just because these are not human relationships doesn't mean that they can't produce pain and grief when people lose their connections to something that they have grown attached to. So, yeah, I think we should be very sympathetic and empathetic toward people who may be having a hard time now that their chatbots aren't talking to them.

So what kind of impact do you expect this move from Character.AI to have on the rest of the industry?

Not much. I think Character.AI was a special case. They actually have lost most of their founding executives and leadership. Noam Shazeer and Daniel de Freitas went back to Google and basically left behind the shell of this company.

So I don't expect Character.AI is going to recover from this. I think they were probably already in a state of losing users and seeing the decline of their platform. So this may be the final nail in the coffin for them.

But I think what has happened is that the rest of the industry is now doing what Character.AI used to want to do. I think the ones that we've talked about on the show, OpenAI and Meta, that are really pushing into this use case, I don't know that they're learning the right lessons from what has been happening with Character.AI.

Yeah, I mean, I have to say, I think it's going to have maybe a bigger impact than you do for this reason. Inevitably, there are going to be congressional hearings about this, and I think Character.AI will probably be there. They're going to say why they did this, and then they're going to go over to whoever is there from Meta or OpenAI and say, “Why do you guys think this is safer than they do? Why do you, Meta, still have Nasty Nancy available on your platform?”

I think that's just going to put a really interesting pressure on them, if for no other reason than to have a response as to how they are justifying what they are doing.

Yeah.

Right? And to try to put some sort of case behind it other than “Everyone else is doing it,” which is an argument I have heard Meta make about why these chatbots are available.

Yeah, and I think one way that the bigger AI companies may respond to that is by saying, “Well, look, we are just so big now. We have so many users, and some small percentage of those people are going to experience mental health crises in their lives, maybe even while using our product, but that number is small relative to the number of users of our products in total.”

This is what I would call the prevalence argument, which we heard a lot from social media companies a decade ago. They would say, “Oh, yeah, there is hate speech and toxicity on our platforms, but if you just look at the overall percentage, it's quite small and meaningless.” So I just think we'll start to see a lot more of that kind of argument.

You know, this whole discussion, Kevin, ties into some really interesting research that OpenAI released this week, where they began to map out the scale of the mental health crisis as it can be seen on ChatGPT itself.

There are now more than 800 million people a week using the platform. That's a pretty decent subset of the population. And while the numbers of people who are having these disturbing or potentially dangerous conversations with ChatGPT are low on a percentage basis, by the company's own estimates, you have 560,000 people a week whose messages to ChatGPT indicate psychosis or mania, 1.2 million people a week who are potentially developing an unhealthy bond to a chatbot, and 1.2 million people who are having conversations that contain “indicators of potential suicidal planning or intent.”

So if you just want to be very cynical about this and think about it only from a legal-liability perspective, if you have more than 1 million people a week who are developing an unhealthy bond to your chatbot, who are expressing thoughts of self-harm, think about the lawsuits that are going to follow. That could be hugely damaging.

So I wonder if the other big labs will look at what Character.AI did this week and decide maybe we actually should build some of these safeguards faster.

Yeah, I don't know. I'm still not that optimistic. I think these companies are trapped because they want the engagement and the depth of connection that people are having with their products.

Any company that makes technology wants people to, if not fall in love with it, at least develop a bond with it and feel very connected to it. So they want that, but they don't want the responsibility for the emotional relationships that people are going to develop with these systems, and already are developing in many cases.

Yeah, I mean, the main conclusion that I have about this whole story, Kevin, is just that this story reminds us that nothing is inevitable when it comes to AI, right? You don't have to build it. You don't have to release it to everyone. You don't have to make it free. You don't have to decline to build any meaningful guardrails.

You can actually just say, “Based on what we've seen, we don't think that this is safe, and we are going to take it off the market.” And at a time when everyone has their foot on the gas pedal and everyone feels like they're in this all-out existential race to a finish line called AGI, I've been so worried that companies were going to cut corners when it comes to safety. We've seen it over and over again.

And so while you hate to give too much credit to a company, particularly one like Character.AI, which did, after all, get itself into this mess in the first place, I do think there is something to be said for saying, “We are going to stop the bleeding here. We are actually going to admit that we don't know how this affects people, and we're going to take it off the market.”

Yeah. I mean, I think it's a really overdue but responsible thing that they did. I think they probably had their hand forced by the lawsuits and the regulators breathing down their necks.

And so I hope that Mark Zuckerberg and Sam Altman and these folks who are building these very persuasive, compelling chatbot companions are looking at this as a cautionary tale for what can happen if you don't think about the consequences of what you're building.

Kevin, maybe one more thing to say about this. We are living through a dramatic contraction in the access that teenagers have to technology online. In the same week that we are announcing this, YouTube, Snap, TikTok, and Meta have all said that they will abide by a law passed in Australia that will ban kids under 16 from having a social media account.

So you can still look at YouTube, but you cannot have your own account. You're going to have to use someone else's. And you can't have an Instagram account. You can't have a TikTok account.

There are many American states that are trying to bring about something similar, with mixed success based on legal rulings. But I think you look at what Character.AI is doing, you look at what some of the states here are doing, you look at what Australia is doing, and I think the social media and AI companies have just lost this argument, right?

There is no longer a consensus that unfettered access to these technologies is good or healthy or safe for teens, and people are finally starting to do something about that.

Yeah, I think that's right.

But I think the question of how to regulate these chatbot companions is only part of what I’m thinking about these days, because I don’t think this is actually something that we’re going to be able to regulate our way out of. I think even if countries do what Australia has done and ban social media use for kids under 16, kids are going to find this stuff. They’re going to get access to it. They’re going to think it’s compelling. Some number of them are going to grow emotionally attached to it. And so I think we do actually have to address the real possibility that there’s really nothing we can do at a regulatory level to prevent every teenager from forming an emotional attachment to a chatbot.

I don’t know. I think that sometimes passing rules like this can be the first step toward a society just changing its relationship with these technologies overall. When I went to high school, there was still a smoking section indoors at my school because it was taken as a given that, well, you can’t stop the seniors from smoking. Of course they’re gonna smoke their cigarettes.

But you go back to my high school today, there’s no smoking section, and banning it for teens, I think, was part of a larger movement of telling the adults, “Hey, this isn’t actually very good for you either,” right? And eventually, that sort of shifted. So I think there may be some hope that if we can take the next generation and not have their primary relationships be with Character.AI, there is some hope that maybe this doesn’t actually change society quite as radically as maybe it otherwise might.

There was a smoking section at your high school?

Can you believe that?

That’s amazing. What year was that?

This was the late 1990s. Now, it was the 1900s, but it was the very end of them.

3. Grokipedia Rewrites Wikipedia

Well, Kevin, I was reading about you recently.

Oh, yeah?

Yes. Let me ask you if this is true, because I read someone say this about you: “That when you were growing up, your upbringing fostered an outsider’s perspective on religious and cultural fringes shaped by a family dynamic that prioritized open inquiry over doctrinal adherence.” How do you respond to these charges?

How’d you get into my therapist’s notes?

Believe it or not, Kevin, that’s not your therapist. That’s from a little something called Grokipedia.

Oh, boy.

Grokipedia, of course, the Wikipedia competitor developed by Elon Musk and xAI as part of a huge culture war that has gone on over the world’s most popular encyclopedia. I thought today we should take a look at this thing and talk about what we think.

So I need you to walk me slowly through this.

I’m gonna Grok you slowly through this.

I am coming to this totally cold, and I want to get into Grokipedia and all the details, but first, I must know: what else does Grokipedia say about me?

Oh, my goodness. Well, first of all, this article is long. There are more than a dozen subsections—

Oh, boy.

—about your life—

Oh, God.

—including your books, your New York Times career, and my personal favorite: notable events and controversies.

What are my notable events and controversies?

Well, you’ll be happy to know that there is an extended section about your interaction with a certain Bing’s Sydney chatbot back in 2023.

Never heard of it.

And then there are also criticisms of your AI reporting—

Oh, boy.

So obviously Grokipedia is part of Elon Musk’s AI chatbot Grok, but what is this project, and why did he decide to make his own Wikipedia?

For well over a year now, conservatives and the right wing have been fomenting this backlash against Wikipedia, which they say is biased against conservatives. That is, of course, a familiar conservative talking point about basically every popular tech platform on the internet. In the case of Wikipedia, they’re particularly concerned that Wikipedia editors have labeled a bunch of conservative media as unreliable and therefore ineligible for inclusion as citations on articles about controversial subjects.

It’s a little too Wokipedia, if you know what I’m saying.

I do know what you’re saying, and I don’t like it. Because embedded in that critique is the idea that, for example, The Heritage Foundation’s blog post, Breitbart, or Fox News’ political coverage deserve to be seen with the same credibility and fact-checking, journalistic rigor as, let’s say, Platformer.news.

Yeah, I feel like this happens every few years, where a group of partisan activists gets very mad at Wikipedia, the crown jewel of the internet. Every 3 years, people are just like, “It’s horrible. It’s biased. We have to destroy it.”

Yes.

Yes.

And this particular backlash seemed to get a lot of fuel after the famous Elon Musk Nazi-salute incident.

What happened there?

Well, you may remember—

I remember the incident, but what was the controversy surrounding it?

Elon Musk says, and has continued to argue across many posts on X, that this was not a Nazi salute. And now whenever any Democratic politician raises their hand in a vaguely Nazi-salute-seeming way, he does a post about it.

But he’s very mad about how this controversy was handled on Wikipedia, where there is an entire page devoted to it. And that appears to be one of the main reasons why Elon Musk says, “We are going to build our own Wikipedia. It is not going to have the same biases baked into it. It is going to be maximally truth-seeking,” to use one of his favorite phrases. And as of this week, it is live.

4. Grokipedia Copies Wikipedia

So can I just ask some technical details about Grokipedia? Is it all written by AI? Is that the premise here?

Well, certainly Grok seems to have played a starring role in this thing. When you read it, it reads very much like Grok output. But as many writers have noted, including Jay Peters at The Verge, when you do side-by-side comparisons of Wikipedia and Grokipedia, there appears to be some pure plagiarism. And Grok does acknowledge that it has used large chunks of Wikipedia under Wikipedia’s license.

So it seems like maybe Elon Musk and his team have ingested some or all of the regular Wikipedia and just given Grok a prompt that’s like, “Rewrite this to be more Grok-like.”

Yes. And at launch, Grokipedia has more than 800,000 articles. That compares to around 7 million on English-language Wikipedia. So it’s a small subset, but presumably that will grow over time.

Right. And can anyone edit the articles on Grokipedia as on Wikipedia?

No, you cannot. What you can do, if you see something on Grokipedia that you think is wrong, is highlight it, and then a little button will pop up that lets you click it. And you can say, “This is wrong,” and you can make your case. And that seems like a great way to waste a lot of time—if you have nothing else to do in your life.

Hang on, give me a second. I have some bones to pick with my Grokipedia entry.

You really haven’t looked at yours yet?

No.

Oh. Well, of course I had to look at mine.

Yeah, what did yours say?

Well, first of all, I’m very proud of us that we made the first 800,000 articles in this encyclopedia, right?

Not easy to do.

I was not one of the first 800,000 entries on Wikipedia, I’ll tell you that much.

So I appreciate that.

Wait, I have to go see this for myself.

Yeah. Go ahead and pull it up. I want to be intellectually honest here. There are parts about my Grokipedia page that I like. It goes into way more detail than my Wikipedia page does, and I think overall presents a pretty good picture of who I am and what I have done.

Oh my God, it’s so long.

It’s incredible—I’ll say it. It’s too long. Nobody actually wants that much information about me.

Wait, can I read the family and relationship section to you?

Yes, please do.

Because this is breaking some news here on this podcast. It says, “Newton is married to a lawyer.”

Yeah.

Congratulations. I thought your boyfriend worked at Anthropic.

Well, he does. And so we have found the first of many mistakes that you will find in Grokipedia.

“He maintains a low public profile regarding his personal relationships.” Not true. Can’t stop talking about it. “With no further details on partnerships or children disclosed in available interviews or profiles.”

Yeah. So I guess I’ll try to say more about my boyfriend to try to help Grokipedia out next time.

Wow, it’s got your Goodreads profile here.

Yeah.

“Newton exhibits a keen interest in reading.” False. He hasn’t read a book in years. “Evidenced by his Goodreads profile cataloging 112 books with ongoing reads, including Cahokia Jazz by Francis Spufford, The Saint of Bright Doors by Vajra Chandrasekera, and others spanning fiction and nonfiction.” Mm.

Let me give a shout-out to The Saint of Bright Doors, by the way. That is the best book I’ve read this year. Super, super good.

There’s something a little creepy about it, right? It’s like we’re going to go in and look at all of your public profiles and see what we can scrape in. But I do think it winds up putting together a decent picture of my life.

For the most part, I’ve been able to stay out of a lot of culture wars and political controversies. So I didn’t see anything in there that made me really roll my eyes and feel bad. If, though, you are an Elon Musk or a Donald Trump, you may find that you’re getting a much friendlier treatment on Grokipedia than you would on Wikipedia.

5. Grokipedia Joins The Culture War

Okay. So let’s talk about the politics of Grokipedia. Where does it differ meaningfully from standard-issue Wikipedia?

Because it is designed as a kind of right-wing alternative, when you pull up articles that have been the subject of a lot of culture warring, you will just find material that is much closer to the conservative or Republican view. So, for example, if you pull up the article on Donald Trump, there is what I would say is a very friendly view of the events of January 6, 2021, that goes out of its way to talk about how Democrats overstated the risk to democracy.

I will say that in some ways I expected Grokipedia to go further to the right. And don’t get it wrong: There is a lot of really racist stuff in Grokipedia. There’s a lot of anti-trans stuff in Grokipedia. But as somebody who has spent more than his share of time reading 4chan and r/The_Donald back in the day, the stuff that I’m seeing in Grokipedia is not as bad as that.

What is the strategy here? What is Elon Musk hoping to accomplish? Is he hoping that people will, instead of going to Wikipedia to learn about stuff, go to Grokipedia, and that we will educate people differently in this country?

Yeah, I think so. I read a quote in The New York Times article about the Grokipedia launch where they had some scholar who said, “Ever since people began to study things, people have wanted to control knowledge and how it is distributed,” right? And I think Grokipedia is just a step in that direction.

If you believe that Wikipedia has a chokehold on the public imagination, if you’re concerned that Wikipedia data is being used as a pillar of most of the big, large language models that we’re now using every day, if you wanted to inject other views into the populace, you might want to create something like Grokipedia.

Yeah, I’m also wondering how much of it has to do with the actual training of Grok, because one thing that we know is that Elon Musk has been frustrated in the past that, despite his best efforts, it keeps Hoovering up all this data from the internet, and that makes it, he thinks, too liberal. And so I’m wondering if this is an effort to give Grok a new kind of substrate of knowledge that it can learn from so that it’s not reliant on Wikipedia.

It’s an interesting idea, but I’m not sure how much original work Grok is really doing. I think that it is almost certainly showing you a wider range of sources than you might find on Wikipedia, or at least a wider range of right-leaning sources. But it’s not as if there are a bunch of right-leaning Grokipedia editors who are going out there, doing original research or something.

This is very much akin to a deep research report that you might get ChatGPT or Gemini to do for you, except that this time it’s Grok.

Right. So how much of a big deal is this? Are you seeing reactions from people who are scared that this actually will replace Wikipedia? Is this just one of Elon’s many passion projects?

So far, I don’t know that Grokipedia has made much of a splash in the mainstream, aside from just being a curiosity. If you follow Elon Musk on X and visit X a lot, it is something that you’ve heard a lot about. And I’ve seen conservatives and the tech right talking up certain Grokipedia pages as, “Aha, this is so much better than Wikipedia.”

So it is having that moment right now. How long does that last? I don’t know. Wikipedia is one of the very most popular sites on the internet, and it’s going to take a lot to displace that, right? I think for a lot of people, going to Wikipedia is just muscle memory.

So barring some sort of massive leverage that Elon Musk is able to get in distributing Grokipedia to more people, I think it’s probably going to remain more of a curiosity.

Yeah, and do you think this contributes to the fears that people have about the decline of Wikipedia? Because we’ve been talking for years now about how generative AI chatbots are increasingly people’s first step toward learning about a new subject, where before, if you wanted to learn about the Franco-Prussian War, you would have gone to the Wikipedia page for it. But now you might pop open ChatGPT and just ask for information, and it would go out and look at Wikipedia and other sources for you and synthesize it.

So is this coming at a time where Wikipedia is already pretty vulnerable as a result of AI?

I think to the extent that Wikipedia is vulnerable, Grokipedia doesn’t really pose much additional threat. I think by far the larger threat to Wikipedia is what you just said. It is that more and more people are accessing the information via chatbots, via Google search results, and the main consequence of that for Wikipedia is that if Wikipedia can’t get you to go to the site, it also can’t get you to contribute.

It can’t get you to update articles. It can’t get you to become an editor. And so the fear is that as traffic to Wikipedia declines, the quality of the site will decline as well. And in fact, just this month, Wikipedia published a blog post in which they said they are starting to see traffic declines due to generative AI.

Hmm.

So that’s a very real threat to the encyclopedia. Grokipedia, I think, isn’t quite that.

Yeah, whether or not Grokipedia takes off as a product, I think that this larger effort to delegitimize Wikipedia is quite possibly going to be successful. I think that people on the right especially have identified that, even though Wikipedia looks like this canonical thing that is just appearing on the internet, it is made by people. A relatively small percentage of people on Wikipedia are actually contributing to it.

And so I think they have recognized that this is another set of levers that they can essentially work: the Wikipedia editors and the moderators who control the rules. Maybe that is the larger victory that they see as being possible here. Maybe they can just undermine the longstanding traditions and norms of the Wikipedia community and get it to behave more like Grokipedia.

We’ll see. I think Wikipedia has been really resilient so far. There actually have been hearings in Congress about this alleged bias in Wikipedia, which I find outrageous because Wikipedia should be able to say whatever it wants about vaccines or January 6 or whatever else, right? It doesn’t have any legal obligation to the federal government to provide one set of views over another.

But that does get, Kevin, at why, on balance, I’m actually glad that Grokipedia exists.

Why?

Because I think if you see something online and you get really mad, and you think that there is a better and smarter view out there, I think the best thing to do is to just put it up on the web, right? Not in every single case. There are some horrible things that I wish you wouldn’t post on the web.

But look, if you want to have a debate about January 6, go ahead and create a webpage, right? That is my preferred resolution to political conversations, as opposed to starting to have hearings to try to pressure Wikipedia into having one particular political view.

So I view Grokipedia, as silly and bad and offensive as it can sometimes be, as still a case of countering speech with more speech. I think that is overall a better way to have a democracy.

Yeah, so I understand that take, but I’m also wondering if the fact that Grokipedia is AI-generated makes it any different. Is the answer to speech that you don’t like really having a chatbot go out there and write a bunch of slop text for you?

I think that is a good question, and maybe we should value Grokipedia less than we do Wikipedia for that reason. At the same time, humans are involved in the creating and shaping of Grokipedia, right? It seems very unlikely to me that what we’re reading in some of these really high-profile entries has not been edited or tweaked by someone. So I think I still see a pretty strong human hand in here, and that’s why, on balance, it still does feel like counter-speech to me.

Wow, that’s a very pro–First Amendment take from you.

Yeah.

Your lawyer husband must be so proud. I just think this inevitably ends with Elon Musk forcing a state government to teach Grokipedia in schools.

I mean, you joke, but that’s probably only a half-joke, and I wouldn’t be surprised if it does become the curriculum in Texas in 2028. So we should absolutely keep an eye on that, and there are places it can go bad.

But at the end of the day, one thing about me as an elder Millennial is I love the web, and so I’m generally in favor of people making websites. Look, at the end of the day, the truth is most websites just get ignored, but they wind up being useful for some people, right? So I’m not going to be visiting Grokipedia every day, but if the uncle you’re going to have your worst conversation with at Thanksgiving this year wants to use it, that’s fine.

Yeah. My bigger question is whether this whole category of the online encyclopedia is just obsolete, right? I love Wikipedia as an idea, as an expression of collective knowledge, as what I consider a true gem of the internet.

It’s a miracle.

It’s a miracle. And I cannot tell you the last time I went to Wikipedia.

Really?

Yes.

Oh, I’m there every day.

Really?

Yes.

So I feel like I go there now when I need to check something that a chatbot has told me, but I do not really go there as my first stop on any given fact-finding mission. My consumption has shifted almost entirely to these chatbots and to search engines, we should say. I still do use Google on occasion.

So I just wonder if there is any future in which Grokipedia, Wikipedia, or any of these sites have a realistic hope of making it, or if they just end up being crammed into the chatbots and that becomes people’s primary way of finding things out.

Well, I think you’re onto something real here because of all of the products that Elon Musk has launched in the past 10 years, Grokipedia does seem like by far the least forward-looking.

Yes.

Right? Now, it is possible that you could take the contents of Grokipedia and find other ways to distribute them, and maybe that’s what he will do. Notably, he is not relying on human contributors to bring in the knowledge because he just stole most of what was on Wikipedia. So those humans already did the work for him.

Yeah.

But he can just take the material that he already has and find new things to do with it, update it via conservative media of whichever flavor he likes. So maybe that’s the way that this thing winds up being a little bit more forward-looking than it seems today.

Yeah.

What’s your Grokipedia take?

Well, let me spend a little more time on it. Let me see what it has to say about The Hard Fork podcast.

Oh, let’s—do we have a page?

We don’t have a page.

Oh, my God.

Okay, I hate this thing.

Oh, my God.

We could do Tuning Fork, Clark Fork River, or pastry fork. Let’s see what it has to say about pastry fork.

6. Jacobs Goes Without AI

Well, Casey, you and I are AI maximalists. We use this stuff all the time. But today we’re going to talk with someone on the show who went 48 hours without using AI at all.

Yeah.

An unthinkable idea to me.

Suffice to say, an experiment that would not have occurred to the 2 of us to try.

Yes. So today on the show, our guest is A.J. Jacobs. A.J. is a great writer and a former mentor of mine. He was actually my first boss in journalism. I helped him with a book many years ago—about 20 years ago now—about the Bible.

He is known for these immersive experiments, where he throws himself deeply into a topic. He wrote a book about following the Constitution literally, and he wrote a book about following the Bible literally. He’s also the host of The Puzzler podcast, and this week he published an article in The New York Times titled “48 Hours Without AI,” in which he sets out to live his life with as little AI contact as possible.

Yeah, and my expectation as I started to read this piece was that it would be pretty easy to go 48 hours without using AI. But for reasons that A.J. gets into, it actually winds up being quite difficult.

Yeah, you essentially have to time travel back to the 1800s to avoid contacting anything that has any form of AI in it. And I think it’s a useful point, in addition to being a very fun article, because it does drive home just how intertwined all this stuff is with the way we live our lives today.

It’s not inevitable that it’s going to continue getting more intertwined, but I think it’s a pretty good bet.

Yeah, so this is a story that begins with A.J. forswearing modern electricity and ends with him foraging for food in Central Park. I think it’s time to bring him in and talk about it.

Yes, his life resembled that show Naked and Afraid, where you have to just find your way out of the forest. That’s what living without AI in the year 2025 is like, according to A.J. Jacobs.

Mm-hmm.

Let’s bring him in. A.J. Jacobs, welcome to Hard Fork.

Delighted to be here. Thank you, Kevin. Thank you, Casey.

So you just did this experiment where you went 48 hours without using AI or machine learning, and I want to talk with you about that. But can we just start with the photos at the top of this story? You are wearing what I would describe as a very loud outfit that has red checkered pants, a paisley flower-print shirt, and glasses that look kind of like Elton John’s, or the ones you get after your pupils are dilated.

Right, I brought them along.

So why the fit? Explain this.

Well, the premise of the article was, as you said, to try not to interact with AI or machine learning for 48 hours. One thing I realized quite early on was that it’s everywhere, especially machine learning.

Clothing designers are experimenting with it in terms of designing, but also anything in the supply chain is totally machine-learning-optimized. They figure out how to route it and how to pack it using machine learning. So I thought, well, anything in my closet that’s 10 years old or less is probably off-limits if I’m really being strict.

But deep in my closet, I did have my grandfather’s 1970s paisley shirt and red-and-white checkered pants. He went through an Austin Powers phase. He was very much a dandy. And I was like, “All right, I gotta do it,” even though it made me very uncomfortable.

Although my wife said that it was the coolest I had looked since we’ve been married, which was insulting and also flattering.

I think you got the timing right.

I think it’s been so long since those clothes were in fashion that they now actually do look fashionable again.

Yeah.

Yeah. A.J., let me ask you this: As you headed into this experiment, what is or was your relationship with AI? Are you the sort of person who was using generative AI tools like ChatGPT every day? Was it a more occasional thing? Where were you on that spectrum?

Yeah, I would say I’m in the middle. I’m not a Luddite, but I don’t have it controlling my life. I did use it for research, and it was actually not bad. I was impressed.

Got it. So you were not trying to prove that AI is bad. I just remember some of your other experiments over the years have included following the rules of the Bible, and that was at a time when people were talking about taking the Bible very literally. I knew that part of why you were doing this was an attempt to say, “Well, here’s what would happen if you just went all the way toward your stated belief.” There was a point in there about the dangers of literalism. Were you trying to make a similar point about the dangers of AI here?

I did not go in with an ax to grind on this one. The thesis was: Where is AI hiding? I don’t believe AI is all good or all bad. I didn’t believe that before, and I don’t believe it now. I think in some cases it’s awesome. Thank God that machine learning checks whether there’s credit card fraud. But on the other hand, it has huge risks and has divided our country.

I was not coming in saying it’s all good or all bad. I was asking, where is it, and what lessons can I learn from spending 2 days without it?

So let’s get into the experiment. I feel like the biggest choice you had to make at the outset was, how am I going to define what counts as AI? You decided to include machine learning. Talk to us a bit about how you set the boundaries for how you were going to run this thing.

Right. All the experts I talked to said AI is a big umbrella. You’ve got generative AI, like ChatGPT, and that’s getting all the heat now. But AI has been around for decades because the umbrella also covers machine learning. It basically covers machines that can evolve, that can look at new data and change.

The way I explained it in a paragraph that was cut was—

This is why I love freelance writers. They harbor so many grudges, and it’s great to have a podcast to air them out. I bet you had a better idea for the headline too, didn’t you, A.J.?

I know. What is this crap? No, he was delightful. It was just a matter of space. But I said—

Yeah.

Traditional programs are input A yields output B, whereas machine learning is more like a recipe that changes. You have a recipe, but then data comes in, and the recipe says, “Oh, people really like sugar. I’m going to add sugar.” So it evolves.

The reason I thought it was important to put both in is because I feel they both have this great potential and great risk. They both have these unintended consequences. When you have machines that can change and you can’t predict what they’re going to do, that is, as I said, sometimes wonderful. Sometimes you end up with YouTube algorithms that turn us all into flat-Earthers.

Right. So let’s talk about some of the things you did in this experiment. My characterization of this up top would be that you had to basically become Amish for 48 hours. You were—

That was a line, again.

Did that also get cut out of the story?

That was in there. I said “Amish cosplay” was one comparison, or Laura Ingalls Wilder was another. Yes, because pretty much anything that’s electric or electronic includes machine learning, including electricity itself, because Con Edison uses tons of machine learning to figure out where the demand is going to be.

So, yeah, I had to go Amish. I did have a solar-powered generator, so I could plug in a lamp for a while. But, yeah, it started the moment I woke up. My iPhone uses facial recognition. That’s AI. But even the iPhone camera uses AI. Gmail uses machine learning, even without the new AI features of Google.

Water was a surprise because the New York reservoir system uses machine learning to help. They want to make clear that humans make the final decisions because they don’t want people to freak out. But the machine learning helps them figure out where the demand is and when they should make repairs.

Well, so how did you stay hydrated for 2 days?

I did plan ahead, which maybe was a little bit of cheating. I put several bowls out on my windowsill in the weeks before to collect rainwater, and I didn’t get giardia or anything, so I feel lucky about that.

How much rainwater were you able to collect before you started the experiment?

Well, that’s why I had several bowls, and it was weeks.

Oh.

So, yeah, it was not ideal. It was not ideal.

There’s another piece of this experiment that I wanted to ask you about, which is that you had to forage for a meal in Central Park. I was not under the impression that food itself was generated by AI, but what am I missing here?

Well, of course it all depends on how you define it. Food is really intertwined with AI. Industrial farms use AI and machine learning to figure out whether to water the crops and when to plant them. Food is, of course, shipped along the supply chain, which is AI-optimized.

So if I’m being really strict, which of course I was, I thought, “Well, maybe I can’t eat anything from the grocery store.” To be super safe, I found a video where a man named Wildman Steve Brill teaches you how to find edible food in Central Park. So I took him up on that, and I went foraging. I got some plantain weeds in Central Park, which I ate, and they were not great. They taste like dirt, but they didn’t kill me. They didn’t kill me.

That’s that park-to-table cuisine that is so popular in New York these days. They’re so ahead of the curve over there.

I ate some plantain weeds from a guy in Central Park once, and I saw the fifth dimension.

That’s right. I’ve seen all those plantain-weed dispensaries that are popping up all over the city lately.

Yeah.

Now, Kevin, as A.J. is describing his experience of depriving himself of so much AI and so much technology, I’m wondering how that lands on you. I think if I were to come to you and say, “Hey, all generative AI services are going to be down for the next 10 minutes,” your heart would seize up. You’d start having palpitations. I’d see a cold sweat running down your forehead. Was this a challenging story for you to read?

You know, it was. I was thinking, “How would I do at this?” And then I was thinking, this would probably be good for me. I should probably do this every couple of months. I should do 48 hours of—

Yeah.

—no technology at all.

But we’ve talked on the show about how you’ve gotten to the point where you’ll put your phone in a prison overnight if you feel like you’re developing too strong an attachment to it. At the same time, I have never seen you run an experiment quite like this.

Yeah, this is true. I have taken breaks from technology. There was a period when I was doing this digital-Sabbath thing, where 1 day a week I would try not to use my phone or send any emails. But that was too hard, so I quit that.

But A.J., this experiment—I mean, we should continue talking about it because I want to get to the takeaways here. You have this 48-hour period where you’re not using AI. What is your emotion upon reaching the end of this period and being able to use this stuff again?

I had a lot of mixed emotions. On the one hand, there was relief from being cut off, like you’ve talked about with the digital detox. There was also annoyance. It is super annoying when you can’t Google. My Encyclopedia Britannica is not up to date.

On the other hand, it was terrifying. That was another feeling I had because I realized how omnipresent AI is. As I’ve said, it’s not all bad, but it has these huge risks. So it was really a mixture of emotions, which maybe is the right way to react to AI. It’s not a monolith. It’s not black and white. It’s super confusing.

How much easier would this experiment have been if you had limited it to generative AI? If you said, “Okay, machine learning is fine,” which is pretty well integrated into all services, as you reported here, what if you just said, “Well, I’m not going to use ChatGPT and other generative AI services”?

I think it would have been easier for now.

Yeah.

In 5 years, I think that line will be erased. I also think it would have been hard to research because, as I confess in the article, I used a ton of ChatGPT to research it. That was another takeaway: how to use ChatGPT.

ChatGPT sensed the thesis of my article. It knew I wanted to find machine learning and AI everywhere, so it was serving me up these half-truths. I had to give it some tough love and say, “ChatGPT, pretend I’ve got the opposite thesis.”

“I don't want AI and ML to be anywhere. Tell me now, what are the reliable sources?” Because, as you know, it's just an obsequious machine.

Yeah. I think if there's one takeaway from your piece for me, it's that the line between classical AI or machine learning and generative AI is thin and getting thinner.

Right now, people are very angry at generative AI. They say, “Oh, it takes all this electricity. It uses all this water. These companies are sort of foisting it onto us.”

So I think there will be people who read this article and say, “Well, he's making this inevitabilist argument that there's nothing we can do, and we have to live in this world, and it's too late to turn back the tide.”

And I think what I came away feeling from your article was that, yeah, in 5 years, the difference between generative AI and classical AI may be so small as to be invisible, and we will just think of this stuff as being on a continuum that starts with Netflix recommendations, image recognition, and self-driving cars and goes through chatbots and all that other stuff.

Well, there's an old joke in the tech industry that AI is just whatever we call whatever the computer can't do yet.

Right.

That's a good one.

Right? We've just been on this advancing frontier forever, and it keeps being able to do more stuff.

Right. Can I just add one thing about the inevitabilist part? Because I don't want that to be the takeaway. I don't want people to give up. I feel we need more transparency. I loved the law that you were talking about, the California law about watermarking AI images.

We need more transparency on what's AI and what is AI-generated. I'm in favor of more regulation. It is such a powerful technology. And I also want more control over my algorithms. I hate that Facebook has so much control.

Maybe there's a way to go in and make it show me articles that I disagree with. Maybe, but it would take me days to figure it out. So there are things we could do. It's not inevitable. We gotta take action because we are somewhat in control of where AI and ML are gonna take us.

I guess I'm curious what you think, A.J., as someone who has spent a lot of time thinking and writing about religion as well as AI, how far that comparison holds. I often tell people that being in San Francisco, in the AI world in 2025, feels a little like being in the Protestant Reformation. You've got all these cults and these groups and these people handing out pamphlets declaring that the end is near and trying to recruit you to their movement, and it just feels like this great blossoming of these ideas about the future and the end of the world.

I also know that the last time we had a big industrial revolution, there were a lot of weird cults and utopian communities and people who opted out of the new technology. I'm curious. I understand this was a joke or a stunt or a piece meant to illuminate some larger point, but I'm wondering if you think there will be people who actually choose to live like this because they will just see all of this AI, and instead of having the reaction that you had, which is, “I wanna dive in and investigate this,” they will just be like, “Screw it. I'm out. I'm going to my haven in the woods, and I'm gonna turn off all my devices, and I'm gonna live like it's 1870.”

Yeah, I think you will have that. You will have some—a big Luddite movement—and I can understand it because it is scary.

As to the religion metaphor, I think it's a good one. I think there is a lot of overlap and this sense of a destiny that AI is destined to create heaven on Earth or even replace us. I would say one difference between religion and science is the idea that science can be falsified, so my hope is that people in the AI industry keep an open mind and look for falsification, look for examples of where AI is actually not doing good, and try to adjust to that so that it doesn't become a religion.

Ooh.

Well, A.J., thanks so much for coming. Thanks for doing this experiment. The piece is called “48 Hours Without AI,” and you can read it at The New York Times. Very scary story to read on Halloween.

Oh, good point.