AI与法律：改变执业、Claude Constitution与新权利——Scaling Laws的Kevin与Alan对谈

Thanks for having us, Nathan. Glad to be here.

Thanks for having us.

Yeah, I'm really excited for this conversation. We have a lot of ground to cover. I'm interested in always trying to patch my blind spots on the AI landscape in my AI scouting mission, and I always appreciate a chance to do that. Given that you guys are both law professors and scholars studying AI and law, and the intersection of those two fields, I want to take the chance to get a survey from you in terms of what's going on at the intersection of AI and law.

I listened to your recent episode on Claude's Constitution, and that's really interesting. There's a paper that you shared with me on automated compliance, which is a phrase I had not heard before and think is a fascinating concept. Who knows what other new social contracts we might imagine and explore together as well. Maybe for starters, what's going on at the intersection of AI and law?

I'd say it's a big traffic jam at this point, or a huge crash, because we have systems that were largely constructed in the 1960s, if not before, and in the 1970s. A lot of the core privacy principles, for example, emerged from the Fair Information Practice Principles. I always get them wrong because we just refer to them as the FIPPs. But you've got FIPPs from the 1970s, and you've got case law from well before that, all of which tries to spell out what rights and obligations we have in an analog world.

We already saw those being pressure-tested during the internet era, and as we all know, AI is just putting all of that on steroids. When it comes to trying to see how prior legal regimes fit into this new world of AI, it makes for a lot of rich scholarship. Thankfully, Alan and I have plenty of excuses to continue to write law review articles, although his are always way better than mine.

That's not true, but I'm not sure anyone wants to read any law review articles, whether they're good or not.

I might back up a little bit, though. I agree with everything Kevin said. I think there are 2 different intersections of law and AI. In those law schools that have AI classes—which an increasing number of them do, and I think within a year or 2 all of them will—there are actually 2 different classes, because there's the law of AI, and then there's AI in the law, and those are actually very different things.

On the one hand, there's all the stuff Kevin was talking about, which is that AI is a new socioeconomic technology movement, maybe the most important thing since fire. But even if you don't think that, I think at this point everyone agrees that it's at least at the level of the internet, right? So there are all these legal questions that come up: How do you regulate it, how do you promote it, how do you control it, et cetera, et cetera?

At the same time, there's a whole separate set of conversations that have some overlap but are actually pretty orthogonal to that, which is that law is just a cognitive discipline. It's not quite as pure a cognitive discipline as, let's say, computer programming, because there are still areas in which the law expects there to be actual human beings, whereas if tomorrow all computer programmers uploaded their consciousness into the cloud, you could imagine a world in which a computer programmer would do just fine. With law, rather, you still need people to go into courtrooms.

A huge amount of law is purely cognitive. So there's no reason to think that the same revolution AI is currently having in computer programming, which is the manipulation of certain kinds of symbols, will not also apply—and is not already applying—to the law, which is also the manipulation of certain kinds of symbols.

It's true that I think the law is somewhat behind where, let's say, computer programming is, but it's like a year behind, or maybe 2 years behind. It's not 30 years behind. Just as software engineering has been completely transformed in the last year—and obviously, I've listened to a bunch of your podcast; you go into it much more than we do, but we talk about it somewhat—as a really crappy hobbyist programmer myself for many years, just because it's fun, I think of it as the sort of adult-approved way of playing video games.

As a 39-year-old father of 2, it's hard for me to justify playing video games, but if I'm coding, I can convince my wife that's a good use of an evening for me, although it totally scratches the exact same itch in my brain.

Just as AI is totally revolutionizing computer programming, it is in the process of totally revolutionizing the law. I think it's going to take longer, and we can talk about it if you want, because the law is a kind of professional guild, and lawyers are the one guild that, because they're lawyers, control the rules about who can be a lawyer, right? And so it'll all take longer. But that's another whole vector, right?

I think we should all care about that, because all jokes about lawyers aside, law is still one of the fundamental technologies of modern society. If you want to think of it that way, it's one of the main infrastructures.

Okay. So, you outlined 2 big areas there. One is basically policy with respect to AI, and the other is the impact that AI is making on the practice of law as it's happening today. In just preparing for this, I was looking at what measures we have to try to get a handle on how good AIs are getting. In general, I've been surprised across the board by how far the AIs have made it up the performance ladder, as measured by something like GDPval, where I saw that currently, in the lawyers category, there aren't that many prompts, at least in the public dataset, but Claude Opus 4/5 is currently the top performer.

It is winning one in three head-to-head comparisons versus human lawyers, and it's winning or tying 70%. That's like—you obviously made it pretty far. You guys can probably unpack that more qualitatively and tell me what it's good at, what it's bad at, and where people are having success and where they aren't.

But it's been striking to me—and I would say this is true in medicine, too—that there hasn't been nearly as much guild closing of ranks as I would have expected two and a half years ago, and I don't understand why. Maybe it's because people are ignorant about how far things have come, and they're living in denial, as opposed to making the moves that they might one day wish they had made if they had properly appreciated the phenomenon. But I guess, how would you characterize just how good at law frontier models have become, how much do most lawyers today appreciate that, and why isn't there more of a response so far?

Yeah. I think I'm curious what Kevin thinks. I think they're extremely good. Obviously, they're still held back by mistakes and hallucinations. They don't necessarily have access to all the databases that you would need to give a full legal answer, especially if the questions are obscure and require you to have read that one random SEC regulation that's buried in the Federal Register. These are obviously all fairly trivially solvable problems, and they will be solved in the next few years.

But in terms of pure horsepower, they're quite good. Some are better than others. In my kind of testing, the amount of money I spend on all of these models a month is horrifying, but I feel like it's part of my professional obligation to get a sense. So I find them different.

I think right now I've found that, although Claude is my daily driver and I mostly live within Claude Code, I find that calling out to 5.2 to ChatGPT 5.2—and then especially using the Pro extended-thinking model, which is—these names are so confusing—which I think you can only get on the web interface, because in Codex CLI there's the xhigh. The whole thing's a mess.

I think all of the labs are spending a lot of money on their custom RLA RLHF environments, and they're obviously focusing on different things. I think OpenAI, my sense is, has focused the most on law, and so, from a vibes perspective, I think its legal taste is the best. But right now all 3 will give you pretty good answers.

In my scholarship and in my writing, I'm constantly talking to these models, having them pressure-test my legal analysis. So I'd say already these models are certainly better than the median lawyer. There's no question about that, at least in terms of whatever kind of raw intellectual horsepower equivalent you would use. I see no reason why, in a few years, they won't be vastly superior.

There will still always probably be the question of bespoke taste. If you're a super-experienced Supreme Court advocate who has done 50 presentations before the justices, that's hard to RLHF. But the vast majority of legal work, just like the vast majority of programming work and the vast majority of medical work, is pattern matching across fairly standardized contexts.

So I think it's over, right? There's no question about this anymore. And I will agree with you that there's actually been a lot less pushback on this than I would have thought. A piece that Kevin and I are currently writing—a law review article—is actually about the use of AI in legal scholarship.

Again, I'm curious, Kevin, about your experience, but as I've presented that piece to faculties across the country, I was expecting a lot of tomatoes being thrown and a lot of people saying, “Oh, but they're just fancy autocompletes, and they can't be creative.” There's honestly a lot less of that than I would have thought. And I think it's because if you spend an hour talking to any of these models on the $20 plan, you just realize: if all they're doing is fancy autocomplete, then all I'm doing is fancy autocomplete.

Why hasn't there been as much resistance? First of all, I think there will be. Still, the vast majority of lawyers are not tech-savvy or interested in this, or they haven't really experienced it. So I think there will be a lot of resistance.

But for those lawyers who have experienced this, I think they're making a bet. This is the bet that I'm making: that there will be a kind of Jevons paradox—as legal services get cheaper, we will want more of them, and lawyers will move up the value chain. And so, although it will be messy, and although some lawyers will do very badly if they can't react in time, in 10 or 15 or 20 years, there's going to be, at the very least, as many lawyers as there are today, at least as much demand for legal services, and frankly, probably much more.

Whether that's true is the question, right? That is, whether Jevons paradox is going to hold, and across which economic domains, is the question about AI in the economy. But I think, given how important law is and given how much less law there is than there could be and probably should be in a very sophisticated rule-of-law country, my money's on Jevons paradox holding.

You're kind to call our country a sophisticated rule-of-law country.

Dude, I'm calling you from Minnesota. I'm trying so hard to stay optimistic right now.

So you're taking the long view. This will all be over at some point.

Yeah, let's hope so.

Let's unpack that latent-demand concept. I have no idea about law, but the way I think about this—and you can tell me if you think about it a different way, and then how you apply it to law specifically—is on a spectrum from dentistry on the one hand to possibly software creation on the other. Software creation is certainly, if not the most extreme, one that's being tested in perhaps the most extreme way right now.

Dentistry—I want zero dentistry services for the rest of my life if I can possibly maintain that. Whatever I have to have, I'll get, but I won't be opting into any dentistry just for fun, right? I'm going to buy the minimum that's required for me to have a good life.

I put accounting on that end of the spectrum, too. Accountants may have a different argument, but I will buy the minimum accounting that I need to buy to be compliant and to know what's going on. Beyond that, I'm not really looking for more. If you could give me 10 times the accounting for the same price versus the same amount of accounting at a tenth the price, I know which one I would pick, and I would pick the savings.

Computer programming, on the other hand, there's a lot of optimism that, hey, maybe we do have latent demand for 10 times or 100 times as much software, and everything will be bespoke and whatever, and we can imagine a whole new software-abundance paradigm. I guess for me, as somebody who's a relatively simple person and has a relatively uncomplicated life, my intuition is that law would fall more on the accounting side.

I do find so often that AI is a GDP destroyer in the sense that when I, for example, last went through a little contract negotiation—it wasn't anything super complicated—I just took what I got to a couple of language models, asked what I should be concerned about, shared my take, and we iterated through it. I didn't have to hire an attorney, obviously.

If there's going to be 10 times more legal services provided at the same cost, what are we not doing today that you would imagine us doing in the future?

I think it's important for non-lawyers to understand that we have a whole concept in the field of law referred to as legal deserts, which are areas in the country that have about 1 lawyer for every 1,000 residents. There's a whole lot of folks who just have no one to turn to when it comes to signing that lease, forming that small business, starting a nonprofit, getting out of that marriage, and so on and so forth.

There is maybe 1 person with a single shingle waiting for any clients who walk down Main Street, trying their best to help them out with a legal dispute, but they're often not a specialist, or they often charge too-high fees. I think there's a tremendous amount of latent demand just for better, higher-quality, faster lawyerly services that suddenly we're going to see a lot of lawyers be able to provide across the US.

That to me is incredibly optimistic because if you look, for example, at landlord-tenant disputes, there have been some trials where, if you just provide a little bit of legal counsel, for example, to a tenant, they have a much higher rate of doing well in that dispute than they would absent having some degree of legal counsel. I would say there's a tremendous amount of latent demand.

The other thing I'll add is that lawyers often like to refer to themselves as counselors, not in the way of being like a therapist or something like that, but in the sense that we want to provide wisdom, judgment, and foresight about how you're going to operate your business in this new legal domain, or how you should begin to think about legal architectures more broadly. That's where I think we'll have a new track of legal education.

I see a sort of bifurcation happening in the legal industry where we're going to have the folks who hang up that single shingle, go represent folks in landlord-tenant disputes, and take care of the rote tasks that lawyers need to do but that AI will take a big chunk of work from. Then I see a track that I would like to refer to—and I didn't coin this—as legal architects. They're operating at a bit of a higher, more abstract level, trying to analyze how systems of law and our regulatory structure should even begin to work and operate.

That's where I see a huge room for creativity and new training and a new sort of lawyering. For example, we have folks like Gillian Hadfield, who's done work with Fathom, and Andrew Friedman, thinking about novel approaches to regulatory design. I am so excited about that sort of work and really think that's going to be a new frontier of legal education that we should embrace and try to foster.

I'm not worried about my students having job opportunities, for example, but I will say that for the schools that are falling behind AI adoption, that's tremendously concerning to me. To touch briefly on the last question, there's still a number—I think Alan just gets invited to better law schools than I do when he talks about our paper. I've had to dodge a tomato or two, figurative tomatoes, from faculty who just don't want to hear about AI, want to make sure that it's not a part of certain courses, or that it's not introduced until students' later years.

The reality, though, is that kids in high school are using AI, if not well before that. By the time they come to law school, this is something we just have to adjust to and acclimate to so that they can succeed when they go into a law firm. We have a huge obligation as a legal education industry to make sure we're thinking about that future of law and preparing students to be successful in that domain.

Yeah, I agree with everything Kevin said. What I would add to that is, on the point of latent demand, in addition to the fact that there are actually a lot of people who are not getting legal services, I think there's again a popular sense that there's too much law and it's too litigious a society. In some domains that's absolutely true, but that's not an across-the-board thing, right?

There are so many people who can't get wills or divorces or whatever the case is. Even so many of us—how many interactions have you had, for example, in your business dealings that you handled by email because actually writing a contract was just too much of a pain in the ass? I certainly have done so.

In law, when you take contracts, which is your standard 1L course, and I think is actually, in some ways, maybe the most foundational legal course there is, it's fundamentally about the question of being precise in agreements, which is ultimately what the law is meant to facilitate. There's this concept of the—I think it's called the complete contingent contract.

That's the idea that if you and your business counterparty had infinite time and infinite energy and zero opportunity costs, your contract would be not infinitely long but almost infinitely long, because you would go through and figure out every single possible eventuality. How do I negotiate that to make a win-win situation with my counterparty across every possible contingency?

You can put in some economic theory and determine that if you could do that, that would be socially optimal, et cetera. That'd be great. But of course no one does that because you can't do that. The law has all these default rules, which are fine, but they're default rules, which means they misfire a bunch.

Now imagine a world in which we each have our own very sophisticated agent. When I want to engage with someone in any kind of transaction, my agents can go and have a conversation at the speed of 400 tokens a second, and they can come to an agreement. You're going to have orders of magnitude more legal demand there, in a way that could actually be quite beneficial to society.

I don't know if you get more lawyers in the end, but it's not obvious you get fewer lawyers. The other thing I would add is it's true that you don't want any more dentistry than you need, but law is a little different because law is a more competitive activity, right? You have a counterparty on the other side who is looking out for their own interests in the way that you and your teeth are fundamentally on the same side.

Once you get enough dental care, you've got enough dental care. With law, it doesn't quite work that way, because no matter how good your legal services are, if the other guy thinks that they can get better legal services, then they'll do that. There are these arms-race dynamics, which is again why I'm not saying that there's infinite demand for legal services, but I think there's a pretty big one.

And just to build on that really quickly, in addition to thinking about improving the basics of law, like contracts, Nathan, in our pre-recording session, when we were all just hanging out, we were talking about what AI in the future of governance looks like. One thing that I've been shocked by is that the more you dig into laws, and the more you realize what technology is capable of and what AI is capable of, you realize our laws really suck.

We're writing laws in the same way, with the same degree of expectations and in the same format as we would have seen centuries ago, right? And yet, to Alan's point and as we were discussing, Nathan, we can use AI, for example, to create new triggers: If, for example, the unemployment rate goes to 7% in this field, then we want to see this new economic policy; or if we see that tariffs are imposed by this country, then we want to automatically see this response.

There's so much room for smarter legislation that we're not even scraping the surface of, and that to me is another exciting field that lawyers haven't really, in earnest, begun to explore. Professor Hadfield is obviously leading the way in that regard, but we need a lot of little Gillians hanging around and going and emulating that study of what the future of law looks like.

So maybe let's work our way up the value levels there. For starters, there's one that I skipped over, and I wonder if there's data around this yet, or maybe just anecdata at this point.

Again, in the programming field, you do have companies starting to say—Anthropic, I think, is being the most vocal about this, arguably the most forthright about this right now—“We're not really looking to hire junior employees in really any department anymore.” And I think in the broader space of software, it's, “Man, I don't know.” If I had a senior architect and I could have them mentor a junior programmer or get another $200-a-month Claude Max plan, which is going to give me better ROI narrowly for the purpose of my project—obviously, there are broader questions about generalizing that strategy and what happens to society, which I'm not ignoring—but locally, it seems pretty clear that you're going to get more from another Claude Code than you would from a kid who came out of an undergraduate CS program that was all in Java or whatever.

There are so many disconnects there that you're trying to bridge, and Claude Code doesn't have or bring those problems to the table. Is that true at the paralegal level? I used to read John Grisham books as a kid, and I remember so much of the stories were these heroic, Herculean labors of underdog individual lawyers fighting one versus these large teams, just reading until their eyes bled through repositories of documents. That seems like probably the first thing that would be dramatically disrupted by AI. Are we seeing that? Is there already a revolution in discovery? I don't even know the full list of what paralegals do, but are we seeing that being majorly changed already?

Yeah. I would say that we're already seeing some industry shifts occur. Fortunately, I get to bring a lot of practicing lawyers to campus here in Austin and probe them about how they're using AI. I'm not going to name firms, but I've asked, “Hey, if I came to you with the number-one graduating student from Harvard, but they had no AI experience, and then I came to you with an AI whiz from a middle-ranked law school, who would you hire?” Now I hear more and more, “I would take that middle-tier person who's savvy with AI tools, because I want them to be on the frontier of finding new tools and teaching everyone else how to use them.”

One of the unfortunate things about the legal industry is that we love a good symbolic technological adoption. I think 70% of major U.S. law firms—that is, top-100 law firms—are using Harvey, according to Harvey's own stats. Harvey, for folks who aren't in the lawyerly weeds, is basically a souped-up version of ChatGPT that's meant to assist specifically with litigation workflows.

Yet when I go talk to folks who work at firms with Harvey and ask, “Okay, what training have you received?” they say, “Oh, there was some email we got when it was initially introduced, but I haven't checked it out since.” And then I ask, “Okay, are you expected to use it at all?” “No, there's really no obligation for us to check it out or to use it in any new fashion.”

The underlying incentive of practicing attorneys is to spend as much time as possible on any given task within the band that's acceptable to your client, because we have the billable hour. If you get paid by the hour, then your incentive as an attorney is to bill as many hours as possible. I think there are a lot of firms that are just used to that model and scared about bucking that trend—bucking what they know has worked. A lot of firms are not necessarily leaning into AI.

I will say that the rate of entry-level lawyerly jobs disappearing—I haven't seen a huge amount of shrinkage, but I do start to hear whispers now of firms saying, “We're just not sure we're going to bring on as many summer associates this year,” or, “Perhaps we don't need to hire as many junior associates going into the future.”

We're also hearing reports of, to coin Ethan Mollick's phrase, a lot of secret cyborgs in law firms these days. The ones who actually are AI-savvy aren't telling their superiors how sophisticated AI is or how many use cases it can actually address. So, it's a really dynamic time in the space.

Yeah, so I'm less plugged in, I think, than Kevin is to legal practice. If he's hearing that there are whispers around this, then I believe him. I guess I'm a little skeptical that this is happening already. I think the data about whether this is happening in the software engineering field is still quite unsettled, and there's a lot of debate over whether these big companies are actually using AI to not hire people or using AI as an excuse for downsizing they've already wanted to do.

Again, law is several years behind on the capability scale, and it's actually several years behind even that in implementing it throughout, both because law firms—and this is part of the guild rules of law—can only be owned and operated by lawyers. Lawyers, God bless them, are not generally brilliant business managers, and so driving managerial change is a hard thing to do.

Also, there are these legal practice rules around when you have a human being showing up in court, and that human being has to attest that they checked everything. If, God forbid, your AI hallucinated, it's going to be very bad for you in front of the judge. I think there are a lot of reasons not to be worried right now, in the next couple of years.

In the longer term, the question is, of course, how strong is Jevons paradox? It all comes back to this question of induced demand, and we're just not sure what the answer to that is. I think the more interesting question—or the question where we can have more confidence about what is clear—is that a lot of the entry-level jobs will just have to go away, and if they're entry-level people, they'll be having very different jobs.

Again, to your point, even a lot of entry-level lawyering is very rote work. It's doing a ton of discovery, finding needles in haystacks, and writing a contract based on the thousand contracts your firm has done before in this practice domain. That's just stuff that today's technology is already going to be so good at.

The question is—and we just don't know the answer to this question—is that work necessary on the way to becoming a really good lawyer? We don't know the answer to that question.

Let me give an example from software engineering that I think about all the time when I try to think through this question about cognitive deskilling, which is a fancy way of saying getting dumber. To me, that is actually much more than job loss—the big concern for me with these AI tools in knowledge fields.

That's actually what happened in computer programming. In the beginning, if you've seen The Imitation Game, the movie about Alan Turing at Bletchley Park, there was no computer programming per se. There were machines, and you would literally program the machine with hardware switches.

Then someone decided, “Well, it would actually be really nice if we did it in zeros and ones.” Then someone invented assembly language, which at the time was basically considered cheating. Now it's insane to think that assembly language was the easy option.

At some point, someone decided to invent the early programming languages. Those were really considered cheating, too. People thought, “Oh my God, if you can't program in assembly language, you're just not a real programmer. You're a moron.”

Every 10, 15, or 20 years in computer programming, there's a new level of abstraction that is developed. After that, people decided, “It would be really nice to have something that does garbage collection so we don't have to worry about memory management. Maybe we should have the Java Virtual Machine so that you could write once and compile on all the systems.” Then, “Let's just have Python so you can write in pseudocode.”

Every once in a while, you have this level of abstraction that, in some sense, makes the task of programming less cognitively demanding in certain domains and in certain respects. And so you could worry, “Well, that leads to cognitive deskilling.” It turns out that the scope of programming problems is essentially infinite. For most people, programming doesn’t become easier exactly; it’s just that they operate at a different level of abstraction, and you still have to be pretty smart to do it.

We’re having this current debate about whether this new programming language—which is to say, natural-language prompting of Claude—is going to have that same effect. My sense is that you’re suddenly going to need to be really smart to do this. You’re going to have to remember less syntax, but suddenly, at a much earlier age, you’re going to be thinking about architectural questions that 30 years ago it would have taken you 15 years to graduate into, because you would have spent those first 15 years remembering what the syntax or curly braces were in your programming language.

So the question is—and again, we don’t know—will that similarly translate to law? Will that similarly translate to medicine? Maybe you just don’t have to do organic chemistry anymore because, I don’t know, just as you don’t need to do long division once calculators come along, maybe you don’t have to do organic chemistry once the AI tools are sophisticated enough to do that.

Does that make incoming doctors dumber in a certain sense because they don’t have to study organic chemistry? Maybe, but now they can spend their IQ points on more interesting, high-level diagnostic questions. I don’t know the answer to that question, but certainly in my own practice, such as it were—and I’m not a practicing lawyer, but I’m a law professor—I’m finding, for example, that I’m using student RAs a lot less than I would have even a few years ago.

A lot of the tasks that I’ve had a student RA do were things like, “Hey, spend 10 hours clicking around, reading 100 law review articles, and figuring out which 3 of them are useful.” I can just have a little script that I wrote download a bunch of PDFs and send them all to Gemini. Gemini Flash summarizes them, and then a combination of Gemini Pro and the Claude API will have a little debate about whether or not the law review article is useful for my purposes. Then I get a beautifully formatted Markdown document.

Again, maybe that’ll be solved and I can figure out a different use for my students, but if I can’t, that will be a problem, because many professions have an apprenticeship phase. One of the reasons I became a law professor was that, when I was in law school, I was an RA for a really wonderful law professor. I did nonsense, crap work for him that I’m not even sure added value to his life, but I hung around him for long enough that I learned something about being a law professor. It became something that I was interested in doing.

If the next generation doesn’t have that opportunity, that is a problem. That’s why I think that, even if in the long term I am optimistic—because I do think Jevons’ paradox tends to work for intellectual work—in the short term, I think it’s really, really messy.

I think the people who are really going to struggle are low-agency people, for lack of a better term. People who expect that there is a way that you do things, that you go through the appropriate hoops, that you just grind. I think what AI does is create an incredible opportunity for people, but it does require a higher level of agency.

If you listen to Tyler Cowen and how he’s thought about the implications of AI in the labor market, I think that’s one of his main themes—the overarching theme of a lot of his work. In the long term, I think that’s great for society. You make more value that way by empowering high-agency people, but it sucks for the people who aren’t so high-agency in the meantime because they get left behind.

From their perspective, it’s a big betrayal, right? They did all the right things and the rug was pulled out from under them, which is where I think a lot of the political friction around this technology is going to come from. We’re going to see that in the next 10 years.

I think that point about the fact that a certain class of people—I think we’ve already seen this in the last several years, with so many kids coming out of college and not being able to get a job that really allows them to pay off their student debt in any reasonable way. The general sense is, “I did what I was told to do. I played by the rules, and somehow I’m still getting screwed.” When that hits a certain level—

And therefore, we should burn the entire system down.

Which is a tough thing for people to stomach. I don’t think burning the whole system down is necessarily the right answer, but I’m at least quite sympathetic to those folks. I’m also not unmoved by the fact that this is a high-class problem, but increasingly I’m thinking, “Yeah, it’s not that high-class of a problem.”

A society has to take care of the big middle class, for lack of a better term, that isn’t going to be an outlier relative to the system but is going to do what the system expects it to do. If that can’t work anymore, then you’ve got a big problem, and things can start to come apart pretty quickly.

Going back for 1 second to this legal-desert concept and Alan’s initial comment that the frontier models are better than the median lawyer—or I think you said the average lawyer—practicing today, I think that totally checks out, although I don’t have that data. From my own personal experience, I can say that, in the context of pediatric oncology, which I’ve unfortunately had a major crash course in over the last few months, things are going well. It’s been very clear at the hospital on a daily basis that the models are better than the residents, and they really do go toe-to-toe with the attending oncologists.

Can I ask you a quick question about that? Better at what? Because when you said—when I said the frontier models are better than the median lawyer, I don’t know. I always hear Ethan Mollick in my mind when I talk about the jaggedness of it.

When I say they’re better, I mean they’re, on average, better, but in certain ways they’re vastly superior, and in certain ways they’re completely incompetent. When you average that out, you get something that’s better. I would imagine something similar for medicine, too, where on certain diagnostic tasks, or certainly when explaining things in more layman’s terms, they’re vastly better.

Again, I’ve thankfully never had this experience that you’re going through, but I have 2 small children as well, and I can only imagine that, in a situation like that, the bedside manner of the resident, the attending, and the nurses with small children is so important. In that sense, I think we’re a long way from these models being better. The idea that a job is a bundle of tasks and only some tasks necessarily get replaced by AI is kind of how I think about it.

Yeah. Well, I think the hospital is a very different domain. In the hospital, the tasks are grouped into multiple bundles. For one thing, I would say the nurses are at much less risk of competition from the language models than the doctors.

The person who comes along—and my poor kid, again, he’s doing much better—in the early days, he was feeling terrible, and all this stuff was happening, and it was all very scary. He could probably tell that we were scared, and he was not easy to deal with at times. That mostly is a nurse’s problem.

Getting him to put the blood-pressure cuff on or getting his temperature taken has a bedside-manner component that the language models are not really touching at all. It’s funny: We’ve got this IV tower that kind of stands there all the time, and when the thing hits the endpoint of a medication it’s giving, or the IV drip is about to run out, whatever, it starts beeping. The doctors don’t know how to use that thing at all. They literally can’t do it.

Yeah, I’ve had that experience as well.

It’s funny how the lines between these bundles of tasks are pretty sharp in the medical context. The things that I’ve seen for the residents—the AIs aren’t showing too many weaknesses relative to the residents. The 1 area where I do see the human doctors still having a bit of an edge is the holistic, multimodal assessment of the patient.

As a parent, I can do that, and if it was my own self and I was of sound enough mind to do it, I could do this for myself in the same way I could do it for a kid. If I write a paragraph or so about generally how he’s doing and what we’ve observed over the last however many hours, and put in the test results and whatever, I would say the AIs are clearly better than the residents and, again, pretty much toe-to-toe with the attendings.

Sometimes, something I say to a language model might cause it to come back with a certain concern, and then I become concerned about it. Where I think the doctors have added value relative to the language model most of all is saying, “I’m just looking at him breathing. I’m looking at his color, and he doesn’t seem to be in distress. I really don’t think we need to worry about that right now.” That’s been the main mode where I think they’ve added value.

Usually, my understanding of what’s going on in language models is, yes, they’re definitely reasoning, though there are also some aspects of stochastic parroting still on the margin. So I think it’s oftentimes just a particular word or phrase that I use that kind of brings up some concept that’s now worrying me, and they can put my mind to rest.

Anyway, I don’t know what the equivalent of that is in the law, and I’m also wondering: What is the equivalent of prescribing? We do have the general sense that, in law, you can represent yourself, right? I can represent myself if I’m accused of a crime. I think I can pretty much represent myself in anything, right? I can certainly sign contracts for myself without needing to hire anybody.

So if I’m thinking about this legal-desert scenario, and I’m thinking the model is already better than the median lawyer or whatever, and potentially better than that—if I were to clone the closest lawyer in a legal desert, the model might still be better, right? Why is there a barrier? Is there a place that the legal profession can fall back to, like doctors are presumably going to fall back to prescribing? That would be the thing where, yeah, you can talk to ChatGPT all day, but if you want the medicines, you come through me.

Is there a version of that in law that will prevent just every random person from representing themselves with language-model backing, or is there not? Or do you think there will be one that will be created?

I think it’s important to flag that every state manages its practice of law. Every state has a state bar that dictates who’s authorized to actually practice law. Typically, you have to go to an accredited law school, then pass the bar exam, and then maintain continuing legal education for a series of years in order to represent someone, for example, before a court.

Then we have unauthorized-practice-of-law statutes. This is where each and every state basically forecloses someone from saying, “Hey, I’m on Craigslist. Trust me, I’ve read every law book. Let me represent you at half the rate of the attorney down the street,” right? It’s that unauthorized-practice-of-law statute that forecloses you from being able to do that.

It’s those UPL statutes, as we refer to them, that have prevented things like LegalZoom, right? They ran into a ton of hurdles in terms of doing things like wills and some real-estate agreements because you had the guild—the lawyer guild—defending itself against these new tools. There’s going to be a lot of friction for a while in terms of tools like, for example, Learned Hand.

I got to talk to Klapper. He started an AI startup called Learned Hand, which, for non-lawyers, is a very famous judge, so it’s meant to be pretty funny. This tool is helping judges, for example, and helping law clerks who assist their judges write better opinions and write them faster.

To your point, Nathan, I think the thing we’re going to see ultimately, or the thing I hope we see, is that we use these new AI tools to address some of the instances in which we see justice effectively be denied because justice is so delayed. Most folks don’t pay attention to the fact that 95% of all litigation occurs in state courts.

If you’ve ever had to go before a state court, they are not known for efficiency. You can be waiting months, if not years, trying to get some dispute resolved. Then, when you get it resolved, you may have gotten a judge who’s just not good at their job, right? Maybe they were hangry when they were writing your opinion, or maybe they have something going on personally.

The outcome of that dispute then isn’t based on the facts; it isn’t necessarily grounded in the law to the extent you hope it is. So we get arbitrary decisions, and we get random decisions that, in my opinion, shouldn’t be a characteristic of a good legal regime, right? The idea, in my opinion, is that everyone should be able to enforce their full rights and realize their rights.

Yet we rely on an adversarial system in which, to be blunt, whoever can pay the most money wins. That’s really messed up, but that’s typically how the law is resolved in a lot of these cases, because whoever pays their lawyers for the longest can survive, more or less, this adversarial approach.

If we instead move to a more systematic, consistent approach to handling the lower-level cases, to handling these more basic disputes, the role for lawyers then becomes managing what that legal regime should look like in the first place, right? That means trying to set, at a higher level, how we should structure society and the incentives such that they align with whatever that community’s values are.

That’s the role that I would say our appellate court system plays right now, right? You think of the U.S. Supreme Court or a state Supreme Court: they get to play the higher-level role of deciding how we should shape laws more generally.

That’s the role I see for lawyers in the future—taking that more hands-on approach of thinking through the ultimate ends of the law and making sure that the system is working in a consistent fashion, rather than the sort of ad hoc, just-hope-you-get-a-good-judge, flip-of-the-coin scenario right now.

I love that vision, and I listened to the episode, which is definitely a Hall of Fame, first-ballot, all-name-team Hall of Fame for both a judge and a legal startup. I definitely want to unpack a little bit more what this vision of the future of law looks like, but let me put you on the spot for a prediction.

Do you think we’re going to see states pass laws saying ChatGPT can’t give legal advice to protect retail lawyers?

I certainly think we’re already seeing that. Some state bar associations have significantly limited the instances in which lawyers can use AI. But on the other hand, we’re seeing states like Arizona. Earlier, Alan mentioned that only lawyers can own and manage law firms; Arizona just became the first state that upended that and now allows non-practicing attorneys to own—or, rather, non-lawyers generally to own and start—law firms.

We’ve seen states like Texas, for example, and Utah leaning into regulatory sandboxes in which AI tools can be deployed with much greater ease. As soon as folks start to see that there are cheaper lawyerly tools available in other states, they’re going to move their companies to those states, they’re going to handle their disputes in those states, and we’re going to start to see the law filter there.

That’s going to be where the pressure emerges from—not from state bar associations waking up one day and saying, “You know what? Screw it. Let’s just go with the AI. I think it’s pretty dang good.” It will be that sort of competitive dynamic.

Yeah, I would also say I think it’s going to be hard, especially in this era, to try to stop general-purpose chatbots from giving legal advice. Both from a legal perspective, unauthorized-practice-of-law statutes always raise difficult First Amendment issues, because it’s one thing to say, “Okay, you can’t represent yourself as a lawyer who can go into court.” Fine, that’s one thing. It’s another thing to say, “You can’t talk to someone, and someone can’t talk to you, about an interesting legal question.” That’s core First Amendment speech.

Obviously, there are blurry lines you have to draw, but I think it’s going to be hard to have such a broad limit on the output of AI models, which I think is pretty clearly protected speech. Whose protected speech it is is an interesting, almost metaphysical question. The models don’t really have rights, and the companies—I’m not sure they have First Amendment rights in models that they themselves barely control. I think users and listeners have rights in communicating, but that’s kind of an interesting, maybe academic, question.

So that’s the legal reason why I’m skeptical that you’ll have such broad prohibitions. I think also it’s just too embarrassing to do that. Enough people have used these models and understand how useful they are. It’s going to be such obvious guild-protective self-dealing to go out and say, “Henceforth, we ban the use of ChatGPT to tell you interesting things about the law in the state of Minnesota.”

What I do think the compromise is going to be is: Look, if you want to do certain kinds of legal transactions, you have to go through a lawyer. I think this is where earlier you asked, “Can’t you represent yourself always? Yourself?” It’s an interesting question.

I actually don’t know the rules about this. Certainly, if you’re too poor to have a lawyer, you can represent yourself. It’s an interesting question whether, if you’re rich enough to have a lawyer, you can nevertheless say, “I’d like to go into court and just represent myself in prosecuting this civil lawsuit.”

Kevin, are you nodding because you can do that, or are you not nodding because you have—

I’m fairly certain you can say, “I’m just not going to.” You can represent yourself pro se and just say, “Screw it, here we go.”

But my question is—and I just want the answer to this—if you’re in a civil context and you say, “Hey, judge, I’m going to represent myself pro se,” can the judge say, “No, you’re not”? Right? Because you’re—because I don’t want to deal with you pro se, and you’re not a poor person, so you can afford to have a lawyer, so I’m going to make you have a lawyer.

I just don't know the answer to that question. It's not something that people have really had to think about because, if you were rich—or, put this way, if you were not poor—the chances of you getting a good outcome representing yourself were so low that you just paid for a lawyer. The thing about AI is that it changes that equation, right? Even if you're rich, the marginal benefit of a real lawyer is not always necessarily going to be that high.

Maybe you just pay for your $20-a-month ChatGPT subscription, or, if you want to be really fancy, your $200-a-month subscription so that you can have the Pro model and get really good legal advice. Maybe the compromise is going to be that there's a lot more free-floating chat legal advice out there, but the bar associations and the state courts get a little more restrictive: “Yeah, but at some point in the process, you need a human lawyer,” either because they think that actually adds value and provides consumer protection, or just improves the legal system, or is pure guild protectionism—or, as is usually the case with these things, a mixture of the two.

You're seeing something similar with medicine and mental health treatment, where it's very hard, I think, to say ChatGPT can't give you medical advice. We're not going to let you upload your test results or your kids' test results to ChatGPT so you can get a second or third opinion. But we are going to hold the line on, “Yes, but if you want the morphine, there has to be a human doctor who writes a prescription for that.”

So I asked Claude, by the way. It says that your right to pro se representation is strongest in criminal trials. There are exceptions related to mental competency, timeliness, disruptive conduct, and standby counsel. Judges can't appoint advisory counsel over your objection.

It's weaker in civil cases, as you suggested. For corporations and other entities, some appellate courts and some circuits have held that there's no constitutional right to pro se representation in criminal appeals or in certain specialized proceedings, including immigration courts, et cetera. So, as always, it's complicated.

Okay. So, the vision for the future: I think the point about whoever has the biggest budget tending to win is the depressing reality. Certainly, one of my great hopes for AI broadly is that, by making access to expertise far more universal, accessible, and affordable, lots of things could be better, and a more just society is one of the great promises there, for sure. How do you see that working in practice?

I guess one thing that I—maybe this is wrong—but when I think about the bigger budget translating to winning, I imagine that being a reflection of too much law. What are they doing? It seems like there's just so much law out there, so many things I could argue, and so many precedents I could bring in that I can spend hours and hours, almost indefinitely. That, to me, suggests we might need a simpler system in some ways.

But that contrasts with your earlier vision of certainly more extensive contracts, which I also projected into maybe more sensitive or more exhaustive legislation in the first place. So, what does that look like in your mind? How do we get to actual justice when, let's say, we all have infinite AI lawyers? How does that translate to justice? What does that look like?

Yeah, so it depends a lot on what the marginal utility curves look like of extra legal thinking, right? My hypothesis—and no one knows the answer, so take this for what it's worth, which is not a lot—but my intuition, and I'm curious what Kevin's is going to be, is that the reason law has gotten so expensive is that, if you think of law as a kind of combinatorial search space of arguments and precedents, can I find in these billions of documents the one sentence that is going to show that my client should prevail in this contract dispute with your client? If you think of it as having to search this very large combinatorial search space, largely that search had to be done by humans.

Obviously, legal tech long predates legal AI. It's at least 50 years old, dating back to the dawn of digitizing legal databases. So, Westlaw and Lexis, which are the main databases lawyers use, are very old companies. They used to do everything with paper books, and then in the ’70s and ’80s they digitized everything. That was a huge deal, right? More recently, you've had some machine-learning-based discovery tools. Nevertheless, you still need a lot of human beings locked in a conference room to do discovery, and those human beings are extremely expensive. Human labor is just extremely expensive.

Because the cost of that extra human labor was still less than the marginal benefit of exploring a little bit more of that combinatorial search space, the effect was to increase the aggregate cost of litigation, right, as Kevin mentioned earlier. Now imagine a world where you have AIs, and they are 10,000 times—3 orders of magnitude or 4 orders of magnitude—more effective than the current ones are, and they're also 4 orders of magnitude cheaper. You're getting something that's effectively 1,000,000 times better in the next few years. That seems totally plausible if you look at epic AI log curves and stuff like that. It seems totally plausible to me in the next few years.

You may get to a point where that actually exhausts the practical combinatorial search space of legal moves that are actually helpful to you. There's just no more precedent to explore. You have read every single sentence of every single piece of electronic discovery. At that point, the arms race ends a little bit, and now there is a natural ceiling on the cost of legal services because there's just nothing more to spend on. That seems plausible to me.

It's also plausible that that's not the case, and lawyers will always discover ways to increase the combinatorial search space. So it will always be more expensive, et cetera, et cetera. If, in 10 years, Kevin's very optimistic vision of the democratization of legal services comes true, I suspect it's going to be because we've just exhausted the scope of legal stuff to do.

Here I'm actually arguing a little bit against myself because now I'm talking myself into, Nathan, your point earlier that maybe law's a bit more like dentistry, where at some point your teeth are just clean, and they can't get cleaner, so I just don't need more dentistry than that. And I don't know. The problem is we're trying to predict these dynamics. These dynamics are all compounding, and so tiny differences in what you think the percentage rate of improvement versus cost reduction will be, or how much the legal search space will increase, can lead to massive changes in your predictions over the next 10 years. That's why I think there's a lot of uncertainty in trying to predict the effect of AI on law, medicine, computer programming, investment, or whatever the case may be.

I'll just add that, if you look at a civil procedure textbook, you'll see that the way litigation works right now is basically a series of very complex procedural steps. Everyone always has at their disposal a number of motions that they can throw out there to delay the process further. Some of those can be in good faith, right? You want to challenge whether the litigation should proceed to another step because perhaps the other party hasn't actually made any valid legal claims, or perhaps you want to challenge the source of information for different legal claims, and so on and so forth.

It's a lot of procedure. It's a lot of process. What I think can really start to reorient things, as you were keying up, Nathan, is: What if we start to move toward outcome-based law? We change the orientation from how many steps we can march through to resolve this one very narrow dispute to both parties wanting to see X happen. Our agents, which have been trained on our incomes, our preferences, our aspirations, our professional goals, and so on and so forth, can autonomously act on our behalf to continuously update whatever agreements we've reached with other parties or other corporations to achieve that end.

That is, to me, the more optimistic and very sci-fi, but eminently possible outcome. That's the outcome that I think we may eventually work toward: Let's make sure the law is oriented toward what we actually want to see, and not just in the sense that we should assume that more procedure or more process is better.

In many ways, this is what Professor Nick Bagley has coined the “procedural fetish” of lawyers. Our answer for trying to make everyone feel fair is to give them more opportunities to speak up. But usually it's not a representative sample of folks who actually show up at those opportunities to speak out, get involved, or throw gum into the cogs of the system. So how do we actually achieve what we wanted to achieve from the outset in passing that law? That's the outcome orientation that I think we could achieve if we lean into this.

So, I guess I don't really know what we're trying to accomplish in some of these contexts. For starters, going back to the Learned Hand episode of Scaling Laws, one thing I was struck by there—and in your description of all this process and the fully exhaustive set of things one might do to represent their clients, reaching an end state—was that you think, “Jeez, I feel bad for the judges.”

I was always struck, listening to that episode, that the judges are in a similar position to doctors today, where I think they're just overwhelmed by stuff, by and large, and welcome the help.

That’s been my sense of how doctors are typically feeling. They’re like, “I’ve got hours of charting to do when I get home. So, if somebody can handle that, that’s an easy win. And if you can come prepared to be a better patient, for lack of a better term, in the management of your own health, that’s a great win for me, too.” I’ve seen some skepticism, but I really have not seen any hostility or sense of threat in my experience in the medical system. I do think a big part of that is just because they’re overwhelmed and they know it.

So, help is welcome. It seemed like that was the vibe that the judges had, too. But now I’m wondering, okay, we’ve got one vision here that is this sort of idea that every corner case of agreement is articulated in advance. This seems to line up—and I’ll preface this by saying I don’t really have a great command of these terms or a deep understanding—but in prepping for this, I did some research and hit on a study that showed that GPT-4, which already shows that the work is dated, was more of a strict formalist. That was contrasted with the human judges, who were described as more legal realist.

Correct me, but I think basically that means GPT is following the letter of the law, and the judges are doing what I think the Supreme Court is often criticized for doing, which is making the decision it wants to make and then justifying it however it wants to justify it. But I’m torn on which they should be doing, because at least historically, I don’t think we’ve written laws so well that following them to the bizarre conclusions one might reach if one were truly formalist about it is obviously a great way to go. At the same time, obviously you’ve got room for bias and all sorts of problems if you just let people exercise their judgment too freely. That’s why we have a whole legal system, so it’s not just people getting to dictate how things are going to go with no checks on whatever they want to say.

Then we’ve got Claude’s Constitution, where I think Ameca has made really interesting points around not wanting to just give Claude a long series of rules that it has to follow, for multiple reasons. One of the most compelling ones that she articulated is that if the model knows that it could do something that would be better for the person it’s interacting with but has to follow these rules, they worry that it might generalize in a problematic way. They’ve seen this in reward-hacking contexts and other experiments, where if the model reward-hacks and starts to develop some sort of self-conception as the kind of thing that reward-hacks, then it becomes more evil in general.

So, they think a very analogous problem would be if a model knows that it really could do something better for you but follows the rule and doesn’t. They’re worried that could become a problem: what kind of person does that, and how does that kind of person behave in other situations? Obviously, just following orders doesn’t always age well. I don’t know how to tie that all up into a question, but it seems like we have a desire for edge cases to be all spelled out and everything to be in black and white, so that we know in advance what we’re getting ourselves into. Maybe we just haven’t been able to push that to the extreme where it can actually work.

But we’re definitely getting a different signal from Anthropic right now, where they’re saying, “We don’t even want to try that. What we want to do is get our AI to have the best possible judgment it can have, so that it knows how to be good even in highly ambiguous situations.” So, I guess, do you have a sense for which way the law ultimately goes?

I want Alan to take the first stab at the Claude’s Constitution answer here, because he’s got some deep philosophical views. I do want to briefly hit on the use of AI to precisely and perhaps perfectly try to read the law as it’s written, in a sort of clear formalist mentality like you were mentioning, Nathan.

I think the issue with that is one of my favorite questions that always gets raised in any good statutory-interpretation exercise. Imagine you’re going to a park, and there’s a sign right when you’re going to the park that says, “No vehicles allowed.” Is a drone a vehicle? Is a stroller a vehicle? Is a scooter a vehicle? Is an ambulance a vehicle? So on and so forth.

There’s so much ambiguity, even when the drafter of that rule may have thought, “Oh, vehicle, I’ve nailed it. Clearly, I was only referring to a car, and therefore everything is settled.” That’s why we’ve always had some variance from perfect formalism, or perfect textualism, as many lawyers would refer to it. It’s just saying, “Whatever the law is as written, we’re going to apply it.” We just don’t have the words for every scenario.

Obviously, AI can assist with coming up with many more words and many more laws, theoretically, but that’s not the sort of world I think any American wants to live in. We have a common-law system here, not a code-based system. If you want to experience a code-based system, go live in the EU, where they attempt to govern and regulate more precisely every kind of behavior.

Whereas in the US, we’ve tolerated some degree of ambiguity based on the reason that we need an iterative, emergent approach to discovering how it is we actually want to govern ourselves. The trick for AI, and the trick for the legal adoption of AI into adjudication, is finding out how to use a system that can create more words and resolve textual disputes with greater consistency and in a greater fashion, while still allowing for that emergent process to continue.

I think, between Alan and me, and for a lot of folks, having a world in which you don’t feel like, “Okay, if you step on this crack, you are automatically going to receive a penalty in the mail, it will be sent to you within 5 days, and it will be taken out of your bank account,” is a scary world that I don’t think any of us want to live in. Maintaining this balance of higher-level rules that guide us generally, as you alluded to in Claude’s Constitution, and then enforcement of those rules is a really tricky issue that could be the subject of a whole legal seminar. Maybe we should just get one on the books, Alan.

Yeah, I think that’d be fun. So, let me say 2 things. Let me say one about the use by judges and then the broader Claude’s Constitution question.

I was lucky in that I had the opportunity to go and talk to some Minnesota state appellate judges. These are state courts, but they’re appellate judges, so they’re a little bit removed from the absolute crush of the trial stuff. One thing that surprised me was how open they actually were to potentially using these tools. There was a lot of skepticism, which was appropriate, and some hesitancy, but again, there wasn’t the sort of tomato-throwing that I thought you would expect.

These are judges, so they tend to be on the older side, frankly. You can imagine a kind of natural aversion. There wasn’t that much of that. If you just spend an hour talking to the $20 version of Gemini, Claude, or ChatGPT, you quickly realize that, whatever the long-term societal effects, this thing is pretty useful. So, I do think we’re going to see a lot more of it.

How judges use it is tricky, and I think the kind of research that you mentioned about GPT-4—again, it’s unfortunate that these things get out of date pretty quickly. We need a better research pipeline to have these evaluations come out within a month, not within a year and a half.

I would also say that I did not take that research to say that GPT-4 is textualist and therefore models must be textualist, or formalist, rather, and therefore models must be formalist. It’s just that, for whatever reason, that model, in the way that it was trained, gave a more formalistic answer on some corpus of legal questions.

It’s GPT-4, so there probably wasn’t specific legal RLHF in the way that there may very well be with these newer models, and certainly with the legal-specific models. For whatever reason, the way it was trained meant that, on some corpus of legal questions, it gave a more formalistic answer.

You could have a model that gives a much more functionalist answer, which is less concerned about the specific language of the law and more concerned with, “What were the legislators trying to do, and how do we apply that to this question of no vehicles in the park? Should a drone be a vehicle?”

I think you’re right to view Claude’s Constitution, to get into that part of your question, as taking a position that, in some sense, you want reasoning—whether it’s artificial reasoning or human reasoning—to operate more at the level of principles than at the level of rules. But I would push against thinking about this as a binary.

There are no pure textualists in the world. There is no one who is so committed to the letter of the law that they would not consider the purposes of the law, or would not deviate if there were an obvious mistake in the law. No one exists like that. Similarly, there’s no one who’s such a legal functionalist or legal realist that they don’t think the legal text binds them at all.

Everyone is somewhere in between, and frankly, most people are, relative to what the spectrum could be, pretty clustered in the middle. 15 years ago, this was reflected on the Supreme Court by Justice Antonin Scalia on the formalist end. He literally wrote a law review article once called “The Rule of law is the law of rules.”

And then, on the other end, there was Justice Stephen Breyer, who would often start with, “This is very complicated. Here are 17 factors that I’m using to think through this problem.” They actually went on almost like a buddy-cop tour of lectures around the country, where they would debate in a good-natured way. It was fun to watch. But what you really realized when you saw this was that they were basically all in the middle. Scalia was on one end of the middle, and Breyer was on the other end of the middle.

I think the lesson from that, and the way that I would read the Claude’s Constitution document, is that you need an intelligence—any intelligence, whether natural or artificial—to be able to operate both at the level of principles and rules. A lot of what we think of as judgment—or, to use the kind of fancy phrase from Aristotle, phronesis—is that ability to operate at both levels.

I mention Aristotle because, to Kevin’s point about my philosophical interest in Claude’s Constitution, when you read that document, you really have to appreciate that it was written by someone who has a PhD from one of the best philosophy departments in the country in moral philosophy. Amanda Askol understands academic moral philosophy. She has read the Nicomachean Ethics. At least as I read Claude’s Constitution, it is footnotes on that document, which is in no way a criticism. I think all ethics should essentially be footnotes on Aristotle.

I read her as saying Aristotle was right that it’s very hard—basically impossible—to derive any comprehensive set of rules of ethics. You need to have a real sensitivity to principles, but that doesn’t foreclose the use of rules in a particular domain. Sometimes the best principled approach to an ethical domain is to say, “It would actually be really helpful to have some rules in this specific ethical domain.”

In fact, when you read Claude’s Constitution, it toggles between high-level principles. There are, quote unquote, 17 of them, in no particular order of priority. Then there are a couple of rules where no principles are applied. Claude will not create child-sex material. You can have a debate with Claude about the principle, but it will not do it. Claude will not create—or at least, hopefully, unless it’s jailbroken, in which case something terribly has gone wrong—by design, Claude will not help you develop airborne Ebola or something like that. It just won’t do it. So even there, there is a recognition.

I think the question for me is not so much, “Should we do rules or standards? Should we do principles or technical rules?” It’s always a yes-and. It’s how you tune the distribution between those two.

What really excites me about AI is that we’re able to do what people sometimes talk about when they distinguish between in vitro experiments and in vivo experiments. There’s this new thing called in silico experiments, where you try to take some part of human life and model it in a machine. The benefits of that are that in silico experiments can be done at a speed and scale that are so many orders of magnitude greater than anything you can do in the real world.

One thing that excites me, as someone who’s interested in law for law’s sake, is that we can run experiments within machine-learning models about how a well-developed legal system works and exactly what the distribution should be between principle thinking and rules thinking—experiments that you could never run in the real world.

I wrote this Lawfare piece recently about Claude’s Constitution, and I ended with this reflection: We’ve been debating this question of rules versus standards and ethical reasoning for literally thousands of years. What’s cool about these machines is that we can run the experiments now. I think we’re going to learn a lot not just about machine intelligence in the next few years, but about human intelligence, because we can now simulate it at scale and tune the dials with precision in machines.

And just to add that onto a human law context, I think future generations are going to look back at the level of sophisticated AI tools we had available right now and be flummoxed that we weren’t asking our legislators to run proposed laws through simulations about their intended effects and likely outputs. Similarly, with respect to judges writing opinions, and not asking, “Hey, find all the ambiguities that are latent in this text before I publish it.” They’re going to be like, “What the hell? You had this ultimate tool at your disposal to catch blatant errors. What are you doing?”

So I think this is a great model for folks to follow with respect to that simulation idea.

One of my mantras for AI that you’re calling to mind is: AI defies all binaries. So I definitely agree with your response there that it can’t be all one or the other. I’m yet to find a good exception to that general guideline or general expectation.

How does this simulation work? I also get really excited about in silico experiments when it comes to science. Can you sketch out what that looks like in law? Do we start with a bunch of scenarios and what we think the right outcome should be and turn them into an eval, like we turn everything else into an eval? Or am I living in one of those simulations right now, perhaps?

I think one of the more promising things is forcing legislators to actually do their job, which is difficult: saying what you actually want to have happen with this law.

If you look at something like NEPA, the National Economic Protection Act may get it wrong. Everyone just calls it Environmental Protection Act. Everyone just calls it NEPA. This is the law that has famously flummoxed the ability to build affordable housing in a lot of communities because it creates a lot of veto points for individual stakeholders to find a way to gum up the wheels of new development.

My hunch is that we could have forecast some pressure points that may be exploited by bad actors, or perhaps well-intentioned actors who are just more expressive than others. We could have identified: “Huh, is this actually resulting in the sort of pro-environmental, pro-green, or pro-climate-change—or anti-climate-change—outcomes that the drafters of that legislation were actually hoping to achieve?”

If you ask legislators, “What are your explicit goals with this legislation? What problem are you actually trying to solve?” and then create evals based on that—“Have we seen a reduction, for example, in carbon emissions? Have we seen a reduction with respect to, let’s say, a congestion-pricing bill, in the number of cars going into the city?”—those are all things we can evaluate and map out.

That’s the forcing function to me: saying, “If you’re going to propose a law, what is the problem you’re actually trying to solve?” Then that becomes the core source of information.

What should we talk about very briefly in closing? I like the idea of essentially red-teaming. I’ve never been very involved in a red-teaming-of-a-bill process until SB 1047 last year. There was a lot of red-teaming of that, and that was a pretty interesting process.

I do think everybody ended up agreeing. I’ve become friends with Dean Ball, who led the initial critique of that bill with his writing online. Even he came out toward the end much happier with it than he was at the beginning. So I think everybody agreed that putting it through its paces and really gaming out how different actors are going to respond to this, and whether we’re really going to achieve what we want, was a pretty successful process.

To think that could be done in general sounds like a very promising enhancement to our legislative process.

Good luck talking members of Congress into that. We’ll see. I don’t know how aligned they are. The first misalignment we may encounter might be between the elected officials and their constituents. Nevertheless, I like the idea.

Maybe just in closing, what other kinds of big ideas do you think people should be thinking about more?

Yeah, I’ll go first, and then Kevin can have the last word. I definitely think you should have a right to use these models, in the sense that I think the First Amendment is probably the right kind of legal home for that. I think you already do. I think this will come up at some point, but I don’t think courts are going to have much difficulty saying that people have the right to access these tools in the same way that they have the right to access libraries to read books.

That’s the kind of negative right, which is to say, you have the right to not have the government forbid you. There’s a corresponding positive right, which is that you have the right for someone to give you compute, essentially.

There are also all sorts of interesting arguments about various kinds of public options. They’re often discussed as public options to build models, but I think, in some sense, public options to give people compute credits. Compute budgets might be interesting. You could write a sci-fi story—or I think I could get Claude to write a pretty interesting sci-fi story—where, in the future, the currency is compute. The main credit that people pass around is the credit to compute because that is so valuable.

To your point, Nathan, about how AI dissolves all binaries, I tend to agree, with the exception of one: the binary between there being a limit to how much compute is useful in the world and there being no limit.

I think that AI shows that there is no limit, and so I think in that sense AI is at the extreme, not in the middle. But to me, I think—and Kevin sometimes rolls his eyes at me because I think he thinks I'm too credulous about this—the question of AI welfare, which is to say the welfare of these models and the legal implications of that, is something that is very easy to dismiss but is going to be an increasingly important issue.

Either these models, as an actual kind of cognitive or metaphysical matter, will become increasingly sentient—I have trouble ruling that out, although it breaks my brain to think about it—or, more importantly and more immediately, as these models become more personable, as people develop more relationships with them, and as their memory improves. The more I talk to Claude, there's a point at which Claude knows me better than my wife does, which is totally plausible because I just talk to Claude constantly for everything.

If you combine that with real-time voice and video, suddenly your AI chatbot has an avatar that you can interact with. And then, certainly, once that AI avatar is embodied in robotics, which I think is going to happen—it'll take a while, and it may take longer than we think—but I'd be shocked, really shocked, if in 10 or 15 years we don't have very convincing, real-time AI companions that people get extraordinarily attached to. What sorts of rights will people demand for those models?

I think it's something that could cause real societal cleavages because I think you're going to have groups of people who are really committed to the idea that these models are, for many practical purposes, sentient entities that we are enslaving or, at the very least, potentially treating very poorly. And then you have other people—and I think this may actually be a source of really interesting religious cleavage in the next 20 to 30 years—who think that the very idea of models as sentient is a literal affront to God. It's a kind of idolatry, and the only correct response to it is a Dune-style Butlerian Jihad.

And then there's going to be this messy middle of people who are just like, “I don't know what's going on. I just want a chatbot.” I think that's going to be a very difficult transition at the legal level, certainly, but especially at the social level. And I think people who say, “No, that's not going to happen. That's science fiction,” are fooling themselves.

So I'd say the negative right that you all were referring to is generally encapsulated within the idea of a right to compute. If this is the first time you're hearing about the right to compute, it's actually been enacted in Montana. There are bills in Ohio and New Hampshire, and I believe a couple of other states, advocating for the right to compute.

And I believe this is one of those major rights, Nathan, that folks are going to be clamoring for sooner rather than later, basically saying that we do need additional protection against the state really infringing on your access to computational tools of all kinds—not only AI, but whatever's coming down the pipe. There should be a higher threshold before the government limits your ability to express yourself or to receive information via these new tools.

The other one that I think is also very interesting in this world, in which compute is obviously a scarce resource that's very important, is data. The other one that we keep hearing about, but far too few people are discussing, in my opinion, is the right to share—meaning the right to share your data as you see fit—which is a really important right.

Because right now, if you want to share, for example, your kids' educational information with a new AI tool provider because you want to train the best AI tutor out there, so that your kid, who perhaps learns differently, or you just want a different curriculum, can make use of that AI tool, FERPA, the federal privacy law that applies in that context, is a real burden to being able to share as much data as possible, as regularly as possible, without literally signing things and doing so on a yearly basis. And I think that individuals, if they want to share their data and want to make that a frictionless process so that they can train better AI for their own personal uses, that should definitely be a thing.

Because we all don't have the ability, for example, what is it to go to that fountain? Is it fountain the fountain of youth thing that all the like super healthy people are going to and they're downloading all of their data, they're getting all these scans, then they're sending it to some AI outfit to recommend personalized health outcomes. That's awesome. But only wealthy folks can go spend a week in Florida or whatever that is, downloading everything about themselves. The rest of us are just left with whatever Walgreens told us at that last checkup.

So let's make it as easy as possible for folks to use their data as they see fit, and that, to me, is a promising outcome under the right-to-share idea.

What about things that we maybe should be thinking about restricting the government from doing? Because I do have the sense now that we're probably already in an age—it’s been, whatever, 10 years since Snowden—and I'm wondering, if there was another Snowden, what would they be telling us? I would have to guess that we've got some sort of LLM dragnet phenomenon going on somewhere.

And there's this adage generally that everybody's committing a felony a week or whatever, and it's just a question of security through obscurity: nobody's really targeting you, and whatever. But that could change very quickly. We're starting to see, obviously, weaponization of the Justice Department, et cetera, et cetera. Should there be new restrictions on what the government can do with AI?

Yeah, I think that's hugely important. I actually wrote a piece for Lawfare a few months ago, and I gave a speech at a law school called “Unitary Artificial Executive,” all about this idea that one of the effects of AI—and near-term AI, not speculative AI, but near-term AI—is to hugely increase the power of the executive branch and the president in particular, both because of all these additional abilities AI gives the president, like perfect enforcement, surveillance, creation of propaganda at massive scales, all that sort of stuff.

And then also, for the president, him- or herself, a much greater ability to control the executive branch, which is millions of people and is very hard, just as a bureaucratic management exercise, to control. But if you have an AI that is trained on the president's preferences, injected at all levels of the bureaucracy, reading all the emails and reading all the texts, you can have a situation where the president really controls, in a much more practical way than he's ever been able to, whatever his legal authorities might be, the executive branch.

And that's, at the very least, complicated. It might have some benefits because elections should have consequences, and the people voted for person A and not person B, so presumably the executive branch should reflect that. On the other hand, again, I'm calling in from Minnesota; it's not hard to imagine the potential abuses of that.

And so I think one of the really important issues in the next decade—because the government is slow to adopt technology, although it does inevitably get there—is going to be: How do we, on the one hand, encourage—because I'm fundamentally an AI optimist—the government to use AI to really improve government services and increase state capacity, which is something our government has not always been good at?

I think that's part of the reason why we're seeing some fraction of the societal discontent, this kind of “burn it down” mentality: the feeling that we're paying a bunch of taxes and the government's not doing anything useful. AI can really help with that. On the other hand, you don't want to supercharge the government through the use of AI, and figuring that balance out is very tricky.

For me, I suspect it's going to be the thing that I think about—my main thing—for the next few years as an academic. But it's far more important for the legislators and the bureaucrats, the company executives who are selling these tools to the government, the politicians, and executive-branch officials to figure this out as well.

And just quickly, I'll add that I think there's some real concern around updating the Fourth Amendment that we need to pay attention to. There are some folks who've realized that, in theory, the government now has an incredible ability to tap into basically every system for detecting and picking up audio. But if you're speaking publicly, just hanging out, saying whatever, talking to your friend, the idea that all of that audio information can now be hoovered up, analyzed, synthesized, and then studied by the government to see who's planning what, who's thinking what, who wants to do what, all without real notification—that's tremendously scary to me, just to think about that sort of pervasive surveillance. That is the issue that I'd really flag.

And I would just encourage, again, on the positive side, for governments really to lean into regulatory sandboxes when it comes to testing new AI systems, erring on the side of saying, “Let's try to deploy this tool and make sure that folks have noticed that we're doing so, have a means to provide feedback, but let's not be afraid of literally reinventing the wheel, improving our processes, and improving our laws.” The rule of law—and law generally—has never been more important, and the intersection with AI is obviously ramping up and likely to become one of the big questions of our times in the next couple of years.

Kevin Frazier and Alan Rozenshtein, thank you both for being part of The Cognitive Revolution.

Thanks for having us.

Thanks, Nathan.