[BidClub_]
Latent Space · · 67 min

Building Legal Agents When You Can't See the Data — Jacob Lauritzen, CTO of Legora

Basil ChathaswyxJacob Lauritzen

AI & SoftwareCompany BuildingTechnical
YouTube ↗
TL;DR
  • Legora’s defense against frontier labs is deep legal workflow ownership. swyx argues that the labs “are never gonna go as deep as we are.” He says Claude may handle smaller corporate matters, but that for a “$400 million M&A transaction,” clients will want Legora.
  • Confidentiality makes improving a legal agent unusually difficult. Basil says Legora cannot inspect customer traces because documents often belong to law firms’ clients, so the team must reconstruct representative workflows for evaluation. Jacob describes an evaluation team of about three engineers, three lawyers and a pool of 100 legal engineers; Basil’s frustration is “I have no idea what people are actually doing with it.”
  • Model gains are commoditizing routine tasks while complex analysis still needs more intelligence. Jacob estimates roughly 60% of existing work can already be done well, making cost and latency the focus for that portion. One citation problem improved from about 70% to about 99% after a model release—“we did nothing”—but regressions elsewhere make task-specific evaluation important.
  • The pressure on junior labor is substantial, but replacement remains contested. The episode’s “Paralegals, gone” claim meets Jacob’s “That’s actually not what we’re seeing”: strong juniors can become more valuable. Meanwhile, competitive pressure is pushing some routine US legal work toward flat fees or free service, with revenue remaining in strategic advice and client relationships.
  • Product expansion requires more people despite reusable software primitives. Jacob’s plan for 20 engineers has become 100, and the team still feels “way too small.” Lists, projects and agent runs provide the desired software scale; legal engineers configure workflows, while forward-deployed engineers write production code and integrations.
  • Law firms could build durable intellectual capital, but partner incentives work against it. Jacob wants firm-specific judgment and precedent codified into assets that outlast individuals. The challenge is getting partners to “build an asset for the company” when they are rewarded for bringing in clients and receive no dividends after leaving.
  • Governance and retrieval engineering remain priorities alongside better models. Basil says Azure’s existing acceptance helped Legora enter law firms, while data-processing agreements constrain its vendor choices. He says Legora has not fine-tuned models so far; the opportunity is “very specific, bounded tasks,” such as cheaper, faster legal-research retrieval, rather than training a frontier-agent competitor.
Digest · the substance, structured for research

1. Legora is moving from legal answers to legal operations

  • Lauritzen describes the starting product as ChatGPT with a Swedish law-search tool: answering research questions by finding precedents. He corrects the chronology—his own YC batch was 2023; Legora’s was 2024—and tentatively dates the assistant-to-agent transition to about a year earlier. The company began as a small Stockholm startup competing as an underdog.

  • The product now drafts, edits Word files, organizes documents and performs complex analysis. Lauritzen’s distinction is between “doing the work” and “orchestrating the work”: Tabular Review handles bulk document analysis, while Lists, Intake and Portal manage tasks and collaboration among clients, lawyers and agents. Word and Outlook add-ins meet lawyers inside their existing tools.

  • The interface can fairly be described as ChatGPT or Cowork for legal work, Lauritzen agrees. The larger question is what replaces “a 200-thread email” with “V5 Word files back and forth.” If capabilities improve and prices fall, he imagines a command center where lawyers react to five things agents have already done, then review and improve the system.

2. The model selector exposes a tension between simplicity and control

  • The ideal, in swyx’s framing as a Cognition adviser, is a domain-specific GPT wrapper that delivers frontier capabilities without making customers model experts. Lauritzen agrees: a lawyer billing $1,000 an hour does not want to judge “Sonnet High or an Opus Mid.” Legora has a primary-model picker; Jacob says the team has gone back and forth on whether it will stay, and that customers want to experiment with new models. Separately, swyx says his team fought a model picker before giving customers the control they wanted.

  • Legora routes individual tools and subtasks to different models; Lauritzen finds subdividing the problem more effective than making one global choice. Even when clients ask for Mythos, “a lot of legal work does not require Mythos-level intelligence”; Jacob says choosing the more powerful option can use credits “10 times faster.”

  • Basil’s pushback is whether models are already good enough. Lauritzen’s answer is a distribution: probably about 60% of current work can be done well, leaving cost and latency as the main optimization. Complex analysis still needs more intelligence, however, and the work available to lawyers might itself expand as capabilities improve.

3. Confidentiality forces Legora to reconstruct work it cannot inspect

  • “We can’t look at any traces, basically,” Basil says: confidential material often belongs to the law firm’s clients. That leaves him saying, “I have no idea what people are actually doing with it.” The evaluations team therefore approximates workflows it believes customers are running rather than freely inspecting production use.

  • The core evaluations team has about three engineers and three lawyers, backed by 100 legal engineers with Big Law experience, typically three or four years and sometimes CS backgrounds. Test “worlds” might contain a 10,000-file diligence room and a target report. Evaluations separate red-flag discovery from drafting quality, checking analysis, wording, inclusions and omissions across practice areas.

  • Lauritzen tentatively agrees Sonnet 4.6 and Opus 4.6 were early agent models; subagent delegation for fan-out retrieval works much better now. One citation issue improved from about 70% to about 99% after a release: “we did nothing.” But the team’s heat map also shows releases improving some tasks while regressing on others, more often than people might expect.

  • Basil says product signals include renewals, repeat use, DAU/MAU and hours spent, which he describes as strong without supplying figures. Cheap online LLM judgments support feature-flag comparisons; restrictive classifications sort failures into 10 or 20 known modes. swyx proposes paraphrased traces, but Basil warns that removing names may not protect confidentiality. They favor coarse categories over detailed inspection.

4. Firm-specific AI could create capital that partnerships struggle to reward

  • Lauritzen introduces his outlook with “I should preface this by saying I don’t know.” He expects lawyers to produce higher-quality work at lower cost and spend more time on judgment, manageable risk and clients. Less time would go into searching huge documents for the smoking gun or performing the menial work that currently absorbs junior effort.

  • His partnership explanation centers on human capital: professional-services firms cannot own their people, so they make them owners to encourage retention. Legora wants to help codify “the X law firm way of doing things” into precedents, processes and possibly a self-improving knowledge graph—durable IP capturing expertise and judgment beyond any individual’s continued employment.

  • The incentive problem comes first. Partners are rewarded for bringing in clients under “eat what you kill” or lockstep arrangements, rather than diverting time into codifying knowledge. Once they leave, they receive no dividends on assets they helped build. Lauritzen wants incentives to “build an asset for the company,” but does not claim partnerships must disappear.

5. The junior-replacement thesis meets pushback as billing changes

  • The episode includes the provocation “Paralegals, gone” and “Junior lawyers, gone”; in the fuller discussion Basil voices the claim, while the opening clip attributes a similar line to swyx. Jacob’s opening response is “That’s actually not what we’re seeing.” The host describes document review previously assigned to paralegals working until around 4 a.m. as work that no longer requires the same staffing. Those workers may be able to take on other valuable work.

  • Lauritzen’s counterargument is talent polarization: good junior lawyers, like good engineers, can be worth more because their output multiplies, and firms still need the next generation. swyx suggests “meat proxy” for the AI operator; Lauritzen resists that description when someone knows the tools and delivers useful results: “you’re actually just doing meaningful work.”

  • Lauritzen’s earlier objection was that hourly billing punishes efficiency: “They earn less money.” He now sees competitive pressure forcing change, with some routine US work moving to project fees, flat rates or free service. Asked where money remains, he points to big work, including litigation, not only M&A. Cheaper routine work also supports client relationships.

6. Tabular Review makes large-scale diligence inspectable

  • The concrete workflow is 10,000 employment contracts, one per row, with questions as columns: termination period, IP assignment, severance or compensation. The tool might fill cells with “30 days” or flag unusual terms. Lauritzen says the grid is the easiest form they have tried for overview and review; lawyers can inspect citations, mark reviewed cells and comment with colleagues.

  • The agent can create the review itself: drop in 10,000 files and ask for due diligence. It launches the table, waits, analyzes results and highlights red flags, such as excessive CEO compensation. It can then draft a client memo or leave the flagged results for the lawyer to examine.

  • Capacity is “up to a million cells, I think,” Basil says. The interface uses AG Grid, and users are happy working there rather than exporting to Excel. Before running the whole job, Tabular Review runs the first 10 rows twice; users can iterate their questions on that small set until results look right, then expand the fan-out.

  • Legora uses its tools for acquisitions, including Kira in Sweden, and diligence in Canada and Australia. Basil says the company does much of the diligence itself but still brings in counsel for strategic questions, including Australian payroll and deal structure. The tools have found red flags that made Legora “pull the brakes”; he declines to discuss those cases.

7. Lists turns legal requests into coordinated agent work

  • Intake is a “storefront thing” rather than simply a chat interface. A request arrives in Slack, kicks off an agent and creates a task in an intake list. Lawyers and agents work inside Legora, mark the task done and return the result to Slack. The person requesting the review does not need to change their workflow.

  • Lists is Linear-like for tasks and Notion-database-like in flexibility, with fields, triggers and automations. Projects hold files; configured agents read and populate those structures. “Agents can create lists, and lists can create agents,” Lauritzen says, describing how one piece of work can trigger another, update fields or add documents without someone initiating every step.

  • swyx asks whether this becomes “end user programming.” Lauritzen’s answer is that flexibility helps even if customers never see it: legal engineers could build fund-management dashboards with generative UI, or a potential marketplace of specialized products. His analogy is “a Lovable inside Legora”—also a way for the company to build its own features faster.

  • The UK acquisition Wexler provides litigation fact intelligence: extract entities, dates and facts from large document sets, then compare corroboration and contradiction. Lauritzen’s example checks testimony against an email: “he said this thing, but actually the email over here says that he wasn’t there.” He sees this as another product buildable on the same primitives.

8. Deployment needs legal power users and engineers who ship

  • Lauritzen distinguishes two roles: legal engineers configure the product as power users and “don’t write code”; forward-deployed engineers write “code into production,” build integrations and feed missing capabilities back into the product. A large, engaged customer might receive both, with the team sitting beside lawyers to teach the product, map workflows and implement them.

  • The data restriction remains: FDEs build the system without touching underlying client data. Exceptionally, they may be onboarded like employees, including receiving a law-firm laptop. There is no fixed Palantir-style rotation between client sites and product work. Lauritzen says it is early, but imagines a circulating group that might spend two months on one project before moving on.

  • Basil says US law firms and corporate customers move faster and want more custom systems; European customers more often start off the shelf. US requests range from Salesforce integrations to entire new products Legora cannot build for them. Jacob asks whether this reflects greater existential fear; Basil says he does not know, and stresses that both regions see AI as strategically important.

9. The 20-engineer plan gave way to reusable primitives

  • Lauritzen once used the 300 Spartans from the film to pitch staying at 20 engineers and beating larger competitors. “We’re 100 engineers now, and it feels like we’re way too small.” Competition and customer demand made that position untenable. His job is now to make customers feel Legora has built many things while actually building a few reusable primitives.

  • Basil says most product work responds to a “mountain of demand,” rather than low-conviction experiments. He identifies two strategic bets customers were not requesting: greater agent agency and collaboration inside Legora instead of email. The team judged these to be the industry’s direction; legal engineers then help firms restructure workflows around those capabilities.

  • Legora started offering APIs around six months earlier after resisting them for a long time; it does not promise the web app’s entire surface through APIs. Lauritzen’s reason is maintenance: changing data models become harder to evolve when external API versions must keep working. That cost felt unaffordable at 20 engineers and more manageable as the company grew.

  • “Can the agent be the API?” is an idea Lauritzen is considering: customers could trigger an agent to achieve an outcome instead of calling a dedicated endpoint. He emphasizes that he is not saying Legora is doing this. swyx raises unbounded execution, cost and latency; Lauritzen adds nondeterminism—“one or two times out of a hundred” it might do something unwanted.

10. Governance changes which models are usable

  • Basil says Legora’s infrastructure is on Azure because early law-firm customers rejected AWS and distrusted Amazon and Google, but already accepted Microsoft through 365 and prior security reviews. He says that has changed: customers understand cloud adoption, and on-premises discussions have “disappeared completely.” The hosts note that this substantial shift took only about three years.

  • Model arbitrage means optimizing capability, cost, latency, governance and regional availability together. Basil says an early Gemini was strong at transcription and speaker attribution; he finds Microsoft’s transcription model better than that older Gemini, “from what I’ve seen.” ElevenLabs is excluded because Legora’s data-processing agreement keeps a short vendor list. Integrated providers can help even without leading every individual task.

  • Basil emphatically rejects the idea that occasional transcription errors are acceptable. swyx recalls a court role, possibly in Australia, consistently becoming “condom”—specialist jargon still breaks these systems. His proposed repair runs an LLM over the transcript to extract keywords, then supplies those keywords to another transcription pass.

11. Search and drafting offer bounded opportunities beyond frontier scaling

  • Basil says Legora has not fine-tuned models “up until now,” but capable open weights make post-training attractive for “very specific, bounded tasks.” Legal research is the example: fan-out retrieval queries could become fast and cheap. He is not proposing that Legora train its own foundation model to be the main agent; he thinks it would most likely lose that contest against Anthropic.

  • Basil sees “low-hanging fruit” in pipelines and the harness before training models. He says the search-focused engineer Song talked to TurboPuffer about multi-vector support; Basil clarifies that Legora is not currently using late interaction. Jacob recommends customized embeddings rather than choosing a general benchmark leader. swyx agrees, adding that Legora’s many languages complicate that work.

  • Drafting is also a search problem. Basil says lawyers do not begin by typing a huge contract from scratch: they find the firm’s previous clauses from similar transactions. “Similar” might mean industry, company type or something else, and the right clauses may come from several precedents. Selecting and combining those pieces is itself non-trivial.

  • Fable and Opus are “extremely chatty” and “very verbose,” usually undesirable in legal writing; Astra is more concise. Lauritzen says prompting has not made verbosity easy to fix. He thinks the next Anthropic model will probably improve substantially on this, while acknowledging uncertainty. The team already gives providers direct feedback on these shortcomings.

12. Citations support review while long-document reasoning stays difficult

  • Legora links written claims to highlighted bounding boxes in source PDFs; an agent’s memo can cite a Tabular Review cell or row, letting users drill through several layers. Asked whether lawyers verify citations, Basil says “No.” He then describes a code-review-like distribution: some check every comma, others trust the agent, and experienced reviewers focus on the points that matter.

  • Long-contract editing remains difficult because changed definitions, clauses or ordering can require changes elsewhere. Basil says Legora chunks the work and orchestrates edits. The discussion notes that the Word add-in has moved through MapReduce and other scaffolding as models change; swyx says his own news workflow recently moved the other way, from MapReduce to putting everything into context, because results had become better.

  • Retrieval has a cross-chunk failure mode: a two-page discussion can be highly relevant as a whole while each individual chunk looks irrelevant. Summarizing chunks separately can lose critical nuance. swyx says customers sometimes anonymize problem documents and supply a reproducible example: “if we can recreate it, we can fix it.”

  • Jacob raises buying a law firm for its data. swyx is uncertain whether that resolves client-data restrictions, including after clients leave. Acquiring inputs and finished outputs also does not reveal how work happened or whether the result was good: someone must “reverse-engineer it.” Real task traces are much more valuable.

13. Lauritzen expects generalist AI and specialist platforms to coexist

  • Asked where lawyers will work in two years, swyx answers “both.” Enterprises might invoke Legora through a broader assistant while still opening its interface, just as he uses Linear through Claude but also visits Linear. For law firms, swyx’s prediction is more direct: “I think law firms will just use Legora” because it is “simpler and better.”

  • Jacob asks whether coding labs cutting off competitors will happen in law; swyx answers, “No, I don’t think so.” Jacob notes that sales representatives do not make that call. swyx’s argument is that frontier labs have broad enterprise knowledge work to cover, while they will not go as deep in legal as Legora.

  • swyx says Claude taking a small corporate client’s legal work is fine. His dividing example is the large transaction: “But if you’re doing a, you know, $400 million M&A transaction, you’re gonna want Legora.” His prediction about the labs is that “they’re never gonna go as deep as we are.”

Full transcript
swyx

People were asking us about Mythos and everything, even though I think realistically a lot of legal work does not require Mythos-level intelligence. Paralegals: gone.

Jacob Lauritzen

Yeah.

swyx

Junior lawyers: gone. I don't need them.

Jacob Lauritzen

That's actually not what we're seeing. We had one problem where it wouldn't cite properly, and we thought, “Next model release, solved.” It went from 70% to 99%. We did nothing, and it was great.

swyx

But if you're doing a $400 million M&A transaction, you're going to want Legora.

Jacob Lauritzen

We're just going to be 20 engineers for the rest of the company's history, and we're going to beat everyone who's much bigger. We're 100 engineers now, and it feels like we're way too small. But it's something I'm toying with: how much of our product can just be the agent versus all the other stuff?

swyx

Okay, we're here in the studio with special guest Jacob Lauritzen, CTO of Legora. Did I get it right, Lauritzen?

Jacob Lauritzen

That's fine. Yeah.

swyx

Lauritzen.

Jacob Lauritzen

That's okay.

swyx

And that's Danish.

Jacob Lauritzen

I'm not super particular about it. It's Danish, yeah, exactly.

swyx

Does it have a special history behind it?

Jacob Lauritzen

No, “sen” is like “son of.”

swyx

Son of Laurits.

Jacob Lauritzen

So it's just like “son of.” So, son of Laurits—that's my name.

swyx

Nice. We've crossed paths here and there. You spoke at AI a few times, and obviously Legora is one of the fastest-growing legal tech companies in history.

Jacob Lauritzen

Yep.

swyx

You said something at the start in our pre-show that I thought was really interesting. You said that your origin is very European, even though obviously you're expanding in New York and SF. What feels different about your culture?

Jacob Lauritzen

I think at our core, we're probably very humble about what we're doing because we've always been the underdog, from the very beginning. No one believed the smallest startup in a basement in Stockholm was going to outcompete this huge behemoth that was way ahead of us in the US. I think we've always been underdogs, so we've always been very humble. We've always sort of had to fight for our existence.

Something we've had to learn as we've entered the US and spent more time there is to be a little bit bolder and tell the world how great we are, because it doesn't come naturally to some of us.

swyx

I want to start by going over your product, because I feel like with a lot of these legal products, they're kind of under wraps. You're not supposed to show them.

Jacob Lauritzen

Yeah, sure.

swyx

If you could just talk about what the product looks like and maybe how you got there—what product iterations got you to where you are today—I think that would be really interesting.

Jacob Lauritzen

Yeah, absolutely. The way everything started was ChatGPT with a Swedish law search tool in it. That was literally everything. We started with legal research, which is a huge market where lawyers spend a lot of time figuring out what previous cases look like this and what the legal precedent is. What have people done in these situations? That's extremely important.

That has always been part of our product. We started with an agent you could ask questions, and it would query laws. That was the very simple beginning. From there, it's expanded in what the agent can do. It will now draft a lot of stuff really well, edit Word files, move your files around, organize everything, and run really complex analyses.

We've added a bunch more features to it. One of our features is called Tabular Review, which is now a name that's being used in the industry. We're very happy to see Claude for Legal using our terminology for what they're building.

swyx

Ooh, we gotta talk about that. We gotta talk about that.

Jacob Lauritzen

It's essentially a way to bulk extract and bulk review many agreements, which you do in due diligence and when you raise money or whatever. That's part of it. We help lawyers do their work much more efficiently and much better.

Outside of just doing the work, we're also helping orchestrate the work a lot more than we were even a year ago. We have a thing called Lists, which is sort of like Linear for us. We have Intake, which is a way to collaborate with clients, and Portal, which is also a way to collaborate: lawyer to lawyer, client to lawyer, lawyer to agent, maybe agent to agent.

We're spending a lot of time there because, frankly, the way you worked with a lawyer 10 years ago—or maybe even today—is really painful, and there's loads of room to do that in a much better way. Those are, I think, the 2 things: doing the work and orchestrating the work.

swyx

When did you start building the agent?

Jacob Lauritzen

We basically had to switch from the assistant to the agent, as we call it. I guess the agent came about a year ago, somewhere around then.

swyx

So this is a rough timeline for people who are trying to understand: YC 2023.

Jacob Lauritzen

Yes.

swyx

And you were a YC founder as well.

Jacob Lauritzen

It was YC 2024.

swyx

YC 2024.

Jacob Lauritzen

Yeah, I was in YC 2023.

swyx

Yeah.

Jacob Lauritzen

Legora was 2024.

swyx

Okay. So this is a little bit before agents were really taking off.

Jacob Lauritzen

Exactly, yeah.

swyx

And then using models obviously changed everything.

Jacob Lauritzen

Yes.

swyx

In one sense, today—we had a preview of the product. Well, we had a private screening.

Jacob Lauritzen

Private screening, yeah.

swyx

We'll put up a video on YouTube. It's mostly like ChatGPT for lawyers.

Jacob Lauritzen

Yep, yep.

swyx

That's a very fair way to put it. The sidebar has a lot of nice custom tools, like Tabular Review and Intake, which I really liked, and a bunch of other things.

Jacob Lauritzen

Exactly. That's exactly what it is. It's Cowork or ChatGPT, just really, really good for legal work.

swyx

And the workflow for in-house corporate lawyers, people looking for upcoming regulations, all these things—we can talk about more of the products there. What models are you guys using underneath?

Jacob Lauritzen

We're using a bunch of different ones, depending on the task.

swyx

Okay. And that's not available for the lawyer to see? You guys pick them yourselves?

Jacob Lauritzen

For the agent, yeah. We've been back and forth on this, just like everyone else, I think. In the agent, there's a model selector for the primary model, but there are certain tools where we use different models. We try different models in Tabular Review. I think we've found it most effective to subdivide the problem and route as we subdivide the task. But right now, we do have a picker in the agent. I'm not sure if that's always going to stay there, to be honest.

swyx

Yeah. I bring a little bit of context to this because I'm an advisor for Cognition. Part of my thesis for what an agent lab is: an agent lab is ideally a domain-specific GPT wrapper.

Jacob Lauritzen

Yeah.

swyx

We're the domain experts. We'll pay attention to you, serve your needs way more than the frontier labs, and give you the best of the frontier labs according to our judgment.

Jacob Lauritzen

Yeah, we'll abstract away all the—

swyx

Our job is to decide for you, because your job is not to dive into all the details.

Jacob Lauritzen

Exactly. We'll always keep you on the Pareto frontier and make sure it's great all the time, and you don't have to dive into all the details. One thing that speaks exactly to this: if you go to a lawyer and ask them, “You're going to draft this contract based on this precedent. Do you think that's a Sonnet High or an Opus Mid task?”

swyx

They won't know.

Jacob Lauritzen

“Just give me the smartest model.”

swyx

Fair.

swyx

Exactly. Just make sure it works.

Jacob Lauritzen

My hourly rate is $1,000 an hour.

swyx

Exactly. Just—

Jacob Lauritzen

Like, what—

swyx

Make it work.

Jacob Lauritzen

So, yes, I originally came in with that. I was like, “Oh, yeah, this is what Devin does differently.” Now we have 5 things in the model picker, including Fable and Opus and Astra. People want to experiment with the fancy new model.

swyx

Yep.

Jacob Lauritzen

Is that the same?

swyx

I think so. It’s definitely way less in legal than it is in—

Jacob Lauritzen

Coding.

swyx

—engineering.

Jacob Lauritzen

Yeah.

swyx

But there are definitely people who just want Fable 1 right away, and people were asking us about Mythos and everything, even though I think realistically a lot of legal work doesn’t require Mythos-level intelligence.

Jacob Lauritzen

Yeah. So, exactly, right? I think basically you’re like, “Okay, we’ll give you the option. You ask for this, but you’re going to run down your credits 10 times faster.”

swyx

Yeah. I would have loved—We fought introducing it for a very long time, and then in the end we just kind of caved in and were like, “Okay, if you want this level of control, we’ll give it to you.”

Basil Chatha

So you think the models are already as good as lawyers need? There’s no need for them to get better?

Jacob Lauritzen

I think it’s a power distribution. There’s probably 60% of the work we can do really well right now, and from now on it’s a cost and latency thing. But for the rest of the work, especially very complex analysis, we still need more intelligence.

I think there’s also the work expanse. That’s the work we have right now, but the work available to lawyers probably expands with intelligence, I imagine.

Basil Chatha

Early on, when you guys first started transitioning to the agents—or building the agents—what models were you using? I guess it would’ve been Opus 4.6 or something.

Jacob Lauritzen

That would’ve been, yeah. Sonnet 4.6, Opus 4.6 is probably right.

Basil Chatha

What were the agents not good at then that they’re much better at now, even though it’s only, what, 9 months later?

Jacob Lauritzen

Subagent delegation was one of them. We’ve gotten it working really well now for fan-out-type retrieval queries. It works extremely well for us now. That was difficult before.

1. Evaluating Legal Agents

We have a team that’s just doing evals, and we track all of the models’ performance in different practice areas, which are subgroups of legal, and across different tasks in all of those. We have a big heat map, and we can see them move. Usually, they improve at most things, but there are a lot of model releases where they improve at some things and regress at others.

That happens a lot more than you’d think.

swyx

What are some of those tasks? Are you familiar with how you guys are running a lot of the evals? What are the different subsegments you look at?

Jacob Lauritzen

Yeah. There’s different framing stuff. We have different worlds, like RL environments, essentially. Lawyers work in projects, more or less, so it’s like: “Here’s a 10,000-file data room. You’re doing due diligence. In the end, a great report looks like this. It contains all these red flags, and it doesn’t contain weird stuff. We want it to be concise,” et cetera.

That’s how we’ve built it. There are different tasks. Part of it is drafting a report: does the draft look good? Does it contain all the right stuff? Is the wording sound? Is the analysis sound? There’s also finding the red flags, the analysis before that. We can subdivide it a little bit, and that’s how we run the evals.

swyx

Interesting. What does your evals team look like? Do you hire lawyers to do that?

Jacob Lauritzen

Yes, it’s a mix. We have engineers and lawyers, and it’s one team. There are 3 engineers and 3 lawyers right now, and then we have a pool of what we call legal engineers. They’re people who have worked in Big Law for 3 or 4 years. Typically, they might have a CS degree, and they’re very tech-forward. They come to us because they want to help change the industry.

Basil Chatha

Are they IT in Big Law, or are they lawyers?

Jacob Lauritzen

They’re lawyers.

Basil Chatha

They’re lawyers in Big Law?

Jacob Lauritzen

They’re lawyers in Big Law.

Basil Chatha

With CS degrees?

Jacob Lauritzen

Yes.

Basil Chatha

Oh my God.

swyx

That must be really hard to find.

Jacob Lauritzen

It’s a Venn diagram. Yeah, exactly. That’s our hardest person to hire for. But they’re also such a force multiplier, because we can pull on them for the eval stuff and for product development. You have a pool of 100 experts who use the product every day and talk to clients every day. They live this and have lived this, and we can pull on them to do evals and help us shape the product. It’s such an advantage.

swyx

When did you guys start that team?

Jacob Lauritzen

Evals grew out of the assistant agent team at some point. Until recently, it honestly wasn’t really worth it, because all the models just got better. It was like, do we want to spend a lot of our time doing really complex evals, or do we just want to build a better harness and lean into the fact that everything’s going to improve? Everything kind of just improved.

Basil Chatha

Yeah, every 2 or 3 months you get a different tier of model.

Jacob Lauritzen

It just got solved.

Basil Chatha

Just pass it through, right?

Jacob Lauritzen

Exactly.

Basil Chatha

Yeah.

Jacob Lauritzen

We had one problem where it wouldn’t cite properly, so it was difficult to follow. The next model release solved it. It went from 70% to 99%. We did nothing, and it was great.

Basil Chatha

Maybe the real work is keeping all the data private, making it taggable and sortable, and adding new features like the tabular view. Is that the—

Jacob Lauritzen

You mean over and above ChatGPT?

Basil Chatha

Well, yes. But obviously now you’re also doing evals. I assume your use of open models is increasing.

Jacob Lauritzen

Yes.

Basil Chatha

We can talk about that. I don’t know what I’m saying here, apart from: what is the job? The job is maybe not—in the early days, it wasn’t so much about doing evals, because it was mostly exposing model capability to lawyers.

Jacob Lauritzen

A huge part of it is figuring out what the legal profession looks like in 5 or 10 years with all of this stuff. Coding is maybe a year or 2 ahead, so we can look to that a little bit. But what are the primitives? How do people work? How do you collaborate if it’s not a 200-thread email with V5 Word files going back and forth? What does that look like?

That’s not even necessarily purely AI. It’s about building a much better product experience and making sure you do it with audit logs, data retention, and all those really important things in a regulated industry. Then there’s the AI part: make the work faster and better, and build proactive agents that react to things when they come in, so you don’t have to kick everything off yourself.

A lot of the work is figuring out what we think it’s going to look like in 5 years. If model capabilities keep increasing and the price keeps coming down, what could we do? Can you log in and have a command center where it’s like, “You need to react to these 5 things the agents have been doing lately,” and review and check and improve the system? Maybe that’s where we’re going—or where we are going.

swyx

I was going to ask if you could go into that in a little more depth. What is the legal profession going to look like in a few years?

Jacob Lauritzen

I should preface this by saying I don’t know.

swyx

Yeah.

Jacob Lauritzen

Who knows, really? But I think we can look to coding a lot. I think all lawyers are going to be way more productive than they were before.

Jacob Lauritzen

They’re gonna be able to do a lot more stuff at a higher quality for a lower cost. I think the profession itself, the work itself, is gonna improve a lot. Lawyers are gonna spend more of their time doing the stuff they became lawyers to do: figuring out the right risk profile, what’s manageable risk, using their judgment and strategic judgment, and working with clients.

They’re gonna do that a lot more, and spend much less time trolling through big documents to find the smoking gun or doing all the menial, super annoying work that a lot of lawyers have been doing.

swyx

What do you think is gonna happen to the partnership model? Do you think that stays, or what happens?

Jacob Lauritzen

I actually gave this a lot of thought just last week.

swyx

Yeah.

Jacob Lauritzen

The partnership model kind of exists because the only type of capital these companies have is human capital. It’s not like us, or a software engineering firm, or a manufacturer, where you have physical assets or some sort of intangible assets that you accrue over time, and they pay dividends to shareholders.

But if you work in an industry like professional services—and it’s the same for consultancies and whatever else—all of your value is in your people, and you can’t really own human capital. You can’t own people; slavery’s abolished. But you can rent them. You can pay them money. So partnerships are essentially a structure where you make the human capital the owners of the company, so that they stay.

swyx

Yeah.

Jacob Lauritzen

Will that change? It doesn’t have to. But what we really want is for law firms to be able to build some sort of intangible, durable capital. You can build a system that learns how you work. A lot of firms think about how there’s the X law firm way of doing things, and we can codify that and build a system.

Whether that’s a self-improving knowledge graph or whatever it is, and precedent files and documents, there’s a way of working that produces a really good output, and that can be their IP. It’s a way to codify human expertise and judgment into a system. That’s something we want to help empower.

swyx

Mm.

Jacob Lauritzen

Does that change the partnership model? The incentives are different in the partnership model. You just have to bring in more clients, and you don’t want to take time away from bringing in clients to codify your information.

swyx

Mm.

Jacob Lauritzen

You get rewarded by “eat what you kill” or by a lockstep model, whatever. That needs to change a little. We need to figure out an incentive structure where partners are incentivized to codify things and build an asset for the company. But right now they aren’t, because as soon as they leave the partnership, they don’t get any dividends from the assets they built.

swyx

Yeah.

Jacob Lauritzen

I think the whole business model—we’ll see.

swyx

Yeah, yeah. It’s all in question right now.

Basil Chatha

My actual other take is more on the bottom end of the industry. The partnership level is the top end.

Jacob Lauritzen

Yeah. What happened there?

Basil Chatha

Paralegals are gone.

Jacob Lauritzen

Yeah.

Basil Chatha

Right? Junior lawyers are gone.

Jacob Lauritzen

Mm.

Basil Chatha

I don’t need them.

Jacob Lauritzen

Yeah.

Basil Chatha

That’s actually not what we’re seeing.

Jacob Lauritzen

Okay.

Basil Chatha

So, I don’t know. Would you say that we’re seeing that in engineering?

Jacob Lauritzen

Yeah.

Basil Chatha

Yeah?

Jacob Lauritzen

Junior engineers are—

Basil Chatha

Really kind of cooked.

Jacob Lauritzen

—kind of cooked.

Basil Chatha

Yeah.

Jacob Lauritzen

You kind of have to come out of university fully baked, or else—

Basil Chatha

Or else you’re not gonna make it.

Jacob Lauritzen

I feel like I’m being a bit dramatic for comedic purposes. But I know a lot of lawyers who started as paralegals, then went through law school and all these things, right?

Basil Chatha

Yeah.

Jacob Lauritzen

And now they’re not needed.

Basil Chatha

Yeah.

Jacob Lauritzen

They’re seriously just not needed.

Basil Chatha

Yeah. Yeah.

Jacob Lauritzen

It’s interesting what happens to the partnership model, the pyramid. I think generally, you still need the next generation. For engineering, the way I think about it is that a good engineer today is worth a lot more than they were 2 years ago, because they can produce so much more stuff, if they’re good. I think it’s the same thing in legal: they can actually do a lot of really good work if they’re good. So I think it’s more that really good junior lawyers are worth a lot more.

Basil Chatha

Yeah, it’s—

Jacob Lauritzen

And—

Basil Chatha

It’s always been the case, right?

Jacob Lauritzen

Yeah.

Basil Chatha

Exactly.

Jacob Lauritzen

But it’s even more so. It’s polarizing even more.

Basil Chatha

Yeah.

Basil Chatha

I’m actually curious about that. The whole law business is basically charging per hour.

Jacob Lauritzen

Mm.

swyx

So—

Jacob Lauritzen

Yeah.

swyx

If each project can be done—

Jacob Lauritzen

It’s faster.

swyx

—a lot faster—

Jacob Lauritzen

Yeah.

swyx

—is it gonna be way cheaper?

Jacob Lauritzen

This was actually a fun story. I was reading Max’s YC application and helping him get into YC, when Leya was called that back in YC. This was my one thing: people have tried this a million times. Legal tech has never been a thing because there’s no incentive for lawyers to be more efficient. Why would they be more efficient? They earn less money. It doesn’t make any sense.

But I think lawyers do want to adopt this because they understand the gravity of the situation. They know the business model and everything is under pressure, and they need to change because the competitive pressure is so high. I think the billing model itself is also gonna change. We’re already seeing that, particularly in the US, where a bunch of stuff is being priced on a project fee and not an hours fee, particularly the lower-end stuff on the spectrum, like small reviews and whatever. Some of it is honestly being done for free—

Basil Chatha

Mm.

Jacob Lauritzen

—and some of it is just being priced at a flat rate.

Basil Chatha

So where is the money being made, then? Because I feel like a lot of what you guys talk about is M&A specifically. Is that where all the money is made?

Jacob Lauritzen

That’s just a very easy example.

Basil Chatha

Oh, okay, okay.

Jacob Lauritzen

Litigation, too. The big work is still there, and it’s very valuable, so lawyers will spend more of their time there. Client relationships are also super important.

Basil Chatha

Like a CRM?

Jacob Lauritzen

No, just spending more time with the client.

Basil Chatha

Oh.

Jacob Lauritzen

A lot of law firms have historically done stuff for free just to have that relationship with the client.

Basil Chatha

Yeah, yeah.

Jacob Lauritzen

So now they can do that, but they don’t have to lose a lot of money doing it.

Basil Chatha

Wait, I thought I just had to pay the lawyer money to be my friend. Isn’t that the retainer thing?

Jacob Lauritzen

Yeah.

Basil Chatha

You’re telling me I can do it for free?

Jacob Lauritzen

Yeah, you just have to find the right lawyers.

Basil Chatha

Oh, goddammit. Okay, cool.

Jacob Lauritzen

Or Legora has to grow much bigger. I don’t know.

Basil Chatha

Yeah, we’re trying. So you’re saying the strategic work and all these high-value-add things are still there, but our legal fees aren’t going down. You also said paralegals aren’t affected, or they’re just shifting. This is something we also worry about with Cognition.

Basil Chatha

Yeah.

swyx

People worry about the phrasing, like, “We will replace your junior engineer,” because nobody likes that kind of statement. But in actual fact, yes, some part of it does happen. Other junior engineers embrace it and become more productive.

Basil Chatha

Exactly.

swyx

Exactly like you just said.

Basil Chatha

Yeah.

swyx

Is there a new term for this role or job that’s gaining traction with your customers?

Basil Chatha

You mean the super-AI-pilled junior legal? Nothing I’ve found yet. Maybe we should coin one.

swyx

Isn’t it a legal engineer?

Basil Chatha

Legal engineer? That’s our term, and it’s also now an industry thing.

swyx

Yeah.

Basil Chatha

Yeah.

swyx

CS JDs.

Basil Chatha

Exactly. Yeah. And maybe those are the people best positioned to do this kind of stuff.

swyx

I think the derogatory term is “meat proxy,” where you just sit in front of the machine and type in something that someone else tells you to type, and then you repeat what the machine tells you to say.

Basil Chatha

Yeah.

swyx

But I do think a very good meat proxy is someone who knows agents very well and knows Legora very well.

Basil Chatha

Yeah.

swyx

And you know, “This is my toolkit.” Every industry has a toolkit.

Basil Chatha

Yeah.

swyx

And I can use my tools to do the thing that you need me to do.

Basil Chatha

But I guess then you’re not really a proxy, right? You’re actually just doing meaningful work, I suppose.

swyx

Yeah.

I’ll talk about Tabular Review, because that’s a thing that happens a lot. I have thousands—

Basil Chatha

We do up to a million cells, I think.

swyx

In a single—

Basil Chatha

In a… Yeah.

swyx

That’s impressive.

Basil Chatha

Yeah.

swyx

We should talk about the table and the windowing.

Basil Chatha

This. Yeah.

swyx

Yeah, I’m working on a big case. I have a million documents. I type it in. Usually, I’ll throw a team of paralegals into a room.

Basil Chatha

Yes.

swyx

Locked in a room.

Basil Chatha

And they’ll sit there till 4:00 AM. Yeah.

swyx

That doesn’t need to happen, right?

Basil Chatha

Exactly.

swyx

Because now they can extract it, and you can go and reprompt everything.

Basil Chatha

Mm-hmm.

swyx

So now I don’t need a paralegal for that. But a good, super-AI-pilled paralegal would do something else, you know?

Basil Chatha

Yes. Yeah, exactly.

swyx

Whatever that is.

Basil Chatha

Yeah. They can do more stuff, or spend more time on the things they didn’t have time for before.

swyx

Or actually sleep. Who knows?

Basil Chatha

No.

swyx

I have a question. You guys bought a company, Kira. Is that how you say it?

Basil Chatha

Yeah.

swyx

And you guys used your own product to do the M&A?

Basil Chatha

Yes, we do that all the time.

swyx

So how did that work?

Basil Chatha

Dogfooding. It works great. We’re doing that a lot. Our legal team is—I mean, intake has been built with our legal team.

swyx

Yeah.

Basil Chatha

Right now, we have a Slack workflow where everyone floats their stuff in, and they’re like—

swyx

Yeah.

Basil Chatha

“This is annoying.”

swyx

Yeah, yeah.

Basil Chatha

Right? And inefficient. So it’s like, great, we should use Legora for that.

swyx

Yeah. Can you talk a little bit about what that process looked like?

Basil Chatha

Yeah, absolutely. We do a lot of the due diligence ourselves because we can do it with Tabular Review and other tools. We still have counsel come in to assess certain risks. For example, there’s payroll. Kira was simple because that was all in Sweden, but we’ve done one in Canada and one in Australia, and there’s payroll in Australia. Do we want someone to look at this more strategically? We’ll still use lawyers for the strategic part, where they can be sparring partners and help us figure out how to structure this the right way. But a lot of the more menial stuff we can do ourselves.

swyx

Mm.

Basil Chatha

A lot of the early things we find—and we’ve had cases where we found something that was a red flag, and then we’ve pulled the brakes.

swyx

Can you talk a little bit about any of those?

Basil Chatha

No, I don’t think so.

2. How Law Firms Adopt AI

swyx

All right. Early on, you said your early customers were Swedish law firms. Have you seen a difference between working with Swedish law firms versus American law firms? You guys are mostly working with big law firms, right?

Basil Chatha

We’re also working with smaller law firms.

swyx

Oh, okay.

Basil Chatha

We definitely started with really, really big law firms, but firms with 50 lawyers use this and get a lot of value too.

Jacob Lauritzen

So have you seen differences between working with the Swedish ones versus American ones? Maybe even Asian ones?

Basil Chatha

Yeah. I think the European ones have been a bit slower to move. The American ones are very forward and move really quickly. I think it’s also much higher on the agenda for American companies than European ones. It’s a strategic, super-important thing that they need to do a lot of. American companies, both corporates and law firms, also have a much higher appetite to custom-build stuff. They all want to do that. They’ll say, “Hey, we want to build this huge thing with you guys and structure it in whatever way.”

Jacob Lauritzen

What—

Basil Chatha

Whereas Europeans are much more like, “We’ll just take the off-the-shelf and figure it out.”

Jacob Lauritzen

What’s an example of something they would want to build themselves?

Basil Chatha

Integrations into their own systems. Some of these customers build their own CRM and their own document management system and all sorts of other things, and they want to build that. Then they want to build triggers, like: you do a review in Salesforce, it goes to this one person for whatever reason, they do something, then it goes into Legora, and then it goes back. That’s the simple end. Then there are some that are extremely ambitious, where it’s a whole new product they want to build with us. We can’t really do that.

Jacob Lauritzen

Yeah, yeah.

Basil Chatha

That’s definitely something where American companies really want to build their own more than European ones do.

Jacob Lauritzen

And the European ones are like, “Hey,” and they give you product suggestions?

Basil Chatha

Yeah, for sure. We have great collaboration with all of them.

Jacob Lauritzen

Yeah.

Basil Chatha

They’ll say, “This sucks, this is great. We want more of this.”

Jacob Lauritzen

Mm-hmm.

Basil Chatha

And we’ll just build a product that does that.

Jacob Lauritzen

Okay. So they don’t see an existential threat to their business, whereas the American ones do?

Basil Chatha

I don’t know if that’s the reason they want a custom build versus not a custom build. I think they both know that this is extremely important and key to their firm’s success and their corporate success. It’s both extremely important. I think it’s just a different way to achieve it.

Jacob Lauritzen

Okay. Another question I think you might want to bring up is product metrics. How do you know that we’re actually doing something good here?

Basil Chatha

Yeah. Mm-hmm.

Jacob Lauritzen

How do you think about that? I’m assuming that’s also related to evals.

Basil Chatha

We have the regular product metrics that we care a lot about. Are people spending a lot of time in our product? Do they seem to get a lot of value out of it? Do they renew? Do they come back a lot? We have really good numbers for enterprise software: our DAU over MAU and hours spent per month on the product are extremely high.

We also do online evals to generally try to figure out whether we solve the problem. We use a super-cheap LLM judge to have an online metric, and we can use that when we roll out new things. We can compare feature-flag-wise, okay.

Basil Chatha

Did it improve, or did it catastrophically decline? One thing that might be interesting—and difficult for our business—is that we can’t look at any traces, basically. We can’t look at any data because it’s all super confidential. That’s extremely frustrating to work with because you basically don’t know what people are doing. It’s like, here’s an agent that can do so many amazing things, and I have no idea what people are actually doing with it.

Jacob Lauritzen

I would’ve assumed you guys signed some sort of agreement with the law firm, and they were like, “Oh, yeah, sure.”

Basil Chatha

Yeah.

Jacob Lauritzen

...with the law firm, and they were like, “Oh, yeah, sure.”

Basil Chatha

I wish. Yeah, I wish. No, but the thing is, a lot of the data they have isn’t even theirs, right? It’s their clients’—

Jacob Lauritzen

Oh.

Basil Chatha

—and it’s super confidential, so we can’t look at it. We have an evals team that’s trying to approximate what we think everyone is doing and build that into eval sets.

Jacob Lauritzen

Wow.

Basil Chatha

So that’s just a super annoying kink of the business.

swyx

Can you paraphrase? For example, ChatGPT has a `/feedback` thing that directly gives you the full trajectory.

Basil Chatha

Mm.

swyx

But obviously, that’s not possible for you guys.

Basil Chatha

Mm.

swyx

But you could use an LLM to paraphrase the real thing, make a synthetic thing that’s close enough, and change all the names and details, but still—

Basil Chatha

Yeah.

swyx

Yeah.

Basil Chatha

We can’t— There’s always a range, right? If you paraphrase all the way down to three values and pick one of them, then obviously that’s fine. If you just remove the names, then maybe that’s not fine.

swyx

Yeah.

Basil Chatha

So we generally pick the safest way.

swyx

Okay.

Basil Chatha

We restrict ourselves as much as possible because there should be no risk at all.

swyx

Yeah, that’s fair enough.

Basil Chatha

But there are things where we know the failure modes are probably these 10 or 20, so we can classify it into one of those, and that’s a helpful signal. We can also classify it into the types of tasks and practice areas, and then we have an idea. It’s something we’re always trying to figure out how to do in a smart way.

Jacob Lauritzen

Also, on the privacy front, I’m assuming even 2 years ago it must’ve been insanely difficult to get law firms to be okay with this.

Basil Chatha

Yes.

Jacob Lauritzen

But—

Basil Chatha

No.

Jacob Lauritzen

—how do they think about their data being sent over to Anthropic or, you know?

Basil Chatha

Yeah.

Jacob Lauritzen

It must be a huge thing.

Basil Chatha

All of our infrastructure is on Azure, primarily for this reason. When we started, law firms were like, “AWS? No.”

Jacob Lauritzen

Yeah.

Basil Chatha

Like, “No way. We don’t trust AWS.”

swyx

Law firms don’t trust AWS?

Basil Chatha

Yeah, yeah.

swyx

Why?

Basil Chatha

It’s different now. But they were like, “Amazon, Google—we can’t trust them with any of our data.” So we basically had to host everything on Microsoft because they already have Microsoft 365.

Jacob Lauritzen

Oh, so they’re okay with Microsoft but not—

Basil Chatha

Exactly, because they’d already done the security review for all of their documents and Word and everything else. Microsoft is an approved vendor in their mind, so we had to host everything on Azure. That’s changed over the past few years.

swyx

Mm.

Basil Chatha

Now people understand that they need to get on the cloud and do all this kind of stuff. There were also a lot more people talking about on-prem back then. That’s disappeared completely. People understand that you should just do it in the cloud, that it’s safe, and that it’s probably safer than what you’re doing.

Jacob Lauritzen

Yeah.

Basil Chatha

That’s changed a lot, but I think we can—

swyx

Yeah, and by the way, it’s only been 3 years, so it’s actually—

Basil Chatha

Yeah.

swyx

It’s a huge shift. It’s great. Good for you guys. I don’t expect the legal industry to have somehow ignored this for the last 10 years.

Basil Chatha

Yeah. I think we needed the AI kick in the butt for everyone to understand that we need to change our ways a little bit.

swyx

Oh, yeah.

Basil Chatha

...our ways a little bit.

swyx

Okay. Okay.

Jacob Lauritzen

So, open-source models are getting really good now. How are you thinking about that?

Basil Chatha

Yeah. We’re thinking about just bringing our clients along on the Pareto frontier.

Jacob Lauritzen

Yeah.

Basil Chatha

And the map of all the models—what they’re good at, their latency, their cost, their governance, and their regional availability—how do we optimize across all of that? Some open-weight models are great for certain tasks, and then we’ll use them there. Some we’ll post-train, and some we’ll just use the frontier models.

swyx

Yeah. You call it model arbitrage. You have different flavors. You did some research on this where you were firmly Anthropic at the start of the year. Grok is improving a lot. Do you want to give an example of a model that wasn’t ranked highly overall, but that you want to shout out as surprisingly good?

Basil Chatha

Yeah. I don’t think it’s good overall, but one of the early Gemini models is extremely good for certain types of transcription and speaker attribution. We use that for transcription, actually.

swyx

Oh, so you have a—

Jacob Lauritzen

Yeah, where are you guys transcribing?

swyx

Like Granola?

Basil Chatha

We actually probably will build a meeting thing.

swyx

Yeah.

Basil Chatha

But this is just speech-to-speech, speech-to-text, text-to-speech—like any other app that has this. We also transcribe files. If you upload hearings or whatever, we’ll transcribe them, put them into text, and then we can do all of our analysis on them.

swyx

No ElevenLabs?

Basil Chatha

No ElevenLabs, for DPA reasons. I mean data processing agreement, I should say. Our data processing agreement has very few vendors on it because we just need to keep it—

swyx

You just need to keep it fresh.

Basil Chatha

Exactly. We need to keep it—

swyx

Tight.

Basil Chatha

—super tight—

swyx

Yeah.

Basil Chatha

—on where any data goes.

swyx

I think that’s one of the benefits of Gemini and OpenAI, where they’re like, “You come to us; we have most of the things that you need.” That’s actually very valuable. It doesn’t have to be state of the art. As long as it’s mostly good enough for all tasks, then who cares?

Basil Chatha

Yeah.

swyx

Like—

Basil Chatha

Yeah, absolutely. I should also shout out Microsoft’s transcription model.

swyx

Ah.

Basil Chatha

They may want—

swyx

Microsoft Transcribe?

Basil Chatha

Yeah.

swyx

Okay.

Basil Chatha

It’s great.

swyx

Okay.

Basil Chatha

It’s better than the old Gemini, from what I’ve seen.

swyx

Yeah. I think usually all of them are in the top two, top three, top four in terms of the Arena scores, and that’s good enough.

Basil Chatha

Yeah, exactly. You have all the governance, it’s super scalable, and that’s probably good enough.

Jacob Lauritzen

At this point, are speech-to-text models pretty much good enough that you’re like, “Oh, one word here or there being off is fine”?

Basil Chatha

No.

swyx

It—

Jacob Lauritzen

No?

swyx

No, no.

Basil Chatha

For sure. I’ve done this a lot.

swyx

Particularly in legal, there are some... We had this thing where, I think it was in Australia, there’s a certain name for a person in court, and it would consistently transcribe it as “condom.”

Basil Chatha

Oh.

swyx

Yeah. There are certain things you don’t want there.

Basil Chatha

I see. Okay.

swyx

Usually you run an LLM layer on top of it. Then you extract some keywords, and you run the transcription model again with the keywords. That’s kind of my workflow so far.

Basil Chatha

Interesting.

swyx

But if you’ve done this at any large scale, you’ll pick up tons and tons of transcription errors on any—

Basil Chatha

Anything.

swyx

domain that is not average day-to-day conversation.

Jacob Lauritzen

Yeah.

swyx

Yeah, absolutely. If there’s any sort of jargon in there—

Jacob Lauritzen

Yeah.

swyx

—it’s gonna be tough.

Jacob Lauritzen

Yeah. I think Whisper and ElevenLabs still transcribe Claude Code as “cloud clo-code.”

swyx

Yeah.

Jacob Lauritzen

And that’s—

Basil Chatha

Yeah, I’ve actually had that problem.

Jacob Lauritzen

Very, very common. So that’s how you know there’s a lot more to improve.

swyx

Yeah. Absolutely. I do want to cover a little bit on the agent product side. We’ve covered a few of the top-level things, right? We’ve already name-dropped Tabular Review, and we’ve name-dropped Intake. You’ve got some other stuff that we haven’t even touched on yet, but I just want to cover the big two, let’s call it.

Jacob Lauritzen

Yeah, yeah, sure.

swyx

Is that fair to say, though? They’re approximately the big two?

Jacob Lauritzen

Agent and Tabular Review.

swyx

Yeah.

Jacob Lauritzen

And our—

swyx

And projects.

Jacob Lauritzen

Probably, yeah. And our Word add-in is also something, because a lot of people work inside of Word, so our Word add-in is huge, and the Outlook add-in too.

swyx

Yeah. I just want to cover the big ones that everyone must know.

Jacob Lauritzen

Yeah.

swyx

So Tabular Review is basically a fan-out across a lot of documents. You can ask the same question across all the documents.

Jacob Lauritzen

Exactly.

swyx

It’s collected in a table, and then you work back and forth.

Jacob Lauritzen

And then you—

swyx

You can—

Jacob Lauritzen

Yeah.

swyx

A million is impressive. What are some other engineering considerations or performance things? What’s hard about this?

Jacob Lauritzen

Uh—

Basil Chatha

Really, before we do that, can you walk through the user experience for a lawyer actually using Tabular Review?

swyx

Okay, so you want us to do a top-level flow? Yeah.

Basil Chatha

Yeah, like, what are they doing with Tabular Review?

Jacob Lauritzen

Yeah.

Basil Chatha

And then we can go into that.

Jacob Lauritzen

Sure.

swyx

Yeah.

Basil Chatha

Yeah.

Jacob Lauritzen

A good example is due diligence, where it’s like, “We need to review all employment contracts, all supply contracts,” or whatever. So you put 10,000 employment contracts in there, and then—

swyx

Each cell is one employment contract?

Jacob Lauritzen

Each row is one employment contract.

swyx

Okay.

Jacob Lauritzen

And then for the cells, you ask questions—sorry, per column. The first column might be, “What’s the termination period?” Then it’s going to fill that out for every one: 30 days, or whatever it is. The next question might be, “Is there an IP assignment clause?” The next one might be, “How much do we have to pay them if we fire them?” or “How much are they getting paid?” Then it’s going to fill out the entire thing.

You can also just ask the agent to do this. If you drop 10,000 files into the agent and say, “Do due diligence on this,” it will spin up a Tabular Review, run all of them, wait for it to be done, and then analyze the results. It can highlight red flags: the CEO is getting paid an outrageous amount, or there’s a weird bonus structure. Obviously, we need to look at that. The agent can then draft a memo to send to the client, or it can just do the review and have a lawyer—the user—say, “Okay, let’s look at this. Here are the red flags.” It’s also collaborative, so you can comment for other people to look at, et cetera.

Basil Chatha

And why did you choose that spreadsheet format?

Jacob Lauritzen

It’s just a lot better to work in. It’s super easy to get an overview of what’s going on and review it. All the stuff we produce also has citations back to source documents, so you can review it easily. If you want to make sure everything’s right, you can highlight cells that have been reviewed or flagged, and you can comment and collaborate. That’s much easier than any other form factor we’ve tried.

swyx

Yeah.

Basil Chatha

Mm.

swyx

I would say that the spreadsheet form factor has a lot of pull in some areas, but somehow hasn’t really spread.

Jacob Lauritzen

Mm.

swyx

I find it very weird. Clay is another one on the GTM side that’s very well known for spreadsheets. You guys—I don’t really know if anyone else does a large spreadsheet AI thing. There was V7, which tried to launch. There’s Sheets. Even in the finance domain, I don’t really have a top name that does spreadsheets.

Jacob Lauritzen

Does this really well. Yeah, no, I agree. If you have fan-out and you do the exact same thing, extracting stuff, it’s such a great form factor to understand what’s going on.

swyx

Do you find that people must export to Excel, or are they happy to live in your spreadsheet?

Jacob Lauritzen

People are happy to live there.

swyx

Do you use AG Grid or your own?

Jacob Lauritzen

We do use AG Grid.

swyx

AG Grid. That seems to be the leading one.

Jacob Lauritzen

Yeah.

swyx

I mean, they’ve done this for, I don’t know, 15 years.

Jacob Lauritzen

Yeah, yeah, they know how to. Yeah.

swyx

Yeah.

Jacob Lauritzen

It’s a vertical list.

swyx

Okay, I’m happy on that spreadsheet angle. I think the only other interesting thing for me engineering-wise is the fan-outs.

Jacob Lauritzen

Yep.

swyx

Engineers are maybe well-equipped to understand the trade-offs. Before you fan this thing out on a million documents, maybe test it on two.

Jacob Lauritzen

Mm.

swyx

Do they do that?

Jacob Lauritzen

Yeah, they will. Our type of review will actually run the first 10 rows twice.

swyx

Just check, right?

Jacob Lauritzen

Yeah. Does this make sense? Did it show us what we thought it was going to show us? That’s really good. It’s also good for people building this to iterate quickly on the first 10 rows until they’re getting the results they want, and then they can go ahead.

swyx

Yeah. I’m happy to move on to the next thing: Intake, which is your Slack bot.

Jacob Lauritzen

Yeah, yeah.

swyx

Everyone has a Slack bot. What’s different about yours?

Jacob Lauritzen

Ours is just legal-specific. What happens with ours is that it’s not actually— We’ll probably also build one that’s just an interface to the agent, but this one is specifically to kick off an agent that then adds a task to our list. We have this Lists thing, which is like Linear—that’s the closest equivalent. It adds it to a specific intake list. Then lawyers and agents can go into Legora and see the things people are sending to us as a law firm or as a legal team in a big corporation. We can kick off agents for them, collaborate on them, and mark them as done. Then it gets sent back in Slack.

It’s not just kicking off an agent. It’s more of a storefront thing: “Hey, here’s the ticket. I want this thing reviewed. Okay, now here it is. It’s back.” No one needs to change anything, and the legal team just sits in Legora and manages the agents there.

swyx

I think the data structure for this kind of thing is really interesting because it’s basically the inbox for all these kinds of queries that can potentially translate into a thing, into something else. Is it hard to integrate all these with the other features, or do you just not worry about it?

Jacob Lauritzen

No, it’s not. I think we have some primitives that work really well. Lists is a great primitive because it’s extremely flexible. It’s almost like a Notion database.

swyx

You could treat it as a Notion database.

Jacob Lauritzen

It’s like a Notion database, so you can configure it for whatever you want. We invest a lot into making it scalable and work really well. You can have triggers and automations on them, and it’s a great glue to tie everything together. You can kick off an agent run based on a list if something happens, and then it comes back and populates a field, adds new documents, or whatever it is.

Jacob Lauritzen

And so, just like you might coordinate a bunch of coding agents with Linear or some other task management tool, we can do something similar with Lists. Agents can create lists, and lists can create agents—

swyx

Wow.

Jacob Lauritzen

—and agents can read lists, so it’s like the agent swarm comes alive.

swyx

Yeah. Do people get excited or overwhelmed by that? I feel like this is basically end-user programming.

Jacob Lauritzen

Yeah.

swyx

You can let people set up crazy automations. Some people get super obsessed by it, and they become influencers. It’d be cool to have Legora influencers.

Jacob Lauritzen

Yeah, right. Clay has influencers.

swyx

Is that worth it? Too overcomplicated?

Jacob Lauritzen

We need the flexibility, even if we don’t actually expose it to the end user. Even if this were all internal, it’d be a really fast way for us to use people like our legal engineers to build a lot of products extremely quickly. You can build custom fund management dashboards on Lists with your own generative UI, and then you have a new product that some people are going to love. We can have a marketplace of specialized products that we can scale super quickly, because all our legal engineers can build new ones. If they’re out with a client and hear, “Oh, you want your own? Easy. We’ll just build one for you.”

It’s a little like having Lovable inside Legora. And if we don’t want to expose it, that’s also how we build our own custom features. We made another acquisition in the UK called Wexler. They’re doing fact intelligence, mostly for litigation: you upload a million documents for a lawsuit or whatever, and then we extract facts with named entities, dates, and so on. Once you have all the facts, you can hold them up against each other and see which ones are corroborated and which ones are in disagreement.

swyx

Yeah.

Jacob Lauritzen

That’s extremely powerful, because then you can also transcribe a court hearing and say, “Oh, he said this thing, but actually the email over here says that he wasn’t there.” If you see it live, it’s so cool.

swyx

Yeah.

Jacob Lauritzen

We can build that on this primitive, too. We can build a lot of really cool things on this one primitive. That’s how we think about it, because as we scale the company, we have to solve so many things. I think a lot about how we build a core primitive that we can scale across many different end products and use to create customizable—

swyx

Mm.

Jacob Lauritzen

—experiences.

swyx

Okay. So what core primitives do you have? Let’s just restate them for people.

Jacob Lauritzen

Lists is a core one.

swyx

Lists.

Jacob Lauritzen

Yeah.

swyx

Projects.

Jacob Lauritzen

Super configurable. Projects are basically containers of files—file systems.

swyx

Yeah.

Jacob Lauritzen

The agent, of course, which you can kick off and run in very different configurations.

swyx

Do you have a name for the agent, by the way?

Jacob Lauritzen

No, we don’t.

swyx

Just Intake?

Jacob Lauritzen

No.

swyx

Okay.

Jacob Lauritzen

You mean, like, did we call it Leia or give it a name? No.

swyx

Ooh. Leia would be cool.

Jacob Lauritzen

Yeah.

swyx

People used to talk about Leia as a named entity, but we stopped that.

Jacob Lauritzen

Okay. Maybe I’ll bring it back.

swyx

It might be confusing. I don’t know.

Jacob Lauritzen

So, Lists, Agent, Tabular Review. We’re considering whether that’s actually an instantiation of something. You can take this as far down as you want, right? But those are probably the big ones.

Basil Chatha

Yeah. I actually wanted to talk a little bit about FTEs, because this sounds kind of like that. I think Max has talked about that a little bit. When you onboard a big customer, you want to get them value ASAP so that they stay on Legora.

Jacob Lauritzen

Exactly. Yep.

Basil Chatha

How has that evolved? When did you guys start that forward-deployed team?

Jacob Lauritzen

We hired our first forward-deployed engineer, our first hire in New York. They were also a YC founder. That was maybe six months ago. I’m not actually sure.

Basil Chatha

Are they considered legal engineers, or no?

Jacob Lauritzen

No.

Basil Chatha

Okay. All right.

Jacob Lauritzen

They’re true engineers. But I do think about how we enable that at scale. A really core thing is: how can we empower our legal engineers? How can we empower our power users and legal engineers to customize the product in a powerful way?

Of course, we’ll still have forward-deployed engineers helping out with custom integrations, scoping and figuring out what’s possible, and feeding all of that back into core product development. That’s how I think about it. It’s almost a first-line, second-line defense—

Basil Chatha

Yeah.

Jacob Lauritzen

—where legal engineers should be able to customize a lot of things and build a great experience. Where they can’t, or where we need to make core product changes or custom integrations, we have forward-deployed engineers who cycle in a little behind them. Sometimes they’ll just work on the product, and sometimes they’ll work—

Basil Chatha

Yeah.

Jacob Lauritzen

—with clients.

Basil Chatha

Can you walk through an example? You’ve just got this new client. What’s step number one? Do you say, “Okay, we have a forward-deployed engineer for you. They’re going to build some kind of product for you”? How does that work?

Jacob Lauritzen

It’s very different from client to client.

Basil Chatha

Yeah.

Jacob Lauritzen

If it’s a big client and they’re super engaged, they have all these dreams of things they want to do. Typically, they’ll have already tried to build something and failed, and they’ll say, “This is a problem we’d like to solve.” Then we’ll staff them with legal engineers and potentially a forward-deployed engineer, depending on what they need. We’ll say, “These people will come and make you successful. They’ll sit with your lawyers, teach them how to use it, figure out your current workflows, and build them into Legora.” Some of that will just involve a legal engineer, and some of it will require a forward-deployed engineer.

Basil Chatha

What’s the separation, if you could talk about that a little bit?

Jacob Lauritzen

Legal engineers configure the product as a power user would.

Basil Chatha

Okay.

Jacob Lauritzen

They don’t write code.

Basil Chatha

Yeah, yeah.

Jacob Lauritzen

A forward-deployed engineer writes code into production, basically. I guess that’s the—

Basil Chatha

Yeah. What kind of background do forward-deployed engineers normally have?

Jacob Lauritzen

Regular engineers.

Basil Chatha

You said YC founder.

Jacob Lauritzen

Yeah. Founders, regular engineers. They’re all—

Basil Chatha

You have, like—

Jacob Lauritzen

Yeah.

Basil Chatha

Dee was your—

Jacob Lauritzen

Yeah, exactly.

Basil Chatha

—was the one with two YC companies and worked on search at Meta.

Jacob Lauritzen

Dee was our first, yeah.

Basil Chatha

Okay. That’s a high bar.

Jacob Lauritzen

She’s awesome. She was the first boots on the ground in the US.

Basil Chatha

What are some examples of workflows or things these people are building for clients? Are there a couple of use cases that all the law firms say, “This is the first thing our FDEs are going to do when they go in”?

Jacob Lauritzen

It’s very different.

Basil Chatha

Okay.

Jacob Lauritzen

There are some core workflows everyone has that we can just build in Legora. That could be anything from how we handle certain types of NDAs. You activate it, and it’s more like skill-building for agents.

For most of the FDEs, it’s more that there’s something we can’t do yet: we need more flexibility in some of the primitives, or we have a really complex system we need to integrate with. Then they’ll go out and build the integration against our APIs, and maybe expand our APIs to make it work.

Basil Chatha

How does the legal structure work there? You said you guys can’t even work on deals.

Jacob Lauritzen

Mm.

Basil Chatha

... yourselves can’t even work on deals.

Jacob Lauritzen

Yes.

Basil Chatha

So, how do they work on client projects?

Jacob Lauritzen

You mean the law firms? How do they work on client projects?

Basil Chatha

Sorry.

How do your FDs—

Jacob Lauritzen

Yeah.

Basil Chatha

... work on client projects if they’re not—

Jacob Lauritzen

They don’t touch the data, still.

Basil Chatha

Oh, okay.

Jacob Lauritzen

They’ll just build the system.

Basil Chatha

Okay.

Jacob Lauritzen

But they’ll also be onboarded like an employee sometimes. They might get a laptop and everything, if they need to.

Basil Chatha

Like, at the law firm?

Jacob Lauritzen

Yeah, yeah.

Basil Chatha

Oh, wow.

Jacob Lauritzen

If they need to.

Basil Chatha

Interesting.

Jacob Lauritzen

That doesn’t happen very often, because usually they just build an integration and it kind of works. But they can, if they need to.

Basil Chatha

And you only onboard FDs into a customer, I’m assuming, if they’re a certain size.

Jacob Lauritzen

Yeah. Yeah.

Basil Chatha

Okay.

Jacob Lauritzen

It needs to make sense.

Basil Chatha

Yeah.

Jacob Lauritzen

Because our FDs can also work on the core product, right? So it’s a constant prioritization thing.

Basil Chatha

Oh.

Jacob Lauritzen

Is there a rule? I guess, does every—Palantir famously had four days you spend on site.

Basil Chatha

Mm.

Jacob Lauritzen

One day you come back and—

Basil Chatha

Yeah.

Jacob Lauritzen

... contribute back to products. Anything like that?

Basil Chatha

No, we don’t—

Jacob Lauritzen

Nothing like that?

To be fully honest with you guys, it’s still early days in a lot of this. Yeah.

Basil Chatha

I mean, where do you see it headed over the next year?

Jacob Lauritzen

I do think it’s going to be more like Palantir, maybe not exactly. I think we’ll have a larger, constantly circulating group of FDs who might sit with a client for two months or whatever to build out one thing, and then they go to the next one, et cetera. I do imagine us rolling in with legal engineers and forward-deployed engineers to work on specific projects.

Basil Chatha

Hmm. Oh.

Jacob Lauritzen

Yeah. The API business is actually interesting, like you said.

Basil Chatha

Mm.

Jacob Lauritzen

Because that’s another role you can have, where you’re the CTO or maintainer of the Legora platform.

Basil Chatha

Yeah.

Jacob Lauritzen

Right?

Basil Chatha

Yeah.

Jacob Lauritzen

We’ll offer everything we have in the Legora web app—

Basil Chatha

Yep.

Jacob Lauritzen

... as an API, and then you can use whatever. Is that a promise you make, or—

Basil Chatha

That’s not something we’re promising now.

Jacob Lauritzen

Yeah.

Basil Chatha

Um—

Jacob Lauritzen

We do have APIs you can build against—

Basil Chatha

Yeah.

Jacob Lauritzen

... but not the entire surface of the product. That’s the curious thing: is that the strategy?

Basil Chatha

Hmm.

Jacob Lauritzen

Obviously, some people will love it because—

Basil Chatha

Yeah.

Jacob Lauritzen

... I don’t have to learn a new tool. I can just use the existing tool I have.

Basil Chatha

Yeah.

Jacob Lauritzen

Right? And then I just—

Basil Chatha

Yeah.

Jacob Lauritzen

... install Legora, for example, with the Word document stuff.

Basil Chatha

Hmm.

Jacob Lauritzen

Other than that, I do think you have to maintain it. Exactly. That’s been my thing, and that’s why we’ve tried not to do it. We haven’t had APIs. I mean, we started our APIs about six months ago. We fought it for a very long time, mostly because we move so quickly in the product, and I don’t want to be slowed down by having to maintain all the API versions for a data model that we’ve changed. Now we’re bigger and things are different, so it’s a cost we can take on. But when we were 20 engineers, we moved so quickly that we couldn’t do this.

swyx

I’ve toyed with this. It’s generally good for all companies to have a platform team and a product team. You have one product team, but you can easily spin up a second one as long as the platform isn’t too tied to a product.

Jacob Lauritzen

Yeah. Yeah, yeah.

swyx

A very simple example is a web app and a mobile app, right? Or a web app, a mobile app, MCP, and other integrations. Then your FDEs are just one more client to the overall platform anyway.

Jacob Lauritzen

Yes. And I think even the agent, you can think about it this way. There’s a harness team that runs the agent, and then you can build a product around it with skills, automations, and whatever else.

swyx

Yeah.

Jacob Lauritzen

Um—

swyx

This is very famously the Amazon Web Services mandate. Bezos was like—

Jacob Lauritzen

Everything’s an API.

swyx

... “Guys, no more backdoor access. You only offer other teams the same APIs you offer customers.”

Jacob Lauritzen

Mm.

swyx

If you don’t have it, you build it.

Jacob Lauritzen

Yep.

swyx

And then the customers also get it.

Jacob Lauritzen

Yeah. I think that’s also changing a little bit. A lot of APIs now—one thing we’re toying with is whether the agent can be the API. I know it sounds weird, but for some of this stuff, what they actually want to do is trigger certain things to happen. For some of it, you could just trigger the agent and ask it to do the thing. I’m not saying we’re doing this, but it’s something I’m toying with: could you just trigger the agent to do this? Obviously, it’s way less efficient than having an API that you always hit. But I’m toying with how much of our product can just be the agent versus all the other stuff.

swyx

Well, you know, unbounded execution, costs, and latency—

Jacob Lauritzen

Yeah. That’s the—

swyx

Is that the only drawback?

Jacob Lauritzen

Yeah, and then determinism. Maybe one or two times out of a hundred, it’s going to do something whack that we didn’t want it to do.

swyx

We’re going to get into harnesses and model fine-tuning in a bit, but I think there’s a product question about restraint—about what you don’t do. You could do all that, and obviously there’s a prioritization issue. Are you at the point of saying no to most things and only doing a small number of things, or do you want to do more things and just need to hire 100 people?

Jacob Lauritzen

I was very strongly in favor of the first thing for a very long time. Let’s just say no to everything and keep the engineering team small. I’ve said this before, but in one of our presentations, after I’d been at Legora for half a year, I put up this slide of the 300 Spartans from the movie, and I was like, “We’re just going to be 20 engineers for the rest of the company’s history, and we’re going to beat everyone who’s much bigger.” We’re 100 engineers now, and it feels like we’re way too small. I want to do that, but in the real world, we’re in fierce competition. People want so much stuff. I could be a puritan and say, “No, we have to stay 20 people, and we can only build whatever,” but then we lose, and people go elsewhere. So we have to build more. My job is to figure out how to build a system where we can build a lot, but the customer feels like we’ve built so much stuff when we’ve actually only built a few things. That’s how I think about my job right now.

Speaker 0

Yeah. I mean, primitives, right?

Basil Chatha

No, exactly. I think that’s really powerful.

Speaker 2

Do you test new products, just to see if there’s any pull? Then do you decide, “No, let’s just focus on our main thing”?

Basil Chatha

We haven’t made—I mean, we’ve made certain strategic bets, but we haven’t tried anything we don’t already have a lot of confidence in. Mostly because we have a mountain of demand: “We want all of these things.” It hasn’t really made sense for us to say, “Hey, maybe this thing could work,” because so many people are asking for the other things. So we mostly build those. Sometimes we’ll disagree with clients and say, “Actually, I think this thing over here will make a lot of sense strategically,” and then we’ll try to build that anyway.

Speaker 2

Strategically for you?

Basil Chatha

Yeah, exactly.

Speaker 2

So what are those things?

Basil Chatha

Or the industry. One of those things was the agent, actually. No one was asking for, “I want the assistant I’m using to have more agency and do things without me asking it.” No one ever did, but we were just like, “This is definitely where the industry is going. This is going to empower you to do so much cool stuff.”

Collaboration was another one. We thought the industry was moving toward collaborating inside the place where you’re working, which is Legora, rather than in emails and text messages. So, yeah, there are a few of those.

Speaker 2

And then once you say, “Okay, yeah, we want you guys to use agents,” I’m assuming it’s going to take a lot for people internally at the law firm to actually learn how to do that.

Basil Chatha

Mm-hmm.

Speaker 2

And so that’s—

Basil Chatha

Yep.

Speaker 2

—where you say, “Hey, FDEs, go in there and teach them what they can do”?

Basil Chatha

Yeah, legal engineers, mostly—

Speaker 2

Oh, legal engineers.

Basil Chatha

Yeah, yeah.

Speaker 2

Okay, that makes sense.

Basil Chatha

Some people call them forward-deployed legal engineers, if you want, but that’s essentially what they are. Then they go in and help structure their workflows around the capabilities of the agent.

Speaker 2

You guys said previously—I think Max said this—but that early on, specifically, you weren’t fine-tuning your own models. Is that still the case?

Basil Chatha

Yeah. We haven’t fine-tuned any models up until now. I think that in the past few months, open-weight models have become extremely capable, so we’ll do it for certain tasks where we think there’s a huge delta between what a frontier model can do in terms of cost and latency and what we think a post-trained model can do.

Legal research is a great example. It’s a fan-out retrieval problem, and you can make a really effective model for those queries very quickly and cheaply, with low latency.

Speaker 2

So you’re post-training them specifically to be better at searching—

Basil Chatha

Very specific, bounded tasks. Yeah, exactly.

Speaker 2

Okay.

Basil Chatha

Yeah, yeah. We’re not trying to train our own foundation model to be the main agent. I don’t think that’s something we’ll win at, versus Anthropic, most likely.

Speaker 2

Yeah. There’s still a lot of search and retrieval research, or RAG research, to do that’s maybe less glamorous. It’s not even always model training; sometimes it’s just architecture and pipelines. You did a case study with TurboPuffer, but do you do late interaction?

Basil Chatha

Yes, absolutely. Yeah.

Speaker 2

You do that?

Basil Chatha

We actually don’t do that.

Speaker 2

Right.

Basil Chatha

But you’re completely right. There’s one guy on the team, Song, who really cares about search. He comes to me all the time and says, “We should do these things. There are these new embedding models, and we can do ColBERT and late interaction.” He actually talked to the TurboPuffer team to get them to build multi-vector support so we could do these types of things.

But there’s loads of low-hanging fruit, generally, even before models. And even in the harness, if you just want to create a better output for our clients, there’s so much stuff we can do that has an immediate impact before training models.

Speaker 0

Yeah.

Speaker 2

What makes a good harness for the legal industry, I guess? I’m assuming you guys are making your own custom harness.

Basil Chatha

Yeah, of course.

Speaker 2

So, yeah.

Jacob Lauritzen

The way we edit files is important. The way we do analysis is important. The way you do legal research is all baked into it. You might have a problem come up, or see something, and then it knows how to do the legal research to figure out what the answer is and what the precedent is. That’s all very important. And then, of course, there’s all the governance stuff around it as well.

Speaker 2

Yeah, it seems like search is probably the most important tool you’re customizing, because if you can’t pull down the right information—

Basil Chatha

Yeah, and then you’re—

Speaker 2

Yeah, yeah.

Basil Chatha

—you’re screwed.

Speaker 2

You’re screwed.

Basil Chatha

Yeah. I think it’s search—

Speaker 2

You guys are searching a lot of documents—

Basil Chatha

—and analysis.

Speaker 2

Yeah.

Basil Chatha

Yeah. It’s search, analysis, and drafting, probably. Drafting huge contracts is actually not super trivial.

Speaker 2

What about it isn’t trivial?

Basil Chatha

If you’re drafting a huge contract from scratch, you have to find precedents. How did we do this previously? Do we need this type of clause or not?

Speaker 2

A specific law firm—how did they do it previously?

Basil Chatha

Yes, yeah. I am the law firm, and I’m about to draft a huge whatever. An NDA is very simple, but there are contracts that are way more complex, and then you need to find the right precedent first. Lawyers don’t just sit and start typing stuff.

They figure out which clauses they’ve used in similar deals, and “similar” can mean a lot of different things, as I’m sure you know in search. It could be the same industry, the same type of company, or whatever. You need to identify the right precedent. You might have multiple documents and then try to figure out which part to take from which document. That’s actually nontrivial. So it’s essentially also a search problem.

Speaker 0

Which model’s best at drafting?

Jacob Lauritzen

I was told I was far from this yesterday, but Fable and Opus are definitely extremely chatty. I’m sure you guys have noticed this. They’re very verbose.

Speaker 0

So that’s not good.

Basil Chatha

That’s generally not good, no.

Speaker 0

Okay.

Jacob Lauritzen

Astra is pretty good at writing—

Basil Chatha

I noticed Astra is pretty good at writing.

Jacob Lauritzen

Yeah, Astra is pretty good at writing.

Speaker 0

And is it promptable?

Basil Chatha

It’s much more concise. We can do certain things, but I think the chattiness and verbosity are actually a little bit difficult for us to address.

Speaker 0

Wow.

Jacob Lauritzen

Let’s see, but the next model from Anthropic will probably be much better at this. Astra is actually very concise.

Speaker 0

Yeah, yeah.

Speaker 2

And I assume they all have a Slack channel with you, and you’re telling them this, right?

Basil Chatha

Yeah, exactly.

Speaker 2

Like, this is—

Basil Chatha

Yeah, we talk about it.

Speaker 2

—this is not news to them.

Basil Chatha

No, no, they know this.

Speaker 2

All right.

Basil Chatha

The other thing I really care about is hallucination.

Speaker 0

Mm.

Basil Chatha

Do they make stuff up? We have a citation system where we basically check that what they’re writing has a link. The user has a link they can click, and it highlights with a bounding box in the PDF where the information was taken from. We can make sure that what it says is always citable somewhere, even multiple layers down. If the agent writes something based on a tabular review, it can cite the cell or the row in the tabular view. You can click multiple layers down and validate it all the way up.

Speaker 0

Yeah.

Speaker 2

Are you finding that lawyers are verifying that these citations are correct or not?

Basil Chatha

No.

Speaker 0

Initially, some of them were.

Speaker 2

It’s kind of like law review, but now are you actually doing it?

Basil Chatha

Yeah, exactly. It’s the same thing.

Speaker 0

People ship hallucinated links to the Supreme Court, and they don’t care.

Basil Chatha

That happens, yeah. I think it’s like coding. You have engineers who just don’t give a shit and send agents, and you have people who sit and read every comma. It’s the same distribution overall.

Speaker 2

There’s just so much. How can one human do that, right?

Basil Chatha

Yeah, but I think they probably just, like we know in code review, know what to look for.

Basil Chatha

Like, what is the system or whatever? I think they also know, “Okay, this is the thing that’s important. Let me double-check these things.”

Speaker 2

Yeah.

Basil Chatha

Then these sound reasonable, so it’s probably fine.

Speaker 2

Okay. Interesting.

Speaker 0

Just like if you receive something from a junior lawyer, it’s the same thing you do, right? You think, “I should double-check this,” and then this probably is fine.

Any other elements of legal reasoning—or, what’s the legal frontier that models aren’t good enough at yet, that you want from the next generation of models?

Basil Chatha

Nothing I can think of specifically right now. Long-document reasoning is still difficult.

Speaker 0

Long, like million-token contracts or something?

Basil Chatha

Yeah, exactly. You have a huge contract, and you’re making edits across it. That’s difficult for a model to do today.

You have to remember that you changed all these things up here, and that the definitions changed, the ordering changed, and whatever else.

Speaker 0

So the simplest approach would be one-shot: you feed everything in, then get a one-shot output.

Basil Chatha

Yes.

Speaker 0

You don’t do that. You break it up into chunks and orchestrate edits.

Basil Chatha

Yeah, exactly.

Speaker 0

This is one of those fun things about being part of this industry, right? Our Word add-in has gone through MapReduce, and we’ve changed. There was one big one, and there was one big one in multiple sites. We keep changing the scaffolding we build around the models as they change.

swyx

Yes. I’d say I’ve recently shifted from MapReduce to throwing everything into context. I do a lot of news reporting, obviously, because of what I do.

Jacob Lauritzen

The entire thing. Yeah.

swyx

And now it’s—

Jacob Lauritzen

Because now they’re—

swyx

—it’s better.

Jacob Lauritzen

They’re good at it now.

swyx

Yeah, exactly. But it’s also different. We had this problem with search, where you’re searching for certain things in huge contracts. You can use MapReduce, but if you’re summarizing something, it might lose important nuance between the chunks, essentially if you just summarize the individual parts.

We’ve been dealing with this problem. We can do 2 types of search. We can also do the other kind, where you need to have reasoning in the different steps in between. I’m trying to think of a really good example. There are some documents where they talk about something at length over, let’s say, 2 pages, and each individual chunk isn’t necessarily relevant to the search term or what they’re trying to figure out. But together, it’s extremely relevant. These are some of the non-trivial problems.

Basil Chatha

And it must be hard, like you said earlier, to actually make the models better at that, because you can’t even look at the contracts, right? These are real contracts.

swyx

Exactly.

Basil Chatha

So do you ever ask them, “Can you make example contracts that are similar to your real ones?”

swyx

We do, yeah. We can receive feedback from them, if they’re willing to engage.

Jacob Lauritzen

Get the company—

swyx

Yeah, exactly. Or they’ll say, “This is super important to us, and it’s not working well.” Then we’ll say, “Okay, great. Can you anonymize the documents?” And then, whatever the problem is, if we can recreate it, we can fix it.

Jacob Lauritzen

One version of this is that you buy a law firm, which some people are considering.

swyx

Yes, but there’s some stuff around whether their data is actually their data or the client’s data. So if the clients are all gone, then maybe it’s fine. I actually don’t know.

Jacob Lauritzen

Interesting.

swyx

But, yes, it’s a way to get a lot of data.

Jacob Lauritzen

Yeah. People like Google are buying Spirit Airlines.

swyx

Google, exactly. Yeah.

Jacob Lauritzen

Right? And, like—

Basil Chatha

Say what?

Jacob Lauritzen

Yeah, they bought—

swyx

Yeah, they bought the—yeah, for like $10 million or whatever.

Jacob Lauritzen

Oh, the data.

swyx

Yeah.

Jacob Lauritzen

If you have a failed company, you still have your Slack. You can sell your Slack. I think there’s a YC company that buys your failed company.

swyx

Yes, there is.

Jacob Lauritzen

Yeah, because it’s just like, “Oh, I just want real—”

swyx

Yeah, people—

Jacob Lauritzen

“—people doing work.”

swyx

Exactly.

Jacob Lauritzen

And then just pull stuff out.

swyx

Exactly. The problem with buying a law firm’s data is that you get all the inputs, the real inputs, which is great. The outputs are a little more difficult, because you just get their end result, and it’s difficult to figure out how they got there. You basically have to reverse-engineer it. In our product, when we get real traces from real use cases, it’s extremely high-value. But if we just get an output and don’t know if it’s good or bad, or how it was created, it’s a little difficult.

Jacob Lauritzen

We’re going to wrap up soon. The last thing I’ll point people to is that there’s also a lot of alpha left on embeddings. If you’re doing a RAG-heavy workload and you’re just looking at MTEB and taking the top model, you’re leaving a lot on the table. You should look at Voyage or some of the custom embeddings. You’ve looked into this.

swyx

Yeah, absolutely.

Jacob Lauritzen

Basically, don’t take default embeddings. Even if they’re OpenAI embeddings or Gemini embeddings, customize your embeddings.

swyx

Yeah.

Jacob Lauritzen

Right? You support that?

swyx

Yes. I think that’s right. The only thing is you need to make sure they’re multilingual. We serve so many languages, so that’s going to be one of the problems.

Jacob Lauritzen

I mean, Europe is king of this.

swyx

Yeah, exactly.

Jacob Lauritzen

Is Mistral helpful there?

swyx

No.

Jacob Lauritzen

Ooh.

Basil Chatha

Not yet. They just raised a ton of money, right? So maybe.

Jacob Lauritzen

Yeah.

swyx

Let’s see.

Jacob Lauritzen

You guys better be Europe’s champions.

swyx

They also raised a ton of money a year ago. I don’t know. I think we need way more model companies out of Europe.

Jacob Lauritzen

Yes.

3. The Next Two Years Of Law

Basil Chatha

I just wanted to ask what predictions you have for the law industry over the next year.

Jacob Lauritzen

We asked a little bit about that earlier. I’ll modify it to your second question: In 2 years, will lawyers do their work inside Legora, or will they use other tools like Claude or ChatGPT and call Legora, like through an API function?

swyx

Yeah, yeah. I think both. It really depends. If you’re a big enterprise, you might have Claude or Copilot, and then you’ll… I think it’s similar to how I use Linear through Claude, but I still go to Linear. I think enterprise will do that. I think law firms will just use Legora because it’s simpler and better.

Basil Chatha

It kind of reminds me of OpenEvidence for medicine. You’re basically that for law.

swyx

Exactly.

Jacob Lauritzen

In coding, it’s gotten to a point where the labs will cut you off for competing with them. This has happened twice.

swyx

Yeah.

Jacob Lauritzen

Do you think that will happen in law?

swyx

No, I don’t think so.

Jacob Lauritzen

Yeah.

swyx

I think—

Jacob Lauritzen

Because they’re releasing legal-services products.

swyx

Yeah, they are. We obviously talk to them a lot, but it’s also like, “Hey, are you going to…” What’s—

Jacob Lauritzen

Your sales reps will say no. They don’t make the call.

swyx

Exactly. That’s not their call to make.

Jacob Lauritzen

Right? And they often don’t even know what’s actually going on. I don’t think that’s the case. They have so much stuff to capture in knowledge work, or in enterprise knowledge work generally, that legal has to be part of that, just like data analytics has to—yeah.

swyx

—and sales and marketing and finance. They have to do things that are good there, but they’re never going to go as deep as we are. If Claude takes small corporate clients and they do some of their legal work via Claude, that’s great. That’s fine. But if you’re doing a $400 million M&A transaction, you’re going to want Legora.

Basil Chatha

I had kind of an unrelated question. I’m curious: are you guys going to law schools and getting them to teach Legora in classes?

swyx

Mm-hmm.

Basil Chatha

How does that work? Are you guys doing that?

swyx

Mm.

Basil Chatha

Like, are you seeing law schools changing their curriculums a lot, or not really?

Jacob Lauritzen

I don’t know. I wouldn’t know, actually.

Basil Chatha

Yeah. Interesting.

Jacob Lauritzen

I would actually do it—

Basil Chatha

I have to—

Jacob Lauritzen

—a different way. You should do scholarships for law school for CS majors, for engineers. We’ll cover your—

swyx

After you’ve done CS—

Jacob Lauritzen

Yes. CS plus some software experience, and we’ll put you in law school.

swyx

Yeah.

Jacob Lauritzen

We’ll cover it.

Basil Chatha

Or you could do the master’s-plus-JD programs.

Jacob Lauritzen

Yeah.

Basil Chatha

I mean, a lot of people didn’t do CS as undergrads, so then they do a master’s just to get caught up.

Jacob Lauritzen

Yes.

swyx

Mm.

Jacob Lauritzen

Yeah. I don’t see that as often.

swyx

I don’t—

Jacob Lauritzen

Yeah.

swyx

Yeah.

Jacob Lauritzen

People do JD/MBA. That’s a thing.

swyx

Yeah.

Jacob Lauritzen

But JD/CS, I haven’t seen it.

swyx

Yeah, it happens.

Jacob Lauritzen

Yeah. Overall, super impressive progress. Congrats on all your success. Are you still going to keep up the pace?

swyx

Absolutely.

Jacob Lauritzen

Yeah. And I think you’re hiring in San Francisco, New York—

swyx

We’re hiring in New York, Stockholm, and London for engineering.

Jacob Lauritzen

Yeah.

swyx

For all other roles, we’re hiring everywhere.

Jacob Lauritzen

How many people do you have in New York right now?

swyx

About 100.

Jacob Lauritzen

Okay. And how many are you planning to have in New York?

swyx

In New York?

Jacob Lauritzen

Yeah.

swyx

Generally?

Jacob Lauritzen

Let’s just say generally for engineering.

swyx

A lot.

Jacob Lauritzen

As many as you can?

swyx

I know. It’s aggressive, yeah.

Jacob Lauritzen

Cool.

swyx

I mean, that’s the home of corporate law, as far as—

Jacob Lauritzen

It is, yeah.

swyx

—as M&A is concerned.

Jacob Lauritzen

New York is—

swyx

So, yeah, I’m looking forward to you buying a law firm and becoming—

Jacob Lauritzen

Yes.

swyx

Yeah. It’s super exciting. I think every industry, especially the top doctors, lawyers, and coders, they all have to have two or three agent labs that just—

Jacob Lauritzen

Yes, I agree.

swyx

—and sort of dominate the space.

Jacob Lauritzen

Yeah.

swyx

So you guys have a great market. Good.

Jacob Lauritzen

Yeah, I think so.

swyx

And I think it’s only going to get much bigger as law changes and becomes more available to more people.

Jacob Lauritzen

Yeah. We’re already way over, but the problems you work on are very much frontier knowledge work.

swyx

Yes.

Jacob Lauritzen

You know?

swyx

Yeah.

Jacob Lauritzen

Unless you’re literally doing science—Navier–Stokes and science—most work basically converges to legal.

swyx

Yeah.

Jacob Lauritzen

Yeah. It’s all legal in the end.

swyx

Yeah. Hiring, finance, everything—I just have to consult my lawyer.

Jacob Lauritzen

Yeah, I know. Which is why I think it’s a really exciting thing to work on, because modern society runs, in some way, on legal.

swyx

Yeah. Yeah.

Jacob Lauritzen

Cool.

swyx

All right. That’s it.

Jacob Lauritzen

All right.

swyx

Thank you.

Jacob Lauritzen

Cool. Thank you.

swyx

Thanks.