Cracking the Medical Code: Why Cleveland Clinic Doctors Love Their Ambience Healthcare AI Scribe

Today my guests are Brendan Fortuner, head of engineering at Ambience Healthcare, and Ben Shahshahani, chief AI officer at Cleveland Clinic. Did you know that the U.S. healthcare system spends $1 trillion per year on administrative tasks? Or that doctors spend hours each day during what they call “pajama time” documenting their patient interactions after hours? Or that human doctors are only 45% accurate when it comes to translating their understanding of patient conditions into the ICD-10 codes used in medical billing?

These codes are then painstakingly reviewed by coding specialists employed by both healthcare providers and insurance companies. I knew there was a lot of room for improvement in the U.S. medical system, but I honestly didn’t realize the magnitude of the opportunity. When the CMO at Ambience initially reached out to suggest this episode, I checked the Ambience website and saw that they offer, among other things, an AI medical scribe for doctor-patient interactions. Remembering a recent chat I’d had with a doctor friend of mine who had been complaining about the inaccuracy and general uselessness of the AI scribe deployed in his clinic, I initially failed to recognize what an interesting conversation this could be.

That changed when I happened to see that Ambience was featured as a successful early adopter of OpenAI’s reinforcement fine-tuning product. Having once earned such a feature myself at Waymark, I know they don’t come easily. When I saw that reinforcement fine-tuning had allowed them to outperform human doctors on the ICD-10 medical coding task by a full 12 percentage points, I knew I wanted to dig in and learn as much as I could.

In the end, this conversation, to which Brendan also invited his customer and friend Ben from the world-class Cleveland Clinic, turned out to be an excellent one, spanning both technical implementation and practical deployment strategies. We get pretty deep into the details of how Ambience has achieved such strong results, including its specialty-by-specialty approach and how it uses reinforcement fine-tuning to optimize the models’ ICD-10 coding F1 score. We also discuss the instances of reward-hacking behavior they observed and how they addressed them.

We even get into the patient-facing products they’re now developing to improve outcomes while further reducing the burden on staff by automating the follow-up calls that nudge patients to get tests done and take their medicines as directed. As an aside, I briefly confused the F1 score with pass@1 for a moment in this conversation, so it’s probably worth mentioning that the F1 score is a way of balancing precision—the percentage of the system’s outputs that are correct—with recall—the percentage of all correct outputs that the system produces—by taking the harmonic mean of those 2 numbers.

On the deployment side, I think Ben’s account of how users develop mental models about which AI tools are worth using, considering both the success rate and the effort required to recover from errors, is a brilliant distillation of things that I and many others have learned the hard way but perhaps never articulated quite so clearly. It was also fascinating to learn that Ben and his team required Cleveland Clinic doctors to use the Ambience medical scribe just once, and that this single interaction was enough to drive 75% voluntary utilization across 4,000 physicians spanning some 60 specialties.

For operational leaders wondering how to think about AI adoption mandates, and for AI product builders wondering what level of reliability is really required for success, this is absolutely something to chew on. The medical scribe company that serves my friend’s clinic has clearly wasted a precious opportunity. There’s a lot more here as well, including a discussion of what happens to the people who are currently employed as medical coding specialists.

I hope you enjoy this outstanding case study of where the rubber of AI product development hits the road of deployment in complex, high-stakes, regulated environments full of understandably skeptical users, with trillions of dollars at stake and the potential to transform American healthcare as we know it.

Happy to be here.

I’m excited for this conversation. It’s been a number of weeks in the making, so just to tell a brief backstory: I got an inbound pitch from, I believe, the CMO at Ambience. I did a quick look and saw the AI scribe concept for the medical context. As it happened, I had just talked to a friend who’s a doctor and was complaining about his medical AI scribe. I thought, “How do I evaluate this? Some of these things might suck out there, and others could be good, but I don’t really know.”

Then, days later, a case study popped up on the OpenAI website, which is a strong signal that you know what you’re doing. Having seen that, I was immediately in. I thought, “All right, you guys are the AI scribe for the medical context that I want to talk to and learn from.” Thank you also for bringing an additional guest, which is incredible. We’ll have the chance to talk about both the technology side and the implementation side, as well as the social context in which all of this actually hits the road.

Maybe, for starters, give us a quick introduction to Ambience Healthcare.

Yeah, for sure. Thanks again for having us. We’re super excited. I think this is going to be a ton of fun.

Ambience was founded about 4 years ago. We’re building an AI platform for hospitals. You could think of us as a clinical intelligence layer that sits on top of the system of record, which is the EHR, like Epic and Cerner. We help augment, enhance, and automate both clinical and administrative tasks to make the overall hospital system more efficient and actually improve the quality of care.

There are 3 different product lines at Ambience. “Platform” is often an ambiguous word—what is a platform?—but I can make it concrete. The first is products for clinical workflows. This is the flagship Ambience Scribe, the one that we first deployed at Cleveland Clinic, which helps doctors take notes.

The second is products that help revenue-cycle teams. Hospitals also have to make money and remain compliant, so Ambience is building out a suite of point-of-care products for coding and billing, which we can talk about. Very recently, we’ve also moved into patient-facing products. Think patient engagement: we have patient instructions that we’ll write for patients, but also more advanced things like voice agents that will call you on the phone and check whether you’ve gotten your labs done and taken your medications.

Those are the 3 different tenants of the company, and we’re growing really fast.

So many things are going vertical in the AI space right now. It’s really quite something to behold. Ben, do you want to tell us how long you guys have been working together? This is all coming at the healthcare industry very quickly.

One of the things that I expected 2 and a half years ago, and that I’ve been very pleasantly surprised by, was that when I first tested GPT-4, I thought everybody was going to unionize like crazy. We were going to see the most protective moves we could possibly imagine, whether it was taxi drivers, lawyers, or doctors. Everybody was going to try to protect their turf and keep AI out of their environment.

Actually, a lot less of that has happened over the last 2 and a half years than I would have expected. One candidate explanation I’ve had is that maybe doctors are all so burned out that they’ll take any help they can get. How would you describe the last couple of years of growing awareness and adoption, as well as the reception and the lack of hostility toward AI that, at least from my perspective, seems to have been the norm in the medical world?

First of all, let me say a little bit about Cleveland Clinic. We are an academic medical center in Ohio. We have basically 3 charters in our mission statement. One is providing healthcare. We have about 25 hospitals. It’s actually a little bit unusual in the sense that we have an international footprint.

We have hospitals in Ohio and Florida. We have a hospital in London and in Abu Dhabi. We also have patient facilities in Toronto and in a few other states. I joined Cleveland Clinic about 9 months ago, and my background was in tech. One of the reasons I came to healthcare was the application and implementation of AI in healthcare.

What you said is interesting because, in terms of the clinical usage of AI, if you think about the work that doctors and nurses are doing, there’s such a shortage of caregivers and so much demand that the issue of “Hey, is this thing going to take away my job?” is really not something that we see, maybe even in our lifetime, because there is so much demand for healthcare.

Doctors are using it. We know from surveys and publications that a lot of doctors are using ChatGPT or other AI systems. What we want to do is implement it in the right way, rather than having them think, “Hey, I’m using it. Am I using it the right way? Am I using it the wrong way?” or mistakenly put confidential or PHI-sensitive information into a chatbot.

We want to bring those things to the forefront and actually make them part of their standard workflow. Obviously, there are a lot of challenges in this, but we think of it as a huge and important productivity tool. The implementations that we've had so far with AI, most of them, I would say, are what you would consider productivity tools.

If you look at it in that way and look at demand and supply, productivity tools don't necessarily lead to a reduction in force, right? It depends on how much demand there is for that service. If there's more demand, it just means that you can serve more people because now you're more productive. There may even be situations where, if you have more productivity tools, demand actually goes up because you can reduce the cost, for instance, and that actually leads to more demand. So, the fact that something automates and helps you be more efficient doesn't necessarily mean that you're in trouble. It really depends on the entire ecosystem.

Yeah, so tell us a little bit more about the pain points. I'll maybe circle back a little later to some of the more aggressive AI-doctor experiments that we're seeing starting to emerge, but staying within the productivity-tool lens for now, where do doctors need the most help? Tell us about productivity.

Documentation is definitely top of the list. There have been surveys. It's very well known that we have a shortage of caregivers—doctors in particular, and nurses as well—and they are burned out. There have been a lot of surveys about what's causing burnout, and in the ones I've seen, the number-one cause—over 53% of respondents—was administrative and nonclinical documentation, or rather, documentation overall.

That's definitely the piece where AI takes time away from documentation so they can do their clinical work. It actually takes away from their after-work time. They call it “pajama time,” right? On average, a doctor may spend up to 3 hours a day doing documentation, and a lot of doctors have to do it after their patient visits are done, in the evening.

When we started working with Ambience and started thinking about an AI scribe, we were looking at it from a caregiver-experience angle. How do we make the work of caregivers, of doctors, better? How do we reduce burnout and give time back to them to do what's important, which is the clinical work they went into the medical field to do?

Along the way, we found out that it has other side benefits that are also pretty damn good. The quality of the documentation actually impacts coding, like Brendan was saying, and that actually impacts financials. That wasn't necessarily the main reason we started, but it all becomes like domino pieces, one at a time. You see that the benefits of AI can be beyond just one area—burnout reduction—to better documentation, revenue aspects, and all that.

What are the stats that Brendan had shared that are kind of mind-blowing?

The math checks out: $1 trillion is spent on administrative tasks in the healthcare sector annually in the United States. You think, how in the world is that possible? But with a $27 trillion GDP, close to 20% of it going to healthcare broadly, it really only was like 14%. If that's $7 trillion, then it's really only 1/7 of that to get to $1 trillion. So, 14% administrative overhead on healthcare doesn't sound crazy when you put it that way. $1 trillion does sound pretty crazy when you put it that way.

My guess is that that's not even including all this time from the doctors. I wonder how—I mean, the accounting doesn't really matter too much—but are there pajama hours being factored into that $1 trillion? I don't know. I doubt it.

I've seen another really interesting report that, if you're interested, I can find out what the actual source was. It shows that from 1975 to 2010, the increase in the number of doctors in the United States was about 150%, which is apparently not that different from population growth. If you look at administrative workers—staff in healthcare—from 1975 to 2010, that went up more than 3,000%.

Okay, yeah. University-like numbers there.

The interesting thing is, just because you have more administrative people doesn't actually mean that the administrative work of doctors went down. They are now having more and more administrative work to do. Nevertheless, for a variety of reasons, the number of administrative, nonclinical staff in healthcare has really ballooned.

Yeah, and this is a little bit off topic for us, but why is that? Is that a function of just complexity? I'm familiar with the rise of the hospitalist, but that's still a medical position, right? What are all these administrative roles?

Because of different regulations. Those regulations require people to actually follow up and make sure that happens. There are hospital systems. Nowadays, actually, I think most of the doctors are employed by hospital systems, so it adds a different level of complexity to running these organizations.

There are a variety of factors—insurance and billing and all these things that need to happen—and they require a lot of documentation. So, all of those things, I think, added up.

Yeah, fascinating. Well, then, that is a lot. I've often asked the same question about universities.

Brendan, do you want to dig a little more in-depth into the product offering and the before and after? We know what the pajama time looks like. What is the post-appointment reality for doctors? How are they saving time? Are they also saving money?

I really want to get into a lot of the technical detail of how you made this work, because I think right now, obviously, reinforcement learning is a megatrend. Reinforcement fine-tuning from OpenAI is relatively new. I would say it's underutilized relative to the potential that it seems to have. So, give us the setup, but then we'll really dig in on how we actually make this happen.

I think maybe it's helpful to start with describing the product, because I think that's the bread and butter of Ambience and really becomes the epicenter of all the other product lines.

Ambience—just to describe the clinical workflow: you and I go into our doctor's office, and that's called an outpatient care setting. The doctor has an Ambience mobile application. We're also natively embedded with Epic. Epic has a mobile application; you can use that for recording with Ambience, but visits are typically 15 minutes or 30 minutes.

You'll bring the phone in, and it'll record that conversation. After the visit, you'll click “End recording.” Ambience will then transcribe that audio into text using fine-tuned models trained for medical speech recognition. We'll then call a bunch of different language models, all tuned for different medical tasks, to generate these clinical notes in different sections of the clinical note.

We then automatically write that back into Epic and the other EMRs. So, when the clinician is done with the visit, they can go back to their desks, refresh what's called their progress note, and boom, all the Ambience documentation is just sitting there for them. That turns out to save them 2 to 3 hours per day, just by automating that task.

But getting into that coding component, Ambience realized very early on: Wait a second. Doctors went to medical school to practice medicine. They did not go to medical school to select the right billing codes. There are 2 cases, which you can talk about with our CEO later, but basically, they make a lot of mistakes, and that leads to a lot of downstream pain for insurers and these revenue-cycle teams.

At the point of care, before they finish that note, Ambience will start to assist them. We’ll make suggestions on the right ICD-10 codes, for instance, and then automatically file them back into Epic. Again, this saves them time and increases what we’ll call the quality of the documentation.

That’s a little bit about the workflow. It’s different for every care setting and even every specialty, so there are some nuances there.

Speaking a little bit to the nuances, one thing that jumps out to me is that, obviously, in the AI space in general right now, there’s a lot of talk and a lot of confusion about agents: what an agent is, the nature of agents, and how agentic things should be.

It seems to me that you are doing what I think most people who are actually realizing major value are doing, which is not letting the AIs choose their own adventure through this problem. Rather, you’re really decomposing the task, setting up intricate workflows—maybe not super intricate in some cases, but step-by-step—where you’ve decided exactly how you want this thing to go, and then measuring and optimizing every node in that workflow until you get to something that’s deployable, and presumably beyond that as well.

You mentioned fine-tuning speech models, which is interesting, as well as fine-tuning multiple different models. I’d love to get as much detail as you can offer in terms of that breakdown, and maybe also how you work with the subject-matter experts, because I’ve seen that be a real stumbling point even in all sorts of Main Street businesses that want to do this.

There’s often the AI person who has some know-how, but then there’s a disconnect: how do you want this task to be done, and can the person articulate how they want it done? Obviously, that’s going to be pretty important to get right in the medical context.

Let’s hear your chain of thought on how you use the context window.

I doubt that.

Yeah, go for it.

Maybe we can start with specialties and some nuances there, because that really informs the architecture of Ambience. Then we can talk about particular components of the architecture, how we power them, and go from there. How does that sound?

That’s great. I first wanted to ask: you said some of your doctor friends weren’t seeing some benefits of scribing. Do you happen to know what specialties they were in?

The one friend I was talking to about this, interestingly, is a family doctor. It’s a pretty diverse set of reasons people come to him. He’s in a highly Spanish-speaking community, so that’s one wrinkle.

He probably speaks about 50% English and 50% Spanish on a day-to-day basis. Beyond that, I don’t know too much, but I think he sees a broad array of things on any given day.

I asked because I think this is something we also observed in early generations of Ambience. They were designed for primary care conditions or internal medicine. Those are typically the providers that you and I go to see once a year: we have a rash, and they go through our problems.

There are over 100 different medical specialties and 3 or 4 different care settings if you include telemedicine. What we found when we tried to expand to more and more complex specialties was that the product didn’t work. Users weren’t using it.

As we dug deeper and deeper, we realized there was tremendously rich heterogeneity across specialties. If you don’t bake that into your fundamental architecture, your product design, and the actual models that you’re fine-tuning, you won’t get adoption.

We had these weird charts where primary care, telemedicine, or urgent care was great, but oncology and cardiology had very little adoption. This was in earlier generations of Ambience. We stepped back and rearchitected our system to allow our team to think: instead of “scribe,” we think about “cardiology scribe,” “oncology scribe,” and “inpatient scribe.”

For each of those, you’d think the first step is different models, and we do have different models. But it’s more than that. It’s actually the user experience itself in the product.

There are different buttons and different elements of the chart that we’re going to pull from Epic into the note. There are nuances between primary care, cardiology, and oncology. Then you have the emergency department and inpatient settings. These are multiplayer settings: there’s more than one clinician, and they go see the patient multiple times over the course of multiple hours and multiple days.

Your whole workflow in the mobile app of “record and then stop” totally breaks. I think that was the first insight that really grounded Ambience’s philosophy and model development.

The benefits are tremendous. Maybe Ben can speak to some things that we saw at Cleveland Clinic, but by tailoring it in this way, you can increase utilization by 2–3×, and that has downstream effects for ROI, value capture, and clinician satisfaction. That’s the bread and butter of how we approach it.

What would you say has most moved the needle? I don’t know exactly what calendar time you were describing when you had primary care working reasonably well but other things weren’t working yet. I wonder what thresholds or big unlocks there were along the way.

I don’t know if that was pre-Whisper or post-Whisper, but it would seem like the open-sourcing of Whisper might have been one notable unlock for you. Once upon a time, although I may be moving into a new era now, I found a huge unlock just in training on chain-of-thought for certain tasks. I’m curious what the big step-change moments were for you along the way.

There’s some interesting ancient history here that I feel comfortable sharing. I joined the company 3 and a half years ago. At the time, this was sort of pre-inflection point for generative-model capabilities, and we were actually using an architecture based on BERT.

We were still doing scribing focused on telemedicine. The particular architecture was called BEAR. Instead of using just a generative, autoregressive model to generate tokens immediately, we broke the task down into an extreme-classification task over transcript chunks.

The chunks would flow in, and then we’d classify them based on this huge ontology of medical concepts, like the symptoms. We’d get this big list of concepts, and at the very end, we’d have a procedural compiler that would put everything together into a very templated note. It was based on BERT and ColBERT, and I think that got us through the early generations of the company.

The big inflection point, of course, was when GPT-3 finally released that instruction-following version. I don’t remember exactly—maybe 3 years ago or so—but nobody really knew what was happening. When you saw that and started playing around with it, you thought, “This is a different era. Everything has changed.” I think that was the first inflection point in our industry.

I do think some of the things with Whisper on the ASR side were another kind of inflection point. Since then, we’ve had other step-function improvements over time.

But it’s one thing to have that technology. Even now, you can see that, across the landscape, many products still aren’t working. As you said, in many specialties and in certain products, it’s still not working.

What Ambience did was take these generative models and, instead of having just 1 pipeline where we tried to have a 1-size-fits-all system that generated for all specialties, build a system where we could plug and play at a very granular level.

Which note sections do you want as a clinician? Do you want a history of present illness, followed by an ROS, followed by physical exam and assessment and plan? We found that cardiologists wanted something a little different, so we created separate models or separate prompts for them.

We let clinicians select what they want. With our users, there are probably 500 different permutations of outputs that we’ll give them just for the Scribe product. Clinicians can choose and compose them, and we give them styles so they can style them.

Do they want the note concise, bulleted, or narrative? Do they want to use layman’s terms? Do they want to use military time or clock time? I think that level of customization, even if it’s built on the same models, is what really unlocked the utilization.

How much simpler has the architecture gotten as the models have improved? In my case with Waymark, we have a similar history. People have heard me describe this before, so, very briefly: for images, we used to compile a bunch of content for a small business from the web, and then we would try to put that into a video form for them.

Around the same time frame, about 3 years ago, you could kind of caption an image, but the captions you would get would be very generic. You might have an image that was clearly a doctor and a patient in a medical setting, and the caption would be, “A man and a woman are speaking.”

Like it would lose all the context. That was the best image understanding we had, so we had all these hacks and whatever. It was pretty gnarly, but pretty fun to build, actually. I look back on that era of hacking on different models and trying to figure out how to make CLIP embeddings useful for sorting the things I was looking to bring to the top fondly.

But now, as you'd expect, we can just dump 100 images into Gemini Flash, Haiku, or GPT-4o mini—or whatever—and ask, “Which of these images should I pick?” It does a much better job than we used to do.

So I imagine that you have a similar sort of simultaneous simplification and expansion of possibility. Could you tell the story of that simplification and expansion as it's unfolded over the last 3 years? I want to hear your story, too.

It's a great question. I would say certain things have gotten easier, but as we've learned more about health care, we've uncovered new use cases and things we didn't anticipate that require additional levels of complexity.

For example, let's say in primary care you want to summarize a note. Off-the-shelf models with a prompt in primary care can get you so much further than they could 3 years ago. It's incredible—an incredible inflection point. But if you want an interactive agent that can safely speak with a patient and collect intake in a safety-critical environment, that still requires a level of craftsmanship, guardrails, and complexity that looks as much or more complex than what I was describing 3 years ago.

I think it's very use-case-dependent. Something simple, like predicting ICD-10 codes based on what was discussed, has shifted in complexity from modeling architectures and tweaking parameters in your loss to asking, “What is the right way to formulate this task? What is the right grader? What are the semantics?” How can we actually annotate a dataset that's high enough quality to model subjectivity, and then deal with all the repercussions of reward hacking?

Complexity is still there; it's evolving. But it really depends on the specific task.

I want to hear more about the patient-facing stuff as well, but let's go down the coding rabbit hole for a minute.

Maybe, Ben, do you want to tee this up again from the provider side? I was also really amazed to see the baseline numbers, and it's a good reminder—I'll let you tell it—but it's a good reminder that the human baseline doesn't mean perfect. That's another common refrain. I sort of like how Biden used to say, “Don't compare me to the Almighty; compare me to the alternative.” That's also apt for AI systems.

Here, obviously, we've got highly educated people, top of the heap in society in terms of their accomplishments, but the success rate on some of these coding tasks is probably not as high as people would think. So, tell us more about the challenges of coding in the world.

Coding is part of larger revenue cycle management in health care, which is actually fairly complex. And back to your question about what changed that caused so many administrative roles in health care, I would imagine that's part of it. It starts early with things like getting preauthorization, then moves to the mid-cycle, which is this kind of coding. After that, once the coding is done, you submit it for reimbursement. In a lot of cases, they may just reject it, and then you have denial processing. Each step is actually human and manual.

There are people calling from payer organizations, from hospitals to insurance companies, asking for preauthorization. Between you and me, a lot of this happens over the phone. Health care has not really adopted technology the way we think it should have.

Coding is basically the process of classifying diseases into a standard set of codes based on a taxonomy. That taxonomy is adopted internationally, I think. Right now, I think the latest is ICD-11, but most organizations use ICD-10. We're talking about in excess of 70,000 different codes.

Every disease and condition is coded based on the evaluation that the doctor does. Then there's another set of codes that classify procedures. They combine these things and say, “Okay, that was a procedure done. That was a diagnosis that was made.” That gets combined, and then it's sent to payers—to insurance companies or to Medicare as well.

This process is fairly technical. At the clinic, we spend in excess of $50 million just doing the coding ourselves. It's partially automated. For part of it, professional coding is typically done by doctors. At the time of care, the doctor may decide and say, “Hey, this is the diagnosis that I made, and that's the code for it.”

But then it's passed through coding people who review the content and the notes that are created either by a system like Ambience and obviously reviewed and approved by the doctor. Based on their knowledge of all these codes, they try to figure out the right comprehensive code to assign to that condition or diagnosis.

Often, they also need to look at the patient's history because comorbidities may have some effect on the documentation. This process may go back and forth. The coders may decide that there's some ambiguity and have to ask the doctor for clarification, which adds to the administrative work that doctors have to do.

For doctors, this isn't really part of what they're meant to do, what they went to school to do, or what they were passionate about. They diagnose a patient and come up with a treatment plan, but now they also have to be aware of what code needs to be assigned to a particular diagnosis for a patient with a particular kind of comorbidity. That's obviously time-consuming.

It's an excellent application for AI and generative AI. It screams for an AI agent to consume the documentation about what these codes mean, consume the documents and notes, look at the patient's history, and then say, “Hey, this is the right thing to do.” The more you can do it at the point of care, the more doctors can think about which option is right in front of them and choose the right one.

Just to describe this process, I think this is also really instructive for people as they think about it. Our number one audience profile, as far as I understand, is the AI engineer. Often, the challenge that the AI engineer has is, “I'm trying to automate some business process.” First, they have to map out the process. I always emphasize the importance of mapping out the process as it exists and really deeply understanding it. That's critical to automating it effectively.

Anybody can slap together a workflow, but that doesn't mean it's actually going to do what the original process was that they were trying to automate. So, to take an extra beat on understanding the process in the pre-AI scenario, tell me what I'm getting wrong here.

The doctor has the visit. They may record audio if they want to revisit it; maybe they don't. After that, either from memory or from whatever they were able to capture, they sit down and write notes. Presumably, their notes are relatively brief by comparison to what the AI is going to begin to output for us in the future. Then they have to code both the diagnosis and the services rendered after the fact into this taxonomy.

Are they doing that through some sort of type-ahead search? They'll be like—I have kids, so the things that we get are like pink eye.

You just start typing “pink eye,” and it pops up with a numeric code. Then, I guess, would there be a similar thing for prescribing drops? “I prescribe drops,” and that pops up with a code, and I click it in.

Then that gets sent off to a boiler room of coders in the basement of the hospital who are receiving the doctors’ notes, looking at these codes, and trying to reconcile them, spot errors, or generally improve things before it gets shipped off to the insurance company, which has people again doing the same thing. Sometimes they say, “Okay, we’ll pay that.” Other times they say, “No, that doesn’t sound right. Back to you to deal with it.” Then, of course, it can get sent back and whatever. How am I doing on the “before,” and what am I missing?

That’s generally the case. I’ve actually sat down myself, and it probably takes a long time to go through the whole process. What happens is that there are professional coders, and they get trained in ICD codes, for instance. Starting from the documentation that the doctors do, they finalize those codes.

Then, if there are procedures, those codes have to be combined together, and they group them together. Those groups are then sent to insurance companies for payment. If it gets denied or if they have questions, that whole denial process is another set of people who have to manage it, and often they may need to pull the doctor back in to provide enough evidence and documentation to justify it.

It’s a fairly manual process, and it does require domain knowledge about what these codes are. There are tools that facilitate it, like you said, searching through it and all that, but nothing that really solves the problem. There are companies getting into that area or something similar now, doing the coding after the documentation processing that doctors do.

It’s an area that you would think you could train because you said, “What is the ground truth? How do you know if that’s the best or not?” It’s hard to say, obviously, but there is a history of documents and associated codes. We have a large number of documents showing what was challenged, what was changed, and what was finally approved. That would give us a baseline of what a human can do.

Now, can I do a better job? One way we can measure that is to have the AI system do it and then have multiple coders look to see whether the AI system’s coding was comprehensive and correct, and compare that to what the humans had done. Often, we may find that there are things that are missing, that the coder or the doctor forgot or didn’t consider, and the system found.

The opposite, by the way, is also bad because upcoding has penalties. We definitely don’t want the system to put wrong or additional diagnoses or procedures that are not factual, because that has a lot of legal consequences.

Okay. So, in reading the case study, the baseline human doctor correctness rate for the OpenAI website was 45%, which was shockingly low to me. Again, I’m coming to this naively, so what do I know? Could we be a little more concrete about what exactly that means?

Is that 45% of the time what the doctor said, the coder didn’t have to change, it was approved by insurance, and then we’re good to go? If that’s right, then how much can the coders move the needle? Do they take that 45% up to 90%, or are we living in a world where more than 1 in 10 things sent to the insurance company are coming back?

No, it’s a great question. I was reading that we waste $20 billion each year as a country just on codes that are not substantiated or are incorrect, specifically ICD-10 codes. Maybe it’s helpful if I start with what Ambience does and show you why this might be challenging and nonintuitive for a clinician.

As mentioned, after the visit, after they’re done talking with the patient—or even sometimes during the visit—they fill out these ICD-10 codes. There’s a search, and they typically use a search engine powered by something called IMO, Intelligent Medical Objects. They’ll type in things like “left ear pain,” and then this big list of codes will come up: H65.195, other acute nonsuppurative otitis media, recurrent, left ear.

You have to get that right, but there are a lot of others, and they all look very similar. It’s very easy for them to make mistakes. Ambience doesn’t do any diagnosing ourselves. We’re just extracting what was discussed in a visit and normalizing that onto the standard set of codes for clinicians.

Doctors know what the diagnosis is, but the gap from that to 70,000 codes that are evolving twice a year is where things break down. We don’t want to say doctors are bad coders; it’s just a nonintuitive task. I think that’s the frame for what we’re actually trying to do.

Yeah, I think we used F1 score, but that’s where the 45% number came from. The first question every time you’re doing fine-tuning or RFT is, “What is actually the maximum that you could possibly get?” It’s not 100%. It’s almost never that.

The reason is that this task has some level of subjectivity. The only way to deal with that is to model that subjectivity into the process and harness it. One way to do that is with a gold panel.

What we did was first establish this baseline. For this task, we look at a transcript and map it to the correct codes. We gave people a search engine in a very similar environment to real life. This included clinicians and auditors from AAPC.

We were able to get a gold panel that showed us what was actually maxed out. Basically, we had 2 gold panels, and we calculated the inter-annotator agreement to see what was actually maxed out. We maxed out around 80% to 85% F1 in inter-annotator agreement. That’s our ceiling.

The models were at around 35% F1, and clinicians were about 45%. There’s a lot of work to do and a lot of progress we can make with RFT. That’s how we got there.

So, is that 85% probably still aspirational, even coming out of the pre-AI workflow? Even after going through the whole coder process, after the doctor does the initial coding, do you know what that number looks like compared with the theoretical maximum?

Here’s the trick: nobody knows. Some of this is actually hard to even know, because Ambience has a very unique artifact. It’s the transcript of what was discussed during the visit.

These RCM teams, coders, and insurance companies are operating on a super-lossy artifact, which is the note. As you know, clinicians don’t love taking notes, and the quality of the notes means they’ve lost all this rich information. There’s this gap in reality.

I think we’re only discovering as a community what good looks like and what is possible to extract from this transcript artifact. It’s hard to know, but we do know there’s a gap there.

Yeah. Okay, that makes sense. I think that’s a perfect setup for the fine-tuning process that not only closed the gap to the human doctor but actually took the AI above it and takes us to a whole brave new world.

I want to learn everything about this. What does the data look like? Do you have to collect chain-of-thought from people? How much are we trying to get a human reasoning prior versus the new paradigm I’m hearing more and more about, which is just the right answer plus a scoring rubric that gets used in reinforcement learning? Maybe in some cases you want to do both, I suppose.

Obviously, we know frontier models have a mix of human SFT data sets and RL combined. How much data do we need? What kind of data do we need? What tricks work in terms of graders? I want to learn as much as I possibly can from your hard work.

Sure. I think maybe it starts with just saying what technique we used. We used a tool from OpenAI that they call RFT, or reinforcement fine-tuning. It’s a proprietary technique. However, there are some parallels in the open-source community, which we could talk about with DeepSeek as well.

We used this API. Our interface for this project is a data set and a grader. I’m sure the folks here are pretty familiar with RFT, but the basic gist is that this is the technique used to get these extremely capable reasoning models. Gemini, Claude, and o3 all use this technique in one of the steps of the post-training process.

Now OpenAI has made it available to all developers who can give this a go. I think it’s very powerful and very novel for a few reasons. In the past, we’ve been doing supervised fine-tuning our entire careers.

You collect this big data set of input and output pairs, and you’re trying to get the model to mimic the outputs. But with RFT, they flip it.

Instead of having this annotation, you can give it a grader—a programmable grader that gives you a reward. You can use any different type of grader. There are a lot of different kinds. You could use a string match, a unit test, or even a language model to grade. You can even use an ensemble of multiple techniques. It's extremely powerful. It allows you to guide the model in very subtle ways toward what you want.

It also allows you, interestingly, to optimize for the end objective. In machine learning, you can actually optimize a model for the end result you want. That's a good place to be. Oftentimes, we're just optimizing for proxy metrics, like loss or F1, but you can actually optimize for the real-world objective.

I think the last thing is that this technique, using reinforcement learning, is extremely sample-efficient. During the training process, the model generates multiple candidate answers—I don't know, 4 or 64. The grader scores the answers, but the model takes 1 example and balloons it through the sampling process into 64 examples. So, in theory, your dataset is getting a lot more signal from every example. You can actually get state-of-the-art results on hundreds or low thousands of examples, where in the past it may have taken 10,000 to get something similar with SFT. So that's the technique. It is a proprietary OpenAI technique, but that's what we're building on.

One huge spectrum with this reinforcement learning is how verifiable the answer is. I think everybody has heard plenty of talk at this point about math and coding. On the far end would be, I don't know, poetry or fiction writing or something.

The more verifiable it is, the simpler your grader can be, right? In theory, we've seen things like with R1, just a straight-up binary signal of whether you got it right or got it wrong can work as well. If you're evaluating poetry or whatever, you have no binary signal, so then you have to start getting into reward design.

So, are you guys on that spectrum? There is a sort of code answer where you could have this sort of F1-like exact-match yes-or-no, but I'm getting the sense that there's more to it than that. It sits between, like, here's the raw transcript of the visit and here's the exact code: did you exact-match or not? What does that answer look like in terms of what more is there, and how does that then feed into your reward design challenge?

It depends on the task. We can talk about ICD-10. It can be different, and we've explored different tasks here, but for ICD-10, it's pretty simple. We have this dataset of transcripts in and these codes out. It's multiple codes for every visit, and we were able to get gains just using string matching.

Ours was fairly vanilla string matching. We were optimizing the F1 score directly, trying to find some balance of precision and recall on the actual codes, which has really interesting downstream impact on the revenue cycle process, but we were trying to model that. This is nice because string matching is actually fairly clean. It's less hackable, which is always a good place to be. It's cheaper to run.

Because there are all these different codes, the model naturally gets partial credit. It gets 1 right, but it doesn't get this other one right, and that actually gives it a richer reward that can lead to learning faster. So, in ICD-10, it was actually fairly simple. We did iterate and try some things, but we got away with string matching.

For any given visit, there's a variable number of codes. How do you think about it if there are, let's say, 5 ground-truth codes? You can have both false positives and false negatives in the AI output, right? Do you treat those differently in your reward process?

Let's say there were 5 ground truths and I got 4 right, missed 1, and then had 1 that wasn't actually there. How do you give me credit for 4? Are those other 2 both minus 1? Is there a need to do something different between the false positives and false negatives?

Yeah, it's a really good question. I think in this particular case, we just did a very naive precision calculation and recall calculation and merged them into 1. We actually optimized that one directly. So it wasn't as clever as we thought it might need to be.

But I think there's actually a really interesting opportunity to extend that. For instance, the model gets a code close; it's just missing potentially 1 subcomponent. Could we use a semantic grader? Could we fuse those together? That's actually a really interesting direction to explore. For this experiment, we didn't have to go there.

So you've not stopped climbing the hill as of the case study on the OpenAI website. The hill is steep. There's a lot of room to grow. It's really exciting.

Maybe unpack a little more, either for that task or for other related tasks, what more you do to climb the hill. Semantic matching, as opposed to just string matching, obviously has a lot of flavors. I don't know how much you would—or how soon you would—have to start worrying about reward hacking. I worry about reward hacking big-time in the big picture of possible superintelligence.

Did you need to take that into account, or did you run into any weird stuff in any of your experiments in this narrower domain?

Yeah, it's a good question. We definitely have some learnings that we got out of ICD-10. I actually think it's an interesting time to introduce the physical exam, because this is where things got really weird.

Whenever you're moving away from string matching to an LLM-based grader, that's where you really have to be careful. You're moving into, I'd say, very dangerous, very hackable territory.

This was initially a learning experiment for us. Could we take RFT and apply it not just to something that was a little bit more objective and string-match, but to a more open and generative task?

For the listeners, when you're in the doctor's office, they check you out. They listen to your heart and your lungs, and they look in your mouth. Afterwards, they document that in the note section. It's typically a structured section, and for each body system, like respiratory, they have to write some prose. It can be sentence fragments—keywords like “respiratory, normal effort, no audible wheezing, and air entry.”

The reason we chose this is that it's not fully open-ended. There is a structure. It is a JSON file, and these sentences are short and fragmented enough that maybe we could do something with just a basic regular expression or semantic grader. But this is where we started to move into actual model-based graders based on rubrics.

We started training the models, and immediately, they started hacking all over the place. The 2 things we found were—again, we were optimizing for F1 score—the models started to gain precision by inflating the number of findings. They started to output the same finding with a different way to describe it. So they started getting all these points for being right, but it was all the same. We fixed that. We penalized it based on redundancy, and we got that going.

Then all of a sudden, it was scoring really highly, but we started to look at the outputs, and there was this tone degeneration. It was no longer using clinical or professional terminology. It was saying things like, “Grandpa’s heart sounds good,” instead of “normal heart sounds.”

This is where the models go, right? We looked again at our rubric and said, “Oh, we’re just checking for clinical quality here. We’re not actually checking for style.” So again, you add that into your grader.

I think we ended up with a weighting here: the rubric was 75% focused on clinical quality and 25% focused on clinical style. Make sure it has good language. Eventually, we were able to harness it and squeeze it out, and we got some good results. Clinicians preferred this version.

But that was just our first foray into a little bit more open-ended work, and that’s where things get a little dicey.

How do you think about this other part of AI automation, broadly? How do you think about choosing what tasks to focus on? You’ve obviously got a mix of what’s feasible, where there’s demand, how big the opportunity is, and the risks if something does go wrong in a particular task.

That’s hard enough for many mainstream businesses. It sounds like a pretty fraught problem in the medical setting. You may also have the trust factor and the openness to this task being addressed in this way, even if everything else checks out.

So, who’s around that table? How do you decide what to tackle and in what order?

Yeah, this is something Ben and I probably think a lot about. Maybe, Ben, you can speak to that.

Yeah, I can tell you how we think about our priorities. Those are the priorities where we think technology, and artificial intelligence in particular, can play a role. I would say we can probably categorize them in 3 areas. One is patient outcomes and overall patient experience.

So, that’s kind of our DNA, right? Anything that can be used to improve patient outcomes, whether that has to do with diagnosis, treatment plans, or any of those things. I think there’s a tremendous amount of opportunity in health care. The ultimate goal of precision, personalized medicine is now within reach because we have a ton of data in the EHR system—in the electronic health record system—that was never used. It’s not available to commercial organizations to train large language models, all right? So if you bring AI in, I think we can do a lot more personalized treatment and diagnosis.

Nevertheless, that’s one area: anything that has to do with clinical outcomes as well as patient experience. For instance, patients wanting to find their way around: What hospital should they go to? Should they go to the emergency room, urgent care, or just make an appointment with their doctor? The second area is caregiver experience. And that’s again because of the shortage, burden, and burnout of caregivers. So anything that we can do to make their work easier and better, and remove all of the nonclinical aspects of the work, I think that’s an opportunity for us.

And the third is overall cost and efficiency. How do we scale our operations better? How do we utilize our resources better? How do we reduce the cost of doing this nonclinical stuff, including things like coding and optimizing our operating rooms in hospitals? All of those are areas that also align with the main stakeholders that we have, which are patients, caregivers, and organizations.

Now, within each of these areas, when you have different use cases, obviously we look at them through the lens of risk, technology maturity, and ROI. We need to have a line of sight, right? We also try to reduce the risk by doing pilots, like what we did in the case of Ambience. We actually ran a pilot of 5 months or more with different providers. We wanted to make sure that the technology was there, that the product-market fit was there, and that we got the gain and value that we expected.

Then we gradually rolled it out. In this case, with our collaboration, it made sense for us to do a phased rollout based on specialties. In other cases, we may decide to start with one of the hospitals, see how it works, and then roll it out to the next hospital and so on.

So how has that reception been? Are you in an environment where people are saying, “Hey, my buddy in the next specialty over has this”? How is it?

Look, this has been—our chief clinical officer keeps calling it a magical experience. In fact, at the end of the pilot, it was so successful that I think some of the doctors were threatening that they would leave if we took it away from them. It just worked out. It’s not very common, by the way, particularly in health care. Finding a product that has that perfect product-market fit is hard, and this one is really a shining example.

Now, I don’t know if it’s going to be an exception. We’ll find out whether other things are not that easy. In my mind, I think of it as a productivity tool. In other industries, I have experienced productivity tools that end up getting adoption, and some productivity tools don’t get adoption.

The way I think about it is that if you’re building something that’s supposed to improve your productivity and you want adoption, the user has some sort of mental model in mind as to how difficult it would be to perform a task, all right? Then they also need to make a mental model as to, if they use this tool, how much more efficiently they would be able to do that task. So that mental model needs to be formed before people actually start using it and keep using it.

A lot of times in AI systems, vendors get that math wrong, in the sense that you can’t only think about the cases where it works. You have to think about the cases—the corner cases—where it doesn’t work and figure out how that impacts the overall equation, because it’s an expectation. It’s the math, right?

Let’s just segment it into 2 scenarios. There’s the expected effort if the product works with no error. Now, you have to multiply that by the probability that it works without any error. Then you have a second piece, which is the probability that it doesn’t work as expected and the effort in the presence of that error.

You may have cases where the error doesn’t happen as often, but the effort to recover from it is so great that your mental math still won’t work, because your expected amount of effort is actually more than not using it.

I’ll give you an example. A long time ago, I was at Yahoo, and we were trying to add voice search to the search box. It didn’t really take off. I think the issue was that the user’s mental model was: If I click on the microphone, is that going to be a faster way for me to get to my search result? At that point, you have to think: How often is there going to be an error? Even if it’s 10% or 20%, what is the effort in the presence of that error? Do you have to go back and retype it and all that kind of stuff? It just didn’t work.

Particularly in cases where it’s hands-off—in other words, for instance, when you’re driving. That’s the case where we thought voice would be a better interface. But guess what? That’s actually the context where there’s typically more noise in the background. The microphone is not near you because you’re in the car, and therefore the error is actually higher.

Now, when it comes to AI Scribe, I shadowed a few of our doctors, and it was really interesting. I went into a visit with a patient for 2 or 3 of these. This is a 15-to-20-minute conversation. In fact, in one case there were 2 people—an elderly couple who came together. They’re chitchatting, they’re talking, all that kind of stuff.

Then I walked with the doctor back to his office. By the time he got there, the clinical note was already generated. He went through it and did a little bit of editing—maybe 1 or 2 items that were actually errors of omission. He was like, “Okay, I would add this or I would add that.” He told me, “I would probably save 10 to 15 minutes.” And that’s when the product didn’t perfectly work. If it had worked perfectly, then he wouldn’t even have to do the edit.

So that means that, if you try it, your mental model is, “Hey, using this thing is actually going to make me more productive.” As part of the rollout plan, one of the strategies that our team came up with, which I think was really clever, was: We’re not going to mandate our doctors to necessarily use this. But as part of onboarding, they are mandated to go through training. They watch a video and have some training material, and they are mandated to try it in at least 1 encounter.

I think the team thought that even 1 encounter would create that mental model for the doctors, who would say, “Ah, actually, yes, I think this thing will help me. It’s a good productivity tool.”

We haven’t seen that yet in another application, which is automated responses to emails. Clinically, doctors also receive hundreds of emails a day: some from patients, some from other doctors, some from labs, all that kind of stuff. So they’re bombarded. There are products out there that try to automatically draft a response to them. Adoption rates are still in the single digits.

Actually, that’s a use case that we might work on together with Ambience. We’ve been talking about how to improve it. But the problem statement is hard there because, when we talk to our doctors about why they don’t use this, they say, “Well, I first have to click and look at the response that the system generated. I have to read it. Often, it needs some editing. I have to go back and edit it and all that.” By the end of that, their mental model is, “I might as well do it myself,” because the entire thing typically is a few seconds, maybe 30 seconds. It’s not a 20-minute thing, right?

There are very many of these 10-to-20-second things, and if you can’t create that mental model that your expected effort is reduced, you’re not going to get adoption. We’re getting a ton of adoption on AI Scribe, and that’s why I think it was just the perfect match in terms of product-market fit.

I do want to give Cleveland Clinic some credit. We’ve been working with them now for about a year. It’s an exceptional organization. It takes not just a product like Ambience that does good AI; it actually takes a lot of organizational work to operationalize and scale this technology.

We were blown away by what we saw. We were able to onboard, I think, around 4,000 monthly active users from zero in 90 days or something, across 60 specialties and 7 languages. Utilization is around 75%. That’s the percentage of visits in which they use Ambience. And I do want to take credit, but I also think it’s incredibly important for your health system partners, including Cleveland Clinic, which has been truly exceptional. Remember, we work with 40 organizations, so we’ve seen what excellence looks like.

Yeah, I think technology is a part of it. Workflow integration and change management are huge. The last mile is probably the longest one. I think your mental-model commentary there is really insightful and important for a lot of people to understand, and it probably helps me understand a little better why so many people are still currently in the “I tried ChatGPT when it first came out and I didn’t think it was that cool, so I haven’t been back since” kind of state.

I wonder if you have any other tricks. I think “try one” is a really interesting one. A lot of organizations could generalize that. It’s not too much of an ask to say, “You have to try this one time.”

Maybe another one would be, “You have to try one once a quarter,” because things keep getting better, and you don’t want to let your first impression outlive its usefulness. I would love to hear more about what the right level of mandate is, the right level of expectation-setting, management, culture-setting—you name it—that helps drive adoption, because I think that is a huge problem in a lot of places.

Yeah. I think maybe one of the differences between a place like Cleveland Clinic and some of the other places that aren’t necessarily academic centers or as forward-looking is that we have doctors who are very passionate and willing to try things. Not everyone, but that’s an advantage, because I don’t think there are very many off-the-shelf solutions that you can buy and just put right into use. It needs a lot of iteration. You need to work with vendors and technology partners.

We provided a lot of feedback during the pilot from these doctors, and that feedback went right back into Ambience for product improvement until we got it to a point where they were satisfied that the metrics from the pilot showed that this thing was working. Then we did the rollout. There are industries where things can maybe be bought off the shelf more easily, but in health care, it’s just much harder.

One thing that I think we maybe gave a little short shrift—because this was a standout for me in the outline—was the war story of how we accidentally burned a bunch of money on the o1 grader at some point. This goes back to the physical exam, but I forgot to cue you on this. Do you want to tell the story of how we burned a bunch of money in the learning process of RFT?

Yeah, I think for builders, especially folks building on OpenAI, this gets into the cost issue. For SFT on their platform, you’ll have a few thousand examples; the job takes maybe a few hours and costs you $100. RFT, maybe you have a few hundred examples; the job could take 1–2 days and cost you thousands of dollars out of the box. It also depends on what grader mechanism you’re using. String matching is obviously going to be tremendously cheaper.

We screwed up. We ended up using o1 as a grader. We ran a really small experiment—about 100 examples—and quickly burned $25,000 on the grader alone. So just watch out for that. Whenever you can, for a number of reasons, go with something very simple, like a straight match or even a unit test. That’s a great direction and probably means it’s better formulated for RFT anyway.

So how do you recommend people work up to that? If the number of data points isn’t that high, and presumably you do want the best grader you can get—especially if you’re in any sort of non-verifiable domain—I would think, right? So if that was o1 at the time, maybe it’s o3 now, is there any way to validate your techniques before you do the $25,000? It doesn’t sound like that was a misconfiguration; it just sounds like that’s what it cost.

No, I think this is great. The OpenAI folks will recommend the same, but there’s a really good way to approach this. It starts with evals, of course. When you jump into RFT, ideally you have a great eval that’s representative of the full distribution, and that’s where you iterate on your grader. You see: is this correlated with human judgment? Is this actually doing what we want?

The side benefit is that you actually see what the hill is to climb. You can see, “The models are only getting this right 35% of the time.” That’s where you see there’s actually a really meaningful hill. You have a grader that’s well correlated with human judgment. Then, absolutely, we have more confidence that we can actually use it without reward hacking in an RFT loop.

Yeah. It’s very interesting. What’s going to happen to the coders?

I think within health care, there are obviously so many different job functions, and yes, I think some of the jobs may change, and those people may work on different types of things. Or maybe they’ll be mostly reviewing the work of the AI systems that are coming back and making sure that the final verification and decision are made by a human. But yes, I think some areas obviously are going to be impacted more than others.

Yeah, and I think this really gets into the question that we’re all asking across all different industries: do you have super people or all AIs? Are they going to take our jobs? I think there’s some early indication, in spaces like coding, that that actually might be true, and it may be happening now. I think in health care it’s a little different for a few reasons. One is that I don’t think the demand for health care is going to stay fixed. I actually think it’s going to grow with the population.

We have about 10,000 people entering Medicare every day. The patients are sicker and sicker, but new technologies are being invented that people want, right? I actually think we’re going to have rapidly increasing demand as the agents themselves improve what we can do with pharmaceuticals and surgeries. So it’s going to be more, not less, and I think we already have a 125,000-physician shortage in the next 10 years. There are going to be plenty of jobs, I think, across coding, administration, and clinical care.

I totally believe the demand will grow, no doubt about that. I don’t anticipate too much slack in the system anytime soon, especially when it comes to actual physical treatments, procedures, and things that are touching human bodies. I think we’ll have plenty of people who want care. Honestly, one of the great hopes I have for the whole AI revolution is that we’ll cure all the diseases and live longer than past generations have lived, and so on. I’m all for that.

On the coding side, it does seem like, boy, if I was sitting there doing coding all day and I saw this result, I would be feeling maybe like I would be feeling if I was driving a taxi or driving a truck and seeing that the Waymos are starting to get pretty good. Even to the point where, from reports I’m hearing from San Francisco these days, Waymos cost more and it’s hard to get them because people just prefer the experience.

Personally, I also think the podcast role is not necessarily going to last very long. NotebookLM can take any topic and immediately kick out a custom podcast that’s also interruptible and interactive. I have a real hard time figuring out how I’m going to match that. When I explore this domain, it’s not with any delusions about how long I’ll be doing what I’m doing. But it does seem like that’s an area where, if you’re going to get this right, you need AGI, and then there are only odd jobs that are going to go away at some point.

Yeah, I think it’s coming for all of us on some timeline. That’s definitely my default position.

What else are you guys looking at in health care broadly? I mean, you said a while back, we don’t do diagnostics ourselves; that’s the doctor’s job. There has been some interesting research out of no less than Google that has shown, at least if you believe their results as presented—which I’ve always defaulted to believing—that AI systems are outperforming doctors even on diagnosis these days, in certain contexts and with certain caveats.

There’s also this question you alluded to earlier, too, of an unbelievable amount of data that is currently locked up in systems where it hasn’t been super useful. I think there’s kind of a mess of it. Partly, it’s because maybe we didn’t know how to use it; partly, it’s because certain institutions or entities think that it’s in their interest not to share it freely. What do you see as the next big unlocks, whether it’s liberating data or otherwise, that could change the whole landscape in even more profound ways going forward?

Yeah, we were talking about this before the episode. I do have thoughts here. You mentioned Google’s excellent AMIE result on diagnostic reasoning. I think one thing I just want to call out that the industry is seeing is that base models are increasingly capable. That’s so true. But there’s also a difference between the results you get in a research lab and when you actually get that into the messy production world and have to operate at scale, processing millions of patient records every day.

You see things, and not all the data in the EHR is perfectly clean like you might see in an eval set. I think what we’re seeing internally and what we’re hearing about are actually 2 different worlds, but that doesn’t take away from the fact that the community is making tremendous progress. Yes, models are getting better just out of the box.

But I think it's important to call out that what we're seeing will shape future architectures here internally. There is this robustness problem. I think Ben was alluding to it. Oftentimes, when you see an academic benchmark, it's reporting the best of them, right? We tried it 64 times, this is the best result, and that's what they show on their plots.

In healthcare, we are more concerned about the worst of them. In fact, in internal evals, we sample a number of times, take the worst one, and that's what we consider is going to go to users. I think the industry is definitely onto this, especially teams like Karan at OpenAI and the HealthBench folks. This is the right way to think about it.

But in the real world, I think there are still gaps. I actually think this data is out of distribution. It really is. Part of the reason is this walled garden, right? I think 80% of patients are somewhere in Epic's database, in their medical records. For regulatory and patient-privacy reasons, we can't access that; it's not on the internet. It's not in the parameters of the model, right?

The second one I would say is that you can go to medical school, but that's just the beginning of your training as a doctor. Then you do another 4 to 8 years of training in residency. And it's not textbook training. It's hands-on in the clinic, observing and watching doctors. That's where a significant chunk of the medical reasoning and learning actually happens. Again, it's just not on the internet, right?

Then layer on top of these esoteric, evolving billing rulebooks. You have medication names that are changing constantly. There are different ones for different countries. I think this research-to-production gap is something we're actively working on as a company and just in the industry overall.

So tell me about the patient-facing thing. Obviously, that's a set of questions and sort of accessibility requirements. And I don't know, especially as it compares and contrasts to all the stuff we've already talked about, what have the challenges and learnings been about actually doing something that's making contact with the patient?

This is a great question. I think this gets into the agents, too, which is another really interesting topic. Imagine if you had ChatGPT inside Epic with access to all the HL7 FHIR private APIs and, of course, extremely high-quality medical reasoning. What could you do? What does medicine actually look like in that world?

We developed very early iterations of the patient-facing one a year or two ago. It was obviously a great demo, but there's actually something here. If you think about other things that stress doctors out, it's not just taking notes; it's actually answering these in-basket messages. In fact, there's often an entire staff at the hospital—often nurses—who are put on calling patients and answering questions. There's this tremendously inefficient back and forth.

I mean, you've probably been to MyChart and asked a question. That's a lot of time. That's a lot of money. And actually, if you think about it, some of these tasks are very tractable, even for the current generations of models, right? With guardrails.

There's a way to do it safely where, for instance, one of the things we're eyeing is that, after the visit, patients don't comply with their plan. If they don't comply with their plan, they get sick, and then they come back to the hospital, and it's more money for the health system. Why don't we just keep following up with them? It's just too expensive. We just can't do it, right? But with AI, we can.

So after the visit, the ambient product will call the patient. Now, this is very early development. I don't want to say this is at scale, but we'll call the patient and ask them, “Did you get your labs done? Did you go take your medication?” Then we will synthesize that back into a note and put it back into Epic for the clinician or the nurse to look at.

Then they know, “This patient's on track,” or, “This patient's not,” and they can then intervene, right? So we're not doing medical care or medical decision-making, but we're automating an increasingly painful task for the system.

Yeah, that's interesting. And that's a voice modality, right? You're calling and having an interactive conversation?

It could be both, yeah.

Right. Any architectural lessons? I think a lot of people are trying to build voice products right now. I don't think we've really coalesced around what the right scaffolding is for that. I've heard approaches including background agents that sort of monitor in real time and feed in coaching to the main voice agent. What have you experimented with, and what, if anything, stands out as a clear winner?

It's a good question. I think first I'll say that, like other medical domains, when you're going to talk with patients, hospitals are more risk-averse, right? They want to have great reputations and great interactions, and you don't want to accidentally mislead patients. So the safety bar is much higher in this space. I think you have to immediately start to think about guardrails, right?

Even if you get the best performance out of a model that's fully end-to-end, like voice, is that actually the model that you want, right? Or do you want a model that's more interpretable at each stage? This is kind of like—I was in self-driving cars—this is what we think about a lot. Do you want those guardrails baked in at checkpoints during that agent interaction?

So I think off-the-shelf voice-to-voice is not ready. I don't think it's as interpretable as we need. There will be some level of fine-tuning, probably of the underlying model. There'll then be guardrails when we bring it into the ASR transcription step, then the actual language-model context processing, and then the voice synthesis, right?

I think this is still under exploration internally, but my guess is the system that's actually landed looks a little bit more heuristic than probably some of the things you're seeing just on the internet and in consumer apps.

More decomposed, more auditable, more guardrails baked in, absolutely. I know we're right at time. Any closing thoughts?

This was really fun. This was a great conversation. I think there's a tremendous amount of opportunity in healthcare. And I think if there's one thing that I could say, particularly to your audience of people in tech, machine learning, and AI, it's to consider healthcare, because there is an opportunity to make a real impact in an area that all of us can relate to.

We're all patients, and we all know patients. No matter if you work even in areas that are kind of a back-office thing—efficiency, cost—healthcare systems can be funny. Over half of the hospitals in the country are losing money. And places like us, Cleveland Clinic, are nonprofit organizations. So when we talk about reducing the cost, it's not about increasing our stock price or shareholder value. That cost savings directly goes back into trying to save more patients.

The opportunity is great, the technology is there, and it's a way to make an impact and make the world better. I had a lot of fun. You're an incredible host. I really appreciate and love the episodes. Thank you. Please keep it up. Hopefully this was interesting to some folks out there.

I agree with Ben. I've worked in self-driving cars, and I had so much fun analyzing millions of bounding boxes on cars. Healthcare is—I believe I'm even more excited about what's happening here. There's the end result of, “Let's drive the cost of healthcare to 0 in the long term and scale it all over the world.” That's phenomenally cool, right?

But also, it's in this very nascent stage where you have a field that has had no innovation for decades, right? They added all of these buttons and dropdowns and clicking to comply with regulatory requirements. Now you have AI agents that can do those things, or are slowly getting capable enough to do them. It opens up this vast design space, a whole candy store of opportunities, right?

That's why we're rapidly growing into this AI platform. We're rapidly hiring machine-learning engineers, machine-learning researchers, clinical scientists, and people from every single discipline here. Then we're creating these small pods to tackle these use cases, which keep emerging the deeper we go.

I do want to say that this is the time to build in healthcare. It is one of the best product-market fits of this new generation of agents and generative technology. I just want more builders in the space. I really think this is cool.

Yeah. So, it's a call for technology people to cross over into healthcare and for doctors, potentially in some cases, to cross over and be the medical experts for technology companies. May we all live to be 500 and beyond. I used to dream about it and feel kind of crazy for dreaming about it. Now I feel maybe a little less crazy.

Keep up the great work. We're all counting on you, and this has been great. Thank you for taking the time, Brendan Fortuner and Ben Shahshahani. Thank you. You have a great day.