The Credibility of AI, the Future of the EHR, and the Cultural Demands with John Halamka

This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

John Halamka: AI has a credibility problem in healthcare. And what I mean by that is, if you buy a can of soup, on it, it says a thousand milligrams of sodium. 500 grams of fat. 2000 calories a serving. My bet is you wouldn't eat that soup. One hopes. You buy an AI algorithm and there's no soup flavor. Right. You have no idea if it was developed on people like the patient in [00:00:30] front of you or not. And therefore you really don't understand its utility, its bias, its likelihood fulfilling what you need it to do.

In 2022 This Week in Health IT doubled in size. We've grown from two podcast channels to four, introduced guest hosts and planned to bring people together in entirely new ways. But I get it. It's a lot of content to keep up with. So we've adapted for that as well. Last year you heard all about our ClipNotes newsletter. [00:01:30] The episode recap of our Newsday and Keynote shows.

Today we have Dr. John Halamka, President of the Mayo Clinic Platform with us. John, welcome back to the show.

John Halamka: Well, hey, thanks so much. When people ask me, what's the weather in Boston today? And I said, snowy, with a chance of Omicron.

John Halamka: So of course this is complicated and it's quite early, so one doesn't want to draw conclusions, but what I know about vaccine durability, Which is that the Madonna, Johnson and Pfizer vaccines have a durability of between 120 and 150 days. Now, when I say durability, I'm not referring to their prevention of serious disease or hospitalization, I'm saying durability against simple infection. [00:03:00] So just think for a moment, if in fact it was eight months ago that you got your last shot, you know, by now you're starting to see the waning of the protective antibodies and therefore it is likely you're going to see an infections spike as folks are that tweener period between second and third shot. So Omicron again it's just too early to know.

John Halamka: Indeed I was. So you remember the best part of that conference is not the speakers. It's all of the events around the conference.

And also just talking about the EHR market. And later on in the show, I wouldn't mind talking to you about the EHR market cause you're one of the few people that is actually written code. Coded an EHR. I think what I want to do is talk to you about the EHR as a platform. [00:04:30] But I don't really want to start there.

I want to start with all right. So you're president of the Mayo platform. We've already covered that on the show. If people are wondering, Hey, what's John doing these days they can listen to the other show. You could give a brief thing on that. Are you still practicing medicine or just predominantly the Mayo platform at this point?

I see about 900 a year. I do toxicology consults across the country. I happen to have a particular expertise in poisonous mushrooms and plants. And so I am doing that virtually, which really was much easier during a COVID waiver and regulatory roll back here. And then I teach about 200 hours a year and I collate a [00:05:30] number of research projects in addition to the administrative duties. So welcome to life as an academic physician.

Bill Russell: You get to do it all. We are going to talk a fair amount about AI. You spoke at HIMSS last year. And you talked about the promise is bright for AI. And you talked about that there's challenges that remain for the adoption of AI. Let's start with that. Give us a little background on some of the things you talked about at the HIMSS conference.

My bet is you wouldn't eat that soup. One hopes. You buy an AI algorithm and there's [00:06:30] no soup flavor. Right. You have no idea if it was developed on people like the patient in front of you or not. And therefore you really don't understand its utility, its bias, its likelihood fulfilling what you need it to do.

But you need more than that. And that second thing, what I would say we need is test ability. So the first thing is transparency. How was it developed and where is it useful testability. So you know what, Hey Bill, I'm gonna run this algorithm against and you know when it says you're likely to wear a red shirt today.

So, you know that algorithms are mostly probabilistic, multi-tiered mathematical equations that don't necessarily have easy explainability. They're a black box. [00:08:00] So, as I debate with those in the industry, they say be transparent and be testable. Explainability, we may not get to.

Bill Russell: All right. So let's break that down. Well, let's start with this. Who's developing the algorithms today. Is it predominantly academic medical centers? Is it third parties? Is it big tech? I mean who is developing these clinical algorithms within our AI models?

You're seeing big tech create some algorithms. You've seen the work coming out of Google and Verily on [00:09:00] various aspects of say image analysis on retinas. Predicting diabetic retinopathy. So we're seeing all of this what I call government academia and industry stakeholders produce these things, but yet we don't really have a set of national guidelines and guardrails as to how these things move from bench to bedside.

Well, it's not that, but Eric just wrote a book with Henry Kissinger called the age of AI, where he reflects on the fact that as we deploy these things, we better have some guidance. And so I think as we see all these [00:10:00] wonderful developers coming up with good stuff, let's wrap it with some guidance.

Bill Russell: The data's really interesting to me in that we can't just pick up an algorithm from Mayo and drop it in Irvine, California, but still we're trying to develop these algorithms that are going to work. And it really depends on the dataset doesn't it? I mean, I was talking to a physician founder of an organization.

And now it's very, it's very basic, but it's a perfect set of data. Whereas when we start talking about that EHR data, we have to take into account the variability in the geographies of those communities. For instance, Irvine, California will be about a 20% [00:11:00] Hispanic, about 15 to 18% Asian populations and various Asian populations, if you really broke it down further. Whereas Mayo population might be Florida if I thought about it.

John Halamka: Yeah. So let me describe three problems with our EHR data sets. So one would be just the fidelity of the data, and that is write a hundreds of data elements you don't know in what workflow, by whom at each institution they're recorded.

Maybe it's not so wonderful. Well then let's also continue on the concept of race, ethnicity. One of the problems you have is the granularity of that data element. It doesn't really break down between, well, you're Asian. Well, are you Japanese? You're Chinese. You're Korean. You're you know my wife is Korean, right.

Well, the challenges were [00:13:00] creating huge amounts of noise in the signal of the EHR data. We're recording through the use of sort of scripts and boiler plate and templates and that kind of thing. And then that's really hard when you're dealing with AI algorithms to separate the signal from the noise.

So you need a much broader coalition of people, but we'll get to that in a minute. What is the promise? What's the promise of AI in healthcare? Why are we pursuing this?

And so one night I was called and I was told, do you know, there's a 24 year old woman running uncloTHed through a Walmart parking lot at three in the morning. And the emergency physician discovered cannabis in the tox screen. Now okay. Every one of us as physicians practices by anecdote and intuition.[00:14:30]

And when I asked her about the cannabis, she said, oh, well, I had the photophobia, the stiff neck and the headache. And then I asked my roommate, if there was anything I could take and the [00:15:30]cannabis had nothing to do with the altered mental status and AI. Would have provided the clinician with augmented decision-making to understand likely diagnoses and likely treatments.

And so it's, you might be able to develop, it might be as limited as you can develop. At Mayo, because you've captured this kind of data in this kind of way, and you structured it and you have the data governance around it to make sure that you are doing and the training for the physicians to capture it in a certain way. But how, I mean, how scalable is that going to be?

And it may take into account certain laboratory tests that are esoteric, certain genomic markers, ideally, right. We'll get to a point where the algorithms say I need these inputs and it becomes a standard of care to gather that kind of information. [00:17:00] But for the moment, it's just, as William Gibson told us, the future is already here, it's just unevenly distributed. And so you're right. That not everyone is going to be able to have all these input variables for now.

Bill Russell: You know, John, I love the example of the simple camera in the room. And this has almost become a joke. Hey, the Fitbit, the Apple watch. But you're probably tracking this stuff.

John Halamka: So as I say, it's a William Gibson problem. So if you ask us say, well, so, Hey, how does Mayo clinic feel about gathering one lead ECG is applying algorithms to them returning a result and improving patient care. Oh, that was a last year problem. Right? And so we have 14 algorithms in the market of which we have predictive A fib. We have [00:18:00] objection fraction with an AUC of 0.92.

For example. So, I think you're gonna look for 2022. I mean, it's just next year, within the next 12 months, you're gonna see many, many more consumer grade devices incorporating these AI algorithms to scale.

John Halamka: So am I allowed to say that it evolved as a transactional system and it has aspirations to be a platform? And the reason I say aspirations is because as you look at FHIR, CDS hooks. You look at various aspects of the interoperability and information blocking rule. You're starting to see the idea that you can have this thing that is [00:19:30] at the core, that has an ecosystem of partners contributing data and using data for novel purposes. It's not there yet, but it's getting there.

Bill Russell: So that's being driven predominantly by policy. 21st Century Cures and some financial penalties and those Catholics. So it's being driven by policy. It feels to me like it's going to be enough. It's still moving a little slow, but it still feels to me like it's going to be enough.

John Halamka: So, yes. But I describe the perfect storm for innovation as an alignment of technology, policy and culture. Right? So the fact is we needed FHIR right?

This was the first incarnation. And they created actually a really nice platform for patients to record their data. And how many people used it? No one. Because it too much effort to get the data in. No value in the [00:21:00] ecosystem. So I think we are approaching a point where, oh, there are enough apps, algorithms maybe even your insurer will offer you a discount or special benefits if there are certain kinds of data you contribute. I'm thinking about what Aetna has done with Apple in the Attain program. When that cultural change occurs, that's really, what's going to get us adoption.

I said, all right, let's, let's start. And so I downloaded their app right there in their booth. It was so simple. It pulled up like 180 different providers. You have to go through there, find your provider. Then you have to remember your login to their portal to actually authenticate that you're you to that health system.

And so for every different specialist and somebody in that medical group that you were to visit, they weren't necessarily on the same system. So you had to do that four, five or six times to get that. It's still not what I would call, what's the word we use. It's it still has a [00:22:30] fair amount of friction to pull that entire medical record together. Is there something on the, on the horizon that's going to reduce that friction? Cause I mean that friction almost makes it not even worth my time.

And does do the linkage to whatever your login and identity is at the provider organization. So here we have a free app that's fully integrating all of my [00:23:30] data and it makes it portable. It's all good. But do we have a canonical identity management program in this country? A mechanism of saying I've linked my biometric fingertip or eye or face to an identity and therefore it's going to go out and fetch all the stuff about me?

Bill Russell: Yeah. Get getting back to the EHR. What do you think the future of the EHR is? I mean, do you think it just sort of goes into the background, acts as a repository, does its transactional work great within the health system itself. It is opened up via FHIR. Or do you think there's going to be a new like ground up something that we're going to see in maybe the next five years or so?

Bill Russell: I have not.

John Halamka: So he was our CMS administrator, but he founded something called the Institute for Healthcare Improvement. The IHI. And Don would use the term sometimes you engineer a system to achieve exactly the result you got. Now, let's think about what we did during the meaningful use era.

And we got burnout. Oh, that's a shock. We got exactly the results we engineered. Now. The dream is not just lipstick on a pig, so to speak. Remember I run a farm so I can say that but to say that we're just doing the EHR as it is [00:25:30] today. And we're putting some FHIR interfaces and maybe a little bit more decision support, not good enough. What I think we need exactly as you described it as a complete paradigm shift in the way that medical records are recorded. How about this? A doctor and a patient have a conversation. The computer, of course, with everyone's consent, whether it's audio video, or both is recording that conversation. And the end result is through NLP and ML.

Bill Russell: Interesting. I wouldn't do this to you, cause you're living your best life at this point, but if if you were some [00:26:30] VC came along through a ton of money at you and said, John, ,we need you to, to lead in this EHR revolution. I assume when you wrote your EHR back in the day, you, you designed it.

Here we are. The EHR market is essentially consolidated to I don't know, let's call it a half dozen players. And we're saying, all right, we're going to fund this. We're going to get to the people we're going to put the team together. Where do you start? I mean, what you just described is a great use case, but I can, I can tack on Nuance onto their DAX ambient clinical listening onto any of the six platforms that are out there today.

John Halamka: So I guess one of our challenges is regulatory complexity. Right. And so think about it. I mean, I worked on this self-built EHR in the 1990s, when clinicians could invent things for clinicians, nurses, for nurses, pharmacists, for pharmacists to [00:28:00] support what they felt was optimal workflow, safety, quality, transparency, and that kind of thing.

It would be tough. So I think we have to take a careful look at what it is we want to achieve and engineer what we want to achieve that will require regulatory change.

John Halamka: Yeah, it's high nineties, but so, but here's the issue and sometimes you have to do the wrong thing to get to the right thing. Right. So think about you may not remember the CCD and the CCDA and all these XML forms that were used as is interoperability.

John Halamka: Yeah. So we argued way back 2005, 2006 for APIs. And no one in the industry thought an API would be a reasonable thing to do. We are so happy with HL7 V2. Why don't we make H7 V2 an XML? Have more data elements? It's very comfortable. It's very safe. Well, it was only when we discovered that anyone can generate a CCD and no one can parse one.[00:30:00]

That anyone was oh wow, we made a horrible mistake. That API thing, it's exactly the right thing to do. Now we got there. And I think we had a market failure in the adoption of EHRs. We needed a regulatory change. And now that we've got market acceptance, we need a radical revision of how those things work.

This isn't even a ten-year projection. It's a three-year projection. What role do you think technology is going to play in addressing this challenge? I realize a lot of other things are going to be at play here, but I want to focus in on the technology. Do you think technology has a role to play?

And then a human reviews it. Cause [00:31:30] the physics and the math, it's just done by an algorithm, which is reasonable. And so the question is, can you augment your humans and make them wildly more productive because they're doing review rather than authorship. So that's certainly something technology can do, but there's another aspect and that is practicing at the top of your license.

Well, if humans can work better, stronger, faster and each human can work on just the stuff they're uniquely qualified to do, we'll probably be able to get through the great resignation.

And it turns out that computers are really good. you know, If we build the right algorithms. I wanted to touch a little bit on, the Google partnership and how that's going. I thought, you shared this before, and I find it very interesting that you've created these layers of access to, [00:33:00] to the data.

And we always sat back like, oh my gosh, that's a huge amount of exposure. It's like, oh, well, we'll run on your, on your network. It's like, all right. Yeah. But we still have all sorts of challenges, but you've addressed that [00:33:30] with your architecture with Google. Can you share a little bit of that with us?

We create sub containers of it and then invite [00:34:30] collaborators, joint ventures, and partnerships into their secure sub container, where then they can run algorithms against the de-identified data. But actually can't take the de-identified data. Nor can they link external databases to the de-identified data to try to re identify.

Bill Russell: That's fantastic. So talk to us about the Google partnership. That was a ten-year arrangement. And if I'm not mistaken, was that two years ago that that came together or was it, was it longer than that?

John Halamka: Just finishing our third year.

John Halamka: Well, so think about the nature of what we want to [00:35:30] do, given that we have this amazing cloud computing and storage environment is creating more multimodal data and more algorithms from that multimodal data, especially in the field of various images.

So I have glaucoma. This is why I'm very interested in your ophthalmology example. I've lost 25% of my vision in my left eye. Mayo clinic. It says, well, that seems a bit odd because you buy a maximum medical therapy, very compliant. Why are you losing vision? We better do some imaging of your brain.[00:36:30]

Maybe there's something funny going on there. So they did 6 MRI sequences in my brain. They found them my third ventricle, which is a cistern of fluid in the brain is larger than a normal human. So I said, Hey, do you have any idea what the Gaussian distribution of third ventricle sizes and a normal human?

Bill Russell: So let's assume I'm a health system CIO at this point. I'm listening to this and I'm going, Hey, [00:37:30] you know what I would like to, I'd like to be a part of that. I'd like to add our data set to that. You have some of these players out there that are collecting datas from data from a lot of different health systems and what not. But if they hear this and they go look, I would like our data set. I'd like to participate. What does that look like? Is that a possibility?

I think that has a whole lot of security, privacy governance, and even psychology challenges. My notion is Federation. And so what I have been working on, and we're just finishing a pilot on this is to [00:38:30] say, and arbitrary data provider, not at Mayo can interact with an arbitrary algorithm. That might be at Mayo or not in a way that protects the IP of the algorithm and the privacy of the data.

Let's try our data set against your algorithm, see how it performs. And I can do it without having you sending me your data.

Bill Russell: So does that require me to move my data into Google's platform? No it doesn't. You're saying federated wherever.

Bill Russell: That's interesting. What if I wanted that architecture that you're talking about with Google? I thought, Hey, this is pretty interesting. Move that data into that, that kind of that kind of architecture. Is that the kind of thing that you've public domain that architecture? Or is that the kind of thing that I can participate some way in that?

Bill Russell: Do they have a skew for anything? I, I wouldn't imagine.

Bill Russell: John, how's the farm?

John Halamka: So far so good. You know, It's cold.

Bill Russell: In Massachusetts. Yeah I would imagine it is cold.

John Halamka: So here's the challenge, right? Keep water liquid for 300 animals when the temperature approaches zero.

Bill Russell: You just gotta keep it moving don't you?

Bill Russell: So do you have a team helping you? I mean, you're taking care of a lot of animals, your sanctuary for these animals. I assume you have some volunteers and some help.

John Halamka: 500 volunteers.

Bill Russell: Wow. Do you have to manage them in any way?

Bill Russell: Wow. The volunteers. Are they educated before they come in or do you, they're just kindhearted people that show up?

And so you can put your finger in the mouth of a sheep or a goat, and you'll still have your finger. As opposed to a horse or a cow full set of teeth, don't do that.

Bill Russell: The level level one is keeping them safe, making sure they don't get injured.

Bill Russell: And this we'll close with this, but in this day and age, I'm talking to so many people that have so much going on and you have a ton going on. I mean, I, I read your name. I see. You're, speaking. You're here. You're there. You're doing a lot of stuff. Is this relaxing for you having 300 animals when you come home?

Or is this, I mean, does this, raise your, your blood pressure as, as you have to deal with that and a pretty demanding job.

Let's go do the next thing. So caring for these is not only a benefit to the animals, but it's a benefit to the whole community because in this time of COVID, do you know that we have actually become a rescue for people? Because the [00:44:00] level of anxiety in society is so high. And when you spend an afternoon with a donkey or a goat or a horse, your anxiety is a lot lower.

Bill Russell: Because those, those animals don't know that COVID is going on do they? They, they just. Man, that's I am going to take you up on that at some point, I'm going to reach out to you and say, I'm going to be in Boston at these times and see if we can coordinate. I'd love to get out there and see what you are doing out there. And the 500 volunteers, I think it would be fantastic.

Bill Russell: Exactly. I would expect nothing less. Hey John, thank you again for what you're doing for the industry and thank you again for taking the time to visit with us. I really appreciate it.

John Halamka: Well, any time, stay in touch and be well.

Bill Russell: What a fantastic discussion. If you know someone that might benefit from our channel, from these kinds of discussions, please forward them a note, perhaps your team, your staff. I know if I were a CIO today, I would have everyone on my team listening to this show. It's conference level value every week of the year.