This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

Interview In Action: Surviving and Understanding the AI Feature Flood with Joseph Evans

Find out more@harmonyit.com.

Bill Russell: Today on the 2 29 podcast.

Joe Evans: i'm not sure that we see a lot of hallucinations, which is what I think originally there was a lot of concern about. But it's more the emissions that I worry about, you know, what is it missing? And as we think about summarization, what some of our incumbent models do, um, are we sure we're getting the whole picture?

Let's jump into today's conversation.

Another episode of the 2 29 podcast. And today we're joined by Joe Evans, CMIO, for Sentara healthcare. Joe, welcome to the show.

Joe Evans: Phil, thanks for having me. A pleasure and a privilege to be here. Appreciate it.

Bill Russell: you know, Joe, the, the, so we ran into each other in LA and I think you saw me after about four hours of sleep and I was moderately coherent. We're recording this Thursday following vi and you've had about, a little bit more than that in sleep, but not much.

Joe Evans: That's correct. Yeah, I am about.

Bill Russell: Perfect.

Joe Evans: eight hours of sleep and two meetings in between and, uh, or not eight hours of sleep, eight hours home, uh, a little bit of sleep, couple meetings. But, uh, yeah, great to be here and, uh,

Bill Russell: Yeah. That,

Bill Russell: having it in LA is so tough. I mean, that flight, that flight out is, isn't bad, but the flight back, you, you, you lose an awful lot of hours on that flight back.

Joe Evans: all the storms we had, the, the, you know, all the people catching up, uh, definitely impacted,

Bill Russell: all right. Hey, Joe, You're just back from the vi event, so I'm gonna take advantage of this. Uh, I mean, what was the, what were the key themes? What were you hearing at, at the event?

Joe Evans: The key themes were, of course, all around AI and you know, what agents are gonna do what, what our major platforms are gonna do around ai. but yeah, that was, I think one of my big takeaways. It was a great opportunity to meet with some of our partners, uh, talk to them about their roadmap, and then also explore, um.

Include people dropping in P-H-I-P-I-I, um, ip, et cetera.

Bill Russell: Well, if, if, if that was the theme there, we're gonna continue that theme on this show. Uh, Uh, oddly, oddly enough. That's, uh, what seems to be top of mind for everybody these days is, uh, AI governance, shadow ai, um, how we're gonna bring AI into the organization. We have our existing vendors, which are. Um, how do I, say this kindly? They're throwing AI features at us. Like, there're, there's so many of them. It's like, alright, slow down, slow down. What, what does this one do? How does it process information? How, what are the guardrails? Um, so let's, let's start there. Let's start with the existing partners that we have in the industry.

I, you guys are an Epic shop, I assume.

Uh, Microsoft, another. Um, and then we have a couple of other platforms to think about. Um. You know, whether they have a solution for it or where it is on the roadmap. Um, and, uh, I mean, I think that is our starting point, but as, as you referenced, um, you know, Epic has a lot of. Generative AI solutions that are, you know, out there, you know, we are still on the, um, a la carte.

But, but yeah. Um, it is, it's a, a crowded space.

Bill Russell: I guess the better question for you is from a clinical standpoint, how, what problems are you potentially looking to AI to solve or to do a little bit better than the, the previous generation of tools that were out there.

Joe Evans: so from a strategy standpoint, the way we're thinking about this is we're looking at some big opportunities around rev cycle management, around what we can do better in care management, being A IDN, and also from a level of care length of stay standpoint. But from a purely clinical standpoint, looking at all the tools in the Epic Box, um.

The health of the.

'cause they wanted to, to know that in order to trust it. What's it gonna take for, for us to trust clinical summarization at scale within a health system?

Joe Evans: Yeah. I mean, Bill, I mean, I think for any of this, um, to work, trust is the foundation and I think to build that trust. There has to be a good governance process in place so that any solution we bring into our production environment, uh, is well vetted. Uh, that's. The very foundation of it. The next layer, uh, for our clinicians is to be able to, you know, create citations.

Bill Russell: It's, I are, are you concerned about the models themselves that are being used? I, I know we're, we're looking for as much visibility, um, dare I say, transparency into these models and, and how they're processing information or how they're. their work, but I, I know that some, some partners push back and they're like, look, that's proprietary.

And then some are using these foundation models and we're never gonna have clear visibility into these, um, non-deterministic models, how they make decisions. Just 'cause we know we can ask them the same question twice and get two different answers. I mean, anyone can. Try that out today with chat GPT ask it the same question, two different sessions, you'll get two different answers.

Joe Evans: as you said, the foundation or frontier models, um. From an AI oversight or governance process, it's not like a machine learning model where you can measure its outputs, see if it drifts, as you said, you might ask it the same question Exactly. Or with slightly different, um, input, um, but get a different output.

You know, we all know that. In our EHR 80%, at least 80% if not more, of our, um, information is unstructured data, you know, scan, PDFs, et cetera. You know, is it pulling those pearls out of there that are gonna influence what I'm gonna do to in the clinic? Right?

That's the promise and the opportunity. but it has to be built on a foundation of trust in order for it to, uh. To, to really be effective. The, you know, I think the next thing I, I, I'd wanna talk about is, so you have your traditional vendors and you know, we'll look to Epic to use Cosmos and summarization and do the things that they do and do effectively and do well. Um, but there's a whole, I mean, you were just on the floor, right? So that floor is loaded with people that are like, Hey, we can solve this one-off problem that nobody else [00:12:00] can solve. How do you determine. Uh, I I How do you determine if you know that vendor's worth talking to? They're worth bringing in the, I mean, what, what criteria sort of did you use as you walked around that five floor to say, Hey, this is somebody we should probably take a look at?

Joe Evans: I think about our kind of incumbent platforms. Where they are in that space. And then, you know, I think one of the crucial things,

Bill Russell: So,

Joe Evans: from,

Bill Russell: your, if, if one of your existing vendors, your platform vendors does something, you just keep walking

Joe Evans: if they do something well. Um, you know, if, if they have maybe a 50 or 60% solution. Now, I think one of the keys is, um, you know, maybe we explore someone else to fill that gap and have a short contract with them. And then when one of the incumbents catch up, you know, then maybe we end that contract and keep open, uh, for exploration.

Sometimes a little bit harder. But if there's a significant gap there, I think they're worth, uh, talking to, uh, at least for the interim. Um, because it is a rapidly changing space.

Bill Russell: let's Lets take one of the problems. Let's take length of stay. That's an easy one to solve, right? Length of stay's an easy one to solve. So, uh, um, you know, what, what kind of, what kind of solutions, uh, are, are you looking at to, reduce the length of stay and, you know, what kind of solutions, what kind of technologies are they, are they using and, and what kind of outcomes are they promising today?

I guess.

Bill Russell: Well there, there's only so far you can push that, right? I mean, they have to stay as long as it's necessary. But no longer than is necessary. I, I, I remember, uh, having an interview with somebody and they were talking about the fact that one of, one of their findings on length of stay was they couldn't get the doctor to discharge them.

Joe Evans: Sense?

Bill Russell: an agile team identifies those things, but a lot of that is organizational change more than a technology solution I would imagine.

Joe Evans: Yep. Yeah, a hundred percent. You know. Kind of process redesign, thinking about it. 'cause as you said, it's such an orchestration. It's, you know, care management. Thinking about what those discharge milestones are. Um, you know, what social determinants, what does it take to get them home or to, what does it take to get them to a skilled facility?

Um, I think that's why it is so, not only complicated but complex. Um, because there's so many. So many variables.

Joe Evans: I think it is just a little bit of an advance. I mean, we all had Dr. Google before this, right? I think this is just the next iteration and I worry more about people that would reach out to GPT Health or. Healthcare and be relieved and say, oh, well they said it's, you know, that burning in my throat is probably just reflux and I should take some malo and I'll be fine, but it's something more significant.

Bill Russell: You know, it's so I take my blood pressure every morning, I'm pointing over there because that's where I take my blood pressure. Um. And for a while there I was logging it and then I took the log and I put it in the chat, GBT, and it must've said three times, consult a physician. I mean, they, they are, they're making sure that it's, it's, uh, they are, are. Making recommendations or, or giving you an an analysis is probably, they're not giving you a diagnosis. They're giving you an analysis of what you're telling it they're saying, Hey, look, your blood pressure's within norms, one or two of these are outside the norms. Um, it starts with, you probably should talk to a doctor.

And the doctor's looking at them like, I have, I have 15 minutes to see you. What do you think I'm gonna do? Read all this and, and talk to you. It's like, uh, they would come in with that, but it, it, Dr. Google didn't necessarily, you know, make. Uh, I'm not gonna say diagnosis again. I'm gonna say do analysis based on the information you gave it.

Joe Evans: Well, I mean, it's good to hear that open AI is heavily on the conservative side with saying, Hey, see your position. Um, but, um,

Bill Russell: But

Joe Evans: but yeah.

Bill Russell: see the physician.

Joe Evans: Well, you know, not always, uh, right. I mean, there's some things that are simple problem focused. Hey, I stepped on this sea urchin. Um, is it gonna be something that I need to worry about?

Or is it benign? You know, you take a picture of it and, uh, tells you what it is and say, put some peroxide on it and you'll be fine.

Bill Russell: I can see that I'm on vacation. I don't wanna call my doctor. And,

Joe Evans: Exactly.

Joe Evans: Not only nursing, but as you said, physicians apps. Definitely short, um, leveraging technology to help us triage or orchestrate the patient journey, um, to get them to the right level of care that they need. I think it's a tremendous opportunity. If we think about our complex specialties, like we'll take cardiology where, you know, a cardiologist is not just a cardiologist.

They might be, you know, ep, they might be a structural heart person, um, but be able to use technology to help see if they get a referral. You know, does this need, number one, does it need an appointment at all or is there a, uh, a PP that can review the chart and say, Hey, you know what? I think everything looks good.

Kind of level of clinician, I think is a, uh, tremendous opportunity for us.

Bill Russell: And, and is that the kind of thing you bake into the portal or how does, where, where does that get baked into the, to the process? Is that, is that in the EHR? Is that in the portal? Is that in, um, you know, some other systems that you have that.

Um. That would be the first step. But then once it's, building on the cardiology example, once it goes down that pathway, customizing, what is the next best step? Is it a A PP? Is it a centralized a PP? Is it our, um, remote telehealth team? Uh, or do they get a in-person visit with one of our, um, cardiology specialists?

Bill Russell: Outta curiosity, just to give people a little flavor. Um, outside of work, what's, what's the, what's the coolest thing you've done with, uh, with one of the, uh, one of the, uh, frontier models?

Joe Evans: So outside of work, um, last year my wife and I went to France for, um, 15 days, went to three regions. Uh, I'm a, uh, Gemini. Fan, uh, use deep research and, uh, basically it. Planned our entire trip. Um, you know, you know, based on some key prompts, I did, um, a prompt for each region. Um, and it was remarkably accurate.

Bill Russell: But, but that's part of the fun of, uh, fun of travel, isn't it?

Joe Evans: Yeah.

Bill Russell: and all that stuff.

Joe Evans: Uh, I still have them, but it, it did, it gave us, you know, in less than a half an hour, um, a good starting point. I still bought the books though, so Rick Steves is safe.

Bill Russell: Yeah, it's, uh, it's, it's really interesting to think, I think travel is one of the things I hear the most. When I ask that question, I, I tend to ask that question at 2 29 meetings just to hear what people are doing. 1, 1, 1 guy for his, uh, four or 5-year-old did a scavenger hunt and, uh, he, he, you know, put all the information in there and then it came up with a scavenger hunt for his kids.

It's like, okay. Um. Especially international travel. 'cause it's, it's not something you do every, you know, every year or even every couple of years. I mean, you'll, my guess is you're not going back to France next year,

Joe Evans: We actually may go back. We'll see. Uh, we felt like, uh, we had been to Paris before, but, uh, yeah, I feel like we missed some Aaron de that, that we might like to look deeper at, but we'll see.

Bill Russell: I don't think there's a more relaxing country than, than France. Like, it, it just, there's something about it that just. I love, uh, obviously Paris is, is to be loved, but when I get into the countryside, I mean, it is, it is just a calming, I I, I think if I took my blood pressure while in the countryside of France, it would, be almost perfect 'cause it's just, it's just wonderful out there.

Bill Russell: Well, Joe, I Hey, I appreciate your involvement in the uh 2 29 project coming to our events City tour dinners. I love coming to Charlottesville. You guys have, uh, a, a great group there and, good conversations. And, uh, the thing that's distinct about the Charlottesville event is people will drive two hours to come into that, that city tour dinner, which, which is, which is kind of fun.

It's, it's good to hear from, uh, all over Virginia when, uh, when we do that. So I wanna thank you for that and I wanna thank you for your time today.

Joe Evans: Yeah, Phil, thank you and always great to talk and really appreciate the opportunity to be here.

If you have a conversation, that's too good not to share. Reach out. Also, check out our events on the 2 29 project.com website. Share this episode with a peer. It's how we grow our network, increase our collective knowledge and transform healthcare together. Thanks for listening. That's all for now.