This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

Today on Keynote

(Intro)   I have recently had multiple patients come up to and share that when they had gone to the clinic, The physician was using ambient technologies in the exam room and how much they appreciated it. To your question of is it going to have a material impact, for those people it already did.

  My name is Bill Russell. I'm a former CIO for a 16 hospital system and creator of This Week Health, where we are dedicated to transforming healthcare one connection at a time. Our keynote show is designed to share conference level value with you every week. Today's episode is sponsored by Artisite, Dr.

First, Gozo Health, Quantum Health, and Zscaler. Now, let's jump right into the episode.

(Main)   All right. Here we are for a keynote, and I'm joined by Seth Hain, SVP R& D at Epic. Seth, welcome back to the show.

Good afternoon, Bill. It's great to be back.

I'm always excited to have this conversation. I think this is the third time you've been on the show.

The first time you were on the show, you were far and away the most listened to show. That we had that year. And what do you think that speaks to your popularity or the fact that you're the SVP of R& D at Epic?

I don't think it's a personal thing. I think that It is evidence of the collection of folks we've got here and the amount of time we're spending to try and help the health system and others.

I feel very lucky to be surrounded by those folks and of that.

Yeah, I think it speaks to the impact that Epic has had on the industry, the number of people that are really looking to Epic to lead the way in a lot of different areas for advancing patient care and advancing the clinician experience and those kinds of things.

I think that's really the reason that so many people want to know what you're thinking in the R& D space. before we go into some of those specifics, let me talk a little bit about your personal journey or let me ask you some questions. Share with us your journey into healthcare and healthcare technology, and then share with us your journey at EPIC once you got there.

Yeah, the journey to EPIC started with the journey to Madison. Which, mathematics is what brought me to Madison originally. My background is in pure mathematics,

guy then?

wear two hats. I grew up outside of Lincoln, Nebraska. So I am a Husker by, I was raised a Husker.

And then transplanted to be a Badger. There are some interesting Big Ten games to watch once in a while as a result of those alliances. It's hard to make a decision which red to cheer for. But yeah I tend to think when I look back of three things that really hinge how I got into what I'm working on today.

And I think that in many ways it echoes some of the themes across the EPIC as well. And that's where I grew up through some of this, right? And the first one was that background in mathematics and Growing up around computer science. I don't know that I've ever told this story in public before, but I was the kid in Nebraska that took his computer programs to the county fair to get ribbons.

So we had a 4 H program that involved programming that I was part of back in the early days. So that background then ultimately led me to Epic about 18 years ago. And a couple of kind of pivotal points there included one working with our systems architecture. So really building out at a low level how these integrated apps.

And working alongside clinicians who are doing a lot for our country and world during that challenging time. But I think it was an important aspect to make sure that I spent a lot of time and got a deep familiarity with end users in that regard as well. And then, those three have really led to my focus, particularly over the last 12 months, but certainly longer than that, is I've been working with our analytics and AI teams.

on embedding that into the core of our software to bring that type of intelligence right directly into workflow to help folks be more efficient and have the opportunity to spend more time face to provide better care. So that's really been the arc, if you will that I've been going through.

There's more I need to continue to learn, more we need to continue to do here, but I imagine it'll keep building on those principles.

it's good to know that Judy does recruiting at the county fairs. Looking for the ribbons and that kind of stuff. That's there's hope for the future.

I want to talk to you a little bit about that, being embedded with the clinicians, especially during the pandemic. how did that influence you? How did that change you? What did you see that you were like, you know what I knew this stuff maybe intellectually, but now I'm seeing it and I, and maybe feeling it for the first time.

Yeah. I think

Right as everything was going into lockdown. I still remember the building I was in as we were walking down the hall and looked at each other and went, vaccines are going to come out. And when they do, we're going to have a real opportunity. to help folks efficiently administer those to large swaths of the population.

And immediately at that point, we already had the ability on mobile devices for nurses to administer vaccines. But it was designed with the intention of It might be multiple vaccines you'd be getting at that point in time, but we realized that there would be a need for one vaccine, maybe, two or three is where we ended up, but it depended on the health system to just be administered in mass.

And so we immediately had a team at that point in time start looking at every single click in the workflow. To say, if we were only going to administer one vaccine via a mobile approach, how would we do this with the fewest number of clicks to make sure that we help save time for folks and get individuals across the country and the world those vaccines more efficiently?

that opportunity And that possibility both brought hope and I think that's part of what we can do when we introduce efficiencies into the software. And it highlighted the importance of kind of rolling up your sleeves and making sure you deeply understand the user experience. So that was one of the keys.

The other one that really stood out to me during that whole time was all of the work and in many cases challenges. around deeply understanding the analytics that was happening across the country and continuing at large scale to figure out how to optimize the use of beds where folks where folks had opportunities to scale up their usage in the hospitals.

Those sorts of things and the collaboration that can happen in those contexts as well. So in the midst of it the folks at NYU build out a machine learning model for predicting who is likely to have favorable outcomes from COVID, and the idea was to help Physicians and others assess patients that might make the most sense to go home while they were recovering from getting COVID rather than occupying a bed that others might need.

This was right at the height of when we were struggling with those capacity challenges as a nation. And not only did they publish on that and put it into practice to run it in real time in their systems, but they then shared those best practices, including the machine learning models. And Oxnard picked it up and put it into their system.

And so you saw both the opportunity for analytics at scale to optimize capacity and to improve operations and in practice AI being put in across the community to improve outcomes. And those are the types of things that saw as I worked through those circumstances with the team here and then the organizations were lucky enough to serve.

as you bring those two areas up. It was interesting during the pandemic doing interviews and the need for these retail like strategies. Hey, all of a sudden Atrium is going to do the vaccinations at the Speedway. You remember this?

So you went to the Speedway, you went on the track and stuff, but when you got to the end, it had to be barcode boom, you're in and you're out. And it wasn't only them. It was a university of. of Colorado. It was, it happened all across the country that we had to learn these new capabilities all together and all at once.

And I thought the sharing was great. And then the other thing I remember you, you talked about the NYU model. I remember, I think it was Parkland down in Texas had a model where they could predict where outbreaks were going to happen in their community. And that kind of analytics, again, it was like they came into IT and said, Hey can we predict this?

Can we, it's like, what can we do? We need to get ahead of this because our rooms are full, we're at capacity, we don't have enough ventilators, we don't have enough, you just went through all this stuff. It's we don't have enough gloves for heaven's sake. It's like, how do we get ahead of this?

And they were looking at IT and saying, traditionally we've come to rely on technology to get us ahead of this, either with knowledge or with systems and processes. that is two of the areas I remember people talking about a lot. You have extensive experience and what have been the most significant challenges you've encountered in healthcare IT over those years.

It's interesting because go back to some of the challenges were just, hey, how are we going to scale this thing up? Like it can handle this many patients today. How is Epic going to be able to handle, 10 times that many patients at a health system and the capacity and whatnot.

when you think back on that question of, The greatest challenges we have encountered to date in healthcare IT. What are some of the things that pop

into your mind? it is a fascinating question and there's both so many and and they're so varied depending on where you are practicing care.

As you noted, a consistent focus of our company here, and in general, I think an important aspect of this has been consistent scale and response times and availability. And I think all three of these are key, and it goes back to the, those early days I talked about when I was on our systems and architecture teams.

Making sure that teams both had in place technologies to make sure that systems were highly available and that they had the IT practices and acumen for testing and validating those approaches was an important aspect of it. But I think that ultimately what has been an interesting challenge and I think will continue to be going forward.

I don't I don't think this is unique and I don't think it'll change. is that there's an ongoing opportunity for a dialogue between on the ground operations. And how that can change and adapt how things like virtual nursing, as an example, or monitoring throughput in a virtual manner across multiple sites of care or care management outreach, including IoT.

Asynchronous means of communication with chronically ill patients, for example, change the models and the ways that we operate, and that it's a handshake with the technology. And that dialogue between folks building technology and imagining what might come next, and the operation teams, And the clinicians on the ground and the challenges that they're facing and the opportunities that they see in front of them and how to adapt to use the technology, I think, is the ongoing theme of this tree.

And that, I think, is where we can all continue to spend more time because that continues to improve. The care that patients get will continue to improve along with it. So you

knew I was going to have to ask you about artificial intelligence at some point. And we're going to delve into this a little bit.

I'm curious to get your perspective. I have my perspective on this quite frankly, I think we've already seen material impact from AI. When people ask me that question, I'm like there's been a lot of advances already. But, how do you envision AI reshaping healthcare delivery and patient outcomes next 24

months?

I'll tell a quick anecdote and then get into some of the longer term ideas. I have recently had multiple patients come up to me and share that when they had gone to the clinic, The physician was using ambient technologies in the exam room and how much they appreciated it. And that is the type of conversation I've never had before.

To your question of is it going to have a material impact, for those people it already did. And now there's other ways to measure this as well. Certainly I don't want to just reference. One off anecdotes, but I also don't want to dismiss them because I think that they started to build out a path for the future.

I think the term AI is interesting, right? There's multiple things underneath that umbrella that include both the predictive analytics that has been in regular use across the system for going on half a decade or more. At this point, we've got over 400 health systems using these predictive models for helping putting nurses in better positions on the med surg floors, to assess patients, to helping identify patients that might not be able to show up for their ambulatory appointments, and to get them help coming in to make sure that they get seen, those types of things.

I think oftentimes when folks are asking this question, though, They're thinking about generative AI the type of technology associated with chat GPT and some of these other consumer tools that folks are starting to look at. I expect over this next year We will see this continuing to increase in scope in regards to improving clinical efficiency.

We've initially seen that in the outpatient settings in many contexts, things like ambient encounters where we have now had tens of thousands of encounters documented using ambient technologies as part of it as well as helping speed up other parts of that clinical workflow. For example, helping a common pattern is drafting suggested text that clinicians be that nurses, physicians, or others might have been typing in response to a message from a patient, for example, a challenge that really increased during COVID as we were talking about earlier.

I think there are also real opportunities that we'll see over the next 12 months. and are starting to see today around automation and backup. Particularly coding for example, but also a variety of other tasks around communication that takes place in those contexts as well. I don't want to miss out on some of the more important approaches with analytics as well in this context.

Because particularly given some of the financial challenges that health systems are facing right now, the combination of analytics, and in some cases these predictive toolings around AI as well, can really help with the capacity challenges. And making sure that folks are fully utilizing FEDS, ORs, have effective levels of nursing, staffing, etc.

And that type of insights from the data can really help put folks in the best position in those regards as well. I think there's a lot coming over the next 12 months, but think the door is going to be open wider. As we look forward think that there's a short term outlack of real value, but there's only more to come.

  📍   In the ever evolving world of health IT, staying updated isn't just an option. It's essential. Welcome to This Week Health, your daily dose of news, podcasts, and expert commentary.

Designed specifically for healthcare professionals like yourself. Discover the future of health IT news with This Week Health. Our new news aggregation process brings you the most relevant, hand picked stories from the world of health IT. Curated by experts, summarized for clarity, and delivered directly to you.

No more sifting through irrelevant news, just pure, focused content to keep you informed and ahead. Don't be left behind. Start your day with insight at the intersection of technology and healthcare. This Week Health. Where information inspires innovation. 📍 Increase 

I do want to talk to you a little bit about the ambient because one of the promises of technology is eventually it fades into the background. It's still there. It's doing a lot of heavy lifting, but it fades into the background, and Ambient listening is such a great example of that. It fades into the background. It can generate the note. It can potentially generate the coding. it can potentially, interact with some other systems and identify drug interactions and whatnot and bring that right to the clinician There's a whole host of things. in the ambient journey. And we're not even talking about computer vision. And you have companies that are, putting cameras in rooms at this point, and they're identifying, bed turns and just a whole host of things that are going on to assist the clinicians in their journey.

I always thought it was integrated. But I think you guys have entered a new new types of relationships with these companies to really embed it into the workflow. Can you talk a little bit about that?

Yeah, I think, it's interesting because I think there's a thread through what you were describing and what we're talking about here.

that hits on sort of two different aspects of this. And one is that deep understanding of the opportunity from a workflow perspective in the clinic. Or the opportunity at the health system level to make and help with recommendations at the operational level. And that understanding is using technology to inform that.

And I think in either of these different contexts, you can think of ambient as an example. Where you start to really transform the workflow. And ultimately you do that in the service of heightening the patient physician interaction and deepening that relationship in a way that will provide better care.

And I think ambient tends to be something people will focus on in these contexts because it's one of the most obvious. And immediately impactful ways where the workflow has dramatically shifted due to this new type of technology. I don't want to, these other cases it's certainly helping save time.

But as I look 12, 24, 36 months out, I think that, These underlying large language models, the technologies behind things, like ChatGPT, or BARD, or these other tools, they open up new styles of interaction, new styles of programming, and ultimately some reimagination of the workflow that really creates new opportunities and hopefully all with the intention of deepening that relationship between the patient and physician.

first time I dealt with ChatGPT, I'm curious what your first experience was with ChatGPT. Mine was, I put my first prompt in there and it came back and I was like, Oh my, and then I put in another prompt and it came back and I was like, Oh man, this is for me, it was like the culmination of what I thought computers would eventually be when I got my first Commodore 64.

It's I would say, Hey computer, give me this information and it would give me that information. That was my experience the first time with ChatGPT. I'm curious what your first experience was.

the first one I really dug in with was discussing philosophy with it. And in that context, I realized that there was this sort of two sides to the coin.

And I've come to appreciate this more having spent more and more time with it because you really got to spend your 10, 000 hours with these tools to deeply understand their possibilities and then continue to roll up my sleeves and continue to do it more. But the opportunity for almost heightening imagination, this, the flip side of the coin of hallucination is imagination.

That opens up some new programming opportunities and some new possibilities in regards to, just simply kindly responding, providing draft text when kindly responding to a patient that a form letter never really was in the same way. And so building out and using these tools and programmatic pipelines.

They both capture that imagination while reducing the risk of accidentally and correctly stating something in these contexts opens up entirely new programming paradigms. And you, I started to see glimpses of that really early on as did others here as well, certainly. And that was how, and when we started rapidly, this was over as I think, Sumit spoke about it.

At our user group meeting over the Christmas holidays last year, started rapidly programming out our first use case. And then having folks live actually on GPT 3. 5, knowing that they would upgrade to 4 within weeks with the in basket use case in April of last year. And it was really that sort of arc.

Okay, we can solve some real new problems. We got a new tool in the tool belt here. Let's get to work.

I was reading it's either Deloitte or Gardner's predictions, but I think they're all predicting the same thing, which is essentially these tools are going to step in and start to do a lot of the menial and mundane.

work that no one really wants to do. It's not, if we use the vernacular of practicing at the top of your license it's the kind of stuff that you really want some, something to take care of for you. And there's this hope that something like ChatGPT might be able to step in and do that.

What stands in the way of material progress for us trusting ChatGPT? to, look at the encounter and generate a bill or look at the encounter and do the coding or whatever, what, obviously it's doing the note today what stands in the way of a significant advancement in the use of large language models?

And I keep saying chat GPT, I shouldn't there's several large language models, and there's even. Open source models I think that people can tap into as well.

I think that, I view the opportunity slightly differently. I think one of the things that was critical as folks rapidly both deployed and adopted predictive analytics was designing it in a manner into the workflow that augmented what folks were doing.

It was less about how do you do the work? That's not the circumstance. It's more about how do you put them, how do you augment what they're doing? And I think some of these studies that you were referencing as well as others have shown that the quality of work folks do when having one of these language models working in concert with them further enhances The ultimate end result.

And I think one of the keys there is continuing to build out deeper user interfaces. that make it straightforward for those to work in a colleague perspective. An example of that with the ambient use case, for example, is that folks can build out note templates that pull both from the ambient conversation in the room, but also from the medical record, right?

These ideas of multimodal, where you're also using images, et cetera, to do that. Is key. So the first piece of the answer is I think it's really about augmentation. And working together with these models to get to the outcome and then the, and then in the workflow. The other key here that I would highlight, and I think that in many contexts, folks have asked one direct question of one of these consumer language models and clearly gotten a wrong answer, as an example.

As a developer, I take a very different approach to this. There is a series of pipelines and checks we build into the software behind the scenes. It's not a single prompt like one might interact with chat GPT on. And as a result, that pipeline increases and improves the quality and builds it out in the context of the workflow.

to better be an assistant to the person that is working through that workflow in that context. And you asked about what are some things as an industry we can continue to build on. I think there's ongoing opportunity for trading tips and techniques for validating these models silently behind the scenes.

Understanding particular workflows at your organization, taking those statistics, evaluating them, and then putting the models into practice, and in these workflows into practice. And we build out a science around that as an industry for predictive analytics, and we're building that out right now around generative AI, but there is still work to do there, and it's work that.

I'm happy to have a lot of colleagues to work with across the healthcare community and have been over the last year. And I think we need to continue to do more of that.

Yeah it's interesting. Programming, we could create those checks and balances all throughout the model.

In AI, especially large language models, I don't want to say AI, but in large language models. I've started to see people start to utilize challenge models, right? And we do this with ChatGPT. We'll ask it a question, it'll come back, and we'll read it, and we'll go that's not really what I was looking for.

Maybe I didn't phrase the question right or whatever. And then I will challenge it, and it'll come back with a better answer. Then I'll Again, prompt it again, it'll come back with a better answer. And one of the articles I was reading was talking about creating these challenge models where the main LLM is coming back with an answer, but it's being challenged by another LLM to say.

Hey, validate that information and that kind of stuff. is this an extension of programming techniques we've been using over the years as we move into these large language models and AI models?

Yeah, I think that in many cases approaches and concepts that we've used for development for decades apply.

Things like building out telemetry and visibility throughout these. Obviously, the importance of both scalability and cost effectiveness as you're doing this development matters. There are a new set of techniques, though, as you're highlighting here. And some of these have been talked about in academic circles for decades.

But with the advances over the last 18 plus months here are now enterprise ready, where you're doing things like having multiple models check the quality of other models, using some of these retrieval augmented generation or RAG for improving the quality of the information of the models. or reasoning against, et cetera.

And, internally here there's a channel that we post. I swear every day there are two or three new prompting techniques that we're pasting into this channel from a research paper that we go and study. Do we implement this one into the pipeline as well? Which use case is this best for? So there is a need to constantly adapt.

As we do this and in that context, continue to also make sure that you're thinking about those other principles around scalability and cost effectiveness as well.

Now, I'm curious, I'm not going to ask you about, are we heading towards a Terminator future or not, but that is one of the concerns that people have.

I'm going to ask you about two concerns. One is the Terminator future, and I'm going to ask you. about artificial general intelligence. And then the second I'm going to ask you about is patient data and privacy. So I'm just giving you a little warning of where I'm going. Artificial general intelligence is when the machine actually starts to generate new thoughts, new ideas based on what's coming in and those kinds of things.

It's widely considered the point at which they become thinking machines. They become machines that are and of themselves evolving and getting smarter. I haven't read anyone who thinks that we've seen that yet. There are times where you get a prompt response where you feel like you're interacting with it.

But how far away from AGI do you think we are at this point? this is not the epic, stated stance I'm just asking you at this point.

I continue to not know what that word means, or that term means. I have started to think about it differently, though. And over the last 12 months, One of the things I've really started to wonder about is actually if we're going to have multiple intelligences.

The way that these different neural nets are both trained as part of their development approach in regards to how as they're developing and learning that they train is what it's called, as well as what data they are trained on. almost ends up creating different styles of intelligences.

And so I think oftentimes people use the term AGI thinking about human intelligence. But I think that there are real opportunities as we look at different types of molecular data sets different types of chemical compounds. Frankly, different board games people play. We can have intelligences that are very highly capable in a variety of different things.

And it may be that we ultimately start talking about these differences in intelligences and not just when is this singular one going to happen.

That's really interesting. I appreciate that response. Patient data and privacy. Vast amounts of patient being handled. you just mentioned training these models.

It's amazing to me when I just interviewed a physician that is utilizing the API for happens to be OpenAI's API for ChatGBT, generating synthetic data. And he's this model knows a lot, like it already knows a lot. so people have a concern about patient data and patient privacy with these models.

How do you balance innovation and privacy security concerns moving forward?

it's a important question. And I think that a key distinction that is necessary when thinking through It's the difference between the application that one is using, which likely may include in these contexts, a generative AI model behind parts of it, and the generative AI model, the foundational model itself.

And that was a key distinction we made as we architected out our capabilities where we are using HIPAA compliant. Fully BAA protected services that do not include any opportunities or rights. or the data to be able to be used for training purposes, et cetera, when being used behind our applications.

And separating those two out give you the opportunity to think about and approach this in a different manner. And then, Addressing each one individually. And that's how we've been thinking through it here.

We're coming up on our end. I appreciate you allowing me to go really deep on the AI questions.

I think if Most people were sitting here. That's where they would have wanted me to go, but team, the team is broadly looking at healthcare innovation and what's going on, what are you excited about? What are you looking at right now saying, Hey, This could really fundamentally impact outcomes for our patients, fundamentally impact the patient experience, fundamentally impact the clinician experience and efficiency, maybe address burnout. What are some of the things that you're looking at, you're saying this has some real promise?

I've been spending a lot of time thinking about the way that integration evolves you're a student of the industry, as am I, and the integration of applications was one thing that we saw provide deep value.

Over time, and over the last couple of years here, we have been spending a heavy focus on something we think of as, and describe as the health grid, which is integration of the larger healthcare ecosystem. With the patient remaining at the center, just like they were at the center of our integrated suite of applications that we built.

The health grid gives the opportunity for that type of integration across sites of care. So it might be dental clinics, which you and I have talked about in the past as an example. It might be specialty diagnostic groups therapeutics. It might be retail clinics. It might be Coordinating clinical trials across these different sites of care as an example.

And I think that network approach and the opportunity that as a patient, as I want to move between those sites of care, but have a consistent view or open up is very exciting. And then I think the layer that will come after that is integrated intelligence. on top of that. And which is what we've obviously a reference to what we've been talking about here earlier.

It's those three layers that I think that I'm particularly excited about. It really starts with my chart at the center of that patient experience. Alright,

We can all project what healthcare is going to look like in three years. That's hard enough. Ten years from now, what will fundamentally look different from the clinician's perspective and what do you think will fundamentally look different from the patient's perspective?

That's ten years. Ten years in technology is that's a hard projection, but. How will it be different for the clinician? How will it be different for the patient?

If anything, I feel like the last 12 months have taught me that all of my predictions need to be taken with a deep grain of humility. So I simply laugh when think about that. I do think we will see a few trends continue. I think that the role and the importance of having a patient at the center will be key.

As an individual that's moving between sites of care becomes increasingly mobile. The opportunities, just simply during as we were talking about back at the beginning. During the pandemic to use telehealth right felt transformative at that point in time and that was only two or three years ago And I think we will see that trend continue and heighten and then on the physician side The opportunity for far more deeply informed and data rich perspective on how different treatments, different cares, different patients have reacted, and using that information to Right at the point of care, alongside my patient, to make informed decisions about what is best for their health and for their goals as an individual, I think it is going to continue to open up new opportunities.

But, as I said, take every prediction with a grain of salt. Yeah,

the health grid is interesting to me because I wonder how much more the health system will know about me, or at least be able to tap in, if I allow them to tap into about me. To help me on my care journey and know that, hey, not only my clinical information, but also that the details around how I make my health decisions and the things I struggle with and those kinds of things and become more of a more of a guide in, in my health journey.

Do you foresee the health grid bringing in these other kinds of data sets alongside the clinical data set to help inform our clinical

journey? I do think that data is part of it. But I think more importantly, the health grid. With that data as a backbone, opens up opportunities for new styles of care delivery and coordination.

I think ultimately it's about workflow and operations coordination across that grid. How does a patient I can have A cohesive experience moving between a specialty diagnostic, a group that I'm working with, likely, maybe even just through the mail, as an example. my health coach and my primary care physician, and know that all of that information obviously will be there heaven forbid, if I need to go to the ED.

And so I think it's that coordination that's ultimately what matters.

Seth, I want to thank you once again. I feel like every time I talk to you, I learn more. So you're part of my training model as a large language model over here. So I really appreciate it.

We're all learning. Enjoy the conversation, Bill.

Thanks. Thank you.   📍

📍 📍 Thanks for listening to this week's keynote. If you found value, share it with a peer. It's a great chance to discuss and in some cases start a mentoring relationship. One way you can support the show is to subscribe and leave us a rating. it if you could do that. Big thanks to our keynote partners, Artisite, Dr.

First, Gozeo Health, Quantum Health, and Zscaler. 📍 You can learn more about them by visiting thisweekhealth. com slash partners. Thanks for listening. That's all for now..