After launching in November 2022, it took ChatGPT just two months to become the fastest growing consumer app in history — proof of the rapid adoption of AI technology. In this conversation, three experts from the American Hospital Association discuss the game-changing applications of ChatGPT and AI for health care data analytics, and some of the potential pitfalls.

00;00;00;01 - 00;00;34;20

Tom Haederle

00;00;34;22 - 00;01;08;10

Tom Haederle

Welcome to Advancing Health, a podcast from the American Hospital Association. I'm Tom Haederle with AHA Communications. These days, our ability to process and analyze enormous volumes of data in new and innovative ways is evolving at warp speed. And AI is at the forefront. What are the implications for health care? Where is all of this going? In this podcast, three experts from the American Hospital Association discuss the game-changing practical applications of ChatGPT and AI for health care data analytics, and the potential pitfalls.

00;01;08;12 - 00;01;20;09

Tom Haederle

Host Stephen Hughes is director of health IT policy for the AHA. Brian Klein-Qiu and John Allison are both associate directors of health analytics policy. Let's join Stephen, Brian and John.

00;01;20;11 - 00;01;47;09

Stephen Hughes

Let's step back for a second to talk about how we got here. With less than 15 months ago, the world changed when ChatGPT kind of just crashed through the room, right? And all of a sudden added AI to the zeitgeist, catching a lot of us off guard. Now, as we know, AI has been in the health care space for decades right? There's diagnostics and in revenue cycle and in several areas, there's been tremendous advancement and practical application.

00;01;47;15 - 00;02;11;00

Stephen Hughes

But ChatGPT really accelerated and changed the way we think about it. It's changing the regulatory conversation around it. It's changed the way that every CEO and every board now is thinking about it. So we step back for a second to talk about what you guys do in data and analytics. There is a Brian and a John that works in some form or another for most health systems, right?

00;02;11;01 - 00;02;39;07

Stephen Hughes

We call them different things. It might be performance improvement team, it might be data analytics team. You know, it might be part of the strategy team. But one way or another, most health systems, whether they're doing it internally or thinking about it through partnerships or consultants, are looking at how they use data, how to analyze data, and how do they either improve patient outcomes or make themselves more competitive, make themselves more efficient.

00;02;39;10 - 00;02;51;26

Stephen Hughes

So Brian and John, let's talk a little bit about what you guys do, and then we're going to talk a little bit about how you guys were using AI and how it was valuable to you and how I think it'll be useful to our members.

00;02;51;28 - 00;03;10;27

John Allison

So, yeah, I'm John Allison. I think the most succinct way to describe what I do, at least the way that I approach my job is, my goal is to really tell compelling stories about hospitals, to advocate well for them. And one of the tools in my toolbag to do that is data, right? There's a lot of it out there.

00;03;10;29 - 00;03;28;10

John Allison

Being able to understand it, being able to make sense of the anecdotes, for example, which is oftentimes really how you're learning, is you're hearing from members or other hospitals being able to kind of convert that into something that's maybe more compelling. It's really important.

00;03;28;12 - 00;03;46;11

Stephen Hughes

Right. I think you guys would agree that health care is unique among all sectors in terms of just the sheer volume of data, right, that you have. I talked at the beginning about, you know, finding use out of that data. But there's a lot of noise inside, right, because it's just by the nature of what we do.

00;03;46;11 - 00;04;08;09

Stephen Hughes

By the nature of patient care, there's just a tremendous volume of data that's captured. And Brian, could you talk a little bit about the the different languages around data? One of the challenges here is that it's you know, it's like there's all this data and there's like several different languages that are being spoken in terms of how that data is understood and analyzed.

00;04;08;16 - 00;04;34;19

Brian Klein-Qiu

Yeah, absolutely. So the most common data analysis or data science languages are going to be your typical languages like Python or R. For historical institutionalized reasons, SAS still kind of has a large, large stranglehold over health care policy data. So, for example, most Medicare claims, a lot of these files they come in SAS formats. It's still kind of the lingua franca in our industry.

00;04;34;21 - 00;04;53;04

Brian Klein-Qiu

That's kind of a challenge because there's not a lot of SAS programmers these days, and that's kind of a hard, hard barrier to access for people to be able to interact with claims data. When you add in the fact that a lot of these data analysts, data scientists, they're not specifically gearing up to work in health care policy the way that John and I work.

00;04;53;12 - 00;04;59;08

Brian Klein-Qiu

What that means is you're going to get a lot of great analysts and programmers who have no prior experience with SAS.

00;04;59;11 - 00;05;08;13

Stephen Hughes

Now, fast forward a little bit. So, Brian, when you came in, tell a little about the story of like you've had a little bit of SAS background, but...

00;05;08;16 - 00;05;09;21

Brian Klein-Qiu

Actually it's opposite.

00;05;09;21 - 00;05;10;12

Stephen Hughes

Oh it's the other way around.

00;05;10;14 - 00;05;12;15

Brian Klein-Qiu

I have a lot of SAS background.

00;05;12;17 - 00;05;13;04

Stephen Hughes

Right, right, right.

00;05;13;11 - 00;05;36;02

Brian Klein-Qiu

00;05;36;06 - 00;05;58;15

Brian Klein-Qiu

00;05;58;15 - 00;06;01;02

Stephen Hughes

And maybe a class. And maybe do a lot of Googling.

00;06;01;02 - 00;06;32;08

Brian Klein-Qiu

00;06;32;11 - 00;06;38;26

Brian Klein-Qiu

So literally from day one, I'm able to plug in and contribute to what John's been handling for I don't know what, the last six months?

00;06;38;29 - 00;06;53;12

John Allison

Yeah, I mean, I would say not only contribute to it, but just like make it better really quickly. Like I remember some of that code you were sending to me and as we were working on and I was like, wow, this is a really good idea. Like, I hadn't thought to do it this way.

00;06;53;12 - 00;07;13;04

John Allison

And, you know, and I think probably the thing to like underscore there is for data analysts, ChatGPT is really quickly becoming a must have in your quiver. It might be the quiver. It is so important for data analysts in terms of like in the toolbag you reach into when you're doing your job.

00;07;13;07 - 00;07;34;22

Brian Klein-Qiu

Oh absolutely. Health care policy, you are working with really a diverse array of languages. But you kind of develop an expertise in one and you kind of just have to make do with whatever knowledge you can get to parse through all the other projects. For example, claims are going to be mostly in SAS files. If you do data and analysis or processing

00;07;34;25 - 00;07;53;04

Brian Klein-Qiu

a lot of people like to use Python for that. But if you look at the claim prices for like how consumers actually prices claims, all those source files are in Java. So if you can have a tool like ChatGPT that can help you process these very different languages easily, that's an incredible boost to productivity.

00;07;53;07 - 00;08;12;22

Stephen Hughes

And there's a couple of important points here. And one, to extend the metaphor of languages out. You know, effectively you have a Rosetta Stone, right? That's understanding these languages. We have two different languages we started with, so we're extending the metaphor out. We've got, you know, Mandarin and English. Now if you're adding Java on top, you're adding, okay, French, right?

00;08;12;29 - 00;08;23;20

Stephen Hughes

We keep adding more and more languages, more and more expertise, more and more time. And someone who's not familiar with something...so ChatGPT effectively was acting as that kind of universal translator.

00;08;23;27 - 00;08;43;17

John Allison

What ChatGPT does is it says, okay, so maybe maybe you weren't as strong at learning programing languages or Java or whatever it is. ChatGPT's going help you get there in a way that I would have said... you talked about weeks to get caught up. That's because you had some experience with some of these languages before.

00;08;43;19 - 00;09;07;19

John Allison

For someone who doesn't you're talking years, right? And so that's where most of the data analysts that would be listening to this or that are if you're a health system C-suite, that would be working for you. Fifteen years ago, ten years ago, even five years ago, it probably took them years and years and years of not only learning the language but getting really comfortable in it, learning how to use it to become the data analyst that they are.

00;09;07;20 - 00;09;09;00

John Allison

It takes years.

00;09;09;02 - 00;09;27;27

Stephen Hughes

00;09;28;00 - 00;09;55;19

Stephen Hughes

You know, there's the crisis, right? We've now gone to this idea of it's not taking jobs. It's making individuals better at their jobs. It's you know, like in our case, like in the AHA's case, rather than having to hire two more Brians or a John and two more Brians, right. I'm now able to take Brian and John and upskill them to a point where I'm getting much more productivity out of two individuals by upskilling.

00;09;55;25 - 00;10;21;13

John Allison

I think with something like ChatGPT that years of like building up this experience and skillset to be able to tackle the problem has really been ameliorated. And to the extent that if I'm an analyst and I'm working at the AHA, which I am, and there's a question, a big question that we have something we want to answer on a topic that maybe not all people have talked about yet, and there's some data out there, but it's just huge.

00;10;21;14 - 00;10;47;18

John Allison

We don't understand it. Instead of me thinking, well, that's a waste of my time. I shouldn't be actually trying to answer that question. Instead, I'm thinking, okay, with ChatGPT, I feel reasonably confident that I can take something that's currently incredibly difficult to understand and I don't have a ton of background in. And I can still ask this big question from the data and reasonably expect to get an answer and not take years to do it.

00;10;47;20 - 00;11;16;04

John Allison

That's just hard to underscore how game-changing that is, at least from my perspective. Whereas before I would have asked a really good question about my data, like what is what is driving this this trend that we're seeing in health care and hospital utilization and things like that. And while the datasets might have been available to me, a lot of them aren't, by the way, but the ones that are publicly available, I might have just shied away from asking questions like, well, I don't have the next ten weeks to devote to this.

00;11;16;06 - 00;11;18;25

John Allison

And nowadays I would say that's a week at most.

00;11;18;27 - 00;11;33;18

Stephen Hughes

But then the other part of which I think again, would resonate with with most health systems is this idea of like they've got a lot of data, that's not the problem. They've captured all kinds of data, it's just in all different places. You have a lot that's in the HR. You have a lot that may have predated the HR.

00;11;33;20 - 00;11;57;06

Stephen Hughes

That you've been hanging on to for, you know, for years for regulatory reasons, but are you actually making a use out of it to create, again, better outcomes, other business opportunities? Who knows? And then sometimes again, there are disparate sources, right? What if you acquired another health system that has a different EHR. You acquired a practice or partnering with a practice, and they've got again, it's a different EHR, or god forbid, paper.

00;11;57;09 - 00;12;18;00

Stephen Hughes

How am I bringing this all in? Or what tools am I bringing in? And in most cases, those answers are going to be, I have a very hard time doing that internally, right? I'm going to have to you know, I'm going to have to either move all of this to some cloud-based service which costs, security risks associated with it.

00;12;18;02 - 00;12;40;09

Stephen Hughes

Time. When am I going to get value back out of this as opposed to, hey, this is my data and now I can take a very narrow approach to my data and solve problems and get value out of my data with my staff inside a health system, right. Which is going to cut the security risk problem. The cost concerns, the do I bring in consultants?

00;12;40;09 - 00;12;49;18

Stephen Hughes

Do I you know, all these other issues? I can actually start solving these problems internally because I have tools like, you know, GPT.

00;12;49;25 - 00;13;26;02

John Allison

Probably something to underscore is if you're looking for an application of AI and you're like, well, I'm not a huge health system. I don't have these resources to spend on these more clinically-based AI applications or maybe even some sort of, you know, chat bot thing, things that maybe are more business-oriented, consumer-facing potentially. A really low, an area of a lot of low-hanging fruit for the use and practical application of AI tools like ChatGPT is going to be data analysis, frankly, and some people might find that boring, other people might find that exciting.

00;13;26;04 - 00;13;52;08

John Allison

I think that's important to contextualize, it's something that ChatGPT does pretty darn well compared to a lot of the other things that it does. It's going to help you think through data problems comparatively pretty, pretty well. And I would say if you're looking for a place to kind of grow your hospital or your health system, whatever it is that your business you're working in, it'd be to be thinking about how to apply these tools to data problems.

00;13;52;08 - 00;14;10;08

John Allison

If your analytics department, your data analysis teams, the people that are helping with decision support, business intelligence, whatever it is, that's the group out of anyone in an organization that I would think would be leading the charge, in a very I hate to use this word because we're in D.C., but in a very grassroots way.

00;14;10;15 - 00;14;15;28

John Allison

You're not being told to do it. It's very organic. This is making me better at my job.

00;14;16;00 - 00;14;45;19

Stephen Hughes

And yes, it starts in data. But I think that, you know, your insight there, John, is good. And I think it's applicable really to even outside of data analytics, right? The idea that the use case came out organically by need. So it's, and I think if there's a lesson there and there's marching orders to then take out, you know, really any level of management in the health system that maybe listening to this all the way up to senior executives and board members is, worry less about trying to shoehorn a solution into the organization.

00;14;45;21 - 00;15;01;19

Stephen Hughes

It's go down to the practitioners, right? Go to your people in the front lines, go to your experts and ask them about how they could use it. And that's where I think you're going to get the most effective, you know, examples and the most effective tools are going to emerge from that.

00;15;01;22 - 00;15;11;13

Brian Klein-Qiu

It's true that this is an incredible tool and you can solve so many issues and increases productivity so much in the hands of somebody who is competent and somebody who is...

00;15;11;13 - 00;15;34;16

Stephen Hughes

In the hands of someone who is competent. Right. You have to have a baseline, competent person to start with that you're upskilling as opposed to...what you're not doing with ChatGBT or really with AI in general. And this is you could extend this metaphor out to the kind of the absurd level of you're not going to turn somebody who didn't go to medical school into a radiologist because they're using an AI tool to look at diagnostic images.

00;15;34;16 - 00;15;55;28

Stephen Hughes

You get you start with a radiologist, add the AI on to that and they get much more productive and much more effective at picking up abnormalities in a mammogram. Just like you take someone who's got good or at least a basis with understanding data languages and good analytical skills, add AI, they're going to be much better at doing decision support.

00;15;55;28 - 00;16;01;03

Brian Klein-Qiu

Exactly. And the criteria for evaluating who's good at analysis has changed with AI.

00;16;01;06 - 00;16;17;21

Stephen Hughes

I'm glad you brought this up, too. So we're not just sounding like that we're a bunch of just ChatGPT and AI cheerleaders, right? There's always a downside with technology. So let's talk a little bit about the quality measures. So how do we do that with ChatGPT and data analytics? How do you apply that safety criteria?

00;16;17;21 - 00;16;19;10

Stephen Hughes

What are the tests or what are the outcomes?

00;16;19;10 - 00;16;40;22

Brian Klein-Qiu

Yeah, that's a good question and that's you hit on a really, really crucial thing because that's something especially in fields like health care systems or health policy in general, there's not a lot of that kind of thinking. A lot of people a lot of organizations, a lot of agencies are not concerned with how do we know that these numbers are right, how we know that these algorithms are correct and we see how many mistakes CMS makes all the time.

00;16;40;24 - 00;17;01;19

Brian Klein-Qiu

It's not limited to CMS. All agencies and companies do that. But one thing that I find shocking and that we need now more than ever is there's no emphasis in the interview process for a lot of these places on how do you sense check your numbers in a project. It's all what are the results of the projects? Give me examples of projects you've done and it's never

00;17;01;26 - 00;17;25;13

Brian Klein-Qiu

how did you know that your project was correct? Explain to me your benchmark process. Explain to me how you were able to contextualize the numbers you got. The answers to those questions are much more indicative of how meticulous someone is and how analytical someone is. Than them being able to list five languages on their resume and say, okay, I created this algorithm and it gave me this many charts.

00;17;25;20 - 00;17;25;27

Brian Klein-Qiu

Here you go.

00;17;25;28 - 00;17;40;08

Stephen Hughes

So would you be saying in a way that the way to actually make sure that you're you're using GPT safely and effectively is to actually double-down on your people hiring process.

00;17;40;10 - 00;17;43;13

Brian Klein-Qiu

Or instill a culture. Instill a culture that is much more...

00;17;43;13 - 00;17;44;16

Stephen Hughes

Get very human centered.

00;17;44;16 - 00;17;46;01

Brian Klein-Qiu

Yeah, right, right.

00;17;46;06 - 00;18;01;04

Stephen Hughes

Interesting. So to ensure quality it's make sure that you're being careful about your your hiring practice,s right, and instilling a culture of quality and you know high levels of ethics in those who are using the applications.

00;18;01;11 - 00;18;15;14

Brian Klein-Qiu

The number one thing you can do if you've already hired a person to improve the culture of checking the quality is just literally ask, how do you contextualize these numbers with other entities' numbers? Can you think outside of GPT, are you right externalizing everything you do?

00;18;15;19 - 00;18;39;15

Stephen Hughes

I love that. To actually ultimately control quality in GPT or tools like GPT think outside of GPT and in the end AI is a piece of technology. Really what we should be thinking about in health systems is a thoughtful application of technology. This is just one more thoughtful application of technology. Overall, I think it's connecting back to what Brian was saying, which again connects us back to the human-centered nature of this.

00;18;39;21 - 00;19;02;05

Stephen Hughes

You know that ultimately the quality control check for, you know, use of AI in data analytics and decision support is do you have a culture of quality, right? Do you have a culture of peer checking around what you're doing? You know, ultimately to get the most out of AI, across the system, across health care, is encourage a culture of innovation and creativity, right?

00;19;02;05 - 00;19;13;06

Stephen Hughes

Know the tools are out there. Encourage people not to be afraid of it. Go look at it and suggest things. You be amazed what you might find. Well, guys, I think that's all the time we have. I really appreciate it. John, Brian, thanks so much.

00;19;13;09 - 00;19;14;16

Brian Klein-Qiu

Thank you. Pleasure to be here.