This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

 Today, on this week in health it, I think electronic health records are the perfect example of why we need more user.

The early versions, um, really did not use user-centered design methods when they designed their systems. And once you have poor usability baked in, it's really hard to recover from that.

Thanks for joining us on this week in Health IT Influence. My name is Bill Russell. I'm a former CIO for a 16 hospital system and creator of this week in Health IT a channel dedicated to keeping health IT staff current. And engaged. Special thanks to our influence show sponsors Sirius Healthcare and Health lyrics for choosing to invest in our mission to develop the next generation of health IT leaders.

If you wanna be a part of our mission, you can become a show sponsor as well. The first step. It's to send an email to partner at this week in health it.com. Common question I get is how do we determine who comes on this week in health it, to be honest, it started organically, it was just me inviting my peer network and after each show I'd ask them, is there anyone else I should talk to?

The network group larger and larger, and it helped us to expand our community. Of thought leaders and practitioners who could just share their, their wisdom and and expertise with the community. But another way is that we receive emails from you saying, Hey, cover this topic. Have this person on the show.

And we really appreciate those submissions as well. You can go ahead and shoot an email to, hello at this week, health it.com. We'll take a look at it and, uh, see if there's a good fit to bring their knowledge and wisdom to the community as well. Today we are joined by Paula Edwards. Paula is who I called when I was setting up data governance for St.

Joe's back in the day. Paula Edwards is a consultant analytics and data governance practice leader for informatics and affiliated professor at the Rollins School of Public Health at Emory University in Atlanta. Paula, welcome to the show. Thank you. I'm excited to be here. It's great to see you again. Are you really excited to be here?

I mean, you just came back from Italy. Are you really excited to be here? I'm, believe it or not, I could talk about analytics and data governance all day long, so this is right up my alley. That's not what you do on vacation, is it? No, I don't look at the computer when I'm on vacation. Well, that's good. It's interesting with this work from home thing, I've been talking to a lot of people 'cause they're not used to this kind of environment and some of 'em are talking about the fact that they just feel like they're being run into the ground.

They're working so many hours. I'm like, but if you do this, like we've done this for a while, you just have to learn to. To draw the barriers and, and live within those barriers. You do, you have to have balance. And so that's why I have my home office set up. And when I shut down at night and leave my office, I'm done with work.

So that's how I manage it. Yep. And I have a little sign at my door that says leaving the office when I go out , it's just a reminder that the family's on the other side and, and they expect me to be leaving the office. So that's what I'm doing. Paula, I'm, I'm really excited to have you on this show. As I said, you, you were the one who worked with me on data governance at St.

Joe's, and we had a lot of stuff to do. Back in the day, you felt like you were out in Southern California more than you would in in Atlanta. You were out there a lot. But here's where we're gonna start. I'd like to unpack your. LinkedIn profile, just a little. You have a couple things which I think are really interesting.

You have analytics aficionado, data governance coach, healthcare data translator, human-centered design advocate. So let's start with your day job and then, and work through these. Tell us about informatics and the type of work that you do. So Informatics is an advisory company and we work with large health systems and community hospitals and other healthcare providers to make sure they're getting the most value out of their technology and data investment.

So we have a range of services and I lead our analytics and data governance practice, and that's where the data governance coach comes in because, uh, that's a lot of what I do. So I work with health systems, like when we came and worked with you at St. Joe's on . Analytics strategy and roadmap, data governance strategy and training and coaching.

Um, and helping people look at their portfolio of analytic solutions and how they've got the people process side of how they provide analytic services set up to really help them get the most value out of their analytic staff as well, and really make sure they're teed up to be able to really innovate and drive change through the organization using data.

I'm, I'm curious what you're seeing these days. What does data governance look like in healthcare? Is it mature or, I mean, would you rate it as mature or would you rate it as still fledgling and progressing? It depends on the organization at this point. Most of the, the places that I've been, they've at least made some effort at establishing some form of data governance.

They either had a data quality problem that was really driving 'em crazy, or they were having trouble to figure out where they should be focusing their analytics resources. And so they made an effort, but then it sort of lost steam and they've had a hard time really getting it to take traction and really be able to use it to provide the level of data asset management the organization really needs, especially given how important data is to running a healthcare organization these days.

So a lot of times when we come in, we're helping 'em look at what do you have already in place that we can build and leverage, and then where are some of the gaps that we need to help, you know, coach you and, and give you some ideas for how you can make things more effective. I'm gonna come back to some of those terms.

Data asset management. I keep hearing the term data supply chain come up. Uh, we're gonna go into that in depth, but I, I want to finish working through, you have analytics aficionado. Is that just part of the game of being in the data world? Uh, you appreciate good analytics when you see some. I really do appreciate good analytics.

I love a good visualization, like a really well thought out visualization that tells a story and is understandable and actionable by the intended audience. And I love seeing a lot of things that are happening these day with all the AI and machine learning and the opportunity to really use that to impact how care is delivered to me.

It's just an exciting time to be in analytics. It's interesting. I used to always try to drive that home that we are telling stories with the data and it, it is important to know what story you are telling or are not telling by the visualization that you put in there. Do, do you have some examples of just some of the things that you've seen through the pandemic?

I would imagine there's a ton of great work that's been done through the pandemic. There's been a lot of really great work that's been done through the pandemic. As I think about some of the things that I'm the most thankful for, from the really innovative cutting edge stuff, what they've been able to do with AI and machine learning to really speed up the vaccine development process is just fantastic.

So we wouldn't be sitting here with vaccines in our arms if that hadn't happened, and normally it takes years to do that. But from a more. Broad, I guess, application of analytics in a pandemic. One of the favorite things that I went back to over and over was one of the maps of the us. I know everybody started out looking at the Johns Hopkins map.

But I think it's a good example of why metrics that you choose matter. Because during the early days of the pandemic, everybody was watching John Hopkins so they could see how the pandemic spread and where was it now. But because it was based on cumulative totals of incidences, you didn't have any idea of what the transmission rate was immediately.

So. You knew, like I live in Georgia, a lot of people in in Georgia had gotten covid, but how many were getting it actively now? And so there's a group called Global Epidemics that published their version of the map that focused on spread. So you could see at the county level. What is the seven day moving average of new cases total and new cases, um, per a hundred thousand people?

So you could say, what's the transmission rate where I am, and based on that, do I wanna go to this event or go to this store, or go to this restaurant and really make your own decision? So it did a good job of just making it actionable for an individual. It's interesting you make that distinct. It's, that's a great example of telling two different stories with the data.

I had a bunch of people call me and say, the pandemic's bad here. And I'd be like, well, look at, because you can break the Johns Hopkins ones down and drill into it and get, get some of that detail and say, but, but look, over the last 30 days, you're actually trending down. And you can look at the CD, C and see some of those numbers and, and say it's heading in the right direction.

But if you just look at the cumulative, it tells one story. Whereas if you're. Time series data, it's something completely different. It's the definitions around data. The data you choose, the quality of the data, all that stuff, and the timeliness of the data. All that stuff matters. But anyway, I get ahead of myself.

We're gonna go into all those things. Healthcare, healthcare data translator implies the need to translate something where, where does this skill come into play? So to me in analytics, it's absolutely critical to have some of these healthcare data translators. And if you think about historically in.

Healthcare it especially, but also in analytics 10 years ago or even less than that, you had the technology people who spoke their language. You had the business and clinical people who spoke a different language, and so when they were trying to communicate requirements and what are we trying to do and what do we need to build, they were talking at each other but not really communicating because they just fundamentally spoke different languages.

And so translators like me. Know enough subject matter expertise about the business and clinical side of the operation and what they're trying to accomplish and what's gonna be valuable to them. And know enough about the data and technology and what they're capable of and what's possible that we can help translate between those groups to really make an impact.

So help the the business visualize what is possible with the analytics tools and the data sets that we have, and how can we really use it to change how we care for populations, and then work with them, with the technology people to say, okay, what do we need from a platform and a data integration and a data standardization perspective to be able to deliver on that promise and get some value out there to our providers and the populations we're caring for.

Fantastic human-centered design advocate. What is human-centered design? Why does it need an advocate, and where does it come into play in healthcare? . So my PhD is actually in human integrated systems, which is about how do you design systems that have. People and technology working together to accomplish a goal.

And human-centered design is what a lot of the methods and tools that are used in that space to make sure that you have your end user front and center at every stage of trying to design your system so that you make sure that it's gonna work efficiently and effectively for the intended user. So a lot of what you hear now with user experience has a lot of its foundation in human-centered design methods and a lot of.

What you see in Agile methods is actually very much founded in some of the user-centered design methods. It's how do you get your user and those key stakeholders involved early and often. Through the design lifecycle so they can provide input, tell you where you got it, right, where you got it wrong.

Really provide valuable insight into the context in which, uh, the system is gonna be used. And I think your question about why is it important in healthcare, I think electronic health records are the perfect example of why we need more user-centered design and healthcare because. The early versions, uh, the early iterations of the current generation of electronic health records really did not use user-centered design methods when they designed their systems.

And once you have poor usability baked in, it's really hard to recover from that. So that's why some of the usability is so bad there. I'm gonna get into data in just a minute, but I'm gonna put you in a time machine. We're gonna go back and sit down with David Breer and his team before they, they put into place the meaningful use.

Whatever it was. I, I was gonna say doctrine, it's not a doctrine but meaningful use program. We're sitting there, it's just, it's close-knit group, four or five people. And what, what are we gonna say to 'em to say, look, let's make sure that we do something here that changes the direction of this. Otherwise we're gonna end up with where we're at today, which is essentially.

Systems are getting better. They are getting incrementally better, and they really need to take a, a. Dynamic leap forward in terms of usability and uh, design around the clinician workflow and those kinds of things. What would we be saying back then? Or did the problem start even before that? I think the problem started before that.

I think it's going back to those early EHR developers to say, you guys need to have doctors and nurses and. Patient registration clerks as part of your design team that are really helping you understand how does a patient flow through our health system and what do we need to do to make it easier on the patient and make it easier for the people who are caring for 'em.

Paula, I did an EHR project. Have you ever put clinicians in a room together and try to get them to agree on what the workflow should be? It's really hard. I mean, what's the trick? How do you drive through that? Go back to the beginning of our conversation. You need healthcare translators that can help facilitate those conversations and really get 'em to focus in on what are the commonalities and how do we build to the standards.

Um, and there is a lot of. A lot of people in healthcare historically have wanted to do it their way, and that kind of dovetails into our data governance conversation. You have to standardize to get efficiency and quite frankly, good user experience. And so you have to be able to facilitate those conversations to get everybody to a workflow that that works well for everyone, even if it's not necessarily their ideal workflow.

You gotta get a little bit of give and take in there. This isn't a, uh, paid for spot for informatics, but I'm gonna give them a shout out because you guys did did great work for me. And the reason I'm gonna give 'em a shout out is 'cause I'm gonna hit you up for some free consulting right now and we're gonna share it with a lot of people.

Um, so we're gonna talk about data definitions, getting to those standards and those kind of things, and. Let's start with what are the characteristics of a high functioning healthcare system with regard to data and analytics? Well, there are a a few characteristics, I think . Number one is culture. You have to have a data-driven culture.

And for me, that culture starts at the top. So I include your leadership in that culture statement. You have to have leaders who are data-driven and who are gonna invest in analytics. And uh, that means both capital and people's time and analytics, and they have to hold all of their direct reports and the organization as a whole accountable for actually using information to inform decisions.

You can't just say, oh, we're gonna put out a balanced scorecard and check the box. You actually have to hold people accountable for their performance on that balance scorecard. And if things are going in the wrong direction, they have to be able to answer why is this happening? And have data to back it up.

So really truly having that data-driven, to me, that's number one. You can't be high performing without that. Obviously, you have to have data platforms. And this, when I think about platforms, it's the platforms to integrate the data and curate it to the degree that is necessary for how you wanna use. And has a range of analytical tools on top of that to meet your target end users.

So for a less mature organization, you might start out with those tools very focused on like information visualization and dashboarding and reporting. But for more mature organizations, you've gotta have those more advanced analytics tools so that really power user analyst and data scientists can really explore and innovate and, and do some really exciting things, um, with the new generation of

Machine learning tools, data prep tools, there's a lot of things people can do with data these days if you give them the right tools and, and have the data available. When we talk about healthcare, sometimes we talk about it like it's one big homogenous thing, but you know, the academic medical center, the community health system, the large IDN, the, the, the two hospitals systems, community hospital, these are very different.

Obviously, the analytic strategies will be different based on budget. So let's do this. Let's start with, you know, the, the two hospital system. They have a set of tools that come with the EHR and a set of tools that come with the various thing. Uh, how do they get started if they're that small of a system?

I think you really do wanna double down on the tools that are available through your core EHR vendor and really take advantage of that reporting content and the dashboarding content that you basically bought with your EHR. With the smaller hospitals. You've got less talent, so you're gonna have less opportunity to do things your way.

So you're gonna have to align more on industry standard metrics, industry standard definitions to the degree that you can, because quite frankly, you just aren't gonna be able to afford and get the talent to be able to do it your way, which in academic medical centers, they always wanna do it their way because their definition.

Is, is better for them than whatever the industry standard, right? To a certain extent, they're driving the industry, the academic medical centers, , they really are, they're, they're thinking down the road and quite frankly they have the staff and the academic knowledge and things like that, and they should be doing those things to kind of push the industry forward.

So I think you really do have to level set based on the realities of your budget and the sophistication of the organization and. Just the availability of analytics talent in your area. Which way are we seeing the academic medical centers go? I mean, clearly we want to consolidate and we wanna standardize, but there's a lot of self-service needs across academic medical centers.

There's a lot of people doing individually funded studies and those kind of things. Are we still pulling everything in and then creating. Ways for them to access that data? Or are we still finding silos of data out in the academic medical centers? Oh, there's lots of silos of data out there. A lot of the academic medical centers that I work with are doing a better job of building platforms that bring the data together.

But what you're seeing now is more, let's, for example, in a cloud analytics environment, let's land it somewhere where everybody can get access to. Let's provide playbooks of how you link different data sets. So they're having this sort of either uncurated or lightly curated data available more broadly.

Because you can do that much faster and give people, you know, faster time to data. And in an academic medical center, a lot of the people who are doing a lot of the modeling and research, they want that raw data. Like they don't want it curated. But when you get to the operational people who are taking care of patients and are really running the business of, of care delivery in those academic medical centers, they need more curated data to be self-served.

Because if you're a . Nurse manager on a unit, you don't have time to go look at raw data. You need to be able to click and pull up a dashboard to see your metrics and see if something's trending in the wrong direction that you need to take action on. So you have to have different levels of analytics for the different self-serve audiences in your organization, even if you're an academic medical center.

So let's, let's push a little deeper. Well, let's start at the top. Let's start in a data governance meeting. Who participates in data governance and what goes on at a good data governance meeting? So. My dog, Lula is very excited that we're moving to a data governance. great. She's, she loves it. When I talk data governance, if I think about it, there's sort of two layers of data governance that every organization needs.

There's an executive level data governance, and then there's a more tactical kind of data stewardship level of data governance. And the people who need to be at the table and what they need to be focused on is different at those two levels. So at the executive level. This is where they should be making strategic decisions about what is important for us as an organization to do with analytics.

How are we gonna allocate our enterprise analytics resources, whether that's capital dollars or staff, in order to get the most value from our portfolio of analytics and, and data assets. So that means you've gotta have executives there and you need to have a cross-disciplinary set of executives there.

So you need somebody from quality finance, IT compliance risk. I'm hearing a lot of organizations these days that one of their big drivers for sort of revamping governance is all sorts of questions are coming up about third party access to data. Somebody wants to buy the healthcares data or there's an IT vendor that wants to partner to develop some cool new machine learning algorithm.

Do we give them access to our data? What do we expect to get in return from it? Are our patients gonna be okay with this? If we give this access to this data? Those are decisions that executives need to be making, not a project manager in IT who gets an extract data request. I can, I can only imagine.

Actually, we could spend all day on that providing data to third parties and how to protect that data, how to anonymize that data, how to ensure that. You're not allowing your data to just run loose in the wild. So I like that meeting. I assume clinical's part of that as well. Clinical absolutely needs to be part of it.

And realistically, you need somebody to represent the acute hospital side and somebody from the physician practice side. 'cause a lot of times the considerations are very different between the two, the different settings of care. And if you have, if you are a large integrated delivery network that has home health or things like that, you'd probably want somebody who can at least talk to those needs and considerations to.

So who's at the tactical meeting and what goes on there? The tactical meeting is gonna be more what I call the data steward level. So these are people who really know and use the data on a regular basis, and they're either gonna be analysts. People who are over the departments, they're actually creating the data.

So a director of revenue cycle and data owners, potentially I, it is gonna be a mix depending on exactly what you wanna focus on. A lot of organizations when they're starting that data steward layer have problems with data standards. 15 different definitions of length of stay, 12 different ways to count visits.

We are getting conflicting data we have to do about something about this. Let's get a group together to decide how are we actually going to count these things and what are the metrics that matter to our organization. And so these are the people who can really say, this is how this group has always counted it.

The other group says, here's how we've always done it. And they have to come to consensus on how are we gonna do it as an organization going forward. Are you saying an organization can get to one definition of length of stay? No, they probably cannot, and that's okay. It's okay As, and my rule is always, if there is a valid business or clinical reason to have two or more definitions of length of stay, that is fine, but we need to know what they are and we need to call them something different.

So you can't call 'em all length of stay. At one of the organizations I worked with, we ended up with length of stay, ended up being the clinical length of stay. That was actually the difference in time between when they showed up on our unit and when they left. That was the length of stay because the clinical operation said that's how long we have to care for this patient.

So we need to know the duration. I need to staff finance wanted to count length of stay as heads in beds that they got to bill for. And so finance ended up having to call that build patient days. And so we solved the problem by saying, okay, there's two needs. We're gonna have two metrics, but we're gonna call 'em something different and this is how we're gonna do it going forward.

Isn't there a, a regulatory definition or ACMS definition for length of stay though, that you have to track? There is, and we had that one too. I know. 'cause it has all these weird nuances. Like if somebody comes back after they got discharged and so there was ACMS length of stay also. So they got to, they, but that's what, that's what goes on at that meeting.

That's what goes on at that meeting. So that's why you don't want your executives in that meeting because they'll check out and never come back. But you don't want that group deciding. Whether or not you're gonna partner with Google on some AI initiative, but that group could use the executive group as an escalation path.

Exactly. If they come to an impasse, and that's typically what that executive group is, is an escalation path for things like policies around who gets access to what. If, if the data steward group can't get to consensus on a standard definition and they need somebody to basically be the tiebreaker and tell 'em how the organization wants to do it, they can escalate that kind of stuff there.

When you're setting up these data governance teams or these groups, I. We used to always have charters for these. Do you recommend a charter? And is there anything specific you would put in those charters? I do recommend a charter and really, and I'm seeing some organizations getting away from charter and more defining a data governance policy, but it effectively has a lot of the same stuff that a charter does.

It's just saying. Here are the roles that are gonna be part of our data governance program. Here's what the different people that are part of our Data Go governance program are responsible for and accountable for. So who gets to decide on budget and resource allocation? Who gets to decide on source of truth and data standards?

Who gets to decide on third party release of data and internal access to information? I. And really having those roles and responsibilities and accountabilities really clearly defined, so everybody's basically operating from the same playbook. Interesting. So, uh, policy-based approach to it and a charter based approach to it.

Our, our charters were pretty simple. It just said, this is what the group is, this is the makeup of the group. Here's what their, their core responsibilities are as a group. That's essentially what that said. We still had policy documents that sort of drove how we approach data governance. Yeah, and the governance groups are gonna be responsible for that policy.

And, and so it's really like how does the organization want to drive their data governance program? I'm for less documents and red tape where that's feasible. So we've kind of, in a couple of cases, turned a data governance policy into sort of mini charter for different groups so that it's all there together and everybody's got the same understanding of what the other groups are responsible for too.

I want to ask you about organizing analytics. I've seen so many models, it's kind of mind boggling at this point. I've seen a chief data officer have the whole thing. I've seen the CIO have it. I've seen the CIO not have it, and it's, I don't know, sort of in a, in a weird space within the organization, not necessarily to a data person per se, but.

Just another group. It's interesting to me how many different models I've seen and I, I haven't really put my finger on what's the most effective. Is there something that's emerging in terms of either principles around how we organize this that works the best? I think so. I mean, when you get into the details, there's a, a very wide range, and quite frankly what will work in an organization is very culture and context dependent.

But I'll tell you what we know doesn't, a fully decentralized model does not work because that's why so many organizations have ended up with silos of information and conflicting definitions. It's just very inefficient and it's not good for having enterprise view of data. If you go to the other end with centralized, that's not great either because once you centralize everything, you can never keep up with demand and, and everybody's always complaining that how come you did their stuff first and I didn't get mine?

So the emerging best practice is really a hybrid model where you've got certain core resources that are centralized so that you get economies of scale, and you can have those resources really focused on that. Enterprise platform build. So what's the 20% of the data that everybody is using that we need to be using a common language and a common platform on?

And then you still have embedded analysts, especially in some of the really analytics heavy departments like finance, like quality, et cetera. Yeah, I, I'm, I'm thinking through how do you take care of that rogue area? The ambulatory group is going off and doing their own thing. They're buying a new tool because essentially you have this hybrid model, you're providing some services.

But what, what you're saying by that hybrid model is essentially you're gonna have headcount out and all the departments, the departments are gonna start hiring analysts. The analysts are going to be working with their constituents. They're gonna identify . Analytics they need for a clinically integrated network or for population health or whatever it is.

And then before long, all of a sudden you have this group that's growing into its own analytics, uh, arm of the organization. How, how do you keep that from happening? It's a constant battle. It really is a constant battle. There's a couple of different strategies. One to your point about going out and buying their own tools.

That's where your executive level governance needs to hold people accountable because they should be the clearing house for purchase of any new analytical tools like that. Budget shouldn't get approved until that group has signed off on it, and so they need to hold the various departments accountable for not going rogue when it comes to technology purchases.

And that's where that whole leadership thing in, in part of my, this is what a high performing analytics organization does. You've gotta have that leadership that's gonna say no. The other thing is you really need to concentrate in your centralized, in your hub of enterprise analytics capabilities on showing them that they can be more efficient and more effective by working with you than by working on their own.

So being able to leverage all of the data sets that are available in the enterprise platform, all of the tools and the subject matter experts in those tools that should be in your hub. If you can say, if you'll work with us and partner with us and be engaged in the data stewardship processes and collaborate with us on projects, you can draw on all of our technical experts and our data experts, and we're gonna accelerate your timeline to insight and your timeline to data.

Then you can get 'em to be more of a partner where they're working with you instead of just going totally off on their own. And that's what you want. 'cause really, eventually what you wanna get to is where those departments that have. Their own little analytics enclave are sort of a, a prototyping and an experimentation platform.

Down in the weeds kind of thing. I, I know that there was always this thought that like a Google was gonna come in, take all the health data, normalize it and present us this really phenomenal interface. And they're doing something right now along those lines, but they've also written a paper that essentially said, no, you gotta clean up your data.

The best tool in the world still can't make sense of your data if it's this messy. The data is still messy, and I think in at its very nature, healthcare data and especially clinical data is just messy. It's unstructured, and it's being captured by a lot of different people, about a lot of different patients that are just a highly, very population.

So it's always gonna be a little messy. I think whenever I have conversations about data quality, what I like to focus on is data quality that is fit for purpose, and so to the Googles and others of the worlds. There's sort of two questions that I would have. One is, given the quality of the data that we have, what can you do that's of value?

What is it? What is the purpose it is fit for? Because I guarantee there's something valuable they could do with the data that they have. Yeah, there is, and I brought that up as an example and they just came out with something. I have this story here. I forget what it is. They just came out with this, this tool that essentially sits on top of this clinical data across a bunch of different EHRs, and it provides value.

I mean, it, it provides a single interface that's searchable for clinicians and it, it does that kind of stuff. So I'm not, I'm not saying they weren't able to create value out of it, but I am saying they also wrote a paper, a peer review, uh, kind of thing they put into the, uh, academic side saying essentially, Hey, here's the problems.

We've looked at a lot of healthcare data now and here's what needs to happen. I think that's what they should be doing, is providing that feedback loop to the people who are capturing the data to say, if, if you want to enable these future use cases, then here's what you need to do to address the data quality problems.

But I think. Making the data quality in healthcare better goes back to one of the things we talked about at the beginning, which is that user-centered design to get better quality data. We've gotta make it easier for the data to be captured cleanly, because right now with the current generation of EHRs, it's really time consuming to capture really good quality data in an EHR, and we've gotta make it easier.

When you're designing an application, you almost have to start with who's it being built for and what's its purpose. And I hear this all the time. The EHR was designed to capture billing and because it was designed to capture billing, and now we're trying to do public health things with it. We're trying to do population health with it.

We're trying to, we're trying to do a lot of different things with it, but that wasn't its original purpose. So now we're retrofitting this whole thing. Uh, to capture better clinical data, more discreet data around the, the clinical event so that we can start to provide value that we need through a pandemic.

And so is it a problem of definition of what is this thing here for and what's it, its core mission as an EHR? Well, I mean, I think that's one way to look at it. What I am really interested to see and excited about the opportunity for is there's a lot going on in the world of natural language processing, and there's a lot of talk about

Amazon Alexa and other ambient listening devices, capturing and doing some of the documentation in the clinic room. And to me, those, if, if they do them right, could just really have a good opportunity to make the data capture easier for the clinician, but capture more useful data and better quality data that can be used for all those downstream uses.

So. Do we wanna spend a lot of time and money going back and trying to retrofit how we capture things in the EHR today, or do we wanna layer some of these newer technologies on to find a better way to capture more useful, cleaner data in an easier way? I hear that and, and, and I'm having flashbacks of you saying.

Layer some things on and I'm sitting there as the CIO going, yeah. Every time I have to put a new tool on, it's, it's a million dollars and I have to hire, you know, five people to feed that tool. And there's another five people times, you know, Southern California, a hundred thousand dollars a person.

There's half a million dollars to maintain it every year. And we haven't even done maintenance or anything to that effect. Health system is the one that has to do that . I mean, could the EHR vendors start building this into their products? So it's part of what you get with their core EHR, they call it modules and they increase by price.

Anyway, we talked about the pandemic and you, and you have public health as as part of what you teach and talk about. So let's talk a little bit about the impact of the pandemic on the way we interact with data, the way we view data practices in healthcare. Will it be fundamentally changed as a result of this?

I think it should be, I think Will is the key question. I think that's gonna require some investment in our public health infrastructure. And quite frankly, I think we need to rethink fundamentally how we're capturing and exchanging data for public health is really interesting. Providence St. Joe's is where patient number one in the US got diagnosed for Covid and talking to some of their analytics folks there, they said the challenge in the first few weeks was there was no ICD 10 code for Covid.

So there was no way to code that diagnosis. And for the first few weeks we didn't really have a valid test that people trusted that would tell us if the co patient was covid positive or not. So how do you use the data that is captured? When it's something new and be able to kind of have that early warning system that, hey, we may not know the name for this yet and be able to code it as a diagnosis, but we know something is going on here that we need to pay attention to.

So I wonder if we need to be thinking about, how do you look at. Variation in symptoms and vitals and things like that as an indicator that something is going on as opposed to waiting till we can actually name it. Well, it's, it's interesting that you bring that up 'cause uh, that was February 29th, first, uh, reported patient at Providence St.

Joe's. I think history will tell us that was not the first patient in the United States. Agreed. But I mean, so we're gonna look at all this data retrospectively. Through, through Epic's platform or through Tru or through wherever we're gonna start pulling this data to, to do this. Are we gonna be able to find this signal that said, Hey, here was the start of the pandemic in the United States?

I don't know. That's a good question. I think if you, now that they know more about the symptoms and the, like the physical presentation of C Ovid 19, they could go back and look at data. Figure that out, but. It's getting access to that data and having the right tools to be able to basically tease that out of the medical record.

Yeah. 'cause a lot of that's gonna be in the notes, isn't it? It it would be clinical notes. So you're gonna have to, you know, look at NLP and I think actually I think Providence St. Joe's did actually create an NLP algorithm that let them target people that they suspected were covid patients and then they could go and sort of validate, um, and confirm whether or not that was the case.

One of the things that happens is we just see this over our lifetime, the federal. Takes a stronger hand during certain times and then a little lighter touch at other times. The pandemic seems to me to be, and the need for better public health seems to be a case where the, the federal and even state will start to weigh in a little bit on this.

Data conversation, defining the data, what data are we gonna collect or report, and those kind of things. Do you think that will be a factor? Do you think that the, the top down factor will be of the federal and state coming to bear on this is going to have an impact on how we collect data at the field level?

I think it will, and I really hope that in addition to funding to do that work, what they really do is they bring the right stakeholders together to talk about how that looks. Because what we've seen through meaningful use and, and other things like that is . You really have to think about what's the impact that data collection is gonna have on the people who are caring for patients.

Because if it's putting a huge burden for the care providers to do manual processes and things that are gonna be time consuming, if you're in a pandemic or other health emergency, you're taking 'em away from patient care. You're taking 'em away from prevention measures and things like that. We've gotta find a way to make it more seamless and require little to no effort on the part of the health systems and the care providers to be able to collect and send that data.

It should be collected in the background as they're providing clinical care instead of being like, we're gonna deliver care and then we're gonna go back and document it up a bunch of stuff on the back end. We are seeing, and, and I'm curious about this, we're seeing a lot of these big data data stores.

We're seeing, uh, Mayo partner with Google, essential partner with Google. We're seeing Tru Veta come out and there's others that have been out prior to that. I guess my question is, I. Is that a trend? Are we gonna keep seeing this consolidation, mass data sets, or is that just for a specific use case? Is that just for pharma and population health and research and those kind of things?

Is that what that's for? And we, we still have a significant need for the operational types of data source and, and data practices. I think you're still gonna have the operational data practices. Because a lot of what those, those larger integrated data sets are for is that discovery and anything that's gonna involve AI and machine learning development.

The more data you have, the, the better you can get in terms of your models. And especially if you're trying to develop something that's gonna generalize across populations, you've gotta have access to data from a lot of different organizations for the model to actually work and generalize. But I think you're always gonna have your local, we've gotta run our business and we need our operational data to manage our populations, manage our financials, manage our patient flow.

I. I don't see a lot of those big aggregators of data really focusing on that. I think they do have a little bit of an opportunity there, and there's a couple of places that that do this, you know, premier and others. If you can give benchmarking data, like how does my revenue cycle performance look versus my peers, do I have an opportunity to improve or am I doing pretty good and what can I learn for those people who are the top performers?

So I, I, I don't see those big aggregators of data focusing on some of this just operational delivery of care. I've started asking this question to close out my interviews, and it sounds like a cop out kind of question, but I've gotten some great answers. What question didn't I ask that you're surprised?

What area didn't we talk about or question, didn't I ask? Well, I think there's two topics that didn't come up that . That I'm kind of surprised. One of them is data literacy. 'cause if we go back to your question about what are the characteristics of high performing, we kind of got off topic, but data literacy is definitely one of those because you can have a great culture and you can have great platforms and great data governance, but if you don't have a data literate population that can put all of those things into practice, you're never gonna have your analytics really impact care delivery or organizational outcomes.

Data literacy for sure. And then the other one that I'm hearing a lot more talk in the industry about, but I'm not seeing a lot of people doing it in practice yet, is model governance. So as people start to put predictive models and machine learning into practice and operational processes, who's gonna decide?

Is this model performing well enough that we will put it into production and who's gonna monitor and manage that over time to make sure that if you have data direct, the model's still performing and we can't roll all the models out at the same time? Like who decides which ones get implemented when.

And a lot of organizations aren't thinking about that yet. All right, so let's hit data literacy real quick. What does an effective data literacy program? I mean, it almost has connotation of. I mean, the word literacy is learning to, learning to read. I mean, it, it almost seems very fundamental. And so we have to give everyone sort of a foundational level of the data, how to read it, how to understand it, and the definitions that we're using and those kind of things.

Do you end up with a training department that does that? I, I think it becomes part of your professional development, but you've got training programs on clinical competencies and other professional competencies. You need to have one on data. There's a lot of parts to it. A lot of organizations where they fail in terms of adoption and utilization of analytics is they train people on the tools, but they don't actually train 'em to understand the data and how to act when they see the data.

I know how to pull up the dashboard, but what do I do when the metrics go in the wrong direction? How do I drill in and figure out what's the root cause and what are the leading indicators that I can start watching so I can intervene early? And liter data literacy is one of those things that if you've got a good program, I think it's not one size fits all the level of data literacy and the specific skills and knowledge that.

Revenue cycle manager or a physician needs to have is gonna be very different than the level of literacy a finance analyst needs to have. And so your program needs to be a little bit modular and flexible to meet the needs of the different constituents in the organization. Are, are the colleges and potentially med schools, are they putting out people with a higher level of data literacy, just general data literacy that we can plug into?

That's a good question. I don't know. I do feel like there's more technical literacy and there are a lot more people coming out that have been exposed more to data and analytics throughout, throughout college compared to like 20 years ago. So I think progress is being made. I think the other thing just for for people to know is you don't have to go create a bunch of content to put together a data literacy program.

There's a ton of really good content out there already. You really just need somebody who's gonna kind of curate that content and put it together into curriculum for organizations that use LinkedIn learning, there's tons of things out there on . Tableau and Excel and how you use those things and some basic concepts around data.

The main content you'd have to develop internally is about understanding your data and your metrics, but you don't wanna have to build it all from scratch. For sure. Are there good programs out there to teach you Lotus 1, 2, 3, and Harvard Graph ? No. Okay. I dunno. I haven't seen any. Model governance. It's interesting to me 'cause model we used to go out and buy models.

Essentially you, you'd hear of another health system that was doing something effective and it was integrated into some tool and you, you ended up. Either layering that into your EHR or layering that into another program, or even some of the analytics platforms now, you could buy some of these models and, and bring 'em across.

It's interesting to me, I, I understand the concept of model governance, but sometimes my concern is about the black box of the model. If you're buying the models, a lot of times people don't want to tell you what's really all in the model. Yeah, and there's a lot of research going on around the concept of explainable ai, and especially when you get into like clinical scenarios, it's gonna be a lot easier to get trust in, in the models and adoption and use of the models if you have explainable ai so that.

You can see what are the factors that drove this risk score, and I can use my experience to judge whether that makes sense or if I think something strange is going on. The other thing is for organizations that aren't gonna develop their own models, they still need to have model governance if they're gonna be buying and implementing models from their core EHR vendor or other vendors because they need to be asking for data about

How was this trained? What's the efficacy? Let's run it on our population and see if it keeps that model efficacy. There's all these publications on, we can use models to do these things, which is great, but to actually put it into production and operationalize it, you really need to understand how effective is it, and was it trained on a population that was comparable enough to mine that it's actually gonna work on my population?

Wow. That's really interesting. Well, thank you for coming back from your vacation to just spend time with us, . We really actually, you may still be on your vacation. That's a, I'm not on vacation. Oh, okay. 'cause people are, if they're watching on, on our YouTube channel, they're probably saying, well, that background can really be anywhere in the, in the world.

Now you really don't know where people are anymore, but you are. Don't, but you're back in Atlanta now. Yeah. Well it was great to talk with you today. I really enjoyed this. Yeah, thank you very much. Appreciate your time. What a great discussion. If you know of someone that might benefit from our channel, from these kinds of discussions, please forward them a note.

Perhaps your team, your staff. I know if I were ACIO today, I would have every one of my team members listening to this show. It's it's conference level value every week. They can subscribe on our website this week, health.com, or they can go wherever you listen to podcasts. Apple, Google. Overcast, which is what I use, uh, Spotify, Stitcher, you name it.

We're out there. They can find us. Go ahead, subscribe today. Send a note to someone and have them subscribe as well. We want to thank our channel sponsors who are investing in our mission to develop the next generation of health IT leaders. Those are VMware, Hillrom, Starbridge advisors, Aruba and McAfee.

Thanks for listening. That's all for now.