This transcription is provided by artificial intelligence. We believe in technology but understand that even the most intelligent robots can sometimes get speech recognition wrong.

 Before we get started, I wanna share with you something that we are extremely excited about here at this weekend, health it, and that is CliffNotes. CliffNotes is the fastest growing email list that we've ever put together. If you can't listen to every show, but you want to know who was on and what was said, the best thing to do is to sign up for CliffNotes.

one paragraph summary, key moments in bullet point format with timestamps and one to four video clips from the show. It's a great way for you to stay current, share insights with your team, and maintain your commitment to their development during these extraordinary times. The best way to sign up, the easiest way to sign up is to just send an email to clip notes C-L-I-P-N-O-T-E-S.

At this week@healthit.com and it'll kick off an automated workflow. You'll get an email back from me, click on that link and you are off to the races. Uh, so don't delay. Send that email. Get signed up today, now onto the show.

Welcome to this week in Health It. Today we do a solution showcase focused on data control. . And data protection. My name is Bill Russell Healthcare, CIO, coach and creator of this Week in Health. It a set of podcast videos and collaboration events dedicated to developing the next generation of health leaders.

This episode and every episode since we started the Covid to 19 series has been sponsored by Sirius Healthcare. Now we're exiting that series. And series has stepped up to be a weekly sponsor of the show through the end of the year. Special thanks to Sirius for supporting the show's efforts during the crisis and beyond.

I enjoy doing solution showcase episodes for two reasons. Primarily. The first is these organizations are contributing financially to, to the production of our program and our mission to develop the next generation of health leaders. And also because it gives me an opportunity to explore these, uh, the solutions, these really interesting solutions that I believe can have an impact.

On our health systems and on the care in our communities. This show dives deep into data protection and data control with guests from Cone Health, our sponsor, Sirius Healthcare, and Rubrik a a Gartner Magic Quadrant leader in this space. Here's our solution showcase. I hope you enjoy. Alright, today we're gonna look at, uh, a really special topic that I've wanted to look at for a while now, and that is data protection.

Especially with all the stuff that's going on with ransomware and disaster recovery, change of things. So I'm, I'm really looking forward to this conversation. We have three distinguished panelists, or I distinguished or not, we'll find out in a minute. We have Josh Peacock, former guest healthcare solutions architect with Sirius.

Good morning, Josh. Morning, uh, Michael Friesen, senior Technical Specialist with Cohen Health. . Good morning, Michael. Good morning. Good to have. Good to have you. And then, although Michael is a true security operations guy, he does not have the, the video, it's all taped up and, and whatnot. We'll superimpose a picture there for him.

And then Zach, I, Zach, I didn't ask you how to say your last name and I should have p Pia. Is that right? Pia? Yeah. Got it. Data Protection specialist with Rubrik. And Rubrik is a Gartner leader in the multi-cloud data control space. Welcome to the show. Thanks. I should ask people how to say their names before I start the show.

I don't know why I keep making that mistake. You would think this is episode like number five, but this is actually gonna be episode like 3 0 5 or something like that. So crazy. This is a, this is a topic that keeps coming up and. It just, it shows itself in different ways and, and a lot of times it's through challenges or, uh, uh, data loss or ransomware or different things to that effect.

And I wanted to dive into this and I really appreciate you guys, um, coming on the show. So let's start with the challenge. And I'm not gonna call on any of you. I'm just gonna throw stuff out and then whoever wants to, to take the question, feel free to, to go after it. Let's start with the challenge. What challenges are we facing in the area of data control today?

I can answer a few things that we're seeing, particularly in, in my experience around some of our Epic clients and some of our other just general healthcare clients as well too. We're seeing. Some significant and uh, just almost uncontrolled growth in data, which I guess we've heard about that for a number of years, but even in that had been traditionally like unstructured, but even structured databases are growing to these massive sizes that we're trying to back up ransomware, even though, and, we'll, you know, we can dive into that a little bit further, but for everyone that we hear about, there's a client, somebody that's getting hit by it daily, that's recovering from things too as well.

So that continues to just be. Area that gets really beat on in healthcare as a target. They're becoming more sophisticated than that as well. So I don't know if Michael or Zach, if you got anything else to add there too, but those are two of the big ones that I see. Michael, I'm gonna, I'm gonna kick it over to you.

The, so you guys, you are, operationally, you're in the role right now. What kind of things are you seeing at Cone Health that causes you to look at this space and, and to look at a, a solution like Rubrik? For us, Rubrik, uh, came into play when we were trying to get away from some of the legacy complex backup solutions.

The automation opportunities with the rest API were huge For us. That goes into the data protection because we automate setting up all of the backups and verifying that they're there. If we need to recover data, there's still some rest API and and automation opportunities for us. The data growth, as you mentioned it, it's not just having one database that's 30 terabytes, it's five copies of that are all slightly different, and so those challenges certainly exist for us as well.

So is that a window problem? So back in the day, back, back when I was CIO, we, we had these, uh, growing data stores and our windows were closing. Like we couldn't back it up between the windows. We needed to back it up. Is that still a problem or is that an old problem? It's still a concern and it's a concern that you still have to work around to some extent.

And obviously getting rid of tape, I don't wanna look at a backup solution that still mentions tape. That helped tremendously Being able to tweak some of the, the threads and having a, a distributed node-based system like Rubrik has helped us get those backup windows down. Yeah. Here's a question I've I read about these ransomware things.

So I think UCSF was the most recent one. And, and Zach, I actually, I, I think I'll call on you for this one. When I look at ransomware, we see the health system get attacked. They pay the ransom. And I'm always like, I'm always sitting there going, I want to talk to somebody. 'cause I, I want to find out like, didn't you have a backup?

Why? How did the ransomware, how were they successful? I understand how they can potentially get in there and lock a certain data set down, but couldn't I just wipe out that data store and restore it? I, I'd be, help me to understand how ransomware works. How are they successful? Yeah, I think it's like Josh said, they're, they're becoming more and more intelligent.

Like they're figuring out that people are just restoring and not paying the ransom. So they said, what can we do to, to prevent people from being able to restore? And the first thing they did is start targeting common backup file extensions. There's a lot of proprietary extensions people use for backup files, so the ransomware actually scan the file system, find all those files first, encrypt those files first.

And then move on to other files. So it's pretty impressive on how intelligent things are getting. Alright, so gimme an idea. Since you're, uh, you are a data protection specialist, you're working with Rubrik, what do, what, what does Rubrik do? What do advanced systems do to protect against that kind of, uh, attack?

Sure. So I mean, at the end of the day, I always ask the question, if you're not, if you don't think that your production NFS or SMB shares are going to be protected in a ransomware intact, then why would you put your backup data there? And that if you look around the industry, that's really what most of the, the competition does.

With Rubrik, we just built it with security in mind, and in order to write to our file system, you first have to authenticate through our software. We use secure RPC calls to get to our file system so there's no IP address on the network where you can access our file system, no NFS, no SMB, none of that, and if you're able to, to basically fake your way out through all that, which, oh by the way, we also have a zero trust methodology.

So we make you re-authenticate each time. If you get to the file system, it's also a pen only or immutable, meaning you can only add change files. You can't delete or modify any of the data in the actual file system. Just pretty, I think we're the only one on the market that has something like that. Yeah, that's, it would take a, an extremely sophisticated attack to go after that and, and what we found is when you have all those kinds of barriers.

People just go find somebody that, that doesn't have those barriers and that's, they just go to the path of easiest attack and the path to easiest money. So that's really what they're after. At the end of the day. I, I want to hit on a, a handful of things. I want to come back to that operational data store growth.

Uh, I, I also wanna talk about cloud and I wanna talk about simplifying the environment, I think are the three things, because we talked earlier. prior to this. And those were the three things that sort of jumped out at me that I, that are interesting. And so Josh, you work with a lot of clients and a lot of different clients.

Give us an idea, you mentioned it in the intro, that the operational data stores and that the data stores just in general are growing. Uh, can you give us some picture of how fast they're growing and, and what you're seeing? Yeah, sure. So we have some clients, um, that are actively in the, let's say, . 40 terabyte range for their core databases, uh, in the epic space.

And, and with that, those are expected to grow to somewhere around 80 to 90 terabytes in the next three years. So that's pretty substantial growth. And then alongside that, we're seeing growth in the reporting side of the world on Epic clients. And then we're also seeing the blob storage or data management systems as well.

So if that's on base or edco or whatever ones you have there too, we're seeing significant growth. Within in those spaces as well. So not just the databases themselves, but then those file storage locations that are ancillary to the core database. So you have the core database, you have the analytics, and then you have the, the document store, the OnBase kind of things.

And so are, are either any of those growing at at different base? It's the document store growing faster than the others. That's pretty organizational specific. I think especially if they maybe had just converted to, um, the new EMR or brought in a new document management system. If they're doing a lot of scanning of old paper, uh, charts and bringing those into that DMS, uh, we see significant growth in those.

Otherwise, it's more of a steady linear growth on those. Do we face any specific challenges in backing up Epic? . So for the most part, the clarity databases are either gonna be some form of Oracle, whether it's Exadata or Oracle on something, or Microsoft sql, and there's good tools around how to support those large databases and back them up in a good, clean manner.

The other items, when you start talking about cache, or as you transition into IRIS, it's not any fault to, it's just the regular. Backups of those are, they don't run into 'em as often, so vendors don't build specific plugins and things like that to make those work in the best fashion. So we go back to a traditional, uh, effort of sweeping files in as a, you know, uh, a way to accomplish those backups.

And that's where we really run into concerns about timing. It can take a while for, uh, large multi terabyte files to be backed up every day. And so that's where I think some of the work we've been doing. Is to document best practices around that with the rubric team and some of our, and then provide white papers out to the industry so that they can, you know, do it in the most efficient fashion as as possible with what we have to work with.

So back in the day, again, I'm dating myself here. We did snapshot, we did full backups, we did snapshots, we did incrementals, we did, there's a lot of different things. Is, is the methodology still the same or has it changed? Uh, yeah, there. So we build a kind of a complete strategy for the organization around, uh, protecting their data.

We start with physical snapshots on, uh, arrays, and then we take those and put those into, uh, like a rubric solution for a longer term and, and more protection in case something were to happen to that array or be affected in some way. But yeah, absolutely. We tend to do fulls and then followed by incrementals after that.

The operational database for Epic, which is the cache slash IRIS, traditionally hasn't played well with Incrementals. Uh, that is one area that we see Rubrik shine. Those is the. Incrementals for their solution. Uh, actually do a really good job on those, those, uh, dot dat files that are make up the, the database.

Alright. Josh, I'm, I'm gonna go to Michael here in a minute, but last question for you. I would assume if you're gonna do ransomware, epic would be something, or the EHR would be something. Have we seen successful attacks on an Epic system? Is that something that they do Target? I haven't seen anything at this point.

I think the, if there was something that was, I. Most applicable to a lot of those tax. It would be the backups, which like Zach explained how to protect some of that. And then there are those file shares that are associated with document Manage, either the Epic Web blob or your OnBase, um, document stores tend to probably be the most targeted type of data just 'cause it sits on, uh, samba or sip of shares.

All right. No, that's helpful. Michael, I wanna go to you on two things. I, I do wanna talk about cloud, but I'll, I'll get to that later it, but I wanna talk about, uh, ease of use. I remember these systems and not being fairly complex, hard to manage. Uh, you mentioned getting rid of tape. That was the first thing is we had these massive tape robots in our data center, uh, that had to be fed and cared for at night and that kind of stuff.

So that's one level of complexity that goes away. How else have we simplified these environments? Or how else have you simplified this environment at at Cone Health with the adoption of the rubric, uh, platform. One of the, the strategies when we were looking at this was trying to get away from the traditional complex backup system where you actually had one or two backup specialists and, and they knew that product.

and, and there's not necessarily anything wrong with that, but you can certainly imagine if you've got a new system that comes in or, or an emergency restore of some sort, and you've got to find the backup guy. Mm-Hmm, . Um, it, it can be a challenge and it, it delays our patient care. We can't restore a system or restore patient data in a timely manner if nobody can just jump in and figure out how this, this system works.

So that was something that really struck. Us with the rubric system was the ease of use. I, I actually had one of the guys on my team, I told him what the URL was. He logged in, he was able to set up a backup for a new system in five minutes. So that allowed us to say there's no reason when we get, uh, a very high profile restore ticket or something like that to come in, that he.

One on my team, uh, we, the team of seven for the data center that includes backups, e everybody can do a restore. Yeah, that's interesting. One of the things I'm, I'm measuring things against now, and we didn't talk about this ahead of time, so this is a curve ball for you guys, but one of the things I like about the systems that I used for, I have a small business here, so I go out and I use cloud systems.

I stand up compute storage and, and those kind of things. But I could just bake the backup right in. Right? As I'm provisioning the, the server, I just click a few handful of things and I set up a, a backup structure. I could choose snapshots, I could choose incrementals and those kind of things as I'm provisioning the system.

So it has a level of automation, and I assume that's all accessible via APIs. Is that something that Rubrik can do? Zach, I guess I'll ask you or, or, or Michael, is that something Rubrik can do? I, I, I can tell you, I, I don't want to jump in front of Zach, so I'm gonna be quiet now. No, please go ahead, Michael.

But I'm excited about this piece. This, for us, was huge. We provision and decommission all of our servers using scripts. We have dabbled with some automation tools, but at the moment, for most of our Windows servers, it, it's all PowerShell scripts. So we are setting up the backup when we deploy . The server using the PowerShell.

Another piece of that PowerShell script sets up the backup, puts it in the right SLA policy then, and, and I think this is just as important because one of the things I love to see is servers go away. And so part of the decommission process, we go out, check that there was a recent backup in the rubric system using the rest API, if there was, maybe we.

We take another snapshot if it's been more than a day, and we just put that in a, an archive, SLA so that we've got it. Otherwise, we just retire those backups and it's done. And, and basically all we're doing is executing some PowerShell scripts that reach out to these pieces and notably one of 'em being the rest AP to get all of this accomplished.

So Ed. Are tasks around building and destroying servers tremendously. Uh, Michael, do you have a cloud deployment as well? Yes. So, uh, how are you looking at backups and data control across a multi-cloud environment or on-prem and cloud? How, how are you thinking about that and how are you using tools around that?

We're, uh, I would say we're getting started with some of that. We've been using the cloud now for several years, but it, it's continuing to grow and we're still continuing to understand and learn how to make the best use of that. That was something that was a very big deal to us about Rubrik as far as the, the.

Kind of, they were, they seemed to be forward thinking about the cloud from the beginning as well. So I have a cloud cluster in Rubrik or in our Azure cloud. I'm using that to back up some file systems that in the past had a Azure backup limit. There was a four terabyte disc backup limit. We didn't have that limitation using the the rubric product in the cloud.

So. Some of those things have changed and that's not, uh, true anymore. We can back up bigger file systems, but it still having it all in one place to manage the cloud backups and the on-prem backups so that we, we just have one place to go to interact with. That is a big strategy for us because we're branching out into AWS, we're using Azure.

We've got on-prem data centers. I, I don't think that it's gonna get simpler from that standpoint. So as long as we can have one interface for all of those things, for our backups and our data protection, that kind of helps keep it a little bit simpler than it could be. Yeah. You know what, Mike, I think you're so far ahead of the game and the, the reason is 'cause you guys are already thinking in terms of automation and scripting.

And that's one of the huge benefits of the cloud. And if you can script your own data center, your on-prem data center, uh, then I, I think the, the sky's limit is gonna be interesting. Zach, you've gotten off pretty easy on this. You've, it's actually nice when you have, uh, you have a, uh, partner talking about your solution and you have a client talking about your solution, but I'm gonna bring it back to you.

A lot of data control, backup specs come from really the regulatory requirements that we have. What kind of. What kind of things does Rubrik or, or, or a system, a data control system have that helps us to adhere to those regulations or even automate some of that, some of that compliance. Sure. So there's quite a few different things that manage like retention that are pretty interesting.

But the really, the two key drivers that I see from Rubrik are, are the ability to detect ransomware is one which doesn't really fit the footprint of what you're talking about, but basically just detecting ransomware and being able to give you a list of all the affected files and let you restore. So just giving you more control of your, your data.

And then the second is an integration where we actually do data discovery. We can actually using backup data without running scans on your production system with no agents. We can actually use the backup data as we're ingesting it, uh, and scan it on the backup system for things like hipaa, PII, data, GDPR, all those types of different things, you can do your own regex files and find that important information, that information that should be in a secure place, and then allowing you to hit your audit or your audits and things of that nature.

So essentially you're applying ai, it it to the data as it flows through. Is that accurate or am I overstating it? It's classification, so it's really built off of Reg Xs, but we're giving you the templates that are the most common within, you know, HIPAA requirements, let's say. So rather than build your own kind of HIPAA template, we've got some prebuilt ones for you and you can add your own after the fact.

We do use machine learning for the ransomware detection, so we're looking down to the directory level, to the file level and saying, Hey, . This is unusual or anomalous behavior. Let's flag this and have you look into it. Thanks for not telling me I was wrong, but I I, what I heard was I was wrong, but I appreciate that.

It, it's fine. I'm wrong on this show Often. I, sometimes I refer to the show as the education of Bill Russell. I, I appreciate you guys educating me further on this stuff, Michael, in our earlier conversation, I, I found it interesting. We were talking about running Epic in Azure, and you guys just did a pilot and I, I, I want you to just share it.

This could easily become a full podcast in of itself. I want you to just share it to just what people's appetite as to what the, the future might hold for running the EHR, uh, in the cloud. Can, can you share a couple details around that? Sure. So we have switched from our DR location for Epic being, uh, our secondary data center to being Azure.

So in, in order for us to really say that is our DR location, epic kind of expects that you actually test it out and, and use it. So as part of that, just, uh, what, two weeks ago we ran for. Our epic prod environment out of Azure for a week. We wanted to gain some insight on a couple of things there. And, and, and I, I should say that this, this was a week of running out of Azure, but it was almost a year.

It was a year and a half of work to try and make that a reality. So it, it was not a, a simple undertaking and it took a lot of help from Epic and from Microsoft, make that work. We were able to run for a week out of there. We wanted to try and figure out, you can look at all of the cost calculations and and so forth.

All you want to try and project what it's really gonna cost. But the reality is there are so many. Extra things. You don't always plug into those calculators. We wanted to see what the cost really would be. One of the challenges around running Epic, I think most people will agree is, is chasing the latest hardware that makes the cloud very attractive to us.

If I can just have a. A quick skew change and now I'm running on the latest hardware. It's a lot cheaper. It seems like it's a lot cheaper than having to replace hardware. One of the things that came out of last week's test was, uh, at the moment that our Citrix performance was actually better than on-Prem.

And again, that goes back to. Our hardware is getting a little older, it's time to replace it, which really breaks my heart 'cause it's only three years old. But that's the pace that Epic drives hardware at. So it, you can't say, we're gonna buy this and it'll be good for five years, any longer. You might get two years out of it.

And, and that's seems unsustainable. , I'll, I'll, I'll actually close this part of the conversation with this question, which was, uh, did your users give you, what was your user's response? Did they, I obviously they knew you weren't gonna pilot this, you would've told them, but was the performance good enough that they weren't like calling every day saying, Hey, when you switch 'em back, or, or how did it go?

So from what I have heard and, and they try and not let me out to talk to people too much, nobody really noticed. Yeah. That's so, that is way better than I expected. Yeah. That would be a win if you could fail over and, and the performance is, you know, as good or better than what you have OnPrem. That is, that's a huge win.

Gentlemen, I'll, I'll, I will leave you guys. The last word, is there anything I didn't ask or, or didn't cover or some topic around data protection and data control that I may have missed or some aspect of the topic?

I'll take it from your silence that I, I asked every question I. I think you did a good job covering it. And I, Michael, I, again, congratulations on your guys' success in Azure and for helping pave the way for some of the other clients that we're working with right now. It's, uh, much easier because of what you guys did.

Yeah. Yeah. The, the first person who runs through the wall, we all appreciate . It takes you a year and a half to do it. There's a lot of bumps and bruises and those kind of things, but when we get to the wall, we just look and somebody goes, oh yeah, don't run through there. Just go through the little opening that they made.

Uh, for you over there. So yeah, we, we do appreciate the, the early adopters and, and getting through there. I, I think there's a whole episode on this. I'll let, I'll let it play out a little bit and you guys get some more learnings around it. I, I, I definitely would like to explore the EHR on Azure and, and AWS and other things and how organizations are approaching it.

But this was, this was really helpful for me. I, I understand ransomware a little bit more. I understand how the attacks happen. I understand how to protect against it. I, the challenge of growing operational data stores as well as the, the blob storage and other things that are going on, it's, it continues to be one of those things that is.

Ever evolving, you would think something like this, we got rid of tapes, which is great, but now we're, we're getting ready for whatever the next set of challenges that, that we are gonna face. Gentlemen, thanks for your time. I really appreciate it. Thank you. That's all for this week. Don't forget to sign up for CliffNotes.

Send an email to CliffNotes at this week in health it.com. Special thanks to our sponsors, our channel sponsors VMware, Starbridge Advisors, Galen Healthcare Health lyrics, serious Healthcare Pro, talent Advisors, and Health. Next, for choosing to invest in developing the next generation of health leaders.

This show is a production of this week in Health it. For more great content, check out our website this week, health.com. Or the YouTube channel if you wanna support the show. Best way to do that is share it with the peer. Actually, another way to do that, sign up for clip notes and forward clip notes onto 'em and say, this is a great show.

I'm getting a lot out of it. Uh, please check back every Tuesday, Wednesday, and Friday for more great content. Thanks for listening. That's all for now.