A Day in the Life of a Ransomed Health System
Episode 11515th June 2021 • This Week Health: News • This Week Health
00:00:00 00:12:10

Transcripts

This transcription is provided by artificial intelligence. We believe in technology but understand that even the most intelligent robots can sometimes get speech recognition wrong.

  Today in health it, the story is a day in the life of a health system experiencing a ransomware attack. My name is Bill Russell. I'm a former CIO for a 16 hospital system and creator of this week in Health IT at channel dedicated to keeping health IT staff current. And engaged. I wanna thank our sponsor for today's Sirius Healthcare.

They are strong supporters of our mission to develop the next generation of health leaders. If you believe in our mission and wanna support the show, please shoot me a note at partner at this week in health it.com. Alright. I found this video out there on YouTube and I'll put it in the show notes and out on social media.

And it is the CIO of Sky Lakes Medical Center in Oregon talking about their six month journey. Following a Rauch ransomware attack back in the fall. Let me start by saying I'm not a fan of this video, mostly because I was taught that you never talk about your security posture with the media. Now I understand fully that the intention is to share this information with the industry in order to help others see the full impact of ransomware and be prepared for such an attack at their health system.

I also know that the bad guys have access to YouTube as well. I do applaud sharing it, but I would suggest more discreet settings to pass the information along. But since it's out there. We're gonna talk about it. In fact, we might split this across multiple days. The first day we might just talk about the situation and what it feels like to be in the midst of a ransomware attack.

The second day, we might talk about what they actually did and how they recovered. And the third day we might talk about the findings and, and what their learnings were from it, and what some of our learnings might be from it. So I might do this for a couple of days. I think this is an important topic. I think a lot of health systems are living this right now.

th,:

And here's what happened. An email came into one of the employees at Sky Lakes Medical Center and the email stated that if you click on this, you'll receive a bonus. And who wouldn't click on that email? 'cause it looked like it came from hr. It had, you might receive a bonus. That person had also had a conversation with HR earlier, as the CIO goes on to talk about.

And so they clicked on that email and within about 30 minutes, the system went out. To a malicious site. It had a zero day payload for them that had been set up by these actors, and they pulled the, the payload back down to the computer, to the person who clicked on it. And what she experienced was that the computer started acting a little slow and the computer froze up and she couldn't control her mouse.

led Ops Strike was enacted at:

ck, and that was it. At about:

The email title is annual bonus report, PDF. This is not something I would click on. It's not that I'm immune to phishing. I, I am not immune to phishing, but this, this seems like a pretty obvious. Email attack, to be honest with you. Annual bonus report. Click here to preview and see it. There's nothing that's personal about it.

It's not personalized to Sky Lakes or the person who is receiving it really in any way. The links are kind of disguised, but they're kind of not, to be honest with you. I think it really, a, a decent email system would've caught this and not even delivered it to the person. But with that being said. We know that around this time there was a significant amount of activity.

In fact, three days after this attack started the FBI did notify about 400 hospitals that there may be attacks on a certain day. Sky Lakes, university of Vermont and St. Lawrence Health Network in New York, succumbeded to those attacks. In fact, St. Lawrence, I think this is an interesting point. St.

Lawrence's health system actually had a person click on the same exact email within a couple of hours of each other. That those two health systems were attacked and infiltrated. Okay. So that's how they got in. So the CIO describes it really as the perfect storm you have. You have covid trending up in the commu community around that time.

In the fall of:

The network itself, they also had AMP endpoints, Cisco AMP endpoints. Which would've caught some of this stuff and slowed it down, if not, uh, stopped it, but it was not yet fully deployed, and they were about a month into the new deployment. And Cisco AMP four endpoints was not yet fully configured even on those systems where it was already out there.

So there was a lot of things that were really influx, which he says created the perfect storm for them. . So I'm gonna get into tomorrow the steps that they took and the things they had to do. I want to go down a little further in the presentation because he goes on to talk about what was actually impacted.

Lemme tell you what happened next. They had to shut everything off, right? They, they went out and they found two companies to help them. Kivu, KIVU, and Talos, which is a, a division of Cisco. And they came in and did a root cause analysis. They also did a response type plan for them on the fly. The first thing was shut it down.

Shut it all down. In fact, their monitors, which were on patients were collecting information, but it wasn't flowing back across the network. Pyxis was down, the EHR was down. Everything was down. Pax was down. So if you wanted to read the images, you actually had to go to the system itself. But even some of those systems were taken offline for a period of time.

To make sure that the malware couldn't get across the network and get to more and more devices and start shutting them down. It was all isolated at that point, if not turned off completely. So they're a community connect site, which means that they are running off of a mothership epic system and that mothership happens to be Asante Health Systems, epic instance.

And they were disconnected from that as soon as the CIO was notified. And we will go into that in a little bit more detail tomorrow. Let me go down a little further 'cause I wanna talk about some of the other things. Let's talk about some of the clinical impacts. When all your systems are down, there's no medical history.

It was almost impossible to find the medical history. Now, they ended up setting up some iPads, going back to the the primary epic instance up at Asante to get medical history. But as the days went on, that medical history got old very quickly. Placing orders was very difficult. You have a generation of people that learned on the EHR.

They weren't around with paper, and they had to go to the people who were around. Prior to the EHR and figure out how to do some of those processes. And some of those processes were good for a couple days. Everybody has downtime procedures for one or two days, but, or maybe even three days. But nobody has downtime procedures for 30 days, 60 days.

So that's what you're looking at here. Some systems were down for an extended period of time and you know, some of the other impacts. Communication was completely crippled. Think about it. I mean, everything's offline. The only thing that was working was uh, Cisco WebEx teams and phone and text messaging.

They actually, at one point tried to bring VRA back online. Very early in the process 'cause they realized how important communication was going to be and they brought it back online and it was immediately hit again by the ransomware attack and taken offline. And they realized that that mechanism of bringing some of those things back online very quickly was not going to work.

And so, you know, all those, all those communication points are down. The stuff that you communicate between shifts, between departments, staffing. You name it all relied on face-to-face. Communication relied on text messages, phone messages, paper. So that's, that's communication's just completely gone for the purposes of time.

I'm just gonna cut some of these things short. Obviously there was a, there was other impacts. There was, uh, time tracking impacts. Kronos is down so you can't track people's times. There's a lot of overtime being done right now. If you could imagine. How difficult this was for nursing, how difficult this was for the physicians and the extra time that was being put in, but you couldn't track any of that stuff, right?

Then there's the financial impact ar ap, you know, you're not paying your bills, you're not receiving income, you're not generating any bills, so you have that kind of thing going on. Plus, and we'll talk about this more tomorrow as well. You have the backend data entry, which has to happen, and you know it's, it's right around the corner.

You're gonna bring that EMR back online and those mounds of paper that are starting to stack up next to each one of the patients, or about each one of the patients at the nurse station or wherever they are, all that stuff's gonna have to go back into the EHR and there's gonna be huge gaps in data. And that information has to go back in, and that's why I wanted to break this one down.

I want people to understand and experience what it would be like to be down for 30 days, and not just the EHR, but all your systems. I. What to say this is significant is a huge understatement, right? There's a clinical impact. There's, uh, care and safety impact. Obviously there, there is a workload and stress impact on the staff.

There's potential data loss. There is, uh, there's so many issues that can arise from this, and so that's the reason we spent so much time on this today. It is just to give you a sense of what it's like to be going through a ransomware attack. Think about it. All your systems down, you don't even know how bad it is.

You don't know which systems are affected. You had to shut 'em all down. We're gonna go through tomorrow a little bit of what their response was, what they did in what order, and then in two days we'll go through what some of the learnings are that they got out of this and some of the learnings we got out of it.

So that's gonna be the So what the, so what's gonna be in two days? So you're gonna have to check back. All right. That's all for today. If you know of someone that might benefit from our channel, please forward them a note. They can subscribe on our website this week, health.com, or wherever you listen to podcast Apple, Google Overcast, Spotify, Stitcher.

I. You get the picture. We are everywhere. We wanna thank our channel sponsors who are investing in our mission to develop the next generation of health leaders, VMware Hillrom, Starbridge Advisors, McAfee and Aruba Networks. Thanks for listening. That's all for now.

Chapters