Today in health, it comes spirit outage, a little bit more detail. And my discussion. With your board members. If I were the CIO of a health system, what would I be saying to the board? My name is bill Russell. I'm a former CIO for a 16 hospital system. And creator of this week health, a set of channels, dedicated to keeping health it staff, current and engaged. We want to thank our show sponsors who are investing in developing the next generation of health leaders, Gordian dynamics, Quill health, how site nuance, Canon medical.

And current health, check them out at this week. health.com/today. All right. So the common spirit outage continues. Let me give you a little bit of detail. This is from their website. Over the course of the past week, we have been managing your response to a cyber attack that has impacted some of our facilities. Patients continue to receive.

The highest quality of care and we are providing relevant updates. On an ongoing situation to our patients, employees. And caregivers. Patient care remains our utmost priority and we apologize for any inconvenience. This matter has created. As previously stated upon discovering the ransomware attack, we took immediate steps.

To protect our systems, contain the incident and begin an investigation. And ensure continuity of care. Our facilities are following existing protocols for system outages, which includes taking certain systems offline, such as the electronic health In addition, we are taking steps to mitigate the disruption and maintain continuity of care, to further assist and support our team in the investigation and response process.

We engaged leading cybersecurity specialists and notified law We continue to conduct a thorough forensics investigation and review of our systems. And we'll also seek to determine if there are any data impacts as part of the process. System serving dignity, health and Virginia Mason medical center have had minimal impacts on operations by this incident.

For the other parts of our health system. , that have seen impacts on operations. We are working diligently every day to bring systems online and restore full functionality as quickly and safely as possible. Let's see central to our decision-making has been and will continue to be. Our ability to carry out our mission in a manner that is safe and effective.

To those we serve at common spirit health. We are dedicated to meeting the needs of the communities we serve and are guided by our core set of values, which includes integrity, excellence, and collaboration. We are grateful to our staff and physicians who are doing everything possible to mitigate the impacts.

On our patients and ensure continuity of care. All right, so that's, what's on their website. , let's see, where , over here at Becker's story. It gives us a little detail on what is shut down. , so this has been going on for a little over a week. And let's see EHR shut down. We've already read that.

Second largest nonprofit hospital chain. That's important piece of information. , comms for said subsidiaries. We already talked about that. And is there anything else new in

No. Not really. , here's what we do know. This is impacting calm spirit hospitals in a state of Washington. Texas. Tennessee. And I think, , another Northern area. , essentially if I'm reading this correctly,

, that should be something that's actually planned. Prior to the. , consummation of the deal. Right. So you should have your security practices all mapped out. It's actually one of the first groups that begins to talk between the two organizations, because there's no. Competitive things that are going on in cybersecurity. We do not compete on cybersecurity.

So as the deals. Sorta going through its process. One of the groups that you get together almost immediately. Is the cybersecurity teams to talk about how you're going to connect up, what are the practices, if there's a different audits and things that are necessary. To move forward. All right. My, so what on that says, I'm glad they're finally getting out there. I was a little concerned that they were. They were really tight lipped. We didn't know a lot about what was going on. They should have a PR person that's way out in front of this thing, because there's people in those communities who were asking questions.

And a PR person should be talking to the local news, talking to , any kind of local media. That's going to get the word out to the patients of what is going on with those hospitals. Think about this there's chronic conditions. It's people with ongoing care there's people with. , surgery set up there's people with just basic appointments set up.

That were getting turned away or still getting turned away and they have questions. They need those answered and it's better to get out in front of that than behind. And the fact that they were not leads me to believe that the. , practices were not practiced. , I was talking to somebody today about tabletop exercises and the value of the tabletop exercises.

Is, , , you don't get attacked every Hopefully something that doesn't happen all that often to you, but when it does happen, you want it to feel like it's something you've done before. And so the tabletop exercises are good. You know, you just have somebody in there it's, it's like playing Dungeons and dragons, except you're doing it with, you know, you have your dungeon master, who is the person conducting the tabletop exercise and you have the team there. And instead of going on a journey, they essentially go on a fictitious here's what's happening to your system.

And it's really interesting because if, you know, if somebody is in that room, like I was in the room once and, you know, answering questions and that kind of stuff, and , the person running the tabletop exercise said. Okay. , your family has been impacted by the disaster and you are no longer available to the team. You can not talk

And, you know, as a CIO, I had a lot of answers. I knew where a lot of things were. All right. So now I'm out of the equation. They have to figure it out. So that's what tabletop exercises do. And they also identify gaps. Do we have a communication plan? Do we have, , , good backups and restores and that kind of, because you're walking through the entire process, a good tabletop exercise.

Is a immensely valuable. So, you know, I understand not knowing a lot of details, like what was the entry point and what was compromised and those kinds of things. So this is a crime scene. This is officially a crime scene. And as we heard in past. , webinars that we've done, we did a webinar. It was a phenomenal webinar. If you have a chance to go back and listen to it, you should.

It was with sky lakes, medical center, and a Santi and skylights was compromised. The Santee was their community connect parent. If you will. And we talked to those two CEO's and we talked to a cybersecurity first responder, and we talked about that entire incident from beginning to end. , it was actually riveting. It would have been good television for heaven sake, and it was a great webinar.

And one of the things that they learned pretty early on is that when you have an incident and you report it and the authorities come in, either the FBI or others You lose control almost immediately. They like tape it off , like a crime scene. And say, okay, your it, people can't touch the keyboards right now until we figure out what's going on because there's, it's a crime scene, right? A crime has been committed. We want to make sure that we preserve evidence and do all that stuff.

And, if you haven't gone through the tabletop exercise, you're just learning this. Now, the other thing they learned in that process, Was it's important to know what your cyber insurance. , says because , one of the first things you do is you go to that policy and you read it and then you realize, oh my gosh,

We can't hire our normal consultants. Come in here. We have to hire one of these five people that we've never worked with before. , there are stipulations in that contract. If you want to file a claim, there's stipulations in that contract that you have Again, Not it shouldn't be the first time you're doing this when you're going through the process. So very important.

To get in front of these things. By the way, all these things I'm talking about are things I. Somebody this week said. , as you know, because I, I do this kind of show and whatnot, people will call me up and say, Hey, what do you know. And, you know, with cybersecurity professionals and whatnot, I've talked to a couple of them this week. I said, look, I don't know any specifics.

, but we knew early on it was ransomware. It's just had too many ear markings of being a ransomware event. So I said, you know, it's a ransomware It's taking down these locations. It's now a crime scene. I gave him all that information and one of the questions was. You know, my board is asking me how we should be looking

And I think here's what I'd be saying to the look. First of all, I'd reinforce the work that you've done to this point. Hey, we have a matrix. We have done an , assuming that you've done this, we did this at St. , we've done an assessment of our security. You're aware of that. The, , the security.

, subcommittee of the board. , it gets briefed every time we come together on what our current security posture is, where we're strong, where we're weak. We cannot be perfect on all of these things. So we know there is going to be an event. We know that someone is going to get into our It's just sheer numbers. You cannot protect against this. It's not if, but when that they are going to get in.

Right. So our system had 20 something thousand employees. 20 something thousand employees. Somebody's going to click on a link. Give away their password or do something right. It's just human error is going to happen. Or somebody into your data center is going to do something. So they're going to get

And so we talked about this on Monday. My job is to control the blast radius. So our investments are about controlling the blast radius. So minimizing the damage they can do if they do get in. And then being able to roll back. Roll back quickly. Roll back effectively. Right. Cause you can't just say, oh, I'm going to restore. Cause if you restore from yesterday and that, , code is already in place, they're just going to relaunch the attack.

Right. So you have to be able to identify, , when you were infected enroll back prior to that. And what I would do is reinforce, Hey, we have this, we keep you informed on this thing. We have made investments in a detection. So we want to know as soon as something odd is going on in our network. We have made investments in remediation and we've made investments.

And recovery. And that's where I would be focused right now. The ransomware event is, is really highlighting and showcasing the need. For strong recovery practices. First of all, you need an, an immutable backup. You need the ability to have a backup that the ransomware attackers can not get Right. Their whole proposition is based on the fact that you cannot recover. If you can recover, they're not getting paid. So they have to make sure you cannot recover. They're going after your backups. You have to make sure you have an immutable backup. You also have to make sure that, , some of the things which are your crown jewels are protected.

So, you know, one of the things they're gonna want to do is they're gonna want to escalate privilege as quickly as possible. They're going to go after your active directory. Active directory runs everything from. Gates to your garages, to access to your buildings, to access to your data assets across the entire enterprise. They're going after doctor directory. If you have not protected your active directory at this point.

, you've made a mistake. If you can't roll that back easily, you've made a mistake. And in a couple of these cases, what we've heard is systems. , their active directory gets compromised. They cannot roll And just short of having to rebuild it. And by the way, rebuilding active directory, almost impossible.

In a. Domain that has 30,000 users let alone 900 applications let alone thousands of servers and other assets, it's just impossible to rebuild. And so you never want to rebuild it. You're going to restore from a backup. The backup could be a couple months back. If you, , are compromised. And so you want something where you can roll it back. Something that is resistant to ransomware.

So, if you're wondering how to protect your active directory, take a look at Sempra. So if you're wondering how to do the immutable backups, Take a look at Rubrik. These are just a couple of companies that happen to be sponsors, but a couple of companies that are out there that you could look at. If I was talking to the board, I would say, Hey, we were protecting our core.

, assets in this way. We've made investments in these things we are doing. , exercises. We are exercising our it organization. We're exercising our practices in the hospital. One Well, a lot of times we take down times at night and you know, our night shift is phenomenal at running on paper or running on backup systems and that kind of stuff. But our daily operation is not that good at it.

Right. So that's one of the things that happens in a ransomware event. Your your day shift is really experiencing a major outage potential, hopefully, but potentially for the first time. And then we find out, do the paper processes really work or do they not work? I mean, one of the things, sky lakes was interesting. They ran out of forms. They ran out of paper.

, you know, it's just some of the things you just don't think about. And, the other thing is their, their applications weren't tiered that well, , the spaghetti, that is a health system, , you think, oh, well, , we're going to restore tier one applications But in many cases, tier one, tier two, tier three applications.

Th they feed each other. Right. So you can bring all your tier one applications back up, but it may not function as a system unless you have the tier two and tier three applications

so again, if I were talking to your board, I would re reiterate our position where we're at. The investments we're making the communication. We're having the work we were doing around, , being prepared and, , exercising our operations. If they ask me specifically about. You know what they're experiencing at common spirit right now, you know, it's a crime scene.

, they are learning some things for the first time. I would tell them, Hey, I've, I've read our cybersecurity policy. I understand the players. They're familiar with us. We are familiar with them. We know what this policy. , entails and what it doesn't entail. We understand what's going to happen when we call the FBI and we know the FBI's phone number. Let's just start there.

, You know, if you're caught flat-footed and we will know pretty quickly if common spirit is flat-footed. If they're flat-footed they are going to take. Greater than 30 days to restore operations. , potentially 60 or 90 days. And then the question becomes how much data do they actually lose? So.

, that's some of the things that, , that you could talk about in terms of the common spirit, and we don't know much. , to be honest with you at this point. , and by the way, that communication plan is so key. As I mentioned earlier, And I would make sure that the board understands the importance of investing.

Time in preparation because the time you spend in preparation, or the money you invest in the tools and technology, to be able to roll back, that is going to minimize that time and impact that data loss. And hopefully. I bring it down to nothing. Hopefully. All right. So that's all for today.

My. , thoughts are with the people at common spirit. I've been through two breaches. , neither of our breaches would even be remotely like the breach that they're currently experiencing. But with that being said, , they're still major events. Everything shuts down, everything gets focused on the event itself.

, you have people working ridiculous hours, trying to get things done, doing some work that they've never done before at a level within the system that they've never had to do before being asked questions. About the system. It's almost like having a forced upon you. Final exam is like, Hey, where's this document? Where's this.

, does anyone have a document that will show us this? Does anyone. No, where we can find this. And to be honest with you, sometimes it's as silly as, oh, that's all on SharePoint. Well, SharePoint's been compromised. We can't get to it. And you're like, Well, that was a mistake. We should add multiple places to get that information. So it's tough. I really do feel for them. I know it's going to be.

A long couple of And, , I wish them the best. All right. That's all for today. If you know someone that might benefit from our channel. Please forward them a note. They can subscribe on our website this week. health.com. Or wherever you listen to podcasts, apple, Google, overcast, Spotify, Stitcher.

You get the picture. We are everywhere. We want to thank our channel sponsors who are investing in our mission to develop the next generation of health leaders, Gordian dynamics, Quill health tau, site nuance, Canon medical, and 📍 current Check them out at this week. health.com/today. Thanks for listening. That's all for now.