This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.
Today on This Week Health.
A digital company thinks differently than a physical company. Most of us are both, and so you have to go with that perspective of the one that is the most vulnerable, which is why we take the same approach to all tier levels.
all right. Welcome to a solution showcase. Today we are gonna be talking about automation in the automation journey that Tivity Health has gone through, and specifically we're gonna talk a bunch around the security automation as you're making this digital transformation, making sure that you have the right software defined architecture in place so that you can maximize the use of your team and maximize your assets and as well as move fast and protect those assets as you move. So great conversation. We have Sarah Richardson, Chief Digital and Information Officer. We have David Giambruno G, who's the IT Operations For Tivity Health and Shane Allen who is with RubriK and my name is Bill Russell. I'm a former CIO for a 16 hospital system and creator of this week Health Set of Channels dedicated to Keeping Health IT staff current and engaged.
We wanna thank our sponsor for today and that is Rubrik we also have, as part of this, there's a download. You can go to gothisweekhealth.com/rubrik. It's a great white paper. In fact, I downloaded it, read it this morning, how healthcare organizations can develop an effective ransomware remediation plan.
Great document, great handout worth reading. I learned a bunch from it and I think you'll get something from it as well. So take a look at that. And now onto 📍 the show.
Today we are going to be doing a solution showcase, and we're going to be highlighting the automation journey Tivity is on. We have Sarah Richardson, Chief Digital and Information Officer with Tivity Health, David Giambruno, affectionately known as G, who is the VP of Infrastructure and Ops, Tivity Health, and Shane Allen, Platform solution Architect.
Welcome to the show everybody. Thank you. Morning. Thank you. I feel like I'm learning more and more about Tivity every week cuz we just had a webinar with you guys and it was a phenomenal conversation, but I'm really looking forward to this one.
We're gonna talk about the automation journey, but clearly with what's going on in the world around ransomware, anytime we're talking about architecture, we have to talk about security and we're going to be talking about that as well.
What I'd like to talk to you about, Sarah, you come in, you're sort of looking at the environment. How do you think about digital transformation as you're looking at the environment from a leadership perspective.
A bit two-sided to a degree. So there's the part of me that before I take on any assignment internally, externally, et cetera, cause I mean, I had a great job when Tivity showed up on the doorstep as I call it, and I had to seriously evaluate, do I wanna leave an environment where I'd been for five years? Establish teams, establish work streams, et cetera, and go do something new. Before these, at this phase is in our career, before we say yes to something.
It has to be compelling to be able to do something new and different. We've all been doing turnaround plans. We've all been doing consolidations, EMR upgrades, like all the things we've done before. I'm in a space where I'm like, I don't wanna do something I've already done before. And that was sort of the value proposition as I brought in.
All three of the VPs are net new. Gee, one of them, we have two others. One has been here 17 years, thank goodness, because she remembers where all the things are buried that we need to avoid in the future. There's that space that says you have to do something you've never done. . And when you bring the individuals together with these diverse and incredibly capable backgrounds, you get to, I'm just gonna make it the word exponentially, that in and of itself.
And so when I looked at Tivity I asked a very honest question, What's the maturity level of it? And I was given a number and I said, What's our appetite to improve per number per year kind of thing. And I was given an honest. We're tracking to that, we're able to start to accelerate components of that because of the introduction of technology with a plan for automation or automation from the get go, with the ability for people like G to run in the direction that you know and create new capabilities at the same time, because I trust him, we trust him.
We as a team are trustworthy. One another. To say you can even be a little bit experimental over here on some of these pieces. With our partnerships with people like VMware, with Rubrik, with aws, we get to play as much as we get to produce. That's a pretty unique space to be in with a team. It's a pretty unique space to be in organizationally, to have the trust of your peers at my level and my boss, who's the ceo, as much as the trust and capability of one another to deliver on those ideas and we go in with a plan that says this is the minimum viable that we're gonna get out of this. And it's still pretty exceptional based on industry standard. We set ourselves to two and three x higher than that on what we're actually gonna deliver. So we set an expectation that's very reasonable and very approachable.
We always endeavor to outperform ourselves. And that's that lens of like curiosity and creativity that has to come with the confidence to really drive true change in an organization That change is the underlying factor of any kind of transformation you wanna do, digital or otherwise.
As a leader one of the things that we're always looking at is how, how can we be the most efficient that we possibly can be? And this automation journey really helps us to address one of the biggest expenses we have in it, which is constantly adding staff. I. especially my analyst team, they would come to me every year and want to double in size. Because healthcare has an insatiable appetite for creating reports that they'll only use once.
but that can happen in operations, it can happen in security, it can happen in a lot of different areas. Talk about what the, the design principles were as you set off as the IT leader at Tivity.
So if you kind of think about it, it was, if we can't automate it, we don't need it. And so it was a fundamental premise and, it was kind of fun, Sarah and I going back and forth, cuz I look at it as a conveyor belt, right?
So you kinda onboard a technology or a capability, you put it on the operational conveyor belt to automate it, so then you can add more, right? Cause to your point, if you don't automate it, you're just gonna have to keep stacking everything. , and then you add people, you add complexities. So the thing that we did, I think incredibly well was automate, And part of the foundational stuff is Dr.
Right? Business continuity. I've got a history of doing it in real life, like I've been through it, right? So I've had a, I've had a factory blow up. So I know what it's like and what it means. We started from the ground up, essentially building all this automation capability.
We talked, you, I kind of mentioned VMware, but. Rubrick and automating the DR so we don't have to think about it. Is one of the foundational qualities and the not think about it is on two levels. A doesn't work in can recover. Cuz we look at as we should have three to four buttons.
Whether it's a file, a server, or a data center, or an application. We don't care. It's the same process. And if for some reason our multi-layered approach to security, something. We've got the ultimate undo button, right? So we have the ability to unwind and restore if we get breached and have ransomware. So taking away the fundamental fear while introducing capability, right, is really foundational. You're free of time, you reduce risk, and you end up having good conversations with leadership teams.of:
And I would imagine that number hasn't gone down much. FBI estimates cyber criminals will earn over a billion dollars in ransom. Advanced ransomware now targets backup files delivery via a range of mechanisms such as phishing emails and exploit kits, et cetera. And it, it just, it sort of lays out the framework. Why is automation, why is Rubrik in this conversation around automation? It would seem to me it's, Really around security, but what are organizations looking at Rubrik to automate and to protect within their environments?
Well, they're looking, they're looking at protecting everything within an environment.We started many years ago in:
So you take that, the automation capabilities of Rubrik and then we've added a whole bunch of data observability on top of Rubrik So you know when you can talk about data security. Now we can very quickly go into Rubrik Rubrik can let you know what's happened to your data, and then be able to surgically recover any type of files or applications very quickly.
So when you talk about ransomware, the whole key of ransomware is to be able to, one know what's affected, make quick decisions, and then recover quickly. So those are the key pieces of RubriK to allow companies to react to a situation that maybe other companies that don't have this visibility into their data can't recover in time.
Yeah. So, gee, let me come back to you. I mean there's a lot of people listening to this going, that's, that's pretty amazing. That software defined data center having a set of APIs, you can program in there. You can have alerts come from one system, kick off a process and do those kinda things.
And then there's a group of people that are hearing this going, Yeah, but my gosh, my legacy environment is not gonna allow me to do a bunch of this stuff. How did you make that transition? How did you go? the legacy that exists to making this a reality, a software defined architecture that can be automated,
we moved to a full SDC once we were there. I have this unconventional view which I'm sure will go over super well. Fundamentally I don't tier recovery. I just copy everything. You end up copying everything.
And in an enterprise it's a spaghetti chart, right? I have this great, like one, this one visual. This is what the back end of a company looks like, right? If it's over 10 years old, it's just like literally thousands of connections. And so the dependency model is horrific. Breaking it all apart and trying to stack rank it is, hard.
And invariably the tier one has somewhere in, its, its application supply chain a, requirement for tier three or tier four. So instead of trying to blow your brains out, we literally just copied everything. And when we did that, it gave the opportunity to use the automation and app flow that they.
To automate all the recoveries. So there are some, like, especially particularly databases, in some of our apps, you have to have the steps, but the data's already there. If the data in the application's already there, everything gets way easier after that. And so the ability for us to, to automate and we're literally at the point where it's a help desk function, it is not an engineering function any longer.
That's how automated it's, and that's always my goal is, If something bad happens. we had an app server bar for a couple weeks ago. it took us longer to do the P one alert than it did to recover the server. And it was done at the help desk level. And then, and then I've been bugging Sarah to the account as a DR test. Right.
Yeah. Well, no, that's, that's a good point. We're gonna get the, if, if by the time you're notifying people that there's a. You've solved the problem. Is that really an outage?
Well, so people notice, so the rule is if they don't notice, I don't call it an outage.
Okay. Yeah, that makes sense.
You don't see it. and to Sarah's part, we're getting there. I mean, that's not far away for us. Right. Where it's completely transparent. And that kinda leads to the, digital transformations. If you're gonna run in the digital world, the metrics and how you hold yourself accountable change greatly because you're now exposing essentially the user experience that people have had or grown used to.
Your internal infrastructure is the same as the external. So you have to think like a cloud, act, like a digital application. It's the user experience, it's the uptime, you're tied to revenue. I'll argue that even. Kind of DevSecOps. We're more GI ops. We're moving into the GI ops world where everybody lives in a, in a get.
And to kind of Shane's point where everything's gonna live in a C I C D, continuous integration, continuous deployment where everything's turning to code, where we can fire our application into our data center, into aws, we really don't care. But at a, it's backed up the entire. B, it's essentially immutable with the rubric platform.
As we get into that digital space, we don't really even care about recovery. We care about the code, we care about the data's there and that it's immutable. But to rebuild an app, to redo anything, it becomes super easy. And then we start using rubric for analysis. And additional like signaling, do we have somebody in the infrastructure?
Is something weird going on in our data? Adding the layers of defense in depth, right? Because we're kind of in the mutually assured destruction world of cybersecurity where everything ratches up every day. And so the combinatorial effect of that, Us being super fast, the ability to make things not matter where Sarah doesn't have to think about the infrastructure or the data or the recovery, and that people could look forward to, again, put time into the transformation and not time into operation.
Shane, I, I wanna come back to you. Because GE used this whole concept of immutable file system, and one of the reasons. Enterprises are unable to recover from ransomware attacks, is that the backups become compromise, which forces the IT teams to either pay the ransom or restore, which can be time consuming as we're seeing with a current ransomware attack that's going on. It's now three weeks without the EHR talk. Talk to us a little bit about the native immutable file system. What is that about and How does that help to at least expedite the recovery from a ransomware attack?
It's true. That started with our initial design. We call it a bunker in the box. We built our own file system, and our file system doesn't have the ability to change data once it's written to that file system, and we don't ex. That file system with traditional protocols like NFS or SMB to the network. So we're not like a storage device where the backup application just dumps to us.
So once we bring data into rubric, we do CRC checks and fingerprints that data. And only after those checks is that internal writer service be able to write it down to our file system. Our file systems never exposed to the end user. There's no access to it. People can't see it. We don't have, I said we don't have the ability to delete it, and we put all these other
things in place On top of that file system. We can't run third party apps, right? We end in encryption between the nodes, so the zero trust architecture. We don't even trust the nodes that talk themselves. Everything is TLS design TLS sign between the nodes. We secure logins with t o, DP and mfa. And on top of that, we have retention lock on those backup policies to where no one can reduce the frequency, reduce the retention, or pull objects outta that backup policy.
So once an object lands in a rubric, it's immutable on that first copy, and we can guarantee the recovery of it because no one can accidentally delete it early or gain access to it through the UI or the admin CLI or any other method. And the only thing that can delete that backup po that backup object is that SLA policy saying expire after X amount of.ent design back in the day in:
that's fascinating to me as a former cio. I wish I had something like that back, back in the day. But as I'm thinking about this one of the things that happens in a ransomware event is we start to restore and we're actually restoring files that have been manipulated or infected, right? How does rubric address that challenge?
They addressed it with everything that Rub does. Everything we bring into Rubric we index. So you think about it, we index all the file systems. We know how many files were added, her change modified or deleted. So now we have that data observability. So now we can start watching the data and so we can see that, hey, if a large amount of change happened, a lot of files were deleted and we see this anomaly.
Sort of look at the interview, the file, has that file been encrypted? We'll let you know if that file has been encrypted and our systems have been encrypted. Now only that we have the ability to. Take and look for IOCs of an affection across the backup history of back within the backup appliance.
Say if, if I have been affected by something, go ahead and look for the signatures of that backup and find the clean backup that I can recover from. So, oh no, only can tell you the point at which the files became encrypted. And then we go to that point. We can also look at the history across and say, When did that system, when did the payload come onto that system? And let's restore it right from that.
Yeah, that's fascinating. Sarah, the world's changed. You're not nearly as old as I am. Today is my birthday actually, so I'm actually older. But the world's changed. I mean, I, back in the day, I remember I was pushing my team to get rid of the tape backups that were in our data center because we had a person dedicated to swapping out those.
We had robots and stuff, but we still had to swap those tapes, send 'em over to Iron Mountain and do that whole thing. Has that whole process just changed the way we think about backups, the way we think about restoring and what's driving.
Well, the modernization of technology is, it's always gonna drive it a capability to take that next step. What we all fail at or have historically is keeping up with the technology that allows us to be as safe as possible for a given moment. Everybody's got budget constraints and everyone's got capital, opex, humans, et cetera. But I do, I remember the days when, heck, I worked at the county hospital. It was my first healthcare job, and we backed up our data, but the safe was still on site.
It was just another building. I'm like, that's not exactly safe. And so I remember saying we couldn't just call Iron Mountain for. that And there's a space where we've all evolved. So the other space too is like, are your vendors evolving with you? Are they creating the solutions that allow you to move into that next generation, the next capability?
If you're not constantly curious, you're not constantly aware, you may not know that you can use RubriK to be as safe as possible in a ransom space. There are other, and there's very few others that give you that same capability, but you have to constantly stay up on it and you have to prioritize your spend in different scenarios.
Like when G and I got here, there wasn't Dr. And I hate to admit that publicly but You're backing things up. We weren't backing them up in a way that was gonna be recoverable. It was literally backing up to the same data center. We're like, that's not disaster recovery, because if a meteorite or some other natural disaster came your way, unlikely in Phoenix.
So you really have to think about the diversification of your assets, of the way you wanna do business, of the way you want your teams to function. One of the things I love about you hear what G and I talk about is jobs need to be about keystrokes. And the automation capability, that it's not about the elimination factor of certain functions.
It's about having people do new and more creative work towards where the organization is headed. A digital company thinks differently than a physical company. Most of us are both, and so you have to go with that perspective of the one that is the most vulnerable, which is why we take the same approach to all tier levels.Heck, back in:
So unless you're always looking at that next action, you need to be taking. You're never going to be good enough. You have to constantly be thinking about how you partner to make things better. So we always are bringing our vendors together to say, Hey, we picked the three of you to be our footprint. How do you best function together?
Some of the best conversations you have are when your three top vendors are all in the same room solving things together with you. And it's not a one size fits all. There's uniqueness in all of us. So how do you meet those needs effectively?
I'd love the fact that they have to, your vendors have to partner with you and keep evolving.
I'll also say time, right? So, Tape won't make it right.
The speed and the amount of data.
The volume of data. I had a, when I first was going to get rid of tape, it was one of the funnest discussions I ever had, right? Cause I was like, We have 12 robots. The recovery time is 25 megs a second. We have 1.2 petabytes. It's about nine months. Is that.
as a cio, I can't tell you how many times I did math problems. It's like, okay, so this much taped, this much bandwidth, this amount of thing. Let's do the math. Here's the math. Is that okay? No, it's not. Okay.
And then as you rotate forward, if the way we have not only crazy data generation, but data driven applications, right? Even APIs service models, they all require data. And so you have to have the data wherever it's needed.
So in a dr, no matter what your model, if your data isn't there, you're not recoverable. You can try and you can wish really hard. So whatever you do, whatever your model. Your data has to be there. And so that's part of the rubric choice was we replicated everywhere. So wherever we need to recover, it's there and available. We can solve almost anything if all the data's there.
All right, so as a former cto, here's, here's some of the design principles I've heard you guys talk about at this point. One is the immutable file system, I think is, is critical in terms of protecting against. The types of attacks that we're having today.
One of the things I heard early on from UG was we've turned it into the help desk, Does the restore? It used to be that has that had to go through the bowels of it and multiple conversations to get to that point, and now we've automated it to the point where somebody can press a button and say, Oh, that's not right.
Let's restore that and we don't have to go to the expert because the experts already weighed in and put the code in. And the automation in there. So it's accessible by someone who doesn't have to have 10 years of architecture experience. And now we've freed up that. We've freed up a lot of things.
One is we've restored a lot quicker because we hadn't, we don't have to go through the bureaucracy and the change management and stuff. We just do it. But the other thing is we freed up that very limited resource, which is those highly skilled experts and their time and now they can work on even. getting Ahead of the next thing that can be automated, whatever that is.
So those are some design principles I've heard. Software defined, I think is another design principle. Software just moves faster than hardware, right? Well, we just talked about the tape backups
and you start, it's abstraction, so I'm not married to hardware any longer. Right. So whether it's in my data centers or in my cloud. I can just move it. Yeah. As long as the data's there, right? I can make it work pretty much anywhere.
So, gee, the last time we got together, you said the quote the reason God could create the world in seven days is cuz he didn't have legacy. but that is one of the problems.
And, and this gets to what you were saying, Sarah, in that we need to evolve the attacks of change, the speed of business and the business change is getting more rapid. The dynamics of the attacks are getting different. So everything about this is agility, speed, security. And it used to be, well, hey, you can move fast, but then you're gonna, if you move fast, you're going to compromise your security in today's day and age.
It's gotta be all of 'em, right? It's not, You gotta move fast and you gotta move securely and you gotta be able to change,
So, in a digital transformation, the ability to do experimentation is huge, right? Cause you're, you're gonna try new things and you're gonna have to go from my think to, I know, right?
And you're gonna do things that haven't been done before. And there's no playbook, there's no best practice. It's literally kind of o j. So one of the things that the, and you've been circling it, is our ability to copy our environments and try new things. So whether it's refactoring, whether it's replatforming, I can now recover into an environment and try those things super safe.
And I just hit the delete button after If it doesn't work so what? Yeah, all of those constraints that, that you had as a C I O and the CTO get. So I can just make a copy, I can launch it, I can play with it. I can try things, I can experiment. No big deal, Super easy. And that's a huge geometry change in time and effort.
Yeah. So if you're a CIO sitting there saying I don't have dev test prod set up for some of my major systems. Essentially, Gee, what you're talking about is DEF dev test prod for the entire system. Yeah. And that's, that's incredibly powerful.
And it's software. I don't need it to run at full capacity. I don't need that. Right. Because almost everything's a logic issue now. Capacity the test later.
this is exciting. This is I love, I love nerding out on this conversation. Shane. What other things should we know that I maybe I haven't hit on at this?
Well, I mean, think we've hit on everything from data resilience, data observation, data remediation, the key things that companies are looking for from a modern backup solution being able to say, Yeah, can I protect my data? What's in my data? And then once I have those decisions, how can I quickly recover from the data? And IT, and rubric makes that very easy. And to get to David's we're software defined, however, to set up a blueprint to set up the recovery is rather, is rather simple.
Define what objects, define what boot order, define how you're gonna fill 'em over. Define what networks are gonna go on. Makes it very simple for someone to say, I need to recover this application. So as is taken and transform just from a backup product, but into a product that companies can rely on through backup, Safeguard the data and recover the data very.
this is an exciting topic. I love what you guys are doing at Tivity. Sarah seems like everywhere you go, you take 'em into the 21st century. So I appreciate the work that you're doing. I appreciate you coming on the show and sharing your. automation journey. And g, always great to talk to you.
Thanks everybody for for being here. And if people want more information, again, you can hit that download. It's go.ThisWeekHealth/Rubric, R U B R I K, and download if you want some more information. And you could also just hit the rubric website and find it there as well. So thanks again for coming on the show. Every.
Thanks, Bill. Thank you. Thank you.
What a great conversation. it's always fantastic to catch up with Sarah and then to have Shane and G there as well to really get technical on the new software defined data center was fantastic. So really appreciate. And really appreciate the conversation. We wanna thank Rubrik, who is our sponsor for this episode, for investing in our mission to develop the next generation of health leaders. And I wanna thank everybody for listening. That's all for now.