Transcripts
W. Curtis Preston: Welcome to the exciting final episode in
Speaker:our series on cloud disasters.
Speaker:During which we conclusively answered the question.
Speaker:Do I have to back up my data in the cloud?
Speaker:The answer is an unequivocal.
Speaker:Yes.
Speaker:Not having a good backup and disaster recovery plan for your data in
Speaker:the cloud is nothing short of an existential threat to your company.
Speaker:Two out of the 10 companies we covered in this series immediately went out
Speaker:of business after what happened.
Speaker:And another killed off an entire line of business.
Speaker:The cloud is an amazing place and I consider myself an
Speaker:advocate, but it is not magic.
Speaker:It is just someone else's computer.
Speaker:In this episode, we will summarize the lessons that we learned and the major
Speaker:arguments that we settled from our coverage of no less than 10 disasters.
Speaker:If you're just joining us, hopefully this episode will wet your appetite
Speaker:enough to have you go back and listen to, or watch those other episodes.
Speaker:We had a good time making them.
Speaker:And we also learned a lot.
Speaker:By the way, if you don't know who I am, I'm w Curtis press an AKA Mr.
Speaker:Backup.
Speaker:And I've been passionate about backup and recovery for over 30 years.
Speaker:Ever since I had to tell my boss that we had no backups of the
Speaker:production database, we just lost.
Speaker:I don't want that to happen to you.
Speaker:And that's why I do this on this podcast.
Speaker:We turn unappreciated backup admins into cyber recovery heroes.
Speaker:This is the backup wrap up.
Speaker:W. Curtis Preston: Welcome to the show.
Speaker:I'm w Curtis Preston, AKA, Mr.
Speaker:Backup, and I have with me my trauma sympathizer Prasanna
Speaker:Malaiyandi, how's it going?
Speaker:Persona.
Prasanna Malaiyandi:I am good, Curtis.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Um, it's interesting of all the things you can complain about or that are
Prasanna Malaiyandi:wrong in this world for you, the biggest issue is different operating
Prasanna Malaiyandi:systems using back slashes differently.
Prasanna Malaiyandi:W. Curtis Preston: Y.
Prasanna Malaiyandi:Yeah, so.
Prasanna Malaiyandi:It's because I'm being forced to operate a Windows based net backup server.
Prasanna Malaiyandi:I.
Prasanna Malaiyandi:And I only know how to work in like Unix land, like when I'm doing automation
Prasanna Malaiyandi:and scripting and stuff, which is the only way I know how to do anything.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:I, I, I don't understand.
Prasanna Malaiyandi:I, I guess people, if they grew up in Windows Land, they learn PowerShell.
Prasanna Malaiyandi:They learn, you know, batch programming and what I didn't learn all that stuff.
Prasanna Malaiyandi:I learned shell scripting.
Prasanna Malaiyandi:And so when I want to administer a Windows-based system, the first
Prasanna Malaiyandi:thing I do is install siggu so that I have a Unix-based environment.
Prasanna Malaiyandi:But when I'm interacting with, um.
Prasanna Malaiyandi:Files.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um, so sig one is nice in that it translates all the file directory names
Prasanna Malaiyandi:and everything into slashes, right?
Prasanna Malaiyandi:So it's, I just have to, um, sort of figure that out.
Prasanna Malaiyandi:And basically, like Deco backslash becomes slash sig drive slash d, so
Prasanna Malaiyandi:you just sort of, that's not too bad.
Prasanna Malaiyandi:But in the case of net backup, I'm doing things like.
Prasanna Malaiyandi:I am running commands that output file names that sometimes are in Windows
Prasanna Malaiyandi:format and sometimes are in Unix format.
Prasanna Malaiyandi:And, um, the, what I've recently discovered is that, and it took me,
Prasanna Malaiyandi:it took me literally just banging my head into the wa into the wall
Prasanna Malaiyandi:a few times to figure this out.
Prasanna Malaiyandi:Is that, um.
Prasanna Malaiyandi:Back slashes, behave differently.
Prasanna Malaiyandi:Um, you know, like I wanna massage them and the first thing I wanna do
Prasanna Malaiyandi:is change 'em and four slashes, right?
Prasanna Malaiyandi:So that I can, so that they're not gonna be escaping everything in my unit.
Prasanna Malaiyandi:She script, right?
Prasanna Malaiyandi:Um, they behave differently on like in standard in then they do in a file.
Prasanna Malaiyandi:So if I take the output of a command, I.
Prasanna Malaiyandi:Such as BPPL include, which is, you know, show me the files that are in this policy.
Prasanna Malaiyandi:Uh, and I pipe that directly into a, uh, you know, into a series of
Prasanna Malaiyandi:pipes to massage the, the output, uh, that backslash behaves differently.
Prasanna Malaiyandi:I have to escape it differently if I'm just doing a, a straight.
Prasanna Malaiyandi:Yeah,
Prasanna Malaiyandi:W. Curtis Preston: if I out, if I take the, the command and actually
Prasanna Malaiyandi:output it to a file and then
Prasanna Malaiyandi:and then look at it.
Prasanna Malaiyandi:Um, and then basically I have to es like if I'm doing it on the command
Prasanna Malaiyandi:line, I have to escape the escape twice.
Prasanna Malaiyandi:If I'm doing, if I'm doing it in a filing, I only have to escape it once.
Prasanna Malaiyandi:And that's been driving me just absolutely bonkers.
Prasanna Malaiyandi:So I think what you do is you just add a whole
Prasanna Malaiyandi:bunch of escapes in there, you know,
Prasanna Malaiyandi:W. Curtis Preston: I
Prasanna Malaiyandi:slashes
Prasanna Malaiyandi:W. Curtis Preston: it works.
Prasanna Malaiyandi:I'm just like, does two will two work?
Prasanna Malaiyandi:No.
Prasanna Malaiyandi:Two, no, no.
Prasanna Malaiyandi:3, 3, 3.
Prasanna Malaiyandi:No.
Prasanna Malaiyandi:And then three, like three, uh, tells me that I, that I have too
Prasanna Malaiyandi:many and I have like, uh, it, it, it, it escapes like something
Prasanna Malaiyandi:later and I'm like, no, no, no, no.
Prasanna Malaiyandi:Okay, so I won't take four.
Prasanna Malaiyandi:You know, and I just, I just do a bunch of them until.
Prasanna Malaiyandi:I get what I
Prasanna Malaiyandi:It works.
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:It's like, it's like, uh, it's like writing code, right?
Prasanna Malaiyandi:You compile it, it fails.
Prasanna Malaiyandi:You're like, oh, I'll fix this, and then it fails again.
Prasanna Malaiyandi:It's like, oh, I'll fix this, and then you just keep going until
Prasanna Malaiyandi:eventually you get zero errors.
Prasanna Malaiyandi:W. Curtis Preston: And, and, and then, so I, I find that as annoying as this is, I
Prasanna Malaiyandi:find that it's actually easier to take a net backup command, output it to a file,
Prasanna Malaiyandi:and then hash through that file, rather than just doing it as a series of pipes.
Prasanna Malaiyandi:And as annoying as that is, it's more, it seems more consistent, um, albeit slower.
Prasanna Malaiyandi:And it's also slower because the first thing I have to do on that
Prasanna Malaiyandi:file, you know what I have to do?
Prasanna Malaiyandi:I have to run the dos2unix command on it
Prasanna Malaiyandi:because of the, the new line.
Prasanna Malaiyandi:Because the new line is also different.
Prasanna Malaiyandi:Yeah,
Prasanna Malaiyandi:W. Curtis Preston: It, it's been fun is what I would say.
Prasanna Malaiyandi:Oh, Curtis
Prasanna Malaiyandi:W. Curtis Preston: Ah, so, uh, so speaking of fun persona, we
Prasanna Malaiyandi:have now terminated our cloud
Prasanna Malaiyandi:concluded.
Prasanna Malaiyandi:W. Curtis Preston: series.
Prasanna Malaiyandi:Concluded,
Prasanna Malaiyandi:W. Curtis Preston: What's that?
Prasanna Malaiyandi:concluded.
Prasanna Malaiyandi:W. Curtis Preston: Why did I say terminated?
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Terminated just seems.
Prasanna Malaiyandi:W. Curtis Preston: said I'm done.
Prasanna Malaiyandi:I'm done with all these people losing data.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:We have concluded our cloud disaster series.
Prasanna Malaiyandi:What did you think?
Prasanna Malaiyandi:I liked it actually.
Prasanna Malaiyandi:It's one of those things, like a disaster happens.
Prasanna Malaiyandi:You read it on the news and then you forget about it, right?
Prasanna Malaiyandi:It says to the next day.
Prasanna Malaiyandi:And or you, there are many of these that like I don't think
Prasanna Malaiyandi:any of us had heard about.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:And just being able to go through and just understand and kind of take a look right
Prasanna Malaiyandi:and getting your perspective right and seeing, okay, what are these disasters?
Prasanna Malaiyandi:And really sort of doing the research ahead of time.
Prasanna Malaiyandi:It was kind of fun.
Prasanna Malaiyandi:I liked them.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, it was fun.
Prasanna Malaiyandi:It was, um, what I liked about it was that enough time had passed with most
Prasanna Malaiyandi:of these disasters that we often had the other half of the story, right?
Prasanna Malaiyandi:When the, when, when they happen, uh, you know, at the mo you know,
Prasanna Malaiyandi:at that moment you check it and you hear like in the case of, um.
Prasanna Malaiyandi:The, uh, the Musi outage, the first thing you hear is that, you know, musi has sued
Prasanna Malaiyandi:Google because, you know, they lost their data and, and that makes the headlines
Prasanna Malaiyandi:and everybody's, oh, you know, there's a lawsuit, you know, but you know, this
Prasanna Malaiyandi:is America, so anybody can sue anything.
Prasanna Malaiyandi:I mean, you may remember that there was a woman who sued.
Prasanna Malaiyandi:Google Maps because she drove her car off a pier.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Because Google Maps told her to go that way.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:You know, you could sue for anything, but that doesn't mean
Prasanna Malaiyandi:you're gonna be successful.
Prasanna Malaiyandi:And it was nice to go back and look at the, um, you know what happened
Prasanna Malaiyandi:after, in some of the cases, we got some additional insight from people
Prasanna Malaiyandi:that were, you know, in and around.
Prasanna Malaiyandi:I, I think of the.
Prasanna Malaiyandi:OOVH incident.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:Uh, that was nice.
Prasanna Malaiyandi:We got some insight into that and how that OVH really was known for the, you
Prasanna Malaiyandi:know, being a low cost provider and so that nobody was pro, nobody was surprised
Prasanna Malaiyandi:when they, uh, didn't have, didn't have.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Well, the other thing also that I came to realize from the series is,
Prasanna Malaiyandi:like you said, time had elapsed.
Prasanna Malaiyandi:It's also interesting to see how the company, some companies try
Prasanna Malaiyandi:to sweep things under the rug,
Prasanna Malaiyandi:W. Curtis Preston: Right,
Prasanna Malaiyandi:right?
Prasanna Malaiyandi:And thanks to the wonders of the internet where nothing is ever gone, right?
Prasanna Malaiyandi:Being able to uncover, okay, really, what was it that was initially
Prasanna Malaiyandi:said versus what do they say today?
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:Uh, thank you very much.
Prasanna Malaiyandi:Um, internet archive, right?
Prasanna Malaiyandi:That was, that was, that was very helpful.
Prasanna Malaiyandi:In some cases we weren't able to get stuff, but, and there were a lot of cases
Prasanna Malaiyandi:where we were able to get basically the first version of the story versus the,
Prasanna Malaiyandi:the later, the, the latter version.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Uh, but, but I would say in most of these cases, right, the initial
Prasanna Malaiyandi:information that people saw was what was allowed to be released, right?
Prasanna Malaiyandi:Whatever the company had at that time, which sometimes they're working off
Prasanna Malaiyandi:incomplete information, but they still wanted to provide some information
Prasanna Malaiyandi:to their users, such as like the OVH incident or Rackspace, right?
Prasanna Malaiyandi:Where it's just, okay, what information is available?
Prasanna Malaiyandi:Let me at least push something out there.
Prasanna Malaiyandi:W. Curtis Preston: Right.
Prasanna Malaiyandi:And that, and that's what we encourage, uh, companies to do, right, is to
Prasanna Malaiyandi:be upfront about what's happening.
Prasanna Malaiyandi:And it's okay to say you don't know yet, but acknowledge what's happening.
Prasanna Malaiyandi:You know, acknowledge what you know when you know it.
Prasanna Malaiyandi:Don't make stuff up.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um.
Prasanna Malaiyandi:And the, and then, and then the, the other thing that I would say is, um.
Prasanna Malaiyandi:Is we like it when they don't try to pass blame.
Prasanna Malaiyandi:That happened a couple of times.
Prasanna Malaiyandi:Uh, when the blame is, you know, clearly on you and your design,
Prasanna Malaiyandi:uh, don't try to pass blame.
Prasanna Malaiyandi:So we've got, I came up with a few lessons that I wrote down, uh, and
Prasanna Malaiyandi:if, and if, and if a few others pop up in our head while we're recording
Prasanna Malaiyandi:this episode, just come out with 'em.
Prasanna Malaiyandi:Um, and the first one is actually, uh, you put that on there.
Prasanna Malaiyandi:So you wanna, you wanna go for it?
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:So I would say that the most important thing for a lot of
Prasanna Malaiyandi:these is if it's important to you, the data, of course, back it up,
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:Which is, which is basically backup lesson number one.
Prasanna Malaiyandi:Is
Prasanna Malaiyandi:Or zero, I would say.
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:Backup.
Prasanna Malaiyandi:It's Rule zero.
Prasanna Malaiyandi:Uh, it's one that I absolutely cannot argue with, right?
Prasanna Malaiyandi:It's something that I say to you, right?
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:W. Curtis Preston: I, I say this all the time, that if, if this data
Prasanna Malaiyandi:really is important to you, then it should be, you know, backed up, right?
Prasanna Malaiyandi:And we can talk in a little bit as to what constitutes a backup and,
Prasanna Malaiyandi:uh, that'll be some of the lessons that we learned along the way.
Prasanna Malaiyandi:But, um.
Prasanna Malaiyandi:I, I'm gonna say in almost all instances, what constitutes a
Prasanna Malaiyandi:backup is you making a backup.
Prasanna Malaiyandi:Not, not saying, oh, I got it because I got it, because it's the cloud,
Prasanna Malaiyandi:or because it's SaaS, or because it's, you know, it's not my problem.
Prasanna Malaiyandi:No, it's always your problem.
Prasanna Malaiyandi:Yeah, and there were companies, right, like Musi went out
Prasanna Malaiyandi:of business because they lost all their data and they couldn't recover it, and
Prasanna Malaiyandi:that was the end of the company, right?
Prasanna Malaiyandi:So,
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:And, and I'd say that that's, that's the first lesson I'm gonna say that
Prasanna Malaiyandi:wasn't actually on the list, is that if you don't get this right.
Prasanna Malaiyandi:Your entire company may just go away, right?
Prasanna Malaiyandi:That happened, uh, in at least two instances in the, the list
Prasanna Malaiyandi:of the companies that we found.
Prasanna Malaiyandi:Either your company or a significant portion of your company may go away.
Prasanna Malaiyandi:I think of Rackspace, right?
Prasanna Malaiyandi:Rackspace is now completely out of the hosted exchange business because.
Prasanna Malaiyandi:Uh, of what happened with the rec based outage, right?
Prasanna Malaiyandi:So either your entire company or a significant portion of your company
Prasanna Malaiyandi:can literally just go, poof, overnight.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:So what's the first one, Curtis, in your opinion, like what's the
Prasanna Malaiyandi:most important lesson from all of the ones that we've looked at?
Prasanna Malaiyandi:W. Curtis Preston: 3, 2, 1.
Prasanna Malaiyandi:Rule still rules.
Prasanna Malaiyandi:And, and I know that there are other versions of the 3, 2, 1 rule.
Prasanna Malaiyandi:I'm mu in this case, what I'm saying is if what you are doing for backup, as I make
Prasanna Malaiyandi:quotes in the air, doesn't at least comply to the 3, 2, 1 rule, it is not a backup.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:There are, there are many situations when we look at the stories that we, that we
Prasanna Malaiyandi:talked about here, where people had what they might have considered a backup.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um, you know, musi is probably the best example of that, right?
Prasanna Malaiyandi:Um, where they had their backups with them, but basically
Prasanna Malaiyandi:they deleted their account.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um, the, the other one would be Code Spaces.
Prasanna Malaiyandi:That's probably the most infamous, I think, where Code Spaces was making
Prasanna Malaiyandi:backups within the, within AWS, but they didn't follow the 3, 2, 1 rule.
Prasanna Malaiyandi:They didn't separate those backups from production.
Prasanna Malaiyandi:And so, when.
Prasanna Malaiyandi:Um, you know, when the, the bad actor went in there, they deleted the entire account.
Prasanna Malaiyandi:And again, poof, they went there, went their primary as well as their backup.
Prasanna Malaiyandi:And the same would be true of the SaaS.
Prasanna Malaiyandi:Um, people that, you know, when I think of specifically Microsoft 365, but this is
Prasanna Malaiyandi:true in so many other cases, is that, um.
Prasanna Malaiyandi:It that if you are not separating that data, if you're not making
Prasanna Malaiyandi:another copy, they don't have, uh, multiple copies of your data, right?
Prasanna Malaiyandi:That, that are accessible to you.
Prasanna Malaiyandi:They may have, may have a DR copy, but as we saw in some of the cases, the DR.
Prasanna Malaiyandi:Copy does doesn't always work.
Prasanna Malaiyandi:Um, that Dr.
Prasanna Malaiyandi:Copy
Prasanna Malaiyandi:might not be to you, and it might not, it might not work, but.
Prasanna Malaiyandi:But the point is that if you're not, if you don't have a whatever they
Prasanna Malaiyandi:have, it's all in the same place.
Prasanna Malaiyandi:And when very bad things happen, uh, your SOL
Prasanna Malaiyandi:And specifically around the Microsoft case, right?
Prasanna Malaiyandi:They may have a DR copy, but like you said, that's for their purposes
Prasanna Malaiyandi:not to allow you to do your restores.
Prasanna Malaiyandi:That's to make sure if something blows up somewhere from an infrastructure
Prasanna Malaiyandi:perspective, they can break things back up and they'll try their
Prasanna Malaiyandi:best to give you back all your data, but there's no guarantees.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, exactly.
Prasanna Malaiyandi:Actually we're gonna get more into detail into that in, in a latter step.
Prasanna Malaiyandi:But yeah, so short version is, I, I have no problem with app
Prasanna Malaiyandi:pending to the 3, 2, 1 rule.
Prasanna Malaiyandi:If you wanna say 3, 2, 1, 1, 0, 7, 9, 5, I don't care.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um, and I, and I don't disagree with any of the versions that I've heard.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Things like I.
Prasanna Malaiyandi:Making sure that, you know, one of the copies, the biggest one is
Prasanna Malaiyandi:making sure that one of those copies is an immutable copy, which is a
Prasanna Malaiyandi:relatively new requirement, and that has to do with, uh, cyber attacks.
Prasanna Malaiyandi:I don't have a problem with that.
Prasanna Malaiyandi:But if what you're doing doesn't comply with the 3, 2, 1 rule, uh, three
Prasanna Malaiyandi:copies of your data on two different types of media, one of which is
Prasanna Malaiyandi:physically separated from the other, um, then it, you don't have a backup.
Prasanna Malaiyandi:I do wonder, and maybe it doesn't have to be for
Prasanna Malaiyandi:this episode, if we should really figure out does the three or have a
Prasanna Malaiyandi:discussion on an episode of does the 3, 2 1 rule really need to be changed?
Prasanna Malaiyandi:W. Curtis Preston: Um, I,
Prasanna Malaiyandi:I think,
Prasanna Malaiyandi:I think just because I know that there's like different,
Prasanna Malaiyandi:like you said, the 3, 2, 1, 1 0.
Prasanna Malaiyandi:Zero.
Prasanna Malaiyandi:Zero.
Prasanna Malaiyandi:It may not be a zero, but you know what I mean.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, well there, there's various versions
Prasanna Malaiyandi:out there and I'm fine with those.
Prasanna Malaiyandi:I do myself, like, you know, a, you know, I add to it.
Prasanna Malaiyandi:One of these copies needs to be immutable.
Prasanna Malaiyandi:I am, I fully agree with that.
Prasanna Malaiyandi:Does it need to be part of the 3, 2, 1 rule?
Prasanna Malaiyandi:I don't know.
Prasanna Malaiyandi:I, you know.
Prasanna Malaiyandi:I like the 3, 2, 1 for, for what it does, because what it does is it proves what
Prasanna Malaiyandi:we've been talking about here is that if you have a cloud service and all of
Prasanna Malaiyandi:the copies of your data are all in the same place, you don't have a backup.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:Speaking of which, uh, I would say that the next lesson learned is really around.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Like you mentioned, the cloud is not backing up your data.
Prasanna Malaiyandi:You may think it is backing up your data.
Prasanna Malaiyandi:They may be doing DR copies for their own purposes, but it's not a backup
Prasanna Malaiyandi:that you as a customer, a user can go restore from, can go back to any point
Prasanna Malaiyandi:in time, recover it all the rest of that.
Prasanna Malaiyandi:Which I think a lot of people mistake, right?
Prasanna Malaiyandi:They're like, oh, I have a Salesforce, or Take your pick and it's a SaaS service.
Prasanna Malaiyandi:They host my data.
Prasanna Malaiyandi:They should be doing my backups.
Prasanna Malaiyandi:Or Microsoft 365, they should be doing my backups.
Prasanna Malaiyandi:And I know Curtis way back in the day when we were first working together, right?
Prasanna Malaiyandi:You were like, have you ever taken a look at Microsoft's
Prasanna Malaiyandi:contracts, terms of service?
Prasanna Malaiyandi:Do you ever see backup in there anywhere?
Prasanna Malaiyandi:W. Curtis Preston: Right.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:It's just, it is just not there, right?
Prasanna Malaiyandi:That and that and that.
Prasanna Malaiyandi:That's the part it is.
Prasanna Malaiyandi:If you think that your SaaS service is backing up your data, please go
Prasanna Malaiyandi:look for it in your contract, right?
Prasanna Malaiyandi:Because you're not gonna find it.
Prasanna Malaiyandi:There may be, there are some exceptions.
Prasanna Malaiyandi:Interestingly enough, one of the exceptions was covered on
Prasanna Malaiyandi:our, on our stories, right?
Prasanna Malaiyandi:Where we had a vendor that was backing up, we're gonna talk, we're gonna,
Prasanna Malaiyandi:we had a vendor who was advertising that part of their service was backup,
Prasanna Malaiyandi:which almost never happens, right?
Prasanna Malaiyandi:You don't see that.
Prasanna Malaiyandi:Go look at what, what Microsoft advertises for 365.
Prasanna Malaiyandi:Look at Salesforce.
Prasanna Malaiyandi:Backup is not part of the offering either in terms of what they
Prasanna Malaiyandi:advertise or what's in your contract.
Prasanna Malaiyandi:And if it's not in your contract, it doesn't exist.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:So the next one, and this is we, we've covered this a little bit already, and
Prasanna Malaiyandi:that is if they are backing up their data center, that backup is not for you.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um, and, and I see this a lot.
Prasanna Malaiyandi:Um, there's again, this, this one guy that I interact with a lot online that,
Prasanna Malaiyandi:that really believes very strongly in the Microsoft way of doing things
Prasanna Malaiyandi:and in his idea that Microsoft has, for example, multiple delayed copies
Prasanna Malaiyandi:of your data, like replicated, uh, delayed, replicated copies of your data.
Prasanna Malaiyandi:That it's, it's gonna use that in the case of disaster to be
Prasanna Malaiyandi:able to recover your environment.
Prasanna Malaiyandi:And that appears to be true.
Prasanna Malaiyandi:I, I've never found it anywhere documented.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:Which, and if it's not documented, if it's not in my contract that I don't care.
Prasanna Malaiyandi:But it appears to be true, however.
Prasanna Malaiyandi:The story of KPMG.
Prasanna Malaiyandi:If that story with the size of KPMG 145,000 employees, if what happened
Prasanna Malaiyandi:to KPMG doesn't prove to you that you don't get to use Microsoft's backup to
Prasanna Malaiyandi:save your butt, I don't know what will.
Prasanna Malaiyandi:This is a, this is a giant company paying tons of money to Microsoft every month.
Prasanna Malaiyandi:Even they weren't able to recover from this giant blunder because it
Prasanna Malaiyandi:was their fault, and that was not enough to have Microsoft use their
Prasanna Malaiyandi:Dr copy, uh, to, to save them.
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:Or, or if we go back and think about like
Prasanna Malaiyandi:the Salesforce example, right?
Prasanna Malaiyandi:Where they changed on permissions and everyone had
Prasanna Malaiyandi:access to every record, right?
Prasanna Malaiyandi:And it was like, how do we undo that?
Prasanna Malaiyandi:And so they tried their best and hopefully if you had a sandbox copy, they were
Prasanna Malaiyandi:able to help you out a little bit.
Prasanna Malaiyandi:But they were basically like, sorry, uh, I can't help you.
Prasanna Malaiyandi:Good luck.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, that's interesting because the Salesforce
Prasanna Malaiyandi:story, it's, it was their blunder,
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:W. Curtis Preston: right?
Prasanna Malaiyandi:It was their blunder that, that, um, I.
Prasanna Malaiyandi:Where they messed up everybody's account.
Prasanna Malaiyandi:And even then, when we know that Salesforce has a, a backup, as I make
Prasanna Malaiyandi:quotes in the air, um, that you can pay for like $10,000 an instance,
Prasanna Malaiyandi:and it takes like weeks to get it.
Prasanna Malaiyandi:But they were like, no, we're not gonna crack open Pandora's box, even though it
Prasanna Malaiyandi:was us that messed up all these accounts.
Prasanna Malaiyandi:So, yeah.
Prasanna Malaiyandi:So th those two incidents, if, if those don't prove to you that these DR copies
Prasanna Malaiyandi:that these vendors are making aren't for your purposes, I, I don't know what will.
Prasanna Malaiyandi:Yeah, and these are large companies, right?
Prasanna Malaiyandi:It's not like a mom and pop cloud vendor, right?
Prasanna Malaiyandi:Doing something or a SaaS application.
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:Going alongside that, so we're saying don't trust
Prasanna Malaiyandi:the cloud vendor to do your backup.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:And instead you really should be using a third party backup service.
Prasanna Malaiyandi:If we looked at some of the cases like Deduce where academics, people
Prasanna Malaiyandi:in academics and research, they lost.
Prasanna Malaiyandi:Studies, right?
Prasanna Malaiyandi:Because, uh, DDU went down or they lost some data, and if they had simply done
Prasanna Malaiyandi:a backup or even simply exported their data, right, they would've been fine.
Prasanna Malaiyandi:They wouldn't have had an issue, but they didn't because there wasn't part
Prasanna Malaiyandi:of the service, there wasn't anything that could easily back that up.
Prasanna Malaiyandi:And so people lost data.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, exactly.
Prasanna Malaiyandi:I mean the, the Deduce story, uh, the Rackspace story, where it would've been
Prasanna Malaiyandi:probably much easier for companies when they, when Rackspace went down and then
Prasanna Malaiyandi:they immediately migrated everybody over to, to 365 would've been much easier
Prasanna Malaiyandi:for them to recover from that situation, uh, if they had their own backup.
Prasanna Malaiyandi:Um, and, and of course the OVH disaster where, uh, you know, the, they, so both
Prasanna Malaiyandi:in the case of OVH and um, deduce, we know that backup was something that
Prasanna Malaiyandi:they advertised as part of the service.
Prasanna Malaiyandi:By the way, specifically in Deduce, you looked, if you look at their
Prasanna Malaiyandi:website from back then, it says they're doing nightly backups.
Prasanna Malaiyandi:And, and we now know that we don't know exactly how or why, but the closest
Prasanna Malaiyandi:available backup was o was a month out and the one that was, and, and there
Prasanna Malaiyandi:was something a little funky with it.
Prasanna Malaiyandi:So they actually went two months out.
Prasanna Malaiyandi:Um, what, how, how is that a nightly backup?
Prasanna Malaiyandi:Right, right.
Prasanna Malaiyandi:So I, I guess what you're saying is, or what I'm saying is that when.
Prasanna Malaiyandi:Bad things happen and bad things happen.
Prasanna Malaiyandi:Admins do wrong things.
Prasanna Malaiyandi:Software engineers do wrong things.
Prasanna Malaiyandi:When you're trusting a single vendor to do both your IT and your backups
Prasanna Malaiyandi:of your it, if they turn out to be incompetent, they're probably also
Prasanna Malaiyandi:incompetent in your backup, right?
Prasanna Malaiyandi:And so this isn't, because both of us used to work for a cloud backup vendor, right?
Prasanna Malaiyandi:This isn't us trying to, you know, get money for our firm employer.
Prasanna Malaiyandi:Uh, this is just common sense man.
Prasanna Malaiyandi:Yeah, and I would also say for those companies where
Prasanna Malaiyandi:they are focused on both your primary IT application as well as the backup,
Prasanna Malaiyandi:typically they're not going to put all the money, all the effort in the backup side
Prasanna Malaiyandi:because their main goal is to sell and add new features to the primary application.
Prasanna Malaiyandi:And so usually you end up with something that's either half baked, not up
Prasanna Malaiyandi:to date, doesn't support all your various scenarios, that someone who is
Prasanna Malaiyandi:specialized in backup is thinking about.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, all those things there.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:There are gonna be lots of important features, both from a security standpoint,
Prasanna Malaiyandi:a recovery standpoint, all of that, that are going to be useful to you if
Prasanna Malaiyandi:you ever actually need to do a restore.
Prasanna Malaiyandi:In the case of any of the events that happened to the people
Prasanna Malaiyandi:in our, in the stories that we covered in the last, what was it?
Prasanna Malaiyandi:Like eight weeks, something like that.
Prasanna Malaiyandi:Uh, what we got next.
Prasanna Malaiyandi:So, I think you did touch on this a bit earlier,
Prasanna Malaiyandi:but be careful when someone sells you a feature and tells you it's
Prasanna Malaiyandi:backup, but it's not backup.
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:Yeah, so this one's interesting because, and I, it sounds like
Prasanna Malaiyandi:we pick on Microsoft a lot.
Prasanna Malaiyandi:I, it's not Microsoft.
Prasanna Malaiyandi:My issue is not Microsoft.
Prasanna Malaiyandi:It's the people that seem to, that seem to, like.
Prasanna Malaiyandi:In the case of Microsoft 365, you will not see the documentation referring
Prasanna Malaiyandi:to retention policies as backup.
Prasanna Malaiyandi:You won't see it.
Prasanna Malaiyandi:Retention policies are an e-discovery feature.
Prasanna Malaiyandi:E discovery and archive have nothing to do with backup, right?
Prasanna Malaiyandi:They behave completely differently in terms of the way they save
Prasanna Malaiyandi:data, the way they pull out data.
Prasanna Malaiyandi:Um, and if you, if you don't understand what I'm saying, you
Prasanna Malaiyandi:don't believe what I'm saying, go and try to restore a mailbox.
Prasanna Malaiyandi:Using 365 retention policies, please go try that.
Prasanna Malaiyandi:Um, the, um,
Prasanna Malaiyandi:tell us if it could turn out to be a plausible point
Prasanna Malaiyandi:in time that that mailbox looked like
Prasanna Malaiyandi:W. Curtis Preston: doesn't do it.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:W. Curtis Preston: It.
Prasanna Malaiyandi:It's all about retention policies and the associated E-discovery feature
Prasanna Malaiyandi:is all about how to get all email that ever went into or came out
Prasanna Malaiyandi:of that box over a period of time.
Prasanna Malaiyandi:It is not designed to restore your mailbox to the way it looked yesterday.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Doesn't restore folders.
Prasanna Malaiyandi:It doesn't restore all kinds of stuff, and it doesn't know how.
Prasanna Malaiyandi:It, again, just you, you gotta, maybe you have to experience it to truly
Prasanna Malaiyandi:understand this, and that is that it doesn't know how to make your mailbox
Prasanna Malaiyandi:look the way it looked yesterday.
Prasanna Malaiyandi:It doesn't understand that concept.
Prasanna Malaiyandi:That is a backup and restore concept.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:And so that's, that's
Prasanna Malaiyandi:Not an archive and retrieval.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, it's about arch archive and retrieval.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:Which, if you don't know what we're talking about, there's a whole
Prasanna Malaiyandi:episode that we talk about why arch archive and backup are different.
Prasanna Malaiyandi:Go, go check that out.
Prasanna Malaiyandi:Uh, put a link in the show notes.
Prasanna Malaiyandi:But the, but the other reason with 365 is when we look at that story of KPMG if
Prasanna Malaiyandi:you, if you have a backup software, um.
Prasanna Malaiyandi:Let's say, you know, pick your favorite backup software.
Prasanna Malaiyandi:If you configure it really, really, really, really wrong, like just the
Prasanna Malaiyandi:worst backup configuration ever.
Prasanna Malaiyandi:You know what it doesn't do,
Prasanna Malaiyandi:Blow away your production.
Prasanna Malaiyandi:W. Curtis Preston: it doesn't blow away your production.
Prasanna Malaiyandi:Which is another key difference between Microsoft retention
Prasanna Malaiyandi:policies and backup, because that's what happened in the KPMG story.
Prasanna Malaiyandi:you almost want a separation of duties, right?
Prasanna Malaiyandi:That's the roles and responsibilities.
Prasanna Malaiyandi:It's like no different than a cloud provider and the customers who are
Prasanna Malaiyandi:on top of the cloud provider, right?
Prasanna Malaiyandi:Each person has their own responsibilities and their roles and what they do
Prasanna Malaiyandi:and what is provided, and it's all clearly articulated in contracts.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, retention policies.
Prasanna Malaiyandi:When this person went to go do what they wanted to do, which was just delete one
Prasanna Malaiyandi:person's chat, it's like, if I get this right, I will delete one person's chat
Prasanna Malaiyandi:and I will have a really, uh, poor.
Prasanna Malaiyandi:Facsimile of backup that is really lousy at Restore.
Prasanna Malaiyandi:That's the best case scenario.
Prasanna Malaiyandi:If I get it wrong, I'm gonna delete the personal chat
Prasanna Malaiyandi:history of 145,000 employees.
Prasanna Malaiyandi:It could, by the way, it could have been worse.
Prasanna Malaiyandi:It could have been worse because.
Prasanna Malaiyandi:Retention policies when you're in them, when you, when you first open them.
Prasanna Malaiyandi:The, by the way, retention policy, the manual is 25 pages, right?
Prasanna Malaiyandi:But you can go in there and you can go for all services, right?
Prasanna Malaiyandi:Meaning
Prasanna Malaiyandi:Prasanna Malaiyandi: It's not just for chat.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:W. Curtis Preston: for all services.
Prasanna Malaiyandi:Please set retention to one copy.
Prasanna Malaiyandi:So,
Prasanna Malaiyandi:W. Curtis Preston: copy.
Prasanna Malaiyandi:Literally in a, in a few mouse clicks, you can blow away your entire 365 environment.
Prasanna Malaiyandi:could you imagine if that had happened?
Prasanna Malaiyandi:And KPMG, which is a huge company, loses everything.
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:I mean, it's bad enough that what they lost.
Prasanna Malaiyandi:People are like, oh, it's just chat.
Prasanna Malaiyandi:Well, chat is often quite valuable.
Prasanna Malaiyandi:Yeah,
Prasanna Malaiyandi:Clearly you've never lived in a company that used chat.
Prasanna Malaiyandi:Uh,
Prasanna Malaiyandi:Prasanna Malaiyandi: don't use smoke signals.
Prasanna Malaiyandi:W. Curtis Preston: what's that?
Prasanna Malaiyandi:They don't use smoke signals
Prasanna Malaiyandi:W. Curtis Preston: Yeah, it reminds me, it reminds me there was a company.
Prasanna Malaiyandi:It was, I'm pretty sure this was a Xerox commercial years ago.
Prasanna Malaiyandi:And it was like, does Xero, is your, is this, describe your company, like
Prasanna Malaiyandi:if your copier goes down, do people say it's okay, we'll use carbon paper.
Prasanna Malaiyandi:Do you remember that?
Prasanna Malaiyandi:That was
Prasanna Malaiyandi:a, that was a, that was a long time ago.
Prasanna Malaiyandi:Um, so what's our next lesson?
Prasanna Malaiyandi:So going alongside what you talked about with Microsoft
Prasanna Malaiyandi:365 and retention policies is, that's like a specific case for Microsoft.
Prasanna Malaiyandi:But also cloud providers in general.
Prasanna Malaiyandi:How about their designs are just awful.
Prasanna Malaiyandi:Like, I think that's actually being too kind to say awful,
Prasanna Malaiyandi:W. Curtis Preston: Yeah.
Prasanna Malaiyandi:right?
Prasanna Malaiyandi:I think it's just downright pathetic.
Prasanna Malaiyandi:Uh, one of the ones that comes to mind is OVH.
Prasanna Malaiyandi:W. Curtis Preston: Right,
Prasanna Malaiyandi:know I hate picking on OVH 'cause we always
Prasanna Malaiyandi:pick on OVH, but I'm picking on OVH.
Prasanna Malaiyandi:Right?
Prasanna Malaiyandi:And it's really around the fact that they offered a backup service and
Prasanna Malaiyandi:they basically had the backup server in the same data center, like just a
Prasanna Malaiyandi:couple rows down in a separate rack.
Prasanna Malaiyandi:And so when they had a fire that damaged both production and their backup data.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, I mean, I mean, that's bad.
Prasanna Malaiyandi:Uh, I, I don't knows some equally bad, similarly bad.
Prasanna Malaiyandi:Um, when I think about the fact, you know, if we go back in time, there was a time
Prasanna Malaiyandi:when Carbonite was everywhere, right?
Prasanna Malaiyandi:There were advertise.
Prasanna Malaiyandi:You would think that Carbonite was the single biggest
Prasanna Malaiyandi:backup vendor on the planet.
Prasanna Malaiyandi:And they were storing customer backup data on raid five arrays.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:I'm gonna say pro prosumer quality raid five arrays with, with no
Prasanna Malaiyandi:dual parity, you know, um, nothing.
Prasanna Malaiyandi:And then they, you know, and it was really just a matter of time before
Prasanna Malaiyandi:what happened to them happened.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um, you know, they had a double disc failure and Right.
Prasanna Malaiyandi:You know, there was that one.
Prasanna Malaiyandi:And then, you know, we have the other one.
Prasanna Malaiyandi:Of, um, you know, code spaces where they stored their backup in the same
Prasanna Malaiyandi:place that they stored production.
Prasanna Malaiyandi:We already talked about that with the 3, 2, 1 rule.
Prasanna Malaiyandi:Yeah, and I think the other one there to also talk about is.
Prasanna Malaiyandi:And I don't know if it's really poor design or just not understanding
Prasanna Malaiyandi:backup and backup requirements is, I would actually say Rackspace,
Prasanna Malaiyandi:right?
Prasanna Malaiyandi:They had a hosted exchange offering, which is great, but they didn't
Prasanna Malaiyandi:really have a backup solution.
Prasanna Malaiyandi:They had never tested it out.
Prasanna Malaiyandi:They were sort of trying things on the fly after they ran into issues.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, the part of that story that was the worst was
Prasanna Malaiyandi:it's like, okay, we've been working on this for a couple weeks now.
Prasanna Malaiyandi:We think we have a good plan for what, you know, for how
Prasanna Malaiyandi:we're gonna get your data back.
Prasanna Malaiyandi:Uh, you know, and then like a little bit more time passes and it's like,
Prasanna Malaiyandi:oh, we've now tested the plan.
Prasanna Malaiyandi:It works.
Prasanna Malaiyandi:Now we need to go do the plan.
Prasanna Malaiyandi:Like weeks are going by, so you're like, okay, so what you're telling us
Prasanna Malaiyandi:is you're just making this up as you go along, that you never tested this before.
Prasanna Malaiyandi:Uh, yeah, not, not good.
Prasanna Malaiyandi:Um, the, um, uh, this next one here is that if your cloud vendor is
Prasanna Malaiyandi:the low cost leader, it will show.
Prasanna Malaiyandi:This was the case of OVH.
Prasanna Malaiyandi:They were well acknowledged to be the low cost leader in that space, and it means
Prasanna Malaiyandi:that they cut corners in a lot of areas.
Prasanna Malaiyandi:They cut corners in terms of firefighting.
Prasanna Malaiyandi:They cut corners in terms of not segregating the backup
Prasanna Malaiyandi:data, all of that stuff,
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:I don't think there's anything wrong with being a low cost offering, as
Prasanna Malaiyandi:long as people understand that and the expectations are there, right?
Prasanna Malaiyandi:You get what you paid for, right?
Prasanna Malaiyandi:Not everyone can afford the Ferrari they may buy, right?
Prasanna Malaiyandi:People go buy the Geo Metro, right?
Prasanna Malaiyandi:Way back in the day, so.
Prasanna Malaiyandi:There are offerings for different people, and that totally makes sense.
Prasanna Malaiyandi:But I think what becomes a challenge is when you're not transparent
Prasanna Malaiyandi:about what people are getting or you set expectations incorrectly and
Prasanna Malaiyandi:misrepresent what they are getting.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, agreed.
Prasanna Malaiyandi:Um, and there are, and well, and, and again, if you're using a low cost
Prasanna Malaiyandi:provider, we go back to rule number one.
Prasanna Malaiyandi:Make sure that you're, make sure that you're backing up the data somewhere else.
Prasanna Malaiyandi:I mean, you should do it like no matter what your provider is, but definitely if
Prasanna Malaiyandi:you're doing a low cost provider, right.
Prasanna Malaiyandi:Um, make sure that you, you know, because they're gonna, they're
Prasanna Malaiyandi:gonna, they're gonna cut corners.
Prasanna Malaiyandi:So the final one, uh, in our lessons learned is.
Prasanna Malaiyandi:SSDs aren't perfect.
Prasanna Malaiyandi:And I know Curtis, we've talked, I think we've had multiple episodes about this.
Prasanna Malaiyandi:In fact, at home when, uh, I was looking for a new sort of, not really backup
Prasanna Malaiyandi:drive, but just extra storage and debating between SSD or just getting spinning
Prasanna Malaiyandi:disc, and I know chatting with you, I was like, yeah, we should probably get SS or.
Prasanna Malaiyandi:Probably you should get a disc drive because we're not using the data often
Prasanna Malaiyandi:and just the issues that we've seen around SSDs, if you're not using it all the time,
Prasanna Malaiyandi:the data isn't guaranteed to be there.
Prasanna Malaiyandi:And, uh.
Prasanna Malaiyandi:This is one of those things where, uh, one of my favorite YouTube channels that
Prasanna Malaiyandi:I watch all the time, life Uncontained, they lost a bunch of data on SSDs
Prasanna Malaiyandi:and people think they're bulletproof.
Prasanna Malaiyandi:Yes, there's less mechanical spinning parts versus traditional hard drives,
Prasanna Malaiyandi:but they could still lose data.
Prasanna Malaiyandi:W. Curtis Preston: Yeah, I, I think we could say that they, they probably
Prasanna Malaiyandi:are more reliable than hard drives, especially if you're moving them around.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:Um, you know, which is the case in your, obviously in your mobile
Prasanna Malaiyandi:device or any sort of camera in the field, but they are not infallible
Prasanna Malaiyandi:and there are some specific.
Prasanna Malaiyandi:Um, ways that they behave that are unique to them.
Prasanna Malaiyandi:Some u you know, some unique problems to SSDs.
Prasanna Malaiyandi:One of them, you know, you alluded to there, is that it does require, um, you
Prasanna Malaiyandi:know, a, a power to ensure that all of the voltages are there and over time.
Prasanna Malaiyandi:If you're not using a given, um, SSD drive, those voltages can drop
Prasanna Malaiyandi:and you can just lose data there.
Prasanna Malaiyandi:There, um, I, I think we're, I think based on, we got a really good
Prasanna Malaiyandi:message from one of our listeners, and I want to thank them for that.
Prasanna Malaiyandi:Uh, I, I wanna do some more research and talk to somebody
Prasanna Malaiyandi:who is a specialist in this area.
Prasanna Malaiyandi:It's not my area of expertise.
Prasanna Malaiyandi:I don't think it's yours either.
Prasanna Malaiyandi:Uh, but, but the short version is.
Prasanna Malaiyandi:That while SSDs and, and various other, you know, not just SSDs, but,
Prasanna Malaiyandi:but, but solid state devices, uh, of every kind that, while they, I
Prasanna Malaiyandi:think they are more reliable than the, uh, a standard hard drive,
Prasanna Malaiyandi:especially if you're moving it around.
Prasanna Malaiyandi:So they're great for laptops.
Prasanna Malaiyandi:Uh, they are not infallible and they still need to be backed up.
Prasanna Malaiyandi:Which brings us back to Rule zero.
Prasanna Malaiyandi:And what's that
Prasanna Malaiyandi:if it's important.
Prasanna Malaiyandi:Back it up.
Prasanna Malaiyandi:W. Curtis Preston: exactly?
Prasanna Malaiyandi:So the cloud isn't perfect.
Prasanna Malaiyandi:SSDs aren't perfect.
Prasanna Malaiyandi:Uh, software engineers aren't perfect.
Prasanna Malaiyandi:Admins aren't perfect.
Prasanna Malaiyandi:All of these are why we back up and, um,
Prasanna Malaiyandi:Don't forget malicious users are out there.
Prasanna Malaiyandi:W. Curtis Preston: Of course malicious users, malicious
Prasanna Malaiyandi:and, bad actors in general.
Prasanna Malaiyandi:Right.
Prasanna Malaiyandi:So, you know, like, like, uh, cyber attacks.
Prasanna Malaiyandi:Only a few of the incidents that we covered were, were cyber attacks.
Prasanna Malaiyandi:The rest were, uh, mistakes.
Prasanna Malaiyandi:Um, there was a fire, trying to think what else, but generally it was just mistakes.
Prasanna Malaiyandi:This is why we make backups.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:and and I think it's also, and I think the other important thing.
Prasanna Malaiyandi:Like these are just cases we found, right?
Prasanna Malaiyandi:There are probably hundreds out there that aren't publicized, right?
Prasanna Malaiyandi:The company's either gone out of business or no one noticed, right, that
Prasanna Malaiyandi:their data was missing or other things.
Prasanna Malaiyandi:These are just things that we have been able to find and at least some information
Prasanna Malaiyandi:to be able to report out to our listeners on, Hey, here's what we see going on.
Prasanna Malaiyandi:W. Curtis Preston: Exactly.
Prasanna Malaiyandi:Well, this has been fun.
Prasanna Malaiyandi:I hope that our, um, you know, listeners have enjoyed it as much as you and
Prasanna Malaiyandi:I have enjoyed making the series.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:No, this has been great, Curtis.
Prasanna Malaiyandi:Thank you for suggesting this.
Prasanna Malaiyandi:W. Curtis Preston: All right, well, uh, thanks to our listeners
Prasanna Malaiyandi:and, uh, be sure to subscribe so that you don't miss an episode.
Prasanna Malaiyandi:That is a wrap