Transcripts
Yeah, I need another friend for my movie fix.
Curtis:Yeah, I have to, I've done multiple
Prasanna:You can have two friends but that's it.
Curtis:Okay.
Curtis:All right.
Curtis:There you go.
Curtis:And I have podcasts with both of them, so it's all good.
Curtis:Hi, and welcome to Backup Central's Restore it All podcast.
Curtis:I'm your host.
Curtis:W.
Curtis:Curtis Preston, AKA Mr.
Curtis:Backup and I have with me, my AirPods fitness consultants.
Curtis:Malia, Prasanna Malaiyandi
Prasanna:yes, I am not sticking my fingers in your ears, just so people know.
Curtis:but you said you could make sure that my AirPods fit properly.
Prasanna:Yes.
Prasanna:And apple provides you with a great tool in the iPhone to actually make sure
Prasanna:because you got the new AirPods pro,
Curtis:I suppose if I actually pulled the little ma the little manual
Curtis:out, it would probably tell me that
Prasanna:Yup.
Prasanna:And actually, when you first paired your AirPods pro with your
Prasanna:phone, It should have asked you, do you want to run a fit test?
Curtis:It may have, but I remember when I first paired it, I was in a
Curtis:hurry and I was on my way somewhere.
Prasanna:Yeah,
Curtis:So if it did ask me, I probably ignored it.
Curtis:But then I was mentioning to you about the the ear tips and
Curtis:that I went with a smaller one.
Curtis:And you were like, did you do the fit test?
Curtis:And I'm like
Prasanna:Yeah.
Prasanna:It's great though, because apple wants to make sure you get the
Prasanna:best sound experience possible.
Prasanna:So they play music, which is annoying music
Curtis:Yeah, it is a little annoying.
Prasanna:but they measure to see how much noise actually leaks into the
Prasanna:microphones on the inside of the tips.
Prasanna:So they can detect, is there too much noise leakage or not,
Curtis:Yeah.
Prasanna:Ingenious.
Curtis:Yeah, it's kinda cool.
Curtis:I Yeah.
Curtis:So I think I have a small and delicate ears,
Prasanna:TMI Curtis TMI,
Curtis:so that's why I went with the smaller ear tips.
Curtis:I have this giant head, but small delicate ears.
Curtis:So
Prasanna:all this power in this itty little, what is it?
Prasanna:Aladdin?
Curtis:infinite power and itty bitty living space.
Curtis:Yeah, exactly.
Curtis:Yes.
Curtis:Speaking of quotable movie lines, I just I've been watching movies
Curtis:that I've never seen before lately.
Curtis:And last night I watched the original name of it was everyone comes to Rick's
Prasanna:I don't know what the actual.
Curtis:Yeah, the original name of this, of the play that became a
Curtis:movie was everyone comes to Rick's or everybody comes to Rick's.
Curtis:You probably know it by its more common name.
Curtis:Casablanca
Prasanna:ah.
Curtis:Yeah, I never saw Casablanca.
Curtis:So I watched Casablanca
Curtis:last night.
Prasanna:I'm surprised you've never seen it.
Curtis:I dunno why I dunno.
Curtis:It just, here's looking at you kid and this is a beginning of a
Curtis:beautiful relationship and it's a lot of great and by the way, you
Curtis:know what line is not in Casablanca,
Prasanna:What
Curtis:play it again, Sam
Prasanna:Say hello to my,
Curtis:no.
Curtis:no.
Curtis:The line that everyone calls from Casa Blanca the, play it again.
Curtis:Sam is not in the movie.
Curtis:It's like the treasure of Sierra Madre.
Curtis:The, we don't need no stinking badges.
Curtis:That's actually not from the treasure of Sierra Madre.
Curtis:It's from the spoof of it, which is, I think it might be blazing saddles.
Curtis:I'm not sure,
Prasanna:Was it?
Prasanna:What's the guy's name?
Prasanna:Not Rob Lowe said it wasn't Rob Lowe.
Curtis:In what blazing saddles.
Prasanna:No, I, because I've heard him say that in like other movies
Curtis:Oh, maybe yeah, it's a line that people quote and misquote but
Curtis:the famous line is not the line that everyone says, just like he
Curtis:does not say play it again, Sam.
Curtis:He says play it.
Curtis:And he says, you played it for her.
Curtis:You can play it for me.
Curtis:It doesn't say play it again.
Curtis:But
Curtis:anyway,
Prasanna:things you learn when you actually closely watch a movie.
Curtis:Yeah.
Curtis:Yeah.
Curtis:Yeah, I've been on this watching older movies kick because of my other podcast
Curtis:with my good friend, Jeff Rocklin his podcast is called the things that
Curtis:entertain us feel free to look that up.
Curtis:And I'm going, having a fun, watching some, current movies as
Curtis:well as I watched citizen Kane for the first time on a big screen at
Curtis:the academy the academy, essentially the academy of motion pictures, museum.
Curtis:They have this beautiful theater and got to see citizen Kane for the first
Curtis:time on this big, giant screen, it was introduced by a guy from the academy
Curtis:and talked about the visual effects and stuff that was in the movie.
Prasanna:That's pretty awesome.
Prasanna:Yeah.
Prasanna:Unfortunately I don't watch movies and so Curtis has to get his
Curtis:Yeah.
Curtis:I need another friend for my movie fix.
Curtis:Yeah, I have to, I've have
Prasanna:it's.
Curtis:multiple
Prasanna:It's OK, you can have two friends but that's it.
Curtis:Okay.
Curtis:All right.
Curtis:There you go.
Curtis:And I have podcasts with both of them, so it's all good.
Curtis:Today I found myself on a Reddit thread.
Prasanna:Why do you do this to yourself?
Prasanna:Curtis?
Prasanna:Why?
Curtis:I was on a Reddit, as is.
Curtis:That's part of my shtick.
Curtis:And I found this Reddit thread and the title was how
Curtis:important is backing up O 365.
Curtis:And he says, I have MSPs always pushing for O 365 backup services.
Curtis:Like Druva how necessary is backing up a 365.
Curtis:Isn't the data geo redundant looking for opinions, thanks
Curtis:and there were a variety of it's a bit like it's a bit like posting, I think
Curtis:which is better Democrat or Republican.
Curtis:It's almost like a troll, right?
Curtis:Because you know that there's going to be people like me that are going
Curtis:to say, dude the cloud isn't magic.
Curtis:You need to back up your stuff.
Curtis:And then, these other guys who seem to be they are certainly less concerned about.
Curtis:What I would consider backup.
Curtis:And so they, there are groups that post the opposite and there are
Curtis:people that blog the opposite.
Curtis:So I thought we'd talk, I know we've covered this in the past, but I,
Prasanna:In fact it was our fifth episode of why backup SaaS services, I think
Curtis:let's see we've that was like a hundred and thousand episodes
Prasanna:I know.
Curtis:If it was our fifth episode, that was a hundred and twenty
Curtis:five, twenty nine episodes ago.
Prasanna:So
Curtis:There you go.
Curtis:There you go.
Curtis:And so I thought we talk about this and first off, I'll throw out
Curtis:our usual disclaimer, Prasanna and I work for different companies.
Curtis:He works for Zoom I worked for Druva.
Curtis:This is not a podcast of either company and the opinions that you hear are ours.
Prasanna:actually Curtis's.
Curtis:I don't let him have his own opinion.
Curtis:And then also rate this podcast at ratethispodcast.com/restore.
Curtis:And we're looking for guests and we're, we're start, we'll start having
Curtis:a guest again here this year, and we would love to have you on, we love
Curtis:to talk about backups, security, data protection, data, privacy barbecue, beer.
Curtis:And backups, barbecue, beer and backups the three BS.
Curtis:So yeah, so please join us.
Curtis:Just message me @wcpreston on Twitter, you can DM me.
Curtis:I accept DMS from everybody.
Curtis:And also you can email me at wcurtispreston@ g-mail . What
Curtis:do you think about this?
Curtis:Now that you've been freed from your associate, you used to be associated
Curtis:with backup for a long time.
Curtis:And now now you're over there at this other company that
Curtis:has nothing to do with backup.
Prasanna:A lot of people think that, yeah, it's running in the cloud.
Prasanna:Why do I need to back it up?
Prasanna:And it's not even just corporate it's.
Prasanna:And not even specifically Microsoft 365, but look at people using
Prasanna:Gmail for their personal email.
Prasanna:How many people, I bet if you go around and ask like everyone, do
Prasanna:you back up your mailbox, right?
Prasanna:Your gmail
Prasann:right.
Prasann:99% of people will be like, wait, what?
Curtis:Yeah.
Curtis:I think it was where we were like 99.9, but
Curtis:yeah.
Prasanna:And I think it's just one of those things where most people, like
Prasanna:we've always said they believe it's a SaaS service backups happen automatically.
Prasanna:They don't need to worry about it.
Prasanna:They don't need to worry about like their laptop dying because it's in the cloud.
Prasanna:They just bring a new device and it all works, but they don't realize that data
Prasanna:is still sitting somewhere on something
Curtis:And it's a bit annoying to me because I've been in
Curtis:the space for a long time.
Curtis:And throughout the years, there have been things that have.
Curtis:Purported as a replacement for backups, right?
Curtis:One of them was or that didn't need backups.
Curtis:One of them was raid, when raid came out, we didn't need backups.
Curtis:And then then there
Curtis:were
Prasanna:and snap and replicate
Curtis:Yeah but even like with snapshot, there were people that just had snapshots.
Curtis:I didn't even have the replication and they would say, oh, I don't need
Curtis:backups because they have snapshots.
Curtis:Dude, those are snapshots of the primary.
Prasanna:Sitting on the primary.
Curtis:Yeah.
Curtis:Sitting on the primary.
Curtis:And that's what I'm going to go for here.
Curtis:So first off I'm going to say the same thing that I often say just a flat-out.
Curtis:Emphatic statement that Microsoft and Google are not backing up your email.
Curtis:They're not backing up your SharePoint, et cetera.
Curtis:Your OneDrive, if you don't know, OneDrive is essentially
Curtis:a an interface to SharePoint.
Curtis:And if you don't believe me, go look at your service contract,
Curtis:everything that they provide for you, everything that they have agreed to
Curtis:provide for you is in a contract.
Curtis:Go look at your contract.
Curtis:You will not find any words that sound like backup, restore, recovery, any
Curtis:of these things, they are not there.
Curtis:And if you look up something called the shared responsibility model, you will
Curtis:see that your responsibility that backups clearly fall under your responsibility.
Curtis:Their responsibility is availability of the platform,
Curtis:not availability of your data.
Curtis:There's very different concept there, right?
Curtis:Availability of the platform.
Prasanna:The same thing holds true for not just Microsoft 365
Prasanna:and like Gmail, but even AWS.
Prasanna:They have the same thing, with the shared responsibility model
Prasanna:where they make sure that yes.
Prasanna:Services available, but backups all the rest is yours.
Prasanna:Unless you explicitly buy service like AWS backup or something else like that.
Curtis:The difference between AWS and Microsoft ,though, is that's clearly
Curtis:spelled out in the documentation of every service that you look at.
Curtis:It will tell you what backups are included.
Curtis:what backups are not included where the backups are and what
Curtis:you can do to make them better.
Curtis:Whereas with Microsoft, they just don't talk about it, which is just really weird.
Prasanna:I think they don't talk about it because they don't want
Prasanna:people to really, they think people don't need to worry about it.
Prasanna:Like I think AWS brings it up because they realize you need to worry about this.
Prasanna:I think Microsoft thinks we don't need people to worry about this.
Prasanna:Why do people have to worry about it's the cloud, everything
Prasanna:should be available, but it's
Curtis:I would say they don't want people to worry about it.
Prasanna:But they don't give you a solution though, either.
Curtis:They don't give you a solution to the problem.
Curtis:Again, proponents.
Curtis:Of the no backup philosophy would state that they do give
Curtis:you ways to protect your data.
Curtis:And the most common thing that is, is touted.
Curtis:So first off they talk about the availability of the platform.
Curtis:They talk about replications and and how that it's part of this availability group.
Curtis:And and that's all great.
Curtis:But again, that's all about availability.
Curtis:That's not about.
Curtis:Backups.
Curtis:And
Prasanna:Backups and restores
Curtis:But th they also tout things like retention policies, the most common
Curtis:thing that I see touted by people that want to th that say, you don't need to
Curtis:back up the data is retention policies.
Curtis:And the thing is, here's news.
Curtis:Most people don't know anything about retention policies, right?
Curtis:Most people don't even know that they exist.
Curtis:Including a lot of backup vendors.
Curtis:They don't know that they exist.
Curtis:And so they just talk about the recycle bin.
Curtis:They're like oh the recycle bin is only two weeks or whatever, and
Curtis:they go after the recycle bin and.
Curtis:Proponents of 365 will then say there are retention policies.
Curtis:And if you really want to store data, if you want to make sure that data
Curtis:doesn't get deleted, even if it's purged from the recycle bin, then you should
Curtis:create a retention policy for that.
Curtis:And and it would save it.
Curtis:My, my big thing though, with the retention policy idea is that all
Curtis:that's really doing is keeping the email or the file or the record in 365,
Prasanna:It's almost like a soft delete, right?
Curtis:It is a delete
Prasanna:It's gone from the user's perspective.
Prasanna:It's gone from the admin's perspective, but it's not yet
Prasanna:gone from the system's persective
Curtis:right?
Curtis:It's just basically, I've said this before 365.
Curtis:And G-Suite and Salesforce, all of these apps, they're all really just a big, fancy
Curtis:database with an interface in front of it.
Curtis:And it's not like you actually have a, it's not like for the Unix folks.
Curtis:It's not like you're your 365 email is in an inbox format.
Curtis:That's in a directory somewhere.
Curtis:It's in a database.
Curtis:And each email is represented by a record in that database.
Curtis:And when you delete the email, what happens is it
Curtis:sets a flag that says deleted.
Curtis:And if you that, and also if it's got a retention policy, it sets a
Curtis:flag that it's going to be retained.
Curtis:So even if it's deleted, it just looks like it's been deleted, but
Curtis:it's really just sitting there.
Curtis:And that, that is not a backup.
Curtis:And why is that?
Curtis:Not a backup.
Prasanna:Because it lives within the same system.
Prasanna:So it doesn't follow the 3, 2, 1 rule.
Curtis:Yes,
Prasanna:See, I got it this time.
Curtis:And by the way, I just want to, I just want to say something and I'm
Curtis:gonna, I'm going to try not to return in kind, but there was this article
Curtis:from practical365.com and right off, they started out with an ad hominem
Curtis:attack in the beginning of the article.
Curtis:Right.
Curtis:Basically saying that people, vendors like Druva that are
Curtis:stating that you need to back up...
Curtis:Oh, here we go.
Curtis:So when you research it, though, you'll find lots of content telling you that
Curtis:you do need to, but more often than not, the content is sponsored, paid or
Curtis:authored by a company selling a 365.
Curtis:backup or continuity solution.
Curtis:Some are written with the express aim of convincing the reader
Curtis:that backups are essential.
Curtis:I would say they all are because they are anyway.
Curtis:But the idea that's literally paragraph two basically, oh anybody
Curtis:who says you need to backup 365, it's just because they sell it.
Curtis:And yeah, I work for Druva, but I've always felt like
Curtis:you need
Prasanna:you should back up.
Prasanna:your day.
Curtis:I've only worked for Druva for four years.
Curtis:I've always felt you need to back up SaaS resources.
Prasanna:And an example would be look at Salesforce.
Prasanna:They brought back in a backup solution.
Curtis:They did
Prasanna:In-house that they had gotten rid of, but they decided that
Prasanna:they're going to continue building out.
Prasanna:And the only reason that they would do that is because their customers are
Prasanna:probably asking them, we need something.
Curtis:So if I look at this article, the first thing that he
Curtis:goes after is retention policies.
Curtis:And it says retention policies are not backups by themselves,
Curtis:but they're part of the picture.
Prasanna:Which I wouldn't disagree with.
Prasanna:Just like
Curtis:right, I don't have any problem with it, but yeah, go ahead.
Prasanna:Yeah.
Prasanna:Just like we talked about with storage arrays, Right, Snapshots are part of
Prasanna:your overall data protection strategy.
Prasanna:It's not just the only thing that you use.
Curtis:Steve Goodman is the author.
Curtis:He talked about that.
Curtis:It's not backups, but it says, it just ensures that data
Curtis:isn't removed from the service.
Curtis:And he then alludes to the fact that people of my ilk will suggest well,
Curtis:the problem with that is that an admin, a malicious actor could go in
Curtis:and change your retention policy and thus delete everything . You could
Curtis:create a retention policy of zero, move everybody into that retention policy
Curtis:and suddenly all their email is deleted.
Curtis:And you know how we know that's possible.
Prasanna:Because it happens.
Curtis:Because it happened there is a giant story
Prasanna:Oh, I remember we did this.
Curtis:Yeah, that was the story.
Curtis:August, 2020.
Curtis:It was KPMG.
Curtis:They created a policy.
Curtis:It was an accident.
Curtis:But that's what backups are for, right?
Curtis:Is they created a retention.
Curtis:They needed to delete one particular person's data.
Curtis:So they created a, but they already had a retention policy that said that data
Curtis:is retained for 90 days or whatever.
Curtis:And so they create a retention policy that says zero, and then they were supposed to
Curtis:move that user into that retention policy.
Curtis:And instead they moved everybody into the retention policy and thus
Curtis:deleted 145,000 users personal chats.
Curtis:While I will just say that a lot of backup providers don't backup personal chats,
Curtis:and the reason for that is that Microsoft does not provide an API to get that data.
Curtis:The, and this does prove the point that retention policies are not perfect.
Prasanna:And they would, of course tout preservation lock.
Prasanna:If you turn on preservation lock, even if a bad actor goes in and shortens
Prasanna:all your retention down to zero that will only apply to data for the future.
Prasanna:Not data from the past.
Prasanna:The only problem with that is people that have turned on
Prasanna:preservation lock, then suddenly find out how much data it takes up.
Prasanna:And then they're like, holy cow.
Prasanna:And,
Prasanna:they can't do anything
Curtis:end up and they can, but they can't fix it.
Prasanna:Yeah, and this reminds me on storage arrays.
Prasanna:Most storage arrays offer two types of retention lock, one called
Prasanna:compliance, and one called governance.
Prasanna:Compliance is the one that you set and you can never unset it.
Prasanna:Like the preservation mode.
Prasanna:And a lot of people would start off with that.
Prasanna:And then they would realize, oh man, this is eating up a
Prasanna:lot more space than I expected.
Prasanna:And there's nothing they could do because it's set that way.
Prasanna:And so you either end up creating a different volume and start using that
Prasanna:other volume with a different mode on it in order to keep the data for as long
Prasanna:as you need, because everyone's Yeah.
Prasanna:I want to keep the data forever.
Prasanna:And then you start looking at your bill and how much data it's consuming.
Prasanna:And you're like, I did not expect that.
Curtis:Yeah.
Curtis:And so I think that again, there's nothing wrong with a retention
Curtis:policy, but it isn't backup because it doesn't copy the data anywhere else.
Curtis:And anything, if anything, catastrophic happens to your configuration?
Curtis:Yeah.
Curtis:It is that data's gone right now.
Curtis:People say, has this never happened?
Curtis:I don't know.
Curtis:But here's the thing.
Curtis:The cloud isn't magic.
Curtis:Microsoft isn't magic.
Curtis:Could it happen?
Curtis:Yes, it could.
Curtis:And by the way, I want to throw out something that, that some commenters
Curtis:on the thread have brought up like, has anything like this ever happen to you.
Curtis:So I've had Microsoft 365 for 10 years, and I've never had a problem.
Curtis:You know what, I've been in the industry for 30 years, I've never
Curtis:had to do a disaster recovery either.
Curtis:That doesn't mean I'm not going to have a DR plan.
Prasanna:Yeah
Curtis:I'm prepared.
Curtis:So the second section here he
Prasanna:The part before you move on,
Curtis:oh yeah.
Prasanna:you brought up a point about the 3, 2, 1 rule,
Prasanna:how keeping all the data local,
Curtis:Yeah,
Prasanna:means that it doesn't meet the requirement, but I believe
Prasanna:some people would also say, yeah, but Microsoft 365 allows you to
Prasanna:replicate the data to other geos.
Curtis:I can't speak to the geo replication capabilities of
Curtis:Microsoft 365, but I will just say replication is not backup right.
Curtis:For the same reason that, so yes, you can replicate, but replication is.
Curtis:Similar.
Curtis:It has other holes that when corruption happens in one place, it can automatically
Curtis:replicate the corruption to other places.
Curtis:This is why we came out with concepts like continuous data protection
Curtis:in the backup world because replication by itself is not helpful.
Curtis:It doesn't appear that they offer this as a service as an
Curtis:additional service, but that.
Curtis:There is a database availability group for exchange and that there are other
Curtis:lagged copies, but I will just say this, I have specifically, and you can call me on
Curtis:this and you can tell me I'm full of it.
Curtis:But I have specifically as a customer of Microsoft, 365, druva
Curtis:is a customer of Microsoft 365.
Curtis:I have specifically asked them, can I use a lagged copy of Microsoft 365
Curtis:exchange online to restore my service.
Curtis:And the answer was an unequivocal.
Curtis:No.
Prasanna:because like you said, it's intended for the service
Prasanna:availability, not for users to be able to restore the backup.
Curtis:So while this author of this article is talking about these lagged
Curtis:copies again, that's the other thing that that folks that are proponents and
Curtis:that understand 365, they often bring up lagged copies, show me documentation that
Curtis:says that I'm allowed to use a lagged copy to restore my database and I'll back off.
Curtis:Because I have documentation directly from Microsoft that
Curtis:says that I cannot use that copy.
Curtis:So
Prasanna:I remember you called into support and actually talked to someone
Curtis:yeah, I did.
Curtis:Yeah.
Curtis:Anyway, so the next section that he has here is it recovery inside the
Curtis:service as possible, but requires skill?
Curtis:The first sentence is that "the weakness in 365 is how complex it is
Curtis:to understand how to recover data."
Prasanna:Why?
Curtis:To which I just want to hang my head
Prasanna:yeah.
Curtis:restore should not be difficult.
Prasanna:Yeah.
Curtis:This is something brought up by the other person
Curtis:that we've often tangled with.
Curtis:When you're having your worst day, the last thing you want is
Curtis:a complicated restore process.
Prasanna:It's like the conversation we had last week
Prasanna:about you restoring backup central,
Curtis:Yes.
Curtis:It was a little more complicated than I thought it was going to be, but it
Curtis:worked out and I went to the people, I went to the people to help me out.
Curtis:They're saying administrators can use things like the search mailbox to
Curtis:recover data or use the e-discovery.
Curtis:By the way, I have tried to use the e-discovery
Curtis:functionality in 365 to restore.
Curtis:Data.
Curtis:And I'll just say this, it like a lot of other tools.
Curtis:It is definitely not a backup tool.
Curtis:E-discovery is, you know what that tool is made for Prasanna.
Curtis:I'm going to give you one guess.
Prasanna:E-discovery?
Curtis:Yeah.
Curtis:So it turns out it's really good at e-discovery and not
Curtis:so good at restoring stuff.
Curtis:First off, I found the e-discovery tool, clunky.
Curtis:I found creating the case was clunky.
Curtis:Getting the data from that case was clunky.
Curtis:Getting the data back into, a mailbox is also clunky
Prasanna:Because It's not intended to be used that wau
Curtis:It's not intended to be used that way.
Curtis:Also it doesn't include the concept of folders.
Curtis:It also doesn't understand the concept of point in time.
Prasanna:It's a little bit like when you have our archive
Prasanna:versus backup discussions,
Curtis:yes, it is.
Curtis:Exactly.
Curtis:E-discovery is an archive
Prasanna:Yeah.
Prasanna:They're very different use cases and therefore they have different semantics.
Prasanna:And when you try to make archive be used for restore or backup use for archive,
Prasanna:the world's collide and bad things happen.
Curtis:Yeah.
Curtis:When I hear someone suggesting that you can use e-discovery to restore a mailbox,
Curtis:I would like to see them do that on video.
Curtis:I really would.
Prasanna:Maybe for one email it's not bad.
Curtis:Yeah, for exactly, for one email, it's not bad but for a bunch of users
Curtis:who have had their email obliterated or OneDrive obliterated, and you're pointing
Curtis:them at e-discovery or that the admin has to go to e-discovery it doesn't
Curtis:understand the concept of point in time.
Curtis:In other words, you can't put your mailbox or OneDrive back to
Curtis:the way it looked two days ago.
Curtis:It just, it doesn't understand that concept.
Curtis:What it understands is give me all.
Curtis:Files or all of the emails that went to Curtis all of them in this timeframe.
Curtis:Which is not the same thing as give me all the emails that were
Curtis:in Curtis's email box yesterday, it doesn't understand that concept.
Curtis:And then the last recommendation was PowerShell commandlets too,
Curtis:which I just want to hang my head.
Curtis:Again, it should be easy.
Curtis:And if the data has been purged and it's only held in a retention policy, it's
Curtis:going to be complicated and the only way to get it is the e-discovery interface.
Curtis:And I found that clunky.
Prasanna:Your last line of defense.
Prasanna:But it shouldn't be your first option.
Curtis:Yeah.
Curtis:I live in this crazy world where people do dumb stuff all the time,
Curtis:and then they need a way out of it and that way should be simple.
Curtis:And the thing is the way we back up 365.
Curtis:We would restore right back to where it came from.
Curtis:We would restore the folder structure that it came from and, et cetera,
Curtis:et cetera, et cetera, it would just, it'd just be restore Curtis
Curtis:to the way he looked yesterday.
Curtis:That's what backups are for, if you don't think that matters.
Curtis:I don't know what to tell you.
Curtis:We're probably not going to share drinks.
Prasanna:well, and if I'm the Microsoft 365 admin and I got this request
Prasanna:and it's I've never really used the e-discovery tool or whatever other tool.
Prasanna:And now I'm trying to figure it out as I have a CEO breathing
Prasanna:down my neck or someone very important or whatever else it is.
Prasanna:If I've never done this operation.
Prasanna:Or it's not dead simple, then there's probably going to be a mistake being done.
Prasanna:And someone's going to be very unhappy.
Curtis:And then there's the final and this one, again, it's a very
Curtis:common thing that people bring up on that side of the discussion.
Curtis:And that is this final heading is Microsoft 365 backup products have
Curtis:key gaps that limit the security and productivity of your organization.
Curtis:So the point being made here is because Microsoft doesn't provide
Curtis:Druva and other companies like Druva APIs to get all of the data.
Curtis:Therefore our backups are invalid I just want to, again, throw my hands in the air.
Curtis:We acknowledge that and we plead with Microsoft to stop offering
Curtis:new services without backup KPIs.
Curtis:But.
Curtis:Because we can't protect something like Yammer, for example, doesn't
Curtis:mean we can't protect exchange
Prasanna:The rest of this stuff.
Prasanna:Yeah.
Curtis:And they also list teams and we do a pretty good job with teams.
Curtis:I know at Druva we do some things that a lot of other companies don't, and,
Curtis:but we are limited by the API APIs
Prasanna:what functionality it allows you.
Curtis:Yeah, and they list some stuff like sensitivity labels, AIP, or MIP
Curtis:functionality and to ask about how they backup and restore that data.
Curtis:And I don't disagree with any of that, but I don't understand how,
Curtis:because we can't get everything.
Curtis:We shouldn't get what we can.
Curtis:And I, as a customer of Microsoft would say to them, Hey, why don't make
Prasanna:Yeah.
Prasanna:And I think the other thing is, as a backup vendor, you need to be open about,
Prasanna:Hey, here are all the things we can't do.
Prasanna:Rather than just saying yeah, we back up everything.
Prasanna:It's good to go.
Curtis:Good point.
Curtis:Yeah.
Prasanna:I think Druva, at least last I looked right.
Prasanna:Was very upfront about here are the Microsoft 365 services we can back up
Prasanna:here are the things that we can't support.
Prasanna:And like you mentioned, it's typically because APIs don't exist.
Curtis:The irony of this particular issue is they're saying that the
Curtis:reason why our backups aren't valid because we can't get everything.
Curtis:Guess what?
Curtis:Their backups don't get everything.
Curtis:And again, I just called what I just called, what they do, backups,
Curtis:what they do, aren't backups.
Curtis:But their Protection mechanisms.
Curtis:Don't get everything.
Curtis:There are a handful of things, just like there are handful
Curtis:of things that we can't get.
Curtis:There are a handful of things that a person who only has 365 tools cannot
Curtis:recover off the top of my head.
Curtis:I don't have them.
Curtis:Vanessa at Druva I know is, is a 365 expert.
Curtis:I'm not one.
Curtis:And we have talked about some things that, again they're not as common as
Curtis:some of the other things, just like Yammer is not that common among the, and
Curtis:if you're listening to this use Yammer, I'm not saying nobody's using Yammer.
Curtis:I'm just saying it's certainly not as common as say exchange online.
Curtis:Teams has become a lot more common in the last couple of years due to COVID.
Curtis:And due to them, adding some functionality to answer slack, right?
Prasanna:And
Curtis:I've used teams and I've used slack and I'll take slack any day, but
Prasanna:I was just going to also mention that you brought up Vanessa.
Prasanna:So we did actually record a couple episodes with her episodes, 85
Prasanna:and 86, where she did talk about the architecture of Microsoft 365,
Prasanna:that needs to be backed up and why you need to back up Microsoft 365.
Curtis:Yep.
Curtis:We should point people to those episodes because I'm pretty sure
Curtis:she went into some of those things.
Prasanna:I learned a lot from that discussion as well about Microsoft 365.
Curtis:When I look at, so there was a part one and a part two.
Curtis:When I look at the second part, he talks about prevention is better than cure.
Curtis:I don't disagree.
Curtis:I would not tell a customer not to use retention policies . But I will say this,
Curtis:if e-discovery is a regular part of your workflow, I will put our e-discovery
Curtis:capability up against the built-in e-discovery tool any day of the week.
Curtis:And if you're paying extra, the ones that I really don't understand are guys
Curtis:that are customers that are paying extra to get E5, and that mainly what they
Curtis:need is to e-discovery functionality.
Curtis:We cost way less than the cost difference.
Curtis:The cost differential is $15 a user
Prasanna:Do you want to mention what E5 is just for
Curtis:I, yeah.
Curtis:E5 is a Microsoft licensing level, so it's E1, E3, E5, right, and the
Curtis:biggest difference between E3 and E5 is the e-discovery capability.
Curtis:And also the eh things like looking for ransomware
Curtis:notification and things like that.
Curtis:And we offer the same functionality.
Curtis:And, but, and again, I would say.
Curtis:I would put our e-discovery functionality up against theirs any day of the week
Prasanna:so you get both backup and e-discovery for less than the price
Curtis:and actual backup not fake backup.
Curtis:I don't know.
Prasanna:It's a little bit like a religious war,
Curtis:It really is a religious war.
Curtis:I will say that, that I don't understand why.
Curtis:The cloud makes this any less of an issue.
Prasanna:Magical!.
Curtis:I don't disagree that 365 is not a really well-designed product.
Curtis:It's just, it's not magic and so bad things can happen.
Curtis:And no, I can't describe all of the bad things that could happen.
Curtis:But that's why we have backup.
Curtis:That's why we have backup everywhere else in IT, and I don't understand why
Curtis:365 is somehow magically different.
Prasanna:Yeah.
Curtis:I want to throw another claim that you hear from people is one of the
Curtis:claims that some people mentioned is what if 365 goes down, you have a backup of it.
Curtis:And these guys were like where are you going to restore it to?
Curtis:Okay.
Curtis:So today, There is an assumption that Microsoft would come back up.
Curtis:Okay.
Curtis:Second is that what you do have is an E discovery capability.
Curtis:You do have ability to search and get access to the most important email
Curtis:that you can download to your laptop.
Curtis:The most important document that you're working, that you can get down.
Curtis:No, you're not going to have the entire service, but you can get the most
Curtis:important things that you were working on.
Prasanna:Yeah.
Prasanna:Or if you had a presentation in an email somewhere, or the email service
Prasanna:down, at least you can go fetch that information while the services.
Curtis:Exactly.
Curtis:We're not going to be running your email for you.
Curtis:It's not DR for email.
Prasanna:But if Microsoft goes down and Microsoft 365 is down, there's a lot of
Prasanna:companies that are going to be in pain,
Curtis:and that was oddly enough that they said the same thing and
Curtis:one of the, one of the articles or comments, it was like, yeah.
Curtis:Okay.
Curtis:So what are you saying?
Curtis:But as long as companies that don't have a third party backup
Curtis:will be essentially dead in the water, unable to work on anything
Curtis:that was recent while 365 is down.
Curtis:And by the way, 365 has gone down,
Prasanna:yeah.
Prasanna:Yup, It reminds me, do you remember when Salesforce did that upgrade
Prasanna:where they ended up messing with a bunch of user metadata?
Prasanna:I think the user objects and they told people go to your backups.
Prasanna:And a lot of companies were like, we don't have anything or go to your sandboxes.
Curtis:That was really weird because.
Curtis:A script of theirs damaged customer data.
Curtis:It changed it so that everybody could see everything
Prasanna:Yeah.
Prasanna:But that's why you need backups.
Curtis:that's why you need backups.
Curtis:And yes, we have had customers Druva has had customers restore data from
Curtis:their backups in Microsoft 365.
Curtis:It's hard to get them to come out and talk publicly, but I know that we have at least
Curtis:one customer that agreed to a case study that we can that we publish on druva.com.
Curtis:But I don't know.
Curtis:I just, I don't understand.
Curtis:I don't disagree it at 365.
Curtis:Isn't a, a solid product.
Curtis:Isn't a, well-designed product and actually does a lot of, it has a lot
Curtis:of really convenient, like what I would call restore convenient features.
Curtis:They mimic backup though.
Curtis:They're not backup.
Curtis:They are essentially snapshots.
Curtis:And everything is all stored inside the same system, and just because
Curtis:a catastrophic failure hasn't happened of some 365 customer yet
Curtis:doesn't mean it won't ever happen.
Curtis:We have had incidents of the other things like ransomware
Curtis:and rogue admins and stuff,
Prasanna:I think just trying to bubble it up even higher level, even though
Prasanna:we've talked a lot about Microsoft 365, I think this applies to any SAS or
Prasanna:cloud solution that a customer is using.
Prasanna:That they may not even realize like service now or Zuora or Zen desk, or
Prasanna:you take your pick of cloud services.
Curtis:Exactly.
Curtis:Any service where you're creating data in that cloud, right?
Curtis:It does.
Curtis:Yeah.
Curtis:Thanks for bringing that up.
Curtis:It really does apply.
Curtis:And the only reason why I'm harping on 365 and have multiple times is it only
Curtis:seems to be with the 365 customer base and their fandom where I get this problem.
Curtis:No one, I don't know anyone arguing that G suite backs up their data.
Curtis:Occasionally we get this, the Salesforce stuff, but that's pretty
Curtis:easy because it's documented Salesforce documented it, amazon documents
Curtis:pretty good on, on their stuff, but
Prasanna:Microsoft
Curtis:just, they just don't say anything.
Curtis:It's annoying and I wish they'd come out and be, they do, by the
Curtis:way in there, this is the weird part is there's 365 for end users.
Curtis:And there's 365 for businesses and the 365 for end-users.
Curtis:They specifically state you should back up your stuff.
Curtis:But in the in the commercial version of the product, they just don't say anything.
Curtis:They don't say we have a backup of your stuff.
Curtis:You're good.
Curtis:You don't need to backup your stuff
Curtis:but they all, they also don't say you do need to backup your stuff.
Curtis:So it's just, it's a combination of the fact that Microsoft
Curtis:is not upfront about this.
Curtis:And then you have multiple people that are saying that it's not needed.
Prasanna:And they may not be upfront about it because maybe
Prasanna:in their minds, they think what they've provided is backup.
Curtis:Then put it in writing, baby.
Curtis:That's all I got to say.
Curtis:I go back to my days of watching judge Judy, it was not in writing.
Curtis:It doesn't count.
Curtis:And I don't know.
Curtis:I, it just seems like on your worst day, the worst, the problem is the worst,
Curtis:the ransomware attack or the malware attack or the bad actor attack is worst.
Curtis:The problem is going to be.
Curtis:And again, I don't care, even if you've turned on retention policies and you've
Curtis:turned on the preservation lock, even if you've done all of that stuff.
Curtis:For anything, that's not in the recycle bin,
Curtis:putting that back is going to be a giant pain in the butt.
Curtis:And do you really want that to be what happens to you on your last day?
Curtis:And again, another claim that the guys may look like because of the
Curtis:APIs and the limits and the throttle limits and everything, it'll take
Curtis:you a really long time to restore an entire Microsoft 365 tenant to which
Curtis:I go, yes, I don't disagree with that.
Curtis:But you could be picky about it.
Curtis:You could just pick the last, week or so is worth of emails and just restore
Curtis:those for now while you continue working and then restore the older stuff
Prasanna:later.
Prasanna:Yeah.
Prasanna:Or prioritize which users get restored first.
Prasanna:There's so many options all
Prasanna:Always yours, Curtis, but I people don't think about that.
Prasanna:The other thing is, at least you're able to restore your data.
Prasanna:Even if it takes 50 days, at least you got your data back versus
Prasanna:what, if you don't get your data back, what are you going to do?
Prasanna:It'll be like, what's that Gmail com or the Google company
Prasanna:that accidentally deleted their account and couldn't restore it.
Prasanna:And then the company closed down because it had all their intellectual property?
Curtis:Yeah.
Curtis:I the name of company is escaping me, but
Curtis:And then they tried to Sue Google.
Curtis:Yeah.
Curtis:Yeah.
Curtis:Nothing happened with that.
Curtis:Did it?
Prasanna:I don't think so.
Prasanna:Yep.
Curtis:Yeah,
Prasanna:So don't be like that
Curtis:public about it.
Curtis:Yeah, don't be like that company.
Curtis:The cloud is not magic.
Curtis:SaaS is not magic.
Curtis:Microsoft is not magic.
Curtis:And Microsoft is not upfront about what they're doing to protect your data.
Curtis:They got a really, a lot of really nice features, but all of the
Curtis:data is still all in one place.
Curtis:Even if it's in an availability group again, that replicated copy that
Curtis:lag copy is not available to you.
Curtis:Don't believe me, ask Microsoft.
Curtis:If the world blows up and I need to use the lagged copy of, my
Curtis:database to restore my environment.
Curtis:Can I
Prasanna:if they said yes, Would your answer change or would your opinion
Prasanna:about Microsoft 365 change Curtis, I'm going to put you on the hot spot
Curtis:Good question.
Curtis:If they said yes, I would ask them to put that in writing.
Curtis:Right as a customer.
Curtis:I would ask them if that, if it is yes, although I'm pretty sure it's not
Curtis:going to be yes, but if it is yes, then why is that not in the documentation?
Curtis:What is the scenario that I need to, what is the process that I
Curtis:need to go through to restore my completely blown away exchange online
Curtis:or SharePoint online environment.
Curtis:And it's not there.
Curtis:Back up your 365 and your G suite and your Salesforce and all the other,
Prasanna:Until Microsoft changes their mind, but until then back it up
Curtis:But even that I'm just gonna say this.
Curtis:So Salesforce, for example, now offers a for-pay backup service.
Curtis:But anyway, it's a little expensive, but cause we have to compete with it now.
Curtis:We were looking at pricing when the pricing came out, was quite surprising.
Curtis:Just how much they're going to charge for the backup service, but.
Curtis:I still would not want to use a backup service built by the people that
Curtis:the original was made from, again, given the choice, I would choose a
Curtis:third party backup service, just my,
Prasanna:I'm going to challenge
Curtis:you on that,
Prasanna:Curtis
Curtis:you, you're going to make the argument the other way, aren't you?
Curtis:Because they know the data best
Prasanna:They know the data.
Prasanna:And I'll give you an example
Curtis:you know what?
Curtis:They don't know best
Prasanna:what?
Curtis:Backup.
Prasanna:I agree.
Prasanna:I will give you a counter example, which is Oracle and RMAN.
Curtis:Okay.
Prasanna:Great.
Prasanna:I think if Microsoft provided a similar mechanism as Oracle does
Prasanna:RMAN,
Curtis:Service to backup Oracle,
Prasanna:No.
Curtis:our Oracle doesn't PR or Oracle doesn't,
Prasanna:So
Curtis:know what I'm saying?
Prasanna:But with RMAN, right?
Prasanna:They understand the data, they understand the format and they say we could either
Prasanna:write it to an NFS target or an S3 target, which Oracle will do on its own.
Prasanna:Or you could plug in a backup vendor's, libraries, if you will into Oracle RMAN
Prasanna:and have it manage moving the data.
Curtis:With respect to my friend.
Curtis:I don't think that's a valid comparison because one is a software tool, right?
Curtis:And the other is a service.
Curtis:So my point is that Salesforce as a service is running on infrastructure
Curtis:right next to that infrastructure will be where the backup service is running.
Curtis:And I'm not talking about a 3-2-1 thing thing.
Curtis:I'm just saying the same people have designed the same two pieces of
Curtis:infrastructure, and they may have made the same catastrophic decisions on both parts.
Prasanna:I agree.
Curtis:That's not the same as
Prasanna:I agree.
Curtis:a software backup tool.
Curtis:anyway.
Curtis:All right did you have fun?
Prasanna:I did.
Prasanna:I always like talking about Microsoft 365 with you, for some reason.
Prasanna:I think it might've been the first topic that you brought up
Prasanna:to me that had your blood boiling before we started the podcast,
Curtis:Yeah.
Prasanna:I do remember when you were on those calls with Microsoft
Prasanna:and you're like, oh my God, I can't believe it did you know?.
Curtis:Yeah.
Curtis:Yeah.
Curtis:Thanks to the listeners.
Curtis:Otherwise it's just me and you talking to microphones.
Curtis:And remember to subscribe so that you can restore it all.