Artwork for podcast The Backup Wrap-Up
Getting approval for a new backup design (Backup to Basics Series)
31st October 2022 • The Backup Wrap-Up • W. Curtis Preston (Mr. Backup)
00:00:00 00:46:59

Share Episode

Transcripts

Speaker:

​ W. Curtis Preston: hi, and welcome to

Speaker:

I'm a host w Curtis Preston, aka Mr.

Speaker:

Backup, and I have with me my software troubles consultant, Prasanna Malaiyandi.

Speaker:

How's it going, Prasanna?

Prasanna Malaiyandi:

I'm good.

Prasanna Malaiyandi:

I'm good Curtis.

Prasanna Malaiyandi:

And like software troubles.

Prasanna Malaiyandi:

You should expect it.

Prasanna Malaiyandi:

You've been in the industry for how long now?

Prasanna Malaiyandi:

Like it

W. Curtis Preston:

I know, but I, I, you know, that I generally don't

W. Curtis Preston:

like, I generally, so like, I like tech and then like, I like working

W. Curtis Preston:

on tech, but then I like using tech and the tech that I'm using.

W. Curtis Preston:

I just want it to work.

W. Curtis Preston:

I'm not TiVo anymore, but you know, when I had the TiVo, that's why I

W. Curtis Preston:

opted for TiVo and not Linux box.

W. Curtis Preston:

With hard drive in some software that will record my TV stream, right?

W. Curtis Preston:

Um, the stuff that I actually use on a day to day basis.

W. Curtis Preston:

I want to just, I want it to just work.

W. Curtis Preston:

And I've broken that trend by, uh, upgrading a tool that I love, right?

W. Curtis Preston:

I love descript, but I've decided to go on the bleeding edge with

W. Curtis Preston:

their new version, which is amazing.

W. Curtis Preston:

But it's in beta and I'm and I'm using a beta tool for the first time.

W. Curtis Preston:

And I'm like, I don't know.

W. Curtis Preston:

I push this button and it doesn't, it doesn't do the thing.

W. Curtis Preston:

Right.

W. Curtis Preston:

And then, yeah,

Prasanna Malaiyandi:

But, but, but at least you have to hand it

Prasanna Malaiyandi:

to the descript support folks for actually like, taking your calls and

Prasanna Malaiyandi:

your chats and your emails, Right.

Prasanna Malaiyandi:

And be like, Hey, let me walk you through that.

W. Curtis Preston:

they're good with live chat.

W. Curtis Preston:

Although there's been a handful where.

W. Curtis Preston:

Where at some point they're like, you know, I think it's

W. Curtis Preston:

best to transfer this to email.

W. Curtis Preston:

That's when you know, you've put, you've asked like one too

W. Curtis Preston:

many questions in a support chat.

W. Curtis Preston:

They're like, I think maybe you should put all your questions in an email.

W. Curtis Preston:

I'm like, But that's not, that's not how my brain works.

W. Curtis Preston:

Like I, I get a question, I get it answered.

W. Curtis Preston:

I work for a little bit a while and then I get another question, you know?

W. Curtis Preston:

Um, yeah.

W. Curtis Preston:

Anyway, so

Prasanna Malaiyandi:

sometimes it unblocks you and then you like hit the next

Prasanna Malaiyandi:

thing and you're like, Oh, what is this?

Prasanna Malaiyandi:

Which is why if you're a good product manager, right, you go in front

Prasanna Malaiyandi:

of customers and you're like, Hey, show me what you're doing and let me

Prasanna Malaiyandi:

figure out like what your problems are, what your pain points are,

Prasanna Malaiyandi:

what features I should be building?

W. Curtis Preston:

Yeah.

W. Curtis Preston:

And, and it is, it is kind of funny though that I now, I'm like

W. Curtis Preston:

on a first name basis with like four or so of their, of their, uh,

Prasanna Malaiyandi:

shout out to the descript support team

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Shout out to for putting up with my, uh, my questions.

W. Curtis Preston:

Cause it's like, it's a combination of, the new version is significantly

W. Curtis Preston:

different than the old version.

W. Curtis Preston:

Um, and so it's a, it's a combination between Is this the way it's

W. Curtis Preston:

supposed to work now or is this the way it's supposed to work?

Prasanna Malaiyandi:

Well, and

Prasanna Malaiyandi:

I

W. Curtis Preston:

think it's broken.

Prasanna Malaiyandi:

And I think for a lot of the features, right,

Prasanna Malaiyandi:

they're building the right things.

Prasanna Malaiyandi:

Like just talking to you like they've optimized some of your flows and some of

Prasanna Malaiyandi:

the things that you've spent so much time working on in the past, right?

Prasanna Malaiyandi:

So it's going to get there, It's just, it's gonna take a little bit of time.

W. Curtis Preston:

But it's been, it's been a difficult

W. Curtis Preston:

couple of weeks, you know?

W. Curtis Preston:

Um, anyway, let's bring on our longtime friend.

W. Curtis Preston:

Uh, I, I've known him for, I don't know, for forever, and he's for, he's

W. Curtis Preston:

been on the podcast multiple times.

W. Curtis Preston:

You may remember him from, from such hits as how does Hollywood do backup?

W. Curtis Preston:

And, uh, he is also.

W. Curtis Preston:

A, uh, a co-author on my book, uh, latest book, uh, Modern Data

W. Curtis Preston:

Protection, which by the way, you can get free by going to druva.com/ebook.

W. Curtis Preston:

And, um, he wrote the chapter that we're gonna talk about today

W. Curtis Preston:

in our Back to basic series.

W. Curtis Preston:

He is now the head of Technical Operations for Frame Store, a

W. Curtis Preston:

visual effects studio in Hollywood.

W. Curtis Preston:

Welcome to the podcast Jeff Rochlin.

Jeff Rochlin:

Hey.

Prasanna Malaiyandi:

Yeah.

Prasanna Malaiyandi:

. . Woo-hoo.

Jeff Rochlin:

Hi guys.

Jeff Rochlin:

How are ya?

W. Curtis Preston:

How's it going, Jeff?

Jeff Rochlin:

It's going.

Jeff Rochlin:

It's going well.

Jeff Rochlin:

It's going.

W. Curtis Preston:

It, it's, uh, it, it is it nice to be back?

W. Curtis Preston:

Cuz I know for a while you weren't, you weren't, you, you've always

W. Curtis Preston:

enjoyed sort of combining the tech side of you and the, and the, you

W. Curtis Preston:

know, the entertainment side of you.

W. Curtis Preston:

And for a while you weren't in that space Right now it's good to be

W. Curtis Preston:

back with, you're a full person.

W. Curtis Preston:

You get to be in entertainment and be in tech at the same time.

Jeff Rochlin:

definitely.

Jeff Rochlin:

Um, I love the creative.

Jeff Rochlin:

Part of it.

Jeff Rochlin:

And I love dealing with the crazy people that make creative things.

Jeff Rochlin:

And so it's, that's, that's why I've spent, you know, the better part

Jeff Rochlin:

of the last 30 years working at, um, uh, some kind of movie studios,

Jeff Rochlin:

preferably in production and not, you know, it cause anybody, this will get

Jeff Rochlin:

me in trouble, anybody can do email.

Jeff Rochlin:

But, you know, dealing, dealing with the, dealing with the executive

Jeff Rochlin:

producer who's trying to get his movie out on time is a different kind of,

Prasanna Malaiyandi:

Yeah, I can only imagine the pressure with that

Jeff Rochlin:

Um, yeah, the, the only place I've worked in the past that

Jeff Rochlin:

where the pressure was worse in a different kind of way was Wall Street.

Jeff Rochlin:

Working on a trading floor in Wall Street is a different piece also,

Jeff Rochlin:

but

W. Curtis Preston:

cuz that's where like you're, you know, you're making a

W. Curtis Preston:

trade or you're trying to make a trade or you're trying to make thousands of

W. Curtis Preston:

trades and suddenly the system goes down.

Jeff Rochlin:

Right?

Jeff Rochlin:

And it's, and it's all time critical, right?

Jeff Rochlin:

Because the price is not sitting static waiting for you to close the trade.

Jeff Rochlin:

If you don't buy it right now, then five minutes from now it's gonna be, you know,

Jeff Rochlin:

a hundred million dollars more in cost and there goes your profit margin, right?

Jeff Rochlin:

So,

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Which is why, at least in in my IT experience, and I spent a, I spent a good

W. Curtis Preston:

time in and outta Wall Street and working with other consultants that I knew that

W. Curtis Preston:

were working on Wall Street, and I, I got the sense, it was the one place that I've

W. Curtis Preston:

seen where it's like money is no object.

W. Curtis Preston:

So they do, they do what is right.

W. Curtis Preston:

To, um, to have fully, highly available systems.

W. Curtis Preston:

Um, and I, I'd say Hollywood might be right behind that, right?

W. Curtis Preston:

Because you've got all these super expensive actors, um, and super expensive,

W. Curtis Preston:

you know, both above and below the line.

W. Curtis Preston:

People see, I learned from you, I learned that

Jeff Rochlin:

You did.

Jeff Rochlin:

you.

Jeff Rochlin:

did.

W. Curtis Preston:

Uh, and, and they're all sitting there and, and

W. Curtis Preston:

you know, and if, and if they can't do it because the computers are down,

Prasanna Malaiyandi:

what?

Prasanna Malaiyandi:

What's above and below the lo line?

Prasanna Malaiyandi:

By the way,

W. Curtis Preston:

oh.

Jeff Rochlin:

anybody in the creative titles, actors, directors, producers,

Jeff Rochlin:

writers are called Above the line.

Jeff Rochlin:

Um, anybody in the skilled trades and the crew, right?

Jeff Rochlin:

The, the camera, the, uh, the gaffer, who's the electrician, uh, you know,

Jeff Rochlin:

those kind of got the stage hands.

Jeff Rochlin:

Those are below the line.

Jeff Rochlin:

Um, and the best way to describe it is, I guess if you watch the credits of a

Jeff Rochlin:

movie, um, at the end, you know, they always run the, the cast gets their,

Jeff Rochlin:

their individual cards and the people who wrote the movie get their cards,

Jeff Rochlin:

and then it goes into that long scroll.

Jeff Rochlin:

Of everybody else that's kind of above and below the line, you know, visually.

Jeff Rochlin:

Um,

Prasanna Malaiyandi:

Interesting.

Prasanna Malaiyandi:

Yeah.

Jeff Rochlin:

so Hollywood is kind of always walking that middle line between,

Jeff Rochlin:

I wanna, I want perfection and I really don't want to have to pay for it.

Jeff Rochlin:

You know what I mean?

Jeff Rochlin:

Which is sort of what the rest of the world is doing as well.

Jeff Rochlin:

Um, how can I strike the balance so that I know I'll get a hundred percent of what

Jeff Rochlin:

I need and yet at the same time, not have to spend the money to, to make it happen?

Jeff Rochlin:

Uh, backups are a, backups and storage in a production are a

Jeff Rochlin:

perfect example of this, right?

Jeff Rochlin:

Because artists iterate their work over and over and over again.

Jeff Rochlin:

They create a lot of files at the end of the day that they don't need because.

Jeff Rochlin:

They've done, you know, they've done 20 versions of the scene and the

Jeff Rochlin:

21st is the one they're gonna use.

Jeff Rochlin:

Um, and then you go back to them and say, We're out of space.

Jeff Rochlin:

Go clean up.

Jeff Rochlin:

And they look at you like, Well, I don't have time to clean up.

Jeff Rochlin:

I'm busy making art.

Jeff Rochlin:

If you want somebody to clean up, then I can't make art.

Jeff Rochlin:

Right.

Jeff Rochlin:

. And you're looking at 'em and going, Well, I can go spend a hundred

Jeff Rochlin:

thousand dollars and buy more storage for you to put more stuff on.

Jeff Rochlin:

But eventually somebody's gotta clean this up cuz you don't need that storage.

Jeff Rochlin:

Um, and that's been a kind of a standard theme throughout my career in Hollywood.

Jeff Rochlin:

Um, they, they are upset when it's broken.

Jeff Rochlin:

They, it costs them a lot of extra money to make up for it when it's broken, right?

Jeff Rochlin:

Because over time on skilled trade and union people is not cheap.

Jeff Rochlin:

Um, and so we do everything we can to make sure that it doesn't break..

Prasanna Malaiyandi:

And I'm guessing, like especially with Hollywood, right,

Prasanna Malaiyandi:

when you're producing or creating right, uh, content, right, Those are

Prasanna Malaiyandi:

like significantly large files, right?

Prasanna Malaiyandi:

Uh, like you said, if you're doing like 21 versions of it, right?

Prasanna Malaiyandi:

The only other industries I could really think of that are similar is

Prasanna Malaiyandi:

probably things in like the, uh, genome space or the mining space, right?

Prasanna Malaiyandi:

Where you just have a whole bunch of data all coming in at once, but.

Jeff Rochlin:

And we're in, uh, from a, like a rendering, the, the final

Jeff Rochlin:

stage of production compute process.

Jeff Rochlin:

It's the same kind of technology being used, right?

Jeff Rochlin:

We run supercomputing clusters on the back end to render large projects, uh,

Jeff Rochlin:

just like oil and gas, use supercomputing clusters to analyze data to figure

Jeff Rochlin:

out where to drill next, right?

Jeff Rochlin:

Um, and as the, the beautiful picture that you get on your TV at home gets

Jeff Rochlin:

better and better when it moves from two K to 4k to eight K to 16 K plugged

Jeff Rochlin:

directly into the headset that's connected into your cerebral cortex so that you're

Jeff Rochlin:

living in the picture , that the files get exponentially larger every time.

Jeff Rochlin:

Right.

Jeff Rochlin:

Um, and I think I did the math for you a long time ago where, think about a.

Jeff Rochlin:

From the perspective of there are 23 frames, still pictures in every

Jeff Rochlin:

second of the film, and the film averages two and a half hours.

Jeff Rochlin:

Right.

Jeff Rochlin:

That's a lot of, that's a lot of images,

W. Curtis Preston:

Right.

W. Curtis Preston:

And, and, you know, and, and I'm glad, I'm glad we sort of started with this

W. Curtis Preston:

part of the discussion because what we wanted to talk about, you know, this is

W. Curtis Preston:

our back to basic series and I'll take this opportunity to do our disclaimer,

W. Curtis Preston:

Um, uh, Prasanna and I and Jeff, we all work for different companies.

W. Curtis Preston:

I work for Druva, Prasanna works for Zoom, and Jeff works for Frame Store.

W. Curtis Preston:

We're not speaking for our companies.

W. Curtis Preston:

We're speaking, uh, our individual, um, uh, opinions.

W. Curtis Preston:

And uh, also if you would like to rate us, we'd love you to do that.

W. Curtis Preston:

We're.

W. Curtis Preston:

Running our Santa Claus special . If you, if you want to see this beard get

W. Curtis Preston:

longer and longer to turn into a Santa Claus beard by Christmas, then uh,

W. Curtis Preston:

you know, you gotta get us a few more comments in, uh, specifically in iTunes.

W. Curtis Preston:

So go to the iTunes, just scroll to the bottom, you know, click on and give us

W. Curtis Preston:

a, give us a rating, give us a comment.

W. Curtis Preston:

And, um, if you'd like to be a part of this conversation, please reach

W. Curtis Preston:

out to me at WC preston on Twitter or w Curtis Preston on gmail.

W. Curtis Preston:

And, um, so we, we've been doing this back to the basics, uh, series, Jeff.

W. Curtis Preston:

And, uh, so basically I'm, I'm pulling out, you know, my favorite book, Modern

W. Curtis Preston:

Modern Data Protection, which you may recall having, having a part in.

W. Curtis Preston:

And, um, we already did chapter one, it was Why We back up.

W. Curtis Preston:

Right, Right.

W. Curtis Preston:

Risks, what are all the things that, that could do damage to data.

W. Curtis Preston:

and what I really want, you know, we talk a lot about, in the backup space, we

W. Curtis Preston:

talk a lot about, um, the fact that too many times, those of us in technology,

W. Curtis Preston:

we focus only on the technology, and then we find ourself asking questions, kind of

W. Curtis Preston:

like you were talking about the stores.

W. Curtis Preston:

Well, what should my retention be?

W. Curtis Preston:

Right?

W. Curtis Preston:

How long should I keep my backups?

W. Curtis Preston:

How long, how, how often should I do backups?

W. Curtis Preston:

Right?

W. Curtis Preston:

What, So, and you know, and how quickly should I backup and how

W. Curtis Preston:

quickly should I be able to restore?

W. Curtis Preston:

And what I'm constantly telling people is, you should not be

W. Curtis Preston:

the one answering this question.

W. Curtis Preston:

You should always go back to the business because the business is

W. Curtis Preston:

why you are doing what you're doing.

W. Curtis Preston:

And so, Chapter two is about, um, gathering and determining

W. Curtis Preston:

your service levels.

W. Curtis Preston:

we have to realize what our organization does, whether it's

W. Curtis Preston:

a government organization or a business organization, and I think

W. Curtis Preston:

you always understood that role.

Jeff Rochlin:

Um, yeah.

Jeff Rochlin:

Well, so one of the things that we do as workers at any particular company we

Jeff Rochlin:

go to is we focus on the job we have to.

Jeff Rochlin:

Learning everything we need to learn about the job and, and understanding

Jeff Rochlin:

that so we could be good at it, right?

Jeff Rochlin:

But very few people started a company and then sit down and say, I wanna

Jeff Rochlin:

understand the product we produce.

Jeff Rochlin:

Right?

Jeff Rochlin:

I wanna understand what the marketing is all about.

Jeff Rochlin:

I wanna understand everything this business does because I don't just see

Jeff Rochlin:

myself as a worker for the company.

Jeff Rochlin:

I see myself as part owner of this business, right?

Jeff Rochlin:

I'm, I'm a critical cog in the wheel that makes the place produce something.

Jeff Rochlin:

And so that's always been my approach to my jobs.

Jeff Rochlin:

It's not good enough to just know I'm a sysadmin.

Jeff Rochlin:

I have to be able to write scripts to run this Unix box.

Jeff Rochlin:

Um, I wanna understand that we rent cars to people all

Jeff Rochlin:

over the, all over the world.

Jeff Rochlin:

And there's a process involved in how that works or.

Jeff Rochlin:

When I worked in, in broadcasting, right, I took the time to understand what

Jeff Rochlin:

the actual broadcast chain is, right?

Jeff Rochlin:

We create an image, it goes into a bunch of systems before

Jeff Rochlin:

it goes out over the air.

Jeff Rochlin:

It passes through a dozen different things that layer different sub carriers on it.

Jeff Rochlin:

Like your Nielsen ratings box, right?

Jeff Rochlin:

There's a, there's a subcarrier running on the signal that tweaks

Jeff Rochlin:

your Nielsen box that says, Hey, he's watching Channel nine, right?

Jeff Rochlin:

Um, and I wanted to understand all of that right up to the, the

Jeff Rochlin:

transmitter, because I'm crazy that way.

Jeff Rochlin:

Um, and what you Yeah, exactly right.

Jeff Rochlin:

Um, and what, what you find as you become that interested in the business

Jeff Rochlin:

also is that the people who are running the business part of the business, they

Jeff Rochlin:

don't know anything about technology.

Jeff Rochlin:

It could be a technology company.

Jeff Rochlin:

The guy who's running the business isn't thinking about

Jeff Rochlin:

how the engine actually operates.

Jeff Rochlin:

He just wants the engine moving fast.

W. Curtis Preston:

It's kind of like, kind of like it in the earlier

W. Curtis Preston:

part, you know, when I was saying I, I just want my technology to work.

W. Curtis Preston:

That's the way they are with tech.

Jeff Rochlin:

exactly.

Jeff Rochlin:

Exactly.

Jeff Rochlin:

And they, and they don't really care about the details.

Jeff Rochlin:

It's not important to them.

Jeff Rochlin:

What they care about is they're gonna get what they expect on the outside

Jeff Rochlin:

of the business right now because it people being who we are, think

Jeff Rochlin:

that the technology is the do all end, all of what makes the business

Jeff Rochlin:

work, especially in big corporations.

Jeff Rochlin:

IT departments tend to do it for the sake of doing it.

Jeff Rochlin:

Right.

Jeff Rochlin:

They make systems for everybody else to use that are complicated,

Jeff Rochlin:

uh, because they work really well inside the machine, right?

Jeff Rochlin:

They take advantage of faster databases, faster storage, so

Jeff Rochlin:

that things on the back end, which are complicated work really well.

Jeff Rochlin:

But then when you and I have to go, you know, fill out our payroll record, you get

Jeff Rochlin:

this screen that looks like it's something from 1954 off of an IBM mainframe, right?

Jeff Rochlin:

In a world where.

W. Curtis Preston:

possibly is

Jeff Rochlin:

Uh, true, but, but you're in a world where everything is,

Jeff Rochlin:

um, you know, graphical interfaces.

Jeff Rochlin:

There's no reason why it has to be that way, and that's why I've always

Jeff Rochlin:

taken it, that you have to understand what the business is doing, and you

Jeff Rochlin:

have to understand what the people in the business are expecting the

Jeff Rochlin:

business to do in order to be able to deliver to them effective systems.

W. Curtis Preston:

I, I've always had that similar opinion and, and it could

W. Curtis Preston:

be because my very first technology job when I was at MBNA, the bank that

W. Curtis Preston:

everybody of a certain level had to spend four hours a month as a customer

W. Curtis Preston:

support rep talking to cardholders,

Jeff Rochlin:

that's cool.

Jeff Rochlin:

That's really great.

W. Curtis Preston:

and, and it really gave you a sense of like,

W. Curtis Preston:

this is what we were about.

W. Curtis Preston:

Right?

W. Curtis Preston:

We are about the cardholder out there spending money that we are lending

W. Curtis Preston:

them because that's how a credit card company is judged, is the outstanding

W. Curtis Preston:

balance of everybody added all up, which in that world was 35 billion.

W. Curtis Preston:

Right?

W. Curtis Preston:

Which is just staggering to think about it.

Jeff Rochlin:

the, the old adage, the customer is always right, is absolutely

Jeff Rochlin:

true all of the time, even when they're wrong and there are pains in your butt.

Jeff Rochlin:

Um, because if you're not going in with that attitude, Then people are gonna

Jeff Rochlin:

say, Well, especially nowadays, right?

Jeff Rochlin:

When there's a big in different choices, they're gonna say, Well, screw this.

Jeff Rochlin:

I'll just go somewhere else.

Jeff Rochlin:

I don't need,

W. Curtis Preston:

We talk about building a framework and building review

W. Curtis Preston:

and advisory boards in that chapter.

W. Curtis Preston:

What, what is, what is that?

Jeff Rochlin:

First of all, um, think about.

Jeff Rochlin:

A pilot flying an airplane, right?

Jeff Rochlin:

He's got a checklist that he goes through to make sure that he's

Jeff Rochlin:

checked all the important critical things about the system, right?

Jeff Rochlin:

And the reason why he has it written on a checklist is because those are

Jeff Rochlin:

things you don't wanna forget,

Jeff Rochlin:

So that's why you build a framework with anything that's important, right?

Jeff Rochlin:

You wanna make sure you capture all of the things you need to understand

Jeff Rochlin:

and have them on a list that you can check off so that you don't find out

Jeff Rochlin:

after you've built the building that, oh, you forgot to dig the foundation.

Jeff Rochlin:

Um, and, and it just assures that, you know, it's, you build things, right?

Jeff Rochlin:

So it's, it's, it's that same concept of you measure twice and you cut once, right?

Jeff Rochlin:

You don't wanna have to go back and have to redo it all again afterwards.

Jeff Rochlin:

Um,

Prasanna Malaiyandi:

question I have for you, Jeff, is I'm guessing the

Prasanna Malaiyandi:

checklist is kind of what the, like you said, the pilot uses, right?

Prasanna Malaiyandi:

In a term that, in this case, it would be like the backup admin would use

Prasanna Malaiyandi:

and they would have their checklist in terms of, Okay, these are the things

Prasanna Malaiyandi:

I need to make sure that I cover.

Prasanna Malaiyandi:

But I, and I know we'll talk about this in a minute, but like that's

Prasanna Malaiyandi:

not necessarily though exactly what you're going to be like word for word.

Prasanna Malaiyandi:

You're not gonna be asking like the business, okay, what is your rto, Right?

Prasanna Malaiyandi:

Because I'm guessing that like what you understand, right?

Prasanna Malaiyandi:

You need some translation between like what's on that checklist

Prasanna Malaiyandi:

into what you're asking the

Jeff Rochlin:

When you're out collecting requirements for what your system

Jeff Rochlin:

needs to be able to do, you may have to translate the question, right?

Jeff Rochlin:

You may not go up to the, the CEO of the company and say, What,

Jeff Rochlin:

what rto are you looking for?

Jeff Rochlin:

But you may go up to and say, So the building burns down tomorrow.

Jeff Rochlin:

At what point does it become no longer effective for you to reopen the business.

Jeff Rochlin:

At what point is it bankruptcy and we're gone versus we can

Jeff Rochlin:

survive this and move forward.

Jeff Rochlin:

Right?

Jeff Rochlin:

And again, that's extreme, but it's translating the question and, and a

Jeff Rochlin:

checklist is like collecting, uh, the, uh, the requirements for any project, right?

Jeff Rochlin:

It's.

W. Curtis Preston:

Right.

W. Curtis Preston:

The context that I remember in the chapter was we talked about that

W. Curtis Preston:

process that we, that we had at that company where you and I worked.

W. Curtis Preston:

Um, that was, it was a very rigid process for developing a new product.

W. Curtis Preston:

And it was, you know, there was the,

Jeff Rochlin:

we were working together in aerospace and you have to remember

Jeff Rochlin:

that aerospace is one of those businesses where if you miss a step

Jeff Rochlin:

in the process, things go bad, right?

Jeff Rochlin:

Everything has to be measured and, and has to be checked

Jeff Rochlin:

off on to make sure it's safe.

Jeff Rochlin:

, Um, they took a lot of that and they've translated that down into their IT

Jeff Rochlin:

processes, which by the way, are the same things that they give you in ITIL right?

Jeff Rochlin:

If you're an IT person, which is a 34 letter word in most certainly

Jeff Rochlin:

creative companies to drop it l um, it teaches you the idea of

Jeff Rochlin:

things have to be structured right?

Jeff Rochlin:

There's a, there's a right way to do things and you just need

Jeff Rochlin:

to try to follow the right.

Jeff Rochlin:

In the case of the aerospace company, part of that process is make sure that to

Jeff Rochlin:

keep you on track, check in with people all the time, and make sure that they

Jeff Rochlin:

agreed that we're still on track, right?

Jeff Rochlin:

I, I spend a month collecting, um, the project plan and what the

Jeff Rochlin:

requirements are, and then I send it out the stakeholders that are

Jeff Rochlin:

gonna need to use this later and say, Just tell me did I get it right?

Jeff Rochlin:

Is this what you wanted?

Jeff Rochlin:

And they come back and go, Yay.

Jeff Rochlin:

Or they come back and go, No.

Jeff Rochlin:

And then you.

W. Curtis Preston:

Right.

Jeff Rochlin:

you check in and you do the same thing, right?

Jeff Rochlin:

You come back and say, Okay, we're moving along this way, and is this still okay?

Jeff Rochlin:

Um, or did your plans change?

Prasanna Malaiyandi:

So totally makes sense having this board that

Prasanna Malaiyandi:

you go check back with, right?

Prasanna Malaiyandi:

Figuring out process.

Prasanna Malaiyandi:

What are some examples of, uh, requirements or

Prasanna Malaiyandi:

questions you would ask Right.

Prasanna Malaiyandi:

For specifically around backup, right.

Prasanna Malaiyandi:

As you're going through this process.

Jeff Rochlin:

yeah.

Jeff Rochlin:

Um, the very first question I always ask, and it's the hardest question

Jeff Rochlin:

for any business to be able to answer, is, what is acceptable downtime?

Jeff Rochlin:

Right.

Jeff Rochlin:

If, if, if something happens, we can't control it, what can you live with?

Jeff Rochlin:

Right?

Jeff Rochlin:

I realize it's always gonna be zero.

Jeff Rochlin:

. But now dig down deep in your heart cuz it's gonna happen and

Jeff Rochlin:

tell me what's acceptable on that.

Jeff Rochlin:

Right?

Jeff Rochlin:

Um, and remember that I can give you as close to zero downtime as humanly

Jeff Rochlin:

possible, but it's gonna cost you a lot

Prasanna Malaiyandi:

That that's, that's exactly what I was gonna ask.

Prasanna Malaiyandi:

It's like, do you then tell them, Hey, by the way, you won zero downtime,

Prasanna Malaiyandi:

it's gonna cost you 5 billion.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

Or something like that.

Prasanna Malaiyandi:

And

Jeff Rochlin:

So for me, you always present it in multiple stages, right?

Jeff Rochlin:

There's the, this is, this is the worst.

Jeff Rochlin:

You don't want to do this.

Jeff Rochlin:

This is survival mode, how much it'll cost.

Jeff Rochlin:

This is.

Jeff Rochlin:

This is like gold plated, the greatest it'll ever be in the world.

Jeff Rochlin:

Once you get this, you'll be the happiest person ever.

Jeff Rochlin:

It's gonna cost you this much.

Jeff Rochlin:

And then this is probably the answer you want.

Jeff Rochlin:

This is the middle, this is the price, right?

Jeff Rochlin:

Um, but you always go back to the business and the business

Jeff Rochlin:

has to decide how critical is it?

Jeff Rochlin:

And it, and it could be for this piece of the business, right?

Jeff Rochlin:

For my, for my call center, I need a hundred percent uptime because

Jeff Rochlin:

worst case, they can write the order down on paper and we can key 'em in.

Jeff Rochlin:

Um, whereas, you know, for the database service, if it's down for an hour,

Jeff Rochlin:

it's down for two hours, maybe that's okay, as long as I can recover it to

Jeff Rochlin:

within a, a certain period of time,

Prasanna Malaiyandi:

And would you say that that's probably

Prasanna Malaiyandi:

the most important question?

Prasanna Malaiyandi:

To always start with

Jeff Rochlin:

Yes.

Jeff Rochlin:

I, I believe it is because, um, it drives everything else right.

W. Curtis Preston:

The thing is downtime is sort of measured in two D.

W. Curtis Preston:

Ways, Right?

W. Curtis Preston:

So there's downtime in terms of like how long the system is down.

W. Curtis Preston:

There's also downtime in terms of how much data we lost, right?

W. Curtis Preston:

So we translate that into RTO and rpo, which I, I know we already did a, uh,

W. Curtis Preston:

a podcast back to basics podcast on, Um, so that, that's the big thing is,

W. Curtis Preston:

is making sure that whatever it is that we're getting ready to design that.

W. Curtis Preston:

We're designing it based on the requirements that business

W. Curtis Preston:

came, you know, they gave us

W. Curtis Preston:

. Jeff Rochlin: But that's why you have

W. Curtis Preston:

you go back and you feed it back to them every once in a while and say,

W. Curtis Preston:

Right, this is what you asked for.

W. Curtis Preston:

Correct.

W. Curtis Preston:

Cause this is what we're gonna build.

W. Curtis Preston:

Right.

W. Curtis Preston:

We're good.

W. Curtis Preston:

And you do it multiple times so that at the end of the day, if something,

W. Curtis Preston:

Assuming you did everything right and the system performs the way you

W. Curtis Preston:

expected to perform, and they come back and go, What the hell, man?

W. Curtis Preston:

You can look at them and go, Six times you came back and said this was acceptable,

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Six times.

W. Curtis Preston:

you and I both agreed that a six hour RTO was acceptable because a two

W. Curtis Preston:

hour RTO was gonna cost us 3 billion.

W. Curtis Preston:

You and I agree, remember that I have it in writing here.

W. Curtis Preston:

And so the, the fact that this, you know, that we met a six hour RTO is good

Jeff Rochlin:

yeah,

Prasanna Malaiyandi:

and it's documented.

Jeff Rochlin:

yeah, yeah, exactly.

Jeff Rochlin:

It's, it's, you know, there's a, there's some element of you're

Jeff Rochlin:

covering your own ass when you're doing it right, But we all have to.

Jeff Rochlin:

Be sure that we're delivering what the business needs.

Jeff Rochlin:

And that's why you have to understand what the business does because if

Jeff Rochlin:

you understand what the business does and somebody comes back and you know,

Jeff Rochlin:

you're in a business where you're talking to customers constantly and

Jeff Rochlin:

somebody comes back when you ask them.

Jeff Rochlin:

Um, what's acceptable and they go, ah, if we're down for a day or two, that's fine.

Jeff Rochlin:

You know, they're wrong.

Jeff Rochlin:

You know that they're not telling you the truth.

Jeff Rochlin:

You know that they're trying to give you the easy answer.

Jeff Rochlin:

And if you understand what the business does, you can look back

Jeff Rochlin:

at them and say, But wait a second.

Jeff Rochlin:

We answer calls 24 hours a day from customers.

Jeff Rochlin:

You're okay with not answering calls 24 hours a day for those customers?

Jeff Rochlin:

Uh, at which point they go, Oh, no, no, I don't mean that

W. Curtis Preston:

I, I did, I did work with a business once,

W. Curtis Preston:

Jeff, that had a two week rto.

W. Curtis Preston:

You know what, You know what it was?

W. Curtis Preston:

They, they were a paper mill.

W. Curtis Preston:

I.

W. Curtis Preston:

And they were like, If the computer sit down for a couple of weeks, we don't care.

W. Curtis Preston:

Like we're, we're gonna keep cutting down trees.

W. Curtis Preston:

We're gonna keep turning it into pulp and we're gonna keep making paper.

W. Curtis Preston:

If, if you can't, if you can't bill anybody, uh, what

W. Curtis Preston:

They'll still need paper.

W. Curtis Preston:

We'll be, that was their attitude.

W. Curtis Preston:

And so for them, a two week RTO was fine, you know?

W. Curtis Preston:

Uh, but yeah.

W. Curtis Preston:

But it goes back to the first thing we started with it, it goes with

W. Curtis Preston:

what kind of business you are.

W. Curtis Preston:

Right.

W. Curtis Preston:

Most businesses, you start using the word day

Jeff Rochlin:

They're,

W. Curtis Preston:

in your rto.

W. Curtis Preston:

They're not gonna be okay with that.

Jeff Rochlin:

If you're in a retail business, you know, and you're down,

Jeff Rochlin:

you know, you're, you're running a big, uh, online catalog service and you've

Jeff Rochlin:

got call centers all around the world taking calls from people or websites that

Jeff Rochlin:

are, you know, taking, uh, selling movie tickets, let's say, which I have a certain

Jeff Rochlin:

amount of experience within the past.

Jeff Rochlin:

Right.

Jeff Rochlin:

And that opening weekend for the next Star Wars movie is coming up and

Jeff Rochlin:

your systems go down for 10 hours.

Jeff Rochlin:

You.

Jeff Rochlin:

Yeah, you're in trouble, right?

Jeff Rochlin:

Cause you know that that money is going to go somewhere else.

Jeff Rochlin:

It's not like they're gonna sit there and go, Well, I'll

Jeff Rochlin:

keep hitting return until I can

W. Curtis Preston:

yeah, it's not like they're gonna not

W. Curtis Preston:

buy tickets for that movie.

W. Curtis Preston:

Right.

W. Curtis Preston:

They're gonna, they're gonna do that.

W. Curtis Preston:

Um, gosh.

W. Curtis Preston:

You remember when we used to stand in line, Jeff

Jeff Rochlin:

Yeah, I do.

Jeff Rochlin:

You know, there was, there was a certain charm to it, but

W. Curtis Preston:

Yeah, I remember standing in line, like, you stood

W. Curtis Preston:

in line to stand in line, right?

W. Curtis Preston:

You, you got up there, there's a big line and you're like, you, you know, it's one

W. Curtis Preston:

o'clock, you want the four o'clock show?

W. Curtis Preston:

And they're like, Oh, the four o'clock show already sold out.

W. Curtis Preston:

Okay, so I guess we're doing the seven o'clock show and uh, and

W. Curtis Preston:

you bought tickets for the seven o'clock show at one o'clock, right?

W. Curtis Preston:

And then you went and did something with yourself for the next six hours.

Jeff Rochlin:

Yeah.

W. Curtis Preston:

Anyway, Sorry, two old guys remeniscing about back in the day.

W. Curtis Preston:

Um, so you, Yeah, you, you, Jeff, in the next section, in the chapter, you

W. Curtis Preston:

talked about drawing up multiple designs.

W. Curtis Preston:

I think that you hinted at that already about the, here's the

W. Curtis Preston:

one you definitely don't want.

W. Curtis Preston:

Here's the one you're probably not gonna pay for.

W. Curtis Preston:

And here, you know, here's the Goldilocks zone.

Jeff Rochlin:

People want to feel like they have choices.

Jeff Rochlin:

It's part of just the way life is, right, And everybody wants gold plated, but

Jeff Rochlin:

they don't want to pay for gold plated.

Jeff Rochlin:

So the best way to show them is there aren't, you know, I'm not narrow minded.

Jeff Rochlin:

There are multiple ways you can do this.

Jeff Rochlin:

You can solve this problem.

Jeff Rochlin:

And at different cost levels, and I'm gonna leave it to you to decide how much

Jeff Rochlin:

money you wanna spend based on that.

Jeff Rochlin:

Right.

Jeff Rochlin:

Um, and then just from experience, uh, I know that you present three layers

Jeff Rochlin:

because there, you know, it's, there's a little psychology involved in it.

Jeff Rochlin:

It's like when you write a budget, there's always something in the budget

Jeff Rochlin:

that you're willing to cut because you know they're gonna come back no

Jeff Rochlin:

matter what, and they're gonna ask you to cut something out of the budget.

Jeff Rochlin:

It's almost like a goodwill kind of thing, right?

Prasanna Malaiyandi:

So far we've kind of been talking about like the requirements

Prasanna Malaiyandi:

are a little bit in stone, right?

Prasanna Malaiyandi:

Sort of, uh, the business comes, gives you the requirements, you

Prasanna Malaiyandi:

come back with the proposals, right?

Prasanna Malaiyandi:

But as we all know, like business needs and requirements change, right?

Prasanna Malaiyandi:

And.

Prasanna Malaiyandi:

How do you sort of account for those changes as you're like,

Prasanna Malaiyandi:

so you had these three options.

Prasanna Malaiyandi:

They picked the middle option, business requirements change.

Prasanna Malaiyandi:

Right?

Prasanna Malaiyandi:

How do you sort of incorporate that?

Prasanna Malaiyandi:

Because you can't be like, Hey, by the way, we already started on our way

Prasanna Malaiyandi:

and so now you have to wait 12 months before we can incorporate those in.

Jeff Rochlin:

We're agile now, right?

Jeff Rochlin:

We don't actually.

Jeff Rochlin:

We don't build projects that we set a two-year goal and then we check

Jeff Rochlin:

in at the end of year two and say, Here's what we've got to find out.

Jeff Rochlin:

We got it wrong.

Jeff Rochlin:

The biggest challenge in that space to me is when you're at

Jeff Rochlin:

the foundation level, right?

Jeff Rochlin:

It's like if I'm building a house and we've designed the house and we've

Jeff Rochlin:

decided the bathroom's gonna go here, the kitchen's gonna go there, right?

Jeff Rochlin:

I'm running pipes now out of the foundation to go to those places.

Jeff Rochlin:

You can't come back to.

Jeff Rochlin:

Three weeks later and say, I've decided to put the bathroom at the totally

Jeff Rochlin:

opposite end of the house cuz I've already poured the concrete and that

Jeff Rochlin:

pipe is not gonna move on its own.

Prasanna Malaiyandi:

Yeah, well, Or well not move on its own.

Prasanna Malaiyandi:

And I guess that's the thing, right?

Prasanna Malaiyandi:

Is it's okay to go back and say, Yeah, this requires sort of like a redesign

Prasanna Malaiyandi:

or a rethought re-implementation, right?

Prasanna Malaiyandi:

It's gonna affect your timelines for sure, but if that's what

Prasanna Malaiyandi:

the business needs, we could.

Jeff Rochlin:

So I think it, I, I think the best way you deal with this kind

Jeff Rochlin:

of thing is in the ordering of how your project is established, right?

Jeff Rochlin:

You've gotta get the core, the solid infrastructure that isn't

Jeff Rochlin:

going to change, um, done first.

Jeff Rochlin:

And you have to make sure the buy off is there and you have to keep

Jeff Rochlin:

going back and saying, Right, we're going to build this network.

Jeff Rochlin:

We're going to buy this kind of storage, We're going to buy these kinds of tape.

Jeff Rochlin:

We can go back and change out systems and how they operate.

Jeff Rochlin:

We can change schedules, we can add, but if you turn around after

Jeff Rochlin:

that check is written and say, I don't want to use that anymore.

Jeff Rochlin:

I can't get you that money back, right?

Jeff Rochlin:

You've spent it.

Jeff Rochlin:

So you need to be sure your foundation is solid in what you're trying to

Jeff Rochlin:

build on, and you need to know that.

Jeff Rochlin:

Um, features can change, right?

Jeff Rochlin:

But the core foundation of it has to be once you've started laying it down.

Prasanna Malaiyandi:

Yeah.

Prasanna Malaiyandi:

And I, I guess Curtis, this would probably be your.

Prasanna Malaiyandi:

Pitch that you would say.

Prasanna Malaiyandi:

It's like, Hey.

Prasanna Malaiyandi:

Yeah, but with the SaaS like service, you may not need to put

Prasanna Malaiyandi:

all that money up front, right.

Prasanna Malaiyandi:

To pay for something.

Prasanna Malaiyandi:

Right.

Prasanna Malaiyandi:

It does give you some amount of flexibility to be like,

W. Curtis Preston:

That that dynamic scaling up and down is definitely a

W. Curtis Preston:

real value of a SaaS product, but like any other product you, you have to take

W. Curtis Preston:

into account a little bit of the future.

W. Curtis Preston:

Right.

W. Curtis Preston:

And you know, the one, the one challenge, I'm gonna say, a backing up to a SaaS

W. Curtis Preston:

provider is to just make sure that we've, we've calculated in the laws of physics,

Jeff Rochlin:

Mm-hmm.

W. Curtis Preston:

Because if you, if you've got, if you've

W. Curtis Preston:

got 10, you know, petabyte and a T1 line, it's not gonna work.

W. Curtis Preston:

Right?

W. Curtis Preston:

So as long as we've ca love, as we're not at the upper edge of your bandwidth

W. Curtis Preston:

utilization, you should be able to grow, uh, significantly on the back end without

W. Curtis Preston:

having to have a change on the, on the,

Jeff Rochlin:

But again, but again, that's the foundation, right?

Jeff Rochlin:

That's the, that's the core infrastructure that you have to agree on up front.

Jeff Rochlin:

How much data do I have?

Jeff Rochlin:

How fast do I need to get it up to that damn service in the cloud?

Jeff Rochlin:

And do I have enough bandwidth and capability to do that?

Jeff Rochlin:

Right?

Jeff Rochlin:

Um, and those are the things that become immutable in the beginning.

Jeff Rochlin:

You lay 'em down.

Jeff Rochlin:

You understand them, and then you can build on top of it whatever

Jeff Rochlin:

you want because you have a foundation that'll support it.

Jeff Rochlin:

You can't turn around though afterwards and say, Well, I didn't really have

Jeff Rochlin:

that much network bandwidth, and it turns out that in order to get that much

Jeff Rochlin:

network bandwidth, I have to have this, pay this guy over here to dig a tunnel

Jeff Rochlin:

underneath, you know, the, the biggest avenue in town to run a fiber optics to my

Jeff Rochlin:

building, and I'm not gonna pay for that.

Jeff Rochlin:

So I guess we'll have to figure out how to do it without the bandwidth.

Jeff Rochlin:

Right?

Jeff Rochlin:

That's when it, that's when the tower collapses and none of it's ever.

W. Curtis Preston:

Yeah, so I, I'd say that, So we, so

W. Curtis Preston:

we, we got the requirements.

W. Curtis Preston:

We've got, um, and we, we've, we've done this iterative process of going

W. Curtis Preston:

back to the stakeholders to make sure we understand the requirements.

W. Curtis Preston:

We've built out theoretical system.

W. Curtis Preston:

The, the next phase that I think is, is really important is to make sure.

W. Curtis Preston:

The product that we, that we're supposed to be buying does what it

W. Curtis Preston:

is that it was supposed to do, right?

W. Curtis Preston:

So we're talking about things like POCs and pilots.

Jeff Rochlin:

I, I do the POC early, right.

Jeff Rochlin:

I do it like before I even, I go back to, if the business comes back to me and

Jeff Rochlin:

says we need to update backups, I go out and start doing the POC on the backup of

Jeff Rochlin:

products before I come back and propose here's what I think the solution would be.

Jeff Rochlin:

Right.

Jeff Rochlin:

Um, you know, There are the sales guys and there are the sales engineers, right?

Jeff Rochlin:

Don't listen to anything the sales guys have to say to you.

Jeff Rochlin:

Cause the sales guys are sales guys.

Jeff Rochlin:

That's what they do.

Jeff Rochlin:

Let them buy you lunch and then don't worry about it.

Jeff Rochlin:

Find yourself a good sales engineer who's attached to the account.

Jeff Rochlin:

Connect up with them.

Jeff Rochlin:

They'll give you this straight answer, and then give them your real

Jeff Rochlin:

world, your real world scenario, that's kind of where I come from.

Jeff Rochlin:

Right.

Jeff Rochlin:

If you're gonna, if you're gonna go to the trouble of selling me a service

Jeff Rochlin:

up in the cloud somewhere, then it should do everything it needs to do.

Jeff Rochlin:

Right.

Jeff Rochlin:

And.

Jeff Rochlin:

Go on a tangent, which I know you'll cut cuz you hate when I go on tangent.

Jeff Rochlin:

So I'm keeping you from doing other

W. Curtis Preston:

It depends on the value of the tangent, Jeff

Jeff Rochlin:

um.

Jeff Rochlin:

When, when people, especially the non-technical people go off

Jeff Rochlin:

and think of the cloud, right?

Jeff Rochlin:

They think the cloud is a solution for everything, right?

Jeff Rochlin:

I want to go store things in the cloud, so I will just copy all of my

Jeff Rochlin:

data up into the cloud and things will be wonderful, and it's like, No, you

Jeff Rochlin:

copy your stuff up into the cloud and all you're doing is you're handing it

Jeff Rochlin:

off to somebody else's data center.

Jeff Rochlin:

That doesn't mean it's properly backed up.

Jeff Rochlin:

That doesn't mean it's redundantly stored in multiple locations

Jeff Rochlin:

in case there's a disaster.

Jeff Rochlin:

You still need a systems engineer to look at the problem and come up

Jeff Rochlin:

with a solution within the context of the cloud as if it was your own.

Jeff Rochlin:

But, but cloud is a marketing term, right?

Jeff Rochlin:

That we sell to people who don't understand technology about a,

Jeff Rochlin:

here the solution is there, baby.

Jeff Rochlin:

Just buy this and, and, and your life will be better and everything will just

W. Curtis Preston:

It, You know what you need, Jeff, In that

W. Curtis Preston:

scenario, you, you need a Prasanna.

W. Curtis Preston:

That is what Prasanna does.

W. Curtis Preston:

So, you know, he's actually, he's actually, you know, I'm gonna say he's

W. Curtis Preston:

more technical than I am, and when we talk about current cloud technology,

W. Curtis Preston:

uh, he's actually making things happen and actually understands that stuff.

W. Curtis Preston:

So you need somebody like that.

W. Curtis Preston:

Like, it's not, like the cloud doesn't just like, Oh,

W. Curtis Preston:

we don't need any IT people,

Prasanna Malaiyandi:

Okay.

W. Curtis Preston:

We just need different kinds of IT people.

Jeff Rochlin:

Right.

Jeff Rochlin:

But you would be surprised at the number of C level and especially

Jeff Rochlin:

financial people out there who think.

Jeff Rochlin:

By buying into a cloud service, they can throw away their data center and they

Jeff Rochlin:

no longer need as big an IT department

W. Curtis Preston:

Yeah.

Jeff Rochlin:

there.

W. Curtis Preston:

might, might

W. Curtis Preston:

need as big, but Yeah.

Jeff Rochlin:

Yeah.

Jeff Rochlin:

you're gonna eliminate certain positions, right?

Jeff Rochlin:

You're gonna eliminate the guys that are handling your hardware.

Jeff Rochlin:

But, um, there, there is this perception and that's why everybody

Jeff Rochlin:

thinks the cloud is cheaper, right?

Jeff Rochlin:

In many cases it's more expensive than running your own data

Jeff Rochlin:

center depending on what you're

W. Curtis Preston:

It, It, definitely depends on how you implemented it.

W. Curtis Preston:

Yeah.

Jeff Rochlin:

Right.

Jeff Rochlin:

But,

Prasanna Malaiyandi:

talk to Corey Quinn or follow him on Twitter, right?

Prasanna Malaiyandi:

That's his big thing.

Jeff Rochlin:

yeah,

W. Curtis Preston:

Yeah.

Jeff Rochlin:

so anyway, that was, that was my tangent, but I thought it might

W. Curtis Preston:

That was a good tangent, Jeff.

W. Curtis Preston:

Um, alright, so the, so the final thing here of, of, you know, of, of,

W. Curtis Preston:

in this chapter was about documenting and implementing the new system, right?

Jeff Rochlin:

Yep.

Jeff Rochlin:

The job, the job that nobody ever wants to have to do is documentation.

W. Curtis Preston:

yeah.

W. Curtis Preston:

Um, you know what,

Jeff Rochlin:

You,

W. Curtis Preston:

that about?

W. Curtis Preston:

Right?

W. Curtis Preston:

Do do I have to, do I have to document?

Jeff Rochlin:

Yeah.

Jeff Rochlin:

You have to.

Jeff Rochlin:

Sorry.

Jeff Rochlin:

You know, cuz it's like that's your proof of life, right?

Jeff Rochlin:

You have, you have it down in writing, you have a plan that you're working from.

Jeff Rochlin:

It's the checklist, right?

Jeff Rochlin:

And it's visible to everybody.

Jeff Rochlin:

You can hand a copy to somebody and say, I want you to read this, and then

Jeff Rochlin:

I want you to sign on the bottom of it so that I know that you've read

Jeff Rochlin:

it and you either agree or you don't.

Jeff Rochlin:

And then if they take it and they just sign it and hand it back

Jeff Rochlin:

to you when there's something in there that they didn't like or

Jeff Rochlin:

that they didn't expect, you look at them and go, You signed for it.

Jeff Rochlin:

Right?

Jeff Rochlin:

Would you sign a contract without reading it?

Jeff Rochlin:

Would you, You know,

Prasanna Malaiyandi:

all the time.

Prasanna Malaiyandi:

I agree.

Jeff Rochlin:

Written.

Jeff Rochlin:

The written documentation, especially the design documentation is actually kind of

Jeff Rochlin:

like a contract with your customer, right?

Jeff Rochlin:

Your customer being the business.

W. Curtis Preston:

right.

Prasanna Malaiyandi:

this design document, right?

Prasanna Malaiyandi:

It, I'm guessing it also sort of restates, Hey, here are all the

Prasanna Malaiyandi:

requirements that I built against, Right.

Prasanna Malaiyandi:

That we had agreed to during the sort of the checklist and

Prasanna Malaiyandi:

those initial discussions.

Prasanna Malaiyandi:

Right?

Prasanna Malaiyandi:

And these are sort of the requirements that this design satisfies.

Jeff Rochlin:

Yeah.

Jeff Rochlin:

When you, when you write that initial document once, once everybody has

Jeff Rochlin:

blessed it and signed off on it, you may never go back to it again.

Jeff Rochlin:

Right?

Jeff Rochlin:

You may never have to go back to the printed document, except for the

Jeff Rochlin:

fact that it's part of your completed set of things you need to have.

Jeff Rochlin:

That's just there for the future.

Jeff Rochlin:

Right?

Jeff Rochlin:

And if you've ever been involved in any kind of construction, right on the back

Jeff Rochlin:

end of the construction, there are the as-builts, right there is the actual

Jeff Rochlin:

design document in the blueprints of the building as it was completed by

Jeff Rochlin:

the contractors who did all the work that you stick in a box somewhere and

Jeff Rochlin:

only go back to when you're trying to figure out where that damn wire that's

Jeff Rochlin:

powering your wifi, uh, device is running.

Jeff Rochlin:

Because that's when you need your as-builts, right?

Jeff Rochlin:

You need to understand how things are wired.

Jeff Rochlin:

This becomes the as-built for the project.

Jeff Rochlin:

It gives you esoteric details that you really don't care about until

Jeff Rochlin:

for some reason you need to know about it, and then it's there.

W. Curtis Preston:

When I, when I think about the value of documentation, I'm

W. Curtis Preston:

speaking in this case more operational documentation than design documentation,

W. Curtis Preston:

but I go all the way back to when my oldest daughter was born, right?

W. Curtis Preston:

I was in Delaware, uh, and uh, I was at the hospital in my wife's delivery room.

W. Curtis Preston:

Nina was born, She's, you know, brand new baby, and I'm, I'm standing in, you

W. Curtis Preston:

know, in New Father Bliss Zone, right.

W. Curtis Preston:

And the phone rings in my wife's hospital room and you know, this

W. Curtis Preston:

is pre-cellphone days, right?

W. Curtis Preston:

And, um, I, um, I picked up the phone.

W. Curtis Preston:

I'm like, Why is this phone, It's the bank on the phone, and I.

W. Curtis Preston:

What, And they said, Well, well we have this restore, uh, that we need to do.

W. Curtis Preston:

I'm like, I am in my wife's hospital room.

W. Curtis Preston:

And they're like, Well, we don't understand abc.

W. Curtis Preston:

And I said, Well,

Jeff Rochlin:

Yeah.

W. Curtis Preston:

did you read the documentation?

W. Curtis Preston:

Is there a problem with the documentation?

W. Curtis Preston:

They said, Oh, we haven't looked at it.

W. Curtis Preston:

Slam.

W. Curtis Preston:

I just slammed down the phone, , I.

W. Curtis Preston:

The, I had good documentation and that was, you know, I, I was like, I'm good.

Jeff Rochlin:

I, I regularly go to developers when they're writing software

Jeff Rochlin:

that I'm involved in having to support and telling them that, um, because

Jeff Rochlin:

I run operations as well, right?

Jeff Rochlin:

And telling them that, um, it is in there enlightened self.

Jeff Rochlin:

To write good documentation for their product because I will call them at

Jeff Rochlin:

two o'clock in the morning on Saturday.

Jeff Rochlin:

If it breaks and there's something broken, not only will I call you, I'm

Jeff Rochlin:

gonna make sure I have your wife's cell phone number as well, and I

Jeff Rochlin:

will call her if you choose not to

W. Curtis Preston:

I call your mom

Jeff Rochlin:

Cause if the, I would, Cause if the business is flat out, down

Jeff Rochlin:

on its ass and I have the CEO of the company who's going to come looking for

Jeff Rochlin:

me because I'm responsible for operations.

Jeff Rochlin:

What am I gonna tell 'em?

Jeff Rochlin:

Sorry, the software doesn't work.

Jeff Rochlin:

We'll get back to you on Monday.

Jeff Rochlin:

Right?

Jeff Rochlin:

It's like, no, it doesn't work that way.

Jeff Rochlin:

Um, and that's why everything, the better the documentation is, the

Jeff Rochlin:

less the operations guys have to go and bother the people who wrote it.

Prasanna Malaiyandi:

It just going back to the earlier thread that

Prasanna Malaiyandi:

I was on, how do you ensure that cuz things will change, right?

Prasanna Malaiyandi:

The system will change.

Prasanna Malaiyandi:

The designs will change.

Prasanna Malaiyandi:

Is that part of the process that any time like you get new

Prasanna Malaiyandi:

requirements or anything else like.

Prasanna Malaiyandi:

Always go back and update the documentation.

Jeff Rochlin:

Yeah.

Jeff Rochlin:

Should be, should be.

Jeff Rochlin:

Without a doubt.

Jeff Rochlin:

And if it's happening during the development process, you take it

Jeff Rochlin:

back to the review boards so that again, everybody's aware of it

Jeff Rochlin:

and everybody checks off on it.

Jeff Rochlin:

Right.

Jeff Rochlin:

That they're okay with the change.

Jeff Rochlin:

That it's not a surprise to finance.

Jeff Rochlin:

Right.

Jeff Rochlin:

And it's not a surprise to, uh, facilities operations that you're,

Jeff Rochlin:

you know, all of a sudden gonna re.

Jeff Rochlin:

Uh, the, the waterlines or something like that to make it work?

W. Curtis Preston:

Yeah, I, I would say it's your job as the technology side

W. Curtis Preston:

of things to watch what's happening with the business side of things and

W. Curtis Preston:

to see if there are changes, right?

W. Curtis Preston:

If all of a sudden to, to use your world.

W. Curtis Preston:

Jeff, you know, you are normally doing 10 movies at a time and you happen

W. Curtis Preston:

to be in some quarterly meeting and you hear that, They're like, Listen,

W. Curtis Preston:

we're gonna double production.

W. Curtis Preston:

We've, you know, we've had some great sales and we're

W. Curtis Preston:

gonna do 20 movies at a time.

W. Curtis Preston:

You're gonna be like, um,

Jeff Rochlin:

Yeah.

W. Curtis Preston:

Right.

W. Curtis Preston:

You know, current capacity is based on 10 movies at a time.

W. Curtis Preston:

I'm gonna need twice as much

Jeff Rochlin:

yeah, I guess examples in my life of that were like, um, I was working

Jeff Rochlin:

at Disney at the time and, uh, I sat in a meeting and they said, Hey, we're gonna,

Jeff Rochlin:

we're gonna release this one in imax.

Jeff Rochlin:

You know, sitting there going, Wait, I have a system that's designed to

Jeff Rochlin:

create two K frames on the back end.

Jeff Rochlin:

I don't even know what IMAX is at this point.

Jeff Rochlin:

Or, Hey, we're gonna start doing 3d.

Jeff Rochlin:

That was like, ah, right.

Jeff Rochlin:

When it comes to things like storage, right, The transition from,

Jeff Rochlin:

we're gonna do two k, we're gonna.

Jeff Rochlin:

4k, we wanna do these special formats.

Jeff Rochlin:

Oh wait, we're gonna start doing live action visual effects.

Jeff Rochlin:

That means we've gotta shoot, you know, real film back plates,

Jeff Rochlin:

uh, that you're gonna put stuff on top

Jeff Rochlin:

of,

W. Curtis Preston:

The point here is that you need to be keeping an

W. Curtis Preston:

eye to going back, sort of bringing this all back to the beginning.

W. Curtis Preston:

Your organization, what is its function, right?

W. Curtis Preston:

This, that idea works for businesses, it works for government, uh, right?

W. Curtis Preston:

So what is the purpose of your organization and what do you, what

W. Curtis Preston:

purpose do you serve within that purpose?

Jeff Rochlin:

Right.

Jeff Rochlin:

I can, I can promise you that the head, the, the, the chief creative

Jeff Rochlin:

officer at Disney is not thinking about the difference in storage capacity

Jeff Rochlin:

between a two K movie and an 8K movie.

W. Curtis Preston:

Yeah, it's not, not their job

Jeff Rochlin:

right.

Jeff Rochlin:

But you have to be listening to him to hear that he just said eight K, and go,

Jeff Rochlin:

Well, okay, I need to go figure out what that means, and then I need to go find

Jeff Rochlin:

the right executive who's willing to go ask him if he's willing to spend money on

Jeff Rochlin:

that or if he was just talking about it.

W. Curtis Preston:

This whole thing was about gathering the requirements,

W. Curtis Preston:

just making sure what your company does, making sure that you are doing

W. Curtis Preston:

the things that you need to do to, to, to meet those requirements.

W. Curtis Preston:

Um, and, and, and that, and that you should absolutely not be

W. Curtis Preston:

determining those requirements.

W. Curtis Preston:

You should not be determining retention.

W. Curtis Preston:

Recovery time, objective, rpo, any of that stuff that should

W. Curtis Preston:

all come back from the business.

Jeff Rochlin:

Yeah, be an active participant in your company, right?

Jeff Rochlin:

It's more, be more at your job than just your job.

Jeff Rochlin:

Understand what your business does, Um, understand why it's doing it,

Jeff Rochlin:

what makes it, you know, special.

Jeff Rochlin:

in whatever it is that it's doing.

Jeff Rochlin:

And when you understand all that stuff, you'll get to know the people

Jeff Rochlin:

in the business better because you know, like I said, the guy in

Jeff Rochlin:

customer service doesn't care about what's going on in the computer room.

Jeff Rochlin:

But if you get to know the guy in customer service because you understand

Jeff Rochlin:

what he's up against, all of a sudden you can build that relationship.

Jeff Rochlin:

That'll make it easier to get the info you need later and then document the hell out

Jeff Rochlin:

of it and make sure everybody signs off.

Jeff Rochlin:

So, Accountability is spread evenly across the whole

Jeff Rochlin:

organization at the end of the day.

Jeff Rochlin:

Cause you don't want to be the guy that everybody points to and

Jeff Rochlin:

says, Well, we're down for a week.

Jeff Rochlin:

Because, you know, he decided that we could be down for a week

Jeff Rochlin:

and go, No, here's the book.

Jeff Rochlin:

Right.

Jeff Rochlin:

And then you document it all through an inch of its life so that a,

Jeff Rochlin:

you have it as a reference, right?

Jeff Rochlin:

Cause other people are going to have to operate it if you don't want to

Jeff Rochlin:

be the only person who gets called at two o'clock in the morning cause they

Jeff Rochlin:

can't figure out how your system works.

Jeff Rochlin:

Then you describe to people how your system works so they can figure it out at

Jeff Rochlin:

two o'clock in the morning without you.

Jeff Rochlin:

Um, and that, um, all of the parameters are covered so that you can now move

Jeff Rochlin:

up to be the president of the company and the next guy coming in behind you

Jeff Rochlin:

will be able to just easily step into.

Jeff Rochlin:

Past role cuz he can just open the books and it's all there for

W. Curtis Preston:

I like it.

W. Curtis Preston:

Well, thanks Jeff.

W. Curtis Preston:

Always, always a treat.

Jeff Rochlin:

My pleasure guys.

Jeff Rochlin:

Thanks for having me.

W. Curtis Preston:

Thanks.

W. Curtis Preston:

Prasanna.

W. Curtis Preston:

I know between Jeff and I, you didn't get a word in edge wise in

Prasanna Malaiyandi:

That's okay.

Prasanna Malaiyandi:

I, I

Jeff Rochlin:

He asks good

Prasanna Malaiyandi:

:

to the old man, old men

Jeff Rochlin:

Hello?

Prasanna Malaiyandi:

By the way, if you wanna listen to the two of

Prasanna Malaiyandi:

them talk more, they do have their, another podcast, right, where you

W. Curtis Preston:

We do, we do

W. Curtis Preston:

the, the things that entertain us Yes.

W. Curtis Preston:

Is the name of our other podcast.

Jeff Rochlin:

May maybe, maybe Curtis will stick a link in in the, uh, show notes.

Jeff Rochlin:

Who

W. Curtis Preston:

maybe, maybe he will, Maybe he will.

W. Curtis Preston:

Uh, all right.

W. Curtis Preston:

And thanks again to our listeners.

W. Curtis Preston:

We're nothing without you.

W. Curtis Preston:

Remember to subscribe so that you can restore it all.

Links

Chapters

Video

More from YouTube