Artwork for podcast FinOpsPod
Jérémy Nancel & John Renne - Optimizations Ideal vs Reality
Episode 14 • 25th October 2022 • FinOpsPod • FinOps Foundation
00:00:00 00:26:35

Share Episode

Shownotes

Episode 14 Jérémy Nancel & John Renne - Optimizations Ideal vs Reality

Rightsizing an under-utilized resource is an easy way to save on cloud costs, right? Or is it? In a conversation taken from the FinOps Foundation Slack channels, Jérémy Nancel & John Renne debate the pros and cons of reducing cloud spend by rightsizing resources vs incurring tech debt via code drift.

Transcripts

Jérémy:

Hi, I'm Jérémy Nancel

John:

Hi, I'm John Renne and this is FinOpsPod.

Stacy:

All right, so let's hold on.

Stacy:

Let's do our thing.

Joe:

Okay.

Joe:

Hi, I'm Joe Daly.

Stacy:

And I'm Stacey Case.

Stacy:

Wait, who says it?

Stacy:

don't.

Stacy:

I usually start

Joe:

this is FinOpsPod.

Stacy:

Wait I start and then you say your name and then you

Stacy:

say, This is FinOpsPod, right?

Joe:

It could be either way.

Joe:

I don't think we have a standardized way of doing it.

Stacy:

Hi, I'm Stacy Case.

Joe:

And I'm Joe Daly.

Stacy:

This is FinOpsPod.

Stacy:

Um, okay, so I'm not gonna lie, I didn't get a chance to listen

Stacy:

to this one yet because you, you were able to get this through a

Stacy:

little bit faster than I thought.

Joe:

but it was made available, was it not?

Stacy:

Yes, Dear listener, Joe made this available to me yesterday, but

Stacy:

I was not able to pre-listen to it, so I will have to listen to it when

Stacy:

it is released with everybody else.

Joe:

And we are in a little bit, You know what, In real life, real

Joe:

world - podcasts don't grow on trees.

Joe:

You gotta make 'em.

Stacy:

What?

Joe:

and there's, you know, just like anything else, we got schedules.

Joe:

We just did our October summit, which was really good.

Joe:

And then next week we're gonna be in Detroit for KubeCon North America.

Stacy:

Yeah I know that, I wanna make this podcast so people can listen

Stacy:

to it later and be like, Why stop talking about what's happening today?

Stacy:

But I am very excited to go KubeCon and see everybody and see the

Stacy:

folks and meet new people, but,

Joe:

it'll be fun.

Joe:

, Here's why it's special, Stacy, because we're gonna, we're gonna meet people and

Joe:

then they're gonna join the foundation, and then I'll be like, We have a podcast.

Joe:

And then they're gonna learn, listen to the podcast, and they're gonna pick

Joe:

this one first, because one, it'll be released, and they'll also be like, Whoa,

Joe:

this topic is really interesting to me.

Joe:

And then they'll be like, Oh my gosh, I so relate.

Joe:

Because they were talking about going to KubeCon and that is where I met them.

Stacy:

It's a full circle moment, folks.

Stacy:

We bring it full circle.

Joe:

And for all of you who aren't at KubeCon Forget you,

Stacy:

No, we still like you.

Stacy:

That is our director of community, everybody.

Stacy:

Director of community

Joe:

All right.

Stacy:

All right.

Stacy:

So talk, talk to me about this one.

Stacy:

Cause I know that the topic for this one came out of a really interesting place.

Joe:

Mm-hmm.

Joe:

Yeah.

Joe:

So.

Joe:

Jérémy Nancel was posting a question in Slack and it got my attention

Joe:

immediately because it was, it's just a real world question.

Joe:

he basically asked the question, Hey, if you're rightsizing resource,

Stacy:

Mm-hmm.

Joe:

At what point do you start worrying about code drift?

Joe:

And if you listen to the FinOoopsPod episode, you know, someone rightsized

Joe:

a resource under my direction and cause the rolling production outage.

Joe:

code drift is when basically the resources don't match the scripts

Joe:

that launch the solution anymore.

Joe:

and it's a great question because that starts to develop a problem in and of

Joe:

itself, which John Ren responded with.

Joe:

In the slack thread as tech debt, and it's true.

Joe:

the more that the solution doesn't recognize the source of truth of the code,

Joe:

the more tech debt you are incurring.

Joe:

But Jérémy makes a great point, is like, yeah, tech debt is not as

Joe:

immediately impactful to my bottom line as this oversized resource is.

Joe:

There is the ideal state, which is you go through and you fix

Joe:

the code and you fix the problem.

Joe:

but there's often a lot more that goes into it.

Joe:

Thus, our favorite answer of whenever anyone asks any FinOps question,

Stacy:

It depends.

Joe:

It depends.

Joe:

FinOps does not happen in a vacuum.

Joe:

That's a t-shirt.

Stacy:

There's a shirt for you,

Joe:

it happens in the real world with lots of different pressures.

Joe:

they had a really great slack thread

Stacy:

So is that why you pulled John into it as well?

Stacy:

John and Jérémy, they don't work together, do they?

Joe:

No, they had never actually talked to each other before.

Joe:

Other than the Slack thread.

Joe:

I just really enjoyed the conversation because this is the debate that if

Joe:

FinOps practitioners aren't having this debate, they need to consider it.

Joe:

because they're both right.

Joe:

Neither of them are wrong in the debate.

Joe:

And they both agree with each other and they both understand each

Joe:

other's point, but they both have to do what's best for their company.

Joe:

Jérémy is a cloud architect at EuroNext in France.

Joe:

And John is a DevOps engineer in the Netherlands working

Joe:

for a company called Bux.

Stacy:

Wait, let me get this right, Joe, John and Jérémy were in a

Stacy:

conversation together talking about this, having never spoken about it

Stacy:

before, and, that's what y'all captured.

Joe:

Yeah, and they really hit it off.

Joe:

And there was actually 10 minutes I removed because they

Joe:

started getting really technical about potential solutions.

Joe:

And I'll be honest, they went way over my head and at the end I

Joe:

think they saw me staring at them and they were like, Perhaps this

Joe:

is too technical for a podcast.

Joe:

I was like, Yeah, I'm gonna remove that part.

Stacy:

So basically reach out to them if you wanna get the technical bits.

Stacy:

You know what is awesome though about this though, is how

Stacy:

this all came about, right?

Stacy:

again, here's folks that are in the community.

Stacy:

It's not someone that you know, that we've had specifically talk anywhere before,

Stacy:

but it's literally just the, this is what you, as our community is going through.

Stacy:

This is the questions that you have, these are the responses that you provide, and

Stacy:

I think that's the whole of this podcast as an extension of the community, is

Stacy:

we're able to amplify those conversations so other folks can learn from 'em.

Stacy:

Right?

Stacy:

Like, you may have missed that slack thread, but hopefully

Stacy:

now listening to this, you're gonna be able to be like, Aha.

Stacy:

Yeah.

Stacy:

Okay.

Stacy:

I got it.

Stacy:

I can relate.

Stacy:

Or, or not, maybe this is not a topic for you, but I think that's just one of

Stacy:

the really awesome things about having this as an extension of the community

, Joe:

I mean, yeah, it's, it was, I mean, I feel very strong.

, Joe:

I joke that, I joke that I tease the community, but I do love them and I

, Joe:

feel strongly about it, and these are the types of conversations that I feel.

, Joe:

I feel don't get the visibility,

Stacy:

Mm

Joe:

from the industry because so much of it, I mean,

Joe:

prescriptive advice is amazing.

Joe:

It's great and it's useful, like I said, you got real world competing priorities.

Joe:

John represents the, here's the ideal way to do it, and Jérémy represents

Joe:

the competing priorities side and how do they work together and mesh.

Joe:

And it's a good conversation and I think we can just let them have it.

Stacy:

Awesome.

Stacy:

Well, take it away gentlemen.

Joe:

I really enjoyed the conversation from Friday, September 9th., Jérémy,

Joe:

you asked, you asked a question.

Joe:

Do you wanna say what question you posted into the slack?

Jérémy:

Yeah.

Jérémy:

So the question was about how to make, work together code drift and automatic

Jérémy:

FinOps actions because we are actually having that issue at the moment.

Jérémy:

So we have resources in Dev and so on that are not used anymore,

Jérémy:

but we want to delete them.

Jérémy:

But if we delete them using some kind of basic automated action

Jérémy:

then we end up with code drift and afterwards end up in technical debt.

Jérémy:

But the thing is sometimes when you do that deletion, it saves you so much

Jérémy:

money that maybe, maybe it's worse to have a little technical depth over that.

Jérémy:

And afterward, John.

Jérémy:

John answered a very, very, interesting response.

Joe:

Yeah.

Joe:

John, I see your responses like, it's so important because it's the ideal.

Joe:

Can you explain your perspective of how, what the answer is?

John:

Yeah.

John:

, I see what technical debt, I see how it, how it occurs.

John:

I see how it exists and, I've been in both sides of the story.

John:

I've a product owner doing a team that did some infrastructure engineering,

John:

and I've been an engineer and what I would consider the most important thing

John:

is, there is a single point of truth.

John:

If for any reason, you have a drift between your infrastructure

John:

and your infrastructure as code.

John:

You might as well not do it.

John:

Cause basically, , people rely on the infrastructure code to do their work and

John:

if there's drift people do stuff that either doesn't work cause it isn't as

John:

expected or even worsens the situation.

John:

It's a big anti-pattern.

John:

Both as an engineer and as a product owner in which it's always been, well,

John:

the goal for me to limit the amount of technical depth and even to, to make sure

John:

it doesn't automatically get created.

John:

Cause that's basically what happens, I think, if you do automatic actions.

Joe:

Right.

Joe:

And John, I have personal experience.

Joe:

I've shared on previous episodes of the podcast where there

Joe:

was infrastructures code.

Joe:

We used a third party automation tool that changed what the

Joe:

infrastructure looked like and it had catastrophic implications

Joe:

because of how the code worked.

Joe:

I always kind of hesitate, I, I, at least I advise like, Hey, before

Joe:

you use automation, check first, don't be so fast with automation.

John:

Oh, I'm, I'm more in favor of automation.

John:

You have to have automation, but make automation do the right stuff if your

John:

infrastructure is completely defined by infrastructure as code in ideal situation

John:

have your automation edit the code.

John:

So basically you edit the source of truth technical debt doesn't get created.

John:

You just have influence on

Jérémy:

I want to say the worst thing is that I definitely agree with you.

Jérémy:

I hate technical debt on going over and over again on the same stuff because it

Jérémy:

gets updated with an out of process way, and I could not agree more on that,

Jérémy:

but sometimes, sometimes it's good.

Jérémy:

It's, it's ideal.

Jérémy:

It's, let's say, I don't know how to say, but when you arrive on an

Jérémy:

infrastructure that has a proper state, you know, proper processes

Jérémy:

and and so on, I would say fixing technical debt is definitely a must.

Jérémy:

But at some points I'm sure there will be a lot of listeners and

Jérémy:

people as well that do work, that do arrive in a place where there's

Jérémy:

already a kind of big technical depth.

Jérémy:

And actually we are talking about FinOps here.

Jérémy:

So if you make some saves by adding some technical depth, it can help

Jérémy:

the managers free some time for you and your colleagues to afterwards

Jérémy:

fix the remaining, technical debt.

Jérémy:

Because at some point, if you spend that much in cloud resources, Then well,

Jérémy:

your margin for doing stuff outside of new features and fixes is limited.

Jérémy:

But , if you show your manager, , that was actually my situation.

Jérémy:

I wanted to show that I was, , giving them something, giving them some,

Jérémy:

, savings on resources that were unused or oversized or things like that.

Jérémy:

And by doing that, I was hoping that it would free the team some time to

Jérémy:

get into a better state, you know, , small step by small step, and so on.

John:

Yeah, but the question is, do you free the team?

John:

Cause basically what you do is you create technical debt.

John:

Which is an extra burden for the team, which, it makes it more difficult

John:

to do troubleshooting cause people actually have to watch what the

John:

truth is instead of relying on the truth that is in their control.

John:

So, yeah, I can imagine on one side you want to do savings, . We should

John:

do that, but, by doing so , on the same time, creating technical debt

John:

and creating more work for the engineers to solve the technical debt.

John:

you really save stuff?

John:

Do do you have savings or do you pay less for infrastructure

John:

and more for human resources?

John:

Cause basically the engineer is often the most expensive part of the infrastructure.

John:

That's not the two or three or tens.

Joe:

I find this so, so, so interesting because it's real life, right?

Joe:

There is a Yeah, of course.

Joe:

Fix the code.

Joe:

But Jérémy, I empathize with you so much because , I understand like,

Joe:

hey, there's multiple pressures, influencing that decision.

Joe:

I think one of the problems I made was, Oh, let's right size and then walk away.

Joe:

Whereas at least you're saying, let's right size or shut it down or delete

Joe:

it and then focus on fixing it.

Joe:

And then John, your point is, No, just fix it because you're

Joe:

creating additional burden outside of your cloud bill, , on that.

Joe:

But there, there's so many different pressures that go

Joe:

into those sort of decisions.

Joe:

It's a debate that I think every FinOps practitioner in every company

Joe:

IT department needs to take when focusing on the FinOps value of their.

Joe:

Perspective.

Joe:

Jérémy, what was the team's reaction, the application team's reaction to this?

Jérémy:

Well, the thing is, the application team, how can I say?

Jérémy:

When we have to do things like that, we first have to we have someone

Jérémy:

system that says, Okay, that resources underutilized or not utilized at all.

Jérémy:

So, one part of the thing that is problematic is sometimes tagging is not

Jérémy:

as good as expected and we have to look into it and find across the organization

Jérémy:

the one guy if he's still there that is responsible for, for the application.

Jérémy:

So, from what I could see when we find some people and say, Okay, you

Jérémy:

have that result that is not good.

Jérémy:

it's okay for them.

Jérémy:

I mean, it's normal.

Jérémy:

It's just that they would find normal to avoid that cost.

Jérémy:

But, we needed to set up the processes, in place in order to fix those.

Jérémy:

So this part is quite easy , when you manage to find the owner, the owner says

Jérémy:

yes, okay, it's maybe time consuming, but then you stay in line with Infr

Jérémy:

code because you do the update yourself, and then you remove the stuff and it's

Jérémy:

automatically directed through your deployment system and everything is fine.

Jérémy:

The complicated stuff is mostly when you don't find any owner.

Jérémy:

And I work in the financial world where regulation is quite heavy and it's very

Jérémy:

sensitive to delete anything, basically in any environment because we have some

Jérémy:

regulation that says that some stuff have to been kept for 10 years and so on.

Jérémy:

So , it's a little bit complicated.

Jérémy:

We have to snapshot stuff and maybe to destroy or to just dispose.

Jérémy:

We have different decision.

Jérémy:

There's no one pattern to solve everything.

Jérémy:

So at some points we need to make some choices and sometimes we say,

Jérémy:

Okay, we're going to take more time and, and solve it through us through

Jérémy:

proper processes, and sometimes we can save so much money by one action.

Jérémy:

Then you say, Okay, doing that now and fixing later.

Jérémy:

Okay.

Jérémy:

It introduces some technical debt, but a month or two, we will not

Jérémy:

have to pay that we would otherwise.

Jérémy:

But I understand John's view and I agree that in general,

Jérémy:

time is is actually hidden.

Jérémy:

Very expensive cost.

Jérémy:

So as I said, noisy choice.

John:

But, I see two things that trigger me.

John:

first one is you modify resources you don't have the owner from yet mean

John:

that, that sounds a bit like, Well, the whole FinOps practice , Was introduced

John:

as a crawl, walk and run stage.

John:

And one of the first stages you always go to is the inform.

John:

So make sure you have the right information and looks

John:

like you are trying to run.

John:

So you're trying to do the automation, which is basically about the last stage

John:

you can get in your maturity while the Inform phase isn't complete yet.

John:

Cause you dunno who the owner is, but you try to take automated

John:

responses based on that resource that, that, that would seriously

John:

scare hell out me and the second is,

John:

When I started my FinOps journey I thought it was the next thing

John:

after DevOps, , You could integrate developers and operations people.

John:

You could use the same tool, use the same speak, just started I

John:

hoped it would be the same thing.

John:

So engineering actually talking to finance, doing the more, and I

John:

should more and more FinOps teams, just as more the central team

John:

without debt engineering engage.

John:

How do you manage that?

John:

Cause basically, uh, yeah.

John:

How do you distinguish the FinOps team from a team like security, quality, etc.

John:

All those team that try to influence the backlog for the infrastructure engineers?

Joe:

Yeah, John, Both points are really good cuz the first point is,

Joe:

You know, like that informed phase if you don't have really good tagging,

Joe:

that is a form of technical debt.

Joe:

and I'm trying not to laugh out loud.

Joe:

As you're describing, taking automation, which is a very

Joe:

mature run state sort of thing.

Joe:

And then, but using, really bad tagging, to drive it, that is kind of dangerous.

Jérémy:

Yeah, indeed, indeed.

Jérémy:

This is dangerous.

Jérémy:

And indeed, we missed out, what would have been the proper processes.

Jérémy:

But the thing is, getting the informed phase can be challenging

Jérémy:

with huge technical debt.

Jérémy:

So waiting for the end of the informed phase before moving to the other ones,

Jérémy:

it's the proper process indeed, but, , management will not see it that way.,

Jérémy:

if all the time you spent if the inform phase is used to reduce your,

Jérémy:

your cloud spent at the end of the month, at the end of the year, they

Jérémy:

are still not seeing improvements, but only by you adding tagging.

Jérémy:

So we also have we also have some initiatives to, fix tagging,

Jérémy:

all over our infrastructure.

Jérémy:

But in the meantime unfortunately, we have some budgets, that we

Jérémy:

were on the point of exceeding.

Jérémy:

And at some point when you have that kind of situation, you cannot help

Jérémy:

but look into your overall costs and see, okay, is there anything I can do

Jérémy:

that can be qualified as a quick win?

Joe:

This is why I absolutely love this conversation because John, you're,

Joe:

you're right, but Jérémy, you're also right, is just like, I don't

Joe:

always have the time because , you know, the boss, the management, the

Joe:

business is like, You know, it's, it's kind of an impossible situation,

Joe:

but they're like, deliver right now.

Joe:

I wanna go back to what you said, John, and you posted it in the Slack thread

Joe:

and I liked how you said it, that, you were saying in two points earlier.

Joe:

The second point was that, you know, you were excited about FinOps cuz

Joe:

you thought it was going to bring the financial impact and engineering

Joe:

engagement together closer.

Joe:

And that's, you often are seeing teams that add just another layer in

Joe:

between finance and engineering and that exists for a reason for a lot

Joe:

of companies and they do a good job translating between the two.

Joe:

However, I think it's really interesting that, if you have a centralized FinOps

Joe:

team, you really do need engineering skillset to jump into these conversations.

Joe:

and I think that's what you're trying to say, John, is

Joe:

there needs to be engineering solutions to problems like this.

John:

Yeah, that's least personal belief is team should have a senior

John:

engineer to just do an impact anlaysis.

John:

to just think in code.

John:

Just like team should have a well educated finance specialist

John:

that really knows the processes.

John:

Because, I remember I studied for the practitioner exam and I

John:

came into terms like amortization.

John:

I was like, Okay, this has been since high school economics.

John:

Think this, please.

John:

I don't anymore.

John:

It's been 20 years or something.

John:

So, no, but every team needs the different disciplines, and I'm not

John:

opposed to a centralized FinOps team.

John:

FinOps teams, require some sort of, helicopter view over the teams,

John:

but it should really have an engineer in them just like it should really

John:

have a finance professional in them.

John:

And preferably a software engineer too.

John:

Not just the engineer but the software engineer.

John:

Now they need a software engineer.

John:

Cause basically the software engineer can also add the

John:

impacts cause what will they do?

John:

Will have serverless function?

John:

Software architect easily make good judgment on whether to use serverless

John:

frameworks to use, for example, he can get the impact on the application.

John:

And FinOps doesn't end with infrastructure.

John:

It's one of my long term thoughts is the, the separation between software

John:

and infrastructure will be gone.

John:

it'll end.

John:

Basically, infrastructure is always about four, five years

John:

behind those software development.

Joe:

I love it.

Joe:

It's so true because , you're talking about infrastructure

Joe:

as code and deployment as code, pipelines , and all that.

Joe:

But there are still plenty of applications that are launching through the console.

Joe:

, but they won't always be and they need that modernization path forward.

Joe:

And software engineers probably really good path of insights there

Joe:

I love the conversation as much as I love the slack thread, this is real

Joe:

life FinOps debates and actions.

Joe:

I think it's so valuable to give attention to.

Joe:

So I wanna thank you both for being here and having this conversation further.

Joe:

me.

John:

Yeah.

John:

But it's really nice to have to talk with peers and to know, to get more

John:

insight in, in, in what the other people.

Jérémy:

Indeed, indeed.

Jérémy:

Even, even the slack is inspiring about, everyone's different use cases and

Jérémy:

being able to talk is actually great at another level, but it's is actually

Jérémy:

great to be able to exchange with everyone, with you guys, but with anyone.

Jérémy:

Best practices, how to, how to do what, what, who we should

Jérémy:

include in teams and so on.

Jérémy:

It's is very inspiring especially for me who's, let's say

Jérémy:

young in, in the FinOps world.

Jérémy:

So really, really, I'm really glad, we had that, that conversation together.

Joe:

Hmm, Hmm, hmm.

Joe:

Things that make you go.

Joe:

Hmm.

Joe:

Now, look folks like we said, podcasts don't grow on trees

Joe:

and neither do music rights.

Joe:

And if I had the money to get the music rights to C&C Music Factory's "Things That

Joe:

Make You Go Hmm", I'd be playing it right now because that conversation makes me go

Joe:

'Hmm, what would I do in that scenario?'

Joe:

How would I handle that?

Joe:

And honestly, rather than me buy the music rights and play it under

Joe:

this right now, I recommend you go and look it up on YouTube.

Joe:

It's a good music video, 1990.

Joe:

The glory days of music videos.

Joe:

But back to the conversation at hand, this is a real debate, the type of

Joe:

thing that when I was a practitioner, you had to consider what was the

Joe:

best way to go about it, you know?

Joe:

Unfortunately, not everything is in code, and the only

Joe:

thing you have is the console.

Joe:

Launching things to the console.

Joe:

you would hope you get more mature and start launching the solution through

Joe:

code, and then you start launching the solution through pipelines.

Joe:

And this conversation gets a lot easier as your application stack matures.

Joe:

but not everyone is at that same maturity.

Joe:

So these are the considerations, and debates that you have to have.

Joe:

Hey, we can right size this now and stop the expense, but we are creating

Joe:

more expense later if we leave this tech debt and code drift behind in our wake.

Joe:

Great conversation, great insights from Jeremy and John.

Joe:

If you have any questions, reach out to Jeremy and John

Joe:

on our FinOps Foundation Slack.

Joe:

And if you don't have access to our FinOps Foundation Slack, then head over to FinOps

Joe:

dot org and click join the community.

Joe:

We'd love to get you involved

Joe:

. Big thanks.

Joe:

Go out to Jérémy Nancel and John Renn.

Joe:

Thanks for being willing to have the conversation, and letting me record

Joe:

it, and share this with the community.

Joe:

Thank you to Stacy Case.

Joe:

Always bringing the energy.

Joe:

Can't wait to do KubeCon with you and Suha and J.R..

Joe:

Stop by the booth and say hello to us if you hear this and

Joe:

you happen to be in Detroit.

Joe:

If not, hey, there'll be other conferences.

Joe:

You could say hello, then we're friendly.

Joe:

Come say hi.

Joe:

And that is all for this episode of FinOpsPod.

Chapters

Video

More from YouTube