Shownotes
Vlad Lukic, BCG’s global leader for tech and digital advantage, and Paul Goydan, global leader of BCG’s cost offer, explain why so many companies spend more on AI than they get back. What is the fix? They argue it isn’t simply cutting AI spend, but assigning clear ownership, categorizing costs correctly, and tying every dollar to a business outcome.
You’ll Learn:
Business owners, not IT, should be accountable for AI’s return, like any other investment.
Many companies give routine tasks to their most powerful and expensive AI models, when a simpler tool could do the job.
Instead of focusing on banning unauthorized AI tools, leaders should turn towards educating employees on sanctioned options.
Learn More:
How Leaders Build an AI-First Cost Advantage: https://on.bcg.com/4c3KrSW
Why We Still Need a CIO in the AI-First Era: https://on.bcg.com/46bBbIV
Chapters
0:00 AI's Bottom-Line Problem
1:01 Why Your AI Costs Don't Add Up
2:16 Can Promoting AI Use Promote Waste?
4:02 Are AI Costs an IT Problem?
4:48 How to Prioritize AI Spend
6:18 How Do CEOs Pay for AI?
7:14 How to Categorize AI Costs?
8:02 Who Owns the AI Budget?
8:30 How Leaders Know AI Is Paying Off
10:50 Strategy vs. FOMO
13:49 Turning FOMO Into Action
14:27 Winners in the AI Era
16:28 Handling Hidden AI Risk
18:05 Now What: Next Steps
Listen to Other Episodes of The So What from BCG podcast
YouTube | https://youtube.com/playlist?list=PLMJgyXjV5gMI9JV-GcF_D1Y6zyf1Eab_0&si=plXqe7-YNzbG56U8
Apple | https://podcasts.apple.com/us/podcast/the-so-what-from-bcg/id1591194141
Spotify | https://open.spotify.com/show/2NSVR7qrAyZ4CaGsnknbBk?si=1d846c2af8784923
Other platforms | https://lnk.to/so-what-general-show12
Follow BCG
LinkedIn | https://www.linkedin.com/company/boston-consulting-group
This podcast uses the following third-party services for analysis:
Podtrac - https://analytics.podtrac.com/privacy-policy-gdrp
Transcripts
- AI cost has gone
from the server room
Speaker:to the boardroom in
the last two years.
Speaker:If we simply go after AI
cost the same way we go
Speaker:after commodity IT spend,
Speaker:we don't get the value
in the business.
Speaker:- AI, especially generative
AI, is not like an Ozempic.
Speaker:It is more like
a gym membership, right?
Speaker:It will cost you to be there, to go there.
Speaker:It will cost you to
actually exercise.
Speaker:You will need to dedicate
time but for awesome results.
Speaker:- Welcome to "The So What
from BCG," the podcast
Speaker:that explores the big
ideas shaping business,
Speaker:the economy, and society.
Speaker:I'm Georgie Frost.
AI was supposed
Speaker:to help businesses become more efficient.
Speaker:Instead, for many, it's adding
a whole new layer of cost
Speaker:and putting pressure
on the bottom line.
Speaker:So how do leaders decide
where to cut back,
Speaker:where to spend more, and
crucially, how to pay for it?
Speaker:Joining me today are Vlad
Lukic, BCG's global leader
Speaker:for Tech and Digital practice,
Speaker:and Paul Goydan, global leader
Speaker:of BCG's AI Cost
Advantage practice.
Speaker:Vlad, Paul, welcome.
Paul, as I said at the top,
Speaker:with AI comes this great promise
Speaker:of greater efficiency,
doesn't it?
Speaker:But for many businesses,
Speaker:the economics just
don't quite seem
Speaker:to be adding up yet.
Speaker:Why is that?
Speaker:- I think first of all, we
love to experiment and pilot.
Speaker:When they don't work out,
Speaker:we don't always stop
paying for those.
Speaker:A lot of companies also
went into tokenmaxxing
Speaker:to get their employees to test and try AI,
Speaker:and that led to exorbitant
bills but low value.
Speaker:- Vlad, isn't that
just simply the cost
Speaker:of adopting a new technology,
Speaker:or are companies spending money,
Speaker:money in the wrong places?
Speaker:- No, it's a little bit of both.
Speaker:Like we are in the
stage where we are,
Speaker:which is you could argue
a lot of those costs,
Speaker:as Paul nicely said,
are in the early stages.
Speaker:I would put it in the bucket
of learning and development
Speaker:where we need to get
our hands on keyboards
Speaker:to get a sense of what
it can, what can be done.
Speaker:But as we mature and progress
in the deployment of AI,
Speaker:the companies that really
unlock the value are the ones
Speaker:that tie, they're, they're
really thoughtful about
Speaker:where they spend the money on AI.
Speaker:They're very clear in articulating
Speaker:where it's adding new capabilities
Speaker:or new services to clients
and it's making a difference,
Speaker:and they're managing it as such.
Speaker:- When does encouraging adoption
Speaker:become encouraging waste?
Speaker:- Yeah, that's a,
that's a great push.
Speaker:And each company will
be in a different,
Speaker:in a different phase.
Speaker:If you're really trying to
encourage your employees to get
Speaker:to learn what's going on and,
Speaker:and just experiment, you
need to be tokenmaxxing.
Speaker:But then you need to very quickly,
Speaker:once you get to a level of
understanding or end usage,
Speaker:quickly start pivoting in
the direction of managing
Speaker:and being very explicit
on where is the AI adding
Speaker:into the, into the process
Speaker:and it's making you more efficient
Speaker:versus it's doing redundant
work or it's giving you a tool
Speaker:that is doing extra
stuff that's not needed.
Speaker:- Valuemaxxing is in.
Speaker:I think part of that is
Speaker:when you need to
learn how to drive,
Speaker:you have to get
behind the wheel,
Speaker:but you don't need to drive
a Ferrari or a Maserati.
Speaker:A frontier model costs 30%
more than a good enough model.
Speaker:Yet when we come
into corporations,
Speaker:we see more often than not
Speaker:very basic rudimentary
tasks are being handed over
Speaker:to frontier models.
Speaker:It's like giving your teenage
driver a brand-new Ferrari.
Speaker:- Which is never a good idea.
Speaker:Vlad, you spoke about the
overspending, underspending.
Speaker:Can you give me some examples?
Speaker:- I'm seeing like, for example,
Speaker:in, in operations companies
Speaker:that are really good operators
in lean where they feel like
Speaker:"I don't need more
technology in here.
Speaker:I'm really running
a lean process,"
Speaker:so therefore, they
don't spend enough.
Speaker:But the mindset they have is
one to two, 3% improvement.
Speaker:With AI, you can be
thinking 30, 40, 50%.
Speaker:And so therefore they underspend
Speaker:into those core workflows
that they're really good at,
Speaker:thinking that they got it,
Speaker:versus if they spent more there
Speaker:and really rethought them
Speaker:and got a step-change
performance improvement,
Speaker:it would be a lot of money
that they are creating.
Speaker:- Paul, I just want to ask you.
Speaker:The AI spending has become
Speaker:a boardroom issue remarkably quickly.
Speaker:Just explain to us why,
Speaker:why this is no longer
just an IT conversation.
Speaker:- AI gives back to
the shareholders,
Speaker:makes companies better
Speaker:by impacting the P&L,
revenue, customer engagement,
Speaker:and making companies
more efficient.
Speaker:None of that touches
the CIO's budget.
Speaker:If we simply go after AI cost
Speaker:the same way we go after
commodity IT spend,
Speaker:we don't get the value
in the business.
Speaker:A rough rule of thumb
we have at BCG is,
Speaker:for every dollar
of value you see,
Speaker:you should be spending about 20 cents more
Speaker:on the tech stack, and then
80% is the efficiency gain.
Speaker:- Vlad, we know that
businesses are being told
Speaker:to spend less, but then at
the same time, spend more
Speaker:because obviously you
need to invest in this AI
Speaker:and the new technologies.
Speaker:So where should organizations
be cutting costs,
Speaker:and where should they
be investing more?
Speaker:And importantly, how
do you figure that out?
Speaker:- You have to do both, right?
Speaker:And one of the things
that I like to use,
Speaker:a few of my colleagues came
up with a really cool analogy
Speaker:that AI, especially generative AI,
Speaker:is not like an Ozempic.
Speaker:It is more like a gym membership, right?
Speaker:It will cost you to
do the, to go there.
Speaker:It will cost you
to actually exercise.
Speaker:You will need to dedicate
time but for awesome results.
Speaker:But it's persistency of the
motion and presence in it.
Speaker:Now, as it gets
to AI specifically,
Speaker:there's a lot of waste.
Speaker:When we come into
companies on the tech side,
Speaker:we usually find 20 to 30%
of the dollars being wasted.
Speaker:You can imagine like the tech
world has been upside down
Speaker:over the last two years,
Speaker:charging for licenses, to
charging to consumption,
Speaker:to charging to tokens.
Speaker:Like the procurement practices
have not evolved as quickly.
Speaker:So there is usually,
Speaker:I call it "sweating the stack" opportunity
Speaker:where you can free up the
cash that you can then deploy
Speaker:to buy some of
the new technology.
Speaker:As you use the new
technology, you also need
Speaker:to link it directly
to business outcomes
Speaker:and free up resources there.
Speaker:So as Paul nicely said, you either need
Speaker:to gain some efficiencies
Speaker:or you need to generate some
momentum on the top line.
Speaker:- You anticipated my
next question, which is
Speaker:where is this money coming from?
Speaker:But you're thinking it can come
Speaker:from efficiencies elsewhere.
Speaker:- It has to, right? It cannot
just come from the tech side.
Speaker:It needs to come from
the new business outcomes.
Speaker:That can be on the, either
Speaker:on the efficiency
and the cost side
Speaker:or that efficiency translated
Speaker:into higher commercial momentum
Speaker:that allows you to, to charge
Speaker:and get better margin
into the business.
Speaker:- A great example of what Vlad
was just illustrating is
Speaker:one of our clients who was
seeing a tremendous return
Speaker:on AI in the marketing business.
Speaker:That AI cost became part
of the marketing budget.
Speaker:And the CMO had to justify why
they're spending more money
Speaker:on marketing through
the traditional metrics
Speaker:of better customer
engagement, higher sales,
Speaker:higher revenue per
customer engagement.
Speaker:It's no different in any
other part of the business.
Speaker:The cost of AI needs to sit
Speaker:with the end business
user of the AI.
Speaker:- So, Vlad, walk me through this.
Speaker:How exactly should businesses
actually be thinking
Speaker:about the cost of AI?
Speaker:- We can categorize pretty
much all the AI spent
Speaker:either as CapEx, which means we've used it
Speaker:to build some sort
of infrastructure
Speaker:within the company,
Speaker:and it should be
categorized that way.
Speaker:Then we have a part of
it that should be OpEx,
Speaker:which is new set of
tools that are part
Speaker:of me just delivering my
daily jobs in a specific way,
Speaker:and I should categorize it as OpEx.
Speaker:And then there are components
of it that are directly tied
Speaker:to the services or
products that we provide,
Speaker:so it becomes a COGS.
Speaker:IT can enable this service
and access to this technology,
Speaker:but if the companies
categorize it this way,
Speaker:it's fairly easy to then link
it into the business ownership
Speaker:that then ties it
to business outcomes.
Speaker:- Paul, does the cost
categorization
Speaker:that Vlad was just
talking about,
Speaker:does that change who's kind of
responsible for the cost
Speaker:and, ultimately, for deciding
whether it's delivering value?
Speaker:- I think it does. The
business owner needs
Speaker:to be accountable for
the cost of deploying AI.
Speaker:The CIO's role is to help you be efficient
Speaker:in the technical choices and
the AI application you make.
Speaker:But the ultimate ROI
sits with the business,
Speaker:just like any other investment.
Speaker:- What should leaders
actually be measuring?
Speaker:How do they know
that it's paying off?
Speaker:- They should measure the outcome
Speaker:they set out to achieve
and value creation, right?
Speaker:I'll give you a silly example.
Speaker:With a client, they tried to measure,
Speaker:or the initial objective
was let's measure
Speaker:can we do this task shorter
than it took us before.
Speaker:And it used to take
them 10 days.
Speaker:Now they were doing
it in a day.
Speaker:But technically,
they had a solution
Speaker:that could do it in a day.
Speaker:In reality, the business
processes were never changed
Speaker:and how they,
Speaker:and the experience of the
customer didn't change.
Speaker:It still took 10 days
to get a response back,
Speaker:and they actually
had an extra cost.
Speaker:And there was no forcing function
Speaker:to have them rethink the process
Speaker:because the metric they
were looking for is,
Speaker:can I do this faster
with technology?
Speaker:Once they've realized
the mistake, they said,
Speaker:"Okay, we're actually going
to measure an outcome,
Speaker:which is that the customer gets a response
Speaker:in a shorter amount of time
and that it costs us less."
Speaker:Once they did that change
Speaker:or in the metrics
they were looking at,
Speaker:they rethought the whole process.
Speaker:They eliminated a
bunch of committees.
Speaker:They eliminated a lot of steps in between
Speaker:and managed to bring the
process down to one day,
Speaker:so you could get your
response much sooner.
Speaker:And they also managed
to optimize the cost,
Speaker:as Paul mentioned,
Speaker:on how you manage the context
windows, et cetera.
Speaker:So they did it cheaper
and much faster,
Speaker:but they had to change the
metric they were looking at.
Speaker:- Would you say that's a typical example?
Speaker:Do you think most
companies get it?
Speaker:- Listen, there's a lot
Speaker:of smart people in
a lot of companies.
Speaker:We are seeing
the profit numbers
Speaker:across the industries
being record high, right?
Speaker:So companies can do this,
Speaker:and when they focus on
it, they get it done.
Speaker:The gap right now in the
market is just the experience
Speaker:of doing that with
the AI in the mix
Speaker:as a new tool, if you will.
Speaker:And what I'm seeing, the ones that persist
Speaker:with the motion quickly feed the learnings
Speaker:that I just shared back
into their workflows
Speaker:and how they manage it.
Speaker:It's not rocket science,
Speaker:but it just requires
that persistence
Speaker:and reflection and
feedback loop.
Speaker:And the good news is that a lot
Speaker:of companies are figuring it out
Speaker:and they're actually
fine-tuning their motions.
Speaker:- Paul, you both speak
to leaders all the time.
Speaker:How much do you sense
Speaker:that a lot of the spending is being driven
Speaker:by a clear business strategy,
Speaker:and how much is, I suppose,
Speaker:a bit of fear of
being left behind?
Speaker:- FOMO drives a lot of spending.
Speaker:Every, every executive
I talk to has been asked
Speaker:by their board, "Are
you applying AI enough?
Speaker:Are you consuming enough AI?"
Speaker:If we flip the question,
like Vlad said,
Speaker:to how much have your earnings
improved because of AI,
Speaker:it changes the ask.
Speaker:I absolutely think there's
a broad distribution,
Speaker:and there are too many companies
Speaker:who are misapplying AI
Speaker:and not getting the
return they could.
Speaker:- When you say misapplying
AI, explain that to me.
Speaker:What are the biggest mistakes?
Speaker:- AI, in some cases,
is no different
Speaker:than any other application of technology.
Speaker:If you go in with a solution
looking for a problem,
Speaker:you're guaranteed to spend money.
Speaker:You're not guaranteed
to get a return.
Speaker:If you have a
laser-like hypothesis
Speaker:of what AI will do better, faster, smarter
Speaker:in your business and how
that hits the bottom line
Speaker:before your fingers ever touch a keyboard,
Speaker:you'll be 10 times
more successful.
Speaker:- Can you give me an example?
Speaker:- One of the most recent
examples I had was applying AI
Speaker:in a manufacturing
setting to reduce waste.
Speaker:The first question I asked
is, what percent of your cost
Speaker:of goods sold goes to waste?
Speaker:And they said less than 1%.
Speaker:And when you do the math
on 100% improvement
Speaker:of 1% of your cost, you
couldn't pay for the AI.
Speaker:That was an application of AI to a problem
Speaker:with incredibly low value.
Speaker:- Vlad, Paul there
spoke about FOMO,
Speaker:the fear of missing out,
and anxieties from leaders.
Speaker:Is that, does that
resonate with you?
Speaker:- Yes, to a degree.
And it is real.
Speaker:And I just want to make
sure that it's not seen
Speaker:as a silly "I need to
quickly copy," right,
Speaker:and "I need to be doing the
same thing as well" kind
Speaker:of high school behavior FOMO.
Speaker:It really is coming from
like really good examples
Speaker:out there where AI
has created value
Speaker:and executives feeling,
whoa, it's real, right?
Speaker:So it's coming from, there's
enough evidence to show
Speaker:that it, when applied
the right way,
Speaker:it can really generate
tremendous value
Speaker:and it can create a really
strong competitive advantage.
Speaker:And therefore, it's in that
context that they're seeking
Speaker:to make sure that they
replicate something like that
Speaker:and to make sure their
competitors don't get
Speaker:to that advantage
before they do.
Speaker:- Georgie, I think FOMO is
fear of market obsolescence.
Speaker:CEOs and boards are afraid
Speaker:their company won't be relevant
Speaker:in five or 10 years
because of AI.
Speaker:It's not missing out because
you're not a cool kid.
Speaker:It's actually that your
company could be left behind
Speaker:in a way that it
becomes entirely obsolete.
Speaker:- Yes. Yes. And that's,
it's visceral, it's real,
Speaker:and it for sure is driving
a lot of this behavior,
Speaker:which is actually, I
would say, very logical.
Speaker:- Without veering into
the realms of psychology,
Speaker:which I'm certainly not
qualified to talk about,
Speaker:but how do you get over that?
Speaker:Because it's absolutely real.
Speaker:It is impacting
leaders' decisions.
Speaker:How do they get past that?
Speaker:- I don't think they need to get past it.
Speaker:It is actually the right driver.
Speaker:The question is
how do you manage?
Speaker:How do you channel that energy
Speaker:where it's going
to create value?
Speaker:But, Paul, I don't know if
you would agree with that,
Speaker:or would you steer
it differently?
Speaker:- I was thinking of
an ostrich analogy
Speaker:of you can't put your head
in the sand and take flight,
Speaker:but otherwise I think you
hit the nail on the head.
Speaker:- Beautiful. Vlad, what are
you hearing from leaders?
Speaker:What are they telling you,
Speaker:and why do they
ask for your help?
Speaker:- Because they're struggling
with a lot of things
Speaker:that we mentioned,
which is, okay,
Speaker:I'm getting pressure from the
board to push this stuff in.
Speaker:I'm getting pressure from
my employees who are saying,
Speaker:give me these tools.
Speaker:I've never faced it before
as a business leader.
Speaker:I didn't learn about it in school.
Speaker:So I don't have an
intuition in my gut
Speaker:that tells me I should lean
in or I should not lean in.
Speaker:How far? How should I
even sanity check?
Speaker:So it's the ignorance
meeting extreme demand,
Speaker:and then the pressure and push from a lot
Speaker:of the tech companies, right?
Speaker:So they're feeling pressure
from all of these areas.
Speaker:And fairly easy way out
of it is, you know what?
Speaker:We'll put some budget aside.
Speaker:Let's start experimenting,
and let's just start testing
Speaker:and building intuition around it.
Speaker:The trick is to then,
at some point, move
Speaker:from those experiments
into lessons learned
Speaker:that are feeding into
the actual interventions
Speaker:and how you manage this.
Speaker:And the companies that manage
Speaker:that transition the faster are
Speaker:actually the ones that are winning.
Speaker:The ones that have leaned in
Speaker:and have figured out
these things are growing
Speaker:one and a half to two times
faster than their competitors.
Speaker:Their stock price, total
shareholder return is
Speaker:three to four times
higher than those
Speaker:that have not figured it out.
Speaker:Their EBIT is one and a half
Speaker:to two times higher
than the other.
Speaker:So it's totally doable.
Speaker:- Paul, what sense do you get
from those leaders that are
Speaker:going to win in this space?
Speaker:- The leaders that are
actually seeing return on AI
Speaker:are the ones that can in
an elevator articulate
Speaker:what they're trying to
change in their business
Speaker:and how it drives earnings.
Speaker:That laser-like linkage and understanding
Speaker:of the technology,
I think, is going
Speaker:to set market leaders apart.
Speaker:- Laser-like articulation,
Paul, why is it so hard?
Speaker:- It requires both
a deep understanding
Speaker:of what the technology
can and can't do
Speaker:and deep operational insight
on your own business.
Speaker:Those two skill sets
are rarely overlapping
Speaker:with the modern
cutting-edge technology.
Speaker:- Vlad, one trend we're
seeing is employees signing up
Speaker:to AI tools themselves
rather than waiting for them
Speaker:to be approved by IT.
Speaker:What challenges does that
create for businesses?
Speaker:- On the good side, it
generates enthusiasm
Speaker:and it generates
momentum, et cetera.
Speaker:The negative side is it
exposes a lot of company data
Speaker:to non-enterprise-grade tools,
Speaker:so your data can leak
out the way it shouldn't.
Speaker:Two, it creates a much
larger cyberattack surface
Speaker:for bad actors to
access the company.
Speaker:Three, it can start generating data
Speaker:that is very inconsistent
Speaker:because people are using different things
Speaker:and feeding it back into
the corporate environments
Speaker:in a very inconsistent way.
Speaker:So the list becomes very,
very, very long,
Speaker:and therefore, companies
need to stay very close
Speaker:to that reality,
which is very true,
Speaker:and channel that energy into
the enterprise-grade solutions
Speaker:that the employees
can get access to.
Speaker:- Is that something that you would,
Speaker:I don't want to say the word
"ban," but is it that serious?
Speaker:Because some of those things
that you said suggests to me
Speaker:that this is a serious problem.
Speaker:- Like, the question is,
can you really ban it, right,
Speaker:given how easily accessible it is, right?
Speaker:You can now take your phone
and take a photo of this
Speaker:and put it in
your personal tool,
Speaker:and it can start doing things.
Speaker:- Yeah.
Speaker:- So it is really hard to ban.
Speaker:I think the key thing
is to acknowledge it,
Speaker:educate the employees
on the dangers
Speaker:of doing it in an
irresponsible way,
Speaker:and then create effective
alternatives to that
Speaker:within your corporate
environment, right?
Speaker:So it's a, you've
got to attack it
Speaker:from a few different ways.
Speaker:You cannot ignore it.
Speaker:- We've covered the so
what, now the now what.
Speaker:What are the next immediate
steps leaders need to take
Speaker:to ensure that
their companies are
Speaker:the leaders of
the future? Paul?
Speaker:- All of our clients
need to start
Speaker:to understand the technology
and the billing mechanisms
Speaker:and treat it just like they do
Speaker:any other large cost category.
Speaker:And there's certainly a lot of efficiency
Speaker:to be had across the board.
Speaker:Model selection is a whole new capability.
Speaker:Likewise, the size of your context window
Speaker:and how much you use AI
in the same prompt grows
Speaker:exponentially to the
amount of usage.
Speaker:How you think about that in
business applications is
Speaker:a whole new capability.
Speaker:We have a whole new
practice emerging, Georgie,
Speaker:that didn't exist six or eight months ago,
Speaker:and it's helping
clients understand
Speaker:how to cost-effectively
apply AI.
Speaker:- The fascinating thing now is
Speaker:things we can do now were
impossible to be done
Speaker:even just two, three
weeks ago, right?
Speaker:So not treating it as a project
Speaker:but rather as an ongoing
muscle that you need
Speaker:to be exercising is the
way I would think about it.
Speaker:- How do you then
think about cost?
Speaker:Same buckets,
same compartmentalization?
Speaker:- Same buckets and same focus
Speaker:and acknowledging that
Speaker:how you get charged
will evolve very quickly
Speaker:based on the different entrants.
Speaker:Some of them will try to
charge you based on outcomes,
Speaker:some based on usage,
Speaker:some based on some
sort of subscription.
Speaker:There will be a lot
of different models.
Speaker:You still need to diligently
think about that, right?
Speaker:And that is part of building
that intuition of how
Speaker:to anticipate those moves and
being part of that ecosystem.
Speaker:- Vlad, Paul, thank you so much.
Speaker:And thank you for listening.
Speaker:If you'd like to find out
more about this subject,
Speaker:you can find links to
Vlad and Paul's research
Speaker:in the show notes.
