Every marketing team wants attribution. But weirdly, it's often not that satisfying when they actually get it.
I led many multi-touch attribution projects as a consultant, and we got really good at implementing tools, creating taxonomies, and making sure that data was clean.
But I found that when you actually showed these reports to a C-level executive, it was usually kind of underwhelming. The data didn't always pass the common sense test.
Today's guest thinks there's a better way — Marketing Mix Modelling. It's basically the application of mathematical techniques to model relationships between different variables.
However, technology now enables it to happen faster and more cost-effectively than ever before.
Many thanks to the sponsor of this episode - Knak.
If you don't know them (you should), Knak is an amazing email and landing page builder that integrates directly with your marketing automation platform.
You set the brand guidelines and then give your users a building experience that’s slick, modern and beautiful. When they’re done, everything goes to your MAP at the push of a button.
What's more, it supports global teams, approval workflows, and it’s got your integrations. Click the link below to get a special offer just for my listeners.
Mark Stouse is CEO of ProofAnalytics.AI. With over 26 years of experience in marketing communications and strategy, he has a passion for transforming GTM performance with data-driven insights and agile decision making. Prior to founding Proof, Mark was CMO at Honeywell Aerospace, CCO at BMC Software, and a marketing leader at Hewlett Packard Enterprise.
https://www.linkedin.com/in/markstouse/
Visit the RevOps FM Substack for our weekly newsletter:
Every marketing team wants attribution.
2
:But the weird thing is that it's often not
that satisfying when they actually get it.
3
:I did a lot of multi-touch attribution
projects as a consultant, and we got
4
:really good at implementing these
tools, technically creating taxonomies,
5
:making sure that data was clean.
6
:But I found that when you actually show
these reports to a c-level executive,
7
:it can be kind of underwhelming.
8
:They don't always pass
the common sense test.
9
:People want to pick on details, and maybe
that's because the idea of dividing up an
10
:opportunity like a pizza between different
touch points isn't actually the way.
11
:Maybe that's not going to
give us the answers we need,
12
:but where does that leave us?
13
:Because we still need to know what's
working in marketing, and today's guest
14
:thinks there's a better way, and he's
founded a company to make marketing mix
15
:modeling or MMM available to B2B marketers
and he's gonna tell us all about it.
16
:So I'm super excited to
welcome Mark Stouse, CEO of
17
:proof analytics to the show.
18
:Thanks a lot for being here, mark
19
:Mark Stouse: Hey, thank you so much.
20
:Justin Norris: Mark, maybe
one clarifying question.
21
:I've seen MMM, spelled out as media mix
modeling and marketing mix modeling, which
22
:is the right way from your point of view?
23
:Mark Stouse: actually.
24
:It really represents the evolution of
it over the last say, 40 to 45 years.
25
:Back when Procter and Gamble first brought
out what was then called econometric, I.
26
:Analysis it was advertising.
27
:that's what it was really all about,
hence the media mix modeling reference.
28
:As, time moved on channels proliferated,
it became still very much within
29
:kind of B2C a reference point.
30
:It became marketing mix modeling.
31
:Today it is really go to market mix
modeling because it includes not
32
:just marketing data and channels and
investments and all that kinda stuff, but.
33
:Sales and customer success and product
data outside data externalities, you
34
:know, the economy, your competitor
actions, really is today much
35
:broader canvas as it should be.
36
:Right?
37
:That's the long and the short of why
people say that MMM means media mix
38
:modeling or marketing mix modeling.
39
:And actually both of 'em are
kind of A little bit passe today
40
:Justin Norris: Maybe let's take a
step back before that and talk about
41
:what was your professional experience
leading up to founding proof analytics
42
:and what brought you to this direction?
43
:I.
44
:Mark Stouse: So I was actually pretty
much a classic marketer and communicator.
45
:I've worked across all of the different
subsets of marketing at one time or
46
:another, and I've been a large company,
CMO, . About a little less than 20
47
:years ago I was at HP and we were all
kind of in the middle of an existential
48
:crisis because the then CEO of hp,
mark Hurd was a very Operations focused
49
:and a very customer focused CEO.
50
:And he wanted to know why there wasn't
more evidence of what marketing was
51
:actually delivering to the company.
52
:It was actually incredibly unpleasant.
53
:And about the only good thing that I can
say about that whole experience that I had
54
:and that other peers of mine had was that
it was, was highly motivating to change.
55
:I kind of got to a point where I
said to myself, look, I either have
56
:to do something to fix this or I
just need to like go do something
57
:else, 'cause it's not just about
budget issues and all that kinda stuff.
58
:It's about credibility,
am I actually doing here?
59
:So in my particular case for whatever
reason, herd gave me a project.
60
:To actually see what could
be done to resolve this.
61
:And I didn't know Jack at
that time about analytics.
62
:I was not even a math enthusiast,
and and so I, I remember I went
63
:home that next Friday I went
through all the stuff in my garage.
64
:. And found a couple of old math
textbooks from university and started
65
:to read them in ways that I never
read them when I was in school.
66
:' cause I was actively seeking an answer
and all of a sudden I got to this and it
67
:was all about multi-variable regression.
68
:Which is absolutely the cornerstone of
causal analytics ? It is the cornerstone
69
:of the scientific method of inquiry.
70
:It has so much credibility that,
it'd be hard to have more credibility
71
:than multi-variable regression has.
72
:Not because it is perfect, but it
is absolutely the best that we have.
73
:And today, is actually
the bedrock of causal ai.
74
:So I started working on this project
for Herd, and he liked it a lot.
75
:And I guess my prize was he
set up a mentoring session.
76
:With the CFO of hp, Bob Leman.
77
:The whole idea was, I want you guys to
talk you from your end and he from his end
78
:and see if we can't get to a meeting of
the mines between marketing and finance.
79
:This whole experience it just changed me.
80
:And before he passed away, I had
the opportunity to talk to Mark
81
:again and thank him for it, right.
82
:Because it was, not fun at all,
but he did me the hugest favor.
83
:So I started climbing the
analytics ladder, right?
84
:The stairway to heaven, so to speak.
85
:Right?
86
:By the time I was CMO at Honeywell
aerospace, pre any kind of
87
:automation for analytics, right?
88
:meant we had to hire a ton of people
in order to get the latency on the
89
:recalculation down, the scalability up
the cost be damned at this point, right?
90
:'cause it was so important.
91
:And get the understandability of
the outputs to the point where the
92
:business goes, yeah, I not only get
it and believe it, but I can make a
93
:better decision today than I could
make before as a result of this.
94
:And then you kind of say, there's not very
many companies that are willing to invest
95
:seven, eight, $9 million a year just in
marketing analytics particularly in B2B.
96
:That point you start to realize
that, that automation was gonna
97
:be absolutely indispensable.
98
:To solving the underlying root problems,
if Data science has has a number
99
:of major problems in the business
context, but they can kind of be summed
100
:up by the fact that it's too slow.
101
:So by the time they get you the
insights, everything has moved on,
102
:you've already had to make the decision.
103
:The information is not comprehensible
many times by normal people.
104
:it's not scalable.
105
:So you end up with three or four
mega models that get updated
106
:once, maybe twice a year.
107
:And so not agile, right?
108
:But big opportunity to appropriately
automate and bring ai into that
109
:whole thing and really elevate
all of this to a new level of
110
:accessibility and meaningfulness.
111
:And so that's why we built proof.
112
:Justin Norris: And so I want to
contrast MMM and regression based
113
:analytics that you're describing with.
114
:What most B2B marketers are
probably familiar with, which is
115
:multi-touch attribution, that's what
pretty much everybody does today.
116
:What is the problem with this form of
attribution from your point of view?
117
:Or is there a problem?
118
:Is it still okay?
119
:Can they be complimentary?
120
:Mark Stouse: as long as it's accurate
data, there's absolutely nothing wrong
121
:with having a really good understanding
of the customer journey, patterns in the
122
:customer journey, things like that, right?
123
:And indeed, a lot of our customers
will include customer journey
124
:data in regression models.
125
:It's valuable.
126
:The problem is that it is not fit for
purpose for what it's being sold to do.
127
:We can just start with the idea of bias.
128
:There's a lot of bias in that data.
129
:There's a lot of bias in
the weightings of that data.
130
:So marketing teams set their
own weightings in advance.
131
:It is antithetical to the mathematical
principles, to say that you can
132
:use MTA data to optimize anything.
133
:Particularly spend
134
:Justin Norris: I just want to
unpack that to understand why
135
:Mark Stouse: mTA data is an effect.
136
:You're measuring an effect.
137
:not capturing.
138
:The cause of that.
139
:And more specifically,
the time lagged cause.
140
:So how are you going to optimize the
money that you spent the past at this
141
:point, based on what you think you're
seeing from effects in the present when
142
:you have no idea what the time lag is?
143
:Right?
144
:So there is this assumption,
that the time lag is near zero,
145
:but that's just so not the case.
146
:Mm-hmm.
147
:I B2B, the time lags can be extensive.
148
:What do I mean by extensive?
149
:Two, three quarters.
150
:we get into brand investments, it's
double that, exactly are you optimizing
151
:and how do you know which part of your
budget back in time you needed to optimize
152
:based on this effect in the present?
153
:Right?
154
:Justin Norris: If we take an MTA
scenario, just to make it like a
155
:little bit more clear, let's say
we have an opportunity and we have
156
:three people in the buying committee
that are part of that opportunity.
157
:And between those three people,
there were sacred brown numbers, 30
158
:different touch points that they had
with different marketing initiatives.
159
:And those could be like, they
downloaded eBooks, they attended
160
:webinars, they clicked on ads.
161
:And so the, traditional way is like,
we're gonna take those 30 touchpoints
162
:and we're gonna take that opportunity.
163
:Let's say it's a 3 million opportunity,
and we're just gonna divvy it up.
164
:And the way that we
divvy it up could vary.
165
:We may pick some touch points, is being
more influential or not influential?
166
:And I totally agree with you there,
that it's kind of like arbitrary that
167
:they give you different models that
you can choose from and it's like, pick
168
:the one you want, like on what basis?
169
:Mark Stouse: I don't think
that marketers understand how
170
:transparently untrue the premise of
MTA is to everyone else but them.
171
:Justin Norris: I wanna get to the
reality of the situation because this
172
:is the, the industry that we're in.
173
:Mark Stouse: The reason why it
went through adoption is that most.
174
:Marketers don't have enough knowledge
of math to be able to recognize
175
:when something doesn't work.
176
:And number two, it was presented to
them as a way of mining data that they
177
:already were generating in their stacks.
178
:So it was easy in quotes, right.
179
:To do this.
180
:And I've been around it long enough
and talked to enough marketers about
181
:it, everybody was shocked when they
went into their first meeting with
182
:MTA data, Thinking that they were
just gonna be greeted as conquerors
183
:and victors and to your earlier
point at the top of the show, right.
184
:It was anything but that.
185
:it was very innervating
for a lot of marketers.
186
:You can actually do it in
Excel if you only have to do
187
:it on a very limited basis.
188
:But if B2B would just adopt B2C analytics
and also I would just add market research
189
:priorities they would see a very rapid
transformation of their credibility and
190
:their precision and everything else.
191
:And.
192
:can always modify it as needed,
but the core principle is ready to
193
:go and has been a very long time.
194
:Justin Norris: So if I recap your
perspective as I understand it, touch
195
:points, the customer journey data,
what the person did and when that can
196
:be interesting, that can be valuable.
197
:The notion of then doling out credit
to those touch points for revenue
198
:outcomes, not mathematically,
statistically defensible.
199
:Mark Stouse: would just add this, right?
200
:MTA at the end of the day
is about pattern matching.
201
:It's about identifying large
scale, repeating patterns
202
:in the customer journey.
203
:And then the assumption is that if a
bunch of people are doing the same thing
204
:over and over again, it must be causal.
205
:That is a complete fallacy, the
same math is used for example,
206
:and in fact, pattern matching too
is used to study climate change.
207
:Some are causal, some are
absolutely not causal.
208
:They happen all the time,
but they don't mean anything.
209
:You can't just assume that
because, a thousand people do it.
210
:Okay.
211
:One of your channels, one of your feedback
loops, you know, it just keeps happening
212
:that ipso facto means that it's causal.
213
:Justin Norris: I've always had this
unease as well where you a great
214
:example from earlier in my career,
we had a self-serve SaaS app, what
215
:would be called product led today.
216
:And everybody that signed up for
that app got a welcome email.
217
:So from that point of view, if you're
looking at touchpoints, like the
218
:welcome email is very influential.
219
:It's very important but,
but everybody got it.
220
:So just because people got it and
some people opened it doesn't mean you
221
:could necessarily infer that that was
driving the outcome in a particular way,
222
:. So everything we've been discussing
is kinda like the current state.
223
:Let's contrast it now with MMM and
why does it not have these different
224
:shortcomings that we've described?
225
:Mark Stouse: Number one, let's start with
the shortcomings that it does have, right?
226
:It is very dependent, as is
every analytic of any kind.
227
:Very dependent on data quality, There's
that old saying about gigo, right?
228
:Garbage in, garbage out.
229
:Everyone is going to be
victimized by gigo equally.
230
:Beyond that though.
231
:It has a lot of major advantages.
232
:It is a lean data mathematical process.
233
:So you do not have to have big data
or even a lot of lean data order
234
:to run these models accurately.
235
:And that is actually a huge factor.
236
:One of the things that CDOs chief
digital officers or data officers are
237
:grappling with right now on private
LLMs is that they big data stocks.
238
:They don't have enough training
data and they don't have enough
239
:operational data to run private LLMs.
240
:Everyone's been so understandably
fascinated with what they can do
241
:with public LLMs that they haven't
really thought it through in terms
242
:of the limitations on private stuff.
243
:Right?
244
:regression does not have this problem.
245
:If you say, oh, I don't have enough
data to run regression analytics.
246
:not even remotely able
to do any kind of ai.
247
:And I think that a lot of c-suites
now Are starting to say, your
248
:creation and maintenance of a high
quality data pool that's relevant to
249
:your function is a core competency.
250
:And if you tell us that you don't have
the right data, you don't trust your
251
:data that's on you, The other thing is
you have to have some basic capability
252
:on the human side of the equation.
253
:So with proof, for example, we have
simplified the whole thing, dramatically.
254
:Automated it significantly
in the right places.
255
:you don't need a full-blown
data scientist to run it.
256
:You need a competent data
analyst and you really don't
257
:even need that person full time.
258
:That's probably a half FTE, right?
259
:Justin Norris: Could we give a
Fisher Price version definition of
260
:what MMM is like, just so people
can conceptualize it in their minds.
261
:Mark Stouse: we live in a multi-variable
world there's tons of potential
262
:causes everything that you control
and everything that you don't control.
263
:That's kind of a really
super easy way to bucket it.
264
:And these all have interactions with
each other across time and space
265
:that produce particular outcomes.
266
:Causally speaking this is a
probabilistic calculation.
267
:So outside of certain physical laws like
gravity, There is no deterministic outcome
268
:that's possible to determine, right?
269
:You'd have to know everything
that there is to know about what
270
:contributes to a particular outcome
to get to a deterministic answer.
271
:That's just not the way that
operates, particularly when you're
272
:talking about human behavior.
273
:So this is the same math that's
being used routinely to study climate
274
:change, to study epidemiology.
275
:To study economics, it captures time lag.
276
:So in the end, the report will tell
you historically the stack rank
277
:of everything that you control and
don't control, that's in the model.
278
:It's relative effect on this outcome.
279
:It will then forecast all that
into the future so that you can
280
:then make different choices.
281
:And because of automation,
we've sped that up to the point
282
:where you can run multi-variable
regression exactly like GPS.
283
:say that you're recalculating the model
every week, new data comes in and is
284
:presented to the model automatically,
it automatically recalculates.
285
:And you see how the present now
is comparing with the forecast
286
:that you have for the same period.
287
:if there's a growing delta, right?
288
:You see, hey, okay, I
need to make some changes.
289
:Stick with the GPS analogy.
290
:need to reroute.
291
:Or man, this is great.
292
:It's just totally tracking.
293
:And it will tell you how long, longer
it's going to take for you to reach your
294
:objective, your goal, your destination.
295
:So there's a countdown.
296
:Regression is a huge part of
the actual GPS that's on your
297
:phone, you use every day.
298
:And if you stop and think about
it for a second, most business
299
:questions, and indeed a lot of life
questions are navigation questions.
300
:Where am I?
301
:Where do I want to go?
302
:What's the best way to get there?
303
:I have enough time and resources
to achieve my destination?
304
:Am I gonna run outta
gas before I get there?
305
:I gonna run out of time
before I get there?
306
:Am if I have to be there at
nine o'clock for a meeting?
307
:And the GPS says, you're
not getting there until 11.
308
:gonna have to make some
choices, that's part of it.
309
:also captures all the headwinds
and tailwinds that may be speeding
310
:you up or slowing you down.
311
:So in many ways, it it gives you
the answer to your questions.
312
:Justin Norris: Take a practical example
along the lines of one of those questions,
313
:let's say a company spends a hundred
thousand dollars a month on display
314
:ads as an act of faith because those
can be notoriously difficult to track.
315
:They're not always resulting
in a click, but they could have
316
:impressions that could have an impact.
317
:So they're spending that a hundred
K month over month, and the CFO
318
:challenges the CMO and says, why.
319
:Are we spending this 100 K every month?
320
:What's it doing?
321
:And that's the question
the CMOs trying to answer.
322
:How would MMM, like what does it compare
that spend versus a particular outcome?
323
:And try to show a causal
relationship between them.
324
:Mark Stouse: yeah.
325
:Multiple variables, right?
326
:So it's highly context oriented.
327
:All of these models are.
328
:Seek to capture as much of
a known context as possible.
329
:what a lot of marketing teams figure
out using exactly that scenario,
330
:is they've been trying to justify
that display ad budget in terms of
331
:demand gen, in terms of, performance
marketing, And that is just not the
332
:way that that typically plays at all.
333
:That is a brand reputation investment.
334
:know, you kind of think about
marketing expense as being
335
:essentially two big chunks, right?
336
:Branded demand brand is easily two to
three times time lagged in its effect.
337
:Than demand is.
338
:It also sticks around a lot longer.
339
:The effects of brand investment
doesn't deteriorate very quickly
340
:unless there is a major scandal of some
sort, something like that, that all
341
:of a sudden breaches the trust wall.
342
:But absent that right, it hangs
around The halflife is quite extended,
343
:whereas the Halflife on demand
investment is highly perishable.
344
:Lasts maybe a couple months and
then it's gone, so the snows melt
345
:a lot faster with demand than brand.
346
:You have to then say to finance,
never discussed time lag with you
347
:before specifically, but we all know.
348
:That marketing takes
time to have an effect.
349
:That simple statement is insufficient
because if we don't know the
350
:time lag, we will never know.
351
:The ROI, so we now have the ability
to say that this investment over
352
:here doesn't really drive demand,
doesn't really pay off on this side.
353
:There isn't a quick return a demand
perspective, but from a brand perspective,
354
:this is how it is improving average
deal size and average deal velocity.
355
:Those are the two big ones that we see
again and again on brand investment.
356
:It is grease on the wheel of the deal.
357
:That's what brand
reputation really is, right?
358
:It makes people buy more than
they otherwise would and buy
359
:faster than they otherwise would.
360
:that That is key in B2B because
it's a higher cost, higher
361
:risk by decision to begin with.
362
:And then today, if we layer in all the
risk factors that people are worried
363
:about it's even more important,?
364
:I mean, If you really want to understand
how trusted you are by your customer
365
:base, look at your average deal velocity.
366
:If you really want to understand how
confident people are in your product's
367
:ability to generate massive amounts of
value for them, look at their average
368
:deal size, particularly year one, right?
369
:Certainly year two, your, a
lot of people, just as a matter
370
:principle today, are buying small
in year one and testing it out.
371
:If you see a major uptick in
year two and deal size, that is a
372
:major vote of confidence in you.
373
:you don't, you need to
really find out why that is.
374
:'cause I guarantee there's a reason.
375
:Justin Norris: Looking at our
example, let's say the dataset that
376
:you had, the company had always
been running those display ads.
377
:Do you need a period where those display
ads are not there in order to demonstrate
378
:the impact of having them or not?
379
:Or is there a way, even though they've
always been there, to somehow demonstrate
380
:the impact that they're having?
381
:Mark Stouse: I think that the answer to
that question is very much contextual.
382
:if we were talking about a correlation
analysis, meaning this ad buy right
383
:against revenue, Then yeah, you would
definitely need some interruptions
384
:to be able to essentially create a
385
:Justin Norris: Oh, control, that's
the word I was looking for too.
386
:Yep.
387
:Mark Stouse: If you're talking about
multi-variable though, where a lot of
388
:things are changing all the time around
it, it's not necessary that's actually
389
:one of the things that's really, really
cool about MVR multi-variable regression
390
:is that because you're bringing in so
many different variables and because
391
:everything changes, particularly the
stuff that you don't control you're gonna
392
:see exactly what you're talking about.
393
:You're gonna see implicit AB test.
394
:Evolving across time.
395
:Justin Norris: We're
talking about limitations.
396
:And one of the things we're talking
about was being an aggregate
397
:versus tracking individual users.
398
:Mark Stouse: Yeah, so one of the things
that's really important to say about
399
:multivariable regression slash go to
market, mixed modeling, et cetera, right?
400
:Is that it is looking at all the
causal relationships in aggregate.
401
:So it's not possible to say that all
these things cause this one identified
402
:person to take the steps that they took.
403
:That's not what go to market
mix modeling is all about.
404
:It's about looking at a population wide
trend that's causal for a lot of people
405
:one of the things that companies really
appreciate though about this is that.
406
:There's no PII in proof
or in, a regression model.
407
:So you don't have the security
concerns that you would have in
408
:MTA or any of the other touch based
approaches, which are trying to tie.
409
:Broad investments, programmatic
investments to impact on a particular
410
:human being or particular company.
411
:And to do that, you have to
pierce the veil on identity.
412
:And that's a problem.
413
:Justin Norris: So it's
looking at the big picture.
414
:So cookies, not an issue,
GDPR, not an issue.
415
:It's really comparing
bulk data sets and the
416
:Mark Stouse: In fact, the only GDPR
thing that we ever have to encounter is
417
:the login information for users of proof.
418
:So we keep that secured, But in
terms of the data that's in the
419
:computations, it's a non factor.
420
:It's just non issue.
421
:Justin Norris: Is there a way to
explain that multi-variate regression
422
:to someone who maybe they've got high
school math, first year university
423
:math, 'cause maybe that's a challenge.
424
:As you said, marketers, some of them
are more quantitatively oriented.
425
:Not all of them are.
426
:And they're gonna need to trust,
this data, so they'll need to be able
427
:to make sense of how it's produced.
428
:You know, If it's just a kind
of computer and it spits out an
429
:answer like, yep, do X or do Y.
430
:Mark Stouse: They're already effectively
thinking in this way because they're
431
:spreading out their investments
across all kinds of different
432
:channels, So they're acknowledging the
multi-variable world in which they live.
433
:What they are not necessarily
acknowledging is the fact that it's not
434
:a list of one-to-one correlations, My
investment in display ads versus revenue,
435
:my investment in social versus revenue,
my investment here versus revenue.
436
:That's not it.
437
:It's a tapestry that weaves back and forth
with different time lags associated with
438
:hmm.
439
:Literally the only way capture that
is with multi-variable regression
440
:or econometrics or, marketing
mix modeling, go to market mix
441
:modeling, right there, it's all
442
:the same thing,
443
:Justin Norris: the fir, the
first thing that you mentioned
444
:Is what we're conditioned to look for.
445
:Like we were conditioned to say like, I
invested a dollar here and it produced
446
:two, like these definitive statements.
447
:I think if I'm interpreting you
correctly, you're saying reality is not
448
:that straightforward most of the time.
449
:Mark Stouse: The final output
can be that straightforward.
450
:Okay?
451
:But the causality elements on
it, are going to vary with the
452
:wind, so a great example of this
is uh, Johnson Controls, right?
453
:So even before Covid really became a
story, uh, their analytics started to
454
:say in terms of forecasting, right?
455
:for reasons we don't understand.
456
:All these different channel investments
that we have historically made are
457
:showing that they're not gonna be as
effective in six months as they are now.
458
:A lot of these were physical events.
459
:, this was not due to marketing
data in these models.
460
:This was all about external data
was already up these early signals,
461
:they decided, you know what?
462
:We are gonna fly by instrument.
463
:We're not going to fly by what we can see.
464
:And so they started removing investments
early a lot of these things and they were
465
:able to claw back quite a bit of money.
466
:And then all of a sudden, real
life proved the analytics accurate.
467
:And so they were very happy about that.
468
:And then finance came later and said, Hey,
so sorry, we're gonna have to cut you.
469
:30 or 40%.
470
:I can't remember exactly
what it was, but it large.
471
:And they said, let us show you something.
472
:Right?
473
:And they modeled the effects of those
kinds of cuts over the next three
474
:years, and it was so detrimental
to the business that finance took
475
:a lot less and went elsewhere to
get whatever they needed, right?
476
:So this is an example, not only of
how a marketing team used it to avoid
477
:waste, but also to avoid cuts that would
be ultimately bad for the business,
478
:Justin Norris: and the sort of external
data you just described, those are things
479
:that's not specific to any one business.
480
:It's common to the macro environment
that many businesses are working in.
481
:Like if I come to you, do I need to
bring that external data on my own?
482
:Or do you have sort of global
external data that you can match
483
:up with my business specific
data and include in the model?
484
:Mark Stouse: So we are not a
data provider, but we can and do
485
:all the time help people locate
data sets that are usually free.
486
:That kind of stuff is usually
free, either from the government
487
:or major financial institutions
or universities, things like that.
488
:So that's not a problem.
489
:And it's great.
490
:Usually just fantastic data.
491
:I mean, It's been totally scrubbed,
so that's not a big deal at all.
492
:In fact, we live in the golden
age of data availability.
493
:Even if it's not free,
someone is measuring it.
494
:Purely speculatively in the belief
that somebody is gonna wanna buy it.
495
:And so today with a credit card,
you can subscribe to all kinds of
496
:data sources, cost effectively.
497
:Particularly given how important
it is to maximizing the investment.
498
:The other thing that I would
just point out is ROI is really
499
:important, but you only know RROI.
500
:Looking backwards, right?
501
:ROI is a historical assessment.
502
:What is really important is
forecasted, ROI, and then the
503
:comparison between it and actuals.
504
:So again, this is very much like
public companies issue guidance
505
:and then they issue regular
updates against that guidance.
506
:That is exactly what the C-suites of many
companies are, demanding today, right?
507
:It's like an investment
uh, to start a new company.
508
:The first question the investor's
gonna ask is, what do you expect
509
:this to do in year 1, 2, 3, 4, 5?
510
:And what's the basis for that projection?
511
:If you're just extrapolating from hope
and best wishes and all that kind of
512
:stuff, that's not much of an argument.
513
:But if you're doing regression based
analysis, that starts to mean something.
514
:Justin Norris: And listening to the
examples that you described, it does feel
515
:like a very enterprise oriented solution.
516
:If I work at a company, 400
people, 50 million a RR, is this
517
:something that can work for us, or
do I need to be of a certain size
518
:threshold for it to be useful?
519
:Mark Stouse: No, actually
it totally scales.
520
:The reasons for investing in it are
going to be different in a small,
521
:medium, or enterprise type business.
522
:But it totally scales.
523
:And the cost is totally approachable,
even for mom and pop, right?
524
:So the reasons for doing it as
a small, and let's say the lower
525
:half of the medium sized business.
526
:Are mainly because if you make a bad
investment in whatever, It has a almost
527
:immediately negative effect on cashflow.
528
:Too much risk, so they are
modeling to avoid that.
529
:We do have some small customers and
that's their main reason for doing it.
530
:If you are an enterprise and
you're spending, I don't know, $200
531
:million a year on marketing, right?
532
:you spend it wrong, what the CFO
is most concerned about is things
533
:like opportunity cost, right?
534
:That's the comparison that's going on
in their mind all the time, particularly
535
:today, is what are all the different ways
that I can spend this dollar and what
536
:are the most effective, or what's most
likely to give me the biggest impact?
537
:That's the competition.
538
:That is the Game of Thrones,
particularly right now in budget season.
539
:budget season right now is
sort of year round now, right?
540
:Because everything is so pressed.
541
:So if you are a marketing leader, you
are in competition to show that money
542
:spent with you is a better return.
543
:Than if they spend it in
it or HR or whatever, right
544
:you also need to really understand
this, you marketing by definition,
545
:is a non-linear multiplier of areas
of business performance that are
546
:linear, one of which is sales.
547
:So in simple language, your leverage,
you are bringing huge amounts of
548
:leverage to sales performance that
sales cannot create for itself, right?
549
:not a ding on sales, it's just the nature
of the reality of the situation, what
550
:do I mean by linear and non-Linear, okay.
551
:It means that if I were to go
to my CRO and double goal for.
552
:For 2024, the first conversation
that they're gonna want to have
553
:with me is about essentially
doubling their Salesforce, right?
554
:Because the relationship between
revenue coming in and the cost of that
555
:revenue in the form of sales team is
it's known it's a linear function.
556
:And that's because it's the
collective performance of a
557
:lot of individual performances.
558
:Okay?
559
:So a bell curve your sales team's
performance is gonna be on a bell curve.
560
:It's just fundamentally linear.
561
:The whole reason why modern marketing
was created in:
562
:to bring non-linear leverage to
that whole equation, even in B2C.
563
:So what does that mean?
564
:It means that because of the way
marketing is, were to have the
565
:same conversation with the CMOA,
we're doubling the revenue goal.
566
:Might have to increase marketing spend
by 25%, maybe 20%, there's already
567
:a ton of leverage built into it.
568
:And if you want an easy understanding
of ROI for marketing, it is the
569
:extent of that multiplier, so
we'll talk about it this way.
570
:You are basically saying that
marketing's mission is to help
571
:sales more product to more people.
572
:That's revenue faster.
573
:That's cash flow from
revenue more profitably.
574
:That's margin impact than
sales could do by itself.
575
:That's the key phrase.
576
:The extent to which that is
true is the ROI to the business.
577
:So does that actually
look like in real life?
578
:Well, and a lot of really,
really great B2B marketing go-to
579
:market kinds of operations.
580
:ratio is somewhere between 10 and 20 x.
581
:So that means that if you take marketing
away entirely, which I think would be a.
582
:Something that not even the most draconian
CFO would ever contemplate, but let's
583
:just say it is the ultimate AB test.
584
:So we're gonna completely
shut down marketing.
585
:You're gonna see a massive fall off.
586
:It may take a year, but you're gonna
see a massive fall off in sales
587
:productivity that is going to reveal
the extent of the marketing multiplier.
588
:It's like literally guarantee able because
sales create the leverage for itself.
589
:It's just not the way it works.
590
:Justin Norris: If a company is
getting started with this solution
591
:what would the process be like?
592
:What data sources would we need to bring?
593
:How much time does it take to build
the model, that sort of thing.
594
:Mark Stouse: the very first step is, and
this is part of the onboarding for us,
595
:is that we sit down with the customer
and we say, look, what are your top
596
:questions that you most want to know
The answer to this is a mixed audience,
597
:usually of marketers and business leaders.
598
:And doesn't matter what your job is.
599
:Nobody has any problem rattling
off their list of questions.
600
:Those questions then generate what's
called a model framework, could easily
601
:analogize that to a recipe card., So this
is gonna be a punch list of data types
602
:that you're going to need to be able to
supply to the model to compute the answer.
603
:then it's gonna outline the model
itself, algorithmically speaking.
604
:When you actually make the dish
from that recipe, that is the model.
605
:Usually the way this actually
works is it's very fast.
606
:It's usually a matter of
weeks, like less than a month,
607
:that kind of timeframe where.
608
:The analyst and the business user.
609
:Could be a marketer, could be
finance guy, could be whatever.
610
:Are collaborating in the tool
on a minimum viable model.
611
:And at some point business user
says, that answers my question.
612
:We need to put this model in production.
613
:You hit the big red button,
it goes into production.
614
:After that, it's pretty autonomous.
615
:There's kind of some DevOps type work,
you you know, you have to maintain the
616
:model and all that kind of stuff, right?
617
:But for the most part, it is
doing its thing on an automated
618
:basis and you're getting whatever
cadence is right for your business.
619
:Daily, weekly, monthly, you're
getting an update on demand.
620
:Justin Norris: the.
621
:. Analyst that you just described, is
that someone that you're supplying
622
:from your team or is this kind
of a bring your own process?
623
:Mark Stouse: we We have a large partner
ecosystem that, we can recommend from.
624
:We also occasionally can do it ourselves
on a managed service basis that is
625
:actually increasingly popular as
teams get thinned out and they know
626
:that they need this capability, but
they don't have the bandwidth or the
627
:expertise to manage it internally.
628
:And so that's actually extremely popular
these days and very cost effective.
629
:the key thing is that there's lot of.
630
:You could easily spend 2020 5K
upfront in time, not in licenses.
631
:Okay.
632
:But in time to get everything
set up but once you've done that,
633
:right, again, the models, unless
you need more models, right?
634
:The models are doing their thing, right?
635
:You don't have ongoing major, investment,
know, every month gotta redo the
636
:whole thing from the ground up.
637
:That's, exactly the kind of thing
that proof was built to eliminate,
638
:Justin Norris: if you add a
new, you add a new channel, do
639
:you have to update the model?
640
:Or the model can be flexible
enough to just say like, oh, you're
641
:doing LinkedIn advertising now.
642
:That's fine.
643
:We can just incorporate that as we go.
644
:Mark Stouse: you could
do it either way, right?
645
:I think be the best practice is that
you clone the model that you have and
646
:then you add the new data streams to it
that you're maintaining the integrity
647
:of the original model for comparison.
648
:And yet you are updating
it that way, right?
649
:And then at some point you're gonna
dispense with the oldest version
650
:of the model altogether, So it's
not like you just see a massive
651
:proliferation of models across time.
652
:But you do need to do this in
an orderly way so that everybody
653
:doesn't lose reference points.
654
:Justin Norris: you mentioned
competitors and I've seen competitors
655
:popping up in the market, even . A
multi-touch attribution vendor
656
:that's adding MMM to their mix.
657
:What's your outlook on how easily other
vendors could recreate these capabilities?
658
:Do you feel that you have a fairly strong
competitive moat around the offering
659
:you've built, or will other people be
able to jump into this environment and
660
:offer similar things fairly easily?
661
:Mark Stouse: So it's really important
to say this, the competitive advantage
662
:that anybody has is not in the math.
663
:So if people propose that they have
some kind of Super Cal Flagal algorithm,
664
:you need to be very careful about that.
665
:' cause the odds also are that it's
not transparent, it's a black box.
666
:And also.
667
:If it is transparent, you probably
don't have the ability to evaluate it.
668
:In our particular case, the IP that really
matters is in how we automate it, how we
669
:scale it, how we make it consumable and
approachable and understandable and how
670
:we do it at a particular price point.
671
:And I think that this is the other
thing that is highly relevant today,
672
:not just in this area, but across
SaaS, is that prices are coming down.
673
:Uh, We're gonna see a fundamental
change in SaaS pricing.
674
:The days of, you know, every year
having a pricing increase are over.
675
:It's just done.
676
:We are very well fixed
competitively speaking.
677
:Do have a, I think, a really solid moat.
678
:So there are a lot of competitors today
that have some great products, but these
679
:are products that were conceived of
and written by and for data scientists.
680
:and, And a lot of 'em
are gorgeous by the way.
681
:Like, graphically, they're gorgeous, but
if you expose them to a normal business
682
:user, a marketer, sales leader, whatever
gonna stare at that screen and go, I have
683
:not a clue in the world what that means.
684
:Like how do I make a better
decision based on that?
685
:You've just now thrown a
lot of friction into it.
686
:It's taking more time.
687
:You've gotta spend a lot of time
translating the data science outputs
688
:into something that is usable.
689
:And we don't do that, we built it with
the end user in mind violating any.
690
:Data science principles.
691
:That's our big advantage.
692
:Think one of the biggest things I can
say about this, just to sum up, is
693
:that there's a reason why large B2C
marketing teams, CPG uh, hotel and
694
:hospitality, retail, whatever, right?
695
:They control all four pss of marketing and
they have, at worst, a defacto authority
696
:over and responsibility over the p and l.
697
:Of their product with their marketing.
698
:There's a reason for that, they are
using econometrics slash Go-to market
699
:mix modeling slash MMM, using that to
optimize and to understand causality
700
:and to optimize based on that causality.
701
:And they're also investing a ton of
money in market research, which a lot
702
:of that ends up in the models, right?
703
:So if B2B marketers want to have the
same attributes their B2C brothers and
704
:sisters, gonna have to do what B2C does.
705
:This is one of those situations where it's
like gravity, You can disagree all you
706
:want to with gravity, and if you throw
yourself off a building, it's not gonna
707
:end well, the same thing is true here.
708
:This is a mathematical principle at work.
709
:It's mathematical law of gravity
with quotation marks around it.
710
:And so everything that I've
said today, I've, and I really
711
:try really super hard to do.
712
:This is not my opinion.
713
:I'm just representing a level of fact
that people can either accept or reject,
714
:but it doesn't change the truth of it.
715
:Justin Norris: We will include a link to
your website so people can uh, check it
716
:out, learn more, look at your resources.
717
:And I'm excited to see where this goes.
718
:Mark, thank you so much
for chatting with me today.
719
:Mark Stouse: You're welcome.
720
:Thank you.