The Transcript

5 Key Concepts to Understand About Tracking, Testing, and Making Decisions

Voiceover: You’re listening to The Digital Entrepreneur, the show for folks who want to discover smarter ways to create and sell profitable digital goods and services. This podcast is a production of Digital Commerce Institute, the place to be for digital entrepreneurs. DCI features an in-depth, ongoing instructional academy, plus a live education and networking summit where entrepreneurs from across the globe meet in person. For more information go to rainmaker.fm/digitalcommerce, that’s rainmaker.fm/digitalcommerce.

Brian Clark: Hey there everyone, welcome to The Digital Entrepreneur. I am Brian Clark, founder and CEO of Rainmaker Digital. Today, as always, we have Jerod Morris, VP of marketing for Rainmaker Digital. We also have our very own data ninja, Loryn Thompson. Don’t ever call yourself a ninja. We’ve talked about this. I can call you a ninja, right?

Loryn Thompson: Exactly.

Jerod Morris: Yes.

Brian Clark: Okay, this is a podcast, you have to talk. Today we’re flipping things around a little bit to close out our series on the elements of the modern website. We’re going to talk about testing. Instead of me rambling on about it, I’ve decided to flip the switch and let Jerod and Loryn talk about testing. As we came out of our launch and build phase — which, when we launched Digital Commerce Institute in the fall, that was the last piece of the puzzle. We don’t plan on building anything brand new or launching something new. What we’re doing now is shifting into our ‘optimize and grow’ phase.

A big part of that initiative that we knew we wanted to tackle as soon as we could — it was at the beginning of the year — was to move from tracking and testing when we thought it might be a good idea, to a culture of tracking and testing. There’s a big difference there. A one-off test here or there can give you valuable information. A process, a system, a culture that basically tracks and tests everything and then tries to make sense of the resulting data and apply that back into our own intuitive creativity, that’s where we needed to go next. That was really what I tasked Jerod and Loryn with, so who better to talk to about it than these two. Are you guys ready?

Jerod Morris: We are ready. Yes.

Loryn Thompson: Yup, ready.

Brian Clark: All right. I’ve got here some notes from you two. This catchy headline, I wonder where you got that from.

Jerod Morris: I know.

Brian Clark: “5 Key Concepts to Understand About Tracking, Testing and Making Decisions. Very nicely done, Jerod was that you?

Jerod Morris: That was me, yes, but with Loryn’s help.

Brian Clark: Good job. Oh, with Loryn’s help, okay. We know Loryn’s the brains behind this but you’re the front man. She’s got to feed you the smart stuff.

Jerod Morris: Right, and I do want to say, Brian, as the lead-in to this conversation — when we talk about data and testing I think it’s so important, as you just said, to create a culture of testing. It’s so important to have two things, number one you’ve got to have commitment from the top. Because if you aren’t committed to this and to doing this, then something like this isn’t really going to happen. I think you’ve really got to have organizational commitment to this being important.

The second thing I think you’ve got to have is enthusiasm. I think that’s really what Loryn has brought, a real enthusiasm for the subject, for learning more, for figuring out where we can use it. It’s really helped and infused what we’re doing with a lot more information and insight. I think as we talk about this, this isn’t listed here as one of the things we’re going to talk about, but it’s very important if you want have this culture of testing. Having that commitment from the top down and then individuals with enthusiasm for actually doing the work that it will take is really important.

Loryn Thompson: Mm-hmm (affirmative).

Brian Clark: Yeah. You not only have my commitment, but I’ve been nerding out right along with you. I think we’re all enthusiastic about it because it’s fascinating.

Jerod Morris: Yeah.

Brian Clark: It’s fascinating when your intuition actually gets proven correct. It’s also just as fascinating when you’re completely wrong. You just have to be open to being wrong because …

Loryn Thompson: Whichever one makes you more money.

Brian Clark: Exactly. Blindly going along with a losing strategy is not that smart in my book. Yeah, I think the enthusiasm is there as soon as people get over their trepidation about it. People think split testing is complicated, but with the right tools it’s becoming easier and easier. Even I can do it when you let me.

Jerod Morris: Yes.

Brian Clark: All right, let’s go through these 5 key concepts. What is number 1?

The Best Time to Start is Now

Jerod Morris: Number 1 is starting now. Again, this is one of those that almost seems so simple you don’t want to say it, but it’s really important. It’s similar to what we tell people on the The Showrunner with podcasting who see that there are people who’ve been podcasting since 2008 so they feel intimidated. It’s like, “Yeah, okay. The best time to start a podcast was back then — 2007, 2008 — but the next best time is now.” Same thing with a blog.

Think about if you haven’t been working out or if you’ve gotten out of shape. Sometimes you’ll think about it and it’s like, “Man, I don’t even want to get on the treadmill for the first time because I haven’t been doing it for so long.” You can’t think that way, because you can take a positive step in that direction today to start moving forward in a better way.

I think it’s the same thing with data. Maybe you’ve had analytics on your site for five years but you’ve never done anything with it and now you get in there and you’re intimidated because there’s all this old data and you don’t know what to do with it. Start today. Take a step today to understand something better or to set up just one simple test to get going.

You’re going to be a lot further along in six months if you take that first step today than if you continue to lament the fact that you haven’t started yet and continue to allow that to keep you from getting going. Again, it’s basic but it’s an important mental hurdle for some people to get over. Whatever you’ve done in the past, it’s okay. Just start today moving forward in a better way.

Loryn Thompson: Yeah, I want to add to that. Even if you don’t think that you really need the data right now or anything, or now is not the best time to devote the energy to putting everything together. It can be really time consuming to get set up. tThe important thing to remember is that with data — having the historical data, having three months of data — that puts you way ahead already. It’s always good to just go ahead and get it set up, even if you think, “Oh, Well I need to devote more time to this right now.” Just put the data in place. You’ll be glad you did in a few months.

Jerod Morris: Mm-hmm (affirmative).

Brian Clark: Yeah, that’s good advice. The next thing I want to talk about here is something I’ve seen over and over. When a company decides, “Okay, we’re going to embrace testing. We’re doing this.” The first thing they then do is look for some turnkey off-the-shelf system that they’re going to slap onto their particular business. Silly or not?

Context is Everything

Jerod Morris: I think the number one thing that you have to realize when you’re going to start doing this is you’ve got to understand what you’re tracking and what data and what reports and what numbers are going to be important for you. I think back to my time when I was running Midwest Sports Fans five or six years ago and all of our monetization was based off of display ads.

For Midwest Sports Fans the page view number was huge. Having Google Analytics driving page views — that’s how we were going to drive revenue, because we weren’t doing anything else with it. But you think about what we’re doing now and think about all of our properties — page views are a nice number and they can tell us something, but it doesn’t tell us very much. We need to convert people onto an email list. We need people to start trials of the Rainmaker Platform.

You’ve really got to start with where you want to end up. What your business objectives are, what numbers drive those objectives. And then what’s going to allow you to track that. The context is everything. I think that’s why you really need to start thinking there and that will help tell you what you need then to be tracking and what tools will help you do that.

Loryn Thompson: Definitely. One thing, when you start looking at that data it’s really useful to start breaking that down into different kinds of segments. You had one session number, you got ‘X’ many sessions last month. That’s great, but what does that actually mean? Maybe you start by looking at people who came from a certain email, or people who landed on the home page, or people who landed on a different page. Looking at how their behavior compares to behavior of people in other segments on this site — breaking that down and figuring out what those sessions mean will get you a lot more mileage out of your data.

Brian Clark: Mm-hmm (affirmative). Yeah, it’s interesting. I think page views are completely irrelevant to a content marketing-driven business. With the state of online advertising they may before irrelevant to everyone. But there are certain key metrics that I think anyone who’s following along with this series … Converting the email, that’s key. Converting to customers based on various adaptive funnels that you create — the kind of stuff that we’re going through here with this series. These are all the things you have to not only see what approach works best, but then you’ve got to be able to tweak and optimize. You can’t do that blindly. You don’t just decide to change a word, you see what word works better.

Jerod Morris: Yeah, exactly.

Loryn Thompson: Yeah.

Understanding What Your Tools are Telling You

Brian Clark: All right, what about tools? People get a little bit hung up on that. What’s your advice about understanding how to use this technology to best effect?

Jerod Morris: Yeah, this is such an important one. Loryn, this is where I’ve learned a lot from just in the time that we’ve been working together. Especially this concept of, “your tools are telling you certain things, but there are also things that they are not telling you.” It’s very important to understand this. Because you can be getting this number, but if you don’t know where this number is coming from and what data is coming in to give you this number, there could be giant glaring holes that mean that it’s painting you an incomplete picture. Again, I’ve learned a lot from you in this sense. And you’ve got a couple of good examples as far as this goes which help illustrate that.

Loryn Thompson: Yeah, for sure. One of the main things is that looking at website data, you really do want to focus on users. It’s nice to know how many sessions you have, how many page views, but really what you’re looking for is when people get to your site, what are they doing? One of the biggest limitations with Google Analytics — which is a super popular website analytics tool — is that the tracking is cookie-based, which pretty much the best we can do right now.

But that means that each user, when you look at that user’s number, it’s only recognized with the exact same browser and device configuration as they were the first time. If you are on Firefox and go to our site one day, and then the next day you go on Safari, Google Analytics isn’t going to know that you’re the same person. That’s why it’s really important to do things like … There are a few things you can do to your website like user ID tracking.

Whenever you look at that user number if you don’t have user ID tracking in place it’s likely over inflated. My best suggestion with things like this is you can’t always fix these huge holes, you just have to be more aware that they exist. As you’re moving through the data if you see something that indicates that, “Look, we had so many more users this time,” it’s, “maybe people are switching from their desktop to their phone to access our site because we started putting more podcasts on,” or something like that.

One of the other things that I want to touch on is that there are a lot of platforms that will give you proprietary metrics that they don’t really tell you how they came up with them. They’re like, “This is the magical impressions reach engagement number that you need to be paying attention to. It’s fantastic.” Okay, what does it mean? If you only have that number and you don’t really have anything to compare it with, then it doesn’t really do anything to help your business. I personally would never rely on any metrics that I don’t know how they got to be there, how they were counted.

Jerod Morris: I think what’s important there is it doesn’t necessarily mean that the metric itself is bad. It could be good, but for you to make an informed decision on it you have to know what it means. It’s very important that you can wrap your arms around it, embrace it, understand what it means so you can use it to make a smart decision.

Loryn Thompson: Exactly. A lot of these companies it’s like a whole ‘business secrets’ thing, and that’s fine. But if you don’t know what’s behind it then how do you know what that number’s limitations are? How do you know what it’s really getting at if you can’t know how it’s calculated?

Jerod Morris: Yeah.

Balancing Qualitative and Quantitative

Brian Clark: That’s interesting. Okay, so everyone is obsessed with big data, the quantitative side of things. Is there still room for qualitative data?

Jerod Morris: Yes, and you need it. I think you can fall into that trap when you start talking about data and start talking about testing. That all you do is look at things from a numbers perspective. Numbers can tell you a lot. They should tell you a lot. But that doesn’t mean that asking questions — having actual conversations — doesn’t matter.

We started putting up a survey on the Rainmaker Platform site to get some information on people when they left a certain page to find out why they were or were not doing something, and we’ve gotten some good insight from that. Now that, obviously, gets boiled down to into data. But it’s not just tracking how someone comes and goes from a site, it’s asking them a question. Brian, just 15 minutes ago we got off the call and we were talking about reaching out to some people to get a testimonial. In that email that I sent out to a few folks I asked them, “Hey, how are things going in Digital Commerce Academy so far?”

I’ve already gotten three replies with people telling me a little bit about their experience and things they liked and things that they wish that it had. It’s really good information, because we’re reaching out to our most active people. Are we going to jump and go do something with that? No, but it’s the kind of information — hearing it from people who are clearly engaged...