Podcast

Five marketing analytics myths debunked

These five marketing analytics myths are so common, they get asked and answered without much consideration as to whether or not they’re the right kinds of things to be asking. In this podcast, we bust the following myths that every marketing leader should know about:

Myth #1: If I’ve got an analyst and I’ve got Tableau, then I can check the box. I’ve got marketing analytics covered.

Myth #2: Display media click-through rate is a metric that matters. It’s a metric that I should be concerned about.

Myth #3: So I’m trying to come up with a budget for what I should invest in analytics, and I’ve decided that I can benchmark it based on what I spend in my media budget.

Myth #4: Industry benchmarks are a good way to measure our performance.

Myth #5: I’ve got big data and I’m doing good analytics, that will provide certainty in predicting outcomes of my, for my business.

Listen here (or subscribe on iTunesStitcherGoogle Play, Google PodcastsPocket Casts or your favorite podcast service):


Welcome to Callahan’s Uncovering Aha! podcast. We talk about a range of topics for marketing decision-makers, with a special focus on how to uncover insights in data to drive brand strategy and inspire creativity. Featuring Jan-Eric Anderson and Zack Pike.

Jan-Eric:
Hi! My name is Jan-Eric Anderson and I’m head of Strategy at Callahan.

Zack:
And I’m Zack Pike, Head of Data at Callahan.

Jan-Eric:
Thanks for joining us on the podcast today. We’re gonna change things up a little bit today. We’re calling this podcast our first ever episode called, “Myth Busters.” So the format we’re gonna try today, we’ve each got a couple of myths that we’re gonna throw out to the other person and just talk about this. And these myths come from questions or tasks that we are given in our day-to-day experience with our clients. And we think that they are, they’re things that are worth challenging or things that are maybe relevant to you and your day-to-day. That’s our hope anyway. And these are things that maybe should be challenged a little bit, but because they’re so common, they get asked and answered without giving much consideration to whether or not they’re the right types of things to be asking. So we’re gonna give this a shot and I’m gonna kick this off. Zack, I’ve got a myth for you to bust. I’ve heard from CMOs that they’ll say that, “If I’ve got an analyst and I’ve got Tableau, then I can check the box. I’ve got marketing analytics covered.” What do you think about that?

Zack:
Well I think that if you have an analyst and you have Tableau, you are set up to do reporting. Every marketing organization has an analyst and a data visualization tool so Tableau software is one of those tools. It could be a host of any of them but on many of our previous podcasts and kind of our whole way of operating is that, those are two very important pieces but it’s often not enough, right? Think about, Tableau is nothing without the data to visualize-

Jan-Eric:
Yeah, you’ve talked about that before. Tableau’s the visualization software.

Zack:
Exactly. And your poor analyst, if they’re sitting there with only Tableau, think about where all their data has to sit. It’s sitting in excel files on their machine. They’re pulling it out of PowerPoint, it’s like all over the place. So, like I said, that sets them up to do reporting really well. So you’re getting your reports every week or every month and they look nice but what that person’s going through on a daily basis is probably a lot of munging and crunching of disorganized data, especially if they’re in the marketing space, pulling data from all these different systems.

Zack:
So to really do analytics, one, we’ve gotta think about our objectives. Am I just trying to do media reporting? Because if that’s the case, I’ve got just a couple of sources and I might be able to get by with Tableau. Tableau alone and one analyst. But if I’m tying to do real analytics, trying to figure out what is happening, why its happening, and how I actually impact it, I’ve gotta think broader than the person and the visualization tool. I need to be thinking about a platform to store all that data into, how I’m gonna get that data into that platform, and then the biggest one is probably do I have the right people to use it? And analyst can mean a lot of different things, right? Someone who’s really good at Tableau, it could be someone who’s good at managing all of the data. But I need someone who can think. And so, the whole thing here is that yes, analytics can mean a lot of different thinks, but Tableau and a person probably isn’t cutting it for you.

Jan-Eric:
It’s a good start but probably not getting you all the way to truly being able to do analytics.

Zack:
Exactly, exactly.

Jan-Eric:
Alright, fair enough.

Zack:
Makes sense?

Jan-Eric:
Yeah.

Zack:
So I’ve got a myth, this is one, we might’ve talked about on the podcast before, but it seems to come up all the time in marketing. So this is the myth: Display media click-through rate is a metric that matters. It’s a metric that I should be concerned about.

Jan-Eric:
Well, that is definitely something that comes up a ton. I would say majority of the time, no. Click-through rate doesn’t matter. It’s not to say that it never matters, but typically it probably doesn’t if you really boil down to it. Twenty years ago, it did. It was kind of a thing you could measure when it came to online advertising. You looked at click-through rates as an indication of people seeing your ads and caring about what the message is and it was an indication that people liked what you had.

Zack:
So why doesn’t it matter now?

Jan-Eric:
Well it doesn’t matter as much now. Unless, I guess, to answer your question, let me talk about when it does matter. It matters in a campaign when the action of clicking on a banner is the single most critical thing that you’re seeking to accomplish in your campaign. And rarely, rarely have we built campaigns where we say that all of our success hinges on whether or not somebody clicks on a banner. So to be looking at click-through as the metric is really chasing a false front. If we’re trying to drive store traffic, I oughta be tracking store traffic rather than click-throughs.

Zack:
Sure.

Jan-Eric:
If I’m trying to track brand affinity or likeability or awareness, I should track that, not click-throughs. So that’s where the issue becomes, when we’re looking at click-through rates as a metric that matters. There’s another fundamental issue at looking at click-through rate anyway. If you look at an industry average on a click-through rate, industry average on a click-through rate’s gonna be less than one percent. So what we’re talking about is trying to optimize and understand a benchmark on something that, out of the gates, has more than a ninety-nine percent failure rate.

Zack:
Right.

Jan-Eric:
And that’s not even taking into account the click fraud that we’re also simultaneously trying to uncover and determine whether or not the clicks are even legit. So it seems as though there’s so many issues at the core of clicks, the nature of clicks, and whether they’re even important or not. It just feels like click-through rates, in most cases, is not going to be the metric you really wanna track, but it’s something we’ve done for a long time-

Zack:
Right.

Jan-Eric:
And so it ends up being a thing a lot of the time. And then we’ll sit there and say, “Well, our click-through rate was fifty percent higher than the industry average, which means that we’re up to a point eight percent click-through rate,” and that just seems like there’s a lot of failure there. It doesn’t really seem like we’ve achieved much.

Zack:
Yeah, a big piece of it is being honest about the medium that it is. It’s display media and it should be used in a certain way. And, the thing that I always throw out is if I am talking about any other measurement quantitatively, and I sat that ninety-nine and a half percent of the time, something does not happen-

Jan-Eric:
We’d be saying, “What’s wrong?”

Zack:
Right, or I would just say, “It never happens.”

Jan-Eric:
Yeah, right.

Zack:
Right? If we don’t get clicks ninety-nine and a half percent of the time, we just don’t get clicks. Right? So think about that medium and what it’s used for which is awareness and eyeballs and stuff like that, and evaluate it from that perspective rather than, you know, the ten people you drove to your website, and half of those were bots anyways.

Jan-Eric:
Right, right, alright.

Zack:
Alright, it makes sense.

Jan-Eric:
Lets go to another myth. So I’m trying to come up with a budget for what I should invest in analytics, and I’ve decided that I can benchmark it based on what I spend in my media budget. So, here’s the myth: I should spend one percent of my media budget on analytics.

Zack:
Yes. That is, I mean, if you read about this subject, that is everywhere. The one percent thing. And, like it’s, this is a hard one because it’s partly true, right? From our perspective, if you’re spending between five and ten million dollars, one percent of that spend from a media or marketing perspective is probably putting you in the range of what you need to be spending or thinking about spending on analytics. But the problem comes in to play when you’re on the low end. If you’re under five million, if you’re spending a million dollars or half a million, you still need analytics.

Zack:
But you’re not gonna be spending five thousand dollars to get that analytics package. We’ve talked about Tableau software earlier. That tool alone is a thousand bucks a year. So now you’ve got four thousand dollars left over to, you know, for the year to either pay someone or a partner or something. So on the low end of that spectrum, you’re gonna be spending a larger portion of your budget, and on the high end, if you’re up in the fifty million dollar range, you’re not gonna be spending one percent on analytics. The best way to go about it is to think about what you’re trying to accomplish. Am I just worried about making the media as efficient as possible? Is that my only concern with this objective? Because in that case, my spend might be really low if I’m just worried about efficiency of how the dollars are being spent from a, in the media space.

Zack:
But in reality, we’re trying to drive business. That’s what we are always focused on. I want dollars coming into the bank account. I don’t care about the impression. And really, I don’t care how efficient it is as long as it’s producing the dollars that warrant the spend that I’m making. And so if we focus on, “What are the business outcomes I’m trying to get?” That gives you two points to kinda measure yourself against. One is just the raw cost of accomplishing those objectives. If I’m trying to understand full business impact and I’m taking into account things like pricing and distribution and things like that, that is a much broader look at the business, which warrants a higher spend because it’s not just marketing anymore. The other side of that is it lets you gauge that cost against how much money’s actually being produced. If I’m doing this analytics on a two billion dollar business, then the hundred and fifty or two hundred and fifty thousand dollars I’m getting ready to spend in this data world, you know, its a little bit easier to swallow than if I’m looking at that up against my one million dollar media budget because that’s just one piece of the full puzzle.

Jan-Eric:
So it seems like establishing a benchmark or establishing a budget for analytics should, it’s less relevant to be as a percentage of your media spent, cause there’s gonna be huge variants there.

Zack:
Right.

Jan-Eric:
You could put yourself in a situation where you’re fiercely under-budgeted given what you’re trying to do, or you could be way over-budgeted for what you’re trying to do. So, you don’t want to be based on that because of the variability. You really gotta price based on what you’re trying to accomplish and then gut check that based on how much money, revenue, you’re looking to analyze and is the “juice worth the squeeze” if you will. What should you be spending as an appropriate level?

Zack:
Right, makes sense.

Jan-Eric:
Absolutely.

Zack:
Alright, I’ve got another myth. Industry benchmarks, we hear this all the time, we get questions on it all the time. Industry benchmarks are a good way to measure our performance.

Jan-Eric:
Well, industry benchmarks … No, the answer is no, in short. Industry benchmarks don’t make any sense.

Zack Pike:
Industry, what’s the industry?

Jan-Eric:
It’s so generic, it’s so non-specific, I don’t even know what to do with it. I feel like that the question around industry benchmarks, and I don’t mean to be smug about this, but I feel that the industry benchmark question, when it comes up, is typically from someone who’s looking to cover their rear.

Zack:
Right.

Jan-Eric:
To give some sort of projection or some sort of point of reference that makes them look good. And it’s not doing anybody good in reality. Because it could be misleading, because these industry benchmarks are often times just an average of the status quo. They are average for, whoever’s reporting it, it’s average of whatever they have visibility to and it’s an indication of the status quo. It is no way, does it answer the question, “Where should we be?” Should we be better than the average? Should we be worse than the average? You gotta know what the average is or where it came from to be even able to understand that, and you don’t have any sort of baring on whether or not, how much above it or how much below it you should be.

Jan-Eric:
So its very difficult, again, I don’t know, I throw caution to the wind of how useful it really is. And you know, then there’s another thing that industry benchmarks are, where they fall way short. If you are trying to innovate, if you are trying to do something different, you are comparing to an industry benchmark that, again, is the status quo and the conventional wisdom or the conventional approach and how everybody else is doing it. So it’s completely irrelevant when you’re in a climate or in a culture that’s trying to push innovation. Industry benchmarks drive me crazy because I spend more time going and trying to find something and having it there as some sort of watermark, but frankly it’s completely irrelevant to what we’re actually doing. But I think it just makes someone’s stomach feel a little bit better having it so we just end up doing it and we get them regardless of what they really mean.

Zack:
Agreed, agreed. I’ve been in this data space for like, ten years, and I’ve seen a lot of reports with benchmarks on them. I have not seen a lot of reports where we are drastically below the industry benchmark. It’s also really easy for a vendor or a partner to pick out benchmarks that serve-

Jan-Eric:
Make the current work look really good?

Zack:
Exactly, exactly. It’s not hard to find. It’s not hurting anything if you have it, but it is if you spent four hours trying to find the right benchmark and the cost of all that time so-

Jan-Eric:
Right, and if you’re making big decisions based on how you compare to something, it’s just dangerous when it gets thrown out there. It’s dangerous how it can be used.

Zack:
Right, exactly. Agreed.

Jan-Eric:
Alright, I’ve got one more myth for you. And this kind of gets into how big data sometimes can be characterized as “The Magic Bullet.” So here’s the myth: If I’ve got big data and I’m doing good analytics, that will provide certainty in predicting outcomes of my, for my business.

Zack:
Yes, certainty is the key word. So, we are in this marketing world. And we’re dealing with humans, and humans typically make decisions in an irrational manner. That’s hard to understand. So that’s why we’re doing marketing. We’re trying to influence decisions, we’re trying to help someone understand that our product is a good product and it meets their needs and its easy to buy and it’s the right price. And then we hope that they’ll buy it. Trying to put certainty into that equation is extremely difficult, and so far, impossible. I mean if it was possible, we all would be paying that person who figured it out a lot of money. Every single one of us would be paying them because they knew the algorithm that predicts human behavior, which doesn’t exist yet. And so, this is coming from a data guy who would love to show certainty.

Zack:
But what we have to think about, this big data, analytics, all of this stuff, is we are focused on driving confidence. We’re trying to increase confidence in the decisions that we need to make or should be making. We’re not driving certainty. So when, forecasting is a really good example because you’re taking attributes of what you think are gonna happen across a business or a marketing campaign or something. You’re putting them into a quantitative type algorithm to spit back some type of number, whether that’s the effect of the media, the growth in business, you know, whatever the objective is. And even in that scenario, it’s an estimation. Based on the smartest way you can do it, it’s an estimation. And then you have the figure out for yourself how confident you are in that estimation. You should be working with your partners to do stuff like that.

Zack:
Where I start to get really upset is, you’ll see a lot of these companies, these big data providers, the tool sellers and manufacturers and things selling certainty, and unfortunately it just doesn’t exist in this industry. In areas where you’re not reliant on human decisions, the sciences, physics, manufacturing, stuff like that, yes, there’s certainty. Because the same thing’s gonna happen every time. But, you and I make decisions very differently. Right? We may look at the same thing, have the same background on it, and make different decisions just cause we’re different people.

Jan-Eric:
Yeah, yeah. These business decisions are, in many ways, are bets. They’re guesses. They’re risks in that regard. Good analytics reduces the risk, but this certainly does not guarantee that its a winning bet. It reduces risk and reduces assumptions and maybe gives you a little bit more confidence as you were saying.

Zack:
But in my eyes, that’s what makes this realm exciting, is you never really know what’s gonna happen. Right? You’re just trying to get smarter about it everyday and more confident. You know, there are a lot of very, very smart people working very hard in the stock market to try and use data to drive buying and selling decisions around stocks. And even today, it’s still really sketchy on how accurate it is. And you still have humans managing hedge funds and you have computers managing hedge funds and there’s not a lot of difference between the two of them.

Jan-Eric:
Yeah, well hey this was fun. Little bit different format than taking one topic. We broke it up. We might have to do this again sometime. Should be cool.

Zack:
Yeah, yeah, yeah.

Jan-Eric:
Well he’s Zack Pike, I’m Jan-Eric Anderson. Thanks again for joining us this week on the podcast.

 

Thanks for listening to our Uncovering Aha Podcast. Callahan provides data savvy strategy, and inspired creativity for national consumer brands. Visit us at Callahan.Agency to learn more.