Podcast

How guest experience reviews can be turned into measurable data

Guest experience reviews are critical to sales performance but those reviews are difficult to quantify. There is real value in online reviews and ratings extrapolated from places like Yelp or Google. When looking at variances over time and comparing with competitor ratings, it can uncover weaknesses, pinpoint low-performing stores and reveal insights that drive marketing campaigns. Zack Pike, vice president of data at Callahan, discusses the benefits in analyzing online reviews.

Listen here:


(Subscribe on iTunesStitcherGoogle PlayGoogle PodcastsPocket Casts or your favorite podcast service. You can also ask Alexa or Siri to “play the Uncovering Aha! podcast.”)

Welcome to Callahan’s Uncovering Aha! podcast. We talk about a range of topics for marketing decision-makers, with a special focus on how to uncover insights in data to drive brand strategy and inspire creativity. Featuring Zack Pike and Jan-Eric Anderson.

Jan-Eric:
Hi, I’m Jan-Eric Anderson, President at Callahan.

Zack:
And I’m Zack Pike, Vice President of Data at Callahan.

Jan-Eric:
Zack, welcome back to the booth.

Zack:
Yes.

Jan-Eric:
I wanted to talk with you today about a topic that’s been a recurring thing and it’s a recurring topic that we talk about with clients all the time and it’s a vague topic and I think that’s something that we need to unpack a little bit. So I want to pick your brain on the idea of how the guest experience in a restaurant category at a restaurant, impacts overall sales and how a company’s business is performing. This is a really important topic for us to talk about because you would be hard pressed to find any marketing leaders, CMO, CEO at a restaurant concept who would sit there and tell you that the guest experience isn’t critical to the sales performance within a store.

Zack:
Yeah.

Jan-Eric:
You’ve got this old assumed truth that the best operators, the operators who are running the best restaurants, the cleanest restaurants, the most efficient restaurants are the ones that are creating the best guest experience are the ones that are going to have the best sales performance. That seems to be universally accepted. Yet when you ask then those exact same people to quantify the benefit of having a good guest experience or quantify the suffering on the business that they’re experiencing due to bad guest experience, very few can actually pin it down. And so it’s I think, it’s been hard to respect the guest experience and the role it plays when it’s difficult to quantify what it actually is.

Jan-Eric:
So what I’d love to get your perspective on today and what I’d like to kind of talk a little bit more about is how a restaurant concept can go about taking their guest experience that they’re creating today and understand what its impact is on sales and their ability to drive sales in the future.

Jan-Eric:
So that’s the topic I really wanted to talk to you about and I think a good place to jump off is just to kind of get your perspective on how do we measure the guest experience and how do we even get our head wrapped around what the guest experience is.

Zack:
So you mentioned something about it being … It’s hard to relate this to sales, and I think it’s hard because it’s hard to get the data right? You’re kind of trying to take something that’s very qualitative and to be able to use it, you’ve got to make it quantitative.

Jan-Eric:
Sure.

Zack:
And that is always difficult. In marketing most the stuff we do is quantitative, right? We’re buying media and we’re driving website visits and we’re making sales. All those are hard numbers. So this stuff gets measured to the nth degree, right? We know how many little smiley faces we get on Facebook after our posts. But in something that’s, I think in my opinion, in a lot of people’s opinion, one of the most important pieces of a business, we don’t have that approach because historically the data’s been really hard to get our hands on.

Zack:
So the way that I’ve seen people measure this customer experience type thing, there’s a few different methods. One is a traditional like customer satisfaction surveys, right? You get a survey in your email after visiting somewhere, you might get a 10% off coupon for filling it out. You even do this, like in retailers, right? If you go to Home Depot, they circle the little thing on your receipt say, “Hey, take this survey. You’re entered for a chance to win a $50 gift card,” or something. So that’s one way. Typically when I’ve looked at that type of data, it’s answers to questions. That’s really hard to turn into usable, measurable data over a long period of time.

Zack:
The other way is there are companies that offer this type of service at a different level, right? They’ve got some form of app that you download and you can provide ratings and things through the app for different locations and even past that, sometimes you’ll see a little kiosk inside the stores and you hit a red, yellow, or green button on your way out and that gives them some indication of your experience.

Jan-Eric:
Again, some concepts you even see like a Dropbox of leave us your comments and there’s a little tear pad and some pencils.

Zack:
Right. That’s old school.

Jan-Eric:
Yeah, that’s super old school. Yeah.

Zack:
So that’s two. The other, the other way that I think people use most often is anecdotal feedback. So I’m the guy in the store every day. I know what my customers are saying, I know what the experience is, which we all know the pitfalls and something like that.

Jan-Eric:
So, let me ask you about that- with what people are saying and some of these, you know, the smiley face, a neutral face, frowning face, those types of things. There are other metrics that are operational indicators that are I think assumed to be indicative of what the guest experience is. And I’ll give you an example of that. If I’m in a fast food or quick serve concept, speed of service is something that I’m trying to deliver. I can look at through my POS, I can look at how how much time elapsed from when the order begins to when the order’s fulfilled. There is an assumption that the speed of service is an indicator of guest satisfaction.

Zack:
Sure.

Jan-Eric:
So yeah, that would be probably another whole set of … Beyond just the guest feedback. But looking at these operational metrics, I’m assuming then that the operational metrics are leading to a guest leaving satisfied with what their experience had been.

Zack:
Right? Yeah. And then the … So I agree with that, but the hard part is we’ve got to understand what the perception of the experience was with the customer because like if you think about, let’s talk about McDonald’s, they could have super fast service, right? All the brand standards could be there, the store could be clean, but if the order was messed up, that could ruin the perception of the entire experience for the customer.

Zack:
So well we’ve actually found a lot of value in is grabbing online reviews that no one really had any influence on. It’s unsolicited reviews on places like Yelp and Google. And typically the ones we’d like to stick with are the ones that are named reviews where you have to leave a comment. Yelp is one of the strongest ones for that. Google does that pretty well too. Now that data requires a different acquisition mechanism than some of these others. But that’s kind of the last, the last of these, these lists of ways to collect this information. And that’s the one that we’ve had a lot of experience with and seen a lot of benefit from.

Jan-Eric:
Now are those comments all just trolls? Haters are going to hate and all you ever see is people reporting the bad. So is it really fair, a fair indication of what the guest experience is?

Zack:
Well, in the data we’ve seen, it’s not all trolls and the real value in that data. There’s really two ways to look at it. One, is look at it over time, right? Cause if … Let’s just talk about Yelp to make it easy. If Yelp is just full of trolls, you would expect the troll factor to be consistent over time.

Jan-Eric:
It’s always hate all the time.

Zack:
Right. So if you’re looking at variances over time and your percentage of negative comments hypothetically is rising over time, you wouldn’t expect that to be caused by the fact that there’s a higher rate of just mad people on the platform in general.

Zack:
So yeah. The other way to evaluate that is to look at it in comparison to your competitors. So if your competitors all have that same factor, so let’s just say half your reviews are negative and half are positive. If you pull your next competitor and 80% of their reviews are positive and 20% are negative, that gives you a bearing to say, “Oh geez, something’s going on here.” Right? “Something is different about my brand or my concept than my competitors.”

Jan-Eric:
So even if it is all hate, there are ways to get into this and figure out degrees of hate. You can look at a number of comments. The number of comments as it relates to the volume of a particular store, right? Hey, this is a store that at crates has tons of transactions, ought to have a higher volume of negative comments. If they are, let’s say they are all negative, which I know that we’ve seen that they’re not all negative, but just kind of going with this for anybody who has this kind of belief that everything’s negative. You can look at the comparisons of that and then you can also be looking at generally the sentiment of what these topics are about and there’s opportunity to kind of learn as you go through that.

Zack:
Yeah, so one thing before I talk about sentiment, even if in my opinion, even if the majority or reviews are negative, my next step would be to start reading those reviews. And if I read through them and they sound like things that could happen, they’re probably not lies, right? It’s probably not people just being malicious and trying to, for whatever reason, bring down a company. They’re probably actual examples of things that have happened inside the restaurant or store or whatever. So my perspective is that if someone is striving for excellence in their customer experience, even the people who are just negative Nancys should be taken seriously.

Jan-Eric:
So one of the biggest hurdles, sort of the first hurdle to get over, around this whole like guest experience thing. If you’re going to do any analytics work with guest experience, you’ve got to turn the guest experience into data. You have to get an insight to what … You have to have access to the guest experience in the form of data. What you’re suggesting is that of all the data that’s out there, perhaps the most valuable source of data would be through unsolicited input from the guest through online reviews and acquiring that information in the form of data. So now you’ve got that data in house and at your, at your fingertips. What do you do with it?

Zack:
Yeah, you mentioned sentiment. That is one of the first steps. So we’re looking at positive and negative reviews over time. That percentage of different star ratings and stuff. Typically all at the location or market level. When you do it broadly it’ll tell you one thing, but when you start looking location by location, it gets really interesting really quickly. But that will only get you so far. Those broad level metrics will only get you so far.

Zack:
The real value is in parsing all of that text in those reviews, which is why I like platforms where you have to leave an actual review rather than just give a star rating cause it tells you what problem or positive experience the customer had. And with a good analytics team and a good piece of technology it gets … I mean you can parse all that text into.

Zack:
And so what we typically do is we’ll go through and parse everything out into categories. So if we think about a restaurant, we may have like a cleanliness category, we may have a food quality category, we may have a take out experience category. And really the way we define those as by looking for different phrases and sentences inside the review that leads to believe that was the experience the customer had.

Zack:
And then we run all that data through a natural language processing algorithm that tells us the sentiment associated with that data. We use Google’s NLP API to do that. We think Google is probably one of the best at figuring out sentiment cause they do this stuff. This is their business and-

Jan-Eric:
And understanding language-

Zack:
Right. Yeah. That’s what they do. And so we run it through there and it comes back and tells us, okay, in the cleanliness category here were the sentiment ratings of every review that you’ve ever had.

Zack:
And now we’ve got all that data to say how many negative cleanliness experiences we had, how many positive cleanliness experiences and what the range was of that. Did I have a bunch of people who were just ecstatic with it? Or a bunch of people who were just kind of like, “Yeah, this place was all right but my table didn’t get cleaned off.”

Jan-Eric:
Okay, great. This is all fascinating. But the whole point of this and what I was trying to kick the podcast off of, maybe you can’t deliver on this is how the guest experience is actually impacting sales. So great. You know what people are saying about the cleanliness of my store, but I’m really trying to understand does that have anything to do with the cash register? Yeah, so can you connect those dots?

Zack:
That’s one of the reasons we need the data at such a granular level. So I’ve got to have it store level, maybe market level if for some reason that’s all I can get. But then we’re putting all of that data up against sales history. And this is one reason we pull a lot of history out of these platforms because you want to do this over multiple years. And what we’re looking for are changes in either sentiment or rating and putting that and looking for that up against changes in sales performance.

Zack:
So a simple example if my, let’s say food quality, if my food quality experience sentiment has been declining over time, number of weeks, years or months, and that lines up with declines in sales at the store level, that starts to point me in a direction that geez, this is either … Basically I’ve got something going on in the restaurant that is making customers unhappy and they’re not coming back because they’re unhappy.

Jan-Eric:
And if you have this at the store or DMA level, you’re able to protect against any nuances that are washed out by looking at things in the aggregate view. If I am looking in Tulsa, Oklahoma as an example, and I have an operator in Tulsa who has really shown a big trend on improving the cleanliness of stores over the last couple of years. And at the same time we have metrics that indicate that sales have gone up and that the effectiveness of media to deliver a very cost efficient cost per store visit, and those things have also been on the rise at the same time? You are led to believe, “Hey, clean stores, running a clean and … Improving the cleanliness of a store impacts a lot of things.” Not only your sales but your ability to drive additional sales through marketing and promotion.

Zack:
Yep. It also helped, sorry, one more thing. It also helps you define where your problem children are. Right? I mean another way to do that same analysis I just talked about is just look at your sentiment or review ratings by store and look for positives and negatives both in the reviews and sales performance and you’ll find that there is a set of stores that have a lot of negative reviews and there are stores that have a lot of positive reviews. Then you can start taking a bit more of a qualitative approach to understand what’s going on in those stores. Right. Do I have a great operator that’s just amazing at what he or she does? Or do I have a store that’s got a really big problem with turnover and they can’t keep employees trained and so it’s causing a bad customer experience. There’s all types of ways you can take this data to fix the problem.

Jan-Eric:
Yeah. Well I think it’s fascinating. Again, the kickoff of this episode of the podcast, this summer summit nebulous, tough to pin down idea of guest experience. You would argue that guest experience has nothing. It has no impact on sales, but it’s very difficult to quantify this and even to understand how far outreaching the guest experience has on other factors. We’re looking at the effectiveness of media or the effectiveness of one particular message or we’re evaluating a marketing calendar. We had this LTO that we launched. Was it good or not? Well, we may determine that it wasn’t good because sales didn’t go up, but it may have nothing to do with the LTO and more to do with the fact that the bathrooms are dirty in the restaurant. And this is not intended to be the classic marketing versus operations battle.

Jan-Eric:
You know when sales are up everybody takes credit when sales are down, ops will blame marketing, marketing blames ops. The reality is this is an ecosystem of things that feed off of one another and we all rise together and we all fall together and so I’m fascinated by this.

Jan-Eric:
It becomes super useful to understand if I’m in a leadership position and I’ve got to make decisions on, “Hey, what are the problems I need to fix? What’s part of the problem? What’s part of the solution and how do I fix things?” Being able to quantify, turn that guest experience into data, leverage that data in analysis that helps me understand what impact it’s having or how it does or does not correlate with sales and other KPIs within marketing and then make decisions and evaluations on how I can fix or optimize the guest experience in a way that benefits sales? That is a giant step forward. Yeah. This is fascinating. Anything else you want to add?

Zack:
Yeah, two things. One, I was just listening to you talk. The thought that popped in my head is it also from a messaging standpoint it helps make sure you’re not lying to your customer. If you know you have a certain type of issue in an area of the country, don’t message that you’re good in that area because then you’re just … It’s like a bait and switch. You’re tricking people and in my opinion, that’s driving an even worse perception. They walk out of there like, “Geez, they said this place was clean. It’s disgusting.” So that’s one piece.

Zack:
The last thing I want to mention is we’ve been talking about restaurants, but this relates to products as well. So if you think about on Walmart, there’s reviews for every product. The same method can apply to that. In fact our a retail strategy director, when she started here, she told me when she was a buyer at a major retailer she used to make her whole team read all of the customer reviews for the products that they managed. And the reason was it gave that buying agent a feeling of how customers were liking it, the problems they were having and stuff like that. So it’s not just restaurants it’s retail as well.

Jan-Eric:
And as a manufacturer or a marketing team, if you don’t understand that and can’t respond to how it’s impacting the other work you’re doing, it strips your credibility. Yep. What’s been fascinating conversation, Zach, thanks for coming in and hopefully our listeners have found it valuable.

 

You’ve been listening to the Uncovering Aha! podcast. Callahan provides data savvy strategy and inspired creativity for national consumer brands. Visit us at callahan.agency to learn more.