More, simply put, is better.
More money. More traffic. More email subscribers. More stars.
Or is it? Medill Northwestern University’s Spiegel Research Center recently conducted research to challenge the “more is better” conventional wisdom, at least when it comes to star ratings’ impact on purchase probability
Read on to see insights from this research.
And in a special bonus this week, my interview with the Spiegel Center’s Executive Director and Research Director was so rich and full of insights, we published it on the MarketingSherpa blog as well.
(As seen in the MarketingSherpa Chart of the Week newsletter. Click to get a free subscription to the latest research and case studies from MarketingSherpa.)
The Spiegel Center conducted research with two online consumer-packaged goods (CPG) retailers, including 22 product categories, over 100,000 SKUs and one high-end retailer with more than 15 million page views over the course of a year.
After publishing its report on the data, the Spiegel Center created this exclusive chart for MarketingSherpa to show star ratings’ impact on purchase probabilities across a range of varied products.
Five stars is too good to be true
The x-axis of the chart shows the amount of star ratings the product has received, and the y-axis shows the effect of those differing star-rating amounts on the conversion rate of that product using a multiple logistic regression (if you’re interested to learn more about that technique, see “About the chart methodology” at the bottom of this article).
Using the “more is better” rationale, you might expect these numbers to be a fairly straight line from the lower left of the chart to the upper right.
In other words, if you think more is better, the conversion rate should increase as star ratings increase.
But, we don’t see that. The conversion rate for most products does tend to increase as the star rating increases. However, conversion plateaus or even drops before the product gets to a five-star rating.
“When consumers see all five-star reviews, they become skeptical that it's too good to be true, that there's some manipulation happening,” said Edward Malthouse, professor at Medill Northwestern and the Research Director of the Spiegel Center.
“Therefore, what we see in that chart is if you have an average star rating in the mid-fours, the purchase probabilities are higher than if it's at five, which means having some negative reviews in there actually adds credibility to you.”
Authentic ratings processes
Tom Collinger, the Executive Director of the Spiegel Research Center at Medill Northwestern University, cautioned brands against buying or incenting five-star ratings.
“The consumer is not stupid,” Collinger said. “What the consumer has more or less said is, ‘We don't believe in nothing but five stars. We don't believe in it.’ That's what their behavior is reflective of. So … not eliminating negative reviews is a business practice. It's not a philosophy.”
And here’s why. We can put anything on our ecommerce websites we want. We can make ridiculous promises. We can even go all Spinal Tap and turn our star ratings up to 11.
However, the customer does not have to believe any of it — and the more far-fetched, the less likely to believe it.
A major reason to provide star reviews is to add anxiety-reducing credibility indicators to your site. If you’re not a fair player in how you get and display these reviews, you will undercut that credibility.
Credibility is king
Once you agree to provide ratings and reviews on your site, you’re also becoming more than an ecommerce site; you’re becoming a publisher on some level as well. And if potential customers are to respect the reviews you publish, they have to perceive them to be unbiased on some level.
“So, it isn't to say you're happy that you're getting negative reviews, but in fact, the presence of some negative reviews apparently increases the validity and authenticity such that people will purchase more when there is the presence of something other than pure fives. That's significant,” Collinger said.
“Reviews work as long as they're credible. Retailers should not suppress negative reviews because it damages the credibility of the review ecosystem. The retailer's interest is in providing useful information to the buyers who come to that website,” Malthouse agreed.
The role of customer reviews
Authenticity goes beyond just the star ratings. It’s important to keep customers’ written reviews unchanged and in their own words as well.
The Spiegel Center’s research found that if customers went just beyond the star ratings and actually engaged with the reviews themselves, they were more likely to purchase.
With those reviews, Collinger suggested that how you display them is important as well. He recommends displaying the most recent reviews first as the default, instead of showing them high to low. This presents the reviews as a more inclusive representative sample and helps build trust with customers.
But, really, Collinger’s biggest piece of advice is somewhat similar to Google’s when it comes to SEO — don’t overly focus on the technical aspects, focus on doing the right thing, and the results will follow.
Or, as Collinger put it in his own words, “I feel like that's a wonderful and encouraging message for marketers, which basically is fix the product, fix the customer experience. Don't worry about fixing or rigging the customer review environment. Fix the way in which it works and good things will happen.”
About the chart methodology
Malthouse explained the methodology this way: “We’re using a multiple logistic regression, which allows for us to control for other factors such as price, category, brand, etc. This makes it a much stronger test but also makes it more difficult to understand. A -2 means that the logit of buying goes down by 2. Probability must be between 0 and 1, and linear functions can go outside that range. The logit function is used to prevent going out of bounds. It maps 0-1 conversion rates to all numbers, which avoids the out-of-bounds problem.”
How Online Reviews Influence Sales (via Northwestern University’s Spiegel Research Center)
Download a free 54-page Customer Satisfaction Research Study to learn about our latest discoveries based on research with 2,400 consumers
MarketingExperiments Quarterly Research Journal (page 99 shares a landing page experiment that included added a real-time, five-star rating widget).
7 Steps to Discovering Your Essential Value Proposition with Simple A/B Tests
Drive better results when you discover what it is about your business that customers love.Get the Guide >
Increase Mobile Conversion Rates
These five free micro classes (each under 12 minutes) apply 25 years of research to help you maximize the impact of your messages in a mobile environment.Get the Course >
Test Discovery Tool
Show business leaders all the results of your testing efforts with this free tool.Get the Tool >
A Model of Your Customer's Mind
These 21 tools and concepts have helped capture more than $500 million in test wins.Download the File >
Research-based Lead Generation Swipe File
22 valid marketing experiments to give you ideas for your next A/B test.Download the File >
Receive the latest case studies and data on email, lead gen, and social media along with MarketingSherpa updates and promotions.