As entrepreneurs and marketers, we are inside the four (sometimes virtual) walls of the organization.
So, it is all too easy for us to focus on the means, and not the end. Focus on the thing we make instead of the value it brings to our customers’ careers and their lives.
To help your organization keep the focus squarely on the customer, we bring you three organizations who engaged in three very different tests in this article.
Read on for quick case studies from the Canadian Red Cross, an online tutoring provider, and a recruitment firm.
This article was published in the MarketingSherpa email newsletter.
“Separate the means from the end. Focus on the customer’s experience of the value,” Flint McGlaughlin taught in Value Proposition Strategy: Everything you need to know in 6 minutes.
To give you ideas for focusing on the customer’s experience of the value, in this article we bring you three quick case studies.
First, an online tutoring provider uses two types of online testing to validate its understanding of its customers’ experience. Then, the Canadian Red Cross tests an expression of personal appreciation to its “customers” (i.e., donors). And finally, a recruitment firm formulates its customers’ experience in a defined value proposition for the very first time, and conducts offline tests in customer pitches to determine its effectiveness.
The GoStudent performance marketing and conversion rate optimization (CRO) team wanted to better understand how consumers are engaging with their website content in the United States. “We want to make sure the landing pages we design capture the user’s attention, improve brand consideration, and ultimately drive action,” said Colton Weaver, Performance Marketing Manager, GoStudent.
While looking at their site conversion rate and engagement rate for their primary landing page, they noticed many American consumers were visiting the landing page but either bouncing or not converting at as high a rate when compared to their counterparts in other nations.
Creative Sample #1: Primary landing page for online tutoring provider
The team hypothesized that adding a stronger CTA message with action words would drive a higher conversion rate when compared to softer messages currently being used. “We leaned into bandit testing to optimize our CTA while highlighting our free trial messaging to capture the customer’s attention which may have been missed historically,” Weaver said.
(Technical Note: I don’t want to get too deep into the complexities of the statistics behind marketing experimentation, but for the purposes of this article it is important to understand that bandit testing, also called multi-armed bandit testing, is an attempt to minimize the traffic that goes to low-performing variations in an experiment by shifting traffic to better-performing variations. A/B testing usually splits the traffic evenly throughout the test to the different treatments.)
To test this hypothesis, the CRO team initially used a four-arm bandit test to test four variations of the CTA – the control and three variations. These CTAs were:
Creative Sample #2: Variation #3 of button copy for online tutoring provider
This test ran for two weeks before reaching statistical significance. Variation #3, “GET YOUR FREE TRIAL,” had the strongest performance driving a 13% increase in sign-up rate compared to the control. Variation #2 also showed an improvement in conversion rate (CVR), but not at the same rate as Variation #3. The results for this four-arm experiment are shown below.
Creative Sample #3: Call-to-action experiment results
After analyzing this data, the team decided to conduct a strict A/B test with the control and the winning variation. It is the team’s general practice to rigorously assure statistical significance before implementing a sitewide change. The A/B test ran for two weeks, and Variation #3 produced the following results as compared to the control.
(Metric Definition Clarity, Engaged Sessions: This user engagement metric shows that your app screen was in the foreground, or your web page was in focus. When your site or app is running but no page or screen is displayed, Google Analytics doesn't collect the metric. The metric can help you understand when users actively use your website or app.)
Looking at the conversion rate of the control over the course of the two tests, during the first test the conversion rate was 7.76%. During the second phase of the test the conversion rate declined -17% (averaging ~6.6%). This difference in conversion rate influenced the performance results as Variation #3 remained relatively constant at ~8.5% conversion rate.
Through these two tests, the team found that a more straightforward call to action – “GET YOUR FREE TRIAL” – helps drive site engagement as the consumer spends more time on the site to understand the brand, product, and services as they prepare to sign up for a free trial. However, with shifts in seasonality and macro-economic trends, the team will be testing this CTA again in the coming months to validate this data further.
“Looking at these results, we know that there is a lot more we can be doing and will leverage A/B testing more rigorously in the future months to come. We are excited to have the agility to test and learn easily which enables us to share with pride across all of our global markets. This is just the beginning,” Weaver said.
During this test, the team also learned that making messaging consistent at each stage of the consumer journey helps drive overall brand continuity and conversion rate. The sales team often uses the free trial message to convert leads into customers.
The customer experience team at the Canadian Red Cross is the steward of long-time donors to the humanitarian charitable organization.
To celebrate World Humanitarian Day during the pandemic, the team wanted to send an email to thank long-time donors and highlight the good work made possible by their contributions. They wanted a high-value touchpoint for loyal donors that give more than average.
“The Canadian Red Cross team created a personalized video email campaign to steward a segment of long-time donors…we tested the personalized video against a control segment and a generic segment who received the same video without personalization,” said Andrew George, Director – Direct & Integrated Marketing, Canadian Red Cross.
The experiment consisted of:
“After the campaign, we saw an 11% lift in year-end giving between the personalized video and control group,” George said.
The personalized video generated an 11% uplift in giving over the control group and a 7% uplift in giving over Treatment #1 – even though there was no direct call-to-action asking for donations. The nonprofit earned a 17x ROI for the personalized video compared with the control group and 13x ROI for the personalized video compared to the generic video. Let’s take a look at the creative samples for the winner…
The email’s subject line was personalized – “<first name>, we made this video for you" and the email was signed by CEO Conrad Sauvé.
Creative Sample #4: Canadian Red Cross email introducing personalized video
The email drove people to a simple landing page that hosted the personalized video.
Creative Sample #5: Canadian Red Cross landing page with personalized video
The personalized video mentioned the donor’s name in the audio track at the very beginning, and also had several visuals that included the donor’s name with thank you messages throughout the video.
“The process to execute the videos was seamless,” George said.
Creative Sample #6: Screen grab from Canadian Red Cross personalized video
"To thank long-time donors, the Canadian Red Cross knew it needed to be personal, but they also wanted to show donors how their contributions make a real difference. By sending a dynamic video rather than, say, a letter, they combined the power of visual storytelling with personalization. This is a proven recipe for loyalty, and it worked here, increasing engagement and donations. As an added benefit, videos are easy to share on social media, helping spread the word further,” said Yaron Dishon, CRO, Idomoo (the Canadian Red Cross’s video personalization platform).
I’m always hesitant to write a case study about value propositions, because MECLABS Institute (MarketingSherpa’s parent organization) has a well-known and well-regarded value proposition methodology, so readers sometimes assume that any value prop we publish in a case study is golden.
This next case study isn’t meant to show you a perfect value proposition. The messaging this organization used could likely be improved (as can the messaging in every case study we bring you) because case studies aren’t meant to give you the copy-and-paste answer, they are meant to inspire you to ask the right questions. These aren’t thought leader opinions about what you should do with your brand, they are real stories about what really worked (and sometimes what really didn’t work). How marketers and entrepreneurs just like you, hard-working people bogged down by day-to-day organizational and client concerns, found a way to produce results for their brands and better serve their customers.
This next case study is another such story. I don’t bring it to you because I think the value proposition messaging is perfect, I bring it to you to show the impact on results a company can achieve by moving from not having a defined value proposition to actually defining that value prop. To inspire you to get your organization to define its own value prop. So let’s get into it.
Nigel Wright Group did not have a defined value proposition. Teams sold on their ability to convince clients that the recruitment firm was good at what it does. There was no consistency between teams and each person used a similar but different approach.
The marketing team realized the recruitment firm needed to define its value proposition. They took the approach to do it internally rather than ask their clients. They involved experienced recruitment consultants from across the business in the workshop (representing the best internal client/market knowledge) to explore “Why clients buy from us?”
After capturing many reasons why clients buy from the firm, these were prioritized into the top four reasons by finding the strongest “proof-point differentiators.”
“These were not based on opinions. These were factual points that made Nigel Wright Group unique within the target market that the value proposition related to. It resulted in a powerful proposition that Consultants were then trained to use in pitches that, as a result, became better structured and more effective at winning new clients,” said Justin Barlow, Marketing Director, Nigel Wright Group.
Here is the value proposition the team defined for Nigel Wright Group:
Proof points were summarized into four significant differentiators. Each is supported by demonstrable client/market examples and case studies (supported by statistics) that reinforce each differentiator:
Once defined, the team sought to test the value proposition by pitching it to clients in real pitches. They also sought feedback from the pitched clients – whether they won or lost – to find out specific elements of the strengths/weaknesses in the proposition.
They discovered that all wins were directly related to the company’s value proposition messaging.
Most losses, however, were price related. Price was not an emphasis in the defined value proposition. The team focused on the value the recruitment firms creates rather than the cost of its fees. The team felt that some clients have a psychological ceiling that they will not accept breaching when it comes to cost.
Overall, they found through this testing that the value prop proof points that support each reason to use the recruitment firm were strong. The team saw a 35% increase in pitch wins using the value proposition messaging in the first six months.
They have decided to revisit and update the value proposition every six months, as they grow and create more value with their business.
Boost conversion. Learn from 150+ CTA experiments.
Customer-First Marketing: Understanding customer pain and responding with action
5 mini case studies about understanding and serving the customer
Marketing Research Chart: Does a good customer experience really affect business success?
Get Better Business Results With a Skillfully Applied Customer-first Marketing Strategy
The customer-first approach of MarketingSherpa’s agency services can help you build the most effective strategy to serve customers and improve results, and then implement it across every customer touchpoint.Get More Info >
MECLABS SuperFunnel Research Cohort
Join us for a live, interactive session and learn along with our community.RSVP >
Marketer Vs Machine
Marketer Vs Machine: We need to train the marketer to train the machine.Watch Now >
Free Marketing Course
Become a Marketer-Philosopher: Create and optimize high-converting webpages (with this free online marketing course)See Course >
Project and Ideas Pitch Template
A free template to help you win approval for your proposed projects and campaignsGet the Template >
Six Quick CTA checklists
These CTA checklists are specifically designed for your team — something practical to hold up against your CTAs to help the time-pressed marketer quickly consider the customer psychology of your “asks” and how you can improve them.Get the Checklists >
Infographic: How to Create a Model of Your Customer’s Mind
You need a repeatable methodology focused on building your organization’s customer wisdom throughout your campaigns and websites. This infographic can get you started.Get the Infographic >
Infographic: 21 Psychological Elements that Power Effective Web Design
To build an effective page from scratch, you need to begin with the psychology of your customer. This infographic can get you started.Get the Infographic >
Receive the latest case studies and data on email, lead gen, and social media along with MarketingSherpa updates and promotions.