Research-based Ecommerce Swipe File

25 valid marketing experiments to give you ideas for your next A/B test

The rise of ecommerce probably has had two effects on your day-to-day life. For one, ecommerce has made it easier to reach customers throughout the country and the world. Fellow evidence-based marketer,

But second, every product you’re selling, every goal you have now exists in a brutally competitive market where your potential customer can quickly and easily leave your website and choose anyone from the behemoth Amazon to an eBay seller operating out of a garage instead of your company.

So how do you compete? How do you increase your conversion rate and sell more products? I can’t give you a specific answer.

But I’ll tell you who can — your customers.

With A/B testing, you can discover what really works on your brand’s website, in your brand’s email, and in your brand’s ads with your brand’s prospective customers.

To give you ideas for tests on your website, we put together this swipe file of 25 ecommerce experiments that MECLABS Institute analysts conducted in Research Partnerships with ecommerce companies to help them learn about their customers and improve conversion rates.

There’s a lot of information here, and different people will want to go through this swipe file in different ways. You can scroll through the webpage you’re on and use the anchor links. Or use the form on the page to download a PDF with all of the experiments, including a table of contents and internal anchor links to help you navigate.

If these experiments inspire your own tests, we’d love to see the results — just drop me a line at d.burstein@meclabs.com.

Here’s to higher-converting ecommerce websites,

Daniel Burstein
Senior Director, Content & Marketing
MarketingSherpa and MECLABS Institute

P.S. If you need help improving conversion, just drop me a line as well. MECLABS analysts can work hand-in-hand with you to apply our patented methodology to your conversion challenges.

 

Download a 181-page PDF with these 25 experiments

plus, you’ll receive regular emails from MECLABS Institute.

Experiment #1: 46% more conversions for furniture company by changing credibility approach

Experiment ID: TP11009
Background: Mid-sized furniture company selling mattresses
Goal: To increase the overall number of mattress purchases
Research Question: Which credibility approach will produce the highest rate of mattress purchases?
Test Design: A/B variable cluster split test

Experiment #1: Control

  • The product is an organic latex mattress. It is one of only a few mattresses that is GreenGuard Gold certified.
     
  • In the control, the certification is present but de-emphasized. There is also no explanation to help the customer understand why they should value this certification. 

Experiment #1: Treatment

Experiment #1: Side by Side

Experiment #1: Results

46% Relative Increase in Conversions
The treatment significantly increased conversions by 45.69%

Design KPI % Rel. Change
Control without GreenGuard copy 0.65% -
Treatment with GreenGuard copy 0.94% 45.69%

What You Need to Understand: The increased clarity around an exclusive source of third-party credibility increased the value exchange and appeal of the product driving more sales.  

Back to Table of Contents

 

Experiment #2: 37% increase in conversions for travel agency by simplifying and sequencing the cart options

Experiment ID: TP1294
Background: B2C company offering package vacations to global consumer audience
Goal: To increase cart completions
Research Question: Which cart page will generate the highest completion rate?
Test Design: A/B split test (variable cluster)

Experiment #2: Control

  • The original cart was simple, but it included three equally weighted options from which the visitor had to select.
     
  • This made the checkout process unnecessarily cumbersome.

Experiment #2: Treatment

 

  • The marketers de-emphasized and integrated the additional options into the product details.

     
  • And they focused the visitor on one main call-to-action here.

Experiment #2: Results

37% Relative Increase in Conversions
The treatment significantly increased total cart conversions by 101.40%

Design KPI % Rel. Change
Control 12.94% -
Treatment 17.66% 36.50%

What You Need to Understand: By moving the secondary CTAs to the relevant cart sections and putting emphasis on a single CTA to move the customer forward friction was reduced and conversion increased.

Not This, But This…

Options Selection
Protocol ID: TP1294

Back to Table of Contents

 

Experiment #3: 12% increase in conversions for multimedia retailer by strategic placement of testimonial and credibility indicators

Experiment ID: TP1070
Background: A national computer hardware multimedia retailer with a significant online and offline presence
Goal: To increase total cart conversions and revenue per cart
Research Question: Which treatment will generate the highest conversion rate and revenue per cart?
Test Design: A/B variable cluster test

Experiment #3: Control

  • A closer look at the control cart page reveals that all supporting content is focused on making an upsell.

Experiment #3: Treatment


The treatment, however, changes focus to reduce potential anxiety.

 

  • Testimonial, customer support and live chat in the supporting column
     
  • Another testimonial and credibility indicators below the call-to-action area

Experiment #3: Side by Side

Experiment #3: Results

12% Relative Increase in Conversion
The treatment significantly increased revenue per conversion by 11.6%

Design KPI % Rel. Change
Control 49.14% -
Treatment 54.84% 11.60%

What You Need to Understand: By addressing anticipated anxiety at the critical decision through the use of testimonials and clear trusted payment options, the treatment generated 3.69% more sales in addition to 11.6% more revenue per cart, resulting in a projected $53,000,000+ annual increase in revenue.

Back to Table of Contents

 

Experiment #4: 87% increase in conversions for online printing company by re-sequencing product and process information on product page

Experiment ID: TP1568
Background: An online printing company that specializes in delivering printed marketing materials with minimal turnaround
Goal: To increase number of purchases online
Research Question: Which product page will result in the largest purchase rate?
Test Design: A/B Variable Cluster Test

Experiment #4: Version A

Experiment #4: Version A

Experiment #4: Version B

Experiment #4: Version B

Experiment #4: Side by Side

Experiment #4: Results

87% Relative Increase in Conversions
The treatment significantly increased conversion by 87.40%

Design KPI % Rel. Change
Version A 4.03% -
Version B 7.55% 87.40%

What You Need to Understand: By resequencing the page to put the form first and better match motivation and clarify the eye path, the new product page template achieved an 87.40% increase in conversions.

Not This, But This…

Eye path
Protocol ID: TP1568

Back to Table of Contents

 

Experiment #5: 20% increase in conversions for Italian cosmetics website by adding an interactive element to product page

Experiment ID: TP1283
Background: Italian ecommerce website offering cosmetics. The researchers were focusing on testing different approaches to the "body" category page.
Goal: To increase rate of conversion
Research Question: Which page will generate the highest rate of conversion?
Test Design: A/B Variable Cluster Test

Experiment #5: Control

  • Is the category list at the top of the page the most user-friendly way to present the information?

Experiment #5: Treatment

Treatment 1 seeks to make the page easier to use by adding an interactive configurator that enables the visitor to customize the products that show up below.
  • By Category
  • By Objective
  • By Product Line

Experiment #5: Treatment

 

 

Treatment 2 seeks to make the page easier by removing the category links and simply featuring the main categories with images.

Experiment #5: Treatment

 

Treatment 3 is a radical approach that seeks to make the process easier by removing the “body” category page altogether, enabling the visitor to choose their category within the navigation of the homepage.

Experiment #5: Treatment

 

Treatment 4 is similar to Treatment 3, only it integrates a more visual approach to the categories within the navigation.

Experiment #5: Side by Side

Experiment #5: Results

20% Relative Increase in Conversion
The configurator treatment significantly increased conversion by 20.00%

Design KPI % Rel. Change
Control 1.04% -
Treatment 1 1.25% 20.00%
Treatment 2 1.10% 6.00%
Treatment 3 1.10% 5.00%
Treatment 4 1.10% 5.00%

What You Need to Understand: By adding an interactive element, the new product page achieved a 20% increase in conversions.

Back to Table of Contents

 

Experiment #6: Projected $500,000+ increase in revenue per year for retail/wholesale collector items website by testing which version of a second step in the conversion funnel will produce the highest conversion rate

Experiment ID: TP1305
Background: A website that sells retail and wholesale collector items
Goal: To increase rate of conversion
Research Question: Which version of a second step in the conversion funnel will produce the highest conversion rate?
Test Design: A/B variable cluster split test

Experiment #6: Background

  • When we analyzed the metrics, we realized there were leaks throughout the checkout process. The credit card submission page stood out as low-cost opportunity for immediate return.
     
  • When we analyzed the metrics even further, we saw that this step also had the highest lost revenue per cart (more than double compared to any other step).
     
  • From this, we hypothesized that optimizing this step would have the highest potential return on our efforts.

Experiment #6: Control

What might be causing the fallout?
  • It is unclear why the credit card is required when payment method is different.
     
  • The complexity of the purchase agreement terms causes confusion and concern.
     
  • There is no indication that the customer’s credit card information is secure.

Experiment #6: Treatment

How we addressed the issues:
  • Third-party security indicators have been added.
     
  • Clearer explanation of why a credit card is required and that it will not be charged.
     
  • “Satisfaction Guaranteed” promise is emphasized.

Experiment #6: Side by Side

Experiment #6: Results

5% Relative Increase in Conversion
The treatment significantly increased revenue per conversion by 11.6%

Design KPI % Rel. Change
Control 82.33% -
Treatment 86.04% 4.51%

What You Need to Understand: While it might seem like a small increase, the changes in this treatment on this step of the funnel resulted in a projected $500,000+ increase in revenue per year. This underscores the potential impact of a properly identified research question.

Not This, But This…

Clarity
Protocol ID: TP1305

Back to Table of Contents

 

Experiment #7: 49% increase in conversions as well as significant increase in email capture for people-search software database company by changing the text and position of the call-to-action and adding an email capture field

Experiment ID: TP1000-13
Background: A company offering people-search software database for consumers
Goal: To increase the number of emails captured
Research Question: Which page will generate the highest email capture rate?
Test Design: A/B variable single factorial split test

Experiment #7: Control

  • When a visitor clicked “Order Now” they were then directed fill out a single-page form with their payment information.

Experiment #7: Treatment

  • The treatment added an email capture field and changed the button copy from “Order Now” to “Continue to Step 2” while sending visitors to the same order page.

Experiment #7: Side by Side

Experiment #7: Results

122% Relative Increase in Email Capture
The treatment path increased email captures by 121.80%

Design KPI % Rel. Change
Control 6.76% -
Treatment 14.98% 121.80%

What You Need to Understand: By changing the position of the call-to-action and adding an email capture field, we were able to significantly increase emails and also increase orders by 49%.

Back to Table of Contents

 

Experiment #8: 29% increase in conversion for fitness company by removing the cart preview

Experiment ID: TP1620
Background: A fitness company that primarily sells fitness training content and gym equipment
Goal: To increase sales
Research Question: Which checkout process will result in a higher conversion rate?
Test Design: A/B multifactor split

Experiment #8: Control

Experiment #8: Treatment

Experiment #8: Results

29% Relative Increase in Conversion
The treatment path increased conversion by 28.60%

Design KPI % Rel. Change
Control 27.70% -
Treatment 35.6% 28.60%

What You Need to Understand: By removing the unnecessary cart preview page, the treatment increased conversion by 28.60%.

Back to Table of Contents

 

Experiment #9: Orders increased 263% for online people-search company by adding the discount incentive

Experiment ID: TP1000-9
Background: An online people-search company that was losing many orders due to cart abandonment. We wanted to find a way to recover as many of these orders as possible with a minimum incremental marketing spend.
Goal: To recover partially completed but abandoned orders through a sequence of cart recovery emails
Primary Research Question: Which cart recovery sequence and offer will generate the most sales?
Approach: A/B split test (variable cluster)

Experiment #9: Control

Experiment #9: Treatment

Experiment #9: Results

263% Relative Increase in Order Rate
The treatment path increased conversion by 263.20%

Design KPI % Rel. Change
Control 19% -
Treatment 69% 263.20%

What You Need to Understand: By adding the discount incentive, the treatment increased order rates by 263.20% and total revenue per email by 133%

Back to Table of Contents

 

Experiment #10: 25% increase in email open rate for organic meals home delivery service by including relevant information about the reduced minimum order

Experiment ID: CS771
Background: This company offers prepackaged organic meals delivered to your home. They believed that the order minimum was hurting repeat sales. They began a promotion that reduced the minimum order. An email was developed to inform previous customers of this new order option.
Goal: To get recipients to open the email
Primary Research Question: Which subject line will receive the higher open rate?
Approach: A/B single-factorial split test

Experiment #10: Version A/B

Experiment #10: Results

25% Relative Increase in Open Rate
Version B outperformed version A by a relative difference of 25.30%

Design KPI % Rel. Change
Version A 35.20% -
Version B 44.10% 25.30%

What You Need to Understand: By including relevant information about the reduced minimum order, prospects opened the treatment email at a relative rate 25.30% higher than the control.

Back to Table of Contents

 

Experiment #11: 26% decrease in open rate, but 60% increase in conversion for large online florist by using offer-oriented subject line in Thank You email

Experiment ID: TP2033
Background: Large florist with a strong online presence seeking to increase the effectiveness of a “thank you” email campaign to previous customers
Goal: To increase the rate of return business from customers who made recent purchases
Research Question: Which email subject line will result in the greatest volume of return business?
Approach: A/B single-factorial split test of the subject line

Experiment #11: Control

  • “Thank You For Making Us Your Florist of Choice” stated intention but did not make a clear offer.

Experiment #11: Treatment

  • “15% Off - Our Way Of Saying Thank You!” stated the purpose and offer of the email message.

 

  • The email graphics and body copy were identical to the control.

Experiment #11: Side by Side

Experiment #11: Results

26% Decrease in Open Rate
The offer-oriented subject line decreased open rate by 25.7%.

Design Open Rate
Control 20.12%
Treatment 14.95%
% Relative Change: -25.7%

 

Experiment #11: Results

  • A deeper analysis of the metrics revealed that despite a significantly lower open rate, the treatment generated a 60% higher clickthrough rate and resulted in a 56% boost in revenue.

Experiment #11: Results

60% Increase in Conversion
The treatment significantly increased conversion by 60.34% and revenues by 56%

Design KPI % Rel. Change
Control 10.11% -
Treatment 16.21% 60.34%

What You Need to Understand: Looking solely at the open rate, one might conclude that the treatment underperformed. However, when drilling deeper into the metrics, it’s clear that the treatment outperformed the control. This underscores the importance of understanding the role of metrics in experimentation.

Back to Table of Contents

 

Experiment #12: 58% increase in conversions for automotive repair company by reducing friction

Experiment ID: TP1429
Background: The company is a leading automotive head gasket repair solution.
Goal: To increase total orders on cart page
Research Question: Which landing page/cart will result in a higher conversion rate?
Approach: Radical redesign of cart page through a variable cluster A/B split test

Experiment #12: Control

Experiment #12: Treatment

Experiment #12: Side by Side

Experiment #12: Results

58% Relative Increase in Conversions
The treatment generated 58.1% more conversions than the control.

Design KPI % Rel. Change
Control 2.1% -
Treatment 3.3% 58.1%

What You Need to Understand: The reduction of friction throughout the process coupled with the single call-to-action led to a 58.1% relative increase in conversion rate.

Back to Table of Contents

 

Experiment #13: 14% increase in cart completions for travel agency by optimizing cart page with call-in data

Experiment ID: TP1368
Background: B2C company offering package vacations. In this test, we focused on improving the checkout process.
Goal: To increase cart completions
Primary Research Question: Which cart page will generate the highest completion rate?
Approach: A/B split test (variable cluster)

Experiment #13: Control

Experiment #13: Control

Experiment #13: Treatment

Experiment #13: Treatment

Experiment #13: Side by Side

Experiment #13: Side by Side

Experiment #13: Phone Numbers

 

  • It is important to note that each of the designs incorporated a phone number that users could call to place an order.
     
  • Conversion tracked through the phone call would make a difference in the results of this test.
     

Experiment #13: Results

14% Relative Increase in Conversions
Without call-in center data, the treatment generated 13.83% more conversions.

Design KPI % Rel. Change
Control 18.73% -
Treatment 21.32% 13.83%

6% Relative Increase in Conversions
With call-in center data, the treatment generated 6.25% more conversions.

Design KPI % Rel. Change
Control 22.63% -
Treatment 24.04% 6.25%

Back to Table of Contents

 

Experiment #14: 43% increase in online purchases for direct-to-consumer printing brand by changing path

Experiment ID: CS31053
Background: Direct-to-consumer printing brand offering custom-printed products
Goal: To increase online purchases

Experiment #14: Funnel Analysis

Experiment #14: Problem

Experiment #14: Solution

Experiment #14: Results

Back to Table of Contents

 

Experiment #15: 40% increase in clickthrough rate for medical provider by adding “Symptoms” to both header and description

Experiment ID: TP4068
Background: Medical provider specializing in treating chronic pain
Goal: To plan a content marketing strategy based on which approach generates more appeal in condition-based searchers
Primary Research Question: Which content approach will achieve a higher clickthrough rate?
Approach: A/B Multifactor Split Test

Experiment #15: Control

Based on what we learned from the previous content approach test, if we use a symptom content approach while matching the control's specificity to each ad group, we can achieve a higher click-through rate.

Experiment #15: Treatment

  • If Treatment 1 wins, we will learn that the symptom content approach is most effective only when used in the headline.

Experiment #15: Treatment

  • If Treatment 2 wins, we will learn that the symptom content approach is most effective when used in the description and when the description is specific to the ad group.

Experiment #15: Treatment

  • If Treatment 3 wins, we will learn that the symptom content approach is most effective when used in BOTH the headline and description and when the description is specific to the ad group.

Experiment #15: Results

40% Relative Increase in Clickthrough
Adding "Symptoms" to BOTH headline and description produced a 40% increase

Version KPI % Rel. Change
Specialty Pain Resources .28%  
Treatment Options .26%  
Causes and Solutions .21% -
Symptoms .39% 40.

What You Need to Understand: Applying insight from the previous tests and inserting "symptoms" into both the headline and description created more successful treatments across all ad groups.

Back to Table of Contents

 

Experiment #16: 40% increase in revenue per order for health drink seller by clarifying value proposition in the copy

Experiment ID: TP1798
Background: A single-product company that sells high quality, all-natural, powdered health drinks
Goal: To provide clarity of value in an effort to better match prospect motivation and increase the CR of the prospects reaching the AG homepage
Approach: A/B Multi-factorial Split Test

Experiment #16: Control

Experiment #16: Treatment

Experiment #16: Side by Side

Experiment #16: Results

40% Increase in Revenues Per Order
The treatment generated an overall 34% increase in the conversion rate.

Design KPI % Rel. Change
Control 3.3% -
Treatment 4.4% 33.37%

What You Need to Understand: By better expressing the value proposition through the copy and limiting imagery distractions, the treatment homepage not only increased conversion by 33.77% but also increased overall revenue per order by 39.95% at a 97% level of statistical confidence.

Back to Table of Contents

 

Experiment #17: 56% increase in revenue per order for precious metals exchange business by adding security seals and testimonials and removing unnecessary form fields

Experiment ID: TP1257
Background: This research partner offers investors a place where they can purchase gold, silver, platinum and palladium for their portfolios.
Goal: Goal of the experiment was to increase registration rate and revenue per visitor
Primary Research Question: Which of the following pages will produce the highest registration rate?
Approach: A/B Split

Experiment #17: Control

  • There are over 15 form fields and many are unnecessary.
     
  • The navigation on the registration page is potentially distracting users from completing the desired task.
     
  • Overall, there is a lack of third-party credibility indicators to alleviate anxiety.

Experiment #17: Treatment

Experiment #17: Side by Side

Experiment #17: Results

56% Increase in Revenues per Order
The treatment generated 56.16% higher revenue per order than the Control.

Design KPI % Rel. Change
Control $10,716.55* -
Treatment $16,734.96 56.16%

What You Need to Understand: Adding security seals and testimonials reduced anxiety and removing unnecessary form fields reduced friction to increase the money each customer was willing to spend.

Back to Table of Contents

 

Experiment #18: 18% increase in rate of conversion for ecommerce textbook site by sequencing the cart and justifying each action the customer is required to take

Experiment ID: TP1434
Background: An ecommerce site selling textbooks to professors in academic institutions
Goal: To increase textbook purchases
Primary Research Question: Which treatment will generate the highest conversion rate for new and existing users?
Approach: A/B/C Split Test

Experiment #18: Control

Experiment #18: Control

Experiment #18: Control

Experiment #18: Treatment

Notice the copy:
  • “… complete registration so that we can verify your instructor status …”
     
  • “Once you have completed registration, you will be able to quickly request exam copies …”

Experiment #18: Treatment

Notice the copy:
  • “Locate the school where you teach … find your department …”
     
  • “Once verified, we will automatically ship your exam copy to this address …”

Experiment #18: Treatment

Notice the copy:
  • “This information helps our publishing program.”
     
  • “Confirm Your Order”

Experiment #18: Treatment

Notice the copy:
  • “… to make sure all of the information that has been entered is correct ...”
     
  • “Send My Samples ...”

Experiment #18: Results

19% Relative Increase in Conversion
The treatment cart flow increased generated an 18.6% increase in conversion.

Design KPI % Rel. Change
Control 33.74% -
Treatment 40.02% 18.6%

What You Need to Understand: By sequencing the cart and justifying each action the customer is required to take, the treatment cart process increased the rate of conversion by 18.6%.

Back to Table of Contents

 

Experiment #19: 3x the projected revenue by increasing email frequency

Experiment #19: Background


Projected monthly revenue rose consistently with increasing send frequency and the number of sends did not have a significant impact on the overall rate of transaction.

Experiment #19: Background


Though projected unsubscribes rise with more sends …

Experiment #19: Background

  • … the unsubscribe rate on a per-message basis does not rise significantly.
     
  • This does not suggest a greater level of irritation, but rather simply more unsubscribe opportunities offered at higher frequencies.

Experiment #19: Background

  • Open rate also does not appear to be significantly influenced by send frequency within the range of frequencies tested.
     
  • There is no significant correlation evident between send frequency and open rate.

Experiment #19: Results

3x Increase in Projected Monthly Revenue
Increasing email frequency yields three times the projected revenue

What You Need to Understand: This company is losing three times its revenue by sending email only once a week instead of every other day. More frequent email sends won’t increase unsubscribes or decrease open rates.

Back to Table of Contents

 

Experiment #20: 20% relative increase in order rate for fitness company

Experiment ID: TP1665
Background: A company offering training tools for professional-grade strength and conditioning
Goal: To increase orders from website
Primary Research Question: Which category page will generate the highest order rate?
Approach: A/B variable cluster test

Experiment #20: Category Page A

Experiment #20: Category Page B

Experiment #20: Side by Side

Experiment #20: Results

20% Relative Increase in Order Rate
Category template A increased visit order rate by 19.9%.

Design KPI % Rel. Change
Version A 1.67% 19.9%
Version B 1.37% -

What You Need to Understand: Removing the copy and moving the customer straight into course selection better matched motivation and increased orders by 19%.

Back to Table of Contents

 

Experiment #21: 13% relative increase in clickthrough rate for fitness company by changing call-to-action, and 61% increase in purchases by changing elements on category page

Experiment ID: TP1631
Background: A company offering training tools for professional-grade strength and conditioning
Goal: To increase orders from the website
Primary Research Question: Which category page will generate the highest order rate?
Approach: A/B variable cluster test

Experiment #21: Category Page A

Experiment #21: Category Page B

Experiment #21: Side by Side

Experiment #21: Results

13% Relative Increase in Clickthrough
The “view details” call-to-action increased email clickthrough rate by 13.04% when compared to the “shop now” call-to-action.

Design Conversion Rate % Rel. Change
Version A 2.3% -
Version B 2.6% 13%

61% Relative Increase in Purchase
The new category template B increased visit order rate by 61.2%.

Design Conversion Rate % Rel. Change
Version A 2.78% -
Version B 4.47% 61.2%

What You Need to Understand: Adding value copy to help the customer understand what a Kettlebell is and how it can benefit them increased the value exchange and increased purchases by 61%. 

Back to Table of Contents

 

Experiment #22: 36% relative increase in sales for auto repair parts company by building the problem on the landing page

Experiment ID: TP1700
Background: An organization that offers car repair products
Goal: To increase overall product sales
Primary Research Question: Which page copy will generate the highest sales conversion rate?
Approach: A/B multifactorial test

Experiment #22: Background

  • A central product page template connected to all channels and visited by all prospects making a purchase

Experiment #22: Treatment

 

 

  • The additional copy was placed at the top of the page.
  • It focused primarily on building the problem.

Experiment #22: Treatment

Experiment #22: Results

36% Relative Increase in Sales
The new page copy increased product sales by 36.1%

Design KPI % Rel. Change
Version A 1.33% -
Version B 1.81% 36.1%

What You Need to Understand: Adding copy to the top of the page that immediately identified the customer’s problem and how the product would fix it increased the value exchange and increased sales by 36%. 

Back to Table of Contents

 

Experiment #23: $3,000,000+ projected increase in revenue per year for storage space company by making simple changes in the sales funnel

Experiment ID: TP1758
Background: A company offering competitively priced, easily accessible storage space for residential and commercial customers
Goal: To increase the number of visitors that complete a storage reservation through the website
Research Question: Which checkout page will result in the highest reservation rate?
Approach: A/B Variable Cluster Split Test

Experiment #23: Version A

Experiment #23: Version B

Experiment #23: Side by Side

Experiment #23: Results

9% Relative Increase in Conversion
The treatment increased conversion rate by 9.10%

Design KPI % Rel. Change
Version A 17.68% -
Version B 19.50% 9.10%

What You Need to Understand: While it might seem like a small increase, the addition of a progress bar in the checkout resulted in a projected $3,000,000+ increase in revenue per year. 

Back to Table of Contents

 

Experiment #24: 36% more total conversions for one-stop vacation planning provider by clarifying the sequence in the checkout process

Experiment ID: TP1621
Background: The research partner is a one-stop vacation planning solution that allows users to book vacation rentals, car rentals, and activities.

Goal: To increase final vacation bookings
Research Question: Which page will yield the highest conversion rate from billing information to confirmation?
Approach: A/B variable cluster split test

Experiment #26: Control

  • The original cart was broken into two (unclear) steps
     
  • The horizontal flow, as well as the blue shading, made it difficult for visitors to get a sense for the sequence of the cart.

Experiment #24: Treatment

  • A simple “step indicator” was added to clearly indicate where a visitor is located in the process
     
  • The treatment also sequenced the two steps vertically.

Experiment #24: Side by Side

Experiment #24: Results

36% Relative Increase in Conversion
The treatment increased conversion rate by 36.10%

Design KPI % Rel. Change
Control 27.40% -
Treatment 37.20% 36.10%

What You Need to Understand: By clarifying the sequence in the checkout process, the treatment generated 36.1% more total conversions than the control.

Experiment #24: Not This, But This

Back to Table of Contents

 

Experiment #25: 45% more Twitter followers and 31% more Facebook fans for ecommerce clothing site by hosting giveaway contests via social media channels

Experiment ID: CS31543
Background: B2C ecommerce site offering men’s and women’s clothing
Goal: To increase engagement and brand awareness among key social media channels
Research Question: What will help grow engagement with our social media channels? How can social media impact sales?
Approach: Giveaway contests via social media channels

Experiment #25: Social Media Campaign

“20 Days of Decent Giveaways”

  • For the duration of 20 days, the brand hosted multiple give-away contests via Twitter and Facebook. To enter, one was required to comment or retweet.
  • Contest timings were random, and entries were only accepted for 30-45 minutes per contest.
  • Winners were selected by a random generator.
  • The contest was mainly promoted on Facebook and Twitter. There was also a rotating banner on their website as well as an initial announcement email.

Experiment #25: Example Messages

Example Facebook Message:
"First Giveaway: We’re giving away 5 pairs of ... Renton and Latika fleece jackets. Reply to this post to enter. We'll pick 5 random winners in 30 minutes. Good luck. LTM Lola“

Example Twitter Message:
"We’re giving away 5 pairs of … Renton & Latika Fleece Jackets. Retweet #WINMJFLEECE to enter to win. We’ll pick 5 randoms at 2:30 EST"

Experiment #25: Results

15% Increase in Sales
The campaign increased sales for products the team used as prizes by 10% to 15%

What you need to understand:

Overall, the team captured 45% more Twitter followers during the effort, bringing their total to more than 5,600. They also captured 31% more Facebook fans, bringing their total to more than 20,000.

“Instead of just a customer re-tweeting a single tweet or replying something random [in Facebook], they really got into it and talked about why they liked the product, why it’s a good product, why they love the brand and why they love [our brand].”

- Gary Wohlfeill, Creative Director

Back to Table of Contents

 

Download a 181-page PDF with these 25 experiments

Related Resources

Drive conversion increases with MECLABS research services
Work with MECLABS Institute scientists to make new discoveries and drive significant increases across your funnel.
Learn More

The MECLABS Guide for Optimizing Your Webpages
This 22-page guide will give you the tactical tools and detailed examples to optimize your campaigns along with a process to support your experimentation.
Learn More