Join thousands of weekly readers and receive practical marketing advice for FREE.
MarketingSherpa's Case Studies, New Research Data, How-tos, Interviews and Articles

Enter your email below to join thousands of marketers and get FREE weekly newsletters with practical Case Studies, research and training, as well as MarketingSherpa updates and promotions.


Please refer to our Privacy Policy and About Us page for contact details.

No thanks, take me to MarketingSherpa

First Name:
Last Name:
Apr 12, 2012
Case Study

Website Optimization: Simple A/B test adds 'thousands and thousands of dollars' to Cars.comís bottom line

SUMMARY: Some marketing campaigns are complicated, while others are relatively simple and produce almost immediate results. As an example of the latter, see how an e-commerce company added an online security badge and ran a one-off A/B split test.

The company changed one simple element on an e-commerce homepage that resulted in a boost in conversion, and translated into thousands of dollars in monthly revenue.
by David Kirkpatrick, Reporter


A large proportion of MarketingSherpa case studies involve many steps, and cover a wide range of marketing activities. And for good reason, as most marketing changes that make an impact are very complex.

However, sometimes an effort comes along that was relatively simple, but ended up making a significant difference for the highlighted marketer.

Todayís case study is one of those efforts that looks simple, and in hindsight maybe even obvious, but resulted in significantly improving the online conversion of one business. has three websites --, and -- that are all lead generation websites with the goal of creating consumer leads for car dealers.

The business model is not complicated. The company drives high amounts of traffic to the site, so any boost in conversion has a direct benefit to the bottom line.


This marketing effort involved an A/B test with a security badge on designed to reduce visitor anxiety and improve conversion.

Step #1. Drive traffic to the website

Vince King, Director of Consumer Publishing,, said the company does a lot of search engine marketing (SEM) to drive traffic to the websites.

A major portion of Cars.comís marketing budget is spent on SEM, and any increase in conversion is significant because the three consumer websites receive around 1,000,000 unique visitors combined each month.

King said, "Weíre always looking to figure out how weíre going to increase conversions. We define conversions as somebody who comes to our website, goes to the form page, chooses some dealers and hits Ďsubmit.í"

Cars.comís search engine marketing is based around paid ads tied to automobile makes and models for all three consumer-facing websites.

"If you were to search for prices for a new Ford Mustang, you would probably see an ad for one of our three websites on Google and Bing," explained King.

Step #2. Test the website for security issues

Before was able to place the security badge on its website for testing, the vendor supplying the badge tested the site for security vulnerabilities. This test scanned from the network perimeter, to the Web application layer.

The scan also looked at potential problems, including:
  • Unintentional links to malicious websites

  • Browser exploit code

  • Accidental sharing of email data with spammers

  • Hacker testing

  • SQL injection

King said an additional benefit for was that after its websites passed the security test, the vendor had a browser plug-in that placed an additional security icon next to search engine ads for anyone who had installed the plug-in.

Step #3. Run the A/B test

After passed the website security test, the security badge was placed on the website for the A/B split test.

For 35 days, website traffic was randomly driven on a 50/50 basis to a homepage with the badge, and to a homepage without the badge.

The test reached statistical confidence after one month, but ran for an additional five days.

King said the vendor provided a number of different badges to choose from and made it very easy to set up the testing protocol.


The homepage treatment with the security badge had a 2.66% increase in online lead conversions.

The badge was then added to all three consumer websites, and given the high volume of website traffic, that level of lift in conversion represents "thousands and thousands of dollars" in monthly revenue, according to King.

"I heard that security, or trust, labels were important to help conversion on e-commerce and lead generation sites, so logos are important to consumers to give them the confidence in moving forward with their personal information," said King.

If you found this case study useful, you can sign up for our free bi-weekly Consumer Marketing newsletter.

Useful links related to this article

CREATIVE SAMPLE: Security badge on

McAfee -- Cars.comís security badge vendor

Optimization 101: How to get real results from A/B testing

Quick Lift Ideas: 8 test ideas to help you increase conversion across your site

Website Redesign: Customer surveys and testing help increase conversion by 21.6%

Anxiety: Use privacy as a competitive advantage

7 signs that youíre overvaluing search engine optimization

Marketing 101: What is conversion?
See Also:

Post a Comment

Note: Comments are lightly moderated. We post all comments without editing as long as they
(a) relate to the topic at hand,
(b) do not contain offensive content, and
(c) are not overt sales pitches for your company's own products/services.

To help us prevent spam, please type the numbers
(including dashes) you see in the image below.*

Invalid entry - please re-enter

*Please Note: Your comment will not appear immediately --
article comments are approved by a moderator.

Improve Your Marketing

Join our thousands of weekly Case Study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions

Best of the Week:
Marketing case studies and research

Chart Of The Week

B2B Marketing

Consumer Marketing

Email marketing

Inbound Marketing

SherpaStore Alerts


We value your privacy and will not rent or sell your email address. Visit our About Us page for contact details.