Close
Join 237,000 weekly readers and receive practical marketing advice for FREE.
MarketingSherpa's Case Studies, New Research Data, How-tos, Interviews and Articles

Enter your email below to join thousands of marketers and get FREE weekly newsletters with practical Case Studies, research and training, as well as MarketingSherpa updates and promotions.

 

Please refer to our Privacy Policy and About Us page for contact details.

No thanks, take me to MarketingSherpa

First Name:
Last Name:
Email:
Text HTML
Apr 16, 2008
Case Study

How Multivariate Test on Banner Ads Boosted Clicks 50.4%

SUMMARY: Running a multivariate test on banner advertising -- instead of relying on industry research -- can prove to be a valuable optimization practice before launching a campaign.

See how a wireless telephone provider tested 36 ad combinations to lift clickthroughs more than 50%. Includes tests on banner ad size, copy and creative ... plus results.
CHALLENGE
Tests that improve creative are *the* most effective online ad tests, according to data from MarketingSherpa’s new 2008 Online Advertising Handbook & Benchmarks. Creative issues were the dilemma that Kevin Kohlmeyer, Emarketing Specialist, Unicel, part of Rural Cellular Corp., faced when it came to optimizing banner advertising across a network.

Kohlmeyer and his team had tried several different copy combinations but didn’t know which worked best. Nor did they know which phone product lured the most consumers to click through and learn more about their services package. Plus, they didn’t know which banner size was optimal.

“For these types of things, we had been looking at industry research to find what certain attributes existed that could be important to consumer prospects,” Kohlmeyer says. “But, we needed to take some steps to verify what really worked for us in particular.”

CAMPAIGN
Kohlmeyer and his team found out “what works” and “what doesn’t work” for Flash banner ads by running an elaborate multivariate test over four weeks. The test included 36 different possible content combinations.

In the end, the goal was to put together a *super banner* with the top-performing creative elements for the campaign. The approach was a stark departure from what they had done in the past with Flash banner advertising.

“We’d put out something like a handset offer or a service-plan promotion across a network,” he says. “A lot of times, it tended to be something limited and something that was consistent with our print ads. But, it was time to do more online.”

The multivariate test showed Kohlmeyer what three key variables, in particular, drive their ads, and what worked best:

-> Key variable #1. Copy

Kohlmeyer and his team tested four lines of copy. Each was attached to an ad with the picture of a cell phone being offered with the service package.

The copy they tested:
o “Buy online and get free shipping on your order”
o “Buy online and we’ll waive your activation fee”
o “Buy online and get free shipping plus no activation fee (over a $30 value)!”
o “Start Shopping Now!”

-> Key variable #2. Product offered

Kohlmeyer knew that pitching consumers with the right name brand can make a big difference. “We looked at what products consumers were selecting once they were already to the site. It made sense that they would bring in prospects as well.”

So, they tested three popular, low-to-medium-cost phones that already were working on their ecommerce site:
o Motorola v197
o Motorola Razr
o Sony Ericsson w200

-> Key variable #3. Banner sizes

Kohlmeyer’s team also knew that using the right banner size could dramatically improve clickthroughs and sales. But which size? (See creative samples below for the winner.)

Here are the three significantly different pixel combinations they included in the multivariate test:
o 300x250 (box ad/often called a “big” ad)
o 160x600 (skyscraper)
o 728x90 (leaderboard)


RESULTS

The testing and numbers crunching paid off big-time for Kohlmeyer and his team. The data generated by the test was used to build the super banner that saw a 50.4% increase in clickthroughs in their next campaign and a “big win” on the sales side, although Kohlmeyer declined to be more specific.

“We’ve seen a healthy overall tick in sales,” he says. “A lot of our sales -- originating from banners -- come over the phone and via other channels. Banner advertising is a significant traffic driver for our multichannel efforts. And this test has helped us improve along those lines.”

Here are the individual test results for each variable as well:
- “Start Shopping Now” copy won out over “Buy online and we’ll waive your activation fee” by 12.2% in clickthroughs.
- “Start Shopping Now” copy got 16.6% more clickthroughs than “Buy online and get free shipping on your order.”

“Getting rid of the ‘loser content’ did help produce a [response] spike. And, what the testing also did was validate some hunches we had or reconfirm what we had seen in the past,” Kohlmeyer says. “At the same time, the fact that ‘Start Shopping Now’ was the best performer was a surprise to us. In the past, we had assumed shipping costs and activation fees were a barrier to purchase and emphasized the fact they were free online. This appears to have been an incorrect assumption. So, we’ll start downplaying it in the future.”

As for which product won:
- Motorola v197 beat the Motorola RAZR by 4.5%.
- Motorola v197 beat the Sony Ericsson by 12.2%.

Kohlmeyer says the margins were enough to feature the v197 in their next banner ad. “It has been a staple in our lineup for a while and has always been a strong seller. It was helpful to know that what sells well is also effective at driving traffic.”

The big winner in the banner size proved that bigger isn’t always better:
- The 300x250 banner (box ad) beat the second-place 160x600 banner (skyscraper) by 27.3%.
- The 300x250 banner trounced the 728x90 (leaderboard) by an even bigger margin.

Data from Sherpa's new Online Advertising Handbook supports this finding. According to a January 2008 survey of marketers by AdJungle for MarketingSherpa, 300x250 ads get the highest clickthrough rate: 62.2% more than skyscraper ads and 27% more than leaderboard ads.

Kohlmeyer says that they had been predominantly using larger pixel sizes like the skyscraper and leaderboard. “We saw that we had to do some downsizing.”

Useful links related to this article

Creative samples from Unicel’s banner ad test:
http://www.marketingsherpa.com/cs/unicel/study.html


DesignBlox - technology provider that made the test possible:
http://www.designblox.com/site/


Kelliher Samets Volk - Unicel’s marketing agency of record:
http://www.ksvc.com


Specific Media Inc. - banner ad network employed on the test:
http://www.specificmedia.com/


Unicel:
http://www.unicel.com


See Also:

Comments about this Case Study

Apr 16, 2008 - Mike Van Winkle of Sleepy Hollow LLC says:
What were the results for: o “Buy online and get free shipping plus no activation fee (over a $30 value)!”


Apr 16, 2008 - Liam Lennon of Aer Arann says:
Great Article - I've also seen the 300x250 and 250x250 banner sizes get a much better response in our most recent campaigns


Apr 17, 2008 - Chris Heine of MarketingSherpa says:
Mike, thanks for the good question. Kevin Kohlmeyer says that offer actually tied for last place with the "Buy online and get free shipping on your order" offer -- also losing by more than 16%.



Post a Comment

Note: Comments are lightly moderated. We post all comments without editing as long as they
(a) relate to the topic at hand,
(b) do not contain offensive content, and
(c) are not overt sales pitches for your company's own products/services.










To help us prevent spam, please type the numbers
(including dashes) you see in the image below.*

Invalid entry - please re-enter




*Please Note: Your comment will not appear immediately --
article comments are approved by a moderator.

Improve your marketing

Join our thousands of weekly Case Study readers. Enter your email address below to receive MarketingSherpa news, updates, and promotions:
Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions