CHALLENGE


His seven years of experience in online marketing told Mark Ogne, VP Marketing RingCentral Inc., a provider of telephony systems for small businesses, that it’s critical to test Web pages that convert visitors into customers.

But at the time RingCentral's management team trusted their intuition and felt their landing page didn't need tests or improvements.

It’s a typical dilemma for marketers. Many company leaders have risen to success by trusting their own gut instincts. Telling them they don’t know the marketplace as well as they’d like to think can be uncomfortable. Plus, your fellow marketers may consider creativity and scientific exactitude opposing values.

No matter what the reason, testing often gets short shrift. Thus, results aren’t as spectacular as they could be.

“We were working with a 2.7% conversion rate to begin with on the site,” says Ogne. “I knew from experience that I could raise that to around 4% with the right analytical foundation.”

First he needed a way to change company culture....

CAMPAIGN


“I ran a black-ops campaign,” he says, with a laugh. Ogne was betting that a few months of behind-the-scenes, low-cost tests could yield such stunning results that he would convert his management team to the wonders of the test.

-> Six tips for running a black-ops testing campaign (or just a quick, affordable, easily managed one):

Tip #1. Test only one page at a time.

This seems obvious, but it’s surprising how many marketers will change every page on a site, test performance and then realize that they don’t know which changes had the greatest effect. By testing one page, you turn the rest of your site into a control mechanism and ensure you get valid results.

Which one should you pick? One that focuses on an action item that's required at some stage of turning a visitor into a prospect, or a prospect into a sale. You need to show that your changes made the difference in actions that lead to the bottom line.

Tip #2. Try fractional factorial testing.

Ogne’s web analytics vendor used the Taguchi Method of fractional factorial testing, but there are others. In this type of test, multiple variations of a page’s elements (for example, headline, graphic, copy length, and background color) are tested in different combinations.

By simultaneously testing multiple variants, one can divine the theoretical best choice in each category and then create a “best recipe” that combines those choices, even if a page with that best recipe was never actually tested. As it tests so many variants at once, the Taguchi Method enables you to draw valid conclusions from just a few test subjects.

This is a huge bonus if you have limited page traffic (as many b-to-b marketers do), limited creative resources (so you're not asking the design department to churn out endless page variations for you) and/or a short test period.

“The Taguchi Method was originally created for manufacturing, and there are people who say it doesn’t apply to marketing,” says Ogne. “But, I think it’s probably even more valid, because there’s really only one outcome in marketing: did they convert or not. In manufacturing there are cost and quality issues that are even more complicated.”

Tip #3. Use an ASP, not enterprise, web analytics package.

If you are going covert, or if your IT team just doesn’t have the time or the inclination to help you, use ASP analytics, where the data is stored and analyzed by your vendor.

This will help maintain marketing control over the project and could vastly reduce the need for IT support. It’s also generally easier to modify tests on the fly using an ASP.

Tip #4. Use a page-tagging (client side) analytics package.

Analytic software generally comes in two flavors: page tagging (client side) and server-side. Without going into all the nuances, if you want to collect data without enormous support and help from your IT team, pick a firm that uses page tagging.

If you do choose server-side collection, pick a testing firm that posts pages themselves onto their own server, not yours, so that you don't need to ask your IT department for much help.

Tip #5. Warn search engine spiders away from extra test pages

You could be penalized by search engines if your site appears to have several duplicate (aka mirror) pages. It's a good rule of thumb to always include basic code that tells search engine spiders that the almost-duplicate test pages are not for their surfing pleasure.

(Note: The software Ogne used didn't have this requirement, but it's important enough that we felt we had to warn you about it.)

Tip #6. Chose an inconspicuous page

Start with a page that your management doesn’t visit often, especially if your organization requires miles of red tape before content changes can be posted to the site. Just be sure that page is critical to the conversion process so you have a measurable result you can use for political gain later!

Ogne decided to test RingCentral's then 450-word search engine landing page. (Link below to before-and-after screenshots.)

He didn't have much time or budget, so he only tested the critical factors that he knew from experience would have large immediate impacts. The goal was to show an obvious difference in results through testing, not to test every possible tweak for a perfectly optimized page. So, the items he chose were:

- Headline tests Switching from the factual phrase "Get a toll-free line" to a benefit-oriented statement which began, "Look like a Fortune 500 Company."

- Pictured person test Switching from a somber white man in a button down dress shirt to a smiling young black woman. (Demographically, women are far more likely to start small businesses than men are.)

- Slashing copy length Ogne shortened the copy by nearly 50% on the page. However, he added a series of three bulleted more info links centered in the copy so visitors who did want more details could click to them. The shorter copy had a slightly higher use of the word "you" as well.



RESULTS


The four-month average conversion prior to testing was 2.7%. Since the tested and optimized page launched in November, average conversions of visitors clicking on a link to start a trial, leapt as high as 62.9% above past conversion rates.

November 2004 conversions reached 3.9%; December lagged a bit at 3.5% as visitors had vacation on their minds more than business. Then, conversions leapt to 4.4% during the first two weeks in January 2005. “I’ll take that!” Ogne says.

As we've heard from other site marketers, visitors who clicked on a "learn more" link wound up converting at a higher rate than average. However, that's not cause-and-effect evidence.

Ogne discovered a data point that was entirely new to us -- turns out weekend visitors were more likely to spend more time on the page and were more likely to convert from longer-copy pages. If you, like most marketers, have lower weekend conversions than weekday conversion percentages, you should try testing out longer copy for weekend visibility only.

When Ogne carefully unveiled his test results to management, they were completely won over.

In fact, there is so much enthusiasm for the concept that Ogne’s next challenge is to make sure that future tests proceed as methodically as the ones that already worked so well. “I’ve opened a bit of a Pandora’s box when it comes to testing,” he jokes.

Management's first test was the site's pricing page. Taguchi methods are perfect for this because so few visitors are required for statistically reliable results. So, you run less risk of confusing your marketplace with conflicting offers.

So far the team has learned that the fewer the pricing offers visitors have to choose between, the better the conversion rate. They also learned in their market at least small business owners prefer variable pricing to flat rate plans. Removing the highest price offer from the page definitely raised conversions... but it also lowered profits, so it wasn't a total win. The team is still testing now.

Useful links related to this article:

Creative samples from the campaign: http://www.marketingsherpa.com/ring/study.html

Harvard Business School article on experimental design related to multivariable testing with small respondent pools (there is a fee for accessing this article): http://harvardbusinessonline.hbsp.harvard.edu/b01/en/common
item_detail.jhtml?id=R0109K

Offermatica, the web analytics firm who performed the tests and analysis for RingCentral.com: http://www.offermatica.com

RingCentral: http://www.ringcentral.com