CHALLENGE
Kate O’Neill, Director, Customer Experience and Product Development, Magazines.com, has 1,800 magazine titles to market with a myriad of potential audiences. A standard tactic is to place a widget on the homepage for limited-time, special offers.

Early this year, her team launched such a campaign for O, The Oprah Magazine. They decided to run an audience-segmentation test to discover the best way to present that special offer to homepage visitors. “Our publisher relations and merchandising groups had such a keen interest in making sure we got exposure to that offer, and my team was interested in making the most out of it.”

Initial data from the homepage test showed that visitors who had specifically searched for O, The Oprah Magazine, were converting at a higher rate than any other segment. O’Neill wondered if what they had discovered was the best overall search landing page experience for visitors.

CAMPAIGN
O’Neill’s team created several multivariate and A/B tests around the limited time 67% discount offer for a year’s subscription to the magazine. They first wanted to optimize the copy and call-to-action that was needed to promote the offer on the Magazines.com homepage.

By funneling search traffic toward that offer, they also examined the impact of using the homepage as a search landing page – instead of using a product-specific landing page.

Here are four steps they took to gain valuable insights into user behavior:

-> Step #1. Multivariate test of homepage offer

O’Neill’s team used their standard tactic for placing limited-time, special offers on the homepage: a “spotlight” widget that was placed on the upper right of the page.

Within that space, they tested three copy variations to promote the offer:

#1: Standard configuration featuring a magazine cover image, the special price, indication of the discount and a “buy” button.
#2: Standard configuration with additional red text that said, “Limited Time Offer!”
#3: Standard configuration with additional red text that indicated a specific end-date for the offer -- “Special Offer! Ends March 31.”

-> Step #2. Segment reporting by visitor categories

For each of the variations, they tracked response rate by user categories:
o First-time visitors
o Repeat visitors
o Previous conversions

O’Neill also wanted to highlight response rates from visitors who had come to the site from a specific search for O magazine. The team’s standard search marketing tactic sends visitors to a product-detail page about each magazine.

Previous data had shown that many visitors who land on a product-detail page end up clicking over to the homepage for more information about Magazines.com instead of converting on the landing page. “We thought they might not be getting oriented very well and were asking themselves, ‘Who is this company? What is this site?”

They ran the homepage test for one week.

-> Step #3. Conduct A/B test of homepage

Visitors who had specifically searched for “O” converted at a much higher rate than other visitors, according to initial data from the homepage test.

“We thought, gee, maybe this is the trick -- put them on the homepage, but make it easy to spot the thing that they were trying to find.”

At that point, O’Neill and her team conducted a two-week test to determine whether the homepage with the special offer would outperform their standard product-detail landing page for search traffic:

- Half of the Oprah specific search traffic was directed to the O magazine product- detail page.
- Half of the Oprah-specific search traffic was directed to the homepage, featuring the special offer in the Spotlight widget.

-> Step #4. Run additional search tests with other titles

To determine how different audiences would respond to a homepage as a search landing page, the team ran similar tests with other magazine titles. Again, they split search traffic between the corresponding product-detail page and the homepage that featured a widget for that title.

They selected some titles that had very different audience demographics than Oprah, such as Money magazine and Golf Digest. They also selected titles that would appeal to a similar demographic, such as Real Simple.


RESULTS
The response from visitors who first came to the site from an Oprah-specific search, but then clicked over to the homepage was overwhelming: They generated a 470% higher conversion rate than the rest of the homepage visitors did.

“This was clearly suggesting, at least in this case, that the homepage look and feel was a lot more orienting to these visitors than the product detail page,” says O’Neill. It also indicates that serving up a relevant product-specific offer on the homepage can be a good conversion source.

Things got murky when the team began sending magazine-specific search traffic directly to a homepage/special-offer combination as the landing page:

- The homepage as landing-page option achieved only single-digit improvements over the product-detail page for Oprah-specific searches.

- The homepage as landing-page option *underperformed* the product-detail page for all other titles tested.

The discrepancy in results shows how much variation there can be in user behavior and reaction to a specific marketing tactic. “We’re still in the position of trying to find that sweet spot,” O’Neill says. “For any given audience or title, what is the effect of feeling oriented in our site, and what is the effect of the thrill of discovery?”

The test is helping inform the development of Magazine.com’s new content-management system and merchandising platform. The results show that O’Neill’s team needs a platform that can accommodate different merchandising and marketing techniques – different pages for different viewers, depending on their search or click activity.

“It’s definitely having a real-life impact in how we’re developing our systems, with the end result being that we can develop a more customized user experience.”

In the process, the homepage test of the three offer variations produced a clear winner. The widget with “Limited Time Offer!” beat the control and the variation with an expiration date. Now, O’Neill’s team is planning additional tests around that spotlight-offer style to produce broader benefits.

Useful links related to this article

Creative samples from Magazine.com’s landing page tests:
http://www.marketingsherpa.com/cs/magazine/study.html

Past Sherpa articles on landing pages -
How to Optimize Landing Pages to Maximize Holiday Traffic Conversions:
http://www.marketingsherpa.com/article.php?ident=30552

How to Increase SEM Conversions 280% - Landing Page Test Results:
http://www.marketingsherpa.com/article.php?ident=30307

Landing Page Report: New Tests & Tactics to Lift Conversions - Special 19-Page Download:
http://www.marketingsherpa.com/article.php?ident=30193

MarketingSherpa Landing Page Handbook:
http://www.SherpaStore.com/LandingPageHandbook.html

Omniture - helped with multivariate and A/B testing:
http://www.omniture.com

Magazines.com:
http://www.magazines.com