Close
Join 237,000 weekly readers and receive practical marketing advice for FREE.
MarketingSherpa's Case Studies, New Research Data, How-tos, Interviews and Articles

Enter your email below to join thousands of marketers and get FREE weekly newsletters with practical Case Studies, research and training, as well as MarketingSherpa updates and promotions.

 

Please refer to our Privacy Policy and About Us page for contact details.

No thanks, take me to MarketingSherpa

First Name:
Last Name:
Email:
Text HTML
Aug 11, 2006
Case Study

Sierra Club Tests Radical Email Newsletter Redesign to Improve Preview Pane Viewing

SUMMARY: Should you revamp your email newsletter for better viewing in the preview pane? The resulting design probably won't be as beautiful as your old one, but could clicks improve?

This January, Sierra Club tested a radical newsletter revamp to learn just that. In this MarketingSherpa Case Study, see their before-and-after design and discover how their click rates changed ... more than expected:
CHALLENGE
When Heidi Hess joined Sierra Club as Online Program Manager in 2004, one of her first tasks was to help launch a fortnightly email newsletter.

During that first year, Hess and the email team focused on building the permission list to hundreds of thousands of opt-ins. Plus, they carefully A/B tested subject lines for months to determine what sorts of topics and wording would get the highest open rates. (Note: See link below to a list of the Club's subject lines over time.)

For 2005, Hess' team raised the bar again, this time focusing on what specific sorts of content would get the most clicks. As a result, average click rates leapt by a remarkable 63% over the year.

In fact, the newsletter became such an internally-recognized success that the Web team's content building decisions began to center on would email recipients want to click to this? (The opposite is true of many organizations, where email content is driven by the site, instead of the site being driven by email metrics.)

Just as the email program was riding this peak of success, Hess became aware of study data showing that 69% of Outlook email users view messages in their preview panes and that Hotmail and Yahoo users would soon follow suit.

She quickly tested the preview pane in Outlook for herself. Turned out the Sierra Club newsletter, which was beautifully designed for a full open, looked pretty bad in preview. (Link to screenshot below.)

What to do? Should you mess with proven success?

CAMPAIGN
Hess decided to A/B test (using randomly segmented names) a preview-pane friendly design vs her standard template for several issues over December 2005-January 2006.

The new design looked almost exactly like the old design, except for two big differences:

#1. Removing admin messaging from the very top

In line with the email newsletter best practices of the time, the old design had several lines of admin messaging regarding whitelisting and how to click to view an HTML version online if the in-box version was garbled.

This was presented in tiny type, but still took up a solid swathe of white space at the top of the newsletter. When viewed in an in-box, that seemed like too high of a percentage of real estate for admin.

So Hess asked the design team to move that information further down.

#2. Tacking a "boring" Table of Contents at the top

Hess asked the design team to add a table of contents section directly above the top of the old newsletter template.

Instead of Sierra Club's cheerful orange template and logo, viewers looking at the issue in the bottom preview pane would only see a list of clickable headlines in 12-point Verdana. The background color was white. The headlines were hotlink blue. (Link to sample below.)

When they clicked, viewers would be brought down to the section of the regular, colorful newsletter the headline referred to. From there, they could read a summary and decide whether to click to the Web site to see more.

Hess didn't know what to expect from the A/B tests.

However, as the Web design world Web has discovered, when you change your look dramatically, your regular visitors are often disconcerted. Even if the new look and navigation is far more intuitively-usable, often Web redesigns see a slump in visitor conversions or page views for a month or two until the audience gets used to the new version.

(This is one of the reasons why most Web developers prefer to tweak their way to a new design gradually rather than launching full throttle.)

Would email results undergo a similar plummet?



RESULTS

"The test results weren't terribly dramatic," says Hess. Average open rates didn't budge, but then this was expected because a design change should affect clicks far more than opens. However, click rates held steady, too.

So, Hess took a deep breath and decided to roll out the new design to the entire list for 2006. She figured that even if the redesign didn't help as much as hoped, at least it didnít hurt.

But then something lovely happened. Month after month in 2006, the click rates began to swing upwards. A newsletter that had already risen 63% in click rates the year before now rose an additional 55%!

We don't know of any popular newsletter that has seen this type of steady dramatic increase for 24 months. Some have held steady, and many have done slightly better Ö but this leap two years in a row is close to unprecedented.

Hess says she canít lay all credit to the design, much has to do with content as well -- and that's as it should be. However, she is feeling a bit triumphant these days about her preview pane leap of faith.

What's next? She's considering redesigns to improve the look of the newsletter in specific email clients such as Yahoo!'s revamped email in-boxes. She'll segment the list so Yahoo!-users only will get specifically-focused design for their unique in-boxes. It's a great idea.

Useful links related to this article:

Creative samples of Sierra Club's newsletters and preview pane views:
http://www.marketingsherpa.com/cs/sierra/study.html


EmailLabs -- Sierra Club's email service provider
http://www.emaillabs.com


Sierra Club
http://www.sierraclub.org


Comments about this Case Study

Aug 14, 2006 - Lisa Schaertl of Tech Savvy Marketing says:
I'm surprised to see such a high percentage of users using the Preview Pane. I wonder if it applies to business users as well. It is still a security issue, as some malicious email messages need only be viewed to cause damage. With Preview Pane open, the message is viewed automatically. It's far safer to turn it off, and open messages manually. For more, see the Information Week article. http://www.informationweek.com/story/showArticle.jhtml?articleID=191902044. Obviously email marketers need to keep the Preview Pane in mind when designing their messages, but I wonder if we'll see these numbers change.


Aug 14, 2006 - Mark Donnelly of BCAA says:
I found the Sierra Club case study to be very timely and relevant. We've just begun testing and optimizing our monthly B2C newsletter in terms of how it is viewed in the preview pane and will be measuring the impact on click thru rates. One other area I'm also beginning to wonder about (and want to test) as well is the impact of viewing email in mobile devices in particular a Blackberry. With the increase in the number of people viewing email (and making a readership decision based on what they see)on their mobile devices, it seems that this is another area that begs some study and analysis.


Aug 14, 2006 - Shahid Khan of Fast Ramp Marketing says:
Biz type: Non-profit sector (www.HDF.com) After making a table of contents start at the top, we have found that by removing all images from the e-newsletter.....more people receive the e-newsletter in their InBoxes. Previously with images (despite having TOC at the top) almost 90% of the email letters were going into the SPAM box.



Post a Comment

Note: Comments are lightly moderated. We post all comments without editing as long as they
(a) relate to the topic at hand,
(b) do not contain offensive content, and
(c) are not overt sales pitches for your company's own products/services.










To help us prevent spam, please type the numbers
(including dashes) you see in the image below.*

Invalid entry - please re-enter




*Please Note: Your comment will not appear immediately --
article comments are approved by a moderator.