Close
Join 237,000 weekly readers and receive practical marketing advice for FREE.
MarketingSherpa's Case Studies, New Research Data, How-tos, Interviews and Articles

Enter your email below to join thousands of marketers and get FREE weekly newsletters with practical Case Studies, research and training, as well as MarketingSherpa updates and promotions.

 

Please refer to our Privacy Policy and About Us page for contact details.

No thanks, take me to MarketingSherpa

First Name:
Last Name:
Email:
Text HTML
Join Our Research Team at DMA 2014
Jan 08, 2008
Case Study

Why It’s Best to Test Your Own Email Practices - Delivery & Subject Line Tests Results

SUMMARY: Homing in on the best day of the week to send an email and writing great subject lines are always key ways to raise response rates. What to do? Test, of course. See how an eretailer tested their enewsletter for the best delivery day and a shorter subject line. And they got some surprises, too, which they turned into a happy opportunity to lift conversions 50%.
CHALLENGE
The Organic Dish was continually disappointed with their monthly enewsletter -- particularly open and clickthrough rates -- even after following industry best practices.

“We had reached the point where we were malcontented about every aspect of our performance,” says Toby Hemmerling, Managing Partner, The Organic Dish. “I was certain that the general content wasn’t the culprit in terms of why we weren’t getting a better response.”

Hemmerling’s enewsletter advertises updated menus for their organic meals, so it’s critical to revenue. He and his team questioned their email delivery day. Days that were early in the week, like Monday and Tuesday, seemed good for their email, but would one day or the other make much of a difference? They also wanted to know which was more important: a stronger call to action or shorter subject lines.


CAMPAIGN
Hemmerling and his team ran two tests to determine the best day of the week for emails and to fine-tune their subject lines:

-> Test #1. Best email delivery day

o Step #1. Review current metrics

Learning the best day of the week to send email was of great importance to them, so the first thing they did was dig into the data they already had.

Metrics from earlier campaigns showed that their email response typically peaked in the beginning of the week, tapered off by Wednesday and then picked up again on Thursday and dropped after that. “From looking at industry reports, I thought that Monday, in particular, was simply a good, safe day for email.”

o Step #2. Review website traffic

After reviewing data about their website traffic, they found that Tuesday was their busiest traffic day, with Monday surprisingly lower. They tested whether email sent out on their busier website day (Tuesday) outperformed messages delivered on Monday.

o Step #3. Send at same time, different days

Their analytics also told them that their peak traffic time was 11 a.m. So, Hemmerling and his team sent an identical message on consecutive days:
o 10:30 a.m. Monday
o 10:30 a.m. Tuesday

“We wanted to send it a half hour before it got busy. We wanted a time [cushion] to hit as many people as possible.”

-> Test #2. Compare subject line length and content

o Step #1: Write different subject lines

Hemmerling and his team wrote two competing subject lines:
- A longer, stronger call-to-action appeal.
- A shorter, less-direct but still eye-catching phrase.

Both subject lines focused on a buyer-friendly policy that hadn’t been widely advertised: switching from requiring four total meals per order to two. Here were the two subject lines:

o The Organic Dish: Now only 2-meal minimum order (47 characters)
o The Organic Dish: A New Way To Order (36 characters)

o Step #2: Split the customer list

They took 20% of their total list for the test and split it 50-50 for a Monday morning mailing. “We wanted the longer version to be a stronger call-to-action, but not the most direct call-to-action at the same time.”


RESULTS

Hemmerling and his team learned that it can be best not to follow the herd and to always test your efforts for yourself.

o Their website’s busiest day of the week, Tuesday, was also their best email day -- not Monday as they had been led to believe.

“We saw a 10% difference in open rates and a 15% disparity in clickthroughs in favor of Tuesday over Monday. In terms of sales, it was no comparison. We did a ton more business on Tuesday. While we are not going to stop testing other days, right now, we are sending our regular email on Tuesday.”

o The strong call-to-action subject line significantly beat the shorter subject line. It had an open rate of 44.12%, outpacing the shorter line’s rate of 35.2%.

“We found out that shorter is not always better. Secondly, we learned that we required the recipient to think too much with the shorter version, which was less direct.”

o Clickthrough rates were even more dramatic in favor of the call-to-action: 17.14% to 5.8%.

“The clickthroughs discrepancy surprised me because it’s hard to gauge how big of an impact subject lines can have [after the open]. It is possible that the stronger call-to-action in the subject line gets folks further down the sales funnel.”

Because the message for both subject lines was identical inside the email, Hemmerling and his team couldn’t gauge exactly how the subject lines affected sales. But after they sent out an email with the longer subject line in the same campaign the next day to the remainder of the list, there was a big boost in sales.

“We were happy to see [great] sales in the potentially quiet week of Thanksgiving with that subject line. We had approximately a 50% increase over our normal campaign.”


Useful links related to this article

Creative samples from The Organic Dish:
http://www.marketingsherpa.com/cs/organicdish/study.htm


MailChimp - email service provider aiding with the tests:
http://www.mailchimp.com


Google Analytics - their Web analytics solution:
http://www.google.com/analytics/


Clicktail - provided heatmaps and other analytics:
http://www.clicktail.com/


The Organic Dish:
http://www.theorganicdish.com/


See Also:

Comments about this Case Study

Jan 16, 2008 - Matt Carroll of www.Placeworld.net says:
Enjoyed this! A short, bad subject line might not be better, but otherwise shorter will usually be better. How about moving "The Organic Dish" to the from line and using something shorter and more concise: "4-Meal Minimum Reduced to 2" (27 characters)? I'm not sure how any of these subject lines are directly related to a call to action. There seems to be some confusion about what a CTA is.



Post a Comment

Note: Comments are lightly moderated. We post all comments without editing as long as they
(a) relate to the topic at hand,
(b) do not contain offensive content, and
(c) are not overt sales pitches for your company's own products/services.










To help us prevent spam, please type the numbers
(including dashes) you see in the image below.*

Invalid entry - please re-enter




*Please Note: Your comment will not appear immediately --
article comments are approved by a moderator.

Improve your marketing

Join our thousands of weekly Case Study readers. Enter your email address below to receive MarketingSherpa news, updates, and promotions:
Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions