Close
Join 237,000 weekly readers and receive practical marketing advice for FREE.
MarketingSherpa's Case Studies, New Research Data, How-tos, Interviews and Articles

Enter your email below to join thousands of marketers and get FREE weekly newsletters with practical Case Studies, research and training, as well as MarketingSherpa updates and promotions.

 

Please refer to our Privacy Policy and About Us page for contact details.

No thanks, take me to MarketingSherpa

First Name:
Last Name:
Email:
Text HTML
Sep 04, 2003
Blog Post

Surprising Email Test Results: Text vs HTML Survey Versions

SUMMARY: No summary available.
"Everyone talks in general about text vs HTML, but hardly anyone has specific test data they can share on it," Joanne Blatte, the editor of our upcoming Email Metrics Guide 2nd Edition, complained to me last week.

I was about to email about 50,000 of our readers a survey to gather data for our media kit, so I figured, why not toss a test cell into it? Heck, why not toss in two?

Here's how the test worked and what I learned:

We randomly generated three cells (I prefer this to slicing the list into parts so there's less chance of skewing results.)

Group A were sent a text-only note with a link to the survey form online.

Group B were sent an identical note with link, except a graphic of our logo was at the very top which meant the message was HTML.

Group C were sent a virtually identical note with our logo at the top, but they also received the actual survey form in the body of their email. If the form worked in their email system (Lotus Notes, some versions of Eudora and others can't use forms), they could submit their answers immediately instead of clicking to reach a survey.

If the form didn't work, they also had a link to click to get to the survey online.

Every test cell received the same "from" and "subject" line, however the size of messages were obviously different. They surveys were also all identical. (And to my shame, rather badly written -- my fault, never copywrite a survey when you are tired late at night.)

Our list is professionals at work - fewer than 10% are
Hotmail, Yahoo or AOL addresses. I culled newbies because
I didn't feel it was the right stage in our relationship to
ask loads of demographic questions, so every name sent to
had been on our house file for at least 30 days and
received at least four newsletters from us (most many more
than that.)

My expectations were: A low 20% or so open rate due to the fact that it was the week before Labor Day (and also a bank holiday in the UK). I also expected the HTML "lite" to win overall for no good reason beyond gut.

Actual results were:

Group A text-only:
Opens - can't tell with text
% click on link - 8.5%
% of sent completed survey - 7.5%

Group B HTML lite:
Opens - 35%
% click on link - 8%
% of sent completed survey - 7%

Group C HTML form:
Opens - 35.4%
% clicked link to use form online - 1%
% of sent completed survey - 7.2%

Color me completely stunned. Never in a million years did I dream the tests would be so similar.

If the list was not regular readers, I suspect there would
have been more profound differences. Perhaps if people have a strong enough relationship with your brand, your
email format doesn't affect response as much as it would a
newbie?

Anyway, if you've ever conducted a text vs HTML test,
please do let me know what you learned at
aholland@marketingsherpa.com. Thanks
See Also:

Post a Comment

Note: Comments are lightly moderated. We post all comments without editing as long as they
(a) relate to the topic at hand,
(b) do not contain offensive content, and
(c) are not overt sales pitches for your company's own products/services.










To help us prevent spam, please type the numbers
(including dashes) you see in the image below.*

Invalid entry - please re-enter




*Please Note: Your comment will not appear immediately --
article comments are approved by a moderator.

Improve your marketing

Join our thousands of weekly Case Study readers. Enter your email address below to receive MarketingSherpa news, updates, and promotions:
Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions