"Everyone talks in general about text vs HTML, but hardly anyone has specific test data they can share on it," Joanne Blatte, the editor of our upcoming Email Metrics Guide 2nd Edition, complained to me last week.
I was about to email about 50,000 of our readers a survey to gather data for our media kit, so I figured, why not toss a test cell into it? Heck, why not toss in two?
Here's how the test worked and what I learned:
We randomly generated three cells (I prefer this to slicing the list into parts so there's less chance of skewing results.)
Group A were sent a text-only note with a link to the survey form online.
Group B were sent an identical note with link, except a graphic of our logo was at the very top which meant the message was HTML.
Group C were sent a virtually identical note with our logo at the top, but they also received the actual survey form in the body of their email. If the form worked in their email system (Lotus Notes, some versions of Eudora and others can't use forms), they could submit their answers immediately instead of clicking to reach a survey.
If the form didn't work, they also had a link to click to get to the survey online.
Every test cell received the same "from" and "subject" line, however the size of messages were obviously different. They surveys were also all identical. (And to my shame, rather badly written -- my fault, never copywrite a survey when you are tired late at night.)
Our list is professionals at work - fewer than 10% are Hotmail, Yahoo or AOL addresses. I culled newbies because I didn't feel it was the right stage in our relationship to ask loads of demographic questions, so every name sent to had been on our house file for at least 30 days and received at least four newsletters from us (most many more than that.)
My expectations were: A low 20% or so open rate due to the fact that it was the week before Labor Day (and also a bank holiday in the UK). I also expected the HTML "lite" to win overall for no good reason beyond gut.
Actual results were:
Group A text-only: Opens - can't tell with text % click on link - 8.5% % of sent completed survey - 7.5%
Group B HTML lite: Opens - 35% % click on link - 8% % of sent completed survey - 7%
Group C HTML form: Opens - 35.4% % clicked link to use form online - 1% % of sent completed survey - 7.2%
Color me completely stunned. Never in a million years did I dream the tests would be so similar.
If the list was not regular readers, I suspect there would have been more profound differences. Perhaps if people have a strong enough relationship with your brand, your email format doesn't affect response as much as it would a newbie?
Anyway, if you've ever conducted a text vs HTML test, please do let me know what you learned at firstname.lastname@example.org. Thanks
The views and opinions expressed in the articles of this website are strictly those of the author and do not necessarily reflect in any way the views of MarketingSherpa, its affiliates, or its employees.