August 14, 2006
Blog Entry

The Annoying Imprecision of Email Open Rate Metrics

SUMMARY: No summary available.
By Anne Holland

I've started calling email open rates "measured opens" because just saying "open" alone is so misleading.

The annoying imprecision of email opens as a metric has been a problem for a decade now. Yet, every 18 months or so, a whole new wave of marketers discover it. Then, many of them email me dismayed queries. "Is it true? Is email open measurement really so vague?"

Yes, I write back. Precise email opens are an unknowable metric according to today's technology.

First of all, obviously no text-only email has a measured open rate. Measurement depends on HTML, which doesn't exist in text-only.

Second, if your recipient's inbox is blocking HTML (as an increasingly large number do these days), they may open but they won't see your lovely graphics, nor will the open be reported back to you.

Third, and this appears to be the biggest area of current confusion, when an HTML email is viewed in a 'preview pane,' that absolutely counts as an open.

According to an EmailLabs study, 69% of at-work email recipients always or frequently use their preview panes when sorting through their inbox. These are mostly Outlook users, but in the consumer world Hotmail and Yahoo are said to be offering preview panes to their users shortly.

I've encountered many myths about this preview pane open. Most center on the idea that preview pane opens can be measured separately (or eliminated completely) from "regular" opens.

"What if we move the HTML pixel that reports opens to the very end of the creative where the preview pane won't trigger it?" asked one marketer recently. Nah, sorry, won't work. When an email is open in preview, according to your stat reports it is completely open.

Besides, 33% of preview pane users admit they personally consider the preview pane to be an "open." They rarely if ever open all the way, preferring to scroll up and down in that small box. (Have you told your email creative team yet to make your templates look great in the tiny box?)

On the other hand, for preview pane users, the email opens
automatically when they are reviewing their in-box. I know I've "opened" many messages without intending to as I surfed my in-box. (And then worried that the resulting "open" might tell the junk mailer in question that my email account was definitely interested in more of their crud. Blech.)

Open rate confusion perhaps arises because most email metrics reports look so darn precise with decimal figures and all. Also, most marketers are used to being able to get incredible Web analytics and can't believe the same isn't available for email.

Your email open reports can be used to spot major problems in delivery (if not a single AOL user opened your last newsletter, you know you have a problem.) And, they're not bad for fairly general, broad "health of this list/campaign" reports where you're watching trends over time. Plus, if you're running subject line tests with an A/B split, a dramatic difference can be quite useful.

But the real metrics you should be watching are clicks and clicks-to-conversions (whatever your conversion activity may be ... from pageviews to purchasing.)

Which means it's time to tie your Web analytics -- and if possible other data such as in-bound calls, printed coupon redemptions, brand perception, loyalty, etc. -- to your email campaigns.

Unfortunately, this is a nasty tech back-end project for many organizations, involving silo-ed systems and databases. If email really matters to your bottom line, you'll do it. Otherwise you'll continue to slide along guestimating success based on general open and click trends.

If you are one of the former -- or if you have some wonderful new tech to measure email that I'm not aware of -- please post a comment to this blog.

(Yes, we're accepting posted comments on *all* MarketingSherpa stories and Case Studies now at our site so that you can share your insights with the community.)

Sponsor: Get $300.00 off MarketingSherpa B-to-B Summit:
Network with your peers in Boston, Oct. 23-24,
or San Francisco, Nov. 13-14:

Case studies from:
B & B Electronics
Everon Technology Services

Learn more:
Or call 877-895-1717

Improve Your Marketing

Join our thousands of weekly case study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions