Are you tearing your hair out worrying about filters stopping your email messages to your house opt-in list? Results from a six-week study conducted by Pivotal Veracity and released exclusively to MarketingSherpa reveal that you're in very distinguished company indeed.

Turns out 54 major mailers from AARP to The Wedding Channel can't get their permission email messages past filters at Hotmail, Yahoo, and/or Gmail all the time either. Even AOL's own outgoing email newsletter department couldn't get all of its bulk mailings past other ISPs' filters.

Plus, even worse news, 18% of filtered messages were transactional -- such as Welcome messages to new opt-ins.

For more details on how the study was conducted and findings, scroll down to the end of this story for a link to your PDF copy. In the meantime, here's our quick summary of findings.

Names of 54 mailers whose mail was filtered

Nonprofit & Govt.: American Red Cross AARP Federal Government: Department of Education Federal Government: FDA National Geographic

Business-to-Business: ePiphany IBM Juniper Networks Postini (yes, the filtering company) SAS WebEx

Retail: 1800 Flowers Academy Sports & Outdoor Bloomingdales Buy.com Coldwater Creek Crutchfield Domino's Pizza DVD (Infinity Resources) LL Bean Macy's Marks & Spencer Neiman Marcus Polo Smart Bargains Target The Wedding Channel

Travel: Expedia Hotwire Orbitz Travelocity

Other: Business-to-Consumer American Consumer Opinion (Decision Analyst) Home Gain Johnson & Johnson Nokia Verizon Postmaster Direct (double opt-in list)

Media: Agora Publishing AOL's CityGuide Businessweek CNET Crain's Discovery Networks HBO Newsweek The Motley Fool The Wall Street Journal

Pivotal Veracity's President & CEO Deirdre Baird confided to us that the mailers with the biggest problem -- folks whose mail almost inevitably went straight to the spam folder -- were Agora Publishing and the US Federal Government. (Your tax dollars hard at work.)

In both of these cases, the mailings were primarily text-only, so the whole avoiding-HTML-to-bypass-filters idea doesn't always work. Do any tactics to avoid filters actually work? Read on. ...

Does outsourcing to an ESP help you avoid filters?

Admission -- We strongly recommend that mailers outsource to a reputable email service provider (ESP) unless you have a significant (read: six-to-seven figure) budget for an in-house department. A good ESP helps you with volume-sending controls, list hygiene, and many more factors critical to delivery.

So we were stunned when this study revealed major mailers using ESPs were just as likely to be filtered as major mailers doing it in-house.

How could this be? Turns out three items affected results:

o Major mailers -- These bigger organizations tend to invest significantly in their email systems, with dedicated staff, servers, and excellent software.

o Content filters -- Although an ESP can help you with technical filter problems (volume, hygiene, etc.), you're deciding what the graphics and text of your email will be. And content often dictates filtering.

o Shared IP addresses -- Did you ask your ESP to send your mail from a dedicated IP address? You may have said "no" because it may cost a little more. But, as we've reported ('til we're blue in the face), if you share an IP address, your filter-risk goes up dramatically. Why? Because if anyone else using that IP address gets in trouble, you're in trouble too because you "look like" the bad mailer to most filters.

So, yes, using an ESP still makes sense for most mailers. But no, don't blithely count on them to get you past filters all by themselves. You've got responsibility, too.

Do TRUSTe clients get past filters?

As we suspected, although TRUSTe seals may help the public feel safer doing business with you online, filters don't track which mailers have TRUSTe seals. (Nor, to be fair, does TRUSTe claim they do.)

Unfortunately the study wasn't able to collect enough data on other mailer certification programs such as Bonded Sender to see if these really work. (Gmail and Yahoo don't use Bonded Sender to evaluate incoming mail.)

The good news is the folks at Bonded Sender and several other programs have agreed to help MarketingSherpa run our own study about deliverability. So we'll run a story with info on this in a few weeks.

In the meantime, if you're investing in any certification program we strongly suggest you track delivery by ISP to see if it's helping you. See below for more details on that.

Does level of permission help you get past filters?

All but one of the mailers studied were opt-in or double opt-in mailers -- meaning an individual had to proactively sign up to get email from them. So, although we've got plenty of anecdotal evidence that opt-out mailers tend to get filtered more (way more), this study doesn't address it.

The good news is double opt-in mailers were 20 points more likely to avoid filters than single opt-in mailers. (Double opt-ins had a 39% filter rate and single had a 59% rate.)

The bad news is filters *did* stop some double opt-in confirmation messages from getting through. So if you want to require double opt-in, getting new names to take the extra step may be impossible because they'll never get your message asking them to. (Can you say, "Arrrgh!"?)

Does your email delivery report reveal the whole truth about filtering?

The data is incredibly confusing.

-> Filter software companies claim they have teensy-weensy false positive rates (under 2%).

-> Most email service providers claim delivery rates of 90%-98%.

-> Multiple industry studies show 20% or more of permission email doesn't reach recipients' in-boxes (and worse for at-work addresses).

Who's telling the truth? As this study reveals (in refreshingly clear language), everyone is. That's because everyone is measuring completely different things. Example, filter companies include all true spam in their data, which is roughly 80% of email.

The key for you as a mailer is to know how much of your own mail is delivered. The problem is, the so-called 'delivery report' most ESPs and email systems provide does *not* track delivery.

The only thing most delivery reports track is mail sent minus bounce messages received in return. So, if you sent an email to 10 people and one of them bounced a "mailbox full" message back, then your email report would tell you that you had a 90% delivery rate.

You probably didn't. Because most filters don't bounce a message back telling your email system that the mail didn't get through ... or if it went into a black hole.

If you require real deliverability data, you have to ask for it. And yes, it will probably cost you extra. You'll need to use a formal delivery tracking service. (And no, the folks at Pivotal Veracity did not pay us to say that.)

If your budget is tight, you can opt for a do-it-yourself program by planting seed names at a bunch of ISP mailboxes and then check results manually. It's not perfect but better than nothing else.

One other idea -- Get your email department to sort open and/or click scores by recipient ISP. It will be awfully obvious if you're not getting any mail through to one of the biggies. But, again that's work.

No matter how you choose to do it, we recommend you budget for a delivery audit on a regular basis if email is important to your organization (or you have a legal obligation to get mail to certain users). Good luck.

Useful links related to this story

Your PDF copy of the complete study: http://www.marketingsherpa.com/Pivotal-Veracity-May-2005.pd


Pivotal Veracity: http://www.pivotalveracity.com

Bonded Sender: http://www.bondedsender.com