"I've been there. I know what's behind the curtain. It's ugly." -- Former major ESP VP who switched to a related industry in 2005.

"This industry has *got* to grow up!" -- Top exec at an independent email deliverability firm.

"Choosing between ESPs is about choosing the lesser of two evils. No matter who you go with, there's going to be some pain. That said it's still better than broadcasting in-house." -- Internet Director at a trade publishing company.

Quick industry background: rocket-like growth 2001-2005

It's pretty darn easy to set up shop as an email service provider. Get a mail server, license some software, whip up a Web site, and you're in business. (Of course that doesn't mean you're any good at it.)

During 2001, that's exactly what plenty of people did because, aside from search marketing, email was one of the few online businesses that was booming. Loads of eager clients hoped to save money by replacing postal mail with email, and very few knew exactly which penetrating questions to ask a would-be vendor.

Then from 2002-2004, a merger and acquisition frenzy took place. Big fish ate small fish; small fish got venture funding for roll-up strategies; and faced with unexpected competition, some vendors disappeared completely.

In 2005, the remaining estimated 65 firms worked frantically to:

- stabilize, hire and train their staff;

- handle an explosion of incoming clients (some gained hundreds of client accounts in 2005 alone);

- develop specialized services to compete in particular niches;

- add on extra bells and whistles ranging from strategic consulting to high-end dynamic content to lift their prices out of commodityland;

- educate clients to avoid all-too-common newbie mistakes;

- and, ensure deliverability in an increasingly difficult ISP landscape.

Yes, you could call it a high-stress industry. And we've heard from many ESP leaders that they expect another round of M&A and marketing wars to occur in 2006 until the industry is boiled down to a few main leaders. The word for 2006 is "consolidation."

ESPs' versus clients' email desires: a disconcerting disconnect

Often we've discovered that what clients yearn for -- and switch firms because they're not finding -- is not what the ESPs themselves think clients really want.

When we talk to CEOs and heads of marketing at ESPs, nearly 100% wax lyrical about extremely advanced email marketing --the stuff most clients would call "bells and whistles".

They passionately gush about microsegmentation, dynamic content, integration, analytics, sequenced automation ... and the current Holy Grail, relevancy. Here are a few sample real-life quotes from earnest, hard-working and intelligent ESP leaders:

"We get very nervous when people come and the first thing out of their mouth is 'I want to be able to send a million emails a month.' We try to get people thinking about their goals in terms of customer relationships -- more focused around integration, analytics, dynamic content."

"I don't sell email marketing, I sell programs. Lead nurturing programs, lifecycle marketing programs.... programs to meet your objectives."

"Rules-based marketing is critical: give me folks who are 30-35 years old who are male and who clicked on my last campaign and bought something."

As their clients and industry analysts will tell them, this is all very lovely. But first the industry has to get the basics down right. When we spoke with clients of all shapes, industries and sizes about how they really wanted their current ESP to improve, not a single one mentioned dynamic content or relevancy. Why? One ESP exec's theory is, "90% of marketers are still doing batch and blast. The reason is, it's too hard for them to do anything else. The lowest common denominator needs to be able to do this sophisticated marketing."

In reality that's only partially right. Yes, 90% are doing batch and blast, and many do look forward to having more sophisticated email marketing capabilities someday. But first they want to be sure ESPs can themselves have the basics nailed down tight.

There's an incredible level of frustration on the side of many clients these days. Common complaints include:

- UIs (user interfaces) are clunky or even buggy.

- List management, segmentation and testing is awkward and time-consuming

- Campaigns are not always sent out on schedule because of a lack of server space during peak send times.

- Customer service and/or tech support are not there live when you need them.

- Integrating between other systems (Web analytics, CRM, external campaign management tools, Web-based email preferences tools) is tougher than promised, especially as the ESP's software goes through upgrades, requiring integration adjustments frequently.

- There's no standardized terminology or metrics practices. For example, many systems report on "delivered messages" without accounting for emails that have been filtered.

Also, the math behind clickthrough rate (CTR) varies widely. Are CTRs a percent of opens? Of sent? Of "delivered"? What about unique versus non-unique CTRs from unique versus non-unique users?

So, what do clients really want? Here are some quotes from our interviews with them:

-- "If our lists are not held by ESP, they need to make it easier to upload lists/changes, connect between databases reliably, make sure lists are cleaned swiftly, make sure the preferences/change account settings/opt-in page on our site communicates with their vendor's database reliably. Does it now? Not always."

- "They need more appropriate infrastructure because the timing of our email is important to us in terms of effectiveness. If we can get them delivered at a certain time in the morning, we'd know they'd perform better than if they're delayed and they go out in the afternoon. If you schedule it for a particular time it should be delivered within half an hour of that time."

"We wanted to be able to copy campaigns as a time-saver and have the ability to preview emails as we're building them."

Several multinational marketers have shared this typical annoyance:

"Our ESP doesn't accept umlauts, the two dots over the "u", all the dipthongs. Therefore it has to use American spellings. For the Germans that's the worst thing you could do. That tells the German audience that you are bush league. It's awful. The other thing is, it doesn't handle international currencies. So none of the Euro or Yen symbols show up. It shows up as garbage. So, our ESP on occasion has embarrassed the hell out of us. The way we get around that is create and save the email as a template. In that case it works. But for our ESP, saving it as a template instead of as a straight HTML file is not as flexible."

And from the interactive agencies, we hear another common complaint:

"I don't want somebody coming offering me creative or content or everything under the sun, because we do creative and content. When I shop for an ESP it has to be a really strong focus on getting that email to the recipient. Minimal bounces, minimal spam blockage. Whitelisted. CAN-SPAM compliancy. Good tracking tools. If they were focused on selling 20 different services I don't think the flexibility would be there and they'd be trying to sell us a bunch of stuff we don't need."

Our summary? ESPs have some incredible ideas to truly revolutionize email marketing. Messaging will someday be far more relevant and valued by each individual recipient. Plus, systems will integrate multiple media metrics (Web conversions, offline coupon redemption, you name it.) And marketers will be able to manage all their permissioned content campaigns -- including RSS, Podcasts, SMS(text messaging), wireless ads, etc. -- from single platforms.

We have seen the future in their eyes and it is a glorious thing.

But first they have to stop tripping over their shoelaces on the way to the corner store to get some milk for mom.

Note: These functional weaknesses do not mean you should keep your email program in-house. If ESPs who are wholly dedicated to email have a hard time managing it perfectly, your in-house team probably can't do any better and may well do a lot worse.

Today's ESPs are doing the best that can be done and making impressive strides in getting better. The industry as a whole is a fairly new one and undergoing growing pains. We expect great strides in the next 24 months.

ESPs' biggest challenge for 2006: reputation management (still)

"I don't care how many tools or features they have or what they claim they can do. If they can't deliver your email it's pointless." - Advertising agency executive

Deliverability problems due to spam filters have been hyped and discussed seemingly endlessly in the past three years. The good news is, best-of-breed ESPs have make significant strides in improving systems and human expertise to deal with deliverability problems. Some improvements include:

- Send volume to ISPs that required a "throttled back" broadcast

- List hygiene systems to quickly remove hard bounces and unsubscribes

- Human deliverability "patrol" teams who monitor blacklists, changing ISP preferences and response to both complaints and challenge/response systems

- Tools that enable clients to see if creative (graphics, copy, design-factors) will cause common spam filters to trip an alarm

- Client selection: picking only clients who are guaranteed to be sending only expected mail to permission-based lists

And, it must be acknowledged that many delivery problems are *not* strictly the ESPs' fault. Marketers' creative and formatting can trip spam filters. Marketers' list-gathering practices may not be truly permission-based. Marketers' ideas of the types and/or frequency of email their customers really want can be badly off base. Marketers mess up.

Do ESPs bear some responsibility for educating their clients? Perhaps. There's already been a heck of a lot of miseducation going on out there. As one ESP VP told us, "Deliverability fear, uncertainty and doubt is a pervasive sales tactic in this industry. The sales rep says 'Those other guys screwed up your deliverability, but we'll do it well.'"

Another ESP CMO said, "There are plenty of ESPs out there that claim to have good deliverability rates but just fabricate their deliverability rates. They don't report their full set of failures, don't report their blocks. Also some ESPs out there are making inroads in respectable senders, but in the ISP community on the receiving end, they're known for their spammer-like behavior and the spammers that use them. Unsophisticated email buyers have to be aware of that and figure out who is really telling the truth and that their reputation is out in the industry."

So, clients are being told their delivery problems will be over if they just sign up with a particular ESP. Plus, clients are then often shown misleading "delivery" reports for their own campaigns that report delivery as total email broadcast minus bounces, sidestepping the entire issue of filters that then can block 20%-40% of permission email from getting through.

(Note: 20% is an average for major ISPs, free Web-based email clients and AOL; 40% is a rough estimate for at-work addresses where corporate IT departments frequently bring less-sophisticated content-based filters into play.)

In 2006 deliverability is going to get worse. Why? Because mailer reputation will soon begin to determine whether mail gets through.

In 2005, first Hotmail and then Yahoo announced launches of authentication (aka SenderID) programs. Other ISPs quickly followed or are in the process of doing so. Authentication doesn't stop spam -- it's designed to stop phishing. These are people who falsify who they are for the purposes of sending emails.

Example: Someone claiming to be a security tech for eBay or Amazon who needs you to verify your credit card info for some reason.

With authentication, the idea is the ISPs will know precisely who each email is *really* from, and they'll stop email from people who are lying about who they are.

Once that step is in place, the next step may inevitably be mailer reputation filters. According to some deliverability experts, the ISPs have been holding off filtering to some degree because they've not been sure if email that was complained about really was *from* the sender it purported to be from.

Now they'll know for sure which marketers are really sending the email campaigns consumers complain about. Now they'll be able to block those senders with a clearer conscience.

When a sender is blocked, the ISP blocks by IP address of the server that the email is being sent from. In other words, they block the server you send mail from at your ESP. Plus, since IP addresses are purchased in groups, sometimes the ISP or blacklist will block the entire group of IP addresses which, in turn, means that ESPs' business is in danger.

ESPs have to watch their reputations as carefully as a clutch of Victorian gentlewomen. Those who accept mailers with opt-out lists (names that are added without individual overt permission to a list) may start shedding those clients or moving them to separate, segmented IP groups.

Others will continue to strengthen their true deliverability reports, engage external third part deliverability reporting systems and add related clauses to client contracts so they can cut any misbehaving clients in a heartbeat.

And, others will continue to lie about true delivery rates and hope no one catches them.

Both ESPs and marketers are now under the gun. New state and federal laws are one thing, the power of a handful of the major ISPs to shut you down are another. They don't have to carry your email, and they won't if they have strong evidence your reputation isn't what it should be.