CHALLENGE
To keep their audience engaged with the brand, ExactTarget sends messages to regular customers and prospects that include product updates, industry articles and Webinars/events info. Recipients can also opt in to receive more specific newsletters about issues, such as agencies, deliverability, retail email, design, integrations and more.

But since the company wasn’t wholly satisfied with the response they were getting, they decided to do what they always tell their customers to do -- test. It was a time to walk the walk.

“We had too many people who were no longer engaging with our emails,” says Erin Howe, Marketing Director at ExactTarget. “During the last four or five years, about 20,000 [professionals] have opted into our newsletter program, but only 8,000 were actively engaging and upgrading their memberships.”

Last summer, Howe and her team looked at a slew of messaging variables that they wanted to test -- some they had employed in the past and some they hadn’t. They were more interested in learning what subject lines, headlines and opt-in/opt-out buttons worked cohesively than in tinkering with color schemes or other nuances.

“We didn’t want any noise in our results,” Howe says. “So we chose to make the colors and the copy [in the sales letter] consistent.”


CAMPAIGN
Howe randomly selected 4,000 names from the database in early October and divided them into 16 groups, each containing 250 people, for a detailed, one-week effort. Every one of the 16 groups received a separate test that involved a combination of four of a possible eight elements.

In all cases, the emails promoted a new service called ExactTarget Passport, which allowed disengaged readers to opt back into regular messages from the company.

The test elements included:

- One of two subject lines: Either “ExactTarget: Please Confirm Your Email Subscription” (seen as *subject line A* in examples below) or “Don’t Miss Out On Your ExactTarget Newsletter” (*subject line B*).
- One of two headlines -- a personalized version that said, “Dear Megan: …” and then simply went into the sales letter, or a generic one, “Would you like to continue ExactTarget communications? Learn valuable email marketing information with the ExactTarget Passport.”
- Eight of the segments saw an opt-in button stating, “Yes, I would like to receive the ExactTarget Passport.” The other half got that button along with an opt-out link that used the copy, “No, I do not want to receive the ExactTarget Passport.”
- To allow the recipient a view of what they might be signing up for, half of the recipients saw a shrunken sample version of an actual ExactTarget Passport message in the upper left-hand corner of the letter format. Meanwhile, the other eight segments didn’t get the sample.

Here’s a rudimentary breakdown of the tests:

Test #1 = Subject line A + generic headline + opt-in button + sample email
Test #2 = Subject line A + generic headline + opt-in button + opt-out button + sample email
Test #3 = Subject line A + generic headline + opt-in button
Test #4 = Subject line A + generic headline + opt-in button + opt-out button
Test #5 = Subject line A + personalized headline + opt-in button + sample email
Test #6 = Subject line A + personalized headline + opt-in button + opt-out button + sample email
Test #7 = Subject line A + personalized headline + opt-in button
Test #8 = Subject line A + personalized headline + opt-in button + opt-out button

Insert subject line B into each of the above equations and you have Tests #9-#16, too.


RESULTS
Howe and her team waited 24 hours after their Oct. 20th mailing before looking at the findings. Although they did not disclose all of the results, they did offer the winning combination and the one that tested the worst.

Let’s get the ugly out of the way -- the worst mix saw only 4% confirm or renew their subscriptions. It included the subject line, “Don’t Miss Out On Your ExactTarget Newsletter,” the personalized headline and the opt-in/opt-out buttons combo.

And the winner was: the subject line, “ExactTarget: Please Confirm Your Email Subscription,” the generic headline and the opt-in/opt-out buttons combo. In this case, an impressive 24% confirmed or renewed their subs.

“We were surprised because we expected that the personalized headline would work the best,” Howe says. “It was also interesting to see the two buttons showing up in both the best test and the worst performer. But in the end, we wanted to capture what elements got the best response in order to allow us to find a great email strategy.”

ExactTarget wasted little time using the winner, launching a campaign to their house file and to two groups of prospects at the end of October. The firm saw a clickthrough rate of 15.1% for email sent to leads with a previously shown interest in the company. Furthermore, 72% requested the newsletter, 19.7% declined the offer and the remaining 7.3% didn’t click either button.

Prospects with no shown interest clicked through at a 12% clip, 64.9% said yes to the newsletter, 29% said no and the other 6.1% didn’t click either button. Customers clicked through 28.5% of the time, with 81% requesting the newsletter and nearly all of the remaining 19% declining.

"The testing allowed us to practice what we preach," Howe says. "We are always telling our customers to test as much as possible. This was an occasion for us to do it ourselves.”

Now, ExactTarget’s sales team isn’t just armed with client results. “We can say, ‘This is something we are using. These are things we’ve been testing. We are using the product every day.’ ”


Useful links related to this article

Creative samples from ExactTarget’s email tests:
http://www.marketingsherpa.com/cs/et/study.html

ExactTarget:
http://www.exacttarget.com