Data Overload? Segmentation, Strategy Tests Increase ROI 377%

Email software systems run the gamut. Some lack standard reporting, list segmenting and A/B testing capabilities or make you build email with HTML coding. Other systems are the polar opposite, offering an array of features and an easy-to-use interface.

Be careful what you wish for. When you transition from a barebones email system to a premium provider, the metrics suddenly at your fingertips can be overwhelming. You might not know where to start.

Erick Barney, VP Marketing, Motorcycle Superstore, sends email offering accessories and specialized content. About three years ago, he switched from software that couldn’t even tell him his open rates to a system that gave him all the metrics he could ask for … and more.

“Right away, we had access to more numbers and reports than we knew what to do with, really,” he says. “Our challenge was to define what to focus on.”

Barney and his team started by defining some key performance indicators and scrubbing the list. They followed this with some basic A/B testing and gradually increased the complexity of their metrics work. They stayed focused on taking small, incremental steps.

The result? Deliverability, open, clickthrough and conversion rates are all up dramatically, and their email revenue has increased 377%.

“Optimization is not something that you can actually achieve, it’s something that you continue to work toward. There’s always something that can be done to improve your strategy,” Barney says.

Here’s the process they followed along with key insights they discovered when suddenly handling a flood of metrics.

Determine What Numbers Matter
First, Barney and his team set out to figure out which metrics were the most important to track.

They chose to monitor:
o Delivery rate
o Open rate
o Click activity
o Unsubscribes
o Conversion rate
o Revenue

“We had little knowledge of what were considered acceptable [key progress indicator] levels, and we began paying closer attention to industry standards,” he says. “We were quickly able to define acceptable KPI levels for our particular business strategy.”

Quick, Automated Benefits
Most premium email systems automatically scrub your list and remove email addresses that bounce. Barney’s software removed names after three attempted sends. After five sends, their delivery rate shot up 30%, and it remains around 98%.

“We would also get reports on the non-deliverable list,” he says. “We were able to identify any problematic domains. For example, we found out that Verizon.net email addresses weren’t accepting our emails. We contacted Verizon and we were able to reverse that, and we were able to reactivate those subscribers.”

Continual Testing
Barney’s new software also made A/B testing so simple that they could test all the time. “We tested everything. Every email we sent, we would be testing at least one thing.” They started with the basics and gradually ramped up to more complex list segmentation strategies.

Here are four of the tests:

-> Test #1. Frequency
Before 2005, Barney and his team emailed their list monthly, despite having no data to determine whether they should email more or less often.

“We were nervous about CAN-SPAM and we didn’t want to get black labeled by any of the domains,” he says. “Now that we had the numbers, we were able to test our frequency. We went to twice a month. Soon after that, we went to weekly and then we went beyond weekly and got a little carried away. But we were able to see what the effects were and make appropriate adjustments.”

Once subscribers started getting two or three emails a week, Barney saw their unsubscribe rate jump significantly and their open rate plummet. His team quickly scaled back the frequency and now send a weekly email with an occasional extra promotion.

The increase in email sends proved very valuable. “That was a huge one for us. I would estimate between 100% and 120% [revenue] growth based on [increased] send frequency alone,” says Barney.

-> Test #2. Day of the week
Next, Barney and his team tested to determine the best day to send an email to maximize the revenue it generated. “What we found is in our industry, weekends do not perform well for email. I think that’s due to the nature of our customer base, being the more outdoor-oriented, weekend-warrior types out doing what they do on the weekends.”

Interestingly, they found that Thursdays and Fridays performed the best, but only for that one day. When Saturday arrived, conversions plummeted. Emails sent on Mondays and Tuesdays had a longer lifespan and higher total conversion rate.

“Mondays and Tuesdays are pretty equal. If there is a calendar event or our materials aren’t ready for Monday, then it’s not the worst thing in the world” to send on Tuesday, he says.

-> Test #3. Time of day
Tests on the time of day to send email show that their recipients favor later in the afternoon.

“We’re finding that our afternoon sends just prior to businesses closing, like 3:30 to 4 p.m., is when we’re getting our best response,” Barney says. “For some reason, particularly in the Pacific Time zone, the afternoon sends really significantly outperform the morning sends.”

-> Test #4. Subject lines
Barney and his team also played with subject lines to see if they could boost open rates. They quickly discovered that a benefit-oriented subject line works best. But they didn’t hammer away with this finding. “We really didn’t like the results we were getting because it really confined our use of the subject line.”

Complimentary shipping outperformed a lowest price guarantee promotion, “but if you use that every time, it’s not going to become a fresh value proposition,” he says. “It’s going to become an expectation. … It’s a limited space. What we feel the best practice is for that is to do the best job you can … to really communicate the value proposition of that email.”

Segment the List
Barney started segmenting the list after completing some basic tests and refining their email strategy. They ended up sending different emails to eight segments. Here’s how they did it:

-> Step #1. Define the segments

“We’re lucky to have some basic segmentation right off the bat. The motorcycle industry is divided into off-road motorcycles, sport-bike motorcycles and more of the cruiser touring motorcycles,” Barney says. “So, we have three basic segments right there. What we ran into is … just because a person is interested in off-road motorcycle gear, that doesn’t mean they don’t also ride a Harley. And we find in our industry there is a lot of crossover.”

From there, they created six segments:
o Sport bike
o Off road
o Cruiser
o Combination of two riding styles (totaling 3 segments)
o Combination of all 3 styles
o Unknown

Unknown subscribers receive the same email as the 3-style subscribers until their behavior indicates a style preference.

-> Step #2. Divide the list
Barney moves a subscriber onto a list by monitoring:
o Purchase history
o Email click activity
o Post-purchase survey

-> Step #3. Send targeted emails
Next, they customized the emails for each segment. Each email features a unique look and feel and different merchandise based on subscriber interest. For instance, after the list was segmented, off-road subscribers only saw off-road products.

“Usually, the differences [between the emails] are just the merchandising of the products. But if a special event comes up that is specific to one of the styles, then we can send a unique one-time promotion just to that one style.”

Overall results after segmentation:
- Open rates on emails doubled (38.6% from 18.5%).
- Clickthroughs more than tripled (20.6% from 6.2%).

Segment to Reduce Costs
Once they switched service providers, each email was costing Barney money. This got him thinking about what they were wasting on subscribers who never bought anything.

After running a report, they found that 23.3% of their list had never clicked on an email, “and we continue to send to these people every week. You start doing the math, and depending on how you pay for email, if it’s per send or per thousand sends, you can start to imagine that that would be adding up.”

But instead of dropping the names entirely, Barney isolated this group and started sending them a quarterly email instead of weekly.

When an email is sent to that list “you get a little activity. The open rate is miserable. You get maybe [5%] to click it. Well, we’re able to take [that 5%] and move them onto the current list. We keep that never-click list clean. We saved a ton of money. It improved the overall deliverability of our program, it improved the open rate.”

This strategy has reduced their annual email send by nearly 30%, Barney says, which has translated into big savings. “We’re kicking ourselves for not thinking of that two years earlier.”

Erick Barney of Motorcycle Superstore spoke at February’s eTail conference in Palm Desert, CA. For details on upcoming conferences, go to http://wbresearch.com/etail/

Useful links related to this article

Creative Samples from Motorcycle Superstore:
http://www.marketingsherpa.com/cs/motorcyclesuperstore/study.html

ExactTarget - Motorcycle Superstore’s current email service provider:
http://email.exacttarget.com/ETWeb/default.aspx

Motorcycle Superstore:
http://www.motorcycle-superstore.com/