September 15, 2005
How To

New Email Test Results & Tactics from Apparel Retailer J. Jill

SUMMARY: Which pulls better in ecommerce email campaigns -- postcard-style creative featuring one hero-shot product or a series of thumbnails of different products to choose from? That's just one of the fascinating email tests the marketers at J. Jill have been working on over the past year. Discover results, plus why the team has stopped straightforward a/b testing. Yes, includes some creative samples to inspire your own email design department.
Just over six years ago, nobody had heard of J. Jill. Now, the multi-channel specialty retailer of women's clothing has a website, a catalog and by the end of 2005 will have close to 200 stores.

How has the company grown so quickly? "Our main objective is to grow the J. Jill brand through retail stores, so our growth strategy is to open 30 to 40 new stores a year," says Andrea Gulli, Operating VP of eCommerce and Credit Marketing.

"The main way we do that is to be profitable through direct channels."

Gulli's team has found success in doing just that, with help from an ongoing series of targeted emails. We talked with Gulli to learn how her team makes the most of the company's email campaigns.

Email tactic #1. Segment, then target

Gulli's team knows if an email subscriber is already a customer, and whether they shop primarily online, at stores, or through the catalog. They know if she's a petite shopper, and if she holds a J. Jill credit card.

"Our goal is to be as synergistic as possible across all channels," Gulli says. The email database is segmented so that each customer receives the emails that are most relevant.

Example:

--Customers who shop mainly via catalog might receive emails notifying them when a catalog has dropped.

--Online customers receive emails with notification of online-only offers.

--A customer near a J. Jill retail outlet might receive notice of offers at the retail level, invitations to store events, or coupons .

Customers can print coupons out and redeem them at the store level, and Gulli uses no technology to prevent them from using a coupon more than once. Forwarding to a friend is also fine with Gulli. As for customers who might print and use a coupon three times, "that would be a busy shopper," she says. Generally the offers last only a few days.

Interestingly, Gulli has learned that the favorite channel of customers remains their favorite despite different offers. "People who tend to buy in a store, even when stimulated to shop by email, they go to the store."

Because of segmentation, customers might receive emails as often as twice a week. "We're looking at frequency and what's most appropriate to different customer segments," she says. "We don't have a hard and fast rule right now."

Email tactic #2. Test often and within campaigns

Gulli is a firm believer in running tests and making changes based on what you find. "The most important thing we've learned about testing is that you need to do it," she says. "You can't make an assumption that one creative is going to be better than another, because you can be fooled."

Gulli's team used to test by splitting each segment in half and sending a different version of the email to each half of the segment. Future campaigns would be based on what they found.

However, last year the company began testing within campaigns.

They take a statistically significant section of a list, then split the segment and send two different emails testing, say, a different subject line in each email. Once a winner has asserted itself, they mail the winner to the rest of the list.

For each test, Gulli's team creates what she calls a "mini P&L," looking at open rates, clickthroughs, conversions and average order value to determine ROI.

"We improve the bottom line by changing the process right in the campaign," Gulli says. "We've been really pleased with it."

Two testing examples:

--Spring's must-have jackets

In this email, Gulli's team tested complimentary shipping on two different order minimums to two different macrosegments of their email file: buyers vs. non-buyers. They sent the different versions to the subsets and monitored results for three days. The "winning" version was then sent to the remainder of the email list.

o For buyers, she tested complimentary shipping on orders of $150 or greater vs. complimentary shipping on orders of $100 or greater.

o For the non-buyers, she tested complimentary shipping on $100 or greater vs. the same on $75 or greater.

Results: For buyers, the $100 order minimum outperformed the $150 minimum. Total conversions on the $100 minimum were 4.6% higher and conversions were 12% higher than the $150 minimum. Open and clickthrough rates were also slightly higher on this version. In the case of the non-buyers, the $100 order minimum also won, this time over the $75 order minimum. The open rate and clickthrough rate were slightly higher on the $75 version, but the average order size was 14.3% higher on the $100 version, making the incremental sales 12.6% higher.

"Gut instinct would tell you that the lower order minimum would work better for the non-buyer segment because you'd expect higher opens and clickthrough rates, and thus overall better results, where really the driver of the overall incremental sales here was the higher order minimum, even on fewer orders," Gulli says.

--March Web-only offer

In this email, they tested two different creative versions: a "postcard" format versus a "thumbnail" format.

Using the same testing methodology -- testing a subset of the email list, monitoring results for 2-3 days and then sending the "winner" to the remainder of the file, they found that the clickthrough rate was higher on the thumbnail version. However, the conversion rate was slightly higher, the average order value was higher by 4.2% and the total purchases were 14% higher on the postcard version.

Gulli has also learned that sending emails earlier in the week is best and that the most direct subject lines get better open rates.

Email tactic #3. Coordinate with other departments

Keeping track of and creating email promotions for various promotions across all channels is a challenge, says Gulli.

How does she make it happen? "We've never been a company where people say, 'It's my order,' or 'It's my promotion,'" she notes. "We work well together."

It helps that Gulli's team, the catalog team, and the store promotions team all work within the same area of the building and are all involved in the planning of the promotions for the season.

Six months before each season, Gulli meets with the heads of catalog and store promotions to come up with campaigns for the season based on key merchandising themes. "Then we embellish [throughout the season] depending on needs," she says.

They also meet weekly to stay organized and often involve other teams, such as retail management, store communications, merchandising group and call center group. "We're nimble," she says. "We're not such a big company that we can't communicate effectively across departments."

Useful links related to this story

Creative samples of some of the test campaigns detailed above: http://www.marketingsherpa.com/jjill/study.html

J. Jill: http://www.jjill.com

Note: J. Jill is a member of Shop.org, a forum for retailing online executives to share information, lessons-learned, new perspectives, insights and intelligence. More info at http://www.shop.org


Improve Your Marketing

Join our thousands of weekly case study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions