CHALLENGE

Peter Meyers, VP of Marketing and Business Development, Over There Interactive, wanted to persuade more visitors to use the search feature on EuroCheapo, a budget European travel website. The site’s main revenue stream is from hotel booking referrals, which is tied directly to its search tool.

Meyers and his team started investigating in late 2008, a few months before the biggest part of the booking season. To really drive an increase in search traffic and revenue, they had to modify the homepage. But, making the wrong change could cause them to lose customers during the most important part of the year.

Unfortunately, the team did not have the resources to do live A/B testing. “We would have preferred to do a lot of pretty aggressive A/B split testing and use a multivariate tool, but frankly, we couldn’t. With our development resources, we had to focus really in certain areas and that didn’t allow us to have multiple iterations of a live homepage, but more so tested variations before it went live,” Meyers says.

CAMPAIGN

Before the homepage re-launch, about 18% of EuroCheapo’s visitors used its search, Myers says. The team had to uncover why more visitors were not searching, and then test new homepage designs before making the final design live. Here are the steps they took:

Step #1: Create a fan list

The team built an email list of EuroCheapo fans, the EuroCheapo Insiders, several months before kicking off the campaign. The Insiders register through EuroCheapo’s newsletter, website, and blog to provide opinions and feedback on products and the website. They live all over the globe and receive no reward other than having EuroCheapo value their opinions.

“They’re really people who represent a wide background and a diversity that I think is really representative of our current user base,” Meyers says.

Step #2: Run a focus group

In Dec. 2008, the team invited about 20 “friends of friends” who had some familiarity with EuroCheapo into their Manhattan office. The subjects who volunteered their time ranged in age from 18 to 54. They were asked to go to EuroCheapo’s homepage (see creative samples below) to find the best budget hotel in any European city. They were not given any additional instructions.

The team observed the subjects’ navigating, such as:
o Whether they used the search
o Whether they used the left-hand navigation
o Whether they went below the fold

“What they immediately would do is click on our left navigation. Prior to our re-launch, we had a link to all of our 27 city guides, as well as links to our different transportation channels. And their first action tended to be to go deeper into the site, but not to perform a search through our Cheapo Search,” Meyers says.

Step #3: Create first test pages

The team created six test homepages designed to encourage more searches. The designs varied by:
- Search-box placement
- Photo use, size and placement
- Navigation placement and style

“The differences [among] the versions, to people who are not as tied up in the product development cycle, would probably look a little minor or moderate. From a real [user interface] design and functionality design, they really were quite distinct,” Myers says.

Step #4: Email survey to fan list

The team emailed the EuroCheapo Insiders a link to the test page they felt was strongest, and a link to a survey (see creative samples below for email and survey questions). The survey included 6 multiple choice and open-ended questions. A few examples:
- What is your initial reaction to this homepage design? (Multiple-choice question)
- What do you like about this homepage design? (Open-ended)
- Any suggestions for how we can further improve the homepage design? (Open-ended)

63.6% of the list took the survey. “The response was really high because we had only communicated with them twice beforehand on product development issues,” Meyers says.

The most notable piece of feedback the insiders provided dealt with images on the page, Meyers says. The old homepage typically had a scenic shot from a European destination that many Insiders felt served as a differentiator from other budget travel sites, and also motivated visitors to travel abroad.

“We were thinking that we might remove it entirely because it took up a disproportionate amount of above-the-fold real estate. But based on their feedback, we gave it pretty prominent placement, though we did crop it slightly,” Meyers shares.

Step #5: Build more pages

The team took the Insiders’ criticism to heart and spent the next five weeks creating three more test pages. Aesthetically, the pages differed little; only their use and size of images changed the page’s look and feel. Functionally, the pages had different navigation structures and placement of the search box. (See creative samples below)

Step #6: Leverage industry contacts

The team’s office is in Manhattan. They have many local contacts in industries, including interactive advertising, digital media and Web design. They reached out to about 15 contacts to review the entire list of test pages and to provide feedback. The team sent the contacts an email with a link to the survey, and encouraged them to email comments, if preferred.

This group was most helpful in encouraging the team to provide fresh relevant content on the homepage to drive repeat visitors, Meyers says. Summaries of recent blog posts, links to email and RSS subscriptions, and regularly updated location-based content were all given prominent placement due to these suggestions.

Step #7: Select final page, announce launch, go live

The team selected a homepage (see creative samples below) from a total of nine test pages. The final choice was based largely on the opinions of the audience, Meyers says. “By the time we got to the final steps of the process, we really did have confidence that the final version was taking into consideration the vast majority of the feedback that we received.”

The team sent an email to the Insiders alerting them to the new homepage launch. They also announced the launch in the blog, newsletter, and RSS feed. The final version went live on Feb. 15.


RESULTS

“When you’re doing fairly large adjustments to any page or template to your site, you’re always a little anxious and nervous about it. But when you’re doing it to the highest trafficked page on your site, we were really hopeful and curious and looking forward to getting data back…And it definitely delivered at, if not higher, than our expectations.”

Results after two and a half weeks:
- Search volume: up 38%
- Volume of reservations: up 35%
- Page views: up 15%

Meyers was surprised by the lift in page views, he says. “We realized that, with a more refined and functional navigational structure that it really helped people who are, say, looking in our blog to be able to dive back into the city guide pages and vice versa.”

Useful links related to this article:

EuroCheapo’s Creative Samples
http://www.marketingsherpa.com/cs/eurocheapo/study.html

8-Point Checklist & Useful Hotlink - How to Improve Your Homepage Performance Significantly (Open access)
http://www.marketingsherpa.com/article.php?ident=30111

SurveyMonkey.com: Provided the team’s free survey
http://www.surveymonkey.com/

EuroCheapo
http://www.eurocheapo.com/