by Courtney Eckerle, Reporter

CHALLENGE

Eventful has been connecting people with their local events for the last six years. Their site features live events and allows visitors to find tickets, favorite an artist, share they are attending a concert on social media and even campaign for an act come to their town with the "demand it!" feature.

Their initial technology didn't allow them to make recommendations based off of the events a subscriber had "favorited" or demanded individually — only through the event's or act's general popularity on the website, the amount of tickets sold and social buzz.

The only alert system they had in place was for artists the user had specifically "favorited" or requested on their website, which the Eventful team felt was missing out on the opportunity to anticipate their audience's interest in other related events.

"It was good … but we always knew that we could do better. So, we really resolved to creating a better customer experience that was driven by our own desire to be a better entertainment publisher," said Paul Ramirez, Vice President of Operations, Eventful.

CAMPAIGN

When a subscriber expresses interest in an artist or event through "favoriting" or demanding, a newly built algorithm gathers data from the data Eventful collected over the past six years, and recommends other acts the subscriber might be interested in.
"The recommendation engineered algorithm enables us to suggest instead of just one event based on what your favorite is … it enables us to provide you a list of events," Ramirez said.

These personalized recommendations were not only built into their existing email newsletter, The Weekly Entertainment Guide, but launched a new email product, Recommended Performer Alerts.

In describing this new level of personalization for their millions of registered email subscribers, Ramirez used the example of a subscriber interested in country singer Carrie Underwood.

"Because of everything we know about you, about Carrie Underwood, about country music and the taste of 20 million subscribers, plus other data that we use from third parties, we can predict with reasonable certainty that there are 20 other performing artists who you also like," he said.

In the launch of the new product combined with integrating this data into the newsletter for increased relevancy and personalization, Ramirez described the two email products as personal "live entertainment city guides that tell people, 'Here is what is going on in San Diego this week.'"

As part of the integration and launch of this recommendation engine, Eventful decided to use it as an opportunity to win-back some of its inactive subscribers. It saw a 400% increase in reactivation rates with the addition of the personalized recommendations.

Step #1. Build algorithm for increased personalization

Ramirez said Eventful has built this algorithm referencing the data store that it has been growing over the past six years.

"That expanded, essentially, the target that we could aim at to better serve our subscribers in making very smart recommendations," he said.

"So in a nutshell, we have three databases," he added, detailing the sources of data that Eventful drew from for this campaign:

• A consumer database capturing PII (Personally Identifiable Information) and information subscribers tell Eventful about a live event they are interested in


• A database of events featuring their times, locations and performers


• A database housing information about performers

In launching the Recommended Performer Alerts, Ramirez said Eventful set out to merge all of this information for the purpose of personalization, and in the constant effort to make Eventful a better entertainment news source for its community of registered subscribers.

Step #2. Test technology

The next step after building the technology to make the recommendations possible was, according to Ramirez, starting to test "it internally with a small community of friends and family that were willing to give us honest user feedback."

Ramirez said profiles were generated for this group, and a live act was "demanded" or favorited for them through Eventful.

From there, Ramirez said, "based on the technology that we have built, [we would show the user] 'here is a list of 17 things going on in San Diego that we think you would be interested in.'"

Ramirez said he knew Eventful was onto something when the reactions to the profiles in testing were "stunning."

"Universally, people said, 'I am interested in most of those, and the ones that I am not interested in … I get why you recommended them to me.'"

He said with the success of the initial test, Eventful expanded the project into two contexts — making a dedicated email product with the Recommended Performer Alert, as well as using the recommendation engine in the context of the Weekly Entertainment Guide.

Step #3. Send out personalized newsletter and alert email

Eventful's Weekly Events Guide showcases a main upcoming event, a sponsored event and recommendation.
Among sections for top venues, trending events and local events, the newsletter encourages the subscriber to "tell us who you like" so they can be alerted when they come to town, as well as adding to the subscriber's recommendations.

Subscribers have the chance to further specify their preferences by clicking an "I'm not interested in [artist's name]" button in each recommendation.

Ramirez said the goal for Eventful is not in the volume of emails sent out, but "matching the preferences of the subscriber for how they want to receive notifications from Eventful."

Send out to inactive subscribers

Eventful pushed out this technology first to their active email population, and later to the inactive population as part of a reactivation campaign.

According to Ramirez, the recommendation engine technology, using normative data, was sent out to those subscribers who had not engaged with an Eventful email in the space of five months.

By using the recommendation engine technology, Eventful was able to achieve a 400% increase in its reactivation rates.

Recommended Performer Alerts

The Recommended Performer Alerts product suggests upcoming local acts the subscriber might be interested in, with the time, date and location of the event, and the ability to find tickets and share the event via social media.

A section below in the alert encourages the subscriber to "Add more performers to get better recommendations," and displays five related acts. The subscriber can press "add" immediately and it will be factored into their next suggestions.

Be aware of alert send frequency

"On the recommended alert product, which is a solo email, here is the challenge," Ramirez said.

One, he said, is even if a user has only "favorited" or demanded one artist with Eventful, if that act is someone like Justin Bieber, Carrie Underwood or Dave Matthews, "that recommendation can spawn scores of alerts so suddenly, you start receiving a lot of emails from Eventful."

Ramirez didn't want to have issues of services like Spotify and Sound Kick, which ingest the entire content of your iTunes library, and, "suddenly … there is now going to be 432 artists that are associated with my profile."

They have recognized this issue, and proceeded with caution when it comes to managing the amount of alerts sent to consumers' inboxes, and have also recently developed a mobile app to alert people as well, if receiving information that way is preferable for them.

"We are having to think fundamentally about how we message subscribers, so we tell them what they have asked us to tell them but in a way that works best for them," he said.

The initial iteration of the triggered alerts product produced a high volume of alerts that was deemed too frequent, according to Ramirez.

To solve this problem, Ramirez said they added criteria to the recommendation emails. Eventful looks at a range of information including subscriber actions towards events and performers across all of Eventful's properties, from the Alerts email, newsletter and website.

Consistently hone service to be more effective

Ramirez said after pushing out the dedicated recommendation alert product, while they saw good encouragement rates, the clickthrough behavior wasn't what they had hoped for.

After coming to the conclusion "the recommendations were just too broad," Eventful consistently tweaked the algorithm over the past year to provide a better, and optimally personalized service.

Ramirez said his team spent time tweaking "the distribution cycles, rethinking whether or not we send that publication … rethinking the frequency with which we send it, do we gain performers if you have so many favorite alerts coming within a horizon of time."

He admitted they were "pretty significant technical challenges," but Eventful was, and still is, dedicated to consistently adjusting and re-addressing issues in an effort to make personalization more effective in its email products.

RESULTS

Ramirez said testing and using this technology in their sends has opened up possibilities for Eventful, saying they "learned really interesting things … and those tests suggested new challenges that we hadn't thought of."

Currently, Ramirez said, they have even "upped the ante" on the data they are collecting for the recommendations in order to continuously tailor recommendations to be more precise, and will be enacting those updates in the beginning of this year.

He added, "What you tell us when you subscribe is only half of the story, you also tell us who you like based on what you click on Eventful.com and what you click on within an email."

Currently, the alerts are only focused on performers, but Ramirez said they have realized it is missing the opportunity to reach people on other events, such as movies or local events without specific "performers."

"Performer alerts … don't give insight into me as a dog lover, and me as a [George] Clooney fan. So, we are creating technologies that add to the data store … of my consumer record on Eventful," he said.

Ramirez added this will ensure not only concerts will be recommended to subscribers through the alerts, but other events subscribers have expressed interest in on Eventful's website, such as when the new George Clooney movie is released, or about a local Dog Surfing competition.

"We have upped the ante on the data that we collect and store so that we can provide a robust and much more comprehensive set of recommendations and I am really excited about that," he concluded.

By introducing the recommendation engine technology, Eventful saw a 400% increase in its reactivation of inactive subscribers.

Weekly Events Guide

• 26% open rate increase


• 97% clickthrough increase


• 56% increase in click-to-open rate

Recommended Performer Alert

• 15% median open rate


• 3% median click rate

Creative Samples

1. Weekly Newsletter


2. Recommended Performer Alerts

Sources

Eventful

Related Resources

Email Marketing: How a triggered alert program maintains 40% open rate, 60% click-to-open rate for millions of subscribers

Email Personalization: 137% increase in open rate from personal note approach

Email Marketing: 17.36% higher average clickthrough rate in 7 personalized subject line tests

Email Marketing: Two ways to add relevance, and why you must be correct (via MarketingExperiments)

Email Marketing: Helzberg Diamonds garners 288% sales lift with animated, personalized promo