Join thousands of weekly readers and receive practical marketing advice for FREE.
MarketingSherpa's Case Studies, New Research Data, How-tos, Interviews and Articles

Enter your email below to join thousands of marketers and get FREE weekly newsletters with practical Case Studies, research and training, as well as MarketingSherpa updates and promotions.


Please refer to our Privacy Policy and About Us page for contact details.

No thanks, take me to MarketingSherpa

First Name:
Last Name:
May 06, 2014

Marketing Research Chart: How frequently do marketers run A/B tests?

SUMMARY: "Changing habits of our website's visitors makes it hard to predict future behavior."

The above quote is from a marketer who replied to the Website Optimization Benchmark Report survey.

To gain a better sense of how marketers react to ever-changing customers, we'll explore split testing frequency in this MarketingSherpa Chart of the Week.
by Daniel Burstein, Director of Editorial Content

A/B testing can produce powerful discoveries about customers, like this example from a marketer who completed the Website Optimization Benchmark Report survey:

"Determined that a carousel with three images in rotation on the homepage was more effective at driving overall conversion than one with five images in rotation."

But you can't rest on your laurels with testing. What worked well a year ago may not produce results today due to an ever-shifting landscape of the economy, your competition, technology and myriad other factors.

In the Website Optimization Benchmark Report survey, we asked marketers:

Q. How frequently does your organization complete an optimization test?

View Chart Online

Click here to see a printable version of this chart

One-third of marketers are not testing frequently

As you can see, almost one-third of marketers test quarterly at best — with 17% testing quarterly and 14% testing annually.

Only 8% of marketers are testing weekly. Also, 45% of marketers are testing at an "other," perhaps random, frequency.

Testing frequently helps you keep up with shifts in the marketplace that may change customer response since you ran your last test. For example:
  • Have competitors radically changed the price of products? How does this affect your price and/or value combination?

  • Has a competitor's (or supplier's) technological breakthrough changed the marketplace?

  • Has a shift in the fundamentals of the economy changed what customers are looking for?

  • Has a major news item had an impact on how customers perceive your product?

  • Are customers researching and shopping for products in different ways because of new technology?

  • As technology evolves, how do customers act differently on different devices?

Numbers can be revealing, but they can also lead us astray with false assurance. Keep in mind, just because you ran a test and achieved a result doesn't mean that what you learned about your headlines or button copy, for example, will always hold true. Revisit these discoveries from time to time.

As one Website Optimization Benchmark Report survey respondent said, "Fundamentals never change, but the technologies and usages do frequently."

Related Resources

Web Optimization Summit 2014 — New York City, May 21-23

Marketing Research Chart: Does A/B testing deliver a positive ROI?

Marketing Research Chart: Measuring website optimization ROI

See Also:

Post a Comment

Note: Comments are lightly moderated. We post all comments without editing as long as they
(a) relate to the topic at hand,
(b) do not contain offensive content, and
(c) are not overt sales pitches for your company's own products/services.

To help us prevent spam, please type the numbers
(including dashes) you see in the image below.*

Invalid entry - please re-enter

*Please Note: Your comment will not appear immediately --
article comments are approved by a moderator.

Improve Your Marketing

Join our thousands of weekly Case Study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions

Best of the Week:
Marketing case studies and research

Chart Of The Week

B2B Marketing

Consumer Marketing

Email marketing

Inbound Marketing

SherpaStore Alerts


We value your privacy and will not rent or sell your email address. Visit our About Us page for contact details.