Marketing Research Chart: Test to improve optimization project success

by Meghan Lockwood, Research Analyst

Chart: Testing project funnel from launch to pushing optimized pages live in 2010

Click here to see a larger, printable version of this chart

Last week’s chart looked at the complications intrinsic to executing a successful website optimization campaign. As we discussed in that article, on average, nearly half of firms who began an optimization project did not ultimately produce a new live website. In this week’s chart, we will look how testing impacts optimization campaign management.

When trying to determine why so few firms produce a live site, we looked to the deployment of testing to see if that had an impact on optimization campaign management.

Among organizations that did not test, the average annual number of projects at Stage 2 of optimization project management, developing optimized treatments, declines from 80% of all organizations (including those who test) to just 52%.

Further, the number of firms producing a live optimized site declined from 52% of all firms to just 31% of those that do not include testing in their optimization life cycle. Not surprisingly, none of the non-test respondents completed what we consider Stage 3 of an optimization campaign, completing a test with statistical validity.

This data indicates that while testing may be more challenging, those companies that do include testing in their optimization campaigns produce more optimized websites. This makes sense, since project management challenges such as expertise gaps require some initial effort to overcome, but demand relatively less attention on an ongoing basis (assuming no turnover).

Other challenges, like competing for IT resources, should also have much less of an impact after they had been resolved for at least one project.

Another stark difference from last week’s chart is that the drop-off in the project funnel is much steeper among companies that do not test. Only half of projects result in building out some sort of a prototype, and less than one-third in a published replacement to the original.

This suggests that without testing, it is impossible to identify the best performing page with statistical confidence (and sometimes not even with testing). If nothing else, test results may simply help marketers make decisions with less anxiety.

For additional research data and insights about landing page optimization, download and read the free Executive Summary from the MarketingSherpa 2011 Landing Page Optimization Benchmark Report.

Useful links related to this research

Optimization Summit 2012 in Denver, June 11-14, 2012

Marketing Research Chart: Improve your optimization project completion rate

Marketing Research Chart: Testing is vital to create an optimization culture

Landing Page Optimization: How to start optimization testing and get executive support