CHALLENGE

Like every other big Web site containing tons of
content, FirstGov had a hard time designing an easy-to-navigate
home page so visitors could find what they wanted.

FirstGov is the US Federal Government's official portal, linking
to more than 180 million pages on 22,000 different government Web
sites.

At first the site's home page was designed to match uber-portal
Yahoo! because the Web team figured Yahoo!-style must be a best
practice.

Michael Messinger, FirstGov's head of marketing and
communications, began to hear lots of anecdotal evidence that the
Yahoo! home page format was not as easy for most visitors to use
as he had hoped. First time visitors in particular, could not find
what they wanted easily enough, and sometimes just abandoned the
site to never return.

It was time for a home page redesign.

Messinger turned to an offline source for inspiration. Back in
the early 1990s, companies publishing phone books revamped the
way they listed government numbers to make it easier for users to
find the number they were seeking.

In the past, phone book white pages were organized by the way the
government organizes itself, rather than by the way people use
government services. For example, to look up the number for the
passport office you used to have to look under "S" for State
Department. Now you can just look under "P" for passports.

Applying this user-centric thinking to the Web, Messinger and the
design team decided to reorganize the home page into three simple
sections: One for each type of person who visits the site:
citizens, businesspeople, and government staff.

How would Messinger know if the redesign was working? He
needed to:

1. Measure both first time and average visitor satisfaction with
the site changes

2. Determine which areas of the site were worth investing even
more redesign cash and energy in

3. Correlate online success metrics with offline satisfaction
metrics already gathered by the Feds for other services, thus
comparing apples to apples.

CAMPAIGN

Since 1999, such agencies as the IRS and the FAA had
been using the American Customer Satisfaction Index (ACSI) as a
user-satisfaction measuring tool offline.

For example, surveyors physically walked up to a Social Security
desk to ask how well customers were being served. They also asked
airline pilots how the FAA was treating them along with querying
customers who file their taxes on paper and electronically about
their experiences with the IRS.

Originally developed by professors at the University of Michigan,
ACSI uses a cause-and-effect modeling technique that tracks not
only how an organization is doing today, but how any planned
improvements will affect customer satisfaction.

According Larry Freed, President and CEO of Foresee Results (a
company which uses ACSI tactics to measure Web site visitor
satisfaction) there is a “strong correlation between the ACSI and
consumer spending, S&P 500 earnings growth and the economic value
in organizations. A study done with Business Week 1000 companies
showed significant market value increase equating to one point of
customer-satisfaction increase.”

FirstGov decided to test using ACSI to track and quantify user
attitudes, loyalty, actions and overall satisfaction for its home
page.

Because Messinger wanted to compare and contrast FirstGov’s user
accessibility before and after the redesign, he had Foresee
launch its ACSI survey beginning six weeks prior to the new home
page launch.

While many site's online surveys are placed in prominent “above
the fold” positions or use pop-ups to garner as many responses as
possible, FirstGov’s survey link was placed under the fold, below
the left navigation bar where it still remains.

That is because Messinger did not want visitors to feel they were
being intruded upon or interrupted; and, also because Messinger
and Freed say ACSI surveys only require 300 responses for
statistically reliability. In fact the average Fortune 500 ACSI
survey results are based on just 250 customer surveys.

Rather than depending on the more traditional tracking of
clickstreams and page views to determine customer usage, the
survey asks visitors to rate the site's:

1. Content
2. Functionality
3. Look and feel
4. Navigation
5. Site performance
6. Image
7. Search engines
8. Satisfaction
9. Likelihood to return
10. Recommendations
11. Frequency of visits

The full survey contains between 30-40 questions. However, most
Web surfers will not sit still for a survey that long, so Foresee
partitions out just 12-20 questions per visitor, which are
compiled via an algorithm that reassembles data into a complete
survey model.




RESULTS

Prior to the redesign, first-time visitors rated the site at a puny 39 (out of a possible 100) points for overall
satisfaction and their likelihood of returning was 45 points.
After the redesign, first-time visitors rated the site at 74
points. That is almost a 100% increase in satisfaction.

Plus, “Likelihood to return to the site” soared from 45 to 83
points for new visitors.

In addition, the survey found:

1. Prior to the redesign, FirstGov's overall visitor satisfaction
was at 64 points. This higher number blends first-time user
scores with returning visitors, and indicates clearly that ease-
of-use varies tremendously with site experience. (Even if
you think your own organization's site is easy to navigate, that
does not mean your new visitors do!)

2. In comparison, Foresee's measurements of overall user
satisfaction for other common portals during first quarter 2002
were:
- MSN 67 points
- AOL 58 points
- Yahoo! 73 points

3. Although about 1,000 visitors per week click on FirstGov's
survey link out of curiosity, just 10% complete the two-minute
survey.

4. FirstGov learned the site can get the biggest return on
satisfaction investment by continuing to improve the site in
three key ways: functionality, navigation, search. Other areas
of improvement will not give the same user-satisfaction bang for the
buck and may not be worth investing in at this time.

Messinger says, “Our mission is to help customers find what
they’re looking for in the way of government information. They’re
shocked at what they can get now. If they’re satisfied when they
leave, that’s the most important thing. We want happy visitors.”

http://www.firstgov.gov
http://www.foreseeresults.com
http://www.theacsi.org