April 23, 2002
How To

How to Conduct an Online Survey Part IV: Getting Responses

SUMMARY: No summary available.
We were dumbfounded the other day to receive a "survey" in the
body of a text email from a newsletter editor. "Please type your
answers here and reply," he wrote. In this day and age of easy,
cheap online survey forms, it does not make any sense to do a
survey in any other way.

Online forms are not only easier for your survey respondents to
use, they are also far, far easier for you to compile results
from because everything's compiled for you. Plus most survey
forms allow you to output results in a variety of useful formats.

-> Choosing the online survey form that is right for you

Yes, if you have got an in-house Web team you can build your own
survey form. However, if you either do not have an in-house team,
or you want to build a survey quickly and easily without getting
in line for Web production help, there are several inexpensive
ASPs offering do-it-yourself survey builders online today.

Some of the best known are Zoomerang, SurveyMonkey and RAOSoft's
EZSurvey (see link to hotlinked list below). We have tried the
first two and been pleased with ease of use and results.

Some considerations when shopping for the best form:

1. Is a landing page required or can you link directly to the
survey? Some online survey forms require that the first page of
the survey be a welcome note that visitors have to click on to
get to the actual survey itself. You definitely lose
responses this way (each additional page loses 10-20% of your
potential responses). So why waste a whole page on a short,
please-take-my-survey note?

2. How is pricing structured? Some online survey firms have a
monthly fee, some an annual. In each case they usually include a
maximum number of responses for that fee, and then you pay an
additional amount for each additional response over that number.

Before picking your vendor, do the math. How many results are
you expecting? Is this a reasonable price for them? Generally a
highly popular Web site or ezine (as measured by amount of
returning visitors or a consistently high issue open rate) will
get 5-10% responses to a survey request.

-> 8 Tips for setting up surveys online

First of all, be sure your questions are exactly as you would like
them before you let anyone take the survey. Most surveying
systems will not let you alter questions after the survey is "live"
because they do not want you to inadvertently taint your response
pool by mixing answers to original and altered questions.

Here are eight more best practices tips for putting surveys
online:

1. Don't make all answers required.

You will get higher results if you make answers optional versus
required. It is not only polite; it is also smart because some
respondents will not have the time or inclination to answer the
entire survey (especially if it has got open-ended questions,
sensitive questions, or it goes on for more than one page). If
they do leave before answering entirely, at least you got
something from them!

2. Use check boxes instead of drop-down menus.

It is a usability thing; most Web users prefer to see a list of
radial buttons or check boxes, rather than clicking on a little
arrow for the drop down menu and then scrolling up and down
looking for the right answer.

However, Web designers often put in drop downs because it makes
the page as a whole appear to be shorter and cleaner. One drop
down versus lots of options spelled out.

Take our advice: avoid drop-downs whenever possible to encourage
response. In fact the only time you should use a drop down is
when your list of optional answers is so long (more than 10) that
seeing them all listed out (even in two columns) would be
even less enticing than a drop down.

3. Pop-ups should be usable for folks with low rez screens.

Many's the time we have started taking a pop-up survey on someone's
site when we got to the end of our screen and could not reach the
bottom of the pop-up in order to submit the survey. That is
because the designer forgot that regular human beings (versus Web
workers and graphic designers) generally do not have their screens
at very high resolutions.

Make sure your pop-up is entirely visible and usable for the 50%
or more of your visitors who set their screens at 800x600 pixels.

4. Make your survey anonymous

You will get more answers if people feel they are not on the line
-- especially if you are asking personally sensitive questions,
or competitive business questions.

So, do not ask for people's email addresses, or names (not even
as your final question) not even if it is not a required
question. It is invasive and many respondents who have dutifully
filled out the rest of your survey will leave your survey rather
than clicking "submit."

If you want to collect emails or names for another purpose, such
as giving away a prize you offered as an inducement to take the
survey, then you can use a second, attached, survey form to do
so.

After respondents submit the first survey, your next page can say
something like, "Thanks for submitting your anonymous answers.
Our survey is complete. Click here to submit your personal
information separately so you can claim your prize."

5. Put the "submit" button on the right - not "reset."

Another case of techies not thinking remotely about usability.
Here is the scoop; unless you are surveying readers of Lefties
News, make sure your submit button is the one farthest to the
right.

That is because it is far easier, and more likely, for right-handed
people to click the button on the right. So make sure the button
you want them to click is on the right. Otherwise they may click
the "reset" button by mistake and delete their responses; happens often even to the smartest people.

6. Do not limit characters in open-ended questions.

No matter how big or small your box appears for open-ended
questions, make sure the characters a respondent can type in them
are not limited, because some people will type on and on and on
and you do not want to lose any part of their answer.

You will need to double-check this with tech support before running
a survey because often a box appears to allow you to write as
much as you would like, but only a limited number of characters show
up in the final results.

7. Think carefully before allowing only one user.
Many survey forms give you the option of only allowing one answer
per user. If there is a chance that some of your readers share
computers -- at libraries, in households, in science labs, etc. -
- then make sure you do not check that option.

There are other ways you can stop multiple respondents from
skewing results; ask your tech team for options.

8. Use your post-submit page for more than a bland "thank you."

People who have bothered to go all the way through your survey,
are generally in two categories; new curious readers and old
passionate readers. In both cases, why waste their visit to your
survey with a vanilla "thank you" when you have this chance to
continue involving them in your brand?

At the very least add a link to go back to your home page.
(You'd be surprised how few publishers do this!) Other ideas
include adding links to "Best of" articles, or special offers in
your online store. Do not drop out of their interaction with you
until they are good and ready to.

-> How to drive traffic to your survey form:

You will get a little response from a publisher's note in your
regular email newsletter. You will get a lot of response to a
stand-alone note that's separate from an issue. So it is worth
sending one; but check your privacy policy wording first to
make sure you can, and do not send more than one because that
crosses the line from interacting with your readers to spamming
them.

If many of your readers are the print-it-out-to-read variety,
then be sure the link you give is an easy one to type in. Most
online survey forms will give you long, nasty-looking links with
lots of numbers. So, you may want to shadow this link with an
easier URL. Just bear in mind, many people stop typing when they
hit a ".com," so adding a "/survey" on the end may not work as
well as putting it in front of the .com. Example:

Bad link -- http://mypub.com/survey
Good link -- http://survey.mypub.com

You can raise your survey responses as much as 100% by offering a
free gift for respondents. If you are using the survey to gather
demographic information to entice advertisers, then go ahead and
offer a free low-cost gift. If you are using the survey to
determine what you can sell your readers (subscriptions, reports,
event tickets), a free gift offer up front may skew your results
because freebie takers and buyers can be different types of
people.

Gifts can be as simple as a special article that only survey
respondents get to read. If you are an online-only publisher and
you are trying to build a relationship with readers so they will buy
high-ticket items from you, an offline gift may work best; it is
a great excuse for you to send them something in the postal mail
that they can touch, and it makes you feel more like a "real"
company they can trust to buy from later on.

In any case, offering a limited number of low-value gifts ("The
first 50 readers to take the survey get this $5 gift free!") or a
100% of respondents a free gift is a better choice than offering
a sweeps entry for a high value prize. That is because people are
more likely to believe they can win a lower-value prize, and they
are just as happy (or even more) with your little prize as they
would be with a chance at a Jaguar.

Some publishers offer free trial subscriptions to survey takers.
This is only a good idea if you already have a proven trial
series in place to convert these trials into buyers. Otherwise,
you should view discounting very dubiously. Right now you should
be educating your readers that your content has value - that it is
worth money (even if they happen to get it for free).
Discounting and freebies run counter to that.

Instead offer something else for free, some additional bit of
survey-takers-only content is fine.

-> After the survey is over

Three quick notes for after-survey activities:

1. Do not allow anyone on your team (especially editorial) to
glance over the results from open-ended questions before these
are re-organized into categories and counted.

Everyone on your team (especially editorial) has preconceived
notions about what results will be. When they skim
unorganized written answers, certain items will leap out at them
and strike them with great force. Unfortunately, these items are
not always the most meaningful ones.

Have one person skim through all the answers with an eye
toward making a list of general categories they fall into. Then
have him or her carefully go through the answers again, moving
each one into its category. Then count how many ended up in each
category and create a report showing the biggest categories at
the top.

Chances are the answer that so impressed you at first will not even
be in the top category. Perception versus reality strikes again.

2. Thank your readers and visitors. Yes, you already thanked
people when they took the survey, but it is well worth thanking
them again in your next newsletter issue or on your home page for
a while. It not only makes survey readers feel special, it also
reminds potential takers for next time that their opinions will
matter.

3. If you make any changes based on your survey results, then let
your readers and visitors know it. People love to feel powerful!


LINK: A list of online survey tool providers (click here and scroll
a bit) http://www.surveymonkey.com/Pricing.asp

Improve Your Marketing

Join our thousands of weekly case study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions