Surveys

From the moment you launch your site, you should create ways to listen to the responses of people who visit it. Sometimes your readers will initiate responses through comment forms. At other times you’ll want to solicit feedback, perhaps through an online or e-mail survey.

Designing Surveys

If you decide to survey your users periodically, not only can you gather demographic data appealing to prospective advertisers, you can also develop a roadmap for improving your design or adding new features. But don’t try to achieve both goals in the same survey.

In all likelihood, such a survey will be too long and so annoying your users will never participate in your future surveys. Surveys that are short and to the point can foster regular participation.

Here are some tips to keep in mind.

• Focus your questions. Decide what the point of the survey is and make sure the information you elicit achieves your goal.
• Keep it short. On the Web, even a 10-question survey pushes the limit. In considering how many questions to ask, imagine quizzing someone at a cocktail party about themselves. For a while, the questions are flattering. If they go on, the questions get wearisome and eventually creepy. Moreover, a long survey risks attracting responses only from enthusiasts, which can bias your results against casual users.
• Don’t pry (at first). Ask the least intrusive questions first. Ease into more probing questions later. For example, consider asking the user’s zip code just before you need the answer to shape your next question. If you are seeking personal information, such as age or income, offer buttons with a range of ages or incomes as responses. Avoid optional questions; your data’s validity goes down each time someone skips a question.
• Weed out kids. If you are in the U.S. and your site might attract children, you need to be especially careful to avoid gathering data from them. The COPPA law requires sites to handle data from children under 13 in certain ways. Be sure to state prominently that children should not respond to your surveys or offer comments. For more details on COPPA, see the Federal Trade Commission website.
• Establish benchmarks. If you plan to survey users on a regular bases (like a Gallup Poll) it’s important to get the questions right on your first survey because every survey that follows will be compared to the first. Avoid tweaking your questions unless an old question is clearly invalid or flawed. Changing a question’s wording or position can introduce new variables into the results. You’re better off knowing that there was a 10 percent change and being certain that the change was 10 percent than getting a new number that you can’t compare to the previous survey.
• Avoid jargon. Steer clear of technical terms and use simple, direct language. Test your survey with a friend who is removed from your field.
• Ask one question at a time. Craft simple, unambiguous sentences. For example, asking: “Do you prefer news updates via e-mail?” is tricky. It assumes that respondents know what news updates you mean and it doesn’t specify what the other options might be. If your question contains an “or” it’s probably best to break it into two questions.
• Solicit a range of preferences. When you ask whether your users would like new features, it’s rare for people to say no. It’s more useful to gauge interest by offering a comparison chart:

• Keep Possible Answers Distinct. Your questions should always encompass a range of possible answers with no overlap. For example, if you list age options as 46-55 and 55+, someone who is 55 might answer either one. Avoid having one answer be a sub-set of another possible answer, too.
• Anticipate Every Possible Answer. When you are asking people to express a preference, “Don’t know / don’t care” might not be what you want to hear, but it might be the answer your respondents want to give. This can be true for other types of questions as well. If you don’t give them the option they want, they might make up something or choose not to finish the survey.
• Don’t ask people to estimate usage. Memory is a tricky thing. People will often respond with the answer that you want to hear or that they wish were true. So avoid questions that ask “How much,” “How often” or “How long” if it’s possible to get that data by actually monitoring site usage. People are more likely to answer questions correctly about their preferences and their problems.
• Limit open-ended questions. Stick with questions that can be answered from a pull-down menu or with radio buttons. Open-ended questions – any question where users fill in a blank – are harder to process. Save them for the final question on a survey, where you give users a chance to add thoughts or comments. Even then, give them some direction to focus their thoughts.
• Be clear about your deadlines. A week should be enough time to get any users to respond who are likely to respond.

How to Get Enough Responses

Offer a reward for completing the survey – maybe a $100 gift certificate to an interesting store (the store might even chip in towards the cost of the certificate). As long as you don’t require a purchase or limit the pool in any way, you won’t run afoul of sweepstakes or lottery laws – though your local laws may vary, so check with a lawyer the first time. Depending on what you have to give, offer free service, free training, free consultation or even offer to give $1 to charity for every completed survey.

Avoiding Multiple Responses

If your survey is a link from your website, ask people to respond only once. If it’s accessible to anyone, you can put some checks in place to discourage one person from taking the survey multiple times. For example, you can set a cookie after the survey is complete, and redirect those people to a different page. You can also block people based on their IP address.

If you ask people to take the survey via an e-mail solicitation, you can make each URL sent out unique and only allow one survey to be filled in from each e-mail. Ask your site developer to make this happen.

Remember that if you promote your survey with a pop-up, many people have pop-up blockers, so you’ll be less likely get a truly random audience – you’ll bias your survey towards web users who don’t have the sophistication to use pop-up blockers.

Third-Party Survey Tools

Survey Monkey

One simple online survey tool is Survey Monkey. The basic service is free for up to 10 questions and 100 respondents per survey. They list a bunch of competitors in their pricing page and they encourage you to compare them.

Building a survey is a simple matter of:

1. Creating an account.
2. Creating a “new survey.”
3. Adding a page for each question or two.
4. Adding the question to the page.

Survey Monkey has some useful help text to get you through the process.

Zoomerang

The interface on Zoomerang is a little more complex, but the basic, free package is about the same: 100 respondents over 10 days for no charge. Additional advanced features cost extra. Zoomerang offers a few more question formats, such as date or time pulldowns, and it also lets you send out the survey to an e-mail list. Otherwise, the two services are about the same.

Building a Zoomerang survey requires you to:

1. Register (with address, name and email).
2. Create a “survey from scratch.”
3. Choose a name for it.
4. Add questions to page one.
5. Add additional pages and questions as necessary.
6. Choose a greeting and a thank-you message (Note: If you make the last “page” of your survey a personal thank-you message, you can blunt the impact of Zoomerang’s unchangeable final Thank You page.
7. Select how you want to distribute the survey.

A Sample Survey

Here are some questions for a typical demographic survey:

Help us get to know our readers

MyNewsSite.com is interested in learning more about you as a visitor to our site. Please answer these eight questions about yourself, and you will be entered in a drawing for a $100 iTunes gift certificate. Your answers will be kept confidential and will not be used for any purpose except in aggregate.

Ending the Survey

Be sure to make it clear that the survey is over. Thank your respondents for their time. Don’t dump people back at the home page. Instead, include a link back to the page that they started the survey on so they can continue on their initial path.

Additional Resources

• 20 Top tips to writing effective surveys
• Survey Question Composition 101

Next Section