Writing Good Survey (Ch 12)

Seven Tips for Good Survey Questions (Video)

1. Specific and Clear 2. Questions that people are willing to answer the question. Disguised questions. 3. Social Desirability 4. Questions with a known answer 5. Avoid double barreled questions 6. Avoid biased terms or wording 7.Pretest your question

Question Sequence Needing Improvement

11. Do you own or rent your current place of residence? Own Rent 12. How long have you lived in this state? -Less than one year -One year to less than 5 years -5 years or more 13. If you have lived outside this state, in which state did you live immediately before moving here?

Personalization

A respondent's name can be entered into the computer keyboard at the start of an interview The name can be automatically inserted into questions and/or instructions throughout the questionnaire to provide a degree of personalization.

Offer an "Other" Category When Appropriate

Allow for "Other" in fixed response questions. BAD Which brands of cola do you consume? Coke ____ Pepsi ____ RC ____ BETTER Which brands of cola do you consume? Coke ____ Pepsi ____ RC ____ Other (please list:) __________________

Avoiding Questions with Implicit Assumptions

Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes? ___Favorable ___Indifferent ___ Unfavorable The above question can be improved as Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes to repair potholes in the city's streets? ___Favorable ___Indifferent ___Unfavorable (explained what the taxes are going to be used for)

Avoid Ambiguity

Avoid ambiguous questions unless they are called for. BAD OR OK "What do you think of Apple's products?" BETTER "What do you think of the Apple iPhone 6s?"

Avoid Double-Barreled Questions

Avoid double-barreled questions. BAD "Would you describe yourself as someone who enjoys cooking, nail treatments, and/or fashion?" BETTER "Would you describe yourself as someone who enjoys cooking?" (then ask about nail treatments and fashion in two separate questions) A revision/clarification is usually accomplished by breaking one question into several, more specific, questions

Avoid Jargon

Avoid jargon (technical terms) and sophisticated words. BAD "Do you consider yourself a metrosexual?" BETTER "To what extent would you describe yourself as someone who enjoys nail treatments?"

Avoid Hypotheticals

BAD "Imagine you were driving a car that handles and rides smoothly..." BETTER Consumer *actually* test-drives car that handles and rides smoothly and then responds to questions.

Avoid Negatives

BAD "Which of the following would you prefer *NOT* to sample?" BETTER "Which of the following would you prefer to sample?"

Arrangement of Related Questions

Cluster questions that focus on the same topic Grouping questions into meaningful clusters can increase respondents' ease in answering the questions and reduce the chance of response errors

Incorporating Complex Skip Patterns

Computerized online interviewing can handle very complex skip patterns. The computer can check as many "if then" statements and previous responses as necessary, decide which question should be asked next, and pose that question on the screen.

Data Collection and Graphs

Data file built in real time Downloadable in several formats At least simple graphs are often readily available

Avoid One-Sided Questions

Do you feel firms today are concerned about their employees? ___Yes ___ No ___ No opinion Would you agree that the deregulation of the airline industry has benefited customers? ___Yes ___ No ___ No opinion

Order Effects

Does The Question Create The Answer?

Would you agree or disagree that the deregulation of the airline industry has benefited customers? ___Yes ___ No ___ No opinion What kind of question is this?

Double barreled questions asking about two issues in one question

Avoid Leading Questions

Ensure questions are unbiased and avoid leading questions. BAD "How much better do you like Donald Trump than Hillary Clinton?" BETTER "Which candidate do you prefer - Donald Trump or Hillary Clinton?"

Avoid Unbalanced Questions

How important is price to you in buying a new car? ___More important than any other factor ___Extremely important ___Important ___Somewhat important ___Unimportant Than answered choice are scud towards important

Balanced Questions

How important is price to you in buying a new car? ___Very important ___Relatively important ___Neither important nor unimportant ___Relatively unimportant ___Very unimportant What type of scale is this? lankert scale

Guarding Against Errors

How often do you eat eggs for breakfast? _____ Frequently _____ Occasionally _____ Rarely _____ Never (Frequently, occasionally, and rarely mean different things to different people) misinterpretations

Complex Questions

In which of the following do you typically invest your liquid assets? ___Insured accounts ___Stock market ___Insured accounts and stock market ___Other accounts (asking about liquid assets when we are talking bout longterm questions) Of the total number of miles you drove during the past month, approximately what percentage was traveled in driving to and from work? ___Percent ( have to do a lot of math)

Question Wording

Is the vocabulary simple, direct, and familiar to all respondents? Do any works have vague or ambiguous meanings? Are any questions "double-barreled"? Are any questions leading or loaded? Are the instructions potentially confusing? Is the question applicable to all respondents? Are the questions of appropriate length?

Simple and Brief

Keep questions simple. BAD "Considering the candidates running for office this election, who is your preferred candidate?" BETTER "Which candidate do you prefer?"

Avoid Leading Questions

Leading a taker to answer a specific way. Don't you think offshore drilling for oil is environmentally unsound? ___Yes ___ No ___ No opinion Do you think the quality of products on the market today is as high as it used to be ten years ago? ___ Yes ___ No ___ No opinion

Specific and Clear

Make questions specific. BAD "Please tell us what you think of this restaurant." BETTER "What did you think of our restaurant's service?"

Adding New Response Categories

Multiple-category questions with an "other" category When a pre-specified number of respondents provide the same open-ended response to the "other" category, that response can be automatically converted and added to the structured categories.

Avoid Order Effects (question order)

Placement/order of questions can influence subsequent responses. Asking income level then price-related questions. Asking detailed questions about service then overall satisfaction.

Sequencing of Questions

Position of Demographic and Sensitive Questions Arrangement of Related Questions Funnel and Inverted-Funnel Sequences Skip Patterns

Questionnaires for Computerized and Online Interviewing

Randomizing response choices Checking for response consistency Incorporating complex skip patterns Personalization Adding "new" response categories Data Collection and Graphs

Computer surveys offer additional benefits to the...

Researcher

The organization and flow of the survey should make sense to the....

Respondent

Position of Demographic and Sensitive Questions

Respondents' personal or demographic characteristics (e.g., age, education level, income, etc.) are placed at the end of the questionnaire. Why do we do this?

Writing good survey question involves....

Several considerations

Inverted-Funnel Sequences

Starting more specific then get more broad

Checking for Response Category

The computer can be programmed to check for consistency between the response given to a question at hand and responses given to certain key questions asked earlier. Prior responses can also be "carried over" into new questions.

Randomizing Response Choices

The computer can be programmed to randomize the order of presentation of the categories for multiple-category questions in which category sequence may influence response choice. Helps alleviate order bias.

Categories Must be Mutually Exclusive

Use mutually exclusive categories. BAD "What is your age?" 18-24 ___ 24-30 ___ 30-35 ___ BETTER "What is your age?" 18-24 ___ 25-30 ___ 31 35 ___

Guidelines for Question Sequencing

Use simple, interesting opening questions Use the funnel approach, asking broad questions first (then move to more complex questions) Carefully design branching questions Ask for classification information last *IMPORTANT* Place difficult or sensitive questions near the end

Avoid Order Bias (order of response choices)

What color do you consider to be your favorite? Blue Yellow Green Orange Respondents have a propensity to choose from the beginning and ends of lists.

Funnel Sequences

Which of the following types of TV shows do you watch? (Check all categories that apply.) ____News shows ____Quiz shows ____Game shows Which of the following types of TV shows *do you like the most*? (Check as many categories as apply.) ____News shows ____Quiz shows ____Game shows Starting broad and get more specific

Double-Barreled Questions

touches upon more than one issue, yet allows only for one answer.