Writing Good Survey (Ch 12)
Seven Tips for Good Survey Questions (Video)
1. Specific and Clear 2. Questions that people are willing to answer the question. Disguised questions. 3. Social Desirability 4. Questions with a known answer 5. Avoid double barreled questions 6. Avoid biased terms or wording 7.Pretest your question
Question Sequence Needing Improvement
11. Do you own or rent your current place of residence? Own Rent 12. How long have you lived in this state? -Less than one year -One year to less than 5 years -5 years or more 13. If you have lived outside this state, in which state did you live immediately before moving here?
Personalization
A respondent's name can be entered into the computer keyboard at the start of an interview The name can be automatically inserted into questions and/or instructions throughout the questionnaire to provide a degree of personalization.
Offer an "Other" Category When Appropriate
Allow for "Other" in fixed response questions. BAD Which brands of cola do you consume? Coke ____ Pepsi ____ RC ____ BETTER Which brands of cola do you consume? Coke ____ Pepsi ____ RC ____ Other (please list:) __________________
Avoiding Questions with Implicit Assumptions
Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes? ___Favorable ___Indifferent ___ Unfavorable The above question can be improved as Are you favorable, indifferent, or unfavorable toward a 10% increase in city taxes to repair potholes in the city's streets? ___Favorable ___Indifferent ___Unfavorable (explained what the taxes are going to be used for)
Avoid Ambiguity
Avoid ambiguous questions unless they are called for. BAD OR OK "What do you think of Apple's products?" BETTER "What do you think of the Apple iPhone 6s?"
Avoid Double-Barreled Questions
Avoid double-barreled questions. BAD "Would you describe yourself as someone who enjoys cooking, nail treatments, and/or fashion?" BETTER "Would you describe yourself as someone who enjoys cooking?" (then ask about nail treatments and fashion in two separate questions) A revision/clarification is usually accomplished by breaking one question into several, more specific, questions
Avoid Jargon
Avoid jargon (technical terms) and sophisticated words. BAD "Do you consider yourself a metrosexual?" BETTER "To what extent would you describe yourself as someone who enjoys nail treatments?"
Avoid Hypotheticals
BAD "Imagine you were driving a car that handles and rides smoothly..." BETTER Consumer *actually* test-drives car that handles and rides smoothly and then responds to questions.
Avoid Negatives
BAD "Which of the following would you prefer *NOT* to sample?" BETTER "Which of the following would you prefer to sample?"
Arrangement of Related Questions
Cluster questions that focus on the same topic Grouping questions into meaningful clusters can increase respondents' ease in answering the questions and reduce the chance of response errors
Incorporating Complex Skip Patterns
Computerized online interviewing can handle very complex skip patterns. The computer can check as many "if then" statements and previous responses as necessary, decide which question should be asked next, and pose that question on the screen.
Data Collection and Graphs
Data file built in real time Downloadable in several formats At least simple graphs are often readily available
Avoid One-Sided Questions
Do you feel firms today are concerned about their employees? ___Yes ___ No ___ No opinion Would you agree that the deregulation of the airline industry has benefited customers? ___Yes ___ No ___ No opinion
Order Effects
Does The Question Create The Answer?
Would you agree or disagree that the deregulation of the airline industry has benefited customers? ___Yes ___ No ___ No opinion What kind of question is this?
Double barreled questions asking about two issues in one question
Avoid Leading Questions
Ensure questions are unbiased and avoid leading questions. BAD "How much better do you like Donald Trump than Hillary Clinton?" BETTER "Which candidate do you prefer - Donald Trump or Hillary Clinton?"
Avoid Unbalanced Questions
How important is price to you in buying a new car? ___More important than any other factor ___Extremely important ___Important ___Somewhat important ___Unimportant Than answered choice are scud towards important
Balanced Questions
How important is price to you in buying a new car? ___Very important ___Relatively important ___Neither important nor unimportant ___Relatively unimportant ___Very unimportant What type of scale is this? lankert scale
Guarding Against Errors
How often do you eat eggs for breakfast? _____ Frequently _____ Occasionally _____ Rarely _____ Never (Frequently, occasionally, and rarely mean different things to different people) misinterpretations
Complex Questions
In which of the following do you typically invest your liquid assets? ___Insured accounts ___Stock market ___Insured accounts and stock market ___Other accounts (asking about liquid assets when we are talking bout longterm questions) Of the total number of miles you drove during the past month, approximately what percentage was traveled in driving to and from work? ___Percent ( have to do a lot of math)
Question Wording
Is the vocabulary simple, direct, and familiar to all respondents? Do any works have vague or ambiguous meanings? Are any questions "double-barreled"? Are any questions leading or loaded? Are the instructions potentially confusing? Is the question applicable to all respondents? Are the questions of appropriate length?
Simple and Brief
Keep questions simple. BAD "Considering the candidates running for office this election, who is your preferred candidate?" BETTER "Which candidate do you prefer?"
Avoid Leading Questions
Leading a taker to answer a specific way. Don't you think offshore drilling for oil is environmentally unsound? ___Yes ___ No ___ No opinion Do you think the quality of products on the market today is as high as it used to be ten years ago? ___ Yes ___ No ___ No opinion
Specific and Clear
Make questions specific. BAD "Please tell us what you think of this restaurant." BETTER "What did you think of our restaurant's service?"
Adding New Response Categories
Multiple-category questions with an "other" category When a pre-specified number of respondents provide the same open-ended response to the "other" category, that response can be automatically converted and added to the structured categories.
Avoid Order Effects (question order)
Placement/order of questions can influence subsequent responses. Asking income level then price-related questions. Asking detailed questions about service then overall satisfaction.
Sequencing of Questions
Position of Demographic and Sensitive Questions Arrangement of Related Questions Funnel and Inverted-Funnel Sequences Skip Patterns
Questionnaires for Computerized and Online Interviewing
Randomizing response choices Checking for response consistency Incorporating complex skip patterns Personalization Adding "new" response categories Data Collection and Graphs
Computer surveys offer additional benefits to the...
Researcher
The organization and flow of the survey should make sense to the....
Respondent
Position of Demographic and Sensitive Questions
Respondents' personal or demographic characteristics (e.g., age, education level, income, etc.) are placed at the end of the questionnaire. Why do we do this?
Writing good survey question involves....
Several considerations
Inverted-Funnel Sequences
Starting more specific then get more broad
Checking for Response Category
The computer can be programmed to check for consistency between the response given to a question at hand and responses given to certain key questions asked earlier. Prior responses can also be "carried over" into new questions.
Randomizing Response Choices
The computer can be programmed to randomize the order of presentation of the categories for multiple-category questions in which category sequence may influence response choice. Helps alleviate order bias.
Categories Must be Mutually Exclusive
Use mutually exclusive categories. BAD "What is your age?" 18-24 ___ 24-30 ___ 30-35 ___ BETTER "What is your age?" 18-24 ___ 25-30 ___ 31 35 ___
Guidelines for Question Sequencing
Use simple, interesting opening questions Use the funnel approach, asking broad questions first (then move to more complex questions) Carefully design branching questions Ask for classification information last *IMPORTANT* Place difficult or sensitive questions near the end
Avoid Order Bias (order of response choices)
What color do you consider to be your favorite? Blue Yellow Green Orange Respondents have a propensity to choose from the beginning and ends of lists.
Funnel Sequences
Which of the following types of TV shows do you watch? (Check all categories that apply.) ____News shows ____Quiz shows ____Game shows Which of the following types of TV shows *do you like the most*? (Check as many categories as apply.) ____News shows ____Quiz shows ____Game shows Starting broad and get more specific
Double-Barreled Questions
touches upon more than one issue, yet allows only for one answer.