Questionnaire Design

Step 6: Evaluate, Retest, and Revise

Always a good idea to get questionnaire approved Evaluate Is the question necessary? Each question must serve a purpose Is the questionnaire too long? No more than 20 minutes Will the questions provide the information needed? Pretest—trial run of questionnaire Pre-testing Specific Questions for Alternative Forms Meaning & Understanding Pre-testing the Questionnaire Respondent interest and attention Flow of the questionnaire Skip patterns Length/time to complete

A Good Questionnaire has..

Ask the questions necessary to achieve the research objective and test hypothesis Involve managers who initiated the research Ask the questions that are high in validity and reliability Well phrased Unambiguous Unbiased Appropriate sequences Consider respondents Ability to understand questions Ability and willingness to answer questions

Step 5: Determine flow and layout continued

Be careful with branching questions 1. Have you ever bought a Blackberry? Yes _____ No _____ If yes, continue with question 2, otherwise go to question 20. -Limit use in self administered surveys. Very useful in Web surveys. Piping: integrate the response from a question into later questions.

Step 2: Determine the Data Collection Method

Effect of method on question format

Open- ended questions

Example: People look for different things in a car. What do you consider most important when buying a car? Why? Advantages: More information Wide range of responses Maybe more accurate description Limitations: Time consuming for respondent Coding and analysis is difficult - often open-ended questions require probing, where the interviewer needs to elicit more of a response

Step 3: Determine the Question Response Format

Form of questions: Open-ended, Close-ended Issues to consider- Cost and time Ease of tabulating & respondent completing Ease of respondent understanding Rule 3: Collectively Exhaustive: together questions must include all examples/ possibilities Rule 4: Question responses should be mutually exclusive Rule 5: Avoid order bias (position bias)- people tend to chose first or last option *Tip to automatically rotate response order*

Questionnaire Design Process

Questionnaire: is a set of questions designed to generate data necessary to accomplish the objectives of the research project. 8 Step Process: 1. Objective & Resource constraints 2. Survey Collection Method 3. Question Response Format 4. Question Wording 5. Flow and Layout 6. Evaluate, Approve, Pre-test & Revise 7. Final Copy 8. Implement

Close- ended questions

Questions that require the respondent to choose from a list of answers. Advantages: Easier to answer Easy to code and analyze More likely to respond for some personal data (e.g. income, age) - asked in multiple- choice, scaled responses Limitations: Information is lost Answers may not be precise Not easy to develop alternatives

Step 5: Determine Flow and Layout

Rule 10: Proper title & introduction Rule 11: Screener questions to identify qualified respondents Rule 12: Use the Funnel Technique Ask general questions first Obtain interest with simple warm-up questions -- Who is your cell phone service provider? Ask questions that require "work" in the middle --What features do you like best about your cell phone? --Please rate the following characteristics in term of importance Rule 13: Position sensitive, threatening, and demographic questions at the end Rule 14: Allow plenty of space for open-ended responses Rule 15: Instructions should be clear and easy to identify. Use CAPITAL LETTERS or bold Rule 16: Be aware of anchoring and order bias Rule 17: Each question should begin and end on same page Rule 18: Layout should be professional, not crowded, and easy to read Rule 19: For web surveys, have a status bar Rule 20: Use a proper closing

Step 1: Determine Objectives, Resources and Constraints

Rule 1: Let the research objectives and hypotheses dictate what questions to ask - Survey Objectives: Outline of the decision making information sought through the questionnaire Rule 2: Avoid unnecessary questions- each questions must serve a purpose

Step 4: Phrasing/ Wording of Questions

Rule 6: The wording must be clear and unambiguous. Rule 7: Avoid biasing the respondent. - Do not use a specific example to measure a broader situation - Avoid Double Barreled Questions - Ask questions 1 by 1 - Avoid leading questions that set perception/ frame of respondents mind Rule 8: Consider the respondent's ability to answer the question - Do not use jargon - Avoid making assumptions that aren't obvious - Do not ask specifics when respondents are likely to remember only generalities Rule 9: Consider the respondent's willingness to answer the question -Don't be pushy- can use 3rd person technique - Counter biasing statements