Chapter 10 - Questionnaire Design
Steps to Questionnaire Design Process
1. Determine survey objectives, resources, and constraints 2. Determine the data-collection method 3. Determine the question response format 4. Decide on the question wording 5. Establish questionnaire flow and layout 6. Evaluate the questionnaire 7. Obtain approval of all relevant parties 8. Pretest and revise 9. Prepare final copy 10. Implement the survey
Data Coding Process
1. List Responses. 2. Consolidate Responses 3. Set Codes 4. Enter Codes (Read responses to individual open-ended questions on questionnaires, natch individual responses with consolidated list of response categories)
What are the 5 key steps of data analysis?
1. Validation and Editing 2. Coding 3. Data Entry 4. Logical Cleaning of Data 5. Tabulation & Statistical Analysis
Conjoint Analysis
A survey based statistical technique used in market research that helps determine how people value different attributes (feature, function, benefits) that make up an individual product or service.
clarity
Achieved by avoiding ambiguous terminology, using reasonable, vernacular language adjusted to the target group, and asking only one question at a time.
mathematical differences
By definition, if numbers are not exactly the same, they are different. This does not, however, mean that the difference is either important or statistically significant.
Put Instructions in ________ ________
Capital Letters
clean the data
Check for data entry errors or data entry inconsistencies
scaled response question
Closed-ended questions in which the response choices are designed to capture the intensity of the respondent's feelin
multiple choice
Closed-ended questions that ask the respondent to choose among several answers; also called multichotomous questions.
dichotomous questions
Closed-ended questions that ask the respondents to choose between two answers.
Were the questionnaires filled out completely? This is an example of of..
Editing
p-value
Exact probability of getting a computed test statistic that is due to chance. The smaller the p value, the smaller the probability that the observed result occurred by chance.
editing
Going through each questionnaire to ensure that skip patterns were followed and the required questions filled out.
important differences for managers
One must be able to distinguish between mathematical differences and statistically significant differences in using the data analysis in managerial decision making.
survey objectives
Outline of the decision- making information sought through the questionnaire.
closed-ended question
Question that requires the respondent to choose from a list of answers.
open-ended questions
Questions to which the respondent replies in her or his own word
screeners
Questions used to identify appropriate respondent
skip pattern
Sequence in which questions are asked, based on a respondent's answer
________________ ________________ early in the interviewing process can distort answers.
Sponsor identification
Were the interviews conducted as requested and specified? This is an example of..
Validation
statistical significance
a difference that is large enough that it is not likely to have occurred because of chance or sampling error
The questionnaire must translate the survey objectives into
a form understandable to respondents and "pull" the requisite information
The research process often begins when...
a manager needs decision-making information that is not available internally or from secondary sources.
What does the bottom box score signal?
a need for managerial action
data anlysis
a set of methods and techniques used to obtain information and insights from the data
regression analysis
a set of statistical processes for estimating the relationships among variables
descriptive statistics
a single number used to make inferences about the population in question
one-way frequency table
a table showing the number of responses to each answer of a survey question
open-ended questions may suggest additional _____________ not listed in a closed- ended response format.
alternatives
dataset
an arrangement of numbers (mainly) in rows and columns
hypothesis testing
an assumption or theory that a researcher or manager makes about some characteristics of the population under study
Managerial _____________ commits management to obtaining a body of information via (questionnaire).
approval
reactive probes
are almost like "natural probes," except that they are more "knee jerk" in character.
what is the purpose of hypothesis testing?
are the research results different enough from the norm that some element of the firm's marketing strategy should change?
wing counter biasing statements
asking the question, that the behavior or attitude is not unusual—makes embarrassing topics less intimidating for respondents to discuss
closed ended
assign numbers to pre-determined categories
What does data analysis help with?
avoid errors, judgement, and false conclusions
How can clarity be improved?
by stating the general purpose of the survey at the beginning of the interview.
Editing open-ended responses require ________ the many response alternatives into some reasonable number.
collapsing
error check routines
computer programs that accept instructions from the user to check for logical errors in the data
marginal report
computer-generated table of the frequencies of the responses to each question,
. A long screening questionnaire can significantly increase the ______ of the study, as more information must be obtained from every contact with a respondent.
cost
How is a dataset created?
data coding
open ended
determine categories assign a number
Up until the 1950s, _______ ____ ________ interviewing was the primary mode of survey data collection, with the respondents interviewed in their home.
door-to-door
Position Sensitive, Threatening, and Demographic Questions at the _____
end
cross tabulation tables
examination of the responses to one question relative to the responses to one or more other questions
logical or machine cleaning of data
final computerized error check of data
scanning
form of data entry in which responses on questionnaires are read in automatically by the data entry device
intelligent data
form of data in which the information being entered into the data entry device is checked for internal logic
A precoded, open-ended question allows the respondent to answer in a
freewheeling format; the interviewer simply checks coded answers as they are given.
What is primary advantage of using scaled response questions?
hat scaling permits measurement of the intensity of respondents' answers.
The portion that would ___________ recommend a business to others or the portion expressing the highest likelihood of doing business again
highly
cross tabulations use statistical techniques to understand the differences between and among ________.
variables
Survey data can be gathered, in a ________ of ways and the survey method impacts questionnaire design.
variety
The researcher designing a questionnaire must consider not only the topic and the type of respondent, but ..
the interviewing environment and questionnaire length as well
alpha level
the probability level used by researchers to indicate the cutoff probability level (highest value) that allows them to reject the null hypothesis
general questions asked to get the person thinking about a concept, company, or type of product; then ..
the questionnaire moves to the specifics.
measures of dispersion
this measures how spread out the data is range, std. deviation
What are the problems with open ended coding?
time consuming, subjective, can have a long list
What is the purpose of statistical significance?
to make inferences about the population based on sample results
What is the marginal report used for?
to monitor entry of valid codes and correct use of skip patterns
What is the primary role of any questionnaire?
to provide the information required for management decision making.
Perhaps the most important criterion for this phase of questionnaire development is the...
necessity for a given question.
In some cases, a respondent may ________ have acquired the information needed to answer the question. Another problem is _________________.
never forgetfulness.
Data collection
phase of the marketing research process during which respondents provide their answers or information to inquiries posed to them by the researcher
The researcher determines whether a hypothesis concerning some characteristic of the _____________ is likely to be true given the evidence
population
When final managerial approval has been obtained, the questionnaire must be _________.
pretested.
Probes fall into three distinct categories:
proactive, reactive, spontaneous (natural probe)
beta level
probability of making a type II error
validation
process of ascertaining that interviews actually were conducted as specified
editing
process of ascertaining that questionnaires were filled out properly and completely
data
process of converting information to electronic format
coding
process of grouping and assigning numeric codes to the various responses to a question
double-barreled questions
questions that attempt to get at multiple issues at once, and so tend to receive incomplete or confusing answers
Top-Box/Bottom-Box Scores
refers to the portion of respondents who choose the most favorable response toward a company
p-value is lower than the significance level..
reject the null
multiple choice questions
replies do not have to be coded as they do with open-ended questions, but the amount of information provided is more limited
a pretest is administered to target ____________for the study.
respondents
What is the logic behind a top-box score?
respondents who choose the most extreme responses are really quite unique compared to others
skip pattern
sequence in which later questions are asked, based on a respondent's answer to an earlier question or questions
questionnaire
set of questions designed to generate the data necessary to accomplish the objectives of the research project; also called an interview schedule or survey instrument
After introductory comments and screens to find a qualified respondent, the initial questions should be ..
simple, interesting, and nonthreatening.
Questionnaires also play a key role in determining
survey costs, discussed in detail later in the chapter
Once a rough draft of the questionnaire has been designed, the marketing researcher should ...
take a step back and critically evaluate it.
Piping
integrates responses from a question into later questions
A questionnaire should be designed explicitly for the ____________ respondents.
intended
A basic problem with open-ended questions lies in the ______________ ______________ _________.
interpretation-processing area
Every questionnaire must have an ____________ and ____________.
introduction and closing
split questionnaire design
involves splitting a questionnaire into one core component that all respondents answer and multiple subcomponents with respondents answering only one subcomponent
proactive probe
is one that can be planned ahead of time, and it might look like the following:
Why should managers examine the bottom box score more?
it is more diagnostic of customer problems
To avoid the problem of a respondent's inability to recall, the researcher should ...
keep the referenced time periods relatively short.
The use of incentives can actually _______ survey costs because response rates increase and terminations during the interview decrease
lower
The researcher must make certain that the questionnaire contains sufficient numbers and types of questions to meet the decision-making needs of ___________.
management
measures of central tendency
mean, median, mode
probing
means that an interviewer encourages the respondent to elaborate or continue the discussion.
Ask Questions That Require "Work" in the ______
middle
