MKT 444 - Exam 2
How do you implement the survey questionnaire?
-Have clear instruction to supervisors and field management companies -Field management companies - Many personal interview studies are conducted by field service companies
What are the 4 measurement processes?
-Nominal scales -Ordinal scales -Interval scales -Ratio scales
What are the 3 no-interview survey types?
-Self-Administered Questionnaire -Mail surveys -Email/Internet
What are the 10 steps of the questionnaire design process?
1. Determine survey objectives, resources, and constraints 2. Determine the data-collection method 3. Determine the question response format 4. Decide on the question wording 5. Establish questionnaire flow and layout 6. Evaluate the questionnaire 7. Obtain approval of all relevant parties 8. Pretest and revise 9. Prepare the final copy 10. Implement the survey
What is a net promoter score (NPS)?
A measure of satisfaction; the percentage of promoters minus the percentage of detractors when answering the question, "would you recommend this to a friend?"
What is a measurement error? What are the 6 types?
A systematic error that results from a variation between the information being sought and what is actually obtained by the measurement process. -Input error -Response bias -Nonresponse bias -Measurement instrument bias -Interview error/Interviewer bias -Surrogate information error
What are the 2 types of mail surveys? What are the pros and cons?
Ad Hoc Mail Surveys: One-shot mail surveys sent to selected individuals with no prior contact. Sample is used fro the single project. Mail Panels: Pre-contacted and pre-screened participants who are periodically sent questionnaires and are usually offered gratuity Pros: -Not too expensive compared to the ones with interviews -Have targeted mailing lists -Longer questionnaires Cons: -Low response rates -Mailing list often out of date -Not sure who completed the survey
What is a semantic differential scale? What are some advantages and disadvantages?
Advantages: -It examines the strengths and weaknesses of a concept by having the respondent rank it between dichotomous pairs of words or phrases that could be used to describe it. -It has been shown to be sufficiently reliable and valid for decision making and prediction in marketing and behavioral sciences. Disadvantages: -The semantic differential suffers from a lack of standardization. -There is no single set of standard scales
What are the advantages and disadvantages of online survey research?
Advantages: -Saves time, cheaper -Convenient for the people taking the survey -Easy to sort data (instant reporting), ready personalization -High response rates -Ability to contact the hard-to-reach (professionals) -Simplified and enhanced panel management Disadvantages: -Not everyone can do it online -Misrepresentation -Unrestricted internet sample -May be no sample frame -Lack of "callback" -Programming errors -Increased spam and online clutter
What is probing?
An interviewer encourages the respondent to elaborate or continue the discussion Powerful probes will advance a discussion quickly away from top-of-mind responses and access deeper information, allowing insight into the baseline thinking that drives behavior EX: What do you think is most in need of improvement at the airport? Probe: Why do you say that? What do you mean? What else...?
What is a stapel scale?
Attributes in the middle and the respondent chooses negative or positive points
What is balanced scale? What is an unbalanced scale?
Balanced: Scales with equal numbers of positive and negative categories. One exception is when the attitudes of respondents are likely to be predominantly one-sided, either positive or negative. Recommended. Unbalanced: Has a larger number of response options on one side, either positive or negative
How do you select a scale?
Based on the scope of the research objectives, select type of scale, and number of scale categories. You want a balanced scale.
What are closed-ended questions? What are the advantages and limitations of them?
Close-ended questions: Require respondents to choose from a list of answers (easier) Advantages: -Providing response alternatives that jog respondent's memories -It eliminates interviewer bias -Coding and data entry can be done automatically Limitations -The researcher must spend time generating the list of possible responses -The range of possible answers can be too long or too short -Position bias
What are commercial online panels? What are panel members?
Commercial Online Panels: Not created for the exclusive use of any one specific company, or for any one particular project Panel Members: Upon joining, members answer an extensive profiling questionnaire that enable the panel provider to target research efforts
What is the most common way to gather primary data? What is the least common way to gather primary data?
Common: Surveys Least Common: Door-to-Door
What are door-to-door interviews? What are the pros and cons?
Conducted face to face in people's homes. This trend is used in some government research (US Census) and most popular form of interviewing in many developing countries Pros: -Direct customer contact -Can demonstrate products Cons: -Expensive -Safety issues -High refusal rate
What is a constant sum scale? What are the advantages and disadvantages?
Constant Sum Scale: Assigning limited points to characteristics based on preferences Advantages: Respondents can value each item relative to all other items Disadvantages: The respondent may have difficulty allocating the points to total 100 if there are a lot of characteristics or items. Most researchers feel that 10 items is the upper limit on a constant sum scale.
What are the 2 different time frames used for survey research?
Cross Sectional Surveys: One time snapshots of population; use the same survey and different sample Longitudinal Surveys: Use the same survey and same sample. Identify market trends. Use tracking panels and groups
What are the 3 types of closed-ended questions?
Dichotomous Questions: Closed-ended questions that ask the respondents to choose between 2 answers (two fixed alternatives, easy to administer, usually evoke a rapid response) Multiple-Choice Questions: Closed-ended questions that ask respondent to choose among several answers (not all possible alternatives are included, spend time generating lists, position bias from settling on a range of possible answers) Scaled-Response Questions: Closed-ended questions in which the response choices are designed to capture the intensity of the respondent's feeling (Permits measurement of the intensity, coding is easy, use much more powerful statistical tools to analyze results)
What is the criteria for a good questionnaire?
Does it provide decision-making information? Does it consider the respondent? The researcher designing a questionnaire must consider not only the topic and the type of respondent, but the interviewing environment and questionnaire length as well (It is best to use simple, everyday language, as long as the result is not insulting or demeaning to the respondent) Does it meet editing, coding, and data processing requirements? Editing, skip patterns, piping, and coding
What is forced vs non-forced?
EX: Friendly 1 2 3 4 5 Unfriendly Do not know -Adding a "Don't know" is considered non-forced and may be viewed as an easy way out for lazy respondents. -A scale without a neutral point or a "Don't know" option forces an answer from respondents who may have no information about an object to state an opinion. This creates a bias. -Use research objective as a guide and make a judgement call.
What is editing? What is a skip pattern (branching)? What is piping? What is coding?
Editing: Going through each questionnaire to ensure that skip patterns were followed and the required questions filled out Skip pattern (branching): Sequence in which questions are asked, based on the respondent's answer Piping: Integrates responses from a question into later questions 1. Do you have a Honda or Toyota? 2. What color is your Toyota? Coding: Process of grouping and assigning numeric code to the various responses to a question (chapter 13)
What are mall-intercept interviews? What are the pros and cons?
Face to face interviews conducted by intercepting people in shopping malls. Trend is declining. Pros: -Direct customer contact -Can test products -Less expensive Cons: -Many distractions are inherent in environment -Respondent may be in a hurry, not in proper frame of mind to answer survey questions -More chance for interview bias -Non-proabability sampling problem
What are some ways to assess validity?
Face validity - The researcher has to make a judgment as to whether the measure "looks like" it measures what it is supposed to. Content validity - This requires the researcher to use systemic process to assess the adequacy. Step in developing content validity: -Literature review -Panel of experts -Pretest with open-ended question -Scale reduction through data analysis Predictive validity - Degree to which a future level of a criterion variable can be forecasted by a current measurement scale. -Example: SAT that everyone takes to enter college is believed to be a predictor of how well an individual will do in college and the person's potential to learn.
What are self-administered questionnaires? What are the uses? What are the pros and cons?
Filled out by respondents with no interviewer present. Usages: Malls, airports, restaurants, or other central locations where the researcher has access to a captive audience. Kiosk-based computer interviewing at trade shows or conventions Pros: -Respondents can take their time -No interviewer bias Cons: -Inability to probe for a more definitive answer -Poor sample
What are executive interviews? What are the pros and cons?
Industrial equivalent of door-to-door interviewing. Trend is moving online. Pros: -Talking with decision maker is valuable -Can have complicated questioning Cons: -Expensive -Requires highly skilled interviewer -Might not be reflective of the population
What are interval scales? What's an example? What are some classifications? What are some limitations? How do you analyze them?
Interval Scales: Have categories which are broken into equal segments. The numbers used as categories do have meaning and measure how much of a trait an observation has. EX: I only eat healthy food Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree Classifications: -"1-7" point interval scale -Age, income, etc as ranges Limitations: With arbitrary zero point, it cannot tell the difference between points Analysis Approach: -Mean -Standard deviation -Statistical analysis (t-test and f-test)
What are the 2 things to keep in mind with evaluating a questionnaire?
Is the question necessary? Is the questionnaire too long? -Mall-intercept should be limited to 20 minutes; internet and telephone surveys should be 15 minutes or less. -Incentives can lower the cost of surveys because the response rates increase -Gift cards are probably the most commonly used incentive today
What is an itemized rating scale?
Itemized rating scales let the respondent selects an answer from a limited number of ordered categories. they are easy to construct and administer. The definitive categories found in itemized rating scales usually produce more reliable ratings.
What is a continuous rating scale?
Let's the respondents select an answer from a limited number of order categories. They are easy to construct and administer. Definitive categories found in itemized rating scales usually produce more reliable ratings
What are the 4 guidelines of questionnaires?
Make sure the wording is clear -Clarity achieved by avoiding ambiguous terminology, using reasonable vernacular language adjusted to the target group, and asking only one question at a time -Avoid overlapping answers and double-barreled question Avoid biasing the respondent (avoid leading questions) Consider the respondent's ability to answer the questions Consider the respondent's willingness to answer the questions -Use generalization ("most people") -Use counter biasing statements to make embarrassing topics less intimidating for respondents to discuss ("do you or anyone in your family...")
What are surrogate information errors? How do you minimize this survey error?
Measurement error that results from a discrepancy between the information needed to solve a problem and that sought by the research Minimize Survey Error: This error results from seeking and basing decisions on the wrong information. It results from poor design and can be minimized only by paying more careful attention to specification of the types of information required to fulfill the objectives of the research
What are nonresponse biases? How do you minimize this survey error?
Measurement error that results from a systematic difference between those who do and those who do not response to a measurement instrument -Potential respondent cannot be reached, -Cannot participate, -Refuses to participate Minimize Survey Error: This error results from the fact that those people chosen for the sample who actually respond are systematically different from those who are chosen and do not respond. It is particularly serious in connection with mail surveys. It is minimized by doing everything possible (e.g., shortening the questionnaire, making the questionnaire more respondent friendly, doing callbacks, providing incentives, contacting people when they are most likely to be at home) to encourage those chosen for the sample to respond
What are measurement instrument biases? How do you minimize this survey error?
Measurement error that results from the design of the questionnaire or measurement; also known as questionnaire bias Minimize Survey Error: This error is minimized only by careful questionnaire design and pretesting
What are input errors? How do you minimize this survey error?
Measurement error that results from the incorrect transfer of information from a survey document. Minimize Survey Error: Result of incorrect keystrokes from a respondent. Use software checks to find illogical response patterns or improperly scanned machine-scored questionnaires
What is interviewer error/interviewer bias? How do you minimize this survey error?
Measurement error that results from the interviewer's influencing consciously or unconsciously - the answers of the respondent. This error occurs because of interactions between the interviewer and the respondent that affect the responses given. Minimize Survey Error: It is minimized by careful interviewer selection and training. In addition, quality control checks should involve unobtrusive monitoring of interviewers to ascertain whether they are following prescribed guidelines
What are response biases? How do you minimize this survey error?
Measurement error that results from the tendency of people to answer a question incorrectly through either deliberate falsification or unconscious misrepresentation. Minimize Survey Error: This error occurs when something about a question leads people to answer it in a particular way. It can be minimized by paying special attention to questionnaire design. In particular, questions that are hard to answer, might make the respondent look uninformed, or deal with sensitive issues should be modified.
What is the most common complaint about online surveys? Is it true?
Most common complaint is about its representation and how Internet users are not the general population as a whole. This comment has largely disappeared in the US - 95.7% of households have access to broadband Internet. Telephones produce similar results.
What are telephone interviews? What are the 3 methods? What are the pros and cons?
Most popular form until 1990. This trend has started to move online. Methods: -Predictive dialing (random digit dialer) -Call center telephone interviews -Computer assisted telephone interviews (CAT) Pros: -Inexpensive -High quality sample -Coverage Cons: Low response rate
What are nominal scales? What's an example? What are some classifications? What are some limitations? How do you analyze them?
Nominal Scales: Partition data into categories that are mutually exclusive and collectively exhaustive. Every observation will fall into one of several categories and only one. Most basic and least powerful scale design. EX: Please indicate your marital status: 1. Married 2. Single 3. Divorced 4. Separated 5. Widowed Classifications: -Yes or no questions -Gender, race, ethnicity -Occupation Limitation: Numbers are simply labels or for identifications. Analysis Approach: -Frequency counts -Percentages -Modes
What are email/Internet surveys?
Now leading survey research methodology. Fast and cheap.
What is sample design error? How do you minimize this error?
Occurs when sampling procedures are incomplete or improper or when appropriate selection procedures are not properly followed. Minimize this selection error: Develop procedures and having quality control checks
What is online qualitative research?
Online focus group -Webcam online focus group -Improving virtual focus groups with Telepresence -Using the web to find focus group participants Online IDI Marketing Research Online Communities (MROC) -It is a carefully selected group of consumers who agree to participate in an ongoing dialogue with a corporation
What are the 4 types of recruiting methods and panel management?
Open Source/Open online panel recruitment: Any person with Internet access can self-select to be in a research panel Initiation-Only/Closed Online Panel Recruitment: Invite only pre-validated individuals or those with shared known characteristics to enroll in a research panel. Incentives: "pay all" or sweepstakes Panel Management: Privacy, positive experiences, keep recruiting
What are open-ended questions? What are the advantages and disadvantages of them?
Open-ended questions: Are those in which the respondents can reply in their own words Advantages: -Enable respondents to give their general reactions -Suggest additional alternatives not listed in a close-ended question Limitations: -The time and money consuming process of editing and coding if done manually
What are ordinal scales? What's an example? What are some classifications? What are some limitations? How do you analyze them?
Ordinal Scales: Have the labels of nominal scales (mutually exclusive and collectively exhaustive) plus the ability to order data. Ordinal numbers are strictly used to indicate rank order. EX: Indicate your age 1. 18-24 2. 25-34 3. 35-44 4. 45-54 5. 55+ Classifications: -Best liked - worst liked -First, second, third -Small, medium, large Limitation: Cannot be used to determine the absolute difference between rankings Analysis Approach: -Mode -Median -Frequency counts
What is a paired comparison? What are the advantages and disadvantages?
Paired Comparison: What characteristic in each pair is more important to you Advantages: First, it is easier for people to select one item from a set of two than to rank a large set of data. Second, there is no pattern in the ordering of items or questions to create a source of bias. Disadvantages: If all possible pairs are evaluated, the number of paired comparisons increases geometrically as the number of objects to be evaluated increases arithmetically.
How do you prepare the final copy of a questionnaire?
Precise instructions for typing, spacing, numbering, and pre-cording Mail survey - Compliance and subsequent response rates may be affected positively by a professional-looking questionnaire
What is a purchase intent scale?
Purchase Intent Scale: Used to measure a respondent's intention to buy or not buy a product. The purchase-intent question usually asks a person to state whether he would definitely buy, probably buy, probably not buy, or definitely not buy under study. The purchase-intent scale has been found to be a good predictor for consumer packaged goods.
What is ranked order? What are the advantages and disadvantages?
Ranked Order: Rank products/brands amongst each other in several categories Advantages: Instructions are easy to understand, and the process typically moves at a steady pace. Disadvantages: Results could be misleading if not all alternative choices are included. Ordinal data does not provide intensity info.
What are ratio scales? What's an example? What are some classifications? What are some limitations? How do you analyze them?
Ratio Scales: Have all the advantages of the other scales, plus it has a meaningful origin (0). It is the most powerful scale. Flat numeric type data. EX: How much did you spend dining out last month at all restaurants? _______ Classifications: -Age = 50 (not an age range) -Income = $25,000 (not an income range) -Number of children: _____ -Indicate how many times you have eaten at the following places in the last month Analysis Approach: -Mean -Standard deviation -Statistical analysis
What is the refusal rate? What is the response rate?
Refusal rate: % of people contacted who refused to participate in a survey Response rate: % of persons who participated in a survey
What is reliability? What is validity?
Reliability: Degree to which measures are free from random error, and therefore, provide consistent data. Validity: Degree to which what the researcher was trying to measure was actually measured.
What is frame error? How do you minimize this error?
Resulting from using an inaccurate or incomplete sample frame Minimize Frame Error: Develop preliminary quality control checks to evaluate the accuracy and completeness of the frame
What is population specification error? How do you minimize this error?
Results from an incorrect definition of the population or universe from which the sample is to be selected. Minimize Specification Error: Careful consideration of the definition of the population
What is a likert scale?
Scale consists of a series of statements expressing either a favorable or unfavorable attitude toward the concept under study. The respondent is asked to indicate the level of their agreement or disagreement with each statement by assigning it a numerical score Strongly disagree to strongly agree
What is scaling? What are the two main types?
Scaling: Procedures for assigning numbers (or other symbols) to properties of an object in order to impact some numerical characteristics to the properties in question. Scaling is a measurement tool. Comparative Scales: Assigning limited points to characteristics based on preferences -Constant sum -Paired comparison -Ranked order Non-Comparative Scales: Scales in which judgement is made without reference to another object, concept, or person -Continuous rating scales -Itemized rating scales (Semantic differential, likert, and staple)
What are these types of questions: -Screeners -Warm-ups -Transitions -Complicated -Classification
Screeners (Qualifying questions): Qualifying or basic questions that lay the groundwork for upcoming questions Warm-Ups (First few questions): Gets the respondent thinking about the topic at hand. Start with general questions and establishes parameters about the respondents' attitudes, behavior, etc. Transitions (First third of questions): Questions that set the tone for the more difficult questions to come. Complicated (Second third of questions): Move to the specific and complex questions. Use of rating scales for attributes, attitudes, beliefs, opinions, etc. Tackling controversial issues. Classification (Last third of questions): Personal and demographic type questions - more invasive.
What is a questionnaire?
Set of questions designed to generate the data necessary to accomplish the objectives of the research project; also called an interview schedule or survey instrument Plays a critical role in the data-collection process. Improper design can lead to incomplete information, inaccurate data, and higher costs
What can you use the Internet for secondary data for?
Sites of interest to marketing researchers -American Marketing Association -American Fact Finder -Bureau of Economic Analysis Newsgroups: A primary means of communicating with other professionals and special-interest groups on internet. It functions like bulletin boards for a particular topic or interest. To participate, a person must subscribe to the group, which is usually free. Blogs: A frequent, chronological publication of personal thoughts and weblinks. Companies use blogs to communicate directly with customers and other businesses. Marketing researchers have used blogs to monitor brands, track trends, profile customers, and identify unmet needs.
What is social media marketing research?
Social media gives opportunities to marketing researchers to understand their consumers and potential customers like never before Social media tracking Conducting a Facebook focus study Conducting surveys -Social media can be used as respondent recruiting tool -All social media have apps for survey research. The decision to use which one depends on the survey population and available tools for data extraction/analysis
What is survey research? What is the goal of it?
Survey Research: Use of a questionnaire to gather facts, opinions, and attitudes; it is the most popular way to gather primary data Have a high rate of usage in marketing research compared to other means of collecting primary data for some very good reasons (need to know why, how, and who) Goal: Make inferences about the total population based on the responses given by respondents sampled
What are the 2 broad types of errors?
Survey error/Random sampling error: Error that results from chance variation, which is the difference between the sample value and the true value of the population mean. Best way to minimize is by having a large sample size. Systematic Error/Bias: Error that results from problems in the research design or flaws in the execution of the research design; sometimes called a non-sampling error -Measurement error -Sample design error -Population specification error -Frame error
What are survey objectives? What are the basic determinate of the estimated data collection costs of survey research?
Survey objectives: Outline of the decision-making information The basic determinants of estimated data collection costs of survey research: -Questionnaire -Incidence rate: Percentage of people screen who are expected to quality (High incidence is better)
What are the 4 interview survey types?
Telephone Door-to-door Executive Interview Mall-Intercept Interview
What are some ways to assess reliability?
Test-retest reliability - Repeat the same survey few weeks later with the same sample of people. Equivalent form - Use similar questionnaires to the same sample of people, and then compare the correlations of similar test items. Internal consistency - use same instrument with two different samples, or sometimes split the sample into halves.
What is measurement? What are the 2 important concepts?
The process of assigning numbers or labels to persons, objects, or events in accordance with specific rules for representing quantities or qualities or attributes Process of measurement involves two important concepts -It is not the person, object, or event that is being measured -The numbers are assigned by rules and/or process specified by the researcher
What is mobile Internet research? What are the advantages and limitations?
The rise in the use of cell phones has spurred an interest in using them as a means of conducting survey research. Keep these surveys short (max 10 questions) and simple (limit open-ended questions). Social videos can help. Currently, about 81% of households in the US own at least 1 smart phone. Advantages: -Increased response rates and convenience -Broader reach -Richer content. Disadvantages: -Distractions -Ownership
True or False: Whenever possible, use the higher order scales to increase the type of analysis that can be conducted.
True.
