Marketing Research

Projective Technique: Sentence Completion

"Complete this sentence with the first word or phrase that comes to mind."

Descriptive Research Methods

Characterizes marketing phenomena without testing causal relationships - frequency of marketing phenomena occurring - degree of association between marketing variables - change in some outcome over time Typically quantitative methods Survey methods (questioning) - Mail, telephone, on-line, in-person Observational methods - Personal or mechanical (e.g., scanner data) Consumer panels - Shopping behavior, television viewing, media usage - Diaries

6. Prepare and Present Results

Clearly and accurately communicate insights to client - Avoid simply telling client what they wanted to hear Display data in easily understandable way

Cross-Sectional vs. Longitudinal Data

Cross-sectional data: "snapshots" of a phenomena at one point in time - Very common for conclusive research methods - Surveys, observational studies, etc. Longitudinal data: measure phenomena repeatedly over several time periods - Consumer panels: monitor performance for a fixed sample measured repeatedly over time - Measure how things change over time

Benefits of Qualitative Methods

Deeper insight - Motivations - Attitudes - Beliefs Unexpected discoveries Ideation Brand positioning Go beyond "functional"benefits of product Ex. Daisy Sour Cream (tub to squeezable tube)

Test-retest reliability

Do we get the same values when we measure the same people more than once? - Positive correlation between T1 (time 1 in Oct) and T2 (time 2 in Nov) - Create an index (i.e. combine items) for each time-point. - Compute correlation between the indexes. - Correlation should be positive and "large"

Convergent validity

Does the measure correlate strongly with things that it is theoretically related to?

Discriminant validity

Does the measure correlate weakly with things that it is theoretically unrelated to?

Designing In-Depth Interviews

Duration: ~30 minutes Number: 15 - 20 (or until theoretical saturation) Guidelines similar to focus group - Create outline of discussion questions - Open-ended questions; follow up unclear responses - Avoid leading questions; remain neutral - Move from general to specific questions -Take notes and/or record Goal is to capture depth of opinions - "Would you explain further?" - "Can you give me an example?" - "Tell me more about that." Stop when you reach "theoretical saturation"(i.e., no more novel insights) - Same for determining number of IDIs

Survey Introduction

Every survey should begin with an introduction describing: - who is running the survey - the general purpose of the survey - how the survey data will be used - how long the survey will take - whether the surveys will be anonymous

Secondary data:

Existing data previously gathered for different purpose that you can utilize - Used in exploratory and descriptive (but not causal) research - Pros: faster and cheaper than collecting primary data (can have initial analysis in a day or two, tons of free data), complex analysis not necessary and flexible (not constrained by specific hypothesis) - Cons: often outdated, often insufficient, often get summary results (ambiguous accuracy), not designed to answer your specific question, contextual variables threaten generalizability (time and changing markets may mean info is no longer relevant), subjectivity (risk of confirmation bias)

Exploratory vs. Conclusive Research

Exploratory research •Open-ended •Many possibilities •Small sample •Unstructured data •Non-statistical analysis Conclusive Research •Well-defined •Few hypotheses •Large sample •Structured data •Statistical analysis

Advantages of Secondary Data

Faster than primary data - Can have initial analysis in a day or two Cheaper than primary data - Reports cost a few hundred dollars - Tons of free data available Complex analysis often not necessary - Reports typically have done some analysis Flexible - Not constrained by specific hypothesis

BEWARE:

Focus groups can reveal key insights that managers were not aware of BUT Focus groups can also reveal idiosyncratic findings that only apply to the selected group at hand.THEREFORE DON'T treat the result of one focus group as generalizable to all consumers

Question Wording Guidelines

Focus on one issue or topic per question - Avoid "double-barreled" questions Keep the question as brief and as simple as possible - Only use vocabulary known to your sample - Stick to simple sentences if possible Make sure there is only one way to interpret the question (avoid ambiguity) Avoid leading or biased questions Use collectively exhaustive response categories (i.e., all possible responses are represented) Use mutually exclusive response categories (i.e., a person should only fall into one category)

3 key decisions

Formulating research design (2), selecting research method (3), and collecting data (4) involve three key decisions: - Exploratory or conclusive research? - Descriptive or causal research? (under conclusive research) - Secondary or primary data? (ex. focus group? in-depth interviews? experiments? observation?

IDIs - Laddering Technique

Goal: connect concrete attributes to abstract emotional benefits and motivations for purchase Why use it? - Most people cannot tell you the core motivations why they buy Product X when asked directly. - So instead, start by eliciting the product attributes that the consumer identifies with - Then ask them why those attributes are important --> benefits/consequences - Then ask why those consequences are important --> values Start by eliciting specific attributes - "What features of X do you like?" - "What features of X make you prefer it to other brands?" - "What is the most important attribute of X to you?" Follow up to reveal consequences/benefits of attributes - "Why is that feature important?" - "Why do you care about that attribute?" Follow up to reveal underlying values - "What does that do for you?" - "How does that make you feel?"

Evaluating Secondary Data

How can we use it? - How relevant is the original question? - How relevant is the original company? - Can this data answer our research question? -- Or do we need more secondary data to corroborate these findings? -- Or should we use it to guide or primary research?

Which survey method to use?

How complex are your questions?Do you need to include physical stimuli? What is desired sample size and representativeness? How much does the survey-taking environment matter (i.e., control)? Is social desirability a relevant issue? Could interviewer bias be a likely issue? How quickly do you need the data? What can you afford ($$)? Ultimately, you want to maximize the willingness and ability of respondents to answer your questions accurately

Evaluating Secondary Data

How was it created? - What was the purpose of the original study? - Who sponsored it? Is there a hidden agenda - What was the methodology? - What was the sampling method?- - When was the study conducted?

In-Depth Interview

Unstructured (i.e., loosely scripted) one-on-one personal interviews Use when you need deeper insights than focus groups provide Use probing questions to uncover details about beliefs and feelings Get beyond simple attitudes, opinions, and behaviors—discover why those attitudes, opinions, and behaviors exist or occur The interviewer is key to getting useful data

Primary data:

New data you collect for specific purpose - Used in exploratory, descriptive, and causal research - Pros: most likely to be actionable; can determine accuracy; customized - Cons: more expensive; takes time to collect - Sources: mail survey, internet survey, personal interview, phone interview, observation

Properties of Measurement Levels

Nominal measures ("unique") - e.g., birthplace, home zip code, college major Ordinal measures ("unique" + "order") - e.g., brand preference ranking, letter grade, education level Near-Interval measures ("unique" + "order" + "distance") - e.g., attitudes, satisfaction, intentions Interval measures ("unique" + "order" + "distance") - e.g., clock time, temperature (C/F), SAT score, IQ Ratio measures ("unique" + "order" + "distance" + "zero") - e.g., revenue, likes, retweets, height *You can convert higher level scales to lower levels, but not the reverse*

Disadvantages of Secondary Data

Not designed to answer your specific question - Questions that seem similar may in fact need different information to answer Often only get summary results - Ambiguity regarding individual responses and how data were collected Contextual variables threaten generalizability - Time and changing markets may mean the information is no longer relevant Subjectivity -risk of confirmation bias

Random vs. Systematic Error

Observed measure = true score + random error + systematic error (e.g., brand attitude = true attitude + random error + systematic error) True Score: what researcher actually wants to measure Random Error: out of researcher's control; due to random variation - e.g., respondents were tired or in a bad mood - How can we reduce random error? Systematic Error (Bias): within researcher's control; due to flawed research methods and procedures - results in consistently higher or lower estimates of population variables *Marketing researchers must make efforts to reduce systematic error*

Face validity:

On the surface, does it seem like you're measuring what you intend to?

Scaling Procedures

What's the right kind of scale for my measure? There are lots of details to work out: - Nominal, ordinal, interval, or ratio? - Number of categories (min. 5 for interval questions) - Should we have a neutral category available? - Balanced positive/negative categories - Category descriptions and numbering - Order from low-to-high or vice versa? - How should we label the endpoints?

Measurement error

Systematic (i.e., nonrandom) difference between the true value and the observed value due to the measurement process

Sample design error

Systematic (i.e., nonrandom) difference between true and observed value due to sampling design and selection process e.g., survey of fitness habits of BC students only includes participants recruited at the rec center

Interviewer error

interviewer consciously or unconsciously influences respondents Ex. Jennifer, a representative of UN Women (the gender equality organization of the UN), administered an in-person survey on gender attitudes and found that 90% of men believe that women should be paid equally

Input error

mistakes when responding to questions or entering survey responses in a database/spreadsheet

EXPLORATORY RESEARCH (formulate research design, select research method for data collection)

no clear research purpose and data requirements

Surrogate Information Error

not measuring what you're supposed to be measuring (operationalization) Ex. Brooks Brothers conducted a survey measuring general attitudes toward the athleisure category and found that people are very enthusiastic about it. Based on this, they released a line of workout clothes that barely sold.

Response bias

participants tend to consciously or unconsciously respond in a certain inaccurate way Ex. An internal survey about workplace dishonesty had a high response rate and found that 95% of workers at Amazon have never called in sick when they actually felt fine Ex. Students who completed the phone version of an alcohol consumption survey reported consuming an average of 2 drinks on a typical night out. Those who completed the online version reported consuming an average of 6 drinks on a typical night out.

Nonresponse bias

participants who don't respond are systematically different from those who did Ex. An American Express survey on credit card bill payment habits among adults in the US had a response rate of 4%. The majority of respondents were people who earn $100,000 or more.

Measurement instrument bias

problems with survey design and questions themselves Ex. Based on the question "Do you approve of having your hard-earned money spent to build a stadium for a team that makes millions of dollars each year?" a survey found that 75% of LA residents oppose using tax revenue to build a new football stadium

IDIs - Projective Techniques

some motivations, beliefs, and attitudes are hard to assess because: - People are unaware of them (exist at subconscious level) - People are aware but unwilling to share (too personal or socially undesirable) - People are aware but think it's irrelevant projective techniques are one way to get around this - origins in clinical psychology (e.g., Rorschach test) - unstructured, indirect form of questioning - participants indirectly reveal their own underlying motivations, beliefs, attitudes, or feelings

Concurrent validity

Can the measure predict current outcomes that it logically should influence?

Causal Research Methods

Causal research gathers evidence on cause-and-effect relationships through... - Marketing experiments: Controlled environments isolating cause and effect - Survey methods: Far less control and *not as good at isolating cause and effect* (ex. guy with beard or no beard -- effect on sale of tshirt)

Third-Person Techniques

Asks respondents to report the beliefs and attitudes of a third person rather than directly expressing their own personal beliefs and attitudes - "What does the typical student think about..." - "How do most people feel about..." - "Why do your peers buy..." Operates by increasing the "psychological distance" between the respondent and their beliefs

Types of Reliability (Precision)

Is our measurement process free from random errors? Is our measurement consistent across situations, respondents, and time?

Respondent Ambivalence and Ignorance

Ambivalence:If many respondents are likely to have mixed feelings, provide a neutral response Ignorance:If many respondents are likely to be ignorant(lacking knowledge) of the topic, provide a "don't know" response CAUTION: neutral and don't-know categories also attract respondents who are not taking the survey seriously (lack of interest, time, ability).

Backward Market Research

Begin with the end in mind! Once the problem has been defined (step 1), ask: 1. How will the results be used or implemented? 2.What should the final results look like? 3.What analyses are necessary to deliver the results? 4.What data is necessary to run those analyses? 5.Conduct the research and deliver the results. (Proceed with steps 2 - 6)

Predictive validity

Can the measure forecast future outcomes that it logically should influence?

Goals

- Conducted after exploratory research to test specific hypotheses - Obtain definitive answers to research questions

Goals

- Conducted at initial stage of research to clarify research problem - Obtain insights about the problem, environment, and consumer rather than answer specific questions - Develop specific hypotheses to test

Measure

- Develop a measurement scale: What are the specific questions or observations? - What is the form of responses and how are they "scaled"? 4 Levels of Measurement: - Nominal / Categorical Measures - Ordinal Measures - Interval Measures - Rational Measures 4 properties of numbers: 1. unique values (1 doesn't not equal to 2) 2. rank order exists (ex. 1 < 2 < 3) 3. intervals (distances) between adjacent numbers are equal (ex. 2-1 = 3-2) 4. ratios are equal when there is a meaningful absolute zero point (ex. 10/5 = 2 (10 is twice as large as 5)

Evaluate

- Evaluate sources of measurement error - Assess the reliability and validity of the measure

How can we use secondary data?

- How relevant is the original question? - How relevant is the original company? - Can this data answer our research question? --- Or do we need more secondary data to corroborate these findings? --- Or should we use it to guide or primary research?

When NOT to conduct research

- Insufficient resources to do proper research - Results would not be meaningful to managers - Opportunity has passed - Decision has already been made (or will not be made) - Managers disagree about the research problem (i.e., what they need to know to make decision) - Needed information already exists - Research costs > benefits

Interval (or Near-Interval)

- Intervals between adjacent values are equal - Attitudes; opinions; satisfaction - %; mode; median; mean; standard deviation Ex. I go out of my to purchase this brand over other brands (ex. 1 (strongly disagree) - 7 (strongly agree)

Advantages

- Less potential for researcher bias (interpreting) - Less effort for respondents - Easier data processing - Easier to compare across respondents

Disadvantages

- More difficult to design (must come up with responses) - Can't capture unexpected responses and explanations - Restrict respondents' frame of reference

Methods

- Observation: ethnography, mystery shopper - Focus groups, in-depth interviews, case studies - Secondary data analysis

Survey Question Sequencing

- Open with easy and non-threatening questions. - Questionnaire should flow smoothly & logically from one topic to next. - Group similar questions together. Don't jump around excessively. - For any given topic in your survey, it's generally better to go from broad (general) questions to specific questions. - Demographics are generally collected last. Why? *Generally better to go from broad (general) questions to specific questions.* Ex. Product category to Brand

Disadvantages

- Potential for researcher bias - Coding responses can be time-consuming and costly - More effort for respondents

Advantages

- Puts respondents in mindset of specific topic - Useful when there are too many possible responses to list - Useful when there are unforeseen possible responses - Can provide insights and explanations (as follow-up) - Useful for exploratory research purposes

Ordinal

- Values can be ordered by magnitude - preference; attribute importance - %; mode Ex. how frequently do you purchase this brand (ex. 1=never, 2= few times a year, 3= few times a month, 4= few times a week, 5=daily) Ex. The percentage of undergraduates on the East Coast who use Tinder 10+ times/week is three times as large as the percentage of undergrads on the West Coast who use Tinder 10+ times/week."

Ratio

- Values have a meaningful absolute zero point - Sales, weight, age - Almost all kinds of statistics Ex. During the past month, on how many days did you use Spotify? For how many years have you been a customer of the brand? (open-ended)

Nominal

- Values represent discrete categories - yes/no questions; classification (e.g., gender) - %; mode Ex. which brand do you purchase most often (select from list)

Types of Validity (Accuracy)

Are we really measuring what we think we're measuring? How close are we to the true score? Ex. Imagine that you are trying to measure word-of-mouth intentions for the restaurant Alden and Harlow. - Evaluate the content validity of this measure - How would you test the predictive validity of this measure? What about the convergent and discriminant validity?

Operationalize

- What can you ask or observe to measure the construct? - Concrete; based on observable or measurable world (ex. purchase occasion of lyft, uber, t) Many possible ways to operationalize "brand loyalty" using *behavioral data*, including (but not limited to): - Frequency of purchases of each brand/most frequently purchased brand - Number of consecutive purchases of brand - Criterion such as "three or more consecutive purchases = loyal" and anything else is non-loyal - Last brand purchased - Total number of different brands purchased (lower# = higher loyalty) - Number of times brand switching occurred (lower# = higher loyalty) You can also measure brand loyalty using self-reported attitudinal data - - Thoughts, opinions, emotions, evaluations - ex. agreement or disagreement with each of the following statements Single item vs. multiple items? - Some constructs are straightforward and require only one item (e.g., age; political ideology; job satisfaction) - Others are more complex and require various items to capture multiple interpretations (e.g., brand loyalty; materialism; narcissism) Operationalization depends on... - Existing scales - Available data - Industry standards - Company standards

Conceptualize

- What concept do you want to measure? - Clearly define the concept in abstract, theoretical terms (ex. defining brand loyalty)

Management Decision Problem

- action-oriented - focuses on symptoms - what should the decision maker do?

What triggers the need for research?

- evaluating alternatives (what message to promote, when should we run a sales promotion, brands to collaborate with) - opportunities (ex. emerging social trends or economic trends, new technology emerges, gov't regulations relaxed or lifted) - problems/threats (ex. new competitor enters market, growing pressure from existing competitors, new tech threatens relevance of product) - market performance symptoms (ex. gap between expectations and reality) - market share is slipping - repurchase rates are falling - store sales declining

Marketing Research Problem

- information-oriented - focuses on root causes - what does decision maker need to know? - how should that information be obtained? symptom: what is happening? (observable and apparent) disease/cause: why is this happening? (not immediately observable or apparent) ex. identify potential causes of brick-and-mortar retailers' symptom (widespread store closings) -- cause from societal/consumer trends, competitors and store themselves

2. Formulate Research Design

- no single ideal research design (each has pros and cons) - generally do a combination of exploratory and conclusive research

The Marketing Research Process

1. Define the Problem or Opportunity 2. Formulate Research Design 3. Select Research Method for Data Collection 4. Select Sample & Collect Data 5. Analyze and Interpret Data 6. Preparing and Presenting the Results

Methods of Administering a Survey

1. Telephone --> traditional telephone, computer-assisted telephone interviewing (CATI) 2. Personal --> in-home, mall intercept, CAP(ersonal)I) 3. Mail --> Mail/Fax Interview, Mail Panel 4. Electronic --> E-mail, Internet Mode of Survey Effects (phone vs. internet (online))

Basic Survey Design Procedure

1.Set your objectives: what do you want to learn? - What constructs do you need to measure? 2.What kind of analysis do you want to do? - Backwards Market Research 3.What format and delivery method? - E.g., online, on paper via postal mail, etc. - What will influence this decision? 4.Construct your survey - Introduction - Questions - Thanks/Debrief/Payment 5.Pre-test it 6.Launch!

MEASUREMENT (yes clear research purpose, design qualitative research, not trying to understand cause and effect, conduct suitable DESCRIPTIVE RESEARCH study -- conclusive research that does NOT establish causal relationships (ex. survey or observation))

A manager may be interested in describing... - Consumer characteristics (What percentage of customers are 18-24?) - Purchase/consumption behaviors (How much does average consumer spend per visit?) - Market characteristics (How much did the market grow last year?) - Marketing mix characteristics (Do customers purchase more in store or online? (place)) - Associations among the above (What's the relationship between age and average expenditure?) Describing requires measurement

Response Scale Bias: Category Numbers

A person's response to a question can be influenced by the numbers assigned to each response. Consumers were asked:"How successful have you been in life, so far?" Responses varied depending on the response scale numbers: - Scale from 0 - 10 - Scale from -5 to 5

Response Scale Bias: Category Labels

A person's response to a question can be influenced by the response scale category labels. Consumers were asked:"How much time do you spend watching TV daily?" Responses varied depending on the response scale labels: low-frequency labels vs. high frequency labels

Question Order Bias

A person's response to one question can be influenced by a preceding question. Consumers were asked: - "How interested are you in purchasing the new Flair combination pen and pencil?" Responses varied depending on the question immediately preceding: 1. No question asked 2.8 2. Asked only about advantages 16.7 3. Asked only about disadvantages 0.0 4. Asked about both advantages & disadvantages 5.7

Focus Groups - Cons

Demand effects Groupthink Socially desirable responding Not necessarily representative Risk of moderator bias (i.e., not neutral) Noisy data - can be hard to distill insights Behavioral intentions (vs. actual behaviors) Can't reveal underlying motivations

IDIs - Cons

Demand effects(especially for direct questions) Projective techniques are weird - People aren't accustomed to such questions - Could yield unreliable responses that don't generalize Highly subjective; runs risk of confirmation bias Small sample sizes aren't necessarily generalizable

Methods

Descriptive research: primarily surveys; observation - Asks who, what, when, where, how many, how often? Causal research: primarily experiments; surveys - Asks what is the effect of x on y?

4. Select Sample & Collect Data

Determine the population - who could (and couldn't) provide the necessary info? - where will you find these people? - population = all of the people who could provide info Select the Sample - sample=subset of population - probability: chosen at random; representative of pop. - convenience: not chosen at random; nonrepresentative

Content validity

Do the items in your measure represent the entirety of the focal concept?

Split-half reliability

Do we get the same values when we divide our measure into two groups and compare values for the same people? - Positive correlation between two halves of the multi-item measure Randomly split your measure in half. - Create an index (i.e. combine items) for each half - Compute correlation between the indexes. - Correlation should be positive & "large"

Focus Groups - Pros

Leverages group dynamics - get more information than you could individually Social setting mirrors reality of buyer decision context (high face validity) Quicker than IDIs and surveys Direct researcher interaction (vs. surveys) - ensures that critical questions are asked, understood, and answered

Measurement

Measurement: assigning numbers (or categories) to abstract or concrete concepts of interest Quantification facilitates analysis and comparison Concepts commonly measured in marketing: - Attitudes - Brand/product attributes - Purchase/usage intentions - Brand attachment/identification - Loyalty - Satisfaction - Behaviors Many marketing variables are "latent" (i.e., unobservable) which makes measurement difficult - Examples: attitudes, beliefs, purchase intentions There is more than one way to define most marketing variables (e.g., loyalty) Thus, there is more than one way to measure most marketing variables

IDIs - Pros

Minimize group pressure and groupthink Heightened state of awareness—respondent is center of attention Flexible location; can conduct remotely Often cheaper than focus groups Ideal for understanding individual decision processes Can get insight about personal topics Can investigate underlying motivations and subconscious factors

What can focus groups accomplish?

Product design - get feedback on current and future products - stimulate new ideas about older products - generate completely new ideas Brand management - Understand emotional response to brand - Identify core brand associations; guide positioning MR feedback loop - Follow-up on conclusive research results (iterative process)

Designing focus groups

Participants - 5 - 10 people per group (multiple groups) - Knowledgeable about focal topic - Avoid "professional" research participants* - Homogeneity within groups; heterogeneity across groups Environment - Quiet and comfortable - Round table (ideally) - Record; one-way mirror Duration: 1-2 hours Moderator - Meant to facilitate conversation; "invisible leader" - Must be adaptable/able to think on their feet - Must seize opportunities to follow up for more information - Must remain neutral - Must have strong social skills - Must build rapport among participants - Must encourage equal involvement Discussion Guide (see exhibit 5.5 on p. 100) - Detailed outline of issues to discuss - Introduction: Set relaxed and casual tone; inform of recording/observers - Start with ice breakers to get people comfortable - Question order: broad (e.g., category level) to specific (e.g., brand level) - Avoid leading questions: "What problems do you have with your boss?" vs. "Tell me about your relationship with your boss." - Avoid overly personal questions - Ask open-ended questions; avoid yes/no questions --> follow up with "why?" if you must ask yes/no questions Goal is to capture breadth of opinions - "Who else has something to add?" - "I see people nodding/shaking their heads—what are you thinking?" - "Does anyone not feel that way?" Stop when you reach "theoretical saturation" (i.e., no more novel insights) - Same for determining number of focus groups

Projective Technique:Personification & Stereotyping

Personification: describe brand/product as though it were a person (Brand --> Person) - If Smirnoff Peach Vodka were a person, what would it be like? Age, occupation, appearance, values, lifestyle, goals, hobbies? Follow up with "why?" Stereotyping: read description of a person and identify brand/product they would use (Person --> Brand) - John is a 26 year-old patent lawyer who lives in San Francisco and spends most of his free time hiking. Environmental consciousness is one of his most important values. What beverages would he have in his apartment? What makes you think that?

Projective Technique: Word Association

Presented with a list of words (ideally read aloud) - Brands, people, concepts, events "What's the first word that comes to mind?" Reveals underlying attitudes and perceptions Example: brand associations - Volvo - Chevy - BMW

Example Exercise: Listerine Social Media Data

Quantitative Analysis - Amount of discussion about Listerine vs. other topics - Consumer sentiment about Listerine vs. other topics - Evaluation of attributes of Listerine vs. other brands Qualitative Analysis - Overall reasons consumers like or dislike Listerine - Different purposes consumers use Listerine for

1. Define the Problem (or Opportunity)

Translate management decision problem into a marketing research problem.

5. Analyze and Interpret Data

Translate raw data into actionable insights using statistical analysis - Descriptive statistics - Crosstabs - Comparison of means - Comparison of proportions - Regression - Factor Analysis - Multidimensional Scaling - Cluster Analysis Many data analysis and visualization software packages of varying sophistication and complexity (ex. excel, tableau, etc)