MKT 450 Exam 2 Garver 2.0
What are best practices to being a great interviewer via interviews or focus groups?
1)Be objective driven 2)Ask about past experiences (open-ended) 3)Ask a lot of questions.
What is the basic format of a maximum difference scaling survey?
1)Introduction 2)Priming and Classification 3)MaxDiff Scaling Questions 4) Demographic Types of Questions 5) Thanks
Research objectives of Customer Satisfaction Research
1. Attribute Importance 2. Satisfaction Strengths and Weaknesses 3. Improvement Opportunities
Relative Data
"Best Competitor"
Non-Relative Data
"No competitor data"
Non - relative Improvement Index
(importance) x (# scale points -satisfaction score) For each attribute
Is CX research focused on customer acquisition or customer retention?
*Customer Retention* Customer acquisition is expensive and customer retention is a profitable strategy
How and why do we measure different situations in maximum difference scaling?
*Priming*: Give a scenario for different choices, things are different Ex: busy, hungry, time urgency
How do we interpret the results for preferences of different levels?
0= Avg Preference | Negative #'s= Below Avg | Positive #'s=Above Avg Preference
What are 3 commonly used scales to measure CX attributes? Why does Garver love the table to present these attributes?
1 (poor)- 10 (excellent) 1 (very dissatisfied)- 10 (very satisfied) 1 (angry) - 10 (delighted) *It is the fastest and easiest, it allows them to read less, emotional, and keeps everything neat and organized*
What is the process for analyzing qualitative data?
1) *Analyze* each interview/focus group/complaint/review *individually* 2) Look across interview/focus group/complaint/review for *common themes*
What are the assumptions of a market simulation? (6)
1) All attributes that affect buyer choices in the real world have been accounted for 2) Each product has equal availability (distribution) 3) Respondents are aware of all of the products 4) Products reach long range equilibriums 5) Equal effectiveness of sales force and advertising 6) No out-of-stock conditions
What are the three probing questions?
1) Can you tell me more about that? 2) What do you mean by that? 3) Can you give me an example of that?
What are 3 different types of CX surveys?
1) Competitive Benchmark 2) Relationship 3) Transaction/Critical Interaction
What are existing public sources of qualitative data? What existing types of qualitative data do companies have access to?
1) Forums 2) Reviews (YouTube, Facebook, and Twitter) Existing Data: 1) Negative (Complain, Review, Surveys) 2) Positive (Compliments, Surveys, Reviews)
What are best types of questions for asking qualitative questions? (2)
1) Open ended 2) Indirect Questions -Ask about past experiences -Let them tell their story in their own way
What are common applications of maximum difference scaling? (4 + Other)
1) Replace most "interval scaled" questions 2) When you need great data 3) Need excellent variation 4) When you need need based segmentation Other: Brand Equity/Positioning/Statements, Brand-Price, Advertising Slogans, Pref. of new features or service
What are best practices for formulating CX attributes? (4)
1) What are the key drivers of a positive CX? 2) Customer & Manager Perspective 3) Words Just Right 4) Did you miss one?
What are the objectives of CX Research? (3)
1) What is most/least important to our customers? 2) How are we performing? 3) What should we fix/improve?
What does it mean when I say, "Can they talk about it?" (4)
1) Will they be prone to social desirability? (lie) 2) Sensitive in nature? 3) Is the sample prone to social desirability? 4) Do they have expertience and expertise, Social Desirability bias, Latent, sub conscious behavior, Memory
Conjoint Analysis:
Attribute Importance - Choice Desired Levels of Performance - Choice
What is a context question? Give Example
Background. Ex: Tell me about life as a ______
How do we *not* ask probing questions?
Biased probing questions Leading the question/witness ex. You must feel ______
Are simulations built on averages or choices? Are they built on attribute importance or preference levels?
Built on individual choices and preference scores
Relative Performance ratios for attributes
Calculate mean attributes scores for "Us" and best other vendor Our mean score divided by best other vendor mean score Always use a best other vendor if you have one Use a table or graph to display performance ratio values
Price Range
Calculate means and medians for all VW questions Line graph means and medians for all VW questions (Is there a difference? Variance?) Sweet spot is between good deal & expensive, but still consider other questions.
Under what situations would you recommend observations to be used instead of focus groups or in-depth interviews?
Can/Will they talk about it? Are they observable? How: Recruit and Ask Permission Ex: coaching an infielder
What is maximum difference scaling related to and what software do we use to create this type of survey?
Conjoint and Sawtooth Software
Be able to graph conjoint analysis data in JASPer
Consistent Scales
What two pieces of information does qualitative research deliver that cannot be delivered via quantitative research? i.e Think about if you wanted to know how a dish is prepared
Context and Process
Customer Retention - Importance:
Correlate independent variable vs dependent variable Correlate attributes vs. Overall Satisfaction or NPS Use a table or graph to display correlation values
Importance =
Correlation Analysis
Importance Scores=
Correlation Coefficient /NPS
MaxDiff: Why cant we just use stated importance ratings?
Absolutely no variation = We know NOTHING People only give very important
Compare your NPS score to:
All competitors, relevant benchmarks, and to Best In Class
why can market simulations with conjoint analysis settle boardroom debates?
Allows us to test out boardroom discussions , it decides which new idea = the best outcomes
What is open coding?
Looking for the *theme* and/or *concept and ideas* ex: ease of use, "great product"
What other names are used for CX surveys? What does CX stand for?
Customer satisfaction/value/loyalty Stands for Customer Experience- Feedback from customers (How are we doing?)
From the customer perspective, what does a maximum difference scaling survey look like?
Customers chose which attributes are most or least important
Performance Ratio of 1
Directly Tied
What is the Right Mindset for Qualitative Interviewing?
Find what is there, *not* what you're looking for Play dumb like a fox Go in energetic and fired up Body language is important
1) Breadth 2) Violates context 3) Group 4) More Synergy 5) More moderate skill 6) More difficult to set up
Focus Groups
How do we interpret the results from maximum difference scaling?
High attributes and their differences Hierarchical Bayes, *results in importance scores for each attribute* for each respondent all attributes scores sum to 100
Can you describe the different levels of abstraction?
Higher Order: Easier to talk to executives, you've got to work hard Concrete Example: You have to put in 2 hours a day to become better
What are important outcomes used in a market simulation?
Highs and Lows: Market Share, Profit/Loss, Revenue, Cost
How do you write the Net Promoter Score (NPS) question?
How likely is it that you would recommend our company/product/service to a friend or colleague? 0 (Not Likely )- 10 (Extremely Likely)
Under what situations would you recommend *focus groups* to be used instead of in-depth interviews or observations?
If it is a group activity and a wide range of topics ex: What bar to go to
What information does conjoint analysis deliver?
Importance of Attributes (means) and Preferences of levels (means)
Relative Improvement Index
Importance x (Inverted Performance Ratio) For each attribute
1) Depth 2) Preserves context 3) Solo 4) Less Bias 5) Less moderate skill 6) Easier to setup
In-Depth Interviews
What are the limitations of MaxDiff?
Time and Effort
Significance score > 0.5
Not significant
What are keys to making qualitative research presentations?
Objective and Data Driven, Make it Stick, Use Verbatim Quotes, Evidence, Hook the Audience *Brief and Deep Understanding of D2*
Best Case Scenario
Use competitive data
Improvement Index
Use to quantitatively prioritize improvement opportunities,use results from prior importance and performance analysis Make sure you use the right formula: Is it relative or NON-relative performance data? Always use a best other vendor if you have one If you are using the relative improvement index, make sure you use the "inverted performance ratio" (them/us) in the formula
What do I mean by what-if analysis?
What if we did this? What would happen?
A 5% increase in customer retention leads to
a 25%-85% increase in profits
What is a good analogy for presenting qualitative results? What should a great presentation look and feel like?
a) Camp stories b) 10 mins, delivers info in an exciting way
What are the characteristics of the Competitive Benchmark Surveys?(5)
a. Marketplace perspective (not only our customers) b. High-level, strategic attributes c. Compare to many competitors d. Once a year e. Analysis: Relative or Comparative
How do you get qualitative research samples? What are best practices for qualitative samples? What are best practices for recruiting qualitative samples? (4) How many do we need, and when do stop?
a. Not random sample, we *hand select* b. Search for the characteristics (talkative, picky,etc) *Representative* c. Let them know that this is *important*, you're *special*, it's *top secret*, Will you *help me*? d. In-Depth Interviews : 10-20 Focus Groups: 4-6 *patterns of redundancy*
What are the characteristics of relationship surveys? (4)
a. Our customers b. *Strategic* in nature c. Quarterly, Bi-Annual, Annual (3 Months to Once a year) d. If possible, *compare to a best competitor* (Relative Road)
What are the characteristics of transaction/ critical interaction surveys?
a. Our customers b. *Tactical* in Nature c. Right after interaction (Continuous) d. NO comparison to a best competitor (Non-Relative Road)
What new technologies exist that help us collect qualitative data? Do the fundamentals of qualitative research change with these new tools?
a. Skype, FaceTime, Cameras (security) b. They do not change the fundamentals, just new tools
How do we detect cause and effect relationships in qualitative data?
customer experience model; Is their context driving certain thoughts and behaviors
What are the three biggest advantages of using maximum difference scaling?
excellent variation improved predictive capabilities allows for need-based segments
Under what situations would you recommend in-depth interviews to be used instead of focus groups
Politics, Drinking, Money, Buying a House, Picking out a wedding ring
What are typical applications of a market simulation? (Things to test-4)
Price Sensitivity, New Feature, Competitive Moves, Brand New Product
Performance Ratio-
Probably a table
Customer Retention - Performance
NPS Calculation NPS = Valid % of Promoters minus Valid % of Detractors Compare to outside benchmarks, easily found by "googling" Use a table or graph to display NPS values
T/F: Conjoint simulator assumptions usually don't hold true in the real world
TRUE: Conjoint simulator assumptions usually don't hold true in the real world
What type of person do we want to recruit for a qualitative sample?
Talkative,Picky, Smart, Leading Edge, Complainer
What new technology tools are used to analyze qualitative data? Is this technology fool proof?
Text Analytics/Mining and it's getting better every year
The higher the improvement index score:
The higher the priority
Performance Ratio <.97
Significant Disadvantage
What is a grand tour question? Give Example
Learn about a process. Think about the last time you ______
Mathematically, how do market simulations work?
"all customers" have a preference score for each level, and then we look at the chart to see" Would Mike choose and amber beer for $13 or a light beer for $7? Then... we look at Mikes overall preference score for a light beer, $7 pitcher? Mikes preference: (light beer=1.2 + $7pitcher= 2) = 3.2) Mike overall preference score for an amber beer? Mikes preference: (amber beer=5.5 + $13pitcher=1) =6.5) *preferred*
How do we calculate NPS?
(% promoters -% of detractors)
Top Box (9s&10s) buy:
2 1/2 times more
Research Objectives of Conjoint Analysis
-Used to measure preferences for product features, to learn how changes to price affect demand for products or services, and to forecast the likely acceptance of a product if brought to market.
7 common mistakes of Bad Graphs
1. Data Overload 2. Font Size - Don't go below size 24 3. Bad Colors 4. No/Bad Data Labels 5. No Sort Ascending or Descending 6. Inconsistent Scales 7. Excel Defaults are dangerous
Choice and Post Purchase strategy
1. How do we get them to buy our product? 2. How do we get them to keep buying our product?
2 methods that use attribute importance and satisfaction results:
1. Improvement Index 2. Quadrant Analysis
How many maximum difference scaling questions are usually asked?
10
Correlate attributes with:
Dependent Variable (NPS, Overall Satisfaction, etc)
Performance Ratio > 1.03
Significant Advantage
T/F:Shares of preference always match real world market shares
FALSE: Shares of preference *don't always* match real world market shares
When analyzing data, how are verbatim quotes used? What purpose do verbatim quotes serve?
It allows you to see what the customers are saying by presenting you the *evidence* for their claims and it *hooks the audience* which *adds to presentation quality* and triggers an *emotional reaction*
Non Relative Performance ratios for attributes
Just calculate our mean scores for attributes Use a table or graph to display performance ratio values
Why is CX important? What is the relationship to financial performance?
Revenue It is *5 to 10* times more expensive to *acquire* new customer than to *retain* customers
Price Range - Waterfall Graph
Run frequencies for "Good Deal" question Use a "Waterfall" line graph to display the values. Where are the steep drop-offs? Where do we lose a large % of customers believing it is a good deal?
Attribute Importance - Customer Choice
Run means for conjoint attributes Just calculate our mean scores for attributes Attributes means will add up to 100 Use a table or graph to display performance ratio values
Preference for Levels - Customer Choice
Run means for levels within each attribute Just calculate our mean scores for levels Zero based numbers, means will add up to 0 Make sure that ALL level graphs have consistent scales points Use a table or graph to display performance ratio values
*QUALITATIVE*: *D^2* Sample:_______ ?'s:______ Data:_______ Analysis:_______ Focus:________
S: Few- Hand Picked ?: Open-Ended D: Meaning -Words A: Interpretive F: Why?
*QUANTITATIVE*: *Generalize* Sample:_______ ?'s:______ Data:_______ Analysis:_______ Focus:________
S: Random-many ?: Close-Ended D: Numeric A: Statistics F: What?
What is a more advanced analysis for conjoint analysis?
Segmentation and Market Simulation
How do we interpret the results for attribute importance?
Share 100pts, The more points the more important and vice versa.
Significance score < 0.5
Significant
Inverted Performance Ratio
Them / Us
Are detecting cause and effect relationships a strength of qualitative research? What is the only method that can "test" for cause and effect relationships?
Yes, Helps explore context Experiments
Performance Ratio 1.03-97
Zone of Indifference
