MKT 450 Garver Exam 2
What is the basic format of a maximum difference survey? (5)
1)Introduction 2)Priming and Classification 3)MaxDiff Scaling Questions 4) Demographic Types of Questions 5) Thanks
What are the 3 objectives of CX Research
1. Attribute Importance 2. Strengths and Weaknesses 3. Improvement Opportunities
What is CX research focused on?
*Customer Retention* Customer acquisition is expensive and customer retention is a profitable strategy
T/F: We don't need conjoint survey data to do market simulations
*FALSE* We *do* need conjoint survey analysis to do market simulations
How and why do we measure different situations in maximum difference scaling?
*Priming*: Give a scenario for different choices, things are different Ex: busy, hungry, time urgency
T/F: NPS is *NOT* a *mean* or *average*
*TRUE* NPS is *NOT* a *mean* or *average*
T/F: To develop a marketing strategy you have to know what is important to customers (driving their behavior)
*True*
What are 3 commonly used scales to measure CX attributes? Why does Garver love the table to present these attributes?
1 (poor)- 10 (excellent) 1 (very dissatisfied)- 10 (very satisfied) 1 (angry) - 10 (delighted) *It is the fastest and easiest, it allows them to read less, emotional, and keeps everything neat and organized*
What is the two step process for analyzing qualitative data?
1) *Analyze* each interview/focus group/complaint/review *individually* 2) Look across interview/focus group/complaint/review for *common themes*
What is the Right Mindset for Qualitative Interviewing? (4)
1) *Find what is there, not what you're looking for* - Ask open-ended questions, then ask the probing questions 2) Plan B: Play dumb like a fox; ask lots and lots of questions Garv Example: Dog Training- he listened to ask questions 3) Go in energetic and fired up 4) Body language 5) Be Interested
What are the assumptions of a market simulation? (6)
1) All attributes that affect buyer choices in the real world have been accounted for 2) Each product has equal availability (distribution) 3) Respondents are aware of all of the products 4) Products reach long range equilibriums 5) Equal effectiveness of sales force and advertising 6) No out-of-stock conditions
What are the three probing questions?
1) Can you tell me more about that? 2) What do you mean by that? 3) Can you give me an example of that?
What are 3 different types of CX surveys?
1) Competitive Benchmark 2) Relationship 3) Transaction/Critical Interaction
Cause and Effect Relationships: What two questions do we ask ourselves?
1) Do you see C&E in their quotes? 2) Is their context (background) driving certain thoughts and behaviors?
What are existing public sources of qualitative data? What existing types of qualitative data do companies have access to?
1) Forums 2) Reviews (YouTube, Facebook, and Twitter) Existing Data for Companies: 1) Negative (Complain, Review, Surveys) 2) Positive (Compliments, Surveys, Reviews)
What is the basic format of a CX Survey?
1) Intro 2) Priming and Classifying 3) Dependent variable - *NPS* 4) Independent variable - *Attr. Importance* 5) Do we need to improve? 6) Improvement Opportunities (rank order/ open ended comments) 7) Demographic Questions 8) Thanks
What are best types of questions for asking qualitative questions? (2)
1) Open ended 2) Indirect Questions -Ask about past experiences -Let them tell their story in their own way
What are the other applications of maximum difference scaling? (4 + Other) hint: When you..
1) Replace "interval scaled" questions 2) When you need great data 3) Need excellent variation 4) When you need, need based segmentation Other: Brand Equity/Positioning/Statements, Brand-Price, Advertising Slogans, Pref. of new features or service
What are best practices for formulating CX attributes? (4)
1) What are the key drivers of a positive CX? 2) Customer & Manager Perspective 3) Words Just Right 4) Did you miss one?
What are the objectives of CX Research? (3)
1) What is most/least important to our customers? 2) How are we performing? 3) What should we fix/improve?
What does it mean when I say, "Can they talk about it?" (4)
1) Will they be prone to social desirability? (lie) 2) Sensitive in nature? 3) Is the sample prone to social desirability? 4) Do they have expertience and expertise, Social Desirability bias, Latent, sub conscious behavior, Memory
What are best practices to being a great interviewer via interviews or focus groups?
Ask probing questions, and LISTEN
What is a context question? Give Example
Background. Ex: Tell me about life as a ______
How do we *not* ask probing questions?
Biased probing questions Leading the question/witness ex. You must feel ______
Are simulations built on averages or choices? Are they built on attribute importance or preference levels?
Built on individual choices and preference scores
What is maximum difference scaling related to and what software do we use to create this type of survey?
Conjoint and Sawtooth Software
What two pieces of information does qualitative research deliver that cannot be delivered via quantitative research? i.e Think about if you wanted to know how a dish is prepared
Context and Process
*Equation* Importance Scores=
Correlation Coefficient /NPS
MaxDiff: Why cant we just use stated importance ratings? hint:John Snow, GOT
Absolutely no variation = We know NOTHING
Compare your NPS score to:
All competitors, relevant benchmarks, and to Best In Class
How do we interpret the results for attribute importance?
All sum up to 100 pts, The more points the more important and vice versa.
why can market simulations with conjoint analysis settle boardroom debates?
Allows us to test out boardroom discussions , it decides which new idea = the best outcomes
What is open coding?
Looking for the *theme* and/or *concept, topics and ideas* ex: ease of use, "great product"
What are the 3 other names used for CX surveys? What does CX stand for?
Customer satisfaction/value/loyalty Stands for Customer Experience- Feedback from customers (How are we doing?)
From the customer perspective, what does a maximum difference scaling survey look like?
Customers chose which attributes are most or least important
In CX Performance, a Performance Ratio of *1* means that you are what compared to your best competitor
Directly Tied
What are the CX Attribute Importance steps for *performance ratio attributes* in JASP and how do we graph our findings?
FIRST: Select the *mean attributes (independent variables)* ex. Ktaste, Kfreshness) scores for "Us" SECOND: Do the same process for our best competitor (Starbucks) and paste those results in Excel as well, a seperate column for each. THIRD: Calculate our *performance ratio* (Us/Them) for each attribute, and form a third column for those LAST: Use a table or graph to display performance ratio values
1) Breadth 2) Violates context 3) Group 4) More Synergy 5) More moderate skill 6) More difficult to set up
Focus Groups
How do we find outside benchmarks of NPS performance?
Can you describe the level of abstraction?
Higher Order: Easier to talk to executives Ex: If you want to be great, you've got to work hard Concrete Example: You have to put in 30 days in a row for at least a half hour a day to become better
What are important outcomes used in a market simulation?
Highs and Lows: Market Share, Profit/Loss, Revenue, Cost
How do you write the Net Promoter Score (NPS) question?
How likely is it that you would recommend our company/product/service to a friend or colleague? 0 (Not Likely )- 10 (Extremely Likely)
Under what situations would you recommend *focus groups* to be used instead of in-depth interviews or observations?
If it is a group activity and a wide range of topics ex: What bar to go to
Under what situations would you recommend observations to be used instead of focus groups or in-depth interviews?
If they can't talk about it and they need to be observed. The picture is worth more than the words.
What information does conjoint analysis deliver? (2)
Importance of attributes and preferences of levels in *means*
1) Depth 2) Preserves context 3) Solo 4) Less Bias 5) Less moderate skill 6) Easier to setup
In-Depth Interviews
What are the limitations of MaxDiff?
Time and Effort
(P) Significance score > 0.5
Not significant
What are keys to making qualitative research presentations (Good Campfire Story)? (6)
Objective and Data Driven, Presentation Zen, Made to Stick, Use Verbatim Quotes: Evidence & Hook the Audience, Bring in quality audio or video *Brief and Deep Understanding(D2)*
Performance Ratio calculation
Us /Them
What are performance ratios and how do we calculate them? How do we write the formulas in Excel? Do we use means or valid % for NPS?
Us divided by Them Us divided by Them Means
Improvement Index
Use to quantitatively prioritize improvement opportunities,use results from prior importance and performance analysis Make sure you use the right formula: Is it relative or NON-relative performance data? Always use a best other vendor if you have one If you are using the relative improvement index, make sure you use the "inverted performance ratio" (them/us) in the formula
What is MaxDiff (Best Worst) scaling and what is it related to?
Used for measuring importance, agreement, etc -customers choose most to least important attributes Conjoint
Research Objectives of Conjoint Analysis
Used to measure *preferences* for product features, to learn how *changes to price affect demand* for products or services, and to *forecast* the likely acceptance of a product if brought to market.
What do I mean by what-if analysis?
What if we did this, what would happen? Which management ideas are good, better, best?
A 5% increase in customer retention leads to
a 25%-85% increase in profits
What is a good analogy for presenting qualitative results? What should a great presentation look and feel like?
a) Camp stories b) 10 mins, delivers info in an exciting way
How do we run NPS scores in JASPer? How do we write the formulas in Excel? Do we use means or valid % for NPS?
a) Display frequency tables b) sum=(valid % of 9/10's & 7/8's & 6/Below) *seperately* NPS (promoters-detractors) c) Valid %
What are the characteristics of the *Competitive Benchmark* Surveys? (sample, strategic/tactical, frequency, best competitor?, and analysis)
a. Marketplace perspective (not only our customers) b. High-level Strategic - Big drivers of CX c. Once a year d. Compare to *many* competitors e. Analysis: Relative or Comparative to a best competitor
How do you get qualitative research samples? What are best practices for qualitative samples? What are best practices for recruiting qualitative samples? (4) How many do we need, and when do stop?
a. Not random sample, we hand select b. Search for the characteristics (talkative, picky,etc) *Representative* c. Let them know that this is *important*, you're *special*, it's *top secret*, Will you help me? d. In-Depth Interviews : 10-20 Focus Groups: 4-6 look for *patterns of redundancy*
What are the characteristics of *relationship* surveys? (sample, strategic/tactical, frequency, best competitor?)
a. Our customers b. *Strategic* c. 3 months to Once a year d. If possible, *compare to a "best" competitor* (Relative Road)
What are the characteristics of *transaction/ critical interaction* surveys? (sample, strategic/tactical, frequency, best competitor?)
a. Our customers b. *Tactical* c. Continuous (Right after interaction) d. *NO comparison* to a best competitor (Non-Relative Road)
NPS People: _______= 9& 10 _______= 8& 7 _______= 6& Below
a. Promoters b. Passives c. Detractors
What new technologies exist that help us collect qualitative data? Do the fundamentals of qualitative research change with these new tools?
a. Skype, FaceTime, Cameras (security) b. They do not change the fundamentals, just new tools
How do we detect cause and effect relationships in qualitative data?
customer experience model; In their quote: "This because of that", "So as a result", "Which lead to..."
What are the three biggest advantages of using maximum difference scaling?
excellent variation improved predictive capabilities allows for need-based segments
What is NPS?
overall performance and strength of the company - look at the valid percents
Most attribute importance research methods have
severe limitations
Under what situations would you recommend in-depth interviews to be used instead of focus groups
Politics, Drinking, Money, Buying a House, Picking out a wedding ring
How do we interpret the results for preferences levels?
Positive #'s= Above Avg Preference 0= Avg Preference Negative #'s= Below Avg
What are typical applications of a market simulation? (Things to test-4)
Price Sensitivity, New Feature, Competitive Moves, Brand New Product
What equation is used in CX Performance (NPS) Analysis in JASP, and how do we display our finidings?
NPS Calculation: Valid % of Promoters minus Valid % of Detractors Use a table or graph to display NPS values
T/F: Conjoint simulator assumptions usually don't hold true in the real world
TRUE: Conjoint simulator assumptions usually don't hold true in the real world
What type of person do we want to recruit for a qualitative sample?
Talkative,Picky, Smart, Leading Edge, Complainer
What new technology tools are used to analyze qualitative data? Is this technology fool proof?
Text Analytics/Mining and it's getting better every year
How do market simulations work?
The imitation of marketing a real-world system over time
Performance Ratio > 1.03
Significant Advantage
Performance Ratio <.97
Significant Disadvantage
What is a grand tour question? Give Example
Learn about a process. Think about the last time you ______
Mathematically, how do market simulations work?
"all customers" have a preference score for each level, and then we look at the chart to see" Would Mike choose and amber beer for $13 or a light beer for $7? Then... we look at Mikes overall preference score for a light beer, $7 pitcher? Mikes preference: (light beer=1.2 + $7pitcher= 2) = 3.2) Mike overall preference score for an amber beer? Mikes preference: (amber beer=5.5 + $13pitcher=1) =6.5) *preferred*
How do we calculate NPS?
(% promoters -% of detractors)
What is the range of NPS scores and what do they mean?
-100: Every customer is a detractor to 100: Every customer is a promoter
Top Box (9s&10s) buy:
2 1/2 times more
7 sins of Bad Graphs
1. Data Overload 2. Font Size - *Don't go below size 24* 3. Bad Colors 4. No/Bad Data Labels 5. No Sort Ascending or Descending 6. Inconsistent Scales 7. Excel Defaults are dangerous
How many maximum difference scaling questions are usually asked?
10
Correlate attributes with the _______ variable:
Dependent Variable (NPS, Overall Satisfaction, etc)
T/F:Shares of preference always match real world market shares
FALSE: Shares of preference *don't always* match real world market shares
What are the steps for *correlation* for the CX Attribute Importance in JASP?
FIRST: Click on *regression* and then select *correlation matrix* SECOND: Select the variables of interest and put your *dependent variable* (ACust.SIndex) *1st*, then select the remaining variables (KTaste) THIRD: Paste in excel and then select the correlation values for graphing Use a table or graph to display *correlation values*
In JASP, What are the steps for *Conjoint Choice Analysis* ? (hint: Which price is the best?Which speed is the best?)
FIRST: Run *means* for conjoint attributes (Variables of Interest- ex.speed of service, price, bev and food options) SECOND: Paste in Excel, Modify means in *descending* order THIRD: Repeat this process for the levels of each attribute,and run each "set of levels" individually (ex. price levels first, speed of service second, and so forth) FINALLY: Use a table or graph to display performance ratio values
What are the steps for MaxDiff in JASP, and how do we interpret the results? (To-Go vs. Stay)
FIRST: Select Variables of interest(Taste, Freshness,Variety, etc) for one situation first ( *To Go*) SECOND: Paste in excel and repeat for the next situation (*stay*) THIRD: Take the *means* of the attributes/variables for each situation and place them side by side to make your graph Interpretation: 100-More points more importance Graph in *descending* order when comparing both situations in one graph
What purpose do verbatim quotes serve?
It allows you to see what the customers are saying by presenting you the *evidence* for their claims and it *hooks the audience* which adds to presentation quality and triggers an *emotional reaction*
Why do we use a best competitor as a comparison standard? What are other relevant comparison standards?
It gives us the best feel to learn how we are doing. To get a marketplace perspective. *Competition*, Past performance, Expectations, Requirements, Others
Why is CX important? What is the relationship to financial performance?
Revenue It is *5 to 10* times more expensive to *acquire* new customer than to *retain* customers
Preference for Levels - Customer Choice
Run means for levels within each attribute Just calculate our mean scores for levels Zero based numbers, means will add up to 0 Make sure that ALL level graphs have *consistent scales* points Use a table or graph to display performance ratio values
*QUALITATIVE*: *D^2* Sample:_______ ?'s:______ Data:_______ Analysis:_______ Focus:________
S: Few- Hand Picked ?: Open-Ended D: Meaning -Words A: Interpretive F: Why?
*QUANTITATIVE*: *Generalize* Sample:_______ ?'s:______ Data:_______ Analysis:_______ Focus:________
S: Random-many ?: Close-Ended D: Numeric A: Statistics F: What?
What is a more advanced analysis for conjoint analysis?
Segmentation and Market Simulation
(P) Significance score < 0.5
Significant
Are detecting cause and effect relationships a strength of qualitative research? What is the only method that can "test" for cause and effect relationships?
Yes, Helps explore context Experiments
Performance Ratio 1.03-.97
Zone of Indifference