MKT 312 #2
Solomon Four-Group Design
EG R O1 X O2 ----------------- CG R O3 O4 ----------------- EG R X O5 ----------------- CG R O6 This design is often prohibitively expensive Provides power to control for before measure effect of O1 on both X and O2
Experimental Research
Manipulation of treatment variable (x), followed by observation of response variable (y) Experiment must be designed to control for other variables to establish causal relationship
Matched Control Group Design
Matches experimental and control groups to reduce selection bias EG M X O1 - - - - - - - - - - CG M O2
Threats to Internal Validity
Maturation: during experiments, it takes time subject might be more fatigued (not the same as at the beginning) Instrumentation: Clearly identify variables, but researcher might have made a mistake (portion sizes, maintaining same variables for each subject) Selection bias: Systematic system bias History: Y's sensitivity might be altered if subjects have been in similar experiments Testing: subjects behave differently because they know they're being tested -Hawthorne Effect: Subjects recognize this is a test -> they behave differently Statistical Regression: Height of parent affects height of child; sampling bias Mortality: Subject dies or doesn't finish experiment Selection-maturation interaction: variables can have relationships with each other so we have to be careful
Success of Nondirective Interviews
(1) establishing a relaxed and sympathetic relationship; (2) the ability to probe in order to clarify and elaborate on interesting responses, without biasing the content of the responses; and (3) the skill of guiding the discussion back to the topic outline when digressions are unfruitful, and (4) always pursuing reasons behind the comments and answers. .
Projective Technique
A fairly ambiguous stimulus is presented to the respondents in reacting to or describing the stimulus, the respondents will indirectly reveal their own inner feelings Presentation of an ambiguous, unstructured object, activity, or person that the respondent is asked to interpret and explain When do we use this? Respondents can't respond meaningfully about -The reasons for certain behavior/attitudes -What the act of buying/using a product means to them Instant Coffee Example: A-->list including maxwell house B--> list including nescafe How do you feel about housewife's behavior Results: Buying nescafe, wife is lazy; Nescafe is useful for working females
Latin Square Design
A simplified design in the absence of interaction effects. E.g., we want to test the effect of three different advertisements (A1, A2, A3), at three different price levels (P1, P2, P3), at three different Store sizes (S1, S2, S3). Using factorial design, how many cells do we need? 27
Major Categories of Acceptable Uses of Qualitative Research Methods
1. Exploratory r Defining problems in more detail. r Suggesting hypotheses to be tested in subsequent research. r Generating new product or service concepts, problem solutions, lists of product features, and so forth. r Getting preliminary reactions to new product concepts. r Pretesting structured questionnaires. ----- 2. Orientation r Learning the consumer's vantage point and vocabulary. r Educating the researcher to an unfamiliar environment: needs, satisfactions, usage situations, and problems. ------- 3. Clinical - Gaining insights into topics that otherwise might be impossible to pursue with structured research methods.
Conditions for Valid Causal Inference
Concomitant Variation: evidence that a strong association exists between an action an an observed outcome Time Order of Occurrence: Evidence that the action preceded the outcome Absence of Competing Causal Explanations: Evidence that there is no strong competing explanation for the relationship-- that a high level of internal validity exists
Two-Group, Before-After Design
Adds a control group to one-group, before - after design Helps control for history and maturation Controls for reactive effect of O1 and O2 EG R O1 X O2 - - - - - - - - - - CG R O3 O4
Advantages and Disadvantages of Drop-Off Questionnaire
Advantages: The interviewer can explain the study, answer questions, and designate a respondent. Response rates tend to be like those of personal interview studies. There is more opportunity to give thoughtful answers and consult records ------ Disadvantages: Costs about as much as personal interviews. A field staff is required.
One Group, After-Only Design
Apply the experimental treatment to a subject or group and measure the results EG X O Leaves open the possibility that the results could be explained by events external to the design
Threats to Experiment Validity in One-group, Before-After Design
Before Measure Effect -May alert respondents to the fact that they are being studied -Results in more socially desirable behavior Mortality Effect -Some subjects may stop participating in the experiment Instrumentation Effect -Results from a change in the measuring instrument
Limitations of Personal Interviews
Bias of Interviewer Response Bias Embarrassing/personal questions Time Requirements Cost Per Completed Interview is High Trained staff of interviewers geographically near the sample required
Advantages of Personal Interviews
Can arouse and keep interest Can build rapport Ask complex questions with the help of visual and other aids Clarify misunderstandings High degree of flexibility Probe for more complete answers Good for neutral questions Do not need an explicit or current list of households or individuals
Limitations of Observational Methods
Can't be used to observe motives, attitudes or intentions More costly and time consuming May yield biased results if there are sampling problems or if significant observant subjectivity is involved
Observational Methods
Casual Observation :Managers continually monitor such variables as competitive prices and advertising activity, the length of lines of customers waiting for service, and the trade journals on executives' desks, to help identify problems and opportunities. Systematic Observation: can be a useful supplement to other methods. During a personal interview, the interviewer has the opportunity to note the type, condition, and size of the residence, the respondent's race, and the type of neighborhood with regard to mixed types and qualities of homes and apartments. Direct Observation: frequently used to obtain insights into research behavior and related issues, such as packaging effectiveness. One firm uses an observer, disguised as a shopper, to watch grocery store shoppers vs. Indirect Observation Natural Observation vs. Contrived Observation: can be thought of as behavioral projective tests; that is, the response of people placed in a contrived observation situation will reveal some aspects of their underlying beliefs, attitudes, and motives. Content Analysis: an observation technique used to analyze written material into meaningful units, using carefully applied rules. Physical Trace Measures: involve recording the natural "residue" of behavior. Empathic Interviewing: 1. Imagine yourself in the person's situation. 2. Avoid self‐referencing. 3. Gently challenge generalizations by asking for specific examples. 4. Ask open‐ended, nonleading questions that start with how, what, and why. Humanistic Inquiry: is a controversial research method that relies heavily on observation but is now being used in marketing with increasing frequency.39 The humanistic approach advocates immersing the researcher in the system under study rather than as in the traditional scientific method, in which the researcher is a dispassionate observer. Behavior Recording Device: overcome particular deficiencies in human observers. The most obvious example is the traffic counter, which operates continuously without getting tired and consequently is cheaper and probably more accurate than humans.
Trends in Survey Research
Computer-Assisted Telephone Interviewing (CATI) -Provides researchers with a way to prevent many interviewer errors -At its best, computer-controlled telephone interviewing can produce faster, more complete, data to the researcher. ------- Computer Interactive Interviewing -In computer interactive interviewing the respondent interacts directly with the computer -Computer interactive interviewing has resulted in better responses from respondents and, in some instances, 30 to 40 percent cost savings.
Unwillingness to respond accurately is due to:
Concern about invasion of privacy: might not want to answer personal questions Time pressure and fatigue: As a lengthy interview proceeds, the accuracy of responses is bound to decline. Prestige seeking and social desirability response bias: respondents will distort their answers in ways that (they believe) will enhance their prestige in the eyes of the interviewer and will not put them at variance with their perception of the prevailing norms of society. Courtesy bias: general tendency to limit answers to pleasantries that will cause little discomfort or embarrassment to the interviewer or to avoid appearing uncooperative. Uninformed response bias: Simply asking someone a question implies that the interviewer expects the respondent to have an answer. Response style: Evaluative questions requiring a good‐bad, positive‐negative judgment are afflicted by systematic tendencies of certain respondents to select particular styles or categories of response regardless of the content of the question.
Limitations of Experiments
Cost Involved Time Considerations Security -Field Experiment exposes marketing program to marketplace -Difficult to hide from competitors Implementation Problems -Difficult to gain cooperation within organization -Contamination may occur in experiments involving market areas due to inability to confine the treatment to designated experimental area -Variability in behavior across test units can be so large that it's difficult to detect experimental effects
Interviewer Error
Depends on: -Respondent's Impression of the Interviewer -Questioning, Probing, and Recording -Fraud and Deceit Solved by: -Improving Interviewer Quality
Descriptive Research
Descriptive research provides a snapshot of some aspect of market environment at a specific point in time No hint of a causal insight to be obtained from descriptive data
Laboratory Research (a type of Experimental Research)
Experiments in which the experimental treatment is introduced in an artificial or laboratory setting Tend to be artificial Testing effect exists as respondents are aware of being in a test and may not respond naturally Results may not have external validity Least costly and allow experimenter greater control over the experiment Alternative explanations of results are reduced, increasing internal validity
Three types of Focus Groups
Exploratory: used at the exploratory phase of the market research process to aid in defining the problem precisely. They can also be viewed as pilot testing: Exploratory groups can be used to generate hypotheses for testing or concepts for future research. Clinical: involve qualitative research in its most scientific form. The research is conducted as a scientific endeavor, based on the premise that a person's true motivations and feelings are subconscious in nature. Experiential: allow the researcher to experience the emotional framework in which the product is being used. Thus, an experiencing approach represents an opportunity to "experience" a consumer in a natural setting.
Random Assignment
For any given assignment to a treatment, every member of the universe has an equal probability of being chosen
One-group, Before-After Design
Improve control by adding before measure EG O1 X O2 Before measure adds sensitivity by adding another method to control for confounding variables
Telephone Interviewing Limitations
Inability to employ visual aids or complex tasks Interviewer must rely solely on verbal cues to judge the reaction and understanding of respondents Can't be longer than 5-10 min. or they get boring Amount of data that can be collected is relatively less A capable interviewer essential Potential for sample bias No phone, unlisted phone or mobile phones
Inaccuracy in Response
Inability to respond Telescoping: an event is remembered as occurring more recently than it actually did. Averaging: something is reported as more like the usual, the expected, or the norm. This is a particular problem for researchers trying to study the exceptions to the ordinary. Omission: where a respondent leaves out an event or some aspect of an experience. Respondent fatigue and poor interviewer rapport are demotivating and result in increased omitting. Cannot formulate an adequate answer: many decisions are made without conscious consideration of the reasons, therefore the reasons they give later on might be incomplete or superficial. --- Some of these problems can be solved by Aided-recall techniques: attempt to stimulate recall with specific cues, such as copies of magazines, pictures, or lists.
Non-matched Control Group Design
Introduce a control group to control for history and maturation EG X O1 ------------- CG O2
Moral of Charity Story
It is important to know the entire information before coming to conclusions
Limitations of Focus Groups
Lack of Generalizability-Results can't be viewed as conclusive Opportunity for Misuse -Occurs when managers yield to a temptation to generalize a few key remarks made by participants Cost -On a cost-per-respondent basis, focus groups are extremely expensive
Individual in-depth interview techniques
Laddering: questioning progresses from product characteristics to user characteristics. Hidden-issues: focus is not on socially shared values but rather on personal "sore spots"—not on general lifestyles but on deeply felt personal concerns. Symbolic Analysis: attempts to analyze the symbolic meaning of objects by comparing them with their opposites.
Observational Methods of Information Collection
Limited to providing info on current behavior
Quantitative Research
Logical place is in conclusive research projects Calls for very specific data, capable of suggesting a final course of action More structure and by larger, more representative respondent samples Primary Role is to test hunches/hypotheses
Self-Administered Survey Advantages
Lower cost Better results, including a shorter response time Reliable answers as no inhibiting intermediary Survey answered at respondents' discretion
Decisions that need to be taken for Self-Administered Surveys
Method of addressing Cover letter Questionnaire length, content, layout, color, and format Method of notification; should there be a follow-up? Incentive to be given
Ethical Issues in Data Collection
Misrepresentation of Data Collection Process Stems From: -Representation of a marketing activity other than research as research -Abuse of respondents' rights during the data collection process, under the rationale of providing better quality research. E.g., >Use of survey for selling purposes >Use of survey to obtain names and addresses of prospects for direct marketing
Telephone Interviewing Advantages
More interviews can be conducted in a given time -Traveling time is saved -Shorter data collection periods More hours of the day are productive Repeated call backs at lower cost (WATS) Absence of administrative costs Lower cost per completed interview Less sample bias due to nonresponse -Intrusiveness of the phone -Ease of call backs
Benefits on 1-1 In-depth interviews
More quality More quantity More depth More representation More efficiency More value
Five Dimensions of Observation Methods
Natural vs. Contrived: Exact real world situation (go to target and observe tide purchases) vs. do something based on set up (move tide products to see what happens) Disguised vs. Nondisguised: do or don't reveal researcher identity Human vs. Mechanical: human researchers vs. computer doing the work Direct vs. Indirect: self explanatory Structured vs. Unstructured: open ended or nah
Refusals can also occur due to:
Nature of questions and place Subject of no interest to the respondent Fear Invasion of privacy Hostility towards sponsor Personal bias Characteristics of the data collection procedure (e.g., Presidential polls)
Basic Symbols and Notations
O: a formal observation or measurement X: exposure of test units to the experimental manipulation or treatment EG: experimental group of test units exposed to experimental treatment CG: control group participating but aren't exposed to experimental treatment R: random assignment of test units and experimental treatments to groups. Randomization increases reliability M: both experimental and control groups are matched on the basis of some relevant characteristics
Focus Group Discussions
Offer participants more stimulation than an interview; makes new ideas and meaningful comments for likely Issues to be addressed: -Outlining the intended direction of the group -Explaining how participants were recruited -Re-educating observers on the concepts of random selection, statistical reliability, and projectability of results
Factors Affecting Response Rate
Perceived amount of work required, length of the questionnaire and ease of completion Intrinsic interest in the topic Characteristics of the sample Credibility of the sponsoring organization Level of induced motivation' An increased unethical use of marketing research to sell products
Methods of Data Collection
Personal Interview Telephone Interview Self-administered survey (email, mail, etc.)
Non-response errors due to refusals
Phenotypic Source: -Characteristics of the data collection procedure >Question asked >How question is asked >Length of interview Genotypic Source: -Indigenous characteristics of the respondents >Age >Gender >Occupation
Key Factors for Focus Group Success
Planning the agenda Recruitment Moderator Analysis and Interpretation of the Results
Guidelines for Conducting Experimental Research
Recognition of and statement of the problem - should be clear and generally acceptable Choice of factors and levels - considering control and measurement Selection of the response variable - must provide useful information about process under study Choice of experimental design - considering sample size, Selection of suitable run order for experimental trials, determination of whether or not blocking or other randomization restrictions are involved Performing the experiment - monitor carefully to avoid errors Data Analysis - use statistical methods Conclusion and Recommendations - draw practical conclusions and recommend a course of action
Qualitative Methods of Information Collection
Recommended to capture the basic feel of a problem prior to conducting more analytical study Four major Constraints: -Volume of data -Complexity of analysis -Detail of clarification record -time-consuming nature of the clerical efforts required Computer tech helps alleviate the use of qualitative research -Transcribing -Coding -Searching and retrieving -Building relationships -Matrix Building
Sources of Survey Error
Population hasn't been defined correctly Respondents selected aren't able or not willing to participate Questions aren't understood by respondents Sample isn't representative of population Interviewer incorrectly understands the response and therefore records it wrong Respondents don't have the knowledge, opinions, attitudes, or facts required
Limitations of Qualitative Methods
Potential Susceptibility of the results to get misused or misinterpreted Results not necessarily representative of whole population Moderator or interviewer's role is extremely critical and can lead to ambiguous or misleading results
Factors Affecting the Choice of a Survey Method
Sampling Type of Population Question Form Question Content Response Rate Costs Available Facilities Duration of Data Collection
Two Group, After-Only Design
Randomization can match test and control groups on all dimensions simultaneously, given a sufficient sample size EG R X O1 - - - - - - - - - - CG R O2 There is no interaction effect of testing as there are no pretest requirements
Field Experiments (a type of Experimental Research)
Research study in which one or more independent variables are manipulated by the experimenter under carefully controlled conditions as the situation will permit Experimental treatment or intervention introduced in a completely natural setting Response tends to be natural Tend to have much greater external validity Difficult to control; therefore, less internal validity Competing explanations for results exist
Personal Interviewing
Researcher -> Interviewer->Interviewee->Interview Environment
Experiments
Studies in which conditions are controlled so that one or more independent variables can be manipulated to test a hypothesis about a dependent variable X(independent variable) Influences ->Y(Dependent Variable) Ex). Coupon -> sales increase Extraneous variable (z): could be influencing Y as well (can't just assume that X is the sole factor)
Trends in Focus Groups
Telephone Video Two-Way Online First, there is the single, one-way or traditional focus group wherein all the respondents are placed in just one focus group to interactively discuss the object of the survey. This typical focus group is composed of 6 to 12 members. On the other hand, the two-way focus group involves two focus groups - one focus group discussing the object, and the other focus group observing and discussing the interactions of the members of the first focus group.
Self-Administered Survey Limitations
The absence of an interviewer means that a large number of variables are controlled inadequately, including the following: -The identity of the respondent -Whom the respondent consults for help in answering questions -The speed of the response -The order in which the questions are exposed and answered -Respondent understanding of the questions
Threats to External Validity
The interaction effect of selection biases and the experimental variable Multiple treatment interference: Each treatment had relationships with the variables so we don't know where the results are coming from (want to tease out interactions between treatments) The reactive or interaction effect of testing: Subject might be embarrassed, can't clearly identify issue Reactive effects of experimental arrangements
Self-Administered Surveys
This type of survey can be emailed, mailed, faxed or simply handed to the respondent No interviewer is involved This technique has one major disadvantage: -There is no one present to explain things to the respondent and clarify responses to open-ended questions. -Most of the open-ended questions being totally useless
Coping with Non-response to Self-Administered Surveys
To achieve high response rate: -Include monetary incentive -Send a follow-up letter -Include return envelope Alternatives: -Mail Panels -Fax Surveys -Web Surveys
Simulated Test Marketing
To overcome limitations posed by experiments such as test marketing in terms of cost, time involved and sabotage or imitation by competition Enables firms to screen new ideas during the early phases of development and channel resources to maximize the potential for success Widely used by the packaged goods industry as an aid for effective business planning Examples of companies involved in STM research - BASES, ASSESSOR and Yankelovich.
Factorial Designs
Two or more experimental variables are considered simultaneously Each combination of the experimental treatment levels applies to randomly selected groups Provides the ability to determine interactive effects of pairs of experimental variables and the main effect Is used to measure the effects of two or more independent variables at various levels. Computation of number of cells needed for a factorial experiment
Information from Surveys
Used to capture: Attitude Decisions Decision process Behaviors Preferences Demographics Aspirations, goals, motivations
Validity
Validity: Good, correct, want this to be as high as possible Internal: ability of the experiment to show relationships unambiguously Whether we can control the extraneous variable or nah - Is the causal relationship between X and Y strong? External: How much we can generalize/extend this result into other settings Field experiments tend to have much greater external validity than laboratory experiments.
Categories of Projective Techniques
Word Association Sentence Completion Test Picture Interpretation Third Person Techniques Role Playing Case Studies Other Projective Techniques
Ten tips for Running a Successful Focus Group
You can never do too much planning for a focus group Manage the recruitment process actively to get the right people in the groups Don't prejudge the participants based on physical appearance The best focus group moderators bring objectivity and expertise to a project Achieving research objectives does not guarantee a successful group project The moderator and client should coordinate their efforts at all stages of the process for the research to achieve its objectives focus groups than are necessary to achieve the research objective One of the most important services a moderator can provide is a fast report turnaround Client observers should be thoroughly briefed about research objectives before the sessions start The most valuable service a moderator can provide is objective conclusions based on the interpretations of the research, without regard for what the client wants to hear
Classical Designs- True Experimental Designs
adopt random assignment procedure and use one or more control groups
Purchase Intercept Technique of Collecting Data
combines both in‐store observation and in‐store interviewing to assess shopping behavior and the reasons behind that behavior. Like a mall intercept, the PIT involves intercepting consumers while they are in a shopping environment; however, the PIT is administered at the time of an observable, specific product selection, as compared to consumers in a mall location.
Lockbox Approach
designed to circumvent the screens that receptionists and secretaries set around busy executives. The mail is used to deliver a small locked metal box containing a questionnaire and other interviewing materials such as flashcards, exhibits, and pictures.
Telephone prenotification approach
essentially a phone call to ask permission to mail or fax a questionnaire. The key is the telephone presentation, which must not only gain agreement to participate but also make sure that the prospective respondent is serious about cooperating.
Qualitative Research
involves collecting, analyzing, and interpreting data that can't be meaningfully quantified, that is, summarized in the form of numbers Techniques are most appropriate in situations calling for exploratory research Typically involves relatively few respondents or units Uses non-structured questioning/ observation techniques
Quasi-experimental Designs
offer the researcher some degree of control (more than preexperimental designs), but there is no random assignment of subjects as there is for true experimental designs. Nevertheless, quasi‐experimental designs usually provide more measurements and more information than a typical preexperimental design.
Drop-off Approach
particularly well suited to studies within compact geographic areas. For public transit studies, the questionnaire can be hand delivered to sampling points, such as every 25th house within areas that have access to a transit line.
Ways to telephone survey
random digit dialing: complete random‐digit dialing is a nondirectory procedure for selecting all 10 (area code, prefix or exchange, suffix) telephone number digits at random. plus-one dialing: consists of selecting a random sample of telephone numbers from one or more telephone directories, then adding the constant "1" to the last four digits of each number selected.
Omnibus Survey
regularly scheduled (weekly, monthly, or quarterly) personal interview surveys with questions provided by a number of separate clients. The questionnaires, based on which the interviews are conducted, will contain sequences of questions on different topics. Each sequence of questions is provided by one client, and the whole questionnaire is made up of such sequences of questions, on diverse topics, from different clients.
Time-series Design (a type of Quasi Experimental Design)
similar to the one group, before-after design except that a series of measurements is employed during which an experimental treatment occurs.
