Chapter 9: Quantitative Research Design
External Validity
"Can the findings from the study be generalized to other populations or environments?" wFactors to consider: selection of subjects, study conditions, and type of observations. wHand and hand with sampling frame. Can the findings be used with other populations or environments. How well does the sample reflect the population. Under what conditions can and with what types of subjects can the same be expected to occur.
Research Designs
-A descriptive correlational design -A cross-sectional nonprobability sample of 90 African-American women stratified by weight (normal, overweight and obese) and their male partners for a total of 180 subjects --Pilot Study of 30 -Quantitative data will be analyzed for the pilot and main study
Data collection Procedures
-African-American women will complete: Demographic and Personal Information Form, the CES-D, Health Beliefs Related to Weight Scale, Cardiac Exercise Self-Efficacy Instrument, Cardiac Diet Self-Efficacy Instrument, Weight Perceptions and Control Scale, and the Heimberg Future Time Perspective Inventory -African-American male partners will complete: Demographic and Personal Information Form, and the Male Significant Other's Perception of a Woman's Weight Scale
Additional criterion in health research: biologic plausibility
-Biologic plausibility: The causal relationship should be consistent with evidence from basic physiologic studies.
Feasibility and control (considerations for the research question and design)
-F: Time Required for Subject Participation Subject Availability Reliability and Validity of New Measurement Tools Facility and Equipment Availability Study Costs Researcher Experience Generalizability -Control is accomplished by ruling out extraneous or mediating variables as an explanation for a studys outcome. This is important and will be questioned in all research designs.
Variables
-Independent Variable (Predictor/Cause) -Dependent Variable (Outcome/Effect) -Extraneous Variables (Confounder) Extraneous variables are controlled by the use of a homogeneous sample, use of consistent data-collection procedures, manipulation of the independent variable, and randomization. It has an unpredictable unexpected impact on the dependent variable. A variable that confounds the relationship between the independent and dependent variables and that need to be controlled either statistically or in the research design.
Control
-Measures the researcher uses to hold the conditions of the study uniform to avoid possible impingement or bias on the dependent variable or outcome. -Goal - rule out competition -Dependent on design selection (ie. Experimental versus NonExperimental) -The greater the amount of control the more credible the study findings. Involves holding the conditions of the study constant and establishing specific sampling criteria. \ Rules out competing explanations for the study results (extraneous variables) Anything that competes with the independent variable as an explanation for the studies outcome.
Three key criteria for making casual inferences
-Temporal: The cause must precede the effect in time. -Relationship: There must be a demonstrated association between the cause and the effect. -Confounder: The relationship between the presumed cause and effect cannot be explained by a third variable or confounder; another factor related to both the presumed cause and effect cannot be the "real" cause.
Limitations
-This convenience sample may have limitations related to its true representation of all African-American women within the weight classes. -Subjects will want to portray their best attitudes and health practices knowing that they are being observed and measured. -The use of some relatively new measurement instruments may threaten validity.
Threats to external validity (3)
1. Selection Effects- generalizability to other populations (Representative sample ?). Threat ex. Researcher cannot attain the ideal sample population. (characteristics or size) 2. Reactive Effects - subject's responses to being studied. (Hawthorne effect). 3. Measurement Effects - when a study is conducted and a pretest is given, it may prime the subjects and affect their ability to generalize to other situations. Selection Effects - The researcher uses a nonprobability method of sampling over a probability method because at times, the number of avaialble subjects is low or unavailable. This is evident when you read a research report and the author makes statements such as: withour a control group, investigators can not conclude with certainty that the benefits reported wer actually the result of the family group therapy; the use of a small convernience sample of primarily young white male of a high socioeconomic status limits the generalizability of findings to those with similar characteristics; or the sample size was small and the findings were descriptive in nature. Reactive Effects - Subjects may respond to the investigator not because of the study procedures but merely as an independent resonse to being studied. Hawthorne effect - named after the Western Electric Corporation's Hawthorne plant, where a study of working conditions was conducted. The researchers developed several different working conditions ( turning up lights, load music, soft music, different work schedules).They found that no matter what was done, worker's productivity increased. Measurement effects - administration of a pretest in a study effects the generalizabilitly of the findings to other populations. The subjects' responses on follow-up testing may differ from those of individuals who were given the education program.
Research Questions-Quantitative
1.Descriptive - describe a single topic/ population which there is theoretical or conceptual material in literature, but no empirical data. 2.Exploratory- examine relationships or differences between concepts and other factors. 3.Test knowledge-asks about cause-and-effect among 2 or more variables. Purpose is to predict what will happen.
Identify threat and how to correct 3
A major increase in cigarette taxes occurs during a one-year follow up study of the impact of smoking cessation program.
Quantitative Research
A systematic, objective process used to gather and statistically analyze information that has been measured by an instrument and converted to numerical data.
Considerations for the Research question and design
Accuracy -Appropriate design accuracy can be accomplished through the theoretical framework and review of the literature. -All aspects of the study systematcally and logically follow from the research questions. -Simple designs require accuracy. -Pilot studies --Invaluable for maitaining accuracy --Provide important information for feasible and well-grounded future inquiry Simple designs require accuracy. Greatly suggested for the novice researcher. Still as useful and vauable as a complicated design. The consumer shoud feel that the researcher chose a design that was consistent with the research question and offered the maximum amount of comtrol. Pilot studys very important when a clinical/research question has not yet been researched.
Inclusion Criteria (MALE)
African-American male partner must: 1) be able to read, write, and understand English 2) be between 18 and 65 years of age and 3) consent to participate
Inclusion criteria (FEMALE)
African-American women of normal weight, overweight or obese weight class who have: 1) self-reported African American heritage; 2) are American-born; 3) are premenopausal; 4) are age 18-44; 5) have the ability to read, speak and write English; 6) who do not have diagnosed and currently treated mental health conditions; and, 7) agree to participate in the study by providing informed consent
Specific Aims
Aim 1: To explore the relationships among cultural factors and weight class in African-American women. Aim 2: To explore the relationship between cultural factors and perceptions of body image in African-American women. Aim 3: To explore cultural factors, future time perspective, and biological variations as predictors of weight class and depression in African-American women with perceptions of body image as a moderating variable.
Idenitfy threat and how to correct 5
Although 30% of the subjects dropped out of a study of employment after a job training program for homeless families, 90% of the persons completing the program were employed within two months and the program was deemed successful for all homeless families. Ninety percent of the dropouts were single women with at least two preschool children.
Research designs notes
As much as possible, equal numbers of women will be recruited for each weight class category. Although a convenience nonprobability sample limits the sample from being a true representation of African-American women in the general population, this sampling approach increases the feasibility of obtaining the required sample size of women and their partners. This sampling approach does not give an accurate representation of the women in the population stratified by weight as there are a disproportionate number of African-American women in the total population who are overweight and obese as opposed to normal weight. Nevertheless, this sampling approach will be used to elucidate the differences in responses related to the study variables based on weight class. The study findings can be used to under gird future interventional and clinical studies where random sampling may be utilized to increase generalizability of the findings to the entire population of African-American women. Quantitative data will be analyzed to elucidate the differences in the various cultural factors and perceptions of body image amongst the subjects.
Controlling the study context
Controlling external factors (such as research context) •Achieving constancy of conditions •Control over environment, setting, time •Control over intervention via a formal protocol: intervention fidelity
Internal validity
Does the independent variable really make the difference or the change in the dependent variable?" wEstablished by: ruling out other factors wIgnoring possible threats to Internal Validity can negate study results. There are a nunber of threats to internal validity , and these are considered by researchers in planning a study and by consumers before implementing the results in practice. Usually seen in experimental designs, but can be considered to some degree in all quantitative designs.
Internal and External Validity
Does the study really measure what it is supposed to measure? Do the results really apply to anyone else? Is the study believable (do the results make sense)? Researchers goal - maximize internal and external validity. used to establish credibility and dependability of results
Specifc Aims notes
H1: There will be a negative relationship between social organization factors (income and education) and weight class with a positive relationship between the social organization factor (family history of obesity) and weight class in African-American women. H2: There will be a negative relationship between environmental control factors (health beliefs about weight and weight control and empowerment through self-efficacy related to diet management and physical activity) and weight class with a positive relationship between the environmental control factor (male significant other perceptions of weight) and weight class in African-American women. H3: There will be a positive relationship between biological variables (African-American women's past history of depression, family history of depression, and age) and weight class in African-American women. H4: There will be a negative relationship between future time perspective and weight class in African-American women. RQ1: What is the relationship between age, weight class, and partner perceptions of weight of African-American women? Aim 2: To explore the relationship between cultural factors and perceptions of body image in African-American women. H5: There will be a negative relationship between social organization factors (income and education) and perceptions of body image with a positive relationship between (family history of obesity) and perceptions of body image in African-American women. H6: There will be a negative relationship between environmental control factors (health beliefs about weight and weight control and empowerment through self-efficacy for diet and exercise) and perceptions of body image with a positive relationship between environmental control factor (partner perceptions of weight) and perceptions of body image in African-American women. H7: There will be a positive relationship between biological variables (age, family history of depression, and the woman's past history of depression) and perceptions of body image in African-American women. RQ2: What is the relationship between future time perspective and perceptions of body image in African-American women? Aim 3: To explore cultural factors, future time perspective, and biological variations as predictors of weight class and depression in African-American women with perceptions of body image as a moderating variable. RQ3: What is the nature of the relationships among depression, cultural factors, perceptions of body image and weight class? H8: Social organization factors, environmental control, future time perspective, and biological variations will predict weight class in African-American women with perceptions of body image as a moderating variable. H9: Social organization factors, environmental control, future time perspective, and biological variations will predict depression in African-American women with perceptions of body image as a moderating variable. RQ4: Is there a relationship between depression and weight class in African-American women?
Threats to internal validity 2
History Selection threat Maturation Mortality Testing Instrumentation
Common threats to internal validity: History effect and selection threat
History effect - the effect of external events on study outcomes; an event that is not related to the planned study but occurs during the time of the study; can influence the responses of subjects to the treatment. Selection threat - biases resulting from the differences between groups. Non-equivalent groups. History - For example a study looking at intervention for smoking cessation is conducted during the period a very high sales tax is levied on cigarette sales. Another example is a study examining the effects of the media on domestic violence being conducted since the beginning of 2009. All of a sudden, there is an event involving Chris Brown and Rihanna with a proliferation of talk show and media coverage dedicated to media violence. Will this affects the way your participants respond? Selection - In the Nelson study, while the investigator tried to control for bias, there was a hardship in terms of finding obese participants interested in participating. She contacted a modeling agency and recruited the majority of the study stubjects who were plus size models. Would this introduce bias. When considering body image, these women may have high level os confidence, self-esteem and self-efficacy.
Control measures in quantitative research
Homogeneity of sampling-inclusion exclusion criteria is utilized to as a straigtforward step of control in order to secure a homogeneous sample. Certain varible such as age, gender, length of time smoked, amount smoked may all affect the outcome of a smoking cessation intervention. These common characteristics of a the study subjects, althoug outside of the study's design may affect the outcome. Constancy: TIMING, INSTRUMENTS, PROCEDURES. is also referred to intervention fidelity. Consistency and standardization in data collection- using reliable and valid instruments, training data collectors. The research will attend to this by describing the the training of study staff or data collectors that took place to ensure constancy. They may also discuss an intervention manual used to script a standardized approach to an intervention or data collection.
Key Features of Quantitative Research Design: Interventions
Key questions •Will there be an intervention? •What specific design will be used? Broad design options •Experimental (randomized control trial) •Quasi-experimental (controlled trial without randomization) •Nonexperimental (observational study) MR. C (Manipulation, randomization, control - 3 things you need to look at when critiquing research study)
Manipulation of the independent variable and randomization
Manipulation of the independent variable Administration of a program, treatment, or intervention to only one group - the experimental group The other group is the control group Randomization - each subject in a population has an equal chance of being selected Manipulation of independent variable (controlling treatment) - in a control group the variables under study are held at a constant or comparison level. Non-experimental groups do not manipulate the independent variable. This does not decrease the usefulness of a nonexperimental design, but the use of a control group in an experimental or quasiexperimental deisgn is related to the level of the research question, the theoretical framework. Independent variable present in one group and absent in the other. Randomization- Every subject selected to be in the study has an equal chance to be assigned to either the treatment or the control group. How does randomization reduce bias? It eliminates bias, aids in the attainment of a representative sample, and can be used in various designs. 1. It neutralizes the effects of extraneous influences on the DV 2. It equalizes subjects in the experimental and control groups to make them comparable on key characteristics at baseline. Blinding Controls observation bias by participant and or investigator
Common threats to internal validity: Maturation and mortality
Maturation - the effect of passage of time; unplanned and unrecognized changes that subjects experience during a study-- such as growing older, wiser, or stronger, that can influence the findings of the study. Mortality - effect on outcome caused by subject attrition or dropping out of a study before its completion. Maturation -Usually seen in a pretest-postest design or a desing that spans a length of time. The growth or change is unrelated to the investigation and may explain the differences between the two testing periods rather than the experimental treatement. Usually when investigating children and learning methods, post-tests of student learning must be conducted in a relatively short period of time. Why? You need to be able to conclude that the results were the resul of the design of the study and not maturation in this population of children who learn skills rapidly. Changes can be due to an age-related developmental process as well as physical changes. Mortality - the loss of subjects from the first data collection point to the second data collection point. Results would not be affected if the participants remaining in the stude are not different from those that dropped out. In a study examining the ways a pharmacist's attitudes affect the prevalence of medication non-compliance among patients, if most dropouts were patients not on prescription meds then the perception may be given that the exposure to a pleasant pharmacist increased patient non-compliance when it really was the effect of experimental mortality that led to the observed results.
Identify threat and how to correct
Nurses on a maternity unit want to study the effect of a hospital-based teaching program on mother's confidence in caring for their infants. They mail out a survey one month after discharge.
Other key features of quantitative research design: Confounding variables
Perform a review of literature before conducting a study Control over confounding variables •How will confounding variables be controlled? •Which specific confounding variables will be controlled? •Randomization, crossover, homogeneity, matching, statistical control
Quantitative Research Design
Plan or blueprint for the conduct of the study Purpose: (1) Guide for answering the research question or testing the hypothesis (2) Maintain a level of control Experimental, QuasiExperimental, & NonExperimental Designs - the type of design affects its application to practice.
Other key features of quantitative research design: Relative timing and Location
Relative timing •When will information on independent and dependent variables be collected—looking forward or backward in time? •Retrospective (case-control), prospective (cohort) Location •Where will the study take place? •Setting choice; single site versus multisite
Common threats to internal validity: testing and instrumentation
Testing - Repeated testing can influence responses. The effects of taking a pre-test on the scores of the post-test. 6. Instrumentation - Changes in the measurement or observational techniques that may account for changes in the obtained measurement. Testing -taking a pretest may sensitize an individual and improve the score of the posttest. Individual generally score higher when they take a test the second time - That would be the case if the items on your midterm and final were the same items you received on your quizzes in the previous weeks. Differences between scores may not be a result of the independent variable but rather of the experience gained through the testing. Instrumentation - if data collectors are not similarly trained, or even if they are similarly trained but unable to conduct the study as planned, a lack of consistency may occur in their ratings, and therefore a threat to internal validity will occur. Another example is using an instruments such as a thermometer where you are comparing the accuracy across different types - mercury as compared to a digital, electronic, tympanic etc. You would need to calibrate each type consistently according to the manufacturers instructions and possibly via the same route and times.
Quantitative Research Design Notes
The design involves a plan, as well as structure and strategy which guide the researcher in writing the hypotheses or research questions, conducting the project, and analyzing and evaluating the data. The design along with the methods and analysis is the mechanism for finding solutions to research questions. Experimental, QuasiExperimental, & NonExperimental Designs - the type of design affects its application to practice. The type of design used in a study is linked to the level of evidence and, in turn, how a study's findings contribute to evidence based practice.
More threats external validity
The more control measures applied in a study the less chance for threats to internal and external validity. More control = more credibility of findings. Minimizing threats to internal and external validity enhances the strength of evidence for quantitative design. External validity can be claimed only after internal validity has been established. Finding threats does not automatically render the study findings useless.
Identify threat and how to correct 6
The researcher measured the confidence and accuracy of nursing students in calculating drug dosage and solutions at the beginning, middle, and end of a two-week course using a standardized calculation exam.
Identify threat and how to correct 4
The smoking cessation rates of an experimental group consisting of volunteers for a smoking cessation program with daily meetings were compared with the results of a control group of persons who wanted to quit on their own without a special program.
Data collection procedures
The subject will be asked to furnish contact information for their male partner, who will be asked to participate. The woman and her male partner will provide informed consent Female subjects weight class determined by BMI calculated using a scale with a stadiometer PI will collect all other data via questionnaire
Sample population and setting notes
The target population for this study will be African-American women who are of varying weight classes: normal, overweight and obese and their partners. A sample of African-American women who meet the inclusion criteria will be recruited from various beauty salons, University and college campuses, and churches in metropolitan Atlanta. The beauty salon has been chosen as a site par excellence, where the attainment of femininity, and its definition and negotiation are being played out. In its generally closed and intimate nature, the beauty salon is not only a feminized space, but also one in which the secret routines of femininity are commoditized and exemplified (Black & Sharma, 2001). Local university and local college campuses will provide a wide sampling of African-American women from various backgrounds needed to account for the variance in cultural factors and perceptions. The Atlanta metropolitan area has several public and private traditionally African-American university and college campuses as well as large predominantly white universities and colleges that have large subpopulations of African-American women for recruitment. The Emory Bariatric Center will ensure adequate access to African-American women who are obese.
Sample population and setting
The target population is African-American women who are of varying weight classes: normal, overweight and obese and their male partners. The sample is a community sample recruited from various sites such as beauty salons, barber shops, University and college campuses, and churches in metropolitan Atlanta.
Limitations notes
The use of a convenience sample may allow the introduction of confounding extraneous variables, which could mask the true nature of the relationships under examination. This may potentially limit the generalizability of the findings to other samples. Threats to validity are present. Subjects will want to portray their best attitudes and health practices knowing that they are being observed and measured. External validity will be affected by the use of a convenience sample.
Potential threats to a study's validity
There are similarities in the names of the selection factos and the testing/ measurement effects between internal and external validity. When considering factors as internal threat, the reader assesses them as they relate to the independent and dependent variables within th study, and when assessing them as external threats, the reader considers them in terms of the study to toher populations ans settings.
Critiquing questions
What are the threats to internal validity? What are the controls for the threats to internal validity? What are the threats to external validity? What are the controls for the threats to external validity?
Which characteristic is a key criterion for causality? a.Cause occurring before the effect b.Third variable involved with the cause and effect c.No empirical relationship between the cause and effect d.Single-source evidence about the relationship
a. Cause occurring before the effect Rationale: Three key criteria for causality include the following: -The cause must precede the effect in time. -There must be a demonstrated empirical relationship between the cause and effect. -The relationship cannot be explained by a third variable. An additional criterion is that evidence of the relationship should come from multiple sources.
Tell whether the following statement is True or False. A true experiment requires that the researcher manipulate the independent variable by administering an experimental treatment (or intervention) to some subjects while withholding it from others. a.True b.False
a.True Rationale: In a true experiment, the researcher manipulates or does something, usually an intervention or treatment, to some subjects and not to others.
Tell whether the following statement is True or False. An experimental research design involves a nonrandomized controlled trial. a.True b.False
b. False Rationale: A quasi-experimental research design involves a controlled trial without randomization.
Tell whether the following statement is True or False. Cross-sectional research designs are helpful in showing patterns of change. a.True b.False
b. False Rationale: Longitudinal studies, in which data are collected two or more times over an extended period, are better at showing patterns of change than cross-sectional studies, which collect data at a single point in time.
Which design is considered a quasi-experimental research design? a.Pretest-posttest design b.Posttest-only design c.Crossover design d.Within-subjects design
d. Within-subjects design Rationale: Quasi-experimental research designs include nonequivalent control group and within-subjects designs. The other research designs are used for experimental research.
Identify threat and how to correct 2
nIn a study of the results of a hypertension teaching program conducted at a senior center, the blood pressures taken by volunteers using their personal equipment were compared before and after the program.
The counterfactual model of causality
•A COUNTERFACTUAL IS WHAT WOULD HAVE HAPPENED TO THE SAME PEOPLE EXPOSED TO A "CAUSE" IF THEY SIMULTANEOUSLY WERE NOT EXPOSED TO THE CAUSE. •AN EFFECT REPRESENTS THE DIFFERENCE BETWEEN WHAT ACTUALLY DID HAPPEN WHEN EXPOSED TO THE CAUSE AND WHAT WOULD HAPPEN WITH THE COUNTERFACTUAL CONDITION.
Advantages and disadvantages of experiments
•Advantages—most powerful for detecting cause and effect relationships •Disadvantages—often not feasible or ethical, Hawthorne effect (knowledge of being in a study may cause people to change their behavior), often expensive
Control group conditions: attention control and delayed treatment
•Attention control—extra attention but not the active ingredient of the intervention •Delayed treatment ("wait-listed controls")—the intervention is given at a later date.
Correlational designs (nonexperimental)
•Cause-probing questions (e.g., prognosis or harm/etiology questions) for which manipulation is not possible are typically addressed with a correlational design. •A correlation is an association between variables and can be detected through statistical analysis. •Correlational studies are weaker than RCTs for cause-probing questions, but different designs offer varying degrees of supportive evidence.
Cross-sectional design, longitudinal design
•Cross-sectional design—Data are collected at a single point in time. •Longitudinal design—Data are collected two or more times over an extended period. •Follow-up studies •Longitudinal designs are better at showing patterns of change and at clarifying whether a cause occurred before an effect (outcome). •A challenge in longitudinal studies is attrition or the loss of participants over time.
Research Questions and REsearch design
•Different designs are appropriate for different questions. •Therapy, prognosis, etiology/harm, and description •Experimental designs (RCTs) offer the strongest evidence of whether a cause (an intervention) results in an effect (a desired outcome). •That's why they are high on evidence hierarchies for questions about causes and effects.
Advantages and disadvantages of nonexperimental
•Disadvantage: does not yield persuasive evidence for causal inferences. This is not a problem when the aim is description, but correlational studies are often undertaken to discover causes. •Advantage: efficient way to collect large amounts of data when intervention and/or randomization is not possible
Experimental Design
•INTERVENTION: THE RESEARCHER DOES SOMETHING TO SOME SUBJECTS—INTRODUCES AN INTERVENTION (OR TREATMENT). •CONTROL: THE RESEARCHER INTRODUCES CONTROLS, INCLUDING THE USE OF A CONTROL GROUP COUNTERFACTUAL. •RANDOMIZATION: THE EXPERIMENTER ASSIGNS PARTICIPANTS TO A CONTROL OR EXPERIMENTAL CONDITION ON A RANDOM BASIS. --THE PURPOSE IS TO MAKE THE GROUPS EQUAL WITH REGARD TO ALL OTHER FACTORS EXCEPT RECEIPT OF THE INTERVENTION.
Nonequivalent control group designs
•If preintervention data are gathered, then the comparability of the experimental and comparison groups at the start of the study can be examined. •Nonequivalent control group pretest-posttest design
Nonexperimental studies
•If researchers do not intervene by controlling independent variable, the study is nonexperimental (observational). •Not all independent variables ("causes") of interest to nurse researchers can be experimentally manipulated. •For example, gender cannot ever be manipulated. •Smoking cannot ethically be manipulated.
Prospective correlational design, cohort study
•In a prospective correlational design, a potential cause in the present (e.g., experiencing vs. not experiencing a miscarriage) is linked to a hypothesized later outcome (e.g., depression 6 months later). •This is called a cohort study by medical researchers. •Prospective designs are stronger than retrospective designs in supporting causal inferences—but neither is as strong as experimental designs.
Retrospective designs
•In a retrospective correlational design, an outcome in the present (e.g., depression) is linked to a hypothesized cause occurring in the past (e.g., having had a miscarriage). •One retrospective design is a case-control design in which "cases" (e.g., those with lung cancer) are compared to "controls" (e.g., those without lung cancer) on prior potential causes (e.g., smoking habits).
Threats to external validity
•Inadequate sampling of study participants •Unfortunately, enhancing internal validity can sometimes have adverse effects on external validity.
Quasi-experiments
•Involve an intervention but lack either randomization or control group. No randomization. Two main categories of quasi-experimental designs: 1. Nonequivalent control group designs: Those getting the intervention are compared with a nonrandomized comparison group. 2. Within-subjects designs: One group is studied before and after the intervention.
threats to construct validity
•Is the intervention a good representation of the underlying construct? •Is it the intervention or awareness of the intervention that resulted in benefits? •Does the dependent variable really measure the intended constructs?
Key Features of Quantitative Research Design: Comparisons
•Key question: What type of comparisons will be made to illuminate relationships? •Some design options •Within-subjects design: Same people are compared at different times or under different conditions. (ex longitudinal) •Between-subjects design: Different people are compared (e.g., men and women).
Threats to statistical conclusion validity
•Low statistical power (e.g., sample too small) •Weakly defined "cause"—independent variable not powerful •Unreliable implementation of a treatment—low intervention fidelity
Causality
•MANY (IF NOT MOST) QUANTITATIVE RESEARCH QUESTIONS ARE ABOUT CAUSES AND EFFECTS. •RESEARCH QUESTIONS THAT SEEK TO ILLUMINATE CAUSAL RELATIONSHIPS NEED TO BE ADDRESSED WITH APPROPRIATE DESIGNS.
Experimental condition
•MUST BE DESIGNED WITH SUFFICIENT INTENSITY AND DURATION THAT EFFECTS MIGHT REASONABLY BE EXPECTED •RESEARCHERS DESCRIBE THE INTERVENTION IN FORMAL PROTOCOLS THAT STIPULATE EXACTLY WHAT THE TREATMENT IS. •ATTENTION MUST BE PAID TO INTERVENTION FIDELITY (OR TREATMENT FIDELITY), THAT IS, WHETHER THE TREATMENT AS PLANNED WAS ACTUALLY DELIVERED AND RECEIVED.
Other key features of quantitative research design: Masking/Blinding and Time Frames
•Masking/blinding: From whom will critical information be withheld to avert bias? Time frames •How often will data be collected? •When, relative to other events, will data be collected? •Cross-sectional, longitudinal design (specific point in time vs timeline)
Advantages and disadvantages of quasi-experiments
•May be easier and more practical than true experiments but •They make it more difficult to infer causality. •Usually there are several alternative rival hypotheses for results.
Control group conditions (Counterfactuals)
•No intervention is used; control group gets no treatment at all. •"Usual care" or standard or normal procedures is used to treat patients. •An alternative intervention is used (e.g., auditory vs. visual stimulation). •A placebo or pseudointervention, presumed to have no therapeutic value, is used. •Attention control condition and delayed treatment (wait-listed)
Descriptive research
•Not all research is cause probing. •The purpose of descriptive studies is to observe, describe, and document aspects of a situation. •Some research is descriptive (e.g., ascertaining the prevalence of a health problem). •Other research is descriptive correlational—the purpose is to describe whether variables are related, without ascribing a cause-and-effect connection.
Experimental designs: •Pretest-posttest (before-after) design
•Outcome data collected both at baseline and after the intervention
Experimental designs: Posttest-only
•Posttest-only (or after-only) design •Outcome data collected only after the intervention •R = Randomization; X = Receipt of intervention; O = Observation/measurement of dependent variable
Controlling participant factors
•RANDOMIZATION - SUBJECTS AS OWN CONTROLS (CROSSOVER DESIGN) •HOMOGENEITY (RESTRICTING SAMPLE) •MATCHING •STATISTICAL CONTROL (E.G., ANALYSIS OF COVARIANCE)
Characteristics of good quantitative research design
•STATISTICAL CONCLUSION VALIDITY—THE ABILITY TO DETECT TRUE RELATIONSHIPS STATISTICALLY •INTERNAL VALIDITY—THE EXTENT TO WHICH IT CAN BE INFERRED THAT THE INDEPENDENT VARIABLE CAUSED OR INFLUENCED THE DEPENDENT VARIABLE •EXTERNAL VALIDITY—THE GENERALIZABILITY OF THE OBSERVED RELATIONSHIPS ACROSS SAMPLES, SETTINGS, OR TIME •CONSTRUCT VALIDITY—THE DEGREE TO WHICH KEY CONSTRUCTS ARE ADEQUATELY CAPTURED IN THE STUDY
Experimental Designs: •Crossover design
•Subjects are exposed to 2+ conditions in random order. •Subjects serve as their own control.
Threats to internal validity ON EXAMMMM
•Temporal ambiguity •Selection threat—biases arising from preexisting differences between groups being compared (ex: all the boys in one group, all the girls in the other on accident). This is the single biggest threat to studies that do not use an experimental design. •History threat—other events co-occurring with causal factor that could also affect outcomes •Maturation threat—processes that result simply from the passage of time •Mortality/attrition threat—differential loss of participants from different groups •Typically a threat in experimental studies
Time-series designs
•Time-series designs gather preintervention and postintervention data over a longer period. •Symbolic representation: O1 O2 O3 O4 X O5 O6 O7 O8
Nonequiv ctrl group designs: Nonequivalent control group posttest only
•Without preintervention data, it is risky to assume the groups were similar at the outset. •Nonequivalent control group posttest only is much weaker. X O1 O1
One group pretest-posttest designs
•typically yield extremely weak evidence of causal relationships. •Symbolic representation: O1 X O2
