Research Method 321: Final Exam Study Guide
Difference between confounding and bias
Bias: Due to investigator/participants. Creates false association Confounding: Misleading true association.
An organization administers an employment test they are developing to current employees. They then relate their scores to existing employee performance ratings to establish _____.
Concurrent Validity
What statistics use nominal data?
Frequency, measures of central tendency
What direction is a negative distribution skewed to?
Tail to the left
Quasi-Experimental Designs:
Used when control features of experimental designs cannot be achieved ◦ For example, the independent variable cannot be manipulated Internal validity may be affected Ecological validity is higher Easier, cost less money.
Reliability & Validity
"Classic" Definitions Reliability - Replicability, consistency Validity - Measuring what it's meant to Redefined for qualitative research.
Conference Talk
-Newest research - Distribution to larger audience - Connect your work to you as a person - More formal
Descriptive vs Inferential Statistics
A study concluded that the average credit card debt of college graduates will increase from the year 2017 to 2018. Inferential
What statistics use ratio data?
All of them
Descriptive vs Inferential Statistics
Average salary of high school teachers in 2017 was $52,400. Descriptive
Ecological Validity
Can the results be replicated in the real world? Do results have implications for real-world issues?
A company is interested in developing a certification test that qualifies employees as emergency team leaders. They gather a group of subject matter experts (SMEs) to evaluate the test items in order to ensure the test is _____.
Content Validity
What statistics use ordinal data?
Frequency, Chi square, measures of central tendency, Spearman correlation
Choosing Tests In general
IV: One independent variable: Nominal, Nominal/ordinal (2 levels), Nominal/ordinal (3 or more levels),Interval/ratio. Two or more independent variables: Nominal/ordinal DV: Nominal, Interval/ratio, Interval/ratio,Interval/ratio. Two or more indepdent variables: Interval/ratio Approach: Chi-square t-test One-wayANOVA Correlation Factorial ANOVA
Samples and Populations
Inferential statistics are necessary because: The results of a given study are based on data obtained from a single sample of research participants. And Data are not based on an entire population of scores. Inferential statistics are used to determine whether individual study results match what would happen if we were to conduct the experiment again and again with multiple samples.
Central Tendency
MEAN (x̄ ) - Average - Adding all scores and dividing by number of scores Interval, Ratio MEDIAN Middle, halfway score Ordinal, Interval, Ratio MODE Most frequently occuring Nominal, Ordinal, Interval, Ratio
Ordinal
Rank ordering numeric values limited. - Example: 2-,3-,and 4 star restaurant ranking, TV programs by popularity. -Distinction: intervals between items is unknown. -Allows us to rank order the levels of the variable being studied. - Categories are ordered first to last. - Example: Letter grades, Movie ratings. Don't provide any info about the distance between the categories. Minimal Quantitative distinctions. Rank Order low to high
Bar Graph
Simple descriptions of categories for single variable. Used when the value on the x-axis are nominal. Discrete, Categorical
What are descriptive statistics?
Summarize data, describe general characteristics of the data
Descriptive vs Inferential Statistics
The average online rating for the book Statistics for Dummies by 26 reviewers is 4.6 on a scale from 1 to 5. Descriptive
To conduct a true experiment...
The researcher manipulates the independent variable to create groups ( experimental and a comparison or control group)
frequency polygon (line graph)
Use a line to represent the distribution of frequencies of scores. Used when the values on the x-axis are numeric. Most useful when the data represents interval or ratio scales.
Implications
What do the results of your study mean? ○ Are the theoretical implications different than the practical implications? ○ What recommendations may stem from your research findings? ○ Did you find anything unexpected? ○ Are there other explanations for your findings?
Control Condition
a condition in within-subjects design experiment that does not contain the experimental manipulation (instead of a control group-in exp. where each subject experiences every condition)
Sample
a small group of persons or elements(observations) selected from the total population
What are the characteristics of qualitative research?
focuses on meaning and understanding; research as primary instrument; inductive process; rich description; emergent and flexible; purposeful; natural setting; understanding of subjectivity/objectivity
Shape: Skew
https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjEofms6vbgAhWki1QKHc2uC28QjRx6BAgBEAU&url=https%3A%2F%2Fcodeburst.io%2F2-important-statistics-terms-you-need-to-know-in-data-science-skewness-and-kurtosis-388fef94eeaa&psig=AOvVaw18Lrs1omCLHQXfBlW-saJJ&ust=1552281753507678
Cross-Sectional Method
persons of different ages measured at the same point in time. shares similarities with the independent group design.
external validity
the extent to which findings may be generalized. Using a census or a random sample will always provide better external validity than using a nonrandom sample. Generalization: other settings larger population other outcomes(DV) other conditions(IV) Issues: College samples WEIRD Sample(Western, Education, Industrialized, rich and democratized) -Can the results be replicated with other participants? In other settings?
Conference Poster
- One-on-one discussions - Networking - Feedback
KEY CHARACTERISTICS OF QUALITATIVE INQUIRY
•Captures human experience in context.•Smaller number of participants (typically).•Guiding research questions.•Hypothesis generating, not testing (often).•Thematic findings.•Conclusions are drawn based on researcher's qualitative analysis (word based) and interpretation of meaning
Nominal
-Categories with no numeric scales and no quantitative value. -Example: males, females,introverts, extroverts. -Distinction: impossible to define any quantitative values. - No numerical or quantitative properties. Categories or groups differ from one another. Most independent variables in experiments are nominal. Example: Eye color, College major, Marital status.
Convergent Validity
Do scores on the measure relate highly (and positively) to score on other measures of the same of similar constructs? -Does the Beck Depression Inventory demonstrate a strong, positive relationship to the center for Epidemiological studies depression measure? - Any given measure is a particular operational definition of the variable being measured. -The extent to which scores on the measure in question are related to scores on other measures of the same construct or similar constructs.
What is the most common method utilized for testing inferential statistics?
Hypothesis testing!
Types of Variables
Nominal,Ordinal,Interval and Ratio
Descriptive Statistics
Precise descriptions about data - Central tendency - Variability
Experimental Method
The researcher attempts to control all extraneous variables. Allows for the study of relationships among variables Isolation of causal mechanisms can help explore cause and effect relationships between IV and DV. Experimental controls eliminate "3rd variable" alternatives
Reflexivity
Attending to the construction of knowledge at every step of the research process.
Generalizing Results
Participants are generally selected from WEIRD samples White Educated Industrialized Rich Democratic This means some groups are over-represented College Students Volunteers Locally relevant Gender considerations
Communicating to Non-Scientists
➢ Know & relate your audience ➢ Avoid jargon ➢ Images, videos ○ Avoid excessive text ➢ Analogies, metaphors- Tell a "story" ➢ "Three Points" approach ➢ Give big picture, avoid detailed analyses ○ Memorable, Meaningful, Miniature
Manuscripts
- Literature Reviews - Meta-Analysis - Theory Articles - Empirical Research Articles - Research Note/Brief
Ratio
-Like interval, but with true zero. Zero indicates absence of variable measured. -Example: reaction time, weight, age, frequency or behavior. Distinction: Can form ratios. -Have absolute zero's point that indicates the absence of the variable being measured. - Used in the behavioral sciences which variables that involve physical measures are being studied-partrically time measures such as reaction time, rate o responding and duration of response.
Interval
-Literal number, Numeric properties are literal assume equal intervals between values. -Example: Intelligence, aptitude, test scores and temperature. -Distinction: No true zero point. - Generally have five or more quantitative levels.
Most Common Types of Qualitative Analysis
1. Interview 2.Focus Group 6-12 Participants 3. Ethnographic Methods ETHNOGRAPHIC METHODS•Researcher conducts observations in a real-world setting.•Researcher defines scope of observations (setting, people, action).•Observations occur over an extended period of time (immersion).•Involve a variety of data collection techniques.-Observe people and events (field notes) -Interview people (key informants) -Examine documents
Reflexive Thinking
Collect Data What were you thinking and feeling during data collection? Acknowledge: bias, assumptions, values, culture, etc What influence did this have on your analysis and findings? Re-evaulate findings Use reflection to inform future research. Collect Data.
Comparison Of Longitudinal And Cross-Sectional Methods
Cross-Sectional is more common because it's less expensive and has immediate results. Disadvantage: Researcher must infer that differences among age groups are due to developmental variable in age.
What are inferential statistics?
Draw inferences about the effects of sampling errors on results, allow for generalizations of finding. Allows researchers to (a) assess just how confident they are that their results reflect what is true in the larger population and (b) assess the likelihood that their findings would still occur if their study was repeated over and over.
Replication and Generalization
Exact Replications: ● An attempt to replicate precisely the procedures of a study. ● Ascertain whether the same results are obtained with replication. ● Important early in the research process (e.g., to determine whether a Type I Error has occurred) or when there are questions about fraud. Conceptual Replications: ● The use of different procedures to replicate a research finding. ● The IV from previous study is operationalized and/or manipulated in different ways. ● Allows researchers to look at the conceptual relationships between variables and to determine how robust the findings are. ● For example, does the relationship only occur under a very specific set of circumstances, or is the relationship more generalizable?
Threats to Internal Validity
History effects: A source external to the experiment causes the change in the DV. Maturation: Natural progression of the individual causes change. Aging, learning, natural developmental processes. 3rd variables (confounds): Change is due to a variable not measured in the experiment. How might be ensure that we include important 3rd variables in our studies? Pre-existing differences: Experimental groups are different at the beginning. How does randomization help defend against this threat?
Quasi-Experimental Designs: Threats to internal validity in quasi-experimental designs include:
History: Unintended event that takes places between first and second measurements ◦ Maturation: Changes that occur naturally over time ◦ Testing: Change of behavior due to pre-test ◦ Instrument Decay: Characteristics of measurement change over time ◦ Regression Toward the Mean: Extreme scores regress or tend to change toward the mean
Concurrent Validity
How does the measure relate to the criterion of behavior measured at the same time? -Does a measure of shyness relate to shy behavior observation the playground? - Demonstrated by research that examines. the relationship between the measure and a criterion behavior at the same time. -Common method is to study whether two or more groups of people differ on the measure in expected ways.
A confounding variable....
Is a variable that varies along with the independent variable. Occurs when the effects of the independent variable and an uncontrolled variable are intertwined. Cannot determine which of the variables is responsible for the observed effect on the dependent variable.
Dependent Variable
Observe and measure effects that the IV has on the DV. --what changes as a result of something else; the scientist cannot directly change it. --the result or outcome --the bases of your data or analysis
A university is interested in the validity of their teacher application process. They require applicants to give a test lecture, which is evaluated by present students. These applicants are hired and their evaluations correlated with mid-term student evaluations to establish _____.
Predictive Validity
A good experimental design...
Requires eliminating all possible confounding variables that could result in alternative explanations.
Inference
Results are based on a sample. Need inference to determine whether the sample reflects the population. Population = sample + random error.
Inference
Results are based on a sample. Need inference to determine whether the sample reflects the population. Population = sample + random error.
Variability
STANDARD DEVIATION (SD, σ) - Square root of the variance - Average deviation (spread) of scores from x̄ - How spread out scores are in scale units RANGE Difference - between highest and lowest scores - VARIANCE (s2, σ2) Variability of scores around the mean Total similarity or dissimilarity - Large variance = large variability
Presenting Findings
Showing, not just telling, your findings - Quotes - Field notes - Images "A-ha" or "Oh-no" moments - Examples of vivid insight
Pie Chart
Simple descriptions for single variable Useful when representing nominal scale. Most commonly used to depict simple description of categories for a single variable. Used in applied research reports and articles written for the general public.
Limitations
What is wrong with your research? What CAN'T it do? ○ Extraneous variables ○ Researcher effects ■ Researcher bias, effect ○ Participant effects ■ Demand characteristics, Hawthorne effect ○ Biased sampling, Access to populations ○ Cross-sectional ○ Time, budget All research has 'em! You should defend your limitations by.... ● Clarify the scope of your research ● Critically reflect upon your knowledge and your study ● Triangulate your results when possible And limitations can inform....
Longitudinal Method
same group is observed at different times (as they age) similar to the repeated measure design. Less Bias Most famous study: The Terman Life Cycle Study(1921)
4) Members of a risk management team at a substance abuse clinic were interested in finding out whether there were differences in the likelihood that a "near miss" (an error that could have led to a serious problem, but did not) would be reported by individuals who had witnessed the near miss error versus those who had committed the error.
➢ Solution ❖ The independent variable is a categorical variable describing employee status (witness or perpetrator of the error). ❖ The dependent variable is whether the error was reported (yes/no). ❖ Both the independent and dependent variables are categorical (nominal). ❖ The most appropriate type of test is a chi square test. ❖ H0: There will be no differences in likelihood of reporting a near miss (i.e., those who commit the error are as likely to report it as those who witnessed it). ❖ H1: Witnesses will be more likely to report a near miss than employees who commit errors. ❖ Type I Error: Concluding that witnesses (or those who commit the errors) are more likely to report a near miss when in fact, there is an equal likelihood of reporting the problem. ❖ Type II Error: Concluding that witnesses and those who commit an error are equally likely to report it when in reality, one group (probably the witness group) is more likely to report the error.
Qualitative vs. Quantitative
Quantitative APPROACH Explore "facts" about behavior/ phenomenon DATA: Numbers SAMPLE: Large, Random DESIGN: Pre-determined AIM: Generalize ANALYSIS: Software, Statistical inference FINDINGS: Precise, Numerical REPORT: Statistical analyses
What are correlational designs?
A relationship between two or more variables; measured, not manipulated.
Sampling + Saturation
Are you discovering new findings? Interviews: 10-12+ Focus Groups: 6+ Observational: ?? Hours? Locations? Other methods: ?
A group of researchers has developed a new measure of state mindfulness. They correlate it with other mindfulness scales (e.g., Mindful Attention and Awareness Scale), in addition to other constructs like emotional intelligence and perspective taking to establish _____.
Convergent Validity
Type II Errors (False Negative)
Made when the null hypothesis is not rejected when the research hypothesis is actually true. We conclude that the IV does not impact the DV when it actually does. Factors related to making a Type II error: Significance (alpha) level • Setting significance level too low. Sample size • True differences are more likely to be detected with a large sample size. Effect size • Smaller effect sizes are harder to detect (especially in small samples).
Experimental Design: Matched Pairs
People are matched based on score on DV or related variable Example: Participants matched on basis of scores on a cognitive ability measure or GPA in a learning experiment u Ensures groups are equivalent The "matching" process: Participants are rank-ordered on a specific measure Researcher forms matched pairs (e.g., highest two participants form first pair, next two form second, and so on..) Members of each pair randomly assigned to experimental conditions
-Does the measure adequately predict the behavior of interest? -Do SAT scores predict who will succeed in college? -Research that uses a measure to predict some future behavior. -Criteria measure is based on future behavior or outcomes. -Important when studying measures that are designed to improve our ability to make predictions.
Predictive Validity:
What direction is a positive distribution skewed to?
Tail to the right
To achieve a high degree of Internal Validity....
The researcher must design and conduct the experiment so that only the independent variable can be the cause of the results.
Chi-Square
Used to analyze nominal data. Can be used to test hypotheses about one variable (e.g., "Are self-described liberals more likely to be registered as Democrats than Republicans?") Can be used to see if two variables share a relationship (e.g., "Are the spouses of veterans diagnosed with PTSD more likely to be clinically depressed?") Compares the proportions of responses observed in the study with the proportions expected by chance.
Future Directions
What's next? We know that no study can stand alone, so what do your results tell us about what should be done next? How can your study's limitations be addressed in future studies? What new research questions came from your results?
Internal Validity
When the result of an experiment can be attributed to the effects of only the independent variable. exists when confounding variables are sufficiently controlled for and the experimental results are attributed to the effect of the IV.
12) Dr. Poll was interested in examining the effect of political party on views of same sex marriage. She gave groups of Democrats, Republicans, and Independents a survey that asked several questions about their attitudes toward same sex marriage. Scores on the survey ranged from 10 (very negative views) to 100 (very positive views).
➢ Solution ❖ IV: political party (3 levels: Democrat, Republican, Independent) ❖ DV: Attitudes toward same sex marriage ❖ The scale of measurement for the independent variable is nominal. ❖ The scale of the dependent variable is interval/ratio. ❖ The most appropriate type of test is a one-way ANOVA. ❖ H0: No differences between groups on mean attitude scores based on party affiliation. ❖ H1: Mean attitude scores will differ between groups. o Presumably,theresearchersarehypothesizingthatonaverage, Democrats will report the most positive attitudes and the most negative views will be seen among the Republicans with the Independents falling between. ❖ Type I Error: Concluding that individuals registered to one party are more likely to hold more (or less) positive attitudes toward same sex marriage when in truth, there is no relationship between party affiliation and attitudes. ❖ Type II Error: Concluding that members of one party hold more positive attitudes toward same sex marriage than another when the reality is that party affiliation and attitudes are unrelated.
1) To examine the relationship between texting and driving skill in students, a researcher uses orange traffic cones to set up a driving circuit in a parking lot. A group of students is then tested on the circuit, once while receiving and sending text messages, and once without texting. For each student, the researcher records the number of cones hit while driving each circuit.
➢ Solution ❖ IV: text messaging condition (two levels: with or without texting). ❖ DV: number of cones hit. ❖ The scale of measurement for the independent variable is nominal. ❖ The scale of the dependent variable (number of cones hit) is interval/ratio. ❖ The most appropriate type of test is a t-test. ❖ H0: Number of cones hit will not vary by condition (i.e., students will hit the same number of cones while texting as they do when driving without distractions). ❖ H1: Students will hit more cones when texting while driving. ❖ Type I Error: Concluding that texting affects driving when the reality is that distraction does not affect driving skill. ❖ Type II Error: Concluding that texting does not affect students' ability to drive safely when in reality, texting does adversely affect driving performance.
Descriptive vs Inferential Statistics
Average cost of a hotel room in Chicago based on hotel costs in Atlanta. Inferential
Communicating to Scientists
Manuscripts Conference Posters Conference Talks
ANOVA (F-test)
Compares group means (similar to a t-test). Used to analyze interval or ratio data when there are more than two groups in a research design. Researchers generally use a significance level of .05. One-way ANOVA determines whether there are significant between-group differences among three or more groups representing levels of an IV. - Example: Do high school freshman, sophomores, juniors and seniors differ on a scale of social support? Two-way ANOVA examines factorial effects, that is, how two (or more) independent variables affect one dependent variable at the same time. • Assesses main effects and interactions.
Program Evaluation
Determining the merit, value, or worth of programs/policies intended to make positive social change. The process of carefully collecting information about a program or some aspect of a program. In order to improve the program. And make informed decisions about the program.
Researchers are in the process of validating a new scale on deviant work behavior. They ask a random group to take the survey and subsequently ask questions on participants' reactions to the scale. This informs them of their scale's _____.
Face Validity
What statistics use interval data?
Frequency, Chi square, central tendency, Spearman correlation, t tests, ANOVA, Pearson correlations
Independent Variable
Has a minimum of two levels, an experimental group and a control group. Manipulation or control of independent variable. Predictor. --different groups set up for the experiment --what, I the scientist, changes --usually changing the "types" or "amounts" of something
Discriminant Validity:
Is the measure unrelated to variables to which should not be related? -Does the measure show little or no relationship with measures of dissimilar constructs? -Is the correlation between a measure of safety motivation at work and measure of extraversion very small(or non-existent)? -When the measure is not related to variables with which it should not be related.
Alpha Level
Probability required for significance. Most common alpha-level probability used is p= .05
Reliability in Qual Research
Trustworthiness & Confidence Inter-Rater: Different researchers, different interpretations
Causality
occurs if variation in the independent variable is followed by variation in the dependent variable, when all other things are equal. 3 criteria to establish
Type I and Type II Errors
■ In reality, H0 is either true or false. ■ Based on data, research makes a decision or reject or fail to reject H0. ■ Errors can occur.
Tom Rogers wanted to test a new "singalong" method to teach math to fourth graders (e.g., "I love to multiply" to the tune of "God Bless America"). He used the singalong method in his first period class. His sixth period students continued solving math problems with the old method. At the end of the term, Mr. Rogers found that the first period class scored significantly lower than the sixth period class on a mathematics achievement test. He concluded that the singalong method was a total failure.
1. What are the independent variable (including its levels) and the dependent variable in this study? Answer: IV: Teaching method (singalong vs. traditional) DV: Scores on the mathematics achievement test 2. What type of design and assignment procedures were used in this study? Answer: Repeated measures 3. Can you identify one or more threats to internal validity in this experiment? • Time of day is the most obvious confounding variable. o Either the students or the teacher (or both) may be unprepared for singing first thing in the morning. o Students may find it difficult to pay attention first thing in the morning. o The teacher may be less alert and less engaging at that hour. • Use of a non-equivalent control group. o More remedial students may be assigned to mathematics during first period while a higher proportion of gifted students may be assigned during sixth period. o Truancy may be more prevalent during first period . 4. How could the experimenter redesign the study to address these problems? • Use a pretest/posttest design. • Run the experiment on multiple classes meeting at different times of day. • Have multiple teachers run the same experiment.
An investigator was interested in studying the effect of taking a course in child development upon attitudes toward childrearing. At the end of the semester, the researcher distributed a questionnaire to students who had taken the child development course. Questionnaires were also given to an equal number of students who had not taken the course. The students who had taken the child development course had different attitudes from the students who had not taken the course (e.g., they had more positive attitudes about having large families).
1. What are the independent variable (including its levels) and the dependent variable in this study? IV: Exposure to a child development class vs. no exposure (at least, that was the plan). Levels:Students enrolled in a particular child development course during a specific term vs. those who were not. • DV: Attitudes toward child rearing 2. What type of design and assignment procedures were used in this study? Check all that apply. Answer: X Posttest only X Independent groups 3. What are the potential problems with this experimental design and the procedures used? • Potentially non-equivalent groups (no random assignment), possible exposure to child development courses in other classes or during different terms. • Given that the students were self-selected into groups, it is very likely that those who choose to take a class in child development already have more positive attitudes toward childrearing (e.g., wanting large families) and any differences observed are not due to exposure to child development classes. 4. What are some ways that the design and procedures could be revised to address those problems? • Use random assignment. • Use a pretest/posttest design
An airport administrator investigated the attention spans of air traffic controllers to determine how many incoming flights the average controller can coordinate at the same time. Each randomly selected controller was tested, without his or her knowledge, by a computer program that fed false flight information to a computer terminal. The controller first "received" information from one plane, and by the end of an hour the controller was coordinating 10 planes simultaneously. The administrator analyzed the errors collected by the computer program. The analysis revealed that the maximum number of planes a controller could handle without making potentially fatal errors was six planes. Also, no errors occurred when only one to three planes were incoming. He concluded that a controller should never coordinate more than six incoming flights.
1. What are the independent variable (including its levels) and the dependent variable in this study? IV: Number of incoming flights the controllers handled DV: Number of errors made by the controllers 2. What type of design and assignment procedures were used in this study? Answer: Repeated measures 3. What are some potential problems with this experimental design and the procedures used? ➢ The introduction of additional incoming flights is confounded with time and (potentially) fatigue. o With the linear increase in number of flights, there may be a practice effect. Increases and decreases in number of flights is more random in reality. There may be a problem with ecological validity and therefore, generalizability. Fatigue may reduce the number of flights that controllers can handle during the second half of the hour. 4. What are some ways that the design and procedures could be revised to address these problems? The number of flights the controller must coordinate should vary randomly throughout the hour's trial. The fatigue variable could be examined by looking at the errors across the hour interval; this would create a mixed factorial design with number of flights as the independent variable and the amount of time elapsed as the repeated measure.
Professor Foley conducted a cola taste test. Each participant in the experiment first tasted 2 ounces of Coca-Cola, then 2 ounces of Pepsi, and finally 2 ounces of Sam's Choice Cola. A rating of the cola's flavor was made after each taste.
1. What are the independent variable (including its levels) and the dependent variable in this study? IV: Type of cola (Coca-Cola, Pepsi, and Sam's Choice Cola) DV: Flavor ratings 2. What type of design and assignment procedures were used in this study? Repeated Measures 3. What are some potential problems with this experimental design and the procedures used? • Order effect: Order of presentation may have influenced preferences. • Demand characteristics: If colas were labelled, participants' ratings may have been influenced. • Experimenter effects: Experimenter may unconsciously signal expectations to participants. • Ecological validity: Were the conditions for the taste test similar to the conditions participants usually experience when drinking sodas (e.g., the soda may have been warm)? 4. What are some ways that the design and procedures could be revised to address those problems? What disadvantages need to be considered for each solution? • Order effect: Randomize the order in which the colas are tasted. o Very few disadvantages (slightly inconvenient for the researcher, more potential for errors in recording data). • Demand characteristics: Remove the labels. o Similar to above. There is a chance that the greater complexity may lead to errors. • Experimental effects: Use a double blind design. o More complex. Requires more resources. • Ecological validity issues: Find out how participants typically drink their colas and serve it the same way. At a minimum, refrigerate the colas. o Greater complexity, may make it difficult to maintain standardized administration protocols.
Researchers are interested in the effects of patterns of TV watching on children's aggressive behavior. They have kids keep a diary of what they are watching and for how long and then compare it to school reports of aggressive actions. They find that the more aggressive TV a child watches the more aggressive schools report they are.
1. What are the independent variable (including levels) and the dependent variable in this study? IV: TV content viewed DV: Instances of aggressive behavior at school 2. What type of design and assignment procedures were used in this study? Check all that apply. Answer: X Correlational design 3. How might experimenter effects or demand characteristics influence the results of this study? What could the researcher do to protect against these effects? Because data recording is completed by the participants at home, away from the experimenters, this design is less vulnerable to experimenter effects. There is a potential for experimenter effects if the instructions given to the children are inconsistent or the coding for TV shows viewed by the children is not standardized. Careful training of experimenters to assure consistency when interacting with participants and coding TV viewing are essential. Demand characteristics are more problematic. Simply asking children to record their television viewing habits may sensitize to the content they are viewing which could in turn change their viewing behavior and possibly even their aggressive behavior. This flaw is not easily countered. 4. Do you see any other flaws in the design for this study? If so, what are they and how do you think the design and/or procedures could be revised to address those problems? There are concerns about accuracy of data recording and testing effects. Children may be unable or unwilling to accurately record their television watching and/or may become less accurate over time as the novelty wears off. Careful explanations and training are needed. Researchers would need to check in fairly frequently with participants to keep engagement levels high. Keeping the duration fairly short would also help (although that creates a new set of problems). What constitutes "aggressive" vs. "non-aggressive" TV needs to be carefully defined. Ditto for aggressive actions at school. Finally, there are questions about direction of causality that are not easily addressed.
Dr. Mnemonic is interested in the effect of type of questioning on memory, she has several subjects watch a video of car accidents and then asks some of the subject's leading questions like "was the car red?" and others are asked open ended questions like "what color was the car?" she then looks at how people remember events based on question type.
1. What are the independent variable (including levels) and the dependent variable in this study? IV: Type of questions asked (leading vs. open ended) DV: Accuracy of recollection 2. What type of design and assignment procedures were used in this study? Check all that apply. Answer: X Posttest only X Independent groups 3. How might experimenter effects influence the results of this study? What could the researcher do to protect against these effects? The experimenter may unintentionally give subtle cues to the respondent. This is particularly likely to occur when she asks leading questions. To protect against this effect, the researcher could practice administering the protocol repeatedly until she is easily able to maintain consistency in presenting the questions to all participants. Using multiple well-trained experimenters would provide additional protection. Finally, well- trained interviewers who are not aware of the goals of the study could be employed. 4. Do you see any other flaws in the design for this study? If so, what are they and how do you think the study could be revised to address those problems? Experimenter effects are the main concern here. However, there are some concerns with generalizability. Car accidents are very dramatic events, which may influence ability to recall details. Different patterns may be observed with more mundane events. Multiple videos showing a variety of situations could be used to address this concern.
A drug company developed a new medication to control the manic phase of bipolar disorder. The firm hired a hospital psychiatrist to test the effectiveness of the drug. He identified a group of patients with bi-polar disorder and randomly assigned them to a drug or placebo group. Nurse Ratched was told to administer the drug and Nurse Johnson was told to administer the placebo. Each nurse made daily observations of her patients during treatment. A month later the observations were compared. In general, patients in the drug group had behaved more "normally" than patients in the placebo group. The drug company publicized its product's effectiveness.
1. What are the independent variable (including levels) and the dependent variable in this study? Answer: IV: Treatment condition (drug vs. placebo) DV: Observations of the patients 2. What type of design and assignment procedures were used in this study? Check all that apply. Answers: Posttest only Independent groups 3. How might experimenter effects influence the results of this study? Answer: Nurse Ratched makes all the observations for the drug group while Nurse Johnson makes all the observations for the placebo. There may be systematic differences in the way they record and interpret patient behaviors. This is especially troublesome because each observer knows the group membership of the patient being observed, thereby increasing the probability of observer effects. Additionally, if one observer consistently interprets behavior differently from the other, the resulting difference may have nothing to do with the variable but only represents scoring bias. 4. What could the researcher do to protect against these effects? • Provide observers with detailed definitions for behaviors of interest. Train the observers carefully. • Both nurses should make observations on all patients. o If that is not feasible, include observations from a third observer, with data to be gathered at random intervals. o Use Cohen's Kappa to examine inter-rater reliability. • Use a double-blind design (i.e., neither nurse would know the group membership of the patients).
Important difference between true experiments and quasi-experiments.
Control - True Experiment: random assignment = High Internal Validity - Quasi-Experiment: no random assignment: more external validity
The Experimental Method involves....
Control over extraneous variable, through either keeping such variables constant (experimental control) or using randomization to make sure that any extraneous variables will affect both groups equally. Casual inferences can be made through experimental control and randomization.
Validity in Qual Research
Credibility (Internal) & Transferability (External) Consensual Validity: everyone is agreeing on the same thing. Triangulation: 3 or more sources of data or research in a single study. Many studies combine both interprevists and positivists methods. Member Checking: The researcher checks the findings and interpretation with the original respondents. This could take place either at the end of research, providing participants with information that ensures their views have been properly captured, or during the research process - here participants can help design questionnaires or interview guidelines, thus being seen as co-researchers.
Basic Experimental Designs: Pretest-Posttest
Measure experimental/control group on the DV before the IV is introduced. Pretest given to each group prior to the experimental manipulation (for experimental group). Assures groups are equivalent in the beginning of the experiment, regardless of group assignment. Used to measure changes that occur due to the "treatment" (experimental manipulation) Advantages: Within subjects change over time can be measured Mortality (dropout) can be assessed for causes and effects of attrition Can be used to select participants for further experimentation Stratified random sample: High/low scorers are stratified and randomized to reduce bias Disadvantages: Time consuming and awkward to administer Sensitized participants to what is being studied Demand characteristics Hypothesis guessing, Hawthorne effect Pre-test should be disguised or embedded among other measures in study
Program Evaluation
Step 1: Identify Goals and Desired Outcomes: Goals: What a project is striving for. Outcomes: Specific targets to be measured Identify: Primary Goals Target group(s): Demographics, developmental transitions, risk processes, locality Desired outcomes: Clearly defined and specific, realistic and attainable, measurable. This step involves a needs and resource assessment. Step 2: Monitors program activities: Organize program efforts Ensure that all parts of the program are conducted as planned Use program resources where they are needed Provides an index of accountability Useful for administrators, funders, directors, stakeholders Provides information about why program worked or not Helpful to review after the outcome evaluation is completed. Determines when outcome evaluation should start Keeps track of changes in conditions that affect implementation of the program (context) Who was supposed to do what with whom and when was it done? Who - delivered services, how many staff, qualifications needed. What - activities the staff actually did. Whom - the target group for each activity. When - time and setting of the activity. Step 3: Outcome Evaluation: Assesses the immediate effects of a program Determine the degree of effectiveness of the planned intervention. Should be planned from the beginning of project. Data collection is often as important as the service for obtaining funding. Common types of outcome measures: Self-report questionnaires Parent or teacher ratings Key informants Observation Step 4: Impact Evaluation Concerns the ultimate effects desired by the program: What are the long-term effects of the program? For example, reduction in overall drug use (prevalence) or new cases of drug use (incidence) Usually requires archival data to measure impact For example, court records, police records, school grades, attendance, medical records, employment income, etc.
Experimental Groups:
Treatment & Control Treatment group (aka experimental group): Group receives "treatment" or manipulation of the IV Allows for comparison of outcome measures (DVs) against the control group. Allows for determination that IV does have effect on DV. Control group: Does not receive any "treatment" (or IV manipulation). Placebo group. Serves as the standard of comparison.
18). A t eam of researchers were interested in examining the relationship between stress tolerance (hardiness) and physical health. Survey data were gathered assessing hardiness (possible scores ranged from 24 to 120) and physical health (scores ranged from 36 to 149 with higher scores associated with better overall health/fewer illnesses).
❖ Predictor: Stress tolerance. Continuous ❖ DV: Physical Health. Continuous ❖ The most appropriate type of test is correlation ❖ H0: No relationship between stress tolerance and physical health ❖ H1: Positive linear- relationship between stress tolerance and physical health ❖ Type I Error: Concluding that there is a relationship between stress tolerance and physical health when there is not one ❖ Type II Error: Concluding that there is not a relationship between stress tolerance and physical health when there is one
5) Susan Sound predicts that students will learn most effectively with a constant background sound, as opposed to an unpredictable sound or no sound at all. She randomly assigns each of the 30 student participants into one of three groups. All students study a passage of text for 30 minutes. Those in Group 1 study with background sound at a constant volume in the background. Those in Group 2 study with noise that changes volume periodically. Those in Group 3 study with no sound at all. After studying, all students take a 10 point multiple choice test over the material.
➢ Solution ❖ IV: background sound (3 levels: constant volume, variable volume, no sound) ❖ DV: scores on the multiple choice test ❖ The scale of measurement for the independent variable is nominal. ❖ The scale of the dependent variable is interval/ratio. ❖ The most appropriate type of test is a one-way ANOVA. ❖ H0: There will be no differences in test scores based on sound level. ❖ H1: Sound level during study time will influence test scores. o Presumably,theresearcherispredictingthatthelowestscoreswill occur in the variable sound condition and the highest scores in the silence condition with the constant volume condition scores falling between. ❖ Type I Error: Concluding that sound level individuals are exposed to during study time influences test scores when in fact, it does not. ❖ Type II Error: Concluding that sound level during study time does not affect performance (ability to learn the material) when in reality, it does.
10) A statistics instructor thinks that doing homework improves scores on statistics exams. To test this hypothesis, she randomly assigns students to two groups. One group is required to work on the homework until all problems are correct, while homework is optional for the second group. At the end of the term, exam scores are compared between the two groups.
➢ Solution ❖ IV: homework condition (two levels: all problems correct, homework optional) ❖ DV: score on the final exam ❖ The scale of measurement for the independent variable is nominal. ❖ The scale of the dependent variable is interval/ratio. ❖ The most appropriate type of test is a t-test. ❖ H0: Mean exam scores will not differ between the homework required and homework optional groups. ❖ H1: Mean exam score will be higher in the homework required group. ❖ Type I Error: Concluding that requiring homework (vs. making it optional) influences exam scores when it does not. ❖ Type II Error: Concluding that exam performance is the same across groups when in truth, those required to turn in homework perform better.
13) Childhood participation in sports, cultural groups, and youth programs appears to be related to improved self-esteem for adolescents. In a representative study, a researcher compares scores on a self-esteem questionnaire for a sample of 100 adolescents with a history of group participation and a separate sample of 100 who have no history of group participation. Scores on the self-esteem measure range from 10 to 50.
➢ Solution ❖ IV: social group participation condition (two levels: history of group participation or no history) ❖ DV: score on a self-esteem questionnaire ❖ The scale of measurement for the independent variable is nominal. ❖ The scale of the dependent variable is interval/ratio. ❖ The most appropriate type of test is a t-test. ❖ H0: No differences in mean self-esteem scores between the community participation and no community participation groups. ❖ H1: Mean self-esteem scores will be higher in the community participation group. ❖ Type I Error: Concluding that adolescents who participate in community programs experience higher levels of self-esteem when in reality, self-esteem is not influenced by community participation. ❖ Type II Error: Concluding that participation in community programs does not enhance self-esteem when the truth is that it does.
17) Researchers working with military spouses were interested in examining the impact of participant depression, perceptions of warrior spouse PTSD symptoms, and social support on marital satisfaction. Participants completed a survey with questions about their own depression (possible scores ranged from 0 to 60), their perceptions of the spouse's PTSD symptoms (possible scores ranged from 17 to 85), adequacy of the social support they received (possible scores ranged from 12 to 60) and marital satisfaction (possible scores ranged from 0 to 45).
➢ Solution ❖ IVs: participant depression, warrior spouse PTSD symptoms, social support. All continuous. ❖ DV: Marital Satisfaction. Continuous. ❖ The most appropriate type of test is multiple regression ❖ H0: No relationship between participant characteristics (depression, warrior spouse symptoms, and social support) and marital satisfaction. ❖ H1: One or more of the participant characteristics variables is associated with marital satisfaction. ❖ Type I Error: Concluding that one or more of the IVs influence marital satisfaction when if fact, there is no relationship. ❖ Type II Error: Erroneously concluding none of the IVs is associated with marital satisfaction.
14) A researcher was interested in finding out if choice of medical specialty differed by ethnicity. She surveyed fourth year medical students and asked them to provide information about their ethnicity and chosen specialty area (cardiology, primary care, internal medicine, or orthopedics).
➢ Solution ❖ The independent variable is ethnicity. ❖ The dependent variable is specialty area (cardiology, primary care, internal medicine, and orthopedics). ❖ Both the independent and dependent variables are categorical (nominal). ❖ The most appropriate type of test is a chi square test. ❖ H0: No differences in proportions of students choosing a given specialty area based on ethnicity. ❖ H1: Different proportions (percentages) of members of various ethnic groups will be drawn to one or more of the various specialty areas. ❖ Type I Error: Concluding that ethnicity and preferences for one medical specialty over another are related when in fact, they are not. ❖ Type II Error: Erroneously concluding that ethnicity is not related to which specialty area an individual is more likely to be drawn to.
6) Final year psychology students were asked about their career plans. The researchers want to investigate if there is any relationship between gender and career preference. They found that 12 females and 26 males wanted to work in the field of clinical psychology, while 24 females and 8 males said they preferred the area of organizational psychology.
➢ Solution ❖ The independent variable is gender (male, female). ❖ The dependent variable is the career preference (clinical, organizational). ❖ Both the independent and dependent variables are categorical (nominal). ❖ The most appropriate type of test is a chi square test. ❖ H0: There will be no differences in proportions of males and females preferring each specialty area. ❖ H1: There will be differences in proportions of males and females reporting a preference for one or more specialty areas. ❖ Type I Error: Concluding that there are gender differences in choices of specialty areas when in fact, gender does not influence preferences. ❖ Type II Error: Concluding that there are no gender based differences in preferences for clinical vs. organizational psychology when the truth is that there are.
8) Researchers were interested in examining the effectiveness of a safety judgment scale for predicting safety compliance (the likelihood that an employee will follow proper safety protocols). To obtain a safety judgment score, participants were asked to read 5 safety-related scenarios and asked how they would respond to each. Responses were coded as correct or not correct and all items were summed. Possible scores on the safety judgment scale ranged from 0 to 5. Scores on the safety compliance scale ranged from 3 to 15.
➢ Solution ❖ The independent variable is safety judgment. ❖ The dependent variable is safety compliance. ❖ The measurement scale of the both the independent and dependent variables is interval/ratio (i.e., continuous variables). ❖ The most appropriate type of test is correlation. ❖ H0: There will be no relationship between safety judgement scores and safety compliance scores. ❖ H1: On average, people who score higher on the safety judgement scale will also display higher scores on the safety compliance scale. ❖ Type I Error: Concluding that people who score high on the safety judgement scale are more (or less) likely to comply with safety procedures when in truth, there is no relationship. ❖ Type II Error: Concluding that safety judgement does not influence safety compliance when in fact, it does.
9) Substance abuse treatment researchers were interested in examining the relationship between counselor characteristics and level of care recommendations. Predictor variables included amount of experience working with substance abuse clients (ranging from 0 to 25 years), formal training in co-occurring chemical dependency/mental health disorders (versus absence of formal training), and recovery status of the counselor (i.e., in recovery versus no personal addiction history). The level of care continuum was operationalized as a continuous variable ranging from 1 to 9.
➢ Solution ❖ The independent variables are: (a) years of experience, (b) training, and (c) recovery status. ❖ Years of experience is an interval/ratio level measure. ❖ Training and recovery status are categorical (nominal) variables. ❖ The dependent variable is level of care recommended. ❖ The measurement scale for the dependent variable is interval/ratio. ❖ The most appropriate type of test is multiple regression. ❖ H0: There will be no relationship between scores on the predictor variables (experience, training in treating co-occurring disorders, and recovery status) and the criterion variable (level of care recommendations). ❖ H1: At least one of the predictor variables will be associated with differences in level of care recommendations. ❖ Type I Error: Concluding that none of these counselor characteristics influence level of care recommendations when the reality is that they do. ❖ Type II Error: Concluding that one or more of these counselor characteristics influence level of care recommendations when they do not.
2) A researcher was interested in investigating the relationships between personal characteristics of medical students and their attitudes toward patients with substance abuse issues. She had medical students complete a survey designed to measure their attitudes toward substance abusing patients. Total scores on the survey ranged from 5 to 20 with higher scores indicating more negative attitudes. Information about demographic characteristics (age and gender) and substance use behaviors (binge drinking and drug use) were also collected. Response options for the substance use behavior questions were yes/no.
➢ Solution ❖ The independent variables are: age, gender, substance use behavior among participants. ❖ The age variable is continuous (interval/ratio measurement scale). ❖ The gender variable is categorical (nominal). ❖ The substance use behavior variables (binge drinking and drug use) are categorical (nominal). ❖ The dependent variable is attitude toward patients with substance abuse issues. The measurement scale is interval/ratio. ❖ The most appropriate type of test is multiple regression. ❖ H0: No relationship between participant characteristics (age, gender, and substance use behaviors) and attitudes toward substance toward patients with substance abuse issues. ❖ H1: One or more of the participant characteristics variables is associated with more negative attitudes toward patients with substance abuse issues. ❖ Type I Error: Concluding that one or more of the personal characteristics of medical providers tested in this study (age, gender, and substance use behavior history) influence attitudes toward patients with substance abuse issues when if fact, there is no relationship. ❖ Type II Error: Erroneously concluding that at least one of these provider characteristics is associated with more negative attitudes.
Qualitative analysis
APPROACH: Understand behavior/ phenomenon from informant's perspective DATA: Words, Images not numbers SAMPLE: Small, Non random DESIGN: Flexible AIM: Understand depth ANALYSIS: Researcher as instrument, Themes FINDINGS: Comprehensive, Richly descriptive REPORT: In language of the informant
If groups are different....
The researcher can conclude the independent variable caused the results.
Manipulation
The researcher manipulates one variable by changing its value to create a set of two or more treatment conditions. -"Doing something" to at least some of the subjects -The independent variable is manipulated when some subjects (experimental group) receive the intervention and others (control group) do not.
16) Previous research has indicated that smoking is associated with low birth weight. An OBGYN is interested in finding out how strong that relationship is. She gathers data from clinic patients on the average number of cigarettes smoked per day during pregnancy and records the weight of the baby at birth.
➢ Solution ❖ The independent variable is the average number of cigarettes smoked per day during pregnancy. ❖ The dependent variable is baby's birth weight. ❖ The measurement scale for the both variables is interval/ratio (i.e., continuous variables). ❖ The most appropriate type of test is correlation. ❖ H0: No relationship between number of cigarettes per day smoked during pregnancy and infant birth weight. ❖ H1: Weight of newborns born to mothers who smoked more cigarettes per day during pregnancy will be lower. ❖ Type I Error: Concluding that number of cigarettes smoked per day during pregnancy does not influence birth weight when the truth is that smoking has a negative effect on birth weight. ❖ Type II Error: Concluding that smoking has a negative effect on infant birth weight when in reality, there is no relationship.
7) Darley and Latané (1969) were interested in whether the presence of other people influences whether a person will help someone in distress and how long it will take. The experimenter (a female graduate student) had the participant wait in a room with 0, 2, or 4 confederates. The experimenter announces that the study will begin shortly and walks into an adjacent room. A few moments later the person(s) in the waiting room hear her fall and complain of ankle pain. The dependent measure is the number of seconds it takes the participant to help the experimenter.
➢ Solution: ❖ IV: number of other people present (3 levels: 0, 2, or 4) ❖ The scale of measurement for the independent variable is ordinal (equivalent to nominal for our purposes here). ❖ DV: number of seconds until the participant moves to help. ❖ The scale of the dependent variable is interval/ratio. ❖ The most appropriate type of test is a one-way ANOVA. ❖ H0: Average number of seconds it takes to respond will be the same regardless of the number of people in the waiting area. ❖ H1: Average response time will be longer when there are more people in the waiting area. ❖ Type I Error: Concluding that average response time differs based on the number of people in the waiting room when if fact, it does not. ❖ Type II Error: Concluding that there are no differences in response times when in fact, people respond more slowly when others are present.
15) A researcher was interested in examining the combined effects of party affiliation and gender on attitude toward tax cuts. She gathered survey data from male and female voters registered as Democrats, Republicans, or Independents. The survey consisted of 10 items asking respondents to rate how favorably or unfavorably they would view different types of tax cuts. Items were rated on a scale of 1 = very unfavorably to 5 = very favorably. All items were summed to obtain a total score (possible scores ranged from 10 to 50).
➢ Solution ❖ There are two factors, or independent variables: (a) party affiliation with 3 levels (Democrat, Republican, Independent) and (b) gender (male or female). ❖ The scale of measurement for both independent variables is nominal/ordinal (i.e., categorical). ❖ The dependent variable is attitudes toward tax cuts. ❖ The scale of the dependent variable is interval/ratio (i.e., a continuous variable). ❖ The most appropriate type of test is a two-way (factorial) ANOVA. ❖ H0: There will be no differences in attitudes toward tax cuts based on gender and party affiliation. ❖ HA: There will be differences in attitudes toward tax cuts based on gender and/or party affiliation. o H1 (Party affiliation main effect): On average, Republicans will report the most positive attitudes toward tax cuts, Independents will fall in the middle, and Democrats will report the least positive attitudes. ▪ Note: This is one possibility. There are several other possible predictions. o H2 (Gender main effect): Mean scores for attitudes toward tax cuts will be higher among males. ▪ You could also predict the opposite, depending on what the existing evidence suggests. o H3 (Party x Gender Interaction): Gender and party affiliation will interact with Independent voters showing a different pattern of relationships across genders than other groups. ▪ Again, there are other possibilities here. Perhaps you think that male Democrats will show more positive attitudes than female Republicans. The most basic alternative hypothesis would be that there is some sort of interaction between gender and party affiliation.
Internal Validity
Ability to determine causation from design: -temporal order -covariation between variables -No plausible alternative -Are there alternative explanations?
Basic Experimental Designs: Posttest Only
Two groups of equivalent participants: Random assignment to condition Individuals participate in the experiment Manipulation of and exposure to IV Individuals are measured on the DV Differences between groups on the DV are attributed to the IV.
t-test
Used to determine whether the mean of one group is significantly different from that of a second group. For example, does the mean of a control group differ from the mean of an experimental group? Small t-value indicates that sample means are similar.
Type I Errors (False Positive)
Made when the null hypothesis is rejected but the null hypothesis is actually true. Our decision was that the IV had an effect on the DV when it actually did not. Probability of making a Type 1 error is determined by the choice of significance or alpha level (α). When the significance level for deciding whether to reject the null hypothesis is .05, the probability of a Type 1 error is .05 That is, there is a 5% chance that the decision is wrong. This can be changed by decreasing or increasing the significance level For example, .1, .01, or .001).
Experimental Methods: Basic Design
One DV (aka outcome variable): Measured posttest or pretest-posttest One IV with a minimum of two levels: Treatment/Experimental group (level 1) Control group (level 2) Involve random assignment to groups and the control of confounding variables. Random assignment vs. Random selection Selection: How participant samples are selected from the population for inclusion. Assignment: How participants are randomly assigned to treatment or control group.
Histogram
Bars to display frequency of quant variable - Continues scales
Which of the statements below describes a population, not a sample? ■ In a national survey on substance abuse, 10% of respondents ages 12-17 reported using illicit drugs in the past month. ■ Ty Cobb is one of the MLB's greatest hitters, with a career average of .366. ■ A study concluded of 6076 adults in public restrooms (in Atlanta, Chicago, NYC, SF) found that 23% did not wash their hands before exiting.
■ Ty Cobb is one of the MLB's greatest hitters, with a career average of .366.
11) A researcher investigated different combinations of temperature and humidity to examine how heat affects performance. The researcher compared three temperature conditions (60°, 70°, and 80°) with a high humidity and a low humidity condition for each temperature. A separate group of participants was tested in each of the 6 different conditions, and for each participant, the researcher recorded the number of errors made on a problem-solving task.
➢ Solution ❖ There are two factors, or independent variables: (a) temperature condition with 3 levels (60, 70, or 80 degrees) and the humidity condition with 2 levels (low or high). ❖ The scale of measurement for both independent variables is nominal/ordinal (i.e., categorical). ❖ DV: Number of errors made on a problem-solving task. ❖ The scale of measurement for the dependent variable is interval/ratio (aka continuous). ❖ The most appropriate type of test is a two-way (factorial) ANOVA. ❖ H0: Mean number of errors will be the same across all 6 conditions. ❖ H1: Mean number of errors will be higher in some conditions than others. o Presumably,theresearchersareexpectingthehighestnumberof errors in the 80°/high humidity group and the lowest in the 60°/low humidity group with the others falling between. ❖ Type I Errors: Since it is possible to have two main effects on one interaction for this design, there are three possible Type I Errors. o Erroneously concluding that temperature affects task performance. o Erroneously concluding that humidity affects task performance. o Mistakenly concluding that there is an interaction between temperature and humidity when there is not (i.e., assuming that the effect of temperature is contingent upon the level of humidity or vice versa when in truth, there is no interaction). ❖ Type II Errors: Three different Type II Errors are possible. o Concluding that temperature affects task performance when it doesn't. o Concluding that humidity influences task performance when it doesn't. o Erroneously concluding that there is an interaction between humidity and temperature (e.g., concluding that humidity adversely affects performance only at high temperatures when in reality, it always influences performance).
Experimental Design: Group design
Between-Subjects Design: Also called independent groups design Participants randomly assigned to and participate in only one group. Within-Subjects Design(Repeated Measures): Also called repeated measures design Each participants experiences all conditions in the study Participants are measured on the DV after involvement in each group/condition Participants' changes on the DV are compared to themselves, as opposed to other participants in the study. Advantages: Fewer participants needed Extremely sensitive to statistical differences because data in all conditions is from the same people Less random error as a result of participant differences Conditions are identical because person is own control group Disadvantages: Order effects: Order of presenting the treatment conditions affects the DV. Specific order effects: Practice effect Fatigue effect Carryover effect How to deal with order effects? Time intervals between treatments Rest period counteracts fatigue Attending to unrelated task reduces practice effects Waiting day or longer to reduce carryover might be necessary Counterbalancing All possible orders of presentation are included in the experiment Can assess the extent to which order influences the results.
A scholastic organization is creating a new mathematics aptitude test. They administer the test along with aptitude tests on reading, writing, biology, and art. Scores on the math test are correlated with scores on the other subjects to establish _____.
Discriminant Validity
3) Researchers interested in the effect of moderate intoxication on driving performance recruited 40 drivers. Half of the drivers drove a driving simulator after consuming a low dose of alcohol and half had not consumed any alcohol. Additionally, half of the participants in each of these groups were highly experienced drivers, while half were relatively inexperienced. The researchers measured the drivers' reaction time to a stimulus presented in their peripheral vision (with longer reaction times indicating worse performance).
➢ Solution ❖ There are two factors, or independent variables: (a) alcohol condition (2 levels: alcohol consumed or not) and (b) the experience condition (2 levels: low or high). ❖ The scale of measurement for both independent variables is nominal/ordinal (i.e., categorical). ❖ The dependent variable is the reaction time to peripheral stimuli. ❖ The scale of measurement for the dependent variable is interval/ratio (aka continuous). ❖ The most appropriate type of test is a two-way (factorial) ANOVA. ❖ H0: Reaction times will be similar across all conditions. ❖ HA: Reaction times will differ across conditions. o H1 ( Alcohol main effect): Mean reaction times will be higher in the groups who consumed alcohol. o H2 (Driving experience main effect): More experienced drivers will exhibit shorter reaction times. o H3 (Alcohol x Experience Interaction): The effect of alcohol on reaction times will be more pronounced among inexperienced drivers. ❖ Type I Errors: o Erroneously concluding that alcohol slows down reaction times. o Erroneously concluding that more experienced drivers react faster. o Mistakenly concluding that imbibing alcohol will have less effect on reaction times among more experienced drivers when in truth, the effect on reaction times is the same. ❖ Type II Errors: o Failing to detect a real difference in reaction times between the group that imbibed alcohol and those who did not (i.e., concluding that alcohol doesn't slow reaction times when it really does). o Failing to detect a true difference in reaction times between experienced and inexperienced drivers (i.e., finding that experience does not influence reaction times when the truth is that more experienced drivers react faster). o Mistakenly concluding that the effects of alcohol are not moderated by experience (i.e., concluding that the effects of alcohol on reaction times are the same regardless of experience level when in truth, the effects of alcohol are more muted among experienced drivers).
