Research Methods Final exam
case study
an observation technique in which one person is studied in depth in the hope of revealing universal principles. NOT generalizable Can involve qualitative or quantitative approaches
ratio scale of measurement
All of the same in interval. 1. EQUAL units of measurement. 2. TRUE ZERO YES. Ex. height, weight, time on task, income, age. number of correct answers on test.
multiple regression
a statistical technique that can be used to analyze the relationship between a SINGLE DEPENDENT variable and SEVERAL INDEPENDENT variables.
dichotomous variable
a variable that has only two values. i.e. MALE or FEMALE.
Archival data
Data initially collected for some other purpose and then analyzed often for another purpose in research. i.e. looking at mental health institution records in 1900's to predict depression rates during that time
Main effects from graph B?
If the dots are at different points then YES main effect. if dots are on top of each other than NO main effect.
Correlation
Positive and negative associations
Issues with Within Subjects Design
Sequence/order effects, progressive effects, carryover effects
matching variable
a characteristic on which we match sets of individuals as closely as possible. i.e. matched on level of depression (variable)
field experiment
an experiment conducted in the participants' natural environment. HIGH CONTROL
two meanings of "multivariate"
can mean analyzing multiple measures. OR analyzing multiple VARIABLES
observation with intervention
participant observation, structured observation, field experiment. something in the environment is manipulated. SOME CONTROL
Attributes of a good theory
productivity, falsification, parsimony
interaction without main effect
you don't always need main effects when there is an interaction.
multivariate analysis
(MVA) is a Statistical procedure for analysis of data involving more than one type of measurement or observation. It may also mean solving problems where more than one dependent variable is analyzed simultaneously with other variables.
Acculturation
(n.) the modification of the social patterns, traits, or structures of one group or society by contact with those of another; the resultant blend. adopting a new culture into one's existing culture
Assumptions of t-test
-Values in the sample are independent observations. -The population sampled must be normally distrubeted. -With large samples, this assumption can be violated without affecting the validity of the hypothesis test.
Purpose of ANOVA
-to measure if Between Group Variance is indeed "substantially" larger than Within Group Variance - BLANK is helpful for testing three or more variables.
Kaysen, Losutter, and Goines (2005):
1. CPT for gay hate cime 2. Natural recovery could have played a role in John's symptom improvement. John was seen within the time period during which victims of traumatic violence tend to see the most spontaneous improvement. 3. IV's = PTSD symptoms DV's = BDI, and other measures used.
true experiment
1. Determine a cause and effect between two variables, where all variables are consistent 2. Manipulated the IV 3. Each subject has equal probability of being selected for either the treatment or control group
successive independent samples design
1. Different samples from the same population collected over time 2. Facilitates studying changes in a population (not individuals) over time 3. Problem = when samples are not comparable
quasi-experiment
1. IV is NOT manipulated 2. Examines differences between pre-existing groups of subjects or between pre-existing conditions 3. Variable that is used to differentiate the groups is called a quasi-independent variable (ex: comparing ethnicity and gender)
Factors affecting power
1. LOWER ALPHA LOWER POWER 2. one-tailed vs two-tailed t-test. less extreme results can be used to draw conclusions but ONLY in one direction. 3. SAMPLE SIZE . LARGER SAMPLE = higher power SMALLER SAMPLE= lower power. 4. confounding variables. LESS confounding variables MORE POWER. 5. participant characteristics. IF GROUPS are VASTLY different than LOWER POWER. 6. STUDY DESIGN.
time-series design controls several threats of internal validity
1. Maturation - cognitive or physical changes occur in participants 2. Testing - repeated tests usually leads to better scores on same test 3. mortality - attrition 4. regression - extreme scores (high/low) on first test. 5. selection- groups were not comparable at start of study. 6. instrumentation- measures aren't appropriate. CANNOT control for threats of HISTORY
Reducing Demand Characteristics
1. Use an experimental disguise. 2. Isolate experimental subjects. 3. Use a "blind" experimental administrator. 4. Administer only one experimental treatment level to each subject.
Roberts et al recommendations for authors on diversity
1. report demographics of samples
Area under the curve (AUC)
THE BIGGER the AREA under the ROC curve THE BETTER! = high sensitivity and higher specificity
Generalizability theory
error can be separated into different factors and can be estimated if method allows (composed of random error, test-retest error, rater error, other sources of error) → True score + multiple error terms
between-subject factorial design
factorial design: participants do not receive all of the conditions. disadvantage = Requires more participants.
Pilot testing why?
field testing a survey can help to identify questions that need to be reworded to improve clarity.
WEIRD science in psychology
for Western, educated, industrialized, rich, and democratic. most psychology research is done with these populations.
Latin Square
form of partial counterbalancing in which each condition of the study occurs equally often in each sequential position and each condition precedes and follows each other condition exactly once
interaction from graph?
if the lines are parallel NO interaction. if lines CROSS each other than YES interaction.
main effect
in a factorial design. the presence of statistically significant differences between the levels of an independent variable IV.
Criterion Variable
in a regression analysis, the variable (Y) Being predicted from the predictor variable.
Samples vs. Populations
population: the ENTIRE group of individuals to which a law of nature applies sample: a relatively small subset OF a population
alpha level
the probability level used by researchers to indicate the cutoff probability level (highest value) that allows them to reject the null hypothesis
Enculturation
the process of acquiring the rules, norms, values, customs, and guidelines of a culture in order to be a part of society.
interrupted time series with switching replications
the program or treatment is replicated at a different time for the two groups. 1. 0-0-T-0-0 2. T-0-0-0
social desirability bias
the tendency to respond to questions in a socially desirable manner. respond to make themselves look good, or how they think they should respond
measures of central tendency
the ways of calculating averages. MEAN, MEDIAN, MODE.
non-interupted time series design
there is NO program or treatment, but performance is measured several times.
A-B design
A-B = baseline THEN Treatment - to see change
A-B-A-B design
A-B-A-B= baseline (A)- Treatment (B)- NO treatment (A) - Treatment (B) - To see change (most Ethical)
repeated- measures design with 2 time points (pre-test vs post-test) appropriate statistical test =
dependent sample t-test
Steps in Hypothesis Testing
1. state hypothesis 2. determine alpha level to define probability of TYPE 1 error that would warrant rejection of the null hypothesis 3. analyze data, and examine P-value 4. make decision about null hypothesis. i.e., reject null or fail to reject null.
Roberts et al recommendations for journals on diversity
1. state whether journal publishes diverse, is it sensitive to diversity 2. include diverse individuals across all levels of publication process 3. justify racial demographics of samples 4. include positionality statements (state their own identity)
null hypothesis REJECTED TRUE
1. type 1 error FALSE POSITIVE probability = a (alpha
"group differences" research Between-group
1.Comparing underrepresented or marginalized population to other groups on variables of interest (first consider: what is the purpose of comparison?) 2. Comparing one or more underrepresented or marginalized groups 3. Illuminating differences and disparities
"group differences" research Within-group
1.Includes only one specific target population 2.Seeks to explain psychological processes and problems 3. Keep in mind within-group diversity!
double blind
A control procedure where neither the participant or the researcher know the conditions of the study. both BLIND example Placebo drug tests
mixed factorial design
A design that includes both independent groups (between-subjects) and repeated measures (within-subjects) variables.
Latin square
A formal system of partial counterbalancing that ensures that each condition in a within-groups design appears in each position at least once.
Factors
A group of INTERELATED VARIABLES i.e. responses subjects give are similar NOT DIRECTLY observed If they respond HIGH on ONE ITEM Q, they will likely answer HIGH on another ITEM Q
race
A group of human beings distinguished by physical traits, blood types, genetic code patterns or genetically inherited characteristics.
manipulation check
A measure used to determine whether the manipulation of the independent variable has had its intended effect on a subject. 1.ask if they know what the study is about 1.ask during experiment if they know what's being studied 3. ASKING if the IV is producing desired affect, i.e. is this making you anxious? (if it's supposed to).
interrupted time series design
A quasi-experiment in which participants are measured repeatedly on a dependent variable before, during, and after the "interruption" caused by some event. 0-0-0-T-0-0-0
Homoscedasticity
A regression in which the variances in y for the values of x are equal or close to equal. points on the scatterplot are about EQUAL distance from each other.
Heteroscedasticity
A regression in which the variances in y for the values of x are not equal. points on scatterplot NOT equal distance from each other
Matching Design
A technique for equating groups on one or more variables, resulting in each member of one group having a direct counterpart in another group. Matching on SOME COMMON VARIABLE. i.e. level of ANXIETY.
covariate variable
A variable that is operating along with the independent variable, which can influence the dependent variable. If the covariate variable is not controlled by the proper statistical procedure, the researcher does not know which of the two variables, the independent or the covariate variable, is influencing the dependent variable. they are of no primary interest in an investigation but are nuisances that must be dealt with.
Sequence (Order) Effects
An effect occurs when more than one treatment is used in an experiment. The order in which participants experience a particular treatment may determine how those participants behave. i.e., the order may impact results.
Basic vs. Applied Research
Applied research is research that seeks to answer a question in the real world and to solve a problem. Basic research is research that fills in the knowledge we don't have; it tries to learn things that aren't always directly applicable or useful immediately.
Nominal scale of measurement
CATEGORIES - of a defined property but the categories cannot be rank ordered. DISCRETE 1. not equal unit of measurement. 2. NO true zero. i.e., gender, ethnicity
ANOVA vs. t-test
DIFFERENCES: t-test: hypothesis test that is used to compare the means of two groups (ONLY) ANOVA: statistical technique that is used to compare the means of MORE than two groups!
ordinal scale of measurement
Data are assigned to categories that can be RANKED with this type of measurement. DISCRETE 1. NO equal unit of measurement. 2. NO true zero. i.e., military rank, the rank of professors.
descriptive vs. inferential statistics
Descriptive statistics uses the data to provide descriptions of the population, either through numerical calculations or graphs or tables. Inferential statistics makes inferences and predictions about a population based on a sample of data taken from the population in question.
Discrete vs. Continuous Variables
Discrete: A finite number of values between any two values. A discrete variable is always numeric. For example, the number of customer complaints or the number of flaws or defects. Continuous: An infinite number of values between any two values. A continuous variable can be numeric or date/time. For example, the length of a part or the date and time a payment is received.
Ethnicity
Ethnicity describes the culture of people in a given geographic region, including their language, heritage, religion and customs.
Main effects from graph A?
If there is a slop then YES main effect. if line has NO slope than NO main effect.
Predictor Variable
In a regression analysis, the variable (X) that predicts the criterion variable.
interaction
In factorial design. occurs when the effect of one independent variable DEPENDS on the LEVEL of another INDEPENDENT VARIABLE. i.e. CBT and drug dosage. if CBT (IV)is affected by the dosage of the drug (other IV).
P x E Interactions
In particular situations, this person behaves in this way because of some internal characteristic. P = person variable E = environment MANIPULATED variable
Labratory vs field research
Lab: allows for greater control and potentially replicable Field: simulates real-life situations (natural behavior in a comfortable environment)
Power (beta)
Power: probability of correctly rejecting the null hypothesis (H0) when the null hypothesis (H0) is false how far the true value is from the null, the standard error of the mean (which depends on the population sd and sample size) in order to compute: specify your best guess as to the value of the true population mean, sd in the population, sample size * power of 0.8 is fairly good! you have a 4 in 5 chance of rejecting the null when null is false
Probability Sampling vs non-probability sampling
Probability sampling involves random selection, allowing you to make strong statistical inferences about the whole group. Non-probability sampling involves non-random selection based on convenience or other criteria, allowing you to easily collect data.
Quantitative vs. Qualitative Research
Quantitative: expressed in numbers and graphs used to test and confirm theories and can establish generalized facts about a topic. Limit is context Qualitative: expressed in words used to understand concepts or experiences and allows for in-depth insights on topics. Context rich!
random assignment vs random sampling
Random sampling (also called probability sampling or random selection) is a way of selecting members of a population to be included in your study. In contrast, random assignment is a way of sorting the sample participants into control and experimental groups.
Applied Behavioral Analysis
Research using various methods to evaluate the effectiveness of conditioning procedures in bringing about changes in the rate of response of some behavior
Problems in "group differences" research diverse populations
Simple categorization and stereotyped assumptions can obscure within-group differences as well as personal experiences of institutional and interpersonal discrimination.
factor loading
Simply put, a variable quantifies the extent to which the variable is related to a given factor. i.e. a lot of factors in a study GROUPED together to explain one factor.
alternative hypothesis
The hypothesis that states there is a difference between two or more sets of data. Your IV DID have an impact on the DV. there was SIGNIFICANCE.
time sampling
The procedure of observing and recording behavior during intervals or at specific moments. i.e. every 10 minutes
interval scale of measurement
This type of measurement uses interval scales, which have equal numerical distances between intervals. CONTINUOUS. 1. EQUAL units of measurement. 2. NO true zero. i.e., IQ score, Temperature.
null hypothesis REJECTED FALSE
True positive probability = 1 -b (beta)
within-subjects factorial design
Within-subjects factorial design: every subject is exposed to all levels of the condition. advantage= Fewer participants
matched group design
a BETWEEN-SUBJECTS experimental design in which participants who are similar on some measured variable are grouped into sets, and the members of each matched set are then randomly assigned to different experimental conditions
P x E Factorial Design
a factorial design with at least ONE subject factor (P = Person variable) and ONE manipulated factor (E = environmental variable)
regression line
a line that describes how a response variable y changes as an explanatory variable x changes. SUMMARIZES the points on a scatterplot and provides means to make predictions.
Item Response Theory
a measurement approach that uses complex statistical modeling of test performance data to make generalizations about item characteristics Based on response patterns of items (which items are more challenging, and weigh some of the scoring based on degree of challenge)
test-retest reliability
a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions. i.e. how consistent are scores for both? correlation between them is calculated.
partial counterbalancing
a method of counterbalancing in which some, but not all, of the possible condition orders are represented
blocked random assignment
a method of random assignment in which participants are assigned to conditions in sequential blocks, each of which contains all of the conditions. ensures an equal number of participants in each condition. used as counterbalancing in within-subjects designs. so all participants experience a condition once before experiencing another condition.
experimenter bias
a phenomenon that occurs when a researcher's expectations or preferences about the outcome of a study influence the results obtained
ROC curve
a powerful tool as a statistical performance measure in detection/classification theory and hypothesis testing, since they allow having all relevant quantities in one plot. Generally the close that a curve is to the top right corner, the better! (High sensitivity and higher specificity)
non-equivalent control group design
a quasi-experimental study that has at least one treatment group and one comparison group, but participants have not been randomly assigned to the two groups. NO RANDOM ASSIGNMENT
Cross-sectional study
a research design in which individuals, typically of different ages or developmental levels, are compared at a single point in time. An example is a study that involves a direct comparison of 5-year-olds with 8-year-olds.
participation observation
a research method in which investigators systematically observe people while joining them in their routine activities. joining the group.
correlational study
a research project designed to discover the degree to which two variables are related to each other. NOT MANIPULATED
response acquiescence
a response set in which participants tend to respond POSITIVELY to survey questions, all else being equal. i.e. answers GREAT to every single question.
factorial matrix
a row and column arrangement that characterizes a factorial design and shows the independent variables, the levels of each independent variable, and the total number of conditions (cells) in the study
A-B-A design
a small N design in which baseline (A) is followed by treatment (B) followed by withdrawn treatment (A).
Pearson's r
a statistic that measures the direction and strength of the linear relation between two variables that have been measured on an interval or ratio scale. ranges +1.00 perfect positive correlation. -1.00 perfect negative correlation. 0= NO relationship between variables
correlation coefficient
a statistical index of the relationship between two things (from -1 to +1) STRENGTH and direction of relationship.
multiple regression analysis
a statistical technique that analyzes the linear relationship between a dependent variable and multiple independent variables .looking at variable X and predicting variable Y. occurs in regression analysis
regression to the mean threat
a threat to internal validity related to regression toward the mean, by which any extreme finding is likely to be closer to its own typical, or mean, level the next time it is measured (with/without intervention) an unusually good performance will regress downward toward the mean next time and an unusually bad performance is likely to regress upward to the mean First test = EXTREME HIGH scores, or EXTREME low scores. second test scores regress to the mean.
History (threat to internal validity)
a threat to internal validity that an experimental group changes over time because of an external factor or event that affects all or most members of the group. aka 9/11 big event that likely changed people in a study the time.
Lykert-Type Scale
a type of rating scale often found on survey forms or questionnaires rating scale 0-5, 1-10 range from disagree- agree- Highly agree type questions
mediator variable
a variable that provides a causal link in the sequence between an independent variable and a dependent variable. EXPLANATORY variable. the reason why.
Type 2 error (beta)
accepting the null hypothesis when it is false. A type II error produces a false negative, also known as an error of omission. concluding there was NO SIGNIFICANCE, WHEN THERE WAS. accepting the NULL.
Regression analysis
allows to see how much a DV is affected by unlimited amount of IV's. allows for specific IV examination, controlling for other IV's. advantage= FLEXIBLE = MULTIPLE IV's, CONTROLS, INTERACTIONS, and MODEL COMPARISONS GIVES MORE INFO THAN correlation and is more flexible
ANOVA (analysis of variance)
an inferential statistical test for comparing the means of three or more groups.
demand characteristics
are cues that might indicate the study aims to participants. These cues can lead participants to change their behaviors or responses based on what they think the research is about.
random assignment
assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups
progressive effects
changes in participants' responses that result from their cumulative exposure to prior conditions. i.e., fatigue..
strategies for collecting demographic information
in general, it is a good idea to put questions about demographic information at the end of a survey. Also, you should include only important demographic categories for the empirical questions that interest you. provide ranges to reduce privacy concerns
design where each participant is randomly assigned to either a treatment group or control group. appropriate statistical test=
independent t-test
3-way factorial design SIMPLE INTERACTION =
interaction of two of the independent variables, with the third variable held constant.
use of factor analysis
is a powerful data reduction technique that enables researchers to investigate concepts that cannot easily be measured directly. By boiling down a large number of variables into a handful of comprehensible underlying factors, factor analysis results in easy-to-understand, actionable data
AxBxC interaction (3-way interaction)
is present when the simple interactions between 2 independent variables are NOT the same at ALL levels of the third independent variable.
measurement issues with diverse populations
it is necessary that measures identify accurately the true prevalence of the construct of interest across diverse groups. Measurement error might lead to biased results, e.g., estimates of prevalence, magnitude of risks, and differences in mean scores.
problems inherent in a 1-question approach to assessing gender
limits data, and also excludes cis-gender identity, and gender identity.
regression analysis
measures the impact of a set of variables on another variable. a reliable method of identifying which variables have impact on a topic of interest. The process of performing a regression allows you to confidently determine which factors matter most, which factors can be ignored, and how these factors influence each other.
reverse counterbalancing
method of control in which conditions are presented in order the first time and then in reverse order. WITHIN-SUBJECTS design. 1-2-3, then 3-2-1
Observation without intervention
naturalistic observation. nothing is done. just observation of subjects. animals or people. NO CONTROL
open-ended questions
not YES/NO, allow respondents to respond in a narrative way. i.e. what do you think of public schools?
participant observation
observational method where the researcher becomes a part of the group being observed
Event sampling
observations are recorded every time a particular event occurs
naturalistic observation
observing and recording behavior in naturally occurring situations without trying to manipulate and control the situation. animals or people.
carryover effects
occur when participants' experience in one condition affects their behavior in another condition of a study
complete counterbalancing
occurs when all possible orders of conditions are used in a WITHIN-SUBJECT design
design where each participant is randomly assigned to one of four different treatment groups. statistical test =
one way ANOVA for independent groups
repeated-measures design with 3 time points (pre-test vs post-test vs 6 month followup) appropriate statistical test =
one way ANOVA repeated measures
Classical Theory
only one category of error (random error on diagram) → True score + (a single) error term
matching
procedure for creating EQUIVALENT GROUPS in which participants are measured on some FACTOR (A MATCHING VARIABLE) expected to correlate with the dependent variable; groups are then formed by randomly assigning to groups, by participants who score at same level on some matching variable
Race vs. Ethnicity
race related to a person's appearance and ethnicity refers to nationality, culture, and ancestry.
creating equivalent groups
random assignment and matching, and block randomization.
Type 1 error (alpha)
rejecting the null hypothesis when it is true (false positive) It means concluding that results are statistically significant when, in reality, they came about purely by chance or because of unrelated factors. AND THERE WAS NO SIGNIFICANCE
Reliability and Validity
reliability (consistency) and validity (accuracy)
longitudinal design
research design in which one participant or group of participants is studied over a long period of time.
confidence interval
statistical range, with a given probability, that takes random error into account. Based on the mean and standard deviation of the sample to make inferences about the population. The lower the CI the smaller the range of scores, thus the less confident you are that the true value or score is within that range of scores
Statistics vs. Parameters
statistics are sample values vs. parameters are corresponding population values
cohort effect
the effect observed in a sample of participants that results from individuals in the sample growing up at the same time
simple effect
the effect of one factor within a level of another factor. differences among particular cell means within the design. More precisely, a simple effect is the effect of one independent variable within one level of a second independent variable.
gender identity
the individual's sense of being male or female
Intersectionality
the interconnected nature of social categorizations such as race, class, and gender as they apply to a given individual or group, regarded as creating overlapping and interdependent systems of discrimination or disadvantage. taking into account MULTIPLE identities of a person and how they impact them.
effect size
the magnitude of a relationship between two or more variables. tells you how meaningful the relationship between variables or the difference between groups is. A large BLANK size means that a research finding has practical significance, while a small BLANK size indicates limited practical applications.
Null Hypothesis (H0)
there is NO relationship between the variables in the population. aka. your IV had no IMPACT on the DV. The null hypothesis is a typical statistical theory which suggests that no statistical relationship and significance exists in a set of given single observed variable
experiment vs quasi experiment
true experimental design -assignment to treatment: The researcher randomly assigns subjects to control and treatment groups. -Control over treatment: The researcher usually designs the treatment and decides which subjects receive it. -Use of Control groups: Requires the use of control and treatment groups. quasi-experimental design -assignment to treatment: Some other, non-random method is used to assign subjects to groups. -Control over treatment: The researcher often does not have control over the treatment, but instead studies pre-existing groups that received different treatments after the fact. -Use of Control groups: Control groups are not required (although they are commonly used). •In an experimental study, the independent variable is randomly assigned. •In quasi-experimental study, the independent variable (such as age) is not randomly assigned.
null hypothesis NOT REJECTED TRUE
true negative probability = 1-a (alpha)
null hypothesis NOT REJECTED FALSE
type 2 error false positive probability = B (beta)
nuisance variables
unwanted variables that can cause the variability of scores within groups to increase. confounding variables that are not part of the study. i.e. hot room, loud environment, these can affect the outcomes of a study.
moderating variable
variable that alters the relation between the independent variable and the dependent variable. the "it depends" variable. relationship of A and B. depends on how much C there is.
unobtrusive measures
ways of observing people so they do not know they are being studied i.e. hidden cameras, searching trash to see what they eat, two-way mirrors etc..
Reactivity
when a participant acts differently when they know they are being observed. to avoid use unobtrusive observations.
