Research Methods Final
What is the social desirability bias?
Over-reporting desirable behaviors, underreporting undesirable behaviors
What is participant reactivity? How can we use unobtrusive measures to prevent it?
Participant Reactivity Remember the Hawthorne Effect? Behavior influenced by knowledge that it is being observed Use of unobtrusive measures Measures taken without participant awareness May be direct measurements May be indirect measurements
What is event sampling?
Record all instances of selected types of behaviors or events, ignoring non-selected events
What is a time sampling?
Record behavior at determined intervals of time (e.g., every 15 seconds)
What is situation sampling?
Record behavior in different, predetermined settings or situations
What is the purpose of random assignment and matching?
Reduce potential confounding variables.
What did the the Westinghouse study show about the limitations of a non-equivalent control group design?
-an evaluation of the Head Start program -documented "fade-out effects" (early gains seemed to fade out) -in order to match for cognitive achievement, they had to select kids in the Head Start group that were scoring well above the mean of that population and they had to select kids scoring well below the mean of that population of kids who are not in head start -therefore, they each showed regression towards their means and distorted results
What is the minimum number of independent variables and levels we must have in an experiment?
2
What is statistical conclusion?
?
Hawthorne effect
A change in a subject's behavior caused simply by the awareness of being studied
What is a crossed-lagged panel correlation?
A longitudinal design Measure variables at multiple points in time Compute correlations across time
What is a correlation coefficient?
A numerical index of the degree of relationship between two variables. R
What is applied research in psychology?
Applied Research is a scientific study within the field of Psychology that focuses on solving problems, curing illnesses, and innovating new technologies. Its main purpose is to conduct scientific research and apply it to real-world situations.
What is formative evaluation?
Begins with assessed need for program On-going as program is implemented basically monitoring the program
What are the characteristics of science as a way of knowing?
Belief:You believe that it is true ... Truth:It is factually true ... Justification: Good reasons for the belief
What is summative evaluation?
Can be follow-up to look at results of program activities Occurs when program complete or at specific time after Was the program effective in meeting goal? Should the program be retained? Do we see the effect we expected? High-stakes testing
What are the key principles of the APA code of ethics?
Consent must be voluntary and well-informed Must avoid unnecessary suffering or harm Benefits of experiment must exceed risks Staff must be scientifically qualified and trained Participant must feel free to quit at any point Staff must stop experiment when dangerous
Different types of observational studies
Controlled Observation-Structured, controlled environment (often a lab) Specific behaviors, many measurement tools Naturalistic Observation-Unstructured, unobtrusive strategy,Observe behavior in natural settings Participant Observation-Unstructured, covert observation strategy,Researcher "joins" observed group
What is the difference between descriptive statistics and inferential statistics?
Descriptive Statistics Tools for organizing and distilling data into a few meaningful numbers we can digest Measures of Central Tendency Measures of Variability ex: mode, mean, median Inferential statistics Tests of Significance(Probability of real effect?) Effect Size(How much of an effect?) Confidence Intervals(Range of probable outcomes for population?)
Are correlational coefficients descriptive or inferential statistics?
Descriptive statistic
What is a quasi-experiment?
Designed to answer cause and effect questions; major difference is that participants are not randomly assigned
What common problems and limitations with observational research?
Ethical Concerns informed consent and right to privacy?Deception in participant observation?Particularly in participant observation WLack of Control Particularly true in unstructured observations Passive observation vs. active manipulation Description ("what") vs. explanation ("why") Observer Bias Preconceptions and expectations Interpreting ambiguous behavior Choices about what to record vs. ignore Use of behavior checklists (operationalizing) Multiple observers (inter-observer reliability) Lack of Representativeness Are your observations really representative?Use of systematic sampling methods
How does correlational research differ from experimental research?
Explore effects of natural variation among individuals Measure variables and look for patterns
Understand the difference between positive and negative correlations
Graph shows positive and negative slopes to demonstrate either correlation
What is a between subjects design?
Groups are completely independent from one another (ex: control group vs. tx group)
What is the difference between a naturalistic observation and a participant observation
In a naturalistic observation you are merely watching w/o interacting/joining group being observed
What is the coefficient of determination?
In a simple linear regression with only one independent variable, the coefficient of determination is the square of the coefficient of correlation. The coefficient of determination is represented by the term r2 (or R2) and it is the percentage of the total amount of change in the dependent variable (y) that can be explained by changes in the independent variable (x).
In psychology, what are some common uses for correlational research?
In psychology, correlational research can be used as the first step before an experiment begins. It can also be used if experiments cannot be conducted. It determines if a relationship exists between two or more variables, and if so, to what degree the relationship occurs.
What is an interrupted times series design?
Longitudinal research design Vaguely similar to within-subjects design No counterbalancing; no control group Series of measurements before and after introducing treatment Look for trends in observations Consistent, lasting changes after treatment?Fluctuations? Brief, temporary changes?
How can researchers strengthen the validity of an interrupted time series design?
Look for trends in observations Consistent, lasting changes after treatment?Fluctuations? Brief, temporary changes?
What are the characteristics of a well-designed survey?
Make items clear and simple; beware vague and ambiguous language Avoid double-barreled questions (combining multiple questions into one) Avoid begging the question (assuming an answer to a question that wasn't asked) Avoid leading questions (questions that suggest a "correct" answer) Avoid using jargon, slang, or colloquial expressions; use appropriate language Avoid negatively phrased questions; they are easily misread and confusing Wait to ask more sensitive questions until later in the survey Place demographic information at end of survey; ask only for essential information and use appropriate terminology
What are the strengths and limitations of applied research conducted outside the lab?
More naturalistic, less controlled
What is multiple regression, and why is it used in psychology?
Multiple regression analysis is used to examine the relationship between one numerical variable, called a criterion, and a set of other variables, called predictors.
How do open-ended questions differ from closed-ended questions?
Open ended allow free response, however are hard to identify correlations in them/ find common themes
When do we use ANOVA?
Same rationale as t test Calculate variance between all groups (total effect) Calculate variance within all groups (error variance) Accounts for sample size (statistical power) Calculates a statistic called an F ratio Minimizes possible Type I error Single test rather than multiple t tests No compounded error
What kind of data do program evaluations rely on?
Scientific mindset to social reform - testing to see if reforms are effective Commitment to flexibility and problem-focus in developing programs and solutions Systematic data is necessary .. May include a pilot study of the program
What does it mean to reject the null hypothesis?
That there is significant difference between control group and intervention group(s)
What is a needs analysis?
The process of identifying and evaluating needs
What is interobserver reliability?
There is consistency across examiners. It's higher is examiners are trained strictly. You want it high!
What is internal validity?
This type of validity is focused on determining whether a study's findings are accurate, or are more the result of the influence of extraneous variables.
What does it mean for a measurement to be valid? To be reliable?
True and capable of being supported Validity in Measurement Does the measurement strategy or tool actually measure what it claims to measure? What evidence can you provide to demonstrate this quality?
Be able to interpret Pearson's r values
Type of Relationship Positive = Direct relationship Negative = Inverse relationship Strength of Relationship Weak correlation: ±.20 to ±.50 Moderate correlation: ±.50 to ±.70 Strong correlation: ±.70 to ±1.00
What is a nonequivalent control group design?
Typically used to evaluate some program or intervention in the real world Similar to a between-subjects design Groups composed of different individuals Control condition vs. Treatment condition BUT ... Groups are not equivalent When random assignment is not possible May use pre-existing (or pre-defined) groups Confident causal conclusions are impossible
What is the third variable problem?
Variables that are causally related are correlated but not all variables that are correlated are causally related.
What is nonresponse bias?
When people do not respond to surveys or answer the phone, the experiment will suffer.
What is an institutional review board?
a body that reviews researchers' proposals before work begins in order to assess potential harm and possible benefits to participants
What is an operational definition?
a carefully worded statement of the exact procedures used in a research study
What is a survey used for?
a measure of people's self-reported attitudes and behaviors aka DATA
What is the directionality problem?
a problem encountered in correlational studies in which the researchers find a relationship between two variables but cannot determine which caused the changes in the other
What is an experiment?
a scientific procedure undertaken to make a discovery, test a hypothesis, or demonstrate a known fact.
cost-benefit analysis
a study that compares the costs and benefits to society of providing a public good
What is a within-subjects design?
all participants get all levels of your independent variable
What is a construct?
an idea about a psychological attribute that goes beyond what might be assessed through any particular method of assessment
factorial design
any experimental design with more than one independent variable
When do we use a t-test?
compare 2 means
demand characteristics
cues in an experiment that tell the participant what behavior is expected
evaluation apprehension
people's concern about how they might appear to others, or be evaluated by them
In regression analysis, what is the predictor value and what is the criterion value?
predictor value ex: number of days missed criterion ex: grade received predictor value uses data to predict outcome which is the criterion value
What is descriptive research?
systematic, objective observation of people
What is construct validity?
the degree to which test scores are consistent with theoretical constructs, concepts or expectations.
What is external validity?
the extent to which the results of a study can be generalized to other situations and to other people
good participant effect
the tendency of participants to behave as they perceive the experimenter wants them to behave
In his 1957 address to the APA, what did Cronbach argue about correlational research?
there is value in both approaches, neither is superior. It is hard to establish cause and effect in correlational
What is regression analysis?
using correlations to make predictions
What is archival research?
using previously compiled information to answer research questions