Research Methods
complex designs notation
#n IV1 (condition 1, condition 2,... condition n) x #n2 IV2 (condition 1, condition 2,... condition n) ... x #nn IVn (condition 1,... condition n)
independent variable
A factor that researchers control or manipulate in order to determine the effect on behavior. A minimum of two levels: The treatment (experimental) condition and the control condition
t-test
A t test is used to test the difference between two sample means. Each t value has a probability associated with it. The test statistic, or t-test, is used to determine the relative likelihood (probability) of observing a given difference between means under circumstances in which both samples have been drawn from the same population Between-subjects t-test Tests for mean differences between groups Within-subjects t-test Tests for mean differences between conditions Single-sample t-test Tests sample versus population parameter When two sample means are different, there can be two explanations Null hypothesis: difference is due to sampling error (from the same population, and not treated differently) Experimental hypothesis: means are from the same population, but were treated differently Within-subjects t-test is more sensitive to differences between conditions than the between-subjects test Smaller differences are more likely to be significantly different when using a within-subjects test
ABA or Reversal design
ABA Design (also called a reversal design) Follow the B with the original conditions Want to know if the IV produced the changes in behavior Going back to the original conditions after your experimental condition can test this If behavior is changed during B then reverts during the final A phase, the IV worked End with a B condition if you want the behavior stopped (ABAB) sometimes you can't undo the effect of the treatment & it gets confusing whether to attribute the changed behavior to the treatment or to confounds if the second A phase does not resemble the first A phase
power
Ability to reject H0 when H0 is false Affected by: Sample size & Magnitude of effect Can determine n necessary for a study This is where you get the 20 Ss per condition Can determine after the fact why you were unable to detect an effect the ability to detect statistical significance if it exists
internal validity
An experiment has internal validity when we are able to confidently state that the independent variable caused differences between groups on the dependent variable (i.e., a causal inference). For an experiment to be internally valid, we must be able to rule out alternative explanations for the study's findings. Threats: intact groups, extraneous variables, selective subject loss, and when demand characteristics and experimenter effects are not controlled for
case study
An in-depth analysis of one particular individual Common research procedure in clinical psychology Allows you to get the vast amount information available in a therapeutic context and find what treatments are effective
ANOVA
Analysis of Variance (ANOVA) is used for null hypothesis testing. Analysis of variance tells us whether the main effects and interaction effect(s) in our complex design are statistically significant. The initial ANOVA is called the "omnibus" (overall) ANOVA. - Main effects & Interaction effect(s) If an interaction effect is statistically significant, the researcher does "follow-up" or "post hoc" tests of statistical significance. Planned vs. unplanned comparisons of two means
relational research
Attempts to determine how two or more variables are related to each other Analysis of data using: central tendency dispersion t tests anova (analysis of variants) x^2 effect size normal distribution
how to calculate a contingency table
Calculate expected frequencies for each cell Put all numbers into the formula Calculate the Chi-Square value Look up significance based on degrees of freedom X2=(22.99+1.56+5.97+13.92+0.94+3.62) X2 = 49.00 df = (2-1)*(3-1) = 2 Critical value = 5.99 (if our X2 is greater than this, we have a significant result and reject the null)
changing criterion design
Change the behavior necessary to obtain reinforcement ie: Interested in investigating the effects of different criteria (press a lever 5 times, then 7 times, etc. before reinforcement) on the participant's behavior could be (but not always) shaping
why do you conduct research?
Contribute to the evolution of knowledge Different researchers ask different questions Different researchers approach the same questions differently
goals of research (according to the scientific method)
Describe a behavioral phenomenon Develop explanations for behavior Attempt to predict behavior or performance Control - Psychotherapy or therapeutic interventions
Correlational research
Determine the degree and direction of a relationship with a single statistic Coefficient of correlation Can be used to evaluate measurements and tests, or can be used directly as a research tool Refers to a category of statistical tests Vary from -1.00 through 0.00 to +1.00 Magnitude of the correlation coefficient determines the strength of the relationship *Just a measure of magnitude and direction of association No determination of causality A could cause B or B could cause A C could cause both A and B Problem of truncated (restricted) range: Need variability to detect differences & If the variability is too small, important differences can be masked
steps of research
Develop a research question. Generate a research hypothesis. Form operational definitions. Choose a research design. Evaluate the ethics of your research. Collect and analyze data; form conclusions. Report research results.
Sensitivity (d')
Difference between mean amount of sensory activity generated by noise and noise + signal trials measured in Z scores d' = ZFA - ZHIT ie: d' = Z(50-20) - Z(50-60) Look up Z score values based on % under the curve 30% and -10% d' = .842 - (-.253) d' = 1.095
range
Difference between the highest and lowest scores in the distribution
multi-method approach
Different areas of psychology require multiple methods clinical, social, industrial/organizational, developmental, counseling, physiological, cognitive, educational, personality, human factors, neuropsychology, etc. No single research method or technique can answer all of the different questions in psychology. No perfect research method Each method or measure of behavior has flaws "Toolbox" with different tools for conducting research
two-tailed test
Do not have a specific prediction about the direction of a relationship (non-directional test)
scientific experiment characteristics
Empirical Judgments based on direct observation and experimentation Skeptical, critical attitude observation is Systematic, controlled Control = essential ingredient of science Greatest control is in an experiment Clear, specific definitions Uses constructs is the term generally used for concepts within research Unbiased, objective reports with inter-observer agreements accurate, precise, valid, and reliable instruments and measurements
directional test
Expect a difference between your two groups/conditions, and you expect that difference to be in a specific direction
non-directional test
Expect a difference between your two groups/conditions, but do not predict a direction
type 2 error
Fail to reject a null hypothesis that should have been rejected
Stages of data analysis
Getting to know the data Summarizing the data Confirming what the data reveal
one-tailed test
Have a specific prediction about the direction of the relationship (directional test) Easier to find a significant result
normal distribution/ curve
High in the middle with symmetrical decline towards each extreme Inflection point Curve starts bending outward (rather than down) One standard deviation from the mean Percentage of scores under different parts of the curve (on each half of the mean)
applied research
Human Factors Industrial/Organizational Clinical/Counseling/Health
main effect
IVs, main thing you're measuring The overall effect of an independent variable in a complex design The main effect is the effect of the independent variable on the dependent variable as if only that variable was manipulated in the experiment. Simple main effects: The effect of one independent variable at one level of the second independent variable. One definition of interaction effects is that the simple main effects of one independent variable are different across the levels of the second independent variable.
covariation of events
If one event causes the other, the two events must vary together (when one changes, the other must change also). We must observe a relationship between the independent and dependent variables. For example, participants who write about emotional events have better health and academic outcomes than participants who write about superficial events. Thus, the two types of writing covary with the different outcomes.
central tendency
Indicates the center of a distribution of scores use mean, median, mode to help analyze Indicates the center of a distribution of scores
discrete trials design
Individual participants receive each treatment condition dozens (or even hundreds) of times Each exposure to a treatment, or trial, produces one data point for each DV Extraneous variables are tightly controlled Order is randomized or counterbalanced if feasible Can compare behavior of participants to provide intersubject replication signal detection theory
intact group
Intact groups are formed prior to the start of an experiment. Individuals are not randomly assigned to intact groups. As a result, individual differences among groups threaten the validity of the experiment.
nonscientific experiment characteristics
Intuitive Judgments based on "what feels right" Accept claims without evidence observation is Casual, uncontrolled Personal biases influence observation Ambiguous concepts Use undefined terms Biased, Subjective, Personal impressions and opinions are reported inaccurate, imprecise, invalid, non-reliable, possibly inconsistent instruments and measurements
interrupted time series design
Involves the examination of a naturally occurring treatment on the behavior of a large number of participants Interesting example: Introduce fluoride and measure school achievement scores Can follow a single class (e.g. , 2001) over a long period Can follow one class (e.g., 3rd grade) over a long period mortality delayed effects on treatments ex: Measure school achievement scores from 1965 to 1995 Fluoride introduced in local water supply in 1979 After this interruption, school achievement scores increased
large n designs
Large numbers of participants are included so that an unusual participant (outlier) does not skew the results (nomothetic)
coefficient of a correlation
Larger numbers indicate stronger relationships The sign (+ or -) indicates the direction of the relationship Pearson product-moment correlation (r)
interpretation
Main effects Interactions Sources of error How would you do the study again? Matching design and analysis to research questions
subject variables
Measurable characteristic of people (Age, sex, height, weight, IQ, nACH, race, Alcoholism, schizophrenia, brain damage) Do not manipulate these variables, and have no control over them Cannot necessarily make clear causal statements when the research involved subject variables Interpretation can be thought of as similar to correlations - there is a relationship, but we cannot determine causality
AB design
Measure initial levels Institute some treatment Measure post-treatment levels A: baseline condition before therapy B: level of the behavior after therapy IV: Therapy DV: tantrum behavior Poor design because it overlooks too many potential confounds (no control) in a small-n: want to find out if we have an effective treatment for the behavior use AB design to do this A- condition B- treatment AB design not necessarily sufficient because other factors that occur between A and B could be confounding variables
median
Middle-most score (50th Percentile)
mean
Most common and familiar (average) Sum of scores divided by number of scores (ΣX/n) Affected strongly by outliers The average score on a dependent variable is computed for each group in the experiment (a measure of central tendency). Most common and familiar (average) Sum of scores divided by number of scores (ΣX/n) Affected strongly by outliers
Pearson product-moment correlation (r)
Most commonly used coefficient r=∑xy / ((∑(x^2)∑(y^2))^-.5) X = scores for the 1st variable and Y = scores for the 2nd variable ∑XY = sum of cross-products of each pair of X and Y scores N = number of pairs of scores Pearson r is not good at detecting relationships that are not linear Linear: straight line relationship Curvilinear: relationship does not fall on a straight line, rather a curve
mode
Most often occurring score
extraneous variables
Practical considerations when conducting an experiment may confound an experiment — these are referred to a extraneous variables. controlled for via balancing and holding conditions constant
type 1 error
Reject a null hypothesis that should not have been rejected
generalizability
Researchers are not interested in just the one sample of people or one set of circumstances tested in a research study. They wish to generalize a study's findings to other People, Settings, & Conditions
inferential stats
Researchers use inferential statistics to determine whether an independent variable had a reliable effect on a dependent variable. Inferential statistics allow us to rule out whether the findings from our experiment might be simply due to chance
alternating treatments
Reversal designs are problematic if you expect to get long-term effects on the DV Alternating treatments design is an alternative that can extend the basic reversal design A small-n experimental design in which two or more IVs alternate ACABCBCB A: Baseline, B: IV1, C: IV2 This design allows you to test more than one IV, and can make determinations about causality Example: test effects of three different diet levels on hyperactivity A: normal diet (baseline) B: introduce cookie into diet C: introduce artificially colored cookie into diet
control
Scientists investigate the effect of various factors one at a time in an experiment. Researchers conduct controlled experiments to identify the causes of a phenomenon. Control requires that researchers manipulate factors to determine their effect on the event of interest — these are independent variables. *Potential IV that is held constant during an experiment
basic research
Social (attitudes, attraction, persuasion, conformity) Cognitive (language, memory, decision making) Developmental (age-specific function levels) Personality (traits, motivation, individual differences)
overlap of basic and applied research
Sometimes we have applied questions that lead to basic research That basic research can then be applied to real problems Research is inspired by critically evaluating phenomena in the world *ie: social loafing
standard deviation
Square root of variance (and used more as descriptive) (Variance= s2= Σ(X-¯X)2/n) The average distance of each score from the mean of a group is computed (a measure of variability).
z score
Standard score units Each increase in Z score represents a change of one standard deviation ≈68% of scores are within one standard deviation ≈96% are within 2 SD, ≈99.74% are within 3 SD Great source of comparison IQ score of 115 is 1 SD above the mean Score is higher than ≈84% of people Can compare across different sample means, etc. when use Z
Contingency tables
Tabular presentation of all combinations of categories of variables that allows relationships to be examined categorical variables (can only be in one square at a time, in one category)
interaction effect
The combined effect of independent variables in a complex design An interaction effect occurs when the effect of an independent variable differs depending on the level of the second independent variable. Interaction effects represent how independent variables work together to influence behavior. An interaction effect occurs when the effect of one independent variable differs depending on the level of the second independent variable. An interaction effect is likely present in a complex design experiment when the lines in a graph that display the means are not parallel - that is, the lines either intersect, converge, or diverge.
dependent variable
The measure of behavior that is used to assess the effect of the independent variable. In most psychology research, several dependent variables are measured to assess the effects of the independent variable. Researchers observe the effect of the independent variable by measuring dependent variables.
time order relationship
The presumed cause must occur before the presumed effect. For example, writing about emotional events (the cause) comes before the beneficial health and academic outcomes (the effect).
effect size
The strength of the relationship between the independent variable and the dependent variable is computed. Cohen's d Guidelines for interpreting Cohen's d are: Small effect: .20 /Medium effect: .50 /Large effect: .80 Different measures of effect size for different tests, but attempting to determine practical significance rather than statistical significance Basically, the magnitude of the effect
X^2 test for independence
There are different kinds of Chi-Square tests Null hypothesis: the number of golfers is independent from school (no difference associated with the variables) Reject the null: there is a relationship Fail to reject the null (you don't accept it, but fail to reject it) Based on observed frequency (how many people appeared in each cell) compared to expected frequency (how many people would you expect in each cell) X2= Σ(O-E)2/E If you do not have an expectation about the expected frequency, you calculate it based on participants E=(row total X column total)/total
null hypothesis testing
This statistical procedure is used to determine whether the mean difference between two conditions is greater than what might be expected due to chance or error variation. We say that the effect of an independent variable on the dependent variable is statistically significant when the probability of the results being due to error variation (chance) is low. Null hypothesis testing is most often used to decide whether the independent variables have produced an effect as measured by the dependent variables.
causal inference
Three conditions must be met before we can make a causal inference: Covariation Time-order relationship Elimination of plausible alternative causes
Sections of a research paper
Title Page Abstract Introduction Method Results Discussion (main body of report) References Footnotes Appendices Tables (if any) Figures (if any)
explanation
Using experiments, researchers can make causal inferences — statements about the cause of an event or behavior. *requires Covariation of events, Time order relationship, and explanation of alternative possible causes
small-n designs
Very few participants are intensely analyzed More information about a smaller number of people Idiographic approach usually in applied settings Allow you to do research when you do not have a large number of participants available to you Because specific conditions, like autism, schizophrenia, hyperactivity are necessary Because specific tests allow for high levels of control (e.g., radar target discrimination) *not all small n are case studies
elimination of alternate plausible causes
We accept a causal explanation only when other possible causes of the effect have been ruled out. Using control techniques, we rule out other possible causes for the outcome. If the two groups differ in ways other than the emotional and superficial writing, these differences become alternative explanations for the study's findings.
confounding variables
When the independent variable of interest and a different, potential independent variable are allowed to covary (go together), a confounding is present. Confoundings represent alternative explanations for a study's findings. An experiment that has a confounding is not internally valid. An experiment that is free of confoundings has internal validity.
external validity
Would the same findings occur in different settings? Would the same findings occur with different conditions? Would the same findings hold true for different participants? Researchers often seek to generalize their findings beyond the scope of their specific experiment. Experiments should include characteristics of the situations, settings, and populations to which researchers wish to generalize (e.g., representative samples and situations). Partial replications are common: Research findings generalize when a similar result occurs when slightly different experimental procedures are used in a subsequent experiment.
scientific method
an abstract concept, not a particular technique or method. Two important aspects of the scientific method are: the reliance on an empirical approach, and the skeptical attitude scientists adopt toward explanations of behavior and mental processes. Way to gain knowledge about behavior and mental processes A general approach to gaining knowledge Not a particular technique or tool Compare scientific method to "everyday" ways of gaining knowledge (nonscientific)
what might a null be due to?
confounding variables, problem with selecting levels of IV
reliability
consistency of an experiment Test-retest reliability: Give same test twice over a short period of time Parallel forms: To avoid practice effects, you can employ alternate forms of the same test Split-half reliability: Correlate scores from two halves of a test improve reliability with: More items, Greater variability among individuals on the factor being measured, Testing situation free of distractions, and Clear instructions
balancing
control technique Some alternative explanations for a study's findings concern characteristics of the participants. For example, what if participants in the emotional writing condition are healthier or smarter than participants in the superficial writing condition, even before they write anything? The goal of balancing is to make sure that, on average, the participants in each condition are essentially the same before the experiment begins. Thus, on average, the groups in the experiment are equally healthy, smart, motivated, conscientious, etc. prior to the independent variable manipulation. *when the two experimenters conduct each condition but are randomly assigned to administer a condition at any particular time.
Description
define, classify, catalogue, or categorize events and their relationships to describe mental processes and behavior. Most psychology research is nomothetic rather than idiographic. Most psychology research is quantitative rather than qualitative.
Interaction
how IVs interact with each other IV1xIV2, IV1xIV3, IV2xIV3, IV1xIV2xIV3
Signal detection theory
how good are you at picking up a signal in some sort of broader noise? If participants need to rate the presence/absence of an item, they can say present or absent and be wrong We need a way to account for hits (correctly indicate present) and misses (fail to identify presence) Allows you to determine ability to detect a stimulus (signal) amongst interference (noise) Calculate d' (d prime) Sensitivity Assumes that noise is randomly distributed (see chart in ppt.)
ideographic approach vs. nomothetic approach
ideographic is better at predicting one person's actions b/c it's tailored to them - The problem of whether to use nomothetic or idiographic approaches is most sharply felt in the social sciences, whose subject are unique individuals (idiographic perspective), but who have certain general properties or behave according to general rules (nomothetic perspective).
delayed effects on treatments
ie: If you expect fluoride to reduce cavities and thereby absenteeism, you would not expect the effects to be immediate
mortality
ie: Students graduate, move away, some die This can be a special type of selection bias You do not have any control over who withdraws from the sample
measures of dispersion
indicate how spread out scores are range and standard deviation
idiographic
individual case studies
theory
integration of facts and ideas
nomothetic
large sample sizes, "average" performance of a group
alpha
level of significance, usually .05
descriptive stats
measures of dispersion and effect size ie: mean, median, mode, range, standard deviation, central tendency, effect size
Observation-treatment-observation design
natural treatment Like an ABA, but the final A is not entirely equivalent to the initial A condition May be interested in measuring reading levels, introducing a new reading program, then measuring reading levels again problem: Children will not be the same age at follow-up that they were at the beginning (see full example in ppt) Not a reversal design because the treatment not under experimenter's control and natural treatments are likely to have long term carryover effects
null
no significant correlation Null results can be a product of problems with the DV Floor effect- Being at the bottom of a scale (does not allow for variability) Ceiling effect- Being at the top of a scale (does not allow for variability) Null results can occur because of a failure to control important control variables Confoundings: unintended effects that occur because their influence confounds the proper interpretation of the results
empirical approach
observation of behaviors that can be observed directly, and experimentation in which scientists employ systematic control in the situation to be observed.
Between subjects design
participant is only in one level of the IV
alternative hypothesis
predicting an alternate outcome for a different causation
p-value
probability associated with an inferential statistic found in a table If the observed p value is less than .05, reject the null hypothesis of no difference. If the observed p value is greater than .05, do not reject the null hypothesis of no difference.
variance
s2= Σ(X-¯X)2/n
quantitative
statistical summaries of performance
X^2 independence test
test the null hypothesis to see whether our outcome rejects or fails to reject the null hypothesis that the variables are independent of each other Based on observed frequency (how many people appeared in each cell) compared to expected frequency (how many people would you expect in each cell) X2= Σ(O-E)2/E
hypothesis
testable prediction about the relationship between 2 or more variables circular- Event itself is used as an explanation for the event
operational definition
the specific procedure used to produce and measure a construct. Define with specificity to allow for clear communication A definition of a concept in terms of the operations that must be performed to demonstrate the concept
what makes research good?
theoretical framework standardized procedures generalizability objective measurement
validity
truth of an experiment Predictive validity: Can you predict an outcome based on some criterion? Construct validity: Does the experiment measure what you intend it to measure? External validity: Can you generalize from your sample results to the population? Internal validity: Do you have causal relationships between your IV's and DV's? Establishing the construct validity of a measure depends on convergent validity & discriminant validity Convergent validity refers to the extent to which two measures of the same construct are correlated Discriminant validity refers to the extent to which two measures of different constructs are not correlated
multiple baseline design
two ways: Establish baseline for dif behaviors and come up with ONE treatment for ONE behavior *look at effects of each behavior, and a combo of the two *the second behavior could be effected unintentionally by the first treatment Different behaviors (or participants) receive baseline periods of varying lengths prior to the introduction of the IV Can establish baselines for different behaviors Crying vs. fighting measured for different lengths of time Can establish baselines for the same behavior for different people Fighting measured for different lengths of time Continuing to measure one baseline while directing a behavioral intervention at a second behavior allows you to tease apart effects of each behavior plus their combination Determine independence of the behaviors or the effect of the IV on the two measures
qualitative
verbal summaries of research findings
minimum number of participants for a study?
~20
