Research Methods Final Exam
normal curve
a distribution of data in which the mean, median, mode are at the same point (center of the distribution) and which 1 sd from the mean is 68%, 2 sd is 95%, 3 sd is 99%
regression line
a line that best describes the trend in the data; is as close as possible to the data points; y'=a+bx where a is the intercept and b is the slope
history
a threat to internal validity when the dependent variable is influenced by an event that occurred during the study; outside of study design, hardest to control and most likely threat; try to control as many things as possible
concurrent validity
a type of criterion validity in which a measuring instrument is correlated with some criterion that is administered concurrently or at about the same time
Factorial ANOVA
an ANOVA in which there is more than one IV; used more often than simple ANOVAs; three different F tests; has main effects and interactions
Repeated measures ANOVA
analysis of scores for the same individuals on successive occasions, such as a series of test trials; problems include carryover effects and practice effects ; individual differences can be identified and separated from the error term, increasing power; reduces variation; usually used over time; DV measured multiple times
ANOVA
analysis of variance among 2 or more means; with 2 mean the F statistic yields the same probability we would obtain with a t-test; measures 2 sources of variation (w/in and between) and compares their relative sizes
interaction of selection biases and treatment
cannot generalize unless they have same characteristic; treatment may only work on participants with specific characteristics; non-randomly selection participants based on a common characteristic; if you use trained subjects, you cannot generalize to untrained subjects
how to combat instrumentation effects
choose the best test that doesn't have calibration issues; train observers well and beware of halo effects
Faulty data gathering
collecting data from participants who are not complying with the requirements of the study; using faulty equipment; treating participants inappropriately; recording data incorrecty
2 way repeated measures ANOVA
common; group x time; 2x2 repeated measures ANOVA
F test and equation
compares the variability between groups to the variability within groups; F = variation between / variation within
reliability
consistency of measure
Nonpublication of data
cooking data; data not included in results because they don't support the desired outcome
poor data storage and retention
data should be stored in its original collected form for at least 3 years after publication; data should be available for examination; confidentiality of participants should be maintained
Independent samples t-test
determines whether 2 unique samples are different; sample size can be unequal; equal or unequal variance
evaluating the null hypothesis for t-tests
establish are the 2 groups different and how meaningful are the differences
how to combat experimental mortality
explain the need for participation well at the beginning
descriptive/intuitive reasoning
general to specific; take an example of something and then vary the example to see if the phenomenon in question is still present or not
external validity
generalizability of a study
2 tailed t-test
if no prediction which direction the samples will differ
multiple-treatment intereference
if subjects participate in multiple treatment groups, the previous treatment may inhibit or enhance the next treatment; more than one treatment
philosophic research methods
inductive reasoning and descriptive reasoning
Large F values
large F values reject the null hypothesis b/c they show more variation between groups than within groups
Fabrication and Falsification
making up or altering data
Central Tendency
measures of central tendency describe the average or common score of a group of scores; common measures include mean, median, mode; a single score that best represents all the scores
correlation coefficient
measures the strength and direction of a relationship between 2 variables; ranges between -1 to 1 and is identified by r
data gathering
most important and most aggravating; always drop non-compliers; fix broken equipment; treat subjects well; store data in a safe and private place for 3 years
cause and effect
no other reasonable explanation exists for the change in the DV except for the manipulation of the IV; due to good theoretical framework, proper experimental design, use of correct stats, controlled variables, appropriate DV, correct interpretation of results
Type 2 error
null hypothesis is false and you fail to reject it; when there is an effect but the stat test comes out as non-significant; due to inadequate power, sampling error
Type 1 error
null hypothesis is true and you reject it; there is no effect but the stat test comes out significant
one-way/simple ANOVA
one independent variable for more than 2 groups; has varying levels
reactive effects of experimental arrangements
one setting vs another setting; lab setting may limit real-world application and influence generalizability; results in a controlled lab setting may not be seen in a real world setting
avis effects
participants try harder b/c they are in the control group; include placebo group
7 areas of scientific dishonesty
plagiarism, fabrication and falsification, nonpublication of data, faulty data-gathering procedures, poor data storage and retention, misleading authorship, sneaky publication practices
Interaction
present when the differences between the groups of one IV on the DV vary according to the level of a second IV
beta
probability of making a type 2 error
Informed consent form
provides potential subject with info to make a sound decision about participating in a study; provides simple but comprehensive info about the study
Sneaky publication practices
publication of the thesis or dissertation - should be regarded as the student's work, committee chair and members may be listed as secondary authors; dual publication - a manuscript should only be published in a single journal
controlling threats to internal validity
randomization of groups (avoid selection bias) and treatments; placebos (combat expectancy); blind setups; double-blind setups; reactive effects of testing (eliminate pre-test and add a control group)
expectancy
researcher threat; researchers anticipate specific responses and then bias or mess with the results; DV needs to be something that is judged for this threat to exist; if researcher cannot manipulate DV, this cannot exist; rating or motivating; use double-blind experiment to fix this
controlling threats to external validity
selecting from a larger population; ecological validity (basic vs applied, lab vs real world, competition vs simulated)
inductive reasoning
specific to general (think megaphone); start with a small group of particular examples of a specific phenomenon and then make generalizable statements about the phenomenon
halo effect
specific type of expectancy effect; research/rater has prior knowledge of participant; ensure rater is not familiar with participant
variance
square of standard deviation; AKA mean square; determines reliability of a test
signal to noise ratio
t-test ratio; difference between groups/variability of groups
main effects
test of each IV when all other IVs are held constant
coefficient of determination
the amount of variability in one measure that is explained by the other measure; R^2
Variability
the degree of difference between each individual score and the central tendency score
logical validity
the degree to which a measure obviously involves the performance being measured; also known as face validity
content validity
the degree to which a test (usually in educational settings) adequately samples what was covered in the course
construct validity
the degree to which a test measures a hypothetical construct; usually established by relating the test results to some behavior
validity
the degree to which a test or instrument measures what it purports to measure; can be categorized as logical, content, criterion, or construct; accuracy of a measure
predictive validity
the degree to which scores of predictor variables can accurately predict criterion scores
criterion validity
the degree to which scores on a test are related to some recognized standard or criterion
Cohen's d
the difference between the means for the two groups divided by an estimate of the standard deviation, often the average of the standard deviations of the samples/pooled standard deviation
standard error of prediction
the error associated with a linear regression prediction equation
correlation
the extent to which 2 variables are related or associated; a statistical technique used to determine the relationship between 2 or more variables
internal validity
the extent to which the results of a study can be attributed to the treatments used in the study
alpha
the probability of making a type 1 error; the significance level associated with a statistical test
Statistical Power
the probability of not missing an effect, due to sampling error, when there really is an effect there to be found; 1-beta; probability of correctly rejecting the null hypothesis when it really is false; ranges from 0 to 1
Calculating statistical power depends on
the sample size, level of statistical significance required, the minimum size of the effect that it is reasonable to expect
standard error
the variability of the sampling distribution of the statistic
hawthorn effect
threat to external validity; participant performs better when being observed
reactive or interactive effects of testing
threat to external validity; pre-test changes behavior of the participant; either don't have a pre-test or do not tell subjects the results; learn something about yourself, then change; cannot generalize results if participants know the results of the pre-test; prior knowledge or exposure to a treatment though a pre-test may make the response to treatment more effective than if a pre-test had not been given
maturation
threat to internal validity; changes seen in participants due to time; DV changing due to the passage of time; things that occur over time that lead to aging; ages 20-55 are stable; see in kids and old adults are effects due to aging or the IV? reduce time of study
experimental mortality
threat to internal validity; drop out of participants for non-random reasons or due to the design of the study; people who stay in the study are different in characteristics than the original sample
testing
threat to internal validity; learning effect; control by eliminating pre-test and by having familiarization days; effects of more than one test administration
selection bias
threat to internal validity; non-random assignment of groups so it is possible difference in results are not due to a treatment, but to group assignments; solve by random assignment to groups or matching groups
statistical regression
threat to internal validity; non-random assignment of groups which are selected based on extreme measures that eventually move towards the average; EX: groups w/ high and low vert jumps --> train --> low scores are higher and high jumpers may be a little lower; EX: motivation
selection-maturation interaction
threat to internal validity; one group has an advantage over another due to time; due to age; Selection differences between the participants in the intervention and the comparison groups lead to differences in maturation effects; non-randomly assigned groups of different ages, old vs young; groups start off different; time effects one treatment group that has a specific characteristic but des not effect the other group w/o that characteristic; solve w/ random assignment of groups
instrumentation
threat to internal validity; varying results due to improper or lack of instrument calibration or variation in the instruments used; use same piece of equipment for tests, only have one judge/scorer
1-tailed test
use if you can predict if the samples will go one direction or the other
descriptive statistics
used to summarize or condense a group of scores; include measures of central tendency (mean, median, mode) and measures of variability (standard deviation, variance, range)
plagiarism
using the ideas, writing, and drawings of others as your own
Misleading authorship
who should be an author? technicians do not necessarily become joint authors, authorship should involve only those who contribute directly; discuss authorship before the project
Effect size formula
(M1-M2)/S pooled ; puts data in standard deviation units
effect size differences
0 = no difference, 0.2 = small difference, 0.5 = medium, 0.8 = large
3 criteria for cause and effect
1. The cause must precede the effect in time 2. The cause and effect must be correlated with each other 3. The correlation between cause and effect cannot be explained by another variable
median is also known as
50th percentile or 2nd quartile; (n+1)/2
Dependent samples t-test
AKA paired t-test; used when samples have equal size and similar characteristics (reduced variance); repeated measures
intertester reliability
Consistency of measuring between two or more different testers