Research Methods Final Exam

normal curve

a distribution of data in which the mean, median, mode are at the same point (center of the distribution) and which 1 sd from the mean is 68%, 2 sd is 95%, 3 sd is 99%

regression line

a line that best describes the trend in the data; is as close as possible to the data points; y'=a+bx where a is the intercept and b is the slope

history

a threat to internal validity when the dependent variable is influenced by an event that occurred during the study; outside of study design, hardest to control and most likely threat; try to control as many things as possible

concurrent validity

a type of criterion validity in which a measuring instrument is correlated with some criterion that is administered concurrently or at about the same time

Factorial ANOVA

an ANOVA in which there is more than one IV; used more often than simple ANOVAs; three different F tests; has main effects and interactions

Repeated measures ANOVA

analysis of scores for the same individuals on successive occasions, such as a series of test trials; problems include carryover effects and practice effects ; individual differences can be identified and separated from the error term, increasing power; reduces variation; usually used over time; DV measured multiple times

ANOVA

analysis of variance among 2 or more means; with 2 mean the F statistic yields the same probability we would obtain with a t-test; measures 2 sources of variation (w/in and between) and compares their relative sizes

interaction of selection biases and treatment

cannot generalize unless they have same characteristic; treatment may only work on participants with specific characteristics; non-randomly selection participants based on a common characteristic; if you use trained subjects, you cannot generalize to untrained subjects

how to combat instrumentation effects

choose the best test that doesn't have calibration issues; train observers well and beware of halo effects

Faulty data gathering

collecting data from participants who are not complying with the requirements of the study; using faulty equipment; treating participants inappropriately; recording data incorrecty

2 way repeated measures ANOVA

common; group x time; 2x2 repeated measures ANOVA

F test and equation

compares the variability between groups to the variability within groups; F = variation between / variation within

reliability

consistency of measure

Nonpublication of data

cooking data; data not included in results because they don't support the desired outcome

poor data storage and retention

data should be stored in its original collected form for at least 3 years after publication; data should be available for examination; confidentiality of participants should be maintained

Independent samples t-test

determines whether 2 unique samples are different; sample size can be unequal; equal or unequal variance

evaluating the null hypothesis for t-tests

establish are the 2 groups different and how meaningful are the differences

how to combat experimental mortality

explain the need for participation well at the beginning

descriptive/intuitive reasoning

general to specific; take an example of something and then vary the example to see if the phenomenon in question is still present or not

external validity

generalizability of a study

2 tailed t-test

if no prediction which direction the samples will differ

multiple-treatment intereference

if subjects participate in multiple treatment groups, the previous treatment may inhibit or enhance the next treatment; more than one treatment

philosophic research methods

inductive reasoning and descriptive reasoning

Large F values

large F values reject the null hypothesis b/c they show more variation between groups than within groups

Fabrication and Falsification

making up or altering data

Central Tendency

measures of central tendency describe the average or common score of a group of scores; common measures include mean, median, mode; a single score that best represents all the scores

correlation coefficient

measures the strength and direction of a relationship between 2 variables; ranges between -1 to 1 and is identified by r

data gathering

most important and most aggravating; always drop non-compliers; fix broken equipment; treat subjects well; store data in a safe and private place for 3 years

cause and effect

no other reasonable explanation exists for the change in the DV except for the manipulation of the IV; due to good theoretical framework, proper experimental design, use of correct stats, controlled variables, appropriate DV, correct interpretation of results

Type 2 error

null hypothesis is false and you fail to reject it; when there is an effect but the stat test comes out as non-significant; due to inadequate power, sampling error

Type 1 error

null hypothesis is true and you reject it; there is no effect but the stat test comes out significant

one-way/simple ANOVA

one independent variable for more than 2 groups; has varying levels

reactive effects of experimental arrangements

one setting vs another setting; lab setting may limit real-world application and influence generalizability; results in a controlled lab setting may not be seen in a real world setting

avis effects

participants try harder b/c they are in the control group; include placebo group

7 areas of scientific dishonesty

plagiarism, fabrication and falsification, nonpublication of data, faulty data-gathering procedures, poor data storage and retention, misleading authorship, sneaky publication practices

Interaction

present when the differences between the groups of one IV on the DV vary according to the level of a second IV

beta

probability of making a type 2 error

Informed consent form

provides potential subject with info to make a sound decision about participating in a study; provides simple but comprehensive info about the study

Sneaky publication practices

publication of the thesis or dissertation - should be regarded as the student's work, committee chair and members may be listed as secondary authors; dual publication - a manuscript should only be published in a single journal

controlling threats to internal validity

randomization of groups (avoid selection bias) and treatments; placebos (combat expectancy); blind setups; double-blind setups; reactive effects of testing (eliminate pre-test and add a control group)

expectancy

researcher threat; researchers anticipate specific responses and then bias or mess with the results; DV needs to be something that is judged for this threat to exist; if researcher cannot manipulate DV, this cannot exist; rating or motivating; use double-blind experiment to fix this

controlling threats to external validity

selecting from a larger population; ecological validity (basic vs applied, lab vs real world, competition vs simulated)

inductive reasoning

specific to general (think megaphone); start with a small group of particular examples of a specific phenomenon and then make generalizable statements about the phenomenon

halo effect

specific type of expectancy effect; research/rater has prior knowledge of participant; ensure rater is not familiar with participant

variance

square of standard deviation; AKA mean square; determines reliability of a test

signal to noise ratio

t-test ratio; difference between groups/variability of groups

main effects

test of each IV when all other IVs are held constant

coefficient of determination

the amount of variability in one measure that is explained by the other measure; R^2

Variability

the degree of difference between each individual score and the central tendency score

logical validity

the degree to which a measure obviously involves the performance being measured; also known as face validity

content validity

the degree to which a test (usually in educational settings) adequately samples what was covered in the course

construct validity

the degree to which a test measures a hypothetical construct; usually established by relating the test results to some behavior

validity

the degree to which a test or instrument measures what it purports to measure; can be categorized as logical, content, criterion, or construct; accuracy of a measure

predictive validity

the degree to which scores of predictor variables can accurately predict criterion scores

criterion validity

the degree to which scores on a test are related to some recognized standard or criterion

Cohen's d

the difference between the means for the two groups divided by an estimate of the standard deviation, often the average of the standard deviations of the samples/pooled standard deviation

standard error of prediction

the error associated with a linear regression prediction equation

correlation

the extent to which 2 variables are related or associated; a statistical technique used to determine the relationship between 2 or more variables

internal validity

the extent to which the results of a study can be attributed to the treatments used in the study

alpha

the probability of making a type 1 error; the significance level associated with a statistical test

Statistical Power

the probability of not missing an effect, due to sampling error, when there really is an effect there to be found; 1-beta; probability of correctly rejecting the null hypothesis when it really is false; ranges from 0 to 1

Calculating statistical power depends on

the sample size, level of statistical significance required, the minimum size of the effect that it is reasonable to expect

standard error

the variability of the sampling distribution of the statistic

hawthorn effect

threat to external validity; participant performs better when being observed

reactive or interactive effects of testing

threat to external validity; pre-test changes behavior of the participant; either don't have a pre-test or do not tell subjects the results; learn something about yourself, then change; cannot generalize results if participants know the results of the pre-test; prior knowledge or exposure to a treatment though a pre-test may make the response to treatment more effective than if a pre-test had not been given

maturation

threat to internal validity; changes seen in participants due to time; DV changing due to the passage of time; things that occur over time that lead to aging; ages 20-55 are stable; see in kids and old adults are effects due to aging or the IV? reduce time of study

experimental mortality

threat to internal validity; drop out of participants for non-random reasons or due to the design of the study; people who stay in the study are different in characteristics than the original sample

testing

threat to internal validity; learning effect; control by eliminating pre-test and by having familiarization days; effects of more than one test administration

selection bias

threat to internal validity; non-random assignment of groups so it is possible difference in results are not due to a treatment, but to group assignments; solve by random assignment to groups or matching groups

statistical regression

threat to internal validity; non-random assignment of groups which are selected based on extreme measures that eventually move towards the average; EX: groups w/ high and low vert jumps --> train --> low scores are higher and high jumpers may be a little lower; EX: motivation

selection-maturation interaction

threat to internal validity; one group has an advantage over another due to time; due to age; Selection differences between the participants in the intervention and the comparison groups lead to differences in maturation effects; non-randomly assigned groups of different ages, old vs young; groups start off different; time effects one treatment group that has a specific characteristic but des not effect the other group w/o that characteristic; solve w/ random assignment of groups

instrumentation

threat to internal validity; varying results due to improper or lack of instrument calibration or variation in the instruments used; use same piece of equipment for tests, only have one judge/scorer

1-tailed test

use if you can predict if the samples will go one direction or the other

descriptive statistics

used to summarize or condense a group of scores; include measures of central tendency (mean, median, mode) and measures of variability (standard deviation, variance, range)

plagiarism

using the ideas, writing, and drawings of others as your own

Misleading authorship

who should be an author? technicians do not necessarily become joint authors, authorship should involve only those who contribute directly; discuss authorship before the project

Effect size formula

(M1-M2)/S pooled ; puts data in standard deviation units

effect size differences

0 = no difference, 0.2 = small difference, 0.5 = medium, 0.8 = large

3 criteria for cause and effect

1. The cause must precede the effect in time 2. The cause and effect must be correlated with each other 3. The correlation between cause and effect cannot be explained by another variable

median is also known as

50th percentile or 2nd quartile; (n+1)/2

Dependent samples t-test

AKA paired t-test; used when samples have equal size and similar characteristics (reduced variance); repeated measures

intertester reliability

Consistency of measuring between two or more different testers