Quiz 3:
compares the variance of the means of the dependent variable between the different levels to the variance of individuals on the dependent variable within each of the conditions.
ANOVA
ANOVA vs paired t-test
ANOVA's can be used for multiple comparisons of two or more groups and for multiple IV's. T-test for comparing DV means between two groups on a single IV.
every subject is scored in 3 different conditions of words; related, abstract, and unrelated. why kind of test would we use to compare the means?
ANOVA--- comparing 3 or more means
using the F statistic, when can we conclude that the manipulation has impacted the dependent measure
If the between-groups variance is significantly greater than the within-groups variance
type 1 error
When we reject the null hypothesis when it is in fact true. We conclude that results are statistically significant so reject h0, but there was actually no significant difference between groups so h0 was in fact true. The Probability of making a type 1 error is equivalent to alpha or your significance level
taking a larger sample will ___ the power of your test and therefore ____ the likelihood of a type 2 error
a larger sample increases the power to decrease the likelihood of a type 2 error (having a larger power means that beta is smaller--- power= 1-beta)
disconfirming evidence
evidence that goes against the research hypothesis
what kind of test is more common, one or two tail tests and why?
non-directional tests (two tailed tests) are more common because we don't always know how things are going to turn out --- it is better to detect both extremes ex) music condition doing much worse or much better than the control
degrees of freedom for between group studies
number of conditions minus 1
you want to know if a new drug is better or worse than another drug. what kind of test would you use? What is h0 and h1?
one-tailed t-test (directional test) h0: new drug is equal to or worse than the new drug h1: new drug is better
within-subject deigns t-test is called
paired samples t-test
mode
the most frequently occurring score(s) in a distribution
within-participant designs
- participants serve as their own control group - also called repeated measure designs within-participants (within-subjects) design = differences across the different levels are assessed within the same participants. Within-participants designs are also called repeated-measures designs because the dependent measure is assessed more than one time for each person.
disadvantages of repeated measure deigns
- practice effects - fatigue - Carryover- When effects of one level of the manipulation are still present when the dependent measure is assessed for another level of the manipulation (the second measure could still be influenced by the first condition)
null hypothesis
Establish the null before analyzing results. each test of a research hypothesis begins with a null hypothesis Assume that the observed data do not differ from what would be expected on the basis of chance The assumption that the observed data reflect only what would be expected under the sampling distribution is called the null hypothesis, symbolized as H0.
If the between-groups variance is significantly greater than the within-groups variance, then what can we conclude? why?
conclude that the manipulation has influenced the dependent measure because the influence of the manipulation across the levels is greater than the random fluctuation among individuals within the levels. A statistic called F is calculated as the ratio of the two variances:
every subject is scored in 3 different conditions of words; related, and unrelated. why kind of test would we use to compare the means?
paired sample t-test--- each subject does both conditions because we are only comparing two means we can use a t-test (if there are more than 2, use ANOVA)
drawback of using the F-statistic
significant F again indicates that there are differences on the dependent variable among the levels and thus that the null hypothesis (that all of the means are the same) can be rejected. BUT a significant F does not tell us which means differ from each other.
test that is a special case of the F test that is used only for comparison of two means.
t-test
In a one-way experiment with only two levels, a statistically significant F tells us what?
that the means in the two conditions are significantly different. However, the significant F only means that the null hypothesis can be rejected.
what is the F-statistic more useful compared to a t-statisitc?
the F test is more general allowing the comparison of differences among any number of means, it is more useful. t-test only allows us to compare two means
mean
the arithmetic average of a distribution, obtained by adding the scores and then dividing by the number of scores
assuming h0 is false, what is the probability of making the correct decision and rejecting the null?
1-beta (power)
significance of the p-value
Each statistic has an associated probability value (usually called a p-value and indicated with the letter p) that shows the likelihood of an observed statistic occurring on the basis of the sampling distribution(th likelihood of getting results by chance) We compare the p-value to alpha to decide whether to accept or reject the null hypothesis
when to reject the null using the p-value
If the p-value is less than alpha (p , .05), then we reject the null hypothesis, and we say the result is statistically significant.
how to account fo rat trade off between type 1 and type 2 errors?
Increase the power of your test: - increase the sample size As N increases, the likelihood of the scientist detecting relationships with small effect sizes increases, even when alpha remains the same.
do within-subject or between-subject designs have greater power?
One major advantage of repeated-measures designs is that they have greater statistical power than between-participants designs.
research hypothesis
a statement that the groups are different groups will have different scores on the dependent variable and these differences are likely not due to chance
probability of a type 1 error
alpha
probability of a type 2 error
beta
If p-value greater than alpha (.05), you_______ the null hypothesis and your result is ______.
fail to reject the null an your results are non-significant
directional test/ one-tailed t-test
if it is one-directional it means you want to test if something is equal or (worse/better) than the old thing A directional test is a hypothesis test that includes a directional prediction in the statement of the hypotheses and places the critical region entirely in one tail of the distribution. ex) you are testing a new drug- is it better than the old drug, or is their no difference?
Means can differ but if the scores within each group are highly variable, T-test will be ____ likely to be significant.
less ideally we want to see more variability between groups than within groups
you want to measure if there is a significant difference between males and female performance on a math test. what type of test would you use?
looking at the difference between two means so use a t-test because it is between subjects (male vs female) use an independent sample t-test
Type 2 errors are more common when the power of a statistical test is ___
low
Distribution
showing all the possible values (or intervals) of the data and how often they occur shoes us the shape of data
a p-value that is less than alpha is considered _____ _____
statistically significant
range
the difference between the highest and lowest scores in a distribution
Research Hypothesis (H1)
the hypothesis that the experiment was designed to investigate
median
the middle score in a distribution; half the scores are above it and half are below it
when F increases, what happens to the p-value?
F has an associated p-value, which is compared to alpha. - a larger F means that the associated p-value will be smaller, and if the p-value is less than our significance level we can say that our results are statistically significant If the p-value is less than alpha, then the null hypothesis (that all the condition means are the same) is rejected.
forumla for f statistic
F= between groups/within groups variance
type 2 error
Failing to reject the null when the null is actually false Type 2 errors occur when the scientist misses a true relationship by failing to reject the null hypothesis even though it should have been rejected Probability of making a type 2 error is Beta Type 2 errors are more common when the power of a statistical test is low
if alpha=.05, ____ null hypothesis (the groups are not equal). this means that results had less than 5% probability of occurring ____ _____
reject by chance
degrees of freedom for within-group studies
number of participants minus the number of conditions
directional tests have ____ tail (s)
one
advantages to repeated measure (between-subject designs)
- greater power - better economy--- require fewer participants because each participant is subject to each condition
basic steps in hypothesis testing (for a t-test)
1- determine your hypothesis (set up a null hypothesis and research hypothesis) 2- set alpha (usually 0.05) 3- Run the study and collect data 4- calculate your t-value (or other stat) and the corresponding p-value 5- compare your p-value to alpha to determine whether or not to accept or reject your null hypothesis
what effects beta?
1- effect size (magnitude of a relationship) 2- sample size (the larger the sample the greater the power)
two types of t-tests
1- independent sample t-test (between participant design) 2- paired sample t-test (repeated measure design)
assuming h0 is true, what is the probability of making the correct decision to fail to reject the null?
1-alpha
When alpha is .05, we know we will make a Type 1 error not more than....
5% of the time; 5 times out of 100`
if you reject the null then your experiment ____ work
DID
trade off between type 1 and type 2 errors
For any given sample size, when alpha is set lower, beta will always be higher. This is because alpha represents the standard of evidence required to reject the null hypothesis, and the probability of the observed data meeting this standard is less when alpha is smaller. As a result, setting a small alpha makes it more difficult to find data that are strong enough to allow rejecting the null hypothesis, and makes it more likely that weak relationships will be missed
you are testing how classical music impacts how people do on a math test: what is the null hypothesis and what is your research hypothesis?
H0 (Null hypothesis): Scores on the math test in the music and non-music conditions are equal. H1 (Research Hypothesis): Scores on the math test for the music and non-music conditions are not equal.
when to fail to reject the null using the p-value
If the p-value is greater than alpha (p , .05), then we fail to reject the null hypothesis, and we say the result is statistically nonsignificant.
what the power of a statistical test?
The power = probability that the researcher will, on the basis of the observed data, be able to reject the null hypothesis given that the null hypothesis is actually false and thus should be rejected. Power is the probability of making the correct decision power can be written in terms of beta: Power= 1- beta As sample size increases so does the power
relationship between the standard deviation and spread?
The standard deviation is small when the data are all concentrated close to the mean, exhibiting little variation or spread. The standard deviation is larger when the data values are more spreadout from the mean, exhibiting more variation.
effect size
a measure of the strength of the relationship between two variables or the extent of an experimental effect An effect size is a measure of how important a difference is: large effect sizes mean the difference is important; small effect sizes mean the difference is unimportant. ex) It normalizes the average raw gain in a population by the standard deviation in individuals' raw scores, giving you a measure of how substantially the pre- and post-test scores differ.
standard deviation
a measure of variability that describes an average distance of every score from the mean it is the square root of the variance
F statistic
a ratio of two measures of variance: (1) between-groups variance, and (2) within-groups variance F= between groups/within groups variance
what is hypothesis testing?
a set of procedures based on inferential statistics that is designed to determine whether of not observed data can be interpreted as support for a research hypothesis
null hypothesis
a statement or idea that can be falsified, or proved wrong a statement that the groups are not different and that they are really equal any difference that shows up in the experiment will be due to chance variations
t-test
a statistical test used to evaluate the size and significance of the difference between two means two kinds of t-tests used to compare means: - either a between-participants design (an independent samples t test) - or a repeated-measures design (a paired-samples t test).
define sample
a subgroup of a population samples can tell us about the greater population
non-directional test
a two-tailed test. either direction of difference or relationship supports the alternate hypothesis just testing if there is in fact a difference between conditions
analysis of variance
also known as ANOVA Basically, you're testing groups to see if there's a difference between them.
what is ANOVA
analysis of variance is used to compare three or more means
why is it important to not have too strict fo a significance level (alpha)?
because as alpha decreases, beta increases so we may be decreasing our chance of making a type 1 error but increase our chance of making a type two error- we may miss a real difference between groups when alpha is too strict
error
errors lead the researcher to draw invalid conclusions, it is important to understand what they are and how we can reduce them. We can wrongly decide to conclude that the null hypothesis may be true, or the null hypothesis may be false
how to report if results are significant
give the degrees of freedom and t-value and corresponding p-value if the p-value is smaller than alpha results are considered significant (if not, not significant) t(degrees of freedom)=t value, p<>alpha.
As the condition means differ more among each other (_____) in comparison to the variance within the conditions(____), F ______.
if between group variances differ more than within-group variances, F increases
if power is low then beta is ____ and the probability of making a type ___ error is ____
if power is low then beta is high and the probability of making a type 2 error is high
as we making alpha lower, then beta gets bigger. SO, as the probability of a type 1 error decreases, the probability of a type 2 error _____
increases --- we may miss significant relationships too strict of an alpha has a cost
t-test that is used for between subject designs
independent sample t-test ex)male vs female; fall vs winter
One-Way ANOVA
is used to test the hypothesis that several independent groups come from populations with the same mean. For example, you can test the null hypothesis that three different mnemonics result in the same average memory performance, or that the average response time is the same for detecting one of four different stimuli. To see which groups are significantly different from each other, you can use the multiple comparison procedures available in this analysis: They tell you which group is different from which other group
T-test takes into account both ____ and ____ in each group.
means and variability
you want to test if there is a difference between taking a test after listening to music compared to just taking the test (and not listening to music). if you were to conduct a non-directional test how many tails? what would h0 and h1 be?
non-directional= two-tailed test (just want to see if there is a difference, don't care about which direction) h0: there is no difference between music and no music h1: music and non-music conditions are not equal
alpha is also known as
the significance level
formula for standard deviation
the square root of the variance
Although setting a lower alpha protects us from Type 1 errors, doing so may lead us to miss the presence of weak relationships
trade off between type 1 and type 2 error
non-directional tests have ____ tail(s)
two
Reject null hypothesis even though it is true. Groups DO NOT really differ, any observed difference was just due to chance (p=alpha).
type 1 error
reject the null even though it is true
type 1 error
the following is an example of: you reject the null hypothesis but the non-music group had more people who were bad at math (We randomly assigned participants but this could still happen). So their scores were lower, but music really has no effect on math performance
type 1 error
assuming h0 is true, what is the probability of rejecting the null (making the wrong decision).
type 1 error p= alpha (usually .05 or 01)
Fail to reject null hypothesis even though it is false. Groups really do differ, but we think they are the same. We don't detecta real difference
type 2 error
fail to reject the null even though it is false
type 2 error
the following are examples of: This would occur when the scientist concludes that the psychotherapy program is not working even though it really is or when you conclude that your friend does not have ESP even though she really can do significantly better than chance.
type 2 error
this error occurs when we fail to notice a true relationship and conclude that the null hypothesis is true when it is really false
type 2 error
assuming h0 is false, what is the probability of failing to reject the null?
type 2 error beta
what is the relationship between power and type 2 errors?
type 2 errors are more common when the power of the test is low if the power is low, then beta is high
multiple comparisons
used in one-way anova testing- tells us which group is different from which other group
what are inferential statistics
using sample data to make inferences about the true state of affairs
what is alpha usually set to? and what does this mean?
usually set to 0.05 or 0.01 this means that there is a 5% chance or a 1% chance that we would get the results by chance
between-groups variance
variance among the condition means
within-groups variance
variance within the conditions
variance
variance= measure of the dispersion of scores on a variable variance is the expectation of the squared deviation of a random variable from its mean. It measures how far a set of (random) numbers are spread out from their average value.
what do we need to know in order to accept or reject the null?
we need to figure out the p-value (probability value)--- the likelihood that results were a result of chance
when do we expect to get a large t-value and thus a small p-value
when variability between group means is large and variability within groups is small