All flash cards for PSY3062
what are the steps to conducting a planned comparison- independent measures
Assign contrast weights for each group- signs of weights, magnitude (numbers) and weights The weights must equal zero We continuing onto the next contrast we must give any contrasts that will not be involved (e.g. control group) a 0. In SPSS, analyse > compare > one-way ANOVA Options > place a tick next to descriptives, homogeneity of variance test, Brown-Forsythe, and Welch, means plot Contrast > assign coefficients - must be in the same order as variables in data > the coefficient total should equal 0 Next > add coefficients for second contrast Press okay
Post hoc tests
No specific hypothesis at the outset. Uses stringent (conservative) aPW (pair wise) for each test to control family-wise type 1 error rate. In order to a post hoc-test properly you must have a significant ANOVA.
SS between subjects
P = total of the scores for each participant K= number of treatment levels G = grand total for all scores N = total number of scores
T-tests
Parametric, tests the difference between two sample means.
what are planned comparisons?
Planned comparisons (priori contrasts): Specific hypothesis at outset. Breaks down treatment variance (model variance) into component parts PC are conducted when we have specific hypotheses about differences amongst the groups means before running the ANOVA You set up your comparisons at the same time as an ANOVA We break down the treatment variance into component variance according to our hypothesises made before et experiment Providing that the hypotheses are independent of one another, the family wise type 1 error will be controlled
keep contrasts independent
a. Contrasts should be orthogonal to each other b. Each unique chunk can only be compared once→ otherwise type 1 error will be inflated c. K-1 possible orthogonal contrasts (K= number of groups) if you follow this rule and always compare only two pieces of variance then you will always end up with one less contrast than the number of groups (k-1)
sphericity
assumes that the variances and covariances of the differences between the levels are approximately the same. if we calculated difference scores for all possible pairs of levels, their variances would be approximately equal special case of compound symmetry - less restrictive form because it is an assumption we want it to be non-significant
Post hoc recommendations
assumptions met and equal sample size use- REGWQ or Tukey HSD safe option Bonferroni- although, quite conservative unequal sample sizes- Fabriels (small discrepancy in n) or Hochberg's GT2 (large discrepancy in n) unequal variances-Games-howell
model sum of squares SSm
broken into three components: SSA, SSB, and SSAB we can calculate SSAB by using subtraction SSAB = SSM - SSA- SSB
using z-scores to find outliers
by converting our data to z-scores we can use benchmarks that we can apply to data sets to search for outliers we go into analyse- descriptive statistics- descriptives dialog box select he variable to convert and tick the save standardised values SPSS will then create a new variable in the data editor to look for outliers we can count how many z-scores fall within certain important limits in a normal discussion we'd expect about 5% to be greater than 1.96, 1% to have absolute values greater than 2.58 and none to be greater than about 3.29
the residual sum of squares, SSr
calculated the same was as in one-way anova SS1 + SS2 + SS3 etc it represents individual differences in performance or the variance that can't be explained by factors that were systematically manipulated
reciprocal transfomr: 1/x
can reduce influence of large scores and stabilise variance, reverses scores, so you may need to reverse before transform
square root transform: sqrt (x)
can reduce positive skew and stabilise variance
transform the data
can sometimes resolve distributional problems, outliers, unequal variances involves applying a mathematical function to all data points must exercise caution when using data transformations as changes hypothesis tested e.g. when using a log transformation and comparing means you change from comparing arithmetic means to comparing geometric means
how many levels?
can use numerical code to designate the number of levels of each factor 2 X 3 means that there are 2 levels of factor A and 3 levels of factor B- because there are two numbers, and therefore, two factors, it is a two-way design 2 X 3 X 3 means there are 2 levels of factor A, 3 levels of factor B, and 4 levels of factor C-because there are three numbers, therefore, three factors, it is a three-way design
what do we use to assess the sample distribution?
central limit theorem
output of two-way repeated anova
descriptive statistics- look at Std. deviation and note whether they are similar across the different levels- can be a way of identifying extreme outliers we the multivariate tests when sphericity is violated and for some reason we can't use the adjusted test hyena-feldt. perhaps they are too conservative and you want to us more powerful tests. if the assumption of sphericity is not violated we view sphericity is assumed row
obtaining good data
designing research- do not alter or embellish standard scales without clearly indicating the changes collecting the data- participate activity in the data collection, check to ensure that research instruments have been completed without omissions coding and analysing the data- check the scoring key against the survey instrument, check for reserve questions, recode so that all items purporting to measure same characteristic scored in same direction, transform scale and item values to be consistent with scoring key, know your measurement instruments (what populations have they been used on), weight scales according to standard scoring procedures, assign unique numbers to missing values to distinguish them from other responses, check the order of data input, make sure data sets are comparable before combining them to make large sample describing the data- label compulsively, data should have clear and permanent labels, do not discard raw data
why factorial design?
enhances external validity: by not limiting the study to examining the influence of only one IV on the DV you can improve the generalizability of the study's findings more efficient: we can use fewer participants than compared to if we did separate studies on gender/depression scores and diagnosis/depression scores. Whilst factorial designs require more participants and resources than a single factor study, compared to conducting several single factor studies to thoroughly examine a particular DV, conducting one factorial design will be more efficient and provide more information. test for interactions between factors: Factorial designs cannot only examine the effects of each IV on its own, but also the combined effects of the IVS- the interaction between factors. used to control systematic and non-systematic variance: Including participant characteristics such as age or gender as IVs and reduce the error variance of the analysis- which will make the statistical analysis more powerful Including additional variables also enables you to test whether these variables are acting as confounding variables
parameter estimation with confidence intervals
if null hypothesis value is outside 95% confidence intervals, can reject he null gives you an idea of how precise your study is
Central limit theorem
if our N is large enough, then the sampling distribution will be approximately normal even if our sample distribution or population is non-normal a large N is >30-40 people if our N is small, we rely on our sample if sample distribution is normal, then the sampling distribution will also probably be normal if can;t assume, use non parametric
spotting linearity
if the graph funnels out, then the chances are that there is heteroscedasticity in the data if there is any sort of curve in this graph then the chances are that the data has broken the assumption of linearity
when do we use greenhouse gessier correction?
if the greenhouse-geisser is less than 0.75 we use this correction - it is a conservative method
how can box plots tell us whether the distribution is symmetrical or skewed?
if the whiskers are the same length then the distribution is symmetrical if the top or bottom whisker is much longer than the opposite whisker then the distribution is asymmetrical
including references for assumption violations
if you have assumption violations or outliers that need to be dealt with - you need to state how you have dealt with them you should a cite a reference to support what you have done
orthogonal means...
independent therefore orthogonal contrasts are contrasts which all tap into different things no redundancy they are unique chunks of variance
chronbach's alpha
is a measure of internal consistency it is used to assess the extent too which a set of questionnaire items tapping a single underlying construct covary values above 0.7 are considered acceptable.
what are the methods of follow up for ANOVA?
multiple t-tests? Not a good idea- increases our type 1 error we could do orthogonal contrasts/comparisons- this what we do when you have a hypothesis post hoc test- use when you DO NOT have a planned hypothesis, compare all pairs of means, in order to use post hoc tests properly we have to have a SIGNIFICANT ANOVA trend analysis- if we believe the means follow a particular shape
non-orthogonal comparisons
non-orthogonal comparisons are comparisons that are in some way related using a cake analogy- non-orthogonal comparisons are where you slice up your cake and then try to stick slices of cake together again the comparisons are related so the resulting test statistics and p-values will be correlated to some extent for this reason you should use a more conservative probability level to accept that a given contrast is statistically meaningful
non-orthogonal comparisons
non-othogonal contrasts are comparisons that are in some way related using a cake analogy- non-orthogonal comparisons are where you slice up your cake and then try to stick slices of cake together again e.g. use placebo in first contrast and then re-use it if you multiple and add the codings from the two contrasts the sum is not zero- the contrasts are not orthogonal with non-orthogonal contrasts, the comparisons you do are related and so the resulting test statistics and p values will be correlated to some extent for this reason you should use a more conservative probability level to accept that a given contrast is statistically meaningful
parametric assumptions
normally distributed: bell curve, there is no skewness & kurtosis, assume that there is no outliers, sampling distribution, residuals homogeneity of variance and homoscedasticity independence
general equation for a linear model
outcome = model + error outcome (bX) + error the model is described by one or more predictor variables (the X in the equation) and parameters (the B) that tell us something about the relationship between the predictor and the outcome variable this model will not predict the outcome perfectly so for each observation there will be some error
________design is more powerful than_________design because they remove individual differences
repeated measures; independent measures
SPSS column (data view)
represents each variable
MSr
represents the average amount of variation explained by extraneous variables- the unsystematic variation
mean squares MSm
represents the average amount of variation explained by the model- systematic variation
SPSS column (variable view)
represents the different options that you can select for each variable
within-subjects design
same participants take part in each condition one sample for level of the IV also known as repeated measures
the effects of kurtosis seem unaffected by whether _______
sample seizes are equal or not
effect size
small effect sizes will be significant with large n larger the sample size the more likely you will find an effect effect sizes indicate how big that effect is it allows comparisons across contexts
In big samples__________ deviations from sphericity can be significant, and in small samples ________violations can be non-significant
small; large
Bonferroni has more power when the number of comparisons is_____, whereas Turkey is more powerful when testing______
small; large number of means
one-way ANOVA
statistical technique for comparing several means at once tests the null hypothesis that the means for all populations of interest are equal we call it the omnibus test - that somewhere there is an inequality- but it doesn't tell us where this difference actually is this is why need a follow up to work out where this difference is- there are two approaches: post hoc or planned comparisons/contrast
what does 'reward > (punishment + indifference)/2'
that reward will be greater than the means of punishment and indifference
if the f-ratio is greater than 1 it is indicates______
that the experimental manipulation had some effect above the effect of individual differences in performance
kolmogorov-Smirnov and shapiro-wilk test
they compare the scores in the sample to a normally distributed set of scores with the same mean and standard deviation ananlyse-descrptive statistics- explore
additivity and linearity
this assumption means that the outcome variable is linearly related to any predictors and if you have several predictors than their combined effect is best described by adding their effects together this assumption is the most important because if it is not true then even if all other assumptions are met, your model is invalid because you have described it incorrectly
trim the data
this means deleting some scores from the extremes could mean deleting the dat from the person who contributed the outlier trimming involves removing extreme scores using one of two rules; percentage based rule and a standard deviation based rule.
leptokurtic distributions make the type 1 error rate ______ and consequently the power is______
too low; too high
SStotal
total variance in the data sum of squared differences between each score and the grand mean
transforming data
transforming the data can sometimes resolve distributional problems involves applying a mathematical function to all data-points transformations can hinder rather than help if the wrong transformation is applied and the interpretation also becomes more difficult
how do we reduce bias?
trim the data winsorizing analyse with robust methods transform the data
when sphericity is definitely not violated ___________can be used
tukey's test
factorial
two or more IVs
reject the null hypothesis when it is true
type 1 error (a)
accept the null hypothesis when it is false
type 2 error (b)
_______ variances create a bias and inconsistency in the estimate of the standard error associated with the parameter estimates
unequal
Scale (interval/ratio data)
values represent ordered categories with a meaningful metric. this means that distance comparisons between values are appropriate e.g. height, weight, years, dollars etc.
SSmodel or SS between
variance explained by the model- groups sum of squared differences between each group mean and the grand mean weighted by group n the more groups means spread out the greater the sum of between will be SStotal- SS within treatments Between groups= our model = SS model = SS between
SSresidual or SSwithin
variance that cannot be explained by the model-groups sum of squared differences between each score and the group mean SSwithin treatments= SS1 + SS2 + SS3 etc. within groups = our error = SS residual = SS within
how do we assess normality
visually: using histograms and p-p plots statistically- tests for normality, test for skew/kurtosis
underlying theory of between-subjects ANOVA
we calculate how much total variability there is between scores- total sum of squares we then calculate how much of this variability can be explained by the model we fit to the data: model sum of squares and then how much cannot be explained- how much variability is due to individual differences or error - residual sum of squares
underlying theory of a two way ANOVA
we still find the SSt and break this variance down into variance that can be explained by the experiment (SSM) and variance that cannot be explained (SSR) in two way ANOVA, the variance explained by the experiment is made up of not one experimental manipulation but two therefore we break the model sum of squares down into variance explained by the first IV (SSA), second IV (SSB) and variance explained by the interaction of these two variables (SSA X SSB)
ANOVA notation
we use subscript notation to represent and locate individual scores in an experiment single subject notation Xi the ith score in a set
confidence intervals
we use values of the standard normal distribution to compute the confidence interval around a parameter estimate. using the values of the standard normal distribution makes sense only if the parameter estimates actually come from one
Are planned comparisons more powerful than post-hoc tests?
yes
planned contrasts
you can do planned contrasts for a two-way design but only for the main effects you can run contrasts involving one IV at a time if you have predictors to test.
non-singificance
you might obtain non-significant findings for the following reasons: there may really be no difference between conditions you study may have been poorly designed/controlled, which resulted in the lack of a difference or a large error term or you study may not have been powerful enough to detect a difference- e.g. because your sample size is too small it is important to note that non-significant results are not necessarily indicative of a failed experiment a non-significant result can be very interesting and important finding
dealing with outliers
you should check you data for outliers you convert all scores into z-scores and if any have a value of Z>3.29 (p<0.001)- they are outliers could also change the score (windorsing) - change the score to the next highest score +1 - convert the score to the score expected for a Z score of 3.29 - convert the score to the mean plus 2 or 3 standard deviations
contrasts in SPSS
• For one-way ANOVA, SPSS has a procedure for entering codes that define the contrasts we want to do. However, for two-way ANOVA no such facility exists and instead we are restricted to doing one of several standard contrasts. • We can use standard contrasts • In reality, most of the time we want contrasts for our interaction term, and they can be obtained only through syntax. • To get contrasts for the main effect of factor 1, click on contrast in the main dialog box.
why can we use planned comparisons instead of post hoc tests?
• Post-hoc tests control for inflated family-wise error o Make critical a smaller to account for type 1 error inflation due to multiple comparisons o This is good→ keeps our experiment-wise (family wise) error at a o It is conservative - reduces our power- makes harder to find our effected interest o It means we require larger effect sizes to reach significance • This may be too conservative, especially if our sample size is small and/or we have many groups or levels to compare
using syntax to do simple effects analysis
• Setup the analysis as normal, but when you're ready to run it, instead of pressing OK, press paste- which will paste the analysis info into a new syntax file • Type an extra command in your syntax file e.g. /EMMEANS=TABLES (GENDER*ALCOHOL) COMPARE (ALCOHOL) ADJ (BONFERRONI) • Compare(alcohol) tells SPS to examine the interaction by comparing the different alcohol levels one gender at a time • ADJ(bonferroni) this will conduct post-hoc tests for the differences between alcohol levels by gender.
custom contrasts
• Syntax is like the underlying code to SPSS • You can generate syntax for any analysis using the paste button o Number of benefits you can see how syntax works or save as a syntax file • Looks something like this→
heterogeneity and sample size
• Unequal sample sizes and heterogeneity do not mix • Gamst et. Al (2008) provide further information about this situation: • Heterogeneity can become a serious issue with unequal sample sizes. For example, large group variances associated with small sample sizes tend to produce a liberal F statistic whose nominal alpha level such as .05 is actually operating at a less stringent level such as .10. • Conversely, when the situation arises where large sample sizes and large group variances are associated, we produce a conservative F statistic, where the nominal alpha level such as .05 is actually more stringent, for example, .01 (Stevens, 2002). (p. 58)
leptokurtic
+ heavy tails
power analysis in SPSS
- Ask SPSS to run (post hoc) power analyses using sample size and effect size estimate based on your sample data - Click on options button and place a tick next to observed power power of 0.80 or greater is considered acceptable
why p >0.99
- By logic, is that reporting a p-value as p = 1.00 implies 100% certainty, which we actually never have. - Since SPSS only reports to 3 decimal places, it fails to capture the fact that the p value may have been. - One the other hand, I think that reporting a p-value as p> 0.99 better communicates the fact that the p value was extremely large, without also implying 100% certainty in the outcome - Just as there is no p= 0.000
tips for choosing contrasts
1. use your control group as a reference 2. only compare two chunks at a time 3.keep contrasts independent
interaction effect
Interaction effects examines two or more factors at the same time, which may not be predictable based on the effects of either factor on their own.
when is a liberal F statistic produced?
Large group variances associated with small sample sizes tend to produce liberal F statistic - more likely to find a significant difference when there isn't one
when is a conservative F statistic produced?
Large groups variances associated with large sample tend to produce conservative F statistic - more likely not to find a significant difference when one does exist.
main effect
Main effect is the effect of one factor (IV) on its own
do we perform both planned comparisons and a post hoc?
No not both Planned comparisons if you have a priori hypotheses
MS between
SSbetween/df between
box plots
a boxplot is a graphical representation of a five number summary 50% of data is contained within the interquartile range, IQR at the top and bottom of the box plot are two whiskers; which show the top and bottom 25% of scores
the scatterplot
a graph that plots each person's score on one variable against their score on another it tells us whether there seems to be a relationship between the variables, what kind of relationship it is and whether any cases are markedly different from others.
how do we run a One-way repeated measures ANOVA in SPSS
analyse- general linear model- repeated measures
outliers and histograms
check the tails of the distribution to see if data points fall away at the extremes
sphericity is sometimes referred to as ________, is a more general condition of ___________
circularity; compound symmetry
between-subjects
compare subjects between each other independent measures different participants take part in each condition
the ANOVA model
compare the amount of variability explained by the model to the error in the model (our model is our groups in ANOVA) if the model explains a lot more variability than it can't explain (the model variance is much greater than the residual variance), then the experimental manipulation had a significant effect on the outcome (DV) the larger F is the less likely it is due to sampling error
if the standard error is biased then the _______will be too because it is based on the standard error
confidence
null hypothesis significance testing
develop an alternative hypothesis H1 (the hypothesis we are interested in) identify the null hypothesis H0 (no change/no significant difference) test the null hypothesis if the data are unlikely under the null (p<0.05), reject H0
the f-ratio ______ tell us whether the F-ratio is large enough to not be a chance result
does not to discover this we can compare the obtained value of F against the maximum value we would expect to get by chance if the group means were equal in an F-distribution with the same degrees of freedom if the value we obtain exceeds this critical value we can be confident that this reflects an effect of ur independent variable
analyse with robust methods
e.g. trimmed means, bootstrapping; methods that don't rely on normality assumptions
the f-ratios
each effect in a two way ANOVA has its own f-ratio to calculate these we have to first calculate the MS for each effect by taking the SS and dividing by the respective degrees of freedom
SPSS row (data view)
each row represents on case or participant
SPSS row (variable view)
each row represents one variable
compound symmetry holds true when both the variances across conditions are
equal
what does a factorial ANOVA do?
examines the effect of each factor IV on its own, as well as the combined effect of the factors main effect is the effect of one factor (IV) on its own interaction effect examines two or more factors at the same time- that is the combined effect, which may not be predictable based on the effects of either factor on their own thus, factorial ANOVA produce multiple F ratios- one for each main effect and interaction term knowing about one f-ratio tells you nothing about the others- each term sits by itself
outliers
extreme scores outliers bias the mean and inflate the standard error
summary of factorial designs
factorial design- examining the impact of two or more factors on a particular dependent variable
stacked histogram
good way to compare the relative frequency of scores across groups
what are the ways to detect outliers?
histograms and box plots descriptive statistics analysis - z-scores in excess of 3.29 (p<0.001)
why two way repeated measures?
more power- detecting effect is high- eliminates individual differences fewer participants
histogram
plots a single variable (x-axis) against the frequency of scores (y-axis)
frequency polygon
same data as the simple histogram, except that it uses a line instead of bars to show the frequency
cohen's d
tells us the degree of separation between two distributions- how far apart, in standard deviation units, are the means 0.2 small 0.5 medium 0.8 large
platykurtic distributions make the type 1 error rate ______ and consequently the power is______
too high; too low
if a test is conservative (the probability of a _____ error is small) then it is likely to lack statistical power (the probability of a _____error will be high)
type 1; type 2
the assumption of normality
we assume that the sampling distribution is normal, not necessarily the sample distribution
when should we use REGWQ test?
when we want to test all pairs of means
SPSS contrasts
• SPSS has a standard set of built-in contrast which can perform- can be useful but more complex • Note: not all are orthogonal
planned comparisons
/WSFACTOR = time 3 SPECIAL (111,2 -1 -1, 000) we replace polynomial with Special () and insert the contrast weights in brackets for group we add in new line after this which takes the form /CONTRAST (group) Special () and insert the contrast weights in brackets
skewness and kurtosis - SPSS
0 indicates normal distribution we convert the value to a z-score (1.96 = p<0.05) to compute the z-score: statistic/standard error e.g. skewness value/standard error
MS error
SSerror/ df error
SS error
SSwithin - SSbetween subjects
MS within
SSwithin/dfwithin
nominal data
a variable can be treated as nominal when its values represent categories with no intrinsic ranking e.g. postcodes, gender, employment status, religious affiliation categorical data
ordinal data
a variable can be treated as original when the values represent categories with some instruct ranking e.g. likert scales, position in race, ranking ranked data
use your control group as a reference
a. If you have a control group as a reference, it is usually because you want to compare it to other groups b. Then compare other groups c. E.g. no diagnosis, depression, schizophrenia
type 2 error
accept the null hypothesis when we shouldn't- failed to reject this is related to power
line charts
are bar chats but with lines instead of bars simple line or multiple line charts
To do planned comparisons, the hypotheses must be derived ____________the data has been collected
before
One non-parametric method used to estimate a statistic's sampling distribution is known as...
bootstrapping
mixed-design ANOVA
compares several means when there are two or more IV and at least one of them has been measured using the same entities and at least one other has been measured using different entities each effect is independent of one another- therefore knowing that one particular effect is significant tells you nothing about whether any other effects are likely to be significant. significant interactions indicate that the main effects do not tell you the whole story of what is going on with your data
_________ can be extremely inaccurate when homogeneity of variance cannot be assumed
confidence intervals
when group with larger sample sizes have larger variances than the groups with smaller sample sizes, the resulting F-ratio tends to be ________
conservative - more likely to produce a non-siginifcant result when genuine difference does exist
reject the null hypothesis when it is false
correct decision (1-b)
error A
error A- used in the f-ratio denominator when calculating the main effect of factor A subjects within A, between subjects error variability, measures differences between subjects in the same group- which includes individual differences how much variance in Factor A it is reasonable to expect based on change
what kinds of factorial designs are there?
independent measures repeated measures or mixed design- a combination of independent and repeated measure factors (mixed models)
between subjects factorial designs
independent measures there are a number of questions we can ask: is there an effect of being in condition A1-3? is there an effect of being in condition B1-2? then there is the extra question of : is there an interaction between A and B e.g. o IV: psychiatric diagnosis AND gender o DV: depression scores o Is there an effect of diagnosis (factor 1) on depression score- ignoring gender? o Is there an effect of gender (factor 2) on depression score- ignoring diagnosis? o Then we can ask the more complicated question: does the effect of diagnosis depend on your gender (interaction)?
skewed distributions seem to have _____effect on the error rate and power for two-tailed tests
little
factorial designs
more often than not, behaviour. affect and cognitions are influenced by more than one factor therefore, we often design studies which involve examining the impact of two or more IVs on a particular dependent variable factorial designs allow us to investigate complicated relationships
sum of squares
need to calculate: SST SSM= SSB (between subject factor) SSR= SSW (within subject factor) work out the overall mean and the group means
what are the two sources of bias?
outliers and violations of assumptions
post hoc tests consist of ______comparisons that are designed to compare all___________
pairwise; different combinations of the treatment groups
type 1 error
reject the null hypothesis when we shouldn't- say theres a difference when there isn't
what are some potential disadvantages of two-way repeated ANOVA?
sphericity temporal confounds- something else different about those times order effects- they just don't care by week 4- we can control this through counterbalancing
mauchly's test
test of sphericity tests the null hypothesis that variances between level differences are equal assumption is violated when it is less than 0.05= we then must look at the Epsilon values the assumption is not violated when it is greater than 0.05
kolmogorov-smirnov test
tests if data differs from the normal distribution the null hypothesis is that the data WILL be normal P<0.05 data vary significant from normal P>0.05 no strong evidence that data vary from normal denoted by D
homoscedasticity
the assumption that the variance are equal across all levels of a factor
homogeneity of variance
the assumption that the variance of one variable is stable at all levels of another variable is the variance similar between two groups? levene's test does this for us H0 - the variances are the same p<0.05 variances are statistically significantly different homogeneity of variance assumption is violated p>0.05 variances not statistically significantly different homogeneity of variance assumption not violated
power
the probability of failing to reject the null hypothesis if it is true/ the probability of detecting an effect if it is there we want more powered studies
SS within (repeated measures)
the variability within an individual- the sum of squared differences between each score and the mean for that respective participant broken down into two parts - how much variability there i between treatment conditions: model sums of squares SSm - how much variability cannot be explained by the model: residual sums of squared SSr
research question and variables
the variables you measure must match your research question
heterogeneity of variance
the variance of one variable varies across levels of another
partitioning the variance
there are two error terms: error A an error B
multivariate tests
these tests conduct a MANOVA rather than an ANOVA these results can be used in pace of the regular ANOVA results i the sphericity or normality assumptions are violated whilst these tests are more robust than ANOVA to the assumption violations they are also less powerful
when do we use lower-bound correction?
this is too conservative- this assumes that sphericity is completely violated avoid using this correction
dealing with normality violations
transformation remove outliers use a nonparametric method- Friedman's test
what happens when we fit a model to the data?
we estimate the parameters and we usually use the methods of least squares when we estimate a parameter we also compute an estimate of how well it represents the population such as a standard error or confidence interval
rules for assigning contrast weights
1. Choose sensible contrasts • You should only compare two chunks of variance • If a group is singled out in a previous contrast, it should be excluded from subsequent chunks 2. Groups coded with positive weights will be compared against groups with negative weights • Assign one chunk of variance with a positive weight and the other chunk with a negative weight 3. The sum of contrast weights must be 0 • If you add up all the contrast weights for a particular comparison, they must sum to 0 in order to be valid • Otherwise you are testing a different null hypothesis 4. If a group is not assigned to a comparison, assign it a weight of 0 • This will exclude the group from calculations 5. For a given contrast, the weights assigned to the groups in one chunk of variation should be equal to the number of groups in the opposite chunk of variation When assigning magnitudes to experimental groups we must look at how many groups there are within the chunk (E1 and E2 = 2) therefore, we should give 1 to each E1 and E2. As a result, the magnitude of the control group should be 2.
when would we use a two-way mixed design ANOVA?
Intervention or drug trails- assessing the main outcome of a new drug e.g. participants have their anxiety levels measured before therapy, at the end of the therapy and again three months after. new therapy or waisting list control. Factor A - group Factor B- time DV- anxiety levels
what are the steps to conducting a repeated measures ANOVA with planned comparisons?
Like above we must choose our contrasts and assign our weights. In SPSS, analyse > general linear model > repeated measures We tell SPSS what is our variables are and how many levels there are in each. We then move our variables cross to the black within-subjects variables > then we click contrasts Note that by default, trend analysis is performed for repeated-measures ANOVA. Therefore, you do not need to click on the contrasts button to set this up. We press continue Next we click plots an move the variable onto horizontal axis Press add then continue Then we click options and place a tick next to descriptive statistics and estimate of effect size > continue Press okay to run the ANOVA
SPSS
analyze > general linear model > repeated measures enter in a within-subjects factor: time between subjects factor: group enter DV specific the levels of time and group we click paste to get to syntax- need to modify it if we want to planned comparisons and simple effects
factorial ANOVA using SPSS
analyze > general linear model > univariate IV placed into fixed factors and DV placed in dependent variable to graph interactions we click on plots and place factor 1 on horizontal axis and factor 2 on separate lines (doesn't matter which way round the variables are plotted)
p-p plot
another graph for checking normality- it plots the cumulative probability of a variable agains the cumulative probability of a particular distribution the actual z=score is plotted against the expected z-score if the data are normally distributed then the actual z-score will be the same as the expected z-score and then you will get a straight diagonal line when values sag consistently above or below this line then kurtosis differs from a normal distribution when values are in a s-shape, the problem is with skewness
variance
assume the variance is roughly equal between groups/levels of factor if it is violated we can use a more strict p value, typically p <0.025 this helps guard against type 1 error transform data- may reduce heterogeneity
error B
error B- used in the f-ratio denominator when calculating the main effect of factor b and the A X B interaction also referred toa s B by subjects within A within each group, measures differences between subjects across time which excludes individual differences how much variance in B it is reasonable to expect based on chance after individual differences have been removed.
post hoc tests
essentially t-tests compare each mean against all others- in general terms they use a stricter criterion to accept an effect as significant they control the family-wise error rate (afw_ simplest example is the bonferroni method
Greenhouse-Geisser estimate
for example, in a situation where there are five contains the lower limit will be 1/ (5-1) or 0.25 (known as the lower-bound estimate of sphericity)
how to run a two-way repeated measures anova in spss
go to analyse- general linear model- repeated measures type in a name for the first repeated measures factor- specific how many levels the factor has type in a label for the DV type in the name for the second repeated measures factor -enter in levels drag variables from the list into within-subjects variables click onto plots and move one IV to one axis and the other on the other axis place ticks next to descriptive stats and estimates of effect size and press continue (unless you want to run post hoc tests or simple effects analysis-if so we move the factors and interaction into the display means for box) then we place a tick next to compare main effects and select bonferroni and then press continue press paste to set up simple effects analysis or custom contrasts press run to perform analysis
what makes a good graph?
graphs should do the following show the data induce the reader to think about the data being presented avoiding distorting the data present many numbers with minimum ink make large data sets coherent encourage the reader to compare different pieces of data reveal the underlying message of the data 2D plot makes a graph easier to compare the values clearly labeled axis minimal or no distractions minimum ink- getting rid of the axis lines
sphericity violations
if Mauchly's test is less than p 0.05 we use an alternative, corrected ANOVA epsilon values: greenhouse-geisser, huynh-feldt, lower-bound each one of them corrects for the violation of sphericity corrects the degrees of freedom
what is the problem of doing multiple t tests?
if we are doing a test it assumes that this is the only test being done decision-wirse error for each comparison assumes it is the only test (a) when we conduct multiple tests we have this 5% rate each time - the more tests we do the more likely that one of those tests will be wrong family-wise error (all of the tests we are conducting) becomes inflated- we need to control this family-wise error rate family wise error= 1-(1-0.05)^4 instance 3 tests our family wise error can be computed = 1(0.95)^4 increases type 1 error- false positive- rejecting the null hypothesis when it is true
simple effects analysis
if you wanted to follow up on a significant interaction with simple effects analysis, add command to to the line which produces the estimated marginal means for the interaction to compare the two groups at each time point compare (group) ADJ (Bonferroni) to compare the three time points for each group separately Compare(time) ADJ (Bonferroni)
underlying theory of repeated measures ANOVA
in a repeated measures search study, individual differences are not random- - because the same individuals are measured in every treatment condition, individuals differences will have a systematic effect on the results - these individual differences can be measured and separated out from other sources of error we partition the variance into: -total variance -between subjects - within subjects- previously the residual or error
Multi-subscript notation Xijk
in two-way ANOVA< three subscripts are needed the first, i, identifies the level of factor a (row) the second subscript, j, identifies the level of factor B- column together (i,j) identify the cell in the data table subscript K identifies the individual score within a score n = number of scores in each cell (in a balanced design) nij = number of scores in a specific cell (i,j) a = number of levels of factor A b = number of levels of factor B N= total number of scores
assumptions
independent observations- someone's score in one group can't affect someone's score in another group interval/ratio level measurement- increments must be equal normality of sampling distribution- remember the central limit theorem homogeneity of variance the group variances should be equal
when the groups with larger sample sizes have smaller variances than the groups with smaller sample sizes, the resulting F-ratio tends to be_____
liberal- more likely to produce a significant result when there is no difference between groups in the population- type 1 error rate is not controlled
main effect with interaction
main effect with interaction: nonparallel lines
main effect with no interaction
main effect with no interaction: parallel lines, there is no crossing over
what test is used to assess sphericity
mauchly's test
double notation, Xij
needed when we have multiple variables for the sample participant the first subject, i, refers to the row that the particular value is in the second subscript, j, refers to the column what is X2,3? the second person (second row) and the third value (third column)
Post hoc tests are done when you have ________________
no specific hypotheses
is there a non-parametric version of mixed ANOVA?
no, but there are robust methods that can be used based on bootstrapping
if the f-ratio is less than 1 it must represent a _________effect
non significant effect this is because there must be more unsystematic than systematic variance our experimental manipulation has been unsucessful
effect size for two-way ANOVA
omega squared- need to first compute a variance component for each of the effects and the error and then use these to calculate the effect sizes for each In these equations, a is the number of levels of the first independent variable, b is the number of levels of the second independent variable and n is the number of people per condition. we also need to estimate the total variability: this involves the sum of all variance component for each of the effects and error plus the residual MS the effect size is then calculated by the effect/total variance
outliers and boxplots
outliers are displayed as little circles with an ID number attached
if the test statistic is biased then so too will its _____
p-value
what are the 3 contexts that can be biased?
parameter estimates standard errors and confidence intervals test statistics and p-values
positive/negative skew
positive skew- scores bunched at low values with the tail point to high values negative skew- scores bunched at high values with the tail point to low values
disadvantages with normality tests
relatively insensitive when N is small- this is when normality matters the most too sensitive when N is large- when this doesn't matter due to central limit theorem therefore it is best to do visual examination
reliability analysis
reliability refers to the consistency or dependability of a measure over time, over questionnaire items, or over observers/raters
managing outliers
remove the participant from your analyses change the participants score- next highest value plus 1, value that has a z-score of 3.29
windorizing
replacing outliers with the next highest score that is not an outlier
planned contrasts- weights
set up contracts- assign positive/negative, magenta and weight to each chunk set up two-way independent ANOVA as usual but press paste instead of OK to copy the syntax into the syntax editor window Add a line like: /CONTRAST (ALCOHOL)SPECIAL (2 -1 -1) Add at top: attractiveness BY Gender Alcohol
first contrast
should compare all experimental groups with the control group
what are the three types of boxplots
simple, clustered, 1-D boxplot
skewness and kurtosis
skew is about the symmetry of the distribution kurtosis is about the heaviness of the tails or pointness of the distribution
f controls the type 1 error rate well under what conditions?
skew, kurtosis and non-normality
reserve scoring
sometimes, questionnaires contain both positively and negatively worded items summing these responses to obtain a total scale score would make very little sense therefore, reversing the response to the second question allows us to calculate a meaningful total, which can be used in subsequent analysis
planned comparisons are done when you have _____________ you want to test
specific hypotheses
levene's test
tests the null hypothesis that the variances in different groups are equal does this by doing a one-way ANOVA on the deviation scores when sample size is large, small differences in group variance can produce a significant in levee's test analyse- descriptive statistics- explore
where does error come from?
the T statistic gets bigger it we increase the difference (the signal) or if we decrease the error (the noise) in the t stat, the error comes from the standard error this is made up of the variability in the sample and the number of people we can reduce the error by increasing the number of people or decreasing the variability e.g. using better measurement instruments the bigger the statistic the smaller the p value
variance ratio method
there is overlap in individual groups- this is where we get our error variance how much variability is there in the total sample group- treatment model in ANOVA we look at this overall variance and we divide it by the error variance if the ratio is greater than 1 - we have evidence that the means are differet
polynomial contrasts: trend analysis
this contrast tests for trends in the data and in its most basic form it looks for a linear trend there are other trends- quadratic, cubic, and quartic trends linear trend- proportionate change in the value of the dependent variable across ordered categories quadratic trend- where there is a curve in the line (to find a quadratic trend you need atlas three groups) cubic trend- where there are two changes in the direction of the grand (must have at least four categories of the IV) quartic trend- has three changes of direction (need at least 5 categories of the IV). each of these trends has a set of codes for the dummy variables in the regression model- if you add the codes for a given trend the sum will equal zero and if you multiple the codes the sum of products will also equal zero- these contrasts are orthogonal
heteroscedasticity
this occurs when the residuals at each level of the predictor variable have unequal variances
what do we do if we violate sphericity?
use the epsilon values
simple histogram
use this when you want to see the frequencies of scores for a single variable
why opt for repeated-measures?
uses a single sample, with the same set of individuals measured in all of the different treatment conditions- one of the characteristics of repeated measures design is that it eliminates variance caused by individual differences individual differences are those participant characteristics that vary from one person to another and may influence the measurement that you obtain for each person- age, gender etc.
bar charts
usual way to display means
problems with repeated measures
usually we assume independence but scores will be correlate between conditions, hence it violates the assumption of independence accordingly, an additional assumption is made: sphericity - assumes that the variances and covariances of differences between treatment levels are the same -related to the idea that the correlation of the treatment levels should be the same
Q-Q plot
very similar to the P-P except it plots the quantiles of the data instead of every individual score in the data
histogram 'bins'
we can determine properties of bins used to make the histogram these bins can be seen as rubbish bins: on each rubbish bin you write a score or range of scores, then you through scores in your data set and throw each score into the rubbish bin with the appropriately label on it when you have finished throwing your data into these bins, you count how many scores are in each bin
effect size for factorial repeated-measures ANOVA
we can use the F ratio s and convert them to an effect size r
compute variable
we can use the compute variable functioning SPSS to sum or average responses to questionnaire items this will create a new variable, and thus a new column of data
assumptions of two-way ANOVA
we have the same overall assumptions we had with one-way ANOVA o Interval/ratio level data for DV o Independence o Normality o Homogeneity of variance • Overall, ANOVA is a relatively robust procedure to violations of assumptions: o This is especially true for violations of normality, especially when sample sizes are large o Normality is not a crucial assumption and can be violated without great consequence
interpreting interactions graphs
whilst you can use graphs to give you an indication of whether there is a main effect or interaction, you cannot use graphs to tell you if main effects or interactions are statistically significant interaction- non parallel lines no interaction- parallel lines/no crossing over
the method of least squares
will produce unbiased estimates of parameters even when homogeneity of variance can't be assumed but better estimates can be achieved using different methods such as weighed least squares
recoding data
with the recode function in SPSS, we can change specific values or ranges of values on one or more variables. this can be used to: collapse continuous variables into categories or reverse negatively scaled questionnaire items
what else can you use when assumption of normality or sphericity are violated?
you can also use the multivariate tests such as MANOVA if the assumptions of normality or sphericity are violated as this provides a more robust test of the null hypothesis that all treatment means are equal you could also conduct a Friendman's test as the non-parametric alternative to a one way repeated measures ANOVA
when planning comparisons...
your first contrast should be one that compares all of the experimental groups with the control groups there is only one control condition and so this portion of variance is only used in the first contrast- because it cannot be broken down any further
data types and statistical tests
your statistical tests must be appropriate to the type of data you are using. we need to consider: how many outcome variables? what type of outcome? how many predictor variables? what type of predictor? if categorical predictor, how many categories? if a categorical predictor, are the same or different entities in each category? are assumptions of linear model met/not met?
normality
• ANOVA is considered to be quite resilient or robust to departures from normality. As Winer et al. (1991) suggest "A reasonable statement is that the analysis of variance F statistic is robust with regard to moderate departures from normality when sample sizes are reasonably large and equal..." (p. 101) • Confidence in the "robustness" of the normality assumption come in part from an important statistical principle known as the central limit theorem, which indicates that as we increase the sample size (n), the sample mean will increasingly approximate a normal distribution. • Hence, Keppel and Wickens (2004) argue persuasively that "once the samples become as large as a dozen or so, we need not worry much about the assumption of normality" (p. 145).
running custom contrasts
• In order to run the analysis, set-up the ANOVA as usual o Instead of pressing OK press Paste o This will create the syntax and copy it to a new syntax window /WSFACTOR = Time 4 Polynomial tells us that you have a within-subjects factor called Time, which has 4 levels, and that, by default, polynomial contrasts will be performed But we now want to perform Special (custom) contrasts rather than polynomial contrasts So we delete polynomial and type 'Special' then place in brackets the contrast weights:
contrasted weights
• Need to assign weights to each group to tell SPSS how to perform the contrast • A planned comparison is linear combination of means • Where a1 to ak are contrast weights for k groups • This simply states that a planned contrast is a weighted sum of treatment means • Assign weights such that if there is no difference, L= 0 • Conduct t-test on 0 to see if it is different from 0
orthogonality
• Orthogonal contrasts are independent o They tap into different things o Unique chunks of variance o There is no redundancy o E.g. hypothesis 1 tests reward in relation to punish and indifferent o It would therefore by redundant to compare reward to punish in hypothesis 2 because you have already compared reward in relation to punish and indifferent o Have already used reward as a unique chunk • You can work out if contrasts are orthogonal to one another by calculating the sum of the cross products.
underlying theory of planned comparisons
• The variability explained by the Model (experimental manipulation SSM) is due to participants being assigned to different groups • This variability can be further broken down to test specific hypotheses about which groups might differ from one another • We break down the variance according to hypotheses made a priori (before the experiment). • Providing that the hypotheses are independent of one another, the family wise type 1 error will be controlled • When breaking down the variance→ cutting up a cake o We cut slices into smaller bits but we cannot stick these bits back together
why should we care if contrasts are orthogonal?
• Type 1 error o If contrasts are orthogonal, we don't need to adjust the a level for multiple comparisons o No inflation of type 1 error rate • Planning o There will always be K-1 orthogonal contrasts for K groups o If the overall ANOVA is significant, at least one of these contrasts must be significant also • Sum of squares o If group sizes are equal then SSmodel can be divided into k-1 components exactly.
simple effects analysis
• When you find a significant interaction between two or more variables, simple effects analysis (i.e. post hoc tests) can be used to examine that interaction. • If there is no significant interaction you cannot do simple effects analysis. • Simple effects analysis looks at the effects of one independent variable at one level of the other independent variable, then repeats the process for all other levels • For example, we could use simple effects analysis to look at the effects of alcohol on attractiveness for males, then at the effects of alcohol on attractiveness for females. • For males we would do a normal ANOVA with the different levels of alcohol and then separate we do a one-way ANOVA for females. • We have to use syntax to do this
In SPSS what does the row and column represent
row- data from on entity column- a level of a variable
how does pairwise comparisons control the FWE?
they control FWE by correcting the level of significance of each test such that the overall type 1 error rate across comparisons rains at 0.05
What are the two options to determine where the difference is in ANOVA?
Post hoc tests and planned comparisons
Polynomial contrasts in SPSS
Set up an ANOVA as usual (analyse > compare means > one-way ANOVA for independent measures; analyse> general linear model > repeated measures for repeated measures) Click on contrasts button. We get an F statistic instead of T statistic
when sphericity is violated the ___________ seems to be generally the most robust of the univariate techniques
bonferroni
df error
(k-1)-(N-k)
platykurtic
- light tails
family wise error rate
1-(1-a)^n
two-way ANOVAs address what three questions?
1. What happens to the DV as factor A (IV1) changes in levels? a. Is there a statistically significant main effect on factor A on the DV? 2. What happens to the DV as factor B (IV2) changes in levels? a. Is there a statistically significant main effect on factor B on the DV? 3. How does specific combinations of factor A and factor B affect the DV? a. Is there a statistically significant interaction between factor A and factor B upon the DV?
you need at least ________ conditions for sphericity to be an issue
3
homogeneity of variance
ANOVA is fairly robust is the sample sizes are equal, but not if the sample sizes are unequal if there is heterogeneity of variance, you can use the Brown-Forsyth or Welch F ratio instead of the regular F ratio - it takes into account the the variances are not equal
what is the alternative to using multiple t-tests?
ANOVA- can compare multiple groups in one test or FWE (family wise error) control
how do we test the interaction?
An interaction tests whether the effect of one factor is the same across all levels of another factor. If there is no interaction the difference between means should be the same. Therefore, if differences are not the sample then there is an interaction — we can reject the null hypothesis because they are not the same. H 0 = (A1B1 - A1B2) - (A2B1 - A2B2) = 0
how is sphericity measured?
By hand- start by calculating the differences between pairs of scores in all combinations of treatments. then we calculate the variance of these differences. the differences between pairs of scores are computed for each participant and the variance for each set of differences is calculated the assumption is met when these variances are roughly equal
post hoc in SPSS
Click on Post Hoc If one variable has only two levels we do not need to select post hoc tests for that variable. If the variable as more than two levels we can conduct post hoc tests. Select the variable and transfer it to the box labelled Post Hoc Tests for.
what do we do when there are two control groups? how do we plan our contrasts?
Contrast 1: control group (C1 and C2) vs. experimental groups (E1 and E2) Contrast 2: E1 vs. E2 Contrast 3: C1 vs. C2
f ratio
F = MS between/ MS within
f ratio for repeated measures
MS between treatments/ MS error
post hoc for unequal sample sizes
Fabriels (small discrepancy in n) or Hochberg's GT2 (large discrepancy in n)
post hoc for unequal variances
Games-howell
heterogeneity of variance
If variance is violated we can use Welch F in one way but this is not available for two way novas. easiest strategy is to adopt a more stringent alpha level- only considered results to be significant if p <0.025 or 0.01
df for total
N-1
df for within treatments
N-k
why can't we use graph interactions as an indication of significance?
We can use graphs to give an indication of whether there is a main effect or interaction but you CANNOT use graphs to tell you if the main effects or interactions are statistically significant. This is because differences between means can be due to sampling error. Therefore, you must ALWAYS use statistical tests to determine significance.
only compare two chunks at a time
a. For simple comparisons b. Compare one chunk against another chunk c. Doesn't apply for polynomial comparisons
omega-squareed
alternative to eta-squared- unbiased estimator and unbiased version of eta-squared 0.01 small 0.06 mediam 0.14 large
how do we run a one way independent ANOVA in SPSS
analyse- compare means- one way ANOVA
total sums of squares, SSt
calculate in the same way as one-way
sphericity
calculate the differences for all the possible participants
log transform: log (x)
can reduce positive skew and stabilise variance
Accept the null hypothesis when it is true
correct decision (1-a)
local sphericity
data with two of the variances of differences that are similar A and B (15.7) B and C (10.7) C and A (10.3)
standard pool
estimate of standard deviation
parameter estimates
estimates of parameters are affected by non-normal distributions. parameter estimates differ in how much they are biased in a non-normal distribution the median is less biased by skewed distributions than the mean if our parameter estimate is normally distributed then test statistics and p-values will be accurate if residuals are normally distrbuted in the population then using the method of least squares to estimate the parameters will produce better estimates than other methods
The REGWQ has _____ and tight control of the ____ error rate
good power; type1
why is it preferable to use a factorial ANOVA instead of two single factor ANOVAs?
interactions ecological validity fewer studies
eta-squared
it is a biased estimator- really good at telling about our sample but doesn't give us a very accurate effect size in our population 0.01 small 0.09 mediam 0.25 large
what does the f-ratio measure?
it is a measure the ratio of the variation explained by the model and the variation explained by unsystematic factors
is ANOVA robust?
it is a robust test meaning that it doesn't really matter if we break he assumptions of the test- the F will still be accurate.
df for between treatments
k-1
what are the tests for normality
kolmogorov-smirnov test or the shapiro-wilk test (better test)
what statistical test is used to assess variance?
levene's test
population pyramid
like a stacked histogram, this shows the relative frequency of scores in two populations
labeling ANOVAs
one way- one IV or factor two-way- two IV or factors three-way- three IV or factors
the three examples of effects sizes
proportion of variance accounted for (r^2) eta-squared (n^2) omega-squared (w^2)
when is anova robust to normality
providing that your same sizes are equal and large (df error >20) ANOVA is robust to the normality assumption being violated if the sample sizes are not equal/small, ANOVA is not so robust to the normality assumption violation- you can transform the data or use a non-parametric alternative to a one-way independent measures ANOVA (Kruskal-Wallis tests)
hartley's fmax
ratio of the variances between the group with the biggest variance and the group with the smallest variance
what statistic are we given for planned comparisons
t
omnibus test
tells us means aren't equal somewhere, but not where
what does /WSFACTOR = Time 4 Polynomial tell us
tells us that you have a within-subjects factor called Time, which has 4 levels, and that, by default, polynomial contrasts will be performed.
if the standard error is biased ______ will be to because they are usually based on the standard error
test statistics
interpretation of the F-ratio
the f-ratio tells us only that the direct or indirect manipulation was successful it does not tell us specifically which group means differ from which we need to do additional tests to find out where the group differences lie
the sample distribution
the frequency distribution of the data in our sample
the sampling distribution
the frequency distribution of the means of random samples taken from the population
why study RDA?
to acquire accurate knowledge about why people think, feel and behave the way they do
when do we use huynh-feld correction?
when greenhouse-geisser is greater than 0.75 it is a less contrastive method than compared to greenhouse-geisser
when shouldn't we use REGWQ test?
when group sizes are different
when is ANOVA fairly robust to violations of the assumption of homogeneity of variance?
when sample sizes are equal
when are assumptions of normality statistically violated?
when the significance value is below 0.05 for assumptions to be met the significance must be greater than 0.05
Polynomial contrasts
• Polynomial trends→ trend analysis - used when there an ordinal pattern in our data • Test specific patters in data • Use specific weights • Make sure levels in your data file are in the correct order
