211 Stats Final
I^2
% of variation across studies that is due to heterogeneity not due to chance.
Within Cell simgahat^2_e
(estimate of the population variance of our sample within each cell, since estimate of population value = hat) theta is not a random effect are almost but not quite variances
main effects model
A hypothetical data model in which the combined effect of two or more independent variables results from their being added together. Any change that can be directly attributed to the IV or treatment after controlling for the other possible influences
General Linear Model (GLM)
A linear model (Y = mX + b) where each object can have multiple measurements. The GLM underlies many different statistical tests including the one-way Analysis of variance and the regression. A way of explaining and predicting a dependent variable based on fluctuations (variation) from its mean. The fluctuations are due to changes in independent variables.
Randomization
A technique for assigning experimental subjects to experimental and control groups randomly without replacement
ANCOVA (analysis of covariance)
A version of the ANOVA used to increase the efficiency of the analysis by statistically removing variability in the DV that is due to an extraneous variable. When using the this test, each person's score on the DV is adjusted on the basis of his or her score on the extraneous variable. To test 2 or more groups while controlling for extraneous variable (co-variates) you use the
Contrasts
ANOVA, and linear regression is a linear combination of variables whose coefficients add up to zerp allowing comparisons of different means quantity sum (a_i theta_1) is a linear combination and is a contrast if sum(a_i) = 0 2 contrasts sum (a_i theta_1) and sum (b_ i theta) are orthogonal of sum (a_i b_i) = 0
Subgroup analysis
An analysis that examines whether statistical results are consistent for different subsets of the sample (e.g., for males and females). Analysis of the treatment effect within a category or a subgroup of participants classified based on demographics or other important characteristics. reanalizes to indentify important differences in treatment effects increase subgroups increase T1 error decrease number of participants in each group leads to false negative
main effect
Any change that can be directly attributed to the independent or treatment variable after controlling for other possible influences. the direct effect of an independent variable on a dependent variable In a factorial design, the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable.
Meta analysis fixed model
Assume all included studies share a common effect size, mu. The observed effects will be distributed about mu, with variance simga^2. assume that the true effect is the same in all studies
Random-effect meta-analysis
Assume each study is estimating a study-specific true-effect, mu observe heterogeneity in the estimates due to 2 sources 1) between-study heterogeneity in true effects 2) within study sampling error
Unequal N in the GLM creates what?
Creates a correlation between IVs.
Repeated measures between subjects
Differences between individuals. This can be between groups of cases when the independent variable is categorical or between individuals when the IV is continuous. It determines if respondens differ on the DV, depending on their group or on their score on a particular continuous IV
multiple t-tests
Easiest approach to compare a subset of means is to do a t-test but you get to leverage the error estimate of the study using everything / all data xbar1 - xbar2 / 2meansquare error / sqrt(n). Based on all people not just the subset of groups. get to use the people in the other groups to estimate the error and the dfs of the detonator of F is based on the of the dfs of the error term not of the numerator to give you more power! MSerror is based on N-K. Every time you conduct a t-test there is a chance that you will make a Type I error. This error is usually 5%. By running two t-tests on the same data you will have increased your chance of "making a mistake" to 10%. The formula for determining the new error rate for multiple t-tests is not as simple as multiplying 5% by the number of tests. However, if you are only making a few multiple comparisons, the results are very similar if you do. As such, three t-tests would be 15% (actually, 14.3%) and so on. These are unacceptable errors. An ANOVA controls for these errors so that the Type I error remains at 5% and you can be more confident that any statistically significant result you find is not just running lots of tests.
If theta^2 = 0 F is what?
F = 1 This assumption is that the interaction is 0, calls the assumption of nonadditivity in model 1, by blocking the treatment or subject by treamtment interaction
Testing for Sphericity
Machly Homogeneity of variance (sphericity): Using a variance-covariance matrix with variance along the dialog and co-variance everywhere else, if no repeated measures we could assume independence observations and then all of the covariances would become zero. Only the variance would be left and the homogeneity of variance would say they are all equal. However, there are co-variances Can use compound symmetry but it is impracticable. So you can use sphericity. And all matrices that are sphericity are also compound symmetric but not the other way around Sphericity: is just a version of the variance sum law. Test for sphericity (*Machly*), not very good (low power with small samples and good power with a large sample)
How to test sphereicity
Machly test
Semi-partial correlation
Measures the relationship between two variables controlling for the effect that a third variable has on only one of the others. used to partial out the effects of a variable that is influencing only one of the other variables also equal to r^2_change
Structural model 2
Model 2: Xij = mu + pi_i +tau_j + pixtao_ij (*interaction term*) + eij tau as a *fixed effect* pi as a *random* effect want to think of treatment as fixed effect
Model 1 Model 2 Model 3
Model 3 has just tau > both pi and tau are random effects Model 2: whether or not you assume the interaction is 0 Model 1: pi is a fixed effect tau is random
Why non-parametric test over parametetric?
Non-parametric tests allow to not make any assumptions. This makes this more the test more robust any parametric. increase roburstness decreases power (though) We get a distrubition of coefficients, by repeatedly running samples (regression coefficients and SEs)
Heterogeneity of variance
Occurs when we do not have homogeneity of variance; this means that our two (or more) groups' variances are not equivalent. Occurs when we do not have homogeneity of variance; means that our two (or more) groups' variances are not equivalent. A situation in which samples are drawn from populations having different variances differences in ES in meta analysis. You use q for this test Differences between studies not due to chance chi-squared I^2
multiple comparisons
Performing two or more tests of significance on the same data set. This inflates the overall α (probability of making a type I error) for the tests. (The more comparisons performed, the greater the chances of falsely rejecting at least one true null hypothesis.) The problem of how to do many comparisons at once with an overall measure of confidence in all our conclusions.
mixed-effects model
Predicts that sometimes media will have direct, sometimes limited, and sometimes a mixture of both Model that assumes the mass media will influence different people in different ways spacerun:yes'> Also known as the hypodermic needle model. model that predicts that media can have a combination of influences A mixed-effects model (class III) contains experimental factors of both fixed and random-effects types, with appropriately different interpretations and analysis for the two types.
mean squares between
SS between/df between In a one-way ANOVA, the value calculated by dividing the between sum of squares by the between degrees of freedom
Benjamini-Hochberg LSU
The Benjamini-Hochberg (BH) procedure is a popular method for controlling the False Discovery Rate (FDR) in multiple testing experiments. one of the best ways to control the type 1 error rate.
Structural model variables
The variables pi_i and e_ij are assumed to be independently and normally distributed around zero within each treatment. Their variances, simga_pi^2 and simga_e^2, are assumed to be homogeneous across treatments. The error term and subject factor are considered to be random, so those variances are presented as simga_pi^2 and simga_e^2. (Subjects are always treated as random.) However, the treatment factor is generally a fixed factor, so its variation is denoted as theta_tao^2 An alternative and probably more realistic model is given by
Method 3
Unique. Lose info because of correlated IVs. If correlation is large could lose a lot of info.
What to do when N is unqueal for the GLM?
Use method 1, 2, or 3
funnel plot
Used to assess *publication bias* Y axis is size of study X axis is effectiveness of drug Image on right shows publication bias b/c it isn't shaped like a funnel. Graphic representations of possible effect sizes (ESs) for interventions in selected studies. A graphical display used in meta-analysis to look for publication bias.
Covariate
Variables that might affect the dependent variable, but are not associated with the independent variable - essentially a control variable. a continuous variable included in the statistical analysis as a way of statistically controlling for variance due to that variable characteristics fo the participants in an experiment. If you collected data before you run an experiment, you could use that data to see how your treatment effects different groups or pops . Can also control covariate influence.
Meta analysis random model
We assume there is a distribution of true effect (not just one). The combined effect therefore cannot represent the one common effect, but instead represents the the mean population of effects.
rank-randomization tests
a class of nonparametric tests based on the theoretical distribution of randomly assigned ranks all randomization test give same value
between-subjects design
a different group of subjects is tested under each condition A research design in which different groups of participants are randomly assigned to experimental conditions or to control conditions. extent to which group means differ from the grand mean
forest plot
a graph that displays the point estimates and confidence intervals of individual studies included in a meta-analysis or systematic review as a series of parallel lines. type of diagram used to present the meta-analysis results of studies with dichotomous outcomes
Meta-anlysis
a procedure for statistically combining the results of many different research studies Combined results from numerous studies to assess the effect of common variables
Interaction effect
a result from a factorial design, in which the difference in the levels of one independent variable changes, depending on the level of the other independent variable; a difference in differences whether the effect of the original IV depends on the level of the other IV
pi tau _ij (interaction effect)
allows different subjects ot change differently over treatments. The assumptions of the 1st model will continue to hold. Assume pi tau_ij to be distributed around zero independently
alphaFW = ?
alphaFW = 1 - (alpha_PC)^C
alphaPC ? alphaFW ? C(alphaPC)
alphaPC < alphaFW < C(alphaPC)
intraclass correlation
an ANOVA technique used for estimating the reliability of a measure a form of correlation used when pairs of scores do not come from the same individual, as when correlations are calculated for pairs of twins how much judges agree within subjects is a judges effect r_I is teh average of all possible Pearson COrrelations
nonadditivity
assess whether the factor variables are additivitely related to the expected value of the response variable (1 degree of freedom )
sphericity assumption
assumption of the repeated measures ANOVA that pairs of scores in the population have equal variance. assumption of the repeated measures (within-subjects) ANOVA that pairs of scores in the population have equal variance condiiton where the variances of the differences between all possible pairs of within subjects ocnditions are equal. Violation of sphereicity occurs when it is not the case that the variances of the differences between all combinations of the conditions are equal (ranking of subjects does not change across experiment)
We estimate _ subject variation and pull it out. This way we can look at only the variation _ subjects (the treatment)
between and within
orthogonal contrasts
contrasts that are independent of one another such that there is no overlap among contrasts
R^2 change
difference between the R^2 value obtained with the second regression equation and the R^2 value obtained with the first regression equation
r^2 change
difference between the R^2 value obtained with the second regression equation and the R^2 value obtained with the first regression equation change in R^2 between 2 equations. > associated with its own F value which tests whether delta (change )is significant
fixed effect meta-analysis
different effect estimates are attributed to the random sample error Assuming each of the studies included are estimating the same underlying parameter. mu
Post Hoc
done after your tests are completed
random effects model
effect sizes of primary studies are weighted as a compromise between sample size and number of primary studies to calculate the summary effect In meta-analysis, a model in which studies are not assumed to be measuring the same overall effect, but rather reflect a distribution of effects; often preferred to a fixed effect model when there is extensive variation of effects across studies. A statistical method that assumes studies in a meta-analysis are measuring different effects Random-effects model (class II) is used when the treatments are not fixed. This occurs when the various factor levels are sampled from a larger population. Because the levels themselves are random variables, some assumptions and the method of contrasting the treatments (a multi-variable generalization of simple differences) differ from the fixed-effects model. can be generalized to the next study assume differences in variation is a true effect
residual vs error
error (something i can estiamte) residual as something left over
Error vs Residual
error is something you can estimate residual is something that is left over
Tau is fixed or random?
fixed
partial effect size
https://www.theanalysisfactor.com/effect-size/
Sheffe test
if concerned about type 1 errors, the most concervative test you can do is the *scheffe test*. if alpha = 0.05 and you have 5 groups k-1 = 4 then (4x) is a method for adjusting significance levels in a linear regression analysis to account for multiple comparisons. It is particularly useful in analysis of variance (a special case of regression analysis), and in constructing simultaneous confidence bands for regressions involving basis functions. Scheffé's method is a single-step multiple comparison procedure which applies to the set of estimates of all possible contrasts among the factor level means, not just the pairwise differences considered by the Tukey-Kramer method. It works on similar principles as the Working-Hotelling procedure for estimating mean responses in regression, which applies to the set of all possible factor levels.
Dunnett's Test
is a multiple comparison procedure developed by Canadian statistician Charles Dunnett to compare each of a number of treatments with a single control. Multiple comparisons to a control are also referred to as many-to-one comparisons.
a priori
is a planned comparison. A world were you have a small number of theorically tests that go to the heart of the study and everything else is posthoc
Most common a priori tests
multiple t-test comparisons linear contrasts
a priori procedures
multiple t-test, linear contrasts, and Bonferroni, & Bonferroni-Holm
Model Comparison
nested models with each other. Model b is nested in model a. Does the reduced model predict just as well as the full model. run an F test to find out, if significant additional predictor of the full model are making a significant prediction.
Repeated measures assumptions
observation are not independent but we assume independence and normally distributed homogeneity of variance > spherical
hierarchical partioning of R^2 leads to semi-partial being called
r^2 change
pi is fixed or random?
random
Randomization gets what? Jackknife and bootstrap get what?
randomization - pvalues J and bootstrap - get SEMs to make confidence intervals > This way one can make their own critical values.
Bootstrapping
randomization with replacement
Repeated measure within subject
represents the variability of a particular value for individuals in a sample. Measure of how much an individual in your smaple tends to change over time.
Simple regression vs fixed vs random
simple regression does not allow for within-study variation fixed does not allow for between study variaiton Random allows both with and between study variation
Tukey HSD (honestly significant difference)
tests all pairwise comparisons
Family-wise error rate
the probability that a family of comparisons contains at least one Type I error When comparing means of multiple groups, pair by pair using a t-test, this error rate is the probability of falsely rejecting the null hypothesis in either one of those tests.
P values do not alone indicate...
the size of an effect
Sum of Squares (SS)
the sum of the squared deviation scores
eta^2
the total variation in y, the proportion that can be attributed to a specific x increase variables, the proportion explained by any one variable decreases
Expected means squares
theta^2_alpha = sum(alpha_i^2) / (a-1) (*The a-1 is a fixed effect*) theta^2_beta = sum(beta_j^2) / (b-1) theta^2_alphaxbeta = sum(alphaxbeta^2) / (a-1)(b-1) *value is without samlping error*
Random study meta-analysis characteristics
weights are a function of tau^2, small studies are given same weight to large studies if there is true heterogeneity, if study i svery large, it gives a precise estimate of mu_i the study specific effect. gives more info about mu_i but since mu_i is not equal to mu, it gives less info than a fixed effect
interaction effect
whether the effect of the original independent variable depends on the level of another independent variable tests the effect of one independent variable at each level of another independent variable in an ANOVA a result from a factorial design, in which the difference in the levels of one independent variable changes, depending on the level of the other independent variable; a difference in differences The affect of A depends on the level of B and visa versa, important for show the limits of any kind of effect we are trying to evaluate.
jackknife statistics
with replacement but decreasing N after each measure
Equations for models general and more specific
x_ij = mu + tau_j + e_ij; 4 x_ij = mu + pi_i + tau_j + e_ij
Model 3 equation also known as what model?`
x_ij = my +tau_j + pi tau_ij + e_ij random effects model
repeated measure structural model
(Model 1)x_ij = mu + pi_i + tao_j + e_ij mu = grand mean pi = constant associated with the ith person representing how much that person differs from the average person. tau = a constant associated with the jth treatment, representing how much that treatment differs from the average treatment e = experimental error associated with the ith subject under the jth treatment
random effect model
- model of meta-analysis - assumes studies are not completely the same - wider CI and more realistic than fixed effect model ES for repeated measure, how big is the difference between subjects differences. How much do the judges agree within subjects is a judge effect. Both subjects and judges are a random effect .
MANOVA (multivariate analysis of variance)
A form of the ANOVA used when a study includes one or more IVs and two or more (3, 4+) DVs, each of which is measured on an interval or ratio scale. Use of this test helps reduce the experimentwise error rate and increases power by analyzing the effects of the IV(s) on all DVs simultaneously. multivariate anova with 2 or more continuous variables with at least 2 factors (DVs). Not very powerful. 1) less power than anova 2) if 1 person does 4 different trials, MANOVA treats them as 4 different variables, not as the same thing
Familywise vs FDR
FW is the probability of making >1 T1 errors, among all the hypthoses when performing mulitple hypothesis tests FDR: the expected proportion of significant tests (not all tests) that will be T1s When all H0s are true FW = FDR.
Unweighted average
Gives equal weight to all values with no regard for other factors. Raw average of data that gives equal weight to all values, with no regard for other factors. unweighted means approximates an ordinal test least squares ignoring the unequal N.
What tests can you use if the Machly test shows that the sphereicty condition is not met.
Greenhouse-Geisser and Hynh - Feldt (e)
Method 2
Hierarichial. Only use iff SS interaction is not significant Otherwise, do not use.
Bonferroni correction
If really concerned about type 1 error, alpha_PC = alpha_FW / C (number of contrasts). 0.05 / 5 = 0.01 (alpha_PC) *Bonferioni correction* what is the appropiate family to compare? arbritary to set C. A multiple comparison procedure in which the familywise error rate is divided by the number of comparisons A procedure for guarding against an increase in the probability of a type I error when performing multiple significance tests. To maintain the probability of a type I error at some selected value (α), each of the m tests to be performed is judged against a significance level (α/m ). For a small number of simultaneous tests (up to five) this method provides a simple and acceptable answer to the problem of multiple testing. It is however highly conservative and not recommended if large numbers of tests are to be applied, when one of the many other multiple comparison procedures available is generally preferable.
what does eplison correct for?
If sphereicty is not meet, uses deviation to s[hereicity to crrect the number of degrees of freedoms of the F distrubiton noted by epsilon corrects for violation of specificity.
Linear contrasts
In statistics, particularly in analysis of variance and linear regression, a contrast is a linear combination of variables (parameters or statistics) whose coefficients add up to zero, allowing comparison of different treatments. Suppose you had 3 means and a prior desire to compare the mean of group 3 to the other 2. .5xbar1 + .5xbar2 - 1xbar3; contrast group 3 to the average of the other 2. The .5 .5 and 1 are called *contrasts groups*. A being contrast for the jth group times the mean of the jth group and then summing them up (L-scores). sum a = 0 want sum(A) = 0. sum(abs(A)) = 2, easier to interpret as a d however not crucial The ratio has to be the same but the numbers can be different such as 50 50 100 if only 1 df in an F ratio, F = t^2 allows us to write a SS for contrasts SS_contrast / df_C / (SS_error / df_D) = F(1, df_D). Creating orthogonal contrasts: sum(A) = 0, sum(a_jb_j) = 0, & sum(abs(a)) = 2, then the contrast are orthogonal . Contrast don't have to be orthogonal but doesn't make sense to add them if not true These two approaches: linear contrast and multiple t-rest are the most common a prior tests.
E(MS) table
Source E(MS) A simgahat^2_e + nthetahat^2_alpha B simgahat^2_e + nthetahat^2_beta AB simgahat^2_e + nthetahat^2_alphaxbeta *now has sources of error*
R^2
Multiple Regression variance explained by the model, the part that is not explained is due to error. Examines the impact of a moderator variables on a study ES he proportion (percent) of the variation in the values of y that can be accounted for by the least squares regression line proportion of the variance in the DV that is predictable from the iVs
Method 1
Sequential. cant decide between a and b, order is aribtoary but it matters. A will always be bigger. Only works if you know which variable will be / should be first
REGWQ Test
Sort of like the Bonferroni logic, except that each test is run at a/(r/k) where k = number of means in the experiment, and r is the number of means from which these two are the largest and smallest. Einot, Gabriel, and Welch fiddled with this just a little bit, but the basic idea is still right. This test keeps FW at alpha regardless of the true null, but is less conservative than Tukey's test.
Sphericity
Sphericity is an important assumption of a repeated-measures ANOVA. It refers to the condition where the variances of the differences between all possible pairs of within-subject conditions (i.e., levels of the independent variable) are equal. The violation of sphericity occurs when it is not the case that the variances of the differences between all combinations of the conditions are equal. If sphericity is violated, then the variance calculations may be distorted, which would result in an F-ratio that would be inflated.[1] Sphericity can be evaluated when there are three or more levels of a repeated measure factor and, with each additional repeated measures factor, the risk for violating sphericity increases. If sphericity is violated, a decision must be made as to whether a univariate or multivariate analysis is selected. If a univariate method is selected, the repeated-measures ANOVA must be appropriately corrected depending on the degree to which sphericity has been violated`
omega squared
Statistical index of the degree to which the independent variable accounts for the variance in the dependent variable determines the strength or magnitude of the difference in the mean scores a less biased measure of the magnitude of effect than eta squared
Example of fixed, random, and mixed effects
Teaching experiments could be performed by a college or university department to find a good introductory textbook, with each text considered a treatment. The fixed-effects model would compare a list of candidate texts. The random-effects model would determine whether important differences exist among a list of randomly selected texts. The mixed-effects model would compare the (fixed) incumbent texts to randomly selected alternatives.
Tukey test of nonadditivity
This assumption that this interaction is 0 which lets to go to model 1, called the assumption of *nonaddivity* by block by treatment or subject by treatment interaction Tukey's test of additivity is an approach used in two-way ANOVA (regression analysis involving two qualitative factors) to assess whether the factor variables are additively related to the expected value of the response variable. It can be applied when there are no replicated values in the data set, a situation in which it is impossible to directly estimate a fully general non-additive regression structure and still have information left to estimate the error variance. The test statistic proposed by Tukey has one degree of freedom under the null hypothesis, hence this is often called "Tukey's one-degree-of-freedom test." We estimate the between subjects variation and pull it out (remove it). This way we can look at only the variation within subjects (the treatment)
multilevel modeling
an approach to analyzing data that have a nested structure in which variables are measured at different levels of analysis; for example, when researchers study several preexisting groups of participants, they use multilevel modeling to analyze the influence of group-level variables and individual-level variables simultaneously statistical techniques that can describe relations between variables when data are varied on multiple levels
multiple regression model
an equation that describes the relationship between a dependent variable and more than one independent variable an employee selection method that combines separate predictors of job success in a statistical procedure
pi and e
are assumed independent and normal around 0 within each mean. Variances are assumed to be homogeneious. pi and e are considered to be random. And treatment factor is generally consider to be fixed, so its variation is noted by theta^2
assumptions: to compute 3 different F's
assume these error valyues are independent, normal with variances that are the same of every level of each of the IVs including their conjuctions (homogeneious variance) error: difference between each observation and the cell (x - xbar_cell) subtract from each observation the cell mean. add up to zero in each cell (by definition) table of effects: estimates of alpha, beta, and alphaxbeta but obtaining a table of effects and residuals, you can get the SS df = number of levels - 1 df(within) = N - A - B F = the MS (A,B,AB) / MS within cell = 3 F's
multiple regression beta and b's
b's allow fo reaiser interpretation of the effects of an IV because b's keep their units. However, can become difficult if the units are very dissimilar. Beta's are hard to interpret because the units are in standard deviations, but allow us to compare the effects of the different IVs on the DVs
fixed effects model
can be used to create a pooled estimate when the studies are fairly homogenous. A statistical method that assumes that differences in results among studies in a meta-analysis are due to chance; in other words, the results are assumed to be similar in meta-analysis, a model in which studies are assumed to be measuring the same overall effect; a pooled effect estimate is calculated under the assumption that observed variation between studies is attributable to chance he fixed-effects model (class I) of analysis of variance applies to situations in which the experimenter applies one or more treatments to the subjects of the experiment to see whether the response variable values change. This allows the experimenter to estimate the ranges of response variable values that the treatment would generate in the population as a whole. can only be generalized to the next person assume differences are due to sampling error
main effects of factorial design
effect of each IV on the DV (effects of each IV) A. The effects of the first independent variable by itself (main effect IV 1) B. The effects of the second independent variable by itself (main effect IV 2) And so on...for all independent variables.
Cohen's f
effect size index for use in Jacob Cohen's power tables for F tests of significance defines an effect in terms of a measure of dispersal among group means A measure of effect size when there are more than two means that defines an effect relative to the degree of dispersal among group means. Based on Cohen's guidelines, an f value of .10, .25, and .40, defines a small, medium, and large effect size, respectively.
Error rate per experiment (PE)
error rates that the researcher could use as a control level in a multiple hypothesis experiment.
Fixed study meta-analysis characteristics
estimates are weighted purely according to their estimates variances (sigma_hat^2). If 1 very large study, smaller studies are given less weight This is because under an assumption of common effects, this weighting results in the most precise estimate of the common effect
why is eta squared biased?
eta^2 is a biased measure of population variance but is acccuate for for the sample. It is always an overestimation increase N decrease bias
Power in factorial design
f: .1, .25, .40 phi_A = f(sqrt(nb)) phi_B = f(sqrt(na)) phi_AB = f(sqrt(n)) phi = fsqrt(number das a mean is based on)
Bonferroni-Holm approach
is used to counteract the problem of multiple comparisons. It is intended to control the family-wise error rate and offers a simple test uniformly more powerful than the Bonferroni correction. It is named after Sture Holm, who codified the method, and Carlo Emilio Bonferroni. Holm Bonferinio treatment is a sequential treatment; arrange family of comparsions from the largest effect to the smallest. Test the biggest one (the most likely to significant) at the Bonferioni value alpha_FW / C. BUt then do the next one at alpha / C-1, and then alpha / C-2 and stop when you hit something non-significant that holds the FW comparison at 0.05. THis procedure is the better than the traditional bonferinoi
Effect size
omega^2 = sigma^2_effect / simga^2_total omega^2_partial = sigma^2_effect / simga^2_error + simga^2_effect 3 possible variances of effect (A,B,AB) eta - biased, omega - unbiased little f (ES measure) f = sqrt(simga^2_effect / simga^2_err) = sqrt(omega^2_partial / 1 - omega^2_partial) used for power analysis
within-subjects design
participants are exposed to all levels of the independent variable a research design that uses each participant as his or her own control; for example, the behavior of an experimental participant before receiving treatment might be compared to his or her behavior after receiving treatment extent to whihc individual scores x deviate from the group mean, mu.
Error rate per comparison (PC)
per-comparison error rate (PCER) is the probability of a Type I error in the absence of any multiple hypothesis testing correction. This is a liberal error rate relative to the false discovery rate and familywise error rate, in that it is always less than or equal to those rates.
Fisher LSD test
requires an F ratio (only post hoc that does) Fisher's Least Significant Difference) is a very strong test in detecting pairs of means differences, it is applied only when the F test is significant, and it is mostly less preferable since its method fails in protecting low error rate. Bonferroni test is a good choice due to its correction suggested by his method. This correction states that if n independent tests are to be applied then the α in each test should be equal to α /n. Tukey's method is also preferable by many statisticians because it controls the overall error rate. On small sample sizes, when the assumption of normality isn't met, a Nonparametric Analysis of Variance can be made by Kruskal-Wallis test, that is another omnibus test example ( see following example ). An alternative option is to use bootstrap methods to assess whether the group means are different. Bootstrap methods do not have any specific distributional assumptions and may be an appropriate tool to use like using re-sampling, which is one of the simplest bootstrap methods. You can extend the idea to the case of multiple groups and estimate p-values.
Tukey's HSD
single # that determines minimum difference between 2 means that is necessary for significance A procedure for the multiple comparison of means after a significant F ratio has been obtained Tukey HSD based on a range statistcic (q) - *pairwise* q is very close to a t-test, use q as an inferential statistic, built to do all pairwise comparisons holding the FW at 0.05.
False Discovery Rate (FDR)
statistical procedures designed to reduce the amount of false positives within a sample. an approach for correcting for many statistical comparisons based on the number of positive results obtained is a method of conceptualizing the rate of type I errors in null hypothesis testing when conducting multiple comparisons. FDR-controlling procedures are designed to control the expected proportion of "discoveries" (rejected null hypotheses) that are false (incorrect rejections). FDR-controlling procedures provide less stringent control of Type I errors compared to familywise error rate (FWER) controlling procedures (such as the Bonferroni correction), which control the probability of at least one Type I error. Thus, FDR-controlling procedures have greater power, at the cost of increased numbers of Type I errors.
mutliple t-tests
t-test to get leverage on the error estimate of the study using all data. Based on all people not just the subgroup of groups. Get to use all ppl in the other groups to estiamte the error and the dfs of the demontor of F is based on the dfs of the error term not the numerator to give you more power
partial eta squared
the amount of variance in the dependent variable uniquely explained by a single categorical independent variable the effect size for a dependent multiple-group design that removes the variability unique to individual participants from the error term
Semi-partial correlation squared
the amount of variance in the dependent variable uniquely explained by a single quantitative independent variable
Partial correlation
the correlation between two variables with the influence of a third variable statistically controlled for
Model 2 (repeated measures anova) more realistic equation
x_ij = mu + pi_i + tau_j + pi tau_ij + e_ij tau = fixed, pi - random, treatment is fixed
Repeated measure anova structural model
x_ij = mu + pi_i +tau_j + e_ij structural model: Xij = mu + pi_i +tau_j + eij mu = grand mean pi_i = constant associated with the ith person or subject, representing how much that person differs from the average person tao_i = a constant associated with the jth treatment, representing how much that treatment mean differs from the average treatment mean e_ij = experimental error associated with the ith subject under the jth treatement.
Model of main effects and interactions
x_ij = mu + tau_j + e_ij x_ijk = mu + alpha_i + beta_j + alphaxbeta_ij + eijk alpha_i = mu_i - mu beta_j = mu_j - mu alphaxbeta_ij = mu_ij - mu_i - mu_j + mu alpha_i = deviation due to treatment level a score of the kth person in the ith level of a and the jth level of b