Biostats exam 3
- modified t tests for orthogonal comparisons (1v2, 3v4) - downward adjustment of alpha for multiple comparisons to avoid type 1 error (divided alpha by # of comparisons to be made) - bonferroni t procedure: can be used for non-orthogonal comparisons (1v2, 1v3, 2v3); increases the critical F value from the table based upon the number of comparisons to be made and the sample size
3 options for a priori or planned comparisons
QQ plots or hypothesis testing (Ho sample: sample distribution = normal distribution)
how do you know if a variable is normally distributed
- difference between individual observations and the mean of the group the subject is in - the group means versus the grand mean (all data)
ANOVA divides the total variation of a variable into...
- random sampling (or at least representative of the population) - normal distribution - equal variance - independence between study groups
F test assumptions
variance of group means from the grand mean (MSa) / variance of subjects within groups (MSe)
F test equation
true
T/F it is incorrect to state a generic product must contain between 80-125% of the brand name product in order to be considered bioequivalent
false, look at r values too
T/F you can judge a study by p value alone
true, look at r values too
T/F you cannot judge a study by p value alone
false, only positive
T/F F value will have a positive or negative distribution
true
T/F F value will only have a positive distribution
false (that would require a test of equivalence)
T/F P > alpha suggest that there is no difference or equivalence
true
T/F P > alpha suggests insufficient evidence to reject Ho
true
T/F analysis of nonparametric tests are not affected much by outliers or the shape of an outcome variables' distribution
false
T/F analysis of nonparametric tests is affected by outliers or the shape of an outcome variables distribution
true
T/F correlation does not prove causation
false
T/F correlation proves causation
false (means that there could be a 45% difference in strength of generic vs brand)
T/F it is correct to state a generic product must contain between 80-125% of the brand name product in order to be considered bioequivalent
statistical normality tests
__________ can be used to determine if sample data is significantly different from the normal distribution
chi square tests
__________ tests compare frequencies (proportions) in two (or more) groups
ANOVA
__________ uses the F test to evaluate the ratio of the mean squared differences
directional
a one-tailed test is an example of a non-directional/directional test
non-directional
a two-tailed test is non-directional/directional
either no difference or a difference too small to matter
alternative hypohtesis of testing for equivalence
experimental treatment is either equivalent to the standard or better; it is not worse
alternative hypothesis for testing for non-inferiority
Ha = u1 > u2
alternative hypothesis for testing for superiority
Ha = u1 - u2 does not = 0 Ha = u1 does not = 0 (non-zero difference, risk ratio is not 1)
alternative hypothesis in testing for differences
- sample randomly selected from population - each subject must contribute data for x and y - two variables x and y vary together - x was NOT used to compute y - x cannot be experimentally controlled - no outliers
assumptions in correlation
linear relationship present
assumptions of person correlation
relationship be monotonic (always increasing or decreasing but not necessary in a straight line)
assumptions of spearman's rho
- no universally approved method for defining ∆ - typically look at prior trials comparing a standard treatment vs prior standards or placebo and determine smallest plausible benefit
how is delta (∆) determined in tests for non-inferiority
- non-inferiority trials cost less than non-directional/superiority trials - even if a novel agent is on non-inferior (not superior) it may be preferred if it is cheaper, safer, or has better tolerability - lower risks of non-FDA approval
benefits of testing for non-inferiority
observed frequency in each cell with expected frequency
chi square test compares what
- fishers z transformation permits formation of confidence intervals - z transformation of r +/- confidence coefficient * standard error
confidence intervals for correlation
- is the standard of care as effective as it was in prior studies? - at the start of the study, were the novel and standard treatment groups at the same risk for the outcome - at the end of the trial, were the groups at similar risk
considerations regarding internal validity in testing for non-inferiority
two continuous variables that are linear
correlation and linear regression explore relationships between...
population: p (rho) sample: r
correlation in a population? sample?
skewness and kurtosis to quantify how far the sample distribution deviates from a normal distribution
d'agostino-oearson omnibus k2 normality test assesses....
columns: J-1 (# groups - 1) rows: N-j (grand N - # groups)
how to determine DF for critical F table value
- if data is skewed or kurtotic, investigators can attempt to "transform the data" into a normal distribution by using LOG(x), Ln(x), or sqrt(x) and used this transformed data during the statistical analysis - after the stats have been run, the measure of central tendency and dispersion need to be transformed back (10^x, e^x, x^2) - allows investigators to use the more powerful parametric tests rather than non-parametric test
data transformation for skewed or kurtotic data
df = (#rows - 1) * (#columns - 1) only count rows/columns with group/outcome specific data (don't include totals)
degree of freedom for chi square test
n - 1
degree of freedom when comparing 2 dependent means
n1 + n2 - 2 = DF
degree of freedom when comparing 2 independent means
- test of superiority - tests of non-inferiority
examples of directional tests
- tests of difference - test of equivalence
examples of non-directional tests
(column total * row total) / grand total
expected frequency equation in chi square test
90%
for non-inferiority studies what confidence interval is used
medians between 3 or more dependent groups
friedman 2-way ANOVA compares...
repeated measures ANOVA or ANCOVA or time series
friendman 2-way ANOVA is the nonparametric alternative to...
- percentiles for each value in a dataset are computed - the number of standard deviations needed to reach each specific value on a guassian distribution is determined - the actual mean and computed standard deviation in step 2 are used to determine a predicted value for each observation (as if it were from a normal distribution) - the actual values (data on y axis) is plotted along with their predicted values (data on x axis) to produce a scatterplot - if the actual data is normally distributed the plotted xy data should fall closely along a diagonal, positively sloped line
how are QQ plots created
observed frequency will be similar to expected frequency value of chi square statistic will be small
if no relationship exists, the observed frequency will be... and the value of the chi square statistic will be...
nonparametric
if the data cannot meet parametric test assumptions, __________ equivalents are available
planned or post hoc approaches
if the global test (F test) suggests one or more pairs differ, what should be used to determine which pairs are differing
considerable variation will occur between groups and the grand mean, compared with the variation within each group
if the groups of mean squares substantially differ, what does that indicate
x = independent variable y = dependent variable
in correlation... x = independent/dependent variable y = independent/dependent variable
equivalence zone or equivalence margin or region of practical equivalence
in equivalence testing, researchers must carefully define what constitutes equivalence when testing if two treatments are indistinguishable, this is referred to as...
- data follows a normal distribution - standard deviations in the population are equal - observations are independent
independent sample t test can be used if we assume the following test assumptions are met...
p > a = Ho: insufficient evidence to suggest the sample distribution differs from a normal distribution (what you want to conclude) p < a = Ha: sample distribution differs from a normal distribution (use nonparametric tests if data transformation is not possible)
interpreting the p value from the statistical normality tests p > a: p < a:
- 0 - 0.25 = little to no relationship - 0.25 - 0.5 = fair degree of relationship - 0.5 - 0.75 = moderate to good relationship - > 0.75 = very good to excellent relationship
interpreting the size of r^2
ANOVA
kruskal-wallis is the nonparametric alternative to....
3 or more independent groups
kruskal-wallis test compares...
- less powerful than parametric tests if the data is normally distributed (don't use these unless you have to) - usually not reported with CIs - not readily extended to regression models
limitations of nonparametric tests
linear regression
linear regression/correlation determines an equation for predicting the value of an outcome (dependent variable) from an exposure/explanatory (independent) variable
linear regression
linear regression/correlation determines strength of relationship but also used for prediction
correlation
linear regression/correlation determines strengths of relationship between two variables
correlation
linear regression/correlation typically involves continuous data, but methods exist to evaluate the relationship between nominal or ordinal data as well
medians of two independent groups
mann-whitney U test compares...
two sample/students t test
mann-whitney u test is the nonparametric alternative to...
usually means poorly performing students were more likely to get the question right (usually by guessing)
negative point-biserial
confidence interval
non-inferiority studies generally use hypothesis testing/confidence interval
tests for ordinal data
non-parametric tests test for..
- spearman's rho for ordinal or non-normally distributed continuous data (nonparametric alternative to pearson's correlation)
nonparametric test for correlation
- rank ordered lists - groups are compared by analyzing their ranks rather than the raw data
nonparametric tests convert raw data into...
- 2 groups: wilcox signed rank (nonparametric alternative to paired t test) - > 2 groups: friedman 2-way ANOVA (nonparametric alternative to repeated measures ANOVA)
nonparametric tests for dependent study groups
- 1 group: sign test (nonparametric alternative to one sample t test) - 2 groups: mann-whitney U (nonparametric alternative to two sample/students t test) - > 2 groups: kruskal wallis (nonparametric alternative to ANOVA)
nonparametric tests for independent study groups
the variances in the F test numerator and denominator are equal
null hypohtesis of F test
Ho = u2 - u1 is greater than or equal to ∆ experimental treatment is worse than the standard treatment
null hypothesis for hypothesis testing for non-inferiority
Ho = u1 - u2 = 0 Ho = u1 = u2 (no difference between treatments)
null hypothesis for testing for differences
Ho = u1 - u2 is less than or equal to 0 (testing if group 1's mean is greater than group 2's)
null hypothesis for testing for superiority
Ho = | u1 - u2 | is greater or equal to ∆ a difference large enough to matter exists
null hypothesis of testing for equivalence using hypothesis testing
lower
p values in parametric tests are typically lower/higher compared to non-parametric on the same data set
tests for continuous data
parametric tests test for what type of data
parametric tests
parametric/non-parametric tests are more likely to reject null hypothesis if appropriate
parametric
parametric/non-parametric tests are statistically powerful
non-parametric
parametric/non-parametric tests are valid in a broader range of situations, (more robust)
non-parametric
parametric/non-parametric tests have fewer required conditions of validity
determines correlation between whether examines get a specific question correct (binary variable y/n) and the examinee's score on the entire exam (a numerical variable)
point biserial correlation used to evaluate test questions ...
indicates that students who answer the question correctly tend to score high on the exam as a whole, whereas students missing the question tend to score low generally
positive point-biserial
- it's possible that X does in fact cause Y - it's also possible that Y causes X - an unknown factor, z, influences x and y proportionately - several other factors influence x and y proportionately - the correlation discovered in the sample is just a coincidence
possible explanations if you are presented with two variables that correlated with each other
tukeys
post hoc comparison that has a moderate threshold to declare significance (commonly used)
scheffes
post hoc comparison that has the highest threshold to declare significance (hardest to show significance)
- scheffes procedure - tukeys HSD - dunnet's
post hoc comparisons
scheffes procedure
post hoc comparisons used to both pairwise and non-pairwise (1v2, 1v3, 2v3, 1v[(2+3)/2], etc
tukeys
post hoc only used to pairwise comparisons, 1v2, 1v3, 2v3
dunnet's
post hoc that has very low threshold to declare significance (easiest to show significance)
dunnet's
post hoc used when treatment means are compared to control only (not to each other)
- chi-square test can be used to test the equality of two proportions - chi square test can be used to test whether one of the variables is associated with the other
research question of chi square test
research questions are framed as directional, but tested using a non-directional test
research questions are often framed as non-directional/directional, but tested using a non-directional/directional test
median of one independent group to a national average/cut point (nonparametric alternative to one sample t test)
sign test compares...
one sample t test
sign test is the nonparametric alternative to...
slide 14-16
slide 14-16 of basic statistical concepts equivalence and non-inferiority studies
slide 9-10
slide 9-10 of basic statistical concepts equivalence and non-inferiority studies
- two ordinal variables - one ordinal variable and one continuous variable - two continuous variables (but one or more which is not normally distributed)
spearman's rho is used to assess correlation between...
- d' agnostino-pearson omnibus K2 normality test - shapiro wilk test - komogorov-smirnov test - darling-anderson test
statistical normality tests
df = N - 2
test statistic for p = 0, how do you calculate the degree of freedom
non-directional tests
test that looks in both positive and negative directions
directional test
test that looks only in one direction
tests whether the difference between two quantities is 0
testing for differences
- hypothesis testing - estimation
testing for differences can be tested using...
test to determine if two treatments are equal/equivalent confidence intervals are used to determine if key parameters (drug peak concentrations, AUC, etc) are equivalent
testing for equivalence
- 2 one-sided t-tests or - confidence interval using twice the stated alpha level (5% alpha with a 90% confidence interval)
testing for equivalence using a hypothesis test is performed using...
tests whether two quantities are within an acceptable range of each other (shown as ∆)
testing for equivalence using hypothesis testing
- t test for two groups - bartlett's for three or more groups
testing for similar variance for < 20 per group or if one or more distribution is skewed or has heavy tails considered
levine's test
testing for similar variance for > 20 subjects per group with a normal distribution
tests whether one quantity is greater than or less than the other
testing for superiority
- one tailed test - confidence interval
testing for superiority is performed using what type of tests
- independent - the variables follow a normal distribution - there are similar sample variances between the groups
the student's t test assumes the 2 groups are... (parametric conditions so that the results reliable)
- joint distribution that is normally distributed (bivariate normal distribution) - covariation needs to be linear [can't use correlation for curvilinear data (J shaped)]
two variables, x and y vary together is an assumption of correlation, what does it mean
point biserial correlation
used to correlate a numerical and binary nominal variable
tests if a novel intervention is no worse than a standard treatment, beyond a specified margin (non-inferiority margin/threshold is symbolized as ∆)
what are tests for non-inferiority
the columns and rows are independent so that you can calculate the expected frequencies
what assumption must you make in a chi square test
t distribution only used when testing p = 0
what distribution can test the null hypothesis p (rho) = 0 for correlation
describes the strength of relationship and ranges 1 to +1
what is r in regards to correlation scatterplot
use fisher's test if any expected cell count is < 2 or if more than 20% of cells have expected cell counts < 5
what test should you use if the expected frequencies are small (so you can't use chi square test)
little to no correlation exists
when r is near 0 there is/is not correlation
when expected frequencies within A-D are small
when should the chi square test not be used
when parametric test assumptions are not met: normal distribution (and unable to transform the data), similar variance
when should you use non-parametric tests
scheffes
which post hoc comparison is mostly limited to non-pairwise data
ANOVA
which test allows comparisons of 3 or more means while holding alpha to 5%
90% allows 5% error at both the lower bound (subtherapeutic) and the upper bound (supratherapeutic) end of the confidence interval since a drug cannot be both
why a 90% confidence interval and not 95% for testing equivalence?
due to +1 and -1 bounds of r causing skewing of distribution
why can't you use t test when null hypothesis p < or > 0
reciprocal of 80% is 125% (1/0.8 = 1.25)
why have AUC at 0.8 and 1.25 and not a symmetrical 0.8 and 1.2
interested in relationship, not if a difference exists
why not just use a t test to correlate a numerical binary nominal variable instead of the biserial correlation
it increases the risk of a type 1 statistical error - 3 tests needed to compare 1v2, 1v3, 2v3 - ~15% alpha
why should you not use multiple t tests to compare 3 or more groups
medians among two dependent groups
wilcoxon signed rank compares...
paired t test
wilcoxon signed rank is the nonparametric alternative to...
correlation
xy scatterplots graph what relationships
