Biostats exam 3

- modified t tests for orthogonal comparisons (1v2, 3v4) - downward adjustment of alpha for multiple comparisons to avoid type 1 error (divided alpha by # of comparisons to be made) - bonferroni t procedure: can be used for non-orthogonal comparisons (1v2, 1v3, 2v3); increases the critical F value from the table based upon the number of comparisons to be made and the sample size

3 options for a priori or planned comparisons

QQ plots or hypothesis testing (Ho sample: sample distribution = normal distribution)

how do you know if a variable is normally distributed

- difference between individual observations and the mean of the group the subject is in - the group means versus the grand mean (all data)

ANOVA divides the total variation of a variable into...

- random sampling (or at least representative of the population) - normal distribution - equal variance - independence between study groups

F test assumptions

variance of group means from the grand mean (MSa) / variance of subjects within groups (MSe)

F test equation

true

T/F it is incorrect to state a generic product must contain between 80-125% of the brand name product in order to be considered bioequivalent

false, look at r values too

T/F you can judge a study by p value alone

true, look at r values too

T/F you cannot judge a study by p value alone

false, only positive

T/F F value will have a positive or negative distribution

true

T/F F value will only have a positive distribution

false (that would require a test of equivalence)

T/F P > alpha suggest that there is no difference or equivalence

true

T/F P > alpha suggests insufficient evidence to reject Ho

true

T/F analysis of nonparametric tests are not affected much by outliers or the shape of an outcome variables' distribution

false

T/F analysis of nonparametric tests is affected by outliers or the shape of an outcome variables distribution

true

T/F correlation does not prove causation

false

T/F correlation proves causation

false (means that there could be a 45% difference in strength of generic vs brand)

T/F it is correct to state a generic product must contain between 80-125% of the brand name product in order to be considered bioequivalent

statistical normality tests

__________ can be used to determine if sample data is significantly different from the normal distribution

chi square tests

__________ tests compare frequencies (proportions) in two (or more) groups

ANOVA

__________ uses the F test to evaluate the ratio of the mean squared differences

directional

a one-tailed test is an example of a non-directional/directional test

non-directional

a two-tailed test is non-directional/directional

either no difference or a difference too small to matter

alternative hypohtesis of testing for equivalence

experimental treatment is either equivalent to the standard or better; it is not worse

alternative hypothesis for testing for non-inferiority

Ha = u1 > u2

alternative hypothesis for testing for superiority

Ha = u1 - u2 does not = 0 Ha = u1 does not = 0 (non-zero difference, risk ratio is not 1)

alternative hypothesis in testing for differences

- sample randomly selected from population - each subject must contribute data for x and y - two variables x and y vary together - x was NOT used to compute y - x cannot be experimentally controlled - no outliers

assumptions in correlation

linear relationship present

assumptions of person correlation

relationship be monotonic (always increasing or decreasing but not necessary in a straight line)

assumptions of spearman's rho

- no universally approved method for defining ∆ - typically look at prior trials comparing a standard treatment vs prior standards or placebo and determine smallest plausible benefit

how is delta (∆) determined in tests for non-inferiority

- non-inferiority trials cost less than non-directional/superiority trials - even if a novel agent is on non-inferior (not superior) it may be preferred if it is cheaper, safer, or has better tolerability - lower risks of non-FDA approval

benefits of testing for non-inferiority

observed frequency in each cell with expected frequency

chi square test compares what

- fishers z transformation permits formation of confidence intervals - z transformation of r +/- confidence coefficient * standard error

confidence intervals for correlation

- is the standard of care as effective as it was in prior studies? - at the start of the study, were the novel and standard treatment groups at the same risk for the outcome - at the end of the trial, were the groups at similar risk

considerations regarding internal validity in testing for non-inferiority

two continuous variables that are linear

correlation and linear regression explore relationships between...

population: p (rho) sample: r

correlation in a population? sample?

skewness and kurtosis to quantify how far the sample distribution deviates from a normal distribution

d'agostino-oearson omnibus k2 normality test assesses....

columns: J-1 (# groups - 1) rows: N-j (grand N - # groups)

how to determine DF for critical F table value

- if data is skewed or kurtotic, investigators can attempt to "transform the data" into a normal distribution by using LOG(x), Ln(x), or sqrt(x) and used this transformed data during the statistical analysis - after the stats have been run, the measure of central tendency and dispersion need to be transformed back (10^x, e^x, x^2) - allows investigators to use the more powerful parametric tests rather than non-parametric test

data transformation for skewed or kurtotic data

df = (#rows - 1) * (#columns - 1) only count rows/columns with group/outcome specific data (don't include totals)

degree of freedom for chi square test

n - 1

degree of freedom when comparing 2 dependent means

n1 + n2 - 2 = DF

degree of freedom when comparing 2 independent means

- test of superiority - tests of non-inferiority

examples of directional tests

- tests of difference - test of equivalence

examples of non-directional tests

(column total * row total) / grand total

expected frequency equation in chi square test

90%

for non-inferiority studies what confidence interval is used

medians between 3 or more dependent groups

friedman 2-way ANOVA compares...

repeated measures ANOVA or ANCOVA or time series

friendman 2-way ANOVA is the nonparametric alternative to...

- percentiles for each value in a dataset are computed - the number of standard deviations needed to reach each specific value on a guassian distribution is determined - the actual mean and computed standard deviation in step 2 are used to determine a predicted value for each observation (as if it were from a normal distribution) - the actual values (data on y axis) is plotted along with their predicted values (data on x axis) to produce a scatterplot - if the actual data is normally distributed the plotted xy data should fall closely along a diagonal, positively sloped line

how are QQ plots created

observed frequency will be similar to expected frequency value of chi square statistic will be small

if no relationship exists, the observed frequency will be... and the value of the chi square statistic will be...

nonparametric

if the data cannot meet parametric test assumptions, __________ equivalents are available

planned or post hoc approaches

if the global test (F test) suggests one or more pairs differ, what should be used to determine which pairs are differing

considerable variation will occur between groups and the grand mean, compared with the variation within each group

if the groups of mean squares substantially differ, what does that indicate

x = independent variable y = dependent variable

in correlation... x = independent/dependent variable y = independent/dependent variable

equivalence zone or equivalence margin or region of practical equivalence

in equivalence testing, researchers must carefully define what constitutes equivalence when testing if two treatments are indistinguishable, this is referred to as...

- data follows a normal distribution - standard deviations in the population are equal - observations are independent

independent sample t test can be used if we assume the following test assumptions are met...

p > a = Ho: insufficient evidence to suggest the sample distribution differs from a normal distribution (what you want to conclude) p < a = Ha: sample distribution differs from a normal distribution (use nonparametric tests if data transformation is not possible)

interpreting the p value from the statistical normality tests p > a: p < a:

- 0 - 0.25 = little to no relationship - 0.25 - 0.5 = fair degree of relationship - 0.5 - 0.75 = moderate to good relationship - > 0.75 = very good to excellent relationship

interpreting the size of r^2

ANOVA

kruskal-wallis is the nonparametric alternative to....

3 or more independent groups

kruskal-wallis test compares...

- less powerful than parametric tests if the data is normally distributed (don't use these unless you have to) - usually not reported with CIs - not readily extended to regression models

limitations of nonparametric tests

linear regression

linear regression/correlation determines an equation for predicting the value of an outcome (dependent variable) from an exposure/explanatory (independent) variable

linear regression

linear regression/correlation determines strength of relationship but also used for prediction

correlation

linear regression/correlation determines strengths of relationship between two variables

correlation

linear regression/correlation typically involves continuous data, but methods exist to evaluate the relationship between nominal or ordinal data as well

medians of two independent groups

mann-whitney U test compares...

two sample/students t test

mann-whitney u test is the nonparametric alternative to...

usually means poorly performing students were more likely to get the question right (usually by guessing)

negative point-biserial

confidence interval

non-inferiority studies generally use hypothesis testing/confidence interval

tests for ordinal data

non-parametric tests test for..

- spearman's rho for ordinal or non-normally distributed continuous data (nonparametric alternative to pearson's correlation)

nonparametric test for correlation

- rank ordered lists - groups are compared by analyzing their ranks rather than the raw data

nonparametric tests convert raw data into...

- 2 groups: wilcox signed rank (nonparametric alternative to paired t test) - > 2 groups: friedman 2-way ANOVA (nonparametric alternative to repeated measures ANOVA)

nonparametric tests for dependent study groups

- 1 group: sign test (nonparametric alternative to one sample t test) - 2 groups: mann-whitney U (nonparametric alternative to two sample/students t test) - > 2 groups: kruskal wallis (nonparametric alternative to ANOVA)

nonparametric tests for independent study groups

the variances in the F test numerator and denominator are equal

null hypohtesis of F test

Ho = u2 - u1 is greater than or equal to ∆ experimental treatment is worse than the standard treatment

null hypothesis for hypothesis testing for non-inferiority

Ho = u1 - u2 = 0 Ho = u1 = u2 (no difference between treatments)

null hypothesis for testing for differences

Ho = u1 - u2 is less than or equal to 0 (testing if group 1's mean is greater than group 2's)

null hypothesis for testing for superiority

Ho = | u1 - u2 | is greater or equal to ∆ a difference large enough to matter exists

null hypothesis of testing for equivalence using hypothesis testing

lower

p values in parametric tests are typically lower/higher compared to non-parametric on the same data set

tests for continuous data

parametric tests test for what type of data

parametric tests

parametric/non-parametric tests are more likely to reject null hypothesis if appropriate

parametric

parametric/non-parametric tests are statistically powerful

non-parametric

parametric/non-parametric tests are valid in a broader range of situations, (more robust)

non-parametric

parametric/non-parametric tests have fewer required conditions of validity

determines correlation between whether examines get a specific question correct (binary variable y/n) and the examinee's score on the entire exam (a numerical variable)

point biserial correlation used to evaluate test questions ...

indicates that students who answer the question correctly tend to score high on the exam as a whole, whereas students missing the question tend to score low generally

positive point-biserial

- it's possible that X does in fact cause Y - it's also possible that Y causes X - an unknown factor, z, influences x and y proportionately - several other factors influence x and y proportionately - the correlation discovered in the sample is just a coincidence

possible explanations if you are presented with two variables that correlated with each other

tukeys

post hoc comparison that has a moderate threshold to declare significance (commonly used)

scheffes

post hoc comparison that has the highest threshold to declare significance (hardest to show significance)

- scheffes procedure - tukeys HSD - dunnet's

post hoc comparisons

scheffes procedure

post hoc comparisons used to both pairwise and non-pairwise (1v2, 1v3, 2v3, 1v[(2+3)/2], etc

tukeys

post hoc only used to pairwise comparisons, 1v2, 1v3, 2v3

dunnet's

post hoc that has very low threshold to declare significance (easiest to show significance)

dunnet's

post hoc used when treatment means are compared to control only (not to each other)

- chi-square test can be used to test the equality of two proportions - chi square test can be used to test whether one of the variables is associated with the other

research question of chi square test

research questions are framed as directional, but tested using a non-directional test

research questions are often framed as non-directional/directional, but tested using a non-directional/directional test

median of one independent group to a national average/cut point (nonparametric alternative to one sample t test)

sign test compares...

one sample t test

sign test is the nonparametric alternative to...

slide 14-16

slide 14-16 of basic statistical concepts equivalence and non-inferiority studies

slide 9-10

slide 9-10 of basic statistical concepts equivalence and non-inferiority studies

- two ordinal variables - one ordinal variable and one continuous variable - two continuous variables (but one or more which is not normally distributed)

spearman's rho is used to assess correlation between...

- d' agnostino-pearson omnibus K2 normality test - shapiro wilk test - komogorov-smirnov test - darling-anderson test

statistical normality tests

df = N - 2

test statistic for p = 0, how do you calculate the degree of freedom

non-directional tests

test that looks in both positive and negative directions

directional test

test that looks only in one direction

tests whether the difference between two quantities is 0

testing for differences

- hypothesis testing - estimation

testing for differences can be tested using...

test to determine if two treatments are equal/equivalent confidence intervals are used to determine if key parameters (drug peak concentrations, AUC, etc) are equivalent

testing for equivalence

- 2 one-sided t-tests or - confidence interval using twice the stated alpha level (5% alpha with a 90% confidence interval)

testing for equivalence using a hypothesis test is performed using...

tests whether two quantities are within an acceptable range of each other (shown as ∆)

testing for equivalence using hypothesis testing

- t test for two groups - bartlett's for three or more groups

testing for similar variance for < 20 per group or if one or more distribution is skewed or has heavy tails considered

levine's test

testing for similar variance for > 20 subjects per group with a normal distribution

tests whether one quantity is greater than or less than the other

testing for superiority

- one tailed test - confidence interval

testing for superiority is performed using what type of tests

- independent - the variables follow a normal distribution - there are similar sample variances between the groups

the student's t test assumes the 2 groups are... (parametric conditions so that the results reliable)

- joint distribution that is normally distributed (bivariate normal distribution) - covariation needs to be linear [can't use correlation for curvilinear data (J shaped)]

two variables, x and y vary together is an assumption of correlation, what does it mean

point biserial correlation

used to correlate a numerical and binary nominal variable

tests if a novel intervention is no worse than a standard treatment, beyond a specified margin (non-inferiority margin/threshold is symbolized as ∆)

what are tests for non-inferiority

the columns and rows are independent so that you can calculate the expected frequencies

what assumption must you make in a chi square test

t distribution only used when testing p = 0

what distribution can test the null hypothesis p (rho) = 0 for correlation

describes the strength of relationship and ranges 1 to +1

what is r in regards to correlation scatterplot

use fisher's test if any expected cell count is < 2 or if more than 20% of cells have expected cell counts < 5

what test should you use if the expected frequencies are small (so you can't use chi square test)

little to no correlation exists

when r is near 0 there is/is not correlation

when expected frequencies within A-D are small

when should the chi square test not be used

when parametric test assumptions are not met: normal distribution (and unable to transform the data), similar variance

when should you use non-parametric tests

scheffes

which post hoc comparison is mostly limited to non-pairwise data

ANOVA

which test allows comparisons of 3 or more means while holding alpha to 5%

90% allows 5% error at both the lower bound (subtherapeutic) and the upper bound (supratherapeutic) end of the confidence interval since a drug cannot be both

why a 90% confidence interval and not 95% for testing equivalence?

due to +1 and -1 bounds of r causing skewing of distribution

why can't you use t test when null hypothesis p < or > 0

reciprocal of 80% is 125% (1/0.8 = 1.25)

why have AUC at 0.8 and 1.25 and not a symmetrical 0.8 and 1.2

interested in relationship, not if a difference exists

why not just use a t test to correlate a numerical binary nominal variable instead of the biserial correlation

it increases the risk of a type 1 statistical error - 3 tests needed to compare 1v2, 1v3, 2v3 - ~15% alpha

why should you not use multiple t tests to compare 3 or more groups

medians among two dependent groups

wilcoxon signed rank compares...

paired t test

wilcoxon signed rank is the nonparametric alternative to...

correlation

xy scatterplots graph what relationships