Heterogeneity (Feddock)
2. All randomized controlled trials examining the effect of methotrexate on mortality in leukemia - What factors would hamper the interpretation of this sytematic review?
(Differences in interventions we may find and patients we may find that may inhibit combining all of these trials) *May be good to combine depending on the studies, but could be challenging*
3. All randomized controlled trials examining the effects of methotrexate vs. placebo (no treatment) on mortality in childhood acute lymphoblastic leukemia - What factors would hamper the interpretation of this sytematic review?
(the things crossed out are clearly defined in the proposal) *Probably much more reasonable to consider in a systematic review where we combine all the study data to get a summary efffect in terms of a generalized goal*
*In what two ways will we see heterogeneity on a forest plot?*
*1. Non-overlap of 95% CI* *2. Disparity in the point estimates*
*+/- Heterogeneity*
*Cochran chi-square (Q-test)* - Statistical significance = definite heterogeneity (doesn't say how much, just if heterogeneity exists or not among study results)
1. All cancer studies using methotrexate to generate an overall effect on morbidity and mortality - What factors would hamper the interpretation of this sytematic review?
*Doesn't make any sense to combine*
*What are one of the most common graphical ways to determine heterogeneity when looking at a meta-analysis?*
*Forest Plot!*
*% variability in effect due to heterogeneity*
*I^2 statistic* *- tells you a degree of heterogenity*
*If you have determined that a meta-analysis has statistical heterogeneity, what do you need to look for next?*
*If those studies have clinical heterogeneity*
*In general, what is heterogeneity used for?*
*To see if we should combine results of studies based on similarity of studies* * - Patients/ Interventions / Outcomes / Study Designs*
What are some reasons for heterogeneity?
1. Chance 2. Differences between studies
Why might there be a difference in studies?
1. Patients 2. Interventions 3. Outcomes 4. Study designs
What are the 4 factors we need to consider in order to assess whether the interpretation of a systematic review would be hampered?
1. Patients 2. Interventions 3. Outcomes 4. Study Methods
Forest Plot: lines
95% confidence interval
Statistical Tests for Heterogeneity
Cochran chi-square (Q-test) I^2 statistic
Forest Plot: Heterogeneity - no
Confidence intervals don't overlap, indicating inconsistent results
Forest Plot: Heterogeneity - yes
Great disparity among point estimates (95% CI overlap, Summary statistic - doesn't overlap with point estimate of 4 studies Two seem to favor control, two seem to favor treatment, but sig overlap, = heterogeneity)
Any kind of variability among studies
Hererogeneity (Patients Interventions Outcomes Study Designs)
Forest Plot: box
Individual Study
Levels of heterogeneity: I^2 = 25-20
Minimal heterogeneity
Levels of heterogeneity: I^2 = 50-75
Moderate heterogeneity
Levels of heterogeneity: I^2 < 25%
No heterogeneity
Forest Plot: size of box
Size of study
Levels of heterogeneity: I^2 >/= 75
Substantial heterogeneity
Forest Plot: diamond
Summary statistic of studies - pooled estimate of effects
Forest Plot: 95% CI
ends of diamond
Forest Plot: mean
middle of diamond
We want to combine study results based on _______ of studies
similarity - Range of characteristics is a matter of judgement