Heterogeneity (Feddock)

2. All randomized controlled trials examining the effect of methotrexate on mortality in leukemia - What factors would hamper the interpretation of this sytematic review?

(Differences in interventions we may find and patients we may find that may inhibit combining all of these trials) *May be good to combine depending on the studies, but could be challenging*

3. All randomized controlled trials examining the effects of methotrexate vs. placebo (no treatment) on mortality in childhood acute lymphoblastic leukemia - What factors would hamper the interpretation of this sytematic review?

(the things crossed out are clearly defined in the proposal) *Probably much more reasonable to consider in a systematic review where we combine all the study data to get a summary efffect in terms of a generalized goal*

*In what two ways will we see heterogeneity on a forest plot?*

*1. Non-overlap of 95% CI* *2. Disparity in the point estimates*

*+/- Heterogeneity*

*Cochran chi-square (Q-test)* - Statistical significance = definite heterogeneity (doesn't say how much, just if heterogeneity exists or not among study results)

1. All cancer studies using methotrexate to generate an overall effect on morbidity and mortality - What factors would hamper the interpretation of this sytematic review?

*Doesn't make any sense to combine*

*What are one of the most common graphical ways to determine heterogeneity when looking at a meta-analysis?*

*Forest Plot!*

*% variability in effect due to heterogeneity*

*I^2 statistic* *- tells you a degree of heterogenity*

*If you have determined that a meta-analysis has statistical heterogeneity, what do you need to look for next?*

*If those studies have clinical heterogeneity*

*In general, what is heterogeneity used for?*

*To see if we should combine results of studies based on similarity of studies* * - Patients/ Interventions / Outcomes / Study Designs*

What are some reasons for heterogeneity?

1. Chance 2. Differences between studies

Why might there be a difference in studies?

1. Patients 2. Interventions 3. Outcomes 4. Study designs

What are the 4 factors we need to consider in order to assess whether the interpretation of a systematic review would be hampered?

1. Patients 2. Interventions 3. Outcomes 4. Study Methods

Forest Plot: lines

95% confidence interval

Statistical Tests for Heterogeneity

Cochran chi-square (Q-test) I^2 statistic

Forest Plot: Heterogeneity - no

Confidence intervals don't overlap, indicating inconsistent results

Forest Plot: Heterogeneity - yes

Great disparity among point estimates (95% CI overlap, Summary statistic - doesn't overlap with point estimate of 4 studies Two seem to favor control, two seem to favor treatment, but sig overlap, = heterogeneity)

Any kind of variability among studies

Hererogeneity (Patients Interventions Outcomes Study Designs)

Forest Plot: box

Individual Study

Levels of heterogeneity: I^2 = 25-20

Minimal heterogeneity

Levels of heterogeneity: I^2 = 50-75

Moderate heterogeneity

Levels of heterogeneity: I^2 < 25%

No heterogeneity

Forest Plot: size of box

Size of study

Levels of heterogeneity: I^2 >/= 75

Substantial heterogeneity

Forest Plot: diamond

Summary statistic of studies - pooled estimate of effects

Forest Plot: 95% CI

ends of diamond

Forest Plot: mean

middle of diamond

We want to combine study results based on _______ of studies

similarity - Range of characteristics is a matter of judgement