Informatics: Meta-Analysis

good empirical evidence that flaws in design, conduct, and analysis of trials may lead to ___

bias

leads to underestimation or overestimation of the true treatment effect

bias

when assessing internal validity you must look for ___

bias

broad scope or narrow scope: advantage is increased external validity

broad scope

broad scope or narrow scope: disadvantage is increased hetereogeneity among trials

broad scope

major advantage of I2 statistic

can look at different subgroups

type of heterogeneity: variability in participants, interventions, or outcomes being studied

clinical heterogeneity

statistical heterogeneity is a consequence of ___ and/or ___ diversity

clinical, methodological

1 disadvantage of a broad scope research question

increase heterogeneity among trials (making interpretation difficult)

advantage of I2: 2. interpretation is __ 3. can be accompanied by an ___ interval 4. simple to ___

intuitive, uncertainty, calculate

most robust type of systematic review

meta-analysis

quantitative synthesis using statistical techniques to combine results of two or more trials with the intent to produce a summary of findings

meta-analysis

uses quantitative methods to statistically combine the results from different studies

meta-analysis

meta-analysis is a ___ review using statistical techniques to combine results of primary studies

quantitative

for Cochran's Q statistic, for a p-value<0.1 ______ null hypothesis and assume that ____ is present among the studies

reject, heterogeneity

type of bias: differences between reported and unreported findings

reporting bias

differences in ___ of bias can help to explain variation in the results of the studies included in systematic review

risks

overall, studies with large __ __ and small __ contribute the most to the pooled estimate

sample size, variability

type of bias: differences between baseline characteristics of the groups

selection bias

type of heterogeneity: variability in treatment effects being evaluated in the different trials

statistical heterogeneity

stage 1 of performing meta analysis involves calculating ___ intervention effect estimate for each study

summary

qualitative assembly, critical appraisal, and synthesis of the results of primary literature addressing a specific topic or clinical problem, explicit methods limit bias and random error

systematic review

____ ___ assemble, critically appraise, and synthesize results of primary literature

systematic reviews

systematic review are ___ literature sources

tertiary (they compile information from primary literature)

often times data is ___ so results can be compared in a common scale (same units) to simplify meta analysis and make meta analysis possible

transformed

T/F: Cochran's Q statistic doesn't tell you what percent of the results can be explained by heterogeneity, just gives a yes or no

true

T/F: an advantage of meta-analysis is that is presents individual trial information in a digestible format

true

T/F: it is also important to search unpublished data aka conference abstracts

true

T/F: it is important to quantify the extent of heterogeneity among a collection of studies

true

T/F: it is usually impossible to ascertain to what degree biases have affected study outcomes

true

T/F: meta-analysis helps to arbitrate between studies that result in different conclusions

true

T/F: meta-analysis provides preliminary data for hypothesis to be tested in definitive large trial

true

T/F: meta-analysis provides the most reliable treatment recommendation in absence of definitive large trial

true

T/F: studies with more individuals will be weighted higher in the pooled estimate than studies will smaller number of individuals

true

T/F: subgroups should be specified before results are ascertained

true

T/F: when evaluating a meta-analysis the authors should specify restrictions like what years were considered, publication status, and language

true

validity assessment should be performed by __ ___ reviewers in masked conditions, validity includes external and internal

two independent

when evaluating a meta-analysis for the search strategy, it should be performed by ___ ____ reviews

two independent

studies with large ___ in results will contribute less to the pooled estimate than studies will less ___

variability, variability

type of bias: differences between groups in withdrawal from study

attrition bias

when calculating the I2 statistic, the answer tells you what percent of results are related to ____

heterogeneity

one major disadvantage of Cochran's Q statistic

poor power in situation of few studies

___% is the best results for the I2 statistic

0 (whereas 100% means 100% heterogeneity)

4 disadvantages of meta-analysis:

1. biases and flaws of individual studies (are incorporated 2. new sources of bias ay be introduced 3. heterogeneity between studies (may not be addressed adequately) 4. clinically relevant secondary outcomes and adverse events may not be fully addressed

4 factors that should be included in research question when evaluating a meta-analysis

1. clearly defined 2. well formulated 3. specifies PICO (participants, interventions, comparisons, outcomes) 4. consider broad vs. narrow scope (research question)

2 advantages of a broad scope research question

1. comprehensive summary of available evidence 2. increased external validity

2 advantages of a narrow scope research question

1. easier review to conduct 2. easier to read and interpret results

3 disadvantages of a narrow scope research question

1. evidence may be sparse 2. limits external validity 3. may increase bias

5 ways to minimize publication bias

1. search multiple databases (***) 2. include abstracts 3. search reference lists 4. include unpublished data 5. search online trial registries

in general, anything above ___% is a lot of heterogeneity

50

3 databases that MUST be searched at minimum when evaluating a meta-analysis

CENTRAL, MEDLINE, EMBASE

____ __ statistic uses a more conservative p-value, uses a cut off point of 0.1

Cochran's Q

this statistic is used to quantify heterogeneity and can be calculated from Cochran's

I2

eligibility criteria for a meta-analysis should include:

PICOS (participants, interventions, comparisons, outcomes, study design)

type of bias: differences between groups in how outcomes are determined

detection bias

for Cochran's Q statistic, for a p-value>0.1 _____ null hypothesis

fail to reject

T/F: the results of a meta-analysis should, in most cases, change practice

false (should NOT)

___ ___ is used to assess the potential for publication bias, using this does NOT help minimize publication bias

funnel plot

advantage of I2: 1. focuses attention on the effect of any ___ on the meta-analsys

heterogeneity

interpretive problems depend on the degree of ____

heterogeneity

what is one of the most troublesome aspects of systematic review?

heterogeneity

type of heterogeneity: variability in trial design and validity

methodological heterogeneity

broad scope or narrow scope: advantage is that its easier to read and interpret results

narrow scope

broad scope or narrow scope: disadvantage is that it limits external validity

narrow scope

if there is significant heterogeneity, should you still do a meta-analysis?

no

when looking at a forest plot, when the 95% confidence interval crosses the line at 1, is there a difference between treatment groups?

no

advantage of I2: 5. does not depend on __ of studies 6. similar interpretations regardless of ___

number, outcome

research question is usually phrased as an ___

objective

consequence of publication bias:

overestimation of treatment effect

type of bias: differences between groups in treatment or in exposure to other factors

performance bias

stage 2 of performing meta analysis involves calculating ___ intervention effect estimate as a ___ average of the intervention effects estimated in the individual studies

pooled, weighted

an advantage of meta-analysis is that it increases ___ to identify more subtle treatment effects

power

meta-analysis is considered a ___ literature source

primary

__ bias: the publication or non-publication of research findings, depending on the nature and direction of the results

publication

___ bias is the main type to be assessed in meta-analysis

publication

____ systematic reviews contain a summary of primary studies that are NOT statistically combined

qualitative

systematic reviews are done by ___ assembly while meta-analysis are completed by __ synthesis

qualitative, quantitative