systematic reviews

What is a Systematic Review?

A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review*

Study Quality Scoring

Done when study quality varies and better quality study should get larger weight • Selection of scoring criteria and weights •Jadad scale/score [Oxford quality scoring system] • Implementation in summary measure or stratify results by quality

The Cochrane Collaboration

International non-profit organisation that prepares, maintains, and disseminates systematic up-to-date reviews of health care interventions

Types of Reviews

Non-systematic reviews systematic reviews

Key Elements of a Systematic Review

Structured, systematic process involving several steps... 1. Formulate the question 2. Plan the review 3. Comprehensive search 4. Unbiased selection and abstraction process 5. Critical appraisal of data 6. Synthesis of data (may include meta-analysis) 7. Interpretation of results All steps described explicitly in the review

critical appraisal

The process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision. •Clearly-focused question •The right type of studies included •Identifying all relevant studies •Assessment of quality of studies •Reasonable to combine studies •What were the results •Preciseness of results •Application of results to local population •Consideration of all outcomes •Policy or practice change as a result of evidence

Allocation of concealment

fairly distributing subject to control vs exposure=subverting the randomization process if biased=

systematic reviews

• Meta-analysis

Non-systematic reviews

• Narrative • Literature • Traditional •Advantages: easy to do, short duration, inexpensive •Lack of scientific method • Methods should be as rigorous as for any other research (e.g., retrieval and quality of evidence) •Only look at published research • Publication bias •**Guidelines are often NOT systematic reviews

types of bias

• Selection bias • Allocation bias • Confounding • Blinding (detection bias) • Data collection methods • Withdrawals and drop-outs • Statistical analysis • Intervention integrity • Selective reporting • Other sources of bias

systematic Review Guidelines

•CONSORT: RCTs • CONsolidated Standards Of Reporting Trials •STROBE: Observational research • STrengthening the Reporting of OBservational Studies in Epidemiology •PRISMA: Systematic reviews and meta-analyses • Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Narrative Reviews

•Depend on authors' inclination (bias) •Author gets to pick any criteria starts fromauthors pov, author then finds aricles that support it or not etc build their case w their conclusion in mind •Search any databases •Methods not usually specified •Vote count or narrative summary •Can't replicate review

unpublished Literature

•Not all known published trials are identifiable in MEDLINE (depending on topic) •Only 25% of all medical journals in MEDLINE •Non-English language articles are under-represented in MEDLINE (and developing countries) •Publication bias - tendency for investigators to submit manuscripts and of editors to accept them, based on strength and direction of results •Hand searching of key journals and conference proceedings •Scanning bibliographies/reference lists of primary studies and reviews •Contacting individuals/agencies/academic institutions •Neglecting certain sources may result in reviews being biased

Registration of Systematic Reviews

•Prospero: International prospective register of systematic reviews • Center for Reviews and Dissemination: • Analogous to clinicaltrials.gov for research protocols to try to improve the quality of systematic reviews, authors regiser their reviews on the site so that it can be seen that authors didnt change menthods as review is being done

Advantages of Systematic Reviews

•Reduce bias •Replicability •Resolve controversy between conflicting studies bc there can be conflcting reviews due to diff methologies or diff evidence at the time review was done •Identify gaps in current research •Provide reliable basis for decision making could also include unpublished lit

Limitations of Systematic Reviews

•Results may still be inconclusive •There may be no trials/evidence •The trials may be of poor quality, review can state thier is only porr quality evidence •Practice does not change just because you have the evidence of effect/effectiveness

Summary

•There are well done and poorly done reviews and criteria to determine what is a well-done review •An up-to-date systematic review can be pivotal in the clinical decision making process •Even well-done systematic reviews can become outdated