13. Systematic Review

Identify Literature, Your development plan should include:

1. Criteria for the literature search ___a. Keywords ___b. Databases ___c. Dates to include (how far back do you go?) 2. Inclusion and exclusion criteria 3. Types of studies you want to include 4. A timeline for your project

Sometimes scoping reviews are also used to decide whether or not it would be usefulto conduct a systematic review:

1. Identify the questions or objectives 2. Define inclusion criteria and exclusion criteria 3. Search for evidence 4. Select evidence 5. Extract evidence 6. Chart evidence 7. Present results

Publication bias means that... A. Journals often prefer to publish results that show significant treatment effects B. Authors prefer to submit results of studies that do not show significant effects C. Published studies with negative results are not read by clinicians D. Clinicians do not believe published results that show significant effects

A

Scoping reviews are different from systematic reviews because they include all forms of input, including consultation of stakeholders and observational studies. A. True B. False

A

What type of study is considered the highest evidence level for a randomized controlled trial or observational designs? A. Randomized control trial B. Observational studies

A

Systematic Review

A document often written by a panel that provides a comprehensive review of all relevant studies on a particular clinical or health-related topic/question.

Class III

All other controlled trials (including well-defined natural history controls or patients serving as own controls) in a representative population, where outcome assessment is independent of patient treatment.

Scoping reviews are similar to systematic reviews because they include methodological assessment of reviewed studies A. True B. False

B

Systematic reviews or meta-analyses serve all of the following purposes except: A. They present methodological reviews of studies to assess their validity B. The present conclusive evidence that should direct practice C. They can address the effectiveness of different interventions D. They can identify risk factors that can be included in general guidelines

B

All of the following criteria are usually specified as part of a search strategy for inclusion in a systematic review except: A. Listing in particular databases. B. Language of publication C. Publication in a particular journal. D. Years of publication.

C

The PRISMA diagram will include all of the following information except A. The number of articles obtained in an initial search B. The reasons for omitting articles. C. The number of journals referenced D. The number of duplicates identified in the search.

C

Which of the following study designs may be inappropriate to include in a systematic review of the effectiveness of pain medication for individuals with arthritis? A. Randomized controlled trials B. Pragmatic clinical trials C. Qualitative design D. Cohort study

C

Appraises quality of studies

Class I Class II Class III Class IV

Step 3: Study appraisal

Critical appraisal of published works can be approached from two perspectives ● The evaluation may focus on whether a study should be included in a systematic review or meta-analysis. _____○ This type of assessment is generally concerned with design and outcome data, producing a quantitative score for methodological quality. ● For judging the clinical applicability of a research report as a source of evidence to determine ifthe quality and substance ofthe paper is valid, meaningful, and relevant to practice.

Systematic

Done or acting according to a fixed plan or system: methodical

Class IV

Evidence from uncontrolled studies, case series, case reports, or expert opinion.

Scoping Review

Exploratory, and they typically address a broad question. _____○ Researchers conduct them to assess the extent ofthe available evidence, to organise it into groups and to highlight gaps. _________■ Does not need to be analyzed, just needs to be information organized to discuss a topic.

Prisma Flow Diagram

Identification Screening Eligibility Included

What are the most common types of systematic reviews?

Intervention Prognostic Measurement

Practice guidelines:

Okay to have a lot of level II studies

The Systematic Review Process

Planning Conducting Reporting and dissemination

Applying systematic review to practice

Practice Standards Practice Guidelines Practice Options

Class II

Prospective matched group cohort study in a representative population with masked outcome assessment that meets a-d above OR a randomized, controlled clinicaltrial in a representative population that lacks one criterion a-d.

Class I

Prospective, randomized, controlled clinicaltrial with masked outcome assessment in a representative population. ● Primary outcome(s) is/are clearly defined. ● Exclusion/inclusion criteria are clearly defined. ● Adequate accounting for dropouts and crossovers with numbers sufficiently low to have minimal potential for bias. ● Relevant baseline characteristics are presented and substantially equivalent among treatment groups or there is appropriate statistical adjustment for differences.

PICOT Format

The best questions to guide a systematic review of the evidence and the development of guidelines are ones that have been developed following ○ Population ○ Intervention ○ Comparator ○ Outcome(s) ○ Time point

Intervention

To assess the evidence about the effects of a healthcare intervention.

Prognostic

To assess the evidence of models or predictors of patient outcomes.

Measurement

To assess the properties of health-status instruments or tools.

Practice standards:

We need a few class I level of evidence to include into the research

Review

critical appraisal of a book, play or other work.

Practice options:

no level I or level II, but some evidence to suggest XYZ, they're options, but need more research

Systematic review is created after...

reviewing and combining all the information from both published and unpublished studies (focusing on clinical trials of similar treatments) and then summarizing the findings.

Describe the types of information that should be included in a study's methods and results sections.

○ Inclusion and exclusion criteria of participants ○ How we sampled the population ○ How we recruited the population ○ How we divided them into groups ○ If the number of participants was sufficient enough for the study ○ Define what will be the outcome → operational definition ofthe dependent variable ○ Whether the participants or researchers were blinded or not

Current guidelines

● "To assess the effects of [intervention or comparison] for [health problem] in [types of people, disease or problem, and setting if specified]." ● Several criteria/frameworks proposed to help guide question development

Step 1: The best and most useful questions are ones that...

● Are specific, not too broad and not too narrow ● Can be answered with evidence (rather than ask for a value judgment) ● Are not questions where all stakeholders agree on what ought to be done ● Instead concern areas of controversy, or issues where we plainly have no idea of what the best course of action is, but there might be literature that might help decide.

What makes a review systematic?

● Based on a clearly formulated question ● Identifies relevant studies ● Appraises quality of studies ● Summarizes evidence by use of explicit methodology ● Comments based on evidence gathered

Data Extraction/Quality Appraisal

● Be clear what information you want from the studies: ___○ Study details ___○ Data for your analysis ● Information will need to be collected relating to: ___○ Methodology ___○ Population ___○ Interventions being compared ___○ Outcomes evaluated

Evaluating Systematic Reviews

● Can you find an important question that the review addressed? ● Was a thorough search done of the appropriate databases, and were other potentially important sources explored? ● Was methodological quality assessed and the trials weighted accordingly? ● How sensitive are the results to the way the review has been done? ● Have the numerical results been interpreted with common sense and due regard to the broader aspects ofthe problems? Applies to meta-analyses

Application to Practice

● Does the research apply to my patient population? ● Would this practice change improve process of care? Patient outcomes? ● Is there support for this practice change?

Advantages of Systematic Review

● Explicit methods limit bias in identifying and rejecting studies ● Conclusions are more reliable and accurate because of methods used ● Large amounts of information can be assimilated quickly ● Delay between research discoveries and implementation may be reduced ● Results of different studies can be formally compared to establish generalizability of findings and consistency of results ● Reasons for heterogeneity (inconsistency in results across studies) can be identified and new hypotheses generated about particular subgroups ● Quantitative systematic reviews (meta-analyses) increase the precision ofthe overall result

Step 4 - Summarizing the Evidence (S)

● Extracting data from trials ● Combining data - Meta-analysis ● Does it make sense to combine?

Example PICOT

● For adults with moderate traumatic brain injury (P), ● Is rehabilitation in a nursing home (I) more effective than ● Inpatient Rehabilitation Facility rehabilitation (C), ● In terms offunctional status (O) ● At one year post injury (T)? ● Example: For older adults with knee osteoarthritis, is blood flow restriction with resistance training more effective than resistance training alone in terms offunctional status one year after program implementation?

Conducting

● Identification of the literature ● Selection of studies ● Appraise the quality of papers ● Abstracting data from papers ● Analysis and interpretation of data

Planning

● Identification of the need for the review ● Preparation of a proposal for a systematic review ● Development of a review protocol

Meta-analyses vs. Systematic Review vs. Scoping review

● Meta-analyses are a more rigorous form of review, we can rely most on their evidence to support our practice ___○ For meta-analyses there will be quantitative data in the end ● Systematic review falls between scoping review and meta-analyses for rigor ● Scoping review is less rigorous

Why do we need systematic reviews?

● Minimise the impact of bias/errors ● Can help to end confusion ● Highlight where there is not sufficient evidence ● Combining findings from different studies can highlight new findings ● Can mitigate the need for further trials

Publication and Reporting Biases

● Positive Results Bias ● Grey Literature Bias ● Time-Lag Bias ● Language and Country Bias ● Multiple Publication Bias ● Selective Citation Bias ● Database Indexing Bias ● Selective Outcome Reporting

Limitations of systematic reviews specific to health promotion

● Results may still be inconclusive ● There may be no trials/evidence ● The trials may be of poor quality ● The intervention may be too complex to be tested by a trial ● Practice does not change just because you have the evidence of effect/effectiveness

Systematic vs. literature review:

● Systematic review - more objective ____○ There will be a conclusion ____○ Written by a panel ● Literature review - more subjective ____○ There will simply be a review ____○ Can be written by one person

Systematic Review In-Class Example

● The coronavirus disease 2019 (COVID-19) pandemic has affected hundreds of thousands of people. Data on symptoms and prognosis in children are rare. ____○ The coronavirus disease 2019 has occurred in children, but they seemed to have a milder disease course and better prognosis than adults. Deaths were extremely rare (Ludvigsson, 2020). ● To assess the quality of evidence supporting occupational therapy's role with homeless people. ____○ The literature suggests that occupational therapy has an appropriate role with people experiencing homelessness (Thomas et al., 2011). ● To summarize the best-available intervention evidence for children with disabilities to assist families and therapists choose effective care.

Reporting and dissemination

● The report and recommendations ● Getting evidence into practice

Inclusion and Exclusion Criteria

● Types of participants ● Types of interventions ● Outcome measures ● Types of studies ● Methodological quality criteria

Core questions to evaluate research studies

● What is the study about? ● Methods: design and implementation ● Results: Were the results presented clearly? ○ Were the data analyzed appropriately? ○ How large was the effect? ● Discussion: Is the author's interpretation of results supported by the data? ○ Does the author acknowledge limitations ● Conclusion: are the findings useful? Were the participants in the study sufficiently similar to my patient? Is the approach feasible in my setting?

Step 2 - Identifying Relevant Publications

● Wide search of medical/scientific databases ○ Medline ○ Cochrane Reviews ○ Ovid ● Relevance to focused question PICO ○ Population ○ Prior test ○ Intervention ○ Comparator ○ Outcome