Research Final Test (Study Guide One)

What information is typically included in the discussion section of a systematic review article?

Discussion Includes • Summary • Clinical Recommendations • Limitations

What are the drawbacks of the traditional narrative literature review?

Drawbacks • Risk and Bias • Difficult to reproduce • Lack of analysis • Low level of evidence

Describe considerations and questions to ask when appraising systematic reviews.

1. General bias: Method of selection, Quality of studies (Find in the Methods Section) a. Were all relevant studies included (i.e. comprehensive search?) b. Were selected articles appraised and data extracted by two or more independent reviewers? c. Were the strength and quality of the primary studies assessed? d. Do the available studies include researchers other than the original developers of the intervention/assessment? e. Are the studies included in the review of the highest level of evidence for a single study? 2. Replication of primary studies (Find In Results Section) a. Is there an adequate number of studies available from which to draw a conclusion? 3. Publication bias: Studies with positive results 4 times more likely to be published. a. Did the reviewers make efforts to obtain unpublished research (e.g. theses, dissertations, research reports from clinical trial registries) b. Was potential for publication bias assessed? 4. Heterogeneity: How similar are the synthesized studies? (Find in Discussion or Results Section) a. Was sufficient detail provided about the primary studies, including descriptions of the patients, interventions and outcomes? b. Are the studies so different that it is difficult to draw conclusions, or does the reviewer include sub analyses to address differences in the studies? c. Was combining the studies logical, or were they too different?

List the steps in conducting a systematic review.

1. Research question 2. Methods Defined 3. Data collection 4. Analysis of Results 5. Writing the Report

Name different types of statistics used in a meta-analysis and when they are used.

1. Scale/Continuous Variables/ aka: Interval or ratio data : Cohen's d 2. Dichotomous Variables (yes or no choices): Odds ratio/ Hazards ratio 3. Correlations (relationship of variables)/Regression (prediction of variables- how well can one set of variables predict another set): r or R2

What information is typically included in the methods section of a systematic review article?

Methods (included in writing the report) • Inclusion/Exclusion criteria • Details of search methods • Who collected data • Information extracted

What information is typically included in the results section of a systematic review article?

Results (included in writing the report) • Number of studies included • Summary table for included studies • Synthesis of information

Compare and contrast a traditional narrative review with a systematic review.

Similarities: Comparison, Summary of the literature Differences: Systematic Reviews are much more rigorous, Meta-analysis is a type of systematic review • Systematic Reviews Answer specific questions, is comprehensive, unbiased synthesis, focus on research reports, Orderly, systematic process

What are the strengths of the traditional narrative literature review?

Strengths • Describe an issue, underlying concepts, theories • Present general knowledge on a topic • Provide historical overview • Identify gaps • Establish concensus or debate • Identify need for further stud • Justify need for further study

What are the steps in the metaaggregation process?

The Meta-synthesis Process: Thematic Synthesis or Metaaggregation 1. Themes identified from each individual study 2. Themes synthesized based on corroborating concepts across studies a. Findings categorized based on similarities of meaning b. Aggregated into a set of statements 3. Attempt to find novel interpretation in the synthesis

What is a meta-synthesis?

• A way of combining qualitative information • The non-statistical technique used to integrate, evaluate, and interpret the findings of multiple qualitative research studies.

What is involved in data collection for a systematic review?

• Comprehensive search • Select relevant studies meeting inclusion criteria • Appraise quality and validity of included studies • Extract relevant data

List the types of data that are extracted from the studies included in a systematic review and give examples.

• Criteria Details o Author(s) o Year • Descriptive Data o Setting/context o Design o Participant characteristics o Details of intervention o Type of analysis • Outcome Data o Results o Findings o Methods used

How are the methods of a systematic review defined?

• Determine inclusion/exclusion criteria (what studies will we include or exclude) • Develop comprehensive search strategy, to include published and unpublished studies

Describe considerations and questions to ask when considering whether to utilize findings from a systematic review in practice.

• Has a clear and accurate summary of findings been provided? • Are the recommendations, both for practice and future research, supported by the data presented? • Do the characteristics of the research participants match the characteristics of my client and his/her situation? If not, can they apply? Are the studies that match too weak to apply? • Which research participants responded to which aspects of the interventions? • What practice changes would you recommend based on this evidence?

What information is reported in the results section of a meta-analysis?

• Measure or estimate of effect • Confidence interval • Heterogeneity: Quantification of the differences between included studies and statistical impact (would rather it be by chance) Want this to be large and the p to be > than .05.

What is a meta-analysis?

• Meta-Analysis: o Type of systematic review o Results of similar quantitative studies are pooled - Statistical methods used to produce overall summary effect size • Every meta-analysis is a systematic review but not every systematic review is a meta-analysis o Quantitative piece

What is a traditional narrative literature review?

• Overview of knowledge, current and historical, derived from a body of literature • Research reports, conceptual or theoretical

What are the characteristics of clinical practice guidelines?

• Provide recommendations for practitioners to address specific clinical situations • Carry more weight than single RCT • Provides level of evidence associated with recommendations and applicability and generalizability of recommendations • Usually developed by organizations • May or may not have a strong evidence base • Often more general than a systematic review

What are the clinical purposes of clinical practice guidelines?

• Quality improvement • Reimbursement decisions • Education of clients and families

List considerations when appraising clinical practice guidelines.

• Reputation and resources of the organization that created them • Author conflicts of interest • Time frame of development and publication • Specific population and setting • Transparency of process used for development and evaluation • Rigor of review process

When can a meta-analysis be used?

• Studies have similar populations • Studies have similar objectives and aims • Studies have similar methods

List characteristics of a systematic review.

• Systematic process of extracting, synthesizing, and combining data • Secondary research (not finding new knowledge from doing a study. Instead it's finding out what other studies say). • Rigorous process (highest level of evidence)

What are the advantages of meta-analysis over other types of studies?

• Takes into account quality of included studies • Increased statistical power • Smaller confidence intervals, i.e., more precise • Estimation of overall effect size • Systematic comparison of differences between studies

How is a meta-analysis done?

• Transform findings into common metric o Calculate effect sizes individually o Pool effect sizes

Name resources for finding systematic reviews.

• Use "systematic review" to limit database search • Evidence databases: PEDro, OTseeker, Evidence Map • Cochrane Collaboration (any type of medical literature, systematic reviews similar to RCT, and open to the public) • Search "systematic reviews" on the AJOT website

List resources for finding clinical practice guidelines.

• Use as a limiter under "article type" or "publication type" • National Guideline Clearinghouse at www.guidelines.gov • AOTA

Describe a forest plot.

• Vertical line = reference point (anything on the left side control group did better than experimental group and right side vis versa) (The bigger squares = higher quality studies) (Horizontal lines are the confidence intervals for each study) (The shorter the horizontal lines are, the better) (Diamond is the overall effect size and the width of it determines its confidence interval). • If there is a square on the right that means the outcome was better