Crit Quiz 1 (weeks 1-5)

Are observational study designs studies of causality or studies of association?

Association

What should happen if clinical heterogeneity is too high?

A meta-analysis should not be done.

If a meta-analysis is not conducted, what type of summary should a systematic review include?

A qualitative summary of included studies.

What is the "sample" of a study?

A reference group to estimate population characteristics.

CPGs may be most helpful in answering clinical questions about what?

About populations of patients

In a forest plot: what does the x-axis represent?

An odds ratio or a difference of means

What type of studies quantify the association between a predictor (independent variable) and an outcome variable (dependent variable)?

Analytic

Define: variables

Anything that has a quantity or quality that varies. "Characteristics that can vary"

What is one established tool for appraising CPGs?

Appraisal of Guidelines Research and Evaluation II (AGREE II)

Describe the process of clinical heterogeneity assessment.

Authors conduct an analysis of the similarities of the study population, interventions, and outcome measures among the studies included in a meta-analysis.

Describe the criteria that the GRADE System uses to decide if a study will have a strong or a weak recommendation, based on the following factors: Balance between benefits and risks, values and preferences, costs.

Balance between benefits and risks: -Strong recommendation: Large difference between desirable and undesirable effects. -Weak recommendation: Small difference between desirable and undesirable effects. Values and preferences: -Strong recommendation: Low variability or uncertainty in clinician/patient preferences. -Weak recommendation: High variability or uncertainty in clinician/patient preferences. Costs (Resource allocation): -Strong recommendation: Low costs/resources demand. -Weak recommendation: High cost/resources demand.

In an evidence base statement, which parts are in bold and which parts are italicized?

Bold: key action statement Italicized: strength of the recommendation

When interpreting quantitative results, if an outcome of interest is continuous, what is commonly compared between treatment and control groups?

the mean difference or effect size (weighted mean difference)

What are some factors that may impact patient preference?

-Absolute benefits -Adverse effects -Cost of procedure, drug, intervention -Frequency/duration of treatment -Religious beliefs, cultural beliefs, desire for intervention (less tangible)

What are examples of common statistical tests used to asses clinical heterogeneity?

-Between study variance: t² -Cochran's Q: X² -Index of variability: I²

In a forest plot: what do the boxes represent? What do the lines attached to the boxes represent?

-Boxes: each study's standardized mean difference. -Lines attached to boxes: the 95% confidence interval.

CPGs can lack the detail that physical therapy clinicians are seeking. What factors can this lack of detail be attributed to?

-CPG's are designed to summarize large body of evidence and give general recommendations, not specific treatment prescriptions. -A small amount of research available for common clinical questions. The body of evidence is growing but gaps still exist between evidence-based information and our clinical questions. -CPGs are usually created by large groups and it is difficult for these groups from related disciplines to agree on specific recommendations.

Define: CPGs (Clinical Practice Guidelines)

-CPG's are systematically developed statements designed to facilitate evidence-based decision-making for the management of specific health conditions. -CPGs incorporate evidence from research, clinical expertise, and ideally patient perspectives. -Scope: Diagnosis, Prognosis, Intervention

What is the main difference between case-control studies and cohort/ cross-sectional studies?

-Case control studies: outcomes come before the exposure. -Cohort and cross-sectional studies: exposure comes before the outcome.

What are different ways that a study can have heterogeneity?

-Clinical diversity: Different metrics/outcomes, participant characteristics, settings. -Methodological diversity (different study designs). -Statistical diversity

What are the functions of the research question?

-Focuses the study -Determines the methodology -Guides all stages of inquiry, analysis, and reporting

The GRADE System describes the quality of evidence used in a CPG. What is the difference between each of the 4 categories of quality used in this system?

-High Quality: Further research is very unlikely to change confidence in the estimate of effect. -Moderate Quality: Further research is likely to have an important impact on confidence in the estimate of effect and may change the estimate. -Low Quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. -Very Low Quality: Any estimate of effect is very uncertain.

What are the key steps to defining a research question?

-Identify a problem/gap/etc. -Systematic literature review -Have a theoretical model

Where are CPGs often published?

-In documents other than peer-reviewed journals. -Professional organization websites (APTA, ECRI, CPG+, PEDro, etc.) -Book/pamphlet form to purchase -Search engines like PubMed may identify CPG's but unlikely to produce a comprehensive list

What are the key factors of a hypothesis?

-Includes the target population -Comparative, indicate a relationship "greater than" "less than" "different from" "related to" -Distinguish the IV and DV -Testable: theoretical model, clinical practice

Within a research study, what is the difference between the inclusion and exclusion criteria?

-Inclusion: The most important traits/characteristics that will qualify someone as a subject, which are important to the aim of the study and it works to restrict the sample into a homogeneous population (ex: demographics, clinical findings, geographical factors, etc.) -Exclusion: Factors that might exclude someone to be part of the study.

Describe each of the 5 levels of evidence for CPGs

-Level 1: Evidence obtained from high-quality diagnostic, prospective trials, or RCTs. -Level 2: Evidence obtained from lesser quality diagnostic studies, prospective studies, or RCTs (ex: weaker diagnostic criteria and reference standards, improper randomization, no blinding, <80% follow-up). -Level 3: Case-controlled studies or retrospective studies. -Level 4: Case series -Level 5: Expert opinion

When determining what the research question wants to accomplish, what is the difference between the main goal and the specific goals?

-Main goal: general objective or aim of the study. -Specific goals: specific objectives / hypothesis.

What is the difference between a research hypothesis and a null/statistical hypothesis?

-Research hypothesis: expectation of the main investigator of the results of the study. It expresses a change. -Null/ statistical hypothesis: express no difference or relationship between the independent variable and dependent variable.

The AGREE II tool that is used to appraise CPGs consists of 23 items organized into what 6 domains?

-Scope and purpose -Stakeholder involvement -Rigor of development -Clarity of presentation -Applicability -Editorial independence

In regards to the causality of an experimental study design, what factors must you consider about the causality?

-Strength of the association (IV and DV) -Time relation: temporal precedence -Covariation of the cause and effect: Dosage relation? -Plausibility: is there any other plausible explanation? -Consistency of the findings

When comparing and appraising several different systematic reviews, what factors should you compare/appraise?

-Study design -Number of studies -Intervention vs. Comparison -Outcomes measured -Results -Main issues/ Bias -Quality of Evidence -Main Conclusions -Clinical Relevance

In which 2 parts do CPGs typically present findings?

-Summary of the quality of available research evidence (numeric level of evidence). -Recommendation that includes the strength of the recommendation (alphabetical grades)

What is the difference between a Meta-analysis and a Systematic Review?

-Systematic Review: describes the systematic/ structured approach for retrieving, appraising, and interpreting the evidence. -Meta analysis: is a statistical analysis combining the results of the individual studies for the purpose of integrating the findings.

Compare the difference between Systematic reviews & Narrative Reviews in: Appraisal

-Systematic Reviews: A rigorous and critical appraisal -Narrative Reviews: Variable

Compare the difference between Systematic reviews & Narrative Reviews in: sources and search strategy

-Systematic Reviews: Comprehensive sources and explicit search strategy -Narrative Reviews: Sources are not usually specified and they have a potentially biased search strategy

Compare the difference between Systematic reviews & Narrative Reviews in: selection criteria

-Systematic Reviews: Criterion-based selection and uniformly applied -Narrative Reviews: Selection criteria is not usually specified and they have a potentially biased selection criteria.

Compare the difference between Systematic reviews & Narrative Reviews in: type of summary

-Systematic Reviews: Quantitative -Narrative Reviews: Qualitative

Compare the difference between Systematic reviews & Narrative Reviews in: type of inferences

-Systematic Reviews: Usually evidence-based -Narrative Reviews: Sometimes evidence-based

Compare the difference between Systematic reviews & Narrative Reviews in: type of question asked

-Systematic Reviews: a focused clinical question -Narrative Reviews: Broad question

About how long do CPGs take to produce if synthesized evidence exists, according to the APTA CPG manual? About how long do CPGs take to produce if no synthesized evidence exists?

9-12 months 2-3 years

Are experimental study designs studies of causality or studies of association?

Causality

What is the difference between clinical and statistical heterogeneity?

Clinical: -Variation in true treatment? -Risk factor effects? -Variation in magnitude or direction Statistical: -Clinical differences between studies -Methodological differences between studies -Unknown study characteristics

What type of studies provide the information on the occurrence of certain outcomes over time?

Descriptive

What type of questions are asked for a case-control study?

Estimate the prevalence of exposure to risk factors

If a researcher manipulates the independent variable, or intervention, what type of study design would this be?

Experimental

Is the following question for an experimental or observational study? For patients post with ACL reconstruction are Quadriceps close kinetic chain exercises more effective than the standard straight leg raising based program for the outcomes of gait speed and muscle atrophy?

Experimental

Is the following question for an experimental or observational study? What is the effectiveness of Pilates for the outcome of pain and function in patients with chronic low back pain?

Experimental

Is the following question for an experimental or observational study? What is the effectiveness of balance reaction training for the outcome of fall risk in the elderly population?

Experimental

What type of study design has the following factors: -Efficacy/effectiveness of intervention to produce outcome -Compare groups of intervention vs no intervention -Hypothesis driven -Causality

Experimental

Are the variables of an experimental study recorded, manipulated, or both?

Experimental= manipulated

What type of studies are conducted to estimate the frequency and distribution of a disease as well as risk factors associated with it?(cross-sectional, longitudinal)

Exposure/Outcomes based studies

When comparing the differences between a foreground and a research question, what are the different acronyms used for each?

Foreground question: P- population/patient/problem I- intervention (C)- comparison O- outcome Research question: F- feasible I- interesting N- novel E- ethical R- relevant

____ is the graphical representation of the results of a Meta-analysis.

Forest Plot

What is the inference of a study?

Generalized conclusions from the sample to the population

Describe each grade/recommendation for GPGs and the level of evidence that each grade/recommendation is supported by.

Grade/recommendation: A -We recommend -Level 1 evidence (strong evidence) -A bulk of level 1 and/or level 2 studies support the recommendation. Must include at least one level 1 study. Grade/recommendation: B -We suggest -Level 2 or 3 evidence (moderate evidence) -A single high-quality RCT or a bulk of level 2 studies Grade/recommendation: C -Option -Level 4 or 5 evidence (weak evidence) -A single level 2 study or a bulk of level 2 and 4 studies including statements by a consensus of content experts. Grade/recommendation: D -Unable to recommend for or against -No evidence or conflicting evidence -Higher-quality studies that disagree with respect to their conclusions. Grade: E -Theoretical/Foundational evidence -A bulk of evidence from animal and cadaver studies, from conceptual models/principles or from basic sciences research. Grade: F -Evidence is based on expert opinion -Best practice based on the clinical experience of the guidelines development team

When a meta-analysis is used, it should be paired with a test of statistical ____.

Heterogeneity

How do CPGs compare to Systematic Reviews?

High-quality CPG includes both a systematic review of research evidence AND explicit recommendations regarding clinical decisions. CPG's are typically broader than systematic reviews and address multiple aspects of care associated with a particular health condition (diagnosis, prognosis, and interventions).

What question should be considered to see if a meta-analysis summary is meaningful?

How similar must studies be to make meta-analytic summary meaningful? (study heterogeneity)

What type of questions are asked for a cohort study?

Incidence, prognosis, risk

Match the variable with its correct definition: Independent variable: ____ Dependent variable: ____ Confounding variable: ____ a) interference caused by another variable b) condition, intervention, or characteristic that will predict or cause a given outcome. c) the measurable outcome; varies depending on the independent variable

Independent variable: B Dependent variable: C Confounding variable: A

What type of questions are asked for a quasi-experimental study?

Intervention effectiveness: treatment, prevention, etiology/causality

What type of questions are asked for an RCT?

Intervention effectiveness: treatment, prevention, etiology/causality

Describe how a treatment would occur in an observational study

Investigator has no control over allocation of treatment or exposure among research subjects. Exposure occurs naturally.

In a forest plot: what does it mean when a study (individual or pooled) has any result with the 95% confidence interval cross the line of no effect?

It is not statistically significant (there is no difference between groups).

____ is a result when important study results are excluded from a systematic review due to language.

Language bias English language is predominant in PT literature, but is not the only language for publication.

In a forest plot: what does the vertical line (y-axis) represent?

Line of no effect (represents the point at which neither treatment is preferred over the other).

What is "Gray Literature"? Examples?

Materials and research that are produced by organizations outside of traditional publications. Examples: conference abstracts, presentations, white papers.

The types of epidemiological studies can be broken down into which 2 categories? What subcategories of studies can be broken down from those 2?

Non-experimental (observational) studies →Population-based or →Individual-based OR Experimental (interventional) studies →Randomized (ex: RCT or clinical trial) or →Non-randomized (ex: quasi, field trial, or community trial)

Is the following question for an experimental or observational study? What are the predictors of recovery (return to work and self perception of disability in patients with recurrent neck pain?

Observational

Is the following question for an experimental or observational study? What factors are related to time to return to play for junior high school basketball players following knee injury?

Observational

Is the following question for an experimental or observational study? What is the prognosis for independent walking for children with diplegic cerebral palsy?

Observational

Are the variables of an observational study recorded, manipulated, or both?

Observational= recorded/ observed

Meta-analyses can only properly quantify results under what circumstances?

Only if studies have been identified and collected in a systematic way.

What is the PEDro scale and how is used in systematic reviews?

PEDro Scale: Physiotherapy evidence database scale. Used in rehabilitation & medical literature to measure the quality of reports of RCTs.

In a forest plot: what does the diamond represent?

The weighted average effect size and confidence intervals when all individual studies are combined. Tips of diamond represent the 95% confidence interval for the effect size.

In a non-experimental (observational) study, if it is population-based, how can this be further broken down into additional subcategories? What if it is individual-based?

Population-based observational: →Descriptive (ex: health survey) or →Analytic (ex: ecological studies) OR Individual-based observational: →Descriptive (ex: case reports, case series) or →Analytic (ex: cross-sectional, case-control, cohort)

What type of questions are asked for a cross-sectional study?

Prevalence, diagnostic accuracy

Below is a list of criteria for a study. Which type of study is likely to use this inclusion/exclusion criteria: systematic review or a primary research study? Criteria: -Health condition -Age -Functional level -Time since injury

Primary research study

____ is the tendency for studies with positive results to be published more often than those studies with negative results.

Publication bias

The research question begins with a ____. Which is an issue someone would like to know more about or a situation that needs to be changed or addressed. What are examples of this?

Research problem -Areas of concern -Conditions that could be improved -Difficulties that need to be eliminated -Questions seeking answers -Gaps in knowledge -Inconsistencies

When interpreting a quantitative study, if an outcome of interest is dichotomous, what is compared between the studies?

Risk ratios and odds ratios can be used to represent each study and the results compared.

In a research question/ study, what is a theoretical model?

Scientific rationale that supports the question that you have. It is a theory to explain the rationale for the question.

What is "Generalizability"?

Shows how the results of a "small" representative group can be used with confidence to make predictions about the large population.

Meta-analysis should only be used to pool results when the studies that are included can be justified as being clinically similar in what 3 ways?

Similar in: population, interventions, and outcome measures.

Define: hypothesis

Statement that predicts the relationship between IV and DV. Assists the researcher in planning the designs and methods of the study. Used for projects with the aim of investigating relations, test a statement "reject it or accept it"

Below is a list of criteria for a study. Which type of study is likely to use this inclusion/exclusion criteria: systematic review or a primary research study? Criteria: -Types of studies (RCTs, etc.) -Types of participants -Types of interventions-Types of outcomes

Systematic Review

What is a standardized mean difference?

The measure of effect size

What is sampling?

The process of selecting a sample from the population to use in a study.

Define: power sample

The sample size required to find significant differences when they exist.

How would stakeholder involvement produce a potential bias in a CPG? What is one way to reduce this risk of bias?

There is a risk of bias when CPG authors translate objective data into recommendations. This risk of bias can be reduced by ensuring that the authors are from broad ranges of interest groups represented in the development process.

In CPGs, what is the role of patient preference?

To form a collaborative effort between the clinician and the patient to facilitate patient understanding of the recommendation, which could lead to better adherence and outcomes.

True or False: Combining individual studies provides a more precise estimate of treatment effects.

True (meta-analysis)

How does the GRADE System (Grading of Recommendations, Assessment, Development, and Evaluation) typically present CPG findings?

Uses similar 2 step process of summarizing quality and recommendation, but uses description rather than numerical or alphabetical ranking.

When a systematic review does an interpretation of qualitative results, it can also include the number of studies favoring one intervention compared to the number of studies favoring an alternative intervention or control. What is this process called?

Vote counting

An evidence base statement reflects both the ____ and the ____ and this statement is followed by an ____.

quality of evidence balance of benefit and harm action statement