Research Methods - Levels of Evidence

How are participants selected in nonrandomized intervention studies?

- Research design that does not randomly assign participants to treatment and control groups - Assign participants largely on the basis of logistics or convenience

Why is level II (randomized clinical trials) considered the "gold standard" of research design?

- The participants are randomly assigned to treatment groups - Clearly suggest the causation between treatment outcomes and patient outcomes

What is a nonintervention study?

- They are known as observational studies because there is not manipulation of any variables - The researcher is not implementing any intervention themselves

Why are nonrandomized intervention studies the most commonly used with SLP and audiology?

- Unable to recruit large numbers/logistics with recruitment - Difficult to randomly assign/blind participants - Your research may be affected by various bodily systems - More labor intensive - Measurability of treatments

What must be done in a systematic review?

- You have to tell the reader what the criteria was for acceptance/rejection, how many you reviewed, etc. - The highest level of evidence includes systematic reviews complete with meta-analysis that aggregate from well-designed clinical studies.

What may case reports challenge?

Accepted notions in clinical practice

What happens in a nonintervention study?

Looking back on previous files of individuals who are getting services/intervention; data-mining

What is a cohort study?

Follow a group of participants over time

What are nonrandomized intervention studies?

Quasi-experimental in nature

What is a retrospective study?

Recruit a group of participants, measure predictor variables from past data, and measure outcomes that occurred at a later date.

What is a prospective cohort study?

Recruit a group of participants, measure predictor variables, identify potential confounders, and follow the cadre over time, measuring outcome variables. Occurs in real time

What are cross-sectional studies

Recruit a sample of participants at one point in time to assess both predictor and outcome variables simultaneously to determine associations or prevalence

What should we remember about research requiring independent confirmation and converging evidence?

Research studies should replicate findings from another study; each data point should look to a trend

What are case-control studies?

Retrospective investigation design that seeks to identify factors or predictor variables associated with particular disorder

What are some characteristics of practice guidelines?

Statements that include recommendation intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits/harms of alternative care options. Establish practice guidelines from a systematic review Not a peer reviewed article

What is effect size?

Statistical calculations that estimate the clinical significance of a particular intervention.

What are the characteristics of a meta-analysis?

Statistical procedures that combine the results of individual studies in order to weigh the evidence regarding the efficacy of a particular procedure or intervention.

What is high internal validity?

You control for the environment, participants, and administration of any procedures employed.

What do we have to control in research in our field?

You have to control for a lot of variables within a certain disorder for research in our field (e.g. age, type, spectrum, etc); make sure researchers define populations in a way that correlate with results

True/False: Level VI (expert opinion) is not published or peer reviewed research.

True

Where are nonrandomized intervention studies usually conducted?

In the real world rather than in ideal conditions

True/False: Randomized clinical trials are not commonly used or popular in the COSD field.

True

True/False: nonrandomized intervention studies are most commonly used in the profession of SLP and audiology.

True

Why isn't randomized clinical trials popular/used in the COSD field?

We deal with behavioral treatment We provide long term treatment, change our treatment protocols → Logistics issues

What are the confidence intervals usually used?

We determine if we are 90%, 95%, or 99% confident these results will be repeated. The majority of researchers use 95%

How are nonrandomized intervention studies quasi-experimental in nature?

We have elements of experimental design but we are not conducting in a lab, our participants are not randomly assigned to a control or experimental group (our control group is typically developing and our experimental group is usually disordered people)

What is included in assessment of systematic review?

- Heterogeneity of the findings - Potential publication bias - Subgroup and sensitivity analyses

Why would you do a case report?

- Challenge norms in clinical practice - Present on case report - There may be someone else with a similar client (lead to collaboration)

What are some characteristics of a systematic review?

- Clear research question that describes the condition of interest, the population, the setting, the intervention and comparison treatment, and the outcome of interest - Comprehensive and unbiased search and retrieval process for studies - Logically defined inclusion and exclusion criteria - Clearly and uniformly presented information from studies: (Characteristics Findings Data) - Calculation of a summary estimate of effect sizes and confidence intervals for included studies (as appropriate) - Assessment

What should we remember about avoiding subjectivity?

- Clinicians and researchers must remain objective - Research should always present arguing sides (typically in the introduction)

What are the three types of nonintervention studies?

- Cohort studies - Case-Control Studies - Cross-Sectional Surveys

Why do we use/do systematic reviews?

- Get to the "bottom line" of all studies involving a particular treatment or intervention. - Validate research - Undertaken to estimate the amount of benefit derived from a treatment or intervention. - Combine the results from patients in clinically relevant subgroups across studies, thereby increasing statistical power. - To resolve discrepancies among the findings of studies. - Plan for future studies

What are the two types of cohort studies?

- Prospective - Retrospective

What does the first common theme entail for highest levels of evidence?

1. Require independent confirmation and converging evidence; essentially they aggregate results of several high-quality studies

What are the five common themes with highest level of evidence

1. Require independent confirmation and converging evidence; essentially they aggregate results of several high-quality studies 2. Studies at the highest levels of evidence require careful planning to ensure high internal validity and random assignment of participants to treatment and control groups. 3. Studies with highest levels of evidence avoid subjectivity and bias through the double-blinding of experimenters and participants 4. Research at highest levels should report both effect sizes and confidence intervals regarding effectiveness of intervention 5. Studies at the highest levels show relevance by recruiting accurate target population and demonstrate feasibility for replication

What is the second common theme with high levels of evidence?

2. Studies at the highest levels of evidence require careful planning to ensure high internal validity and random assignment of participants to treatment and control groups.

What is the third common theme for high levels of evidence?

3. Studies with highest levels of evidence avoid subjectivity and bias through the double-blinding of experimenters and participants

What is the fourth theme of high levels of research evidence?

4. Research at highest levels should report both effect sizes and confidence intervals regarding effectiveness of intervention

What is the fifth theme of high levels of research evidence?

5. Studies at the highest levels show relevance by recruiting accurate target population and demonstrate feasibility for replication

Memorize the Levels of Evidence pyramid!

:)

What is a systematic review?

Aggregation or collection of studies based on a clinical or research question

Why is the quality of research based on a hierarchy of levels?

All hierarchies use numbers, plus or minus signs, and/or letters to define the particular levels of evidence. The lower the number, the higher the scientific rigor. - We determine if particular research meets the qualities of rigor

What is the purpose of the case study reports?

Although one of the lowest levels of evidence, they can serve as the stepping stone for future research or a new clinical question

What are case reports?

Describes and interprets an individual case. Often describe: Uniqueness about the client that is different than a typical client (e.g. the disorder/behavior)

What is the caveat of cross-sectional surveys?

Inability to establish causality between predictor and outcome variables.

True/False: We want a small effect size.

False, we want large effect sizes

What happens if research studies cannot replicate findings?

If it cannot, this questions the validity of one of those studies

What is level 6 (expert opinion)?

Lowest level of evidence because it is based on opinion of leaders in the profession who have established themselves through clinical work and research

What are confidence intervals?

Measures of the precision of the summary effect estimate that should be reported.

How are systematic reviews and meta-analysis different?

Meta-analysis is only a quantitative analysis and a systematic reviews can be quantitative or qualitative

What happens when there is a failure in using random assignments?

Runs the risk of sampling bias because confounding factors are not equally distributed between treatment and control groups.

What is prevalence?

The number of persons who have a disease as a function of the number of those at risk.

How might the results of a cross-sectional survey be used?

The results may serve as a foundation for a cohort study and prevalence and relative risk rates may be determined.

True/False: You must have a balance of internal and external validity.

True, they have a reciprocal relationship and if you control too much for one it skews the other

What is the function of a systematic review?

Validate research because causality between treatment and positive patient outcomes cannot be established on the bases of a single investigation