Evidence-Based Practice and Introduction to Literature Evaluation

PICOTS

*P*atient *p*opulation/problem *I*ntervention or *i*ssue of *i*nterest (risk factors, tests etc) *C*omparison intervention or *c*ontrol group *O*utcome - ECHO model *T*ime frame of outcomes or f/u *S*etting

Source of Information

- A clinician should ALWAYS be able to support recommendations for therapy with EVIDENCE - Books are used as reference materials to provide core knowledge about a topic - Books are NOT the optimal source in providing up-to-date recommendations for therapy - Primary literature is the best source for information of this type

Article Discussion / Conclusion

- Discusses, interprets and evaluates the results from the study - May compare or contrast the findings of other relevant studies - May describe limitations of the research - May include ideas for future studies - The conclusion(s) should be drawn solely on the study results, and should not go beyond what the data can support

Article References

- Discusses, interprets and evaluates the results from the study - May compare or contrast the findings of other relevant studies - May describe limitations of the research - May include ideas for future studies - The conclusion(s) should be drawn solely on the study results, and should not go beyond what the data can support -If possible, the references should not be inclusive of only previous publications from the authors of the study in question -Format of references is dependent on the journal • Alphabetical order • Order encountered in publication

Relative Risk Reduction

- Drug A has an event rate of 30% (30 patients develop a MI out of every 100 treated) = EER - Drug B has an event rate of 40% (40 patients develop a MI out of every 100 treated = CER - Relative risk reduction is the ratio in event probabilities of exposure group with control group RRR = (CER - EER)/CER = 1 - RR = 1 - [30% / 40%] = 25% Drug A decreased risk of developing MIs by 25% compared to those who received Drug B

Absolute Risk Reduction

- Drug A has an event rate of 30% (30 patients develop a MI out of every 100 treated) = Experimental Event Rate (EER) - Drug B has an event rate of 40% (40 patients develop a MI out of every 100 treated = Control Event Rate (CER) - Absolute risk reduction is the difference in event rates between 2 groups = CER - EER - ARR= 40% - 30% = 10% = patients with Drug A have a 10% decreased risk of developing MI compared to Drug B

Article Acknowledgement

- Individuals contributing to the study, but do not meet the requirements for authorship - Financial support

Authors Section

- Most research articles list the investigators associated with the study - Authorship credit should be based on substantial contributions to (1) conception and design, or analysis and interpretation of data; (2) drafting the article or revising it critically for important intellectual content; and (3) final approval of the version to be published - Complex studies requires multiple investigators - typical drug study involve the following: - statisticians: study design and data analysis - physicians: patient recruitment, monitoring, medical care - nurses: medication administration, record keeping, nursing care - pharmacists: blinding, randomization of subjects, formulation of test drugs - analytical chemists: drug assays - scientists and other trained personnel: lab testing, assessments of drug effects - The following information is typically contained in the author's section • Name • Degrees • Title • Affiliation

Abstract of Article

- Provides a complete but brief review of the study - Includes a summary of the study: • Backgrounds • Objectives • Main methodology • Main results • Conclusion - The abstract should never be used by itself to evaluate the validity of a study! There is insufficient information in the abstract to evaluate a study's validity - The abstract simply gives an overview of the study so that the reader can choose whether or not to read/evaluate the entire study - Abstracts are typically limited in size by the journal - A typical abstract limit is about 250 -500words - This limited space prevents a complete review of the methodology - Readers or reviewers need to assess the abstract carefully to make sure that the results and conclusions are consistent with those in the body of the paper and not overstated by the authors

Journal Club Hunting (2)

- Start with a clinical case/issue or clinical question of interest to the individual or group - Question is then converted into a PICOTS question - A literature search to find relevant research study is performed and brought back to the group to critically evaluate and discussion application to clinical decision making and practice

Article Disclosure

- Statement that describe any potential conflicts of interest between the authors and the study • Grant funded by manufacturer of one of the study medications • Authors are employed by the manufacturer of one of the study medications • Authors have received financial compensation by the manufacturer of one of the study medications (paid speaker or consultant) • Authors have significant financial interest in one of the pharmaceutical companies sponsoring the study (e.g., stocks)

Introduction of Article

- The main purpose of the introduction is to describe background information relevant to some unknown problem or issue Why the study was undertaken as well as what the study purposed to accomplish - This may be accomplished by citing previous research - The last sentence of the introduction typically defines the study objective(s), sometimes the objective may be stated more in the form of a hypothesis e.g., we postulated that...... - Good studies do not contain too many objectives • Increases complexity • Requires greater sample size - The objective(s) should be specific: • Ex: Poor objective = evaluate RxA and RxB in heart failure • Ex: Good objective = compare differences in ejection fraction, LVEDP and NYHA classification after 6 months of treatment with RxA and RxB - The study hypotheses may or may not be specifically defined but should be obvious - Any objective listed in the introduction should be accomplished and reported in the study results - If a study is replicating a previously published study, the rationale for conducting such an investigation needs to be provided: • Ex: Previous study had insufficient sample size or insufficient duration of follow-up - Any information that is omitted provide a sense that unfavorable results are being hidden - Be cautious of publication bias in the introduction as readers can be easily manipulated by such practices

Title of Article

- The title is an important piece of information for any study - A good title accurately describes the nature of the research, unbiased, specific and concise e.g., "Improved bronchodilation with levalbuterol compared with racemic albuterol" - no significant differences were reported in lung function " A randomized clinical trial of bronchodilator from levalbuterol compared with racemic albuterol in patients with asthma" Since we often seek information by looking at the title, a title that is misleading could result in: • Obtaining a paper that is not what the reader expects Missing critical information because the reader chose not to obtain a study because he/she thought it was not useful

Steps of EBP

0. Cultivate a "spirit of inquiry". 1. Ask the burning clinical question in PICOT format. 2. Search for and select the most relevant best evidence. 3. Critically appraise the evidence (i.e., rapid critical appraisal, evaluation, and synthesis). 4. Integrate the best evidence with one's clinical expertise and patient preferences and values in making a practice decision or change. 5. Evaluate outcomes of the practice decision orchange based on evidence. 6. Disseminate the outcomes of the EBP decision or change.

Four Options to Identify the best evidence available to answer a well framed question.

1. Ask a colleague for his or her expert opinion. 2. Review practice guidelines (evidence-based or expert opinionbased) or a textbook for appropriate disease management e.g., UpToDate, Harrison's Online, Scientific American Medicine Online, Clinical Evidence Concise e-text, Pharmacotherapy textbook, Applied Therapeutic etc. 3. Consult electronic databases of systematic reviews and/or meta-analyses e.g., Cochrane library; and/or consider Evidence-based abstraction service e.g., ACP journal club, EBM, Journal watch series, PNN Pharmacotherapy line 4. Conduct a literature search using an electronic database e.g., MEDLINE, Turning Research Into Practice (TRIP), OpenText Federated Query Server

Evidence Based Abstraction Services

Abstraction services consist of an editorial team that scans dozens of journals, usually organized by specialty. They identify articles of potential clinical relevance, critically appraise the studies, and provide commentary on the quality, validity, and clinical significance of the results reported.

Risk Reduction

Always look at the outcome in the study (negative vs.positive effect) to ensure correct interpretation of the RR and OR, e.g., gastric CA, remission status, hospitalization, diabetes prevention, in order to determine the true clinical meaning of the results. These measures are often presented with 95% CI. Optimal treatment of patients requires use of the scientific literature to support clinical decisions Evidence-based medicine should be the norm Clinicians often want to know the bottom line Simple calculation of absolute and relative risk reduction may aid in making clinical decisions

Journal Club Appraisal (3)

Are the results of the study valid? (Internal Validity) What are the results? (Reliability) Will the results help me in caring for my patients? (External Validity - Applicability and Usefulness)

Usefulness of Medical Information Points

Audience Scope of practice Patient situation and/or clinical relevance Frequency of problem Types of outcomes, Evidences are accurate (valid) Journal's reputation Impact factor Original research

CINAHL Subjects

Covers 13 nursing and allied health disciplines

MEDLINE subjects

Covers medicine, health, and biomedical sciences

PsycInfo Subjects

Covers psychology, behavioral sciences, and mental health

Medical Research Sections

Discovery - Pre-clinical - Clinical • Phase I • Phase II • Phase III • Phase IV

Why use EBP

Highest quality of care and best patient outcomes Reduction in healthcare costs, Vast amounts of available information/healthcare literature Reduction in geographic variations in care Promotion of retention of healthcare workers

Journal of Citation Reports

Impact factor Publications

Barriers to Evidence Based Practice

Lack of knowledge and skills Information overload and/or overwhelming patient loads Lack of role models Lack of time, resources, or money to search for and appraise evidence Resistance to change Misconceptions or negative attitudes about research and evidence-based care - require education

NLM Produces...

MEDLINE and PubMed, both produced by the NLM, are the largest and best known bibliographic databases of biomedical journal literature. PubMed's in-process records provide basic citation information and abstracts before the citations are indexed with NLM's Medical Subject Headings (MeSH) Terms and added to MEDLINE.

EMBASE Subjects

Major European biomedical and pharmaceutical

Journal Club Evaluation of Outcomes, The practice Decision or Change Based on Evidence (5)

Measurement of outcomes is important to determine and document impact of the EBP change on healthcare quality and/or patient outcomes Essential in determining whether the change based on evidence resulted in the expected outcomes when implemented in the real-world clinical practice setting

Odds Ratio

Odds ratios (OR) are defined as the ratio of the odds (P/[1-P]) of having outcome in exposured group compared to control group. • OR compare the odds of developing an event/outcome between 2 groups, NOT risks or likelihoods • The OR and RR are interpreted very differently • What is the meaning of OR: - OR=1 means no difference in the odds of having outcome - OR<1 means decreased the odds of outcome/event occuring in group 1 vs 2 - OR>1 means increased the odds of event occurring in group 1 vs 2 • When the rate of event occuring in the control group (baseline event rate) is low/rare (<10%), OR is a good approximation of RR

Problems with Peer Review

Peer-review may be absent (editorial review only) - The referees may lack expertise in the topic - The referees may lack expertise in research methodology - The referees may lack expertise in biostatistical analysis - The referees may be a professional competitor of the author of the article - create reviewer bias, conflict of interest, ideas may be stolen during review process - Bias against the null hypothesis - Lack of time or motivation by referees • The reader is thus expected to review and self-evaluate all published research

Article Results

Presents the findings of the research e.g., subject demographics, dropout and compliance, endpoints and safety, subgroup analysis - Tables, graphs and pictures may be used to clarify the data • These should be included if they simplify reporting of the data - " A picture is worth a thousand words" • Beware of bias or misleading representation of the data - Statistical significance or insignificance of the findings - Result should be consistent with the methods; there should be NO results for which no methods were described. - There should be NO evaluation or discussion of the findings here

TRIP Database Subjects

Simultaneous searching of multiple sites, speeding up the question answering process

Soft vs. Hard Outcomes

Studies that affect "hard outcomes or patient-oriented evidence/outcome" have a greater impact than just those that affect "soft outcomes or disease-oriented evidence/outcome"' Morbidity and mortality versus treating lab values

Peer Review Validity

Sufficient validity and importance to warrant publication: - Clinicians - Methodologists - Biostatisticians - Epidemiologists • Final decisions on publication are made by the editors of scientific journals, after input by the referees

Facilitators of Evidence Based Practice

Support from leaders and peers Adequate time to critically appraise studies and implement their findings Research reports that are clearly written Resources Journal clubs and EBP rounds Determined effort is often required in order to address barriers and facilitate the implementation of EBP

Journal Club Integration (4)

This is the "implementation" of a decision to patient situations EBP is not based solely on research and published evidence Patients' histories and circumstances have a significant bearing on the clinician's choice of intervention Availability of resources must also be considered

Journal Club Disseminate the Outcomes of the EBP Decision or Change (6)

Too often in healthcare, positive outcomes are not shared with others There are numerous strategies for accomplishing this goal, from personal communication to conference presentation

Journal Club Grazing (1)

can be single article or multiple articles - Start with a group of individuals - Select the articles based on the goal e.g., care of a specific population or a topic of general interest to the audience - Dividing the relevant articles among individuals - Each individual is then responsible for perusing or grazing on the assigned articles of interest to the group - At the small-group meeting, each individual firstly takes his/her turns in presenting the information to the rest of the group members, then evaluating and leading the discussion of the study validity, biases or flaws, applying to patient care

Impact Factor

indicates the impact of the published articles relative to the medical field (scientific influence) - Impact factor is a tool used for ranking the relative quality of journals and is based on the number of times that published papers are cited in subsequent papers -the number of all the citations in the two previous years of a journal's articles divided by the number of articles published in those two years

Absolute Risk Difference

• ARD is the difference between the incidence in the exposure group and the incidence in the control group • Classified as ARI or ARR and generally presented a a positive (absolute) number • What is the meaning of ARD: - ARD = 0 means no difference between exposure group and control group - If exposure group has a smaller proportion of patients having outcome compared to control group, then a decreased risk of the outcome occurring = ARR ,in most instances dealing with prevention of bad outcome - If exposure group has a larger proportion of patients having outcome, then an increased risk of the outcome occurring = ARI, in most instances dealing with an increase in harm - Always look at the outcome in the study (negative vs. positive effect) to ensure correct interpretation of ARD, sometimes is presented with 95% CI

Intervention Effect / Assessment of Clinical Relevance

• After assessing internal validity of the study, we will then consider "Will these findings change the way I will treat or prevent this disease in my practice—and specifically for the patient sitting in front of me right now?" • First step, consider the clinical value of the beneficial outcomes reported. Are the outcomes demonstrating improvements important to the patients? Disease-oriented outcome vs. Patient-oriented outcome

Basic Elements of Research Articles

• Authors • Title • Abstract • Introduction • Methodology • Results • Discussion / Conclusion • References • Disclosure

Article Methodology

•Assess internal validity - The single most important section of the published research article - The study validity is dependent on the methodology • Internal validity • External validity - The methods should be described in sufficient detail that one could theoretically duplicate the experiment - Studies with poorly-described methods will not be published in substantial peer-reviewed journals The methodology should describe in detail: • The research design • Time period and location of the study • How patients were recruited and sampled • The randomization process • Inclusion / exclusion criteria • All independent variables in the study • All outcomes measured in the study • Necessary definitions (e.g., "Success") • Necessary follow-up schedule, data collection, compliance • Necessary sample size information • All statistical tests used in the study