M6/M7 Quiz
****Classify the use of EBM to address clinical questions such as harm
-Question is not whether someone exposed to agent gets ill, but whether those ALREADY WITH illness have higher rate or amount of exposure? What causes disease? RCT's would be ideal but impractical/unethical so Observational studies done, but have limitations Issue w/assessing harm/etiology--- not whether someone who is exposed to some kind of agent gets ill but whether those who have that illness have had a higher rate or amount of exposure.
4 categories of foreground questions (IDHP)
1-Intervention (or therapy)—Benefit of treatment or prevention 2-Diagnosis—Test diagnosing disease Ex: x-ray, labs, diagnostics 3-Harm—Etiology or cause of disease 4-Prognosis—Outcome of disease course
****Hierarchy of evidence for HARM
1-Randomized controlled trial (strongest & best form) best form of evidence, and often, we can use a randomized controlled trial to determine harm. 2-Cohort study 3-Case-control study 4-Case series/report (weakest)
Elements of Cochrane Reviews (meant to be dynamic)
1-Review is a systematic review, so it includes a statement of the clinical problem or question and sources of evidence, which are typically gathered from a literature search -->Statement of clinical problem or question 2-Sources of evidence Literature search Non-experimental data if included (controversial) 3)Inclusion and exclusion criteria Results in tabular and graphical form 4-Conclusions Date of last update → goal to continuously updated Last substantive update or significant new evidence Example of report: "A discussion of approaches to knowledge synthesis" (Hartling, 2014)
CDS Roadmap 3 pillars
1-best knowledge available when needed 2-high adoption and effective use 3-continuous improvement of knowledge and CDS methods under AMIA
3 reasons why published research studies in health informatics have not been Evidence-based
1-early hype ---not isolated to HIT vendors/techno-enthusiast, only govt & academia (aggravated by "technology pressure") 2-methodological challenges -early studies suffer from internal validity /external validity (whether results generalized to other locations and patients) issues. 3-failure to anticipate unintended consequences R/t HIT adoption
6 ways we arrive @ false beliefs (thomas kida) "dont believe everything you think"
1-stories>statistics. 2seek to confirm, not to question, our ideas. 3-rarely appreciate the role of chance and coincidence in shaping events. 4-misperceive the world around us. 5-oversimplify our thinking. 6-memories are often inaccurate.
****Limitations of EBM
1. evidence applies to *populations* not your patient. 2. *no evidence for Dx intervention or Tx doesn't mean doing it is a bad idea.* 3. EMB is still our best guide. "culture pushback": Not everyone agrees with the experimentally oriented approach of EBM Some criticisms of EBM are valid Challenges physician-patient autonomy Focuses on large-scale, randomized controlled trials that homogenize individual differences Concerns about manipulations of clinical trials data and reports Want other approaches: experience/observation Challenges physician-pt autonomy by having very rigid view of what evidence is to be applied in decision making for patient care.
decision trees
2 types classifications -attributes of pp input and algorithm classifies to one class or the other decision tree: decision analysis-derives optimal action , or event a flowchart of action
ISABEL
2002 differential dx web based worldwide s/s inputted as free text or imported from EHR paid RX mobile app
arden syntax (Standard)
A standard language for encoding medical knowledge representation for use in clinical decision support systems maint, library, knowledge and slots "curly braces problems" several EHR vendors use but @ low
SUMMARY: 2 main approaches for making reccmondeations based on evidence
Clinical practice guidelines provide steps and decision points for providing clinical care Decision analysis allows elucidation of a framework for making optimal decisions
Using a Decision Analysis
Elicit utility values for outcomes from patient, such as risk of adverse events from disease or treatment Calculate probabilities of events based on best evidence We plug evidence values and utility values into tree→ then "Fold back" decision tree to calculate overall utility (where we basically determine optimal pathway through the tree) Ex: afib, options: stroke or no stroke 4 permutations: no stroke w/no bleed, a stroke /no bleed, no stroke w/bleeding, and stroke with bleeding.
****Synopsis
Highly summarized information appropriate for clinical setting, such as 1-Critically appraised topics (CATs) 2-Clinical Evidence, InfoPOEMS, PIER 3-Clinical practice guidelines 4-Provide answers to specific questions
****Truth table
Mathematical table used to tell whether an expression is true for all legitimate input values Shows the expected results for a given set of input conditions and an operator (such as AND or OR) Why truth tables? Writing out the logic will help you when you get ready to write your SQL statement. Also, you need to understand what results you expect to get "convenient method organizing truth values of statements "
Continuing Medical Education (CME)
Traditional CME desired but evidence shows ineffective and does NOT lead to changes in practice online, CME comparable if not SUPERIOR to tradiional CME
Example of Case Report/Case series
amous example was Bendectin for nausea in pregnancy, where adverse publicity led to removal from market of safe and effective treatment Was combination of two agents, both of which were effective and neither of which were harmful (Magee, Mazzotta, & Koren, 2002; Hale, 2012) Re-introduced use of ingredients in a delayed-release combination pill after US FDA approval in 2013 (Nuangchamnong, 2014) (doxylamine & pyridoxine)
****Logic gate
an arrangement of electronically-controlled switches used to calculate operations in Boolean algebra. Physical model /implementation of a Boolean function Determines whether signal is passed through or stopped In computers, electrical signal sets bits in RAM or on disk AB = A And B AB= A or B A is A? (stopped)
unsupervised machine
analyses data without any classes (clustering) cluster analysis- technique that organizes large data set into disint groups gene sequences association rules- IF THEN statements--> "market basket analysis " (if man buys diapers ther's a 90% chance he will buy a beer)
Background Questions
ask for general knowledge about a disorder Usually answered with textbooks and classical review articles usually asked by students come up in the care of patients, and they are usually answered without using evidence-based medicine techniques. General information not specific to a given patient Distinction from foreground questions can be blurry New etiologies of disease
Foreground questions (PICO) & example
ask for knowledge about managing patients with a disorder, or contemplating test/treatment have 4 questions Usually answered using EBM techniques Have three or four essential components (PICO) -Patient and/or problem -Intervention -Comparison intervention (if appropriate) -Outcomes usually asked by subspecialist Example In an elderly patient with congestive heart failure, are beta blockers helpful in reducing morbidity and mortality without excess side effects?
SUMMARY ON QUESTION OF HARM AND ANSWERED W/ WHAT KIND OF STUDY?
assess whether exposure to some natural or manmade agent causes disease and is usually answered with a case-control study
methodology challenges
knowledge use knowledge management knowledge acquisition (expert/data based knowledge) knowledge representation (configuration & table based) Rules based Basyeian networks knowledge maintenance
supervised machine learning
machine learning that requires humans to provide input and desired output as well as feedback about prediction accuracy during the beginnings of the system
Types of SYSTEMATIC REVIEWS and define one of them
meta-analysis (combines results of multiple similar studies (systematic review NOT EQUAL to meta analysis --> too heterogenous
Precision
narrower confidence intervals or statistical significant results indicate higher precision
SUMMARY OF QUESTIONS ABOUT PROGNOSIS TELL US WHAT?
natural course of disease
****TJ Hooper decision
tug boats, negligence for no radio, even though not commonplace Conforming to custom will not insulate from liability if the custom is unreasonable or there are better, reasonable alternatives "lagged in the adoption of new and available devices precautions so imperative that even their universal disregard will not excuse their omission. ertainly in such a case we need not pause; when some have thought a device necessary, at least we may say that they were right, and the others too slack."
****AND
two conditions
limits on Decision analysis
-idealized situation that may not apply to a patient but give a framework for making decisions and/or deviating from standard approach -time-consuming on individual level - dependent on quantification of values and fuzzy situations
Unfiltered
-non-selected evidence (PUB MED) -a classification of LOE (level of evidence)
Cochrane Collaboration
"An international collaboration with the aim of preparing and maintaining systematic reviews of the effects of health care interventions" (Hersch, 2008) Largest producers of systematic reviews, limited to INTERVENTIONS/THERAPY/TREATMENT questions (not diagnosis or harm (unless the harm takes place in the context of an intervention or prognosis).
****Reasons to study EBM
"Effectiveness" 1/ 6 of attributes advocated in IOM's Crossing the Quality Chasm Report advocates in detail the use of informatics for a "learning health care system" We learn from what we do: main way to learn from what we do is to collect data and analyze it, which can be done without computer-based information systems (but allow us to analyze data and trends) "Descriptions of Methodological Details and Challenges for EBM," in Medical Care EBM key points in JAMA
***IOM defines evidence based Medicine as what? And from what report?
"Patients should receive care based on best available scientific knowledge. Care should not vary illogically from clinical to clinical or from place to place." Crossing the Quality of Chasm
****QMR
"Quick medical reference diagnostic CDDS of extensive knowledge base of dx, symptoms, lab findings. -Leader randolph miller from INTERNIST 1 -combines s/s using evoking strength and frequency type weights to arrive at a list of possible diagnoses. -had to ship CD's "knowledge maintenance" DECISION SUPPORT NOT expert system VALUED AUTONOMY "Systems should enhance not replace MD" DISCONTINUED IN 2001. (stand alone) integrated w/VANDERBILT EHR Mckessions horizon expert orders
**Federal Coordinating Council definition for CER-
"Research comparing different interventions and strategies to prevent, diagnose, treat and monitor health conditions" Key aspect of CER- directly compares two or more existing TREATMENTS OR INTERVENTIONS to determine the effectiveness, benefits, and risks of each. Ex: comparing 2 sleep meds to determine which is best under specific conditions WOULD NOT draw conclusions from separate studies comparing each sleep med w/placebo "Must assess a comprehensive array of health-related outcomes for diverse patient populations" Otherwords CER must consider factors like gender/ethnic diversity, pt's with chronic conditions/multiple conditions that interact w/each other "Necessitates the development, expansion, and use of a variety of data sources and methods" (informatics!) such as clinical trial and electronic health records, such as data mining algorithms, that gather useful information while also adhering to confidentiality laws.
Filter
"an expert has appraised and selected the BEST evidence /up to date" -a classification of LOE (level of evidence)
I****Internist-1
"focused on production rules(IF THEN statements) used pt observations to generate possible dx knowledge base was used in QMR" -Jack Myers of diagnostic expert, computer scientist named Harry Pople) "doc in a box" -randolph miller---> QMR successor -user entered findings 1-controlled vocabulary 2-weighted positive and negative findings —evoking strength, frequency and importance Ex: pt had flu-like symptoms, but NOT fever. → system used a weighting system that combined the data to arrive at a diagnosis. The weighting terms included evoking strength,( how strongly a given finding suggested a particular disease.) Frequency was how often you would expect to see the finding if the patient had the disease and importance was how significant the finding was. -stand alone system: required separate data entry from the data entry into the medical record for routine clinical care. These CDS programs were not connected to any other systems and were basically run on a single computer
SYNOPSES (source of summarized evidence)
"make the data available to users in highly digested form"-Haynes 4S Clinical Evidence: "Evidence formulary" bills itself as an evidence formulary draws on Cochrane Reviews and other syntheses & individual studies to summarize evidence. InfoPOEMS: "Patient-Oriented Evidence that Matters" Physician's Information and Education Resource (PIER) from the American College of Physicians
technology pressure
"natural tendency" to try to fit new technology into healthcare, even when benefits not proven
barriers to Clinical decision support systems
"no single 1 entity to oversee strategic development" (CDS collab, AHRQ, DHHS) -clinicians need to provider several pieces of information -impact lower than expected due to bypassing -Physician behavior on implementing guidelines -struggles to provide higher-quality care at LOW COST
***Clinical Decision Support & who defined it?
"provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care"-ONC for HIT -any resource that aids clinician/hc member or pts'. -provides treatment reminders for disease states that may include use of more cost-effective drugs. (ex: TIerny studies- failed to demonstrate improved compliance to guidelines using Computer reminders for HTN , Diurex-w/reminders, ortho surgeons improved compliance guidelines to prevent DVT)
****Clinical Practice Guidelines (CPG) defined by IOM in 2011
-"statements that include reccs intended to optimize patient care that are informed by systematic review of evidence and an assessment of the benefits and harms of alternative care options"-IOM -method of gaining medical knowledge; publishing one does not change how medicine is practice & quality of CPGS is variable and inconsistent
Cost
-A clinical question -Suggest Type of Study: Economic analysis and modeling -
****Diagnosis
-A clinical question -Suggested Type of Study: Prospective, blind comparison to a gold standard
****Prognosis
-A clinical question on the OUTCOME OF DISEASE COURSE (NATURALLY) -Suggest best type of study: Cohort Study>case control> case series "PURE" prognosis are rare: -Prognosis is "natural history" of disease But very little "history" is "natural" in modern era with our abundance of diagnostic tests, interventions, harmful agents, and so on -Many studies measure prognosis after a test or intervention -Prognosis usually measured by Survival Curve "Later we get cancer, worse prognosis, but stage 4 likely to die within 7 years
***Evidence-based medicine (EBM) definition
-A set of tools and a disciplined approach to informing clinical decision making -an approach to informing clinical decision making that applies the best evidence available -Allows clinical experience (art of medicine) to be integrated w/ best clinical science -Makes medical literature more clinically applicable and relevant -Usually answers foreground questions (asked by subspecialist) -expert opinions complement not replace EBM -most common type of question in EBM is INTERVENTION (AKA treatment or Therapy)
Clinical prediction rules (A Variation on diagnosis clinical questions)
-A systematically derived and statistically tested combination of clinical findings that provides meaningful predictions about an outcome of interest -Use of results of multiple "tests" to predict diagnosis (Adams, 2012) -Best evidence establishes rule in one population and validates in another independent one -Techniques in clinical prediction rule- can be used to evaluate new markers for disease as they are developed as well as to compare results between studies. (ex: Predicting DVT's involved ---> high sensitivity (good at ruling out dx) moderate specificity ---> better for ruling out than ruling IN disease Coronary RIsk prediction- newer risk markers dont add more to known basic risk factors
****Comparative effectiveness research (CER)
-The new EBM Mantra -prominence when ARRA allocated 1.1 billion for CER -2 reports required to inform operational plan: 1) federal coordinating council for comparative effective research 2009 (FCCCER)(they defined CER!) & Called for development not only of research but of human and scientific capital, data infrastructure, and dissemination 2)IOM report for prioritizing research: (Committee on Comparative Effective Research Prioritization, Board on healthcare services & IOM) A- identified top 100 research priorities B- address not only common DZ but issues on HCD & health disparities
****Regenstrief Medical Record System (RMRS) "R stands for reminders" CDS
-Type of CDS is : REMINDER Developed by informatics experts at the Regenstrief Medical Institute in Indiana and used at multiple hospitals and clinics Original intent: to develop decision aids to "remind" physicians of things they want to do but might forget" Integrated with the Regenstrief Medical Record System (RMRS) Gradual expansion of rules and sites this system is one of the models often pointed to for its sophistication
****Define clinical practice guidelines & how used in EBM & steps of clinical practice guideline?
-a synopses -evidence that they can work if address MULTIPLE COMORBIDITIES (concurrent chronic diseases) do work. -despite evidence RAND study in NY England journal of medicine shows only 54% of patient's receive recc. care" -hardest part is implementing CPGS and achieving buy in by all healthcare workers, particularly physicians -Series of steps for providing clinical care May consist of text/tables or algorithms Algorithm steps (Ohno-Machado et al., 1998) (usually) 1-Action: Perform a specific action 2-Conditional: Carry out action based on criterion 3-Branch: Direct flow to one or more other steps 4-Synchronization: Converge paths back from branches
examples of CDS
-alerts to abnormal lab values -alerts to ADE's -Dx screening reminders -immunization reminders -alert -infobutton -reminder -orderset
****Dxplain
-early CDS 1980's, stand alone program STILL USED! -Diagnostic decision support system developed at Massachusetts General Hospital -"based on clinical findings, justifies diagnosis suggest further steps,describes atypical manifestations" -Similar structure to QMR COMBINES S/S evoking strength and frequency type weights to arrive at a list of possible diagnoses. -AMA Sponsored -Individual going to a particular Website and entering data into the CDS system
****Harm AKA
-etiology or cause of disease -a clinical question -1/4 of the categories of a Foreground Question Suggested Type of Study: RCT & COHORT > case series
IT helps CPGS how?
-expediting search for best evidence & linking results to EHR's and smartphones for access -2 areas disease management & quality improvement strategies - important role in improving care and lowering costs
****Randomized Controlled Trial (RCT)
-experimental method in which participants are randomly assigned to TREATMENT or CONTROL group (placebo or not) -assures pts in two groups balanced in both known & unknown prognostic factors, only difference is intervention -trials are homogenize -DOUBLE BLINDED- meaning that both investigators & subjects do not know whether they got the drug or not --> less biases GOLD STANDARD to test therapeutic interventions
*Clinical decisions (evidence based) 3 aspects
1) EVIDENCE -randomized trials -systematic reviews -patient data -basic/clinical & epidemiologic research EVIDENCE+PATIENT/MD FACTORS=KNOWLEDGE EVIDENCE+CONSTRAINTS=Guidelines 2)PATIENT /PHYSICIAN FACTORS -cultural beliefs -personal values, -experience, and -education. patient's physical reaction to treatment and medications -role inc Clinical decisions (CD) Physicians may have limitations in skills, if rural areas, or entire medical services not provided FACTORS+CONSTRAINTS=ETHICS FACTORS+EVIDENCE=KNOWLEDGE 3)CONSTRAINTS -Formal policies, laws -community standards -time (no time to treat if acute illness) -reimbursements (MEDS,TX, TEST covered by insurance) CONSTRAINTS+FACTORS=ETHICS CONSTRAINTS+EVIDENCE=GUIDELINES
3 asPECts of what goes into making clinical decisions
1) Evidence - what does the research show? randomized trials and systematic reviews. 2) Patient/physician factors cultural beliefs, personal values, experience, and education. patient's physical reaction to treatment and medications -role inc Clinical decisions (CD) Physicians may have limitations in skills, if rural areas, or entire medical services not provided 3) Constraints Formal policies and laws as well as community standards may influence decisions Time-if acutely ill, no time to treat based on EBM reimbursement-what medications, treatments, and tests are covered by a patient's insurance
****Challenges for CDS
1) Funding -Older innovative programs were granted funding, once ran out, abandoned before being put to use in practice setting -some evidence that CDS has the potential to save costs, that potential has not usually been enough MOTIVATION for hospitals or other larger organizations to fund the needed research and sustained development 2) Knowledge maintenance -changes rapidly, & using out-of-date CDS is dangerous -When QMR commercial, company not only had fund experts to update knowledge base→ had to ship CD's w/ update to their user base (even when not guarantee users would take time to install & update to their systems) -Cost w/keeping systems up-to-date 3)Optimal Timing -figuring out how to provide timely information that will be used and yet not overwhelm the user with too many alerts or reminders, inducing what has been called 'alert fatigue.' 4)Alert Fatigue -Research studies over the years have shown that when there are numerous alerts, the doctors tend to ignore them. 5)Motivation for use -Incentives within the 2009 HITECH Act for meaningful use of EHRs (pronounced E-H-Rs) will very likely spur development of CDS, and there are examples from history that give us hope that there will be more use in the future
*******Added value of CDS
1) Improve Quality: a) adherence to guidelines b) avoid inappropriate procedures c) avoid dx therapy errors (by alerts) --> drug interactions/delay in dx d) Potentially Minimize severity/complications --alerts to abnormal lab values --alerts to ADE's --Dx screening reminders --immunization reminders 2)Great return on investment
******3 Parts of Structure of CDS (CIK)
1) Knowledge based ----Compiled information If-then rules EX: IF a new order is placed for a particular blood test that tends to change very slowly, AND IF that blood test was initially ordered within the previous 48 hours, THEN alert the physician. In this case, the rule is designed to prevent duplicate test ordering. ---Other types of associations probabilistic associations of signs and symptoms with diagnoses. (Some more likely present with diagnoses, not 100% there) Ex: flu s/s fever, cough, body aches but not always all 3) 2)Inference Engine Model/formulas for combining the knowledge base with patient-specific data 3) Communication Mechanisms -Input of patient data -Output of information to user
4 phases of CDS architecture
1) STAND ALONE required separate data entry from the data entry into the medical record for routine clinical care. These CDS programs were not connected to any other systems and were basically run on a single computer 2)Integrated to a certain extent with other systems, usually hospital electronic health records, so that they could use the patient data included in them and all people who used the hospital system could take advantage of the integrated CDS The Abx assistant program that's part of HELP system at LDS Hospital in Utah is typical of that phase 3)STANDARDS-BASED development of knowledge bases with agreed upon technical standards so some of the knowledge could be shared across sites. Different sites with different systems could apply the knowledge if they both have systems that use the same standards. Ex: of this standard was known as the Arden Syntax for describing decision rules Arden - not another technical abbreviation, named after Arden NY where standard was done 4)SERVICE MODELS what is likely in the future, is providing the CDS through the World Wide Web and having it operate behind the scenes. Rather than an individual going to a particular Website and entering data into the CDS system( like DXplain) the knowledge might reside at a central place and the CDS would be programmed to access that knowledge and use it to provide decision support locally. Need to have local programs set up w/web-based systems
5 rights of CDS
1) right information (what) - quality of knowledge -based on highest level of evidence possible w/validity, 2) right person (who)- target of cds, -healthcare member/patient 3) right format (how) implementation of cds, -alert -infobutton -reminder -orderset 4)right channel (where) - mode of cds, -EHR alert?, text? email? alert? 5)right time (when)-workflow integration, -new info should appear early in order entry process before they complete. -electronic Rx (adds usability of technology) Jerry Osheroff described what he calls the five rights of decision-support rule of thumb for deciding if a proposed decision-support intervention accents are not Dr. Shaw suggests that good decision-support interventions require the right information being delivered to the right person in the right format via the right channel and at the right time
National Quality Forum (NQF) 4 components (TIIA), and goal?
1) triggers, 2) input data, 3)intervention, 4)& action steps goal was to create taxonomy for future quality performance measures and ma quality data set (QDS)
Appropriate Times for CDS
1)At the point of decision making ex: drug dosage information is best to have available at the time the doctor is prescribing medications. 2)When new data arrives exa: if the report form monitoring a patient's blood level shows the patient has too much of a prescribed medication in his or her system. Need to take immediate action based on that information. 3)To stop dangerous decisions Ex: might be if the physician has just ordered a drug that the patient is allergic to; the warning should be presented before the order goes through. 4) When clinician requests it- "ON demand" Ex: might be references related to a particular treatment question. 5)Appropriate frequency Ex: Too frequent repetitions of similar information can tend to get ignored. (alert fatigue)
***UNintended consequences of CDS (not by ash but in book)
1)JC used Setinal Event alert in 2008 to alert HC workers that 25% of med eros were R/t to technology issue 2-Alert fatigue cause drug/lab test alers to be ignored 3-Alarm fatigue is big issue as alert fatigue 4-Distraction while using mobile devices & social media issue while on the job 5-Upcoding w/EHR use could ^ HC cost & raise thorny ethical/legal issues 6-HIT may raise, not lower long-term HC costs 7-Privacy & security issues on increase due to widespread HIT adoption 8-e-iatrogenesis (ADE r/t technology
****Bates "Ten Commandments for Effective CDS"
1)SPEED 2)ANTICIPATE NEEDS/DELIVER IN REAL TIME 3)FIT WORKFLOW 4)USABILITY MATTERS (little things make a difference) -if you make it complicated to ignore people will use it -no use of free text or no reminders sent 5)MD'S RESIST (IV ketoralac) 6)CHANGE DIRECTIONS EASIER THAN STOPPING (ex: pharm & therap committee- changing MD behavior computer order entry system & choice of views decision support- order what direction of abd view to order 7)SIMPLE INTERVENTIONS WORK BEST Must fit guide on 1 single screen -use asa when post MI unless otherwise contraindicated 8)ASK FOR ADDITIONAL INFORMATION ONLY WHEN YOU REALLY NEED IT weight of patient nephoro drug for CHF -computerized guideline success RT number of extra data eleents needed 9)MONITOR IMPACT, GET FEEDBACK , RESPOND -"action-oriented suggestions" -drug to drug interactions 10) MANAGE AND MAINTAIN YOUR KNOWLEDGE critical to successful delivery critical to keep up w/pace of change of medical knowledge
3 high level categories based on function for CDSS developers
1)ordering facilitators 2)point of care alerts/reminders 3)relevant information displays (order faciliators) (order sets) (therap support) (smart formstemplates)- electronic encounter notes that sturucures questions alerts and reminder
Barriers/limits to CPG's
1)practice setting -7.4 hr workday jsut to comply w/US preventive services task force reccomendations 2)comorbids MAY NOT APPLUY TO COMPLEX PATIENTS 3)contrary opinions -experts don't agree -MIND LINES INSTEAD--> Their own rational which is not research evidence 4)sparse evidence -tight DM2 turned out did not result in improved outcomes 5)knowledge and attitudes -MDS DONT FOLLOW -lack of confidence to not promote a test -information overload 6)too long, imperative/confusing where should be posted/format? -should be no longer than 2 pages, they are 50-100 pages 7)less buy-in if data reported is not local 192 authors of 44 CPGS had 87% tie to pharmacy company 8)No uniform evidence (LOE) rating system -non-filtered searches 9)lack of champions to promote MD'S WERE NOT AWARE 10)excess influence by drug companies 11)quality of national guidelines 12)no patient input -difficult to implement in EHR (coding logic/integration workflow -EX: LIN Found lack of adherence to recc. of major guideline on use of stress testing before PCI .
** more reasons to study EBM (book)
1-Current methods of keeping medically/educationally UP TO DATE DON'T WORK 2-TRANSLATION of research into practice often SLOW LACK OF TIME & volume of published material results in information OVERLOAD 3-The pharm & medical device industries bombard clinicians and pt's everyday w/misleading & BIAS information: -They heavily influence research & publication Issues resulting from their influence include: Treating questionable or early diseases before evidence is in -Overpower studies so there is statistical but not clinically significant outcomes -Establishing inclusion criteria so patients most likely respond to treatment are included -Using surrogate & not clinical endpoints & only selecting studies w/positive results 4-Much of what is considered "standard of care" everyday practice has yet to be challenged and could be wrong ---> without proper training clinicians will not be able to appraise best information resulting in poor clinical guidelines & wasted resources
**How to use EBM to assess clinical questions about diagnosis? involves what 2 things? what 2 essential steps? what are 2 variations?
1-Diagnostic process involves: logical reasoning and pattern recognition Consists of two essential steps: 1)Enumerate diagnostic POSSIBILITIES and ESTIMATE their relative likelihood, generating DIFFERENTIAL diagnosis 2)Incorporate NEW INFO from diagnostic tests to CHANGE probabilities, RULE OUT some possibilities, and CHOOSE MOST LIKELY diagnosis Two variations on diagnosis to be discussed: 1)Screening 2)Clinical prediction rules
external validity AKA
APPLICABILITY indicates that results reported in study can be generalized to the patients of interest.
Antibiotics assistant
Advising and critiquing system for use of antibiotics developed at LDS Hospital, Utah Integrated with the LDS Hospital information systems as part of HELP system (Health Evaluation through Logical Processing) Around since 1960's Variety of decision support apps The full name came later Provided advice on orders for antibiotics to prevent infections By using data in patients record Includes resources to inform MD's Currently in use in LDS Hospital and other hospitals part of the Intermountain Health Care (IHC)
Effectiveness studies
Ascertain whether something works in the "real world" when put in hands of larger segment of clinical provide population
What does "intelligently filtered" mean in the defintion of CDS?
Aspect of information--> 1)For the particular clinician -reminders for certain preventive measures for patients that the clinician wants to do but may forget. 2) Usable Form -Ex : Correct dosage for med→ 20 page article on med too much information 3) Context sensitive -Ex: displaying a patient's drug allergies at the time that the physician is ordering medications 4)Tailored to patient -Ex: medications require special monitoring for elderly patients, but not for children. If a pediatrician treating a child is using the medication, the pediatrician does not need a warning about how the medication should be monitored for an eighty-year-old.
HITECH ACT impact
Change standard of care Does not require use of CDS Provision in HITECH Act for MU of EHR"S will financially reward use initially→ impose financial penalties for non-use Based on history's of other technologies & if implementation challenges met→ future ^ use of CDS to improve care looks bright.
dashboards
CDS, "population based" decision making include patient summary, flowchar, lab resultsm vs or disease registry.
Traditional methods for gaining medical knowledge
CME (see other card) CPGS (see other card) Expert Advice-different ways, needs to be evaluated w/knowledge that their reccs not relevant to primary care population Reading-info from pharms, too much to keep up, leaves clinicians confused
examples of systematic reviews of clincial question : TREATMENT/INVERVENTION/THERAPY
Cardiac risk factors: meta analysis found benefits to lowering cholesterol & blood pressure & homocysteine POLYPILL (6 meds )- was created and could reduce cardiovascular dz by 80% & POLYMEAL clinical trial done in India confirmed
Systems
Decision support within EHR systems Best way to provide evidence to clinicians at point of clinical decision making Designed to facilitate the decision-making process at the point of care.
steps in creating a *SYSTEMATIC REVIEW (guyant, rennie meade, cook)
Define the question—Population, intervention, comparison, outcome(s) Conduct literature search—Define information sources and searching strategy Apply inclusion and exclusion criteria for articles retrieved, and measure reproducibility Abstract appropriate data Conduct analysis—Determine method of pooling, explore heterogeneity, and assess for publication and other bias
3 Questions asked about Clinical Practice guidelines in order to appraise?
Developers carry out comprehensive ? reproducible literature within last 12 months?
**evolution of CDS
Evolution of CDS CDS systems began to proliferate Late 1960's Diagnostic and reminder CDS & major focus for informatics research and development. 1980s to early 1990s Drug Interaction CDS 1990s to Present Quality Indicator CDS 2010 and beyond CDS for Precision Medicine (to prevent problems when ordering medications) Late 1990's
example of foreground questions
Example: In an elderly patient with congestive heart failure, are beta blockers helpful in reducing morbidity and mortality without excess side effects?
Cochrane Database of Systematic Reviews (CDSR)
Full-test database containing systematic reviews and protocols (reviews still in progress) of the effects of healthcare interventions "It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials."—Archie Cochrane, 1972 CDSR embodies Cochrane's vision About 2,000 reviews done but many more needed to cover medicine comprehensively
HELP
Health eval through logical processing -integrated w/abx assistant full name came later Provided advice on orders for antibiotics to prevent infections By using data in patients record Includes resources to inform MD's Currently in use in LDS Hospital and other hospitals part of the Intermountain Health Care (IHC) developed medical logic modules for CDS
****Therapy AKA WHAT/WHAT
INTERVENTION/TREATMENT/THERAPY!- question concerning Benefit of treatment or prevention -MOST COMMON type of question in EBM is INTERVENTION (AKA treatment or Therapy) Can include drug therapy, diet therapy, surgery, alternative medicine, and so on ***Best evidence comes from a randomized controlled trial (RCT) or meta-analysis of RCTs Key aspect: Patients similar in all regards with exception of intervention applied Ideal conditions are not always present - category of question asked in FOREGROUND questions -**Suggested Best type of study: RCT>cohort>case control>case series
unintended consequences per Sitting & singh on patient safety
JC used Setinal Event alert in 2008 to alert HC workers that 25% of med eros were R/t to technology issue Alert fatigue cause drug/lab test alers to be ignored Alarm fatigue is big issue as alert fatigue Distraction while using mobile devices & social media issue while on the job Upcoding w/EHR use could ^ HC cost & raise thorny ethical/legal issues HIT may raise, not lower long-term HC costs Privacy & security issues on increase due to widespread HIT adoption
Clinical Decision SupportSYSTEMS
Information technology systems that support electronic CDS. underused tool for managing utilization & improving both efficiency & quality degress of computerization US on 1st level-no assistance need to be on level 5 - executes selection of human approves
EBM history
Initial focus was clinical epidemiology, & detection of bias But remember the caveat: "Absence of evidence is not evidence of absence" (Martin Rees-cosmo/astrophyscist)
Validity/internal validity
Is the study believeable? If apparent biases or errors in selecting patients, measuring outcomes, conducting the study, or analysis are present , then study is less valid. If study poorly designed, will have poor INTERNAL VALIDITY.
FHIR (Standard)
JASON task force astronaut project webservice approach ( gacebook and google)
****Boolean
Logical Operator Named after 19th century mathematician, George Boole Fundamental to computer operation and data retrieval
EHR's without CDS (LAC) & what's the added value w/it?
L- Legible A-Accesible C-Complete (not guaranteed) added value of CDS: improve QUALITY Greater return on investment
de deombals system for acute abdoinal pain
Leeds in UK 1970's Bayesian probability to assist in differential dx of abd pain
Case Control Study & EXAMPLES
Most common form of observational study Retrospectively identify cases of diseases (that we think) and match to otherwise similar controls, looking to see if different rate or amount of exposure Problem is when controls create SPURIOUS ASSOCIATION: e.g., Coffee drinking associated with miscarriages 2016 NIH article says yes, Slate magazine article disputes findings pancreatic cancer (MacMahon et al., 1981), but controls were patients with other GI diseases whose symptoms were exacerbated by coffee (so they drank less) Differences were not present when other appropriate controls were used (Zheng et al., 1993) Useful when condition is VERY RARE or has long development time Classic case was demonstration that DES causes vaginal cancer (reviewed in Swan, 2000
Limits of Systematic reviews
Not everyone accepts use of meta-analysis; Feinstein (1995) calls it "statistical alchemy" Meta-analyses on same topic sometimes reach different conclusions due to methodologic differences (Hopayian, 2001) "Truth" determined by meta-analysis has the shortest "half-life" of all knowledge (Poynard et al., 2002) Effect of publication bias may be exacerbated in systematic reviews (Dickersin, 1997; Dwan, 2013)
***Classic CDS origin
Origin: Expert systems research (AI, aim to build computer program that could simulate human thinking which could fxn as expert consultant) TO Medical process improvement -adapted from expert systems to CLINICAL DECISION SUPPORT SYSTEMS -moved from expert systems to CDSS to apply in real life pt care process INTENT:assist MD (not simulate) Provide info to user--> not to come up w/answer User's role: - filter information & discard erroneous/useless information -interact w/system (be active not passive on recept of output) *REFERENCE INFORMATION AND GUIDANCE *REACTIVE ALERTS AND REMINDERS
Results
Results should be assessed in terms of magnitude of treatment effect and precision (narrower confidence intervals or statistical significant results indicate higher precision).
Primary prevention
aim to keep disease/complications from occurring
application of EBM Steps
Phrasing a clinical question that is pertinent and answerable Identifying evidence to address the question Critically appraising the evidence to determine whether it applies to the patient ---> Spend less time but go into detail about when we talk about interventions
****Systematic review
Protocol driven comprehensive reproducible searches that aim at answering a focused question --> so multiple RCT's are evaluated to answer specific question -extensive literature searches are conducted to reduce selection bias of references, to find sound methodology benefits: multiple RCT's are analyzed not just 1 study. (Jadad scale used to evaluate quality of individual RCT. PRSIMA can also be used- a 27 item checklist ) - EXAMPLE of the 2nd to bottom of Hierarchy of Evidence 4S model; part of SYNTHESES "exhaustive review of data on topic" (available in PUBMED)
Odds
RATIO of the probability that an event WILL occur TO THE probability that it will NOT occur
Observational Studies
Researchers carefully and systematically observe and record behavior without interfering with behavior. --most common form is a case-control study.
**Hierarchy of Evidence
SYS come first, YNO, THESE STUDS.
****NOT
Simplest operator Takes single input & produces its opposite as the output
Mycin
Stanford Dr. Ted Shortliffe as focus for his PhD dissertation in 1975. It was designed to behave like an expert medical consultant. Provided diagnostic and therapeutic advice like an expert consultant Infectious diseases (focus to treat these & recc.) Diagnosed causal organism Suggested drug to treat infection Certainty factors-Nothing 100% certain (labs not accurate, s/s common to wide range of dx) Created so that the rules in the system did not have to rely on perfectly certain data. Performed well compared to experts Model expanded to oncology and other areas Stand-alone system not connected to a hospital information system or medical record system. Data had to be entered manually into the system. Never used in practice Lack of interest from physician so never used in practice Remains most cited examples of these systems
recommendations on consequences on patient safety
Study IT registries Improve publication quality Create incentives for publishing negative studies to counter publication bias Health IT systems on taxonomy Improve indexing of health IT systems taxonomy Improve index of HIT eval papers Migrate from meta analysis to emta summaries→ synthesizing both qualitative/quant studies Include HIT eval competencies in curricula Create frameworks from EB implementation Stablish post-market surveillance for health IT
**Centre of EBM defines EBM
Systematic approach to clinical problem solving which allows integration of best available research evidence w/clinical expertise & patient values
Usual meta analysis summary stats: (Odds ratio) OR
Used for binary events, e.g., death, complication, recurrence Usually configured such that OR < 1 indicates treatment benefit (COCHRANE APPROACH) If confidence interval (CI) does not cross OR = 1 line, then results are statistically significant Can calculate number needed to treat (NNT) from OR
Usual meta analysis summary stats: (Weighted Means difference)
WMD Used for numeric events, e.g., measurements Usually configured such that WMD < 0 indicates treatment benefit If CI does not cross WMD = 0 line, then results are statistically significant
examples of background questions
What causes pneumonia? When do complications of type I diabetes usually occur? At what stage of the disease do the complications of type 1 diabetes usually occur? In what order do complications such as neuropathy, heart disease, and kidney failure generally occur in type 1 diabetes?
****Alert fatigue
When an excessive number of alerts are used in an information system, users get tired of looking at the alerts and may ignore them -Defined as simply disrupting clinical workflow -Can result in dismissed alerts -Contributes to resistance to CDSSs "
Case Reports/case series (SERIES like a show you collect but with out the REMOTE (CONTROL) , so it sucks)
collections of reports on treatment of patients without control groups; much less scientific significance -the last choice of study for HARM -No comparison group
Screening test for disease (a variation on a DIAGNOSIS clinical question using EBM) define. Requirements?
definition-" identification of unrecognized disease" Aim-to keep disease (or complications) from occurring (primary prevention) or stop progression (secondary prevention) Requirements for a screening test: 1-Low cost 2-Intervention effective—ideally shown in randomized controlled trial 3-High sensitivity—do not want to miss any cases; usually follow up with test of high specificity Ex: false-positives; positives usually f/u w/high spec. To reassure. -(American love anyways although harmful (breast cancer) (prostate cancer) -ovarian cancer 43% had atleast 1 FALSE POSITIVE -contributes to medical spending -
Risk
describes probability an adverse event WILL occur
Summarizing Evidence
does NOT mean collecting & combining findings, rather, methodological challenges
Simulconsult
dx program based on BAYSEIAN Network w/strength in pediatrics, genetics, neuro, rheumatology
****Decision analysis
enables us to explicitly lay out the factors that go into decision making and assign numerical values and calculate a value and quantitative measure to guide decision making. Applies a formal structure for integrating evidence about beneficial and harmful effects of treatment options with associated values and preferences They can be applied to guide decision making of single patient or to inform decisions about clinical policy allows elucidation of framework for making optimal decisions -help standardize, improve care, lower cost -will increase w/use of EHR & Quality improvement effots
Example studies of HARM
examples: "Do silicone breast implants cause autoimmune disease? such as lupus? (got connective tissue dz but not at higher rate than no boobjobs" Do anti-obesity drugs (fenlafuramine & phenteramine ? cause heart valve abnormalities?" (yes they did. at higher rate than those who did not.
Methodology Expert & process of CPGS brief
experts in EBM, epidimology, statistics, cost-analysis, . Panel--> PICO--> disclose any conflicts of interest--> evidence synthesis--> evidence graded--> reccomendations (benefits, harms, cost)--> use rating system--> voting needed (development process made by National Academy of medicine and GUidelines International Network
snap dx
free mobile app that dx CDS for clniicans based on positive and negative ratios
Info buttons (standard)
function of linking pt data to general information web-linked icons that permid downloading of guidelines, articles, monograph canonical
2 main types of basis clinical questions
harm prognosis
CDS Architecture
how CDS constructed how ACCESSED how SHAREABLE the knowledge base is 4 phases: 1)stand alone-internist 2)Integrated-Abx assistant 3)STANDARDS-based-Arden syntaxrmr 4)SERVICE models
corollary orders
orders or information like "ordering getmaycin sulfate " (draw levels pre & 1 hour post third dose?? yes or no?)
neural networkds
popular approach and capable of both supervised & unsupervised machine learning networks arranged in layers
examples of studies of prognosis
preterm babies- UK & ireland born @ 25 or fewer weeks of gestation VS full term babies --> 43% had serious impairment and control group only had 1.3% untreated early localized prostate cancer -men were followed from 77 to 84 17% developed GENERAL disease and 16% DIED of disease
secondary prevention
prevent complications from occuring when disease already present
National Guideline CLearinghouse
program initiative of Agency for Healthcare Research & quality (AHRQ) and is larges , most comprehensive of alll CPGS resources
Meta-analysis
quantitate summary of systematic reviews that take systematic review step further by using statistical techniques to combine results of several studies as if they were one large single study (pooling & exploring heterogenity) 2 advantages: -larger # of events = more precise -results apply to wider range of patients because inclusion criteria of all included studies
examples of communication mechanisms
see M7 notes
CQL clinical aulity language
standard HL& draft
Bayes Theorem in diagnosis
statistical formula that gives the post-test probability (posterior probability) of disease being present. Post-test (posterior) probability a function of pre-test (prior) probability and results of test Post-test probability variable increases with positive test and decreases with negative test know what the prior or pre-test probability is because that information is used to calculate a new probability when test results are added
Bayesian networks
structures use forms of bayes theiorem (conditional probs) to calculate (posterior) probs of dz baed on pretest probs prev of each disease
Efficacy studies (brian haynes)
studies that investigate whether something works BUT take place under "ideal" circumstances Ex: randomized controlled trial
Cohort Studies (like in my nursing COHORT)
study patients w/specific conditions (retrospective or after the fact) & compare w/ppl who do not. -less reliable than RCT & cohort studies because showing a statistical relationship does not mean 1 factor necessarily caused the other Harm- (2nd choice): 1-Prospective study WITHOUT randomization 2-Particularly useful when poor outcomes are rare and huge sample size would be required—e.g., upper gastrointestinal hemorrhage with NSAIDs 3-Problematic when groups are dissimilar—e.g., people who take NSAIDS may be sicker than or otherwise different from those who do not (no benefit of randomization)
internal validity
study poorly designed
SYNTHESES
the 2nd to bottom level of the pyramid (has systematic reviews & evidence reports) performed due to lengthy-ness of SR/Evidence reports *growing trend towards systematic reviews or EVIDENCE reports to bring together all of evidence -found in CDSR (cochrane collaboration on INTERVENTION Q'S), & PUB MED -a source of summarized evidence *bring together primary data
****Some Unintended Consequences of Clinical Decision Support System, as described by Joan S. Ash et al.
unintended consequences RT to CONTENT: 1)elimination or shifting of human roles 2)currency of CDS content--- updating 3)wrong or misleading CDS content 4)medication ordering 5)alert content problematic 6)medication reconcilation UC RT to presentation: 1) presentation to user 2)Rigiditiy of systems--> needs numerical entry -interuppted workflow 3)alert fatigue 1a-drug 2 drug interactions ignored 2b-weight based --- cant be incorporated, difficult to override, OVER ALERT, 4d-d/c lab, pop up occurs 5e-clicking okay & not reading 6f-seems every drug has an alert 4)sources of potential errors -timing 5)updating can lead to errors- eliminating in a pick list by accident or making typo in some drug name (alerts cry wolf)
alerts and reminders
warn clinican about potentiol problems during or after patient visit pop up boxes triggered by ERx ordering of lab/images/return of results interruptive or non-interupptive
Evidence appraisal
when evaluating evidence, one needs to assess its validity, results, & applicability
