research final

Heterogeneity Continuous Outcome

"Combining apples and oranges" Reflects the variation of results of studies included in meta-analysis Generally want low heterogeneity and lesser variability in results Chi-squared statistic P-value less than 0.05 = statistically significant heterogeneity Cochrane's Q statistic Tests the hypothesis that results across studies are homogeneous Only informs the presence or absence of heterogeneity Does not report the extent of heterogeneity Higgin's I2 statistic or index Quantifies the degree of heterogeneity in meta-analyses Represents amount of variability in the effect sizes across studies that can be explained by between-study variability Reported as a percentage between 0% and 100% Low (25%), moderate (50%), and high (75%) Significant statistical heterogeneity is considered to be present if ≥50% Greater I-squared percentage = greater heterogeneity

Application Review

FDA has 60 days to file the NDA for review If the FDA files the NDA, a review team (advisory committee) is assigned to evaluate the research FDA evaluates the professional labeling and manufacturing facilities Reviewers will approve application and issue a complete response letter

mue

1. Familiarize yourself with the medication in question. I consulted the 2016 Update of the Clinical Practice Guideline for the Management of Candidiasis by the Infectious Diseases Society of America (IDSA) and highlighted every scenario where it was appropriate to use micafungin. I kept the appropriate uses of micafungin in the back of my mind as I was looking through patient charts and quickly identified any "red flags." 2. Create an Excel spreadsheet. My column headings were: Patient name/medical record number, ordering physician, diagnosis for admission, indication for micafungin use, culture +/-, dosing schedule, micafungin start/stop date, total length of therapy, empiric therapy (yes or no), de-escalated or discontinued (yes or no), reason for discontinuation (if applicable), and overall appropriate therapy (yes or no). 3. Review patient charts thoroughly. Keep your answers simple and easy to understand. Initially, you may have to spend at least 30 minutes to 1 hour on each patient chart, especially if you have a MUE with gray areas. In my project, it was often debatable whether or not the length of therapy was appropriate because the proper time frame of micafungin treatment is a broad range that can be modified based on clinical judgment, according to IDSA clinical guidelines. Be prepared to go back through patient charts after discovering other possible questions that need to be answered. 4. Write a short summary for complicated patient cases about their hospital stay. This will help you organize key details when you are presenting the MUE to your preceptor. 5. Proofread and refine your chart. Schedule an appointment with your preceptor to review your findings. Consider meeting with nurses, hospitalists, or other clinicians who may have seen your patients or frequently order the medication. 6. Compile and evaluate data. By this stage, you have made a final clinical decision on whether or not the therapy was appropriate for each individual patient (and your preceptor has agreed with your decisions). Your data should answer some of the most important questions asked by your preceptor or the director of pharmacy, such as: In what percentage of the cases was the micafungin use appropriate? It was determined that the use of micafungin was appropriate in 78% of cases. 7. Build your presentation! This is the opportunity to present your findings to the department of pharmacy and/or hospital staff. Was there a need for more pharmacist intervention? For example, in three of the patient cases, I found that the incorrect dose of micafungin was used for patients with endocarditis, and a pharmacist could have intervened to ensure the correct dose was being used. In my presentation, it was especially crucial to highlight how pharmacists could coach hospitalists to only use micafungin in high-risk Candidemia, to de-escalate to fluconazole whenever possible, and to use the right doses based on different disease states.

Different Types of Case-Control Designs

Hospital- or Clinic-Based Case-Control Cases have the disease Controls happen to receive care at the institution Population-Based Case-Control Has a defined base population Draws samples from a potentially more diverse population, which increases generalizability (external validity) Example: Nested Case-Control

Medication Use Evaluation

A performance improvement method that focuses on: Evaluating and improving medication-use processes Goal of optimizing patient outcomes. MUE may be applied to a medication or therapeutic class, disease state or condition, a medication-use process (prescribing, preparing and dispensing, administering, and monitoring), or specific outcomes Ultimate question - which drug is best for my patient? What medication should be kept "on formulary" at my hospital/clinic/pharmacy? What would I recommend for treatment? DUE/MUE are designed to accomplish the following objectives: Ensure drug therapy meets current standards of care Create guidelines (criteria) for appropriate use Enhance responsibility/accountability in the drug use process Control drug costs Provide optimal medication therapy Prevent medication-related problems Evaluate the effectiveness of medication therapy Identify areas in which further information and education may be needed for health care providers

Research Protocol

A standardized document Covers all aspects of a study Provides specific details Leads to standardization for implementation across investigators and sites For Investigational New Drugs (IND), FDA must approve all clinical protocols before administering the investigational agent to humans All protocols involving human subjects must be approved by an Institutional Review Board (IRB)

New Drug Application (NDA)

Submitted to the FDA after Phase III trial Includes all data from animal and human testing, as well as pharmacokinetic and pharmacodynamic information and manufacturing processes

Fixed Effects Model

Assumes that variability across studies is due to random variation Assumes that there is a single true treatment effect and that all trials provide estimates of that true effect Assumes that all studies represent the same population, intervention, comparator and outcome Assigns weight to each individual study

Random Effects Model

Assumes there is a different underlying effect for each study, and the analysis takes this into consideration Assumes each study there are multiple true treatment effects related to differences in populations and that each trial provides an estimate of its own true effect Incorporates both "between study variance" and "within-study variance" Main difference between fixed effects and random effects is the method used to calculate the total overall effect Random effects meta-analysis produces wider confidence interval for the total overall effect than a fixed effects less accurate total overall effect size

Study Measurements

Baseline measurement Information collected prior to randomization (demographics, co-existing conditions, meds, social history, etc.) May be used for stratification purposes to ensure potential confounding variables are evenly distributed Process measurement measures degree of adherence to study protocol Outcome measurement An RCT should have only 1 primary outcome (usually efficacy); measured objectively The outcome in part determines sample size and the hypothesis. Why? E.g. "The urine albumin concentration in the first morning urine sample using a specified urine detection assay at 30 days after beginning drug therapy. Biomarkers Measures of disease progression; secondary outcome or "surrogate" measure Linked to underlying disease process but do not directly measure disease activity E.g. measuring LDL cholesterol as a marker for heart disease. Safety monitoring Extremely important - adverse events are documented and reviewed looking for a temporal relationship and likelihood of association Could lead to discontinuation of study if serious side effects emerge Sometimes use an independent data and safety monitoring board (DSMB)

Minimizing Bias

Bias is a systematic error in study design that can lead to incorrect findings It is a tendency of measure to deviate in one direction from a true value Can lead to disastrous underestimation or overestimation of the true effects of the intervention Because true results are always unknown, it is never really possible to fully eliminate bias

Bias vs. Confounding

Bias= systematic deviation of a study's result from the truth Usually introduced during the design/implementation and cannot be remedied later Examples: selection bias and information bias Confounding = the creation of a relationship that may be factually right, but that cannot be interpreted causally become some underlying, unaccounted for factor is associated with both the exposure and the outcome Example:

Blinding

Blinding (or masking) is designed to minimize bias (particularly ascertainment, measurement and detection bias) Single blind One of the three categories (usually study subjects) do not know which arm of the trial they have been assigned to Double blind Both the study subjects and the researchers are unaware of the randomization schedule; necessary in phase III trials Triple blind This is the most objective design; subjects, investigators and those collecting the outcomes data are all unaware of the randomization schedule; the data collection group is usually an external group Open label Everyone is aware of the randomization plan; usually restricted to phase I or early pharmacokinetic studies

Relative Risk

Can only be used in prospective studies (need to know the baseline risk) Calculated by dividing the incidence rates in each group (exposed ÷ nonexposed) Can't distinguish between a small and large true difference in groups Can overestimate the risk

Selection of Cases and Controls

Cases Source population is defined based on how you are sampling and the eligibility criteria Example: patients in a specific geographic location, having a particular insurer, or receiving care at a particular health facility Search existing records Claims data, disease registries, medical records Incident cases= newly diagnosed Usually preferred because you have an easier time establishing your timeline between disease and exposure Prevalent cases=known existing disease Controls must represent general population

Unpublished Data

Drug information that has not been published in medical literature Why is it important? Minimizes publication bias Published studies are more often of a positive nature than unpublished studies About 50% of studies submitted to the FDA in support of drug approvals are not published after 5 years Includes more data from which to draw a conclusion, but has not been peer-reviewed Overall result is published data overestimates efficacy and underestimates harm

Dichotomous data Dead or alive Response or non-response Meeting goal vs. not meeting goal

Continuous data Mean change in clinical measure: blood pressure, total cholesterol Change in score on symptom scale

Other Observational Studies

Cross-sectional studies (AKA Prevalence studies) All individuals in a sample at the same POINT IN TIME Examines the prevalence of exposures, risk factors, or disease Analyzed like cohort studies by comparing disease prevalence between exposure groups May also work like case-control studies by comparing odds of exposure between groups with or without disease Address the issue of establishing when an exposure preceded the disease Not ideal for studying rare diseases Not to be confused with crossover study design in RCTs

Continuous Outcome

Data presented can help calculate an overall, pooled weighted mean difference for comparison of Drug A and B Statistical calculation is not as simple as an "average" Meta-analysis will "weight" each study Inverse proportion to its variance Heavier weighting for larger, more precise studies, and down-weighting for smaller studies with lower confidence Allows for adjusting for the degree of variance associated with each study

Methods

Design of the study Large amount of information that includes: Type of subjects enrolled Comparative therapy description Outcome measures Statistics Poor study design = reduced internal validity Majority of critique comes from methods section

External Validity

External validity indicates that the findings of a given study can be generalized to other settings or populations For example, can a study in the Scandinavian Cardiovascular Journal in a small population of medication-naïve patients be generalized to patients with chronic hypertension in the USA?

Crossover

Each subject receives all interventions based on specified sequence of events Takes longer but has greater statistical power since each subject participates twice Often has high dropout rate Carryover and Washout (can create problems in crossover design) Carryover effect - effects that linger after the first phase of the crossover Washout period - time needed for outcomes of first phase to dissipate prior to beginning second phase of the crossover Especially problematic if drug has a long half-life Can lead to increased drop-out rates in subjects enrolled

Designing the Intervention

Effectiveness vs. Efficacy Efficacy (phase III studies) measures physiology, survival and quality of life in near-perfect setting Effectiveness measures how well the intervention performs in the clinical setting (real world) Safety Measures of safety are chosen based on preclinical toxicology studies Dose-response study - relationship between dose of the drug and the efficacy or toxicity Adherence Adherence (or lack thereof) can have significant effects on a trial Taking accurate dose histories is an important aspect of the monitoring of clinical trials Pharmacists often assist in this aspect of the study by implementing measures to improve adherence (education, counseling, adherence aids)

Discussion

Evaluate and/or interpret the results Begins with summary of key findings Discuss internal and external validity Comparison of study results to other trials Discuss limitations of study Clinical importance of results and how they can be used in practice Conclusions should align with results of the trial

Continuous Outcome

Example: Systematic Review identified 5 studies Compare treatment A with treatment B in patients with a given disease Outcomes assessed with symptoms scale (0 to 50) Lower scores = less symptoms; higher scores = more symptoms Endpoint is 12 weeks Do we include endpoints past this mark?

"Exposure"

Exposure is typically used to describe an innate trait, contact, or experience with a potential risk or protective factor Outcome examined is exposure to treatment of interest versus another that is unexposed Cohort design parallels clinical trial Treatment groups are formed based on exposure Unlike clinical trial, this exposure is not randomized

Study Design

Factorial Design Evaluates multiple interventions in a single experiment; e.g. different doses, different drugs Group or Cluster Randomization All members of the group or cluster are randomized together Adaptive Design Conditions of the study or analysis plan are prospectively planned to be modified over time based on results of interim analysis at pre-specified time points Can reduce the number of participants needed

Data Synthesis Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)

Flow diagram used to present all information obtained Shows flow of information through the different phases of systematic review How studies were identified Results of abstract screening Results of full text eligibility assessment Reasons for exclusion of studies Details of included studies

FDA

Food and Drug Administration (FDA) Federal agency that decides which drugs, biologics, and medical devices are marketed in the United States Monitors the manufacture, import, transport, storage, and sale of $1 trillion worth of goods annually Centers of the FDA: Center for Biologics Evaluation and Research (CBER) Center for Drug Evaluation and Research (CDER) Center for Devices and Radiological Health (CDRH) Since 1940, more than 1000 new molecular entities (NME) have been approved An estimated $802 million is spent to get a new drug product on the market Range from $500 million to $2 billion Most products that undergo preclinical (animal) testing do not make it to testing in humans Very important that clinical trials conducted for drug approval are accurate and complete Pharmacists may play a role in this process and ensure clinical trials meet the goals set forth by the study sponsor, local investigator and ultimately the FDA

GRADE

Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group Developed system for grading the strength of the body of evidence (rather than component studies) Risk of bias Consistency Directness Precision Overall strength is then characterized Insufficient Low Moderate High

Internal Validity

Internal validity is the degree to which the outcome (efficacy or safety) can be explained by differences in assigned intervention (treatment) The process of controlling the study design factors, implemented by the scientist before conducting the study is often referred to as the internal validity of the study One way to strengthen internal validity is to include a control group Best way to strengthen internal validity is to randomly assign study participants (randomization)

Continuous Outcome

In Study 1 the mean change for Drug A is -14.1 and the mean change for Drug B is -11.8 So difference in mean change for Drug A vs. Drug B is -14.1 minus -11.8, so the difference is -2.3 Interpretation Drug A has a 2.3 greater improvement (since lower scores are better) in symptom control than Drug B

Acknowledgements

Individuals contributing to the clinical trial Financial support Publication bias

Five major types of bias

Investigator bias Errors in study design, implementation or data analysis Ascertainment bias: those analyzing the results are aware of which subjects in control or intervention group Minimized by blinding Selection bias Preferential enrollment of specific patients into one treatment group over another Patients do not have equal chance of being allocated to groups Combat this bias through strict adherence to randomization scheme Performance bias Systematic differences in care between treatment groups, or in exposure to factors other than the intervention being studied Treatment group may get more attention because of the nature of the treatment Minimized by blinding Attrition bias Differential dropout of patients in treatment and control groups Often seen in the treatment group due to non-compliance with medications Can be minimized using the "intention-to-treat" method Patients are analyzed as if they completed the study in their originally assigned group Detection bias Systematic differences between groups in how outcomes are determined Can occur if investigator knows treatment allocation, therefore an objective assessment cannot be made Can lead to an overestimation of treatment effect Minimized by having non-study personnel assess outcomes or by appropriate blinding

Quality Rating

Jadad Scale Assess methodological quality of clinical trials Three questions: Was the study described as randomized? Was the study described as double blind? Was there a description of withdrawals and dropouts? Additional points given if: The method of randomization was described in paper and was appropriate The method of blinding was described in paper and was appropriate Zero = very poor; five = rigorous

Internal/External Validity

KEY CONCEPT!!!!!!!!! Internal validity is the degree to which the outcome can be explained by differences in the assigned interventions/treatment groups. How effectively and appropriately a study examined what it was intended to examine. Internal validity is strengthened by randomization. External validity means the findings of the results can be generalized, or applied to other settings or patient populations.

How to Minimize Confounding

Matching cases with controls Controls are selected to parallel selected characteristics of cases to reduce the influence of factors other than the exposure of interest Example: match a white male case with a white male control of similar age Can be performed up to 1:4 2 Main Benefits: Reduces potential for confounding More statistically precise estimation of the association Requires different statistical analysis than a simple Odds Ratio for an unmatched case-control study design Mathematical Modeling wwwn.cdc.gov/epiinfo Odds Ratio calculations for this course will be for unmatched case-control study examples

Phase IV Trials

May be requested by the FDA or initiated by the sponsor to gather more data on safety and efficacy Long term safety and efficacy data Rare adverse effects may develop FDA may initiate further restrictions on medication uses to ensure that benefits outweigh risks Risk Evaluation Mitigation Strategies (REMS) Can be required before or after a drug is approved Developed by drug sponsors FDA reviews and approves them May include: Medication guide (patient package insert) Communication plan for health care providers Elements to assure safe use (ETASU) - training, limitations on use, monitoring, enrollment of patients in a registry REMS must be assessed for adequacy at least by 18 months, 3 years, and 7 years after approval

Meta-Analysis

Meta-analysis of studies identified in systematic review Included in systematic reviews Statistical analysis that quantitatively includes all data derived from included studies Used to provide overall estimate of benefit or harm Benefits: Conclusions may be more generalizable Single randomized controlled trial = may limit application to other settings or individual patients Combining studies in meta-analysis increases sample size Benefits clinicians May not have time or resources to gather and evaluate a large body of evidence Focus on a narrow, clearly defined topic Includes ALL eligible studies

Sample Size

Must be determined before the study is conducted The goal is to determine the appropriate number of subjects that are needed to test the primary study hypothesis Important Considerations Effect Size The degree of difference between treatment groups that is clinically important; e.g. a BP of 10 mm Hg Sample size takes effect size into consideration Power Capacity to detect a difference in the study groups if a true difference exists Studies are typically designed to have 80% power If a study is not sufficiently powerful, we run the risk of a "False Negative" study, or a type II error Finding of "no difference" can be explained by insufficient number of subjects

Interpreting the NNT/NNH

NNT = 1 ÷ ARR NNH = 1 ÷ ARI NNT should be low (ideal <10) NNH should be high (ideal >200) NNT should be < NNH Exceptions can include: minor harm, benefits significantly outweigh the harm (e.g., severe nausea with a chemotherapy agent), few treatments exist

Analytical Approach-Number Needed to Treat (or Harm)

Number needed to treat describes effectiveness of an intervention Refers to the number of patients that must receive the treatment in order for one patient to experience a desired outcome 𝑁𝑁𝑇= 1/(𝑅𝑖𝑠𝑘 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒) (NNT should be LOW) From previous example: 𝑁𝑁𝑇= 1/0.24 = 4.16 patients Can also calculate a number needed to harm (NNH should be HIGH to be considered a safe medication) The lower the better! Ideal number needed to treat = 1!!! Everyone improves with the treatment! 𝑁𝑁𝑇= 1/(𝑅𝑖𝑠𝑘 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒) A measure of the risk to a patient We want the NNH to be as high as possible, to minimize risk to our patients! NNH = 1/(𝐴𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑎𝑏𝑙𝑒 𝑅𝑖𝑠𝑘)

Cohort Studies - "Observational Associations"

Observational studies are the only ethical way to gain insight into the effects of exposures known to have harmful effects For example, tobacco, illicit drug use, or a nutrient poor diet RCTs, while the gold standard, are still prone to bias and other limitations generalizability conducted in "ideal" settings giving insight into efficacy vs. effectiveness lack adequate power to detect less common events such as elevated risk of cardiac death from azithromycin Intention-to-treat analyses can produce biased estimates

Dichotomous Outcome

Odds of response: Number of patients who responded DIVIDED BY number of patients who did not respond Study 1, Drug A: 180/54 = 3.333 Study 1, Drug B: 134/96 = 1.396 Odds ratio: Calculated as odds of response with Drug A DIVIDED BY odds of response with Drug B 3.333/1.396 = 2.388 Relative risk Alternatively, relative risk can be calculated: Number of patients who responded DIVIDED BY the total number of patients Study 1, Drug A: 180/234 = 0.769 Study 1, Drug B: 134/230 = 0.583 Risk ratio Risk with Drug A over risk with Drug B 0.769/0.583 = 1.320

Abstract

Overview or synopsis of the study DO NOT solely make a decision based on the abstract you should read the entire study Addresses the objective, methods, results and conclusions Primary use of abstract is to determine if the article should be read

Study Design Parallel

Parallel Most common design for Phase III comparative trials Each subject is randomized to either intervention or control group Advantage is "shorter" time to complete trial Typically requires a large sample size compared to crossover trials

Selection of Control Group

Placebo Concurrent Control A placebo is an inert substance that is identical in appearance to the active treatment Active Controls If there is a known or accepted standard of care or treatment, then patients are randomized to either the intervention or active control; this is an ethical consideration (non-maleficence) Historical (External) Controls Comparison to a group observed at a different time or in a different setting; advantage is that all study participants will be in the intervention group Suffer from internal validity concerns and is not a well-accepted study design (bias, lack of blinding) Non-Inferiority Trials Show that the effect of a new treatment is not worse than an active control (usually an FDA-approved drug) by some specified margin

PICOTS

Population Underlying characteristics of the population Age, sex, race, previous treatment, early vs. advanced disease, other conditions, current medications Intervention Drug, medical procedure or other healthcare intervention being assessed Comparator Compared to placebo, compared to usual care or added to existing treatment Outcome Endpoint measurement Should reflect a true health outcome no surrogate endpoints! Timing When the outcome will be measured How long the outcome will be measured Setting How healthcare is being delivered in the population Access and delivery - urban as opposed to rural

Review Meeting

Post-marketing and commitment studies are agreed upon by the sponsor Conducted after FDA approval FDA uses post-marketing data to continuously monitor the safety and efficacy of products on the market

Results

Primary and secondary endpoint results Patient demographics Dropout information Safety information Data often represented by visual interpretation

Forest Plots

Primary mechanism for conveying results of meta-analyses Graphical representation of individual studies and pooled estimate Visually shows results, amount of variation between the results, and estimate of overall result

Odds vs. Probability

Probability of flipping heads is 50% or 0.50 Odds of flipping heads is the ratio of the probability of flipping heads divided by the probability of not flipping heads Odds= 0.50/0.50= 1

Informed Consent

Process by which a potential study subject is presented with information about the study and willingly volunteers to participate Must be presented in an understandable manner Understanding must be verified Must not be under coercion, undue influence, or duress Requirements Study purpose and expected benefits Methods to be employed Anticipated inconveniences and discomforts Potential temporary and permanent risks Contingent compensation All reasonable alternatives Right to withdraw

Bias

Publication bias Most widely acknowledged form of bias in meta-analysis Results from likelihood that studies with positive findings are more likely to be published How can we help overcome this? Unpublished data Grey literature! BUT need to make sure these sources provide sufficient data to assess internal and external validity of study

Phase II Trials

Purpose: Evaluate efficacy of the agent Small number of subjects with the disease or condition that the drug is proposed to treat (100 - 200 patients) Average 2 years in duration Help to develop preliminary evidence before larger trials Continue to evaluate safety

Phase III Trials

Purpose: Further define efficacy and safety of the agent New agent is compared to current therapy Usually multicenter studies that treat 600 - 3,000 patients Average 3 years in duration Some Phase III trials are pivotal studies that serve as the basis for the New Drug Application (NDA) for marketing approval

Meta-analysis

Quantitative synthesis of data derived from individual studies that address a key question Utilized when there are multiple studies (usually ≥3) identified through systematic review Combines results from studies to provide overall estimates

Randomized Controlled Trials"The Best Research"

RCTs provide strong evidence to support evidence-based practice Designed to measure a primary outcome in a highly selective group of individuals that are randomly assigned to one or more interventions Drug therapies, prevention strategies, medical procedures, educational research Designed to determine the effect of a specific intervention on health-related outcomes, disease prevention, or progression of disease Strongest type of study design in clinical research Most common type of trial to compare the efficacy of an experimental intervention compared to a standard therapy or placebo Required in phase III studies or "pivotal trials" Trials establish efficacy and safety under near perfect conditions Required as part of an NDA (New Drug Application) Trial design, adherence to pre-defined criteria, and minimization of bias are key

Interpreting the Relative Risk

RR < 1 = a decreased risk of having the event Called the relative risk reduction (RRR) Calculated as 1 - RR RR = 0.6 means there is a 40% reduction in risk RR > 1 = an increased risk of having the event Called the relative risk increase (RRI) Calculated as RR - 1 RR = 3.4 means there is a 240% increase in risk

Randomization

Randomization is the process of assigning patients to a treatment or control group by chance alone (random.org) Simple Randomization Use of a random number generator; can lead to unequal groups if there is a small sample size which can affect validity of the results Block Randomization Subjects are divided into blocks prior to randomization and then the blocks are randomized Helps maintain equal group size Stratified Randomization Allows for equal subgroup sizes based on certain factors (e.g. age, disease severity, gender, etc.) Not necessary in very large sample sizes Adaptive Randomization Chances of being assigned to the treatment group change based on the responses of the prior patients (e.g. if early results show a marked tendency toward effectiveness) Not a widely accepted technique in the scientific community

Trial Registries and Results Databases

Readily accessible online source for unpublished data www.ClinicalTrials.gov Data is self reported with a set of mandated requirements: Study purpose, recruitment status, design, eligibility criteria More than 100,000 clinical studies Created to promote greater public access to trial data Limitations No mechanism for ensuring adherence Not all trial results will be included Quality of registered information is variable

Regulatory Agencies

Scientific reviews Contain narrative summary of the clinical trials that form the basis for approval of regulated drugs Drugs@FDA

Factors that threaten internal validity

Selection Differences in patient's baseline characteristics (e.g. disease severity or demographics) between the two groups Randomization decreases selection bias History If an external event occurs during the course of the study that can impact the outcome (e.g. a person works at an employer who changes their health insurance, or loses their job, which impacts visits to a physician for regular care) Maturation If a patient changes over the time of the study (e.g. progression of disease from initial entry into study) Randomization decreases this risk, because similar changes should happen to both treatment and control groups Mortality/Attrition Study participants are lost to follow up or withdraw from the study Particularly problematic if occurs in one arm of the study more than the other (e.g. due to side effects in intervention group) Testing If a study participant changes their behavior because they know they are being tested, or self-testing Instrumentation If the testing method changes due to improvements in technology or sensitivity of instrumentation (e.g. different methods for testing antimicrobial MICs) Statistical Regression If repeated testing leads to regression to the mean

Bias in Case-Control Studies

Selection Bias: Eligible population is poorly defined Poor sampling Unequal diagnostics in the target population Example: Detection bias/medical surveillance bias Patients with type II diabetes might have more routine eye examinations and subsequently are more likely to be diagnosed with glaucoma compared with non-diabetic patients Information Bias Systematic differences in the completeness of accuracy of data lead to differential misclassification of individuals regarding the exposure and/or outcome Examples: Recall bias: Cases recall past exposure in different rates than the controls do Observational bias: subjects may change their behavior because they know they are being observed

Drug Approval Process

Standardized by FDA review Preclinical testing Investigational New Drug (IND) application Phase I - IV testing Preclinical testing In vitro or in animals Before filing an IND, must develop a pharmacologic profile of the drug Determine acute and subacute toxicity FDA decides if it is safe to test in humans IND Application Clinical trials in humans may only be conducted after an IND has been approved by the FDA and investigational review board (IRB) IRB approves clinical trial protocol on how the study will be conducted Phase I Trials First use in humans Purpose: Determine safety and toxicity Low doses are used in a small number of healthy subjects (20 - 80 patients) Average 6 months - 1 year in duration Also test various dosage forms and develop pharmacokinetic data

Which of the following statements, if any, are true? a) Not one of the four trials showed a significant difference between antibiotic treatment and appendicectomy in the risk of complications. b) The forest plot is drawn on a linear scale. c) A relative risk less than 1.0 represents a reduced risk of complications for antibiotic treatment compared with appendicectomy. d) The meta-analysis of complicationsshowed a relative risk reduction of 31% for antibiotic treatment compared with appendicectomy. e) No significant heterogeneity existed between the sample estimates of the population relative risk.

Statements a, c, d, and e are all true, whereas b is false For each treatment group in the four trials, the number of participants who experienced a complication and the total number in each group are shown in the column headed "Events/total." These data were used to calculate the estimated relative risk and 95% confidence interval, shown on the right for each trial and represented graphically in the centre. Because the data for "Events/total" for antibiotic treatment are presented first, followed by those for appendicectomy, the presented relative risks therefore represent the risk of complications for antibiotic treatment relative to appendicectomy. The sample relative risk is represented by a square and its associated 95% confidence interval by the horizontal line. The size of each square is proportional to the sample size of the trial. For all four trialsthe 95% confidence interval for the population relative risk included 1.0, and therefore not one of them demonstrated a significant difference between treatment groups in the risk of complications (a is true). The relation between the 95% confidence interval for a relative risk and 5% level of significance when hypothesis testing has been described in a previous question. The graphical representation of the sample relative risks and associated 95% confidence intervals are plotted on a logarithmic scale (b is false). As the 95% confidence intervals were originally calculated on a logarithmic scale, they therefore appear symmetrical about the sample relative risk in the forest plot. The solid vertical line in the centre of the graph isthe "line of no effect"—that is, a relative risk of 1.0, which represents no difference in risk between antibiotic treatment and appendicectomy. A relative risk smaller than 1.0 would imply that the risk of complications was reduced for antibiotic treatment, relative to appendicectomy (c is true), whereas the risk would be increased if the relative risk was larger than 1.0. Therefore, as indicated on the forest plot, a relative risk less than unity "favours antibiotic treatment," whereas one greater than unity "favours appendicectomy." A total overall estimate of the population relative risk was obtained by pooling the relative risks from the four trials. However, the trials did not contribute equally to the pooled result, as the total estimate was not an average of the individual estimates. The contribution of each trial is indicated under the heading "Weight (%)." The percentage weight contributed by a trial is determined by the precision of its sample estimate of the population parameter, and trials with more precise estimates—those with narrower confidence intervals—contributed more. The total overall estimate of the population relative risk is presented in the row labelled "Total" and is given as 0.69 (95% confidence interval 0.54 to 0.89). It is graphically represented by a diamond; the centre of the diamond equals the total overall relative risk, whereas the extreme points indicate the limits of the 95% confidence interval. The vertical dotted line through the centre of the diamond and graph represents the overall estimated relative risk. Therefore, the meta-analysis of complications showed a relative risk reduction of 31% for antibiotic treatment compared with appendicectomy (d is true). As the 95% confidence interval did not include 1.0, the total overall estimate was therefore significant at the 5% level of significance—that is, there was a significant difference between treatments in the risk of complications. The P value for the test of significance of the total overall estimate was 0.004, as shown in the text "Test for overall effect: z=2.91, P=0.004," corroborating the inference from the 95% confidence interval. The value for z is the test statistic resulting from the statistical test used to derive the P value. A meta-analysis must incorporate a statistical test of heterogeneity to assess the extent of variation between the sample estimates. Statistical tests of heterogeneity have been described in a previous question.5 Statistical homogeneity would have existed in the above example if the sample relative risks were similar in magnitude and the variation between them was no more than expected when taking samples from the same population—that is, any variation between them was minimal. If statistical homogeneity did not exist, then statistical heterogeneity would be present, and the sample estimates would differ substantially. The result of the statistical test of heterogeneity influences how the total overall estimate would have been obtained. Furthermore, the presence of heterogeneity might suggest that the relative risk of complications differed between subgroups in the population. The results of the statistical test of heterogeneity are shown in the text "Test for heterogeneity: χ 2=1.08, df=3, P=0.78, I 2=0%." The test is performed in a similar way to traditional statistical hypothesistesting, there being a null and alternative hypothesis. Simply, the null hypothesis indicates that homogeneity exists, while the alternative hypothesis states that heterogeneity is present. The P value for the test of heterogeneity was 0.78, indicating that there was no evidence to reject the null hypothesis in favour of the alternative. Therefore, the conclusion was that homogeneity existed between the sample estimates (e is true). The value for χ 2 is the test statistic resulting from the statistical test used to derive the P value. The value for degrees of freedom ("df") equals the number of trials minus one and is used along with the test statistic to calculate the P value. Higgins I 2 statistic is often also used to test for heterogeneity. This statistic represents the percentage of variation between the sample estimates that is due to heterogeneity. It can take values from 0% to 100%, with 0% indicating that statistical homogeneity exists between the sample estimates. Significant statistical heterogeneity is often considered to be present if I 2 is ≥50%. The value of I 2 in the above example is 0%, corroborating the inference of the statistical test that statistical homogeneity existed (e is true).

Discussion

Statistical significance versus clinical difference NOT ALL p-values are clinically important!! Make a decision in determining if the intervention is worth using instead of the control therapy ALL p-values should be presented in conjunction with associated endpoint results CORRECT: the clinical trial concluded that drug A lowered mean DBP more than placebo, -12 versus -3mmHg, respectively [p=0.001] NOT CORRECT: the clinical trial concluded that drug A lowered mean DBP more than placebo [p=0.001] Should be prompted to ask: "Were the endpoint results not presented with the p-value because the actual difference in the effect was minimal?"

Relative Risk & Risk Difference

Statistical tests to rule out chance variation Simplest way to report the difference between the groups is relative risk or risk difference 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑟𝑖𝑠𝑘= (𝐴/((𝐴+𝐶)))/(𝐵/((𝐵+𝐷))) 𝑅𝑖𝑠𝑘 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒= 𝐴/((𝐴+𝐶)) −𝐵/((𝐵+𝐷))

Strengths and Weaknesses of RCTs

Strengths Methodologically strong research design Prospective design allows tight control of design, setting, environment Can infer causality in some cases - no other design can do this Weaknesses By definition, the design is limited to specific patients and therefore cannot ensure generalizability Very expensive and time-intense Ethical concerns due to randomization High dropout rates Hawthorne effect Patients performing better in other areas of their life because they are being studied

Factors that threaten external validity

Subject Selection For example, studies of military veterans will involve mainly older males Is the group of patients included in the study representative of the patients that I see in my clinic every day? Pretesting If a pretest is required for entry into a study, those who took the pretest may respond differently to the intervention than those who, in clinical practice, may not take the pretest. The Research Setting If the study occurs in a highly specialized or artificial setting (EFFICACY) Hawthorne effect: patients modify behaviors (become more compliant as a result of longitudinal learning that takes place over the course of the study, or because they know they're being observed (e.g. eating a better diet while in a study for a diabetes medication) History Results in the past may not apply today due to changes in the healthcare system, access to care, standards of practice, or socioeconomic issues Example: surfactant use in neonates in the 1980s Multiple treatments If the treatment protocol is complex involving multiple treatments, it may not transfer easily into the clinical setting Clinical trial is "near perfect"

Selecting Participants

Target Population The group of people with the desired clinical and demographic characteristics that will ultimately benefit from generalization of the study (i.e. the largest sample size) Study Sample A more specific subset of the target population that is accessible to the investigators and that participates in the study Inclusion/Exclusion Criteria Inclusion - the specific characteristics that the investigator is most interested in studying Exclusion - factors that would confound or impair the ability to interpret the study results (also safety factors - e.g. if a drug was known to have nephrotoxicity, we would exclude patients with significant kidney disease) Recruitment Must be based on ethical principles Recruitment depends entirely on the inclusion and exclusion criteria Patients may take an active role in seeking out participation through www.ClinicalTrials.gov published by the NIH Advertisements are sometimes used

Maximizing Validity

The validity of a study refers to the degree to which the findings are correct. Face validity - on the surface does it seem correct? Content validity - does it appear correct to experts? Criterion validity - does it correlate with other measures or events (e.g. does it match up with a "gold standard" - concurrent validity) Construct validity - does it match up with theoretical constructs This is the highest level of validity

Case-control and cohort studies are categorized as observational epidemiologic studies in the literature

They inform patients, clinicians, and policy makers on a wide variety of subjects Effects of drugs Influence of pharmacy services on patient outcomes They may be a more ethical means to gain insights over a randomized controlled trial (RCT) They do offer useful information on use of an intervention in a typical setting Illuminate rarer disease states and under-represented populations It is still a stretch to use observational studies to establish causation!

Analytical Approach - Risk Ratios

Three main types used in studies: relative risk (RR) - for prospective studies odds ratio (OR) - for retrospective studies hazards ratio (HR) Calculated by dividing the statistic in the exposed group by the statistic in the non-exposed group (control group)

Introduction

Two purposes: Discussing the study rationale What do we already know on the subject studied? Issues that are the basis of the study Study purpose What are our outcomes? Study objective stated Null hypothesis: no difference between groups Research hypothesis: difference between groups under investigation Not always included

Dichotomous Outcome Example

Using the previous example with a continuous outcome measure, assume that for the purposes of measuring symptom improvement in clinical studies, the scale is often categorized so that scores less than 15 are classified as responders to treatment Now the data can be treated as dichotomous (i.e., response or non-response) Data may be used to calculate odds ratio (OR) or relative risk (RR)

Summary and Conclusions

Well-designed observational studies can provide useful information regarding the effects of drugs, effect of exposures, and incidence of disease They cannot replace the level of evidence provided by randomized controlled trials Case-control studies are conducted by selecting cases, identifying a set of controls, and looking back in time to identify differences in exposure. Researches need to be aware of potential biases that influence the validity of these types of studies

Analyzing the Results

When to analyze Endpoints analyzed after all assessments are completed Interim analysis plans are included for an "escape strategy"; decision to conduct an interim analysis done by the DSMB at pre-specified intervals How to analyze Intention-to-treat analysis Patients are analyzed as if they completed the study; this helps to minimize attrition bias On-treatment (per protocol) analysis Used when only those subjects who are completers are analyzed Introduces risk of bias Benefit is that it allows for a broader sensitivity analysis where using other analyses threaten internal validity Interim analysis Used to monitor safety and maintain ethical standards Goal is to include the minimum number of subjects required for valid results of the primary objective and not expose subjects to unnecessary risk or delays in treatment Subgroup analysis Analyzing the primary outcome dependent on specific demographic (or other)factors. What type of error can be produced and why?

Odds Ratio (OR)

ad/bc

Medication-use evaluation (MUE)

evaluating and improving medication-use processes with the goal of optimal patient outcomes.

Interpreting Risk Ratios

𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑟𝑖𝑠𝑘= (𝐴/((𝐴+𝐶)))/(𝐵/((𝐵+𝐷))) Ratio of 1 means there is no difference between the groups Ratio < 1 means the risk is decreased in the exposed group Ratio >1 means the risk is increased in the exposed group