Module 4 - Part 2
What would you use run charts for a competing event for?
*Are the changes being made what is affecting the change or is there a competing event?
How do you create a Histogram?
1. Accumulate at least 25 data points 2. Rank data points from smallest to largest 3. Calculate range of data by subtracting smallest value from largest 4. Estimate number of bars to be displayed 5. Determine width of bars by dividing range by calculated number of bars 6. Draw vertical axis to show number of times a value of data falls with each bar 7. Draw horizontal axis to show number of times value falls with each bar 8. On left, draw bars with heights equal to number of times data fall within bounds of each bar Label histogram
When is a frequency plot (histogram) used?
1. Have a data set of related values (i.e. arrival times in ED) 2. Want to see central location, shape, spread of data to learn about system } Does all of process fit within needs? (Our standards) Any patterns that bear looking Into?
How do you create a Pareto Chart?
1. Identify independent categories and way to compare, either by frequency (count), time, cost, other unit of analysis 2. Rank order data in descending categories 3. Calculate percentage of total each category depicts 4. Draw left (vertical) axis with unit of comparison 5. Draw horizontal axis with categories from largest to smallest 6. Draw a bar for each category 7. Draw right vertical axis from O to 100 8. Draw a line graph of cumulative percentage 9. Label axes and diagram
What are the 6 Histogram shapes?
1. Normal 2. Bimodal 3. Skewed 4. Truncated 5. Outliers 6. Uniform
How to Create a Scatter Diagram?
1. Select the two measures of interest 2. Record pairs of measures (patient's satisfaction and patient's wait time) on a graph 3. Plot pairs on a scale so that the range of variation takes up the full range of data 4. Axes should be of approximately equal length for each variable Label scatter diagram
How do you interpret a Histogram?
A. Vertical axis — Frequency of data points in each class B. Frequency — Number of data points in each class C. Class — Each bar is one class D. Modal Class — Class with highest frequency Horizontal axis — Scale of measure
What are Reporting Measures?
Additional examples of reporting include: • National patient safety goals • Other safety indicators • Infection control information to Centers for Disease Control and Prevention (CDC) • Healthcare Effectiveness Data and Information Set (HEDIS) measures reported by health plans • Outcome and Assessment Information Set (OASIS) measures reported by home health agencies
What is the interquartile range in the interpercentile measure of variability?
Although there are several interpercentile measures of variability, the most common is the interquartile range, a stable measure of variability based on excluding extreme scores and using only middle cases. Divide the data into quarters and identify where 50% of the data lies (between the 25 and 75 percentile). An example of the interquartile range is a growth chart for children.
How do you use Stratification Using a Run Chart?
Analyzing different shifts on one chart
When do you use a Flow Chart?
As part of a Root Cause Analysis (RCA) or Failure Mood and Effects Analysis (FMEA) • Anticipate any problems before implementing an action plan • Illustrate current state and envision future state • Develop a new process or redesign a current process • Identify problem, analyze flow chart, and plan solution(s)
What is Rule 4?
Astronomical • Astronomical point } Obviously different from other points } Universal agreement that it is unusual • Every data set has a high and low point; not all high and low points are astronomical
What chart would you use? How to prepare and pass CPHQ examination?
Cause and Effect Diagram
What is this an example of?
Cause and Effect Diagram
What are Confidence Intervals?
Confidence Interval (CI) provides a range of values that describe how much a sample statistic, such as a mean, deviates from the population statistic; most commonly 95% or 90%. • A range for the 95% or 90% probability around a sample mean can be obtained. • You are 95% or 90% confident that the true population mean lies between those two values.
What chart would you use? Statistically significant patient fall rate identified.
Control Chart
What type of tool? We've made a lot of safety changes. Is our percent of Adverse Drug Events improving?
Control or Run
What are External Reporting of Findings?
Core measures reported quarterly by hospitals to: Centers for Medicare & Medicaid Services (CMS) The Joint Commission (TJC) • Managed care reports Healthcare Effectiveness Data and Information Set (HEDIS) measures annually • Other organizations report different measures to accreditation/regulatory bodies • Reported quarterly through vendors which analyze data and send back comparison reports
How do you Identify Rule 3?
Count the number of data points that do not fall on the center line • Use a table to determine if there are too many or too few runs on a run chart • Too few runs demonstrates a nonrandom signal which can be improvement or degradation • Too many runs signals non-randomness generally and often may be due to organization or stratification of data (by shift, days, etc.)
What is negative correlation in a scatter plot?
Data goes from upper left of chart to lower right of chart Not an indication of good or bad (what you want or don't want); just the direction of the line
How do you Create a Flow Chart?
Decide on the starting and ending points of the process and place these as ovals on the chart • Brainstorm to record all steps in the process and place as rectangles on the chart • Brainstorm to record all decision points in the process and place these on the chart as diamonds with a positive/negative response available for each one • Connect the ovals, rectangles, and diamonds with lines and arrows to show the flow of the process
What is a Cause and Effect Diagram?
Displays possible cause and effects identified through brainstorming • Also called a Fishbone or Ishikawa Diagram • Used by teams to organize ideas from brainstorming about potential causes of a problem or source of variation or elements required for improved performance • Used during root cause analysis to identify root causes • Look for topics generating the most ideas } Potential areas to start improvement projects Generally at least 4 categories of causes Some pre-label the categories before completing the diagram } Manpower, Methods, Machines, Materials Patron (user of system), People (workers), Provisions (supplies), Places to work (work environment), Procedures (methods and rules)
How do you Interpret a Control Chart?
Five rules can be used to identify special cause • Rule 1 - Single point outside limits • Rule 2 - Shift • Rule 3 - Trend • Rule 4 - 2 of 3 Points in outer 1/3 • Rule 5 - 15 or more points in inner
What chart would you use? Where problem areas in a particular process are located
Flow Chart
What is this an example of?
Flow chart
How is the age of our primary care providers distributed?
Histogram
What IS the abandonment rate for each hour of the day?
Histogram
What chart would you use? Medication administration errors for ordering, dispensing and administering medication
Histogram
What type of tool? What is the age at which kids say they smoked their first cigarette?
Histogram
What is Common Cause in a control chart?
Inherent in the system (process or product) over time • Affects everyone working in the system • Affects all outcomes of the system • Process is stable and predictable with random distribution of plotted points within limits (common cause) • Control chart provides basis for taking action to improve the process • Action should be taken to identify important causes of variation common to all of the points
How do you tell the correlation in a regression analysis?
Like a scatter diagram, in a regression analysis, the values range from -1 to +1 . If the value is closer to -1 , then it is a negative correlation. If the value is closer to +1 , then it is a positive correlation. If the value is close to zero, then there is no correlation.
What is an Interpercentile Measure?
Measure of variability Interquartile range (most common) • Extreme scores excluded • Measures lined up in order of size and divided into quarters • Only middle 50% of cases used • Growth charts a common example — normal growth between 25th and 75th percentile
What is an example of a Confidence Interval?
Men are 2 times as likely to miss appointments as women 95% confidence interval around the referent rate of 2 = 1.27 - 3.13 90% confidence interval around the referent rate of 2 = 1.44 - 2.77
What is a Chi-Square Example?
Null hypothesis is that men and women miss appointments at the same rate Null hypothesis is that there is no difference between the groups Rate = 1 • Chi-square will tell us if the rate = 2 (from men missing appointments 2x as much as women missing appointments) is statistically significant • Actual chi-square value for this data corresponds to statistical significance (p value) of 0.0
What are the major reasons for delays in claim tum around times?
Pareto
What type of tool? What are the biggest reasons for re perfusion delays?
Pareto
What chart would you use? Where to begin looking at over one-hour delays in recovery room leading to a backlog
Pareto Diagram
What are some qualities of core measures?
Regarding core measures, there is often a lag in receiving data. First quarter data that compare one organization to another may not be received for six to eight months, for example. Core measures are important because the focus is on providing the best care for every patient. Patients can choose the organizations they believe will best meet their healthcare needs by reviewing organizational performance on core measures.
What is the benefit of Run Chart Displaying Multiple Statistics for the same Measure?
Reviewing different statistics for the same outcome
What run chart rule does this graph show?
Rule 1
What is the Rule 1 for Determining Special Cause in a Control Chart?
Rule 1 = Single point outside the control limits (UCL or LCL) A point exactly on the limit is not considered outside the limit When there is not a lower or upper limit on one side of the center line, Rule 1 does not apply Quickly identifies sudden changes
How do you Detect Nonrandom Signals of Change
Rule 1: Shift · Six or more consecutive points either all above or all below the center line · Values falling on the center line are not counted · Values falling on the center line do not make a shift · Values falling on the center line do not break a shift · Skip values falling on the center line Rule 2: Trend · Five or more consecutive points all going up · Five or more consecutive point all going down · If two or more points are the same, count only the first point · Ignore points of the same value after counting the first one · Like values do not make or break a trend Rule 3: Number of Runs · A run is defined as a series of points in a row on one side of the center line · Too few or too many runs (crossings of the center line) signal a nonrandom pattern of change · Some points fall right on the median making it difficult to determine which run they are in · Data must actually cross the center line in order to identify that a new run has started How to Identify Rule 3 • Count the number of data points that do not fall on the center line • Use a table to determine if there are too many or too few runs on a run chart • Too few runs demonstrates a nonrandom signal which can be improvement or degradation • Too many runs signals non-randomness generally and often may be due to organization or stratification of data (by shift, days, etc.) Rule 4: Astronomical • Astronomical point } Obviously different from other points } Universal agreement that it is unusual • Every data set has a high and low point; not all high and low points are astronomical
What rule does this chart show?
Rule 2
What is Rule 2 for Determining Special Cause in a Control chart?
Rule 2 - Shift • Run of eight or more points in a row above or below the center line • Known as a shift • A point exactly on the center line does not cancel a shift or count toward a shift
What rule does this run chart show?
Rule 3
What is Rule 3 for Determining Special Cause in a Control chart?
Rule 3 - Trend • Six or more consecutive points increasing (going up) or decreasing (going down) • Known as a trend • Ties between two or more consecutive points do not cancel a trend or add to a trend
What is Rule 4 for Determining Special Cause in a Control chart?
Rule 4 — 2 of 3 Points in Outer 1/3 Two out of three consecutive points in outer one-third (sigma) of a control limit Points do not have to be in the same third of an outer limit (one point can be in outer third near upper control limit and one point can be in outer third near lower control limit) When there is not an upper or lower limit on one side of the center line Rule 4 does not apply
Has caller satisfaction been increasing since we changed our customer service process?
Run Chart
What type of chart is this an example of?
Run Chart
What is this an example of?
Run Chart with Shifts Rule 1
What chart would you use? Urinary tract infection rates over one year.
Run chart or Control Chart
What is this chart an example of?
Run chart with shift
How Do We Tell a Change is an Improvement in a run chart?
Run charts speak for themselves • Analyze With probability based rules for evidence of statistically significant change } Improvement or degradation } We don't use special/common cause terminology
What type of tool? Does the number of staff on duty appear to drive the re perfusion time?
Scatter
What chart would you use? Amount of calories and weight
Scatter Diagram
When do you use a Control Chart?
Use a control chart to answer the following questions • What are we trying to accomplish? • How will we know that a change is an improvement? What changes can we make that will result in improvement?
What are the benefits of Small Multiples?
What are the benefits of Small Multiples? • Number of components rolled together • Same scale for all graphs • All graphs on one page Can compare satisfaction for multiple components
When to Use a Pareto Chart?
} When it is important to identify the most frequent factors contributing to an issue or problem } When it is important to know where to focus improvement efforts to make the biggest impact
What is a Run Chart?
· A line graph of data plotted over time · Data is kept in time order · Can see flow of data · Helps answer questions: } How much variation do we have? } Is process changing significantly over time? } Has our change resulted in an improvement? } Did I hold the improvement?
What is Rule 3 — Number of Runs
· A run is defined as a series of points in a row on one side of the center line · Too few or too many runs (crossings of the center line) signal a nonrandom pattern of change · Some points fall right on the median making it difficult to determine which run they are in Data must actually cross the center line in order to identify that a new run has started
How do you Create a Cause and Effect Diagram?
· Determine the effect or the label for the diagram and put it on the far right · Draw a horizontal line to the left of the effect · Determine the categories (4 M's or 5 P's) · Draw a diagonal line for half of the categories above the line and half below the line · Brainstorm the list of each of the categories · Organize each of the causes on each bone · Draw branch bones to show the relationships
What is Rule 2: Trend
· Five or more consecutive points all going up · Five or more consecutive point all going down · If two or more points are the same, count only the first point · Ignore points of the same value after counting the first one Like values do not make or break a trend
When would you use a Flow Chart?
· Graphic display of a process as it is known to the authors, owners, or team · Outlines the sequence and relationship of the pieces of the process · Different individuals may have slightly different views of how the process really flows
What does a Flow Chart Tells You?
· How to identify inefficiencies, omissions, gaps, redundancies, barriers, etc. · How to determine what steps in the process are needed to make improvements or redesign the process · How to determine where risk factors lie that could lead to adverse events
How do you use Run Charts?
· It can be misleading to place a trend line on a run chart if there is no signal of change · Place a trend line on a run chart only if that chart provides a signal of change · Stratification (separation and classification of data according to selected variables or factors) can be used on a run chart Different symbols can be used to represent different variables
What are Run Charts with Multiple Measures?
· Multiple measures usually required for improvement · Display run charts as small multiples } Set of run charts } Each has same measure but different subject (location, provider, etc.) } Presented on same page Display more than one measure on a chart
What is the Center Line for a Run Chart?
· Run charts contain a center line that is usually a mean or a median · Mean: Average (all up all values and divide by total number) Median: Physically the middle number (rank all values and count from each end to find the middle; if there are two middle values, add them and divide by 2
What does a run chart tell you?
· Run charts speak for themselves · If a run chart does not speak for itself, it can be analyzed further using probability-based rules · Always analyze charts using all 4 rules · Any one rule activated is evidence of a non- random signal of change Signal could be improvement or degradation
What is Rule 1: Shift ?
· Six or more consecutive points either all above or all below the center line · Values falling on the center line are not counted · Values falling on the center line do not make a shift · Values falling on the center line do not break a shift Skip values falling on the center line
How do you Create a Run Chart?
· State the question the run chart will answer and get data · Develop horizontal scale (usually time) · Develop vertical scale · Plot data points · Label graph completely } Useful title } Horizontal access - sequence of data } Vertical access - what you are measuring · Place a center line on the chart Add additional information (goal if appropriate, annotate events, changes, etc.)
When would you use a Cause and Effect Diagram?
· When you have ideas identified through brainstorming · When you need to identify and organize possible causes of problems or issues · When you need to determine potential areas to explore
What is an example of Chi-Square?
• 15 of 30 men (50%) fail to keep appointments • 10 of 40 women (25%) fail to keep appointments • Referent rate of missed appointments in men compared to missed appointments in women } Rate 50%/ 25% = 2 (men 2x as likely not to show up as women) • Is this 25% difference due to chance?
What is a Pareto Chart?
• A Pareto chart is a display of data without knowledge of the cause of the variation • A Pareto chart made with data from an unstable process may lead to erroneous conclusions • Stratification can be used with a Pareto chart } Create a secondary Pareto chart from the largest bar in the primary chart • Pareto chart identifies areas to focus for improvement
What is a Histogram?
• A histogram (or frequency plot) is a bar chart (usually displayed with bars touching) • Presents information about one measured variable in 6 — 12 groups that are equal and mutually exclusive (no overlap) • Always about one value } Location } Spread } Shape } Patterns
What is a Run Chart?
• A line graph of data plotted over time • Data is kept in time order • Can see flow of data • Helps answer questions: } What is our baseline variation? } How much variation do we have? } How is process changing over time? } Has our change resulted in an improvement? } Did I hold the improvement? • Very low on math Evidence of improvement after changes
How do you use the 5 Rules for Detecting Special Cause in a control chart?
• Always apply all five rules to each chart • Any one rule activated indicates special cause in that area • Common cause is determined by ruling out special cause (none of the five rules are activated) Specific numbers vary depending on different statistical references
What is a Control Chart (Shewart)
• Control chart (Shewhart Chart) is a statistical tool used to distinguish between variation due to > Common cause > Special cause Statistical tool to determine if a process is stable (and predictable)
How do you Establish and Revise Limits in a control chart?
• Create initial limits for a control chart using 20 to 30 data points • If baseline data stable, freeze and extend initial limits into the future • With fewer than 20 points, calculate trial limits and revise when 20 points reached Revise limits when special cause is identified to analyze future data or when improvements made to a process result in special cause
How do you establish and revise limits in a Control Chart?
• Create initial limits for control chart using 20 to 30 data points • If baseline data stable, freeze and extend initial limits into the future • With fewer than 20 points, calculate trial limits and revise when 20 points reached Revise limits when special cause is identified, to analyze future data, or when improvements made to a process result in special cause
What is a Positive Correlation in a scatter plot?
• Data goes from lower left of chart to upper right of chart Not an indication of good or bad (what you want or don't want); just the direction of the line
What is the benefit of putting related Improvement Measures on one page?
• Different process measures and an outcome measure How does the process corelate with the outcome
What does a Histogram tell you?
• Distribution of the data collected } Spread of the data (close to mean, extending out away from mean) } Shape of the data of what it looks like (normal distribution or bell-shaped curve, two peaks or bimodal, three peaks or trimodal, etc.) } Location of the data (on one side or the other, in the middle, evenly spread out, etc.) Does NOT tell you if the process is stable
How is a frequency plot (histogram) interpreted?
• Evaluate central location • Evaluate spread • Learn from shape
What is Rule 5 for Determining Special Cause in a Control chart?
• Fifteen consecutive points close to the center line (in the inner one third or one sigma) of the chart
How do you Interpret a Run Chart?
• Four rules can be used to identify nonrandom signals of change Rule 1 - Shift Rule 2 - Trend Rule 3 - Too many or too few runs Rule 4 - Astronomical data point • Evidence of nonrandom signal of change in run chart if one or more rules is seen; can be evidence of improvement or degradation • Learn from signal and take appropriate action There is not universal agreement on the numbers to use for the rules. Different statistical references use different numbers. If a signal is detected, it's important to investigate what is causing the signal and take action to either make or sustain an improvement.
What are the qualities of a Run Chart?
• Graphical display of data plotted in some type of order, generally over time • Also known as a trend chart or time series chart • Tool for understanding variation • Universal tool for improvement • Easy to construct Simple to interpret and versatile
What does a Control Chart Tell You?
• How much variation exists in a process • Type of variation (common or special cause) • Which type of improvement strategy is needed (common or special cause) • Whether a process is stable and predictable • If changes yielded improvements If improvements are being sustained
What is a Level of Significance?
• Level of significance (p) gives the probability of observing a difference as large as the one found in the study when there is no true difference (when the null hypothesis is true) • Based on probability, when p values <.05, results are statistically significant
How a Pareto Chart Interpreted?
• Look for the Pareto effect •We won't always find it! } Is entire chart speaking to us? } Can we re-stratify? Last choice is selecting a column and tackling it!
What are Mistake 1 and Mistake 2 in a control chart?
• Mistake 1 = Reacting to an outcome as if it came from a special cause when actually it came from common causes of variation • Mistake 2 = Reacting to an outcome as if it came from common causes of variation when actually it came from special cause • Mistake 1 is made more often Control charts help minimize these two mistakes
What is a Special Cause in a control chart?
• Not part of the system (process or product) all the time or does not affect everyone • Appears due to specific circumstances • Process is not stable or predictable • Identify when special cause occurred • Learn from the special cause
What are core measures?
• Protocols based on evidence-based research that identify the best patient results for given types of disease processes • Initially began with heart failure, pneumonia, acute myocardial infarction (AMI), and surgical site infections • Now expanded to multiple disease processes
What is the purpose of a Common Cause Improvement Strategy in a Control Chart?
• Realize that process is performing as well as possible • Making it better requires process redesign • Identify all aspects of process to change • Test changes using PDSA cycle • Implement successful changes using PDSA cycle
What are the 5 charts on the Tool List?
• Run Chart: How much variation do we have? Is our change an improvement? Are we holding it? • Pareto: Which variables out of many are occurring most? • Frequency plot: How is this one variable distributed (what is the spread of LOS, Cost. etc.) • Scatter Plot: Is variable A related to variable B? Shewhart Chart: Is our process stable? Do we have common or special cause variation?
How to Create a Control Chart?
• Select a measure and plot it on a run chart • When enough data is available, calculate a center line and upper and lower limit • Three sigma limits most commonly used Upper Limit = Center Line + 3 Sigma Lower Limit = Center Line — 3 Sigma • Varying limits are used when subgroup size is unequal Stratification can be used in control charts
How is a Histogram set up?
• Should be developed with 25 or more data points • Number of data points can be represented by bars, stacked symbols, or lines • Very flexible in the amount of data that can be displayed Stratification can be used with histograms
What are Some Basics on Building a Run Chart?
• Start with a single data point • Somewhere around 12 data points determine median • Apply rules to detect statistically significant change • When 20-30 data points revise median After this revise your median when the old median is no longer relevant
What is the purpose of a Special Cause Improvement Strategy?
• Take action based on the special cause • Can be favorable or unfavorable • If distribution or pattern of points is not random, process is unstable (special cause) • Action should be taken to learn about special causes of variation
What does a Scatter Diagram Tell You?
• The type and degree of relationship between two variables Possible cause and effect relationships
When would you use a Scatter Diagram?
• To determine whether there is a relationship between variables • To identify the strength of the relationship } Tight } Loose } Outliers • To determine the type of the relationship } Positive } Negative No relationship
When would you use a Histogram?
• To present information about location, shape, spread, and patterns of data • To find patterns in data • To show the data's shape • To show the data's distribution or spread • To show whether data are symmetric or skewed To show whether there are extreme data values
What is a scatter diagram or scatter plot?
• Tool for learning about associations or relationships between two variables • Identifies cause-and-effect relationships between variables • Identifies patterns of relationship exists between variables (positive, negative, no relationship) • Stratification can be used with scatter diagrams • Can illustrate the strength of relationships between variables • Correlation = relationship between two variables • Independent variable: Leader variable Dependent variable: Follower variable
What are examples of t-tests?
• Two-sample independent t-test } Group of 20 department heads } 10 receive training on QI tools } 10 do not receive training on QI tools } Scores of each group compared • Paired sample t-test } Group of 20 department heads } 20 given pretest } 20 receive training on QI tools } 20 given posttest } Scores on pretest compared with scores on posttest
What are the properties of Control Chart Selection?
• Type of data determines appropriate type of control chart used • Count or Classification (Attributes) Data } Count Data - C Chart - Equal area of opportunity - U Chart - Unequal area of opportunity } Classification Data - PChart • Continuous (Variable) Data } I (X) Chart — Each subgroup composed of a single data value } X and S Chart — Subgroups have more than one data value
When would you use a Run Chart?
• Use a run chart in improvement efforts } Display data to make process performance visible } Determine whether a change resulted in improvement } Determine whether gains made through improvement effort are being sustained Determine how much variation is present
When would you use a Run Chart?
• Use a run chart when you want to } Use few data points } Rapidly detect signals of improvement } Use simple tool readily accepted } Make amount of process variation visible } Detect improvement Determine whether improvement has been maintained
What are the benefits of a Pareto Chart?
• Used to help focus improvement efforts • Display of the 80/20 rule (80% of problems due to 20% of causes) • Data (problems, defects, adverse drug events, patient complaints) can be organized into categories or classifications • Categories ordered from most frequently occurring to least frequently occurring on horizontal axis of chart • Useful to have 30 or more data points
What are Nonparametric Tests?
• Used with categorical or count data • Chi-square } Measures statistical significance of a difference in proportions } Conceptually same as t-test (analyzes difference between 2 means) } Most commonly used statistic in medical literature (easiest to calculate) Nonparametric tests include chi-square tests which are used with categorical (attributes) data. Much of the data collected by healthcare quality professionals are counted, not measured. Since the data is counted, it means that many math operations don't apply. It is possible, however, to describe the ratio of the counts or to compare proportions with counted data. The chi-square test measures the statistical significance of a difference in proportions. It is a statistical test commonly used to compare observed data with data that one would expect to obtain according to a specific hypothesis.
What are Parametric Tests?
• Used with data measured on a continuous scale • Include t-test, regression analysis, and multiple regression analysis • t-test } Used to analyze difference between two means (scores) } Used when determining whether difference between two group means is statistically significant, meaning difference is not by chance • Regression analysis is based on statistical correlations, associations among variables } Simple linear regression } One variable (x) used to predict second variable (y) (e.g., weight used to predict height) } Similar to a Scatter Diagram • The higher the correlation, the more accurate the degree of prediction
