IHI:Q103 Testing and measuring changes with PDSA cycles
QI 103 Lesson 3: How to Build Your Degree of Belief Over Time Your Turn: Acting on Test Results #1 Based on the data collection form, which of the following explanations do you believe best explains what happened in our example scenario? - There was a problem with data collection. - The change was conducted as planned but the predicted results did not occur. - The change was not conducted as planned. #2 For the next PDSA test cycle, the team proposes having another English-speaking nurse (not Nurse Peters) try using the PHQ-9 survey with 10 English-speaking patients on a different day of the week. What do you think of the proposal? - This plan sounds reasonable. - It would be best to adjust this plan. #3 For the next test cycle, another idea is to have Nurse Peters test the PHQ-9 form again with five English-speaking patients in one afternoon. This time, a back-up nurse will be available who can step in if Nurse Peters has to leave for any reason.What do you think of this idea? - This plan sounds reasonable. - It would be best to adjust this plan
#1 The change was not conducted as planned. The best answer is that the change was not conducted as the team planned it. You can see in the notes that Nurse Peters, a critical part of the process, had to leave, and there was no one to finish executing the test.Fortunately, thanks to successful data collection, you know that until Nurse Peters had to leave, the test was going great. Even though the team needs to adjust the support processes, your degree of belief that the change will lead to improvement has increased. #2 It would be best to adjust this plan. This is a change in scope (change in staff person and day of the week) and scale (number of patients). This is probably too much, too soon. #3 This plan sounds reasonable. Even though the test didn't totally work last time, there is good reason to be confident that with a better support system in place (the addition of a back-up nurse), the team will achieve its desired results. A cycle with the same size and scope is a good way to show that a five-patient test can be successful and further increase the degree of belief that the change is a good one.
Post-Lesson Assessment: QI 103 Lesson 2 Which of the following statements is true? - You need a computer program to draw a run chart. - It is possible to draw a run chart by hand. - Computer programs can be helpful for drawing run charts but are not always necessary. - It is possible to draw a run chart by hand AND computer programs can be helpful for drawing run charts but are not always necessary.
- It is possible to draw a run chart by hand AND computer programs can be helpful for drawing run charts but are not always necessary.
Post-Lesson Assessment: QI 103 Lesson 2 During a clinical rotation on the medical-surgical floor of a hospital, you notice several patients have developed urinary tract infections (UTIs) associated with their Foley catheters (tubes inserted into the bladder to drain urine). Your staff physician agrees that this is a problem and offers to help with an improvement project. Together, you work through several PDSA cycles to reduce the rate of UTIs on your floor. When designing the run chart, it is important to include: - Units of time on the Y axis - The rate of UTIs on the X axis - Units of time on the X axis - Units of time on the Y axis AND the rate of UTIs on the X axis
- Units of time on the X axis The run chart should display units of time — whether it's days, weeks, or months — on the X axis. The Y axis is where you plot the key variable you are measuring, which in this case is the rate of UTIs.
QI 103 Lesson 1: How to Define Measures and Collect Data Your Turn: Establishing a Data Collection Plan You've got a summer job working at the headquarters of Jen & Berry's, an ice cream company. One day, the CEO herself (Jen, of course) walks by your desk. "We're testing a new flavor," she says, "Peanut Buttery Banjo Jamboree." She has high hopes customers will like the new flavor, and that it can replace "Chocolate Mocha Polka Party," the company's lowest-selling product. "But," she says, "we need some data to show that it's really an improvement." 1. How would you figure out which flavor the company should sell? - Grab a couple of pints of ice cream and sample them both yourself! - Poll a small sample of customers in the region, including proportional numbers of people who read "Chocolate Lovers Monthly" and "Peanut Butter Today." - Talk to an equal number of readers of "Chocolate Lovers Monthly" and "Peanut Butter Today." - Ask everyone in your region which flavor they prefer. 2. Which of the following is the best plan for collecting data? - Select a sample of 10 charts from each of the four months to review. - Review a sample of the first 40 charts in chronological order. - Review all the charts for a four-month period, of which there are 210. - Skip baseline data collection because it's not important for this project.
1. Poll a small sample of customers in the region, including proportional numbers of people who read "Chocolate Lovers Monthly" and "Peanut Butter Today." Although you might be tempted to sample the ice cream yourself, the best answer is to poll a small number of customers in the region. Polling either the Chocolate Lovers Monthly or the Peanut Butter Today group alone is going to result in a "sampling bias" — a sample that results in either a more positive or a more negative picture than that of the overall population. And it's not realistic to ask everyone in the entire region to try both flavors. Your best bet is to ask a small sample that is representative of the total population, with the correct proportions of both types of magazine subscribers and people who subscribe to neither publication. 2. Select a sample of 10 charts from each of the four months to review. Knowing that there are 210 charts in total, you can save time by selecting a smaller sample, such as 40 charts, to give you "just enough data" to proceed.In conducting the review, you discover that 15 of the 40 patients in the sample (37.5 percent) had a hemoglobin A1c greater than 8. When the information systems folks get back to the team several weeks later with the summary data from running all the charts for the same four-month period, the percentage they give you is almost identical. In conducting the review, you discover that 15 of the 40 patients in the sample (37.5 percent) had a hemoglobin A1c greater than 8. When the information systems folks get back to the team several weeks later with the summary data from running all the charts for the same four-month period, the percentage they give you is almost identical.
How many data points do statisticians recommend to draw a median?
12
When increasing the number of patients or events from one PDSA cycle to the next, it is usually helpful to multiply by what number?
5
Post-Lesson Assessment: QI 103 Lesson 1 Why should you consider collecting a family of measures when undertaking an improvement? - It makes the project more publishable. - A single measure may not be enough to determine the impact of a change on the system. - All improvement projects are so complex that they require multiple measures. - All of the above
A single measure may not be enough to determine the impact of a change on the system. A single measure may not be enough to determine the impact of a change on the system.Health care systems are extremely complex. A small change in a complex system can lead to many unexpected results, so using only one measure may not capture the effect of the change upon the system. Using more measures will not necessarily increase the likelihood of publication. Finally, it is health care that is complex, not necessarily the improvement project itself. The most successful projects are often the simplest ones.
Post-Lesson Assessment: QI 103 Lesson 2 Which of the following is a reason for using a dynamic display for data? - To show variation - To make the case for improvement over time - To look for patterns - All of the above
All of the above The best answer is all of the above. Summary statistics that are static in nature don't give you the appropriate picture of the variation that lives in your data. By plotting data over time, you can observe patterns and find evidence of improvement.
Which of the following statements is true about using data for improvement?
All of the above.
When graphing your data, you should:
Annotate the tests of change.
Which of the following might be an outcome measure for this effort?
Average number of minutes between patient arrival at the clinic and completion of check-in
Post-Lesson Assessment: QI 103 Lesson 3 A hospital is trying to implement a new patient assessment form. They want to first test the usability and efficacy of the form.When determining sample size for the first test, it is most important to: (A) Look at similar research to see what sample size other organizations use. (B) Weigh the potential consequences of a test that does not lead to improvement against the belief in success. (C) Use a random sampling technique, so results can be extrapolated. (D) Ask all staff members what sample size they think should be used.
B With improvement work, you should weigh the potential consequences of a test that does not lead to improvement against the belief in success. How small your first PDSA cycle should be rests on your degree of belief and the stakes involved.
Post-Lesson Assessment: QI 103 Lesson 2 Which of the following describes data stratification? - Plotting observations to show the relationship between two sets of data - Plotting data over time - Illustrating the relative frequency of occurrence - Classifying and separating data according to specific variables
Classifying and separating data according to specific variables Classifying and separating data according to specific variables — a practice called stratification — is a helpful way to understand the story data is telling. The goal of stratification is to find patterns in data that will help you understand the causal factors at work. Stratification helps inform teams' decisions about what changes to make, where, and when.
Post-Lesson Assessment: QI 103 Lesson 1 You work on a surgical unit that receives 87 new patients per week. In your spare time, you are trying to improve pain control across the unit. As you prepare to collect your baseline data before testing your change, you consider your plan to gather this information. Which of the following would be the best data collection plan for this project, and why? - Collect data for at least a year, focusing only on patients who have undergone especially painful surgeries. - There is no reason to collect baseline data. - Collect data on a representative selection of patients for two weeks. - Collect data on the pain level of every patient for at least a year.
Collect data on a representative selection of patients for two weeks. Collecting data on a representative portion of your population — that is, sampling — allows you to collect baseline data and start making improvements faster, using fewer resources, than would be possible if you tried to look at every patient. Baseline data is important so that you can see if the changes you are testing are leading to improvement. Collecting data about a nonrepresentative sample of your population, such as by looking only at patients with the most severe pain, won't show the broad impact of your project on your unit.
Post-Lesson Assessment: QI 103 Lesson 3 Imagine you're a member of a newly formed improvement team that has taken up the challenge to reduce health care-associated infections at your hospital. You have an idea for a change to the room cleaning process that you want to test, but you're slightly nervous because improper cleaning and disinfection can carry a high risk for patients with compromised immune systems. You haven't run any PDSA cycles yet. Which of the following would be the best next step?
Confirm the "face validity" of the new cleaning process by demonstrating it with a couple of housekeeping staff members and a supervisor.
Post-Lesson Assessment: QI 103 Lesson 3 Imagine you're a member of a newly formed improvement team that has taken up the challenge to reduce health care-associated infections at your hospital. You have an idea for a change to the room cleaning process that you want to test, but you're slightly nervous because improper cleaning and disinfection can carry a high risk for patients with compromised immune systems. You haven't run any PDSA cycles yet. Which of the following would be the best next step? (A) Have one housekeeper use the process with one room cleaning.(B) Have all housekeepers use the process for a week. (C) Have one housekeeper use the process on five room cleanings.(D) Confirm the "face validity" of the new cleaning process by demonstrating it with a couple of housekeeping staff members and a supervisor.
Confirm the "face validity" of the new cleaning process by demonstrating it with a couple of housekeeping staff members and a supervisor. Based on your concern about patient safety, you'd likely first want to confirm the "face validity" of the new cleaning process by demonstrating it with a couple of housekeeping staff members and a supervisor. When testing changes that involve patients, it can be helpful to conduct an initial test using staff only. A simulation or practice session before going live is often a good way to uncover issues with high risk.
Post-Lesson Assessment: QI 103 Lesson 3 A hospital is trying to implement a new patient assessment form. They want to first test the usability and efficacy of the form. When will the organization know the assessment form has been implemented on Unit Y? - When nurses on Unit Y automatically use the form to assess all their patients - When nurses on Unit Y use the form when they remember to - When Nurse Moss uses the form for all her patients - When the organization has tested the form on all its patients for a month
Correct Answer: When nurses on Unit Y automatically use the form to assess all their patients Rational: The best answer is when nurses on Unit Y automatically use the form to assess all their patients. Implementation occurs when a change is embedded into daily practice
QI 103 Lesson 2: How to Use Data for Improvement Your Turn: CLABSIs Here's Dr. Cheriyan's data again — the same improvement project presented in the slides you saw on the first page — but displayed dynamically on a run chart: When the data is presented in this format, what do you see? - The CLABSI rate has steadily INCREASED since the tests of change (PDSAs). - The CLABSI rate has steadily DECREASED since the tests of change (PDSAs).
Correct Answer:The CLABSI rate has steadily DECREASED since the tests of change (PDSAs). It's clear that the changes the team is testing have led to a decrease in the CLABSI rate. Before we end this lesson, we'll look at one more helpful technique for interpreting data for improvement work.
QI 103 Lesson 3: How to Build Your Degree of Belief Over Time Your Turn: Depression Screening Returning to the example of testing depression screening for patients with diabetes: The team wants to conduct two small-scale tests on the same day: one involving the same nurse who did the first successful test with one English-speaking patient, this time with five more English-speaking patients. And another with a Spanish-speaking staff member and one Spanish-speaking patient. What do you think of this idea? Does this test plan sound reasonable? Yes, the plan sounds reasonable. No, adjust the plan.
Correct Answer:Yes, the plan sounds reasonable. In the 1:1:1 test cycle with the Spanish version of the depression screening, you're broadening the scope (Spanish instead of English) but maintaining the same scale of the previous cycle. This test could be run in parallel with the test cycle of the English-speaking nurse and five English-speaking patients, which keeps the scope the same but increases the test size.
Which of the following methods would you recommend to display your improvement data?
Draw a run chart.
Let's say the hospital has an English-speaking nurse (Nurse Moss) assess one English-speaking patient with the new form. It is a successful test and the improvement team wants to increase the scope of the next test. What would they do?
Have a Spanish-speaking nurse give the assessment to one of her Spanish-speaking patients.
Let's say the hospital has an English-speaking nurse (Nurse Moss) assess one English-speaking patient with the new form. It is a successful test and the improvement team wants to increase the scale of the next test. What would they do?
Increase the number of patients Nurse Moss assesses by a factor of 5.
Post-Lesson Assessment: QI 103 Lesson 3 A hospital is trying to implement a new patient assessment form. They want to first test the usability and efficacy of the form.The hospital has an English-speaking nurse (Nurse Moss) assess one English-speaking patient with the new form. It is a successful test and the improvement team wants to increase the scale of the next test. What should they do? - Have a Spanish-speaking nurse give the assessment to one of her Spanish-speaking patients. - Have a different English-speaking nurse give the assessment to one of her English-speaking patients. - Increase the number of patients Nurse Moss assesses by a factor of 5. - Increase the number of patients Nurse Moss assesses by a factor of 10.
Increase the number of patients Nurse Moss assesses by a factor of 5. The best answer is to increase the number of patients Nurse Moss assesses by a factor of 5. Scale is the number of interactions within the test — in this case, the number of patients receiving the assessment, and the 5X Rules recommends an increase by a factor of five in each subsequent test. Changing the conditions of the test — such as the language involved or the staff involved — would be a change in scope, rather than scale.
Post-Lesson Assessment: QI 103 Lesson 2 Which of the following is a problem with static data? - It doesn't adequately portray variation. - It is often inaccurate. - It can't display mean, median, or mode. - All of the above
It doesn't adequately portray variation. Rational: Summary statistics that are static in nature don't give you the appropriate picture of the variation that lives in your data. Although you can accurately display data such as the mean, median, or mode, it is not a good way to observe change over time.
Which of the following statements is true?
It is possible and sometimes easiest to draw a run chart by hand. Other times, it can be helpful to use a computer program. The next course will teach you to draw a run chart using a spreadsheet computer program.
Post-Lesson Assessment: QI 103 Lesson 1 You're trying to improve patient satisfaction scores in your hospital's emergency department. You decide to poll 40 patients a day for three weeks. Your charge nurse says that the quality of service really fluctuates depending on the time of day. So your daily sample consists of 10 patients polled at 8 AM, another 10 at noon, another 10 at 5 PM, and the final 10 at 9 PM. This is an example of what kind of sampling? - Simple random - Proportional stratified random - Judgment - None of the above
Judgement The best answer is judgment sampling. Judgment sampling chooses samples based on subject matter expertise.
Which of the following is the best way to collect baseline data for this improvement project?
Look at a few patients every day for a week.
Post-Lesson Assessment: QI 103 Lesson 1 You volunteer at a student-run clinic associated with your academic health center. As a member of the student board, you are constantly looking for ways to improve the clinic. One common complaint is that it takes too long to check patients in once they arrive, and you decide to tackle this problem. Which of the following is the best way to collect baseline data for this improvement project? - Look at a few patients every day for a week. - Look at 10 percent of patients for a year. - Look at 100 percent of patients for a month. - There is no reason to collect baseline data.
Look at a few patients every day for a week. The best answer is to use a small sample and gather the data quickly. When measuring for improvement, it's often unnecessary (and may defeat the goal of rapid, iterative testing) to collect all available information over an extended period of time. Baseline data is important for knowing whether changes you are making are, in fact, leading to improvement.
When designing your run chart, it is important to include:
None of the above
When planning a sequence of PDSA cycles for a change that involves patients, which of the following is a true statement?
None of the above
Post-Lesson Assessment: QI 103 Lesson 1 You volunteer at a student-run clinic associated with your academic health center. As a member of the student board, you are constantly looking for ways to improve the clinic. One common complaint is that it takes too long to check patients in once they arrive, and you decide to tackle this problem. Which of the following might be a process measure for this effort?
Number of students helping to check a patient in
When designing the run chart, it is important to include:
Units of time on the X axis
Which of the following is an example of an effective measurement technique for improvement?
Use quantitative and qualitative data. interviews or focus groups
Post-Lesson Assessment: QI 103 Lesson 3 When planning a sequence of PDSA cycles for a change that involves patients, which of the following is a true statement? - Patient characteristics in each PDSA cycle should be as uniform as possible to allow valid comparisons. - The number of patients in each cycle should stay fixed, to allow valid comparisons. - We would expect the number of patients involved to grow rapidly from early cycles to later cycles. - None of the above
We would expect the number of patients involved to grow rapidly from early cycles to later cycles.
When determining sample size for the first test, it is most important to:
Weigh the potential consequences of a test that does not lead to improvement against the belief in success.
Imagine you're a member of a newly formed improvement team that has taken up the challenge to reduce health care-associated infections at your hospital. You have an idea for a change to the room cleaning process that you want to test, but you're slightly nervous because improper cleaning and disinfection can carry a high risk for patients with compromised immune systems. Which of the following is most important to determine the best size for your initial PDSA test?
Weigh the potential consequences of a test that does not lead to improvement against the degree of belief in success.
Post-Lesson Assessment: QI 103 Lesson 3 Imagine you're a member of a newly formed improvement team that has taken up the challenge to reduce health care-associated infections at your hospital. You have an idea for a change to the room cleaning process that you want to test, but you're slightly nervous because improper cleaning and disinfection can carry a high risk for patients with compromised immune systems. Which of the following is most important to determine the best size for your initial PDSA test? - Apply the 1-2-3 rule. - Weigh the potential consequences of a test that does not lead to improvement against the degree of belief in success. - Apply the 5X rule. - Weigh the potential consequences of a test that does not lead to improvement against the possible benefit of a test that does lead to improvement.
Weigh the potential consequences of a test that does not lead to improvement against the degree of belief in success. With improvement work, you should weigh the potential consequences of a test that does not lead to improvement against the belief in success. How small your first PDSA cycle should be rests on your degree of belief and the stakes involved.
Post-Lesson Assessment: QI 103 Lesson 1 You're working on an improvement project at a community mental health center. Your project aim: "Within two months, 100 percent of our patients will wait less than 30 minutes to be seen by a physician." You decide to gather data on patient wait times over a week-long period in order to establish a baseline. What might be an important consideration as you plan your data collection strategy? - How to establish consensus among all the clinic's caregivers about the value of the project before gathering data - How to inform the supervisors of individual physicians quickly when those physicians' patients wait more than 30 minutes - Who will collect the data and what method will be used for collecting it - All of the above
What exactly you mean by "wait less than 30 minutes to be seen" — does this include the time the patient spends checking in, for instance? Who will collect the data and what method they will use for collecting it It is crucial to clearly define not just what the data is that you want to collect, but how you will collect it. You are gathering data for improvement, not accountability, so for this project, it doesn't make sense to notify supervisors about the performance of individual caregivers. It's not necessary to establish consensus at the outset about the value of the project; by doing small tests of change, you are likely to gain buy-in as you go.
When will the organization know the assessment form has been implemented on Unit Y?
When nurses on Unit Y automatically use the form to assess all their patients
In a run chart, the variable being measured is typically placed on what axis?
Y axis
Post-Lesson Assessment: QI 103 Lesson 2 During a clinical rotation on the medical-surgical floor of a hospital, you notice several patients have developed urinary tract infections (UTIs) associated with their Foley catheters (tubes inserted into the bladder to drain urine). Your staff physician agrees that this is a problem and offers to help with an improvement project. Together, you work through several PDSA cycles to reduce the rate of UTIs on your floor. 1) Which of the following methods would you recommend to display your improvement data? a) Draw a bar chart. b) Write a list of numbers. c) Create a two-column table. d) Draw a run chart.
d) Draw a run chart.
