Patient Safety: IHI QI 103 Testing and Measuring Changes with PDSA Cycles

In a run chart, the variable being measured is typically placed on what axis? (A) X axis (B) Y axis (C) Either axis (D) Neither axis; the run chart does not compare variables.

B The measured value is usually represented on the Y axis of a run chart. The X axis is usually the time — minutes, hours, days, weeks, months, etc. — or a numerical sequence in cases where data doesn't correspond to units of time.

Use the following scenario to answer the next three questions. A hospital is trying to implement a new patient assessment form. They want to first test the usability and efficacy of the form. When determining sample size for the first test, it is most important to: (A) Look at similar research to see what sample size other organizations use. (B) Weigh the potential consequences of a test that does not lead to improvement against the belief in success. (C) Use a random sampling technique, so results can be extrapolated. (D) Ask all staff members what sample size they think should be used.

B With improvement work, you should weigh the potential consequences of a test that does not lead to improvement against the belief in success. How small your first PDSA cycle should be rests on your degree of belief and the stakes involved.

When planning a sequence of PDSA cycles for a change that involves patients, which of the following is a true statement? (A) Patient characteristics in each PDSA cycle should be as uniform as possible to allow valid comparisons. (B) The number of patients in each cycle should stay fixed, to allow valid comparisons. (C) We would expect the number of patients involved to grow rapidly from early cycles to later cycles. (D) None of the above

C As improvement work progresses and the number of cycles increases, we would expect the scope and scale of the tests to increase, meaning both a rapidly growing number of patients involved in the tests as well as increasing diversity in the test population.

Use the following scenario to answer the first two questions : You volunteer at a student-run clinic associated with your academic health center. As a member of the student board, you are constantly looking for ways to improve the clinic. One common complaint is that it takes too long to check patients in once they arrive, and you decide to tackle this problem. Which of the following might be an outcome measure for this effort? (A) Average number of minutes between patient arrival at the clinic and completion of check-in (B) Number of patients seen by the clinic (C) Average number of students helping to check a patient in (D) None of the above

A Outcome measures tell you how the system is performing. In this case, the aim of the project is to decrease the time it takes to check in patients, so an appropriate outcome measure for this project could be "average number of minutes between patient arrival at the clinic and completion of check-in." The average number of patients seen by the clinic and the average number of students helping to check in patients might be useful to track as balancing and process measures, respectively.

Which of the following is a problem with static data? (A) It doesn't adequately portray variation. (B) It is often inaccurate. (C) It can't display mean, median, or mode. (D) All of the above

A Summary statistics that are static in nature don't give you the appropriate picture of the variation that lives in your data. Although you can accurately display data such as the mean, median, or mode, it is not a good way to observe change over time.

Let's say the hospital has an English-speaking nurse (Nurse Moss) assess one English-speaking patient with the new form. It is a successful test and the improvement team wants to increase the scope of the next test. What would they do? (A) Have a Spanish-speaking nurse give the assessment to one of her Spanish-speaking patients. (B) Have a different English-speaking nurse give the assessment to one of her English-speaking patients. (C) Increase the number of patients Nurse Moss assesses by a factor of 5. (D) Increase the number of patients Nurse Moss assesses by a factor of 10.

A The best answer is to have a Spanish-speaking nurse give the assessment to one of her Spanish-speaking patients. Changing the conditions of the test — such as the language involved — would be a change in scope. A change in scale would be a change in the number of interactions within the test.

Which of the following is the best way to collect baseline data for this improvement project? (A) Look at a few patients every day for a week. (B) Look at 10 percent of patients for a year. (C) Look at 100 percent of patients for a month. (D) There is no reason to collect baseline data.

A The best answer is to use a small sample and gather the data quickly. When measuring for improvement, it's often unnecessary (and may defeat the goal of rapid, iterative testing) to collect all available information over an extended period of time. Baseline data is important for knowing whether changes you are making are, in fact, leading to improvement.

Which of the following describes data stratification? (A) Plotting observations to show the relationship between two sets of data (B) Classifying and separating data according to specific variables (C) Plotting data over time (D) Illustrating the relative frequency of occurrence

B Classifying and separating data according to specific variables — a practice called stratification — is a helpful way to understand the story data is telling. The goal of stratification is to find patterns in data that will help you understand the causal factors at work. Stratification helps inform teams' decisions about what changes to make, where, and when.

Why should you consider collecting a family of measures when undertaking an improvement? (A) It makes the project more publishable. (B) A single measure may not be enough to determine the impact of a change on the system. (C) All improvement projects are so complex that they require multiple measures. (D) All of the above

B Health care systems are extremely complex. A small change in a complex system can lead to many unexpected results, so using only one measure may not capture the effect of the change upon the system. Using more measures will not necessarily increase the likelihood of publication. Finally, it is health care that is complex, not necessarily the improvement project itself. The most successful projects are often the simplest ones.

You're working on an improvement project at a community mental health center. Your project aim: "Within two months, 100 percent of our patients will wait less than 30 minutes to be seen by a physician." You decide to gather data on patient wait times over a week-long period in order to establish a baseline. What might be an important consideration as you plan your data collection strategy? (A) Whether you'll provide food for the patients who wait more than 30 minutes. (B) What exactly you mean by "wait less than 30 minutes to be seen" — does this include the time the patient spends checking in, for instance? (C) How to establish consensus among the clinic's caregivers about the value of the project before gathering data. (D) How to inform the supervisors of individual physicians quickly when those physicians' patients wait more than 30 minutes.

B It is crucial to clearly define your measure before you begin gathering data, so that you and your team members measure the same thing each time — and so that others understand what you are measuring. It's not necessary to establish consensus at the outset about the value of the project; by doing small tests of change, you are likely to gain buy-in as you go. Finally, you are gathering data for improvement, not accountability, so for this project, it doesn't make sense to notify supervisors about the performance of individual caregivers.

When increasing the number of patients or events from one PDSA cycle to the next, it is usually helpful to multiply by what number? (A) 2 (B) 5 (C) 10 (D) 20

B The 5X Rule recommends an increase by a factor of five whenever you finish one successful test and move on to the next.

Which of the following is an example of an effective measurement technique for improvement? (A) Always strive for perfection. (B) Use quantitative and qualitative data. (C) Always set aside designated time for data collection. (D) All of the above

B The best answer is to use qualitative and quantitative data. Qualitative data, which is not so much about numbers as it is about the depth of the information collected, can be a rich source of knowledge in improvement projects. Interviews or focus groups are common sources of qualitative data. Measurement is not the goal; improvement is the goal. In order to move forward to the next step, a team needs just enough data to know whether changes are leading to improvement. ("Seek usefulness, not perfection" is a mantra at IHI.) To save time, integrate data collection into the daily routine as much as possible.

You work on a surgical unit that receives 87 new patients per week. In your spare time, you are trying to improve pain control across the unit. As you prepare to collect your baseline data before testing your change, you consider your plan to gather this information. Which of the following would be the best data collection plan for this project, and why? (A) Collect data on the pain level of every patient for at least a year. (B) Collect data for at least a year, focusing only on patients who have undergone especially painful surgeries. (C) Collect data on a representative selection of patients for two weeks. (D) There is no reason to collect baseline data.

C Collecting data on a representative portion of your population — that is, sampling — allows you to collect baseline data and start making improvements faster, using fewer resources, than would be possible if you tried to look at every patient. Baseline data is important so that you can see if the changes you are testing are leading to improvement. Collecting data about a nonrepresentative sample of your population, such as by looking only at patients with the most severe pain, won't show the broad impact of your project on your unit.

You're working on an improvement project at a community mental health center. Your project aim: "Within two months, 100 percent of our patients will wait less than 30 minutes to be seen by a physician." You decide to gather data on patient wait times over a week-long period in order to establish a baseline. What might be an important consideration as you plan your data collection strategy? (A) How to establish consensus among all the clinic's caregivers about the value of the project before gathering data. (B) How to inform the supervisors of individual physicians quickly when those physicians' patients wait more than 30 minutes. (C) Who will collect the data and what method he or she will use for collecting it. (D) All of the above.

C It is crucial to clearly define not just what the data is that you want to collect, but how you will collect it. You are gathering data for improvement, not accountability, so for this project, it doesn't make sense to notify supervisors about the performance of individual caregivers. It's not necessary to establish consensus at the outset about the value of the project; by doing small tests of change, you are likely to gain buy-in as you go.

You're trying to improve patient satisfaction scores in your hospital's emergency department. You decide to poll 40 patients a day for three weeks. Your charge nurse says that the quality of service really fluctuates depending on the time of day. So your daily sample consists of 10 patients polled at 8 AM, another 10 at noon, another 10 at 5 PM, and the final 10 at 9 PM. This is an example of what kind of sampling? (A) Simple random (B) Proportional stratified random (C) Judgment (D) None of the above

C The best answer is judgment sampling. Judgment sampling chooses samples based on subject matter expertise.

Let's say the hospital has an English-speaking nurse (Nurse Moss) assess one English-speaking patient with the new form. It is a successful test and the improvement team wants to increase the scale of the next test. What would they do? (A) Have a Spanish-speaking nurse give the assessment to one of her Spanish-speaking patients. (B) Have a different English-speaking nurse give the assessment to one of her English-speaking patients. (C) Increase the number of patients Nurse Moss assesses by a factor of 5. (D) Increase the number of patients Nurse Moss assesses by a factor of 10.

C The best answer is to increase the number of patients Nurse Moss assesses by a factor of 5. Scale is the number of interactions within the test — in this case, the number of patients receiving the assessment, and the 5X Rules recommends an increase by a factor of five in each subsequent test. Changing the conditions of the test — such as the language involved or the staff involved — would be a change in scope, rather than scale.

When will the organization know the assessment form has been implemented on Unit Y? (A) When Nurse Moss uses the form for all her patients (B) When nurses on Unit Y use the form when they remember to (C) When nurses on Unit Y automatically use the form to assess all their patients (D) When the organization has tested the form on all its patients for a month

C The best answer is when nurses on Unit Y automatically use the form to assess all their patients. Implementation occurs when a change is embedded into daily practice.

When designing the run chart, it is important to include: (A) Units of time on the Y axis (B) The rate of UTIs on the X axis (C) Units of time on the X axis (D) A and B

C The run chart should display units of time — whether it's days, weeks, or months — on the X axis. The Y axis is where you plot the key variable you are measuring, which in this case is the rate of UTIs.

Imagine you're a member of a newly formed improvement team that has taken up the challenge to reduce health care-associated infections at your hospital. You have an idea for a change to the room cleaning process that you want to test, but you're slightly nervous because improper cleaning and disinfection can carry a high risk for patients with compromised immune systems. Which of the following is most important to determine the best size for your initial PDSA test? (A) Apply the 5X rule. (B) Apply the 1-2-3 rule. (C) Weigh the potential consequences of a test that does not lead to improvement against the degree of belief in success. (D) Weigh the potential consequences of a test that does not lead to improvement against the possible benefit of a test that does lead to improvement.

C With improvement work, you should weigh the potential consequences of a test that does not lead to improvement against the belief in success. How small your first PDSA cycle should be rests on your degree of belief and the stakes involved.

When planning a sequence of PDSA cycles for a change that involves patients, which of the following is a true statement? (A) Patient characteristics in each PDSA cycle should be as uniform as possible to allow valid comparisons. (B) The number of patients in each cycle should stay fixed, to allow valid comparisons. (C) We would expect the number of patients involved to very slowly grow from early cycles to later cycles. (D) None of the above

D As improvement work progresses and the number of cycles increases, we would expect the scope and scale of the tests to increase, meaning both a rapidly growing number of patients involved in the tests as well as increasing diversity in the test population.

Imagine you're a member of a newly formed improvement team that has taken up the challenge to reduce health care-associated infections at your hospital. You have an idea for a change to the room cleaning process that you want to test, but you're slightly nervous because improper cleaning and disinfection can carry a high risk for patients with compromised immune systems. You haven't run any PDSA cycles yet. Which of the following would be the best next step? (A) Have one housekeeper use the process with one room cleaning. (B) Have all housekeepers use the process for a week. (C) Have one housekeeper use the process on five room cleanings. (D) Confirm the "face validity" of the new cleaning process by demonstrating it with a couple of housekeeping staff members and a supervisor.

D Based on your concern about patient safety, you'd likely first want to confirm the "face validity" of the new cleaning process by demonstrating it with a couple of housekeeping staff members and a supervisor. When testing changes that involve patients, it can be helpful to conduct an initial test using staff only. A simulation or practice session before going live is often a good way to uncover issues with high risk.

Use the following scenario to answer the following two questions: During a clinical rotation on the medical-surgical floor of a hospital, you notice several patients have developed urinary tract infections (UTIs) associated with their Foley catheters (tubes inserted into the bladder to drain urine). Your staff physician agrees that this is a problem and offers to help with an improvement project. Together, you work through several PDSA cycles to reduce the rate of UTIs on your floor. Which of the following methods would you recommend to display your improvement data? (A) Draw a bar chart. (B) Write a list of numbers. (C) Create a two-column table. (D) Draw a run chart.

D Run charts are an effective way to view changes over time. They are much easier to interpret visually than a list of numbers or a static display of data such as a bar chart.

Which of the following statements is true about using data for improvement? (A) Both quantitative and qualitative data can be useful. (B) The data should tell a story. (C) A run chart is one of the most helpful tools for displaying data. (D) All of the above.

D The best answer is all of the above.

You volunteer at a student-run clinic associated with your academic health center. As a member of the student board, you are constantly looking for ways to improve the clinic. One common complaint is that it takes too long to check patients in once they arrive, and you decide to tackle this problem. Which of the following might be a process measure for this effort?

Number of students helping to check a patient in In this case, you're trying to decrease wait times by increasing the number of staff members on duty, so an appropriate process measure for this project would be number of students helping to check a patient in. Keeping track of costs associated with a new process could be a balancing measure. The average number of minutes between patient arrival at the clinic and completion of check-in may be a useful outcome measure.