Fun With Control Charts Asaph Rolnitsky, MD

FUN WITH CONTROL CHARTS OR: HOW TO IMPRESS YOUR FRIENDS WITH PROCESS CONTROL

Asaph Rolnitsky, MD Sunnybrook Health Sciences Centre NICU, University Of Toronto MSc candidate, Queen’s University VAQS Graduate

DISCLOSURE • No conflicts to disclose

April 5, 2016

1

Fun With Control Charts Asaph Rolnitsky, MD

OBJECTIVES • To understands what SPC is • To understand a run chart • To understand a control chart • To find tools for use • To have fun

CONTENT • • • • • • • •

April 5, 2016

History SPC simplified Basic definitions Run charts examples Control charts examples Analysis by rules Available tools Demonstration?

2

Fun With Control Charts Asaph Rolnitsky, MD

“YOU IDIOTS!!.. WE’LL

You idiots!GET We’ll never DOWN get this thing NEVER THING down the hole! THE HOLE.”

HISTORY • Walter Shewhart (1891 – 1967), a physicist, engineer, and statistician.

• The father of statistical quality control, the control charts and the Shewhart (PDSA) cycle.

• Working in Bell labs, Shewart developed a methods of measuring, analyzing, and presenting changes in product quality.

April 5, 2016

3

Fun With Control Charts Asaph Rolnitsky, MD

WHAT’S WRONG WITH DESCRIPTIVE STATISTICS?

THE PROBLEM WITH “BEFORE AND AFTER” • Before and after can be very impressive and bring your improvement project to its desired goal: presenting favourable data. • And maybe publication.

• In fact, descriptive statistics (before and after) does not show how you got there, and cannot predict where you’re headed.

April 5, 2016

4

Fun With Control Charts Asaph Rolnitsky, MD

PROCESS CONTROL VS DESCRIPTIVE STATISTICS Belly massage to reduce mor tality in hernia repair patients 12.00%

10.00%

Massage Massageq8h q8h 8.00%

6.00%

4.00%

6.3% 2.00%

3.4% 0.00% 1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Average Average Mortality Average Linear (Average) Linear (Average)

TREND LINE VS SPC

April 5, 2016

5

Fun With Control Charts Asaph Rolnitsky, MD

WHAT ARE CONTROL CHARTS? • Control charts present data dispersion over time. • They show the limits of the process, based on standard deviation of the data and where data is within the limits.

CONTROL CHART- WHY? • To distinguish between common cause and special cause variation. • (Out of control/special cause can be good)

• Center line often the mean. • UCL and LCL similar to standard deviation. • Type of analysis depends on distribution of data.

April 5, 2016

6

Fun With Control Charts Asaph Rolnitsky, MD

Visual display of data over time.

CONTROL CHART: REDUCING HYPOTHERMIA AT ADMISSION

DeMauro Pediatrics 2013

April 5, 2016

7

Fun With Control Charts Asaph Rolnitsky, MD

BELL CURVE (NOT TRUE FOR ALL CHARTS)

99.7% OF POINT IN A SAMPLE FALL WITHIN 3SDS.

April 5, 2016

8

Fun With Control Charts Asaph Rolnitsky, MD before

after

12

10

8

6

4

2

0 Jul-09

Jan-10

Aug-10

Feb-11

Sep-11

Apr-12

Oct-12

May-13

Nov-13

Jun-14

Dec-14

Jul-15

NICU i Chart 16 14.670 14

12

UCL

NICU

10

8 4

6.830 4

6 CL

4

2

0

DATE/TIME/PERIOD

April 5, 2016

9

Fun With Control Charts Asaph Rolnitsky, MD

COMPONENTS OF CONTROL CHARTS Data points

Data Control points limits

Central line

SD lines

COMPONENTS OF CONTROL CHARTS

PDSA1: new order set

target

April 5, 2016

10

Fun With Control Charts Asaph Rolnitsky, MD

ZONES Zone A Zone B Zone C Zone B Zone A

RUN CHART • • • •

April 5, 2016

The simplest chart. Simple to see major trends. Easily drawn, no necessary tool. Easy to present on a scoreboard • How many km I ran • How many calories I ate • How long it took to arrive to work • Etc…

11

Fun With Control Charts Asaph Rolnitsky, MD

RUN CHART • A “run” is a point or a cluster of points in any side of the MEDIAN line (and not ON the median).

• Special cause variation indication: • • • •

April 5, 2016

Too many/few runs (available table) A shift: ≥8 points on any side of the median. A trend of ≥6 (or 7?) up or down. “Astronomical point”

12

Fun With Control Charts Asaph Rolnitsky, MD

•

Special cause: • Too many/few runs? • 13/13. • A shift: ≥8 points? • no • A trend of ≥6? • No • Astronomical point? • ?No

Run Charts

Control Charts

Simple & easy to understand and interpret complex, not just one type to consider More Easy to create with paper or in Excel Need a special template or special Less sensitive and can miss some special

software

cause signals

More sensitive and powerful tool—control

No measure of the amount of variation or

limits

‘precision’ in the data

provide additional tests

Minimum 10 data points

Control limits show the precision and more accurately predict future behaviour Minimum 25 data points

April 5, 2016

13

Fun With Control Charts Asaph Rolnitsky, MD

TYPES OF CONTROL CHARTS •

Each chart looks at different characteristics of your data.

• • •

Weight loss

•

Each has different charts.

Medication errors Proportion of ROP

WHAT ARE WE MEASURING?

April 5, 2016

• • •

Sepsis episodes/month (some months have less patients)

•

Days between sepsis

•

Line insertions between infections

Patients with sepsis/month? (one patient can have>1) Sepsis episodes/100 patient days

14

Fun With Control Charts Asaph Rolnitsky, MD

WHAT TYPE? Discrete counts, different distribution

Continuous variables, normally distributed

WHAT TYPE? Counts for EVENTS, or AFFECTED subjects (ie: intubation attempts vs intubated patients Measurement for one observation (ie: Time to admission)

April 5, 2016

15

Fun With Control Charts Asaph Rolnitsky, MD

Type?

continuous

attribute

What are you counting?

Subjects (“defectives”) i.e.: children that had sepsis

XmR (I)

Events (“defects”) i.e.: Sepsis episodes

P

nP if constant sample or sample=1

U

C if constant sample or sample=1

EXAMPLES:

April 5, 2016

KQI

Type of chart Why?

Temperature at admission

X

Measured continuous variable

Number of blood transfusions given in the NICU

C

Counted (“defects”) or “events”

Sepsis workups, intubation attempts

C

Counted “events”

Abx days/100 patient

U

Counted “defects”, corrected/patient

EBM feeds/Total feeds

P

Counted defects, corrected/variable denominator

TPN days/100 admission days

Np

Counted defects, corrected/constant denominator

Days between line sepsis

T

Counted time between event

IVH>2

C

Counted “events”

Complaints/patient

P

Counted defects, corrected/variable denominator

16

Fun With Control Charts Asaph Rolnitsky, MD

WHY IS THE TYPE IMPORTANT? • Each control chart has specific calculation of control limits. • Choosing the wrong chart may falsely show your process is in control or not. • Some expert suggest to use I chart (XmR) as much as possible.

COMMON CAUSE VS SPECIAL CAUSE VARIATION • Common cause variation is the expected, random variability in data signals. • If process is stable but not optimal, may mean need for redesign. • i.e.: Sepsis rate is stable, but high.

April 5, 2016

17

Fun With Control Charts Asaph Rolnitsky, MD

COMMON CAUSE VS SPECIAL CAUSE VARIATION • Special cause is a signal that is statistically significantly deviating from the previous collection of points. • May mean a change to study, or may mean that a change is indeed causing an effect.

RULES OF SPECIAL CAUSE Westgard

AIAG

Montgomery Western Electric

Healthcare

Control Chart Rules 1. Points above UCL or Below LCL

1

1

1

1

1

1

2. Zone A n of n + 1 points above/below 2 sigma

2

2

2

2

2

2

3. Zone B n of n + 1 points above/below 1 sigma

4

4

4

4

4. n points in a row above or below center line

8

9

7

8

8

5. Trends of n points in a row increasing or decreasing

7

6

6

6

6

6. Zone C - n points in a row inside Zone C (hugging)

15

15

15

15

7. n points in a row alternating up and down

14

14

14

8

8

8

8. Zone C - n points in a row outside Zone C 9. Zone B n points above/below 1sigma; 2 points one above, one below 2sigma

April 5, 2016

NelsonJuran

8

4

18

Fun With Control Charts Asaph Rolnitsky, MD

CONTROL CHART RULES

120%

Rule 1: 1 Point above/Below CL

100%

P chart: Feeding Error Rates Rule 6: 15 Points in zone C (HUGGING)

Rule 5: 6 Points trending Rule 4: 8 Points in one side of CL (SHIFT)

80%

60%

40%

20%

0%

April 5, 2016

Rule 2: 2/3 Points above/Below 2SDs

Date/Time/Period

19

Fun With Control Charts Asaph Rolnitsky, MD

RULES TRANSLATING TO SPECIAL CAUSE VARIATION • The rules give description of when there is low probability that your measurements are due to common cause, and more likely represent special cause.

• The rules translate probabilities to points clusters.

EXAMPLES IN PLAIN LANGUAGE: • The chance to have a signal out of the 3SDs is