Quality Improvement in Hospitals OHA Survey Results: September 2013 Executive Summary The goal of continuous quality improvement (QI) within health care, which is the focal point of Ontario’s Excellent Care for All Strategy, is a driver of the significant reforms currently being implemented across the system. There is enormous opportunity in this initiative, which already has had a major, positive effect on hospitals. In order to better understand how hospitals are doing with respect to their QI efforts and the development of their QIPs, in April 2013, the Ontario Hospital Association (OHA) administered a QI survey to all Ontario hospitals. One-hundred and four hospital corporations responded to the survey (70% response rate), which consisted of 60 questions. The survey asked hospitals to evaluate their QI capacity, their 2013/14 QIP development process, and the degree to which they conduct critical incident and quality of care reviews. The following results have been shared with the QIP Task Group, a tripartite group with membership from the OHA, the Ministry of Health and Long-Term Care (MOHLTC) and Health Quality Ontario (HQO), as they develop and redesign hospital supports for future reporting periods. Results: Overview Overall, the results were positive, and in particular, showed strong leadership from hospital boards in terms of time invested in overseeing their organizations’ QI efforts. Over 60% of respondents reported that their Board spent at least 25% of their meeting time discussing quality, the benchmark established by the Institute for Healthcare Improvement.1 Additionally, Boards meeting this benchmark reported that their Quality Committee experienced fewer barriers when developing their annual QIP. The results also revealed the need to further strengthen QI capacity within the system. This will be critical to advancing change. QI capacity was evaluated in the context of the percentage of front-line staff formally trained in QI and the structure of QI resources across the organization. Only a small number of respondents reported that more than 25% of their front-line staff was trained in QI. Additionally, less than half of the respondents reported that there were sufficient opportunities for staff to participate in formal QI learning and education. Building QI capacity is central to leveraging the system’s limited resources in the most effective and efficient manner, particularly when it comes to the dissemination of best practices and providing implementation supports, including quality improvement training for front-line staff. QIP Development With respect to hospitals’ QIP development process, almost 50% of respondents engaged with external partners and other sectors when developing their QIP, and we found that various parts of the hospital took on different roles in the QIP preparation process. Over 75% of respondents indicated that the process is often led by QI staff and senior leadership. Over 70% of respondents indicated that Board members and the Quality Committee of the Board typically performed a review role. Over 60% of respondents indicated that the community/patients informed QIP development. Finally, more than 75% of respondents indicated that critical incident data informed the high priority objectives and initiatives within their QIP. These data show a high level of involvement from leadership, and the growing importance of community/patient involvement in the delivery of high-quality care. They also suggest a high uptake of the QIP supports made available for the 2013/14 reporting period, which point to the need and usefulness of such resources. This includes supports made available by HQO, such as use of the Navigator tool, the OHA’s Frequently Asked Questions and the QIP Guidance Document. While external supports were utilized by more than 70% of respondents, a number of 1

Institute for Healthcare Improvement. (2007). IHI Calls on Boards to Lead on Quality and Safety. Retrieved from: http://www.greatboards.org/newsletter/reprints/GB-Summer07-conway.pdf

1 September 3 2013

hospitals indicated that earlier dissemination of the various guidance documents would facilitate earlier review by internal quality committees, senior management and the Board. Regarding the final section of the survey, hospitals were not asked to report the number of incidents and critical incidents, as defined in the Public Hospitals Act. Rather, they were asked to report the number of incident reviews and Quality of Care Information Protection Act (QCIPA) reviews conducted annually. In both cases, the number of reviews conducted was relatively low. This finding may be reflective of the fact that a low number of incidents are occurring in hospitals, or that hospitals are moving towards incident review processes that are structured around reflective learning and patient involvement. The survey also allowed hospitals to provide recommendations on how to improve the QIP. Comments were received from 60% of respondents, and ranged from greater flexibility in indicator reporting periods, reassessing the submission deadline and providing more QIP guidance focused to Boards and Quality Committees. Limitations Limitations for each section are located in footnotes throughout the report.

2 September 3 2013

Table of Contents Respondent Profile ....................................................................................................................................................4 Section 1: Quality Improvement Capacity ...................................................................................................................4 Section 2: QIP Development .......................................................................................................................................9 Section 3: QIP Guidance and Supports ...................................................................................................................... 15 Section 4: Impact on Quality Improvement ............................................................................................................... 23 Section 5: Future QIP Directions ............................................................................................................................... 25 Section 6: Critical Incident Reporting ........................................................................................................................ 27 Section 7: Quality of Care Reviews............................................................................................................................ 29

3 September 3 2013

Respondent Profile Survey Respondents by Hospital Type Hospital Type Academic Addiction & Mental Health CCC & Rehab Community Small TOTAL

Number of Respondents 11 3 11 43 36 104

Provincial Response Rate* 69% 60% 79% 74% 65% 70%

*total hospitals in each type / number of survey respondents (data source: OHA Hospital Listing (2013))

4 September 3 2013

Section 1: Quality Improvement Capacity The findings in this section reflect the investments hospitals have made in quality improvement (QI) knowledge, skills and activities. QI capacity was examined in two ways: the percentage of front-line staff trained in QI and how QI is structured throughout the hospital. 1. QI Training and Structure The literature suggests that QI skills are increasingly necessary for all hospitals and medical staff as they increase the readiness of staff to participate in QI projects2. It also suggests that a more broadly shared QI skill base enables staff to identify improvement opportunities in the course of their work3. Figure 2. Organizational structure of QI capacity (N=104)

Figure 1. Percentage of front-line hospital staff who have formal QI training (N=104)

70%

1%

11% < 25% 26-50% 51-75% 76-100%

Percentage of Respondents

2%

63%

60%

52%

50% 40% 30% 20% 10%

86%

0% Centralized

Decentralized

Note: Check all question. See Footnote #4



As shown in Figure 1, the vast majority of hospitals (89 hospitals; 86%) reported that less than a quarter of their front-line staff4 lacks formal training in QI.



Figure 2 suggests that hospitals structure their QI resources both centrally and throughout their organization5.

2

Adler et al. (2003). Performance improvement capability: keys to accelerating performance in hospitals. California Management Review: 45(2). Weiner et al. (2006). Quality Improvement Implementation and Hospital Performance on Quality Indicators. Health Services Research: 41(2). 4 Limitation: Formal QI training’ was not explicitly defined in the survey. Other than critical analysis methods and process mapping, examples of formal QI training were not provided. As such, hospitals may have interpreted formal QI training in different ways. 5 Limitation: The survey asked hospitals ‘How is QI capacity structured in your organization?’ The options were centralized, decentralized, other and not sure. Respondents were able to select more than one option. 27 hospitals (26%) selected BOTH decentralized and centralized. 3

5 September 3 2013

Percentage of Front-Line Formally Trained in QI

Figure 3. Profile of Hospitals with Less than 25% front-line trained in QI (n = 90)

32 35

< 25%

10

Small Community

3

CCC & Rehab Addiction & Mental Health

10

Academic

0

5

10

15

20

25

30

35

40

Number of Hospitals



Of those hospitals reporting that less than 25% of their front-line staff had formal QI training, the majority were from small or community hospitals.6

2. QI Education and Supports Figure 4. Hospitals Reporting Sufficient Opportunities to Participate in Formal QI Learning and Education (N=104) 7%

43%

50%

Yes No Not sure



As shown in Figure 4, half of the survey’s respondents reported that there were insufficient opportunities for staff to participate in QI learning and education (45 hospitals; 43%).

6

Note that most of the survey’s respondents were from small and community hospitals. The 32 small hospitals represent 89% of small hospital respondents; the 35 community hospitals represent 81% of community respondents; the 10 CCC/Rehab hospitals represent 91% of CCC/Rehab respondents; the 3 mental health hospitals represent 100% of mental health respondents; and the 10 academic hospitals represent 91% of academic respondents.

6 September 3 2013

Figure 5. Types of QI education and/or training used in hospitals for hospitals with suffient opportunities for staff participation in formal QI learning and education (n=45)

100%

96%

98% 93% 89%

90% 80%

Percentage of Respondents

80%

78% 67%

70% 60% 50%

42% 40% 30% 18%

20%

18%

10% 0% Model of Improvement (PDSA)

Process mapping, process/ systems redesign

Critical Measurement Change Project analysis (types of management management methods (e.g. measures, skills skills root cause data analysis, collection, Failure Mode analysis) and Effects Analysis)

LEAN methods

Six Sigma methods

IHI Open School

*Other

Note: "Check-all’ question. Each response option (i.e. LEAN methods) has an “n” of 45



As shown in Figure 5, of the 45 hospitals that indicated that there were sufficient opportunities for QI education or training, almost all used: o Critical analysis methods (98%), o Process mapping (93%), and the o Plan-Do-Study-Act model of improvement (96%)



A small number of hospitals (18%; 19 hospitals) reported use of ‘other’ types of QI education and training. These included training from Accreditation Canada (Tracer training) and the Canadian Patient Safety Institute (PSEP training), along with the use of some project and strategic management tools such as the balanced scorecard and A3 methodology.

7 September 3 2013

External QI Supports

Figure 6. External Sources of QI Education and Training for organizations that used external sources and information to support and develop QI initiatives (n=73)

90%

84%

80%

77% 70%

Percentage of Respondents

70% 60%

60%

67%

58%

50% 41%

40% 30% 21%

20% 10% 0% Private sector Ministry of Ontario Hospital Health Quality consultants Health and Long- Association Ontario Term Care

Accreditation Canada

Quality Healthcare Network

Institute for Safe Medication Practices Canada

*Other

Note: "Check-all’ question. Each response option (i.e. Accreditation Canada) has an “n” of 73



Of the 70% of respondents that indicated their organization used external sources of expertise and information to develop QI initiatives (73 hospitals), the majority used supports from Accreditation Canada (84%), the Ontario Hospital Association (77%) and Health Quality Ontario (70%) (Figure 6).



A small number of hospitals (21%; 22 hospitals) reported they used other external groups for QI support, such as the Centre for Healthcare Quality Improvement, ThedaCare, Safer Healthcare Now!, and the NHS Change Framework.

8 September 3 2013

Section 2: QIP Development The findings in this section provide an overview of the processes hospitals used to develop their Quality Improvement Plans (QIPs) for 2013/14. These processes are categorized into four sections: (1) the hospital groups involved in developing the QIP, (2) contribution of the Board and Quality Committee to the QIP development process, (3) QIP timelines and external engagement, and (4) LHIN support. 1. Hospital Groups Involved in the QIP Development Process Hospitals were asked how members of their organization participated in developing their 2013/14 QIP. Hospitals did this by assigning roles7 for each group of staff in their organization. The roles were defined as lead, review, inform or no role. “Lead” refers to the initiation and execution (or delegation of execution) of all QIP activities. “Review” is an oversight role in which the group ensures accuracy and appropriateness of the QIP and its components. “Inform” is a functional role in which the group contributes data, ideas or concepts to formulate the QIP and its components. Results are summarized below and in Figures 7-9. Figure 7. Groups that LED the 2013/14 QIP Preparation Process (n=104) *Other Community/patients Front-line care providers Quality improvement staff Clinical and service provider leaders and managers Senior leadership Board members Medical Advisory Committee Quality Committee of the Board under ECFAA 0%

10%

20%

30%

40%

50%

60%

70%

80%

Percentage of Total Responses



Figure 7 suggests that QI staff and senior leadership typically lead the QIP preparation process (over 70% of hospitals). Front-line staff, Board members and the Medical Advisory Committee (MAC) rarely took on leadership roles.



The 5 hospitals (5%) that provided an “other” response commented that middle management, senior physician groups, the internal Quality/Patient Safety/Risk Management Committee and the Family Health Team led the QIP preparation process.

7

Limitation: These roles were not explicitly defined in the survey. For example, hospitals may have been unable to distinguish between a ‘review’ and ‘inform’ role and selected both for a particular group.

9 September 3 2013

Figure 8. Groups that had a REVIEW role in the 2013/14 QIP Preparation Process (n=104) *Other Community/patients Front-line care providers Quality improvement staff Clinical and service provider leaders and managers Senior leadership Board members Medical Advisory Committee Quality Committee of the Board under ECFAA 0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Percentage of Total Responses



Figure 8 suggests that the Board and the Quality Committee of the Board usually take on a review role in the QIP preparation process (over 70% of hospitals). The MAC, clinical/service provider leads and senior leadership also take on a review role at almost 50% of hospitals.



“Other” QIP reviewers (6%; 6 hospitals) included the Family Health Teams, physicians and various internal hospital committees (Quality and Patient Safety Committee, Quality and Risk Committee and the Family Advisory Committee).

Figure 9. Groups that had an INFORM role in the 2013/14 QIP Preparation Process (n=104) *Other Community/patients Front-line care providers Quality improvement staff Clinical and service provider leaders and managers Senior leadership Board members Medical Advisory Committee Quality Committee of the Board under ECFAA 0% 10% Note: "Check-all’ question. Each response category (i.e. Front-line care providers) and option (i.e. inform) has an “n” = 104

20%

30%

40%

50%

60%

70%

Percentage of Total Responses



As shown in Figure 9, approximately 60% of hospitals reported that the community/patients and front-line providers informed QIP development. Almost 50% of hospitals indicated that clinical/service provider leads and the MAC also informed the QIP preparation process.



A small number of hospitals (15 hospitals; 14%) indicated that the LHIN, HQO’s feedback sessions at HealthAchieve and other heath sector providers also informed their QIP preparation process. 10

September 3 2013

2. QIP Contributions by the Board and Quality Committee 25% Threshold for Discussing Quality The Institute for Healthcare Improvement (IHI) 8 recommends that a Board should spend at least 25% of its meeting time discussing quality. Figure 10. Percentage of Hospital Board Meeting Time Spent Discussing Quality (n=104) 34%

/= 25%

66%



As shown in Figure 10, the majority of hospitals (63; 66%) reported that their Board met or exceeded the 25% threshold for discussing quality.

Contributions by the Quality Committee Figure 11. How the Quality Committee Contributed to the development of the 2013/14 QIP (n=104) 100% 90% 90%

85%

Percentage of Respondents

80%

75%

71%

70% 57%

60% 50% 40% 30%

18%

20% 10%

1%

4%

0% By raising priority By reviewing the By discussing and By ensuring issues for inclusion drivers, indicators, analyzing details of alignment of the QIP into the QIP for and people the QIP in more with the organization’s Board approval responsible for the depth than the big dot indicators board at-large does strategic plan and priorities

By implementing QIP priorities

By monitoring and making recommendations to the Board regarding QI initiatives and policies

Not sure

*Other

Note: "Check-all’ question. Each response option (i.e. by implementing QIP priorities) has an “n” of 104

8

Institute for Healthcare Improvement. (2007). IHI Calls on Boards to Lead on Quality and Safety. Retrieved from: http://www.greatboards.org/newsletter/reprints/GB-Summer07-conway.pdf

11 September 3 2013



Figure 11 suggests that Quality Committees contribute to QIP development by discussing details of the QIP in more depth than the Board (90%; 94 hospitals) and by ensuring alignment with organizational priorities (85%; 88 hospitals). Few hospitals (18%; 19 hospitals) reported that their Quality Committee implements QIP priorities.



The contributions that Quality Committees make to the QIP development process are consistent with their role as a reviewer of the QIP, as found in Figure 8.

Barriers Faced by the Quality Committee Figure 12. Barriers Faced by the Quality Committee when developing the 2013/14 QIP

50%

46%

45% 40% 35%

Percentage of Respondents

35%

32% 29%

30% 26%

25%

Board Spends Less than 25% of their Time Discussing Quality (n=34)

23% 21%

20% Board Spends At Least 25% of its Time Discussing Quality (n=70)

14%

15%

12%

10% 6%

6%

6%

5% 0% Insufficient time for Difficulty in measuring Poor internal data Lack of QI knowledge Lack of direction planning and current performance quality for current around expecations completion and targets performance/targets

No barriers



As shown in Figure 12, almost half of hospitals (46%; 47 hospitals) that devoted at least 25% of their meeting time to discuss quality reported that their Quality Committee faced no barriers when developing their QIP. Boards that did not meet the 25% threshold reported this to be true only 35% of the time.



Boards that did not meet the threshold indicated that their Quality Committee had difficulty measuring current performance and targets, lacked QI knowledge and reported poor internal data quality on current performance. Hospitals that did not meet the threshold reported having no barriers 35% of the time versus 46% for hospitals that met the threshold.

12 September 3 2013

3. QIP Timelines and External Engagement Figure 13. When hospitals begin development of their 2013/14 QIP (N=104)

10% Q2 (July - September)

36%

Q3 (October - December) Q4 (January - March) 54%



As shown in Figure 13, more than half of the respondents (52 hospitals; 54%) indicated that they developed their QIP in the third quarter of the fiscal year (Q3; October-December). 38 hospitals (36%) indicated that they developed their QIP in Q4 (January – March). Only 10 hospitals (10%) developed their QIP in Q2 (July – September) and no hospitals reported developing their QIP in Q1 (April – June).



More than half of the respondents (59 hospitals; 57%) submitted QIP Progress Reports to the Quality Committee quarterly. The other half either submitted them monthly or based on the frequency of their Quality Committee meetings (bi-monthly or five times a year).

Sectors Engaged in QIP Development Figure 14. Sectors Engaged in Developing the QIP 70%

(of the organizations that engaged with other sectors to develop QIP, n=50) 58%

Percentage of Respondents

60%

56%

50% 40% 40% 30% 22% 20% 10% 0% Primary Care

Long-term Care

Home Care/Community Care

Other hospitals

Note: "Check-all’ question. Each response option (i.e. by primary care) has an “n” of 50



About half of the respondents (50 hospitals; 48%) indicated that they engaged with external partners and other sectors when developing their QIP (Figure 13). Of the 50 hospitals: o 58% (60 hospitals) engaged with the home care/community care sector o 56% (58 hospitals) engaged with other hospitals

13 September 3 2013

4. LHIN Support The ECFAA implementation survey (2011) revealed that most hospitals did not engage their LHIN when developing their QIP. While this survey did not ask this exact same question, it did ask hospitals to indicate whether their LHIN requested a draft of their 2013/14 QIP prior to submission to HQO, whether or not it requested changes to the QIP, and if it facilitated any LHIN-wide QIP discussion. Figure 15. Hospitals Reporting that their LHIN Requested a Draft of their QIP Prior to Submission (N=104)

Figure 16. Percentage of respondents whose LHIN requested changes to their QIP (N=104) 2% 3%

4%

33%

Yes

Yes

No

No

Not sure

Not sure 63% 95%

Figure 17. Did your LHIN facilitate LHIN-wide QIP discussion or information sharing? (n=104)

Percentage of Respondents

60%

55%

50% 40% 30%

29%

20% 10%

14% 6%

0% Yes; with other hospitals Yes; with other sectors

No

Not sure



As shown in Figure 15, one-third of respondents (33%; 34 hospitals) reported that their LHIN requested a draft of their QIP prior to submission.



Almost all of the respondents (99 hospitals; 95%) indicated that their LHIN did not make changes to their QIP prior to submission (Figure 16).



As shown in Figure 17, more than a quarter of respondents (30 hospitals; 29%) reported that their LHIN facilitated LHIN-wide QIP discussion with other hospitals.

14 September 3 2013

Section 3: QIP Guidance and Supports This section provides an overview of the suite of Quality Improvement Plan (QIP) supports that were made available to hospitals during the 2013/14 reporting year. The results are organized according to five sections: (1) QIP supports, (2) the quality of the supports and responses received, (3) the impact of the release date of the QIP Guidance Document, (4) HQO supports and (5) additional supports. The findings are shown below. 1. QIP Supports Figure 18. Quality Improvement Plan Supports Accessed for the 2013/14 Reporting Year All Survey Responses (n = 104, for each question)

100%

90% 90%

83%

Percentage of Respondents

80% 67%

70%

64%

64%

60% 50% 40%

36%

30%

36%

23% 17%

20% 10% 10%

4%

0% Ministry of The Ministry Health and Longof Health Term Care

Health Quality Ontario

Ontario Onatario Hospital Hospital Association Association

Organizations Accessed for QIP Support*

Yes

No

Use of HQO’s Analysis Report

Yes

No

Use of HQO’s Benchmarking Update

Yes

No

Use of HQO’s Navigator Tool

Yes

No

Participation in HQO’s Feedback Sessions

*Note: ‘Support’ refers to email or phone assistance provided by these organizations. This does not refer to accessing specific guidance documents or attending feedback sessions.



As shown in Figure 18, there was a high uptake of QIP supports made available for the 2013/14 reporting period.

15 September 3 2013

2. Quality of Supports and Responses Received

QIP Guidance HQO Analysis for OHA QIP Frequently Document (2013/14) Improvement Report Asked Questions (n=104) (2012/13) (n=104) (2013/14) (n=104)

Figure 19. Helpfulness of Guidance and Support Documents (n=104)

1 3

20

45

27

8 Not helpful Slightly helpful Moderately helpful

1

6

26

42

26

3 Very Helpful Extremely helpful No response was received

02

0%

11

10%

47

20%

30%

42

40%

50%

60%

70%

80%

2

90%

100%

Percentage of Respondents



As shown in Figure 19, guidance materials from HQO, the MOHLTC and the OHA were helpful for a majority of respondents.

3. Timing of QIP Guidance Document 

Hospitals were asked to comment on whether or not the November 2012 release date of the QIP Guidance Document (2013/14) allowed sufficient time to develop their QIP.9



While the majority of hospitals (71%) said ‘yes,’ 35 hospitals (34%) provided comments in three areas: 1) Timing: hospitals reported that the November release date did not leave enough time for sufficient internal approvals, LHIN review, or stakeholder consultation. Some hospitals also reported having begun their QIP development process before the guidance document was released. 2) Indicators: the misalignment between internal reporting periods and those indicated in the guidance document and lagging baseline data were raised as concerns. 3) Distribution: a few hospitals mentioned that guidance materials should be released as they become available rather than as a batch in November.

9

Limitation: Hospitals that replied both ‘yes’ and ‘no’ to whether or not the November release date of the QIP allowed for sufficient time to complete the QIP were asked to provide additional comments. It is thus possible that a hospital responded ‘yes’ to the question and also provided a contradictory/negative comment.

16 September 3 2013

Figure 20. Did the Release Date of the 2013/14 QIP Guidance Document Allow for Sufficient Time to Complete Your QIP? By Hospital Type (n=104) Small

33

Community

3

28

CCC & Rehab

15

4

7

Addiction & Mental Health

Yes

3

Academic

0

6 0%

10%

20%

No

5

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of Respondents



As shown in Figure 20, almost two-thirds of CCC/rehab hospitals that responded to the survey (7 hospitals; 64%) indicated that the November release date of the guidance document did not allow for sufficient time to complete their QIP. One the other hand, almost all small hospitals responding to the survey indicated that the release date was sufficient.

4. HQO Supports a) HQO Analysis for Improvement Report (2012/13) 

As shown earlier in Figure 18, a very small number of respondents (10 hospitals; 10%) indicated that they did not use HQO’s Analysis for Improvement Report (2012/13) when completing their 2013/14 QIP.



The 94 hospitals that used HQO’s Analysis report when completing their QIP were asked how they used it. Figure 20 displays these results. Figure 21. Use of HQO's Analysis for Improvement Report (2012/13) When Completing the 2013/14 QIP (n= 94)

For assistance in selecting priorities for improvement

19%

22% To clarify the difference between proprity levels 1,2,3 and which areas should be focused on To understand how to set appropriate targets

22%

17% For assistance in developing change ideas 20%

To understand and incorporate components of exemplary plans

17 September 3 2013



5 of the 10 hospitals that did not use HQO’s Analysis Report commented that they were either unaware of the report, did not feel that the report was applicable to their organization, or felt that the timing of the report was insufficient for completing their QIP (Figure 22). The 5 hospitals who provided an ‘other’ response commented that they did not use the report due to availability of other useful resources, and that they had engaged with other hospitals when developing their change initiatives and priorities. Figure 22. Reasons Why Hospitals Did Not Use HQO's Analysis for Improvement Report when completing their 2013/14 QIP (of hospitals that did not use HQO's Analysis Report, n=10)

10%

20%

Unaware of Report Timing

50%

Not applicable to my organisation Other 20%

b) HQO Benchmarking and Target Setting Update Figure 23. Reasons why organizations did not use QIP Benchmarking and Target Setting Update in developing their 2013/14 QIP (of hospitals that did not use the Update, n=18)

30% 35% Unaware of update Timing (released too late) Not relevant to indicators on my QIP

35%



As shown in Figure 23, a small number of respondents (18 hospitals; 17%) indicated that they did not use the HQO QIP Benchmarking and Target Setting Update when developing their QIP for 2013/14. These 18 hospitals were asked why they did not use the update, to which they responded that they were unaware of the update (30%), the update was released too late (35%), and/or the update was not relevant to indicators on their QIP (35%). 18

September 3 2013

c) HQO QIP Navigator Figure 24. Reasons for not submitting by QIP Navigator (for hospitals that did not use QIP Navigator to submit 2013/14 QIP, n=37)

8%

Difficulties logging-on and accessing the Navigator

8% 29%

Difficulties maintaining version control 14%

Staff preference towards traditional submission format Already began developing QIP in Excel

2%

Lack of support and technical assistance Other

39%



As shown in Figure 24, initial development of the QIP in Excel was often cited as the top reason why hospitals did not use Navigator tool to develop and/or submit their 2013/14 QIP (37 hospitals; 36%). The 11 hospitals that provided an ‘other’ response indicated difficulties with incorporating internal hospital data and functionality issues with the tool.

Figure 25. How the QIP Navigator Was Used* for the 2013/14 QIP (n=104) 6%

As a space to facilitate inter-departmental collaboration and QIP completion

22%

As a tool to submit the finalized QIP

12% 60%

As a resource to confirm reporting periods, targets amd indicator definitions As a repository for QIP guidance materials



Respondents were also asked to indicate how they used10 QIP Navigator for the 2013/14 QIP. As shown in Figure 25, more than 50% of hospitals used the tool to submit their finalized QIP.

10

Limitation: The survey asked hospitals how they used the Navigator tool for developing their 2013/14 QIP regardless of whether a hospital used the tool to develop/submit their QIP. These results are thus subject to limitations as only 36% of respondents indicated that they did not use Navigator to develop and/or complete their 2013/14 QIP.

19 September 3 2013

Figure 26. Hospitals Intending to Use Navigator to Submit 2014/15 QIP (n=104)

27% Yes No 4%

Not sure 69%



As shown in Figure 26, the majority of respondents (72 hospitals; 69%) indicated that they intend to use the QIP Navigator to develop and/or submit their QIP for 2014/15.

Figure 27. When Did Your Hospital Begin Developing the 2013/14 QIP? Respondents who used QIP Navigator (n=65)

Figure 28. When Did Your Hospital Begin Developing the 2013/14 QIP? Respondents who did not use QIP Navigator (n=35) 3%

14% 28%

Q2: Jul- Sept Q3: Oct-Dec

Q2: Jul- Sept 51%

Q4: Jan- Mar

46%

Q3: Oct-Dec Q4: Jan- Mar

58%



As shown in Figures 27 and 28, the hospitals that used the Navigator tool to develop and/or submit their 2013/14 QIP began developing their QIP earlier than those that did not use the Navigator tool. A possible reason for this is that those hospitals beginning the QIP development process later may have had less time to get accustomed to the new Navigator tool, as it was formally released on December 5, 2012.

d) HQO Feedback Sessions The questions in this section of the survey asked hospitals to evaluate their participation in HQO’s individualized feedback sessions and/or the peer-to-peer feedback sessions, in accordance with the sessions listed in the table below. The questions asked in this section were not specific to the individualized or peer-to-peer feedback sessions. Table 1. HQO Feedback Sessions Individualized Feedback Sessions

Peer-to-Peer Feedback Sessions

HQO Health Quality Transformation Conference 2012 October 23, 2012

Session #1 December 4, 2012, CE LHIN

20 September 3 2013

OHA HealthAchieve 2012 November 5, 6, 7, 2012 Various virtual sessions provided by HQO as requested by hospitals

Session #2 January 9, 2013, NE LHIN Session #3 January 16, 2013, OHA Region 2 (Ottawa)

Figure 29. Hospitals that Participated in HQO's Feedback Sessions (as listed in Table 1) (n=104)

Figure 30. Were Sessions Helpful in Developing the 2013/14 QIP (for the hospitals that participated in any of HQO's feedback sessions, n=67)

13% 36% Yes

64%



Yes

No

87%

No

As shown in Figure 29, the majority of respondents (67 hospitals; 64%) reported that they participated in HQO’s feedback sessions. Of these hospitals, almost 90% of them found the sessions helpful in developing their 2013/14 QIP. The reasons why these hospitals found these sessions helpful is outlined in Figure 31.

Figure 31. Reasons Why HQO Feedback Sessions Were Helpful (Respondents that participated in the HQO's QIP feedback sessions (n=58)) Helped to clarify aspects of the QIP development and submission process

9% 33% 21%

Assisted in setting targets and developing change ideas Provided useful information about the performance of my organisation's peer hospitals Other

37%



As displayed in Figure 31, 9% of respondents (16 hospitals) provided an “other” response. Some members also commented that the sessions provided a great networking opportunity and the sessions assisted in improving various problem areas in their QIP.



A very small number of hospitals that attended the feedback sessions found them unhelpful in developing their 2013/14 QIP (9 hospitals; 13%). Most of these hospitals reported that the sessions did not provide them with any new information or that they were not prepared with a draft of their QIP.

21 September 3 2013

Figure 32. Would you recommend feedback sessions to continue? (Respondents that participated in

Figure 33. Would you recommend feedback sessions to continue?

HQO's feedback sessions; n=67)

(Respondents that did not participate in HQO's feedback sessions; n=37)

3%

14% Yes

Yes

No

No 86%

97%



As shown in Figures 32 and 33, both hospitals that participated and did not participate in the feedback sessions recommended that they should continue.



A small number of hospitals (15; 14%) provided suggestions as to how the feedback sessions could be improved. These ranged from offering these sessions earlier, the need to involve the LHINs in the sessions and the offering of alternative modes of delivery (e.g., videoconference).



In terms of timing of the sessions, the majority of respondents (77 hospitals; 73%) indicated that the feedback sessions should be offered between December and February.

5. Additional Supports

90%

Figure 34. Preferred ways of accessing on-going supports pertaining to QIP development (n=104) 82%

80%

Percentage of respondents

70%

69%

68% 58%

60%

55%

50% 41% 40% 30% 20% 10% 0% Web-based platforn/ community of practice

Conferences/ forums/ quality congresses

Webcasts

Regional groups

Groups of comparative hospitals

Individual feedback

Note: ‘Check-all’ question. Each response option (i.e. webcasts) has an “n” of 104



As shown in Figure 34, the majority of hospitals preferred to access on-going QIP development supports through webcasts (82%; 85 hospitals), web-based platforms (69%; 72 hospitals) and from groups of comparative hospitals (68%; 71 hospitals).

22 September 3 2013

Section 4: Impact on Quality Improvement The findings in this section help in informing how the Quality Improvement Plan (QIP) has impacted hospitals in Ontario. Where applicable, data from the ECFAA Implementation Survey (2011) was included as a point of comparison and to determine whether or not the impact has changed. 1. Impact on QI Activities Figure 35. How the QIP Has Impacted Your Organization (comparision of 2010/11 to 2013/14) 40% 36%

2013/14 (n=104)

35%

2010/11 (n=77)

31% Percent of respondents

30% 26% 25%

24% 20%

21%

20%

17% 15%

15% 9%

10% 5% 1% 0% Encouraged our Encouraged our Encouraged staff to Encouraged the Board to organizationtototalk talkabout brainstorm around new organisation talk about quality and about quality and quality and quality quality improvement quality improvement quality improvement to improvement to a greater initiatives a greater degree degree than wethan were we were doing before doing before



Encouraged stronger interdepartmental collaboration

Other

As shown in Figure 35, there are two notable shifts in how the QIP has impacted hospitals between the 2010/11 and 2013/14 reporting periods. The first is that a larger number of respondents indicated that the QIP has “encouraged their organization to talk about quality and QI to a greater degree than before” for the 2013/14 reporting year. The second is that a fewer number of respondents reported that the QIP has “encouraged their Board to talk about quality and QI” for the 2013/14 reporting year.

2. Participation in Other QI Initiatives 

Almost all of the respondents (95 hospitals; 91%) indicated that their organization participated in other QI initiatives in addition to those included in their 2013/14 QIP. The majority of these hospitals stated that these initiatives were not included in their QIP (87 hospitals; 91%). One hospital commented that only initiatives for priority areas are included in the QIP.

23 September 3 2013

3. Patient Awareness of the QIP Figure 36. How Patients Are Made Aware of the 2013/14 QIP (n=104) 100% 91% 90%

Percentage of respondents

80% 70% 60% 60% 50% 40% 26%

30% 20% 10%

13%

12%

9%

2%

0%



As shown in Figure 36, most respondents (95 hospitals; 91%) indicated that their patients were made aware of their 2013/14 QIP through communication on their website. A large number of respondents (62 hospitals; 60%) indicated that patients were made aware through the QIP Short Form.



A small number of respondents (9 hospitals; 9%) provided ‘other’ responses that listed other ways patients were made aware of their QIP. Comments included: through a local newspaper, community advisory committee, youth advisory committee, media release, CEO forum, radio talk show and through information screens throughout the hospital.

24 September 3 2013

Section 5: Future QIP Directions Hospitals were asked to provide general feedback on all aspects of the Quality Improvement Plan (QIP) and recommendations for how the QIP can be improved in future years. The results have been grouped into four themes and are summarized in Table 2. Table 2. QIP Feedback and Improvement Opportunities Theme

Challenge with QIP (n = 64)

QIP Goals

1.

Utility  QIP is not reflective of all QI initiatives  Internal corporate and strategic planning tools are more meaningful to staff, physicians and the Board.

2.

Alignment  Reporting requirements associated with QIP, Quality-Based Procedures (QBPs), the Canadian Hospital reporting Project and quality indicators for the LHIN should be aligned.  QIP does not align with internal quality frameworks such as the Balanced Scorecard. Availability of Guidance  QIP Guidance Document should be released earlier.  Unclear where to access resources.

QIP Development

3.

4.

5.

QIP Submission

6.

Quality of Supports  QIP Excel Spreadsheet not user-friendly or easily understood by the public.  Promote collaborative feedback. Quantity of Supports  More individualized/peer to peer feedback.

LHIN Involvement  Low LHIN involvement in QIP.  Difficult to submit a meaningful QIP one month before the April 1 deadline.

7.

Prescriptiveness of Navigator  Pre-populated indicators cannot be removed  300-word limit for change ideas is insufficient  Lack of French templates.

8.

Reporting  Difficultly achieving and appropriately setting goals, change ideas and stretch targets on an annual reporting cycle.  The QIP should not be publicly disclosed in the same format in which it is submitted to HQO.

Improvement Opportunities/ Recommendations (n = 70)  Streamline QIP reporting with other provincial reporting requirements (e.g., performance accountability agreements, Senior Friendly Plan).



Timelier availability of QIP guidance and communication about modifications to the submission process.



Increase opportunities for hospitals to receive specific, hospital-level feedback.



Creation of a central repository for everything related to the QIP (i.e. notification emails, Bulletins, specific resources and schedules for feedback sessions).



Offer more QIP education and webcasts to address any issues throughout the QIP development process. QIP as a multi-year plan (2-3 year reporting requirement).





Reporting requirements should be modified for smaller hospitals with fewer QI staff.

25 September 3 2013

Theme

Challenge with QIP (n = 64) 

9.

Indicators and Reporting Periods

Improvement Opportunities/ Recommendations (n = 70)

Performance-based compensation should be an optional inclusion.

Performance-based Compensation  The requirement to link compensation to the QIP has left front-line staff with the impression that senior management are only interested in QIP targets tied to compensation.  Requirement is unfair for hospitals that do not have a framework for “bonuses.” This requires the claw-back of wages and at-risk base compensation.  At-risk pay (not bonus) minimizes the incentive for hospitals to establish stretch targets.

10. Indicator Selection  Resources required for more meaningful selection in small hospitals.  Lagging baseline data is a concern and causes misalignment in internal goal setting processes and QIP planning.  Lack of clarity regarding the inclusion of priority 2 and 3 indicators in QIP.



Continue to categorize core indicators by hospital type and make these available earlier.



Create an indicator/metric development section in the QIP Guidance Document.



Timelier availability of indicator data and standardized reporting periods.

11. Indicator Reporting Periods  Lack of consistency in indicator reporting periods caused confusion.



Creation of a central repository for all indicators specific to hospital types.



Re-assess the validity of the priority-level framework.

26 September 3 2013

Section 6: Critical Incident Reporting The results from this section inform how hospitals report critical incidents within their QIP. The results are split into two sections: (1) frequency of critical incident reporting to the Quality Committee and (2) the aggregation of critical incidents. 1. Frequency of Critical Incident Reporting to Quality Committee Figure 37. How often is aggregated critical incident data provided to your Quality Committee of the Board (n=104) 50% 44%

Percentage of respondents

45% 40% 35% 30%

29%

25%

21%

20% 15% 10%

10% 5% 0% Twice a year



Quarterly

Monthly

Other

About half of the respondents indicated that their critical incident data is aggregated and provided to the Quality Committee of the Board quarterly (44 hospitals; 42%). Twenty-nine hospitals (28%) provided this data to the Quality Committee twice a year, and 21 hospitals (20%) provide this data monthly. o A small number of hospitals commented that they provided aggregated critical incident data to the Board at every Board meeting (i.e., standing agenda item) or as soon as possible following the critical incident.

2. Aggregation of Critical Incidents 

Most hospitals (90%) indicated that critical incident data was aggregated by incident type or severity. o A small number of hospitals commented that their organization aggregated critical incident data individually, by department/program, or that there were not enough incidents to aggregate the numbers.

27 September 3 2013

Figure 38. How aggregated critical incident data is incorporated into the QIP (n=104) 80%

76%

Percentage of Total Respondents

70% 60%

58%

50% 40% 30% 20%

24% 18%

10%

6%

3%

0% Critical incident data The contributing Addressed in the QIP Reporting trends/ informs the high factors/ root causes narrative results are tied to priority identify several executive initiatives/objectives methods/ process compensation within the QIP measures for template improvement within the QIP template

Other

Not sure



As shown in Figure 38, more than 75% of the responding hospitals indicated that critical incident data informed the high priority initiatives/objectives within their QIP. About 60% of the hospitals stated that the contributing factors and root causes associated with the critical incidents informed process measures for improvement within the QIP template. About a quarter of the respondents stated that aggregated critical incident data was addressed in the narrative portion of the QIP.



A small number of respondents (2 hospitals; 2%) commented that there were not enough critical incidents to inform the QIP.



A quarter of respondents (27 hospitals; 26%) reported that they require additional supports regarding the aggregation of critical incident data. Supports requested included: o Development of a webinar or guidance document that clarifies how critical incidents should be incorporated into the QIP o Development of a consistent, province-wide definition of a critical incident o Examples of how other hospitals present their critical incident data to their Board

28 September 3 2013

Section 7: Quality of Care Reviews Quality of care reviews may reveal significant improvement opportunities for care processes and structures. In 2004, the Quality of Care Information Protection Act (QCIPA) was introduced. The Act protects information that is collected by or prepared for a quality of care committee from disclosure, subject to limited exception. The OHA conducted a survey in November 2010 to better understand the degree to which hospitals were using QCIPA when conducting incident reviews. Where applicable, the results from the 2010 survey are included as a point of comparison. 1. Incident Reviews Hospitals were asked to indicate the total number of incident/adverse event reviews conducted during the 2010/11 and 2012/13 years.

Figure 39. Incident Reviews Conducted Annually (comparison of 2009/10 and 2012/13) 80%

Percent of Total Responses

70%

68%

2012/ 2013 (n = 104)

60% 50% 52% 40%

2009/ 2010 (n = 53)

30% 21% 20% 10%

12%

12%

13%

5% 4%

0% 0-20

21-40

2%

41-60

61-80

1% 0% 81-100

10% >100

Number of Incident Reviews



As shown in Figure 39, the number of incident reviews conducted by hospitals has remained relatively constant between 2009/10 and 2012/13 when we last asked the question.



The greatest change lies in the 0-20 incident range and the 41-60 incident range.

29 September 3 2013

2. Incident Reviews under QCIPA Figure 40. Incident Reviews Conducted under QCIPA, as a Percent of All Incident Reviews Conducted (comparison of 2009/10 and 2012/13) 80% 71% 70% 70%

Percent of Total Responses

60%

2012/13 (n = 104)

50% 40%

2009/10 (n = 53)

30% 20% 13%

11% 7%

10%

5% 6% 0-20%

21-40%

8%

5%

4%

0%

41-60%

61-80%

81-99%

Percent of Total Incident Reviews



Figure 40 suggests that hospitals continue to conduct the majority of incident reviews outside of QCIPA protections. Over 70% hospitals reported that between 0-20% of all incident reviews are conducted under QCIPA.

3. Critical Incidents Hospitals were also asked to indicate the percentage of critical incidents (as defined in the Public Hospitals Act or as applicable in their organization) that were reviewed under QCIPA in 2012/13.

60%

57%

Figure 41. Percent of Critical Incidents Reviewed Under QCIPA (n=104)

Percentageof total respondents

50% 40% 30% 22% 20%

13%

10%

4%

4%

26-50%

51-75%

0% 0-25%

75-99%

All

Percent of Critical Incidents Reviewed Under QCIPA

30 September 3 2013



More than half of respondents (59 hospitals; 57%) indicated that 0-25% of all critical incidents are reviewed under QCIPA.



Approximately a quarter of respondents (23 hospitals; 22%) indicated that all critical incidents are reviewed under QCIPA.

4. Use of QCIPA Figure 42. How has your use of QCIPA reviews changed since January 1, 2012? (n=104)

22%

Increased Decreased

15%

63%



Stayed the same

More than half of the respondents (65 hospitals; 63%) indicated that their use of QCIPA reviews has stayed the same since January 1, 2012 (Figure 42).

Figure 43. Reasons Why QCIPA Use Has Decreased (n=16)

Introduction the Freedom Introduction ofof the Freedom ofof Information and Protection Information and Protection ofof Privacy ActPrivacy Act

12% 25%

Culture change

Change in the number and/or type of critical incidents 21%



42% Other

The small number of hospitals that reported that their use of QCIPA had decreased (15%; 16 hospitals) cited a culture change (42%; 7 hospitals) or a change in the number/type of critical incidents (21%; 3 hospitals) as a reason for this change. A small number of hospitals indicated that the introduction of FIPPA caused QCIPA use to decrease (12%; 2 hospitals). 31

September 3 2013



Four hospitals (16%) provided additional comments, stating that their use of QCIPA has decreased due to: o Movement towards a reflective learning process that is more transparent o Increase in staff awareness; staff are more open and comfortable in the full disclosure of adverse events



22 hospitals provided additional comments regarding QCIPA. The majority of hospitals that provided comments raised concerns around the understanding and use of QCIPA. o

Concerns around QCIPA (15 hospitals; 68% of responses):  Limits ability to share openly across hospitals and with a patient’s family  Lack of clarity around when to use a QCIPA review  Hospital has not felt the need to develop a formal QCIPA review process  The legislation has set the expectation/culture that every quality improvement conversation needs protection under QCIPA  Lack of QCIPA understanding is impeding the engagement of physicians in reviews  Lack of understanding around physician liability impeding the participation of physicians in reviews  Small organizations do not rely on QCIPA

o

Support for QCIPA (5 hospitals; 23% of responses):  Essential for facilitating incident reviews in a non-threatening environment  Allows staff to communicate freely with the safety and quality improvement team  Facilitates frank root cause analysis in an environment free from legal repercussions  Modification of QCIPA in any way would impede our organization’s quality improvement process  Difficult to balance QCIPA, community expectations, and ECFAA requirements with the supports and guidance currently available

o

Additional Comments:  Develop guidance or tools that profile the lessons learned from QCIPA since 2004  Develop a standardized aggregate QCIPA reporting template.

32 September 3 2013