A Call for Proposals NOTE: This RFA and grant application was generously provided by Dr. Paul Haidet for posting, solely for the purpose of providing an example of a successfully-funded application in the field of education research. It does not have any direct bearing on the Woodward application process, other than to provide an illustrative example of successful grantsmanship.

Retooling for Quality and Safety: An initiative of the Josiah Macy, Jr. Foundation and IHI Open School for Health Professions

Deadline for proposal receipt

August 14, 2009

This initiative is generously supported by the Josiah Macy, Jr. Foundation

Project Goal The Josiah Macy, Jr. Foundation and IHI believe that every health professional entering the workforce needs the right skills to help lead and participate effectively in the continual improvement of quality and patient safety. IHI is seeking six progressive academic institutions with medical and nursing schools to participate in a twelve-month demonstration project. The aim of the initiative is to develop and implement innovative methods to assure the integration of health care improvement and patient safety content within existing required curricula at each participating school.

Project Requirements Starting with current best practices, IHI will help sites develop curricula and learning designs that incorporate quality and safety as well as strategies for implementing these changes. IHI will help the project teams test the most promising new ideas and models within their organizations. As part of the demonstration project during the 2009 – 2010 academic year, each participating team will: 1. Articulate specific outcome and process goals for incorporating quality improvement and safety into the medical and nursing curricula; 2. Participate in a collaborative process to develop a set of core competencies; 3. Test the change ideas that they believe are most promising (e.g., integration of improvement into ongoing courses, practicum development ideas, new virtual teaching methods, etc.); 4. Expand the scope of testing through iterative cycles; 5. Refine change strategies, and incorporate new opportunities and strategies that arise; 6. Submit monthly site progress reports; 7. Help synthesize a set of robust change ideas that are successful across a range of settings and conditions; 8. Engage students in hands-on improvement work in partnership with a local health care provider. Each improvement project team will write a report of the project, regardless of outcome, and share this experience with the other sites, the academic community, and the IHI faculty; 9. Participate in demonstration project support activities: a. IHI faculty coaching b. Monthly cross-site Web-Ex sessions c. Two in-person project-wide meetings d. On-site visits from IHI faculty members 10. Agree on demonstration project authorship principles, to be finalized by the selected sites and IHI. As part of the synthesis phase of this effort (in the fall of 2010), each participating team will 1. Assess their progress toward the project goals they articulated at the outset of the initiative; 2. Develop a strategy for sustaining and spreading the gains accomplished to date; 3. Identify priorities for possible further development; 4. Develop a final report of findings to submit to IHI;

5. Present final report and celebrate results at the project summative meeting at the IHI Annual National Forum on Quality Improvement in Health Care in December 2010; 6. Consider submission of results for presentation at relevant professional and academic association conferences (e.g., AAMC); 7. Assist the IHI to: a. Produce and publish a summative report detailing the demonstration process and findings, with strategies for integration within academic curriculum structures; b. Convene a summative Congress event, at which IHI will engage relevant stakeholders such as academic institutional leaders, certifying entities, students, and IHI Open School Chapter leaders to discuss the findings. 8. Agree to participate in the cross-site evaluation, sharing project reports, numbers of students affected, faculty and student feedback, and site team assessment of progress. In addition, the following may be tracked: a. New or enhanced curricula offerings; b. Engagement of senior leadership; c. New teaching methods; and d. New inter-departmental or inter-professional partnerships.

Eligibility Criteria Each application must be submitted jointly by a medical school and a nursing school committed to working together. Participating applicants must: • Be located in the United States; • Have both a nursing and medical school; • Have full support from academic leadership; • Have full support from health system senior leadership; and • Have at least one dedicated faculty leader in the medical school and one in the nursing school.

Selection Criteria The aim of the initiative is to develop and implement innovative methods to assure the integration of health care improvement and patient safety content within existing required curricula at each participating school. To meet this aim, project applications will be selected on the following criteria: Project Plan with Measurable Outcomes: Participating schools will develop and test changes in curricula and learning methods that will lead to the incorporation of quality and safety knowledge in the curriculum. Applicants should: • Specify goals for integrating improvement and safety knowledge within academic curricula; • Describe a plan for meeting the project requirements noted above;

• •

Describe a plan for actively engaging students in improvement activities within the health care delivery system; and Identify measures of success.

Sustainability Plan: Participating schools must be prepared to enhance and sustain new approaches beyond this demonstration initiative. Applicants should: • Demonstrate how the project team will best sustain changes for the long term, including their intention to incorporate quality improvement and safety knowledge into the standard curriculum; • Describe a plan for continued engagement of senior leadership; and • Identify project team members that will continue to work on sustaining changes, and a plan for engaging new students in the work. Partnering and Inclusion: Each project team will represent both a medical and a nursing school. In addition, project teams should expect to develop broader partnerships within their university and their local community. Applicants should: • Describe a plan for partnering between the medical and nursing school within the university and, if applicable, other departments within the school; and • Identify a health care provider partner and a plan for engaging the partner in “real-world” improvement work or opportunities. Senior Leadership Support: Because of the strategic and system-level focus of this initiative, participating schools must demonstrate full support of their academic and health system senior leadership. Applicants should: • Describe in detail how this work connects to the overall priorities of the school; • Demonstrate evidence of current investment to develop medical and nursing faculty and staff knowledge and skill in improvement and patient safety; • Demonstrate evidence of the committed involvement of a Dean or Associate Dean in both the medical and nursing schools; • Identify faculty champions to spearhead these efforts within each medical and nursing school and demonstrate how the faculty champions will stay actively connected to the team’s work throughout the demonstration and synthesis phases; • Indicate possible barriers and how to address them; and • Ensure that there is dedicated “space” in the existing curriculum for these enhancements. Dedicated Project Resources: Each project team, along with the faculty champions, should appoint a project leader to oversee the day-to-day activities—someone who has dedicated time for this work. Applicants should: • Demonstrate the commitment of a project leader, including a respected faculty member with sufficient time and experience, to drive this work; • Describe the project team that will support the endeavor, including engaging students in all phases of program development; and • Describe in detail how the team will make use of the $35,000 allocated funds to support their work. Identify the additional local resources they will apply to the work. (Please note: A portion of the $35,000 site allocation is intended to support required travel and

meeting expenses for the three in-person project-wide meetings. Any institutional overhead costs also must be included. )

IHI Project Team •

•

• •

•

• • •

Executive Sponsors: o Penny Carver, Med, Senior Vice President, IHI o Don Goldmann, MD, Senior Vice President, IHI Lead Academic Advisor: Linda Headrick, MD, MS, Senior Associate Dean for Education and Faculty Development, School of Medicine, University of Missouri, Columbia, Missouri Nursing Academic Advisor: Amy Barton, PhD, RN, Associate Dean of Clinical and Community Affairs, College of Nursing at the University of Colorado Denver Senior Advisor for Academic Programs: Jonathan Finkelstein, MD, MPH, Associate Professor, Ambulatory Care and Prevention and Pediatrics, Harvard Medical School and Harvard Pilgrim Health Care Academic Advisor: Greg Ogrinc, MD, MS, Director of the Office of Research and Innovation in Medical Education and Assistant Professor of Community and Family Medicine, Dartmouth Medical School; general internist at White River Junction VA Medical Center. Improvement Expert: Robert Lloyd PhD, Executive Director of Performance Improvement, Institute for Healthcare Improvement Project Director: Jill Duncan, RN, MS, MPH, IHI Open School for Health Professions Project Manager: Carly Strang, IHI Open School for Health Professions

Key Dates Phase 1: Monday, July 13, 2009 - IHI begins accepting proposals Monday, July 20, 2009- Optional informational call for applicants 2:00-3:00 PM Eastern Daylight Time Connection Information: 1. Go to http://ihi.webex.com 2. Enter 355 310 415 where it asks, “If you know the meeting number, join here:” 3. A pop-up box will appear with a telephone number, session number, and attendee ID number. Please be sure to enter the entire attendee ID number. If the phone number does not appear, go to “Communicate” Menu and select “Join Teleconference.” If you do not have an internet connection, just dial: 866-469-3239 Passcode #: 355 310 415 Attendee ID #: Hit #

Friday, August 14, 2009 – Deadline for proposals Friday, August 28, 2009 – Sites selected and notified Phase 2: Tuesday, September 8, 2009 – First all kick-off project call Friday, October 2, 2009 – First all site face-to-face meeting Monday, May 24 2010 – Second all site face-to-face meeting Phase 3: December 2010 (Date TBD)– Third all site face-to-face meeting

Inquiries All inquiries and communications should be addressed to the Project Manager, Carly Strang, at [email protected] or 617-301-4858.

How to apply Proposals must be received no later than August 14, 2009, 5:00 PM Eastern Daylight Time in any of the following formats: • • •

Fax to the attention of Carly Strang at Fax (617) 301-4848 Email electronically to [email protected] Mail to: Carly Strang Institute for Healthcare Improvement 20 University Rd, 7th Floor Cambridge, MA 02138

Retooling for Quality and Safety: An initiative of the Josiah Macy, Jr. Foundation and IHI Open School for Health Professions

Proposal submission packets must be received no later than August 14, 2009 and should include the following items: Cover letter of expressed interest (maximum: 1 page) Description of the current programs offered and resources available (maximum: 1 page) Plan of action to meet the criteria (maximum: 10 pages) o Senior leadership support Describe in detail how this work connects to the overall priorities of the school; Demonstrate evidence of current investment to develop medical and nursing faculty and staff knowledge and skill in improvement and patient safety; Demonstrate evidence of the committed involvement of a Dean or Associate Dean in each participating school; Identify faculty champions to spearhead these efforts within each medical and nursing school and demonstrate how the faculty champions will stay actively connected to the team’s work throughout the demonstration and synthesis phase; Indicate possible barriers and how to address them; and Ensure that there is dedicated “space” in the existing curriculum for these enhancements. o Dedicated project resources Demonstrate the commitment of a project leader, including a respected faculty member with sufficient time and experience, to drive this work; Describe the project team that will support the endeavor, including engaging students in all phases of program development; and Describe in detail how the team will make use of the $35,000 allocated funds to support their work, travel, and sustainability plan and what additional resources they may apply to the work from other sources. o Partnering and inclusion Describe a plan for partnering between the medical and nursing school and, if applicable, other departments within the school; and Identify a health care provider partner and a plan for engaging the partner in “real-world” improvement projects. o Proposed project goals and measures of success Describe goals for integrating improvement and safety knowledge within academic curricula; Identify settings and methods for content delivery that may include large and small group teaching venues, and independent study by groups or individuals

(though preference will be given to innovative programs in each of these settings); Identify measures of success; and Describe a plan for actively engaging students in improvement activities within health care delivery systems. o Sustainability plan Demonstrate how the project team will best sustain changes for the long term, including their intention to incorporate quality improvement and safety knowledge into the standard curriculum; Describe a plan for continued engagement of senior leadership; and Identify project team members that will continue to work on sustaining changes, and a plan for engaging new students in the work. Budget (please see the attached budget template) o The $35,000 stipend should cover any travel and overhead costs References or letters of support from the Dean of the Medical School, Dean of the Nursing School, health care provider partner, and any other collaborating entities CV for the proposed project leader(s) W-9 Tax Form (attached)

Proposals must be received no later than August 14, 2009, 5:00 PM Eastern Daylight Time in any of the following formats: • • •

Fax to the attention of Carly Strang at Fax (617) 301-4848 Email electronically to [email protected] Mail to: Carly Strang Institute for Healthcare Improvement 20 University Rd, 7th Floor Cambridge, MA 02138

Beginning of the proposal. Note that the proposal provides a NEED, a RATIONALE, GOALS and HYPOTHESES, a VISION, a PLAN OF ACTION including: IMPLEMENTATION and OUTCOMES ASSESSMENT with DATA COLECTION and DATA ANALYSIS STRATEGIES, credentials and responsibilities for KEY PERSONNEL, a DETAILED BUDGET and a TIMELINE.

A. Project Goals and Curriculum Overview A.1. Project Goals We propose a project to implement and evaluate an interprofessional quality and safety curriculum in health sciences education. A large and growing literature on medical errors and quality improvement demonstrates that system issues have a large impact on patient safety and quality regardless of the motivation or clinical skills of the individuals within a given system.1-7 This finding, along with data that demonstrate the widespread nature of medical errors and the costs that these incur,8 underscores the need for every health professional who enters the workforce to have the right skills to help lead and participate effectively in the continual improvement of quality and safety.9-11 We believe that education to foster such skills should mirror the practice environment as closely as possible.12 Specifically, such education should be: a) interprofessional, b) team-based, and c) grounded in the application of core concepts to real-world problems and issues. Given these factors, we propose a project to design and implement an 8-hour curriculum on quality and safety for Penn State (PSU) nursing and medical students. Interprofessional work on quality and safety problems will be a central feature of this curriculum. Further, the curriculum will build on existing IHI Open School courses,13 and will create a classroom environment that fosters productive relationships between the nursing and medical professions. A.1.1. PSU Quality and Safety Curriculum Process Goals 1. Design and implement an 8-hour curriculum on quality and safety using an interprofessional TeamBased Learning approach 2. Use the quality and safety curriculum as a foundation to build future required interprofessional curricula at the Penn State-Hershey campus. 3. Collect evaluation data that will inform: a) the design of ongoing curricula in quality and safety, and b) planning for future interprofessional education activities A.1.2. PSU Quality and Safety Curriculum Outcome Goals At the end of the quality and safety curriculum, students will be able to: A. Articulate a systems approach to quality and safety issues and value the benefits of such an approach. B. Apply the Plan-Do-Study-Act (PDSA) approach to devise practical solutions to quality issues drawn from real-world cases at the Penn State Hershey Medical Center C. Given a particular quality improvement issue, outline the steps (including aims, measures, and change ideas) of a project to address the issue D. Appreciate differences in role expectations between nurses and physicians, and work together to devise solutions to given problems that draw upon the strengths and conceptual backgrounds of both professions A.2. Curriculum Overview We will create a series of two required workshops that will occur in the fall of 2009 for medical and nursing students at the Penn State – Hershey campus (the main health sciences education campus for Penn State). As part of the preparation of this proposal, the project team has already secured dates, venues, and space in the medical and nursing curricula for these workshops. In the sections that follow,

we will briefly describe our plan to conduct these workshops in an interprofessional, team-based, application-oriented fashion. A.2.1. Quality and Safety Education Should be Interprofessional Because nurses and physicians both have roles in the care of patients, they also have roles in improving the quality and safety of the healthcare system.14 However, achieving a coordinated response to quality and safety issues can be hindered by the fact that nurses and physicians are often operating under different sets of assumptions about their own roles, the roles of other healthcare workers, and the best way to approach problems.15,16 In order to work together to address quality and safety issues efficiently and effectively, nurses and physicians need to: a) approach quality and safety issues using a common framework, b) understand how this common framework fits into their conceptualization of their own role (the small arrows at the top of the figure), and c) understand differences in role expectations between nurses and physicians (the large arrows at the bottom of the figure).17-19 In the PSU Quality and Safety curriculum, medical students and nursing students will work together to use a common framework, based on the quality and safety curriculum of the IHI Open School,13 to devise solutions to real-world problems based on actual cases drawn from the Penn State Milton S. Hershey Medical Center. In doing so, the curriculum will provide a common frame of reference for nursing and medical students to discuss and study quality and safety together. In addition, it will provide a bridge for students to reflect upon their own frame of reference with respect to quality and safety content, and will foster understanding of the frame of reference of their colleagues from another profession.20 We will build activities into the curriculum to foster reflective debriefing. For example, as part of an NHLBI-funded project to use reflective practice to foster students’ cultural competence, Dr. Haidet has used reflective debriefings to create student reflection about stereotyping and bias and their effects on decision making.21-23 A.2.2. Quality and Safety Education should be Team-Based Since healthcare is usually delivered by nurses and physicians working together, their educational experiences should provide an opportunity to learn how to work effectively on a team. Such experiences go beyond the delivery of course content to also provide students with fundamental communication and leadership skills that maximize effective collaborations. We will organize the Penn State quality and safety curriculum using a revolutionary pedagogical method called Team-Based Learning (TBL).24-26 Team-Based Learning is an instructional format that was originally developed in business education, but has increasingly been used in medical and nursing education.27-29 TBL allows multiple teams (of 5-6 students each) to work in parallel within a single lecture hall, facilitated by a single instructor. It thus combines the interactivity of small-group collaborative learning with the efficiency of large group

teaching.30,31 A central concept in TeamBased Learning, as shown in the conceptual model in the figure to the left, is learner engagement.32 TBL incorporates key design elements to foster engagement, both with course content and within one’s team. One of these elements is constructive controversy,33 a situation where multiple teams make decisions about a common real-world problem, and then , using course concepts, work collaboratively as an entire class to address disagreements and discuss alternative solutions to the problem. Constructive controversy creates emotional and cognitive energy which fosters excitement, identification with one’s team, and a drive to deepen one’s understanding of course concepts. TBL creates open, safe discussions among multiple teams in the community of the classroom, and gives learners experience in working with, acknowledging, and managing the uncertainty of complex problems, such as those encountered in the world of quality and safety. Team-Based Learning uses repeating cycles of learning to move students from exposure, to acquisition, to the application of course concepts. We will use this cycle for each of the two workshops in our curriculum. For each workshop, we will first expose students to concepts by requiring that they view elements of the web-based IHI Open School Quality Improvement 101, 102, and 103 curricula13 in advance preparation for the workshop (approximately 90 minutes of assigned viewing prior to each workshop). Next, during the first hour of each workshop, we will strengthen their acquisition of this knowledge and assure readiness to apply it through a series of individual and team activities and clarifications. Finally, in the last hour of each workshop, students will apply this knowledge in teams to realworld problems that we have drawn from the quality improvement case files of our own medical center. Dr. Gregory Caputo, Chief Quality Officer and Amy McCowan, Nursing Magnet Director for the Milton S. Hershey Medical Center, will identify suitable cases and will work with the project team to build application activities that will require the student teams to make decisions under real-world conditions of uncertainty, complexity, and competing priorities. A.2.3. Quality and Safety Education Should be Oriented Toward Application Quality improvement and patient safety are applied sciences.34 Using core concepts, quality and safety scientists plan unique strategies that address complex problems. In our curriculum, we will create

opportunities for students to practice this skill of application within their teams. During the application phase of each workshop (phase 3 of the Team-Based Learning cycle), each team of students will be asked to answer a question about a real-world, complex problem drawn from the case files of our hospital. During this intra-team discussion, all of the student teams will work on the same problem, and the problem will have several plausible answers. The utility of such an exercise is that all of the teams will have a common frame of reference, since they are all working on the same problem, but different teams may select different answers. At a single point in time, the instructor will ask all of the teams to simultaneously reveal their answers. This will create a moment of high energy, when all of the teams can look around the room and see what other teams think, relative to their own. During the ensuing interteam discussion, teams will defend and discuss their choices using underlying course concepts, and the instructor will facilitate and pose provocative questions to deepen and increase the sophistication of the discussion. Finally, the instructor will wrap up the exercise by summarizing lessons learned. A draft of an application activity, built from a case in our medical center, appears in the box below. DRAFT TEAM-BASED LEARNING APPLICATION ACTIVITY A 72 year-old man with a history of severe osteoarthritis (treated with naproxen) is admitted to the hospital for pneumonia and treated with intravenous antibiotics. Instead of using naproxen for his arthritis pain during the hospitalization, the hospital team prescribes ibuprofen, a similar drug in the same class (NSAIDs) as naproxen. His pneumonia improves, and he is discharged on the third hospital day. His discharge medication list includes both ibuprofen and naproxen, a redundant combination that can increase the chances of NSAID side effects. Without asking questions, he takes his medications as prescribed, and is readmitted to the hospital 5 days later with gastrointestinal bleeding, which is determined to be a result of NSAID-induced gastritis. “Medication reconciliation” is the process of comparing a patient's medication orders to all of the medications that the patient has been actually taking. Reconciliation should occur at every instance in which the patient's care is transitioned from one care provider to another, including changes in the levels and locations of care. In this case, the patient’s NSAID-induced gastritis was attributed to a failure to adequately reconcile his medication orders upon discharge, leading to him taking two redundant medications from the same class. Imagine that you are student members of a hospital quality improvement team. Several proposals are made to your team to reduce the number of medication errors due to breakdowns in the medication reconciliation process. Your team is tasked with choosing which proposal to move forward. The hospital CEO will only fund one proposal. From the following proposals, choose the ONE PROPOSAL that your team would like to move forward with. Be prepared to explain the rationale for your choice. Feel free to base your team’s rationale in practical, theoretical, logistical or any combination of factors that seems appropriate. A) Assign a pharmacist to each ward team in the hospital. The pharmacist’s primary job will be to perform medication reconciliations at all care transitions. B) Institute mandatory training for all physicians and nurses about medication reconciliation, and remind personnel that documentation of reconciliation is already required at each care transition. C) Create a training program for patients and families that teaches about the importance of medication reconciliation and empowers them to discuss this with their care team; offer this program when patients are admitted to the hospital. D) Institute a pay-for-performance program that will reward physicians and nurses with salary bonuses for documenting completion of medication reconciliation in the hospital chart. E) Institute caps on the numbers of patients that can be assigned to nurses and doctors, so that the care teams have time to complete medication reconciliation. F) Develop a monthly reporting system that identifies physicians and nurses who have not completed medication reconciliation

In this example, students may apply a number of concepts, including importance, ease of implementation, cost, strength, “carrot” vs. “stick”, targeting the system vs. targeting the individual, and others to make their choice. They will need to examine tensions between competing concepts and will begin to build a framework for using multiple concepts to make sense of a problem and select an appropriate plan of action.

B. Leadership Support and Project Resources B.1 Leadership Support Our work to develop an interprofessional curriculum in quality and safety connects to overall priorities in both the School of Nursing (SON) and the College of Medicine (COM) at Penn State. The following excerpts are drawn from the SON and COM strategic plans, both of which were developed in 2008 and intended to chart the institutions’ courses through 2013: School of Nursing Strategic Action 3a: “Develop strategic partnerships to advance the development of knowledge and the translation of science to the bedside in the area of improving quality…” College of Medicine Goal II: “To ensure high-quality state-of-the-art health sciences education across the learning continuum through curricular innovation;” Objective A: “Develop interdisciplinary health sciences education programs.” Upon receipt of the request for proposals from the IHI and the Macy Foundation, Drs. Paula MiloneNuzzo and Harold Paz, the deans of the nursing and medical schools, respectively, identified this program as an ideal fit for the strategic objectives of health sciences education at Penn State, and, working with the Offices of Dr. Gregory Caputo, Chief Quality Officer and Amy McCowan, Nursing Magnet Director at the Penn State Milton S. Hershey Medical Center, identified and supported the team that has developed this proposal. As is detailed in their letters of support and the in-kind section of the project’s budget, Drs. Milone-Nuzzo and Paz have already made administrative and technical support NOTE! available to the project team, and will support all of the educators on the team to have protected time to fulfill the project’s goals. This support has freed up resources in the budget for the team to hire a dedicated methodologist to assist in the development of a rigorous evaluation plan, detailed later in this proposal. B.2. Organizational Structure and Plan for Partnering Between Nursing and Medical Educators The organization of faculty involved in this project is shown in the figure below. From the outset, we sought to build an organizational structure that would emphasize equal representation, and an equal partnership between nursing and medical school faculty at every level. Our intent is that the curriculum development team’s processes, which we will make transparent to students, will model the kinds of collaborations that we hope they will experience in the classroom, and that they will pursue in their careers. The project team is uniquely qualified to build this curriculum. Dr. Clark is the coordinator for the clinical nursing curriculum at the Penn State Hershey campus, and oversees all of the didactic and clinical teaching for junior and senior nursing students. Dr. Frazer is the director of

the Nursing 400 “Professional Role Development” course. This required senior course forms the nursing students’ context for our quality and safety curriculum. Dr. Haidet was recently recruited by the Vice Dean for Academic Affairs to fill the position of Director of Medical Education Research at Penn State. An expert in communication and organizational influences on education, Dr. Haidet is also one of the first medical educators to implement and evaluate the Team-Based Learning (TBL) Method, and is internationally known for his work on integrating TBL into health sciences education. Dr. Cherry is medical director of the Penn State shock/trauma center, and Associate Chief Quality Officer for the medical center. Drs. Frazer and Cherry will each lead one of our quality and safety workshops, with Drs. Haidet and Clark assisting. A second distinguishing feature of our organization is the inclusion of an advisory panel. This panel includes nationally known figures (Donna Havens Ph.D. from the University of North Carolina at Chapel Hill,35-39 Mark Keroack M.D., M.P.H. from the University HealthSystem Consortium,40-44 Richard Frankel Ph.D. from the University of Indiana45-49); local leaders and experts in evaluation, interprofessional education, and quality & safety; and students. Letters from all members of the advisory panel appear in Appendix B. We will ask the advisory panel and the local deans to provide feedback on our curriculum materials and evaluation strategy in the fall of 2009, and will convene the panel for two conference calls in the spring of 2010 to review course evaluations and participate in the planning of the second iteration of the course. Our curriculum will therefore undergo four cycles of evaluation and revision during 20092010: 1) advisory panel feedback on the curriculum, 2) advisory panel feedback on the evaluation strategy, 3) implementation of the curriculum and examination of evaluation data, and 4) presentation of evaluation data to the advisory panel with feedback and planning of curriculum revisions. The final distinguishing feature is our emphasis on evaluation. Drs. Frankel, Dellasega, and Shapiro all have experience and expertise in evaluating interprofessional education activities in a variety of content areas, including quality and safety.46,50 Dr. Haidet is an accomplished qualitative researcher, having performed both educational research and educational program evaluation using a variety of qualitative data collection and analytical methods.51-59 Dr. Yang is a new faculty member in the Department of Public Health Sciences at Penn State. Dr. Yang is experienced in measurement and biostatistics, particularly in the social sciences, and has an impressive early track record of publications in this field.60-63 In addition, as is outlined in his letter in Appendix B, Dr. Chinchilli, Chair of the Department of Public Health Sciences, will donate time to the project to mentor Dr. Yang and ensure that the quantitative evaluation plan is rigorous, timely, and accurate. B.3. Utilization of Resources and Space in the Curriculum As noted above, the deans and medical center quality officers have already devoted considerable resources to the development of this project. As is outlined in detail in the budget justification, the allocated funds from IHI and the Macy Foundation will be used to support travel to IHI events, honoraria for the advisory panel, instructional materials, a dedicated masters’ level or higher research assistant to provide project management, prepare curriculum and evaluation materials, and collect evaluation data, and 10% FTE for Dr. Yang to plan and implement the quantitative evaluation of the curriculum. In addition to the in-kind and program support, we have already convened the leadership of both the nursing and medical schools to identify a suitable venue for an interprofessional curriculum in quality and safety. We have devised a structure that will combine 50 senior nursing students with 150 first-year medical students to participate in two required 2-hour workshops, on November 10 and

December 1, 2009. The December 1st date will include a third hour at the end for reflection on the curriculum and aspects about working in teams. We toured the facilities and classrooms of the Penn State Hershey campus, and have identified the medical center’s auditorium as the most suitable space in which to conduct the workshops. We have already scheduled the auditorium for the workshop dates. For each workshop, students will view elements of the online IHI Open School “Quality Improvement 101, 102, and 103” courses as advance preparation (approximately 90 minutes of assigned viewing prior to each workshop). For the nursing students, these sessions will occur as part of their required Nursing 400 “Professional Role Development” course, which Dr Frazer (Co-I) directs. For the medical students, these sessions will comprise a required seminar that will exist outside the structure of concurrent courses. The medical curriculum at Penn State Hershey has a number of “island” courses where students focus intensely on important topics in medicine. This required seminar structure is consistent with other “islands” in the medical curriculum. The letter in Appendix B from Drs. Richard Simons, Vice Dean for Academic Affairs, and Carol Whitfield, Associate Dean for Preclinical Curriculum, outlines their support for this space in the medical curriculum. Our rationale for combining senior nursing students with first-year medical students is that this combination will ensure that students are similar in age (senior nursing students average approximately 21 years of age, first-year medical students average approximately 23). In addition, we hypothesize that the considerable clinical exposure that the nursing students will have had by this point may help to offset a tendency toward hierarchical behavior that may position medical students “above” nursing students. We also expect that the differences in clinical backgrounds may provide for an important learning experience about working in teams, and drawing out the strengths of all members of the team. The quality and safety curriculum will be attended by the entire project team, plus Dr. Caputo (Chief Quality Officer) and Amy McCowan (Nursing Magnet Director). At the end of the second workshop, Dr. Caputo and Ms. McCowan will present the combined class with detailed data from the Hershey Medical Center’s “Quality and Safety Dashboard,” an online tracking system which allows all employees to access detailed, up-to-date information on the medical center’s quality indices. They will identify both inpatient and outpatient areas that the medical center is currently actively working to improve quality, and will invite interested students to engage in quality improvement projects related to ongoing efforts. Both the medical and nursing school curricula have structures that provide incentives for students to participate. Penn State medical students are required to complete a research project, and we have ascertained that a quality improvement project would qualify for this requirement. Senior nursing students are required to take a number of elective courses, and one popular elective is a senior research elective, which could be used to work on a quality and safety project. The project team will track student interest and participation in project activities as one of the process outcomes of the curriculum.

C. Evaluation Strategy C.1. Overview of Evaluation Structure In September 2009, we will work with our advisory panel and personnel from the IHI to finalize our data collection and evaluation strategy. We will also seek an educational exemption from the PSU Institutional Review Board to use evaluation materials for both program evaluation and potential

scholarly communications. In this section, we present the results of our background assessment work and initial discussions with Vernon Chinchilli, Ph.D., Chair of the Department of Public Health Sciences. Our evaluation structure appears in the figure below. A fundamental principle of Team-Based Learning is that student learning teams should be formed by the teacher (rather than letting students form their own teams), and should have at least 5, but not more than 6 students.64 Teams with less than 5 students or more than 6 students are more susceptible to dynamic influences from either aggressive or reticent students. Because the senior nursing class has only 50 students compared with 150 students in the medical class, there is a mismatch in numbers between nursing students and medical students, leading to the potential for teams where the nursing students are significantly outnumbered. Because of the potential for adverse interpersonal dynamics in such teams, we have elected instead to randomly assign nursing students to only half of the 34 teams in the classroom, as shown in the figure. This will produce 17 interprofessional teams with equal numbers of nursing and medical students, and 17 teams consisting of only randomly assigned medical students. There are no data in the interprofessional or TBL literature to suggest whether one population of teams will be at an advantage, given the course content and the random assignment of students to teams. While we will collect evaluation data on student knowledge, we will not assign grades for this curriculum, in the event that one population has significantly better learning outcomes than the other. This dual-population structure will allow us to begin to build not only evidence about the effectiveness of interprofessional education in quality and safety, but will also allow us to compare the effects of interprofessional versus medical only teams on a number of outcomes, detailed below. C.2. Measurement Instruments Our evaluation strategy and the measures that we will use will be guided by our outcome goals (section A.1.2), the conceptual model for interdisciplinary quality and safety education (section A.2.1), and the conceptual model of Team-Based Learning (section A.2.2). A preliminary inventory of measures appears in the table on the next page. The instruments in this table represent measures that we found on an initial Medline search of the quality and safety, interprofessional, and team education literature. Validity evidence is available for all of these, albeit in varying degrees. Dr. Haidet (co-PI) led or participated on the development teams that created the Classroom Engagement Survey, the Value of Teams Survey, and the Team Performance Scale. We note that two systematic reviews of interprofessional education or collaboration found a paucity of validated instruments to measure interprofessional attitudes or communication processes.65,66 We will expand our search to include additional databases and fields other than medicine, conversations with our advisory panel members,

and conversations with IHI personnel in an effort to identify additional instruments relevant to the outcomes assessment for this project. We will also measure demographic variables including, but not limited to, student age, gender, ethnicity, and dual degree status (e.g., MD/PhD, etc). TABLE: PRELIMINARY INVENTORY OF MEASURES Construct Instrument Timing of Administration Knowledge of QI/Safety concepts Application of QI/Safety concepts Attitudes toward QI/Safety concepts Self-assessment of QI/Safety skills Classroom engagement Attitudes toward working in teams Team communication processes Role expectations Conceptualization of the other’s profession Problem solving approaches

To be developed The Quality Improvement Knowledge 67 Application Tool (QIKAT) Attitudes Instruments by Madigosky et 68 69 20 al, Schnall et al, Cox et al 68 Skills Instrument by Madigosky et al Classroom Engagement Survey 20

70

Cox et al, Value of Teams Survey 71 Team Performance Scale Qualitative assessment

26

Pre/Post Pre/Post Pre/Post Pre/Retrospective Pre/Post After each workshop session Pre/Post Post Post

Qualitative assessment

Post

Qualitative assessment

Post

In addition to quantitative data collection, we will develop qualitative interview guides and openended survey questions to explore outcomes and processes for which survey instruments do not exist. Examples of such outcomes include role expectations, conceptualization of the other’s profession, problem solving approaches, and individuals’ experiences with communication within their teams. We will plan to conduct 6 focus groups after the curriculum is completed (2 each for nursing students, medical students from interprofessional teams, and medical students from medical-only teams). This qualitative data collection and analysis will not only provide valuable evidence about the outcomes of our curriculum, it will also provide a foundation for developing additional survey instruments for constructs that lack adequate measurement tools. C.3. Analysis Strategy Dr. Yang will lead the quantitative analysis for the project. The table details the timing of administration of surveys for various outcomes. After examining descriptive statistics, we will conduct three sets of quantitative analyses. First, we will examine changes in outcome variables from before- to after the curriculum. For this pre/post analysis of outcomes, we will apply a mixed-effects linear model that accounts for (1) the nesting of individual students within teams, and (2) the repeated measurements. Second, we will examine differences between the interprofessional cohort and the medical-only cohort on both pre-curriculum and post-curriculum measures, again via a mixed-effects linear model that accounts for the nesting of students within teams. Finally, we will examine differences between the interprofessional and the medical-only cohort in change from pre-curriculum to postcurriculum, again using the mixed-effects linear model with the construction of linear contrasts for the appropriate comparisons. We will adjust all models for relevant demographic variables. Given that there are 34 teams (17 each for two types), a conservative estimate is that there will be 80% statistical power with a two-sided, 0.05 significance level test to detect a one standard deviation unit of change between the two types of teams. The statistical power for the three statistical analyses described above, however, will be higher than 80%, because of the multiple students within each team.

For our qualitative data, Dr Haidet will lead the analytical team. We will audio record the focus group interviews and professionally transcribe the recordings. We will use Atlas.ti qualitative analysis software to organize and code qualitative transcript and survey data. Dr. Haidet owns a multi-user license for this software, and will donate its use for the analysts for this project. We will approach the qualitative data using a narrative framework, which identifies stories in the data and examines elements such as character, setting, and plot to derive themes and meanings. Dr Haidet has previous experience in performing narrative analysis of medical students’ stories, using a variety of text and pictorial data.72 We will present all conclusions drawn from the qualitative data back to a randomly-chosen sample of nursing and medical students, and ask the students to comment on the plausibility of our conclusions. We will also conduct member-checking with the nursing and medical students on our advisory panel.

D. Sustainability Plan There are several forces that will facilitate the sustainability of our curriculum, both from a nursing and a medical school standpoint. First, there is a recognized desire by both schools to partner with each other to create required interprofessional curricula in a variety of content areas relevant to team processes in medical care. Our curriculum will provide a foundation on which to build such future curricula, and will be a model for the processes to use in such curricula. The deans of both schools are invested in the outcome of this project, and will provide support for early adopters and opinion leaders among both sets of faculty to join our team in this effort. Second, the process of writing this proposal has created connections between the nursing and medical faculty that will hopefully lead to future collaborations. For example, Dr. Clark has introduced Dr. Haidet to most of the nursing faculty on the Hershey campus, and Dr. Haidet’s position as Director of Medical Education Research provides him with protected time to develop collaborations on scholarly projects with both nursing and medical educators. Finally, both the nursing and medical schools are in the process of accreditation renewal, and these processes provide opportunities for growth of our curriculum in safety and quality. For example, the medical school has just completed its 18-month self-study in preparation for a 2010 Liaison Committee for Medical Education (LCME) review. As part of this self-study, the school compiled a detailed and comprehensive list of topics taught during every hour of the medical curriculum. During our background work for this proposal, we reviewed the curriculum database for the medical school. Nearly all teaching time devoted to quality and safety currently occurs during the third year of medical school (mostly in the pediatric clerkship), and there is little coordination between the venues and teachers of quality and safety. We see our proposed curriculum as a foundation for these later teaching sessions, and we will seek to form a coalition of educators in the medical curriculum to coordinate and present a coherent longitudinal curriculum in quality and safety. In conclusion, we propose to build and implement a foundational 8-hour curriculum in quality and safety, using Team-Based Learning to promote effective interprofessional communication and application of core concepts. We will use resources provided by the IHI and Macy Foundation to enable a rigorous evaluation strategy that will provide critically needed data on what works and what does not. Our team represents a true partnership between the nursing and medical schools at all levels, and will serve as a role model for nursing and medical students in our curriculum, as they are introduced to the world of multidisciplinary clinical care.

Budget Justification Project Implementation Plan The project will be implemented in the fall of 2009 and all of 2010. During the fall of 2009, we will design the curriculum and evaluation materials, implement the curriculum, and collect evaluation data. During the winter and spring of 2010, we will collect additional qualitative evaluation data, analyze quantitative and qualitative evaluation data, and revise the curriculum. We will also support students who elect to pursue quality improvement projects. During the summer of 2010, we will continue to refine the curriculum, tailor the evaluation plan to be sensitive to adjustments made to the curriculum, and track the progress and outcomes of student quality improvement projects. We will implement a second iteration of the curriculum in the fall of 2010. The Gantt chart below provides a timeline for planned project activities.

Project Activity Hire project manager Create curriculum materials Identify and create measurement tools Develop qualitative surveys and interview guides Feedback from advisory panel Implement curriculum Collect post-curriculum surveys Conduct post-curriculum focus groups Analyze qual and quant data Catalogue and track outcomes of student projects Curriculum revision Second Curriculum Implementation Prepare reports, present findings IHI/Macy meetings

Timeline of Project Activities (Gantt Chart) 2009 2010 Sept Nov Jan Mar May Jul Oct Dec Feb Apr Jun Aug

Sept Oct

Nov Dec

Please note how clearly each key personnel is described, including: % effort, who they are, their credentials, roles and specific responsibilities within the project. Project Personnel Mary Beth Clark, R.N., Ed.D., Co-Principal Investigator (10% Effort, In-Kind) Dr. Clark is the Hershey Campus Coordinator for the Penn State University School of Nursing. Dr. Clark has a long track record of curriculum development and scholarship in the field of nursing education. Specifically, she has worked in a number of areas related to the quality of nursing care, including nursing telephone triage, nursing roles in patient satisfaction and trust, nursing simulation, interpersonal nursing issues in pediatrics and adolescent medicine, and undergraduate honors nursing education.73-75 Dr. Clark is the recipient of several awards, including the Hershey Medical Center Nursing Recognition Award (three times), and is a member of the Sigma Theta Tau Nursing Honor Society. As the ranking officer for nursing education at the Penn State –Hershey campus, Dr. Clark will act as liaison between the project team and the School of Nursing, and will communicate project activities and results regularly to the dean of the School of Nursing. Dr. Clark will be responsible for hiring the project manager, and will supervise the manager for the duration of the project. Dr. Clark will manage the nursing school’s portion of the budget and will oversee all project operations, including ensuring that the project follows its timeline and that scholarly products are produced from the project. Paul Haidet M.D., M.P.H., Co-Principal Investigator (10% Effort, In-Kind) Dr. Haidet is the Director of Medical Education Research for the Penn State College of Medicine. A recognized leader in the fields of patient-physician communication and medical education research,76-85 Dr. Haidet is an internationally recognized authority on implementing and researching Team-Based Learning in medical education. Dr Haidet authored the first full TBL paper in the medical literature and is an author or co-author on over a third of the published reports on TBL in medical education. Dr. Haidet is a deputy editor for the journal Medical Education, and is the recipient of the 2007 Society of General Internal Medicine National Award for Scholarship in Medical Education, and the 2008 American Academy on Communication in Healthcare Lynn Payer Award for Outstanding Contributions to the Published Literature on the Research and Teaching of Healthcare Communication. Dr. Haidet will act as liaison between the project team and the College of Medicine, and will communicate project activities and results regularly to the Vice Dean for Academic Affairs, and the dean of the medical school. Dr. Haidet will train the team in TBL facilitation techniques, and will co-lead the sessions with Drs. Frazer and Cherry in 2009. Dr. Haidet will co-lead the evaluation team (along with Dr Yang), and will oversee the qualitative evaluation for the project. Dr. Haidet will oversee the medical school’s portion of the budget. Christine Frazer, Ph.D, C.N.S., Co-Investigator (10% Effort, In-Kind) Dr. Frazer is the Director of the Nursing 400 “Professional Role Development” course for senior nursing students. For nursing students, the quality and safety workshops will take place within the context of this course. Dr. Frazer’s expertise is in educational strategies and evaluation, and she has studied a number of different methods to foster role development and professionalism among nurses.86-88 These include gaming strategies and a number of other reflective techniques. Dr. Frazer is a member of the Sigma Theta Tau Nursing Honor Society, and was a 2005/2006 appointee to the Manchester Who’s Who Among Executive and Professional Women in Nursing and Healthcare. Dr. Frazer will develop teaching materials for the workshop sessions, and will lead the December 1st session. She will also lead the

reflective debriefing during the third hour of the December 1st session. Dr. Frazer will participate in the analysis of qualitative data for the project, and will act as liaison to the nursing members of the advisory committee. Robert Cherry, M.D., M.S., Co-Investigator (10% Effort, In-Kind) Dr. Cherry is the Associate Chief Quality Officer for the Milton S. Hershey Medical Center and the Chief of the Section of Trauma and Critical Care in the Department of Surgery. Dr Cherry has pioneered the use of simulation in training trauma teams, and was the architect of one of the world’s first Master’s Degree Programs in Homeland Security and Health System Disaster Preparedness.89-93 This online degree program is in the process of being expanded throughout the entire Penn State University system, and has become a model for similar programs throughout the country. Dr Cherry will develop teaching materials for the workshop sessions, and will lead the November 10th session. Dr Cherry will participate in the analysis of quantitative data for the project, and will act as liaison to the medical members of the advisory committee. Chengwu Yang, M.D., Ph.D., Co-Investigator (10% Effort) Dr. Yang is a physician, measurement specialist, and biostatistician newly hired in the Department of Public Health Sciences at the Penn State College of Medicine. Dr. Yang was the Director of Statistics for the First People’s Hospital of Shunde City in Guangdong, China, and has worked on the development of a number of measures of patients’ perceptions, neuropsychological assessment, and program evaluation.60-63 Dr. Yang is a member of the American Statistical Association and the Institute of Mathematical Statistics. Dr. Yang will co-lead the evaluation team, and will lead the design, collection, and analysis of all quantitative evaluation data for the project. Dr Yang will serve as liaison to the evaluation members of the advisory committee. Vernon Chinchilli, Ph.D., Mentor (2% Effort, In-Kind) Dr. Chinchilli is Distinguished Professor and Chair of the Department of Public Health Sciences at the Penn State College of Medicine. Dr Chinchilli has held leadership positions in a number of local and national organizations, and is a Fellow of the American Statistical Association. He is an associate editor for the journals Biometrics and the Journal of Biopharmaceutical Statistics. Dr. Chinchilli will serve as mentor to Dr. Chengwu Yang, and will consult with the team on quantitative analysis strategy and interpretation.

Pleas note how there are no ambiguities in the proposed use of funds.

Additional Project Resources Instructional Support/Materials: $9,875.00 is requested for instructional support and materials. $8840.00 of this item will be used to hire a masters-degree level project manager (260 hours x $34 per hour). The project manager will create and print all documents for the project, including curricular and evaluation materials, will coordinate and collect evaluation surveys, will enter data into an electronic database (with 10% double-entry data checking), will coordinate logistics and communication with the advisory committee, and will assist with production of manuscripts. The rest of the funds ($1035) will be used to purchase office supplies and supplies specific to TBL (e.g., Immediate Feedback Assessment

Technique forms for the phase 2 readiness assurance process), and to professionally transcribe focus group transcripts. Travel and Lodging: $5000.00 of project funds will support travel and lodging for the study team. We will plan to send at least one member of the team to the two all-site face-to-face meetings, and will plan to bring the entire project team to the third all-site face-to-face meeting. Paula Milone-Nuzzo, Dean of the School of Nursing, has offered support for travel and lodging for the third meeting for the nursing members of the team (Drs. Clark and Frazer). Advisory Panel (National Consultation Panel): $5250.00 is budgeted for honoraria for seven of the members of the advisory panel (7 x $750 per member). Dr. Caputo and Ms McCowan have offered to waive their honoraria in support of the project. As is described in the letters of support, the Advisory Panel has deep expertise in the fields of quality, safety, communications, evaluation, humanities, and interprofessional education, as well as including the voices of both medical and nursing students. The Advisory Panel consists of the following individuals: Paul Navarre, former nursing student Michael McShane, first year medical student Richard Frankel, The Indiana University School of Medicine Mark Keroack, The University HealthSystem Consortium Donna Havens, The University of North Carolina at Chapel Hill Daniel Shapiro, Chair, The Penn State College of Medicine Department of Humanities Cheryl Dellasega, The Penn State College of Medicine Department of Humanities Amy McCowan, The Milton S. Hershey Medical Center Office of Quality and Safety Gregory Caputo, The Milton S. Hershey Medical Center Office of Quality and Safety Indirect Costs: The Macy Foundation’s allowable indirect rate is 0.10, or $3182.00. These funds will be evenly divided between the School of Nursing and the College of Medicine.

Resource Sharing between the School of Nursing and the College of Medicine As noted above, the indirect costs will be divided equally between the nursing and medical schools. In addition, roughly half of the project’s budget will be administered through each school. Specifically, the College of Medicine will be responsible for administering funds for Dr. Yang’s salary support, and travel and lodging ($16691.00), and the School of Nursing will be responsible for administering funds for instructional support and the advisory panel ($15,125.00).

Appendix A – References

Appendix A: References 1. Institute of Medicine Committee on Quality of Healthcare in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 2. Singh H, Naik AD, Rao R, Petersen LA. Reducing diagnostic errors through effective communication: harnessing the power of information technology. J Gen Intern Med 2008;23:489-494. 3. Ashton CM, Khan MM, Johnson ML, Walder A, Stanberry E, Beyth RJ, Collins TC, Gordon HS, Haidet P, Kimmel B, Kolpakchi A, Lu LB, Naik AD, Petersen LA, Singh H, Wray NP. A quasi-experimental test of an intervention to increase the use of thiazide-based treatment regimens for people with hypertension. Implement Sci 2007;2:5. 4. Tang Z, Weavind L, Mazabob J, Thomas EJ, Chu-Weininger MY, Johnson TR. Workflow in intensive care unit remote monitoring: a time and motion study. Crit Care Med 2007;35:2057-2063. 5. Shojania KG, Ranji SR, McDonald KM, Grimshaw JM, Sundram V, Rushakoff RJ, Owens DK. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA 2006;296:427-440. 6. Miller WL, McDaniel RR Jr, Crabtree BF, Stange KC. Practice jazz; understanding variation in family practices using complexity science. J Fam Pract 2001;50:872-878. 7. Cabrera D, Colosi L, Lobdell C. Systems thinking. Evaluation and Program Planning 2008;31:299-310. 8. Institute of Medicine. To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press;2000. 9. Seiden SC, Galvan C, Lamm R. Role of medical students in preventing patient harm and enhancing patient safety. Qual Saf Health Care 2006;15:272-276. 10. Moskowitz EJ, Nash DB. Teaching the tenets of quality and safety; an annotated bibliography. Am J Medical Quality 2009;24;333-343. 11. Varkey P, Karlapudi S, Rose S, Swensen S. A patient safety curriculum for graduate medical education: results from a needs assessment of educators and patient safety experts. Am J Medical Quality 2009;24:214-221. 12. VerheyMP, Haw MA. Teaching quality management in a nursing graduate program: a collaborative university-agency quality team. J Nurs Care Qual 1994;8:48-54. 13. www.ihi.org/IHI/Programs/IHIOpenSchool/IHIOpenSchoolforHealthProfessions.htm?TabId=4, accessed August 10, 2009. 14. Headrick LA, Khaleel NI. Getting it right: educating professionals to work together in improving health and health care. J Interprof Care 2008;22:364-374. 15. Headrick LA, Wilcock PM, Batalden PB. Interprofessional working and continuing medical education. BMJ 1998;316:771-774. 16. Haidet P, Fecile ML, West HF, Teal CR. Reconsidering the team concept: educational implications for patient-centered cancer care. Patient Education and Counseling 2009; in press. 17. Sievers B, Wolf S. Achieving clinical nurse specialist competencies and outcomes through interdisciplinary education. Clinical Nurse Specialist 2006;20:75-80. 18. Grey M, Connolly CA. Coming together, keeping together, working together. Interdisciplinary to transdisciplinary research and nursing. Nurs Outlook 2008;56:102-107.

19. Hamilton P, Gemeinhardt G, Mancuso P, Sahlin CL, Ivy L. SBAR and nurse-physician communication: pilot testing an educational intervention. Nurs Admin Q 2006;30:295-299. 20. Cox KR, Scott SD, Hall LW, Aud MA, Headrick LA, Madsen R. Uncovering differences among health professions trainees exposed to an interprofessional patient safety curriculum. 21. Teal CR, Gill A, Shada RE, Thompson BM, Fruge E, Villarreal GB, Patton C, Haidet P. The implicit association test: a good trigger for reflective discussion about biases and healthcare disparities? Submitted. 22. Thompson BM, Teal CR, Rogers JC, Paterniti DA, Haidet P. Opening the ‘black box’ of reflection: a model to guide educators efforts. Academic Medicine 2009; in press. 23. Teal CR, Shada RE, Gill A, Thompson BM, Fruge E, Villarreal GB, Haidet P. When best intentions aren’t good enough: helping medical students develop strategies for managing patient bias. Submitted. 24. Michaelsen LK, Parmelee DX, McMahon KK, Billings DM, Levine RE. Team-Based Learning for Health Professions Education: A Guide for Using Small Groups for Improving Learning. Sterling, VA: Stylus; 2007. 25. Haidet P, Fecile ML. Team-based learning: a promising strategy to foster active learning in cancer education. J Cancer Education 2006;21:125-128. 26. Levine RE, O’Boyle M, Haidet P, Lynn DJ, Stone MM, Wolf DV, Paniagua FA. Transforming a clinical clerkship with team-based learning. Teaching and Learning in Medicine 2004;16:270-275. 27. Searle NS, Haidet P, Kelly PA, Schneider V, Seidel CL, Richards BF. Team Learning in medical education: initial experiences at ten institutions. Academic Medicine. 2003;78:S55-S58. 28. Thompson BM, Schneider VF, Haidet P, Levine R, McMahon K, Perkowski L, Richards BF. Team learning at ten medical schools: two years later. Medical Education 2007;41:250-257. 29. Clark MC, Nguyen HT, Bray C, Levine RE. Team-based learning in an undergraduate nursing course. J Nursing Education 2008;47:111-117. 30. Haidet P, O’Malley K, Richards B. An initial experience with ‘Team Learning’ in medical education. Acad Med 2002;77:40-44. 31. Hunt DP, Haidet P, Coverdale J, Richards B. The effect of using Team Learning in an evidence-based medicine course for medical students. Teach Learn Med 2003;15:131-139. 32. Haidet P, Schneider V, Onady G. Team-Based Learning Research and Scholarship in Medical Education. In: Michaelsen LK, Parmelee DX, McMahon KK, Billings DM, Levine RE. Team-Based Learning for Health Professions Education: A Guide for Using Small Groups for Improving Learning. Sterling, VA: Stylus; 2007. 33. Johnson DW, Johnson RT, Smith KA. Constructive controversy. Change 2000;32:29-37. 34. Wu B, Klein C, Stone TT. Healthcare systems engineering: an interdisciplinary approach to achieving continuous improvement. Int J Electron Healthcare 2006;2:201-222. 35. Havens DS, Wood SO, Leeman J. Improving nursing practice and patient care: building capacity with appreciative inquiry. J Nurs Adm 2006;36:463-470. 36. Havens DS, Vasay J. The staff nurse decisional involvement scale: report of psychometric assessments. Nurs Res 2005;54:376-383. 37. Kellogg VA, Havens DS. The shift coupon: an innovative method to monitor adverse events. J Nurs Care Qual 2006;21:49-55.

38. Kellogg VA, Havens DS. Adverse events in acute care: an integrative literature review. Res Nurs Health 2003;26:398-408. 39. Havens DS, Vasey J. Measuring staff nurse decisional involvement: the Decisional Involvement Scale. J Nurs Adm 2003;33:331-336. 40. Meurer S, Keroack M, Sabel A. Re: Comparison of risk adjustment methodologies in surgical quality improvement. Surgery 2009;146:128. 41. Keroack MA, Youngberg BJ, Cerese JL, Krsek C, Prellwitz LW, Trevelyan EW. Organizational factors associated with high performance in quality and safety in academic medical centers. Academic Medicine 2007;82:1178-1186. 42. Keroack MA. The power of collaboration. Am J Med Qual 2005;20:119-120. 43. Leas BF, Goldfarb NI, Browne RC, Keroack M, Nash DB. Ambulatory quality improvement in academic medical centers: a changing landscape. Am J Med Qual 2009;24:287-294. 44. Maio V, Goldfarb NI, Keroack M, Browne RC, Nash DB. Outpatient quality improvement in academic faculty practice plans: does it exist? Am J Med Qual 2004;19:235-241. 45. Blanch DC, hall JA, Roter DL, Frankel RM. Is it good to express uncertainty to a patient? Correlates and consequences for medical students in a standardized patient visit. Patient Education and counseling 2009; Jul 13 (epub ahead of print) 46. Flanagan ME, Patterson ES, Frankel RM, Doebbling BN. Evaluation of a physician informatics tool to improve patient handoffs. J Am Med Inform Assoc 2009;16:509-515. 47. Braddock C 3rd, Hudak PL, Feldman JJ, Bereknyei S, Frankel RM, Levinson W. “Surgery is certainly one good option”: quality and time-efficiency of informed decision making in surgery. J Bone Joint Surg Am 2008;90:1830-1838. 48. Cottingham AH, Suchman AL, Litzelman DK, Frankel RM, Mossbarger DL, Williamson PR, Baldwin DC Jr, Inui TS. Enhancing the informal curriculum of a medical school: a case study in organizational culture change. J Gen Intern Med 2008;23:715-722. 49. Rodriguez HP, Anastario MP, Frankel RM, Odigie EG, Rogers WH, von Glahn T, Safran DG. Can teaching agenda-setting skills to physicians improve clinical interaction quality? A controlled intervention. BMC Med Educ 2008;8:3. 50. Dellasega C, Milone-Nuaao P, Curci KM, Ballard JO, Kirch DG. The humanities interface of nursing and medicine. J Prof Nurs 2007;23:174-179. 51. Haidet P, Stone DA, Taylor WC, Makadon H. When risk is low: primary care physicians counseling about HIV prevention. Pt Educ Couns 2002;46:21-29. 52. Paterniti DA, Price MD, Haidet P. Social Problems, Aging, and Self-Reported Assessments of Quality of Life after Total Knee Replacement The Occupational Therapy Journal of Research 2002; 22(Suppl):1-2. 53. Tran A, Haidet P, Street RL Jr, O’Malley K, Martin F, Ashton CM. Empowering communication: a community-based intervention for patients. Patient Education and Counseling 2004;52:113-121. 54. Levine RB, Haidet P, Kern DE, Beasley BW, Bensinger L, Brady DW, Gress T, Hughes J, Marwaha A, Nelson J, Wright SM. Personal growth during internship: a qualitative analysis of interns’ responses to key questions. Journal of General Internal Medicine 2006; 21:564-569. 55. Weissmann PF, Branch WT, Gracey C, Haidet P, Frankel R. Role modeling humanistic behavior: learning bedside manner from the experts. Academic Medicine 2006; 81:661-667.

56. Haidet P, Kroll TL, Sharf BF. The complexity of patient participation: lessons learned from patients’ illness narratives. Patient Education and Counseling 2006; 62:323-329. 57. Jodlowski D, Sharf BF, Capistrano LC, Haidet P, Woodard LD. ‘Screwed for life’: examining identification and division in addiction narratives. Communication & Medicine 2007;4:15-26. 58. Lown BA, Chou CL, Clark WD, Haidet P, Kemp White M, Krupat E, Pelletier S, Weissmann P, Anderson MB. Caring attitudes in medical education: perceptions of deans and curriculum leaders. The Journal of General Internal Medicine 2007;22:1514-1522. 59. Morrow AS, Haidet P, Skinner J, Naik AD. Integrating diabetes self-management with the health goals of older adults: a qualitative exploration. Patient Education and Counseling 2008;72:418-423. 60. Yang C, Garrett-Mayer E, Schneider J, Gollomp S, tilley BC. Repeatable batteries for assessment of neuropsychological status (RBANS) in early Parkinsons’ disease. Movement Disorders 2009; in press. 61. Ford DW, Zapka JG, Gebreziabher M, Hennessey W, Yang C. Investigating critically ill patients’ and families’ perceptions of chance for survival. J Palliative Med 2009; in press. 62. Evatt ML, Chaudhuri KR, Chou KL, Cubo E, Hinson VK, Kompoliti K, Yang C, Poewe W, Rascol O, Sampaio C, Stebbins GT, Goetz CG. Dysautonomia rating scales in Parkinson’s disease: sialorrhea, dysphagia, and constipation – critique and recommendations to movement disorders task force on rating scales for Parkinson’s disease. Movement Disorders 2009; Feb 9 (epub ahead of print). 63. Carlozzi N, Horner M, Yang C, Tilley BC. Factor analysis of the repeatable battery for assessment of neuropsychological status (RBANS). Applied Neuropsychology 2008;15:274-279. 64. Michaelsen LK, Bauman Knight A, Fink LD. Team-Based Learning: A Transformative Use of Small Groups In College Teaching. Sterling, Va; Stylus: 2004. 65. Hammick M, Freeth D, Koppel I, Reeves S, Barr H. A best evidence systematic review of interprofessional education: BEME guide no. 9. Med Teach 2007;29:735-751. 66. Zwarenstein M, Goldman J, Reeves S. Interprofessional collaboration: effects of practice-based interventions on professional practice and healthcare outcomes. Cochrane Database Systematic Review CD000072 2009; Jul 8. 67. Varkey P, Karlapudi SP, Bennett KE. Teaching quality improvement: a collaboration project between medicine and engineering. Am J Med Qual 2008;23:296-301. 68. Schnall R, Stone P, Currie L, Desjardins K, John RM, Bakken S. Development of a self-report instrument to measure patient safety attitudes, skills, and knowledge. J Nursing Scholarship 2008;40:391-394. 69. Madigosky WS, Headrick LA, Nelson K, Cox KR, Anderson T. Changing and sustaining medical students’ knowledge, skills, and attitudes about patient safety and medical fallibility. Academic Medicine 2006;81:94-101. 70. Haidet P, Morgan RO, O’Malley K, Moran BJ, Richards B. A controlled trial of active learning versus didactic lecturing in medical education. Advances in Health Sciences Education 2004;9:15-27. 71. Thompson BM, Levine RE, Kennedy F, Naik AD, Foldes CA, Coverdale JH, Kelly PA, Parmelee DX, Richards BF, Haidet P. Characterizing the quality of learning team processes in medical education: development and validation of a new measure. Academic Medicine 2009; in press. 72. Haidet P, Hatem DS, Fecile ML, Stein HF, Haley HA, Kimmel B, Mossbarger DL, Inui TS. The role of relationships in the professional formation of physicians: case report and illustration of an elicitation technique. Patient Education and Counseling 2008;72:382-387.

73. Light P, Hupcey JE, Clark MB. Nursing telephone triage and its influence on parents’ choice care for febrile children. Journal of Pediatric Nursing 2005;20-424-429. 74. Hupcey JE, Clark MB, Hutcheson CR, Thompson VL. Expectations for care: elders satisfaction and trust in healthcare providers. Journal of Gerontological Nursing 2004;30:37-45. 75. Thompson VL, Hupcey JE, Clark MB. The development of trust in parents of hospitalized children. Journal of Specialists in Pediatric Nursing 2003;8:137-147. 76. Haidet P, Paterniti DA. ‘Building’ a history rather than ‘taking’ one: a perspective on information sharing during the medical interview. Archives of Internal Medicine 2003;163:1134-1140. 77. Street RL Jr, Krupat E, Bell RA, Kravitz RL, Haidet P. Beliefs about control in the physician-patient relationship: effect on communication in medical encounters. J Gen Intern Med 2003;18:609-616. 78. Braddock CH, Eckstrom E, Haidet P. The ‘new revolution’ in medical education: fostering professionalism and patient-centered communication in the contemporary environment. J Gen Intern Med 2004;19(5 part 2):610-611. 79. Haidet P, Kelly PA, and Chou C for the C3 Investigators. Characterizing the patient centeredness of medical schools’ hidden curricula: development and validation of a new measure. Academic Medicine 2005;80:44-50. 80. Haidet P, Stein H. The role of student-teacher relationships in the formation of physicians: the hidden curriculum as process. The Journal of General Internal Medicine 2006;21:S16-S20. 81. Beach MC, Inui T, and the Relationship Centered Care Research Network (Frankel R, Hall J, Haidet P, Roter D, Beckman H, Cooper LA, Miller W, Mossbarger D, Safran D, Sluyter D, Stein H, Williamson P). Relationship-centered care: a constructive reframing. The Journal of General Internal Medicine 2006;21:S3-S8. 82. Haidet P, Kelly PA, Bentley S, Blatt B, Chou CL, Fortin AH VI, Gordon G, Gracey C, Harrell H, Hatem DS, Helmer D, Paterniti DA, Wagner D, Inui TS. Not the same everywhere: patient-centered learning environments at nine medical schools. Journal of General Internal Medicine 2006;21:405-409. 83. Haidet P. Jazz and the ‘art’ of medicine: improvisation in the medical encounter. Annals of Family Medicine 2007;5:164-169. 84. Haidet P. Where we’re headed: a new wave of scholarship on educating medical professionalism. The Journal of General Internal Medicine 2008;23:1118-1119. 85. Haidet P, O’Malley KJ, Sharf BF, Gladney AP, Greisinger AJ, Street RL Jr. Characterizing explanatory models of illness in healthcare: development and validation of the CONNECT instrument. Patient Education and Counseling 2008;73:232-239. 86. Frazer C. The effect of gaming as an instructional strategy on baccalaureate nursing students' immediate knowledge and knowledge retention. Dissertation, Widener University School of Nursing; 2007. 87. Fisher K, Frazer C. A qualitative study of emergency nurses’ perceptions and experience in caring for individuals with intellectual disabilities in the United States. International Journal of Nursing in Intellectual and Developmental Disabilities 2007;3:1. 88. Kolanowski A, Frazer C, Fick D, Penrod J. It’s about time and the non-pharmacological interventions in the nursing home. Journal of Nursing Scholarship 2009, submitted. 89. Ali J, Al Ahmadi K, Williams JI, Cherry RA. The standardized live patient and mechanical patient models – their roles in trauma teaching. Journal of Trauma 2009;1:98-102.

90. Cherry RA, Trainer M. The current crisis in emergency care and the impact on disaster preparedness. BMC Emergency Medicine 2008; 8:7. 91. Cherry RA, Ali J. Current concepts in simulation-based trauma education. Journal of Trauma. 2008;5:1186-1193. 92. Cherry RA, Davis T. The road to an advanced degree program in public health preparedness. Academic Medicine2007;82:773-780. 93. Cherry RA, Williams J, George J, Ali J. The effectiveness of a human patient simulator in the ATLS shock skills station. Journal of Surgical Research 2006;139:229-35.