Measuring Resident Progress: Competency Milestones in Internal Medicine
Writing Sub-committee Eva M. M Aagaard, Aagaard MD Univ. of Colorado – Denver...
Measuring Resident Progress: Competency Milestones in Internal Medicine
Writing Sub-committee Eva M. M Aagaard, Aagaard MD Univ. of Colorado – Denver
Kelly Caverzagie, MD Henry Ford Hospital
Davoren Chick, MD Univ. of Michigan
Daisy Smith, MD Lankenau Hospital
Gregory C. C Kane, Kane MD Thomas Jefferson University
Michael L. Green, MD Yale University
William F. Iobst, MD ABIM
Eric S. Holmboe, MD ABIM
1
Workshop Objectives
Identify the principles of competencybased education and training and why milestones are important Explore resident behaviors that define milestones for curriculum development and evaluation Identify assessment methods that can evaluate resident achievement of levelspecific milestones
Agenda 1:30 pm – Background of CBET 1:45 pm – Intro to Milestones 2:00 pm – Small group #1
Application of milestones
2:25 pm – Small group #2
Evaluation of competence
2:50 pm – Large group discussion
2
Background
ACGME Outcomes Project
Moved accreditation from process and structure to outcomes Programs must demonstrate their actual accomplishments Programs must demonstrate resident competence in six dimensions of practice (Core Competencies) Curriculum and evaluation (i.e. resident education and training) now focus on demonstrating resident competency
Competency-Based Competency Based Education and Training (CBET)
Goal is to demonstrate achievement of educational outcomes necessary for advancement to the next phase of a career
Competency-Based Education and Training (CBET)
…is is an outcomes-based outcomes based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing o ga g framework a o of o competencies. The International CBME Collaborators 2009
3
Structure/Process vs. CBET Educational Program Variable
Structure/Process
Competency-based
Driving force: Curriculum
Content Knowledge acquisition
Outcome – Knowledge application
Goal of educ. encounter
Knowledge acquisition
Knowledge application
Typical assessment tool
Single subject measure
Multiple objective measures
Proxy
Authentic (mimic real tasks of profession)
Setting for evaluation
Removed (gestalt)
Direct observation
Evaluation
Norm-referenced
Criterion-referenced
Emphasis on summative
Emphasis on formative
Assessment tool
Timing of assessment
Adapted from Carracchio, et al. 2002
Competency-Based Education and Training (CBET)
Fundamental Requirement
The program director knows that the trainee is truly competent to progress to the next stage of their career
Programs are struggling to operationalize the competencies to achieve CBET
Unclear and complex nature of core competencies Difficulty in assessment Varied needs for each learner Summary: Having trouble moving from structure/process to competency/outcomes
4
Developmental Milestones and CBET
Developmental Milestones
ACGME engaged all disciplines to “articulate milestones of competency development” Purpose:
Describe the developmental progression of observable behaviors (i.e. help to interpret the core competencies) Assist with assessment of competency Provide specific feedback to trainee regarding progression Provide ACGME framework with which to make accreditation actions
Milestones Task Force
Members included PD PD’ss, evaluation experts and representatives from Med Ed stakeholders
Initial Meeting (November 2007)
Convened by ABIM and ACGME Literature review, presentation of three ongoing milestones initiatives, discussion of conceptual frameworks of competency Sub-groups Sub groups assigned to develop initial milestones and evaluation strategies for each competency
Second Meeting (May 2008)
Achieve consensus for minimum standard of competence Align evaluation strategies with individual milestones Establish writing committee
5
Milestones Task Force
Writing Committee
Third Meeting (December 2008)
Reconcile, refine and standardize milestones language Produce document that described need, process and context of milestones in medical education reform Review and provide feedback to writing committee in preparation for external review
Task Force had editorial independence Journal of Graduate Medical Education (www.jgme.org)
Milestones Conceptual Framework
Organized by competency sub sub-divisions divisions (i.e.“bullets”)
Each milestone framed in behavioral terms
IM-RRC 2009 proposed revisions Criterion-based Competence can only be observed in performance
Dreyfus model of skill acquisition and learning
Competence is the minimum expectation for completion of residency
6
Milestones Inherent Challenges
Approximate timeframes for achieving each milestone over three years
Flexibility for program-specific curricula
Suggested general evaluation strategies
Specific evaluation tools beyond the scope of this project
Page 9
7
Page 9
RRC sub-bullets
Page 9
8
RRC sub-bullets
Page 9
RRC sub-bullets
Page 9
9
Video
The following video is of a second year resident obtaining a history from an elderly man presenting to the ED with chest pain. Using the milestones for Historical Data Gathering, evaluate this resident in order to provide formative feedback. feedback Has this resident demonstrated behaviors expected of the 18 month “milepost” for this milestone?
Small Group #1 - Scenario You have a scenario detailing a “critical critical moment” in a residency training when a resident transitions into a new role or responsibility. In small groups, consider the set of skills a resident should demonstrate in order to make this transition.
10
Small Group #1 - Task Circle all of the developmental milestones that the resident should demonstrate before making this transition. Do the milestones appear to reflect your expectations of a resident at this point in their training?
Report of Small Group #1
11
Small Group #2 Using the milestones that you have circled, write down how you would evaluate that the resident has demonstrated that behavior. Do you have a specific evaluation tool in mind?
Report of Small Group #2
12
Large Group Now take the milestones document in total … does the compilation add up to the minimum of what you expect of a resident ready to transition to independent practice?
