Training Evaluation: Part 1

CHL Self-Study Lesson Plan Lesson No. CHL 342 (Supervisory Continuing Education - SCE) Sponsored by: by Jack D. Ninemeier, Ph.D. Professor Emeritus ...
Author: Hillary Barker
18 downloads 0 Views 181KB Size
CHL Self-Study Lesson Plan Lesson No. CHL 342 (Supervisory Continuing Education - SCE) Sponsored by:

by Jack D. Ninemeier, Ph.D. Professor Emeritus The Eli Broad Graduate School of Management Michigan State University

Training Evaluation: Part 1 Learning Objectives 1. Explain reasons to evaluate training 2. Review basic training evaluation concerns 3. Describe when training should be evaluated

Supervisory Continuing Education (SCE) lessons provide members with ongoing education focusing on supervisory or management issues. These lessons are designed for CHL re-certification, but can be of value to any CRCST in a management or supervisory role.

C

entral Service (CS) leaders rely on a well-trained, knowledgeable and dedicated workforce to address the numerous critical responsibilities required of their departments. New employees must receive initial training and their more experienced counterparts must be regularly updated about new methodologies, regulations and numerous other concerns. Significant time, money and expertise are required to provide necessary training, and it is important that these resources be wisely expended. As well, training results must be assessed and entered into the employees’ personnel records for subsequent assessment by regulatory and guideline-setting organizations. Training results are also important because they are part of the supporting structure for a quality management system, such as ISO 9001 or ISO 13485. Is training evaluation important? Yes, and this first lesson in a two-part series discusses the reasons to evaluate training, basic training evaluation concerns, and times when training should be evaluated.

You can use these lessons as an in-service with your staff, or visit www.iahcsmm.org for online grading at a nominal fee. Each lesson plan graded online with a passing score of 70% or higher is worth two points (2 contact hours). You can use these points toward either your re-certification of CRCST (12 points) or CHL (6 points). Mailed submissions to IAHCSMM will not be graded and will not be granted a point value (paper/pencil grading of the SCE Lesson Plans is not available through IAHCSMM or Purdue University; IAHCSMM accepts only online subscriptions).

www.iahcsmm.org

Overview of Training Evaluation Evaluation is the final step in the training process, and it is used to determine the extent to which the training was successful (if goals were attained). If the training was successful, trainers can move on to plan and deliver training that considers other knowledge and skill domains. If, however, the evaluation indicates that trainees cannot perform according to expectations, the trainers can refocus and reemphasize applicable aspects of the training. Sometimes the evaluation might suggest more extensive post-training knowledge or skill performance challenges, and then it may be necessary to revise and repeat the entire planning process. Consider a training program to improve supervisory skills to reduce employee turnover. Assume the initial definition of training needs leads to the development of a training program to ad-

dress improvements in how supervisors interact with employees. Also assume that post-training evaluations indicate that the goal of a specified departmental turnover rate was still not satisfactory within six months of training completion. Instead, further analysis revealed the turnover rate for newly hired staff members had not changed and had even worsened. CS leaders may then determine the problem actually relates to selection, orientation and early training activities of new staff members. In this example, trainers may need to plan and implement an entirely different training program to properly address turnover. It is likely that the “real” problem might never have been correctly identified without reassessment driven by an after-training evaluation to measure training success. In another situation, post-training evaluation may indicate the program was

MARCH / APRIL 2014

Communiqué

CHL Self-Study Lesson Plan

less-than-totally-successful. It did focus on the correct problems, but the training plans and/or training lessons didn’t provide the best content. In this example, only selected steps rather than the entire training process, may need revision to improve training results. Reasons to Evaluate Training Time, money and labor are increasingly in limited supply. CS leaders must determine whether their commitment of resources to planning and implementing training procedures is a better tactic than alternative uses for the resources. This is one reason training evaluation is important. More specific reasons to evaluate training include to: • Assess the extent to which training achieved planned results – Training

goals identify competencies to be addressed in training and provide a benchmark against which training can be evaluated. Assume one objective of a training program is to teach a new CS technician how to properly operate a specific item of instrument processing equipment. The exact operating instructions in the manufacturer’s operating manual can form the basis for the training procedures. This objective drives the training (it provides material for a training lesson that indicates how to properly operate the equipment), and it provides a way to evaluate training effectiveness (the extent to which each required operating step is sequentially and correctly used). • Identify Training Strengths and Weaknesses – Few training programs are 100% effective or ineffective. Some training lessons are better than others, some training activities are more useful than their counterparts, and some trainers may be more effective than their peers. Successful evaluation can identify training aspects that should be continued Communiqué

MARCH / APRIL 2014

and elements that may require revision or replacement. • Determine success of individual trainees – Trainees who are successful (achieve planned results on the job) will not require remedial training. However, training for others may need revision and/or repetition. Assessment of individual trainees is relatively easy when an individualized, on-job training method is used, and it is more difficult with group training. The importance of the assessment is, however, equally important. • Gather information to help justify future programs – When the success of a training activity is quantified, objective information becomes available to help justify future training efforts. Alternatively, CS leaders can determine whether resources are better invested in other improvement efforts. • Determine trainees eligible for future training – Some healthcare facilities provide educational or training activities in career development programs that require prerequisite training. Other facilities may have less formal programs, in which selected trainees who successfully complete training programs are eligible for additional training opportunities. These, in turn, lead to increased promotional considerations as vacancies occur. n both of these instances, CS leaders must know whether and to what extent individual trainees successfully completed the training. • Assess costs/benefits of training – This reason for training evaluation has already been suggested. The expenditure of any resource must generate a return greater than the cost of resources allocated for it. Some benefits of training including improved employee morale and increased interest in attaining quality

goals are difficult to quantify. Others including improved performance knowledge and skills and reduced operating costs for a specified task may be easier to quantify, and both could be assessed by effective training evaluation. • Reinforce major points for trainees – Some training evaluation methods including objective tests and performance appraisal interviews allow evaluators to reinforce the most important training points. For example, questions on a written assessment likely address the most important training concepts. If the assessments are self-graded or reviewed by the trainer, reinforcement of these important points becomes possible. Likewise, if performance appraisal interviews address training concepts contained in training assessments, additional opportunities for reinforcement arise. • Assess trainees’ reactions to training – Trainers who are interested in improving training programs want to gain the trainees’ perspectives about the programs. Anonymous input gained before, during and after training can be helpful in this assessment. • Assess trainers’ reaction to training – “There’s always a better way” is an old saying that applies to training as well as to other management tactics. Trainers who have used a training lesson, for example, may have ideas about ways to improve it in the future, and training evaluation can help to identify and suggest changes to existing programs and materials. Training Evaluation Concerns Any training evaluation method must address at least five assessment concerns: • Valid – Training evaluation methods must be valid; they must measure what they are supposed to measure. Assume a

www.iahcsmm.org

CHL Self-Study Lesson Plan

FIGURE 1: PRE-TRAINING SELF-ASSESSMENT FORM Training Topic: Name of Trainee (Optional): To be completed before training To be completed after training If this training is ideally effective, I will learn the following:

To what extent did you learn what wanted to learn (check one)? Not all Somewhat

Very Much

1. _________________________________________________ _________________________________________________ _________________________________________________

q

q

q

2. _________________________________________________ _________________________________________________ _________________________________________________

q

q

q

3. _________________________________________________ _________________________________________________ _________________________________________________

q

q

q

4. _________________________________________________ _________________________________________________ _________________________________________________

q

q

q

5. _________________________________________________ _________________________________________________ _________________________________________________

q

q

q

Comments:

training objective focuses on the ability of trainees to successfully complete an inventory count using methods taught during training. Trainee demonstration of scanning equipment operation and proper use of inventory counting procedures suggests the training was successful (at least at the time of the demonstration). In contrast, if the training assessment queried trainees about issues such as “Did you like the training?” and “Did the trainer seem enthusiastic?” the input would not allow trainers to determine whether training objectives were attained. • Reliable – Training evaluation methods are reliable when they consistently provide the same results. Training activities that are implemented in the same way by

www.iahcsmm.org

the same trainers using the same training resources and procedures to train employees in the same position may be consistent. Will the results be the same or similar each time it is replicated? Trainers do not know unless the same evaluation methods are used. • Objective – Objective evaluation methods provide quantitative (measurable) training assessments. Acquisition of knowledge can be objectively measured by performance on a well-designed test. Efficiency in a skill might best be assessed by observing the trainee’s performance of the task after training. Then performance can be considered “acceptable” if the procedures used are in concert with those taught during training.

• Practical – A training evaluation method is practical when the time and effort required for the assessment is “worth” its results. Knowledge assessments that require trainees to memorize mundane facts, and skill demonstrations that are benchmarked against staff with extensive experience and efficiency in performing the task are not practical. • Simple – An evaluation method is simple when it is easily applied by the trainer, easily understood by the trainees, and when results are easy to assess and analyze by those evaluating the training. When Should Training Be Evaluated? Many CS leaders think about training evaluation in the context of an after-training assessment. Training should be evaluated at its completion, and it is also useful several months or even longer after the training is completed to determine if the training information is being applied. Evaluation can also be helpful before training even begins and while it is being conducted. Let’s review some before, during and after training methods that incorporate these assessment concerns. Evaluation Before Training Assume a trainee participated in an instructor-led group training program relating to inventory management methods and completed a knowledge assessment test at its conclusion. Assume further that the trainee missed only two questions out of the 20 that were asked. Many trainers would likely conclude that the training was successful because the participant scored 90% on the quiz (18 questions ÷ 20 questions). In fact, the training could really have been a waste of the facility’s resources and the trainee’s time if the trainee already knew the concepts addressed by the eighteen questions

MARCH / APRIL 2014

Communiqué

CHL Self-Study Lesson Plan

before the session began. In actuality, the after-training evaluation really measured what the trainee knew when the training was completed rather than what he/she learned from the training session. To address this concern, some trainers use a pre-test/post-test evaluation. A trainer can identify key concepts to be addressed during the training. These concepts are addressed in a pre-test administered before the training begins. This same measurement tool using the same questions is then administered at the end of training. The improvement in scores between the pre-and post-test represents a measure of training effectiveness. Other advantages of pre-test/post-test evaluation include: • It provides trainees with an overview (preview) of the training • It helps trainees identify some of the most important concepts that will be addressed in training • It presents an opportunity for trainers to preview the lesson and suggests priority learning points before the training begins As is true with other training evaluation methods, it may also be helpful to administer the post-training assessment several months (or even longer) after training is completed. This tactic can help determine the extent to which training information has been retained and actually applied in the workplace. Another pre-training evaluation tactic involves an exercise that requests trainees to answer the following concern: “If this training is ideal, I will learn the following from it…” This activity can be included in an introductory session that previews subject matter to be addressed in the training. Trainees might be asked to retain their responses to this “ideal training” exercise until the end of training. Then they could undertake a post-training assessment to Communiqué

MARCH / APRIL 2014

Should Training Evaluations Be Anonymous?

Input from any training evaluation method is not useful unless it is truthful. It is generally easier to obtain anonymous input from group trainees than from those participating in individualized training. An obvious reason: “There is safety in numbers.” The best evaluations are often provided by unidentified trainees who do not fear on-job retaliation (especially if they are supervised by the trainer!), and those who believe the request for input is genuine and that the results will be used. Healthcare organizations with a culture of respect for employees and a history of utilizing their input for improvements will likely obtain useful feedback. By contrast, their counterparts that often have adversarial relationships between supervisors and staff may have great difficulty in obtaining useful information by any evaluation method. Given the above, should names of any evaluators be requested? The answer is, “probably not,” unless there is unique and specific reason to do so. A compromise: some trainers request anonymous input, and then seek out trainees to provide more detailed one-on-one input. Mentors in some organizations receive trainee input, and summarize this information in reports to human resources or other applicable staff members.

note their perceptions about the extent to which the training was ideal. Figure 1 illustrates a self-evaluation form that could be used for this purpose. Evaluation During Training During an introductory session, trainers can indicate they will ask for feedback during the session. As this feedback is solicited, trainers can obtain a “first-half ” reality check to discover information helpful to improve the remainder of the training. Trainers facilitating group sessions can ask trainees to write anonymous responses to questions such as: • I wish you would stop doing (saying)… • I hope we continue to… • I don’t understand… • I hope you will begin to… • A concept that I wish you would discuss further is… • A concept I want to learn more about that has yet to be discussed is…

Evaluation After Training After-training evaluation can help assess whether training achieved its planned results. It may also identify how training sessions might be improved and assess the trainees’ success. Numerous evaluation methods can be used. Experienced trainers often employ more than one method and analyze the combined results to yield a comprehensive assessment of training results. In Conclusion This lesson is the first in a two-part series and has explained the importance of training evaluation. It has also noted that the process is necessary before and during the training process, as well as at its conclusion. The second and last lesson in the series will describe and provide examples of training evaluation methods and review follow-up documentation efforts.

The major point is to learn how to maximize use of the remaining training time. Then revisions to training content and/or delivery methods, if necessary, can better assure attainment of training objectives.

www.iahcsmm.org