Evaluation involves the assessment of the effectiveness of the training programs. This assessment is done by collecting data on whether the participants were satisfied with the deliverables of the training program, whether they learned something from the training and are able to apply those skills at their workplace. There are different tools for assessment of a training program depending upon the kind of training conducted. Since organisations spend a large amount of money, it is therefore important for them to understand the usefulness of the same.
For example, if a certain technical training was conducted, the organisation would be interested in knowing whether the new skills are being put to use at the workplace or in other words whether the effectiveness of the worker is enhanced. Similarly in case of behavioural training, the same would be evaluated on whether there is change in the behaviour, attitude and learning ability of the participants. | Benefits of Training Evaluation Evaluation acts as a check to ensure that the training is able to fill the competency gaps within the organisation in a cost effective way.
This is specially very important in wake of the fact the organisations are trying to cut costs and increase globally. Some of the benefits of the training evaluation are as under: * Evaluation ensures accountability – Training evaluation ensures that training programs comply with the competency gaps and that the deliverables are not compromised upon. * Check the Cost – Evaluation ensures that the training programs are effective in improving the work quality, employee behaviour, attitude and development of new skills within the employee within a certain budget.
Since globally companies are trying to cut their costs without compromising upon the quality, evaluation just aims at achieving the same with training. * Feedback to the Trainer / Training – Evaluation also acts as a feedback to the trainer or the facilitator and the entire training process. Since evaluation accesses individuals at the level of their work, it gets easier to understand the loopholes of the training and the changes required in the training methodology. Not many organisations believe in the process of evaluation or at least do not have an evaluation system in place.
Many organisations conduct training programs year after year only as a matter of faith and not many have a firm evaluation mechanism in place. Organisations like IBM, Motorala only, it was found out, have a firm evaluation mechanism in place. The Way Forward There are many methods and tools available for evaluating the effectiveness of training programs. Their usability depends on the kind of training program that is under evaluation. Generally most of the organisations use the Kirk Patrick model for training evaluations which evaluates training at four levels – reactions, learning, behaviour and results.
After it was found out that training costs organisations a lot of money and no evaluation measures the return on investment for training, the fifth level for training evaluation was added to the training evaluation model by Kirk Patrick which is called as the ROI. Most of the evaluations contain themselves to the reaction data, only few collected the learning data, still lesser measured and analysed the change in behaviour and very few took it to the level of increase in business results. The evaluation tools including the Kirk Patrick model will be discussed in detail in other articles.
Training Evaluation When we talk about evaluation in training, we tend to think of tests administered in the class to check our understanding of the content. However, let’s look at assessments from a broader perspective. Let’s look at a system of evaluation that has become a training industry standard: the Kirkpatrick model of evaluation. This model acknowledges that we evaluate for a number of different — and unique — reasons and in a number of different — and unique — ways. Level 1: Reaction Evaluation reaction to training usually takes the form of end-of-course evaluations.
While they can be used to improve the training, they should NOT be interpreted to show how effective the training was. End-of-course evaluations (or “smile sheets”) identify whether someone liked the training or whether they thought the training was effective. Here are some tips for making the most out of this type of evaluation: * Determine the purpose. * Create specifically for each training: Avoid using “one-size fits all”. If you have a standard form, customize it by adding questions specific to your course. Write majority of questions close-ended: Getting anecdotal quotes from students is interesting and makes for good marketing. However, few of us have time to conduct a content analysis on the comments. Use more quantitative measures. * Balance positive and negative questions: Asking participants to list the most important things about the training without asking about the least important things leads to an imbalanced, biased feedback. * Ask questions about transfer and impact: Go beyond the traditional smile sheet.
Ask “What, if anything, will make it difficult for you to use your new skills on the job? ” “Will your manager be able to help you with your new skills? ” “How confident are you that you will be able to use your new skills on the job? ” “How do you expect your job to change as a result of using these new skills? ” * Provide time: Give time and expect participants to use the allotted time to complete the feedback thoroughly and thoughtfully. * Consider handing the evaluation out at the beginning of the training and encourage participants to complete it throughout the training.
Level 2: Learning Just because we know that participants like our training, doesn’t mean that they’ve learned anything. We need to determine whether they learned something. Level 2 evaluations can be used to determine if learning occurred. This information is good for three primary reasons: Does the participant have the required skill and knowledge? These level 2 evaluations are also known as summative tests or certification exams. Where does the participant require remediation or additional development?
These level 2 evaluations are known as formative tests or developmental evaluations. How can the training be improved? Sometimes that absence of learning indicated in level 2 evaluations is because of the teaching style or the materials. Level 2 assessments can tell you WHAT needs attention; and with time and analysis, can tell you WHY some participants are picking up the content and others aren’t. Tips to using level 2 evaluations effectively: * Determine the purpose: Why are you evaluating the learning? Who needs to know (students, managers, the instructor)?
Why (improve the training, hire/fire/promote, give participants a gauge)? * Link tests to performance: If participants need to know something, design the test to measure knowledge (multiple choice, true-false, essay, matching, short-answer). If participants need to do something, design behavioral evaluations (i. e. , role plays or demonstrations) with a measurement instrument (Behavior Checklist, Behaviorally Anchored Rating Scale, Behavior Frequency Checklist, Best-Solution Approach). See the examples on the right for simple performance-based checklists. We’re making a clear distinction between knowledge and performance for a very good reason. Just because you know something doesn’t mean that you can do it. Having knowledge of brain surgery is not the same as having experience performing brain surgery. * There are two primary types of Performance-Based Evaluations: labs for procedural performance and role plays/demonstrations for non-procedural (principles). * Pilot the test and validate: Strong performers should do better than weaker performers; the test should give similar results over time.
Level 3: Transfer Just because we know participants liked our training (level 1) and they learned something (level 2), doesn’t mean that they can apply it back on the job (level 3) As performance consultants, we need to measure if participants are using their newly acquired skills and knowledge on the job. Level 3 evaluation can be used to improve the training and to identify work environment barriers that prevent participants from using their skills and knowledge.
Level 3 evaluation can be as simple as observing performance; gathering information from participants, managers, and others; or accessing records/artifacts such as performance data, error data, customer satisfaction information, etc. While evaluating learning transfer may be an expensive and thankless task involving control groups and quantitative statistics, the goal is to ensure that learning is transferring to the job. Here are some suggestions for making sure that training transfers: If you are management (any level above supervisor): * Use an advisory committee to ensure training addresses business needs. Communicate the importance of training in general and this training specifically. * Coach supervisors appropriately; model coaching behavior. * Establish standards for training follow-up and coaching by supervisors. * Provide time for training and follow-up. If you are a supervisor (the trainees’ immediate manager): * Develop performance measurements to measure gains from training. * Attend a training orientation for supervisors. * Select trainees based on specific criteria. * Conduct a pre-training meeting with trainees. * Use graduates of previous training to prepare trainees. Allow time to prepare for the training. * Plan for after training follow-up. The trainee: * Self-assess his/her skills to identify and focus needs. * Complete precourse work. The trainer: * Identify performance needs; conducts task analyses to determine content and flow. * Identify other elements impacting performance (job design, information, resources, feedback, etc. ) * Provide an advance letter to the supervisor. * Coach the supervisor on ways to increase training transfer and effectiveness. * Provide an advance letter to the trainees. Provide precourse work. * Plan for after training follow-up. Level 4: Business Impact Just because we know that 1) participants liked our training, 2) learned something in our training, and 3) applied what they learned on the job, we still don’t know if the business is better off. Training impact refers to the measurable change in profits, revenue, errors, or other operational factors linked to your training. This is also called Return on Training Investment. (See Entelechy’sReturn On Training Investment Calculator. ) How do we determine the benefits of our training? Work with the client to identify — before the training — what performance measures would be important and credible. * Determine what data would point to success in your training. Determine how best to collect the data (without excessive administrative work). * Determine what other factors are important to the desired performance. Evaluation Summary Knowing WHAT you’re evaluating and WHY you’re evaluating will help you determine HOW best to evaluate. Using Kirkpatrick’s four levels of evaluation will help you use training evaluations effectively. For assistance with YOUR evaluations contact Entelechy.
