What is the relationship among Kirkpatrickâ€™s four levels of evaluation? Would you argue for examining all four levels, even if your boss suggested you should look only at the last one (results) and, that if it improved, you would know that training had an impact? Provide at least one example (e.g., tool, device, etc.) that would be used to gather evaluation data at each level.
Your initial post should be 250 to 300 words. Use this weekâ€™s lecture as a foundation for your initial post. In addition to the Blanchard and Thacker (2013) text, use at least one additional scholarly source to support your discussion.
Week Six Lecture
Evaluation should not be underestimated as part of the training process. However, as Blanchard and Thacker (2013) point out, â€œmany rationalizations for not evaluating training continue to exist, and evaluation of training is often not doneâ€ (p. 315). This is often because training managers are resistant to the process of evaluation for reasons that Blanchard and Thacker (2013) have delineated in the course text, as well as for myriad other fears related to their own sense of self-efficacy and/or job security. However, it is important to note, that the evaluation process should be considered from the very beginning of the training process.
Figure 1: Copyright 2013 Pearson Education, Inc. publishing as Prentice Hall
When a training needs analysis is conducted, it is wise to begin considering all of the aspects of the training process as well as the intended outcomes of the training process. As an example, consider the layout of Chapter Nine of the Blanchard and Thacker (2013) text. It begins with a graphic that depicts the evaluation phase of training. Then the learning objectives are stated, clearly, succinctly, and achievably (SMART). They are:
- Describe the pros and cons of evaluation and indicate which way to go on the issue.
- Explain what process evaluation is, and why it is important?
- Describe the interrelationships among the various levels of outcome evaluation.
- Describe the costs and benefits of evaluating training.
- Differentiate between the two types of cost-effectiveness evaluation (cost savings and utility analysis).
- Describe the various designs that are possible for evaluation and their advantages and disadvantages.
- Define and explain the importance of internal and external validity (Appendix 9-1).
As you review the learning objectives and then do the same with the chapter, it becomes increasingly evident that Blanchard and Thacker (2013) have laid out precisely what they intend to do. They have followed through, with as many examples and means of addressing diverse learning styles as they perceive necessary toward design and delivery of the material for purposes of learning effectiveness and efficiency. It is obvious that their goal â€“ as well as your instructorâ€™s goal (which is in alignment) â€“ is to ensure you learn about ways you can successfully impart transferrable skills to others. The goal is for you to learn valuable information that teaches how to evaluate the effectiveness of future training sessions that you design and deliver.
Figure 2: Copyright 2013 Pearson Education, Inc. publishing as Prentice Hall
Just as Blanchard and Thacker (2013) have described levels of learning that are expected of you as a result of engaging with the chapter, your training should begin with a TNA, and then progress with design and delivery that will enable participantsâ€™ learning to be evaluated at the appropriate levels necessary for them to use the KSAs effectively, as is depicted in the graphic above.
Begin with the end in mind â€“ Steven Covey
Click the link below if you would like to know more about Stephen Coveyâ€™s important book, The 7 Habits of Highly Effective People, as it pertains to leadership and personal effectiveness.
Evaluate Both Training Process and Outcomes
Both the training process and learning outcomes should be measured to ensure that the knowledge participants are intended to gain is occurring as it is intended both individual employees and the organization. Just as an athlete cannot complete a task to win a game, or an artist cannot create a masterpiece without a vision for the creation, a trainer cannot do justice to evaluation â€“ either of process or outcomes â€“ without beginning with the end goals (objectives) in mind.
Figure 3: Copyright 2013 Pearson Education, Inc. publishing as Prentice Hal
Just as the TNA is a form of data collection for determining what gaps exist in training and performance, evaluation is a form of data collection and analysis to determine if the training needs have been met. This can be done using a variety of methods, levels of learning, and other important criteria, such as costs and benefits of the training experience.
Kirkpatrickâ€™s Four-Level Approach to Training Evaluation
This is a widely used conceptual framework for training evaluation consisting of the four levels of learning listed below:
||Level of Analysis
||Reaction (lowest, often referred to as â€˜smile sheetsâ€™)
||Did participants enjoy the training?
||Participants may be open to more training if needed. Process was pleasurable.
||Learning (next level, shows knowledge was gained)
||What did participants learn?
||Shows what has been learned by participants.
||Behavior (level 3, shows KSAs can and are being used)
||Have participants changed behavior as a result of the training?
||Did they gain hard or soft skills? How, if at all, have their attitudes changed? Can they apply learning gained?
||Results (highest of the 4 levels, demonstrates positive behavior changes for the organization)
||How does behavior or other change from training benefit the organization?
||The employee training has positively affected the organization.
Kaufmanâ€™s Five Levels of Training Evaluation
In a dynamic global environment wherein technological advances and information sharing are increasingly crucial concepts, it is important to consider training evaluation from both intra and inter organizational perspectives. Therefore, Kaufmanâ€™s five levels of evaluation have been shared below.
||Level of Analysis
||Enabling (first level)
||Are resource inputs available? (human, financial, physical)
||Determines feasibility for individuals, group, organization.
||Reaction (first level, comparable to Kirkpatrick)
||Reactions to methods, means, processes.
||Acceptability and efficiency.
||Acquisition (comparable to Learning above)
||Have participants mastered the competencies?
||Do they â€˜get itâ€™?
||Application (compare to behavior above)
||Can participants do what was intended?
||Individuals and groups can use the training in the organizational context.
||Organizational Output (compare to results above)
||How does the behavior or other change benefit the organization?
||Organizational payoffs, contributions from training.
||How, if at all, does the training benefit society?
||Does the training transfer to participant responses, societal consequences, or other payoffs (such as environmental)?
Phillips Five Level Return on Investment (ROI) Framework
Jack Phillipsâ€™ training model is an holistic approach that goes beyond Kirkpatrickâ€™s four levels to examine both cost and utility in training evaluation.
||Level of Analysis
||Reaction & Planned Action
||Did participants like the training and can they use it?
||Preparation for organizational implementation.
||Are there KSA changes?
||These were goals of training in terms of transference of KSAs.
||Changes in job behavior or other specific application of materials.
||These were goals of training in terms of applying KSAs.
||What is the impact on the business or other organization?
||Did the training achieve what was intended for the organizational goals?
||Costs (financial and other)
||Were the costs worth the benefits to the organization?
Peter Block is a renowned consultant who has been engaged in training, teaching, and writing for about three decades. He is an active partner in Designed Learning, a training company. When it comes to evaluation, Block (2001) has given time tested advice. He states:
Evaluation is not about ratings, it is about learning. It should be a conversation among participants. Get it in writing first, but then make it the beginning of a conversationâ€¦It gives people information they can act on, engages the people who are drifting away, and thus reinforces the belief that we can shape our destiny, not just observe and remember it (Block, 2001, p. 370).
Evaluation of training is the culmination of the wonderful processes that you have learned about and practiced in this course.
Forbes School of Business Faculty
Blanchard, P.N., & Thacker, J.W. (2013) Effective training: Systems, strategies, and practices (5th ed.). Boston, MA: Pearson.
Block, P. and 30 Flawless Consultants (2001). The flawless consulting fieldbook & companion: A guide to understanding your expertise. San Francisco, CA: Jossey-Bass/Pfeiffer.
Callibrain (July 17, 2013). Video review for the 7 habits of highly effective people by Stephen Covey [Video file]. Retrieved from http://www.youtube.com/watch?v=eOpKziGrxSE
Covey, S. (n.d.). The seven habits of highly effective people: Habit 2: Begin with the end in mind. Retrieved from https://www.stephencovey.com/7habits/7habits-habit2.php
Phillips, J. J. (1997). Handbook of training evaluation: Measurement methods (3rd ed.). Houston, TX: Gulf Publishing Company.