SAVE | EMAIL | PRINT | MOST POPULAR |  RSS | | REPRINT
|
Measurements for Evaluation and Management of the Training Department (PART 2 OF 4)
April 27, 2009
By Dan Klein, Ph.D.
The following article presents a comprehensive evaluation framework based on a regular managerial rationale, i.e., improvement of effectiveness and efficiency based on four types of data: effectiveness, efficiency, cost, and quality. This concept differs from the current evaluation framework (Kirkpatrick), which deals mainly with effectiveness, and is based on learning rationale (i.e., the first three levels of Kirkpatrick's model are measures of changes in learner's behavior as a consequence of the learning solution). The article will be presented in four parts. Last week, Part 1 was published; click here. The following is Part 2. The next two parts will follow in the coming weeks.
2 Evaluating the Training Department's Effectiveness (Cont.)
In order to be effective, the Training Department has to attain two types of goals:
a. Improving the organization's competence level
b. Reducing learning processes' costs
Improving Competence Level
One can evaluate the contribution of the training department to the level of organizational competence by combining the following two types of measures:
a. Amount of acquired competences—An effective training department enables the employees to acquire more competencies. There are several ways to measure the amount of acquired competencies such as by the number of employees who acquire competencies or the reduction of the overall organization's competency gap.
b. Contribution of acquired competencies—An effective training department will invest resources in competencies that relate to important performance, i.e., those that contribute the most to organizational goals.
Reducing Learning Processes' Costs
The process of acquiring competence begins at the point at which a competence gap is created, and ends at the point at which the employee receives the required competence. In instructional terms, one can call the starting point "Instructional need," and the end point "Certification point" (see Definition 6 in Appendix 1). The time between the emergence of the instructional need and the certification point is called Time-To-Competence or TTC. Figure 2 (shown at beginning of this article) presents a typical case of TTC. Note that under this conception the training process also includes situations in which the training department has only partial control over the learning situation, as in on-the-job training (OJT). The responsibility of the training department must start in the need and end in the certification.
The length of time to competence is directly related to the amount of damages to the organization. Following are the three types of such damage:
• Working with partial competence—Until such time as the worker receives the necessary certification, he or she works with only a portion of the skills and knowledge needed for the job. This means that some of the tasks will not be accomplished properly or will take more time. The likelihood of making harmful mistakes also increases.
• Learning instead of producing—While the employee is learning, his or her production is reduced. Learning can take many forms: the employee may read a workbook, may look for information on the organization's Intranet or can participate in a course.
• Coaching costs—In many cases, the last phase of learning involves an OJT period. This means the involvement of a colleague, subject matter expert, or another employee whose time is now spent on coaching rather than producing.
These three problems negatively affect the resources of the line manager and his or her chances of attaining the group goals. Thus, the manager should do whatever he or she can to reduce the time to competence.
Competence can be achieved only by learning, and learning takes time. Thus, time to competence costs are inevitable, and the aim is to reduce them as much as possible. This can be done by managing the time to competence in the best method. When the TTC is managed properly, the instructional need is identified as it appears and the employee immediately receives the appropriate instructional solution. The alternative to properly managed TTC is unmanaged TTC. When the TTC is not managed, the employee learns while working in an unstructured haphazard fashion. This type of learning involves high costs because it takes longer to accomplish properly and it involves experts' time investments.
The damage caused by TTC can be evaluated by using the following three types of measures. All three measures are based on time, and thus can be easily translated to monetary terms:
a. Time of partial functioning—The damage caused to the organization is, in fact, the gap between the level of production of a competent employee and the level of production of the partially competent employee.
b. Learning time—The damage caused to the organization is the time the employee was involved in learning instead of producing. It is relatively easy to measure this time when the employee learns by taking a course. It is more complicated to measure the time spent in self-learning or OJT. A learning management system can help a great deal in managing this type of time.
c. Coaching time—The damage caused to the organization is the cost of the expert’s time.
Following is an example of reducing the organization's damages by reducing the TTC. Let us assume that an organization consumed 1,000 course days in a year. If the employees were learning by themselves (i.e., no training department) the time to reach a certification would have tripled. This means the training department saved 2,000 days to the organization. This is a rather simplistic example because it does not specify all of the factors that should be taken into consideration. For example, it assumes that each course ends with a certification; it ignores the contribution of an employee with partial competence; it doesn't take into consideration the involvement of the coachers and mentors, and so on. Nevertheless, it shows the essence of this type of contribution to the organization.
This type of measurement possesses the following three meaningful advantages:
• ROI—The measure can be easily translated into financial terms. Since management is attentive toward any reduction in costs it is relatively easy to show the contribution of the training department.
• Good opportunity to prove effectiveness—Almost every course (with at least fair quality) will produce good ROI. This is so because any professional training is of shorter duration relative to haphazard learning. Even a simple activity, such as locating and collecting all of the relevant documents by the instructor, saves the time that each learner would have invested in a similar activity.
• Change in the status of the training department—Employee time is a resource that belongs to the business line managers, not to the training department. The length of the TTC is inversely related to the time resources the managers will have for attaining their business goals. If the organization evaluates the business line managers by the TTC's costs of their employees, their only option to reduce time will be obtaining assistance from the training department. This type of dynamic will dramatically change the training department's status in the organization.
Please check back next week for Part 3 of this article.
Appendix 1 - definitions
1. Learning process—An experiential cognitive process during which the learner's behavior changes as a consequence of acquiring knowledge and/or skill and/or attitude.
2. Teaching process—Planned processes that facilitate learning processes.
3. Training process—Teaching that is carried out in a business organizational environment. Following are the main characteristics of a training process:
• Aims to acquire a defined business goal
• Created by planned exposure to structured stimuli
• Financed by the organization to which the learner belongs
4. Training department—The organizational body that is responsible for the training processes. The training department, as any other department, is based on cost-effectiveness considerations. In many organizations, the training department has various customers, e.g., the organization's employees, customers, or distributors. The current paper focuses on the organization's employees as the training department's customers.
5. Competence—Competence does not have a clear accepted definition. For our purposes, competence is the ability to perform a job at the required professional standard. In the current context it is important to differentiate between two types of competences: "acquired competence" and "not acquired competence." Acquired competence is a change in the behavior as a result of acquiring new skill, knowledge, or attitude. For example, learning to program in Java. Not acquired competence is a personality trait or physiological characteristics such as patience or coordination. In the current paper the term "competence" refers to acquired competence.
6. Certification—Formal organizational approval that an employee has acquired a competence that was defined by the appropriate manager. The approval is based on assessing the employee's capability to independently perform parts of the role.
7. Efficiency—Producing maximum output with minimum input.
Dan Klein is a specialist in learning and training strategies and consultant to the high-tech business sector. He received his Ph.D. from the Department of Educational Psychology - the University of Illinois at Urbana-Champaign, and currently lectures at the department of Instructional Systems Technologies at the Holon Institute of Technology in Israel. He previously served as a major in the research and development section / training department of the Israeli Air Force. For more information, e-mail Klein at kleindr1@netvision.net.il
|
SAVE | EMAIL | PRINT | MOST POPULAR | RSS |
|
|
| Back to Training Index |
|
|
