SAVE | EMAIL | PRINT | MOST POPULAR |  RSS | | REPRINT
|
Measurements for Evaluation and Management of the Training Department (PART 3 OF 4)
May 04, 2009
By Dan Klein
The following article presents a comprehensive evaluation framework "based on a regular managerial rationale, i.e., improvement of effectiveness and efficiency based on four types of data: effectiveness, efficiency, cost, and quality. This concept differs from the current evaluation framework (Kirkpatrick), which deals mainly with effectiveness, and is based on learning rationale (i.e., the first three levels of Kirkpatrick’s model are measures of changes in learner’s behavior as a consequence of the learning solution)." The article will be presented in four parts. Last week, Part Twowas published. The following is Part Three. The last part will follow next week.
3. Evaluating the Training Department’s Efficiency
An organization evaluates the contribution of the training department by the return (i.e., effectiveness) it gives on the organization’s investment (training budget). The ratio between effectiveness and costs actually means efficiency (see definition 7 in Appendix 1). Note that it is not enough for the manager to improve the ratio between cost and effectiveness; he or she also must show data that clearly proves the efficiency. When the department’s resources are used maximally, any cut in the budget will inevitably harm the department’s outcomes (i.e. will increase the organizational competence gap). In other words, when the department proves it is efficient, the organization has a good reason not to cut the budget.
To be efficient, the training department must perform its work processes efficiently. There are many work processes in the training department. Each process produces a product and incurs concomitant costs. The most important processes are those that contribute directly to the instructional solution. These processes are called "core work processes." Following are examples of core work processes: collecting and prioritizing needs, mapping the training contents, designing the instructional strategy, developing instructional solutions and training environments, and operational processes. Figure 3 shows examples of work processes and their relation to the overall measurement framework.
The challenge facing the training manager is how to reduce costs without hurting effectiveness. To achieve this, the manager must tightly control the quality of the processes’ products while trying to reduce the costs. Thus there are two types of factors the manager controls in order to improve the department’s efficiency. These are:
a. Quality: Collecting data on the quality of the department’s products. When the products possess the required quality the chances to be effective are higher.
b. Costs: Collecting data regarding the resources invested in the work processes.
3.1. Quality and efficiency
"Quality" means compliance of a process or its outcome with a predetermined set of excellence criteria. Quality in itself is not a departmental goal; quality data are mainly used to control the training product’s attributes. Quality affects effectiveness in the following ways:
a. Impact on time to competence: Inadequate quality of products lengthens the time it takes to learn and reach certification, and creates other related problems.
b. Impact on competence level: Lengthening the time to competence means more resources will be invested in the training processes. Since resources are limited, there will be fewer resources for other instructional solutions. In other words, fewer employees will be certified.
Note that good quality does not necessarily mean a high didactic level. A high quality solution means a solution that fits the instructional needs and the organizational situation. A passive lecture in a convention can be a quality solution for professional, experienced, employees.
The quality of the products can be measured in many ways. For example, one can evaluate the training materials by SMEs or didactic experts. The quality of the lesson’s delivery can be evaluated by feedback from learners or by expert observers. The overall quality of the instructional solution can be measured by final testing or by the application of the course content at the job location. Once data regarding quality is collected, one can check which product contributed the most (least) to the effectiveness of the training solution. This kind of check enables the training manager to evaluate where the resources should be invested in order to improve effectiveness with minimal costs.
3.2. Cost and efficiency
One can use the term "training costs" in two contexts:
a. Training department costs: These costs are part of the training budget. For example, a trainer or room costs. Reducing these costs (while keeping the same quality level) improves the department’s efficiency.
b. Organization's training costs: Costs the organization incurs because of the learning activities. For example, learning instead of producing. Reducing these costs is one of the goals of the training department.
In terms of ROI, the training budget is the organizational "investment" in the training department, and the reduction of the organization's training costs is part of the "return" on that investment. The current section deals with the "investment," i.e., training budget-related costs.
Efficiency can be improved by producing more effective outcomes per specific costs (or the same outcomes per fewer costs). To control costs, the manager has to collect data regarding each factor that influences the cost. For example, cost of a developer hour, overhead costs, etc. Note that any attempt to measure costs has to take into consideration the type of instructional solution that is required. Solutions differ in regard to the costs needed to produce them. For example, planning a simulation costs much more than planning a frontal lesson. Thus, before we measure costs we have to define the type of solution.
Based on the costs and solutions types, one can define efficiency measures. Such measures express the ratio between cost and product. Efficiency measurements can be used for a global evaluation of the training department or a specific work process. Examples of efficiency measures are, cost per one learner, average number of learners per solution, and number of learners per instructor.
Evaluating the training department's efficiency is a difficult mission. There are no accepted types of solutions with accepted costs per each solution, and there are no accepted benchmarks of what is considered a "good" cost-effective ratio.
Appendix 1: definitions
1. Learning process: An experiential cognitive process during which the learner’s behavior changes as a consequence of acquiring knowledge and/or skill and/or attitude.
2. Teaching process: Planned processes that facilitate learning processes.
3. Training process: Teaching that is carried out in a business organizational environment. Following are the main characteristics of a training process:
• Aims to acquire a defined business goal
• Created by planned exposure to structured stimuli
• Financed by the organization to which the learner belongs
4. Training department: The organizational body that is responsible for the training processes. The training department, as any other department, is based on cost-effectiveness considerations. In many organizations, the training department has various customers such as the organization’s employees, customers, or distributors. The current article focuses on the organization’s employees as the training department’s customers.
5. Competence: Competence does not have a clear accepted definition. For our purposes, competence is the ability to perform a job at the required professional standard. In the current context it is important to differentiate between two types of competences: "acquired competence" and "not acquired competence." Acquired competence is a change in behavior as a result of acquiring a new skill, knowledge, or attitude. For example, learning to program in Java. Not acquired competence is a personality trait or physiological characteristics such as patience or coordination. In the current article, the term "competence" refers to acquired competence.
6. Certification: Formal organizational approval that an employee has acquired a competence that was defined by the appropriate manager. The approval is based on assessing the employee’s capability to independently perform parts of the role.
7. Efficiency: Producing maximum output with minimum input.
Dan Klein is a specialist in learning and training strategies and consultant to the high-tech business sector. He received his Ph.D. from the Department of Educational Psychology-the University of Illinois at Urbana-Champaign, and currently lectures at the department of Instructional Systems Technologies at the Holon Institute of Technology in Israel. He previously served as a major in the research and development section / training department of the Israeli Air Force.
|
SAVE | EMAIL | PRINT | MOST POPULAR | RSS |
|
|
| Back to Training Index |
|
|
