SAVE | EMAIL | PRINT | MOST POPULAR |  RSS | | REPRINT
|
How Meaningful Are Your Learning Evaluations?
May 29, 2009
By Dan Hawthorne
It's become a ceremonial ritual. At the end of the training program, the instructor hands out the dreaded "Smile Sheet" that includes a wide range of questions on the trainer, the course, the room's temperature, and even the catering.
The smile sheet has gotten a bad rap over the years—not so much because of what it is, but more often by how it's used. Human nature tends to dictate that we do things as we've always done them, and that's what has happened with many post-training evaluations. Essentially, smile sheets aren't bad; badly written smile sheets are bad.
Psychology's signal theory tells us that what an organization does sends signals to the people in the organization. Those signals either can be positive or negative. Imagine that people are told they're required to attend a program, but they receive no real information on what they are expected to learn, and then they receive no follow-up feedback. These actions send some clear signals about how the organization values the program, even if that was not the intent. In dealing with trainees, intentions mean very little—it's all about how they perceived the program.
Thus, a post-training evaluation can be one of those signals to a trainee. When an evaluation asks generic questions on a standardized form, trainees get the message that the program isn't important. It's just another form (corporate junk mail). So, trainees pay little attention to the survey. If learners see questions that don't relate to the course or their job function, should training managers expect thoughtful, relevant answers?
Here are some examples of how questions can incorporate relevancy and get a trainee's attention:
• "Which seemed more pertinent to your job, (1) the role-plays of customer to salesperson, or (2) the group discussion exercises?"
• "Regarding using the MODE model of sales at your home office, do you feel that more customers will be responsive to what you've learned?"
Generic questions generate vague and often misleading data. When the evaluation involves at least a few specific, pointed questions about the training content, trainees are more likely to pay attention to the whole form. This simple change generates immediate reactions from trainees that yield powerful information about how knowledge will be retained and, more importantly, used on the job.
When employees perceive that the organization doesn't value the course's content, then buy-in and engagement also will decline. Drive behavioral change by conducting relevant evaluations that show the organization's real interest in learner reactions and training outcomes. Effective evaluations can serve dual functions: gauging knowledge retention while also activating engagement.
Another key function of well-built post-training evaluations is they can measure factors that impact a training program, but may not be the fault of the program itself.
For example, imagine a company that says it treasures innovation in its employees. It wants employees who actively test new ideas in order to ferret out better processes or products. The company sends its employees to great innovation workshops where the learners master innovation mind-sets and skills. Yet, when they come back from the workshop, no one applies those skills. Why? It's well known throughout the company that anyone who launches an unsuccessful idea will see their career stall. They may even be fired. So, no one innovates. Soon, senior leadership decides the whole training program failed (and assigns blame to the training department).
But the problem doesn't lie within the training program, the trainers, whether the trainees enjoyed the program, benefited from the material, or retained what they learned. The problem lies in an organizational value misalignment that a generic post-training smile sheet never will reveal.
Thus, researchers recommend examining some key points when measuring training retention, including:
• Organizational satisfaction: Are trainees satisfied with the organization they are working for? Do the company's espoused values align with those it puts into practice?
• Job satisfaction: Are trainees satisfied with the jobs they are working in? Do trainees feel like they have a future in their current jobs?
• Perception of having available resources to implement training: Do trainees have everything they need to be able to use what they are learning?
• Support by upper management: Is upper management fully behind the training program? Do employees know it? Will their supervisors be pushing them to use what they learned when they return to their jobs?
Many training evaluations ask the wrong questions, and so they fail to identify important non-training factors. Look critically at each question on the post-training evaluation and determine whether it adds value or generates noise. By asking smart questions in the post-training evaluation and analyzing them through statistical procedures, a training director can become a hero to his or her organization.
Imagine the same scenario as before. You are the training director for that company. During the pilot test, trainees report they enjoy the innovation training program and think it's a great course, but they never are going to use the information at work—because if they do, they could lose their jobs. Essentially, they "check out" of the training program before it ever starts. Now, imagine you deliver this information to senior leadership—so they can adjust the organizational climate on innovation before the training rolls out. You've likely just saved them a large amount of money, and yourself a lot of headaches trying to figure out why the program didn't work. And all of this would occur because you asked the right questions, instead of doing what's always been done.
Dan Hawthorne is director of research at Intulogy LLC., an outsource training provider specializing in design and delivery of training programs.
|
SAVE | EMAIL | PRINT | MOST POPULAR | RSS |
|
|
| Back to Training Index |
|
|
