SAVE | EMAIL | PRINT | MOST POPULAR |  RSS | | REPRINT
|
A Fresh Look at Measuring Learning Retention
May 28, 2009
By Dan Hawthorne
Training officers often struggle to glean good information about a training program's effectiveness. It's hard to nail down what to look for while also determining what will apply in every situation.
A recent article in the March 2008 issue of the Journal of Applied Psychology provides information about trainee reactions that offers indispensable insights to anyone facing these decisions. In the article, researchers Sitzmann, et al. performed a meta-analysis (a statistical process that pools research data to look for patterns in similar studies) on 136 studies from the last 30 years, involving a total of 27,020 trainees.
One of their main findings speaks to the influence of the instructor on learner reactions. Instructors have the most impact on trainee reactions, accounting for 37 percent of the variance. That puts instructor style above the factors of organizational support (32 percent) and individual trainee characteristics (31 percent). And while an instructor's style doesn't seem to have a direct relationship with post-training motivation and self-efficacy, there is a direct relationship between trainee reactions and those two outcomes. So, if the instructor can impact those reactions, then post-training motivation and self-efficacy have a good chance of following suit.
Because of this effect, the researchers recommend practitioners keep in mind how instructors can best affect reactions. The researchers discovered anxiety levels could destroy how well a trainee rates a course. When trainees are anxious about course material (or outside influences), their minds are elsewhere. They don't focus on the coursework in front of them.
To deal with anxiety in the classroom, instructors should:
• Create an open and relaxing atmosphere in the classroom. This can enhance trainee satisfaction with the course.
• Be aware of what behaviors indicate anxiety, and be on the lookout for them.
• When leading challenging coursework, consider including some relaxation techniques on breaks.
Additionally, post-training self-efficacy (which looks at learners' self-reported confidence in their capabilities) emerged as a useful predictor of learning outcomes. Particularly, self-efficacy is a predictor of long-term learning of procedural skills, accounting for 24 percent of the variance in measurements of long-term procedural knowledge. Ideally, you want to measure learning outcomes through an objective test. However, if you can't do that, ask trainees a few survey questions about their confidence to perform a skill. It offers a useful (and indirect) way to measure how well they have learned the skill. Ask trainees pointed and specific questions about the training to make them think about performing the skill. Doing this makes them run though performing the skill in their minds, and helps them call up how confident they truly feel.
Another important finding is that trainee reactions can predict motivation during training as well as changes in attitude after training. So, learner reactions may be useful in predicting enrollment rates or the course's reputation within the organization. In a nutshell, reactions can be a window into how receptive a trainee is to attitude the change you want to achieve. If the course's goal involves attitude change or motivation, then it becomes important to measure reactions in a focused fashion. Try to gauge how well the attitude change has taken hold by asking clear and direct questions in the post-training assessment.
An essential take-away of this research is that reaction measures have to be used appropriately. In many cases, organizations use generic post-training reaction sheets for every training program they implement because doing so makes it much easier to use the reaction sheets. However, "easier" is rarely better, and in some cases, creates other problems. It can be very important to target a specific training program with a training evaluation that speaks directly about the program in question. Trainees usually can detect generic questions, and many of them will start ignoring the questions, and just start making marks in a straight line down the sheet. But if the sheet asks pertinent questions that relate directly to the training, learners are more likely to read it more closely. They'll answer more thoughtfully as well.
Many times when I've spoken to clients about post-training evaluation sheets, I hear them talk about the same problems and disappointments. Frequently, these evaluation sheets are used because trainees expect to see them, and management wants to see numbers that say people enjoyed the training. However, in many cases, these sheets are thrown into a file drawer, and the data on them is never used.
You can get powerful data from these post-training evaluation sheets when you use evidence-based measures that incorporate information such as what's been discovered by researchers Sitzmann, et al. Effective post-training evaluations can serve multiple functions: measuring training efficacy, stimulating employee buy-in, and increasing engagement and behavioral change.
In the second part of my article on measuring learning retention, we'll discuss ways to create meaningful evaluations, generate relevant responses, and gather powerful data. These processes can mean the difference between your training program's ROI being written in black ink or red ink.
Dan Hawthorne is director of research for Intulogy LLC.,
an industry-diverse outsource training provider specializing in design and delivery of training programs built around its clients' proprietary subject matter.
|
SAVE | EMAIL | PRINT | MOST POPULAR | RSS |
|
|
| Back to Training Index |
|
|
