Training Effectiveness Measurement for Large Scale Programs - Demystified!

A 4-tier Practical Model for Technical Training Managers

Raman K. Attri

• Publisher: Speed To Proficiency Research: S2Pro©
• ISBN:
  9789811189906
• Topic: Business
• Subtopic: Human Resource Management
• Index: Business

This book addresses the challenges typical technical training managers, and other technical managers face in justifying the return on investment of their programs, particularly for large-scale, investment-intensive programs. This book describes a very intuitive and practical model for the measurement of the effectiveness of technical training programs. The book is based on a 4-tier Return on Effectiveness (ROE) model developed through years of research, observation, and experience. The ROE model uses four simple indices: training reaction index, improvement index, effectiveness index, and impact trends to convey a full picture of the effectiveness of a training program. This feedback-based and data-based approach enables training managers with a step-by-step approach to collect data, measure important indicators and compute the indices to present evidence of the effectiveness of their training programs to the executives.

Chapter 1

TRAINING ROI MEASUREMENT CHALLENGE

WHY DO YOU NEED IT ANYWAY?

MEASURING TRAINING ROI

---

Modern businesses rely heavily on the training of the workforce. Take the example of a company selling and repairing complex hardware. They are tightly pressed to reduce the time-to-resolution to keep the profit margins high and to keep their customer satisfaction high. The skills of the workforce supporting such functions are critical to the organizations. Typically, complex troubleshooting and higher order technical skills are affected through large-scale training programs. These are investment-intensive training programs that require money and resources. The substantial investments on training legitimately drive executives to ask: What is the Return on Investment (ROI) for a training program?

ROI is expressed by assigning dollar values to investment and tangible/non-tangible benefits. Measuring worthiness of an investment using ROI and equivalent dollar value is a much-loved preference at the executive level because it gives a quantified figure based on which it is much convenient to decide quickly. Executive staff and business managers typically are not training professionals and hence cannot relate well with soft or intangible benefits of a training program.

Therefore, they need training ROI as crucial metrics for them to show the overall business value of the training. That is where the challenge comes. The traditional training and learning managers typically either are not business managers, or they have a minimal background in finance. Such business acumen takes years to develop for traditional training managers/professionals. However, they are still forced to do and have little choice.

While the basic premise of ROI measurement to translate the value of training into dollars equivalent is very appealing but is not always practical. The foremost challenge with ROI approach is that it may take away the full perspective for which a training program is introduced in the first place – the skill gaps or skill enhancement. Admittedly skill gap must have some business or financial implications that would have led to the development or implementation of a training program under question.

The biggest argument given against ROI measurement approach is that training and learning is a multi-dimensional process whose value cannot be articulated simply in dollars. Though the role of training in improving performance is well understood, researchers argue that performance during a training event is short lived. The performance within a training intervention may not be the desired performance required at the job. In a study, Bjork (2009, p. 313) expressed that performance during a training event may not be the right indicator: ‘Performance during training is often an unreliable guide to whether the desired learning has actually happened. Considerable learning can happen across periods when performance is not improving and, conversely, little or no learning can happen across periods when performance is improving markedly.’ However, what matters is the job performance of an individual after training. Bjork (2009, p. 319) highlights a challenge as: ‘The problem for a training organisation is to maximise performance when it matters, that is, after training and, specially, when individuals are deployed’. Sonnentag and Frese (2002, p. 6) suggested that ‘One might argue that what ultimately counts for an organisation is the individuals’ performance and not their learning—although learning might help to perform well. This line of reasoning stresses that learning is a highly relevant predictor of performance but is not performance itself.’ During any training intervention, learning is contended as more important than in-training performance, while after the training, at the job performance is contended to be more important than learning.

The real effectiveness of training or learning is the performance someone displays in the field. However, performance in a job role cannot be attributed only to a training program, rather is attained through collective learning from several sources. Any training program does not work in isolation to produce outcomes or impacts during the actual job. The outcomes and final job performance in a job role are attained through close interactions with other support functions like the manager’s involvement, performance support systems, on-the-job mentoring, on-demand continuously learning that happens after a training program. Thus, the ROI of a training program alone provides little to no context for true effect and true contributors toward the performance.

Therefore, evaluating training interventions in...