Handbook of Training Evaluation and Measurement Methods

Jack J. Phillips, Patricia Pulliam Phillips

• Publisher: Routledge
• Year: 2016
• Print ISBN: 9781138797307
• eBook ISBN: 9781317632559
• Topic: Business
• Subtopic: Human Resource Management
• Index: Business

Today's economic climate means that anyone involved in training and development must be able to measure its effect on business performance. With a focus on costs, benefits, and return on investment, this book provides a comprehensive reference for those who are learning about or implementing an evaluation system.

This new edition is fully revised and updated to reflect current developments, with step-by-step guidance on a range of vital topics, including:

• Developing a results-based approach to HRD
• Evaluation design
• Data collection and measuring success
• Calculating program costs and ROI
• Increasing management support for HRD programs.

With end-of-chapter discussion questions and an accompanying online Instructor Guide, this fourth edition provides sound theory and practical solutions.

The Handbook of Training Evaluation and Measurement Methods is a complete and detailed reference guide suitable for HRD professionals and students in advanced courses in HRD, training evaluation, and program evaluation.

Part I The Measurement Imperative

The Need for Results-Based HRD

DOI: 10.4324/9781315757230-1

The Conference Board’s CEO Challenge^® 2015 reports that human capital remains top of mind for CEOs. CEOs explain that organizational growth is possible when human capital programs are in place.¹ Yet, the human resource (HR) function as a whole continues to fail in demonstrating results that resonate with stakeholders. This is evident in the July/August 2015 issue of Harvard Business Review cover story: “It’s Time to Blow up HR and Build Something New.” This is not the first time HR has come under fire. Consider Fast Company’s August 2005 cover story: “Why We Hate HR.”² As the author describes, after years of rhetoric about becoming “strategic partners,” HR professionals aren’t there. The overarching issue is failure to align HR programs, policies, and practices with what matters to the organization: operational excellence, customer relationships, innovation, and sustainability.³

Human resource development (HRD) is a significant target in this HR debate. With an estimated $164 billion spent on training and development in the U.S. alone, it is no wonder.⁴ Top leaders of organizations agree to invest in programs; yet programs fail to yield results that resonate with these leaders. This positions the HRD function not as a true business partner, but rather a nice-to-have, but easily dispensable function. To become a true business partner, HRD must meet five fundamental criteria:

1. Human resource development must be integrated into the overall strategic and operational framework of the organization. It cannot be an isolated, event-based activity, unrelated to the mainstream functions of the business.
2. HRD leaders and professionals must know and understand operations. Classically, the HRD function has been viewed as being out of touch, and not understanding of the changes and opportunities in the organization. The HRD team must fully understand the organization’s core business and the various intricacies of relationships, issues, and concerns. This knowledge will position the HRD function in such a way that it can develop and offer programs that are aligned with business objectives.
3. The HRD function must perform in an outstanding way, and this is a fundamental issue. HRD executives and managers must be seen as capable, dependable, and good communicators. Their performance must be stellar, focusing on results relevant to operations.
4. HRD must establish partnerships with key operating managers. The HRD staff must be proactive in building these relationships, even when managers are reluctant to partner. These key clients are crucial to the overall success and well-being of the HRD function.
5. A comprehensive measurement and evaluation process to capture the contribution of human resource development must exist. The process must be comprehensive yet practical and feasible as a routine function in the organization. It must be utilized to its fullest, providing important data to all stakeholders. The data must communicate the value human resource development delivers to the organization, using conservative standards, systematic steps, and a consistent process.

Most HRD executives believe that human resource development is an important part of strategy, and it has become a mainstream function in many organizations. Professional designations such as the Association for Talent Development (ATD) Certified Professional in Learning and Performance (CPLP), and the International Society for Performance Improvement’s (ISPI) Certified Performance Technologist (CPT) have positioned HRD, learning, and performance improvement as solid professions worth pursuing. The same executives give good marks to the successful establishment of partnerships with key management. In fact, over the past few decades, tremendous strides have been made in working with managers to build the relationships that are necessary. However, these executives also indicate that progress in measurement and evaluation in most organizations has not kept up with the need for the data it can produce. While evaluation of HRD programs is now routine, and ROI Institute’s Certified ROI Professional (CRP) designation has elevated the practice to a sought-after profession, Chief Learning Officer Business Intelligence Board’s 2015 Measurement and Metrics survey reports that 58 percent of those responding are dissatisfied with the extent of training measurement that occurs within their organization.⁵ In addition to HRD leadership cries for improvement in measurement and evaluation, senior executives routinely ask for more information about their investments in people. ROI Institute and ATD report in their 2010 survey of Fortune 500 CEOs that a gap exists between the type of data senior executives want and the type of data HRD provides.⁶ Respondents to the ATD/i4cp 2013 Value of Learning survey admit there is opportunity to improve what and how they measure learning effectiveness, with only 18 percent of respondents reporting they do a good job.⁷ So, in spite of the strides being made to position HRD as a business partner, a critical thread that ties together the relationship between the business and HRD process is broken—that thread is measurement and evaluation.

This book describes how to develop a comprehensive, feasible measurement and evaluation process that helps ensure HRD can become a true business partner.

Global Evaluation Trends

Measurement and evaluation have been changing and evolving—in private and public sectors, as well as nongovernmental organizations. This change is occurring across organizations and cultures worldwide. The following are a few of the most obvious trends.

1. Organizations are increasing their investments in measurement and evaluation, with best practice groups spending 3 to 5 percent of the HRD budget on measurement and evaluation.
2. Organizations are moving up the value chain, away from measuring reaction and learning to measuring application, impact, and ROI (Return on Investment).
3. Needs of the clients and sponsors of training and development projects, programs, initiatives, and solutions are the primary drivers for the increasing focus on measurement and evaluation.
4. Evaluation is an integral part of the design, development, delivery, and implementation of programs.
5. A shift from a reactive approach to a proactive approach is occurring as organizations address evaluation early in the cycle.
6. Measurement and evaluation processes are systematic and methodical, often a seamless part of the delivery process.
7. Technology is significantly enhancing the measurement and evaluation process, enabling the data collection, processing, analysis, and integration of large amounts of data.
8. Evaluation planning is becoming a critical part of the measurement and evaluation cycle.
9. The implementation of comprehensive measurement and evaluation processes usually leads to emphasis on initial needs analyses.
10. Organizations with comprehensive measurement and evaluation systems in place have seen increases in their program budgets.
11. Organizations without comprehensive measurement and evaluation systems see the reduction or elimination of their program budgets.
12. The use of ROI is emerging as an essential part of many measurement and evaluation systems. It is a fast-growing metric—70 to 80 percent of organizations have it on their wish lists.⁸
13. Many successful examples of comprehensive measurement and evaluation applications are available in all types of organizations and cultures.

These trends are creating a never-ending appetite for more information, resources, knowledge, and skills in the measurement and evaluation process.

Evaluation Myths

Sixteen assumptions keep measurement and evaluation at bay for the HRD community. Unfortunately, many well-intentioned leaders and evaluation practitioners perpetuate these beliefs, and often cause further delay for those who need a strong measurement practice the most. By discerning myth from reality, HRD executives and practitioners can move forward more easily with their measurement practice than when they take the myth-makers at their word. Table 1.1 presents the assumptions. Some are perceived as myths; others are reality. Before reading further,...

Table of contents

• Cover Page
• Half Title Page
• Title Page
• Copyright Page
• Contents
• List of figures
• List of tables
• List of boxes
• About the Authors
• Preface
• Acknowledgements
• PART I The Measurement Imperative
• PART II Evaluation Design Issues
• PART III Data Collection and Analysis
• PART IV Implementations Issues
• Appendices
• Index