There is a growing dilemma faced by those of us in the learning and development industry. With e-Learning playing an increasingly important role in the training of employees, what is the best way to assess the effectiveness of our efforts? Is calculating the ROI of e-Learning the best method, and if so is it even possible to do? Or is it better to measure e-Learning using an objective tool such as Kirkpatrick’s four-level training evaluation model?
This is a debate that has supporters on both sides, and valid arguments for each point. Let’s start by saying we can all likely agree that a well-trained workforce is essential to the success of an enterprise. However, training must be done within a reasonable cost structure, and must have the intended impact on the employees it’s meant to serve. This is the return on investment argument put forth by those in charge of budget decisions. At the same time, it’s very difficult to quantify things like the long-term benefits of e-Learning, not to mention the opportunity costs of not doing it, or of doing it poorly. This is the argument generally supported by e-Learning specialists.
So who’s right?
In 1959 Donald Kirkpatrick published his Four Level Evaluation Model in the Journal of the American Society of Training Directors. But more than half a century after it was first conceived, is Kirkpatrick’s model still relevant? And even more to the point, is it the best way to evaluate the effectiveness of e-Learning? At the time his model was considered primarily academic, and wasn’t truly popularized until the mid-nineties when his book, Evaluating Training Programs was published. Interestingly, this was also the beginning of wide-spread use of e-Learning within the corporate training environment.
Kirkpatrick’s four-level model for evaluating the learning process is both simple and comprehensive. It consists of a series of observed steps:
- Reaction – How did the participants react to the training?
- Learning – What skills and knowledge did they gain from the training?
- Behavior – In what ways was their behavior (job performance) changed by the training?
- Results – What tangible or measurable results were achieved by the training?
It’s this last step that holds the most promise for evaluating the effectiveness of e-Learning. Potential measurable outcomes include increased productivity, higher customer satisfaction ratings, more sales, improved employee retention, etc. However, it’s still difficult to quantify these tangible effects as a direct result of training. So now let’s take a look at how we might evaluate e-Learning from an ROI perspective.
In a basic sense ROI is a measure of business benefit that evaluates the performance of an investment based on dividing the results by the input cost. In the case of evaluating e-Learning effectiveness, the investment is the commitment to training the workforce, the input cost is what it takes to develop and deliver an e-Learning course, and the results are how well-trained the workforce is after participating in the training. But as you might see, there are a few problems with this calculation.
The first problem is in figuring the results. The benefits of e-Learning once it is developed and delivered are ongoing, making the measurement of its results an ever-increasing value. The second problem is in how to evaluate improvements in business performance and assign a percentage of these results to the delivery of e-Learning. And a third, but by no means final, problem with the use of ROI in evaluating e-Learning effectiveness is the opportunity cost. If no e-Learning is done or if it is done haphazardly, what impact will it have on the business?
Using ROI to measure a business investment is a purely financial calculation. Using it to measure the effectiveness of e-Learning introduces variables that can’t be readily quantified, and therefore leave out a substantial part of the benefit of e-Learning. The debate over this subject will surely continue, but let’s leave it with this final thought. While the expense of traditional training is measured one employee at a time, the cost of e-Learning is spread over the entire workforce.
Here at Designing Digitally, we’re experts at delivering e-Learning that is effective, affordable, and delivers the best value to your workforce. We know through experience the positive impact e-Learning can have on employee development when it’s done effectively. Trust us to deliver employee training that has a positive impact on your workforce now and in the future. Call us to request more information about how our e-Learning design team can help your organization.