Top Ten Workplace Training Effectiveness Measures
Often, even the best organizations struggle to effectively measure workplace training. For example, multiple Baldrige recipients in recent years continue to rely on course participation and completion rates as primary training effectiveness indicators. From an overall results and work systems design perspective, these companies are high performers. Why don’t they measure training differently?
Some do. They simply do not want to use their limited application space to show results they can more effectively, and completely, share on site. However, only a small percentage of workplaces can use such an excuse. Most don’t measure training effectiveness at all. Some struggle to provide evidence of any formal training.
Organizations may measure training cost, training hours, and completion rates, but rarely do they combine these metrics for the purpose of work systems effectiveness review. In this post, I share a progressive list of possible training effectiveness metrics. Keep in mind those at the top of the list may seem familiar, but they are also the weakest gauges.
LISTEN to my ‘How to Measure Training Effectiveness’ Real Life Work podcast NOW!
Measure #1: Total Training Cost and Time
Because we budget training time as an expense, most organizations can show you how much time and money they invest annually in training. Plus, they may divide this total by the total number of staff to give you a training cost or training time per staff member ratio.
Measure #2: Participation Rates
Participation rates look at how many people attend the training versus who was supposed to be there. For this metric to work, one must have a list of courses each job position requires. Ideally, a competency matrix dataset exists to define course requirements for each job role.
CHECK OUT my 37-video ‘Facilitating and Leading Teams’ workshop playlist on my Great Systems YouTube channel.
Measure #3: Completion Rates
Completion rates are a step above participation rates in that they show who finished a course. Yes, there are cases where many more people attempt certain courses versus complete them. Additionally, good course designs require course work outside of the classroom. In turn, such courses are not complete until the work outside of the classroom is done.
Measure #4: Smile Sheets
Smile sheets are another term for end of course surveys. Donald Kirkpatrick refers to this ‘reaction’ metric as a Level One training effectiveness evaluator. As there are four levels in Kirkpatrick’s levels of training effectiveness, this means end-of-course surveys are something, but not good enough.
Also, it is common to include one or more statements with a training focus on the periodic culture survey most companies conduct. One common example is ‘I receive the training I need to do my job effectively.” It is also common to inquire about the general effectiveness of training people attend.
With the time that passes since the training, it is likely these ‘learning’ indicators are superior to smile sheet results.
Measure #5: End-of-Course Tests
This is the classic formal education measure. Did you pass the test? To evaluate training, we can give such tests at the end of an entire course or course segment. The effectiveness of this measure largely depends on test design. For example, a ‘hands on’ skill demonstration test is a more effective indicator of skill acquisition than a simple True – False test.
Measure #6: Post-Training Position Observation Checklist (POC) Scores
Now, we move into the above average training effectiveness metrics. The Position Observation Checklist (POC) is a best practice I learned from national Malcolm Baldrige Award recipient Pal’s Sudden Service many years ago. Since then, it has served both me and my customers well at multiple locations.
Both the trainee and their supervisor use the Position Observation Checklist (POC) to assess skills on the job post-training. A typical checklist has five areas of performance, with an average of five skills per area. Each person evaluates skill application on a 100 point scale for each area.
This tool serves as a Level Three ‘behavior’ training effectiveness indicator. CLICK HERE to download an example Incident Investigator Position Observation Checklist!
Measure #7: 90-day Post-Training POC Score
This ‘behavior’ measure is more effective than the previous one simply because it demonstrates longer skill retention and effective application. Typically, both checklist assessments are used. Comparison made between the two assessments help show which skills people tend to retain better than others over time.
At this point in training work system design and measurement maturity, it is likely that a certificate process exists to help integrate different skill sets, needs, and training results. Both end-of-course and 90-day assessments of a skill demonstration nature are common with such certifications.
RELATED CONTENT: The AURA of Effective Training
Measure #8: Reduction in Process Errors and Waste
The final three types of training effectiveness measures reflect correlation between the training and work process performance. In a proactive process improvement system, each process owner captures key errors and failures daily at the process level. In turn, they can connect daily process results with the staff who support that process.
Begin with a simple ratio that looks at average POC scores (post-course and 90-day) versus daily process error and failure rates. For the best results, track errors at the process non-conformance level. Process-specific scrap and rework reduction rates also work. Avoid the use of lagging, aggregate measures (such as the site safety recordable rate).
For equipment failures, look for downtime duration and frequency relationships, in addition to the non-conformance that results in process downtime. Avoid the use of downtime percentages in favor of downtime minutes and frequencies.
Measure #9: Improved Workforce Engagement Scores and Retention Rates
An additional level of results correlation exists when effective training satisfies key workforce engagement factors. Such satisfaction helps reduce turnover and increase staff retention. Plus, the reduction of daily errors and failures in the workplace creates a more positive attitude towards work in general.
If you are fortunate enough to have datasets that allow it, look for a variety of correlations between the different inputs and outcomes I present. The design of your annual work culture survey should help support training effectiveness data capture.
Plus, try to find ways to practice lean data capture for your training completion information. For example, QR code scans – one from the trainee’s security badge and one code from the post-course test – provide a quick way to update a training database.
POPULAR CONTENT: How to Increase Workforce Engagement
Measure #10: A Training Work System Effectiveness Index
The ultimate gauge of training system effectiveness is an index that constitutes multiple effectiveness measures. Plus, the use of such an index allows results ‘credit’ to be given for foundational course completion efforts. A typical index would include metrics from essentially each option in this list.
As with all process measurement, it begins with effective dataset construction. What training data do you need to capture for each course someone attends? What type of training profile should you build for each staff member?
How will you set up a similar course effectiveness profile for each course? How do you want to progressively improve your approach to training effectiveness measurement over time?
Would You Like Help to Improve Your Training Work System?
Effective training is a passion of mine. I try to improve both training content and delivery with every course I teach. When I serve in a training capacity, my ultimate goal is sustained, and effective, skill application on the job. What are your goals as a trainer?
Please take a look at my ‘Teach Your Teams’ performance improvement workbooks. I created each one over time as my own work demands drove the need. Most of the places I worked did not have huge (or much of any) training budgets. In turn, I did my own research and created my own ‘real life work’ training products. Currently, you can find full video and audio content for three of these workbooks on my Great Systems YouTube channel.
Please Connect with Kevin
If you would like more information on effective training measurement, please email me at firstname.lastname@example.org. Better yet, connect with me on LinkedIn to get my freshest content as soon as it is released.
Kevin McManus, Chief Excellence Officer, Great Systems LLC
LIKE Great Systems on Facebook
CONNECT with me on LinkedIn
CHECK OUT my Amazon.com Author Page
FOLLOW me on Twitter: @greatsystems
© 2023, Great Systems LLC, All Rights Reserved