Measure and Improve Investigation Process Effectiveness
How Do You Measure and Improve Your Incident Investigation Process?
Investigating problems, like most work activities, is a process. Have you read my Vital Signs measurement book, sat in on one of my measurement workshops, or simply done a lot of process measurement yourself?
If so, you know that processes produce a variety of results with each cycle, and not just one or two. At a minimum, all processes have vital signs that relate to time, cost, quality, morale / teamwork, and safety dimensions.
All too often however, we place too much focus on the time measure. In doing so, we often compromise other results like quality, morale / teamwork, and safety. For example, I frequently see people do poor root cause analysis and write weak corrective actions in order to complete their problem-solving work on time.
Is investigation timeliness the primary measure you focus on? If so, you might not find the true root causes of your problems. Worse yet, you might fail to notice that your fixes fail all too often.
Ironically, timeliness is a quality measure. We want to meet one of our customer’s key requirements. Unfortunately, we often compromise our ability to meet other requirements when we focus too much on time.
There are many ways to measure investigation process effectiveness. However, I want to focus on the basics here. If you want to talk about this topic in more detail, feel free to connect with me. Then, we can dig into your challenges in more detail. With no further ado, here are some key investigation process measures most people should track, but often don’t track.
If so, you know that processes produce a variety of results with each cycle, and not just one or two. At a minimum, all processes have vital signs that relate to time, cost, quality, morale / teamwork, and safety dimensions.
All too often however, we place too much focus on the time measure. In doing so, we often compromise other results like quality, morale / teamwork, and safety. For example, I frequently see people do poor root cause analysis and write weak corrective actions in order to complete their problem-solving work on time.
Is investigation timeliness the primary measure you focus on? If so, you might not find the true root causes of your problems. Worse yet, you might fail to notice that your fixes fail all too often.
Ironically, timeliness is a quality measure. We want to meet one of our customer’s key requirements. Unfortunately, we often compromise our ability to meet other requirements when we focus too much on time.
There are many ways to measure investigation process effectiveness. However, I want to focus on the basics here. If you want to talk about this topic in more detail, feel free to connect with me. Then, we can dig into your challenges in more detail. With no further ado, here are some key investigation process measures most people should track, but often don’t track.
How Much Investigation Process Time Do You Invest?
I like to track two time-based measures when I look at investigation process performance. First, what is our average investigation process cycle time in days? This often the measure leaders focus on the most. How long did it take to complete the investigation?
Second, how many people hours did we invest to achieve this ‘number of days’ cycle time? Sadly, few problem-solving teams track this basic measure. In turn, it is even more difficult to optimize future investigation process cycle times.
We often make a mistake when we use groups that are too large to try to solve problems. However, you can involve many people in evidence collection. Unfortunately, we lose process efficiency and effectiveness when we try to put them all in the same room at the same time.
Additionally, we often lack good group facilitators or fail to collect good evidence. This only magnifies the potential for lost time and money when a group meets.
Second, how many people hours did we invest to achieve this ‘number of days’ cycle time? Sadly, few problem-solving teams track this basic measure. In turn, it is even more difficult to optimize future investigation process cycle times.
We often make a mistake when we use groups that are too large to try to solve problems. However, you can involve many people in evidence collection. Unfortunately, we lose process efficiency and effectiveness when we try to put them all in the same room at the same time.
Additionally, we often lack good group facilitators or fail to collect good evidence. This only magnifies the potential for lost time and money when a group meets.
Effective investigations explore how failed or missing safeguards increase the likelihood of human performance problems or equipment failure. Not all safeguards are created equal however. How do you measure workplace safeguard effectiveness?
What is the Cost of Your Incident Investigation Process?
I firmly believe that if managers knew the real costs of their process challenges, they would make very different decisions. Do you track the costs of your investigation process?
At a minimum, we should know the costs of the people hours we invest as we try to solve a problem. Ideally, we should also include at least some of the cost of process errors (waste) in our cost totals as well.
You don’t have to have an exact number here. For example, you can multiply a weighted average wage rate by the total people hours you invest to give you a good reference. However, cost is not a value to always minimize! Instead, we should try to optimize the hours and dollars we invest trying to solve a problem.
To do this, look for correlations between our time and cost measures and other key investigation process measures over time. Value is the goal. What quality of fixes do we get for the time we invest?
At a minimum, we should know the costs of the people hours we invest as we try to solve a problem. Ideally, we should also include at least some of the cost of process errors (waste) in our cost totals as well.
You don’t have to have an exact number here. For example, you can multiply a weighted average wage rate by the total people hours you invest to give you a good reference. However, cost is not a value to always minimize! Instead, we should try to optimize the hours and dollars we invest trying to solve a problem.
To do this, look for correlations between our time and cost measures and other key investigation process measures over time. Value is the goal. What quality of fixes do we get for the time we invest?
How Safe is Your Incident Investigation Process?
This should be one of the easier measures to obtain. If you have an injury because of the investigation process itself, shame on you. However, this can, and does, happen. Additionally, one could argue that we should capture similar past injuries as part of the investigation.
At a minimum, we should track whether we experience ANY type of injury, no matter how minor, because of our investigative efforts. We should keep in mind that if we fail to secure a scene, release a scene too quickly, or continue to work when hazards exist, we will raise the potential for this to happen.
At a minimum, we should track whether we experience ANY type of injury, no matter how minor, because of our investigative efforts. We should keep in mind that if we fail to secure a scene, release a scene too quickly, or continue to work when hazards exist, we will raise the potential for this to happen.
How Does Your Incident Investigation Process Impact Morale and Teamwork?
How often do we damage our company culture when leaders investigate problems in an accusatory manner? How often do we blame people instead of systems and damage our leadership credibility?
A 'blame' work culture makes it difficult to obtain good data and evidence. Such a culture makes it even more difficult to put effective fixes in place.
Morale and teamwork are not that hard to measure. Unfortunately, we often fail to even give these factors consideration when we look at investigation process effectiveness.
A problem-solving team might work very hard to collect a well-rounded set of evidence, only to get the business when they take too long to do so.
Surveys serve as the primary tool for measuring an investigation process from a morale and teamwork perspective. For example, you can use a post-investigation survey to gauge team and customer satisfaction with the investigation process itself. These surveys do not have to be long - 4-5 questions are typically adequate.
Plus, you can include a couple of questions with an investigation process involvement and effectiveness focus in your regular workforce climate survey. When you combine these big picture results with investigation-specific results, you get a better picture of true process outcomes.
A 'blame' work culture makes it difficult to obtain good data and evidence. Such a culture makes it even more difficult to put effective fixes in place.
Morale and teamwork are not that hard to measure. Unfortunately, we often fail to even give these factors consideration when we look at investigation process effectiveness.
A problem-solving team might work very hard to collect a well-rounded set of evidence, only to get the business when they take too long to do so.
Surveys serve as the primary tool for measuring an investigation process from a morale and teamwork perspective. For example, you can use a post-investigation survey to gauge team and customer satisfaction with the investigation process itself. These surveys do not have to be long - 4-5 questions are typically adequate.
Plus, you can include a couple of questions with an investigation process involvement and effectiveness focus in your regular workforce climate survey. When you combine these big picture results with investigation-specific results, you get a better picture of true process outcomes.
How Do You Measure Incident Investigation Process Quality?
The key question to ask here is “What indicators serve as proof that our fixes worked?” Many people feel that if the incident or problem being investigated does not occur again, then the fixes were effective.
There is nothing wrong with the use of such an indicator. However, consider the use of more error-sensitive leading indicators on their investigation process effectiveness scorecard.
Behavior-based observation rates provide a better indication of fix effectiveness. Waiting for another bad thing to happen is not the best way to know if your fixes work or not.
Instead, we should search for trends that show reductions in the errors, safeguard gaps, and equipment failures that led to the incident. Similarly, we should measure if the root causes we found now occur less frequently.
There is nothing wrong with the use of such an indicator. However, consider the use of more error-sensitive leading indicators on their investigation process effectiveness scorecard.
Behavior-based observation rates provide a better indication of fix effectiveness. Waiting for another bad thing to happen is not the best way to know if your fixes work or not.
Instead, we should search for trends that show reductions in the errors, safeguard gaps, and equipment failures that led to the incident. Similarly, we should measure if the root causes we found now occur less frequently.
Six Key Incident Investigation Process Measures
Investigation Process Cycle Time: This metric looks at the time required to complete the investigation. The clock starts when the incident is reported, or the problem is defined.
Three options exist to gauge when the process cycle ends. Some end the process when a final report is issued. A better option is to stop the cycle when corrective actions have been put in place. Some people prefer to wait until preventive actions have been installed to end the investigation cycle.
Total people hours invested: Once the process cycle start and end points have been defined, one can capture the number of people hours spent within the cycle, A better practice is to track the number of people hours spent to complete each process step.
# of staff engaged in process completion: It is important to look at how you invest your people hours. In general, when you engage fewer people in the process, it takes longer in terms of calendar time to complete the investigation process.
Total investigation labor cost: If you capture the number of people-hours you invest, you can use a weighted average multiplier to convert hours into labor cost. Typically, there are privacy issues related to the use of person-specific wage rates.
Customer satisfaction with investigation results: An end-of-process survey provides this measure in most cases. They key lies in including all affected stakeholders in the survey process.
Customer satisfaction with investigation analysis: Similarly, you should survey key customers as the effectiveness of the investigation analysis. For this to happen, key analysis expectations / practices must first be defined. The investigation team itself should also complete this survey (and possibly the ‘results’ version).
Three options exist to gauge when the process cycle ends. Some end the process when a final report is issued. A better option is to stop the cycle when corrective actions have been put in place. Some people prefer to wait until preventive actions have been installed to end the investigation cycle.
Total people hours invested: Once the process cycle start and end points have been defined, one can capture the number of people hours spent within the cycle, A better practice is to track the number of people hours spent to complete each process step.
# of staff engaged in process completion: It is important to look at how you invest your people hours. In general, when you engage fewer people in the process, it takes longer in terms of calendar time to complete the investigation process.
Total investigation labor cost: If you capture the number of people-hours you invest, you can use a weighted average multiplier to convert hours into labor cost. Typically, there are privacy issues related to the use of person-specific wage rates.
Customer satisfaction with investigation results: An end-of-process survey provides this measure in most cases. They key lies in including all affected stakeholders in the survey process.
Customer satisfaction with investigation analysis: Similarly, you should survey key customers as the effectiveness of the investigation analysis. For this to happen, key analysis expectations / practices must first be defined. The investigation team itself should also complete this survey (and possibly the ‘results’ version).
Five More Incident Investigation Process Measures
Investigation Process impact on morale and team cohesion: This measure can also come from an end-of-cycle survey. In a more general sense, the process should be assessed annually as part of a work site’s annual work culture assessment.
Potential injuries or significant near misses during the process: Ideally, this value is always ‘zero.’ However, I have seen cases where people were injured or almost hurt during the investigation itself. The same can be said for potential or actual equipment damage that may occur or could have occurred.
% root cause elimination: Effective corrective and preventive actions minimize, if not eliminate the one or more root causes they were designed to target. This measure summarizes the net effect of the changes you make on the set of root causes that were defined.
% reduction in ‘high risk’ error rates: This metric helps determine the effectiveness of the corrective and preventive fixes that are put in place. Keep in mind that the errors themselves must be defined as part of the investigation process to gauge the degree to which their occurrence and/or severity rates declined.
Increase (improvement) in mean time between equipment failures / error occurrence: This is an additional measure of corrective and/or preventive action effectiveness. An effective investigation process overall should reduce the rate at which a given problem occurs.
Additional measures exist for use, but these examples are most common.
Potential injuries or significant near misses during the process: Ideally, this value is always ‘zero.’ However, I have seen cases where people were injured or almost hurt during the investigation itself. The same can be said for potential or actual equipment damage that may occur or could have occurred.
% root cause elimination: Effective corrective and preventive actions minimize, if not eliminate the one or more root causes they were designed to target. This measure summarizes the net effect of the changes you make on the set of root causes that were defined.
% reduction in ‘high risk’ error rates: This metric helps determine the effectiveness of the corrective and preventive fixes that are put in place. Keep in mind that the errors themselves must be defined as part of the investigation process to gauge the degree to which their occurrence and/or severity rates declined.
Increase (improvement) in mean time between equipment failures / error occurrence: This is an additional measure of corrective and/or preventive action effectiveness. An effective investigation process overall should reduce the rate at which a given problem occurs.
Additional measures exist for use, but these examples are most common.