Measuring Investigation Process Effectiveness
By Kevin McManus, Chief Excellence Officer, Great Systems
Recently, I received a great e-mail question from one of my TapRooT® root cause analysis course participants. His question was basic, but meaningful. “What are your thoughts on measuring investigation process effectiveness?” he asked. He went on to explain that he wanted my insights partially because he knew I was a measurement freak. More importantly, he was being pressured by his boss to focus on investigation timeliness over everything else.
How do You Measure Investigation Process Effectiveness?
Investigating problems, like most work activities, is a process. If you have read my Vital Signs measurement book, sat in on one of my measurement workshops, or simply done a lot of process measurement yourself, you should know that processes produce a variety of results with each cycle, and not just one or two. At a minimum, all processes have vital signs that relate to time, cost, quality, morale / teamwork, and safety dimensions.
All too often however, we focus on the time measure too much. In doing so, we often compromise other results like quality, morale / teamwork, and safety. For example, I frequently see people do poor root cause analysis and write weak corrective actions in order to make sure their problem solving work is completed on time.
If investigation timeliness is the primary measure we focus on, we may not even realize that we are not finding the true root cause of a problem. Worse yet, we may not notice that our fixes are failing too often. Ironically, timeliness is a quality measure. We are trying to meet one of our customer’s key requirements. Unfortunately, we often compromise our ability to meet other requirements by focusing too much on time.
Key Measures of Investigation Process Effectiveness
There are many different ways to measure investigation process effectiveness, but I will just focus on the basics here. If you want to talk about this topic in more detail, feel free to connect with me and we can dig into your particular challenges in more detail. With no further ado, here are some key investigation process measures most people should be tracking, but often don’t track.
Investigation Process Time Invested
I like to track two time dimensions when I look at investigation process performance. First of all, what is our average process cycle time in days? Second, how many people hours did we invest in achieving this ‘number of days’ cycle time? Sadly, this basic measure is rarely tracked, let alone optimized, for each investigation that is performed.
We often make the mistake of using groups that are too large when trying to solve problems. Many people can be involved in collecting evidence. We lose process efficiency and effectiveness however when we try to put them all in the same room at the same time. Additionally, we often lack good group facilitators or fail to collect good evidence. This only magnifies the potential for lost time and money when a group meets.
Investigation Process Cost
I firmly believe that if managers knew the real costs of their process challenges, they would be making very different decisions. Do you track the costs of your investigation process? At a minimum, we should know the costs of the people hours we invested in trying to solve a problem. Ideally, we should also include at least some of the cost of process errors (waste) in our cost totals as well.
You don’t have to have an exact number here. Multiplying a weighted average wage rate by the total people hours invested will give you a good enough reference to use. Cost however is not a value to minimized! Instead, we should try to optimize the hours and dollars we invest trying to solve a problem. We do this by looking for correlations between our time and cost measures and other key investigation process measures over time. Value is the goal. What quality of fixes do we get for the time we invest?
Investigation Process Safety
This should be one of the easier measures to obtain. If you are having injuries occur as a result of the investigation process itself, shame on you. This can, and does, happen however. Additionally, one could argue that having similar injuries post-investigation because our fixes failed should be included.
At a minimum, we should track whether or not we experienced ANY type of injury, no matter how minor, as a result of our investigative efforts. We should keep in mind that failing to secure a scene, releasing a scene too quickly, or continuing to work when hazards exist, will raise the potential for this to happen.
Investigation Process Morale and Teamwork
How often do we damage our company culture by investigating problems in an accusatory manner? How often do we damage our credibility as leaders by blaming people instead of systems? Morale and teamwork are not that hard to measure, but we often fail to even give these factors consideration when we are looking at investigation process effectiveness.
Surveys serve as the primary tool for measuring an investigation process from a morale and teamwork perspective. A post-investigation survey can be used to gauge team and customer satisfaction with the process that was used, for example. You can also include a couple of questions related to investigation process involvement and effectiveness in your regular workforce climate survey.
Investigation Process Quality
The key question to ask here is “What indicators serve as proof that our fixes worked?” Many people feel that if the incident or problem being investigated does not occur again, then the fixes must have worked. There is nothing wrong with using such an indicator, but I feel it needs to be supplemented with more error-sensitive leading indicators.
Behavior-based observation rates provide a better indication of fix effectiveness. Waiting for another bad thing to happen is not the best way to know if your fixes worked or not. Instead, we should be looking for trends that show if the errors, safeguard gaps, and equipment failures that led to the incident are diminishing. Similarly, we should measure if defined root causes are occurring less frequently.
Do You Need to Improve Your Investigation Process?
Other investigation process measures can be tracked. Trending the measures presented above over time will give you some great initial indicators of investigation process effectiveness. Problem complexity and other factors will introduce some variation into this data, but you can always segment your results to help account for such effects. We really don’t know if, and where, improvement is needed until we begin tracking these basic measures.