How to Measure Investigation Process Effectiveness
By Kevin McManus, Chief Excellence Officer, Great Systems
Recently, I received a great e-mail question from one of my TapRooT® root cause analysis course participants. His question was basic, but meaningful. “What are your thoughts on how to measure investigation process effectiveness?” he asked. He went on to explain that he wanted my insights partially because he knew I was a measurement freak. More importantly, he was being pressured by his boss to focus on investigation timeliness over everything else.
How do You Measure Investigation Process Effectiveness?
Investigating problems, like most work activities, is a process. If you have read my Vital Signs measurement book, sat in on one of my measurement workshops, or simply done a lot of process measurement yourself, you should know that processes produce a variety of results with each cycle, and not just one or two. At a minimum, all processes have vital signs that relate to time, cost, quality, morale / teamwork, and safety dimensions.
All too often however, we focus on the time measure too much. In doing so, we often compromise other results like quality, morale / teamwork, and safety. For example, I frequently see people do poor root cause analysis and write weak corrective actions in order to make sure their problem solving work is completed on time.
If investigation timeliness is the primary measure we focus on, we may not even realize that we are not finding the true root cause of a problem. Worse yet, we may not notice that our fixes are failing too often. Ironically, timeliness is a quality measure. We are trying to meet one of our customer’s key requirements. Unfortunately, we often compromise our ability to meet other requirements by focusing too much on time.
Key Measures of Investigation Process Effectiveness
There are many different ways to measure investigation process effectiveness, but I will just focus on the basics here. If you want to talk about this topic in more detail, feel free to connect with me and we can dig into your particular challenges in more detail. With no further ado, here are some key investigation process measures most people should be tracking, but often don’t track.
How Much Investigation Process Time Do You Invest?
I like to track two time dimensions when I look at investigation process performance. First of all, what is our average process cycle time in days? Second, how many people hours did we invest in achieving this ‘number of days’ cycle time? Sadly, this basic measure is rarely tracked, let alone optimized, for each investigation that is performed.
We often make a mistake when we use groups that are too large to try to solve problems. However, you can involve many people in evidence collection. We lose process efficiency and effectiveness however when we try to put them all in the same room at the same time. Additionally, we often lack good group facilitators or fail to collect good evidence. This only magnifies the potential for lost time and money when a group meets.
What is the Cost of Your Investigation Process?
I firmly believe that if managers knew the real costs of their process challenges, they would make very different decisions. Do you track the costs of your investigation process? At a minimum, we should know the costs of the people hours we invest as we try to solve a problem. Ideally, we should also include at least some of the cost of process errors (waste) in our cost totals as well.
You don’t have to have an exact number here. If you multiply a weighted average wage rate by the total people hours you invest, you will have a good enough reference to use. Cost however is not a value to always minimize! Instead, we should try to optimize the hours and dollars we invest trying to solve a problem. We do this by looking for correlations between our time and cost measures and other key investigation process measures over time. Value is the goal. What quality of fixes do we get for the time we invest?
Investigation Process Safety
This should be one of the easier measures to obtain. If you have an injury as a result of the investigation process itself, shame on you. This can, and does, happen however. Additionally, one could argue that we should capture similar past injuries as part of the investigation.
At a minimum, we should track whether or not we experience ANY type of injury, no matter how minor, as a result of our investigative efforts. We should keep in mind that if we fail to secure a scene, release a scene too quickly, or continue to work when hazards exist, we will raise the potential for this to happen.
Investigation Process Morale and Teamwork
How often do we damage our company culture by investigating problems in an accusatory manner? How often do we damage our credibility as leaders by blaming people instead of systems? Morale and teamwork are not that hard to measure, but we often fail to even give these factors consideration when we are looking at investigation process effectiveness.
Surveys serve as the primary tool for measuring an investigation process from a morale and teamwork perspective. A post-investigation survey can be used to gauge team and customer satisfaction with the process that was used, for example. You can also include a couple of questions related to investigation process involvement and effectiveness in your regular workforce climate survey.
Investigation Process Quality
The key question to ask here is “What indicators serve as proof that our fixes worked?” Many people feel that if the incident or problem being investigated does not occur again, then the fixes must have worked. There is nothing wrong with using such an indicator, but I feel it needs to be supplemented with more error-sensitive leading indicators.
Behavior-based observation rates provide a better indication of fix effectiveness. Waiting for another bad thing to happen is not the best way to know if your fixes work or not. Instead, we should be looking for trends that show if the errors, safeguard gaps, and equipment failures that led to the incident are diminishing. Similarly, we should measure if defined root causes are occurring less frequently.
Do You Need to Improve Your Investigation Process?
Other investigation process measures can be tracked. Trending the measures presented above over time will give you some great initial indicators of investigation process effectiveness. Problem complexity and other factors will introduce some variation into this data, but you can always segment your results to help account for such effects. We really don’t know if, and where, improvement is needed until we begin tracking these basic measures.
Please email me your questions at firstname.lastname@example.org