Top Five Tips for Effective Evidence Collection
By Kevin McManus, Chief Excellence Officer, Great Systems
Several years ago, I contributed a chapter to a John Wiley textbook on loss prevention engineering methods. My chapter focused on conducting effective incident investigations, which includes evidence collection. I also find that a lot of my TapRooT® root cause analysis course students often struggle with effectively collecting usable and necessary evidence. In recent years, I have continued to learn about what it takes to collect quality – necessary and usable – evidence in less time.
In this post, I want to share what I feel to be the top five tips for effectively collecting evidence for problem solving, including incident investigations. I have found that if you don’t collect the right evidence, the rest of your problem solving process will be compromised to some degree. How effective are your evidence collection efforts?
How Effective are Your Evidence Collection Efforts?
Very few organizations measure the effectiveness of their investigation processes at all, let alone the crucial evidence collection step. When traditional root cause analysis tools are used, such as the Five Whys, fishbone analysis, and/ or fault tree analysis, the failure of countermeasures to validate a given root cause, or causes, represents a process failure. With the TapRooT® root causes analysis process, a lack of evidence results in missed opportunities to improve.
If you don’t have the evidence, you can’t justify selecting a given root cause. Do you gain the evidence through trial and error, testing countermeasure after countermeasure, or do you improve your evidence collection efforts up front, making the weak or missing safeguards much easier to spot? How do your investigators know the best questions to ask? How effective are your investigators at spotting the human factors concerns that affect the likelihood of human error each day?
Though quantitative evidence does not exist to prove it, I think it is safe to say that the quality of the evidence collection process drives the effectiveness of the root cause analysis process, and ultimately, the selected corrective actions. If nothing else, effective evidence collection reduces the process cycle time required for root cause analysis significantly, while typically also yielding better results.
Tip #1 – Detect the causal factors to be analyzed as early as possible
In the TapRooT® root causes analysis process, causal factors represent the equipment difficulties (failures) or human performance problems (errors) that require root cause analysis. With traditional root cause analysis approaches, causal factors would be similar to the process failures (non-conformances) that occurred and produced the defect. Finding causal factors as early as possible in the problem solving process helps provide focus to the evidence collection effort.
Too much time is spent collecting evidence that is interesting, but not really that necessary or useful, in too many investigative efforts. Valuable interview time, for example, is often lost exploring the interesting, but largely irrelevant aspects of the problem’s story, even though time limitations exist. Having at least 2-3 preliminary causal factors defined early on helps focus the evidence collection effort.
Keep in mind that defining causal factors is an iterative process. Expect to re-word those factors you initially define, add a new one or two as your problem exploration reveals them, and possibly even drop one or two.
Tip #2 – Utilize a pit crew approach to evidence collection
Imagine if only one person was allowed into the pit in motor sports to change tires, make suspension and engine adjustments, refuel the car, and clean the windshield. How long would it take to get in and out of the pit during a race? This example sounds silly, but all too often we expect one person to collect most, if not all, of the evidence for a given problem solving effort. Delays in evidence collection naturally lead to enhanced evidence erosion.
I learned years ago, when working with Quality Circles, that engaging multiple team members throughout the problem solving process produces better results. Involving more people in your evidence collection efforts does not mean everyone has to learn how to effectively interview others. Spreading out the evidence collection workload does help collect more usable and necessary evidence in less time.
Certain team members can help tremendously simply by quickly collecting the right documents that are related to the problem and organizing them in a usable form for the investigation team. Having a team member who can quickly take great pictures at the scene for later analysis often represents a more effective use of time versus trying to ensure interviews are conducted immediately.
Both of these tasks require a different skill set. Utilizing the best skills of each team member by matching them to similar evidence collection activities can help collect more evidence in less time. An example ‘swim lanes flow chart’ for an investigation process can be found here. This flow chart format makes the roles that different people play in the effective execution of each process step clearly evident.
Tip #3 – Use punch lists and question scripts to trigger evidence collection actions
What are your ‘key question triggers’ that prompt your investigators to ask the right questions? One of the main reasons investigators fail to collect the right amount of ‘necessary and usable’ evidence is that their questions are driven largely by only the design of the investigation form and their own past experience. Think about it. How do your investigators know the ‘right’ questions to ask when investigating a problem? How do they know what to look for and what to expect?
Including a well-rounded set of ‘starter’ questions on the problem definition, or incident investigation, form is a great start. Additional options exist for helping to ensure key questions are asked. Job aids, such as lanyard checklist badges and pocket cards, are one affordable choice. Soon, ‘heads up displays’ on our safety eyewear will give us such prompts. The goal will remain the same though. Get better, and more, questions into the hands, and minds, of our problem investigators.
Tip #4 – Look upstream for initiating errors and stop / catch errors
At the start of the movie ‘Now You See Me’, there is a saying that sets the premise for this movie about magicians. It goes “the closer you look, the less you see.” One reason magicians are successful is that they distract you. They get you to look closely at things they want you to see and away from those things they don’t want you to see. Most investigators ‘get fooled’ into focusing primarily on the injured party or most visibly broken process.
As a key step of the formal problem solving process, one should verify the degree that the problem solving or investigation team ‘looked upstream and downstream’ for contributing factors. The goal of any investigative effort should be to find and fix all significant system failures or gaps – not just those that are closest to the most evident problem.
For example, what types of errors might have let the problem exist, or grow too large, in the first place? Were there opportunities to catch the error that were missed, such as inspection steps or hold points? If time is going to be invested looking at a process problem, the entire process should be reviewed for improvement opportunities, as opposed to only focusing on certain steps or people.
Tip #5 – Formally measure evidence quality with each problem solving cycle
Too few organizations measure the effectiveness of their investigation process, let alone their evidence collection efforts. The resources invested in such efforts however are usually significant, as are the potential consequences of ineffective corrective and preventative actions. In turn, value is to be gained from formally measuring the ‘vital signs’ of both the investigation process as a whole, and perhaps more importantly, the evidence collection effort.
Specific to evidence collection, one should formally assess the degree that the desired breadth of evidence was collected. What percentage of our key questions were adequately answered? How often did we collect evidence from all three sources – people, paper, and place? How timely were each of our individual evidence collection efforts?
You can also score the quality of the evidence captured using a five-point Likert scale, or simple ‘A-B-C’ grade assignments. For example, a simple ‘Pass / fail’ assessment could be made on each package of evidence collected. To what degree were the basic evidence collection criteria met?
How Can You Improve Your Evidence Collection Efforts?
As a contract TapRooT® root cause analysis process trainer, I see hundreds of investigations each year. I am confident that taking advantage of the five evidence collection tips defined in this post will help you significantly improve your evidence collection process. If you have any questions, please do not hesitate to contact me at firstname.lastname@example.org.
Kevin McManus, Chief Excellence Officer, Great Systems
If you like this post, please check out the other posts on my Great Systems website or on LinkedIn. My books can be purchased in both e-book and hard copy form from Amazon.com.