Explore My Process Measurement Madness

Explore My Process Measurement Madness

By Kevin McManus, Kaizen Facilitator and Coach

In this podcast episode, I let you explore my process measurement madness. My hope is that this podcast inspires you to engage a little more with the practice of proactive, process-based daily measurement. I love measurement.

When I would teach the TapRooT® 5-day root cause analysis course, I would always chomp at the bit to teach the Trending module. Why do I love measurement so much? How has measurement helped me improve my own work processes as a small business owner?

What types of things do you measure at work?

Most, if not all, organizations measure something.  Unfortunately, a very small percentage of organizations have what I would consider to be a high-performance measurement system.  Some organizations measure too many things. Others focus primarily on those numbers that they must track in order to satisfy regulatory and legal requirements.

Some leaders tend to focus only on certain input and output numbers, such as those of a sales revenue, total customer volume, total expenses, and total units produced.  Very few organizations trend their performance information in an effort to better understand their work systems.

My 40-plus years of experience have led me to conclude this. Less than 10% of the organizations out there have high performance measurement systems.  My experiences as a national Baldrige Examiner, Judge, and leader in organizations that aspire to be high performers, further validate this belief. Even the best still struggle to capture, crunch, and improve daily, transaction-based process results.

More importantly, these experiences help me identify what I consider to be the Top Ten Measurement Work System Weaknesses. These weaknesses exist in a majority of the world’s businesses, hospitals, schools, governmental agencies, and service industries. You can find the list in the graphic on this page.

LISTEN to my ‘Measurement Madness’ post as a podcast!

How effective is your measurement work system?

If you would like to challenge my assumptions, feel free to do so. However, as you do, please think about two things.

First of all, how effective is your existing measurement system?  What criteria and factors do you consider as you answer this evaluative question?

Second, as you read over the list of measurement system weaknesses I provide, which ones sound familiar to you?  What negative experiences or waste do you witness in the workplace where one or more of these weaknesses might exist?

How much do you need to improve your measurement system? Where do you need to make improvements first?

If you want to make your measurement system more effective, start with the installation of balanced scorecards.  It is one of the first places I start when I help others make measurement system improvements.

The lack of a balanced look at our work systems drives many of our ineffective leadership behaviors and practices. When you use a balanced scorecard, it helps you focus on the right things.

It helps you understand your work systems to a much higher degree.  How do you know if you make improvements in the right places?

In short, I can sum up the main measurement system design and use mistakes made by a high percentage of organizations into three main areas:

#1: We use the wrong types of measures

Simply put we rely too heavily on lagging counts to gauge our performance. It is bad enough to depend on lagging indicators more than leading indicators (what Dr. Deming called ‘driving by using the rear-view mirror with the windshield papered over’). However, we make matters much worse when we use lagging counts as our primary type of measure.

Counts don’t provide a connection between the measure and the system it measures. No ratio equals no evident relationship.

Sadly, you can convert any count into a ratio if you use time or dollars as a divisor. Also, you can use percentage of total ratios.

The goal here is to have a high percentage of leading ratios in your measure mix. The tendency is to focus on measuring things that are relatively easier to measure or count. Examples include work hours, headcount, costs, and units produced or served.

Also, we tend to focus on higher level, aggregate measures, instead of process-based measures. This practice introduces variation into the chart that does not actually exist.

Unfortunately, such a practice can actually hide problem areas over time. Even worse, leaders accept the higher level of variation as the norm and waste levels rise.

#2: We measure the wrong things

The prevailing norm is to measure at the company, business unit, or site level. Instead, we should rely mainly on process-based measures to drive our improvement efforts. Measures that are not process-based lose their relative value quite quickly.

Why? Such measures represent only an aggregation of multiple processes or work systems.

When we group measures from different processes together, we can either introduce extra variation or washout (hide) excessive variation that might exist in one system versus another. We might need fewer calculations to create these higher-level measures.

However, at the same time, decision quality suffers significantly. Bad decisions in turn, result in the use of limited time and financial resources to make relatively low value changes.

It is better to roll up into these measures, with the use of process-based measures as the foundation.

One major measurement error is the practice of comparing monthly counts – usually dollars – to each other as a performance review technique. Also, the absence of process-level data makes it tough to troubleshoot problems.

In turn, we usually fall back on debates as to why a given number goes up or down. We argue about why our results don’t meet expectations.

However, how often do our expectations exceed the capability of the process?

#3: We use our measures in the wrong way

Most people use measures to monitor other people – to hold them accountable in layman’s terms. Leaders should use measures to predict future process performance, to determine if improvements have the desired impact, and to identify high leverage opportunities for change.

We might think that our measurement practices track whether improvement occurs or not. However, when we use the wrong measures (lagging counts), and fail to measure at the process level, we only fool ourselves.

Most people measure for the wrong reasons. They use measures in an attempt to control, or at least keep track of, people. Of course, people figure this out. Plus, they soon learn how to dodge the system through data distortion, or system distortion, to keep the ‘pain’ to a minimum.

Process owners should use measures to monitor system performance (keep it between the lines). Measures can help us predict future system performance (what are the odds of goal achievement?). Step change shifts in our trends should indicate when innovations begin to gain traction. Finally, measures should help us make wiser resource investment decisions, as they give us feedback on the performance of past investments.

How sensitive is your work measurement system?

Because our measurement systems are less sensitive than they should be due to process aggregation, it would be tough to tell if a given fix was having the desired effect even if the data was available in trend form. Better measurement system designs can show us how well a fix works.

However, we often fail to track errors at the process level. We only track the big problems. That’s sad, because we waste a lot of limited time when we attempt to fix common, nagging problems with weak fixes that are unfortunately all too common as well (such as verbal reminders and longer procedures).

Best in class measurement systems measure from the inside out. They start at the process level and roll up to the department, site, and higher levels. In addition to measuring at too high of a level, we often make a mistake when we look at monthly trends for only a few measures. All too often, we rely on table-based or pie chart ‘snapshot’ comparisons as our diagnostic data source for process troubleshooting.

BUY my “Vital Signs – Power of Meaningful Measurement” workbook NOW at Amazon.com

How do you use your measures to predict future success?

Measuring is all about looking for fact-based patterns and relationships. Fact-based feedback affects human behavior, and in turn, performance. Also, without reliable and consistent fact-based feedback, how do we know where to invest our limited resources?

How will we really know if our investments are paying off? How will we know if our processes are performing as expected, let alone improving?

To most, measures and prediction go together usually on the weekends, in settings that often include beer and sports. Most organizations fail to use their measures as indicators of future performance.

The proof is evidenced by the limited use of control limits on charts. It is further supported by the even more limited use of the term ‘sigma’ in everyday conversation to help explain the weather.

Most importantly, process variation cannot be understood without trend lines and control limits. If we don’t understand process variation, we can’t predict future process performance with any accuracy.

Also, when we fail to explore process variation through trend-based questions, we might miss chances to improve measure sensitivity (better collection or calculation methods) or find and fix common cause process errors.

How the best use and improve their measures

As an Alumni Examiner and Judge for the Malcolm Baldrige National Performance Excellence Award, I have experienced the use of some simple, but great, measurement systems. Also, for almost fifty years, I have been directly involved with process measurement, process analysis, and process improvement approaches that worked to a large extent.

However, there have been learning moments.  Fifty locations later, I know that meaningful measurement is possible.

I admittedly made more mistakes than I would like to admit with my measurement system design experiments over the years. However, you don’t need to make those same mistakes. There are effective design practices that help you learn the right way.

My goal is to share the most common attributes of great measurement systems I have seen in this podcast. I want to include example approaches that can help make measurement systems actually work.

Remember: Effective measurement boils down to three things

Right measures – Do you use leading ratios as your primary type of measure? How often do you use percentages, time, and currency to essentially convert ANY count into a meaningful ratio?

Do you capture key counts daily at the transaction level? Do you turn this transaction data into ratios and trend them over time?

Right Place – How often do you capture key counts daily for each key process? Can you split your existing ‘mixed systems’ charts apart so that the process behaviors they reflect are more meaningful?

Where can you switch to more sensitive measures, such as daily process non-conformance or human error rates, to help better understand individual performance?

Right Use – How often do you use measurement to predict future performance instead of using numbers to look backwards into the past? How might you change the way you use data to recognize, reward, reprimand, and otherwise influence behavior in the workplace?

Systems give you what you design them to give you. This is definitely true for measurement systems.

However, leaders need simple, well-designed measurement systems if their goal is to sustain daily kaizen and operational excellence.