Measuring Process Improvement Effectiveness

By Kevin McManus, Chief Excellence Officer, Great Systems

This post began as an Industrial and Systems Engineer magazine  article that I wrote a couple of years ago, just after I wrapped up another national Malcolm Baldrige Performance Excellence Program site visit. On that particular site visit, my focus was directed at the organization’s information analysis and sharing processes.

As I prepared questions for the visit in this area, some new questions popped into my own mind relative to measuring organizational progress in cultivating ideas, best practices, and innovations. Unfortunately, few organizations do little in this area beyond counting ideas captured and implemented over time.

For starters, what is the difference between an idea, a best practice, and an innovation? Does it make sense to track the frequencies at which we are generating, evaluating, and implementing each of these items? These questions helped me generate a variety of possible process improvement measures in my mind. Taken as a set, these metrics represent an ‘effectiveness snapshot’ of an organization’s process improvement system.

Example Process Improvement Effectiveness Measures

Exploring the Idea – Best Practice – Innovation Difference

I suspect that most people would struggle to explain the difference between an idea, a best practice, and an innovation. I did at first, but the more I thought about it, clarity and distinctions emerged. Ideas come first. Some ideas are good, and some are not. Some have been tried before, and worked, while others have failed time after time. We all get ideas, all of the time. It is the ones that give us great results that we’re after.

Best practices are ideas that have been proven to work. Their results compare well against relevant benchmarks. If you want to be one of the best hospitals, school systems, or small businesses nationally for a given measure, your results for that measure should compare favorably with national top percentiles. Best practices are needed to sustain such results. A caution exists here however. As with innovations, best practice claims are made much more often than they are proven.

True innovations are even harder to find. Innovations lead to breakthrough results – what some call step changes in performance, or hopping ‘S curves.’ As with best practices, innovative claims tend to be overstated. People often think things are innovative simply because they have not seen them before. In other cases, a change may be new within a given business sector, but commonplace in other sectors. Best practice or innovation? I would have to see the results before I make a choice.

EXPLORE MORE: In Search of Innovation

How Many Ideas, Best Practices, and Potential Innovations are You Implementing?

As I thought about ideas, innovations, and best practices being generated daily across an organization, possible effectiveness measures came into my mind. First, the three improvement types have to be counted, captured, and trended. The status of each of the three improvement types must also be factored in. Once captured, what becomes of your ideas, best practices, and innovations as they are evaluated? How many of your key work processes are being impacted?

At what rate are best practices and innovations being identified, versus ‘mere’ ideas? Most importantly, what are the different implementation rates for each type of improvement? High performance organizations should know both their process improvement rates and the degree that all key processes are being affected. What do you think?

How are You Measuring Process Improvement Effectiveness?

Test yourself. Can you describe the difference between an idea, a best practice, and an innovation?  If you can, do you know the rates at which each type of improvement is being generated across your work groups?  Do you also know your respective evaluation and implementation rates?

Don’t worry if the use of such rates sounds a bit foreign to you. Few organizations can provide evidence of such process improvement rates. One can only wonder how their leaders determine if their current rate of improvement is sustainable or if they are staying ahead of the competition without trending these measures.

Most organizations struggle to provide evidence of their process improvement effectiveness. They don’t know the rates at which ideas of any type – good or bad – are being generated, evaluated, and implemented over time. Ask for similar rates for higher order improvements, like best practices and innovations, and the reaction tends to be the ‘deer in the headlights’ look. How fast are we are improving? Do you have enough great ideas in the process improvement bank for future use?

LEARN MORE: Process Improvement Strategies

How Can You Measure Process Improvement Effectiveness?

A variety of rates for gauging process improvement effectiveness can be created from the basic ‘how many ideas are we capturing?’ monthly counts. You could simply sort the total ‘improvement options’ captured for a given time period into nine groups and calculate a ‘percent of total’ rate for the nine different idea, best practice, and innovation counts. One could also use a time period as the denominator, such as ‘per day’ or ‘per month.’

Since the nine count totals exist, I would trend these nine percentages over time as a baseline indicator. More importantly, I would also look at (1) percentage of total work processes affected and (2) ideas per employee work hour. For starters, I would only track three rates to gauge process improvement effectiveness – ideas, best practices, and innovations generated. Over time, all nine types of improvement rates would be trended using total employee work hours as a denominator. Such an approach provides a much better reflection of effective staff engagement.

Example Process Improvement Effectiveness Measures

Using Process Improvement Rates to Enhance and Sustain Competitiveness

This is where I became even more intrigued. Can someone claim to be, let alone prove they are, a high performer if they can’t produce trend lines, with benchmarks, for process improvement effectiveness rates such as these? Are we willing to focus on outcome gains alone, such as fewer lost customers and lower costs, more revenue and profit generated, or a reduction in accidents reported, to tell us if our work systems are sustainable and agile enough for the future? Don’t we need leading indicators of potential process excellence to remain sustainable in the future?

As with most measures, these process improvement rates tell us different things about our idea generation and stakeholder engagement processes. Our current evaluation and implementation rates give us insight into the degree that process changes are occurring right now. In too many organizations, idea backlogs exist because the evaluation and implementation processes have not been streamlined. People will quickly stop contributing potential improvement options if they don’t think their ideas are going to promptly, and fairly, considered.

Our improvement potential is defined both by the three generation rates, along with the ratios that exist between the three rates. Typically, the percentage of ‘all ideas’ that represent possible innovation is the lowest, followed by the best practice percent of total rate. Most organizations, however, struggle to consistently generate any form of improvement idea. How many ideas per person are generated in your company each month, week, or day?

DISCOVER MORE: Exploring Barriers to Process Improvement

How are You Finding, Sharing, and Using Ideas, Best Practices, and Innovations?

Process improvement effectiveness rates are needed to give us insight into who we are involving, and how we are involving them, in our formal process improvement efforts. Ad hoc, poorly aligned improvement efforts produce inconsistent results and ‘pockets of excellence’ at best.

Typically, only a small percentage of the organization is engaged in idea generation on a consistent basis. What percentage of your workforce is actively engaged in a formal process improvement effort? In turn, only a small percentage of work processes is being consistently improved, to any degree, year over year. What is the case in your organization?

Coordinated idea capture, evaluation, and implementation efforts contribute significantly to accelerated progress on the road to operational excellence. Sadly, evidence of best practice implementation and true innovation is tough to find in too many companies. A small percentage of organizations are, however, beginning to figure out the improvement angles. Check out the application summaries from past national Malcolm Baldrige Performance Excellence Program recipients for possible ‘great work system’ examples.

Here are some key first steps you might want to take:

  • Use a common database for all potential improvements. Include database fields for classifying each submittal as an idea, best practice, or potential innovation and for tracking current improvement status.
  • Create operational definitions for each of the three terms. Include specific examples of each improvement type to aid your team in making distinctions between each term.
  • Include process improvement effectiveness rates in your set of Key Performance Indicators (KPIs). Use trend and statistical process control (SPC) data to help set short- and long-term goals for improving each rate annually.
  • Make it easy for any employee to submit an improvement for consideration. Design the submittal form to help the idea, best practice, or innovation submitter align their proposal with the strategic intent of the organization, as defined by your strategic objectives, mission, vision, and values.
  • Require EVERY process owner to devote regular time to promoting improvement generation and supporting evaluation and implementation efforts. Include related strategies and measures in that person’s personal development plan.
  • Meaningfully recognize anyone who makes some form of improvement contribution.

How Fast is Your Organization Improving?

How are you finding, sharing, and using ideas, best practices and innovations? What do your process improvement effectiveness trend lines look like? How fast are you improving? What’s your true improvement potential? What percentage of your work processes are being improved each year, and to what degree? Only time will tell.

Stay tuned for my third post in this three-post series on the innovation topic. In my next post, I will share some best practices for developing the capability and capacity for more effective innovation generation across your workforce.

If you want some help installing effective work systems of this nature, please let me know. Also, be on the alert for my high-performance podcasts, which I will start sharing in the next few months.

Keep improving!

Kevin McManus, Chief Excellence Officer, Great Systems  

FOLLOW me on Twitter: @greatsystems

LIKE Great Systems on Facebook

CONNECT with me on LinkedIn