How to Measure Process Improvement Effectiveness
By Kevin McManus, Chief Excellence Officer, Great Systems
This ‘how to measure process improvement effectiveness’ post began as an Industrial and Systems Engineer magazine article I wrote a few years ago. Back then, I had just completed another national Malcolm Baldrige Performance Excellence Program site visit. On that particular site visit, my role was to evaluate the organization’s processes for information capture, analysis, and use.
As I prepared questions for the visit in this area, some new questions came into my own mind. How should an organization measure organizational progress in idea, best practice, and innovation generation? Unfortunately, few organizations do little more in this area than count the number of ideas they capture and implement over time.
For starters, what is the difference between an idea, a best practice, and an innovation? Also, does it make sense to track the frequencies at which we generate, evaluate, and implement each of these items? Such questions help me generate a variety of possible process improvement measures in my mind. As a set, these metrics represent an ‘effectiveness snapshot’ of an organization’s process improvement system.
Exploring the Idea – Best Practice – Innovation Definition Difference
I suspect that most people would struggle to explain the difference between an idea, a best practice, and an innovation. I did at first. But the more I thought about it, clarity and distinctions began to emerge. Ideas come first. Some ideas are good, and some are not. Some have been tried before, and worked. Others have failed time after time. We all get ideas, all of the time. It is the ones that give us ‘step change’ results that we’re after.
Best practices are ideas that have been proven to work. Their results compare well against relevant benchmarks. Do you want to be one of the best hospitals, school systems, or small businesses nationally for a given measure? If so, results for that measure should compare favorably with national top percentiles. You need best practice work systems to sustain such results. However, a caution exists. As with innovations, best practice claims are made much more often than they are proven.
True innovations are even more difficult to find. Innovations lead to breakthrough results. Such results are what some call step changes in performance, or hopping ‘S curves.’ As with best practices, innovative claims tend to be overstated. Often, people often think something is innovative simply because it is new to them. In other cases, a change may be new within a given business sector, but commonplace in other sectors. Best practice or innovation? I want to see the results before I make a choice.
How Many Ideas, Best Practices, and Potential Innovations Do You Implement?
As I thought about ideas, innovations, and best practices an organization generates daily, possible effectiveness measures came into my mind. First, we have to capture, count, and trend the three improvement types. Also, we must consider the status of each of the three improvement types. Post-idea capture, what becomes of your ideas, best practices, and innovations as they move towards evaluation? How many of your key work processes do the ideas you capture impact?
At what rate do you identify best practices and innovations, versus ‘mere’ ideas? Most importantly, what are the different implementation rates for each type of improvement? High performance organizations know their process improvement rates. Plus, they know the degree that improvement affects all key processes. What do you think?
How Do You Measure Process Improvement Effectiveness?
Test yourself. Can you describe the difference between an idea, a best practice, and an innovation? If so, do you know the rates at which you generate each type of improvement across your work teams? Do you also know your respective evaluation and implementation rates?
Don’t worry if the use of such rates sounds a bit foreign to you. Sadly, few organizations can provide evidence of such process improvement rates. One only wonders how their leaders determine if their current improvement rate is sustainable. What keeps them ahead of the competition without trends for these measures?
Most organizations struggle to provide evidence of their process improvement effectiveness. They don’t know the rates at which they generate ideas of any type – good or bad. Also, evaluation and implementation rates are unclear, or unknown. Ask for similar rates for higher order improvements, like best practices and innovations. Often, the reaction tends to be the ‘deer in the headlights’ look. How fast do your processes improve? Do you have enough great ideas in the process improvement bank for future use?
How Can You Measure Process Improvement Effectiveness?
Leaders can create a variety of rates to gauge process improvement effectiveness. These rates go beyond the basic ‘how many ideas do we capture?’ monthly counts. To start, simply sort the total ‘improvement options’ you capture for a given time period into nine groups. Then, calculate a ‘percent of total’ rate for the nine different idea, best practice, and innovation counts. Also, you can use a time period as the denominator, such as ‘per day’ or ‘per month.’
Since the nine count totals exist, I trend these nine percentages over time as a baseline indicator. More importantly, I look at (1) percentage of total work processes in play and (2) ideas per employee work hour. For starters, only track the idea, best practice, and innovation generation rates to gauge process improvement effectiveness. Over time, you can trend all nine types of improvement rates. Consider the use of total employee work hours as a denominator. Such an approach provides a much better reflection of effective staff engagement.
Using Process Improvement Rates to Enhance and Sustain Competitiveness
This is where I become even more intrigued. Can someone claim to be, let alone prove they are, a high performer if they can’t produce trend lines, with benchmarks, for process improvement effectiveness rates such as these? Are we willing to focus on outcome gains alone, such as fewer lost customers and lower costs, more revenue and profit generated, or a reduction in accidents reported? How effectively do such measures tell us if our work systems are sustainable and agile enough for the future? Don’t we need leading indicators of potential process excellence to remain sustainable in the future?
As with most measures, these process improvement rates tell us different things about our idea generation and stakeholder engagement processes. Our current evaluation and implementation rates give us insight into the degree that process changes occur right now. In too many organizations, idea backlogs exist. Such backlogs occur when the evaluation and implementation processes contain too much waste. People will not contribute potential improvement options if they don’t think their ideas will be given prompt, and fair, consideration.
The three generation rates, along with the ratios that exist between the three rates, help define our improvement potential. Typically, the percentage of ‘all ideas’ that represent possible innovation is the lowest rate. Next, the ‘best practice percent of total’ rate is often found. Most organizations struggle to consistently generate any form of improvement idea. How many ideas per person do you generate in your company each month, week, or day?
How Do You Find, Share, and Use Ideas, Best Practices, and Innovations?
Leaders need process improvement effectiveness rates. Such rates give them insight into who they engage in their formal process improvement efforts. Ad hoc, poorly aligned improvement efforts produce inconsistent results and ‘pockets of excellence’ at best.
Typically, most companies only engage a small percentage of their staff in idea generation on a consistent basis. What percentage of your workforce do you actively engage in a formal process improvement effort? In turn, we only consistently improve a small percentage of work processes, to any degree, year over year. What is the case in your organization?
Coordinated idea capture, evaluation, and implementation efforts contribute significantly to accelerated progress on the road to operational excellence. Sadly, evidence of best practice implementation and true innovation is tough to find in too many companies. However, a small percentage of organizations are beginning to figure out the improvement angles. Check out the application summaries from past national Malcolm Baldrige Performance Excellence Program recipients for possible ‘great work system’ examples.
Key Ways to Improve the Effectiveness of Your Process Improvement Work System
Here are some key first steps you might want to take:
- Use a common database for all potential improvements. Use database fields to classify each idea submittal as an idea, best practice, or potential innovation. The database also helps track current improvement status.
- Create operational definitions for each of the three terms. Include specific examples of each improvement type to help your team make distinctions between each term.
- Include process improvement effectiveness rates in your set of Key Performance Indicators (KPIs). Use trend and statistical process control (SPC) data to help set short- and long-term goals to improve each rate annually.
- Make it easy for any employee to submit an improvement for consideration. Design the submittal form to help the idea, best practice, or innovation submitter align their proposal with the strategic intent of the organization. Your strategic objectives, mission, vision, and values help define such intent.
- Require EVERY process owner to demonstrate HOW they personally promote improvement generation and support evaluation and implementation efforts. Include strategies and measures to support such skill development in that leader’s personal development plan.
- Meaningfully recognize anyone who makes some form of improvement contribution.
How Fast Does Your Organization Improve?
How do you find, share, and use ideas, best practices and innovations? What do your process improvement effectiveness trend lines look like? How fast does each of your processes improve? What’s your true improvement potential? What percentage of your work processes do you improve each year, and to what degree? Only time will tell.
Stay tuned for my third post in this three-post series on the innovation topic. In my next post, I will share some best practices for developing the capability and capacity for more effective innovation generation across your workforce.
Do you want some help to design and install effective work systems of this nature? If so, please let me know. Also, be on the alert for my high-performance podcasts, They will begin in the next few months.
Please email me your questions at firstname.lastname@example.org