ISO 9000:2000 places a new and significant emphasis on "Measurement, Analysis and Improvement". This emphasis (like many other changes in the year 2000 revision) reflects the growing maturity of quality management experience around the world. There is a strong trend amongst the better companies towards the integration of the quality system and the management system. The resulting "Business Model" is firmly based on the collection and analysis of appropriate metrics in significant areas of the company’s activities. This paper explores the opportunities and challenges in implementing a metric management system in the context of the new ISO 9000:2000 revision.

All companies have metrics - activities that are measured, reviewed and then adjusted as necessary in the light of the results. For many companies the only formal metrics are financial ones such as annual turnover, taxation returns or end-of-quarter results. A smaller number of companies have sophisticated systems in place to define, measure, analyse and improve their "Key Performance Indicators". Such indicators typically go beyond the strictly financial measures to address operational issues like on-time delivery, design lead-time, rework levels, customer satisfaction etc.

It is not unusual for these metric management activities to be completely separate from the quality system, as if they lived in some kind of "parallel universe". Management regard the metrics as essential information for running the company, while ISO 9000 is what they have to do "to keep the Registrar happy". In many cases the quality system was simply not designed to support an effective metric management system. A small number of companies, on the other hand, have managed to integrate their management and quality systems so that the two are practically indistinguishable. Metric management plays a key role in the development of such a "Business Model" approach to the use of ISO 9000.

The ISO 9000:2000 revision of the Standard will encourage many more companies to follow this "Business Model" approach. Starting with a strong customer focus, the overall approach is to link the quality management system to the business needs of the organisation. The Standard places a great emphasis on the use of the quality system to achieve measurable continual improvement and requires quality objectives to be "established at relevant functions and levels within the organisation". This requirement will pose a significant, and hopefully positive, challenge for many companies.

Some companies will need to introduce for the first time a formal system to measure and improve their own performance. Others will need to integrate existing metric management systems with the quality management system. The relatively small number of companies that are already doing what the new standard requires will have little or no difficulty in complying. However, they still face the challenge of ensuring that their metrics continue to provide them with adequate insight into key parts of their business.

ISO 9004:2000, "Guidelines for Performance Improvements", reflects the experience of more mature quality management systems and provides excellent insight into many aspects of an effective metric management system. The challenge of implementing the broader, less detailed requirements in ISO 9001:2000 is greatly facilitated by this guideline document. One of the core ideas that seems to be reflected throughout ISO 9004:2000 is that of the Balanced Scorecard.

The Balanced Scorecard is a powerful tool for companies who wish to develop a metric management system. The approach focuses on the minimum amount of essential information necessary for each level of the organisation to achieve its objectives. It starts with the strategic perspective that all key "players" have interests that need to be addressed. This is clearly reflected in ISO 9004:2000 which describes "interested parties" as including customers, employees, shareholders, suppliers as well as the wider community. A typical Balanced Scorecard might include metrics on

• Financial performance
• Product/service quality
• Supplier performance
• Customer satisfaction
• Process and operational performance
• Employee satisfaction
• Community/Environment impact

The challenge is to identify suitable metrics under some or all of these headings that will carry clear significance for the reviewer – at whatever level of the organisation. In this way, each area of activity within the company can be consciously aligned with the organisation’s strategic goals.

Implementing a metric management system raises many challenging issues:

• Too much data quickly defeats the purpose of the exercise. Too little data can be very dangerous.
• It is important that the data reviewed is not too short-term. Where possible, the metrics should show past-present-future characteristics. (Eg: Cumulative Sales up to the end of last month, Sales for last month, projected sales for next six months).
• The level of detail should be appropriate to the user. Generally speaking, the higher up the organisation, the more summarised the information needs to be if it is to convey useful information. The process by which high-level metrics and goals are decomposed or "cascaded" down to lower levels needs to be continually reviewed.
• Targets and goals need to be based on rational criteria and proper research. Poorly designed metric management systems end up as little more than glorified "number games", where meaningless targets lead to the creation of equally meaningless reports.
• Need to focus on process efficiency rather than performance. If targets are met purely on the basis of short-term performance (eg working 16-hour days "just to meet the target") they will not last and the reported results, accurate as they may be in one sense, will be totally misleading from a systems point of view.

A major challenge facing many companies after registration to ISO 9000 is how to align the quality system with the key business goals of the organisation and avoid the problems of "maintenance fatigue". One very effective approach is to ensure that the Balanced Scorecard (or equivalent metric management system) provides the direction and that the quality management system provides the engine to get there.

ISO 9000:2000 represents a fundamental change in the orientation of the Standard. Previous versions of ISO 9000 described the components of a quality management system (under each of the 20 clauses). Now, at last, ISO 9000:2000 with its process approach and the five major headings, describes how these components fit together to form a real system. Let us look at how the traditional ISO 9000 components (documentation, action tracking, records, audits, management review) can be used to drive an effective metric management system.

• Documentation

The first point is that metric management is itself a process and should be mapped accordingly, with well-defined inputs and outputs, together with criteria for the selection of suitable metrics and the review of performance. Like any other part of the business, if the metric management process is not defined, it cannot be systematically improved.

Many quality procedures are written with inadequate attention to the main role of the documentation. From a Balanced Scorecard point of view, this role is very clear. It is to capture the practice that protects the current level of performance for any particular metric. A simple but powerful reminder of this is to include a heading like "Measures of Effectiveness" in all procedures. This can help strengthen the link between the quality and metric management systems.

• Action Tracking

Mechanical implementations of ISO 9000 include "Corrective Action Request" (CAR) systems to address things that go wrong in their quality systems (ie non-conformances). Such systems reflect and perpetuate a very negative mindset in relation to ISO 9000. By definition they are purely reactive and cannot easily be used to track the pro-active changes that result from the Balanced Scorecard approach. Many companies have derived significant benefit from renaming these reactive mechanisms with more pro-active titles such as "Controlled Action Request" or simply "Action Tracking" systems.

Such systems can help to cascade high-level objectives down to "relevant functions and levels within the organisation" (see ISO 9001:2000 5.4.1). In fact they can track the entire life-cycle of a target or goal related to the Balanced Scorecard. Higher level CARs (Controlled Action Requests) can be broken down into a number of lower level ("child") CARs and this process can be continued until the units of activity necessary to achieve the target are reflected in a "family-tree" of such CARs. Layer by layer, starting from the bottom, each CAR in the family is monitored, tracked, reviewed, recorded and ultimately closed. This cycle is repeated until the original, high-level target (hopefully) has been achieved.

This pro-active use of the corrective action mechanism can help invigorate the quality management system and move it from "maintenance mode" towards being a key driver for continual improvement. This is the key to a successful "Business Model".

In many ways the Balanced Scoreboard is no more than a formalised way of collecting and analysing "quality records… to demonstrate the effective operation of the quality management system" as has always been a requirement of ISO 9000. But more importantly, it can provide the "missing link" that so often prevents management from grasping the full potential of the quality system.

Using the Balanced Scorecard literally (i.e. like a scoreboard at a football match) provides the organisation from top to bottom with a clear, visible focus for the collection and analysis of quality records. Properly designed, it helps organisations to face up to failure in a systematic, constructive manner.

• Internal Audits

Internal quality audits are supposed to "determine whether the quality management system has been effectively implemented and maintained" (ISO 9000:2000 8.2.2b). Traditionally internal audits focus on fairly stable aspects of the business (as reflected in documented procedures) and not on areas that are going through major change. Using Controlled Action Requests to implement goals and targets allows audits to confirm that "change is conducted in a controlled manner and that the integrity of the quality management system is maintained during this change" (ISO 9000:2000 5.4.2b). Under a Balanced Scorecard approach, such audits can provide powerful, objective, timely, independent feedback on the current stage of implementation of the various changes that result from the high-level objectives.

Of course, an important area to audit would be the metric management system itself - to ensure that the organisation is following best practice and that performance improvements are being implemented in a systematic and sustainable way.

• Management Review

The theory of ISO 9000 is that management own and sponsor the quality management system and take an active leading role in its development and improvement long after registration. In practice, executive management tend to withdraw from the quality system shortly after certification, to get on with "the real business" of running the organisation. Their direct involvement in ISO 9000 tends to be limited to the "management review" (as required by the Standard) - typically once or twice a year. Such reviews tend to be rather mechanical, largely because the quality system under review is not well aligned with the business.

Using ISO 9000 to implement the Balanced Scorecard approach can transform the management review and, more importantly, the entire relationship between the management system and the quality system. In fact they can both merge into something far more powerful than either on its own – a Business Model. In such a model, the management review begins to ask much more relevant questions, such as:

• Did we achieve our business objectives as reflected in the Balanced Scorecard?
• Did we achieve them because of the quality system/business model – or was it in spite of it?
• Do we need to improve our process for identifying and achieving the goals and targets defined on our Balanced Scorecard?

All the above "components" of a quality system have been included in earlier versions of ISO 9000 but the "engine" was never clearly described. The new revision makes it very clear that the components are meant to be part of a system that drives continual improvement in a systematic and reviewable manner. The process model for continual improvement in ISO 9000:2000 provides an interesting framework for the implementation of an effective metric management system. Let us review the four major steps of that process.

• Management Responsibility

Under this heading management articulate their policy of putting a Balanced Scorecard approach at the very heart of their management system.

• Resource Management

Issues of resources apply as much to the metric management process as to any other. The main resource is usually time - time to collect and analyse data. Many companies find themselves in the vicious circle of being so busy "achieving results" that they have no time to prove that the results they achieve are worth the effort. Management commitment to an effective metric management system must be followed by the allocation of sufficient resources to make that system work.

• Product Realisation

The metric management process needs to be defined. Documentation should capture how metrics are selected, identified, broken down or "cascaded" to various functions and levels, monitored, reviewed and updated at appropriate intervals. In particular it should address the key events of reporting and responding to any deviations from targets or goals.

• Measurement, Analysis and Improvement

A mature system should "measure, analyse and improve" the metric management system itself. This will not happen unless the system is designed to make it happen. Useful things to measure might include the following:

• the number of metrics established versus the planned number
• The degree to which the high level targets/goals have been met.
• the number of metric-based activities that are completed on time
• the main reasons for failure to achieve targets/goals.
• main reasons for other deviations in the metric management process.

When the continual improvement loop is closed, and top management (at management review) ask the big question: "Is this metric management system working for us...?" the answer should be very obvious.

Being in a position to ask the question is the hallmark of a world class organisation.

As the world shrinks under the influence of information technology and globalisation, new challenges emerge for the future direction and structure of quality management systems. Many large corporations have implemented ISO 9000 on a site-by-site basis, with little or no regard to sharing best practice between sites or using the ISO 9000 toolbox as a framework for continual improvement across the corporation. Many such corporations do indeed operate good metric management systems, often with a Balanced Scoreboard approach, but usually outside the framework of the "ISO system". This is a major challenge.

For many organisations the process of streamlining the quality system within one site seems difficult enough. The idea of aligning all their sites is so overwhelming that few even address this issue. And yet they do operate a metric management system. They do have an established best practice in identifying, monitoring, reviewing and responding to trends in their key performance indicators. And the changes that they initiate do have a direct and predictable impact on the integrity of the quality systems in the various sites.

© Copyright Rody Ryan 2000. All rights reserved.