EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL
Introduction to KPI Models
AIM/AEP/S-LEV/0004
Edition Edition Date Status Class
: : : :
0.1 29 Oct 2001 Draft General Public
EUROPEAN AIR TRAFFIC MANAGEMENT PROGRAMME
DOCUMENT IDENTIFICATION SHEET DOCUMENT DESCRIPTION Document Title Introduction to KPI Models
EWP DELIVERABLE REFERENCE NUMBER PROGRAMME REFERENCE INDEX
EDITION :
0.1
EDITION DATE :
AIM/AEP/S-LEV/0004
29 Oct 2001 Abstract This document summarises KPI models existing in the literature. It covers three models that can required during the deployment of a performance measurement system: KPI categorisation, KPI correlation and improvement models.
KPI Improvement
CONTACT PERSON :
Keywords Categorisation
Model
Paul Bosman
Correlation
TEL : 3333
UNIT :
DOCUMENT STATUS AND TYPE STATUS Working Draft Draft Proposed Issue Released Issue
o þ o o
CATEGORY Executive Task Specialist Task Lower Layer Task
INTERNAL REFERENCE NAME :
þ o o
CLASSIFICATION General Public EATMP Restricted
AHEAD Electronic Filing System
þ o o
AIM/AEP/S-LEV/0004
Introduction to KPI Models
DOCUMENT APPROVAL The following table identifies all management authorities that have successively approved the present issue of this document.
AUTHORITY
NAME AND SIGNATURE
DATE
Ertan Ozkan
29 Oct 2001
Conrad Cleasby
29 Oct 2001
Author/Editor
Programme Manager
Quality Assurance
Page iv
Draft
Edition: 0.1
Introduction to KPI Models
AIM/AEP/S-LEV/0004
DOCUMENT CHANGE RECORD The following table records the complete history of the successive editions of the present document.
EDITION
DATE
0.1
29 Oct 2001
Edition: 0.1
REASON FOR CHANGE Creation
Draft
SECTIONS PAGES AFFECTED All
Page v
AIM/AEP/S-LEV/0004
Page vi
Introduction to KPI Models
Draft
Edition: 0.1
Introduction to KPI Models
AIM/AEP/S-LEV/0004
TABLE OF CONTENTS
1.
INTRODUCTION ...................................................................................................... 1 1.1
Purpose and scope ................................................................................................ 1
1.2
References.............................................................................................................. 1
2.
WHERE ARE KPI IN THE CONTEXT OF ISO9001:2000? ...................................... 3 2.1
Organisation as Production Process .................................................................... 3
2.2
ISO 9001:2000/8 Measurement, Analysis and Improvement................................ 4
2.3 3.
Element 8.1 - General................................................................................. 5
2.2.2
Element 8.2.1 - Customer Satisfaction........................................................ 5
2.2.3
Element 8.2.3 - Measurement and monitoring of processes ....................... 6
2.2.4
Element 8.2.4 Monitoring and measurement of product .............................. 6
Why a Performance Measurement System?......................................................... 7 KPI CATEGORISATION .......................................................................................... 9
3.1
Balanced Scorecard ............................................................................................... 9
3.2
Intangible Assets Monitor .................................................................................... 11
3.3
KPI Categorisation Proposal ............................................................................... 12
4.
KPI CORRELATION .............................................................................................. 15 4.1
Organisational Hierarchy ..................................................................................... 15 4.1.1
Downward information flow ....................................................................... 16
4.1.2
Upward information flow ........................................................................... 16
4.2
Process Hierarchy ................................................................................................ 17
4.3
Semantic Correlation ........................................................................................... 18
5.
6.
2.2.1
IMPROVEMENT MODEL ....................................................................................... 20 5.1
3-Tier Improvement Model ................................................................................... 20
5.2
Plan-Do-Check-Action (PDCA) Cycle .................................................................. 21 CONCLUSIONS ..................................................................................................... 22
Edition: 0.1
Draft
Page vii
AIM/AEP/S-LEV/0004
Page viii
Introduction to KPI Models
Draft
Edition: 0.1
Introduction to KPI Models
1.
INTRODUCTION
1.1
Purpose and scope
AIM/AEP/S-LEV/0004
This document summarises the theory behind the Key Performance Indicators (KPIs). It covers the existing models in the literature (i.e., the models used in the balanced scorecard [5] and intangible assets monitor [10]) for KPI categorisation and correlation. A simple improvement model completes the study. The document is organised as follows: • Chapter 2 summarises ISO9001:2000 requirements related to measurement, analysis and improvement; • Chapter 3 presents performance measurement models and proposes a KPI categorisation scheme; • Chapter 4 discusses issues related to KPI correlation; • Chapter 5 presents simple improvement models and how KPIs are used in these models; • Finally chapter 6 gives conclusions. Throughout this document the term “performance measurement system” is used to denote the decision support system that contains a set of KPIs with well-defined measurement, monitoring, analysis and improvement procedures.
1.2
References [1] Foundations of Service Level Management, R. Sturm, W. Morris and M. Jander, ISBN 0672317435, SAMS Publications, 2000. [2] Change Management the 5-step action kit, C. Rye, ISBN 0749433809, Kogan Page Limited, 2001. [3] Operational Performance Measurement Increasing Total Productivity, W. Kaydos, ISBN 1574440993, CRC Press LLC, 1999. [4] Service Level Management for Enterprise Networks, L. Lewis, ISBN 1580530168, Artech House, 1999. [5] What is the Balanced Scorecard, P. Averson, http://www.balancedscorecard.org/basics/bsc1.html, The Balanced Scorecard Institute,1998. [6] Top-10 Reasons for a Performance Measurement System, P. Averson, http://www.balancedscorecard.org/appl/top_ten.html, The Balanced Scorecard Institute,1998. [7] Deployment of the Balanced Scorecard Measurement System, P. Averson, http://www.balancedscorecard.org/appl/deployment.html, The Balanced Scorecard Institute, 1999.
Edition: 0.1
Draft
Page 1
AIM/AEP/S-LEV/0004
Introduction to KPI Models
[8] ISO9001:2000 FAQs, http://isotc176sc2.elysium-ltd.net/FAQs.html [9] KPI Portfolio, Edition 0.4, AIM/AEP/S-LEV/0005, 12 Nov 2001. [10] Measuring Intangible Assets, http://www.sveiby.com.au/IntangAss/MeasureIntangibleAssets.html, Karl-Erik Sveiby, 1 March 1997. [11] ISO 9001:2000 Discussion, ISO 9001 Support Group, http://www.isogroup.net/
Page 2
Draft
Edition: 0.1
Introduction to KPI Models
AIM/AEP/S-LEV/0004
2.
WHERE ARE KPI IN THE CONTEXT OF ISO9001:2000?
2.1
Organisation as Production Process An AIS organisation, like any other organisation, is a complex set of processes with multiple inputs, production processes, outputs and stakeholders. In simple terms it produces publications and provides various types of services to its customers. If it is viewed as a production process its input, output and stakeholders will be very similar to the following: Capital, people, energy, material
Inputs
Orders, changes AIS
Competitors Vendors Regulations, Standards Economic forces Environmental Factors
O R G A N I S A T I O N
Products, services
Customers
Profits, dividends, growth
Shareholders
Wages, benefits, security, opportunity
Employees
Taxes, compliance with laws
Governments
Returns, waste (material, labour, capital)
Outputs
Stakeholders
Figure 1. AIS Organisation as a Production Process
To measure the performance of this model AIS organisation the following must be determined: • How well it is serving its different stakeholders. A good performance measurement system must reflect what is important to all of its stakeholders: the AIS organisation itself, its customers, its shareholders, its employees, and arguably, governments and society as a whole; • AIS organisation consists of complex set of processes, it is required to understand how it is performing. Success comes both having a good strategy and executing it well. However how one can determine if a strategy is correct without having factual information.
Edition: 0.1
Draft
Page 3
AIM/AEP/S-LEV/0004
Introduction to KPI Models
The new ISO 9001:2000 standard makes life easier and gives the outline of measurement, analysis and improvement activities [8]: • Monitoring of information on customer satisfaction as a measure of system performance; • Measurements extended to system, processes, and product; • Establishment of measurable objectives at relevant functions and levels; • Analysis of collected data on the performance of the quality management system; • Continual improvement. If the box “AIS organisation” in the model (See Figure 1) is extended with the requirements of the ISO9001:2000 standards, the location of measurement, analysis and improvement requirements will become more clear:
Sales and Marketing
I N P U T
Production Planning and Development
O U T P U T
Etc…
ISO 9001:2000/7 Product Realization
ISO 9001:2000/6 Resource Management
S T A K E H O L D E R
ISO 9001:2000/8 Measurement, Analysis and Improvement
Figure 2. Process Model Extended with ISO9001:2000 Requirements
As shown in the above figure the performance measurement system should have interfaces with internal processes, output and stakeholders. Before going into the details of the KPI models lets explore the ISO 9001:2000/8 Measurement, Analysis and Improvement.
2.2
ISO 9001:2000/8 Measurement, Analysis and Improvement This section is compiled from ISO 9001:2000 Discussion Group [11].
Page 4
Draft
Edition: 0.1
Introduction to KPI Models
2.2.1
AIM/AEP/S-LEV/0004
Element 8.1 - General “The organisation shall plan and implement the monitoring, measurement, analysis and improvement processes needed a) to demonstrate conformity of the product, b) to ensure conformity of the quality management system, and c) to continually improve the effectiveness of the quality management system
This shall include determination of applicable methods, including statistical techniques, and the extent of their use.” Planning states that your organisation must plan, define and implement the activities (inspection and testing/measurement and monitoring) to assure conformity and achieve improvement throughout your organisation. Processes are also subject to the same inspection and test requirements as products and services. Any process that is deemed to have an impact on conformity is subject to verification. However, not only do products, processes and services have to conform, they must improve. Inspection and Test Plans will have to be set up for critical processes. Acceptance criteria have to include improvement specifications. It also states that your organisation shall determine the need for, and use of, applicable methodologies including statistical techniques. In comparison to the old standard, statistical techniques are no longer a separate element. But the significance of this requirement is its application to improvement and non product/service related process. Organisations still only require techniques that are appropriate. If there are no techniques that are appropriate to your organisation, then no techniques are required. Statistical tools such as chart and table analysis should be used as a minimum to measure the achievement of quality objective and goals. Your procedure should reference the procedures that use statistical tools or it could reference a list that identifies the tool and the person responsible for its implementation. 2.2.2
Element 8.2.1 - Customer Satisfaction “As one of the measurements of the performance of the quality management system, the organisation shall monitor information relating to customer perception as to whether the organisation has met customer requirements. The methods for obtaining and using this information shall be determined.” The ISO 9001:2000 standard requires that you must monitor information on customer satisfaction (and/or dissatisfaction) as one of the measurements of performance of the quality system. The new standard helps clarify whose opinion is most important in regards to the quality system: customer's opinion. The standard requires that you define ways of obtaining and using information pertinent to your customer's satisfaction and monitoring this information as a
Edition: 0.1
Draft
Page 5
AIM/AEP/S-LEV/0004
Introduction to KPI Models
minimum form of measuring quality management system performance. However it does not necessarily mean that it requires the implementation of an entirely new measurement system. Being proactive and finding out issues before they arise is the underlying idea. To which extent you want to take this is up to you. 2.2.3
Element 8.2.3 - Measurement and monitoring of processes “The organisation shall apply suitable methods for monitoring and, where applicable, measurement of the quality management system processes. These methods shall demonstrate the ability of the processes to achieve planned results. When planned results are not achieved, correction and corrective action shall be taken, as appropriate, to ensure conformity of the product.” Element 8.2.3 states that your organisation must apply suitable methods for measurement and monitoring of those processes specific to your organisation that allow you to deliver your product to your customers (realisation processes). The standard states that these methods must confirm the continuing ability of each of these processes to satisfy its intended purpose. The creation of a product occurs through the operation of one or more processes. You have to make sure that the processes are set up to meet customer requirements (i.e. capability studies). You continually monitor these processes and make sure they are always set for meeting customer requirements. At the same time you use the data you are collecting to look for opportunities for improvement. One critical point of the new standard is the monitoring component. When one "monitors" a process, one must compare the collective variance from one set of measurements to the next, or a series of measurements. The standard is clear on how this data must be used: process measures must show the continual ability to satisfy customer requirements (monitoring is an obvious component in identifying this). Action will have to be initiated when processes fail to satisfy requirements.
2.2.4
Element 8.2.4 Monitoring and measurement of product “The organisation shall monitor and measure the characteristics of the product to verify that product requirements have been met. This shall be carried out at appropriate stages of the product realization process in accordance with the planned arrangements (see 7.1.). Evidence of conformity with the acceptance criteria shall be maintained. Records shall indicate the person(s) authorizing release of product (see 4.2.4). Product release and service delivery shall not proceed until the planned arrangements (see 7.1) have been satisfactorily completed, unless otherwise approved by a relevant authority and, where applicable, by the customer.” The first paragraph states that your organisation must measure and monitor the characteristics of your product and verify that the requirements of the product are met. You have to do this at appropriate stages of the product realisation process. The second paragraph states that evidence of conformity to the characteristics of your product that verify the requirements must be
Page 6
Draft
Edition: 0.1
Introduction to KPI Models
AIM/AEP/S-LEV/0004
documented. The standard also states that you must indicate the "authority" responsible for the release of the product. The third and final paragraph states that product release and service delivery must not proceed until all the "monitoring and measurement of product" activities have been completed, unless of course your customer says it is ok (or otherwise approved by them). The ISO 9001:2000 standard is more specific and comprehensive regarding ensuring a quality product leaves your facility. The new standard now requires you to "measure and monitor the characteristics of the product" to ensure the product requirements are met, as oppose to requiring "inspection and testing activities" to verify the requirements are met, as in the old standard. Your organisation will have to identify the significant product / service characteristics that effect quality and then develop methods to verify conformity to these characteristics. This includes characteristics that may not be identified by your customers.
2.3
Why a Performance Measurement System? In the previous sections the requirements of ISO9001:2000 are analysed. However operational and practical gains are still not clear. Top-10 reasons to deploy a performance measurement system are as follows [3, 6]: 1. Basis of quality management and service level management: KPIs provide factual basis for quality and service level management. In fact the monitoring and measurement of customer satisfaction, products and processes are clear ISO9001:2000 requirements. Furthermore KPIs and their associated objectives are used to define a service level agreement. 2. Improved Control: Visibility provides accountability and incentives based on real data, not anecdotes and subjective judgements. This serves for reinforcement and the motivation that comes from competition. 3. Better Planning and Forecasting: The visibility provided by a performance measurement system supports better and faster budget decisions and control of processes in the organisation. This means it can reduce risk. Collection of process cost data for many past projects allows us to learn how to estimate costs more accurately for future projects. 4. More Efficient Allocation of Resources: Measurement of process efficiency provides a rational basis for selecting what business process improvements to make first. A good carefully selected set of KPIs clarifies in which areas service/product did need improvement and in which areas they were satisfactory, so resources could be channelled accordingly and efficiently. In case of outsourcing they are useful to understand whether your suppliers are violating the legal agreements they have made with you. 5. Understanding Business Processes: It allows managers to identify best practices in an organisation and expand their usage elsewhere. Knowing what a process can do (i.e.,, its capability) 6. Improved Quality and Productivity: It improves the bottom line by reducing process cost and improving productivity and mission effectiveness. 7. Benchmarking: It permits of process benchmarking performance against outside organisations.
Edition: 0.1
Draft
Page 7
AIM/AEP/S-LEV/0004
Introduction to KPI Models
8. Clear Responsibilities and Objectives: Good KPIs clarify who is responsible for specific results or problems. They specify what “good performance” means for each person, manager or operating unit. This has several distinct advantages: 1) Everyone knows what they are supposed to accomplish, 2) Everyone knows how well they are performing, 3) Everyone becomes accountable for only their performance and not problems created by someone else. 9. Strategic Alignment of Objectives: A performance measurement system allows an organisation to align its strategic activities to the strategic plan. It permits real deployment and implementation of the strategy on a continuous basis. Through KPI measurements an organisation can get feedback needed to guide the planning efforts. 10. The Freedom to Delegate: Managers are often reluctant to delegate because they are afraid things might get out of control without knowing about it. However, when managers can stay in touch with what is happening through KPIs, the fear of delegating and the tendency to micro-manage disappear.
Page 8
Draft
Edition: 0.1
Introduction to KPI Models
3.
AIM/AEP/S-LEV/0004
KPI CATEGORISATION The first two sections of this chapter summarise balanced scorecard and intangible assets monitor. These two approaches contain a performance measurement model. Our objective is to understand their basic ideas and to apply feasible ones in our KPI model. In the last section a KPI categorisation model derived from ISO 9001 requirements and these two models is provided.
3.1
Balanced Scorecard The balanced scorecard (BSC) was initially developed around 1990 in the USA by Kaplan & Norton. It is a management system (not only a measurement system) that enables organisations to clarify their vision and strategy and translate them into action. It provides feedback around both the internal business processes and external outcomes in order to continuously improve strategic performance and results. The BSC suggests that the organisation is viewed from four perspectives, and to develop KPIs, collect data and analyse it relative to each of these perspectives: • The Learning and Growth Perspective • The Business Process Perspective • The Customer Perspective • The Financial Perspective The four perspectives used in the BSC can be summarised as follows [5]: • Learning and Growth Perspective: This perspective includes employee training and corporate cultural attitudes related to both individual and corporate selfimprovement. In a knowledge-worker organisation, people (i.e., the only repository of knowledge) are the main resource. In the current climate of rapid technological change, it is becoming necessary for knowledge workers to be in a continuous learning mode. KPIs can be put into place to guide managers in focusing training funds where they can help the most. In any case, learning and growth constitute the essential foundation for success of any knowledge-worker organisation. • Business Process Perspective: This perspective refers to internal business processes. KPIs based on this perspective allow the managers to know how well their business is running and whether its products and services conform to customer requirements (the mission). These KPIs have to be carefully designed by those who know these processes most intimately. • Customer Perspective: Recent management philosophy has shown an increasing realisation of the importance of customer focus and customer satisfaction in any business. These are leading indicators: if customers are not satisfied, they will eventually find other suppliers that will meet their needs. Poor performance from this perspective is thus a leading indicator of future decline. In developing KPIs for satisfaction, customers should be analysed in terms of kinds
Edition: 0.1
Draft
Page 9
AIM/AEP/S-LEV/0004
Introduction to KPI Models
of customers and the kinds of processes for which a product or service are provided to those customer groups. • Financial Perspective: This perspective covers the traditional need for financial data. Timely and accurate funding data will always be a priority and managers will do whatever necessary to provide it. It may include additional financialrelated data such as risk assessment and cost-benefit data, in this category.
Figure 3. Four Perspectives of Balanced Scorecard
As stated before the BSC is a management approach, KPIs designed within the context of BSC are used • In fine-tuning activities • To assist in choosing activities • In decision making to refine and revise decisions of strategic choice. • To provide feedback about the effectiveness of activities within the organisation to achieve desired outcomes is also required. The idea of “Strategy-Objective-Measurement”in the BSC is illustrated in the following figure by using a simple example “room temperature”. (For more complete example please see Figure 14. Objectives for a Furniture Vendor)
Page 10
Draft
Edition: 0.1
Introduction to KPI Models
AIM/AEP/S-LEV/0004
Strategic Control - What Temperature?
Strategy
Which Activities? Goals
Operational Control - Keep the room warm
Thermostat: Compare with target
Feedback
Activity
Which KPIs?
Run heater if room is below target temperature.
Measurements
Performance Measurement
Measure Room Temperature
Figure 4. Loop in the Balanced Scorecard
3.2
Intangible Assets Monitor The Intangible Assets Monitor (IAM) is based on the notion of people as an organisation´s only profit generators. Human actions are converted into both tangible and intangible knowledge "structures". These structures are directed outwards (external structures) or inwards (internal structures). These structures are assets, because they affect the revenue streams. The profits generated from peoples’actions are signs of that success, but not the originator of it. The IAM is a stock/flow theory. It assumes that some of the organisation's assets are intangible assets and the purpose of the IAM is to guide managers in how they utilise the intangible assets, identify the flows that are growing them and renewing them and guard against the risk is of losing them. The IAM has three main categories and each category is further divided into three sub-categories in order to reflect growth, renewal and stability. Since the basic notion of IAM is “people”, The IAM categories are defined in terms of people. • External Structure: It covers all the time the employees who work for customers. They work represent a potential for maintaining, building and developing relations with customers in direct projects for them. • Internal Structure: It covers employees who work in general management administration, accounting, personnel, reception, filing, etc. Their role is to maintain the internal structure. Activities like routine maintenance of
Edition: 0.1
Draft
Page 11
AIM/AEP/S-LEV/0004
Introduction to KPI Models
computer systems and databases are also be classed under this category, unless they refer to a specific customer or group of customers. • Competence of People: It covers professionals who plan, produce, process or present the products or solutions client asks for. The term includes all those directly involved in client work, whether or not they are professionals in the field of competence that constitutes the company´s business idea. It does not, however, include members of the company´s support functions, i.e. those who work in accounting, administration, reception, etc. They are part of the internal structure, and should be accounted for under that category.
Intangible Assets Monitor External Structure
Internal Structure
Competence of People
Indicators of Growth/Renewal
Indicators of Efficiency
Indicators of Stability
Figure 5. Categorisation of Intangible Assets Monitor
The following figure gives some example KPIs for the category “competence of people”: Indicators of Growth/Renewal Number of Years in the Profession Level of Education Training and Education Costs. Competence Turnover
Competence of People
Indicators of Stability Professional Turnover Seniority Indicators of Efficiency Proportion of Professionals Value Added per Employee. Profit per Employee
Figure 6. Example KPIs for IAM
3.3
KPI Categorisation Proposal The ISO 9001 clearly states that system (including customer satisfaction), product and processes must be measured.
Page 12
Draft
Edition: 0.1
Introduction to KPI Models
AIM/AEP/S-LEV/0004
Balanced Scorecard (BSC) and Intangible Assets Monitor (IAM) gives the subcategorisation required for system measurement: • Human Factors (i.e., internal structure and competence of people of IAM and learning and growth perspective of BSC) • Customer Satisfaction (i.e., external structure of IAM and customer perspective of BSC) • Financial (i.e., financial perspective of BSC) In order to fulfil the ISO 9001 requirements there should be another category that covers processes and products: • Process (i.e., business process perspective of BSC)
Customer Satisfaction
Human Factors
Financial
Process
Figure 7. AIS Main KPI Categories
What to measure in the first three categories, which are related to system measurements, is relatively clear. On the other hand measurements related to process needs a further look. The following figure shows the inputs and outputs that might be measured to understand how a process is performing and why it is performing that way. Resources In Product Output Quantity
Work Input Quantity Work Input Quality Input Procedural Quality
Production Process – Value Adding Activities
Environmental Factors
Product Output Quality Output Procedural Quality Productivity
Waste Out Figure 8. Process Model
Edition: 0.1
Draft
Page 13
AIM/AEP/S-LEV/0004
Introduction to KPI Models
Each variable of the process is defined as follows [3]: • Work Input Quantity: The amount of work to be accomplished. • Work Input Quality: The quality of work units coming into the process. • Input Procedural Quality: In addition to product quality problems, they may be problems that are not directly attached to the product (e.g., late delivery) • Environmental Factors: It covers events or variables that are outside of the process but can still affect its performance. For examples the unemployment rate can impact absenteeism, turnover and the length of time it takes to hire new employees. • Resources in: It covers resources consumed in the process such as materials, labour and energy. • Product Output Quantity: The amount of product produced. • Product Output Quality: The quality of products and services relative to the customer’s requirements. • Output Procedural Quality: Procedural quality problems passed on the subsequent customers. • Productivity: It isn’t a tangible output of a process but it must be measure to provide complete picture of performance. • Waste: Waste is any resource consumed that does not add value to the product. No process is 100% efficient so it generates some waste. Scrap and rework are common forms of waste. Idle time of equipment and people, overproduction, excess motion, excess capacity and unnecessary movement of materials or people are also forms of waste. Above process model provides a good coverage for the performance measurement. However it is too complicated to be used in the categorisation. The following categorisation scheme is proposed to measure performance of a process: Process
Efficiency
Quality
Capacity
Productivity Resources In Waste
Product Output Quality Work Input Quality Output Procedural Quality Input Procedural Quality
Work Input Quantity Product Output Quantity
Figure 9. Sub-categorisation for KPIs related to processes
Page 14
Draft
Edition: 0.1
Introduction to KPI Models
4.
AIM/AEP/S-LEV/0004
KPI CORRELATION In this chapter the issues related to KPI correlation are covered. There are three possible contexts where KPIs can be correlated: • Organisational Hierarchy • Process Hierarchy • Semantically The context to be used depends on the structure of organisation and KPIs. However it should be noted that semantic correlation needs a clear set of strategic objectives.
4.1
Organisational Hierarchy There are different possibilities to correlate KPIs depending on the angle that we are looking at. First correlation is closely related to the organisation hierarchy. There are two sets of more-or-less continuous data flows required in a performance measurement system. These two information flows are illustrated in the following diagram using a notional organisation chart [7]: Organisation Level Strategic Goals, KPIs, Targets
Organisation Level Information Analysis and Evaluation
Strategic Decision Making
Director
Department
Branch Figure 10. Information Flow in a Performance Measurement System
In the above figure, the top-level strategic goals, KPIs and targets are in blue and they flow downward from the strategic goals. The KPI measurements, in red, are collected starting at the bottom (branch level) and flow upward. At each level of the organisational hierarchy, the data are aggregated across the lower levels.
Edition: 0.1
Draft
Page 15
AIM/AEP/S-LEV/0004
Introduction to KPI Models
Aggregation serves to reduce information overload. Periodically, measurements are collected, aggregated and analysed at each management level. These data provide the managers with knowledge of performance within their own part of the organisation. In other words, the performance evaluations are not only for the toplevel managers. Each manager, at each level, benefits by seeing the same KPIs as they apply to his or her own area of responsibility. 4.1.1
Downward information flow Line managers at the directorate, department and Branch levels define goals, desired outcomes, initiatives, KPIs, targets, and schedules. The goals, KPIs, targets and schedules are aligned with those specified in the top-level strategic plan. Some of these parameters may have to be translated from general to more missionspecific to apply to the work being performed at each organisational level. Specific desired outcomes and initiatives to attain them are developed by these managers, and the metrics, targets and schedules are then developed. This results in a hierarchical set of KPIs that are pertinent at each organisational level. It also has the effect of giving the organisational stakeholders a meaningful role in performance evaluation.
4.1.2
Upward information flow At each organisational level collection methods for each of the KPI must be defined. This is the most expensive, labour-intensive, aspect of a performance measurement system. Furthermore it has the most impact on the rank-and-file employees. KPIs are aggregated without filtering or loss of data and reports at each line management layer are created from KPI analysis. These will provide feedback to the managers on KPIs that are pertinent to their own interests. At the senior management layer, KPIs are combined into the vital few needed to give an overall picture of the organisation, on a continuous basis. As a summary the information flow implies that strategic goals, KPIs and targets have an effect on the operational control and operational control provides necessary feedback for the strategic control:
Strategic emphasis
Operational emphasis Organisational Hierarchy
Figure 11. Strategic Control versus Operational Control
Page 16
Draft
Edition: 0.1
Introduction to KPI Models
4.2
AIM/AEP/S-LEV/0004
Process Hierarchy The second correlation comes from the process hierarchy. This correlation depends on the fact that a process may be divided into sub-processes and/or activities.
External Customers
Quality Factors Organisational Requirements & Strategic Goals
Key Deliverables
Organisation KPIs
Principal Processes
Output Level 1 – KPIS
Internal Processes Level 2 - KPIs
Diagnotic Measures Level 3 +… N Figure 12. KPI Measurement Sequence
As in the organisational view, the system (organisation) KPIs are derived from customer requirements, organisational requirements and strategic goals. Then KPIs are extended for principal processes and further for internal processes. As in the organisational hierarchy at each level of the process hierarchy, the data are aggregated across the lower levels in order to create information required for the higher level processes. In case of root-cause analysis the information collected at the lower layers are used to identify the problematic sub-process(es).
Edition: 0.1
Draft
Page 17
AIM/AEP/S-LEV/0004
Introduction to KPI Models
Typical example KPI for the process hierarchy correlation is the “end-to-end response time”. Response time is generally defined as the period of time between the occurrence of an inquiry for information and the delivery of data. With regard to AIS, responsiveness can be described as the period between the end-user's request for certain AIS products and services and the point in time when he receives them. If it is assumed that there is a primary process “Handle Customer Request” and it is divided into the following sub-processes:
Handle Customer Request
Record Request
Collect Required Data
Produce
Print
Deliver
Subprocesses Figure 13. Example Process for Response Time
In order to calculate response time for the primary process “Handle Customer Request” the response times of its sub-processes should be added. Measurement of sub-processes may provide valuable information. For example if the response time of the primary process is very high but it is just due to bad postal services. The only way to recognise it is to measure response time of the sub-process “Deliver”.
4.3
Semantic Correlation The third and last correlation comes from the balanced scorecard (BSC). In the BSC a set of objectives are determined from vision, mission and strategy and these objectives are classified in the financial, customer, business process and learning and growth perspectives. However these objectives can be semantically related to each other. Lets take an example of furniture vendor of which BSC objectives are illustrated in the following figure. The financial objective “increase shareholder value”can be achieved through “reducing average warehouse value”, “reducing cost of administration activities”and “increasing revenue growth”. This is an example that shows how the correlation takes place within same perspective. Similarly objectives in the different perspectives can also be correlated: the financial objective “increase overall market share” can be achieved through the objective “provide popular models” and “be trusted, friendly, convenient and affordable” in the customer perspective. The semantic correlation between the goals implies a correlation between KPIs that are designed to measure these goals.
Page 18
Draft
Edition: 0.1
Introduction to KPI Models
AIM/AEP/S-LEV/0004
VISION We will be the number one furniture vendor in the town MISSION We will make money by providing popular, high quality furniture at affordable prices. We will use our knowledge, creativity and fun spirit to make every customer feel special, and to make sure every customer that leaves our premises have enjoyed shopping with us."
STRATEGY Increase Shareholder Value through Revenue Growth and Increased Productivity
Increase shareholder value
Reduce average warehouse value
Increase revenue growth
Reduce cost of administration
Increase overall market share
Popular models
Be trusted, friendly, convenient and affordable
Friendly and welcoming staff
Reduce number of items in selection
4 weeks selection turn-over max
Quality furniture
Delivery within 24 hrs
Fast credit approval
Professional service and complaint handling
Keep quality complaints at minimum
Deliver right products to right person within 18 hrs
Handle credit request in 10 minutes
Handle all complaints on the spot
All staff members through basic sales training
Financial Perspective
Customer Perspective
Reduce number of administration employees
Business Process Perspective
Learning and Growth Perspective
Figure 14. Objectives for a Furniture Vendor
Edition: 0.1
Draft
Page 19
AIM/AEP/S-LEV/0004
5.
Introduction to KPI Models
IMPROVEMENT MODEL The objective of a performance measurement system is “not only to measure” but also “to use measurements for improvement”. Therefore an improvement model is natural part of any KPI study.
5.1
3-Tier Improvement Model The main objective of a performance measurement system is to provide basis (i.e., facts) for continual improvement: Business Improvement (Achieve the Objective) Performance Measurement System (Define KPIs, Collect and Analyse Data) Technological Know-how (Databases, Intranet, O/S, etc.) Figure 15. 3-Tier Improvement Model
The above improvement model summarises basic skills required to deploy a performance measurement system. At the lower layer technological know-how is required to support data collection and distribute data to the related persons (i.e., reporting). In the middle layer KPIs are defined, collected, monitored and analysed. The top layer, simply the objective, is to improve the business (i.e., product, process or system). However this is not a one-shot approach. There is a continuous cyclical process. This cycle focuses on the internal processes (diagnostic measures) and external outcomes (strategic measures). The system's control is based on KPIs that are tracked continuously over time to look for trends, best and worst practices and areas for improvement. Define KPIs
Refine the KPIS
Monitor KPI Compliance
Improve Product/Service, Process, System
Collect and Analyse Data
Figure 16. Loop in a Performance Measurement System
Page 20
Draft
Edition: 0.1
Introduction to KPI Models
5.2
AIM/AEP/S-LEV/0004
Plan-Do-Check-Action (PDCA) Cycle The loop given in the previous figure illustrates the analysis of KPIs may yield improvement. If there is a need for improvement, the well-known PDCA Cycle1 can be used:
Do
Plan
Check
Action
PDCA Cycle
PLAN
Define the Problem
DO
CHECK
ACTION
Implement a Change
Test the Change
Take Permanent Action
Identify Possible Causes Evaluate Possible Causes
Figure 17. PDCA Cycle
The PDCA cycle has four major steps: 1. PLAN - Plan a change • Define the problem • Identify possible causes • Evaluate possible causes 2. DO - Determine what kind of a change may improve and then implement a change 3. CHECK -Test the change in order to determine whether the change worked. 4. ACTION - If the change worked as expected then take permanent action and embed the change. KPIs provide factual base in identifying possible causes of the problem. However they play another important role during testing the change in order to decide whether there is an improvement after the change.
1
Though there are many elaborated models for improvement, the basic cycle is more-or-less same: Plan-Do-Check-Action
Edition: 0.1
Draft
Page 21
AIM/AEP/S-LEV/0004
6.
Introduction to KPI Models
CONCLUSIONS It may be ambitious and expensive to deploy a performance measurement system in the entire organisation. It may not be possible to afford such large-scale efforts. Therefore it should not be deployed across the organisation all at once. Rather, it should start small, in a business unit and be allowed to develop incrementally. Experience will be gained before organisation-wide deployment is considered. This reduces cost, risk and disruption. KPI category and correlation models given in this document may provide good starting points. The selected set of KPIs should cover at least ISO9001:2000 requirements: system (including customer satisfaction), products and processes. Three correlation models are important in determination of data collection methods (at which level? who will collect? how the collected information will be gathered, etc.) While designing and deploying a performance measurement system the following four points should always be remembered: • What gets measured gets done. • What you measure is what you get. • KPIs only as good as the use made of them (e.g., weighing yourself 10 times a day won't take off the pounds). • Performance measurement is a means not an end.
End of Document
Page 22
Draft
Edition: 0.1
