Quality Management KPI Metric Determination. February 2013

Quality Management KPI Metric Determination February 2013 Roadmap for KPI Identification (an example) Quality Scope Business Continuity Area Auto...
Author: Jodie Harrison
0 downloads 1 Views 325KB Size
Quality Management KPI Metric Determination February 2013

Roadmap for KPI Identification (an example) Quality Scope

Business Continuity

Area

Automation

Mission

Replace manual procedures

Challenge

Automation in job scheduling management

Metric

# manually planned or scheduled jobs

Target

© 2011 SAP AG. All rights reserved.

KPI

Reduce # manually planned or scheduled jobs by 25% until end of 2013.

2

QM-BC: Areas for Improvement

Area

Mission

Automation

Process Quality

Replace manual procedures

Defined process KPIs

Proactive Support

Adjusted and improved documentation

© 2011 SAP AG. All rights reserved.

Managed Integration

Mainstream new products / technologies

Reduced costs of operations

Efficient Operations

Increase IT Value

3

QM-POI: Areas for Improvement

Area

Solution Transition Quality

Development Management

Security

Project Control

Mission

Solution is maintained up to date

custom development stability

Authorizations maintained

Managed quality gates

Changes are analyzed for impact

Custom development standards

Authorizations reviewed

Integrated partners

Tested business processes

Custom development documentation

Managed access (firewall)

Established project methodology

Master Release Plan

ICC

Communication

Updated footprint

© 2011 SAP AG. All rights reserved.

4

QM-BPI: Areas for Improvement

Area

Mission

Continuous Improvement

Business Alignment

Innovation

Business deliverables

Business feedback

Key Drivers

Quality Awareness

Business signoff

ICC

Communication Plan

© 2011 SAP AG. All rights reserved.

5

QM-SIV: Areas for Improvement

Area

Risk Mitigation

Handover to Operations

Operations Planning

Mission

Integration Validation

Operations readiness

Operations procedures

Risk Awareness

Operations solution review

Documented business processes

Operations skills

© 2011 SAP AG. All rights reserved.

6

Standard Metrics for BC: Automation Area

Mission

Challenge Automation in job scheduling management

Metric Ratio of automated versus manually scheduled jobs (a) Ratio of resource inolvment in job scheduling (m)

Replace manual procedures

Automation in test management

Coverage of business processes in test cases (a)

Automation

Coverage of EEM Proactive Support

© 2011 SAP AG. All rights reserved.

Ratio of automated test cases vs. overall test cases performed (a)

Coverage of Business Process Monitoring

# EEM locations (a)

Ratio of business processes in Business Process Monitoring (a)

7

Standard Metrics for BC: Process Quality Area

Mission

Challenge

Metric Coverage of E2E standards (m)

Process Maturity # of Process maturity assessments (m)

Defined process KPIs

Ratio of KPIs per IT process (m)

Process Improvement

Ratio of IT processes with KPIs (m) Frequency of adjustments of process KPIs (m)

Process Quality

Adjusted and improved documentation

© 2011 SAP AG. All rights reserved.

Change completeness – Updated Solution Documentation and Operation Handbook

% incidents with Priority 1/2 that are caused by missing or wrong solution documentation change or operations handbook (m)

Change completeness - test case adjustment with change request / implementation and transport/ E2E processes

% test cases reviewed or changes (after identification of affected ones) (m)

8

Standard Metrics for BC: Managed Integration Area

Mission

Challenge

New product incidents

Metric

# incidents with priority 1/2 after go-live of new products / technologies within first 3 months (m) Coverage of product trainings sessions for end users (m)

New product skills assessed Managed Integration

Mainstream new products / technologies

Ratio of incidents identified as training issues (m) Coverage of product trainings sessions for IT staff (m)

New organizations reviewed

New tool integration

© 2011 SAP AG. All rights reserved.

# up-to-date skill reviews (m)

# maturity assessments performed (m)

# maturity assessments performed (m)

9

Standard Metrics for BC: Reduced Costs of Operations Area

Reduced costs of operations

Mission

Challenge

Metric

Incident effort reduction

Time for incident resolution (priority 1/2) (a)

Incident solving experts

# experts required to solve critical incidents within customer’s organization (m)

Efficient Operations

IT Value

Reduce TCO

Increase transparency of IT Value

© 2011 SAP AG. All rights reserved.

% TCO vs. revenue (m)

Trend in escalation / Days with an outage (m)

10

Standard Metrics for POI: Solution Transition Quality Area

Mission

Challenge

Solution is maintained up to date

Solution Maintenance

Changes are analyzed for impact

Change impact analysis

Solution Transition Quality

Metric distance from your current release, EhP, and switch (m)

# change impact analysis projects in SAP Solution Manager (a)

.

Tested business processes

Master Release Plan

© 2011 SAP AG. All rights reserved.

Business processes tested

Master Release Plan frequency

# critical business processes steps tested for performance regularly (m) # of MRP updates documented per annum (m) MRP update documented per SAP Release Strategy update (m)

11

Standard Metrics for POI: Development Management Area

Mission

Challenge

Metric % of incidents (prio 1/2 ) caused by custom developments (m)

Custom development stability

Upgrade change impact analysis

Custom code clearing

Development Management

Custom development standards

Custom development documentation

Custom development standards

Custom development documentation

Ratio of completed analysis projects in Upgrade Change Impact Analysis (m) Ratio of completed analysis projects in Custom Code Clearing (m) ratio of incidents caused by custom development (a) Level of clear development standards documented (m) # development projects without review of development standard compliance (m) % custom developments documented according to SAP Standard (Sol. Documentation for Custom Development) (m) % of custom code objects not compliant to documentation standard (m)

ICC

Reduce modification

% of custom code objects reviewed incl. documentation standard (m) % of reduction of unnecessary modifications (m)

© 2011 SAP AG. All rights reserved.

12

Standard Metrics for POI: Security Area

Mission

Challenge Profile changes

Authorizations maintained Security notes

Security

© 2011 SAP AG. All rights reserved.

Authorizations reviewed

Security notes

Managed access (firewall)

Infrastructure assessment

Communication

Alignment Meetings

Updated Footprint

Updated Standards/Policies

Metric Ratio of profile changes for users in one year (m) Distance to implemented security note from released security note (m) # incidents caused by not implemented security notes (m) % security reviews vs new releases (m) # reviews of assigned roles vs. actual requirements (m) # infrastructure assessment per annum (m)

# alignment meetings (m)

Ratio of changes in standards/policies leading to IT reviews (m)

13

Standard Metrics for POI: Project Control Area

Mission

Challenge

Quality Gates Defined Managed quality gates Quality Gate Criteria Project Control

Metric

% projects with quality gates defined (m) % of quality gates passed by 1st attempt (m) # of criteria defined for each quality gate (m) # of quality gates passed by mgmt. decision (m) # incidents caused by ignored quality gates (m)

Integrated partners

Risk Management of Partners

Alignment meetings/ Participation in risk mitigation reviews

Established Project Methodology

Project Methodology aligned

% projects executed according to project methodology (m)

© 2011 SAP AG. All rights reserved.

14

Standard Metrics for BPI: Continuous Improvement Area

Continuous Improvement

Mission

Challenge

Metric

Usability - Enhancements

# usability enhancements resulting from Business Feedback (m)

Usability - Problems

# review of usability with users (m)

Business deliverables Functionality - Assessments

# SMA per major release (m)

Trend: # prototypes

Functionality - Prototyping Ratio of prototypes built in sandbox systems to implemented ones in Business (m)

© 2011 SAP AG. All rights reserved.

15

Standard Metrics for BPI: Continuous Improvement Area

Mission

Challenge

Establish Quality Roles

Continuous Improvement

Metric

% Completeness of Quality Challenges (m)

Quality Awareness

Execution of Quality Plan

Measurement frequency of KPIs (m)

Continuous Learning

# Lessons Learned performed with Business (m) # Success Stories with Business communicated in the organization (m)

© 2011 SAP AG. All rights reserved.

16

Standard Metrics for BPI: Business Alignement Area

Mission

Challenge

Metric # offered (service catalog) IT-services

Service Catalog % usage of offered services (m)

Business feedback

# alignment meetings between Business & IT (m)

Alignment meetings

# action items from alignment meetings between Business & IT (m) average duration until completion of action items (m)

Business Alignement

Business testing

% test cases performed by Business (m)

Business change acceptance

% change requests confirmed/accepted by Business (m)

Business signoff

Communication Plan

© 2011 SAP AG. All rights reserved.

Informed business

# newsletters /events o inform business users (m)

17

Standard Metrics for BPI: Innovation

Time for completion of Rapid prototyping (m)

Innovation

Key Drivers

Speed to Respond to market opportunity

# projects resulting from Rapid prototyping (m)

Business Feedback rating (m)

ICC

© 2011 SAP AG. All rights reserved.

Reduce modification

% of reduction of unnecessary modifications (m)

18

Standard Metrics for SIV: Risk Mitigation Area

Mission

Challenge Integration test without critical issues

Integration Validation

Risk Mitigation

Volume/load test without critical issues

Metric # of new issues of type functional/technical identified during integration test phase (m)

# of issues of type *…* identified during performance test phase (m)

Reduced duration of stabilization period

Days after go live until stabilization/hypercare is no longer required (m)

Minimize costs for stabilization

Stabilization by add. Resources/incidents or end user (m)

Risk Awareness

Documented business processes

© 2011 SAP AG. All rights reserved.

Risk tracking

Frequency of risk reviews (m)

Completeness of solution documentation

% critical business processes documented

19

Standard Metrics for SIV: Handover to Operations Area

Handover to Operations

Mission

Operations readiness

Challenge

Completeness of solution operations documentation

% of changes of operation handbook after GoLive

SignOff

% of completion SignOffs by Business

Sufficient skills

Monitoring procedures established

© 2011 SAP AG. All rights reserved.

Metric

% of training schedules performed

Coverage of automated monitoring for critical business processes

20

Standard Metrics for SIV: Operations Planning Area

Mission

Challenge

Metric Ratio of business processes in Business Process Monitoring (m)

Business Process Monitoring Operations procedures

Operations Planning

Operations solution review

Operations skills

# business processes for which monitoring objects have been defined and instrumented (m)

Job Scheduling

# jobs for which monitoring objects have been defined and instrumented (m)

Operations review

# operations reviews for solutions to be implemented (SMA approach) (m)

Operations trainings

# operations handbook training / skills transfer for new topics (m) % of employees with proven skills (m)

© 2011 SAP AG. All rights reserved.

21