Experiences in Root Cause Analysis and Defect Prevention Methods

Experiences in Root Cause Analysis and Defect Prevention Methods Kelly L. Lanier Raytheon Network Centric Systems [email protected] Copyright 2003...
Author: Lee Goodwin
1 downloads 0 Views 1MB Size
Experiences in Root Cause Analysis and Defect Prevention Methods Kelly L. Lanier Raytheon Network Centric Systems [email protected]

Copyright 2003 -- Raytheon Company,

Case for Action Deliver Quality and Reduce Costly Rework! Six Sigma Specialist Project that focused in reducing escaping defects. Design defects that escape to Integration are 10 times more expensive than if they were caught in the Design stage. Another thing to note is that one of the biggest problems we have found with cost and schedule is in integration. Copyright 2004 -- Raytheon Company,

Output of a R6S Project Characterization

Set the Foundation Define and agree to the the following: – Defect A flaw or imperfection that results in incorrect software. A defect may or may not be detected during software use. A deficiency which has the potential of producing incorrect response or undesired effect.

– Comment Type Assignment, checking, performance, etc.

– Comment Priority 1 through 5

Copyright 2004 -- Raytheon Company,

When to Count a Defect Count a defect if: – The requirements documentation could lead to incorrect source code By being incorrect Or by leading to incorrect design Etc.

– The design documentation could lead to incorrect source code By being incorrect itself Or by being easy to misunderstand

– The source code is incorrect Copyright 2004 -- Raytheon Company,

Defect Analysis and Prevention Defect Analysis is the process of analyzing a defect to determine its root cause. Defect Prevention is the process of addressing root causes of defects to prevent their future occurrence.

Copyright 2004 -- Raytheon Company,

Process Focus Defect Containment focus

Defect Prevention focus – Preventing the occurrence of an individual defect or group of defects Copyright 2004 -- Raytheon Company,

Requirements

Stage Detected

– Finding defects in the stage they were introduced and as early in the lifecycle as possible – Eliminating escaping defects

Stage Originated

Design

Code & Test

SW Integration

Qual Test

SYS Integration Post-Release

Totals

Requirements

3

Design

1

15

2

10

45

1

2

12

20

Qual Test SYS Integration

0

0

5

0

5

2

3

2

2

0

2

Post-Release

0

1

2

0

0

0

5

8

Totals

9

31

66

22

5

2

5

140

Code & Test SW Integration

3 16 57 35 10

Focus on Defect Containment

AND Defect Prevention

11

Defect Analysis and Prevention Process Defect Analysis and Prevention

Select Defects for Further Analysis CCBs, Peer Reviews, Defect Containment Metric Pareto Charts

Re-evaluate

Select Defects

Monitor Against Planned Measurement Continuous Improvement

Copyright 2004 -- Raytheon Company,

Implement Action Plan

Get Detailed Defect Data from Defect Logger and Synergy

Develop Action Plan and Measurement for Success

Perform Causal Analysis

Identify Root Causes and Solutions

Submit Improvement to Organization

Identify and Prioritize Improvement Opportunities

Next Priority

Select Defects for Further Analysis CCBs, Peer Reviews, Defect Containment Metric Pareto Charts

Select Defects

Select Defects for Further Analysis

The program metrics analysis team regularly reviews defect data to determine if defect analysis is necessary – Defect metric has exceeded threshold – Defect data shows a trend – Individual defect is flagged for analysis

Copyright 2004 -- Raytheon Company,

Defect Analysis and Prevention

Get Detailed Defect Data from Defect Logger and Synergy

Perform Causal Analysis

Analysis Tools

Identify Root Causes and Solutions

Pareto

http://homext.ray.com/sixsigma http://homext.ray.com/sixsigma Five Whys

????? Fishbone

Defect Containment Matrix Stage Originated

Stage Detected

Requirements

Design

Code & Test

SW Integration

Requirements

3

Design

1

15

2

10

45

1

2

12

20

Code & Test SW Integration

Qual Test

SYS Integration Post-Release

Totals 3 16 57 35

Qual Test SYS Integration

0

0

5

0

5

2

3

2

2

0

2

Post-Release

0

1

2

0

0

0

5

8

Totals

9

31

66

22

5

2

5

140

Copyright 2004 -- Raytheon Company,

10 11

Monitor Against Planned Measurement

Implement Action Plan

Develop Action Plan and Measurement for Success

Submit Improvement to Organization

Continuous Improvement

Identify and Prioritize Improvement Opportunities

Next Priority

Continuous Improvement (1 of 3)

Identify and prioritize improvement opportunities based on the potential solutions to defect root cause Submit improvement opportunities to the organization via the Organizational Improvement Website

Copyright 2004 -- Raytheon Company,

Monitor Against Planned Measurement

Implement Action Plan

Develop Action Plan and Measurement for Success

Submit Improvement to Organization

Continuous Improvement

Identify and Prioritize Improvement Opportunities

Next Priority

Continuous Improvement (2 of 3)

Organizational Improvement Website – A proposal is a request to have the SWEC organization evaluate, select, and adopt a recommended improvement and to provide a funding and evaluation path for the proposal. – An advisory is an informational message to the SWEC organization advising that a program is providing a funding and evaluation path for an improved process, method, tool, technology, etc to support its own business/project goals. However, this improvement may have relevance to the organization at large and have strategic importance to the enterprise Copyright 2004 -- Raytheon Company,

Monitor Against Planned Measurement

Implement Action Plan

Develop Action Plan and Measurement for Success

Submit Improvement to Organization

Continuous Improvement

Identify and Prioritize Improvement Opportunities

Next Priority

Continuous Improvement (3 of 3)

Work as a Six Sigma Project – Create an action plan for the improvement Include how to measure success of the improvement – Implement the action plan – Monitor the progress of the action plan Track progress for measurement of success – Communicate the results to the program & organization

Copyright 2004 -- Raytheon Company,

Monitor Against Planned Measurement

Implement Action Plan

Develop Action Plan and Measurement for Success

Submit Improvement to Organization

Continuous Improvement

Identify and Prioritize Improvement Opportunities

Next Priority

Defect Prevention

Examples – – – – –

New checklists Modified processes Updated plans Scheduled training Tool support

Copyright 2004 -- Raytheon Company,

Monitor Against Planned Measurement

Implement Action Plan

Develop Action Plan and Measurement for Success

Submit Improvement to Organization

Continuous Improvement

Identify and Prioritize Improvement Opportunities

Next Priority

Defect Prevention

Check to mark a defect

Check to analyze

Copyright 2004 -- Raytheon Company,

Organization Defect Analysis and Prevention Process Defect Analysis and Prevention

Select Defects for Further Analysis Out of Threshold Org Defect Metric, Quarterly Process Capability Analysis

Get Detailed Program Defect Data

Re-evaluate

Select Defects

Level 5 Behaviors Monitor

Institutionalize

Experiment

Perform Causal Analysis

Identify Root Causes and Solutions

Identify and Prioritize Improvement Opportunities

Submit Improvement Proposal

Next Priority Continuous Improvement Copyright 2004 -- Raytheon Company,

Program Defect Analysis and Prevention

Results SW EC Defect Containm ent T rend 100% 90%

84%

81%

80% 70% 60%

Upper DROV

Copyright 2004 -- Raytheon Company,

Nom inal value

Lower DROV

4Q03

3Q03

2Q03

1Q03

4Q02

3Q02

2Q02

1Q02

Dec 01

1 Sep 0

Jul 0 1

May 01

1 Apr 0

Mar 01

Feb 0

1

50%

M ean P erform anc e

Defect Density SWEC Defect Density Trend

Upper DROV

Nominal value

Lower DROV

4Q03

3Q03

2Q03

1Q03

4Q02

3Q02

2Q02

1Q02

CMMI Level 5 September 2003

Jul 0 1 Sep 01 Dec 01

01 May

Apr 0 1

Mar 01

Feb

01

CMM Level 4 June 2001

Mean Performance

Improved Defect Density by 44 percentage points, and reduced variation by 31% Copyright 2004 -- Raytheon Company,

Summary Out of phase defects cause expensive rework Use common definitions and counting approach Focus on defect detection and prevention Analyze metrics at least monthly using R6 tools Prioritize and implement defect prevention activities Promote lessons learned to the organization

Copyright 2004 -- Raytheon Company,