Using the ISBSG Database in an Agile Project Context Customer Orientation

Lean Six Sigma

Agile Processes

Thomas M. Fehlmann, Zürich, Euro Project Office AG E: [email protected] – H: www.e-p-o.com

Project Estimations

Transfer Functions

3°International Conference on IT Data collection, Analysis and Benchmarking Florence (Italy) - October 19, 2015 1

Dr. Thomas Fehlmann Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

l l l l l l l l l l

1981: 1991: 1999: 2001: 2003: 2004: 2007: 2011: 2012: 2013:

Dr. Math. ETHZ Six Sigma for Software Black Belt Euro Project Office AG, Zürich Akao Price 2001 for original contributions to QFD SwissICT Expert for Software Metrics, ICTscope.ch Member of the Board QFD Institute Deutschland – QFD Architect CMMI for Software – Level 4 & 5 Net Promoter® Promoter® Certified Certified Associate Associate Member of the DASMA Board Vice-President ISBSG

Transfer Functions

2

Predictive Projects = Waterfall? Definition

Customer Orientation

l Traditional project development has become the “enemy” called Lean Six Sigma

Agile Processes

Project Estimations

Waterfall

Analysis Design

Implement

l The inspector stays on the bridge crossing the waterfall and assesses progress

Install

l Who’s afraid from the Big Bad Inspector???!?

Transfer Functions

Operate 3

The Wonderful New World of Agile l Twelve Principles of Agile Software 1. Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. 2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 4. Business people and developers must work together daily throughout the project. 5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 7. Working software is the primary measure of progress. 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 9. Continuous attention to technical excellence and good design enhances agility. 10. Simplicity – the art of maximizing the amount of work not done – is essential. 11. The best architectures, requirements, and designs emerge from selforganizing teams. 12. At regular intervals, the team reflects on how to become more effective, 4 then tunes and adjusts its behavior accordingly.

Scrum Development Cycle Customer Orientation

fail

Independent Testing

pass

Lean Six Sigma

24h Agile Processes

10 day Project Estimations

Transfer Functions

Product Backlog “User Stories”

Sprint Backlog “Work Items”

Sprint

Work Increment of finished software 5

Why Predictive ≠ Waterfall Customer Orientation

l What do you need to predict? Functionality? No…, it’s evolving Cost, Yes!

Lean Six Sigma

39 Entry (E) + 45 eXit (X) + 12 Read (R) + 14 Write (W) = 110 CFP Get Certifcate

Traveler

Get SIM Certificate

Mobile Phone

SIM Card

SIM Certificates

Telecom User Management

Payment

User ID

1.// Request Certificate fail

pass

2.// SIM Certificate

Independent Testing

3.// Identification

Agile Processes

4.// Telecom Clearance 24h

5.// No Valid Certificate 6.// Phone Connected?

7.// Device ID 10 day

Project Estimations

8.// Ask for Subscriber ID 9.// Return Subscriber ID 10.// Read Subscriber ID Product Backlog “User Stories”

Sprint Backlog “Work Items”

Sprint

Work Increment of finished software

11.// Read Device ID 12.// Ask for PIN Code

Transfer Functions

13.// Enter PIN Code

l How many Sprints are needed?

14.// Ask for clearance

6

Predict Cost! Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

l You need to know

l In what units?

Velocity

Functional Size per Sprint

Team Size

Person Days per Sprint

Sprints Needed

Number of Sprints

l Cost = Team Size * Sprints needed * Salaries

l Vision & Goals? When are Goals Reached?

Transfer Functions

How do we know the Project is Finished? 7

Definitions Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

l ISBSG

= International Software Benchmarking Standards Group

l FSU

= Functional Size Units (IFPUG, COSMIC, etc.)

CFP IFP

= COSMIC Function Points = IFPUG Function Points

l PDR

= Project Delivery Rate in Hours per FSU

l StP

= Story Points, as defined by the team (project-specific)

l sPDR

= Story Point Delivery Rate: the number of story points per FSU

l PD

= Person Days; 1 day = 8 hours

l PWE

= Project Work Effort, measured in Sprints, or sometime still in PDs

Includes Definition, Analysis, Design, Implementation, Test and Installation as well as project management and project administration loads

l Velocity = FSU delivered per Sprint 8

The ISBSG Data Portal – Entrance Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

9

The ISBSG Data Portal – Search for Projects Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

10

The ISBSG Data Portal – Search Results Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

11

The ISBSG Data Portal – Project Details Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

4.25

Transfer Functions 4.25

12

Set Vision and Goals: Sample Helpdesk Project Customer Orientation

Lean Six Sigma

l A Transportation Company asked customers what they expect from a helpdesk Net Promoter Score analysis unveiled five major business drivers responsible for attractiveness of using the helpdesk in case of any service interruption or difficulties Improvements are dearly needed… Involving Software as well as Services

Agile Processes

Business Drivers Project Estimations

Transfer Functions

a) Helpdesk BD1 BD2 BD3 b) Software BD4 BD5

Topics Responsiveness Be Compelling Friendliness Personalization Competence

Attributes Fast Response Able to resolve issue Keeps calm Knows who's calling Can figure out solution

Few Selections Committed to help Unagitated Knows travel plans According my preferences

Known Issue Knowledgeable Cool Knows delays

Priority 0.34 0.44 0.41 0.64 0.33

13

Functional User Requirements – Vision Count 39 Entry (E) + 45 eXit (X) + 12 Read (R) + 14 Write (W) = 110 CFP Customer Orientation

Get Certifcate

Traveler

Get SIM Certificate

Mobile Phone

SIM Card

SIM Certificates

1.// Request Certificate 2.// SIM Certificate

Lean Six Sigma

110 CFP

Telecom User Management

User ID

3.// Identification

Agile Processes

This extract shows how Telecom provides Identity Services for Travelers

4.// Telecom Clearance 5.// No Valid Certificate 6.// Phone Connected?

7.// Device ID Project Estimations

8.// Ask for Subscriber ID 9.// Return Subscriber ID 10.// Read Subscriber ID

Transfer Functions

11.// Read Device ID 12.// Ask for PIN Code 13.// Enter PIN Code

14

Is that the Whole Story? Customer Orientation

Lean Six Sigma

Agile Processes

l Functional Requirements (FUR)

l Non-functional Requirements (NFR)

Represented by initial sequence diagram – or any other kind of Epics

NFR describe Business Goals indeed

Cost per Function Point known

Most NFR become functional over time!

FUR do not describe business goals in project

No Database available for Benchmarking

Project Estimations

Transfer Functions

15

Non-Functional Requirements become Functional over Time Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

l In Agile, the transformation of NFR to FUR happens within each Sprint Functional Size growths during the Project NFR can be assessed before project start Initial FUR count is the project vision Predictive counts require NFR!

Size by analogy or expert judgement Approximate FUR

Outline FUR

Precise COSMIC size measurement

Approx. COSMIC size measurement Detailed FUR

Software

Implemented System

True NFR

Outline NFR

System Project

Requirements Analysis

Design & Definition

Hardware

Build, Test & Implement

Source: COSMIC Guideline on Non-Functional & Project Requirements and Charles Symons 2014

Other deliverables

16

Initial Buglione-Trudel Matrix Business Drivers Customer Orientation

Story Cards

Business Driver Goal Profile / Achieved Response Profile

BD1: Responsiveness BD2: Be Compelling

Lean Six Sigma

BD3: Friendliness BD4: Personalization

Agile Processes

BD5: Competence

Convergence Gap

User Story Priorities

0.23

Project Estimations

Transfer Functions

Functional Effectiveness (ISO/IEC 19761)

Functional Story Cards

0.02

User Stories

Q001 Customer Story

Q002 Helpdesk Story

Q003 Enrollment Story

Q004 Social Story

Q005 Get SIM Certificate

17

Functional and Non-Functional Story Cards Story Card for Helpdesk Story Customer Orientation

Lean Six Sigma

Q002-02F: Get Additional Information

Story Points:

3

Name: Peter

Functional Size:

2

Sprint:

Test is Draft is Review Ready Ready Done

Finalized

Approved

Functional

#03 - Adagio

Business Impact:

As a Helpdesk staff, I want to identify a client without having to ask for the name or get credentials, regardless whether calling by phone, sending email, or when chatting, such that I can charge service fees or ticket sold according the clients’ preferred payment method, so that in case of urgency I can help more customers faster and more effectively

Traveler

Helpdesk Story 35.// Respond to Traveler

36.// Traveler explains Situation

Contact traveler and ask for additional information about his or her travel needs

Agile Processes

Story Card for Helpdesk Story Q002-09Q: Apologies

Project Estimations

Story Points:

8

Name: Heidi

Functional Size:

0

Sprint:

Business Impact:

BD2: 1

Test is Draft is Review Ready Ready Done

Finalized

Approved

Functional

#04 - Rondo BD4: 6

As a Helpdesk staff, I want to identify a client without having to ask for the name or get credentials, regardless whether calling by phone, sending email, or when chatting, such that I can charge service fees or ticket sold according the clients’ preferred payment method, so that in case of urgency I can help more customers faster and more effectively Find a way to express apologies in a personal way

Transfer Functions

18

Final Buglione-Trudel Matrix Business Drivers Customer Orientation

Story Cards

Business Driver Goal Profile / Achieved Response Profile

BD1: Responsiveness BD2: Be Compelling

Lean Six Sigma

BD3: Friendliness BD4: Personalization

Agile Processes

BD5: Competence

Convergence Gap

User Story Priorities

0.03

Project Estimations

Transfer Functions

Functional Effectiveness (ISO/IEC 19761)

Functional Story Cards

0.02

User Stories

Q001 Customer Story

Q002 Helpdesk Story

Q003 Enrollment Story

Q004 Social Story

Q005 Get SIM Certificate

19

Business Impact Cards become Functional Customer Orientation

Lean Six Sigma

l The “Apologies” Story Card becomes functional after involving the Customer Experience and the Legal department Adding 9 CFP to the 110 CFP already agreed for basic functionality Happens within the sprint “#04 – Rondo” Unplanned, additional functionality resulting from an NFR

Agile Processes

Story Card for Helpdesk Story Q002-09Q: Apologies

Project Estimations

Story Points:

8

Name: Heidi

Functional Size:

0

Sprint:

Business Impact:

BD2: 1

Test is Draft is Review Ready Ready Done

Finalized

Approved

Functional

#04 - Rondo BD4: 6

As a Helpdesk staff, I want to identify a client without having to ask for the name or get credentials, regardless whether calling by phone, sending email, or when chatting, such that I can charge service fees or ticket sold according the clients’ preferred payment method, so that in case of urgency I can help more customers faster and more effectively Find a way to express apologies in a personal way

Transfer Functions

20

Initial and Final Metrics for Goals Reached

0.34

BD2 Be Compelling

0.44

BD3 Friendliness

0.41

BD4 Personalization

0.64

BD5 Competence

0.33

3 6 6 6 3

0.29

BD1 Responsiveness

0.34

0.52

BD2 Be Compelling

0.44

0.56

BD3 Friendliness

0.41

0.51

BD4 Personalization

0.64

0.28

BD5 Competence

0.33

Solution Profile for User Stories 0.74 0.53 0.10 0.33 0.22 Convergence Gap 0.75

0.10 Convergence Range 0.20 Convergence Limit

0.56

0.09

0.31

0.17

4 4 3 3 3 3 6 9 6 6 1 6 6

Achieved Profile

Q005 Get SIM Certificate

7 7 6 6 3

Q004 Social Story

Business Drivers

Q003 Enrollment Story

Goal Profile

Achieved Profile

Q005 Get SIM Certificate

Q004 Social Story

4 4 3 3 3 3 6 6 6

User Stories

Deployment Combinator

Q002 Helpdesk Story

Transfer Functions

BD1 Responsiveness

Q003 Enrollment Story

Business Drivers

Project Estimations

Business Drivers

User Stories

Q002 Helpdesk Story

Agile Processes

l Final Coverage of Business Drivers

Q001 Customer Story

Business Drivers Deployment Combinator

Q001 Customer Story

Lean Six Sigma

l Initial Coverage of Business Drivers

Goal Profile

Customer Orientation

0.33 0.44 0.42 0.66 0.30

Solution Profile for User Stories 0.65 0.52 0.27 0.42 0.25 Convergence Gap

0.23

0.65

0.10 Convergence Range 0.20 Convergence Limit

0.51

0.26

0.42

0.26

0.03 21

How predicting Cost of “Non-Functional” Requirements? User Stories

Agile Processes

Project Estimations

Business Drivers

Collect the Experts

BD1 Responsiveness

0.34

• Business • Developers

BD2 Be Compelling

0.44

BD3 Friendliness

0.41

Agree on a value 1 – 3 – 9 for each cell indicating how much the User Story impacts the respective Business Driver

BD4 Personalization

0.64

BD5 Competence

0.33

Make sure the Convergence Gap closes

9 9 3 3 3

1 3 3 3 3 9 9 3 9 1 1 3 3 3

Achieved Profile

Q005 Get SIM Certificate

• Ideally by Net Promoter Score

Q004 Social Story

Goal Profile

Get the Business Driver’s Goal Profile

Q003 Enrollment Story

Cost Prediction

Find a QFD Moderator Lean Six Sigma

User Stories

Q002 Helpdesk Story

l QFD Workshop

Q001 Customer Story

Customer Orientation

0.35 0.44 0.42 0.65 0.30

Solution Profile for User Stories 0.63 0.51 0.23 0.45 0.30 Convergence Gap

0.6 0.10 Convergence Range 0.20 Convergence Limit

0.5

0.2

0.4

0.3

0.03 Business Impact

81

Transfer Functions

22

Early Estimation by a QFD Workshop

Transfer Functions

BD1 Responsiveness

0.34

BD2 Be Compelling

0.44

BD3 Friendliness

0.41

BD4 Personalization

0.64

BD5 Competence

0.33

0.35

BD1 Responsiveness

0.34

0.44

BD2 Be Compelling

0.44

0.42

BD3 Friendliness

0.41

0.65

BD4 Personalization

0.64

0.30

BD5 Competence

0.33

Solution Profile for User Stories 0.63 0.51 0.23 0.45 0.30 Convergence Gap

0.6 0.10 Convergence Range 0.20 Convergence Limit

0.5

0.2

0.4

0.3

81

Achieved Profile

4 4 3 3 3 3 6 9 6 6 1 6 6

0.33 0.44 0.42 0.66 0.30

Solution Profile for User Stories 0.65 0.52 0.27 0.42 0.25 Convergence Gap

0.03 Business Impact

7 7 6 6 3

Q004 Social Story

Business Drivers

Q003 Enrollment Story

Goal Profile

Q004 Social Story

1 3 3 3 3 9 9 3 9 1 1 3 3 3

Achieved Profile

Q005 Get SIM Certificate

Q003 Enrollment Story

9 9 3 3 3

User Stories

Deployment Combinator

Q002 Helpdesk Story

Project Estimations

Business Drivers

Q002 Helpdesk Story

Goal Profile Agile Processes

Business Drivers

User Stories

Cost Prediction

Q001 Customer Story

User Stories Lean Six Sigma

l Final Convergence Gap after a few sprints Q005 Get SIM Certificate

l Result of QFD Workshop

Q001 Customer Story

Customer Orientation

0.65

0.10 Convergence Range 0.20 Convergence Limit

0.51

0.26

0.42

0.26

0.03 Business Impact

89 23

Predict Total Functionality Needed and thus Effort Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

l The Idea: Do a QFD Workshop and count the total amount of correlation values in the matrix

l FUR count yields 110 CFP

Assume 4.25 PDR (h/CFP) Story Point Delivery Rate 110 * 4.25 = 468 h

l 1 CFP = 2.0 Story Points l 1 Story Point = 1 Impact l NFR total for 81 Impacts Therefore NFR add 81 * 2.0 = 162 CFP

This yields 4.25 * 162  689 h

l Total 468 h + 689 h = 1156 h

24

Predict Total Functionality Needed and thus Cost User Stories

0.44

Friendliness

0.41

BD4

Personalization

0.64

BD5

Competence

0.33

1 3 3 9 1

Solution Profile for User Stories 0.63

0.6

Q003 Enrollment Story

9 9 3 3 3

Q005 Get SIM Certificate

0.34

Be Compelling

BD3

Q004 Social Story

l The Idea:

Responsiveness

BD2

Q002 Helpdesk Story

Goal Profile

Customer Orientation

BD1

Q001 Customer Story

Business Drivers

3 9 3

9 1 3

Achieved Profile

User Stories

Cost Prediction

0.35

3

0.44

3 3

0.42 0.65 0.30

0.51 0.23 0.45 0.30 Convergence Gap 0.5 0.2 0.4 0.3 0.03

0.10 Convergence Range 0.20 Convergence Limit

Business Impact

81

Do a QFD Workshop and count the total amount of correlation values in the matrix Lean Six Sigma

Somehow, they will correlate to the amount of functionality added by tasks imposed by customer’s needs, to satisfy the business driver’s goal profile 39 Entry (E) + 45 eXit (X) + 12 Read (R) + 14 Write (W) = 110 CFP

Get Certifcate

Traveler

Basic Functionality is counted the usual way using a standard Functional Sizing Method Agile Processes

Get SIM Certificate

Mobile Phone

SIM Card

SIM Certificates

Telecom User Management

User ID

1.// Request Certificate 2.// SIM Certificate

3.// Identification 4.// Telecom Clearance 5.// No Valid Certificate 6.// Phone Connected?

7.// Device ID 8.// Ask for Subscriber ID

l The Problem:

9.// Return Subscriber ID 10.// Read Subscriber ID

11.// Read Device ID 12.// Ask for PIN Code 13.// Enter PIN Code

How does work for business impact compare with PDR for functionality?

14.// Ask for clearance

Project Estimations

l The Solution: Transfer Functions

Use the Team’s Story Points to “translate” Business Impact into FUR Calibrate pure functional Story Cards with pure Impact Story Cards 25

Payment

Calibrating with Story Points Customer Orientation

Story Points by the Team

Lean Six Sigma

Agile Processes

Project Estimations

FUR for Customer Needs

Business Impact NFR

ISBSG Benchmark

Total Estimate €

Transfer Functions

COSMIC Functional Size

26

Conclusion Customer Orientation

Lean Six Sigma

l Effort Estimation without knowing the Customer’s Needs is only incidentally correct Simply because all “Change Requests” originate from customer’s need and the product’s business drivers

l Embracing Agile seriously allows predicting project cost much better and simpler than traditional cost estimation techniques Agile Processes

By predicting the agile software development process Not the product!

Project Estimations

l It’s still benchmarking but at much better granularity, taking the team and the environment into account

Transfer Functions

27

Questions? Customer Orientation

Lean Six Sigma

Agile Processes

Project Estimations

Transfer Functions

28

Controlling Variations CN  VoC Customer Orientation

FUR LT

Voice of the Customer (VoC)

Competitive Analysis (LT)

#NPS, #Kano

#Market Share

Lean Six Sigma

UC BD Agile Processes

Business Drivers (BD)

UC FUR Functional User Requirement (FUR)

Project Estimations

Non-Functional Transfer Functions

Functional User Stories (UC) #Priority

29