Test Improvement in Our Rapidly Changing World “Continuous Test Improvement for any Situation” By Martin Pol

[As presented at StarWest2014 Anaheim, USA 10-15 2014]

• Since 2000

• 125 + consultants • 100% test specialist • Netherlands, Amsterdam • Belgium, Brussels • World-wide services www.polteq.com [email protected]

• Test consultancy – – – – –

Process improvement Process implementation Automation Cloud computing Outsourcing support

• Test training • Test R&D • Test projects

The Evolution

Improvement of Improvement Continuous Test Improvement for any Situation

The Evolution of Testing

Pioneering Optimizing

 Structure

Unstructured Undisciplined Flexible

Structuring Structured Disciplined Inflexible

Then

Today

The Evolution of IT and Testing

Prototyping Waterfall Agile/Scrum V-model Spiral model DEVOPS Iterative RAD TDD ATDD BDD W-model XP Continuous Integration/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB Packages SOA Devices Social media Virtualization Cloud Mobile Internet of Things Web Localization Big Data

SPI and TPI

Test Process Software Process Improvement

SPI and TPI

Test Process Improvement Software Process Improvement

SPI and TPI Initiatives

IEEE MIL-STD CMM PSP TSP TPI TI4Agile CMMI TMM TMMI Juran SEI BS 5750 TSM STEP V2M2 TI4Auto CTPI TPI-Next TQC Deming IDEAL ISO9000-9126 TIM TOM STEP CTP Cloutest

Models for Test Improvement • • • • • • •

SQR TOM V2M2 CTP TI4Agile TPI (Next) STEP

• • • • • • •

TSM TIM TI4Auto TMMi CTPI TPS CMMI

Main Issue: Current TPI-models don’t match

Prototyping Waterfall Agile/Scrum V-model Spiral model DEVOPS Iterative RAD TDD ATDD BDD W-model XP Continuous Integration/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB Packages SOA Devices Social media Virtualization Cloud Mobile Internet of Things Web Localization Big Data

And also: • High scores – … but does it really help?

• Great improvement plans – … but poor results

• Solid improvement policy – … low acceptance (people)

• Ivory tower – … resistance Improvement Center

Worse than the ivory tower!

More suitable improvement approach required • Situational Fast

Flexible

– Context Driven

• Continuous • Joint effort • Change proof – Proactive

• …… Lean

Effective

Continuous Test Improvement for any Situation

What helps? • Learning from the past • Using successful “recent” innovations – – – – – – –

Agile Scrum Kanban Exploratory Context Driven Collaborating Continuous

• Good practices

Test Improvement at two levels Test Improvement Intake (Objective – Scope – Approach matching)

Assessment

Continuous Improvement ‘Release’

Continuous Improvement

Assessment tasks Improvement tasks

Improvement backlog

Improvement sprints

Test Improvement using Scrum Test Improvement Intake (Objective – Scope – Approach matching)

Assessment

Improvement Stories

Improvement backlog

Improvement sprints

Continuous Improvement ‘Release’

Test Improvement using Scrum Continuous Improvement ‘Release’

Test Improvement Intake Assessment

(Objective – Scope – Approach matching)

Improvement Stories

Scrum

2-x weeks

Improvement increment

Improvement Backlog

Sprint Backlog

Sprint

DoD DoD

Why Agile/Scrum? • • • • • • •

Small increments Ease of Control Ease of Change Ownership of participants Focus on people ………… Increased use of Agile

Is Agile/Scrum a must? • Don’t forget the implementation

Using IDEAL?

Learning

Acting

Initiating

Diagnosing

Establishing

Learning

Continuous Improvement ‘Release’

Using IDEAL

Acting

Assessment

Scrum

2- X weeks

Initiating

Improvement increment

Test Improvement Intake (Objective – Scope – Approach matching)

Improvement Backlog

Diagnosing

Sprint Backlog

Sprint

DoD

Establishing

Roles

Improvement Architect

Test Improvement Intake (Objective – Scope – Approach matching)

Assessment

Continuous Improvement ‘Release’

Continuous Improvement

Improvement Owner

Assessment tasks Improvement tasks

Improvement backlog

Improvement sprints

Scrum Master

Team members

Test Improvement Intake (Objective – Scope – Approach matching)

Assessment

Continuous Improvement ‘Release’

Continuous Improvement

Assessment tasks Improvement tasks

Improvement backlog

Improvement sprints

Test Improvement Intake • Objectives – – – – – – – –

Time Costs Quality (product) Quality (testing) ………… improve “Agile Testing” ………… improve level of Test Automation ………… prepare for Outsourcing …………

Test Improvement Intake (Objective – Scope – Approach matching)

Test Improvement Intake

Test Improvement Intake (Objective – Scope – Approach matching)

• Objectives – T, C, Q(P), Q(T), …

• Scope – Area of consideration • • • • • •

Organization, department, project, product, … Test level Test type (e.g. security, performance) Agile Testing Cloud services ……………

Test Improvement Intake

Test Improvement Intake (Objective – Scope – Approach matching)

• Objectives – T, C, Q(P), Q(T), …

• Scope – Area of consideration Mark on the Horizon

Test Improvement Intake • Objectives – T, C, Q(P), Q(T), …

• Scope – Area of consideration

• Approach matching – Context – Models – Selection

Test Improvement Intake (Objective – Scope – Approach matching)

Context

Prototyping Waterfall Agile/Scrum V-model Spiral model DEVOPS Iterative RAD TDD ATDD BDD W-model XP Continuous Integration/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB TMap-Next Packages SOA Devices Social media Virtualization Cloud Mobile Internet of Things Web Localization Big Data

Models Bound – – – – – –

TPI Next TMMI STEP CTP GQM …… Tailor-made

– – – – – –

TI4Agile TI4Automation CTPI SFAI-web Belbin ……

Models Unbound – Experience based – Heuristic • Questioning • Discussing

– – – – – –

Implicit knowledge Exploring Idea raising sessions Out-of-the-Box Mind mapping Good Practices

Using heuristics, etc.

How do you feel? Ever felt this before? How about your work? Doing sports? How is your family?

In the morning? When was that? Need to walk? At what level? Simular complaints?

Before breakfast? … … … …

Heuristics for Test Improvement • • • • •

Observation Image of testing History, earlier improvement attempts Discuss potential solutions ………

Idea raising session • Organize a meeting (lean, informal, 3-10 people, < 2 hours, coffee, post-it’s)

• Introduction – Define objectives

– Set the right focus • Gather ideas – Individual “brain dump” – – – –

Use post-it’s Explain, explore, question, discuss Cluster and group ideas, … Use Mind-mapping

• Prioritize according to objectives – E.g. term, costs, impact

• List results, actions, epics, stories, …… – Who, what, why, …

Advantages – – – – –

Efficient 100% focus Invented here! Morale Buy-in

Models Unbound

Bound – – – – – –

TPI Next TMMI STEP CTP GQM …… Tailor-made

– – – – – –

TI4Agile TI4Automation CTPI SFAI-web Belbin ……

– Experience based – Heuristic • Questioning • Discussing

– – – – – –

Implicit knowledge Exploring Idea raising sessions Out-of-the-Box Mind mapping Good Practices

Selection of the Approach • Objectives

Test Improvement Intake (Objective – Scope – Approach matching)

•

Fast

• Scope/Size

•

Flexible

• Resources

•

Lean

•

Effective

• • • • •

Situational Continuous Joint effort Change proof ……

– T, C, Q(P), Q(T), …

– Budget – Time – TI/domain knowledge

• Context • Maturity (best guess) • Culture

Bound

(Tailor-made)?

Assessment

Assess according to the model, but … • Apply Scrum for implementation • Translate recommendations into improvement stories

Improvement backlog

Unbound?

Assessment

Assessment – – – –

Interviews Heuristics Idea raising sessions Study

– – – –

Walk around Test along Attend meetings Feedback and react

• Apply Scrum for implementation • Translate recommendations into improvement stories

Improvement backlog

Combine whatever contributes! Unbound

Bound – – – – – –

TPI Next TMMI STEP CTP GQM …… Tailor-made

– – – – – –

TI4Agile TI4Automation CTPI SFAI-web Belbin ……

– Experience based – Heuristic • Questioning • Discussing

– – – – – –

Implicit knowledge Exploring Idea raising sessions Out-of-the-Box Mind mapping Good Practices

Improvement backlog

Assessment

Improvement Backlog

Continuous Improvement ‘Release’

• Translate Improvement Suggestions into Improvement Stories (or Epics) As senior IT-director, I want to increase test efficiency, so that the testing cost is reduced by 20%

As scrum team member, I want to automate the regression tests, so that the effort for regression testing is reduced As product manager, I want to increase the release frequency, so that we will be more competitive

Improvement Release planning • Divide and prioritize in logical portions for Improvement Backlog – Contribution value (objectives) – Ease of implementation – Dependency

Continuous Improvement ‘Release’

Towards Implementation level Continuous Improvement ‘Release’

Test Improvement Intake Assessment

(Objective – Scope – Approach matching)

Improvement Stories

Scrum

2-x weeks

Improvement increment

Improvement Backlog

Sprint Backlog

Sprint

DoD DoD

Continuous Improvement

Continuous Implementation

Assessment tasks Improvement tasks

Improvement backlog

Improvement sprints

• Refinement and prioritizing

Scrum

2- x weeks

Improvement increment

Improvement Backlog

Sprint Backlog

Sprint

DoD

Continuous Improvement

Continuous Implementation

Assessment tasks Improvement tasks

Improvement backlog

Improvement sprints

• Refinement and prioritizing • Towards tasks, effort and time Assess the level of unit testing Implement a testing dashboard Hire a test automation specialist Investigate open source tools for regression testing

Continuous Improvement

Continuous Implementation • Refinement and prioritizing • Towards tasks, effort and time • Integrated with Business as Usual (BAU) • Competition with BAU •

Assessment tasks Improvement tasks

Improvement backlog

Improvement sprints

Conclusion • Improvement of improvement is required • The situation sets the (hybrid) improvement approach

• Continuous improvement integrates with BAU using Scrum • Tearing down the ivory tower Test Improvement Intake (Objective – Scope – Approach matching)

Assessment

Continuous Improvement ‘Release’

Continuous Improvement

Assessment tasks Improvement tasks

Improvement backlog

Improvement sprints

Thank you!

Martin Pol [email protected] www.polteq.com