Successful Introduction of a Practical Test Strategy

Europe’s Premier Software Testing Event Stockholmsmässan, Sweden “Testing For Real, Testing For Now” Successful Introduction of a Practical Test Str...
Author: Philip Berry
3 downloads 4 Views 10MB Size
Europe’s Premier Software Testing Event Stockholmsmässan, Sweden

“Testing For Real, Testing For Now”

Successful Introduction of a Practical Test Strategy Jurian van de Laar, Improve Quality Services, The Netherlands WWW.EUROSTARCONFERENCES.COM

A practical Test Strategy Jurian van de Laar [email protected] EuroSTAR Conference 2009 Stockholm

Improve Quality Services Accredited provider ISTQB Foundation & all Advanced modules

Improve Quality Services Waalre (near Eindhoven) The Netherlands

Training provider IREB Requirements Engineering Accredited Lead Assessors formal TMMi assessments

Eindhoven

 Leading in Testing & Quality Management  Consultancy, Subcontracting and Training  Founded in 1998 by Erik van Veenendaal www.improveqs.nl

Policy? Strategy?  What does it mean ??  Do you need this ??

 

Case study Critical success factors

Sounds familiar ?

Definitions Test Policy & Strategy  Test Policy – High level description of principles, approach and major objectives of the organization regarding testing.

 Test Strategy – High level description of test levels for an organization or programme, and the testing to be performed within each level. Business goals

Generic risks

Projects

Test Policy

Test Strategy

Master Test Plan

Test goals

Blueprint

Test Policy 20XX

Definition of testing

Mission / Ambition

Customers

Test targets

Test process

based on business goals

Organization

Test Process Improvement - objectives, performance indicators, model

Employees

Test Strategy  Based on the test policy  Addresses the generic product risks

 Contains description of test levels:  

  

Purpose, responsibility Level of independence (organizational) Entry and exit criteria Test process and deliverables Approach (top-down, bottom-up)

 

  

Test design techniques Tools to be used Standards to be used Test environment Typical non-functionals

The challenge  Test policy & strategy are important …  Provide common basis for projects  What is really important for business / managers  Leading in models like TMMi, TPI, and TMap Next

 … but also difficult to realize !  ‘We have no business goals or policy’

 ‘We don’t have time / interest / priority’

Practical case study  Principal: IT-Solutions Mgr. Solutions Architecture  Draft Test Strategy guidelines as starting point  My assignment: coaching creation of Test Strategy  Themes: strategy, terminology, organization  Scope: IT Express BeNeLux  Business, Business-IT and IT-services

Approach 1 Roadmap and planning 2 Organize meeting structure

3 Kick-off with management group 4 Training day for work group 5 Strategy workshops (4 sessions) 6 Elaboration Test Policy & Strategy 7 Reporting to management group

Kick-off with management 3  Purpose: get feedback and constraints  Presentation using provocative arguments

 Results:  Feedback:  How ‘testing’ is perceived in the organization

 Effects of insufficient testing (= business case for improvement)  Important quality characteristics

 Constraints  Realistic (acceptable) costs and organization structure

Strategy workshops 1. Agree on purpose and scope  using management feedback and constraints as input

2. Determine policy themes  reliability, customer facing, usability, performance, efficient testing, time to market

3. Definition of testing 4. Business goals  not our responsibility, proactive thinking is (rationale)  7 values DHL taken as starting point

5. Test strategy  V-model, determine test levels  RACI*, responsibilities, who is ‘main contractor’ * = RACI: Responsible, Accountable, Consulted, Informed

5

Elaboration: results 6  Test Policy document  Quality goals + rationale (2 A4)  Test policy (2 A4)

 Test Strategy document    

Principles V-model Characteristics per level Risk based testing

 Management presentation  Results and recommendations

Elaboration: Test Policy 6

Elaboration: V-model Full Roll-OUT

6

Management presentation  Recommendations     

Agree on common test strategy Apply risk based testing Install a test competence cell Start a test process improvement project Agree on common test terminology (e.g. ISTQB)

 Recommendations have been implemented  Test competence cell: competence manager installed  Interviews / meetings with stakeholders  Plan for deployment written

7

Lessons and success factors  It requires time to determine a good test strategy!  If there is no policy available, then propose one.  Involve management (feedback, constraints).

 Broad representation of organization as workgroup.  Foundation test knowledge in workgroup (training).

 Bring sufficient flexibility into your V-model.  Who is responsible for what (RACI) ?  Pioneer for implementation from own organization.

Epilogue (1) The assignment ended in 2008, what happened afterwards…

 Follow-up by test competence manager:  Further elaboration and deployment of test strategy (templates and tools)  Organized sessions/training on test awareness

 Assistance in projects to apply test strategy  Has set up method to manage test environment

Epilogue (2) The assignment ended in 2008, what happened afterwards…

 Current situation (mid 2009):  Competence cell (0.5 FTE*): central organization of testing expertise.  Close cooperation helps to get clear requirements

 Clearly defined test levels are helpful  Test strategy currently applied in a project for European standardisation of one of DHL’s most important product lines * = FTE: Full-time equivalent

Conclusions  A sound Test Policy and Test Strategy …  are the important starting points for projects  help to have good discussions …and decisions  can be very clear and specific  give answers to relative questions  are the first step in test process improvement

… are worthwhile to give a try in your organization ?!?!