Europe’s Premier Software Testing Event Stockholmsmässan, Sweden
“Testing For Real, Testing For Now”
Successful Introduction of a Practical Test Strategy Jurian van de Laar, Improve Quality Services, The Netherlands WWW.EUROSTARCONFERENCES.COM
A practical Test Strategy Jurian van de Laar
[email protected] EuroSTAR Conference 2009 Stockholm
Improve Quality Services Accredited provider ISTQB Foundation & all Advanced modules
Improve Quality Services Waalre (near Eindhoven) The Netherlands
Training provider IREB Requirements Engineering Accredited Lead Assessors formal TMMi assessments
Eindhoven
Leading in Testing & Quality Management Consultancy, Subcontracting and Training Founded in 1998 by Erik van Veenendaal www.improveqs.nl
Policy? Strategy? What does it mean ?? Do you need this ??
Case study Critical success factors
Sounds familiar ?
Definitions Test Policy & Strategy Test Policy – High level description of principles, approach and major objectives of the organization regarding testing.
Test Strategy – High level description of test levels for an organization or programme, and the testing to be performed within each level. Business goals
Generic risks
Projects
Test Policy
Test Strategy
Master Test Plan
Test goals
Blueprint
Test Policy 20XX
Definition of testing
Mission / Ambition
Customers
Test targets
Test process
based on business goals
Organization
Test Process Improvement - objectives, performance indicators, model
Employees
Test Strategy Based on the test policy Addresses the generic product risks
Contains description of test levels:
Purpose, responsibility Level of independence (organizational) Entry and exit criteria Test process and deliverables Approach (top-down, bottom-up)
Test design techniques Tools to be used Standards to be used Test environment Typical non-functionals
The challenge Test policy & strategy are important … Provide common basis for projects What is really important for business / managers Leading in models like TMMi, TPI, and TMap Next
… but also difficult to realize ! ‘We have no business goals or policy’
‘We don’t have time / interest / priority’
Practical case study Principal: IT-Solutions Mgr. Solutions Architecture Draft Test Strategy guidelines as starting point My assignment: coaching creation of Test Strategy Themes: strategy, terminology, organization Scope: IT Express BeNeLux Business, Business-IT and IT-services
Approach 1 Roadmap and planning 2 Organize meeting structure
3 Kick-off with management group 4 Training day for work group 5 Strategy workshops (4 sessions) 6 Elaboration Test Policy & Strategy 7 Reporting to management group
Kick-off with management 3 Purpose: get feedback and constraints Presentation using provocative arguments
Results: Feedback: How ‘testing’ is perceived in the organization
Effects of insufficient testing (= business case for improvement) Important quality characteristics
Constraints Realistic (acceptable) costs and organization structure
Strategy workshops 1. Agree on purpose and scope using management feedback and constraints as input
2. Determine policy themes reliability, customer facing, usability, performance, efficient testing, time to market
3. Definition of testing 4. Business goals not our responsibility, proactive thinking is (rationale) 7 values DHL taken as starting point
5. Test strategy V-model, determine test levels RACI*, responsibilities, who is ‘main contractor’ * = RACI: Responsible, Accountable, Consulted, Informed
5
Elaboration: results 6 Test Policy document Quality goals + rationale (2 A4) Test policy (2 A4)
Test Strategy document
Principles V-model Characteristics per level Risk based testing
Management presentation Results and recommendations
Elaboration: Test Policy 6
Elaboration: V-model Full Roll-OUT
6
Management presentation Recommendations
Agree on common test strategy Apply risk based testing Install a test competence cell Start a test process improvement project Agree on common test terminology (e.g. ISTQB)
Recommendations have been implemented Test competence cell: competence manager installed Interviews / meetings with stakeholders Plan for deployment written
7
Lessons and success factors It requires time to determine a good test strategy! If there is no policy available, then propose one. Involve management (feedback, constraints).
Broad representation of organization as workgroup. Foundation test knowledge in workgroup (training).
Bring sufficient flexibility into your V-model. Who is responsible for what (RACI) ? Pioneer for implementation from own organization.
Epilogue (1) The assignment ended in 2008, what happened afterwards…
Follow-up by test competence manager: Further elaboration and deployment of test strategy (templates and tools) Organized sessions/training on test awareness
Assistance in projects to apply test strategy Has set up method to manage test environment
Epilogue (2) The assignment ended in 2008, what happened afterwards…
Current situation (mid 2009): Competence cell (0.5 FTE*): central organization of testing expertise. Close cooperation helps to get clear requirements
Clearly defined test levels are helpful Test strategy currently applied in a project for European standardisation of one of DHL’s most important product lines * = FTE: Full-time equivalent
Conclusions A sound Test Policy and Test Strategy … are the important starting points for projects help to have good discussions …and decisions can be very clear and specific give answers to relative questions are the first step in test process improvement
… are worthwhile to give a try in your organization ?!?!