WHITEPAPER

EMPOWERING ORGANISATIONS WITH SPICE Software process improvement – the practitioner’s way

RAINER ANDERS Senior Consultant & SPICE Assessor Champion of Innovation Group Automotive Engineering [email protected] Rainer Anders has been working for SQS since November 2000. As an ISO15504/SPICE Assessor and Senior Consultant, he is responsible for process improvement as well as project, quality and test management in large and complex projects and organisational environments. He interprets the relevant norms from a practitioner’s point of view and integrates this interpretation into the customer’s organisation. For more than 20 years, he has been involved in process improvement and testing for various enterprises in the automotive industry, IT service providers and financial services.

Empowering Organisations with SPICE

Page 2

1 MANAGEMENT SUMMARY The development of software-based electronic and mechanical systems and their software components faces an increasing variety of challenges, such as: Ever-increasing time-to-market demands, i.e. continuously aggressive release cycles Increasing amounts of safety requirements and simultaneously increasing overall system complexity Distributed development across company locations, even on different continents New regulations for safety during the development of systems Expensive recall campaigns due to software failures Loss of reputation due to the publication of software failures in the media

To achieve quick response times to changing market conditions and regulations, and simultaneously keeping up high quality standards for products, companies must have reliable and repeatable methods and processes. For the development of software and electronic systems, ISO/IEC 15504 (known as ‘SPICE’) is the state-of-the-industry model, which nowadays is requested by almost all major European car manufacturers. SPICE (Software Process Improvement and Capability dEtermination) is an international standard for the assessment of company processes with a focus on software development. The model is not limited to a specific industry sector, and for various sectors in the industry, derivatives of the model have been established, including first and foremost Automotive SPICE and Test SPICE. The benefits of a structured approach following established process models have been underpinned with statistical evidence for projects and organisations in terms of:

Increased efficiency Defect reduction in (software) development Increased productivity Increased predictability

In order to transform an organisation towards the SPICE model, a Software Process Improvement (SPI) project establishing the foundations and introduction of new processes is required. For the success of an SPI project, a number of prerequisites must be fulfilled. A typical SPI project consists of the following different phases: Starting phase: Assessing the current state and conducting project planning Establishing the fundaments of software development: · Version management · Traceability of requirements · Project management Constructing the ‘house’: · Software development and test processes · Quality management during development · Implementation of new processes Project closure

The focal area of the SPI project are the employees of the organisation rather than merely the processes themselves: to be successful, the individuals must learn and accept the new processes and apply them in daily life. When running an SPI project, senior management support must be present and be demonstrated. An SPI project imposes an additional effort and workload onto the organisation and its employees. This effort is required in order to realise the benefits derived from the improvement steps, bringing these benefits to daily work. Senior management must be aware of this effort from the very beginning and must consider it during

Empowering Organisations with SPICE

the run-time of the project to achieve realistic and pragmatic solutions. After project closure, the additional effort will cease to exist and workload will be reduced to the previous level (including the aforementioned benefits in terms of productivity).

Page 3

Many companies have successfully improved their development processes and consequently improved their market potential in collaboration with SQS AG. A significant proportion of this success stems from SQS’ methods, which are presented and explained in this whitepaper.

2 INTRODUCTION 2.1 DEFINITIONS SPICE – ISO 15504 SPICE stands for Software Process Improvement and Capability dEtermination. IEC/ISO 15504 describes a method for software development. A number of processes, base practices and work products of the processes are defined (e.g. project management, software design and system testing). Automotive SPICE is an industry sectorspecific derivative for the automotive industry. The so-called HIS focus (‘Herstellerinitiative Software’) (() is based on Automotive SPICE and considers only a limited number of processes pertaining to Automotive SPICE. Improvement projects following SPICE are called Software Process Improvement (SPI) projects. Definition 1: SPICE – ISO 15504 (')

CAPABILITY LEVELS Each process in SPICE can be assessed for its so-called Capability Level (1 – 5): Level 0: Incomplete Process The process is not executed or does not fulfil its purpose. At this level, there rarely is evidence confirming that the processes’ goals have been met. Level 1: Process Executed The process fulfils its purpose. Base practices are fully implemented. Level 2: Process Controlled Processes are executed in a planned manner (planned, controlled, adjusted) and the deliverables are produced adequately, controlled and maintained at project level. Level 3: Process Established Processes are executed with respect to a defined approach. The process achieves the defined results at organisation level. Level 4: Process Predictable Level 5: Process Optimised (Level 4 and 5 are not referenced in this whitepaper.) Definition 2: SPICE – capability levels

Empowering Organisations with SPICE

EVALUATION SCHEME FOR ASSESSMENTS F ully adequate – the criteria are met in full (86 – 100 %) L argely adequate – the criteria are largely met (51 – 85 %) P artially adequate – the criteria are partially met (16 – 50 %) N ot adequate – the criteria are not met (0 – 15 %) A capability level is achieved if all process attributes of a lower capability level are fulfilled (fully adequate) and the process attributes of the current level are largely or fully adequate. Definition 3: Evaluation scheme for assessments

2.2 THE CHALLENGE The SPICE standard describes a maturity model for development organisations with a focus on software development. The standard allows different industry sectors to derive refinements of SPICE. The refinements can incorporate additional requirements or restrict themselves to a subset of processes. The automotive industry has taken this refining step and has defined ‘Automotive SPICE’ as a development standard for the software development for electronic control units. Original Equipment Manufacturers (the car manufacturers as the system integrators, OEMs) are now increasingly demanding a Capability Level 3 from their suppliers. This demand results from the realisation that more mature organisations typically are in a better position to produce highquality products, which is what the OEMs are dependent on due to the ever-increasing complexity of the systems they build.

Page 4

Only a small number of OEMs and a small number of suppliers have achieved Capability Level 3 for individual development teams. In the course of a development project, SPI projects are initiated for the suppliers on a regular basis with the aim of improving or establishing existing and new development methods. From a broader view, the SPI projects aim at promoting the improved procedures and processes throughout the respective organisation (beyond the scope of a project). This approach bears high risks for organisations because the desired improvements cannot be achieved in this setting. To SQS’ experience, it is SPI projects with a dedicated mission, budget and resources which have a reasonable chance to be successful. In order to be successful with SPICE, processes must not remain lip service but have to be alive within the organisation. OEMs typically force their suppliers to confirm that they have reached a certain capability level, and enforce the right to audit their suppliers accordingly. The audits are conducted by the OEMs on a regular basis. If a supplier does not achieve the required capability level, his rating as a supplier is decreased and he faces the risk of losing future contracts or being forced into less profitable orders – both economically undesirable options. 2.3 BENEFITS OF PROCESS IMPROVEMENT A typical issue that needs to be addressed at the beginning of an SPI project is the question of tangible benefits for the organisation. More often than not, resistance is expressed in the following phrases: ‘We created great products in the past without focusing on processes and their improvement.’ ‘My colleagues learned best practices at their university.’ ‘We are running an agile organisation. This doesn’t match with static processes.’

Empowering Organisations with SPICE

Page 5

Despite the initially negative attitude, many organisations still decide to improve their software development projects so as to produce highest quality products even when the workload is high or very high. The following benefits of process improvement – which have been demonstrated in many projects by independent evaluations – have convinced the companies: 1. Efficiency improvements The following chart indicates the potential efficiency improvement with respect to the maturity improvement from one level to another, for example from Level 1 to Level 2. The efficiency improvement stems from a reduction of development effort. The chart shows the efficiency improvement for 95 % process compliance, where a 4 % to 10 % increase in efficiency can be achieved.

2. Defect reduction in software development Based on the structured approach for development, defect reduction can be achieved in particular at Levels 2 and 3 (cf. ())). At Atthe thesame same time, the rework efforts for defect elimination are reduced from level to level, summing up to an overall positive effect on the organisation. 3. Productivity gain As per capability level achieved, productivity gains can be realised within an organisation. In particular, the efforts for defect fixing are significantly reduced (cf. (*)). 4. Increased predictability of projects The precision of predictions for the project course can be much higher because of the availability of historical data from other projects the organisation handled in the past. The successful controlling of projects is facilitated starting from Capability Level 3 (cf. (+)).

% Reduction in effort per one-level change in process maturity

16 14 Upper 95 % confidence limit 12 10 One-level average change in process maturity

8 6 4

Lower 95 % confidence limit

2 0 0

100

200

300

Thousands of lines of code Figure 1: Efficiency improvement due to process improvement per maturity level

400

500

Empowering Organisations with SPICE

Page 6

By indicating the benefits of an SPI project in this way, it is shown that a structured approach with references to best practices and standards clearly supports organisations. The following conclusions can be drawn from the aforementioned results: 1 Deployment of resources becomes more efficient. 2 Processes become more transparent and comparable.

3 Projects are more easily predictable. 4 The respective quality goals can be achieved more easily and the product in development can be delivered in time. 5 Due to a reduced defect rate, the maintenance costs are reduced. 6 Documentation is assured and thereby the repeatability of the development is guaranteed.

35 %

3.5 Percentage of Rework

30 %

3.0

Defects per KLSOC

25 %

2.5

20 %

2.0

15 %

1.5

10 %

1.0

5%

0.5 0.0

0% Level 2

Level 3

Level 4

Level 5

Figure 2: Defect reduction in software development due to process improvement

Effort per development day

Customer onsite error fixing after deployment

4

Internal error fixing Testing

3

Reviews Preventive QA

2 1 0 Maturity

1

2

3

4

5

1.0x

1.4x

2.2x

2.9x

3.4x

Level Relative Productivity Increase Figure 3: Productivity gain due to process improvement

Source: Motorola, 2004

5

Empowering Organisations with SPICE

Page 7

Over / under percentage

140 %

0%

-140 % Without Historical Data Variance between +20 % and -145 % Level 1 & 2

With Historical Data Variance between -20 % and +20 % Level 3

Figure 4: Precision of prediction of projects

3 THE MARKET VIEW – CURRENT STATE AND FUTURE 3.1 WHY DO OEMS DEMAND DEFINED PROCEDURES? The use of electronic parts as technically embedded systems in vehicles has seen a dramatic rise in recent years. Not only are the convenience functions controlled by such devices, but more and more safety functions are realised by means of electronic control units which are integrated with mechanical parts of the vehicles. As mechanical and electronic parts become ever more integrated, the systems are called mechatronic devices. Typically, these mechatronic devices are interconnected via realtime bus systems, thereby inducing additional dependencies that again increase the complexity of development. In this context and due to liability regulations, the demands for detailed and traceable documentation are increasing heavily.

In order to manage the complexity of the products, a structured approach must be followed. A well-defined, structured approach ensures that all suppliers work and deliver based on the same principles and that the processes for documentation and development become fully compatible. For the customer, this allows for more precise planning of resources, interfaces, time and quality. For various industry sectors, standards have been developed following the same principles and varying in the respective industry-specific differences. This also holds true for the standards for the development of safety-critical systems (in general, IEC 61508, ISO 26262 for the automotive industry, DO-178B for avionics, IEC 60601 for medical devices, EN 50129 for railways, IEC 62061 for mechanical engineering, IEC 60335 for household devices, IEC 61511 for process industry, IEC 50156 for heating, IEC 51513 for nu-

Empowering Organisations with SPICE

clear power plants). For the development of a safety-critical system, the developing organisation must be at or above a maturity corresponding to SPICE Capability Level 2. If this process maturity level cannot be reached, the development of a safety-critical system will not be possible with an adequate technical and economic success. 3.2 WHO IS IN SCOPE?

Page 8

There are specific challenges for the productaccountable party of a safety-critical system. In coming years, more and more safety-critical functions will be implemented in software, with the safety-critical functionality being in focus for the development organisation. This is not a task for the individual developer or a small development team; it rather requires an organisation-wide refocusing on ‘functional safety’.

Every manufacturer and supplier taking part in a product development process, in particular those producing software components, are in scope.

4 PROCESS IMPROVEMENT PROJECTS 4.1 PREREQUISITES FOR SUCCESSFUL SPI PROJECTS SPI projects have manifold layers of activities comprising various tasks. In the following, a short excerpt of the latter focusing on the most important aspects is provided: It is necessary to acquire in-depth business knowledge with regard to the processes in order to develop an understanding of them and reflect on them. Methods and procedures have to be identified, analysed, and revised. Economical goals must be achieved. Employees have to be educated and convinced that the SPI project is in their proper interest, so as to motivate their contribution.

Apart from the various tasks, it is essential for an SPI project to establish the necessary environment. To this end in particular, the management of the organisation are in charge. Successful SPI projects have a number of prerequisites:

Commitment of senior management from the very beginning. Management actively take part in the project and its implementation in the organisation. Management need to accept that · SPI projects require a budget; · Other projects may be impacted and delayed; · Employees need training; · In the course of the project, resources will be redeployed to the SPI project to a certain extent; · Improvement of organisations and processes is a continuous task. The SPI project is a project of its own, independent from other product development projects. Realistic time and budget planning Establishment and monitoring of KPIs Integration of and transparent information for employees at all times in the course of the project Realistic assessment of the on-top workload for the organisation in the roll-out phase of new processes Project risk management

Empowering Organisations with SPICE

Typically, a process improvement project does not start on a green field. It is embedded in a living organisation having established procedures and routines. Therefore, there are many potential obstacles and risks for the project which may have a negative impact on its success, e.g.: The pilot project is already behind schedule and has overrun the budget. Project members are assigned tasks and roles in addition to their current tasks and roles. The SPI project must be executed as a ‘side activity’. The SPI project was started in the phase of an economic boom and all employees already work above 100 % workload. There is resistance with regard to the SPI project among management and staff. Cultural change induced by new processes is neglected. The company has been bought by an investor. Target markets or target segments are moving.

The aforementioned challenges must be addressed by management, and if so required, adequate measures (e.g. slowing down the project) must be taken. 4.2 THE SET-UP SQS’ methodology will be demonstrated in the context of the following project scenario: A supplier in the automotive industry has ramped up a development department for an innovative product in a short time. The product is an electronic controlling unit for a well-defined task and the market has responded positively to the product. The supplier has successfully sold his product to several OEMs. The development of a production-ready product for a car family based on a product line is a big challenge for the suppliers. They have to estab-

Page 9

lish and/or revise all processes for the product, namely: New or changed processes for hardware and software New or changed procedures for testing and validating hardware and software New or changed production processes and product lines New tools for production including procurement New or changed logistics processes Training of employees

For all the aforementioned processes, the respective OEM must be integrated. A supplier can develop the product for serial production with a single OEM with good processes and typically would achieve Capability Level 2. With additional OEMs and product variations, the complexity of the processes increases and most suppliers’ capability level falls to 1, or in the worst case even to 0. If further car families or further OEMs are added, the development processes of a supplier are increasingly under pressure. Therefore, more advanced concepts are required for the following processes and activities: Multi-project management Risk management Version management Configuration management Release management Branching and merging Traceability Reusability and platform development

In our example, one of the OEMs conducted an assessment according to Automotive SPICE in the context of a supplier selection process. The result was disappointing and did not meet the expectations of the OEM. The supplier’s capability level – depending on the specific process – was either 0 or 1. As a consequence from the assessment,

Empowering Organisations with SPICE

several actions were imposed on the supplier, the key request being the improvement of product development processes with the target of Capability Level 2 in a specified period of time. In case the supplier did not achieve this level, the orders for the product would be at risk. SQS AG was asked by the supplier to set up an SPI project, to improve the capability level of the respective processes in the given time frame to Level 2, and to prepare the processes for Capability Level 3. The project was conducted on the basis of an Automotive SPICE assessment with HIS focus ((). Project Projectpayment paymentwas wastied tiedtotothe the success of the improvement measures being validated through an external assessment. 4.3 THE APPROACH In order to run a successful SPI project, it is necessary to choose a holistic approach. The approach must consider the following: In the starting phase, the organisation and its processes must be investigated. According to the results from the starting phase, a detailed planning for the respective processes must be laid out. If the fundamental processes (e.g. project management) are not stable, they have to be stabilised. All other processes are built on top of the fundamental processes. In conjunction with the conceptual phases, the organisation is prepared to take over the processes for daily work. After completing these phases, the project is closed and the processes are subject to regular maintenance by the quality manager. SQS AG has a long track record of successful SPI projects, as the company favours a pragmatic approach . This approach is adapted to the customer’s needs and his respective project situation, as will be

Page 10

shown in the following sections. In the case study, a supplier of electronic control units had been requested by a customer to improve his development processes. Since the supplier had no in-depth experience with SPICE or with SPI projects, he was in need of external support. To this end, SQS was chosen to assist the improvement efforts. 4.4 THE STARTING PHASE In the starting phase of an SPI project, it is crucial to gain a broad overview of the current processes and procedures in software development. This overview must be achieved in a structured and traceable form. An assessment according to SPICE or CMMI is a convenient way of doing this. If only the test processes of an organisation are in focus, an assessment according to Test SPICE, or for smaller organisations an SQS Health Check, is fit for purpose. During this phase, the SPI project is initiated and equipped with resources, teams, a budget, and project goals. In the case study, the SPICE assessment according to Automotive SPICE with HIS focus was conducted in a several-day effort. As the results of the assessment are internal results for the supplier only, it can be assumed that the employees were more open and more realistic in the course of the interviews than in the OEM’s assessment. The supplier’s management explicitly asked the employees in the kick-off meeting to be as open as possible. Line managers of the interview partners are not present at the time the interviews take place, and the results are evaluated and presented anonymously. The assessment analysed the areas of project management, configuration management, change management, requirements engineering, software development and testing. A typical result from such an assessment looks like the table in Figure 5.

Empowering Organisations with SPICE

Page 11

PROCESS

1.1 2.1 2.2 3.1 3.2 4.1 4.2 5.1 5.2

SPICE

IPQ LEVEL

ENG.2

System requirements analysis

L

N

N

N

N

N

N

N

N

1

1.00

ENG.3

System architectural design

L

N

N

N

N

N

N

ENG.4

Software requirements analysis

F

N

P

N

N

N

N

N

N

1

1.00

N

N

1

1.00

ENG.5

Software design

L

N

P

N

N

N

N

N

N

1

1.00

ENG.6

Software construction

P

N

N

N

N

N

N

N

N

0

0.50

ENG.7

Software integration test

P

N

N

N

ENG.8

Software testing

P

N

N

N

N

N

N

N

N

0

0.50

N

N

N

N

N

0

0.50

ENG.9

System integration test

L

N

N

N

N

N

N

N

N

1

1.00

ENG.10

System testing

P

N

N

N

N

N

N

N

N

0

0.50

MAN.3

Project management

L

L

P

N

N

N

N

N

N

1

1.50

SUP.1

Quality assurance

N

N

N

N

N

N

N

N

N

0

0.00

SUP.8

Configuration management

L

N

N

N

N

N

N

N

N

1

1.00

SUP.9

Problem resolution management

N

N

N

N

N

N

N

N

N

0

0.00

SUP.10

Change request management

L

N

N

N

N

N

N

N

N

1

1.00

Figure 5: Example results from a SPICE assessment

The result from the assessment brings transparency and allows for identifying weak points. For each of the processes addressed in the assessment, detailed knowledge about open points and proposals for improvement are collated. This is the basis for the planning of the SPI project in terms of time, budget and resources. 4.5 THE FUNDAMENT The fundament of good software development has three essential requirements: Versioning of information and artefacts such as a source code and documentation, to make changes traceable Traceability of requirements throughout all parts of the development process Concrete planning for all tasks

All three requirements had not been met or sufficiently met in the project set-up. Without having the fundament in place, it is not possible to produce high-quality software for a long time and with reasonable effort. This becomes transparent only after the knowledge and the expertise of individuals is no longer available and motivated project members are no longer around, or in the late maintenance phases, when knowledge has been lost over time. The question of how to create a solid foundation for these requirements is key to an organisation’s success. 4.5.1 VERSIONING According to SPICE, all development artefacts (documents, source code, etc.) are filed with versioning information, and the archived version must not be changed without explicit notice. SPICE does not refer to any specific tool for ver-

Empowering Organisations with SPICE

sioning. In practice, typical configuration management tools are available on the market and fulfil these criteria very well. By this means, it can be assured that Versions of individual artefacts can be summarised into releases; Old versions are no longer used; Links between programmes are possible; Release management can be implemented; Accidental deletion does not occur.

In the case study, development artefacts had simply been stored on a file server, and various problems occurred in particular when releasing software: Old versions had been used, so that functionality was missing. Old versions had been used, so that already removed defects were reintroduced. Usage of versions for a different customer with a different hardware configuration led to communication failures in the vehicle. Source code had been accidentally deleted and had to be recovered imposing late delivery of the product.

For the purpose of software reuse, individual software artefacts are built into different software product variants. To properly model and manage these dependencies, a file system is insufficient. In collaboration with the supplier, we collected and evaluated his requirements to a configuration management tool and accordingly decided on a tool. The tool selection followed the relevant process as described in ISTQB. As a result, an open-source tool was recommended that already is in use in many companies. A qualification of the tool for the sake of functional safety was not necessary.

Page 12

Then, a concept for using the tool for configuration and release management was defined, the software was installed and set up, the employees and the configuration and release manager were trained, and the tool was rolled out. To this end, a set-up of a library and modularisation concept for the software was conducted, resulting in efficient reusability of software modules following predefined rules. The benefits were perceived early and clearly by the users as many manual and error-prone tasks were replaced by a safe and automated tool. By the time of the roll-out of the tool, the role of the configuration and release manager was established. 4.5.2 TRACEABILITY OF REQUIREMENTS In software development, the coverage of all requirements must be transparent at all times – during implementation as well as in the testing phase – by means of defined successful test cases. Through this transparency, any progress of the development is precisely trackable. Furthermore, it is ensured that nothing is implemented without a defined requirement. Therefore, SPICE emphasises the end-to-end traceability of any development artefact. It is not recommended to manage the dependencies in a spreadsheet application. For tiny projects, this might be an option with reasonable effort. But as soon as the project size exceeds two or three developers or approximately 100 requirements, the effort of manually maintaining the spreadsheets is no longer sustainable, and the investment in a requirements management tool should be made. Traceability must be preserved across the entire toolchain. In our case study, the management of requirements was done in MS Word. We identified a requirements management tool the supplier had installed but left unused because he had not yet had the time for the roll-out. The same was true for the test management tool, and in contrast to

Empowering Organisations with SPICE

REQUIREMENTS TOOL

Page 13

SOFTWARE CODING TOOL

TEST MANAGEMENT TOOL

CONFIGURATION MANAGEMENT TOOL

Figure 6: Toolchain and traceability

software development there was no configuration management system in use. In collaboration with the supplier, SQS AG mapped out the requirements for the requirements management tool and the test management tool. In this context, the exchange formats with the OEMs were an important criterion. According to the requirements, the interfaces and functionality of the tools were examined for their suitability. The analysis revealed shortcomings in the interface from the requirements management tool to the test management tool – even though they were produced by the same software vendor. This led to a change request for the software vendor, and an improved version had to be developed and deployed. The traceability from the software development tool to the requirements tool had to be managed as a manual interface since it could not be implemented by an automated solution. A pragmatic solution to this problem was the invention of a unique ID for all software modules, that was then documented as part of the requirements description. If a software module addressed several changes, it was sufficient to refer to the ‘higher level’ requirements. This way, traceability was ensured for all requirements in a pragmatic and yet SPICE-compliant way.

4.5.3 PROJECT MANAGEMENT In order to produce a concrete plan for a development effort, it is required that an organisation defines clearly scoped projects. Altogether, projects need a clear project scope, a project lead, a timeframe, and a dedicated budget. These days, one often encounters matrix organisations, where more often than not projects are unfocused and the project ideas are not realised. Without a clear project organisation and a dedicated project lead with respective competencies and mandates, SPICE requirements cannot be fulfilled. In our case study, the organisation did not have a dedicated project organisation. Projects typically were part of the line organisation, and there was no centralised overview of the real number of projects. This entailed an unstructured, poorly planned work approach. From this approach, time and again, firefighting activities had to be triggered which caused an additional workload and critical situations. Moreover, all developers were involved in the maintenance of the series products. All in all, there was no transparency regarding real effort, tasks and timelines. The number of projects was estimated at three to five. With the support of SQS AG (guided by SQS’ Project Management Expert Group), the supplier analysed his current situation. In this analysis, em-

Empowering Organisations with SPICE

ployees were allocated to roles, tasks and effort by means of a work breakdown structure (WBS). With the WBS as a tool, a number of findings became apparent: There were · Seven projects for new products, · Four maintenance projects for product evolution, · Two projects in the sales process. Some roles were not present in the projects. There was a permanent work overload of project members.

Several measures could be derived and realised: Set-up of a Project Management Office (PMO) for the development department, which is responsible for the planning of critical resources and provides overall governance for development projects Development of a customised generic standard project plan to be used in all projects Installation of projects and project teams in alignment with the respective OEM Formal documentation of all required roles with tasks and competencies/mandates Set-up of phase reviews in projects Introduction of risk management and risk reviews as part of the projects Establishment of KPIs derived from respective quality requirements Introduction of a modularisation for the software in order to reduce maintenance efforts A priori resource planning in collaboration with OEMs and senior management

4.6 THE CONSTRUCTION WORK 4.6.1 SOFTWARE DEVELOPMENT AND TESTING PROCESSES Once the fundament is laid out, further processes can be built on top. In particular, these are the

Page 14

software development and testing processes making use of requirement definitions, the requirements management and the configuration management. It is key that requirements are derived from the specification of the OEM and are described correctly and unambiguously. If the requirements are captured correctly, it is possible to derive test cases und identify gaps and defects at the earliest point in time. In the subsequent test processes, the following topics are touched: Separation of individual testing phases: · Component testing · Software integration testing · Software testing · System integration testing · System testing Early development of test cases according to the requirements for each test phase Set-up of a stable testing environment Planning for a well-ordered test execution

In order to increase the quality of the work products and identify errors or gaps early, a stringent review approach is paramount. All essential work products from the software development and test management processes have to be reviewed. Reviews have the additional benefit that the knowledge contained in the documentation is shared across different people in the organisation, thereby avoiding the installation of knowledge islands. In our case study, the description of requirements was of a low quality and sometimes even ambiguous, which caused debates with the OEM during the later project stages. Employees were made aware of the issues and received training. After the requirements had been reviewed, the quality improved significantly and the OEM’s and supplier’s different points of view were harmonised. Accordingly, the number of change requests decreased.

Empowering Organisations with SPICE

Initially, there was no coherent test concept – it had to be designed and implemented in cooperation with the supplier. In this context, the existing testing tool was integrated into the test process. In collaboration with the employees, a review approach was designed and implemented. The approach was based on templates and patterns established within SQS AG, which were customised to the supplier’s needs. 4.6.2 QUALITY MANAGEMENT PROCESS A quality management process is only indirectly accountable for the quality of a product. The purpose of this process is to establish, improve and evolve quality procedures and measures in development. Furthermore, the quality manager ensures that quality is measured and documented. In order to assure quality, he needs the quality criteria of the organisation as a guideline. These quality criteria are either defined by management or have to be drawn up by management in collaboration with the quality manager. The latter then develops a quality management plan for the organisation, accordingly. On the basis of this plan, he can subsequently define a quality strategy and determine the measures required for the projects.

Page 15

on the facilitation of the phase reviews of projects. As an external task, the quality manager functions as the single point of contact with the quality management of the OEM. One of the quality manager’s most important tasks regards monitoring compliance with the processes in order to identify deviations and initiate required changes. To this end, regular process reviews are conducted and these reviews are documented by KPIs. 4.6.3 ROLL-OUT OF PROCESSES INTO THE ORGANISATION An essential part of an SPI project is the roll-out of the (re)designed and documented procedures and processes into the everyday working life of the organisation. Only if the stakeholders of the processes learn and use these new processes, the benefits for the individuals and for the organisation can be leveraged. If the processes are followed half-heartedly or not at all, overhead effort is generated and resistance is stirred among the stakeholders. Therefore, processes must be defined in a way that fixes them on a high level yet with enough built-in flexibility also to accommodate project-specific circumstances. Due to changes in the environment of the organisation, it may be necessary to adapt the processes, too, either at the level of single projects or at the level of the whole organisation.

In our case study, the role of the quality manager had not yet been defined and the necessity of a quality assurance process had been neglected. But the organisation could be convinced, and management went on to establish and staff the role of a quality manager. In collaboration, the The roll-out requires detailed planning. For each quality manager and management then derived process, it must be determined What to do and on what basis the roll-out the relevant quality criteria from DIN ISO/IEC should take place; 25000 (,), refined refinedtheir theirinterpretation interpretationfor forthe the organisation and identified measurement criteria. What risks might be involved; Starting from these criteria, a standard quality What mitigation actions might be necessary; plan for the supplier’s project was developed for and future project-specific customisation and use. On what timeline the process should be At the same time, the quality manager took over rolled out. the management and documentation of reviews and the evaluation of the results. He further took

Empowering Organisations with SPICE

In order to monitor the progress of the SPI project, regular (quarterly) short assessments are conducted to determine process maturity. This way, it is possible to take corrective action early. At the same time, regular documentation of the progress of the process roll-out ensures that the project management can exercise sufficient control. SQS AG’s approach to process roll-out is borrowed from the SPI manifesto. (7) Pursuant to the latter, there is more than one successful way to perform a roll-out: the demands and needs of

ACT

Page 16

the organisation and its members must determine the focus. All measures have to be tailored to the organisation. Accordingly, the concrete approach to the SPI roll-out in an SPI project will vary in its individual actions: Creation of process descriptions and templates Training of employees in new processes, templates and tools Training on the job Implementation of the processes following the Triple-A Principle

PILOT PROJECT Consultant coaches pilot project on new processes. Client staff is work-shadowing.

SPI PROJECT PROGRESS (TIME)

ASSIST

PILOT PROJECT Client staff coaches pilot project on new processes, being assisted by the consultant.

ANSWER

PILOT PROJECT Client staff coaches pilot project on new processes. Consultant is involved only to answer difficult questions.

Definition 4: Triple-A Model

Empowering Organisations with SPICE

The Triple-A Model supports the introduction of new processes into an organisation by a continual approximation via three stages. This facilitates the ramp-up for all stakeholders and keeps the effort of the individual within reasonable bounds. In our case study, SQS AG rolled out the new processes over a timeline of nine months. To most of the processes, the Triple-A Model was applied. One part of the process could be rolled out with training-on-the-job methods, as only a few individuals were affected and they were already familiar with the procedures and followed them. During the roll-out phase, progress was assured by mini-assessments every three months. 4.7 PROJECT CLOSURE The closing of an SPI project includes the handover of all project artefacts, templates, and documentation to the respective client. With the help of these artefacts, the client is in a position to maintain, evolve, and manage his processes. One of the standard outcomes of an SPI project is an assessment of the capability level achieved. This assessment can be conducted by SQS AG; a valid alternative option would be an external audit by one of the OEMs, if the respective OEM intended to audit the processes anyway.

Page 17

It is recommended to have a lessons-learnt session sometime after project closure, with the aim to assess the SPI project itself and reflect the questions: What went well? What needs improvement?

From this assessment, all project participants can benefit further, in particular with regard to future SPI projects and improved methods. Our case study concluded with a successful assessment by SQS AG, with the results confirmed a year later in a follow-up audit by one of the supplier’s OEM customers. This way, SQS AG received its bonus payments. Further, SQS AG and the supplier conducted lessons-learnt sessions to improve future collaboration.

Empowering Organisations with SPICE

Page 18

5 CONCLUSION AND OUTLOOK SPI projects offer high potential for the evolution of a development organisation in terms of the following: Achievement of commercial objectives of the organisation Cost transparency Quality Process transparency Customer satisfaction

In this context, the benefits for the organisation can be identified and realised. With a structured, pragmatic approach, SPI projects can be a success for all stakeholders. SQS AG has Long-term experience from many projects; Domain experts and consultants; and an Internal continuous improvement process in order to make the most of the lessons learnt in client projects.

Bibliographical References

'

The SPICE User Group. AutomotiveSIG_PAM_v24. 2009.

(

HIS. www.automotive-his.de. Herstellerinitiative Software. [Online] 2011. http://www.automotive-his.de/.

)

How CMM Impacts Quality, Productivity, Rework and the Bottom Line. M. Diaz and J. King. March: Crosstalk, 2002.

*

Motorola. Relative productivity increase. 2004.

+

John Vu. Software Process Improvement Journey From Level 1 to Level 5. Proceedings of the SEPG Conference. 1997.

,

ISO. DIN ISO/IEC 25000 Software-Engineering – Qualitätskriterien und Bewertung von Softwareprodukten (SQuaRE) (Software Product Quality Requirements and Evaluation). 2010.

-

Jan Pries-Heje. Roskilde University, Axiom, Jørn Johansen, DELTA. SPI Manifest. [Online] 2010. http://www.qm-infocenter.de/qm/o_ pb.asp?navid=201005201113362&pb_id=20091214104939-129.

Page 19

SQS Software Quality Systems AG Phone: +49 (0)2203 9154-0 Stollwerckstrasse 11 51149 Cologne / Germany www.sqs.com