A case study on process composition using Enterprise SPICE model Amalia Alvarez1, Santiago Matalonga1, Tomás San Feliu2, 1

Universidad ORT Uruguay. Cuareim 1471. 11100 Montevideo, Uruguay [email protected] ; [email protected].

2 Universidad Politécnica de Madrid. Facultad de Informática. 28660 Boadilla del Monte. Madrid. España [email protected]

Abstract. Process improvement models include the best practices from relevant disciplines in comprehensive sets. Their purpose is to convey knowledge that will help organizations through their process improvement journey. In this paper we argue that process improvement practitioners do not take advantage of this knowledge. Hence we propose a process to evaluate and extract knowledge form a process model in order to improve an existing process. This paper presents an application of this process using the Enterprise SPICE model to improve an existing training process. Keywords: Process improvement, Process definition, Process Composition, Enterprise SPICE Model

1. Introduction Process improvement models like ISO/IEC 15504[1] or CMMI[2], have grown in popularity and adoption in recent years[3]. These models bring together knowledge and best practices from each of its target domains. Many organizations are convinced about the usefulness of these process models. Ideally, process models harbor the promise of shortcutting the process improvement journey to high performance[4]. By providing best practices and guidance, process models enable knowledge sharing and knowledge transfer from state of the practice to the organization. In order to do so, organizations must harness the value of the information contained in those process models. More often than not, organizations fail to understand the intention of the process models guidance and set out looking for solutions to problems already solved by these process models[5]. One of these problems is how to define a process or how to include process elements into a process definition. We understand process composition as the act of harnessing best practices included in references and incorporate them into existing process or into new processes. This paper presents a case study on process composition. A process for capturing knowledge from process models was defined using Enterprise SPICE model[6], which was also used to improve an existing process.

Enterprise SPICE is an initiative of the SPICE User Group that brings together best practices from several disciplines and several models and standards into a comprehensive improvement model [6]. The enterprise process used as starting point is the ROI+training process[7]. This process links production defects to training interventions. ROI+training support the training department within a software development organization by helping them justify the investments in terms of Return on Investment. The ROI+training process is defined in [7], and its application in the software industry has been published [8]. This paper is organized as follows. Section 2 presents background research on process definition, Enterprise SPICE model, and the ROI+training process. Section 3 presents the case study on process composition using the Enterprise SPICE process model. Finally in Section 4, we present our conclusions.

2. Research background 2.1. Elements of a process description The most comprehensive research on process definition has been carried a out by [10]. According to [10] a process description has to satisfy criteria that includes a set of information needed for it to be usable by people performing the process. That set of information is organized as process elements. The basic process elements are: Purpose of the process, Input and Output work products of the process, Activities that should be done, Roles that those Activities should perform, Entry and Exit criteria that state the limits of the process and a Procedure that describes the Tasks needed to perform the activities. 2.2. Enterprise SPICE Overview The Enterprise SPICE model integrates and harmonizes selected process models and standards into a single enterprise improvement model. By bringing together best practices from several disciplines and several models and standards into a comprehensive improvement model, that provides an efficient and effective mechanism for assessing and improving processes deployed across a typical, large or small, enterprise[6]. In the development of the Enterprise SPICE model the major, essential and widelyrecognized process standards and models that cover most of the relevant disciplines that IT enterprises perform [6] were considered. The result of this work was the Enterprise SPICE Process Reference Model, PRM, and Process Assessment Model, PAM[6], we will refer to it as the “Enterprise SPICE model” to address the all-inclusive PRM and PAM in this paper. The model is organized in four categories: Governance/Management (GVM), Cycle (LFC), Support (SUP) and Special Applications. Each category has a group of processes that are numbered in a unique way. Process dimensions include Purpose, expected Outcomes, Base Practices (BP), Work Products, Relationship Notes with

other processes and Mappings with other references[6]. The code to identify a Base Practice of a process is: PPP.N.BPX, where PPP is the codification of the group of processes, N the number of the process in the group and BPX, the number of Base practice. For instance, GVM.2.BP1, refers to the first Base Practice, of the second process in the Governance/Management category. 2.3. Return on training investment This section details the motivation for the research that leads to the ROI+Training process. Training investment is of outmost importance for software development organizations. Organizations rely on training for the continuous improvement of their processes or to maintain the set skill of their workforce update to the ever-changing technology [11]. It has been reported that organizations invest over 100 million US dollars per year on training activities[12]. This investment requires organizations to find reliable ways to evaluate the return on their investment in training. Methods of return on investment (ROI) have been proposed as a solution for this problem[13]. The ROI+training process[7] builds on top of these proposals by linking production defects with ROI. We have successfully deployed the ROI+training process in a software factory and found that it brings insight into the training effort[8] and also helps align the training to the organization’s business needs[14].

3. Process composition using Enterprise SPICE model This section details the steps taken by the researchers to improve the ROI+training process definition. Most often new processes are defined by observation and formalization of the activities of people doing their work. For this purpose, several techniques have been developed (i.e. Value-stream mapping [15]). But, when improving an existing process, references models like Enterprise SPICE model or the CMMI[2] can be useful for formalizing and improving processes that are already defined. The ROI+training process was originally defined in [7] (see Table 1). The goal of the process is to link production defects to return on training investment. ROI+training takes as input defects registered during the software production process, and through causal analysis[16] designs training interventions which are aimed at reducing those defects. Observed reduction is taken as input for return on investment calculation. This process has been piloted at a software factory in Montevideo Uruguay[8]. Step 0 1

Name Validate Entry Criteria. Validate that the organization’s measurement infrastructure can support the process Execute the Production Process

2 3 4 5 6 7 8

The process needs software defects to take as input. Analyse Defects Root cause analysis of defects to select training interventions. Plan Training Interventions. Establish Agreement on Measurement Objectives Agree with ROI stakeholders how benefits will be measured. Deliver Training Training must be delivered according to the organisation’s standard training process. Evaluate Return on Investment Calculate ROI by taking into account costs and benefits. Communicate Results Communicate the results of the training interventions in terms of ROI. Validate Exit Criteria Describes the process takeaways.

Table 1 ROI+training original definition 3.1. Case study Our intention with this case study is to enhance the definition ROI+training process in order to improve its usability and readability. For this purpose we decided to base our discussion by using Enterprise SPICE model as reference model, specifically the Process Improvement process. The Process Improvement process was tailored to suit the case study needs, were some of the best practices suggested by Enterprise SPICE model did not apply. The process followed by the researchers is detailed in Table 2. Purpose: Improve process definition by taking advantage of documented best practices, in order to improve the capability of the original process so that business can be conducted more efficiently and effectively. Inputs Outputs  Organization’s improvement  Improved process goals  Process definition  Best practices Activities and tasks: 1. Identify process improvement opportunities 2. Analyze process status 3. Plan improvement. a. Select best practice reference model b. Map best practice chapter to process goals. 4. Implement improvement. a. Incorporate best practices into process definition 5. Communicate results of the improvement. 6. Evaluate the results of improvement. Table 2 Process redefinition process

1. Identify process improvement opportunities and 2.Analyze process status After studying the process definition guidelines presented in section 2.1, the following shortcomings were identified on the process definition. First of all, the purpose of the process is not stated in its definition (Table 1). Hence the process has to be accompanied with a description that clarifies its intention. Secondly, inputs and outputs of the process are also not defined. This is a major shortcoming, since a process is “a set of integrated activities that uses resources to transform inputs into outputs”[17]. We might even argue that without a clear statement of them, a definition can hardly be called a process definition. In the case of this process, the lack of definition of inputs and outputs makes for confusion on the role of Return of Investment in the process. We claim that it would make for better communication to have ROI as a measure of the process and not as an output. This will allow for comparison between different instances of the process. Finally, there is no reference to the roles that will be executing the activities of the process. Without them there is no clear guidance for the practitioners to assign the task and activities of the process to the resources of their own enterprises. 3. Plan improvement. In order to map the best practice chapters to the original process activities we proceed with Crawford slip method with post it notes on a large whiteboard[18]. The resulting process description integrates several best practices from a set of process of the Enterprise SPICE model. These processes and their rationale for inclusion are described below:  Investment Management (GVM.2). “The purpose of the Investment Management process is to ensure that organizations realize optimal value from strategically aligned business investments at an affordable cost with a known and acceptable level of risk”[6]. Investment management was included to provide guidance on the use of Return on Investment as a financial indicator. It would support steps six and seven of the original process.  Human Resource Management (GVM.3). “The purpose of the Human Resource Management process is to provide the organization with individuals who possess skills and knowledge to perform their roles effectively and to work together as a cohesive group”[6]. Human resource management was included to provide guidance on establishing developer’s knowledge and skills goals. These activities provide guidance for step three of the original process.  Quality Assurance and Management (SUP.3). “The purpose of the Quality Assurance and Management process is to assure the quality of the product or service and of the processes used, and provide management with appropriate visibility into all relevant quality aspects.”[6]. Quality Assurance and Management was included in order to provide guidance on how to analyze the causes of the defects, and to guide the implementation of corrective actions. These activities support Step two and four of the original process.  Training (SUP.7). “The purpose of the Training process is to develop and maintain the skills and knowledge of staff so they perform their roles effectively and efficiently”[6]. Training was included to provide guidance on how to plan

and provide the training interventions. This process should support steps three and five of the original process. 4. Implement improvement. In this stage the authors started redefining the process. The first insight to improve the definition was the need to clearly determine the process inputs and outputs. Then, a clear statement that represented the purpose of the process was written. Finally, the roles that a typical organization should need to implement this process were defined. The improved process definition will include the following three roles: Quality Management (QM), Human Resources Management (HRM) and Trainer (Tr) 5. Communicate results of the improvement. The resulting process definition is presented in Table 3. Notice the aforementioned division between inputs and outputs, the clear statement of the process´s purpose, and the mapping to the reference model in the description of the activities and tasks, and the roles assignment to activities and tasks. Purpose: Provide specialized training based on causal analysis of production defects. Communicate the results of the training intervention in terms of Return on Investment. Inputs Outputs  Software defects records  Trained individuals classified as “lack of training”  Training records  Average cost of a software defect Activities and tasks: 1.

2.

3.

4.

5.

Determine needed skills and competences a. Defect root cause analysis (SUP.3.BP6 “Analyze quality”)  Role: QM b. Identify skills by profile (GVM.3.BP2)  Role: HRM Define training investment evaluation criteria  Role: QM a. Determine training objectives (GVM.3.BP3) b. Establish criteria (GVM.2.BP1) – Establish criteria for selecting and evaluating potential investments. Evaluate and determine training investment alternatives  Role: QM a. Develop a strategy for training (SUP.7.BP1) b. Identify Investment Proposals (GVM.2.BP2) c. Prioritize and evaluate investment proposals (GVM.2.BP4) d. Establish the investment portfolio (GVM.2.BP5) e. Identify and allocate resources (GVM.2.BP6) Deliver Training  Role: Tr a. Establish training plan (SUP.7.BP3) b. Establish training mechanism (SUP.7.BP4) * If necessary c. Prepare for training execution (SUP.7.BP5) d. Train individuals (SUP.7.BP6) e. Establish and maintain records (SUP.7.BP7) Evaluate results  Role: QM a. Assess training effectiveness (SUP.7.BP8)

b. Review/evaluate performance (GVM.2.BP7) Process Measurement: Return on training investment Table 3 ROI+Training enhanced definition In order to calculate ROI costs and benefits are necessary. The Trainer will provide cost of preparing and executing the training while the Quality Manager should provide the information about benefits. 6. Evaluate the results of improvement. Ideally the resulting process should be given to practitioners for their evaluation and acceptance.

4. Conclusions This paper has shown a method for improving an existing process by taking advantage of the knowledge available in a process improvement model. In this paper we have presented a process to help practitioners “Improve process definition by taking advantage of documented best practices, in order to improve the capability of the original process so that business can be conducted more efficiently and effectively” (Table 2). Furthermore, the tasks and activities encapsulated in this process have been obtained from the best practices of a process improvement model. In addition to this, we have shown a case study of how to use the knowledge in process improvement models to improve an existing process definition. The process model we have used as reference is the Enterprise SPICE model. The Enterprise SPICE model is a novel process improvement model that brings together guidance from several models and experiences from the IT sector. To the best of our knowledge, there have been few applications of the Enterprise SPICE model to this date. In this paper we have used it both to design the process for process composition, and as reference to improve an existing process definition. Future lines of research will include defining guidelines and process assets (like checklists) for supporting the process composition process to help practitioners adopt this process. Successful deployment to an organization will require guidelines that are tailored to the organizations process, roles and technology. Our intention is to take this process to the enterprise in order to pilot the process in a live organization.

5. References [1]

ISO/IEC, “ISO/IEC 15504-2:2004 Information technology Process assessment Part 2: A model for Process Management.” 2004.

[2]

CMMI Product Team, “CMMI for Development, Version 1.3. Improving processes for developing better products and services,” no. CMU/SEI-2010TR-033. Software Engineering Institute, 2010.

[3]

K. El Emam and I. Garro, “Estimating the extent of standards use: the case of ISO/IEC 15504,” The Journal of Systems & Software, vol. 53, no. 2, pp. 137143, 2000.

[4]

J. P. Womack, D. T. Jones, and D. Roos, The machine that changed the world. Rawson Associates, 1990, p. 339.

[5]

W. S. Humphrey, “Why don’t they Practice what we preach,” 2007.

[6]

The Enterprise SPICE Project Team, “Enterprise SPICE ® An Integrated Model for Enterprise-wide Assessment and Improvement,” 2010.

[7]

S. Matalonga, “Un proceso para la evaluación del entrenamiento organizacional,” Universidad Politécnica de Madrid, 2011.

[8]

S. Matalonga and T. San Feliu, “Calculating Return on Investment of Training using process variation,” IET Software, vol. Accepted f, 2011.

[9]

G. Valdés, M. Visconti, and H. Astudillo, “Tutelkan: A Software Process Reference Model,” in EuroSPI 2011, 2011, pp. 179-190.

[10]

T. G. Olson, N. R. Reizer, and J. W. Over, “A Software Process Framework for the SEI Capability Maturity Modelsm,” Pittsburg, 1994.

[11]

T. Hodges, Linking Learning and Performance. Masachusetts: Butterworth Heinmann, 2002.

[12]

L. Patel, “ASTD State of the Industry Report.” Asociation for Training and Development, 2010.

[13]

J. J. Phillips, Return on Investment in Training and Performance Improvement Programs, Second Edition (Improving Human Performance). Butterworth-Heinemann, 2003, p. 344.

[14]

S. Matalonga and T. SanFeliu, “Linking Return on Training Investment with Defects Causal Analysis,” in 20th Conference of Software Engineering Theory and Practice, 2008, pp. 42-47.

[15]

L. Hambleton, Treasure chest of Six Sigma growth models, tools & best practices: a desk reference book for innovation and growth. Upper Saddle River. New Jersey: Prentice Hall, 2008.

[16]

D. N. Card, “Learning from Our Mistakes with Defect Causal Analysis,” IEEE Software, vol. 15, no. 1, p. 7, 1998.

[17]

ISO, “ISO 9001:2008(E). Quality Management Systems - Requirements.” 2008.

[18]

W. Dettmer, Brainpower Networking Using the Crawford Slip Method. Bloomington: Trafford, 2003, p. 178.