Identifying Issues and Concerns in Software Reuse in Software Product Lines Meena Jha1 and Liam O’Brien2 1

CQUniversity and the University of New South Wales, Sydney, NSW 2052, Australia [email protected] 2National ICT Australia Limited and the Australian National University, Canberra, ACT 2601, Australia [email protected]

Abstract. One of the reasons for introducing software product lines (SPL) is the reduction of costs through reusing common assets for different products. Developing assets to be reused in different products is often not easy. Increasing complexity due to the multitude of different functions and their interactions as well as a rising number of different product variants are just some of the challenges that must be faced when reusing software and other assets. In an attempt to understand the obstacles to implementing software reuse in SPL we have conducted a survey to investigate how software reuse is adopted in SPL so as to provide the necessary degree of support for engineering software product line applications and to identify some of the issues and concerns in software reuse. This survey also gathers information from SPL practitioners on what influences the selection of software to reuse within a software product line. This paper reports the results of that survey. Keywords: Software Reuse, Software Product Line, Domain Engineering

1.

Introduction

The study of software product line addresses the issues of engineering software system families, or collections of similar systems [2, 5, 6]. The objective of a software product line is to reduce the overall engineering effort required to produce a collection of similar systems by capitalizing on the commonality among the systems and by formally managing the variation among the systems. This is a classic software reuse problem [8]. Software reuse has been practiced since computer programming began. Reuse as a distinct field of study in software engineering, however is often traced to McIlroy’s paper which proposed a software industry on reusable components [1]. The primary focus of software product line research has been on a number of issues such as domain analysis and modelling, architecture modelling, process definition, etc [3]. Research in software product lines [9, 10, 11] shows that there are success stories, but there are no strong rules that can be derived from these success stories about how software is reused. Systematic reuse is one of the goals of software product lines, meaning that components must not only be copied but actually shared among several

1

subprojects. Also a software product will not only consist of shared components for the whole software product line, but some of the components will be specific to a single product. This means that the ability to reuse software depends on the commonality and variability requirements. Software Product Line practitioners come across many software reuse problems. Up to now the software reuse issues and concerns have not been surveyed, compared and documented in a systematic way. We conducted a survey to determine if domain knowledge is one of the keys to software reuse in SPL, to identify areas of concern for software reuse within SPL and to identify if the reliability of software assets/components is an issue for reuse in SPLs. This survey builds upon our previous survey on software reuse in the conventional software community [4] to identify issues and concerns in software reuse. The remainder of the paper is structured as follows: Section 2 discusses the methodology, activities and milestones adopted in undertaking the survey. Section 3 gives an overview of the questions presented to software engineers, developers, managers and researchers working in SPLs. The key results and analysis are described in detail in Section 4. Section 5 concludes the paper and outlines potential future work derived from the survey.

2. Methodology The survey was email based and carried out in mid 2008. The development and administration of the survey questionnaire and the analysis of the results were conducted in five different activities. These activities are as follows: • Step 1: Gather good candidate questions; • Step 2: Select questions that were of most interest; • Step 3: Determine how the questions would be asked and in which order; • Step 4: Select the people from SPL community whom we would like to contribute to our survey and send them the survey; • Step 5: Analyze the results by putting all responses together to point out areas where the respondents identified issues and concerns.

3. Presentation A total of 51 questions were put together and organized into five subsections: general type (2 questions), reuse measurement type in SPL (16 questions), reuse technical issues in SPL (10 questions), testing and the reliability of reused software in SPL (10 questions), and development environment in SPL for reuse (13 questions). The general type questions were asked to capture the educational and experience level of the respondents. Reuse related questions included, for example, identifying the level of reuse being done in SPL, measuring the reuse level, testing of reused

2

code, development environment used, etc. The survey was targeted specifically at the software product line community and people active in software product line efforts.

4. Result and Analysis 4.1 Section 1: General Questions Of the 29 survey respondents 17% had less then 5 years experience in SPL, 37% had experience of 5-10 years, 37% had experience of 10-20 years and only 6% had more than 20 years experience. The survey result shows the years of experiences for the majority of the respondents ranges from 5 to 20 years. 65% of the respondents were researchers & managers in SPL, 20% were product line architects, 6% each were core asset architects, and 9% were R&D project leaders. Simply put these numbers could mean that we have a more mature population in the sense of working experience in an SPL context. This could indicate that we have gathered a good sample population that could answer our software reuse questions, thus reflecting the average software product line engineer and developer. 4.2 Section 2: Software Reuse Management and Measurement Before an organization fully commits itself to actually convert into a functioning product line organization, the advantages, disadvantages and factors influencing reuse in software product line should be addressed. This section of the survey has captured answers about reuse management and measurement, its advantages, disadvantages and factors influencing reuse in the software product line community. The key benefits of reuse in SPL have been widely accepted and our survey result reflects the same. 100% of the respondents believe that reuse in SPL will achieve increased quality, planned productivity, capture of domain knowledge, and cost reduction. The advantage thus is that reuse is considered to be of strategic importance in organizations adopting SPL practices and tools. Our respondents believe that SPL necessitates effective strategic planning and product line road mapping; market analysis; change from bespoke customer relationships and projects toward market, product and service orientation; effective requirements, scope, and release management of products and services. Moreover, the scope of reuse is broadened from software (code) reuse to the reuse of all domain artifacts. Disadvantages of Software Reuse in SPLs However, the disadvantages associated with reuse in SPL were maintenance cost and start up cost. The major disadvantages respondents have highlighted are complexity – the “gravity” of software engineering: reuse can add complexity by creating dependencies between previously autonomous organizational units. Some of the problems with the dependencies identified by one of the respondents are web of dependencies, coordination cost, cost of offering integration, process and tool divergence.

Is Software Reuse domain based? Respondents believe that software reuse “is not sufficiently domain based”. Some of the comments from the respondents include “Documentation may be missing, the trace between requirements and design artifacts may be missing, and the understanding of the interplay between different functions may be missing”, “There is no real way of translating practice into theory”, “There can be widespread reusable software with inappropriate design”, “People reuse software without solving architecture mismatches”, “There is too much reusable software that is large grained such as in Service Oriented Architecture (SOA). Loosely coupled software modules are hard to be developed due to the difficulty to define the right interfaces, since SOA usually erodes other quality attributes such as performance and scalability”, and “Software developers are interested in developing software for-reuse but not developing a product with reusable software components”. Reuse might also hinder new ideas and the making of innovations. Balance between innovation and reuse should be determined by the company’s strategy. The opinion that ‘copy and paste’ is THE reuse strategy should be removed from SPL. Respondents are of the opinion that to maintain asset health, regular investments should be done to keep code healthy and keep the number of variations manageable. The Impact of Product Line Engineering Participants were also asked if they feel current software product line engineering practice is influencing reuse. Responses to this question (which are negative) should change the way we do reuse currently. 87% of the respondents feel reuse education will definitely help them learn more about recent reuse technology in making reuse possible in their organization. Also companies need to have development policies that mandate reuse. The companies that are serious about SPL must educate all levels (technical management, higher level management, and developers) about what SPL is, how the company plans to achieve it and what their role will be in making the transition successful. However, more then half of the respondents (55%) feel that work may be increased when they are reusing other’s code. 20% feel that reusing other’s code will not increase their work. However, 26% believe that it will definitely increase their work. 100% of the respondents believe that there is increased recognition of reuse, but software reuse will not be achieved with just education. Reuse Planning The respondents felt that SPL reuse has to be planned in advance. In their view, the central factor is to anticipate common and variable artifacts in the domain of the product line to provide useful reusable artifacts. The alignment of four views of SPL (business, architecture, process and organization) is fundamental for achieving benefits from SPL based product development. According to the respondents reuse is part of SPL, and its facilitators are: high expertise in domain knowledge, technical skills in SPL techniques, tool support, the number of software units produced in an SPL, motivated personnel, and high management commitment and support for integrating domain and technical expertise. Long term strategies and commitments are required. Standards should be established on how to communicate all properties of reusable components.

4

Respondents said that the reuse policy should be tracked and managed by engineering managers and/or quality managers and/or configuration managers; include planned variation and variation mechanisms; include mechanisms to deal with change control conflicts on assets; include rotation of product team members among different products, and/or core asset teams. Respondents agreed that SPL is very challenging to institutionalize as long as the costs of implementing an organizationwide SPL program are perceived to outweigh the benefits. The most important factor is to achieve the break-even point between costs and benefits as quickly as possible. The Influence of Software Engineering Practices 85% of the respondents felt that software engineering practice influences reuse. Using core assets, the configuration management philosophies, the amount and type of documentation, requirements elicitation, and many other software engineering practices all influence reuse. 90% of the respondents felt the increased recognition of reuse as the software product line success stories have an impact on companies. It was felt that the use of a common software process would promote reuse in a single organization. There is no real benefit in having a common process across companies but there is some benefit in having a common process across products within an SPL. A common process will help if there is an equal emphasis on the technology and the product. A Reuse Repository The responses to the use of a reuse repository improving code reuse were mixed. A clear idea behind the reuse repository in SPL could not be gathered. Several of the respondents commented that a reuse repository would improve code reuse as long as there are proper methods and practices around its use. The fundamental premise of software product lines is that code repositories without the context provided by a software product line do not work. Influence of Organization or Project Size on Reuse The majority of the respondents believe that a company’s divisions or project’s size is not predictive of organizational reuse. People are of the opinion that the size has nothing to do with reuse. This is very contradictory with the theory already established where the complexity of the reuse increases with the size of the project [10]. People are also of the opinion that product line approaches must be tailored to organizations. Size is one of the customization factors; but far from being the only one. A rich (large) domain has potential for reuse but usually has more challenges to coordinate. A small company developing sophisticated and customized products does not benefit from reuse. But a small company that is developing products which have 50-80% of the same components, 20-50% for customized software, reuse and SPL are beneficial. Products targeted into mass-markets have high potential for reusable software components. So, business and target markets are more important than the size of organization. Reuse and Software Quality The majority of the respondents felt that software quality does not inhibit reuse. Most code in each product is already of high quality as it is reused across multiple products.

Quality concerns should actually support SPLs. But several of the respondents felt that conflicts on core assets (usually arising because of different stakeholders’ quality concerns) can limit the modification and/or use of core assets within products. Especially for mass marketed, deeply embedded systems where memory and CPU power are very limited due to cost constraints, reuse (and even modular software design) is inhibited. Also for some real-time systems with specific performance requirements reuse might be inhibited. So performance in terms of efficiency and response time are most inhibitive for reuse. Of course all the domain artifacts that are explicitly designed (typically in an SPL context) to be reused must be tested extensively to uncover both functional and non-functional flaws. In an environment where reuse is not explicitly and strategically planned, quality problems may seriously hamper reuse. Domain Knowledge and Software Reuse It has been shown that domain knowledge is still the key to reuse of software [10]. All of our respondents also agreed on this question. Without domain knowledge common and variable artifacts cannot be anticipated. Actually in SPL, the domain knowledge is the thing that is reused, not code. However, not every application engineer must have that knowledge if the reuse infrastructure supports the production process adequately. Several of the respondents believe that domain knowledge is necessary but not sufficient. Organizations that have successfully adopted an SPL approach tend to report the solutions to their problems and underplay what they were already doing well. It has been seen that SPL efforts generated from management, architects, and process engineers are ultimately necessary. Domain knowledge is a significant factor, but not the main key. Management commitment and process discipline to follow through on a reuse agenda that’s driven by business goals are more significant. 4.3 Section 3: Software Reuse Technical Aspects in SPL Many respondents believe that software engineers need to change their programming language to promote reuse. For instance, classes and components are key aspects for reuse. The use of high-level programming languages makes it easier to develop reusable software and use it. However, opinion on the importance of the choice of programming language is divided. The first major reuse store were math libraries written in FORTRAN. The large volume of literature on reuse CASE tools and their growing market show that many organizations regard CASE tools as a way to improve reuse. To study this question, participants were asked whether they agreed with the statement, “CASE tools have promoted reuse across projects in their organization”. The data shows that 50% of the respondents generally feel that CASE tools have not promoted reuse across projects in their organization; while 50% agree that they do. Model Driven Tools and environments are the way to go and less so with the traditional CASE tools. In Model-Drive Development (MDD) and Model-Driven Architecture (MDA) reuse happens at the model level. CASE tools can help reuse design artifacts. Tools with round trip reengineering can help reverse engineer code to better understand the properties of software to enable reuse. We conclude that CASE tools are not currently

6

very effective in promoting reuse. 50% of the respondents said the SPL community is using CASE tools and the other 50% said they are not using them. However approaches such as SOA and some product line tools promote reuse [10]. When asked if given an opportunity to build from scratch or reuse respondents believe that in a product line organization reuse is built in and is clearly not an option. Arbitrarily reusing assets that have not been prepared and proactively planned to be reused for a given task is of course problematic as we know from ad hoc reuse experiences. Respondents are of the opinion that if they find reusable software that fits to the architecture they would prefer to reuse rather than build from the scratch. So, the architecture is the main key issue that facilitates reuse. However one very interesting response was received: “Reuse can be done effectively if we know (1) which domain artifacts are available and (2) we can trust the domain artefacts to do the things we want them to do, why not reuse them? The issue then is do we know and how can we know these issues.” This is a very challenging question posed to SPL community. All the participants are of the opinion that domain knowledge is the key to reuse of software. One cannot decide what software to build as reusable software without understanding the domain and identifying common functionality/features that can be developed as reusable software. According to our respondents the main advantages of the SPL approach are: • The product line’s wide engineering vision can be shared among the projects easily • Development knowledge and corporate expertise can be utilized efficiently across projects, and • Assets can be managed systematically. The SPL approach often requires a large upfront capital investment to create an organizational structure dedicated to implementing a reuse program and it takes time to see a return on investment [9]. 4.4 Section 4: Testing and the Reliability of Reused Software In spite of the enthusiasm of the components community, finding in a library of components, the precise one that will solve the problem at hand is nearly an impossible task. The problem is two fold. Firstly, we cannot find the perfect component in the library of reusable parts. Secondly even if we find the perfect component, is the component reliable enough to use. Having a documented architecture of the software can improve understanding of a system and show how system reliability depends on the components reliability and its interfaces. The behavior of the software with respect to the manner in which different modules of the software interact is defined through the software architecture. We considered several responses in our survey in investigating this issue. Participants were asked to rate their agreement with the statement, “Software developed elsewhere is reliable” and were also asked, “Do you test a component in any way before you reuse it?” Our survey has shown a strong correlation between these variables suggesting that quality concerns are very much related to amount of external reuse. Almost 63% of our respondents stated that they had some element of

reuse in their code but only 40% of these claimed that they tested the reused code in any way. Usually people spend more time testing product specific components. And usually the problems that are discovered are related to reusable components that were not properly adapted for a specific product. When the participants were asked if they “understand system and component reliability, their interactions and the process to identify critical components?”, the respondents said that the architecture of the system helps in the understanding of the above mentioned attributes. There are several piece of research that document some of the methods typically used (and demanded by customers) in certain domains for example in the automotive area. Examples include FMEA (Failure Mode and Effects Analysis), FTA (Fault Tolerance Analysis), and the upcoming ISO WD 26262. In some cases scenario-based approaches like ATAM (Architecture Tradeoff Analysis Method) [5] have been used to identify critical components of a system. In most SPL teams the identification of critical component is based on experience. The understanding of the critical components is mainly derived from source code and documentation, or actual testing of the component. There is no systematic/formal process to understand system and component reliability and the components interactions. System reliability is determined by (field or internal) defect reports – component reliability is determined by how those defect reports are caused by any component. Informal architectural reviews are used to analyze systems and component interactions. The majority of the survey respondents have stated that to find reliable components an architecture review is the answer. When the participants were asked if they “test a single component, class or core component in any way?” the respondents said that single classes are typically not tested. Most of the respondents said they do not test single components by themselves. Sometimes single components are tested, but most of the time some integration with other components is already done and the integrated components are tested. Sometimes unit testing is performed on an assembly of components, as the simplest “unit” of test. Unit testing is not widespread. One of the respondents said that perhaps 10% of the components have some sort of unit test. JUnit, Window Tester, httpUnit, and Rational Test Manage are the main test frameworks used in SPL. A traditional type of structured approach is used when testing software which involves unit and integration testing. Unit testing focuses on internals of a component. Integration testing is carried out on the combined usage of components. The same testing is done at the model and code levels. Some respondents also said that they don’t do component testing as they have trust in the Eclipse process. Some of the respondents are of the view that, because a component is always developed from the requirements of a product, it must be tested to determine if it fulfils those requirements. Reusing software components may also alter time allocated in the software development phases. If a component is already available for reuse then the analysis and design phase time can be minimized. Test cases can be reused if documented properly and in sync with the current version of the core asset. The results we have collected say that between 5%-20% of the time is spent in the analysis phase, 5%60% of the time is spent in the design phase, 0%-50% in the implementation phase and 10%-30% for the testing phase. Some of the respondents said they spend 0% of time in implementation and 50% in testing.

8

4.5 Section 5: Development Environments for Reuse Survey participants were asked what sort of development environment they usually use. The majority of the respondents preferred Java Eclipse, Netbeans or a simple programming IDE, e.g. JCreator. Some of the respondents use a model based approach, UML2 as a standard modelling language extended by ontology oriented design models and tools. Nowadays, Eclipse is used quite a lot as a tooling platform. A specific environment for embedded systems/microcontrollers e.g., Java projects Eclipse and for PC-based applications typically Visual Studio is used. One of the responses from a project designer was that the environment or programming language depends on the Programmable Logic Controller (PLC) used, as every PLC brand has its own environment for the software created for that PLC. When asked if in their opinion, the choice of framework (e.g., .NET, EJB, CORBA, etc.) affects the possibility for the software to be easily upgradeable in the long term, the responses we received were mixed. One of our respondents was not familiar with these frameworks, some did not agree and the rest believed that the mainstream frameworks would probably be more upgradeable than others. Most of our respondents also agreed that complexity of a component does not affect the decision to develop or reuse the component. If a component has been previously tested, it is used regardless of its complexity. However, one of our respondents believed that complexity and domain knowledge could have an effect on such decisions but this really depends on the business strategy. We received a very positive response on the need of well documented software architecture of the system in order to reuse the code. Well documented software/system architectures are very important to support decision making about reuse and to correctly integrate the different components to reduce the testing time. However, few of the respondents believe that as long as domain expertise remains adequate it is not essential to have architectural details. Experts are good at storing domain knowledge. Architecture helps a lot in reuse but may not always be the case. This leads us to a question what if the experts leave an organization. This is an important unsolved issue with legacy systems where experts have left and organization and the documentation is out of sync [7]. All of our respondents believe that most software product lines must be maintained and must evolve over time. Core asset architect respondents believe that maintaining and evolving core assets and integrating core assets in application engineering are the biggest problems for SPL.

5. Conclusion and Future Work In this paper, the issues and concerns of software reuse in SPL were gathered from a survey of members of the SPL community. Some of the key findings are that domain knowledge is not the only key to reuse in SPL. While many assets within the SPL can be reused, priority should be given to the product line architecture. It is not essential to have a well documented architecture for reuse in SPL as SPL has a well defined reuse process. However an SPL process should enforce documentation of the architecture. The issue with the reuse is that the variability should be described in domain requirements artefacts and adequate traceability of the artefacts within and between interacting domain and application requirements engineering life-cycle needs

to be maintained. Also there is no systematic/formal process to understand system and component reliability and the components interactions. Software architecture may help identify critical components Existing tools and techniques do not address some of the most important practical needs in software product lines. We believe this paper will be of interest to software product line practitioners as well as those organizations struggling with reuse as they will gain the following from the paper: • Software architecture can be used in a product line setting to support better identification, reuse, and integration of reliable components. Documentation of software architecture requires a substantial amount of effort. • The list of issues and concerns given in this paper can be used to implement a new software reuse process in SPL. • Current product engineering and derivation processes in SPL can be analyzed for systematic reuse of software in the SPL. Today’s software product lines can be large and have complex variability that they must be supported by a Knowledge Base Software Reuse System (KBSRS), otherwise a systematic approach to software reuse may not be possible. Part of our future work is based on the development of a KBSRS that will be used to capture expertise on software reuse within organizations. We have previously conducted a similar survey to find issues and concerns of software reuse in the conventional software engineering community. We will be comparing the findings of this survey and the previous survey to compare both communities and to identify lessons that can be shared across both communities.

References 1. McIlroy D., “Mass Produced Software Components”, Software Engineering Concepts and Techniques: Proceedings of the NATO Conferences, Buxton, J M, Naur, P., and Randall, B. eds., Petrocelli/Charter, 1969, Pages: 88-98. 2. Hoffman D. M., and Weiss D. M., Software Fundamentals: Collected Papers by David Parnas, Addison-Wesley 2001. 3. Torkar R., and Mankefors S., “A Survey on Testing and Reuse”, Proceedings of the IEEE International Conference on Software-Science, Technology & Engineering, 0-7695-2047-2/03, 2003. 4. Jha M., O’Brien L., and Maheshwari P., “Identify Issues and Concerns in Software Reuse”, Proceedings of Second International Conference on Information Processing (ICIP’08), Bangalore, India, 2008. 5. Clements P. and Northrop L., Software Product Lines: Practices and Patterns. Addison Wesley: Boston, MA, 2002. 6. Birk A., Heller G., et al., “Product Line Engineering: the State of the Practice “, IEEE Software, Vol. 20(6): Pages: 52-60, 2003. 7. Jha M. and Maheshwari P., “Reusing Code for Modernization of Legacy Systems”, IEEE International Workshop on Software Technology and Engineering Practice (STEP), September 24-25 Budapest, Hungary, 2005.

10

8. Kruger C.W. “Software Reuse”, ACM Computing Surveys, Vol. 24 No. 02 June, Pages 131-183, 1992. 9. Weiss D. M. and Lai C. R., Software Product Line Engineering: A FamilyBased Software Development Process. Addison-Wesley, 1999. 10. Pohl K., Bockle G., and Linden F. v.d., Software Product Line Engineering: Foundations, Principles, and Techniques, 1st ed. New York, NY:Springer, 2005. 11. Linden F. v.d, “Software Product Families in Europe: The ESAPS & CAFÉ Projects”, IEEE Software, Vol. 13, No. 3, Pages: 41-49, 2002.