Information and Software Technology

Information and Software Technology 53 (2011) 121–136 Contents lists available at ScienceDirect Information and Software Technology journal homepage...
0 downloads 0 Views 2MB Size
Information and Software Technology 53 (2011) 121–136

Contents lists available at ScienceDirect

Information and Software Technology journal homepage: www.elsevier.com/locate/infsof

Components meet aspects: Assessing design stability of a software product line Leonardo P. Tizzei a,⇑, Marcelo Dias a, Cecília M.F. Rubira a, Alessandro Garcia b, Jaejoon Lee c a

Institute of Computing, University of Campinas, Campinas, SP, Brazil Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, RJ, Brazil c Computing Department, Lancaster University, Lancaster, LA, UK b

a r t i c l e

i n f o

Article history: Received 6 October 2009 Received in revised form 25 August 2010 Accepted 26 August 2010 Available online 15 September 2010 Keywords: Product Line Architecture Component-based Development Aspect-Oriented Software Development Design stability

a b s t r a c t Context: It is important for Product Line Architectures (PLA) to remain stable accommodating evolutionary changes of stakeholder’s requirements. Otherwise, architectural modifications may have to be propagated to products of a product line, thereby increasing maintenance costs. A key challenge is that several features are likely to exert a crosscutting impact on the PLA decomposition, thereby making it more difficult to preserve its stability in the presence of changes. Some researchers claim that the use of aspects can ameliorate instabilities caused by changes in crosscutting features. Hence, it is important to understand which aspect-oriented (AO) and non-aspect-oriented techniques better cope with PLA stability through evolution. Objective: This paper evaluates the positive and negative change impact of component and aspect based design on PLAs. The objective of the evaluation is to assess how aspects and components promote PLA stability in the presence of various types of evolutionary change. To support a broader analysis, we also evaluate the PLA stability of a hybrid approach (i.e. combined use of aspects and components) against the isolated use of component-based, OO, and AO approaches. Method: An quantitative and qualitative analysis of PLA stability which involved four different implementations of a PLA: (i) an OO implementation, (ii) an AO implementation, (iii) a component-based implementation, and (iv) a hybrid implementation where both components and aspects are employed. Each implementation has eight releases and they are functionally equivalent. We used conventional metrics suites for change impact and modularity to measure the architecture stability evaluation of the 4 implementations. Results: The combination of aspects and components promotes superior PLA resilience than the other PLAs in most of the circumstances. Conclusion: It is concluded that the combination of aspects and components supports the design of high cohesive and loosely coupled PLAs. It also contributes to improve modularity by untangling feature implementation. Ó 2010 Published by Elsevier B.V.

1. Introduction Software product line (SPL) engineering aims to improve development efficiency for a family of software systems in a given domain. This concept promotes large-scale reuse through Product Line Architecture (PLA) that is common to a variety of similar products in terms of their architectural elements [1,2]. The combination of SPL and Component-based Development (CBD) is a well-known technique to rapidly and efficiently derive products from a set of reusable assets [3]. In the CBD, software systems are developed composing interoperable and reusable blocks called software components [4]. Moreover, software components explicitly expose their ⇑ Corresponding author. E-mail addresses: [email protected] (L.P. Tizzei), [email protected]. unicamp.br (M. Dias), [email protected] (C.M.F. Rubira), alessandro.garcia@ puc-rj.br (A. Garcia), [email protected] (J. Lee). 0950-5849/$ - see front matter Ó 2010 Published by Elsevier B.V. doi:10.1016/j.infsof.2010.08.007

services as provided interfaces and their dependencies as required interfaces. Component-based PLA fosters the explicit representation of component specification and contributes to reduce coupling and increase cohesion, thereby improving SPL modularity and evolvability. It is known that software systems evolve, otherwise their use becomes progressively less satisfactory [5]. In particular, in the context of PLA, there are several different types of evolutionary changes, such as: introduction and removal of features, and changing a mandatory feature to an optional one and vice-versa. They can potentially end up with architecturally-significant changes and thus cause high maintenance costs for the products. As such, it is important for organisations to control the SPL evolution systematically and a stable PLA lies at the heart of it. The reuse of PLA, among other factors, depends on the ability of its design to remain stable [6]. Stable PLA means that it can endure evolutionary changes by sustaining its modularity properties [7].

122

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

As advocated by some authors (e.g. [7–9]), Aspect-oriented (AO) design decompositions is an approach to support modularity and stability of crosscutting features. Nevertheless, there are also initial evidences that the isolate use of AO decompositions can lead to modularity instabilities in various circumstances [10–12]. There has been some studies combining AOSD and SPL [13–16], but most of them focus on structuring implementation assets through the use of aspects to implement features. Few works discuss modularity and assets stability combining the use of aspects on PLA design. This paper presents a comparative study to evaluate the positive and negative impact of different techniques for designing PLAs. The objective of its evaluation is to quantitatively and qualitatively assess to what extent the use of component and aspect promotes architecture design stability in the presence of various types of SPL change. We evaluate the PLA stability of a hybrid approach (i.e. combined application of aspects and components) against the isolated use of component-based, OO, and AO approaches. Our empirical study involved four different implementations of a PLA: (i) an OO implementation, (ii) an AO implementation, (iii) a component-based implementation, and (iv) a hybrid implementation where both components and aspects are employed. Each implementation has eight releases and they are functionally equivalent. We used conventional metrics suites for change impact [17] and modularity [18] to measure the architecture stability evaluation of the 4 implementations. We observed that the hybrid approach provided a more stable PLA in most of the eight releases. In this paper, we have adopted a component implementation model called COSMOS [19,20], which is a representative component model [4], since it has explicit required and provided interfaces, it can be independently deployable and is subject to composition by third parties. This paper is organised as follows: Section 2 presents some concepts about Software Product Lines, Aspect-oriented software development, and COSMOS implementation model. Section 3 describes the empirical study, followed change impact analysis (Section 4) and modularity analysis (Section 5). Section 6 discusses some limitations of our study. Section 7 discusses lessons learned and Section 8 presents related works. Section 9 draws conclusions and directions for future work.

2. Background 2.1. Software product lines A Software Product Line (SPL) is a set of software-intensive systems sharing a common, managed set of features that satisfy the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way [2]. A SPL promotes a systematic reuse approach to generate similar products and it is usually described by features that represent systemic properties relevant to a stakeholder. It is possible to create different products from a set of core assets due to the concept of variability. Variability is the capacity that a software system or a software artefact has to be modified, customized and configured to a specific context [21]. Since variability is a key issue for SPL engineering, feature modelling is usually applied to represent the commonalities and variabilities of a software system [22]. Features are attributes of a system that directly affect end-users and they can be categorized in three types: (i) mandatory features, which are common for all products in a SPL, (ii) optional features represent selectable features in a given domain, and (iii) alternative features indicate that only one feature in a group of two or more features can be selected [22,23]. In this way, the variability of a SPL can be represented in a feature diagram, which is a hierarchical decomposition of feature

that captures structural or conceptual relationships among features [22]. A key to the success of SPL implementation is structuring its commonalities and variabilities in Product Line Architecture (PLA) in terms of variable architectural elements, their interfaces and its configuration. In traditional systems, software architecture is a high-level definition of the system in terms of its architectural elements (components, connectors, and configuration) and their functionalities and attributes [1]. In PLAs, there is also variability, which is achieved postponing architectural design decisions, described by architectural variation points. A variation point is the place on PLA where a decision is taken. These variation points should be consistent with the variants in the feature model, its constraints and dependencies [24]. Gomaa [25] extends UML to represent these variation points in the architectural level using stereotypes. Mandatory components have kernel stereotype and optional components have optional stereotype. Since this notation extends UML, it is supported by UML modelling tools and it is the notation we adopt in this paper for representing the PLAs. 2.2. Aspect-Oriented Software Development Aspect-Oriented Software Development (AOSD) [26] is an approach that aims to modularize the crosscutting concerns of both software product lines and single-systems, which employ configuration management to manage different versions of development assets and different software products [27]. These concerns are widely-scoped properties and usually crosscut several modules in the software system. Aspects are the abstractions used to encapsulate otherwise crosscutting concerns [28]. In this paper, the notion of concern is considered equivalent to feature. Therefore, the use of aspects rely on three majors mechanisms to modularize and vary crosscutting concerns: (i) join points are the identifiable executions points, such as method calls or object attribute assignments; (ii) a pointcut is a predicate over dynamic join points, meaning that given a certain dynamic join point, a pointcut can either match this join point or not (at runtime); and (iii) advices are the new behaviour that extend, refine or replace the computation at selected join points. There are a variety of aspect-oriented programming languages with different join point models, such as AspectJ [28,29], Caesar [30], and Hyper/J [31,32]. In AspectJ, a call to a method, a method execution, or an assignment to a variable are examples of join points. For instance, using AspectJ, it is possible to intercept a call to an interface method in order to check whether any parameter is null. 2.3. Component implementation models This section presents the COSMOS implementation model and its extension, the COSMOS-AO implementation model, which combines components and aspects. 2.3.1. COSMOS implementation model According to Szyperski [4], a software component is a unit of composition with explicit provided and required interfaces. It can also be deployed independently and is subject to composition by third parties. The COSMOS implementation model is representative of component models because it supports all these characteristics. The main advantages of COSMOS, when compared with other component models such as Corba Component Model (CCM) [33] and Enterprise Java Beans [34], is twofold. First, COSMOS supports the explicit representation of architectural units, such as components, connectors and configuration, thus providing traceability between the software architecture and the respective source

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

code. Second, COSMOS is considered a platform-independent model, since it is based on a set of design patterns. Based on these advantages, we chose the COSMOS implementation model to realize the component-based PLAs involved in our empirical study. Nevertheless, the study presented in this paper could also be applied to other component models, such as CCM and Enterprise Java Beans. COSMOS defines five sub-models, which address different perspectives of component-based systems: (i) a specification model specifies the components using UML; (ii) an implementation model explicitly separates the definition of the provided and required interfaces of the components from the implementation of its provided services; (iii) a connector model specifies the link between components using connectors, thus enabling two or more components to be connected in a configuration; (iv) a composite components model specifies high-granularity components, which are composed by other COSMOS components; and (v) a system model defines a software component which can be executed straightforward, thus encapsulating the necessary dependencies. Fig. 1a shows an architectural view of a COSMOS component called CompA and Fig. 1b shows the detailed design of the same COSMOS component. We have used UML notation in this paper to model architecture and detailed design and, in Fig. 1, we have omitted operations and attributes for the sake of clarity. This component is internally divided into specification (CompA.spec package) and implementation (CompA.impl package). The specification is the external view of the component, which is also sub-divided into two parts: one that specifies the provided services (CompA.spec. prov package) and the other makes dependencies explicit (CompA. spec.req package). For instance, IManager and IA interfaces are provided interfaces and IB is a required interface. The COSMOS implementation model also defines classes to support component instantiation and to implement provided interfaces. The CompA.impl package has three mandatory classes: (i) a ComponentFactory class, responsible for instantiating the component; (ii) a Facade class that realizes provided interfaces, following the Façade design pattern [35]; and (iii) a Manager class that realizes IManager interface and provides meta-information about the component. ClassA, which supports the implementation of the IA interface, and ClassB, which requires services from other components via IB interface, are example of auxiliary classes. 2.3.2. COSMOS-AO implementation model Crosscutting concerns can hinder the reusability, maintainability, and evolvability of applications [36]. It has been shown that

(a)

(b)

123

(a)

(b)

Fig. 2. (a) An architectural view of a COSMOS-AO component; (b) a detailed design of the COSMOS-AO component.

these concerns also arise at component-based decompositions [30,37], which explains why it is an interesting approach to combine components and aspects. The COSMOS-AO model integrates components and aspects approaches taking advantage of their characteristics in order to improve modularity. Components improve the modularity of software architectures [38], because they are loosely coupled and entail high cohesive modules, while aspects support the separation of crosscutting concerns improving encapsulation and, consequently, the modularity. Fig. 2a shows an architectural view of a COSMOS-AO component called CompB, which advises CompA component. Fig. 2b shows the detailed design of the same COSMOS-AO component. It has a structure similar to the COSMOS component (Fig. 1b), with a CompB.aspects package, which is highlighted in gray. Aspects that advise other components stay inside the CompB.aspects package. For instance, ConcernA aspect advises CompA component. Aspects may also have some auxiliary classes, such as ClassC used via Facade class. For the sake of clarity, we omitted all operations, attributes, and classes inside CompB package as well. CompB package can have a structure similar to CompA package. The COSMOS-AO model uses good AO design practices, such as encapsulation of advices by components [36,39] and symmetric components, that is, there is no difference between components with and without aspects [36,39,40]. In contrast to other approaches, COSMOS-AO model does not depend on any programming language or library. We have used the AspectJ programming language [28] to implement the COSMOS-AO components. Thus, we inherited some limitations from AspectJ, for instance, JavaME [41] does not allow using cflow, an important mechanism in AspectJ. However, AspectJ limitations have a minor influence on the results of this study. 3. Empirical settings

Fig. 1. (a) An architectural view of a COSMOS component; (b) a detailed design of the COSMOS component.

The stability assessment of PLAs involved in this empirical study is based on conventional metric suites for change impact [17] (Section 4) and modularity [18] (Section 5). These metrics were chosen as they have been worked as effective stability indicators in a number of previous studies (e.g. [7,18,42–45]). We also relied on the same change impact and modularity measures from the study of Figueiredo et al. [7], so that we could directly contrast

124

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

their observations with our findings. The study of Figueiredo et al. [7] was based only on the comparison of conventional OO and AO implementations of the MobileMedia SPL (Section 3.1), not including the complementary role of components and aspects on the design of stable SPLs. Then, we have replicated and extended their evaluation in order to compare the PLA stability of a hybrid approach (which combines the use of components and aspects) and the component-based approach to the original implementations of the target SPL (Section 3.1). The first release of both OO and AO versions were componentized thus creating the first release of COSMOS and COSMOSAO MobileMedia, respectively. We evolved the component-based implementations following the same evolution scenarios documented in Figueiredo et al. [7]. Four implementations of MobileMedia were involved in this empirical study: (i) an OO implementation, (ii) an AO implementation, (iii) a COSMOS implementation, and (iv) a COSMOS-AO implementation. The first two implementations were already provided by a previous study conducted by Figueiredo et al. [7] while COSMOS and COSMOS-AO implementations were derived from OO and AO implementations, respectively. 3.1. Target software product line In order to exemplify and evaluate the techniques, we present a SPL called MobileMedia [7], which manipulates photo, music, and video on mobile devices, such as mobile phones. The system uses various technologies based on the Java ME platform, such as SMS, WMA and MMAPI. It is possible to derive 200 products from the last release of MobileMedia SPL. It has two implementations with the same functionalities but implemented with different approaches: the AO implementation has approximately 12 KLOC (for all LOC metrics in this paper, blank lines and comments were ignored) and the OO implementation has 11 KLOC. MobileMedia endured seven evolution scenarios, which led to eight releases. The scenarios comprise different types of changes involving mandatory, optional, and alternative features, as well as non-functional concerns. Both AO and OO implementations of MobileMedia follow the same sequence of evolution scenarios, whose objective is to expose the alternative PLA decompositions to common changes. Table 1 summarises the evolution scenarios in MobileMedia. The first column describes the release in which the changes were applied. The second column describes the features added or changed in the MobileMedia application, and the third column describes the type of change and the type of feature (e.g. mandatory, optional, alternative) included or changed in that release. Fig. 3 presents a partial feature model, following the notation proposed by Ferber et al. [47]. It shows the features of the MobileMedia release 4, such as Add Photo, and Label Photo (which in later releases become Label Media due to the inclusion of Music

Fig. 3. A partial feature model of MobileMedia SPL.

and Video media). To illustrate an evolution scenario, the Favourites optional feature is shown in gray, because it was added in release 4. 3.2. Study definition and execution The objective of this comparative study is to assess quantitatively and qualitatively to what extent the synergistic use of components and aspects promotes architecture design stability in the presence of various types of changes. Thus, we compare 4 design and implementation techniques: (i) OO, (ii) AO, (iii) componentbased, and (iv) a combination of component-based and AO. The results of this comparison allow us to know whether a technique to design and implement has influence on PLA stability. The null and the alternative hypotheses of this study are: H0: there is no difference in PLA stability with respect to the design and implementation technique. H1: the combination of component-based and AO techniques supports PLA stability better than the isolate use of each technique. We evaluate the two hypotheses by analysing the change impact on PLAs (Section 4). First, we evaluate whether the null hypothesis (H0) is true or not. The alternative hypothesis (H1) is evaluated in case the null hypothesis is false (i.e. rejected). The two original implementations of the MobileMedia were the input for our empirical study. In order to execute the comparative study, we have performed the following steps:  Step 1. Refactor the first release (R1) of the original OO implementation to COSMOS implementation.  Step 2. Evolve the refactored COSMOS implementation according to the evolution scenarios described in Table 1, thereby building releases R2–R8.  Step 3. Refactor the first release (R1) of the original AO implementation to COSMOS-AO implementation.  Step 4. Evolve the refactored COSMOS-AO implementation according to the evolution scenarios described in Table 1, thereby building releases R2–R8.

Table 1 Summary of evolution scenarios in MobileMedia. Release

Description

R1 R2

MobilePhoto core Exception handling included (in the AO implementations, exception handling was implemented according to Castor et al.[46]) New feature added to count the number of times a photo has been viewed and sorting photos by highest viewing frequency. New feature added to edit the photo’s label. New feature added to allow users to specify and view their favourite photos. New feature added to allow users to keep multiple copies of photos New feature added to send photo to other users by SMS New feature added to store, play, and organise music. The management of photo (e.g. create, delete and label) was turned into an alternative feature. All extended functionalities (e.g. sorting, favourites and SMS transfer) were also provided New feature added to manage videos

R3 R4 R5 R6 R7

R8

Type of change Inclusion of non-functional concern which is also a mandatory feature Inclusion of optional and mandatory features Inclusion of optional feature Inclusion of optional feature Inclusion of optional feature Changing of one mandatory feature into two alternatives Inclusion of alternative feature

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

125

 Step 5. Collect change impact and modularity metrics for eight (R1–R8) COSMOS releases.  Step 6. Collect change impact and modularity metrics for eight (R1–R8) COSMOS-AO releases.  Step 7. Compare the results of COSMOS-AO against COSMOS, AO, and OO implementations. As a result of Steps 1–4 execution, two new implementations were created (COSMOS and COSMOS-AO implementations), each one with eight releases. The sum of LOC of all eight releases for COSMOS implementation is approximately 33 KLOC and 38 KLOC for COSMOS-AO implementation. During the execution of Steps 1–4, we were strict in following the same design and implementation decisions made by the original MobileMedia developers. For instance, all SPLs follow the same architecture pattern, namely the Model-View-Controller (MVC) [48] and the component-based PLAs were also supported by a component-based process [49]. Regarding the feature implementation of the COSMOS-AO SPL, an optional or an alternative feature is encapsulated by a component and its implementation relies on aspects and auxiliary classes (see COSMOS-AO in Section 2.3.1). It is equivalent to the decision that an optional or an alternative feature implementation is encapsulated by a module in AO SPLs. Thus, every time an optional or an alternative feature is included, at least a module is created in the aspect-oriented PLAs. The implementation of mandatory features in the COSMOS-AO does not use aspects and is realized by multiple PLA elements. In COSMOS SPLs, the implementation of all types of features is realized by multiple PLA elements, which is equivalent to OO SPLs. Another decision was to extract exception handling code according to Castor et al. [46]. Exception handling is the only mandatory feature implemented using aspects in COSMOS-AO and AO SPLs. In order to implement all SPL versions, we have also relied on the same programming languages of Figueiredo’s work [7]: Java and AspectJ [28]. All implementations are functionally equivalent and were also tested to guarantee the same level of correctness and robustness. Fig. 4 illustrates a partial result of the Step 4, a fragment of COSMOS-AO PLA in the release 4. It is highlighted in gray the component FavouritesMgr, which was added in release 4 due to the inclusion of the optional feature Favourites. However, it does not mean that a single module always encapsulates completely the implementation of a feature. A feature can be implemented by several components (feature scattering) or one component can implement two or more features (feature tangling). For instance, the Add Photo feature is implemented by the PhotoMgr and PersistenceMgr components and the PersistenceMgr component implements Delete Photo feature as well. Feature scattering and feature tangling occurs in all MobileMedia implementations (i.e. OO, AO, COSMOS, and COSMOS-AO), but the level

Fig. 4. A partial product line architecture of MobileMedia SPL.

Fig. 5. The number of modules per release for each MobileMedia implementation.

of scattering and tangling differs from each other as it is presented later in this paper (Section 5). We also used the same variability programming mechanisms of the original MobileMedia implementation, namely conditional compilation and aspects. On COSMOS implementations, the use of conditional compilation defined which parts of code should be compiled or not based on the value of preprocessor variables. For instance, to implement the Favourites feature, pieces of code enclosed by #IF Favourites and #ENDIF Favourites directives were added. These pieces of code are triggered by the Favourites preprocessor variable located on a configuration file. On COSMOSAO releases, those parts of code which were enclosed by directives on COSMOS releases, were implemented by aspects. Instead of using directives, pointcuts select join points on the components whose behaviour should be modified. As a result of features inclusion (Steps 2 and 4), the number of modules of each implementation has increased from R1 to R8. Fig. 5 shows the number of modules per release of each implementation of MobileMedia SPL. There are two main causes for different number of modules in each implementation: (i) modules granularity and (ii) different implementation techniques. Regarding the (i) cause, both component-based implementations are more coarse-grained than non-component-based implementations. That is, fewer modules implement the same number of features. There is a trade-off between fine-grained and coarse-grained modules. Finegrained modules increase intermodule communication, which may hinder system’s performance. On the other hand, coarsegrained modules have more complex interfaces and are more affected by changes [38]. Modules granularity was influenced by the component-based process used for developing componentbased MobileMedia implementations (namely UML Components [49]) and also architect’s ability to design components with a proper level of abstraction. It explains why both component-based implementations have fewer modules than non-component-based implementations in each release. We further discuss modules granularity in Section 7. Regarding the (ii) cause, optional and alternative features were implemented by creating a new module which modify the behaviour of existing modules on aspect-oriented implementations. On the other hand, on non-aspect-oriented implementations, the inclusion of new features was not necessarily followed by the creation of new modules, because sometimes these features were implemented by modifying existing modules. It explains why AO PLAs have more modules than OO PLAs, and why COSMOS-AO PLAs have more modules than COSMOS PLAs. Despite these differences, Fig. 5 shows some similarities regarding the number of modules per release of each implementation. For all of them, the increase from R1 to R4 was significantly minor than the increase from R4 to R8.

126

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

During the execution of Step 5 and Step 6, we used the same metric suites of the original MobileMedia empirical study [7]. The majority of the measuring data were collected using tools (e.g. Aopmetrics [50]) and the execution of Step 7 was supported by a statistical analysis tool [51] to double-check the correctness of our calculations.

addition of alternative features (R7 and R8). In R3, both a mandatory and an optional feature were included (see Table 1), however, we have separately analysed the impact caused by mandatory features (presented in Fig. 6) and the impact caused by optional features (presented in Fig. 7). 4.1. Addition of mandatory features

4. Change impact analysis In this section, we discuss the change impact on modules and operations. The more resilient is a PLA design, the less is the change impact on its modules and operations. The change impact on modules is measured by the number of modules added, changed, and removed in each release. The measurement of the changed impact on operations is similar to the change impact on modules. The greater the number of modules or operations affected (i.e. changed, removed or added), the greater is the impact on the PLA. On COSMOS and COSMOS-AO PLAs, a module is equivalent to a component. The change impact metrics have been collected comparing each release to its previous one (e.g. comparing R2 to R1). In the following, we discuss addition of mandatory features (R2 and R3), addition of optional features (R3, R4, R5, and R6), and

Fig. 6 presents the change impact on modules and operations in R2 and R3. Fig. 6a shows the number of added modules in R2, R3, and the total (i.e. the sum of R2 and R3 measures) for each implementation. The other charts in Fig. 6 shows the number of changed and removed modules (Fig. 6b and c, respectively) and also the number of added, changed, and removed operations (Fig. 6d, e, and f, respectively). The charts in Fig. 6 have different scales, and Figs. 7 and 8 follow the same structure. The overall results shows that component-based (i.e. COSMOS and COSMOS-AO) approaches were consistently less impacted than non-component-based approaches. For instance, Fig. 6a shows that the number of added modules in R2 was significantly less for component-based than for non-component-based approaches. In R3, only one module was added in AO PLA. Thus, the

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 6. Change impact due to mandatory feature inclusion.

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

(a)

(b)

(c)

(d)

(e)

(f)

127

Fig. 7. Change impact due to optional feature inclusion.

total number of added modules was higher for the two noncomponent-based solutions than for component-based solutions. Similar differences in the results can be seen in Fig. 6b, d, and e. The use of aspects supports the modularisation of Exception Handling feature, which was included in R2, as well as the use of components [46,52,53]. The combined use of aspects and components improved the modularisation even more than the use of either aspects or components alone. The reason is that while AOP improves separation of Exception Handling feature, components encapsulate the aspects and classes created to handle the exception. Both Fig. 6c and f do not present significantly distinguishable measures because the numbers of removed modules and operations were small for all approaches. COSMOS-AO PLA has slightly better measures in R2 than COSMOS PLA (Fig. 6a and e) and slightly worse measures in R3 than COSMOS PLA (Fig. 6b, d and e). The overall results are alike for addition of mandatory features on COSMOS-AO and on COSMOS PLAs, because the majority of mandatory features in COSMOS-AO were implemented using pure OO, which led to a code structure similar to the COSMOS implementation. Furthermore, the results indicate that component-based approaches accommodate changes

better than non-component-based approaches, for implementing mandatory features. 4.2. Addition of optional features The measures related to the addition of optional features were similar for all implementations. There is a slight advantage for the two non-aspect-oriented approaches, which have the best results. For instance, Fig. 7a presents the number of added modules due to optional features being included. The total number of added modules in COSMOS PLAs was the smallest one. The results in R5 were significantly influenced by a modification on all PLAs in order to use the Chain of Responsibility pattern [35] to handle mobile phone commands. The aim was to increase maintainability of the PLAs. Thus, first we analyse the results ignoring R5 and afterwards we analyse the measures of R5 in detail. A pair wise comparison shows that the number of added modules is either slightly higher or equal on aspect-oriented PLAs than on non-aspect-oriented PLAs (Fig. 7a, R3 and R4). Because aspects are used as glue-code between the base code and the code that implement the optional feature on aspect-oriented PLAs, a high

128

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 8. Change impact due to alternative feature inclusion.

number of modules were added and a low number of modules were changed (Fig. 7b, R4 and R6). Despite the high number of changed operations on AO PLA (Fig. 7e), these changes and removals affected a few modules. In R5, the number of changed operations on component-based PLAs was substantially lower than on non-component-based operations (Fig. 7e, R5). On the other hand, the number of removed operations was lower on non-component-based PLAs than on component-based PLAs (Fig. 7f, R5). Due to the architectural modification, some component interfaces were modified which led to the removal of several operations. On non-component-based PLAs, the architectural modification led to changes in operations instead of their removals. Another consequence of the architectural modification was the high number of changed modules on COSMOS-AO PLA (Fig. 7b, R5). It is the opposite of what happened to all other releases (see Sections 4.1 and 4.3), when the number of changed modules on COSMOS-AO PLA was either the smallest (R2, R6, R7, and R8) or the second smallest (R3 and R4). The impact of the architectural modification was particularly strong on the pointcuts of aspectoriented PLAs. Note that in R5 not just the number of changed modules was high on COSMOS-AO PLA, but also the number of

added and removed operations (see Fig. 7d and f, R5). Although the number of changed modules was low on AO PLA (Fig. 7b, R5), the number of changed operations was the highest (Fig. 7e, R5). 4.3. Addition of alternative features Fig. 8 shows that the two component-based approaches achieved the best measures for the addition of alternative features. The COSMOS-AO PLA was the least impacted on the number of changed modules and operations (see Fig. 8b and e, Total column) and the number of removed modules and operations (see Fig. 8c and f, Total column). The COSMOS-AO approach takes advantage of the obliviousness properties of aspects and the high cohesion of component-based approach to avoid change propagation on other architectural elements. For example, in the R8 of COSMOSAO PLA, we have created a cohesive component that supports and mediates the use of all alternative media (photo, music, and video) decreasing the impact on its architectural elements. It was not possible to create such a component in other PLAs. In the COSMOS and OO PLAs, the code necessary to combine the use of different media types was scattered in several modules. In the AO PLA, it was necessary to create an aspect for each combination of media

129

L.P. Tizzei et al. / Information and Software Technology 53 (2011) 121–136

types. Although COSMOS-AO has the best results for addition of alternative features, the COSMOS-AO PLAs added more modules and operations than COSMOS PLA, because the COSMOS-AO approach relies on pointcuts and advices to extend components creating new ones instead of changing them.

4.4. Discussion The null hypothesis H0 states that there is no difference in PLA stability with respect to the design and implementation technique (Section 3.2). We have analysed PLA stability of MobileMedia in Sections 4.1, 4.2 and 4.3. Section 4.1 shows that the two component-based approaches have the best results for Addition of Mandatory Features. Section 4.2 presents similar results for all approaches, with a slight advantage for OO and COSMOS approaches while adding optional features, and Section 4.3 shows that COSMOS-AO approach has the majority of the best results for Addition of Alternative Features. In this section we statistically analyse change impact metric in order to test H0. Thus, we calculate the total change impact on modules (i.e. the sum of the added, changed, and removed modules) and on operations for each release from R2 to R8. Table 2 shows the total change impact on modules from R2 to R8, for each implementation. For instance, in Table 2, OO row, R2 column, the number 14 represents the sum of added, changed, and removed modules in release 2. A similar table was build for the total change impact on operations and is presented in the MobileMedia study [54]. We use the Friedman two-way analysis of variance by ranks to discover whether the change impact data for each implementation is significantly different [55]. The Friedman analysis tests statistically whether k different samples come from the same population or not. This analysis is suitable for our study because we want to find out whether 4 different design and implementation techniques, namely COSMOS, COSMOS-AO, OO, and OO, are from the same population, that is, have similar results for PLA stability. We refer to Siegel and Castellan Jr. [55] for further details on the procedure. In order to test the null hypothesis, it is necessary to find the probability p of rejecting correctly H0 based on observed data. If p 6 a, where a = 0.10 is the significance level (i.e., the probability of rejecting H0 when H0 is in fact true), then we can reject H0. Details on the calculations are available at MobileMedia study webpage [54]. According to the results both probabilities p for change impact on modules (pmodules = 0.01) and on operations (poperations = 0.07) were less than the significance level (a). Therefore, the results provide us evidence to reject H0. Since we rejected H0, we can test H1. This initial analysis indicates that at least one of the techniques differs from at least one other technique, but it neither determines which one is different nor it tells how many of them are different from each other. By using an extension of Friedman analysis, we can compare the techniques in a pairwise way with the same level of significance (i.e. a = 0.10). Table 3 shows which technique differs from each other based on Friedman analysis extension. The first and the third column show the compared techniques and the second column shows the result of the comparison. The not equal symbol (– ) means that

Table 2 Total change impact on modules in releases R2–R8. Implementations

R2

R3

R4

R5

R6

R7

R8

OO COSMOS* AO COSMOS*-AO

14 5 18 6

9 5 12 8

5 4 5 5

13 13 14 17

13 12 13 8

37 26 45 31

29 19 25 14

Table 3 Friedman analysis for change impact on modules. Technique *

COSMOS -AO COSMOS*-AO COSMOS*-AO COSMOS* COSMOS* AO

Comparison

Technique

– ? ? – – ?

AO COSMOS* OO AO OO OO

two techniques differs from each other and the question mark (?) means that it is not possible to claim that two techniques are different from each other based on the data of this study. For instance, it is possible to claim that the results for change impact on modules are significant when comparing COSMOS-AO against AO techniques. On the other hand, the difference when comparing COSMOS-AO technique against COSMOS technique is not significantly different. The results presented in Table 3 does not suffice to either accept or reject H1, because the amount of data collected is not enough to claim that COSMOS-AO technique is different than COSMOS and OO techniques regarding the change impact on modules. In order to provide a more accurate analysis, we have performed a paired test called Wilcoxon signed rank test [55]. This test allows us to compare two techniques against each other and determine which has the greatest change impact on PLAs. We compared only the pairs of techniques whose differences are significant according to Friedman analysis, namely, COSMOS-AO and AO, COSMOS and AO, and COSMOS and OO (Table 3). Details on the calculations are available at the MobileMedia study webpage [54]. Table 4 presents the results for the comparisons between the techniques. The structure of the table is similar to the Table 3, it is used an inequality symbol (less than (