The Waterfall Model and Agile Methodologies : A comparison by project characteristics Wilfred van Casteren

Academic Competences in the Bachelor 2 assignment: Write a scientific article on 2 Software Development Models September 17, 2016

conferences played a major role in gaining general acceptance for the term software engineering and as a result had a profound effect on how programmers produced code. The motivation for these conferences was that the computer industry at large was having a great deal of trouble in producing large and complex software systems. The essence of this software crisis was that the errors in software systems and the cost of writing that software tended to grow geometrically with the size of a software system. The effect of considering software development an engineering discipline results in serious ramifications on the way in which software was created. Because programmers had their roots in an engineering discipline, an extreme amount of care was taken when crafting code. The cost of computing was rather high so rather than wasting precious computing time, programmers practiced hardware concepts as ’measure twice, cut once’, before running their code on the computer. This cautious nature caused the development of methodologies that enabled project teams to slowly and methodically create their plans for the creation of software systems."

Abstract—Complex software development projects have long been plan-driven processes, but the coming of age of Agile Methodologies has given rise to a more adaptive approach to software/system development. The purpose of this paper is to give a short presentation of two Software Development Models (SDMs), The Waterfall Model and the Agile Methodologies (AM), and compare both SDMs based on project characteristics. Keywords: Software/System Development Model (SDM), Waterfall Methodology, Agile Methodologies, plan-driven approach to software development, adaptive approach to software development, comparison of SDMs by project characteristics, software engineering history

I. I NTRODUCTION Developing complex software systems has been documented and described in depth for the first time in ’Managing the development of large software systems’ by Royce [10] in 1970. Several modern software development concepts were introduced in this writing for the first time. One of these concepts formed the basis for the Waterfall Model. In Section II: The Waterfall Model: An introduction to the history of software engineering, a short explanation and a history of its inception will be given for this Software Development Model (SDM).

At this juncture disclosed by Ganis, Dr. Winston Royce wrote an article on managing large and complex software developments (see Royce [10]). In this article, based on experiences in developing software for planning spacecraft missions, commanding and post-flight analysis, W.Royce describes the fundamentals of software/system development. Part of these fundamentals are still applicable today. In an intermediate phase of his elaboration Royce presents a sequence of phases which form a software development cycle. Bell and Thayer [2] refer to this sequence as the concept of the "waterfall" of development activities and unintentionally were the originators of the designation for the first SDM, the Waterfall Model.

’The Manifesto for Agile Software Development’ by Fowler and Highsmith [4], also known as ’the Agile Manifesto’, and first proclaimed in 2001, is a summary of the values and principles on which Agile development is based. Together with ’The Art of Agile Practice: A Composite Approach for Projects and Organizations’ by Unhelkar [11, chap. 2 and 3] it forms the basis for Section III: Agile Methodologies: Motivation, Principles and Practice. Comparing the Agile Methodologies with the Waterfall Model methodology will be part of Section IV: Agile Methodologies and the Waterfall Model: A comparison by project characteristics. This section mentions some typical Agile projects and one that is typically plan-driven. Its main contribution to this paper is to offer clear characteristics for both typical plan-driven and typical Agile projects. This section is based on Unhelkar [11, chap. 2 and 3] and ’Using risk to balance agile and plan-driven methods’ by Boehm and Turner [3].

B. The Waterfall Model As nobody ever proclaimed the Waterfall Model as such, it is susceptible to multiple interpretations. Literally quoting Ganis [5], "In the Waterfall Model there is often only a small set of artifacts (also called work products, which can include documents, models, or code) that is produced in each phase, validated at the end of the phase, and then used as input for the next phase. These artifacts are considered complete, almost frozen, and revisited only to fix a major issue.". Ganis continues: "What the Waterfall Model method emphasized predominately was the freezing of requirement specifications or the high-level design very early in the life-cycle, prior to engaging in more thorough design and implementation work."

In Section V related work will be presented. Finally a conclusion will be presented in Section VI. II. T HE WATERFALL M ODEL : A N INTRODUCTION TO THE HISTORY OF SOFTWARE ENGINEERING

This, while Munassar and Govardhan [6], in their ’A Comparison Between Five Models Of Software Engineering’, mention two Waterfall Models: "the pure Waterfall Model", illustrated by Figure 1: The Waterfall Model, and "the modified Waterfall Model". In this "pure Waterfall Model", the Software Development Life-cycle (SDLC) can only continue to the next phase when a phase is "signed off" completely. The "modified Waterfall Model" allows iterating between consecutive phases.

To explain the Waterfall Model it is appropriate to give an introduction on the history of software engineering. A. Software engineering history According to Ganis [5], the term software engineering can be traced back to a set of historical conferences hosted by NATO in the late 1960’s. In the fall of 1968 and again in the fall of 1969, NATO hosted a conference devoted solely to the subject of software engineering. At the time, the term software engineering was not in general use, nor widely accepted. Using Ganis [5] words "the prevailing assumption at the time was that it would be beneficial to consider software development as engineering". As for the reasons why Ganis [5] states: "These NATO

Munassar and Govardhan [6] describe the phases in their "pure Waterfall Model" as follows. In the system requirements phase the components for building the system, the hardware requirements, software tools, and other necessary components

1

System Requirements Software Requirements

Architectural Design

Detailed Design

Coding

Testing

Maintenance Fig. 1: The Waterfall Model

are determined and documented. The software requirements phase establishes the expectations to be met by the software and identifies which system requirements the software affects. The software requirements phase takes into consideration the interaction needed with other applications and databases, performance requirements and user interface requirements. The architectural design phase determines the software framework to use, defines the major components and the interaction of those components. The detailed design phase examines the software components defined in the architectural design stage and produces a specification for each component’s implementation. During the coding phase the detailed design specification gets its implementation. The testing phase determines whether the software meets the specified requirements and finds any errors present in the code. The maintenance phase addresses problems and enhancement requests after the software is released.

over comprehensive documentation", "customer collaboration over contract negotiation", "responding to change over following a plan". Though the Agile Manifesto’s principal concern is developing valuable software, Unhelkar [11, chap. 1] states: "Agility has a vital, strategic role to play across all dimensions of business. The close intertwining of almost all business functions with corresponding underlying software systems, causes Agile businesses to require corresponding Agile software systems." A. Agile Principles Unhelkar [11, chap. 2] presents the Agile principles, as stated in ’the Agile Manifesto’ by Fowler and Highsmith [4] in the following understandable way: "The highest priority of the Agile Methodologies is to satisfy the customer through early and continuous delivery of valuable software. This customer satisfaction is achieved by engaging the customer in regular, ongoing conversations. The time spell between requirements elicitation and demonstration of the output is shortened, thereby leading to rapid customer feedback. In addition to accepting the changes that the customer (user) will ask for, this Agile principle encourages the teams following Agile approaches to harness the change for the customer’s competitive advantage.

Munassar and Govardhan [6] consider the Waterfall Model as a baseline for many -more contemporary- Software Development Life-cycle Models, while Dr. Winston Royce can be seen as the founding father of plan-driven software development methodologies. III. AGILE M ETHODOLOGIES : M OTIVATION , PRINCIPLES AND PRACTICE

’The Manifesto for Agile Software Development’, also known as ’The Agile Manifesto’, by Fowler and Highsmith [4], officially announced the foundation of modern Agility. It literally states that "developing Agile is all about uncovering better ways of developing software by doing it and helping others do it".

The frequency of delivery can range from weekly to a couple of months. The shorter timescale of delivery enables visibility, absorption, and feedback by the customer. As the interval between planning and delivery reduces, the customer gets the opportunity to view and understand the product and suggest changes before the development is completed.

The authors/subscribers of the manifesto value: "individuals and interactions over processes and tools", "working software

2

team is better utilized in easing the team’s function, and reduced supervision inculcates a higher degree of trust.

Developers, users, and business stakeholders work together closely. Such close collaboration results in a common understanding between the business problem and the corresponding solution that is meant to help the business. Such direct collaboration is far more valuable than written, contracted documents because it helps the individuals understand the implied meanings behind requirements. These individuals need to be provided with the right environment and support to deliver the solution. Thus, Agile projects give credence to the skills and enthusiasm of the individuals and leverage them to produce visible outcomes. Such self-motivation eliminates the need for minute planning of tasks and their tracking by project managers.

Reflection is an important Agile principle that encourages teams to review and reflect on their output at regular intervals. As a result of this, teams are able to adjust their activities and fine-tune their output. Such teams plan dynamically and collaboratively to improve the way the resources, namely, people, processes, and technologies, are being used. Reflection also encourages team members to consider the many architectural, design, operational, and functional constructs resulting from previous projects that can easily feed into the current project and help improve the quality of the output." B. Agile Practices ’A Survey of Agile Development Methodologies’ by Williams [13] explains that each iteration in an iterative product, is a self-contained, mini-project with activities that span requirements analysis, design, implementation, and test. It sets forth that each iteration leads to an iteration release, which may be only an internal release, that integrates all software across the team and is a growing and evolving subset of the final system.

The principle of face-to-face conversations among various stakeholders creates a common understanding of the problem as well as the goal of the project. In particular, an on-site face-to-face conversation between the users and the developers is the most efficient and effective method of conveying information within the project. These conversations reduce misunderstandings and subjective interpretations and thereby reduce the eventual errors within the deliverables.

Still according to Williams [13], the customer adaptively specifies his or her requirements for the next release based on the observation of the current release, rather than speculating at the start of the project. Williams [13] claims evidence exists of frequent deadlines reducing the variance of a software process and, thus, possibly increasing its predictability and efficiency.

The working software (or product) is maintained in its executable form right from the start of an Agile project. So, when the final delivery of the product occurs, that delivery is only an increment over what was already a working piece of software. When an Agile project commences, the concentration of the team members is on the actual delivery of the working product that is also tested.

Also following Williams [13], the predetermined iteration length used when developing Agile serves as a time-box for the team. Thus scope is chosen for each iteration to fill the iteration length. So, rather than to increase the iteration length to fit the chosen scope, the scope is reduced to fit the iteration length.

Agile aspires for ease in development resulting from collaboration and communication among the sponsors, developers, and users. This opens up opportunities for long-term sustainable, evenpaced development. According to this principle, software (or any product, for that matter) should be produced through a work–life balance rather than excessive hours every week.

The key difference between Agile methods and past iterative methods, following Williams [13], is the length of each iteration. In the past, iterations might have been three or six months long. With Agility, iteration lengths vary between one to four weeks, and intentionally do not exceed 30 days. Moreover Williams [13] claims research has shown that shorter iterations have lower complexity and risk, better feedback, and higher productivity and success rates.

The principle of technical excellence promotes regular and ongoing attention to technical details that would result in excellence in design and development. This principle brings together the motivation of the individuals and their technical know-how to bring to fruition the product resulting from an Agile project. Technical excellence encourages individuals to take up the opportunity and the challenge of excelling in technologies. This individual technical brilliance is encouraged in terms of exploring and applying new technologies to the solution, ongoing improvement of code and design, regular testing, and incorporation of feedback from the customer within the solution.

C. An Agile Methodology example: Scrum Scrum is a widely accepted Agile Methodology. It is more sophisticated and elaborate than other Agile methods. So it makes for a good candidate to be described in some detail. On their website Rawsthorne and Shimp [9] describe a Scrum process from the perspective of the Scrum team. They claim the goal of a Scrum team is to produce and release results that meet the goals and priorities that have been set down by the product owner, typically as a result of working with stakeholders. Normally, they add, before the Scrum team can start producing results, there is a visioning phase (this could also be the first phase in a project), during which the business owner, product owner, and the stakeholders produce a product vision and a product road map. The vision provides the overall focus for the

Simplicity in design, relating to the design of a system, has been based on an understanding that complex systems are never designed or developed as complex systems from the outset. Even the most complex system needs to start off as a simple, basic system with basic constructs. Self-organizing teams are teams that require minimal supervision as the team members align themselves to a common goal and produce excellent outputs. The effort required in managing a

3

Fig. 2: Scrum process as in Awad [1]

phased approach to software development. Unhelkar [11] continues, "while a contemporary Agile approach may seem to be diametrically opposed to this traditional development life cycle, there are some important elements in this life cycle (such as controlling and reporting) that have a positive contribution to make in a composite Agile approach". Furthermore Unhelkar [11] adds to this: "In a waterfall-based life cycle, deliverables such as the analysis model, designs, code, and tests are sequentially dependent on the previous deliverable in this approach. Thus, system design, in such an approach, will not proceed until the analysis model is signed off, and coding will proceed only after the design is signed off."

project, while the road map gives guidance about releases and their goals. Futhermore Rawsthorne and Shimp [9] state, The Scrum team’s purpose is to create a result that satisfies the stakeholders’ needs, wants and desires. Often, they continue, so that more demand for business owners’ services is generated. Moreover they state, this production is done through a series of relatively short, fixed-length iterations, called sprints, in which results are produced by working on items. They add, the sprint length can vary, but this requires more discipline on the part of the team and is not advisable for new, less mature teams. Rawsthorne and Shimp [9] continue, the backlog contains items at all levels of fidelity, from vague wishes/wants/needs to finely detailed requirements. They add, the higher the priority of the item, the more detailed the request should be, so that it will be ready for planning and execution. They set forth, as the project progresses, the product owner and the Scrum team should continuously work with the stakeholders to (re)prioritize the backlog, identify new backlog items, eliminate noise, and refine and generally clean the items list to get it ready for planning.

A. Archetypical projects According to Unhelkar [11], projects that benefit the most from an Agile approach are "greenfield" development projects. Furthermore he adds: "Agile approaches to software development are most conducive in these projects wherein coding is at the center of activities surrounded by Agile best practices. Typically, a small project comprising five programmers and lasting for a period of about 6 months would be ideally placed to make extensive use of Agile principles and practices. An experimental or pilot project in the small category is also likely to benefit with Agility. Such a project can be used to experiment with Agile itself, or may be a part of a new, major development."

The fundamental process flow of Scrum is the sprint, which is a relatively short period of time in which backlog items are converted into results. Figure 2: The Scrum process shows a schematic overview of a Scrum process. IV. AGILE M ETHODOLOGIES AND THE WATERFALL M ODEL : A COMPARISON BY PROJECT CHARACTERISTICS

Unhelkar [11, chap. 3] observes lack of scalability as one of the earliest criticisms of Agile methods. While the Agilists refute this criticism, it is still difficult to visualize some of the Agile practices applicable to large-scale software projects, he beliefs. Unhelkar sees difficulties in Agile practices like

In ’The Art of Agile Practice: A Composite Approach for Projects and Organizations’ by Unhelkar [11, chap. 2] the traditional, waterfall life cycle is described as a linear,

4

TABLE I: Agile and plan-driven home grounds. Project characteristics Application Primary goals Size Management Customer relations Planning and control Communications Technical Requirements Development Test Personnel Customers Developers Culture 1 and 2 3 and 4

Agile home ground

Plan-driven home ground

Rapid value, responding to change Smaller teams and projects

Predictability, stability, high assurance Larger teams and projects

Dedicated onsite customers, focused on prioritized increments Internalized plans, qualitative control Tacit interpersonal knowledge

As-needed customer interactions, focused on contract provisions Documented plans, quantitative control Explicit documented knowledge

Prioritized informal stories and test cases, undergoing unforeseeable change Simple design, short increments, refactoring assumed inexpensive Executable test cases define requirements, testing

Formalized project, capability, interface, quality, foreseeable evolution requirements Extensive design, longer increments, refactoring assumed expensive Documented test plans and procedures

Dedicated, colocated Crack1 performers At least 30% full-time Level 2 and 3 experts; no Level 1B or Level –1 personnel 3 Comfort and empowerment via many degrees of freedom (thriving on chaos)

Crack2 performers, not always colocated 50% Level 3s early; 10% throughout; 30% Level 1B’s workable; no Level -1s 4 Comfort and empowerment via framework of policies and procedures (thriving on order)

Collaborative, Representative, Authorized, Committed, and Knowledgeable. See Table II ’Staff software skill level’.

frequent releases in short development cycles, pair programming, starting the development before completing requirements, and an ever-evolving project direction in a large-scale project. Similarly, development of large and complex software packages (typically, Enterprise Resource Planning packages) that are developed for further customization will find it challenging to use pure Agile, he predicts, as these large-scale packages are developed in a setting that is far removed from the actual users.

mentioned ’Managing the development of large software systems’ by Royce [10] adds depth to any study on SDMs. ’A Survey of Agile Development Methodologies’ by Williams [13] could be a good starting point for a more detailed insight in the different flavors of Agile developing. ’The Waterfall Model in large-scale development’ by Petersen et al. [8] describes how team members experience a Waterfall development in a large-scale environment in practice, and adds to already known flaws and strengths in Waterfall development.

B. Agile and plan-driven project characteristics An overview of typical characteristics in terms of project goals and size, project management style, project tools usage, personnel and culture for both Agile projects and plan-driven projects is presented in Table I, drawn from ’Using Risk to Balance Agile and Plan-Driven Methods’ by Boehm and Turner [3]. Table I refers to Table II when it comes to defining staff developers’ skill levels.

A case study for a company developing Agile large-scale is presented in ’A comparison of issues and advantages in Agile and incremental development between state of the art and an industrial case’ by Petersen and Wohlin [7] elaborates on developing Agile with multiple teams and systems in practice. A framework for deciding whether a project should be developed Agile or in a Waterfall Model or through a model somewhere in between as presented in ’Using risk to balance Agile and plan-driven methods’ written by Boehm and Turner [3], draws the lines along which to think to decide on which Software Development Model to go by.

TABLE II: Staff software skill level Level 3 2 1A

1B

Charasteristics Able to revise a method, breaking its rules to fit an unprecedented new situation. Able to tailor a method to fit a precedented new situation. With training, able to perform discretionary method steps such as sizing stories to fit increments, composing patterns, compound refactoring, or complex COTS integration. With experience, can become Level 2. With training, able to perform procedural method steps such as coding a simple method, simple refactoring, following coding standards and CM procedures, or running tests. With experience, can master some Level 1A skills. May have technical skills, but unable or unwilling to collaborate or follow shared methods.

’Agile development: Mainstream adoption has changed agility’ is a good quantifying article by West et al. [12] to get ideas on which Agile or plan-driven methodologies are popular, how methodologies are perceived by developers, how methodologies could be combined, etc.. VI. C ONCLUSION

V. R ELATED WORK

Since ’Managing the development of large software systems’ Royce [10] many Software Development Models have emerged and evolved. These SDMs could be perceived as being mainly plan-driven or mainly Agile. Both models have their uses, advantages and disadvantages.

The Waterfall Model as a part of a more complete view to software/system development as described in the already

Up to now, only a few publications on companies developing

-1

5

Agile large-scale have seen the light, of which I have already mentioned Petersen and Wohlin [7], this doesn’t mean it is impossible to do so, it merely makes it hard to generalize on too few known cases. This could be an interesting research opportunity.

[8]

R EFERENCES

[9]

[1] MA Awad. A comparison between agile and traditional software development methodologies. University of Western Australia, 2005. [2] Thomas E Bell and Thomas A Thayer. Software requirements: Are they really a problem? In Proceedings of the 2nd international conference on Software engineering, pages 61–68. IEEE Computer Society Press, 1976. [3] Barry Boehm and Richard Turner. Using risk to balance agile and plan-driven methods. IEE Computer Science, 2003. [4] Martin Fowler and Jim Highsmith. The agile manifesto. Software Development, 9(8):28–35, 2001. [5] Matt Ganis. Agile methods: Fact or fiction, 2010. [6] Nabil Mohammed Ali Munassar and A Govardhan. A comparison between five models of software engineering. IJCSI, 5:95–101, 2010. [7] Kai Petersen and Claes Wohlin. A comparison of issues and advantages in agile and incremental development between

[10]

[11]

[12]

[13]

6

state of the art and an industrial case. Journal of systems and software, 82(9):1479–1490, 2009. Kai Petersen, Claes Wohlin, and Dejan Baca. The waterfall model in large-scale development. In International Conference on Product-Focused Software Process Improvement, pages 386–400. Springer, 2009. Dan Rawsthorne and Doug Shimp. Scrum in a nutshell. https://www.scrumalliance.org/community/articles/ 2009/december/scrum-in-a-nutshell, 2009. Winston W Royce. Managing the development of large software systems. In proceedings of IEEE WESCON, number 8, pages 328–338. Los Angeles, 1970. Bhuvan Unhelkar. The Art of Agile Practice: A Composite Approach for Projects and Organizations. CRC Press, 2016. URL http://www.ittoday.info/Excerpts/Art_of_Agile_ Practice.pdf. Dave West, Tom Grant, M Gerush, and D D’silva. Agile development: Mainstream adoption has changed agility. Forrester Research, 2(1):41, 2010. Laurie Williams. A Survey of Agile Development Methodologies. 2007.