Virtual Software Inspections over the Internet

Virtual Software Inspections over the Internet Lasse Harjumaa and Ilkka Tervonen Department of Information Processing Science University of Oulu, P.O....
Author: Clyde Curtis
2 downloads 0 Views 71KB Size
Virtual Software Inspections over the Internet Lasse Harjumaa and Ilkka Tervonen Department of Information Processing Science University of Oulu, P.O. Box 3000, FIN-90014, Oulu, Finland Fax: +358 8 553 1890, e-mail: [email protected], [email protected]

ABSTRACT Software inspections, although they have proved cost-effective, are still practised infrequently or not at all. Their value may be understated and the effort required for arranging inspection meetings is too often considered to be a waste of time. One approach to removing the resistance that inspections frequently meet with is organizing them on a network and automating the most laborious tasks in the process by means of a collaborative software package or software especially intended for reviewing or inspecting documentation. We suggest that the inspection process should be adjusted to allow for increased flexibility and smooth computer support. Furthermore, we are of the opinion that the World Wide Web provides a natural and effortless framework for an inspection tool. Our experiments with web-based inspection tools have pointed to some important challenges that inspection software must face. The tools that we introduce are capable of handling any software documentation presented in HTML. They endeavour to follow the metaphor of seeking defects in paper documents for maximal ease of use, but their most important features are automated material distribution, and instruments for participants to perform the inspection anywhere and at any time. Keywords: Software inspection, World Wide Web, inspection tool, virtual meeting 1

INTRODUCTION

Software inspection, since it was introduced by Fagan in [1], has been widely acknowledged as an effective technique for uncovering defects in all sort of software documentation. Establishing and running the operation may become expensive, however, particularly if the organization lacks previous experience of the process, which often is the case. The costs essentially derive from the laborious nature of software inspection, involving the expenditure of a certain amount of time on related activities not only individually but also in a number of joint meetings. Bringing in inspections always involves the introduction of new working practices to some extent and it may take a while before everybody becomes accustomed to the rigorous inspection routines. Bearing in mind the current fast pace of development cycles in the software industry, it may seem appealing to drop some of the inspection formalities. Arranging the meetings required by traditional software inspection is often tricky, because all participants must come together at the same time in the same place. For a

geographically distributed development team finding a place and time suitable for all members could be virtually impossible, or it could only be done at unreasonable expense [2][4]. Computer support for software inspections has been suggested as a way of removing the bottlenecks in the inspection process. Software tools intended for inspections are typically client-server applications allowing data gathering during inspections, work flow control and management of the inspection material. The tools often focus on technical aspects of inspection data administration. The results of experiments with computerized inspections are interesting. According to Johnson and Tjahjono [7], 72% of inspectors preferred performing reviews using computers instead of the traditional manual method. Correspondingly, Stein et al. state in [10] that distributed, asynchronous software inspections can be a practicable method, although they emphasize that traditional meetings should not be eliminated. Porter [9] reports that face-to-face meetings do not provide any substantial advantages over meetingless inspections, whereas Johnson reports in [6] that thoughtless computerization of the manual inspection process may in fact increase the costs. Existing tools evidently lack some important features, however. The inspection systems may be compelled to fit only a limited part of the actual process, and they may be able to handle just one document format, which is usually plain text. The CSRS system, for example, only provides a means for reviewing code, even though it is certainly one of the most impressive tools for inspection collaboration. Other systems, like hyperCode, are capable of presenting more sophisticated material, but lack adequately defined inspection process or annotation classification capabilities. We recognize that the tools we will introduce in this paper also have considerable insufficiencies, and we will present some ideas for overcoming these difficulties [7][8]. Our approach to software inspection has two objectives: to increase the flexibility of the inspection process and yet to retain the discipline needed to gain the most out of the inspections. At the first glance, these two aims may appear contradictory, but by introducing a comprehending software tool for inspection collaboration, we are convinced that we can achieve both at the same time. The two dimensions of the reorganized inspection process are depicted in Figure 1. This paper will discuss in particular the virtual logging meeting, which embodies the most flexible part of the process, and yet highly disciplined in form, as can be seen in the figure. Conventional inspections are disciplined, but not very versatile. If meetings are replaced by e-mail interaction, for example, the resulting inspection process will have a high flexibility level but will lack rigor. Walkthrough sessions reside in the middle, involving both rigor and flexibility to some extent. Further information about the characteristics of these two dimensions along with several types of inspection aligned to this diagram can be found in article [5].

Inspection with virtual Virtual logging meeting inspection (Virtual inspection)

Conventional Conventional inspection inspection

Walkthrough Walkthrough Inspection Inspection with no meeting without meeting

Flexibility

Figure 1. Discipline and flexibility as dimensions of software inspection. Bringing computer support into the process smoothens metric measurement considerably and should make the management of inspection material and gathered data a straightforward matter. On the other hand, it enforces a proper level of rigor in the form of deadlines and fairly strict workflow control. The most important aspect of computerization, however, is that it enables the most laborious and costly parts of the process to be reorganized into a more effective form. We will discuss this in more detail in the next chapter. The World Wide Web further enhances the feasibility of computerized inspections since the web is global and work in a virtual environment is not limited to normal working hours. The HTML document format is supported by all of the most popular word processors, so that virtually all written material becomes effortlessly available for inspection. Web browsers are already widely installed and used, so that the need for investments in software products is extremely limited. As practically all developers are already familiar with the web platform, the only training needed consists of instruction in acting as a moderator or inspector in the process [4]. In the next chapters we will describe how we have fitted the flexible inspection process into the World Wide Web paradigm. We will then give a brief introduction to two software inspection tools especially designed for the web, and summarize the interesting findings of a couple of experiments assembled with those tools in an organization working in the area of Internet technology. 2

ADJUSTING THE INSPECTION PROCESS FOR THE WEB

What we consider the main benefit achieved by initiating computer-supported – and especially WWW -tool-based – inspections, is the transformation of the process to comply with modern software development demands and principles. The traditional inspection process model does not fit into a web environment without modifications, however [4][6]. First of all, the traditional inspection logging meeting is eliminated and replaced with a public inspection, which is virtual and distributed. This should reduce not only the

costs occasioned by face-to-face meetings but also the time needed to carry out the inspection cycle, because bringing the inspection team members together to one location at the same point of time usually requires major efforts by the moderator. The web approach makes inspection meetings much more elastic, in the form of asynchronicity and geographical dispersal. Furthermore, distribution and storing of the inspection artefacts, including the document to be inspected, related documents and checklists, becomes more convenient and manageable than with printed documents. In fact, dealing with the material electronically is natural, because the documents are always created in electronic format originally. It is no longer necessary to copy all the material and deliver it to each inspector manually. We have previously proposed a modified inspection model which is specially tailored for computer-supported inspections. The refined steps of the inspection process and corresponding phases defined from the inspection tool viewpoint are depicted in Figure 2. Discipline is maintained in the procedure by establishing a set of roles, checklists and overall work flow control. As can be seen in the diagram, the inspection leader carries the main responsibility for keeping the procedure running undisturbed, including setting deadlines for each phase, determining the entry and exit criteria for moving from phase to phase and, of course, starting and closing the inspection.

Inspectors

Legend: =

setting deadline

Inspection leader

Author

Choosing participants, delivering material

No

Kickoff

Ready for inspection? Yes

Individual inspection

Checking and on-line recording

No

Ready for cleaning?

Planning

Rechecking

Yes Removing duplicate issues

Sharing issues

Public inspection

Commenting and on-line recording

Further commenting (may include face-to-face meeting)

No

Ready for classification? Yes

Classification of defects

No

Classification

Ready to proceed? Yes

Studying

Estimating quality & reporting results

Edit & follow-up

Studying

Edit Estimating & & reporting follow-up

Figure 2. Phases of the inspection process. An inspection tool needs to implement five stages to comform fully to this diagram. The kickoff phase is the official starting point of the inspection. Here the inspection leader chooses people for the inspector roles and initiates the schedule for the inspection. The latest version of the material to be inspected must be conveyed into the system and appropriate checklists chosen. In the individual inspection phase, the main concern is exposing suspicious items in the material. Inspectors read through the document and report their findings on-line. Potential problems are initially categorized by the inspector when raising the issues concerned. The inspection leader ensures that all issues reported are unique and relevant, and eliminates those that are not. The list of unique findings is then published to all participants, a new deadline is set and the commenting phase, or public inspection, begins. The inspection leader is also responsible for establishing and maintaining

flexible communication links between the participants during the asynchronous discussion. After they have commented on the issues and reached agreement, the author of the original document walks through the list of findings and performs the final classification of each issue using a similar graphical, on-line tool to that employed in the individual and public inspection stages. Finally, the inspection is closed and becomes available in the form of statistics and study material for subsequent inspections. The defects identified are corrected (by the author) and the solutions are documented and validated. Our process model allows a face-to-face meeting to be arranged if an evident need for one emerges. This could be the case if the domain of the material to be inspected is complex or it is difficult to achieve agreement on the defects [4][11]. 3

THE TOOLS

This chapter will privide concise descriptions of the two tools which we have developed and experimented with at the University of Oulu Web inspection Tool (WiT) and Inspection Window. 3.1 WiT On the basis of some experiments with an early prototype, we have developed a more comprehensive version called WiT (Web inspection Tool), the main feature of which is the ability to inspect any HTML document, including documents containing images. This is a major advantage compared with most inspection software tools, which allow only reviews of plain ASCII documents. The WiT approach is document-oriented. Instead of reading a document through an application or applet embedded into a web page, we are now able to inspect the web page itself. This makes the organization of the client side of the tool more convenient and complies better with the web metaphor, which has proved to be easy to use and has been much appreciated by millions of WWW users. In addition to the document to be inspected, inspectors are presented with the checklists selected for the inspection. Automating the distribution of the inspection material and enabling its on-line viewing are among the main objectives of the tool, along with orchestrating the recording and logging of the issues discovered and the serviceable handling of all administrative data. Altogether, the aim is to provide full support for arranging virtual inspection meetings. WiT accomplishes the process model described in Figure 2 above, with the exception of the follow-up phase. It has recently been integrated with a testing tool called TEWS (Test tool for Windows), however, which provides a means for ascertaining the progress of the inspection outcome, thus enabling the follow-up of defects. Together these tools form a powerful quality assurance aid. Inspection starts with some planning actions by the inspection leader, including scheduling, deciding who should participate, preparing the document to be inspected, and ascertaining that all the other entry criteria have been met. This must be done partly manually. The user interface of the WiT tool in the individual and public phases is depicted in Figure 3. It is divided into two frames, one containing the document to be examined and the other checklists related to the artefact. In many cases, checklists might be

ignored because they are not available in the inspection material distributed or are out of date. The user interface arrangement in WiT, however, makes sure that the checklists are accessible for the inspectors at all times during the reviewing.

Figure 3. Marking a suspect item in WiT. All annotations made to the document are recorded on-line in the inspector's WWWbrowser. No extraneous client software is needed, and the locations of findings are presented as vertical marker lines in the left-hand margin of the document. As soon as an annotation is created, a dialogue box opens, prompting the user to give further information about the finding. The annotation dialogue is presented in the middle in Figure 3. If there were more annotations related to the same location in the document, their identification numbers would be presented in the dialogue and they would be available for browsing through. The location of the suspect item in the document being inspected is marked simply by drawing a marker line in the left-hand margin. These markers are also colour-coded, so that the more severe the classification of the annotation linked to a location is, the more cautioning the tag colour is. Major issues, for example, are indicated in red, minors in blue, and so on. In addition to a free-form description of the annotation, the user is required to state the checklist entry to which a newly raised issue relates and to make a suggestion as to the severity of the issue. Findings are typically categorised as minor or major faults or questions to the author, which are usually misunderstandings between the reader and the designer.

The amounts of time spent individually and in total are recorded automatically by the tool, as well as the number of defects, grouped by types. Statistics composed using this information are useful when examining the effectiveness of inspections, and provide helpful advice for the inspection leader when selecting the most effective inspectors in the kickoff phase, or when determining the most appropriate types of document for inspection. Information on completed inspections can also be used by the inspectors for educational purposes, to learn what to expect when inspecting documents. The inspection workflow is controlled by the tool in the form of e-mail notifications about the actions inspectors need to take at a given point in time. Once the document to be inspected has been stored in the system, material including the artefact, checklists and other documents related to it, and data gathered during the inspection are managed automatically. The WiT tool has been experimented with by computer science students taking part in a Reviewing and Testing course at the University of Oulu. Approximately 50 people used the tool to inspect a requirements specification document. All of them had previously performed traditional inspection with a logging meeting on a similar document. After trying the tool, the students were asked to answer a questionnaire about its usability compared to the traditional inspection. We did not aim at any exact statistical comparison between the methods, but tried to detect informally the strengths and weaknesses of the tool. The results were not surprising; the familiar and easy-to-use user interface was much appreciated, and so was the independence of place and time, but a few reliability problems emerged. These were solved along with the TEWS integration by means of a technology update. The results of the questionnaire are briefly summarized in Table 1.

Pros

Cons

+ Making comments on-line

− Reliability problems

+ Freedom of time and place + Ease of use + Seeing comments made by other people during the public phase Table 1. Results of the student experiment with WiT. The WiT emphasizes the individual inspection phase (traditionally referred as preparation) even more than in Fagan’s or Gilb’s [3] process descriptions. The virtual meeting phase concentrates more on achieving agreement and ensuring joint understanding of all the issues that arise, although it is quite possible to create and report new issues during the debate. 3.2 Inspection Window We have also experimented with the Inspection Window application, which could be called a light version of the WiT tool, implementing only a limited portion of the virtual inspection process. While our foremost aspiration concerning WiT was

completeness of the underlying process, the approach in Inspection Window was fast, painless execution of an inspection. The aim was to make inspector collaboration as effective as possible. In Inspection Window the review is performed during a relatively short period of time selected by the moderator. In this time interval the material to be inspected is available to the inspectors to browse through and comment on. Unlike conventional inspection processes, which typically include about five phases, the process in the Inspection Window tool is mainly implemented by means of a single inspection step, the public phase. For the end user, the Inspection Window tool is exactly the same in appearance as WiT’s public inspection phase. There are two additional actions that are taken by the moderator: starting of the inspection and closing it after adequate agreement has been reached on the quality of the material. This form of inspection is more of a supervised debate than a formal technical review, but as meaningful as any inspection, assuming the inspectors are motivated to do their job – as they should be. As all the issues found in the material are immediately accessible to all the inspectors, the reviewing is a great deal more communicative than in WiT. This should encourage the inspectors to search for errors similar to those already reported, thus uncovering more defects than would have been the case in an investigation carried out completely alone. The Inspection Window tool has been briefly experimented with in a company developing Internet applications, where development is fast-paced, the teams are working extremely dynamically and independently of each other in this hectic environment and there is not necessarily time to carry out the whole inspection cycle. Thus Inspection Window was a more suitable option. The focus of the experiment was on the role of the inspection moderator. The organization in which the tool was tested did not have software inspections mentioned at all in their process model before the introduction of the Inspection Window system, nor had they ever done inspections. They had noticed the potential of the method, however, and were willing to implement inspections on condition that the process was tailored to be more light-weight and flexible than traditional software inspections with face-to-face meetings. While a one-step inspection is very simple to prepare and manage, it still strengthens communication between developers, thus making the development more transparent and public among the team members. Furthermore, with virtual inspections it is easier to make use of outside expertise, even requesting comments from people who may be working even in other organizations. Some experiences gained from installation of the Inspection Window tool are listed in Table 2. The most appreciated features of the tool were its adaptability to a variety of operating system platforms and its attachment to a web browser. Initiating the tool was not a straightforward operation, however, largely because of the weak linkage between Inspection Window and the systems already in use in the development projects. Some of the work previously saved in reviewing and material management is now spent on the extraneous administrative tasks required to transfer documents from the version control system to the inspection repository, forwarding annotations made by inspectors to change requests, etc. The job of the moderator may become quite burdensome, as happened in our experiment.

Pros

Cons

+ Ease of use

− Inadequate integration into other systems

+ Platform independence

+ No investments in additional − Administration of the tool software needed − Inspection of large document bundles − Some reliability problems Table 2. Results of the experiment with Inspection Window. We consider that the integration of inspection tools into other development applications within an organization will be the next great challenge for the implementers of inspection software. This will be a difficult task because of the great divergence in tools and technologies used in development software. Seamless cooperation between separate systems is important for full exploitation of the advantages of computerized inspections. 4

CONCLUSIONS

The research that we have done with our experimental tools indeed backs up the assumption that arranging software inspections in the World Wide Web is a reasonable method of dealing with the bottlenecks encountered in traditional inspections. Replacing troublesome face-to-face logging meetings with procedures entirely performed in a virtual environment is especially suitable for dynamic, geographically distributed development teams, most of which already are communicating over the Internet and probably have other jointly used systems operating in the Web. For them, the setting up and exploitation of a web-based inspection tool is a straightforward matter. The inspection process should be refined to achieve increased flexibility and suitability for computer support and the web. At the same time, however, the desired level of rigor for ensuring the effectiveness of the process can easily be maintained by appropriate design of the supporting tool. The most widely appreciated features of the two tools that we have tested with either students or a real-life software company were the liberty to perform the inspection anywhere and at any time, not necessarily during office hours and the ease of use and availability of the checklists were also appreciated. We hope in future to further enhance the functionality of Web-based inspection tools by introducing technologies such as XML. The next generation of inspection tools should be capable of handling linked documents, and in fact of reviewing of an entire web site instead of just a single document. The current inspection tools – including WiT and Inspection Window – mostly ignore co-operation with other development software used within the organization. Even though inspection tools have sophisticated features, it seems that they concentrate on automating the detection of syntax errors in code and other technological aspects instead of supporting the inspection process as a whole as an integrated part of the

development cycle of a software artefact. Computer-supported inspections relate at least to version control, bug report repositories and project management. A great number of software tools and administrative tasks are involved in each of these categories, and the coordinating of tool co-operation requires further research. There is a great deal of potential in web tools for boosting the popularity of software inspections. They are not necessarily as laborious or as difficult to arrange and coordinate as is often believed. If appropriately integrated into the development environment and other systems, inspection tools, and consequently inspections, can become natural, coherent parts of the cooperative communication and development processes that take place within the organization. 5

REFERENCES

[1] Fagan, M.E.: Design and Code Inspection to Reduce Errors in Program Development, IBM Systems Journal, vol 15, no 3, 1976, pp. 182-211 [2] Glass, Robert: Inspections – Some Surprising Findings, Communications of the ACM, vol 42, no 4, 1999, pp. 17-19 [3] Gilb, T., and Graham, D.: Software Inspection, Addison-Wesley, Wokingham, England, 1993 [4] Harjumaa, L., and Tervonen, I.: A WWW-based Tool for Software Inspection, in Proceedings of the 31st HICSS Conference, vol III, IEEE Computer Society Press, Los Alamitos, CA, 1998, pp. 379-388 [5] Iisakka, J., Tervonen, I, and Harjumaa, L.: Experiences of painless improvements in software inspection, in Project Control for Software Quality, ESCOM-SCOPE’99, Shaker Publishing B.V., 1999, pp. 321-327 [6] Johnson, P.M.: Reengineering Inspection, Communications of the ACM, vol 41, no 2, 1998, pp. 49-52 [7] Johnson, P.M., and Tjahjono, D.: Does Every Inspection Really Need a Meeting?, Empirical Software Engineering, no 3, 1998, pp. 9-35 [8] Macdonald, F.: Computer Supported Software Inspection, PhD thesis, October 1998 [9] Porter, Adam., and Johnson P.: Assessing Software Review Meetings: Results of a Comparative Analysis of Two Experimental Studies, IEEE Transactions on Software Engineering, vol 23, no 3, 1997, pp. 129-145 [10] Stein, M., Riedl, J., Sören, J.H., Mashayekhi V.: A Case Study of Distributed, Asynchronous Software Inspection, in Proceedings of the 19th International Conference on Software Engineering, 1997, pp. 107-117 [11] Tervonen, I., Iisakka, J., and Harjumaa L.: Software Inspection – a Blend of Discipline and Flexibility, in Project Control for 2000 and Beyond (Proceedings of the ESCOM-ENCRESS 98 Conference), Shaker Publishing B.V., The Netherlands, 1998, pp. 157-166 [12] Tervonen, I., Harjumaa, L., and Iisakka, J.: The Virtual Logging Meeting: A Web-based Solution to Resource Problems in Software Inspection, in Proceedings of Sixth European Conference on Software Quality, Vienna, 1999, pp. 342-351