A Quality Assurance Approach to Technical Debt Software Engineering Research and Practice (SERP) 2014 Zadia Codabux*, Byron J. Williams, Nan Niu Department of Computer Science and Engineering Mississippi State University Starkville, Mississippi [email protected], [email protected], [email protected] Abstract - Technical debt is a metaphor used to reason about lingering issues in software development. It is the result of decisions taken during the development process, which are often made due to resource constraints and aggressive schedules. The consequences of technical debt are that it can impede future development and incur increasing costs. However, technical debt is not always bad as, in some cases, respecting deadlines is more important than clean code. Nevertheless, for achieving quality software, it is crucial to prevent the amount of debt from increasing. In this paper, we describe a method for addressing technical debt using a quality assurance (QA) classification scheme and focus on prevention, reduction, and containment activities. We also highlight techniques and processes that are used to apply quality assurance to technical debt.

Keywords: technical debt, software quality, software testing I. INTRODUCTION Software development is prone to failure. One of the root causes of this failure is the use of predictive processes for complex software. Predictive processes attempt to fully define all requirements upfront. Traditional software development models such as the waterfall model, which involves predictive processes, are not the most appropriate when business needs and technology change rapidly. It is often difficult to establish a complete vision for the software product at inception, list all the requirements clearly, and devise a detailed plan to convert these requirements into the finished product [1]. A shift in thinking occurred when development began to focus on customer needs and the product rather than the processes. This shift gave rise to agile software development, which involves an incremental, iterative approach to project development where the requirements and their solutions evolve through collaboration between development teams and customers [1]. In addition, as stated by Williams and Cockburn, agile development is “about feedback and change,” and agile methodologies are developed to “embrace, rather than reject, higher rates of change [2].”

*Contact Author

One of the main foci of agile software development is quick release of functionality. While so doing, technical debt can arise whenever issues not solved in the current release will have to be addressed in later releases [1]. This technical debt metaphor was coined by Ward Cunningham in the 1992 OOPSLA experience report to describe how long-term code quality is traded for short-term gains such as increased productivity. Cunningham stated, “Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise” [3]. In other words, technical debt is the consequence of tradeoffs made during software development, as illustrated in Figure 1. These tradeoffs include not preserving architectural design; not using good programming practices, conventions and standards; no longer updating documentation; and not performing thorough testing [4]. However, every minute spent on “not-quite-right” code counts as interest towards the debt [3]. The metaphor has since been extended and refers to any imperfect artifact in the software development life cycle [5]. Technical debt can be classified as unintentional or intentional. Unintentional debt is acquired inadvertently, for example, due to low quality of work or the lack of adherence to good programming practices. However, intentional debt is accumulated for strategic reasons, e.g., compromising on proper coding standards in order to deliver software on time. Intentional debt is further classified as short-term or long-term debt [6]. Another classification of technical debt is based on traditional lifecycle phases such as testing debt, design debt, and documentation debt [7]. Documentation debt is the result

of inadequate or outdated documentation. Design debt is the result of any anomaly or imperfection in the source code. Testing debts are tests that were planned but not implemented or executed.

opportunity cost of releasing software now compared to some point in the future. For instance, McConnell points out that when time to market is critical, incurring technical debt might be a good business decision. Other instances where debt can be incurred may be due to time and resource constraints where the software or feature needs to get out of the door and the software will be “fixed” after the release. In addition to timeto-market factor, he also explained how preservation of startup capital contributes towards technical debt. In a startup company, expenses that can be delayed should be, as opposed to expending startup funds on technical debt now. Another case where it might be justifiable to incur technical debt would be near the end of a system’s lifecycle because when a system retires, all the debts retire with it [6]. As explained earlier, technical debt is not always bad if it allows a business to achieve a competitive edge and market share. In such cases, it makes sense to delay the moment when the software is brought in line with standards and best practices. Good technical debt has one or more of the following characteristics [11]:

Figure 1: Technical Debt [8]

Over the last decade, technical debt has gained a lot of attention from the software engineering research community. However, the minimal amount of empirical studies existent in the field indicates that there is still need to bridge the gap between research and practice [9]. This can be achieved by carrying out more empirical studies involving practitioners and identifying best practices from the software engineering literature to address technical debt. Technical debt if not handled, can become a problem to organizations. Seaman et al. [10] enumerated some examples of the consequences of technical debt as  disastrous consequences  large cost overruns  quality issues  inability to add new features without disrupting existing ones  premature loss of a system This paper proposes different practices to prevent, reduce and contain technical debt in an agile software development environment context. The rest of the paper is structured as follows: Section II addresses the impact of technical debt. Section III focuses on the relation between quality assurance and technical debt while presenting a quality assurance approach to managing technical debt. Section IV presents some recommendations for managing technical debt. Finally, Section V concludes the paper. II. THE IMPACT OF TECHNICAL DEBT At the start of a new project, it is common to intentionally incur technical debt in order to achieve some goals. When technical debt is incurred for strategic reasons, it is due to the



 

It has a low interest rate – technical debt with low interest rate can be supported for a longer time frame as the development costs won’t increase over time. Repaying the debt at a later time might cost insignificantly more than it will cost to repay it now. It is being regularly serviced - when technical debt is regularly serviced, the amount of debt will decrease over time, thereby increasing the quality of the software. The work that preceded it had a very high opportunity cost – if there is a high opportunity cost related to the debt, it is preferable to release the software and incur some debt, rather than not releasing and losing money.

The decision to assume the debt must be made explicit and should be the result of a collective decision based on the key stakeholders concerned. Such a decision will normally be taken after assessment of the risks and benefits identified. In addition, the stakeholders must ensure that a process exists to manage the debt and that it is paid back within a set period of time. However, non-strategic debt can be detrimental to the quality of the software. Such situations might include when the debt is taken on without stakeholder approval and the impact of the debt is not properly assessed. It is harder to track and manage technical debt accumulated due to shortcuts taken as a result of the lack of quality assurances processes [6]. This type of debt is comparable to credit card debt as it can be easily incurred and adds up very fast due to compounding interest. Nonetheless, a process to pay back technical debt is needed. The longer the debt repayment is deferred, the harder and more costly it will be to pay it back as the interest charges keep compounding. Technical debt is often quantified as principal and interest. Principal refers to the cost of completing the task at present. Interest is the extra cost added to the principal that

will be needed to complete the task at a later time. These are the basic factors for calculating the Return on Investment (ROI) on resolving the debt if the costs can be determined. Over time, technical debt can lead to degeneration of the system’s architecture. The lack of prompt debt payment can result in technical bankruptcy where an organization’s resources are spent dealing with the inefficiencies created by the debt that has accumulated over time and can no longer keep up [12]. In the worst case, the software might need a complete redesign or need to be rewritten, entire departments are outsourced, customers and market shares lost and customer confidence is lost as the company will be spending more resources on debt servicing rather than focusing on new features [13]. A famous example is Netscape Navigator, which experienced architecture decay over a short time period. Netscape developers wanted to release a newer version of the software but could not because the code was harder to change than expected and the system became unmaintainable. The architecture was difficult to understand and it became almost impossible to add new components [14]. Another example is Visual Query Interface (VQI), a software package that degenerated as the programmers made changes to the system without following the architectural guidelines provided. The programmers introduced design pattern violations which cause unnecessary couplings, misplaced classes (i.e., classes placed in the wrong package), and imported classes not used in the package [15]. This degeneration is referred to as code decay [13]. Code decay can have several root causes. One cause is violating architectural design principles. For example, in a strictly layered system, where a layer can only use the services provided by the layer below. If a developer does not follow the constraint, this change is considered a violation of the architecture. Other causes for code decay include: Time pressure that causes programmers to knowingly postpone refactoring  Writing code without following proper programming conventions  Debugging code improperly  Taking shortcuts to get a working solution as fast as possible 

The above examples illustrate that technical debt gives rise to code decay, which makes code changes harder than they should be. Hochstein et al. [13] reported that in order to find areas of code decay, they used a tool to detect code smells which helped to identify code areas where good design principles were breaking down. A code smell is a surface indication that usually corresponds to a deeper problem in the system [16]. As

a result, code smells are useful to identify areas accumulating technical debt. As illustrated above, technical debt can be disastrous if not handled properly. The next section proposes different techniques to handle technical debt with the aim of producing better quality software. III. TECHNICAL DEBT AND QUALITY ASSURANCE Technical debt can be used to reason about software quality. Seaman et al. pointed out that metrics such as cohesion, coupling, complexity and depth of decomposition for software quality can be used to quantify technical debt. Some examples include: the software has defects and needs to be reworked, the software does not fulfill its requirements, all test cases have not been executed, or missing documentation. Example indicators of technical debt include the number of defects in the system, the number of requirements that remain unfulfilled, the number of test cases that need to be executed or automated and documentation that requires completion updating. Technical debt can be characterized by the amount of work that needs to be done and associated costs in order to address these problems [17]. In order to better select, adapt and use QA techniques most appropriate for specific applications, Tian proposed the following Quality Assurance classification scheme for defects [18]:  Defect prevention  Defect reduction  Defect containment Defect prevention activities consisted of activities whose aim is to prevent certain types of faults from being injected in the software. These activities include eliminating technical ambiguities and correcting human misconceptions about functionality. Another activity is fault prevention or blocking that involves directly preventing missing or incorrect human actions. Defect reduction activities have the objective of detecting and removing faults once they have been injected in the system. This is commonly achieved using inspections, testing, static analysis or observations during dynamic execution. Defect containment activities involve containing the failures to local areas to limit their impact. Activities include using fault tolerant techniques to break the relationship between fault and failure. The QA classification described above has been adapted to technical debt. The following section will depict how technical debt can be prevented, reduced and contained with the overall aim of managing the amount of technical debt in the software, thereby increasing its quality.

A. Technical Debt Prevention The aim of technical debt prevention activities is to proactively reduce the chance of technical debts being injected in the software. Different strategies are proposed to handle the different sources of debt injections. 



Education and Training - In cases that human misconceptions are sources of error, the software professionals can be trained on the organization’s development methodology, technology and tools, and development process. Education and training will reduce the probability of implementing the wrong solutions and subsequent rework in design, coding and testing phases. These issues are potential contributors to the technical debt backlog. Pair Programming - Pair programming is an agile technique whereby two software developers work side by side at a workstation, one writing the code while the other reviews the code. A study carried out by Cockburn et al. found that pair programming improves design quality, reduces defects and enhances technical skills [19]. Such best practices will help reduce unclear, unreadable, duplicated code which is often the source of technical debt.



Test-driven Development - Test-driven development is an approach to software development that requires writing a failed automated test before writing the code that makes the test pass. Once the code passes the test, the cycle is repeated including refactoring the existing code to ensure clean code that always works. Thus, test-driven development reduces risk because developers have a better understanding of what the software should achieve and the code has fewer defects. Consequently, this code does not contribute to the technical debt backlog.



Refactoring - Refactoring is a technique for restructuring code by improving its internal structure without affecting its external behavior. Refactoring improves not only aspects of code quality but also productivity [20]. Examples of refactoring involve deleting unused code and duplicated code.



Continuous Integration - Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily, leading to multiple integrations per day [21]. Continuous integration speeds development and the team is able to deliver at any moment a version of the software suitable for release. In addition, it encourages feedback between customers and the development team, which assists with ensuring proper functionality before set deadlines.



Conformance to process and standards - When the source of the debt is due to non-conformance to processes, standards and conventions then enforcement either at the

organization or project level will help prevent technical debt injections. Such activities theoretically contribute to high quality software, thereby reducing the amount of defects. 

Tools - Moreover, tools that can aid in reducing the chance of debt injections should be utilized; for example, a syntax-directed editor can help to reduce syntactical problems.



Customer Feedback - Getting customer feedback can help clarify any requirements misinterpretations early. This feedback will assist in preventing the wrong solution from being implemented, thus increasing technical debt. Prototyping is a tool that can be used to aid in getting customer feedback by allowing them to interact with the “working system” and communicating their satisfaction or concern before the software is actually deployed [22].



Others – There are other best practices that can contribute to technical debt prevention: o Bug bashes which involve the entire team putting aside their daily activities to review and exercise a part of the code base with the objective of quickly revealing defects. o Having dedicated teams whose primary focus is on reducing technical debt [9]. o Having each development team dedicating one iteration during a set release period towards debt reduction [9]. o Allowing Slack – planning some lower priority tasks that can be dropped if the team gets behind. In this way, the team will deliver the most important functions. o Reflective improvement – developers take a break from software development and try to find ways to improve their processes. This helps to pinpoint processes that are and are not working and modify them to develop a strategy that works better.

However, none of the techniques listed above are 100% foolproof and cannot guarantee prevention of technical debt. In addition, it is very difficult for a company to adopt each and every best practice suggested above. For example, prototyping can be a costly technique and might not be suitable for small companies. A company with limited manpower might not find pair programming a very attractive solution. Refactoring requires some level of expertise and experience with the code. Novice programmers may not be able to refactor large portions of the code. A company might adopt one or more of the suggested techniques and best practices, according to what they think will be most fruitful to them, based on the resources at their disposal and the types of system being developed.

B. Technical Debt Reduction The aim of technical debt reduction activities is to remove as much of the injected debt as possible. There are numerous good practices in agile software development that can benefit technical debt reduction. Such agile practices include code reviews (using static analysis tools), test automation and customer feedback. 

Code reviews using static analysis tools assess the quality of the code without executing the code to reveal violations of recommended programming practices that might affect the quality of the software [4]. These tools help to pinpoint problematic code areas such as long methods, large classes, and duplicated methods and help reduce technical debt [23].



Automating unit tests act as a safety net by providing immediate feedback to the team. In addition, automating regression testing in an automated build process permits growth of solid code while quickly iterating. Automated tests protect the software in the sense that it prevents certain defects from reoccurring without being detected [23].



Customer feedback is an essential agile practice. After an iteration, the new features should be demonstrated to the customers for feedback. Such an initiative will ensure that the development team is on the right track and not accumulating any specification debt.

Performing regular code reviews and automated testing are techniques that can be used to identify and reduce technical debt. Involving the customer provides the added benefit of validating the software before its release. However, despite the availability of static analysis tools, it might not always be possible to carry out extensive code reviews. Often, these tools report many false positives that limit the effectiveness of the tools. Similarly, when all unit tests cannot be automated due to resource constraints, a team can accumulate test debt. Based on their resource availability, a company can decide which of the proposed techniques work best for them. C. Technical Debt Containment Technical debts that bypassed the debt prevention and reduction activities intentionally or unintentionally remain present in the software. It is common practice to have working software deployed at a customer’s site with known technical debt present. For these situations, technical debt containment is the quality assurance approach that must be taken. It is imperative to isolate the impact of known debts so that other parts of the software are not impacted. While there is a lack of techniques proposed in existing literature to contain technical debt, N-version programming (NVP) is one applicable approach.



N-version programming – NVP is the independent generation of functionally equivalent programs from the same initial specification [24]. It is a technique that can be adapted to contain technical debt by introducing duplications in the software. Often, developers are hesitant to refactor complex code for fear of introducing additional defects. When there are independent versions of the software, programmers will be able to refactor one version of the code. If there is a failure in that particular version, the software will still function correctly, due to the multiple versions.

NVP is a technique that can be used for technical debt containment. However, NVP has its limitations. First, it is a technique that relies heavily on the accuracy of the requirement specifications. Also, it is based on the assumption that the software built differently will result in few similar errors. Lastly, the cost with implementing N versions of a program instead of one increases development cost [25]. These restrictions may prevent a challenge to the adoption of NVP, but the need to contain the impact of lingering debts makes it a viable solution. IV. RECOMMENDATIONS FOR MANAGING TECHNICAL DEBT Despite a company’s desire to prevent and reduce technical debt, it is difficult to get rid of all the debt. Gradually, the software development team needs to pay off the debt as a commitment towards ensuring quality software. Laribee proposed a four-phase approach to tackle technical debt [26]: 1. 2. 3. 4.

Identify the technical debt and its location. Prioritize the debt with the help of the concerned stakeholders and the team members. Fix the debt. Repeat phases 1-3.

Phase 1 is a crucial step as it is imperative to perform a technical debt analysis to assess how much technical debt is present in the code base and make it visible. Free tools such as Sonar [27] can aid in the process of code quality transparency. Sonar has a technical debt plug-in that identifies potential issues before they affect delivery and shows the approximate cost to pay back all the technical debt in a module or across different modules as in Figure 2.

Figure 2: Sample Information Extracted from Sonar [23]

Such tools can help build up a technical debt list, each item in the list representing an uncompleted task. For each debt item, the list should contain a description of where the debt is, why the tasks need to be addressed, estimates of the principal and interest, and effort estimates. It is common practice to

maintain the list within the company’s defect tracking system as a debt backlog.

to reduce the debt and transform a system of marginal quality into a sustainable, high quality set of artifacts.

Once technical debts have been identified, the next step is to communicate with the concerned stakeholders so that they are aware of the issues. Next, the decision can be made on how to prioritize the technical debt list such that the most critical items are addressed first.

V. CONCLUSION

Seaman el al. [10] propose 4 distinct decision–making approaches for prioritizing technical debt. In the Cost/Benefit Analysis approach, the principal, interest and interest probability of each technical debt item is assigned ordinal scales of measurements such as low, medium and high, which can help make coarse-grained preliminary decisions. For example, a company may decide to tackle 75% of debt with high interest and 25% of debt with medium interest and defer the ones with low interest. Secondly, the Analytic Hierarchy Process (AHP) assigns weights and scales to different criteria which are used to measure technical debt. Then, a series of pair-wise comparisons is performed between the alternatives to get a prioritized ranking of the technical debt items. The Portfolio approach is based on the return on maximization of investment value and investment risk minimization to decide what technical debts should be addressed first. Lastly, the Options approach is analogous to investing in refactoring the debt item with the long-term objective of facilitating maintenance in the future and thereby saving money. Snipes et al. assess deferred defects to determine the amount of technical debt and prioritize fixes. [28]. They identified the factors that influence the decision as follows:

Technical debt is part of every project and affects different artifacts in the software development life cycle. It is important that technical debt is visible to all concerned stakeholders. An additional good practice is to repay the technical debt within a tolerated time frame. Indeed, if not handled promptly, the consequences can be drastic. Proactively applying prevention and reduction QA activities will ensure that quality artifacts are being produced. Reactively, the technical debt needs to be identified, located and contained. In this paper, we have proposed a set of quality assurance activities that focus on prevention, reduction and containment of technical debt. Prevention activities include pair programming, test-driven development, continuous integration, formal methods, amongst others; reduction activities include customer feedback, code reviews and automated testing; and containment activities include N-version programming. This paper provides a single source for common best practices for technical debt and can help organizations that have newly adopted agile methods to better manage their technical debt and prevent the debt backlog from growing. More mature companies can also benefit from this work as they may decide to include more best practices into their iterations or shift from one technique to another and this list can help guide their choices. REFERENCES [1]

     

Severity - assesses the impact of the defect on the customer’s normal system operations Existence of a workaround - defers the defects to a later release and temporarily fixing the issue using an alternate approach Urgency of fix required by a customer - occurs when the customer requests a fix for a particular defect Effort to implement the fix - refers to the effort and time required to fix and test the defect Risk of the proposed fix – refers to the extent to which the code and functionality will be changed when the defect is fixed Scope of testing required - determines whether regression tests need to be run on the resulting defect fix

The factors listed above are of decreasing order of importance. After technical debt has been identified and prioritized, the technical debt items need to be integrated into the existing development backlog. Identifying and prioritizing technical debts are the initial phases in the whole process. The real challenge for the team is to actually work on the debt backlog

[2] [3]

[4] [5]

[6] [7]

K. Schwaber and J. Sutherland, Software in 30 Days: How Agile Managers Beat the Odds, Delight Their Customers, And Leave Competitors In the Dust. John Wiley & Sons, 2012. L. Williams and A. Cockburn, “Agile software development: it’s about feedback and change,” Computer, vol. 36, no. 6, pp. 39–43, 2003. W. Cunningham, “The WyCash portfolio management system,” in Addendum to the proceedings on Object-oriented programming systems, languages, and applications (Addendum), Vancouver, British Columbia, Canada, 1992, pp. 29–30. A. Vetro, “Using automatic static analysis to identify technical debt,” in 2012 34th International Conference on Software Engineering (ICSE), 2012, pp. 1613 –1615. C. Seaman and N. Zazworka, “IEEE/Lockheed Martin Webinar on Identifying and Managing Technical Debt,” Nov2011. [Online]. Available: http://www.nicozazworka.com/research/technical-debt/. S. McConnell, “Technical Debt.” [Online]. Available: http://www.construx.com/10x_Software_Development/Techni cal_Debt/. [Accessed: 14-Apr-2013]. J. Rothman, “An Incremental Technique to Pay Off Testing Technical Debt.” [Online]. Available: http://www.stickyminds.com/sitewide.asp?Function=edetail& ObjectType=COL&ObjectId=11011. [Accessed: 14-Apr2013].

[8] [9] [10]

[11] [12] [13] [14]

[15]

[16] [17]

[18]

[19]

[20]

[21]

[22] [23] [24]

G. Lipka, “The UX of Technical Debt,” Feb-2011. [Online]. Available: http://commadot.com/the-ux-of-technical-debt/. [Accessed: 14-Apr-2013]. Z. Codabux and B. Williams, “Managing technical debt: An industrial case study,” in 2013 4th International Workshop on Managing Technical Debt (MTD), 2013, pp. 8–15. C. Seaman, Y. Guo, N. Zazworka, F. Shull, C. Izurieta, Y. Cai, and A. Vetro, “Using technical debt data in decision making: Potential decision approaches,” in 2012 Third International Workshop on Managing Technical Debt (MTD), 2012, pp. 45–48. S. Chin, E. Huddleston, W. Bodwell, and I. Gat, “The Economics of Technical Debt,” Cutter IT Journal, vol. 23, no. 10, 2010. T. Theodoropoulos, “Technical Debt – Part 1: Definition.” . L. Hochstein and M. Lindvall, “Diagnosing architectural degeneration,” in Software Engineering Workshop, 2003. Proceedings. 28th Annual NASA Goddard, 2003, pp. 137–142. M. W. Godfrey and E. H. S. Lee, “Secrets from the Monster: Extracting Mozilla’s Software Architecture,” in In Proc. of 2000 Intl. Symposium on Constructing software engineering tools (CoSET 2000, 2000, pp. 15–23. R. T. Tvedt, P. Costa, and M. Lindvall, “Does the code match the design? A process for architecture evaluation,” in International Conference on Software Maintenance, 2002. Proceedings, 2002, pp. 393–401. M. Fowler, K. Beck, J. Brant, W. Opdyke, and D. Roberts, Refactoring: Improving the Design of Existing Code, 1st ed. Addison-Wesley Professional, 1999. C. Seaman and Y. Guo, “Chapter 2 - Measuring and Monitoring Technical Debt,” in Advances in Computers, vol. Volume 82, Marvin V. Zelkowitz, Ed. Elsevier, 2011, pp. 25– 46. J. Tian, “Quality Assurance Alternatives and Techniques: A Defect-Based Survey and Analysis,” 01-Jun-2001. [Online]. Available: https://secure.asq.org/perl/msg.pl?prvurl=http://mail.asq.org/p ub/sqp/past/vol3_issue3/sqpv3i3tian.pdf. [Accessed: 10-Mar2014]. A. Cockburn and L. Williams, “The Costs and Benefits of Pair Programming,” in In eXtreme Programming and Flexible Processes in Software Engineering XP2000, 2000, pp. 223– 247. R. Moser, P. Abrahamsson, W. Pedrycz, A. Sillitti, and G. Succi, “A Case Study on the Impact of Refactoring on Quality and Productivity in an Agile Team,” in Balancing Agility and Formalism in Software Engineering, B. Meyer, J. R. Nawrocki, and B. Walter, Eds. Springer Berlin Heidelberg, 2008, pp. 252–266. M. Fowler, “Continuous Integration,” May-2006. [Online]. Available: http://martinfowler.com/articles/continuousIntegration.html. [Accessed: 12-Mar-2014]. E. Allman, “Managing technical debt,” Commun. ACM, vol. 55, no. 5, pp. 50–55, May 2012. B. Barton and C. Sterling, “Manage Project Portfolios More Effectively by Including Software Debt in the Decision Process,” Cutter IT Journal, vol. 23, no. 10, 2010. L. Chen and A. Avizienis, “N-Version Programming: A FaultTolerance Approach to Reliability of Software Operation,” in Twenty-Fifth International Symposium on Fault-Tolerant Computing, 1995, Highlights from Twenty-Five Years, 1995.

[25] [26] [27] [28]

V. Bharathi, “N-Version programming method of Software Fault Tolerance: A Critical Review,” in National Conference on Nonlinear Systems and Dynamics, NCNSD, 2003. D. Laribee, “Using Agile Techniques to Pay Back Technical Debt.” [Online]. Available: http://msdn.microsoft.com/enus/magazine/ee819135.aspx. [Accessed: 22-Apr-2013]. “SonarQubeTM.” [Online]. Available: http://www.sonarqube.org/. [Accessed: 16-Jan-2014]. W. Snipes, B. Robinson, Y. Guo, and C. Seaman, “Defining the decision factors for managing defects: A technical debt perspective,” in 2012 Third International Workshop on Managing Technical Debt (MTD), 2012, pp. 54 –60.