Defect Prevention: Reducing Costs and Enhancing Quality

“Prevention is better than cure” applies to defects in the software development life cycle as well as illnesses in medical science. Defects, as defined by software developers, are variances from a desired attribute. These attributes include complete and correct requirements and specifications as drawn from the desires of potential customers. Thus, defects cause software to fail to meet requirements and make customers unhappy.

And when a defect gets through during the development process, the earlier it is diagnosed, the easier and cheaper is the rectification of the defect. The end result in prevention or early detection is a product with zero or minimal defects.

How serious are defects in software development? In the United States, up to 60 percent of software developers are involved in fixing errors, Computer Finance Magazine reported in 1998. This fact alone, without consideration of providing the quality needed to please customers, shows the value of preventing software defects.

Advantage of Early Defect Detection

Data to support the need for early fixes of software defects is supplied by several reports. The National Institute of Standard Technology (NIST) published a study in 2002 noting that the cost of fixing one bug found in the production stage of software is 15 hours compared to five hours of effort if the same bug were found in the coding stage.

The Systems Sciences Institute at IBM has reported that the cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase (Figure 1).

Figure 1: Relative Costs to Fix Software Defects (Source: IBM Systems Sciences Institute)

Can software defects be prevented by simply logging them into some “defect tracking tool/system,” documenting them and providing fixes for them? The obvious answer is no, though this is the first step toward defect prevention.

Defect prevention involves a structured problem-solving methodology to identify, analyze and prevent the occurrence of defects. Defect prevention is a framework and ongoing process of collecting the defect data, doing root cause analysis, determining and implementing the corrective actions and sharing the lessons learned to avoid future defects.

Principles of Defect Prevention

How does a defect prevention mechanism work? The answer is in a defect prevention cycle (Figure 2). The integral part of the defect prevention process begins with requirement analysis – translating the customer requirements into product specifications without introducing additional errors. Software architecture is designed, code review and testing are done to find out the defects, followed by defect logging and documentation.

Figure 2: Defect Prevention Cycle (Source: 1998 IEEE Software Productivity Consortium)

The blocks and processes in the gray-colored block represent the handling of defects within the existing philosophy of most of the software industry – defect detection, tracking/documenting and analysis of defects for arriving at quick, short-term solutions.

The processes that form the integral part of the defect prevention methodology are on the white background. The vital process of the defect prevention methodology is to analyze defects to get their root causes, to determine a quick solution and preventive action. These preventive measures, after consent and commitments from team members, are embedded into the organization as a baseline for future projects. The methodology is aimed at providing the organization a long-term solution and the maturity to learn from mistakes.

Most of the activities of the defect prevention methodology require a facilitator. The facilitator can be the software development project leader (wearing another hat of responsibility) or any member of the team. The designated defect prevention coordinator is actively involved in leading defect prevention efforts, facilitating meetings and communication among team and management, and consolidating the defect prevention measures/guidelines.

The five general activities of defect prevention are:

1. Software Requirements Analysis

Table 1: Division of Defects Introduced into Software by Phase
Software Development Phases	Percent of Defects Introduced
Requirements	20 Perecent
Design	25 Percent
Coding	35 Percent
User Manuals	12 Percent
Bad Fixes	8 Percent
Source: Computer Finance Magazine

Errors in software requirements and software design documents are more frequent than errors in the source code itself, according to Computer Finance Magazine. Defects introduced during the requirements and design phase are not only more probable but also are more severe and more difficult to remove. Front-end errors in requirements and design cannot be found and removed via testing, but instead need pre-test reviews and inspections. Table 1 shows the defects introduced during different phases of the software development life cycle.

From the studies made by various software development communities, it is evident that most failures in software products are due to errors in the requirements and design phases – as high as 64 percent of total defect costs (Figure 3), according to Crosstalk, the Journal of Defense Software Engineering.

Figure 3: Origin of Software Defects (Source: Crosstalk, the Journal of Defense Software Engineering)

Hence, it is important to have a proper process of analyzing the requirements in place to ensure that the customer needs are correctly translated into product specifications. Perhaps two or three iteration of interactive sessions with the customer can be of great help in verifying the understanding of the developer about actual requirements.

2. Reviews: Self-Review and Peer Review

Self-review is one of the most effective activites in uncovering the defects which may later be discovered by a testing team or directly by a customer. The majority of the software organizations are now making this a part of “coding best practices” and are really increasing their product quality.

Often, a self-review of the code helps reduce the defects related to algorithm implementations, incorrect logic or certain missing conditions. Once the developer feels they are ready with the module code, a glance through the code and understanding what it does compared to what it is supposed to do, would complete the self-review.

Peer review is similar to self-review in terms of the objective – the only difference is that it is a peer (someone who understands the functionality of the code very well) who reviews the code. The advantage is that of a “fresh pair of eyes.”

3. Defect Logging and Documentation

Effective defect tracking begins with a systematic process. A structured tracking process begins with initially logging the defects, investigating the defects, then providing the structure to resolve them. Defect analysis and reporting offer a powerful means to manage defects and defect depletion trends, hence, costs.

A defect logging tool should document certain vital information regarding the defect such as:

◈ Correct and complete description of the defect – so that everyone on the development team understands what it is and how to reproduce it.

◈ Phase at which it is found – so that preventive measures can be taken and propagation of the defect to next phase (software build) is avoided.

◈ Further description of the defect by screenshots.

◈ Names of those who uncover defects – so everyone knows who to contact for a better understanding of the defect.

4. Root Cause Analysis and Preventive Measures Determination

After defects are logged and documented, the next step is to analyze them. Generally the designated defect prevention coordinator or development project leader facilitates a meeting to explore root causes.

The root cause analysis of a defect is driven by three key principles:

◈ Reducing the defects to improve the quality: The analysis should lead to implementing changes in processes that help prevent defects and ensure their early detection.

◈ Applying local expertise: The people who really understand what went wrong are the people present when the defects were inserted – members of the software engineering team. They can give the best suggestions for how to avoid such defects in the future.

◈ Targeting the systematic errors: There may be many errors or defects to be handled in such an analysis forum; however, some mistakes tend to be repeated. These systematic errors account for a large portion of the defects found in the typical software project. Identifying and preventing systematic errors can have a big impact on quality (in terms of defects) for a relatively small investment.

With these guidelines, defects are analyzed to determine their origins. A collection of such causes will help in doing the root cause analysis. The defects are classified based on their types. A Pareto chart is prepared to show the defect category with the highest frequency of occurrence – the target. An example of defect classification in a Pareto chart is shown in Figure 4.

Figure 4: Example of Defect Classification in a Pareto Chart

Root cause analysis is the process of finding and eliminating the cause, which would prevent the problem from recurring. Finding the causes and eliminating them are equally important.

Each defect category and the causes making those defects happen can be represented using a cause-and-effect diagram, as shown in Figure 5.

Figure 5: Cause-and-Effect Diagram for a Defect

The cause-and-effect diagram, also known as a fishbone diagram, is a simple graphical technique for sorting and relating factors that contribute to a given situation. A team usually develops the cause-and-effect diagram in a facilitated brainstorming session. Once the root causes are documented, finding ways to eliminate them requires another round of brainstorming. The object is to determine what changes should be incorporated in the processes so that recurrence of the defects can be minimized.

5. Embedding Procedures into Software Development Process

Implementation is the toughest of all activities of defect prevention. It requires total commitment from the development team and management. A plan of action is made for deployment of the modification of the existing processes or introduction of the new ones with the consent of management and the team. Some of the other activities in this phase of defect prevention are:

◈ Monthly status of the team should mention the severe defects and their analyses.

◈ Fortnightly or monthly (based on the project schedule) meetings to make the team aware of the systematic errors/defects, their symptoms and solutions.

◈ Embedding the defect prevention measures in software development life cycle processes.

◈ Learning from the previous project’s root cause analysis of defects should be used as the baseline for future projects.

◈ Monitoring the defect prevention progress. Is the number of defects decreasing? Is the development team learning from past mistakes?

Finally, defect prevention is not an individual exercise but a team effort. The software development team should be striving to improve its process by identifying defects early, minimizing resolution time and therefore reducing project costs.