2
Understanding Defects

To prevent and control software defects, we need to understand them. This chapter explains the nature of software defects, including where they enter into the system, what effects they can have, how to detect them, and what causes them.

To reduce the number and impact of defects in our software, it is important to understand the nature of errors and defects. Almost any error on a project can affect the reliability of the software. Anything that makes it more difficult for project personnel to perform their tasks can negatively impact reliability, even if it does not directly result in placing a defect in the software code. A frustrated, angry, or confused programmer is more likely to make an error resulting in a software defect than a motivated, generally happy, and well-informed programmer. A poor work environment and a lack of good software development tools are examples of defect precursors. Defect precursors do not directly cause a software defect, but they make defects more likely and so are considerations for software reliability. Projects that produce high-quality software tend to be well-run projects. Not all errors or defect precursors result in defects, but reducing errors and precursors reduces the likelihood of defects. Similarly, not all defects produce software faults, and not all software faults result in software failures, but again, reducing them improves our chances of reliable software.

As we want to produce reliable software, our understanding of software defects needs to be tailored to that purpose. To this end, we consider the following:

1. Where defects enter the project system
2. Effects defects can have on the project system
3. How we can detect defects
4. What causes defects
5. How we can handle defects

The first four of these are addressed in Sections 2.1–2.4, while the fifth is covered in Chapter 3. Chapter 4 covers the material in more detail by addressing it for specific phases of a project.

2.1 Where Defects Enter the Project System

Knowing where defects can enter a project system is important because we can use this information to design mechanisms to prevent or detect them. When we think of software defects, we typically think of specific types of errors, such as typographical errors, logical errors, synchronization errors, resource errors, or interface errors, to name just a few, and the software defects that may result from them. These types of errors are obviously important, and we must be able to handle them; however, defects affecting the software can enter a system in almost any phase and through almost anything used to design or produce the software product. Processes and products in one phase are used by later phases to produce the final product, so defects in an early phase may propagate to the final product.

In Chapter 4, we describe six phases that are typical for a project. They are as follows:

1. Concept Development and Planning
2. Requirements and Interfaces
3. Design and Coding
4. Integration, Verification, and Validation
5. Product Production and Release
6. Operation and Maintenance

We also consider management impacts. All of these use processes and produce products that create opportunities to introduce defects. Examples of potential defect sources include a poor understanding of customer needs, imprecise requirements, and not following good configuration control processes. The first two examples are typically from the Concept Development and Planning phase and the Requirements and Interfaces phase, respectively, while the last example can be from any phase. It is also important to realize that defects can be introduced into software that has a low defect density, but these defects may have very serious consequences. Also, correcting a detected defect or adding a feature to mature software may introduce defects. Chapter 4 takes each of these phases and describes it, outlining what defects are typical for each phase and how they can enter the project system. It describes techniques and processes to mitigate these defects and lists some metrics to help monitor progress in each phase.

2.2 Effects of Defects

Software defects manifest themselves in many ways, and understanding this helps us produce more reliable software. Of course, a defect may never manifest itself. For example, if the defective part of the code is never executed, the defect never causes a fault or failure. As we generally try not to write unused code, we will assume that defects have some likelihood of being executed.

We commonly think of software defects as causing software crashes, infinite loops, or incorrect software results. Crashes and infinite loop tend to be readily visible. Incorrect results may be obvious or may be subtle. Other types of defects, such as memory leaks, may manifest themselves even more subtly. Software defects, or “bugs,” are sometimes classified into two types:

1. Mandelbugs: A mandelbug is a software defect whose activation and subsequent behavior is complex and its behavior appears chaotic. An example of a mandelbug is a type of defect jovially referred to as a “heisenbug.” Heisenbugs are altered by the attempts to find them. They may be affected by the timing of the execution, by the memory addresses used, by having debugging tools connected to the system, or any of a large number of other factors. Once introduced into the software, heisenbugs, and mandelbugs in general, can be notoriously difficult to find.
2. Bohrbugs: A bohrbug is a software defect whose behavior is repeatable and predictable. Although the cause of the incorrect behavior may be unknown, they are repeatable if the right conditions are found and applied.

Knowing about these various types of defects helps us plan, carry out, and analyze software tests. However, the possible existence of these subtle and hard-to-find defects is one of the reasons why we should not rely solely on software testing to detect defects. It also adds emphasis to the fact that software testing can only show the existence of defects in software, not the absence of defects. Ultimately, it supports the idea that we need to put an emphasis of defect prevention.

If the only defects that we consider are defects in the software, we are missing opportunities to prevent defects from being introduced into the project system. As previously mentioned, almost any error or defect can increase the likelihood of software defects. For example, a poorly worded requirement may be interpreted differently by different software developers. If two developers are writing different software modules affected by this requirement, the different interpretations may mean that these modules do not work together correctly. Furthermore, the effects may be subtle and difficult to find, meaning that the most cost-effective and schedule-effective way to deal with the defect is by ensuring that the requirements are as clear and precise as possible.

Finally, not all defect effects are equally important. Defects that never manifest themselves are less important than defects that cause critical failures. Improving the reliability of software involves focusing on the defects that are most likely to occur and also on the defects that have the most serious consequences if they do occur.

2.3 Detection of Defects

An effective and efficient software reliability effort requires well-thought-out defect detection and monitoring. Good defect detection and monitoring should:

1. Find errors and defects early when it is most cost-effective and schedule-effective to correct them.
2. Be as complete as practical, finding a high percentage of the errors and defects, and finding them in all processes and products that can significantly affect the software product.
3. Be reliable by not missing too many errors and defects while also not creating too many false alarms and the ensuing unproductive effort.
4. Be cost-efficient and schedule-efficient to perform.

Good defect detection and monitoring should also add confidence in the software and related products. It should provide evidence that it is working, and project personnel should be able to trust the detection and monitoring processes and execution enough that the results can be used as a part of the final sign-off of the software.

Recognizing defect precursors is critical for preventing and removing defects efficiently. For example, knowing that a software defect may be due to a requirement defect informs us that we need to detect requirement issues and therefore institute appropriate processes for doing this. Process and product monitoring is important at each phase of the project, and Chapter 4 covers each in more detail.

There are many ways to identify an error, defect precursor, or defect. Some ways identify weaknesses or problems with the processes that produce a product and others identify issues with a product. Techniques to detect process defects and weakness include the following:

1. Use a process failure modes effects and analysis (FMEA)/failure modes, effects, and criticality analysis (FMECA).
2. Use process reviews, inspections, and independent assessors.
3. Use error brainstorming sessions. Those responsible for a task brainstorm on what errors could occur while performing the task. The list can be used to develop checklists for the errors, and the brainstorming process sensitizes the task...