Software Project Estimation

Alain Abran, PhD, is a Professor and the Director of the Software Engineering Research Laboratory at Université du Québec, Canada. He is also Co-Chair of the Common Software Measurement International Consortium. He was the international secretary for ISO/IEC JTC1 SC7. Dr. Abran has over 20 years of industry experience in information systems development and software engineering.

Foreword xiii

Overview xvii

Acknowledgments xxiii

About the Author xxv

Part One Understanding the Estimation Process 1

1. The Estimation Process: Phases and Roles 3

1.1. Introduction 3

1.2. Generic Approaches in Estimation Models: Judgment or Engineering? 4

1.2.1. Practitioner’s Approach: Judgment and Craftsmanship 4

1.2.2. Engineering Approach: Modest–One Variable at a Time 5

1.3. Overview of Software Project Estimation and Current Practices 6

1.3.1. Overview of an Estimation Process 6

1.3.2. Poor Estimation Practices 7

1.3.3. Examples of Poor Estimation Practices 9

1.3.4. The Reality: A Tally of Failures 10

1.4. Levels of Uncertainty in an Estimation Process 11

1.4.1. The Cone of Uncertainty 11

1.4.2. Uncertainty in a Productivity Model 12

1.5. Productivity Models 14

1.6. The Estimation Process 16

1.6.1. The Context of the Estimation Process 16

1.6.2. The Foundation: The Productivity Model 17

1.6.3. The Full Estimation Process 18

1.7. Budgeting and Estimating: Roles and Responsibilities 23

1.7.1. Project Budgeting: Levels of Responsibility 23

1.7.2. The Estimator 25

1.7.3. The Manager (Decision-Taker and Overseer) 25

1.8. Pricing Strategies 27

1.8.1. Customers-Suppliers: The Risk Transfer Game in Estimation 28

1.9. Summary – Estimating Process, Roles, and Responsibilities 28

Exercises 30

Term Assignments 31

2. Engineering and Economics Concepts for Understanding Software Process Performance 32

2.1. Introduction: The Production (Development) Process 32

2.2. The Engineering (and Management) Perspective on a Production Process 34

2.3. Simple Quantitative Process Models 36

2.3.1. Productivity Ratio 36

2.3.2. Unit Effort (or Unit Cost) Ratio 38

2.3.3. Averages 39

2.3.4. Linear and Non-Linear Models 42

2.4. Quantitative Models and Economics Concepts 45

2.4.1. Fixed and Variable Costs 45

2.4.2. Economies and Diseconomies of Scale 48

2.5. Software Engineering Datasets and Their Distribution 49

2.5.1. Wedge-Shaped Datasets 49

2.5.2. Homogeneous Datasets 50

2.6. Productivity Models: Explicit and Implicit Variables 52

2.7. A Single and Universal Catch-All Multidimensional Model or Multiple Simpler Models? 54

2.7.1. Models Built from Available Data 55

2.7.2. Models Built on Opinions on Cost Drivers 55

2.7.3. Multiple Models with Coexisting Economies and Diseconomies of Scale 56

Exercises 58

Term Assignments 59

3. Project Scenarios, Budgeting, and Contingency Planning 60

3.1. Introduction 60

3.2. Project Scenarios for Estimation Purposes 61

3.3. Probability of Underestimation and Contingency Funds 65

3.4. A Contingency Example for a Single Project 67

3.5. Managing Contingency Funds at the Portfolio Level 69

3.6. Managerial Prerogatives: An Example in the AGILE Context 69

3.7. Summary 71

Further Reading: A Simulation for Budgeting at the Portfolio Level 71

Exercises 74

Term Assignments 75

Part Two Estimation Process: What Must be Verified? 77

4. What Must be Verified in an Estimation Process: An Overview 79

4.1. Introduction 79

4.2. Verification of the Direct Inputs to An Estimation Process 81

4.2.1. Identification of the Estimation Inputs 81

4.2.2. Documenting the Quality of These Inputs 82

4.3. Verification of the Productivity Model 84

4.3.1. In-House Productivity Models 84

4.3.2. Externally Provided Models 85

4.4. Verification of the Adjustment Phase 86

4.5. Verification of the Budgeting Phase 87

4.6. Re-Estimation and Continuous Improvement to the Full Estimation Process 88

Further Reading: The Estimation Verification Report 89

Exercises 92

Term Assignments 93

5. Verification of the Dataset Used to Build the Models 94

5.1. Introduction 94

5.2. Verification of DIRECT Inputs 96

5.2.1. Verification of the Data Definitions and Data Quality 96

5.2.2. Importance of the Verification of the Measurement Scale Type 97

5.3. Graphical Analysis – One-Dimensional 100

5.4. Analysis of the Distribution of the Input Variables 102

5.4.1. Identification of a Normal (Gaussian) Distribution 102

5.4.2. Identification of Outliers: One-Dimensional Representation 103

5.4.3. Log Transformation 107

5.5. Graphical Analysis – Two-Dimensional 108

5.6. Size Inputs Derived from a Conversion Formula 111

5.7. Summary 112

Further Reading: Measurement and Quantification 113

Exercises 116

Term Assignments 117

Exercises–Further Reading Section 117

Term Assignments–Further Reading Section 118

6. Verification of Productivity Models 119

6.1. Introduction 119

6.2. Criteria Describing the Relationships Across Variables 120

6.2.1. Simple Criteria 120

6.2.2. Practical Interpretation of Criteria Values 122

6.2.3. More Advanced Criteria 124

6.3. Verification of the Assumptions of the Models 125

6.3.1. Three Key Conditions Often Required 125

6.3.2. Sample Size 126

6.4. Evaluation of Models by Their Own Builders 127

6.5. Models Already Built–Should You Trust Them? 128

6.5.1. Independent Evaluations: Small-Scale Replication Studies 128

6.5.2. Large-Scale Replication Studies 129

6.6. Lessons Learned: Distinct Models by Size Range 133

6.6.1. In Practice, Which is the Better Model? 138

6.7. Summary 138

Exercises 139

Term Assignments 139

7. Verification of the Adjustment Phase 141

7.1. Introduction 141

7.2. Adjustment Phase in the Estimation Process 142

7.2.1. Adjusting the Estimation Ranges 142

7.2.2. The Adjustment Phase in the Decision-Making Process: Identifying Scenarios for Managers 144

7.3. The Bundled Approach in Current Practices 145

7.3.1. Overall Approach 145

7.3.2. Detailed Approach for Combining the Impact of Multiple Cost Drivers in Current Models 146

7.3.3. Selecting and Categorizing Each Adjustment: The Transformation of Nominal Scale Cost Drivers into
Numbers 147

7.4. Cost Drivers as Estimation Submodels! 148

7.4.1. Cost Drivers as Step Functions 148

7.4.2. Step Function Estimation Submodels with Unknown Error Ranges 149

7.5. Uncertainty and Error Propagation 151

7.5.1. Error Propagation in Mathematical Formulas 151

7.5.2. The Relevance of Error Propagation in Models 153

Exercises 156

Term Assignments 157

Part Three Building Estimation Models: Data Collection and Analysis 159

8. Data Collection and Industry Standards: The ISBSG Repository 161

8.1. Introduction: Data Collection Requirements 161

8.2. The International Software Benchmarking Standards Group 163

8.2.1. The ISBSG Organization 163

8.2.2. The ISBSG Repository 164

8.3. ISBSG Data Collection Procedures 165

8.3.1. The Data Collection Questionnaire 165

8.3.2. ISBSG Data Definitions 167

8.4. Completed ISBSG Individual Project Benchmarking Reports: Some Examples 170

8.5. Preparing to Use the ISBSG Repository 173

8.5.1. ISBSG Data Extract 173

8.5.2. Data Preparation: Quality of the Data Collected 173

8.5.3. Missing Data: An Example with Effort Data 175

Further Reading 1: Benchmarking Types 177

Further Reading 2: Detailed Structure of the ISBSG Data Extract 179

Exercises 183

Term Assignments 183

9. Building and Evaluating Single Variable Models 185

9.1. Introduction 185

9.2. Modestly, One Variable at a Time 186

9.2.1. The Key Independent Variable: Software Size 186

9.2.2. Analysis of the Work–Effort Relationship in a Sample 188

9.3. Data Preparation 189

9.3.1. Descriptive Analysis 189

9.3.2. Identifying Relevant Samples and Outliers 189

9.4. Analysis of the Quality and Constraints of Models 193

9.4.1. Small Projects 195

9.4.2. Larger Projects 195

9.4.3. Implication for Practitioners 195

9.5. Other Models by Programming Language 196

9.6. Summary 202

Exercises 203

Term Assignments 203

10. Building Models with Categorical Variables 205

10.1. Introduction 205

10.2. The Available Dataset 206

10.3. Initial Model with a Single Independent Variable 208

10.3.1. Simple Linear Regression Model with Functional Size Only 208

10.3.2. Nonlinear Regression Models with Functional Size 208

10.4. Regression Models with Two Independent Variables 210

10.4.1. Multiple Regression Models with Two Independent Quantitative Variables 210

10.4.2. Multiple Regression Models with a Categorical Variable: Project Difficulty 210

10.4.3. The Interaction of Independent Variables 215

Exercises 216

Term Assignments 217

11. Contribution of Productivity Extremes in Estimation 218

11.1. Introduction 218

11.2. Identification of Productivity Extremes 219

11.3. Investigation of Productivity Extremes 220

11.3.1. Projects with Very Low Unit Effort 221

11.3.2. Projects with Very High Unit Effort 222

11.4. Lessons Learned for Estimation Purposes 224

Exercises 225

Term Assignments 225

12. Multiple Models from a Single Dataset 227

12.1. Introduction 227

12.2. Low and High Sensitivity to Functional Size Increases: Multiple Models 228

12.3. The Empirical Study 230

12.3.1. Context 230

12.3.2. Data Collection Procedures 231

12.3.3. Data Quality Controls 231

12.4. Descriptive Analysis 231

12.4.1. Project Characteristics 231

12.4.2. Documentation Quality and Its Impact on Functional Size Quality 233

12.4.3. Unit Effort (in Hours) 234

12.5. Productivity Analysis 234

12.5.1. Single Model with the Full Dataset 234

12.5.2. Model of the Least Productive Projects 235

12.5.3. Model of the Most Productive Projects 237

12.6. External Benchmarking with the ISBSG Repository 238

12.6.1. Project Selection Criteria and Samples 238

12.6.2. External Benchmarking Analysis 239

12.6.3. Further Considerations 240

12.7. Identification of the Adjustment Factors for Model Selection 241

12.7.1. Projects with the Highest Productivity (i.e., the Lowest Unit Effort) 241

12.7.2. Lessons Learned 242

Exercises 243

Term Assignments 243

13. Re-Estimation: A Recovery Effort Model 244

13.1. Introduction 244

13.2. The Need for Re-Estimation and Related Issues 245

13.3. The Recovery Effort Model 246

13.3.1. Key Concepts 246

13.3.2. Ramp-Up Process Losses 247

13.4. A Recovery Model When a Re-Estimation Need is Recognized at Time T > 0 248

13.4.1. Summary of Recovery Variables 248

13.4.2. A Mathematical Model of a Recovery Course in Re-Estimation 248

13.4.3. Probability of Underestimation −p(u) 249

13.4.4. Probability of Acknowledging the Underestimation on a Given Month −p(t) 250

Exercises 251

Term Assignments 251

References 253

Index 257