Domain-Specific Model-Driven Testing (eBook)

Dr.-Ing. Stefan Baerisch completed his doctoral thesis at the Department of Software Engineering at the Christian Albrechts University, Kiel. He works as a consultant in Oldenburg.

Foreword 6
Acknowledgements 7
Abstract 8
Contents 10
1 Testing of Software 13
1.1 Testing and Software Quality 13
1.2 Acceptance Testing 24
1.3 Quality of Test Automation 25
1.4 Chapter Summary 27
2 Use of Models in Software Engineering 29
2.1 Roles and Properties of Models 30
2.2 Model-Driven Software Development 33
2.3 Model-Driven Testing 36
2.4 Chapter Summary 38
3 System Families and Variability 40
3.1 Variability in Software Engineering 41
3.2 Feature Modeling 45
3.3 Testing of System Families 49
3.4 Chapter Summary 51
4 Information Retrieval and Digital 52
4.1 Information Retrieval 52
4.2 Digital Libraries 61
4.3 Chapter Summary 66
5 The MTCC Approach 67
5.1 The MTCC Process 67
5.2 Overview of the MTCC Models 81
5.3 Chapter Summary 84
6 Models for System and Test Representation 86
6.1 Models for the Behavior of a Testee 87
6.2 Feature Models for System and Test Representation 93
6.3 Composition of the Application Test Model 103
6.4 Chapter Summary 105
7 Application of MTCC Models 107
7.1 Representation of Test Steps in the Editor 108
7.2 Configuration of Test Models 118
7.3 Reuse of Tests 124
7.4 Test Execution Based on Test Configuration Instances 130
7.5 Chapter Summary 135
8 Application of MTCC to the Digital 136
8.1 A System Family of Digital Libraries 136
8.2 Test-Relevant Services and Test Steps 143
8.3 Generic Services 146
8.4 Domain-Specific Services 148
8.5 Chapter Summary 155
9 Validation Goals and Design 157
9.1 Validation in Software Engineering 157
9.2 Goals and Questions of the MTCC Validation 163
9.3 GQM Application for MTCC 165
9.4 Chapter Summary 179
10 Results of the Validation 181
10.1 Participants in the MTCC Validation 181
10.2 The Prototype of the MTCC Editor 181
10.3 Considered Systems 185
10.4 Capability to Represent Tests 185
10.5 Validation of the Executability 187
10.6 Validation of Reusability 191
10.7 Interpretation of the Validation 197
10.8 Chapter Summary 202
11 RelatedWork 204
11.1 Testing Based on Usage Scenarios 204
11.2 Modeling of Tests 208
11.3 Abstract Representation of User-Interfaces 210
11.4 Testing of System Families 211
11.5 Involvement of Domain Experts in Software Development 214
11.6 Novelty of MTCC 216
11.7 Chapter Summary 217
12 Summary and Contributions 218
12.1 Summary 218
12.2 Contributions 220
13 Outlook and Conclusion 222
13.1 FutureWork 222
13.2 Concluding Remarks 223
A Test Sets 225
Test Set TS1 225
Test Set TS2 225
Test Set TS3a 226
Test Set TS3b 227
Test Set TS4a 227
Test Set TS4b 228
Test Set TS5 228
Bibliography 230

1 Testing of Software (p. 1)

Testing is defined as the execution of a system with the purpose of finding faults in the system. It encompasses a wide range of different approaches with different goals and motivations. In this chapter, we discuss both the role of testing for software quality and the role of MTCC in the field of testing.

We start with an examination of the role of testing in Section 1.1. In order to be more efficient and effective, often, various parts of the testing process are automated. The extent, to which testing is automated, varies with different approaches, but the execution of tests implemented as test scripts and the recording of test results form the core of test automation. Section 1.1.3 examined the goals of test automation and the challenges that must be met to establish an automated testing regime. Acceptance tests involve domain experts in system testing.

Section 1.2 motivates the involvement of domain experts into the testing process and discusses acceptance tests and their automation. The quality of a test is defined by its ability to find faults. Tests must therefore be based on the requirements as defined by the domain experts for a system and must be able the assess a testee based on this requirements.

Section 1.3 discusses the meaning of quality for tests and test software and applies the quality attributes of software engineering to test software.

1.1 Testing and Software

Quality Testing is a method of quality assurance that verifies the behavior of a system against a set of requirements that are expressed as tests. The notion of quality and its relation to the requirements of a system, the specifications that represent these requirements and the needs of the users of a system are central to testing.

If quality is defined as the capability of a system to fulfill the requirements of its users or customers [JG88], the question must be considered how these requirements can be captured and formalized as tests. One source of requirements are domain experts. Domain experts are customers or users of a system who are familiar with the functions that the systems needs to fulfill and who are often experienced in the use of similar systems.

The direct and continuous involvement of domain expert in the software development process as a methodology belongs to the group of agile approaches to software development [BA04] but is considered as part of other approaches to software development as well [Eva04].

One method to make the knowledge of domain experts available for the software development process is the definition of acceptance tests. In acceptance tests, domain experts either test a system themselves or specify the functionality to be tested and the conditions for the success of a test in a format that is either suitable for execution by a human tester or for execution by an automated testing system. One purpose of test automation is to allow the execution of tests without human involvement. Automation is desirable both for economic reasons and to allow for greater effectiveness in testing.

1.1.1 Software Quality

Juran [JG88] defines quality as fitness for use. This definition raises a number of questions: fitness for what use? What constitutes fitness? How can it be determined, if and to what degree a system is fit to be used? In order to evaluate the fitness of a system, some sort of specification is needed.

A specification is a document that describes the requirements for a system. The quality of a system can also be defined as the degree to which its implementation complies with its specification. The specification must therefore reflect the requirements for a system. Functional as well as non-functional quality attributes are relevant to software systems. Software engineering defines a number of methods that serve the purpose of creating a high-quality products or evaluating its quality.

1.1.1.1 Different Notions of Quality

Two questions have to be addressed in the discussion of quality assurance.