In this chapter we presented a formalization and a software prototype for the automatic test generation using model checkers for reactive component system. We contributed ap-proaches for formalizing the artifacts and execution semantics of the test generation. Our contributions were a refinement of the common MBT metamodel with UML based test
4.10. Conclusion & Discussion 111
models, test suites, test selection criteria, and test case specifications. A novel test selec-tion criterion covering parallelism in Statecharts, called All-Configuration-Transitions,was presented, and its semantics were shown. In addition, we showed how other typical test selection criteria, likeAll-StatesandAll-Transitions, can be modeled using our approach. To round up the automatic test generation, we contributed a common workflow for testing with model checkers, which we used in our default implementation of the workflow steps together with the model checker NuSMV to generate test cases.
To achieve the above results, we first refined our formalization to a specific target system type (Sec. 4.2). To classify the system type, we used a taxonomy with seven dimensions where we positioned the systems we target. While this classification clearly shows what system we support, it also spots the general limits of our approach. For example, we choose structural model coverageandtest case specificationsfor selecting interesting test cases. This decision leads us to the design of specific test selection criteria. Based on them, we defined model-to-model transformations to test case specifications, and finally to trap properties and verified counterexamples for these test case specifications. All of these conversions and op-erations depend on the initial decision in the taxonomy, and it is leaved unanswered if our formalization approach is applicable for different test selection methods likedata coverage orstochastic selection. In addition to these limitations, we are also aware of limitations in the supported UML constructs. For example, we selected a subset of possible UML State-chart concepts, and leave concepts liketransition prioritizationandhistory nodesaside. The main reason for this limitation is the complexity of the transformation of the UML based test model to a semantically equal model checking problem. However, we think that our approach can be extended to support more test selection criteria and UML constructs in future with justifiable efforts.
113
CHAPTER 5
Model-Based Testing Of Dynamic Component Systems
5.1. Introduction
The last chapter served us as an intermediate step to show the hypothesis of this dissertation.
We showed that using a common workflow for testing with model checkers, UML as the modeling notation for our test cases, and model transformations for the implementations of the workflow steps, we can systematically generate test cases for reactive component sys-tems. In this chapter, we show how this approach can be applied to generate test cases for DCSs. The core of DCSs is the support of component configurationevolutionduringruntime of the system. Two characteristics allow this evolution, namely theadaption of a component instanceand thereconfiguration of the overall system(Sec. 2.1). In these systems, component instances can withdraw provided functionality at any time, leading to thedynamic availabil-ityof functionality. An important aspect of dynamic availability is that it is not under the control of the component instance using the functionality. This requires that the compo-nent instances have to be prepared to respond at any time to arrival and/or departure of required functionality (Cervantes and Hall, 2003). This poses additional quality require-ments to the development of software components, since missing or incorrect handling of dynamic availability can have an impact of the functionality of the overall system. In terms of software testing, any component code that is not prepared to handle dynamic availabil-ity correctly containserrorswhich might becomefailuresof the overall system. Therefore, a systematic approach to test a component with focus on dynamic availability is desirable.
The overall goal of this chapter is therefore to show how MBT can be used to systematically generate test cases for components with focus on dynamic availability.
The goals we aim to reach in this chapter are the following:
• Show how our formalization of MBT for reactive component systems can be adapted for automatic test generation of test cases for DCSs. The adaption should focus on testing the component’s behavior to dynamic availability of functionality.
• The modeling notation for test models should be kept in the UML to avoid that test designers have to use yet another language.
To reach these goals, the following steps are taken in this chapter:
• We first show what the differences of test cases for common component systems and DCSs are (Sec. 5.2).
• Based on the identified characteristics of test cases for DCSs, we show how a UML profile can be used to enhance the test model to contain information about dynamic availability (Sec. 5.3).
• We then show how special test selection criteria can be derived from the extended test model. These test selection criteria guide the test generator in generating test cases which test the component’s preparation to dynamic availability of functionality (Sec. 5.4).