5. Testing
Objectives of this unit:
• Notions: fault management, software testing, dependability
• A (very brief summary of ) Methods and techniques of Fault
Management
Fault “Management”
Exception Handling
. . .
Process Restart/Check- pointing
Recovery Block Techn.
Process Replication Fault Recovery
Formal Methods . . .
Non-formal Methods . . .
N-Version Programming Diversitary Programming
Defensive Programming
Fault Masking Fault Avoidance
Analysis Fault Removal/Test Construction
Fault Exclusion Fault Tolerance
Fault-Intolerant Development Fault-Tolerant Operating Quantification
Metrics/Reliability
Mathematical Methods Software Engineering Methods
...
Metrics/Reliability
Mathematical Methods
Analysis (static) Test (dynamic) Formal Methods
(Correctness Proof, Symbolic Execution, Algebraic Methods, (Dynamic)Assertions),
Non-formal Methods, (Review/Walkthrough/
Inspection)
Semi formal/Empirical Methods
Hybrid Methods (KE)
Software Testing – Techniques and Tools
Summary of Validation & Verification Methods
Dependability and Fault Management
Faults
Errors
Malfunctions
Failures Undesirable Situations/
Events („Impairments“) Deviations
. . .
Measures
Reliability
Availability Safety
Security
. . .
Countermeasures/
Means
Assessment Procurement
. . .
Introduction
Undesirable events
Faults
Failures
Non-occurence of unauthorized
disclosure / improper alteration of
information
Non-occurence of catastrophic consequences onto environ- ment
Security / Safety
Functional View
x aa AA
E A
EN
AN
e xn x an
x ea EA
P
P
e
x af P´
P x af´
fx
test case = (input, expected output)
Example:Test cases: tc1 = (en,an), tc2 = (ea,aa), tc3 = (ef,af´) Test spec:= {(en,an ), (ea,aa), (ef,af´)}
Why Testing?
Example: (G. Myers, 1975)
Given: A program P
accepts inputs a, b, c as edges of a triangle
∆ and determines whether ∆ is-
equilateral - isosceles, or - scalene.-
Find: Test cases to validate P.
2,2,3
3,2,2 (a = b ∨ b = c ∨ a = c) leads to ∆ isosceles 3,2,3
2,2,2 (a = b ∧ b = c) leads to ∆ equilateral
Test Cases: I. Regular (desirable) situation
(Input leads to Output)
2,3,4 (a ≠ b ≠ c) leads to ∆ scalene
2,2,4 (a + b = c)
2,4,2 (a + c = b) error message 2
4,2,2 (b + c = a) 0,2,3 (one arc = 0)
2,0,3 one arc = 0) error message 3 2,3,0 (one arc = 0)
0,0,3 (two arcs = 0)
0,3,0 (two arcs = 0) error message 4 3,0,0 (two arcs = 0)
Test Cases: II. Irregular (undesirable) situations
2,2,5 (a + b < c) 2,5,2 (a + c < b)
5,2,2 (b + c < a) error message 1
Test Cases: II. Irregular Cases (Contd.):
-2, 3, 4 (negative figure)
2,-3, 4 (negative figure) error message 6
2, 3,-4 (negative figure) -2,-3, 4 (negative figure)
-2, 3,-4 (negative figure) error message 7
2,-3,-4 (negative figure)
-2,-3-4 (negative figures) error message 8
1.2,3 ,2 (no integers)
1 ,3.1.,2 no integers) error message 9
1 ,3 ,2.1 no integers) 1,2,# (no numbers)
1,B,3 (no numbers) error message 10
X,2,3 (no numbers) 1,2 (only two inputs)
1 (only one input) error message 11
Programs
– 2 selections, executed (iterated) once:
{abe, ace} → 21 = 2 paths – 2 selections, executed twice:
{abe, ace, abdbe, abdce, acdbe, acdce} → 22 + 21 = 6 paths
– 2 selections, executed three times:
{...} → 23 + 22 + 21 = 14 paths
Example
sel1
sel2
event1 event2
Y Y N
N
b c
d
e a
∑= n
i
d
1
– General: p := i p : number of paths d : number of selections n : number of executions
Filtering the relevant...
Coverage-Oriented „Debugging“
Module Test Sub-System Test Integration Test System Test . . .
„Bottom-up“-Test
„Top-down“-Test
„Sandwich“-Test Hypothesis/ Method Selection Component Level Direction
Test Strategies
Control Flow-oriented (C0-,C1-, ..., C-Test) Data Flow-oriented (dd-, dk-, kk-, ... -Test) . . .
White-Box(Implementation/Structure-oriented)
Black-Box(Specification/Function-oriented) Operational Profile
Sensitive
Cause-Effect Analysis
Equivalence Classes/Partition Test Special Value-/Domain Test
Diversity:Mutations-/Back-to-Back Test Gray-Box Testing
Node/Arc-Coverage (double/triple-, ...,
edge-coverage -Test)
Random-/Anti-Random Testing
