Qualitätssicherung in der Softwareentwicklung VU 8 DI Dr. Bernhard K. Aichernig

(1)

Qualitätssicherung in der Softwareentwicklung

VU 8

DI Dr. Bernhard K. Aichernig

Institut für Softwaretechnologie (IST) TU Graz

Sommersemester 2012

(2)

Outline

1

Conformance Testing Properties

2

Labelled Transition Systems Equivalence

Preorder ioco Examples

3

Test generation

(3)

Input/Output Conformance Relation

Jan Tretmans - 1996

Based on IO-Labeled Transition Systems

(4)

Input/Output Conformance Relation

Jan Tretmans - 1996

Based on IO-Labeled Transition Systems

(5)

Conformance Testing

(6)

Conformance Testing - Soundness

Test suite is sound:

conformance ⇒ all tests pass

(7)

Conformance Testing - Soundness

Test suite is sound:

conformance ⇒ all tests pass

(8)

Conformance Testing - Exhaustiveness

Test suite is exhaustive:

conformance ⇐ all tests pass

(9)

Conformance Testing - Exhaustiveness

Test suite is exhaustive:

conformance ⇐ all tests pass

(10)

Conformance Testing - Completeness

Test suite is complete:

conformance ≡ all tests pass

(11)

Conformance Testing - Completeness

Test suite is complete:

conformance ≡ all tests pass

(12)

Conformance Testing with ioco

System (Implementation) is modeled as IOTS

weakly input enabled

Specification is an IOLTS

possibly incomplete possible non-deterministic

!g1

τ τ

!g1 ?g2

?g2

!g3

!g3 !g1

(13)

Conformance Testing with ioco

System (Implementation) is modeled as IOTS

weakly input enabled

Specification is an IOLTS

possibly incomplete possible non-deterministic

!g1

τ τ

!g1 ?g2

?g2

!g3

!g3 !g1

(14)

Input Output Labeled Transition Systems

Input Output Labeled Transition System

An IOLTS is an LTS M = (Q

^M

, A

^M

, →

_M

, q

₀^M

) with Q

^M

a finite set of states

A

^M

= A

^M_I

∪ A

^M_O

∪ {τ } where

A^M_I andA^M_O are input and output alphabets τ6∈A^M_I ∪A^M_O is an unobservable, internal action

→

_M

⊆ Q

^M

× A

^M

× Q

^M

is the transition relation q

₀^M

∈ Q

^M

is the initial state.

QUESTION?

What means: The implementation conforms-to the

specification?

(15)

Input Output Labeled Transition Systems

Input Output Labeled Transition System

An IOLTS is an LTS M = (Q

^M

, A

^M

, →

_M

, q

₀^M

) with Q

^M

a finite set of states

A

^M

= A

^M_I

∪ A

^M_O

∪ {τ } where

A^M_I andA^M_O are input and output alphabets τ6∈A^M_I ∪A^M_O is an unobservable, internal action

→

_M

⊆ Q

^M

× A

^M

× Q

^M

is the transition relation q

₀^M

∈ Q

^M

is the initial state.

QUESTION?

What means: The implementation conforms-to the

specification?

(16)

How to relate 2 LTSs?

Equivalence Relations (=)

Bisimulation

Trace Equivalence Testing Equivalence ...

Preorder Relations (≤)

Trace Preorder Testing Preorder ...

Input-Output Relations

ioconf

ioco ...

...

(17)

(Weak) Bisimulation

Two states are bisimilar iff they simulate each other and go to states which are bisimilar

Bisimulation is not suited for testing!

(18)

Trace Equivalence

A trace is an observable sequence of actions

Two states are trace equivalent iff they have the same traces

Trace equivalence is the weakest notion of conformance

(19)

Equivalence vs. Preorder Relations

Equivalence Relation (R)

reflexive (sRs)

symmetric: iRs→sRi transitive: iRs∧sRt →iRt

Preorder Relations (≤)

NOTnecessarily antisymmetric:

iRs↔i ≤s∧s≤i simplifies testing

e.g.: Trace Preorder

i ≤_tr s↔traces(i)⊆traces(s)

(20)

Some Notations: Transitions

q →

^a_M

q

⁰

=

df

(q, a, q

⁰

) ∈→

_M

q ⇒

q

⁰

=

df

(q = q

⁰

) ∨ (q →

^τ _M

q

₁

∧ · · · ∧ q

n−1 τ

→

_M

q

⁰

)

q ⇒

^a

q

⁰

=

df

∃q

₁

, q

₂

: q ⇒

_M

q

₁

→

^a_M

q

₂

⇒

_M

q

⁰

(21)

Some Notations: Quiescence

δ is used to represent quiescence

q−→^δ q=_dfq is a quiescent state.

Quiescent state = no edge labeled with an output or an

internal action

(22)

Some Notations: Quiescence

δ is used to represent quiescence

q−→^δ q=_dfq is a quiescent state.

Quiescent state = no edge labeled with an output or an

internal action

(23)

Some Notations: Suspension Automaton

∆(M) = (Q

^M

, A

^∆(M)

, →

_∆(M)

, q

₀^M

) where:

A^∆(M)=A^M∪ {δ}withδ∈A^∆(M)_O

→_∆(M)is obtained from→_M by adding loopsq→^δ qfor each quiescent state

(24)

Some Notations: After

q after

_M

σ =

df

{q

⁰

| q ⇒

^σ_M

q

⁰

} Q after

_M

σ =

df

S

q∈Q

(q after

_M

σ).

(25)

Some Notations: Out

Out

_M

(q) =

df

{a ∈ A

^M_O

| q →

^a_M

} Out

_M

(Q) =

df

S

q∈Q

(Out

_M

(q))

(26)

ioco

Definition: ioco

Let IUT = (Q

^IUT

, A

^IUT

, →

_IUT

, q

₀^IUT

) be weakly input enabled with A

^IUT

= A

^IUT_I

∪ A

^IUT_O

∪ {τ } and S = Q

^S

, A

^S

, →

_S

, q

₀^S

be strongly responsive with A

^S

= A

^S_I

∪ A

^S_O

∪ {τ }. Then:

IUT ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(IUT ) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

IUT ioco S iff outputs (and quiescences) of the IUT are

possible in S after an arbitrary suspension trace of S.

(27)

ioco

Definition: ioco

Let IUT = (Q

^IUT

, A

^IUT

, →

_IUT

, q

₀^IUT

) be weakly input enabled with A

^IUT

= A

^IUT_I

∪ A

^IUT_O

∪ {τ } and S = Q

^S

, A

^S

, →

_S

, q

₀^S

be strongly responsive with A

^S

= A

^S_I

∪ A

^S_O

∪ {τ }. Then:

IUT ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(IUT ) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

IUT ioco S iff outputs (and quiescences) of the IUT are

possible in S after an arbitrary suspension trace of S.

(28)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(29)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(30)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(31)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(32)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(33)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(34)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(35)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(36)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(37)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(38)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(39)

P ioco S?

P ioco S =

df

∀σ ∈ traces(∆(S)) :

Out

_IUT

(∆(P) after

_IUT

σ) ⊆ Out

_S

(∆(S) after

_S

σ).

(40)

Test Cases

A test case is an IOLTS

Inputs = Outputs IUT, Outputs = Inputs IUT Equipped with verdict states (pass, fail) In each state (except Pass, Fail):

Single output and all inputs

All inputs andθ

(41)

Formal Test Execution

(42)

Formal Test Execution

(43)

Formal Test Execution

(44)

Formal Test Execution

(45)

Formal Test Execution

(46)

Formal Test Execution

(47)

Formal Test Execution

(48)

Formal Test Execution

(49)

Formal Test Execution

(50)

Formal Test Execution

(51)

Formal Test Execution

(52)

A Complete Test Generation Algorithm

Given the suspension automaton of a specification as an LTS S = (Q

^S

, A

^S

, →

_S

, q

₀^S

)

1

Initially compute K = q

₀^S

after

_S

2

Do non-deterministically, either:

Stop test case with verdictpass

Let the test case produce an output (!a) with K⁰=K after_S?a6=∅.

Also accept all inputs at the same time and add fail states for unexpected results.

Accept all inputs (and quiescence) and add fail states for unexpected results. Compute newK⁰for valid inputs.