Applied Automata Theory

(1)

Applied Automata Theory

Roland Meyer

TU Kaiserslautern

(2)

Table of Contents I

1 Regular Languages and Finite Automata Regular Languages

Finite Automata Equivalence

Determinism and Complementation Decidability and Complexity

2 Weak Monadic Second-Order Logic Syntax and Semantics of WMSO B¨uchi’s Theorem

3 Star-free Languages

Ehrenfeucht-Fra¨ıss´e Games Star-free Languages

McNaughton and Papert’s Theorem

4 Presburger Arithmetic

Syntax and Semantics of Presburger Arithmetic Representing Solution Spaces

Quantifier Elimination

Existential Presburger Arithmetic

5 Semi-linear Sets

Definition of Semi-linear Sets

Roland Meyer (TU KL) Applied Automata Theory (WiSe 2013) 2 / 161

(3)

Table of Contents II

Closure Properties of Semi-linear Sets Theorem of Ginsburg and Spanier

6 Parikh’s Theorem

7 𝜔-Regular Languages and B¨uchi Automata 𝜔-Regular Languages

B¨uchi Automata Determinism

8 Linear-time Temporal Logic Syntax and Semantics of LTL From LTL to NBA

9 Model Checking Pushdown Systems

Syntax and Semantics of Pushdown Systems Representation Structure: P-NFA

Computing Predecessors Model Checking LTL

10 More on Infinite Words

11 Bottom-Up and Top-Down Tree Automata

Syntax and Semantics of Bottom-Up Tree Automata Determinism and Complementation

(4)

Table of Contents III

Document Type Definitions

Unranked Trees and Hedge Automata

(5)

Part A Finite Words

(6)

1. Regular Languages and Finite Automata

(7)

Basic Notions

Definition (Words)

Finite alphabet= finite set ofletters Σ ={a,b,c, . . . ,n}

Finite word over Σ= finite sequence of letters w =a0·. . .·an−1withai∈Σ for alli∈[0,n−1]

Length of wordw is |w|:=n Empty word𝜀with|𝜀|:= 0

i-th symbol inw denoted byw(i) :=ai

Set of all finite words over Σis Σ^*

Set of all non-empty words over Σis Σ⁺:= Σ^*∖ {𝜀}

Concatenation of wordsw,v ∈Σ^* isw·v∈Σ^*

(8)

Basic Notions

Definition (Languages and operations)

Languageis a (typically infinite) set L⊆Σ^*

Set-theoretic operationsapply to languagesL1,L2⊆Σ^*: L1∪L2

union

L1∩L2 intersection

L1∖L2 difference

L1:= Σ^*∖L1 complement

ConcatenationL1·L2:={w·v ∈Σ^* | w ∈L1andv∈L2} Kleene starL^* :=⋃︀

i∈NLⁱ withL⁰:={𝜀}andLⁱ⁺¹:=L·Lⁱ for all i∈N:={0,1,2, . . .}.

(9)

Regular Languages

Definition (Regular languages)

The class ofregular languages over alphabet Σ, denoted by REG_Σ, is the smallest class of languages that satisfies

(1) ∅ ∈REGΣ and{a} ∈REGΣfor alla∈Σ and

(2) ifL1,L2∈REGΣ then alsoL1∪L2,L1·L2,L^*₁∈REGΣ.

So every regular language is obtained by application offinitely manyoperations in (2) from the languages in (1).

Notation

Avoid brackets: ^* binds stronger than·binds stronger than∪ Write{a}asa

Example: 𝜀∪(a∪b)^*·b. We have𝜀since{𝜀}=∅^*.

(10)

Closure Properties of Regular Languages

Observation

Finite sets of words form regular languages Regular languagesnotclosed under infinite unions By definition, regular languages closed under^*,·, ∪

Goal

Show that REGΣ also closed under remaining operations on sets: ∩, , ∖.

Note thatL1∖L2=L1∩L2.

Needalternative characterizationof regular languages

It is not only about proving closure: need a representation where operations can be computed efficiently

Languages are infinite sets. Finite representations not always easy to find (one of the sports of TCS)

(11)

Finite Automata: Syntax

Definition (Finite automaton)

Anon-deterministic finite automaton (NFA)is a tupleA= (Σ,Q,q0,→,QF) with alphabet Σ,

finite set ofstatesQ,initial stateq₀∈Q,final statesQ_F ⊆Q, and

transition relation→ ⊆Q×Σ×Q. Writeq−→^a q^′ rather than (q,a,q^′)∈ →.

Size ofAis|A|:=|Σ|+|Q|+ 1 +|Q_F|+| −→ |. Note

|A| ≤ |Σ|+|Q|+ 1 +|Q|+|Q|²|Σ| ∈O(|Q|²|Σ|).

For Σ fixed, this is inO(|Q|²). Number of states is important.

(12)

Finite Automata: Semantics

Definition (Run and language)

RunofA= (Σ,Q,q0,→,QF) is a sequence q0

a₀

−→q1 a₁

−→. . .q_n−1−^a−−ⁿ⁻¹→qn. Also say this is arun ofAon wordw :=a0. . .an−1. We writeq₀−^w→q_n if there are intermediary states.

Run isacceptingifq_n∈Q_F. Language ofAis

L(A) :={w ∈Σ^* | q0

−w→qwithq∈QF}.

IfL=L(A) we sayLisacceptedorrecognized by automatonA.

(13)

From Regular Languages to Finite Automata

Goal

Show that regular languages are recognizable by NFAs.

Idea

Apply operations from REG to NFAs.

Proposition (NFA languages are closed under · and ∪)

Consider two NFAsA1andA2.

(1) There is an NFAA1·A2so thatL(A1·A2) =L(A1)·L(A2).

(2) There is an NFAA1∪A2so thatL(A1∪A2) =L(A1)∪L(A2).

(14)

From Regular Languages to Finite Automata

Proposition (NFA languages are closed under

^*

)

Consider an NFAA. There is an NFAA^* withL(A^*) =L(A)^*.

Construction

LetA= (Σ,Q,q₀,→,Q_F). Define

A^*:= (Σ,Q∪ {q^′₀}),q^′₀,−→ ∪ −→^′,Q_F∪ {q₀^′}) whereq₀^′ −→^a ^′ qifq0

−a

→q andqf

−a

→^′qifq0

−a

→qfor allqf ∈QF. An illustration is given in the handwritten notes.

Theorem

If L∈REG_Σthen there is an NFA A with L=L(A).

(15)

From Finite Automata to Regular Languages

Goal

Show the reverse: NFA languages are regular.

Idea

Represent NFA withn∈Nstates by system ofnequations Solve this system using Arden’s lemma

Lemma (Arden 1960)

Let U,V ⊆Σ^* with𝜀 /∈U. Consider L⊆Σ^*. Then L=U·L∪V iff L=U^*·V.

Proof.

Please see the handwritten notes.

(16)

From Finite Automata to Regular Languages

Observation

Only-if direction (⇒) in Arden’s lemma means such an equation has a unique solution.

Use this as tool to construct regular language for a given NFA.

Theorem

If L is recognized by an NFA, then L is regular.

Proof sketch.

Example

(17)

Deterministic Finite Automata

Definition

An NFAA= (Σ,Q,q₀,−→,Q_F) is calleddeterministicorDFAif for alla∈Σ and allq∈Q

there is precisely one stateq^′ ∈Q withq−→^a q^′. Deterministic automata are convenient in applications.

Goal

Show that for every NFAAthere is adeterministicfinite automatonA^′ with L(A) =L(A^′).

(18)

Powerset Construction

Theorem (Rabin & Scott 1959)

For every NFA A with n∈Nstates there is a DFA A^′ with at most2ⁿ states that satisfies L(A) =L(A^′).

Construction: Powerset

LetA= (Σ,Q,q0,−→,QF). SetA^′:= (Σ,P(Q),{q0},−→^′,Q_F^′) with Q₁−→^a ^′ Q₂whereQ₂:={q2∈Q | q₁−→^a q₂ for someq₁∈Q₁} and moreover

Q_F^′ :={Q^′ ⊆Q | Q^′∩QF ̸=∅}.

Note thatA^′ isdeterministic. For everya∈Σ and Q1⊆Q there is a goal state (which may be∅ ∈P(Q)). This goal state is unique.

(19)

Closure under Complementation

Consequence of Rabin & Scott: closureof regular languagesunder complementation

Note

Consider NFAA. It is not easy to find NFA for L(A). Why?

L(A) =w ∈Σ^* so thatthere isan accepting run of Aonw. L(A) =w ∈Σ^* so thatallruns ofAonw do not accept.

To give an automaton forL(A), we thus have to translate this∀-quantifierinto an

∃-quantifier. For DFAsA^′, this works:

L(A^′) =w ∈Σ^*so thatthere isan accepting run ofA^′ onw.

L(A^′) =w ∈Σ^*so thatthere isa run ofA^′ onw that does not accept.

(20)

Closure under Complementation

Proposition (Closure under )

Consider a DFAA. Then there is a DFAAwithL(A) =L(A).

Construction: Swap final states

LetA= (Σ,Q,q₀,→,Q_F). DefineA:= (Σ,Q,q₀,→,Q∖Q_F).

(21)

Summary

LetL=L(A) for an NFAAwithn∈Nstates

There are DFAs forLandLwith at most 2ⁿ states

The bound is optimal: there is a family (L_n)_n∈_Nof languagesL_n that are recognized by an NFA with n+ 1states but

that cannot be recognized by a DFA with<2ⁿ states.

Only considering states reachable fromq₀often yields much smaller automata

(22)

Decidability and Complexity

Problems

Consider an NFAA.

Emptiness: L(A) =∅?

Universality: L(A) = Σ^*?

Membership: Given alsow ∈Σ^*. Doesw ∈L(A) hold?

Focus on emptiness and reduce remaining problems to it

Emptiness

Theorem

Emptiness for NFAs can be solved in timeO(| → |).

Idea

Compute reachable statesR₀⊆R₁⊆. . .until fixed pointR_k =R_k+1

Proof.

LetA= (Σ,Q,q0,−→,QF). DefineR0:={q0}and

Ri+1:=Ri∪ {q^′∈Q | q∈Ri andq−→^a q^′ for some a∈Σ}

Considerk ∈NwithRk =Rk+1. IfRk∩QF ̸=∅returnL(A) not empty.

Otherwise returnL(A) empty.

Reaches fixed point after at most|Q|steps. GivesO(|Q|| → |).

Sufficient to use eachq−→^a q^′ at most once. Linear in | −→ |.

(24)

(25)

2. Weak Monadic Second-Order Logic

(26)

Weak Monadic Second-Order Logic

Goal

NFAs (and also regular languages)operationalmodels

Logics aredeclarative: specifications often more intuitive and more concise Solve decidability problems in logic: satisfiability and validity

With automata: emptiness checks

(27)

WMSO: Syntax

Fix alphabet Σ (parameter of the logic) NeedsignatureSig = (Fun,Pred)

Here,purely relational signaturewithFun=∅ DefinePred:={< /2,suc/2} ∪ {Pa/1 | a∈Σ}.

Consider two countably infinite sets

V1={x,y,z, . . .} offirst-order variables V₂={X,Y,Z, . . .} ofsecond-order variables

Definition (Syntax of WMSO)

Formulas inWMSO (overSig,V1,V2 )are defined by 𝜙::=x<y p suc(x,y) p Pa(x)

⏟ ⏞

Predicates from signature

p X(x) p ¬𝜙 p 𝜙1∨𝜙2 p ∃x:𝜙 p ∃X :𝜙

wherex,y ∈V₁andX ∈V₂.

(28)

WMSO: Syntax

Definition (Notation and abbreviations)

Notationto make signature explicit:

WMSO = WMSO[<,suc]: all WMSO formulas

WMSO[<],WMSO[suc]: formulas that only use predicates <andsuc FO[<,suc],FO[<],FO[suc]: first-orderformulas (overV1, only) Abbreviations: Let𝜙, 𝜓∈WMSO. We set

𝜙∧𝜓:=¬(¬𝜙∨ ¬𝜓) 𝜙→𝜓:=¬𝜙∨𝜓

∀x:𝜙:=¬∃x :¬𝜙 ∀X :𝜙:=¬∃X :¬𝜙 x ≤y :=¬(y <x) x=y :=x ≤y∧y≤x first(x) :=¬∃y :y <x last(x) :=¬∃y:x<y

(29)

WMSO: Syntax

Definition (Bound and free variables)

Consider formula𝜙∈WMSO.

Variablex∈V₁ isbound in𝜙if syntax tree contains occurrence of∃x above x. Similar forX ∈V₂.

Variable that occurs in𝜙and is not bound isfree in𝜙

Write𝜙(x1, . . . ,xm,X1, . . .Xn)to indicate that free variables of𝜙among x1, . . . ,Xn

Formula without free variables calledclosedorsentence

Assume bound and free variables disjoint. Can always be achieved by 𝛼-conversion of bound variables:

(Bad) x<z∧ ∀x:x <y x <z∧ ∀x^′ :x^′ <y (Good)

Example

¬∃y :y <x y bound,x free, notationfirst(x)

∃x :first(x)∧X(x) x bound,X free

(30)

WMSO: Semantics

Intuitive meaning

First-order variables: natural numbersN(positions in a word) x<y,suc(x,y): usual<and successor onN

Second-order variables: finite sets of natural numbers X(x): x is in setX

What does WMSO stand for?

W = Weak: quantify overfinitesets

M = monadic: quantify overelementsof the domain. Polyadic = quantify over tuples.

SO = second-order: with quantification oversets of elements.

Third-order with quantification over sets of sets of elements.

(31)

WMSO: Semantics

Example

∃X : (∃x:first(x)∧X(x))∧(∀x:X(x)→ ∃y :x<y∧X(y)) There is afiniteset of natural numbers

that contains 0 (and thus is not empty) and for every element contains a larger one.

Such a set has to be infinite Formula is unsatisfiable

(32)

WMSO: Semantics

To give semantics, needSig-structuresS = (D^S, <^S,suc^S,(PaS

)_a∈Σ) with D^S =domainof elements (to talk about and quantify over)

P_a^S ⊆D^S, <^S,suc^S ⊆D^S×D^S =interpretation of predicate symbols Restrict ourselves to particularSig-structures that are associated to words

Definition (Word structures)

Letw ∈Σ^*. Itsword structureisS(w) := (D^w, <^w,suc^w,(P_a^w)_a∈Σ) with D^w:={0, . . . ,|w| −1} <^w :=<^N∩(D^w×D^w) suc^w:={(0,1), . . . ,(|w| −2,|w| −1)} P_a^w :={k ∈D^w | w(k) =a}

(33)

WMSO: Semantics

Definition (Satisfaction relation | = for WMSO)

Letw ∈Σ^* and𝜙∈WMSO. To define whether𝜙holds inS(w), need an interpretationI :V1∪V29D^w∪P(D^w) that assigns (sets of) positions to free variables in𝜙(maybe to others, not important). With this:

S(w),I |=𝜙1∨𝜙2 if S(w),I|=𝜙1orS(w),I |=𝜙2

S(w),I |=∃x:𝜙 if there isk∈D^w so thatS(w),I[k/x]|=𝜙 S(w),I |=∃X :𝜙 if there isM⊆D^w (potentially empty)

so thatS(w),I[M/X]|=𝜙.

Here,I[k/x](x) :=k andI[k/x](y) :=I(y) fory ̸=x. Similar forX.

(34)

WMSO: Semantics

Definition (Equivalence)

Two formulas𝜙, 𝜓∈WMSO are calledequivalent, denoted by𝜙≡𝜓, if for all w ∈Σ^* and allI :V₁,V₂9D^w∪P(D^w) we have

S(w),I |=𝜙 iff S(w),I |=𝜓.

Remark

The empty word𝜀has theempty word structure withD^𝜀=∅.

The empty worddoes not satisfyfirst-order existential quantifiers.

Itdoes satisfyall first-order universal quantifiers:

S(𝜀)̸|=∃x :x =x S(𝜀)|=∀x :¬(x=x) The empty worddoes satisfysecond-order existential quantifiers

S(𝜀)|=∃X:∀x:X(x)→Pa(x)

(35)

WMSO: Semantics

Interested inclosed formulas

For𝜙closed,S(w),I |=𝜙does not depend onI Yet needI for satisfaction of subformulas

Definition (Satisfiability, validity, model)

Consider closed formula𝜙∈WMSO

Say𝜙issatisfiableif there isw ∈Σ^* so thatS(w)|=𝜙 In this case, callS(w) amodel of𝜙

Formula without model isunsatisfiable IfS(w)|=𝜙for allw ∈Σ^*, then𝜙isvalid

Observation

𝜙is valid iff¬𝜙is unsatisfiable.

(36)

WMSO: Semantics

Set of words that satisfy a formula form a language

Definition (Language defined by 𝜙, definability)

Consider closed formula𝜙∈WMSO. Thelanguage defined by𝜙is L(𝜙) :={w ∈Σ^* | S(w)|=𝜙}.

LanguageL⊆Σ^* isWMSO-definableif there is a formula𝜙∈WMSO with L=L(𝜙).

NotionsWMSO[suc],WMSO[<],FO[suc],FO[<]-definableby restricting𝜙.

Example

(37)

First Hierarchy of Languages

Distinguish between

FO[suc],FO[<],FO[<,suc],WMSO[suc],WMSO[<],WMSO[<,suc]-definability

Lemma

L is FO[<,suc]-definable iff L is FO[<]-definable (1) L is WMSO[<,suc]-definable iff L is WMSO[<]-definable (2) L is WMSO[<,suc]-definable iff L is WMSO[suc]-definable (3) L is WMSO[<,suc]-definable iff L is WMSO₀-definable. (4) WMSO₀=WMSO without first-order variables but with new predicates:

X ⊆Y,Sing(X),Suc(X,Y),X ⊆P_a with a∈Σ

Meaning: X is subset of Y , X is a singleton set, X and Y are singletons X ={x}

and Y ={y}with suc(x,y), all positions in X have letter a.

WMSO vs. FO: later. FO[suc] vs. FO[<]: not this lecture.

(38)

From Finite Automata to WMSO

Goal

Establish REG = WMSO-definable.

First Subgoal: ⊆

Show that regular languages are definable in WMSO

Theorem (B¨ uchi I, 1960)

Let A be an NFA. We can effectively construct a WMSO-formula𝜙_A so that L(𝜙_A) =L(A).

Proof.

Please see handwritten notes.

Example

(39)

From WMSO to Finite Automata

Second Subgoal: ⊇

Show that WMSO-definable languages are regular

To this end, represent all models of a WMSO-formula by an NFA

Approach

Proceed by induction on structure of𝜙

Problem

∃X :𝜙(X) is closed but𝜙(X) contains X free

Theorem (B¨ uchi II, 1960)

Let𝜙∈WMSO. We can effectively construct an NFA A_𝜙 that satisfies L(A_𝜙) =L(𝜙).

(40)

B¨ uchi’s Theorem

Theorem (B¨ uchi I+II, 1960)

A language L⊆Σ^*is regular iff it is WMSO-definable.

Corollary

It is decidable whether a WMSO-formula is satisfiable/valid.

Worst-case complexity of automata construction

Consider NFAsAandB with at mostn∈Nstates.

A∪B 2n+ 1 states A 2ⁿstates 𝜋x(A) nstates.

Thus, formula withk ∈Nconnectives may yield automaton of size 2²^{. .}

.2c

⏟ ⏞

k-times

withc∈N.

(41)

Consequences of B¨ uchi’s Theorem

Observation

Construction from NFAs to WMSO gave formulas of particular shape.

Existential WMSO, denoted by∃WMSO, is restriction of WMSO to formulas

∃X0:. . .∃Xn:𝜙, where𝜙does not contain second-order quantification.

Corollary

Every closed formula𝜙∈WMSO has an equivalent closed formula𝜓∈ ∃WMSO.

Thus a language is WMSO-definable iff it is definable in∃WMSO.

Proof.

Let𝜙∈WMSO. BuildA𝜙with B¨uchi II. Build𝜓=𝜙A_𝜙 with B¨uchi I.

(42)

(43)

3. Star-free Languages

(44)

Star-free Languages

Goals

(1) Show that FO[<] defines astrictsubclass of regular languages (2) Find alternative characterization:

FO[<]-definable iff represented bystar-freeregular expression

Recapitulation

First-order formulas are WMSO-formulas without second-order variables Example over Σ ={a,b,c}:

𝜙:= ∀x :P_a(x)→ ∃y :x <y∧P_b(y) States that every letterais followed by a letterb:

L(𝜙) ={a,b,c}^*·b· {b,c}^*∪ {b,c}^* Note: first(x),last(x),x =y still in FO[<]

(45)

Star-free Languages

Towards Goal (1)

Known: FO[<]-definable languages are regular Show: Language (aa)^* isnotFO[<]-definable:

For all𝜓∈FO[<] we haveL(𝜓)̸= (aa)^*.

Hence: FO[<]-definable languages formstrictsubclass of regular languages

(46)

Ehrenfeucht-Fra¨ıss´ e Games

Tool from finite model theory (logic) forproving inexpressibility results

The game — informally

Set-up:

Two players: spoilerandduplicator Two words: v andw over Σ Number of rounds: k ∈N

Potentially some existing edges between positions Per round

Spoiler selects position inv orw

Duplicator selects fresh position in other word and connects them by a line

I Positions must have same letter (preservePa)

I New line not allowed to cross existing lines (preserve<) Next round

Winning

Duplicator loses if cannot reply

Duplicator wins if number of rounds passes without loss

(47)

Ehrenfeucht-Fra¨ıss´ e Games

Definition (Partial isomorphism between word structures)

ConsiderS(v) and S(w). Apartial isomorphismbetweenS(v) andS(w) is a partial functionp:D^v 9D^w so that

(1) Functionpis injective.

(2) For allx ∈dom(p) and alla∈Σ we haveP_a^v(x) iffP_a^w(p(x)).

(3) For allx,y ∈dom(p) we havex <^v y iffp(x)<^w p(y).

Lets= (s₁, . . . ,s_n) and t= (t₁, . . . ,t_n) two vectors of positions inD^v andD^w. Writes↦→t for partial functionp:={(s₁,t₁), . . . ,(s_n,t_n)}.

Understanding requirements (1) to (3) wrt. informal game

(1) = fresh position (2) = identical labels (3) = no crossing edges

Interpretation of EF-games

LetS(v),S(w) two word structures with designated positionss,t Duplicator tries to establish partial isomorphism, starting froms↦→t

(48)

Ehrenfeucht-Fra¨ıss´ e Games

Definition (EF-Game)

ConsiderS(v),S(w) withs,t vectors of positions inD^v andD^w. Letk ∈N. AnEF-gameG_k((S(v),s),(S(w),t)) has the following elements and rules:

k rounds

Initialconfigurations↦→t

Given configurationr, a round consists of the following moves:

I Spoiler choosess∈D^v ort∈D^w

I Duplicator choosest∈D^w ors∈D^v

I Game continues withr∪ {(s,t)}as new configuration

Duplicatorwinsk rounds if last configuration is partial isomorphism.

DuplicatorwinsGk((S(v),s),(S(w),t))if has awinning strategy: whatever moves spoiler does, duplicator can wink rounds.

(49)

Ehrenfeucht-Fra¨ıss´ e Theorem

Where is this going?

Now we know what an EF-game does: compares word structuresS(v) andS(w).

So what? Overall goal isEF-theorem:

duplicator winsGk((S(v),s),(S(w),t)) iff v andw cannot be distinguished by FO[<]-formulas of quantifier-depth≤k.

(50)

Ehrenfeucht-Fra¨ıss´ e Theorem

Definition (Quantifier-depth)

Thequantifier-depth qd(𝜙) with𝜙∈FO[<] is the maximal nesting depth of quantifiers in𝜙:

qd(x<y) := 0 qd(Pa(x)) := 0

qd(¬𝜙) :=qd(𝜙) qd(𝜙₁∨𝜙₂) :=max{qd(𝜙₁),qd(𝜙₂)}

qd(∃x:𝜙) := 1 +qd(𝜙)

Definition (k-equivalence)

ConsiderS(v),S(w) withs,t. Then (S(v),s) and (S(w),t) arek-equivalent, denoted(S(v),s)≡k (S(w),t), if for all𝜙(x) withqd(𝜙)<k we have

S(v),I[s/x]|=𝜙 iff S(w),I[t/x]|=𝜙.

In the case of empty sequencess=𝜀=t, equivalenceS(v)≡_k S(w) means the structures satisfy same sentences of quantifier-depth up tok.

(51)

Ehrenfeucht-Fra¨ıss´ e Theorem

Theorem (Ehrenfeucht, Fra¨ıss´ e, 1954, 1961)

Duplicator wins G_k((S(v),s),(S(w),t))iff(S(v),s)≡_k (S(w),t).

Why is this cool?

Because it gives a pumping argument!

Proposition

Language (aa)^*is notFO[<]-definable.

Lemma

Duplicator wins Gk(a²^k,a²^k⁺¹).

Proof (of lemma and proposition).

(52)

Proof of the Ehrenfeucht-Fra¨ıss´ e Theorem

Lemma (How to win an EF-game?)

(1) Duplicator wins G0((S(v),s),(S(w),t))iff s↦→t is a partial isomorphism.

(2) Duplicator wins Gk+1((S(v),s),(S(w),t))iff

(2.a) ∀s ∈D^v :∃t∈D^w:Duplicator wins Gk((S(v),s.s),(S(w),t.t))and (2.b) ∀t∈D^w:∃s∈D^v :Duplicator wins Gk((S(v),s.s),(S(w),t.t)).

Intuition

Gk((S(v),s.s),(S(w),t.t)) gives arbitrary first step inGk+1((S(v),s),(S(w),t)).

Proof (of Ehrenfeucht-Fra¨ıss´ e Theorem).

(53)

Star-free Languages

Towards Goal (2)

Findsubclass of REG that characterizes FO[<]-definable languages Wantalgebraic characterization(as opposed to logical) that highlights closure properties

(54)

Star-free Languages

Definition (Star-free Languages)

The class ofstar-free languages over alphabet Σ, denoted by SF_Σ, is the smallest class of languages that satisfies

(1) ∅,{𝜀} ∈SFΣ and{a} ∈SFΣfor alla∈Σ and (2) ifL₁,L₂∈SF_Σ then also L₁∪L₂,L₁·L₂,L₁∈SF_Σ.

Remark

Complement is not an operator on REG, but it can be derived.

Complement may yield^* in alternative representations of the language.

Example

(55)

From Star-free Languages to FO[<]

Goal

Establish SF = FO[<]-definable.

Theorem (McNaughton and Papert I, 1971)

Let L∈SFΣ. We can effectively construct a FO[<]-formula𝜙Lso that L(𝜙L) =L.

Proof.

Homework.

(56)

From FO[<] to Star-free Languages

Goal ⊇

Establish SF⊇FO[<]-definable.

Insights

Relation≡k hasfinite index, i.e., finitely many classes.

Every class of≡k can be characterized bysingleformula.

With this, give inductive construction of SF-representation for FO[<]-defined language.

(57)

From FO[<] to Star-free Languages

Lemma

Consider structures(S(s),s)with|s|=n∈N. For every k ∈N, equivalence≡k

has finite index.

Proof.

Lemma

For every equivalence class[(S(v),s)]≡_k there is a formula𝜙[(S(v),s)]_≡

k of qd(𝜙_[(S(v_),s)]_≡

k)≤k so that

(S(w),t)∈[(S(v),s)]_≡_k iff S(w),I[t/x]|=𝜙_[(S(v_),s)]_≡

k.

Proof.

(58)

McNaughton and Papert’s Theorem

Theorem (McNaughton and Papert II, 1971)

Let𝜙an FO[<]sentence. We can effectively construct L∈SFΣso that L(𝜙) =L.

Proof.

Theorem (McNaughton and Papert I+II, 1971)

A language L⊆Σ^*is star-free iff it is FO[<]-definable.

(59)

The World of Finite Words ... as we know it now

WMSO-definable languages = regular languages B¨uchi

FO[<]-definable languages

= star-free languages McNaughton and Papert

(aa)^* Ehrenfeucht-Fra¨ıss´e

(60)

Part B Natural Numbers

(61)

(62)

4. Presburger Arithmetic

(63)

Presburger Arithmetic

Goal

State properties of sets of natural numbers

Use restricted language of first-order arithmetic: addition, no multiplication, quantification

Compute solution space (free variables) Compute truth value (closed formulas)

Two approaches

Automata theoretic: Represent solution space via automaton Logical: Establish quantifier elimination result

(64)

Presburger Arithmetic: Syntax

SignatureSig = (Fun,Pred) withFun={0/0,1/0,+/2}andPred={< /2} Infinite set offirst-order variablesV

Definition (Syntax of Presburger arithmetic)

Termsbuilt from variables and function symbols:

t ::= 0 p 1 p x p t1+t2 withx∈V. Formulas inPresburger arithmeticdefined by

𝜙::=t1<t2 p ¬𝜙 p 𝜙1∧𝜙2 p ∃x:𝜙.

Set of all formulas denoted byPA.

(65)

Presburger Arithmetic: Syntax

Definition (Abbreviations)

Abbreviations: Consider termst₁,t₂,n∈N, andx ∈V. We set t₁>t₂:=t₂<t₁ t₁≤t₂:=¬(t₁>t₂) t1≥t2:=t2≤t1 t1=t2:=t1≤t2∧t1≥t2

n:= 1 +. . .+ 1

⏟ ⏞

n-times

nx:=x+. . .+x

⏟ ⏞

n-times

Abbreviations for formulas: as before.

Definition (Bound and free variables)

Like for WMSO. Sentences have no free variables.

(66)

Presburger Arithmetic: Semantics

Fixedstructure (N,0^N,1^N,+^N, <^N)

Definition (Satisfaction relation | = for PA)

Consider formula𝜙∈PA. AninterpretationI :V 9 Nassigns a natural number to each free variable in𝜙(and maybe to others, not important). With this:

I|=t1<t2 if I(t1)<^NI(t2) I|=¬𝜙 if I ̸|=𝜙

I|=𝜙1∧𝜙2 if I |=𝜙1andI |=𝜙2

I|=∃x :𝜙 if there isn∈Nso thatI[n/x]|=𝜙.

Interpretation of terms (note thatI(x)∈N):

I(0) := 0^N I(1) := 1^N I(t₁+t₁) :=I(t₁) +^NI(t₂).

Definition (Equivalence)

Formulas𝜙, 𝜓∈PA areequivalent,𝜙≡𝜓, if for allI :V 9 Nwe have I |=𝜙 iff I |=𝜓.

(67)

Presburger Arithmetic: Semantics

Definition (Truth, solutions, definability)

Consider closed formula𝜙∈PA.

Say𝜙istrueif satisfied by all interpretations.

Otherwise𝜙satisfied by no interpretation and call itfalse.

Consider formula𝜓∈PA withn∈Nfree variablesx.

Restrict ourselves to interpretationsI :V 9 Nwithdom(I) =x.

Assume variables are ordered, writeI as vectorv ∈Nⁿ. Callv∈Nⁿ withv|=𝜓amodelorsolutionof𝜓.

Formula𝜓 issatisfiableif there isv∈Nⁿ withv|=𝜓.

If allv∈Nⁿ satisfy𝜓, call𝜓 valid.

Solution spaceof𝜓is

Sol(𝜓) :={v∈Nⁿ | v|=𝜓}.

A setS⊆N^k isPresburger-definableif there is𝜓∈PA withS =Sol(𝜓).

(68)

Representing Solution Spaces

Goal

RepresentSol(𝜓) by aDFAA_𝜓.

Problem

A𝜓 accepts words whereasSol(𝜓) contains numbers.

Definition (Least-significant bit first encoding, language of a formula)

Relationlsbf ⊆N× {0,1}^* encodesk ∈Nby the setlsbf(k) :=binary(k)·0^*. Binary notation hasleast-significant bit first. Extend relation to vectors:

lsbf ⊆Nⁿ×({0,1}ⁿ)^* withn∈N. Thelanguage of𝜓∈PAis

L(𝜓) := ⋃︁

v∈Sol(𝜓)

lsbf(v).

(69)

Representing Solution Spaces

Theorem (B¨ uchi 1960, Wolper & Boigelot 2000, Esparza 2012)

Let𝜓∈PA. We can effectively construct aDFAA𝜓 with L(A𝜓) =L(𝜓).

Corollary

It is decidable, whether𝜓is satisfiable/valid.

Approach

A_¬𝜓 :=A_𝜓 A_𝜙∨𝜓:=A_𝜙∪A_𝜓 A_∃x:𝜓:=𝜋_x(A_𝜓)

(70)

Representing Solution Spaces

Remains to construct automaton for solutions of atomic formulas.

Notation

Atomic formulas can be assumed to be in form 𝜓=a1x1+. . .+anxn≤b

witha1, . . . ,an,b∈Z. Witha∈Zⁿandx∈Vⁿvectors, write as a·x≤b.

For the construction, please see handwritten notes.

Lemma (Termination)

Let𝜓=a·x≤b and s =∑︀n

i=1|ai|. The states j ∈Zadded to the worklist satisfy

−|b| −s≤j ≤ |b|+s.

(71)

Quantifier Elimination

Goal

Decide truth of a sentence𝜙∈PA.

Approach (Replace quantifiers by concrete values)

A logicadmits quantifier eliminationif for any formula of the form

∀/∃x1. . .∀/∃xn:𝜙(x1, . . . ,xn,y1, . . . ,ym) there is an equivalent formula𝜓(y1, . . . ,ym).

To obtain quantifier elimination for Presburger arithmetic, we extend the signature by≡mfor allm≥2. The semantics is as expected.

Remark

Note that PA[<] and PA[<,(≡m)m≥2] are equally expressive:

x≡my iff ∃z: (x≤y∧y−x=mz)∨(x>y∧x−y=mz).

(72)

Quantifier Elimination

Theorem (Presburger 1929)

Consider∃x:𝜙(x,y1, . . . ,ym)∈PA[<,(≡m)_m≥2]. We can effectively construct 𝜓(y1, . . . ,ym)∈PA[<,(≡m)_m≥2]with

∃x:𝜙(x,y1, . . . ,ym) ≡

logical equivalence 𝜓(y1, . . . ,ym).

Proof.

Corollary

Given a sentence𝜙∈PA, we can decide whether it is true or false.

Phrased differently, the theory of structure(N,0^N,1^N, <^N,+^N)is decidable.

(73)

Existential Presburger Arithmetic

Existential Presburger arithmeticavoids negation, but introduces equality and disjunction.

Definition (Existential Presburger arithmetic)

Consider the signatureSig = ({0/0,1/0,+/2},{≤/2,=/2}). Formulas in existential Presburger arithmeticare defined by

𝜙::=t1<t2 p t1=t2 p 𝜙1∧𝜙2 p 𝜙1∨𝜙2 p ∃x :𝜙.

We use∃PA to denote the set of all formulas in existential Presburger arithmetic.

(74)

Existential Presburger Arithmetic

With quantifier elimination, every Presburger formula is equivalent to an existential formula.

Theorem

For every formula𝜙∈PA there is a formula 𝜓∈ ∃PA with𝜙≡𝜓.

Proof.

With the quantifier elimination result,𝜙∈PA has an equivalent formula𝜙≡𝜌 with𝜌∈PA[<,(≡_m)_m≥2] quantifier-freeandnegation-free.

We remove the congruences in𝜌by

x≡my iff ∃z: (x≤y∧y−x=mz)∨(x>y∧x−y=mz).

The resulting formula is𝜓∈ ∃PA.

(75)

Existential Presburger Arithmetic

Motivation

The interest in∃PA is thatsatisfiabilityhas alow complexity.

The proof encodes satisfiability intointeger linear programming(ILP), which is the following problem:

Given: A matrixA∈Z^m×n andb∈Z^m.

Problem: DoesA·x≥bhave an integer solution x∈Zⁿ?

Theorem (von zur Gathen and Sieveking 1978)

ILP is NP-complete.

(76)

Existential Presburger Arithmetic

Lemma

Satisfiability in∃PA is NP-complete.

To check satisfiability of𝜙∈ ∃PA, first move the existential quantifiers to the top.

This takes linear time and yields a formula𝜙^′ ≡𝜙.

In the next step, replace every disjunct𝜓1∨𝜓2 by one of its components, either 𝜓1or𝜓2. This turns𝜙^′ into a formula𝜙^′′, again in linear time. Intuitively,𝜙^′′

guesses the disjuncts that will be satisfied.

The resulting formula𝜙^′′ actually is an ILP problem. Extending it by lower bound constraints ensures we find a solution inNⁿ. The formula is now𝜙^′′′.

Use Theorem 59 to solve the ILP problem𝜙^′′′ inNP.

Altogether, this yields a non-deterministic algorithm that runs in polynomial time and reports positively iff∃PA is satisfiable.

NP-hardness is byNP-hardness of 0/1-ILP.

(77)

5. Semi-linear Sets

(78)

Semi-linear Sets: Definition

Goal

Show that semi-linear sets are precisely the sets of numbers that are Presburger-definable.

Consequences

Closure of semi-linear sets undercomplement(cool).

Closure of Presburger-definable sets under iteration.

Definition (Semi-linear sets)

Letc∈Nⁿ be a vector andP⊆Nⁿ afinite set of vectors. We define L(c,P) :={v∈Nⁿ | for eachp∈P there iskp∈Nso thatv=c+∑︁

p∈P

kpp}.

Here,c is calledconstantandP is the set ofperiods.

A setM⊆Nⁿis linearifM=L(c,P) for some c∈NⁿandP⊆Nⁿ finite.

A setS⊆Nⁿ issemi-linearif it is a finite union of linear sets.

(79)

Semi-linear Sets: Definition

Remark

(1) Given a linear setL(c,P)⊆Nⁿand a vectorv∈Nⁿ, it isdecidable whether v∈L(c,P) holds. The same decidability holds for semi-linear sets.

(2) Linear sets are not closed underanyof the Boolean operations: if

M1,M2⊆Nⁿ are linear, thenM1, M1∪M2, andM1∩M2need not be linear.

(3) The class of semi-linear sets properly includes the linear sets, i.e., every linear set is semi-linear.

(80)

Semi-linear Sets: Closure Properties

Definition (Linear functions)

A functionf :Nⁿ→N^m is calledlinearif

f(x+y) =f(x) +f(y) and f(kx) =kf(x) withk∈N.

Lemma (Closure under linear functions)

Let S ⊆Nⁿ be semi-linear and f :Nⁿ→N^m be linear. Then f(S)⊆N^m is semi-linear.

(81)

Semi-linear Sets: Closure Properties

Definition (Iteration)

LetA⊆Nⁿ. We define

A^*:={v1+. . .+vk ∈Nⁿ | v1, . . . ,vk ∈A}.

Lemma (Closure under iteration)

If S⊆Nⁿ is semi-linear, so is S^*.

Proof.

LetS =L(c₁,P₁)∪. . .∪L(c_l,P_l). One can show that S^*= ⋃︁

J⊆{1,...,l}

L(∑︁

i∈J

ci , ⋃︁

i∈J

Pi∪ {ci}).

(82)

Semi-linear Sets: Closure Properties

Lemma

If S⊆Nⁿ is semi-linear and c∈Nⁿ, then

c+S :={c+x | x∈S} is semi-linear.

Theorem (Closure under ∪ and ∩)

Let S1,S2⊆Nⁿbe semi-linear. Then S1∪S2and S1∩S2are semi-linear.

Proof.

For∪there is nothing to do.

For∩, it is sufficient to show that the intersection of linear sets forms a semi-linear set. For a semi-linear setM1∪M2, we then use

M∩(M₁∪M₂) = (M∩M₁)∪(M∩M₂).

The proof is on the board.

(83)

Semi-linear Sets: Closure Properties

Anapplicationof the above closure properties is the following result.

Lemma (Closure under taking the inverse of linear functions)

Let S ⊆N^m be semi-linear and f :Nⁿ→N^m be linear. Then f⁻¹(S)⊆Nⁿis semi-linear.

(84)

Theorem of Ginsburg and Spanier

Theorem (Ginsburg and Spanier)

A set S⊆Nⁿ is Presburger-definable if and only if it is semi-linear.

The proof is on the board.

Corollary (Closure properties)

If S⊆Nⁿ is semi-linear, then S is semi-linear.

If S⊆Nⁿ is Presburger-definable, then so is S^*.

(85)

6. Parikh’s Theorem

(86)

Parikh Images

Goal

The Parikh image of a word𝜓(w) counts the occurrences of letters.

The goal is to show that𝜓(L(G)) is semi-linear for every context-free grammarG. The classical proof of Parikh directly shows semi-linearity of Parikh-images.

We present a different approach due to Verma, Seidl, and Schwentick from 2006:

the Parikh image can be captured directly by asmallPresburger formula.

What we show

Given a context-free grammarG, we construct inlinear timeanexistential Presburger formula𝜙G so thatSol(𝜙G) =𝜓(L(G)).

This is interesting as satisfiability for existential Presburger arithmetic is only NP-complete.

(87)

Part C Infinite Words

(88)

Where are we?

Learned so far...

REG/Finite automata, WMSO/FO formulas, Presburger arithmetic/Semilinear sets/Parikh images.

Now followingmodel checking problemmakes sense:

A|=𝜙 defined by L(A)⊆L(𝜙).

Ausually calledsystem,𝜙usually calledspecification, check whetherAis model of𝜙(in the sense of|=).

Systemsfeatures: regular or regular + counting.

Sometimes, finite words are not sufficient...

Operating systems typically not meant to terminate: ♦req New class of automata: B¨uchi automata—system.

New logic: Linear-time Temporal Logic (LTL)—specification.

New systemfeatures: B¨uchipushdownautomata –recursion.

(89)

7. 𝜔-Regular Languages and B¨ uchi Automata

(90)

Goals and Problems

Goal

Recognize infinite words with finite automata What is an accepting run? Final states fail!

B¨uchi condition: visit final states infinitely often.

Solve algorithmic problems

Emptiness: Does the automaton accept a word?

Language equivalence: Do automata AandB accept the same language?

Key challenges

Determinisation/complementation.

Applications

Model checkingMSO — second-order variables range over infinite sets.

Model checkingLTLas syntactic fragment of MSO.

(91)

Basic Notions

Let Σ be a finite alphabet.

Definition

𝜔-word over Σ= infinite sequencew =a0·a1. . .withai ∈Σ for alli∈N. Set of all infinite words over Σis Σ^𝜔.

𝜔-languageL⊆Σ^𝜔 = set of𝜔-words.

Letw ∈Σ^𝜔anda∈Σ. Then|w|_a∈N∪ {𝜔}= number ofa inw. Concatenation

Impossible to concatenatev,w ∈Σ^𝜔 Ifv∈Σ^* andw ∈Σ^𝜔, thenv·w ∈Σ^𝜔.

LetV ⊆Σ^* andW ⊆Σ^𝜔, thenV ·W :={v·w | v ∈V,w ∈W} ⊆Σ^𝜔. Letv ∈Σ⁺. Thenv^𝜔:=v·v·v·. . .

LetL⊆Σ^* withL∩Σ⁺̸=∅. Then

L^𝜔:={v0·v1·v2·. . . | vi ∈L∖ {𝜀}for alli∈N}.

(92)

Basic Notions

Example

Set of all words with infinitely manyb

so that twobare separated by even number ofa:

a^*·((aa)^*·b)^𝜔.

Next step

Define𝜔-regularlanguages

Choose𝜔-iteration of regular languages.

“Correct definition” as follows: has natural corresponding automaton model.

(93)

𝜔-Regular Languages

Definition (𝜔-regular languages)

A languageL⊆Σ^𝜔is 𝜔-regular if there are regular languagesV0, . . . ,V_n−1⊆Σ^*, W0, . . . ,W_n−1⊆Σ^* withWi∩Σ⁺̸=∅for alli ∈[0,n−1] so that

L =

n−1

⋃︁

i=0

Vi·W_i^𝜔.

Example

Lemma

𝜔-regular languages are closed under union

concatenation from left with regular languages.

(94)

B¨ uchi Automata

Syntactically finite automata Acceptance condition changed

Definition (B¨ uchi automaton (syntax and semantics))

Anon-deterministic B¨uchi automaton (NBA)is a tuple

A= (Σ,Q,q₀,→,Q_F) with the usual statesQ,initial stateq₀∈Q, final statesQ_F ⊆Q, transition relation→ ⊆Q×Σ×Q.

Run ofAis an infinite sequence

r =q₀−→â⁰ q₁−→â¹ q₂−→â² . . . Ifw =a₀·a₁·a₂·. . ., we have arun ofAonw. Writeq0

−w→to indicatethere isa run ofAon w. (States not important.) LetInf(r) := states that occur infinitely often inr.

Runr isacceptingifInf(r)∩QF ̸=∅.

𝜔-language ofAis

L(A) :={w ∈Σ^𝜔 | there is an accepting run ofAonw}.

(95)

B¨ uchi Automata

Comment

Acceptance = one final state visited infinitely often

= set of final states visited infinitely often (⇐asQF finite set).

Example

The automata can be found in the handwritten notes. Let Σ ={a,b}.

L1:= (a^*·b)^𝜔 Infinitely many b.

L2:= (a∪b)^*·a^𝜔 Finitely manyb.

Note thatL2=L1= Σ^𝜔∖L1.

AutomatonA2forL2is non-deterministicwhileA1forL1is deterministic.

(96)

Deterministic B¨ uchi Automata

Definition (Deterministic B¨ uchi automaton)

An NBAA= (Σ,Q,q0,→,QF) isdeterministic (DBA)if for alla∈Σ and all q∈Q there is precisely one stateq^′∈Q withq−→^a q^′.

Not by accident thatA2is NBA whileA1 is DBA.

L2cannotbe recognized by a DBA.

In sharp contrast to NFA = DFA-recognizable languages.

Theorem

There are𝜔-languages that are NBA-recognizable but not DBA-recognizable.

Consequence

There are NBAs that cannot be determinized into DBAs.

SinceL2= (a∪b)^*·a^𝜔, one may assume that 𝜔-regular languages

⏟ ⏞

expressions/closure

= NBA-recognizable languages

⏟ ⏞

automata

This in fact holds.

(97)

8. Linear-time Temporal Logic

(98)

Linear-time Temporal Logic

Specification language for model checking:

in a model checking problem A|=𝜙, formula𝜙is typically in LTL Used in industry as PSL = property specification language (variant of LTL, like statemachines in UML are derived from finite automata)

Proposed by Amir Pnueli in 1977, Turing award 1996

(99)

Linear-time Temporal Logic

Idea of LTL

Subset of MSO useful for specification

No quantifiers, more complex and intuitive operators

Understand word as a sequence of (sets of) system actions over time Interpret formula at a single moment/point in the word

𝛼 a ^𝛽

aisnow, 𝛽 is the future,operatorsonly make claims about thefuture

Remark

LTLis alinear-timetemporal logic that talks about words

CTLis abranching-timetemporal logic that talks aboutcomputation trees E○(x∧A○z).

CTL^*unifies and generalizes LTL and CTL

(100)

Linear-time Temporal Logic

Goal

Translate LTL into NBA for model checking LTL can be understood as a subset of MSO

Therefore, we know this translationcan be done But it is strictly less expressive than MSO

Therefore, we obtain afaster and easieralgorithm

Applied Automata Theory