Theory of Computer Science B7. Context-free Languages: Normal Form and PDA Gabriele R¨oger

Gabriele R¨oger

University of Basel

March 29, 2021

Context-free Grammars

Definition (Context-free Grammar)

Acontext-free grammar is a 4-tuplehV,Σ,R,Si with

1 V finite set of variables,

2 Σ finite alphabet of terminal symbols (with V ∩Σ =∅),

3 R ⊆V ×(V ∪Σ)^∗ finite set of rules,

4 S ∈V start variable.

Short-hand Notation for Rule Sets

We abbreviate several rules with the same left-hand side variable in a single line, using “|” for separating the right-hand sides.

For example, we write X→0Y1|XY for:

X→0Y1and X→XY

We have used the pumping lemma for regular languages to show thatL={aⁿbⁿ|n∈N0}is not regular.

Show that it is context-free by specifying a suitable grammarG with L(G) =L.

Questions

Questions?

Chomsky Normal Form

Chomsky Normal Form: Motivation

As in other kinds of structured objects,normal formsfor grammars are useful:

they show which aspects are critical for defining grammars and which ones are just syntactic sugar

they allow proofs and algorithms to be restricted

to a limited set of grammars (inputs): those in normal form Hence we now consider anormal form for context-free grammars.

Definition (Chomsky Normal Form)

A context-free grammarG is inChomsky normal form (CNF) if all rules have one of the following three forms:

A→BC with variables A,B,C

andB andC are not the start variable, or A→awith variableA and terminal symbol a, or S →εwith start variable S.

German: Chomsky-Normalform

formally: rule setR ⊆(V ×((V \ {S})(V \ {S})∪Σ))∪ {hS, εi}

Chomsky Normal Form: Theorem

Theorem

For every context-free grammar G there is a context-free grammar G⁰ in Chomsky normal form with L(G) =L(G⁰).

Theorem

For every context-free grammar G there is a context-free grammar G⁰ in Chomsky normal form with L(G) =L(G⁰).

Proof.

The following algorithm converts the rule set ofG =hV,Σ,R,Si into CNF:

Step 1: Add new start variableS⁰.

Add a new variableS⁰ which will be the start variable, and add a ruleS⁰ →S, whereS is the original start variable.

Afterwards, the (new) start variable does not occur on the right-hand side of a rule.

We will writeV⁰ for the new variable set (V⁰=V ∪ {S⁰}) and R⁰

for the new rule set. . . .

Chomsky Normal Form: Theorem

Proof (continued).

Step 2: Eliminateε-rules of the form A→ε(A6=S⁰).

Let V_ε be the set of variable from which one can derive the empty word. We find this set Vε by first collecting all variables A∈V⁰ with rule A→ε∈R⁰ and then successively adding additional variablesB if there is a rule B →A₁A₂. . .A_k ∈R⁰ and the variablesAi are already in the set for all 1≤i ≤k. Add rules that obviate the need forA→εrules:

for every existing ruleB →w ∈R⁰ withB ∈V⁰,

w ∈(V⁰∪Σ)⁺, letI_εbe the set of positions wherew contains a variable A∈V_ε. For every non-empty set I⁰ ⊆I_ε, add a new rule B →w⁰, wherew⁰ is constructed fromw by removing the variables at all positions in I⁰.

Remove all rules of the form A→ε (A6=S⁰).

. . .

Proof (continued).

Step 2: Eliminateε-rules of the form A→ε(A6=S⁰).

Let V_ε be the set of variable from which one can derive the empty word. We find this set Vε by first collecting all variables A∈V⁰ with rule A→ε∈R⁰ and then successively adding additional variablesB if there is a rule B →A₁A₂. . .A_k ∈R⁰ and the variablesAi are already in the set for all 1≤i ≤k. Add rules that obviate the need forA→εrules:

for every existing ruleB →w ∈R⁰ withB ∈V⁰,

w ∈(V⁰∪Σ)⁺, letI_εbe the set of positions wherew contains a variable A∈V_ε. For every non-empty set I⁰ ⊆I_ε, add a new rule B →w⁰, wherew⁰ is constructed fromw by removing the variables at all positions in I⁰.

Remove all rules of the form A→ε (A6=S⁰).

. . .

Chomsky Normal Form: Theorem

Proof (continued).

Step 2: Eliminateε-rules of the form A→ε(A6=S⁰).

Let V_ε be the set of variable from which one can derive the empty word. We find this set Vε by first collecting all variables A∈V⁰ with rule A→ε∈R⁰ and then successively adding additional variablesB if there is a rule B →A₁A₂. . .A_k ∈R⁰ and the variablesAi are already in the set for all 1≤i ≤k. Add rules that obviate the need forA→εrules:

for every existing ruleB →w ∈R⁰ withB ∈V⁰,

w ∈(V⁰∪Σ)⁺, letI_εbe the set of positions wherew contains a variable A∈V_ε. For every non-empty set I⁰ ⊆I_ε, add a new rule B →w⁰, wherew⁰ is constructed fromw by removing the variables at all positions in I⁰.

Remove all rules of the form A→ε (A6=S⁰).

. . .

Proof (continued).

Step 2: Eliminateε-rules of the form A→ε(A6=S⁰).

Let V_ε be the set of variable from which one can derive the empty word. We find this set Vε by first collecting all variables A∈V⁰ with rule A→ε∈R⁰ and then successively adding additional variablesB if there is a rule B →A₁A₂. . .A_k ∈R⁰ and the variablesAi are already in the set for all 1≤i ≤k. Add rules that obviate the need forA→εrules:

for every existing ruleB →w ∈R⁰ withB ∈V⁰,

w ∈(V⁰∪Σ)⁺, letI_εbe the set of positions wherew contains a variable A∈V_ε. For every non-empty set I⁰ ⊆I_ε, add a new rule B →w⁰, wherew⁰ is constructed fromw by removing the variables at all positions in I⁰.

Remove all rules of the form A→ε (A6=S⁰).

. . .

Step 2: Example

ConsiderG =h{X,Y,Z,S},{a,b},R,Si with rules:

S→ε|XY

X→aXYbX|YZ|ab Y→ε|b

Z→ε|a

Proof (continued).

Step 3: Eliminate rules of the form A→B with variablesA,B. If there are sets of variables{B₁, . . . ,B_k} with rules

B₁ →B₂,B₂ →B₃, . . . ,Bk−1 →B_k,B_k →B₁, then replace these variables by a new variableB.

We useV⁰⁰ to denote the resulting set of variables.

Define a strict total order<on the variables such that a rule A→B implies that A<B. Iterate from the largest to the smallest variableAand eliminate all rules of the form A→B while adding rulesA→w for every ruleB →w withw ∈(V⁰⁰∪Σ)⁺. . . .

Step 3: Example

ConsiderG =h{X,Y,Z,S},{a,b},R,Si with rules:

S→ε|X

X→aZbY|Y|ab Y→Z|b

Z→Y|bXa

Context-free Grammars Chomsky Normal Form Push-Down Automata Summary

Chomsky Normal Form: Theorem

Proof (continued).

Step 4: Eliminate rules with terminal symbols on the right-hand side that do not have the formA→a.

For every terminal symbola∈Σ add a new variable Aa

and the ruleA_a →a.

Replace all terminal symbols in all rules that do not have

the formA→awith the corresponding newly added variables. . . .

Chomsky Normal Form: Theorem

Proof (continued).

Step 5: Eliminate rules of the form A→B1B2. . .Bk with k >2 For every rule of the formA→B₁B₂. . .B_k with k >2, add new variablesC2, . . . ,Ck−1 and replace the rule with

A→B1C2

C₂→B₂C₃ ...

C_k−1→B_k−1B_k

(Example taken from textbook by Sipser)

ConsiderG =h{A,B,S},{a,b},R,Si with rules:

S→ASA|aB A→B|S B→ε|b

Specify a grammar G⁰ in CNF with L(G⁰) =L(G).

Chomsky Normal Form: Length of Derivations

Observation

LetG be a grammar in Chomsky normal form,

and letw ∈ L(G) be a non-empty word generated byG.

Then all derivations ofw have exactly 2|w| −1 derivation steps.

Proof.

Exercises

Questions?

Push-Down Automata

q0 0 q1 q2

0,1

0

Language Lis regular.

⇐⇒ There is a finite automaton that acceptsL.

What information can a finite automaton “store”

about the already read part of the word?

Infinite memory would be required for

L={x₁x2. . .xnxn. . .x2x1|n>0,xi ∈ {a,b}}.

therefore: extension of the automata model with memory

Limitations of Finite Automata

q0 0 q1 q2

0,1

0

Language Lis regular.

⇐⇒ There is a finite automaton that acceptsL.

What information can a finite automaton “store”

about the already read part of the word?

Infinite memory would be required for

L={x₁x2. . .xnxn. . .x2x1|n>0,xi ∈ {a,b}}.

therefore: extension of the automata model with memory

q0 0 q1 q2

0,1

0

Language Lis regular.

⇐⇒ There is a finite automaton that acceptsL.

What information can a finite automaton “store”

about the already read part of the word?

Infinite memory would be required for

L={x₁x2. . .xnxn. . .x2x1|n>0,xi ∈ {a,b}}.

therefore: extension of the automata model with memory

Stack

Astack is a data structure following thelast-in-first-out (LIFO) principle supporting the following operations:

push: puts an object on top of the stack

pop: removes the object at the top of the stack

Pop Push

German: Keller, Stapel

Input tape

I n p u t

Read head

Push-down automaton

Stack access

Stack

German: Kellerautomat, Eingabeband, Lesekopf, Kellerzugriff

Push-down Automaton for {a

b

| n ∈ N

}: Idea

As long as you read symbols a, push anA on the stack.

As soon as you read a symbol b, pop an Aoff the stack as long as you read b.

If reading the input is finished exactly when the stack becomes empty, accept the input.

If there is no Ato pop when reading a b, or there is still an A on the stack after reading all input symbols, or if you read an afollowing a bthen reject the input.

PDAs arenon-deterministic and can allow several next transitions from a configuration.

Like NFAs, PDAs can have transitions that do not read a symbol from the input.

Similarly, there can be transitions that do not pop and/or push a symbol off/to the stack.

Deterministic variants of PDAs are strictly less expressive, i. e. there are languages that can be recognized by a (non-deterministic) PDA but not the deterministic variant.

Push-down Automata: Non-determinism

PDAs arenon-deterministic and can allow several next transitions from a configuration.

Like NFAs, PDAs can have transitions that do not read a symbol from the input.

Similarly, there can be transitions that do not pop and/or push a symbol off/to the stack.

Deterministic variants of PDAs are strictly less expressive, i. e. there are languages that can be recognized by a (non-deterministic) PDA but not the deterministic variant.

q0 ε,ε→# q1 q2 q3

a,ε→A b,A→ε

b,A→ε ε,#→ε

Push-down Automata: Definition

Definition (Push-down Automaton)

Apush-down automaton (PDA) is a 6-tuple M =hQ,Σ,Γ, δ,q₀,Fi with

Q finite set of states Σ the input alphabet Γ the stack alphabet

δ :Q×(Σ∪ {ε})×(Γ∪ {ε})→ P(Q×(Γ∪ {ε})) the transition function

q0∈Q the start state

F ⊆Q is the set of accept states

German: Kellerautomat, Eingabealphabet, Kelleralphabet, Uberf¨¨ uhrungsfunktion

LetM =hQ,Σ,Γ, δ,q0,Fi be a push-down automaton.

What is the Intuitive Meaning of the Transition Functionδ?

hq⁰,Bi ∈δ(q,a,A): If M is in stateq, reads symbola and has Aas the topmost stack symbol,

then M cantransition toq⁰ in the next step

poppingA off the stack and pushing B on the stack.

q a,A→B q⁰

special casea=εis allowed (spontaneous transition) special caseA=εis allowed (no pop)

special caseB =εis allowed (no push)

Push-down Automata: Transition Function

LetM =hQ,Σ,Γ, δ,q0,Fi be a push-down automaton.

What is the Intuitive Meaning of the Transition Functionδ?

hq⁰,Bi ∈δ(q,a,A): If M is in stateq, reads symbola and has Aas the topmost stack symbol,

then M cantransition toq⁰ in the next step

poppingA off the stack and pushing B on the stack.

q a,A→B q⁰

special casea=εis allowed (spontaneous transition) special caseA=εis allowed (no pop)

special caseB =εis allowed (no push)

q0 ε,ε→# q1 q2 q3

a,ε→A b,A→ε

b,A→ε ε,#→ε

M =h{q₀,q1,q2,q3},{a,b},{A,#}, δ,q0,{q₀,q3}iwith

δ(q0,a,A) =∅ δ(q0,b,A) =∅ δ(q0, ε,A) =∅

δ(q₀,a,#) =∅ δ(q₀,b,#) =∅ δ(q₀, ε,#) =∅

δ(q0,a, ε) =∅ δ(q0,b, ε) =∅ δ(q0, ε, ε) ={(q1,#)}

δ(q1,a,A) =∅ δ(q1,b,A) ={(q2, ε)} δ(q1, ε,A) =∅

δ(q1,a,#) =∅ δ(q1,b,#) =∅ δ(q1, ε,#) =∅

δ(q1,a, ε) ={(q1,A)} δ(q1,b, ε) =∅ δ(q1, ε, ε) =∅ δ(q₂,a,A) =∅ δ(q₂,b,A) ={(q2, ε)} δ(q₂, ε,A) =∅ δ(q2,a,#) =∅ δ(q2,b,#) =∅ δ(q2, ε,#) ={(q3, ε)}

δ(q₂,a, ε) =∅ δ(q₂,b, ε) =∅ δ(q₂, ε, ε) =∅ andδ(q3,x,y) =∅for allx ∈ {a,b, ε},y ∈ {A,#, ε}

Context-free Grammars Chomsky Normal Form Push-Down Automata Summary

Push-down Automata: Accepted Words

Definition

A PDAM =hQ,Σ,Γ, δ,q₀,Fi accepts inputw

if it can be written asw =w₁w₂. . .w_m where eachw_i ∈Σ∪ {ε}

and sequences of statesr0,r1, . . . ,rm ∈Q and stringss₀,s₁, . . . ,s_m ∈Γ^∗ exist

that satisfy the following three conditions:

1 r₀=q₀ ands₀ =ε

2 For i = 0, . . . ,m−1, we have (r_i+1,b)∈δ(r_i,w_i+1,a), wheres_i =at ands_i+1=bt for somea,b∈Γ∪ {ε}and t ∈Γ^∗.

3 r_m∈F

Definition

A PDAM =hQ,Σ,Γ, δ,q₀,Fi accepts inputw

if it can be written asw =w₁w₂. . .w_m where eachw_i ∈Σ∪ {ε}

and sequences of statesr0,r1, . . . ,rm ∈Q and stringss₀,s₁, . . . ,s_m ∈Γ^∗ exist

that satisfy the following three conditions:

1 r₀=q₀ ands₀ =ε

2 For i = 0, . . . ,m−1, we have (r_i+1,b)∈δ(r_i,w_i+1,a), wheres_i =at ands_i+1=bt for somea,b∈Γ∪ {ε}and t ∈Γ^∗.

3 r_m∈F

The stringss_i represent the sequence of stack contents.

Push-down Automata: Accepted Words

Definition

A PDAM =hQ,Σ,Γ, δ,q₀,Fi accepts inputw

if it can be written asw =w₁w₂. . .w_m where eachw_i ∈Σ∪ {ε}

and sequences of statesr0,r1, . . . ,rm ∈Q and stringss₀,s₁, . . . ,s_m ∈Γ^∗ exist

that satisfy the following three conditions:

1 r₀=q₀ ands₀ =ε

2 For i = 0, . . . ,m−1, we have (r_i+1,b)∈δ(r_i,w_i+1,a), wheres_i =at ands_i+1=bt for somea,b∈Γ∪ {ε}and t ∈Γ^∗.

3 r_m∈F

The stringss_i represent the sequence of stack contents.

Definition

A PDAM =hQ,Σ,Γ, δ,q₀,Fi accepts inputw

if it can be written asw =w₁w₂. . .w_m where eachw_i ∈Σ∪ {ε}

and sequences of statesr0,r1, . . . ,rm ∈Q and stringss₀,s₁, . . . ,s_m ∈Γ^∗ exist

that satisfy the following three conditions:

1 r₀=q₀ ands₀ =ε

2 For i = 0, . . . ,m−1, we have (r_i+1,b)∈δ(r_i,w_i+1,a), wheres_i =at ands_i+1=bt for somea,b∈Γ∪ {ε}and t ∈Γ^∗.

3 r_m∈F

The stringss_i represent the sequence of stack contents.

Push-down Automata: Accepted Words

Definition

A PDAM =hQ,Σ,Γ, δ,q₀,Fi accepts inputw

if it can be written asw =w₁w₂. . .w_m where eachw_i ∈Σ∪ {ε}

and sequences of statesr0,r1, . . . ,rm ∈Q and stringss₀,s₁, . . . ,s_m ∈Γ^∗ exist

that satisfy the following three conditions:

1 r₀=q₀ ands₀ =ε

2 For i = 0, . . . ,m−1, we have (r_i+1,b)∈δ(r_i,w_i+1,a), wheres_i =at ands_i+1=bt for somea,b∈Γ∪ {ε}and t ∈Γ^∗.

3 r_m∈F

The stringss_i represent the sequence of stack contents.

q0 q1 q2 q3

ε,ε→# a,ε→A

b,A→ε b,A→ε

ε,#→ε

The PDA accepts inputaabb.

Acceptance: Exercise

q0 ε,ε→# q1 q2 q3

a,ε→A, b,ε→B

ε,ε→ε a,A→ε b,B→ε

ε,#→ε

Show that this PDA accepts inputabba.

Definition (Language Recognized by an NFA) LetM be a PDA with input alphabet Σ.

Thelanguage recognized by M is defined as L(M) ={w ∈Σ^∗ |w is accepted by M}.

Recognized Language: Exercise

q0 ε,ε→# q1 q2 q3

a,ε→A, b,ε→B

ε,ε→ε a,A→ε b,B→ε

ε,#→ε

What language does this PDA recognize?

Theorem

A language L is context-free if and only if L is recognized by a push-down automaton.

PDAs: Exercise (if time)

Assume you want to have a possible transition from state q to stateq⁰ in your PDA that

processes symbolcfrom the input word, can only be taken if the top stack symbol is A, doesnotpopAoff the stack, and

pushesB.

What problem do you encounter? How can you work around it?

Questions?

Summary

Every context-free language has a grammar in Chomsky normal form. All rules have form

A→BC with variables A,B,C (B,C not start variable), or A→awith variable A, terminal symbola, or

S →εwith start variableS.

Push-down automata (PDAs) extend NFAs with memory.

The languages recognized by PDAsare exactly thecontext-free languages.