Theory of Computer Science C5. Context-free Languages: Normal Form and PDA Gabriele R¨oger

Theory of Computer Science

C5. Context-free Languages: Normal Form and PDA

Gabriele R¨ oger

University of Basel

April 1, 2019

Theory of Computer Science

April 1, 2019 — C5. Context-free Languages: Normal Form and PDA

C5.1 Context-free Grammars and ε-Rules C5.2 Chomsky Normal Form

C5.3 Push-Down Automata

C5.4 Summary

Overview

Automata &

Formal Languages

Languages

& Grammars

Regular Languages

Context-free Languages

ε-rules

Chomsky Normal Form

PDAs

Pumping Lemma Closure Properties

Decidability Context-sensitive &

Type-0 Languages

C5. Context-free Languages: Normal Form and PDA Context-free Grammars andε-Rules

C5.1 Context-free Grammars and

ε-Rules

C5. Context-free Languages: Normal Form and PDA Context-free Grammars andε-Rules

Repetition: Context-free Grammars

Definition (Context-free Grammar)

A context-free grammar is a 4-tuple hΣ, V , P , S i with

1

Σ finite alphabet of terminal symbols,

2

V finite set of variables (with V ∩ Σ = ∅),

3

P ⊆ (V × (V ∪ Σ) ⁺ ) ∪ {hS, εi} finite set of rules,

4

If S → ε ∈ P , then all other rules in V × ((V \ {S}) ∪ Σ) ⁺ .

5

S ∈ V start variable.

Rule X → ε is only allowed if X = S and S never occurs on a right-hand side.

With regular grammars, this restriction could be lifted.

How about context-free grammars?

C5. Context-free Languages: Normal Form and PDA Context-free Grammars andε-Rules

Overview

Automata &

Formal Languages

Languages

& Grammars

Regular Languages

Context-free Languages

ε-rules

Chomsky Normal Form

PDAs

Pumping Lemma Closure Properties

Decidability Context-sensitive &

Type-0 Languages

C5. Context-free Languages: Normal Form and PDA Context-free Grammars andε-Rules

ε-Rules

Theorem

For every grammar G with rules P ⊆ V × (V ∪ Σ) ^∗ there is a context-free grammar G ⁰ with L(G ) = L(G ⁰ ).

Proof.

Let G = hΣ, V , P , S i be a grammar with P ⊆ V × (V ∪ Σ) ^∗ . Let V _ε = {A ∈ V | A ⇒ ^∗ ε}. We can find this set V _ε by first collecting all variables A with rule A → ε ∈ P and then successively adding additional variables B if there is a rule B → A ₁ A ₂ . . . A _k ∈ P and the variables A _i are already in the set

for all 1 ≤ i ≤ k. . . .

C5. Context-free Languages: Normal Form and PDA Context-free Grammars andε-Rules

ε-Rules

Theorem

For every grammar G with rules P ⊆ V × (V ∪ Σ) ^∗ there is a context-free grammar G ⁰ with L(G ) = L(G ⁰ ).

Proof (continued).

Let P ⁰ be the rule set that is constructed from P by

I adding rules that obviate the need for A → ε rules:

for every existing rule B → w with B ∈ V , w ∈ (V ∪ Σ) ⁺ , let I ε be the set of positions where w contains a variable A ∈ V ε . For every non-empty set I ⁰ ⊆ I ε , add a new rule B → w ⁰ , where w ⁰ is constructed from w by removing the variables at all positions in I ⁰ .

I removing all rules of the form A → ε (after the previous step).

. . .

C5. Context-free Languages: Normal Form and PDA Context-free Grammars andε-Rules

ε-Rules

Theorem

For every grammar G with rules P ⊆ V × (V ∪ Σ) ^∗ there is a context-free grammar G ⁰ with L(G ) = L(G ⁰ ).

Proof (continued).

Then L(G ) \ {ε} = L(hΣ, V , P ⁰ , Si) and P ⁰ contains no rule A → ε. If the start variable S of G is not in V _ε , we are done.

Otherwise, let S ⁰ be a new variable and construct P ⁰⁰ from P ⁰ by

1

replacing all occurrences of S on the right-hand side of rules with S ⁰ ,

2

adding the rule S ⁰ → w for every rule S → w , and

3

adding the rule S → ε.

Then L(G ) = L(hΣ, V ∪ {S ⁰ }, P ⁰⁰ , S i).

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

C5.2 Chomsky Normal Form

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

Overview

Automata &

Formal Languages

Languages

& Grammars

Regular Languages

Context-free Languages

ε-rules

Chomsky Normal Form

PDAs

Pumping Lemma Closure Properties

Decidability Context-sensitive &

Type-0 Languages

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

Chomsky Normal Form: Motivation

As in logical formulas (and other kinds of structured objects), normal forms for grammars are useful:

I they show which aspects are critical for defining grammars and which ones are just syntactic sugar

I they allow proofs and algorithms to be restricted

to a limited set of grammars (inputs): those in normal form

Hence we now consider a normal form for context-free grammars.

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

Chomsky Normal Form: Definition

Definition (Chomsky Normal Form)

A context-free grammar G is in Chomsky normal form (CNF) if all rules have one of the following three forms:

I A → BC with variables A, B, C , or

I A → a with variable A, terminal symbol a, or

I S → ε with start variable S . German: Chomsky-Normalform

in short: rule set P ⊆ (V × (VV ∪ Σ)) ∪ {hS, εi}

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

Chomsky Normal Form: Theorem

Theorem

For every context-free grammar G there is a context-free grammar G ⁰ in Chomsky normal form with L(G) = L(G ⁰ ).

Proof.

The following algorithm converts the rule set of G into CNF:

Step 1: Eliminate rules of the form A → B with variables A, B . If there are sets of variables {B ₁ , . . . , B k } with rules

B 1 → B 2 , B 2 → B 3 , . . . , B k−1 → B _k , B _k → B 1 , then replace these variables by a new variable B.

Define a strict total order < on the variables such that A → B ∈ P

implies that A < B. Iterate from the largest to the smallest

variable A and eliminate all rules of the form A → B while adding

rules A → w for every rule B → w with w ∈ (V ∪ Σ) ⁺ . . . .

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

Chomsky Normal Form: Theorem

Theorem

For every context-free grammar G there is a context-free grammar G ⁰ in Chomsky normal form with L(G) = L(G ⁰ ).

Proof (continued).

Step 2: Eliminate rules with terminal symbols on the

Step 2:

right-hand side that do not have the form A → a.

For every terminal symbol a ∈ Σ add a new variable A _a and the rule A a → a.

Replace all terminal symbols in all rules that do not have

the form A → a with the corresponding newly added variables. . . .

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

Chomsky Normal Form: Theorem

Theorem

For every context-free grammar G there is a context-free grammar G ⁰ in Chomsky normal form with L(G) = L(G ⁰ ).

Proof (continued).

Step 3: Eliminate rules of the form A → B ₁ B ₂ . . . B _k with k > 2 For every rule of the form A → B ₁ B ₂ . . . B _k with k > 2, add new variables C ₂ , . . . , C k−1 and replace the rule with

A → B ₁ C ₂ C ₂ → B ₂ C ₃

.. .

C k−1 → B k−1 B _k

C5. Context-free Languages: Normal Form and PDA Chomsky Normal Form

Chomsky Normal Form: Length of Derivations

Observation

Let G be a grammar in Chomsky normal form,

and let w ∈ L(G ) be a non-empty word generated by G .

Then all derivations of w have exactly 2|w | − 1 derivation steps.

Proof.

Exercises

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

C5.3 Push-Down Automata

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Overview

Automata &

Formal Languages

Languages

& Grammars

Regular Languages

Context-free Languages

ε-rules

Chomsky Normal Form

PDAs

Pumping Lemma Closure Properties

Decidability Context-sensitive &

Type-0 Languages

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Limitations of Finite Automata

q

0 q

q

0,1

0

I Language L is regular.

⇐⇒ There is a finite automaton that accepts L.

I What information can a finite automaton “store”

about the already read part of the word?

I Infinite memory would be required for

L = {x ₁ x 2 . . . x n x n . . . x 2 x 1 | n > 0, x i ∈ {a, b}}.

I therefore: extension of the automata model with memory

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Stack

A stack is a data structure following the last-in-first-out (LIFO) principle supporting the following operations:

I push: puts an object on top of the stack

I pop: removes the object at the top of the stack

I peek: returns the top object without removing it

Push Pop

German: Keller, Stapel

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Visually

Input tape

I n p u t

Read head

Push-down automaton

Stack access

Stack

German: Kellerautomat, Eingabeband, Lesekopf, Kellerzugriff

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Definition

Definition (Push-down Automaton)

A push-down automaton (PDA) is a 6-tuple M = hQ , Σ, Γ, δ, q 0 , #i with

I Q finite set of states

I Σ the input alphabet

I Γ the stack alphabet

I δ : Q × (Σ ∪ {ε}) × Γ → P _f (Q × Γ ^∗ ) the transition function (where P _f is the set of all finite subsets)

I q 0 ∈ Q the start state

I # ∈ Γ the bottommost stack symbol

German: Kellerautomat, Eingabealphabet, Kelleralphabet,

German: Uberf¨ ¨ uhrungsfunktion

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Transition Function

Let M = hQ, Σ, Γ, δ, q 0 , #i be a push-down automaton.

What is the Intuitive Meaning of the Transition Function δ?

I hq ⁰ , B ₁ . . . B _k i ∈ δ(q, a, A): If M is in state q, reads symbol a and has A as the topmost stack symbol,

then M can transition to q ⁰ in the next step while replacing A with B ₁ . . . B _k (afterwards B ₁ is the topmost stack symbol)

q a, A → B

. . . B

q

I special case a = ε is allowed (spontaneous transition)

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Example

q q

a, A → AA a, B → AB a, # → A#

b, A → BA b, B → BB b, # → B#

a, A → ε b, B → ε

a,A → ε b,B → ε ε, # → ε

M = h{q, q ⁰ }, { a, b }, { A, B, # }, δ, q, # i with

δ(q, a, A) = {hq, AAi, hq

, εi} δ(q, b, A) = {hq, BAi} δ(q, ε, A) = ∅ δ(q, a, B) = {hq, ABi} δ(q, b, B) = {hq, BBi, hq

, εi} δ(q, ε, B) = ∅ δ(q, a, #) = {hq, A#i} δ(q, b, #) = {hq, B#i} δ(q, ε, #) = ∅ δ(q

, a, A) = {hq

, εi} δ(q

, b, A) = ∅ δ(q

, ε, A) = ∅ δ(q

, a, B) = ∅ δ(q

, b, B) = {hq

, εi} δ(q

, ε, B) = ∅

δ(q

, a, #) = ∅ δ(q

, b, #) = ∅ δ(q

, ε, #) = {hq

, εi}

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Configuration

Definition (Configuration of a Push-down Automaton)

A configuration of a push-down automaton M = hQ, Σ, Γ, δ, q ₀ , #i is given by a triple c ∈ Q × Σ ^∗ × Γ ^∗ .

German: Konfiguration Example

I n p u t

q

Configuration

hq, ut, BAC#i.

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Steps

Definition (Transition/Step of a Push-down Automaton)

We write c ` _M c ⁰ if a push-down automaton M = hQ, Σ, Γ, δ, q 0 , #i can transition from configuration c to configuration c ⁰ in one step.

Exactly the following transitions are possible:

hq, a 1 . . . a n , A 1 . . . A m i ` _M



 

 



 

 



hq ⁰ , a ₂ . . . a _n , B ₁ . . . B _k A ₂ . . . A _m i if hq ⁰ , B 1 . . . B _k i ∈ δ(q, a 1 , A 1 ) hq ⁰ , a 1 a 2 . . . a n , B 1 . . . B k A 2 . . . A m i

if hq ⁰ , B ₁ . . . B _k i ∈ δ(q, ε, A ₁ ) German: Ubergang ¨

If M is clear from context, we only write c ` c ⁰ .

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Reachability of Configurations

Definition (Reachable Configuration)

Configuration c ⁰ is reachable from configuration c in PDA M (c ` ^∗ _M c ⁰ ) if there are configurations c 0 , . . . , c n (n ≥ 0) where

I c 0 = c ,

I c i ` _M c i+1 for all i ∈ {0, . . . , n − 1}, and

I c _n = c ⁰ .

German: c

ist in M von c erreichbar

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Recognized Words

Definition (Recognized Word of a Push-down Automaton) PDA M = hQ, Σ, Γ, δ, q 0 , #i recognizes the word w = a 1 . . . a n

iff the configuration hq, ε, εi (word processed and stack empty) for some q ∈ Q is reachable from the start configuration hq ₀ , w , # i.

M recognizes w iff hq ₀ , w , #i ` ^∗ _M hq, ε, εi for some q ∈ Q.

German: M erkennt w , Startkonfiguration

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Recognized Word Example

q q

a, A → AA a, B → AB a, # → A#

b, A → BA b, B → BB b, # → B#

a, A → ε b, B → ε

a,A → ε b,B → ε ε, # → ε

example: this PDA recognizes bbabbabb blackboard

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

Push-down Automata: Accepted Language

Definition (Accepted Language of a Push-down Automaton) Let M be a push-down automaton with input alphabet Σ.

The language accepted by M is defined as

L(M ) = {w ∈ Σ ^∗ | M recognizes w }.

example: blackboard

C5. Context-free Languages: Normal Form and PDA Push-Down Automata

PDAs Accept Exactly the Context-free Languages

Theorem

A language L is context-free if and only if

L is accepted by a push-down automaton.

C5. Context-free Languages: Normal Form and PDA Summary

C5.4 Summary

C5. Context-free Languages: Normal Form and PDA Summary

Summary

I Every context-free language has a grammar in Chomsky normal form. All rules have form

I

A → BC with variables A, B, C, or

I

A → a with variable A, terminal symbol a, or

I

S → ε with start variable S.

I Push-down automata (PDAs) extend NFAs with memory.

I PDAs accept not with end states but with an empty stack.

I The languages accepted by PDAs are exactly

the context-free languages.