16. Binary Search Trees

16. Binary Search Trees

[Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap. 12.1 - 12.3]

418

Dictionary implementation

Hashing: implementation of dictionaries with expected very fast access times.

Disadvantages of hashing: linear access time in worst case. Some operations not supported at all:

enumerate keys in increasing order next smallest key to given key

419

Trees

Trees are

Generalized lists: nodes can have more than one successor Special graphs: graphs consist of nodes and edges. A tree is a fully connected, directed, acyclic graph.

Trees

Use

Decision trees: hierarchic representation of decision rules

syntax trees: parsing and traversing of expressions, e.g. in a compiler

Code tress: representation of a code, e.g.

morse alphabet, huffman code

Search trees: allow efficient searching for an element by value

Examples

start E

I S

H V

U

F U

A R

L A

W

P I

T N D

B X

K

C Y

M G

Z Q

O Ö CH

short long

Morsealphabet

422

Examples

3/5 + 7.0 +

/

3 5

7.0

Expression tree

423

Nomenclature

Wurzel

W

I E

K

parent

child inner node

leaves

Order of the tree: maximum number of child nodes, here: 3 Height of the tree: maximum path length root – leaf (here: 4)

Binary Trees

A binary tree is either

a leaf, i.e. an empty tree, or

an inner leaf with two treesT_l (left subtree) andT_r (right subtree) as left and right successor.

In each nodev we store a keyv.key and

two nodesv.leftandv.rightto the roots of the left and right subtree.

a leaf is represented by thenull-pointer

key left right

Binary search tree

A binary search tree is a binary tree that fulfils the search tree property:

Every nodev stores a key

Keys in the left subtreev.left ofv are smaller than v.key Key in the right subtree v.rightofv are larger thanv.key

16 7

5 2

10 9 15

18

17 30

99

426

Searching

Input : Binary search tree with rootr, keyk Output : Nodevwith v.key =k ornull v←r

whilev6=null do if k=v.key then

return v

else if k < v.key then v←v.left

else

v←v.right return null

8

4 13

10 9

19

Search (12)→^null

427

Height of a tree

The height h(T)of a treeT with rootris given by h(r) =

(0 ifr =null 1 + max{h(r.left), h(r.right)} ^otherwise.

The worst case run time of the search is thusO(h(T))

Insertion of a key

Insertion of the keyk Search fork

If successful search: output error

Of no success: insert the key at the leaf reached

8 4

5

13 10 9

19

Insert (5)

Remove node

Three cases possible:

Node has no children Node has one child Node has two children

[Leaves do not count here]

8 3

5 4

13 10 9

19

430

Remove node

Node has no children

Simple case: replace node by leaf.

8 3

5 4

13 10 9

19

remove(4)

−→

8 3

5

13 10 9

19

431

Remove node

Node has one child

Also simple: replace node by single child.

8 3

5 4

13 10 9

19

remove(3)

−→

8 5

4

13 10 9

19

Remove node

Node has two children

The following observation helps: the smallest key in the right subtree v.right (thesymmetric successorof v)

is smaller than all keys inv.right is greater than all keys inv.left and cannot have a left child.

Solution: replacev by its symmetric suc- cessor.

8 3

5 4

13 10 9

19

By symmetry...

Node has two children

Also possible: replacev by its symmetric predecessor.

8 3

5 4

13 10 9

19

434

Algorithm SymmetricSuccessor( v )

Input : Nodevof a binary search tree.

Output : Symmetric successor ofv w←v.right

x←w.left

whilex6=null do w←x x←x.left returnw

435

Analysis

Deletion of an elementv from a tree T requiresO(h(T)) fundamental steps:

Findingv has costsO(h(T))

If v has maximal one child unequal tonullthen removal takes O(1) steps

Finding the symmetric successor nof v takesO(h(T))steps.

Removal and insertion of ntakesO(1) steps.

Traversal possibilities

preorder: v, then T_left(v), then T_right(v).

8, 3, 5, 4, 13, 10, 9, 19

postorder: T_left(v), then T_right(v), then v.

4, 5, 3, 9, 10, 19, 13, 8

inorder: T_left(v), thenv, then T_right(v). 3, 4, 5, 8, 9, 10, 13, 19

8 3

5 4

13 10 9

19

Further supported operations

Min(T): Read-out minimal value in O(h)

ExtractMin(T): Read-out and remove minimal value inO(h)

List(T): Output the sorted list of elements

Join(T₁, T₂): Merge two trees with max(T₁) <min(T₂) in O(n).

8 3

5 4

13 10 9

19

438

Degenerated search trees

9 5

4 8

13

10 19

Insert 9,5,13,4,8,10,19

ideally balanced

4 5

8 9

10 13

19

Insert 4,5,8,9,10,13,19

linear list

19 13 10 9 8 5

4

Insert 19,13,10,9,8,5,4

linear list

439

17. AVL Trees

Balanced Trees [Ottman/Widmayer, Kap. 5.2-5.2.1, Cormen et al, Kap. Problem 13-3]

Objective

Searching, insertion and removal of a key in a tree generated fromn keys inserted in random order takes expected number of steps O(log₂n).

But worst caseΘ(n) (degenerated tree).

Goal: avoidance of degeneration. Artificial balancing of the tree for each update-operation of a tree.

Balancing: guarantee that a tree withnnodes always has a height of O(logn).

Adelson-Venskii and Landis (1962): AVL-Trees

Balance of a node

The heightbalanceof a node v is de- fined as the height difference of its sub-treesT_l(v)andT_r(v)

bal(v) := h(T_r(v))−h(T_l(v))

v

T_l(v)

T_r(v) h_l

h_r

bal(v)

442

AVL Condition

AVL Condition: for eacn node v of a treebal(v) ∈ {−1,0,1}

v

T_l(v)

T_r(v)

h h+ 1

h+ 2

443

(Counter-)Examples

AVL tree with height

2 AVL tree with height

3 No AVL tree

Number of Leaves

1. observation: a binary search tree withnkeys provides exactly n+ 1leaves. Simple induction argument.

2. observation: a lower bound of the number of leaves in a search tree with given height implies an upper bound of the height of a search tree with given number of keys.

Lower bound of the leaves

AVL tree with height1has M(1) := 2leaves.

AVL tree with height2has at leastM(2) := 3leaves.

446

Lower bound of the leaves for h > 2

Height of one subtree≥h−1. Height of the other subtree≥h−2. Minimal number of leavesM(h) is

M(h) =M(h−1) +M(h−2)

v

T_l(v)

T_r(v)

h−2 h−1

h

Overal we haveM(h) =F_h+2 withFibonacci-numbersF₀ := 0, F₁ := 1,F_n := F_n−1+F_n−2forn >1.

447

[Fibonacci Numbers: closed form]

Closed form of the Fibonacci numbers: computation via generation functions:

1 Power series approach

f(x) :=

X∞ i=0

F_i·xⁱ

[Fibonacci Numbers: closed form]

2 For Fibonacci Numbers it holds thatF₀ = 0,F₁ = 1, F_i =F_i−1+F_i−2∀i >1. Therefore:

f(x) =x+ X∞

i=2

F_i·xⁱ =x+ X∞

i=2

F_i−1·xⁱ+ X∞

i=2

F_i−2·xⁱ

= x+x X∞

i=2

F_i−1·xⁱ⁻¹+x² X∞

i=2

F_i−2·xⁱ⁻²

= x+x X∞

i=0

F_i·xⁱ+x² X∞

i=0

F_i·xⁱ

= x+x·f(x) +x²·f(x).

[Fibonacci Numbers: closed form]

3 Thus:

f(x)·(1−x−x²) =x.

⇔ f(x) = x 1−x−x²

⇔ f(x) = x

(1−φx)·(1−φx)ˆ with the rootsφandφˆof1−x−x².

φ= 1 +√ 5 2 φˆ= 1−√

5 2 .

450

[Fibonacci Numbers: closed form]

4 It holds that:

(1−φx)ˆ −(1−φx) =√ 5·x.

Damit:

f(x) = 1

√5

(1−φx)ˆ −(1−φx) (1−φx)·(1−φx)ˆ

= 1

√5

1

1−φx − 1 1−φxˆ

451

[Fibonacci Numbers: closed form]

5 Power series of g_a(x) = ₁₋¹_a·x (a ∈R):

1

1−a·x = X∞

i=0

aⁱ·xⁱ.

E.g. Taylor series ofg_a(x)atx= 0or like this: LetP_∞

i=0G_i·xⁱa power series ofg. By the identityg_a(x)(1−a·x) = 1it holds that

1 = X∞

G_i·xⁱ−a· X∞

G_i·xⁱ⁺¹=G₀+ X∞

(G_i−a·G_i−1)·xⁱ

[Fibonacci Numbers: closed form]

6 Fill in the power series:

f(x) = 1

√5

1

1−φx − 1 1−φxˆ

= 1

√5 X∞

i=0

φⁱxⁱ− X∞

i=0

φˆⁱxⁱ

!

= X∞

i=0

√1

5(φⁱ−φˆⁱ)xⁱ

Comparison of the coefficients withf(x) =P_∞

i=0F_i·xⁱ yields

Fibonacci Numbers

It holds thatF_i = √¹

5(φⁱ−φˆⁱ)with rootsφ,φˆof the equation x² =x+ 1(golden ratio), thusφ = ¹⁺₂^√5, φˆ= ¹⁻₂^√5.

Proof (induction). Immediate fori= 0, i= 1. Leti >2: Fi=Fi−1+Fi−2= 1

√5(φⁱ⁻¹−φˆⁱ⁻¹) + 1

√5(φⁱ⁻²−φˆⁱ⁻²)

= 1

√5(φⁱ⁻¹+φⁱ⁻²)− 1

√5( ˆφⁱ⁻¹+ ˆφⁱ⁻²) = 1

√5φⁱ⁻²(φ+ 1)− 1

√5

φˆⁱ⁻²( ˆφ+ 1)

= 1

√5φⁱ⁻²(φ²)− 1

√5

φˆⁱ⁻²( ˆφ²) = 1

√5(φⁱ−φˆⁱ).

454

Tree Height

Becauseφ <ˆ 1, overal we have

M(h) ∈Θ



 1 +√ 5 2

!h

⊆ Ω(1.618^h)

and thus

h ≤1.44 log₂n+c.

AVL tree is asymptotically not more than44%higher than a perfectly balanced tree.

455

Insertion

Balance

Keep the balance stored in each node

Re-balance the tree in each update-operation New noden is inserted:

Insert the node as for a search tree.

Check the balance condition increasing from nto the root.

Balance at Insertion Point

=⇒

+1 0

p p

n

case 1:bal(p) = +1

=⇒

−1 0

p p

n

case 2: bal(p) =−1

Finished in both cases because the subtree height did not change

Balance at Insertion Point

=⇒

0 +1

p p

n

case 3.1: bal(p) = 0right

=⇒

0 −1

p p

n

case 3.2: bal(p) = 0, left

Not finished in both case. Call ofupin(p)

458

upin(p) - invariant

Whenupin(p) is called it holds that the subtree frompis grown and bal(p) ∈ {−1,+1}

459

upin(p)

Assumption: pis left son ofpp¹⁷

=⇒

pp +1 pp 0

p p

case 1: bal(pp) = +1, done.

=⇒

pp 0 pp −1

p p

case 2: bal(pp) = 0,upin(pp) In both cases the AVL-Condition holds for the subtree frompp

upin(p)

Assumption: pis left son ofpp

pp −1

p

case 3: bal(pp) =−1,

This case is problematic: addingn to the subtree frompphas

Rotationen

case 1.1bal(p) =−1. ¹⁸

y x

t1

t2

t3

pp −1

p −1

h

h−1 h−1

=⇒ rotation

right

x y

t1 t2 t3

pp 0

p 0

h h−1 h−1

18pright son:bal(pp) = bal(p) = +1, left rotation

462

Rotationen

case 1.1bal(p) =−1. ¹⁹

z

x y

t1

t2

t3

t4

pp −1

p +1

h

h−1 h−1 h−2

h−2 h−1

h−1

=⇒ double rotation left-right

y

x z

t1 t2

t3

t4

pp 0

h−1 h−1 h−2

h−2

h−1 h−1

19pright son:bal(pp) = +1,bal(p) =−1,double rotation right left

463

Analysis

Tree height: O(logn).

Insertion like in binary search tree.

Balancing via recursion from node to the root. Maximal path lenght O(logn).

Insertion in an AVL-tree provides run time costs ofO(logn).

Deletion

Case 1: Children of noden are both leaves Letpbe parent node of n. ⇒ Other subtree has heighth⁰ = 0,1or2.

h⁰= 1: Adaptbal(p).

h⁰= 0: Adaptbal(p). Callupout(p).

h⁰= 2: Rebalanciere des Teilbaumes. Callupout(p).

p n

h= 0,1,2

−→

p

h= 0,1,2

Deletion

Case 2: one childk of nodenis an inner node Replacen byk. upout(k)

p n

k −→

p k

466

Deletion

Case 3: both children of noden are inner nodes Replacenby symmetric successor. upout(k)

Deletion of the symmetric successor is as in case 1 or 2.

467

upout(p)

Letppbe the parent node ofp. (a) pleft child ofpp

1 bal(pp) =−1 ⇒ bal(pp)←0. upout(pp)

2 bal(pp) = 0 ⇒ bal(pp)←+1.

3 bal(pp) = +1⇒next slides.

(b) pright child ofpp: Symmetric cases exchanging+1and−1.

upout(p)

Case (a).3: bal(pp) = +1. Letq be brother ofp (a).3.1:bal(q) = 0.²⁰

y

pp +1

x

p 0 ^q z 0

1 2

3 4

h−1 h−1

=⇒ Left Rotate(y)

z −1

y +1

x 0

1 2 2

h−1 h−1 h+ 1

upout(p)

Case (a).3: bal(pp) = +1. (a).3.2: bal(q) = +1.²¹

y

pp +1

x

p 0 ^q z +1

1 2

3 4

h−1 h−1

h

h+ 1

=⇒ Left Rotate(y)

z 0 r

y 0

x 0

1 2 3 4

h−1 h−1 h h+ 1

plusupout(r).

21(b).3.2:bal(pp) =−1,bal(q) = +1, Right rotation+upout

470

upout(p)

Case (a).3: bal(pp) = +1. (a).3.3: bal(q) =−1.²²

y

pp +1

x

p 0 ^q z −1

w

1 2

3 4

h−1 h−1 5

h

=⇒ Rotate right

(z) left (y)

w 0 r

y 0

x

z

0

1 2 3 4 5

h−1 h−1 h

plusupout(r).

22(b).3.3:bal(pp) =−1,bal(q) =−1, left-right rotation + upout

471

Conclusion

AVL trees have worst-case asymptotic runtimes ofO(logn)for searching, insertion and deletion of keys.

Insertion and deletion is relatively involved and an overkill for really small problems.