CS 3343 -- Spring 2009

Search Trees • A binary search tree is a binary tree. Each node stores

a key. The tree fulfills the binary search tree property: For every node x holds: • y≤ x , for all y in the subtree left of x • x < y, for all y in the subtree right of x 77

Red-black trees

33

Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk 3/5/09

CS 3343 Analysis of Algorithms

88

1

3/5/09

CS 3343 Analysis of Algorithms

2

Abstract data type (ADT) Dictionary (also called Dynamic Set): A data structure which supports operations 77 • Insert 33 18 18 10 22 22 10 • Delete 88 15 15 • Find 12 17 12 17 Using balanced binary search trees we can implement a dictionary data structure such that each operation takes O(log n) time.

• Balanced search trees (guarantee height of log n for n elements) • k-ary search trees (such as B-trees, 2-3-4-trees) 77 33

18 18 10 10 88

22 22 15 15

12 12 CS 3343 Analysis of Algorithms

17 17

ADT Dictionary / Dynamic Set

Different variants of search trees:

3/5/09

22 22 15 15

12 12

Search Trees

• Search trees that store the keys only in the leaves, and store additional split-values in the internal nodes

18 18 10 10

17 17 3

3/5/09

CS 3343 Analysis of Algorithms

4

1

Balanced search trees

Red-black trees

Balanced search tree: A search-tree data structure for which a height of O(log n) is guaranteed when implementing a dynamic set of n items.

Examples:

3/5/09

This data structure requires an extra onebit color field in each node. Red-black properties: 1. Every node is either red or black. 2. The root is black. 3. The leaves (NIL’s) are black. 4. If a node is red, then both its children are black. 5. All simple paths from any node x, excluding x, to a descendant leaf have the same number of black nodes = black-height(x).

• AVL trees • 2-3 trees • 2-3-4 trees • B-trees • Red-black trees

CS 3343 Analysis of Algorithms

5

3/5/09

CS 3343 Analysis of Algorithms

Example of a red-black tree

Example of a red-black tree

77

77

33 NIL

18 18 NIL

10 10 88

6

33

h=4

22 22 11 11

NIL NIL NIL NIL

NIL

NIL

26 26 NIL

18 18 NIL

10 10 88

NIL

22 22 11 11

NIL

NIL NIL NIL NIL

26 26 NIL

NIL

1. Every node is either red or black. 3/5/09

CS 3343 Analysis of Algorithms

7

3/5/09

CS 3343 Analysis of Algorithms

8

2

Example of a red-black tree

Example of a red-black tree

77

77

33 NIL

18 18 NIL

10 10 88

33 NIL

22 22 11 11

NIL

NIL NIL NIL NIL

26 26 NIL

NIL

CS 3343 Analysis of Algorithms

9

bh = 1 10 10 bh = 1

88

bh = 0 NIL NIL NIL NIL

NIL

26 26 NIL

NIL

26 26 NIL

NIL

3/5/09

CS 3343 Analysis of Algorithms

10

Theorem. A red-black tree with n keys has height h ≤ 2 log(n + 1). Proof. (The book uses induction. Read carefully.) INTUITION: • Merge red nodes into their black parents.

22 22 11 11

11 11

Height of a red-black tree

18 18 bh = 2 NIL

22 22

4. If a node is red, then both its children are black.

77 bh = 2

NIL

10 10

NIL NIL NIL NIL

Example of a red-black tree 33

NIL

88

2., 3. The root and leaves (NIL’s) are black. 3/5/09

18 18

NIL

5. All simple paths from any node x, excluding x, to a descendant leaf have the same number of black nodes = black-height(x). 3/5/09

CS 3343 Analysis of Algorithms

11

3/5/09

CS 3343 Analysis of Algorithms

12

3

Height of a red-black tree

Height of a red-black tree

Theorem. A red-black tree with n keys has height h ≤ 2 log(n + 1). Proof. (The book uses induction. Read carefully.) INTUITION: • Merge red nodes into their black parents.

3/5/09

CS 3343 Analysis of Algorithms

13

Theorem. A red-black tree with n keys has height h ≤ 2 log(n + 1). Proof. (The book uses induction. Read carefully.) INTUITION: • Merge red nodes into their black parents.

3/5/09

Height of a red-black tree

CS 3343 Analysis of Algorithms

14

Height of a red-black tree

Theorem. A red-black tree with n keys has height h ≤ 2 log(n + 1). Proof. (The book uses induction. Read carefully.) INTUITION: • Merge red nodes into their black parents.

3/5/09

CS 3343 Analysis of Algorithms

15

Theorem. A red-black tree with n keys has height h ≤ 2 log(n + 1). Proof. (The book uses induction. Read carefully.) INTUITION: • Merge red nodes into their black parents.

3/5/09

CS 3343 Analysis of Algorithms

16

4

Height of a red-black tree

Proof (continued)

Theorem. A red-black tree with n keys has height h ≤ 2 log(n + 1). Proof. (The book uses induction. Read carefully.) INTUITION: • Merge red nodes h′ into their black parents. • This process produces a tree in which each node has 2, 3, or 4 children. • The 2-3-4 tree has uniform depth h′ of leaves. 3/5/09

CS 3343 Analysis of Algorithms

17

• We have h′ ≥ h/2, since at most half the vertices on any path are red. • The number of leaves in each tree is n + 1 ⇒ n + 1 ≥ 2h' ⇒ log(n + 1) ≥ h' ≥ h/2 ⇒ h ≤ 2 log(n + 1). 3/5/09

Query operations

18 18 10 10 88

22 22 NIL 11 11

NIL NIL NIL NIL 3/5/09

18

The operations INSERT and DELETE cause modifications to the red-black tree: 1. the operation itself, 2. color changes, 3. restructuring the links of the tree via “rotations”.

77 NIL NIL

h′

Modifying operations

Corollary. The queries SEARCH, MIN, MAX, SUCCESSOR, and PREDECESSOR all run in O(log n) time on a red-black tree with n nodes. 33

CS 3343 Analysis of Algorithms

h

CS 3343 Analysis of Algorithms

26 26 NIL NIL 19

3/5/09

CS 3343 Analysis of Algorithms

20

5

Rotations RIGHT-ROTATE(B)

BB

AA

LEFT-ROTATE(A)

AA αα

Red-black trees

ββ

γγ

BB

αα

ββ

γγ

• Rotations maintain the inorder ordering of keys: a ∈ α, b ∈ β, c ∈ γ ⇒ a ≤ A ≤ b ≤ B ≤ c. • Rotations maintain the binary search tree property • A rotation can be performed in O(1) time. 3/5/09

CS 3343 Analysis of Algorithms

21

This data structure requires an extra onebit color field in each node. Red-black properties: 1. Every node is either red or black. 2. The root is black. 3. The leaves (NIL’s) are black. 4. If a node is red, then both its children are black. 5. All simple paths from any node x, excluding x, to a descendant leaf have the same number of black nodes = black-height(x). 3/5/09

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: • Insert x =15.

88

22 22 11 11

Insertion into a red-black tree

Example: 33 • Insert x =15. • Recolor, moving the violation up the tree.

18 18 10 10

26 26

77 18 18 10 10 88

15 15 3/5/09

CS 3343 Analysis of Algorithms

22

IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring.

77 33

CS 3343 Analysis of Algorithms

22 22 11 11

26 26

15 15 23

3/5/09

CS 3343 Analysis of Algorithms

24

6

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: 33 • Insert x =15. • Recolor, moving the violation up the tree.

IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring.

77

Example: 33 • Insert x =15. • Recolor, moving the violation up the tree. • RIGHT-ROTATE(18).

18 18 10 10 88

22 22 11 11

26 26

15 15 3/5/09

CS 3343 Analysis of Algorithms

Insertion into a red-black tree

25

3/5/09

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: 33 • Insert x =15. • Recolor, moving the violation up the tree. • RIGHT-ROTATE(18). 3/5/09

CS 3343 Analysis of Algorithms

18 18

15 15

10 10 88

22 22 11 11

26 26

15 15

CS 3343 Analysis of Algorithms

26

Insertion into a red-black tree

Example: 33 • Insert x =15. • Recolor, moving the violation up the tree. • RIGHT-ROTATE(18). • LEFT-ROTATE(7)

10 10

11 11

18 18

IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring.

77

88

77

22 22 26 26 27

3/5/09

77

CS 3343 Analysis of Algorithms

10 10 88

18 18 11 11 15 15

22 22 26 26 28

7

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: • Insert x =15. • Recolor, moving the violation up the tree. • RIGHT-ROTATE(18). • LEFT-ROTATE(7) 3/5/09

IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring.

10 10

33

Example: 77 • Insert x =15. • Recolor, moving the 33 88 violation up the tree. • RIGHT-ROTATE(18). • LEFT-ROTATE(7) and recolor.

18 18

77 11 11

88

22 22

15 15

26 26

CS 3343 Analysis of Algorithms

Insertion into a red-black tree

29

3/5/09

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 4 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: 77 • Insert x =15. • Recolor, moving the 33 88 violation up the tree. • RIGHT-ROTATE(18). • LEFT-ROTATE(7) and recolor. 3/5/09

CS 3343 Analysis of Algorithms

18 18

15 15

18 18 11 11 15 15

22 22 26 26

30

Pseudocode RB-INSERT(T, x) TREE-INSERT(T, x) color[x] ← RED ⊳ only RB property 4 can be violated while x ≠ root[T] and color[p[x]] = RED do if p[x] = left[p[p[x]] then y ← right[p[p[x]] ⊳ y = aunt/uncle of x if color[y] = RED then 〈Case 1〉 else if x = right[p[x]] then 〈Case 2〉 ⊳ Case 2 falls into Case 3 〈Case 3〉 else 〈“then” clause with “left” and “right” swapped〉 color[root[T]] ← BLACK

10 10

11 11

CS 3343 Analysis of Algorithms

10 10

22 22 26 26

31

3/5/09

CS 3343 Analysis of Algorithms

32

8

Graphical notation Let

Case 1

denote a subtree with a black root.

All

Recolor

CC DD

AA

x

’s have the same black-height.

y

CC

Recurse

new x DD

AA

BB

BB

(Or, A’s children are swapped.) Push C’s black onto A and D, and recurse, p[x] = left[p[p[x]] since C’s parent may be y = right[p[p[x]] red. color[y] = RED 3/5/09

CS 3343 Analysis of Algorithms

33

3/5/09

CS 3343 Analysis of Algorithms

Case 2 CC AA

x

BB

p[x] = left[p[p[x]] y = right[p[p[x]] color[y] = BLACK x = right[p[x]] 3/5/09

34

Case 3 LEFT-ROTATE(A)

CC

y

BB

x

CC

y x

AA

y = right[p[p[x]] color[y] = BLACK x = left[p[x]] 35

3/5/09

AA

AA

p[x] = left[p[p[x]]

Transform to Case 3.

CS 3343 Analysis of Algorithms

BB

RIGHT-ROTATE(C) y (and recolor)

BB CC

Done! No more violations of RB property 4 are possible. CS 3343 Analysis of Algorithms

36

9

Analysis

Pseudocode (part II) else 〈“then” clause with “left” and “right” swapped〉 ⊳ p[x] = right[p[p[x]] then y ← left[p[p[x]] ⊳ y = aunt/uncle of x if color[y] = RED then 〈Case 1’〉 else if x = left[p[x]] then 〈Case 2’〉 ⊳ Case 2’ falls into Case 3’ 〈Case 3’〉 color[root[T]] ← BLACK

• Go up the tree performing Case 1, which only recolors nodes. • If Case 2 or Case 3 occurs, perform 1 or 2 rotations, and terminate. Running time: O(log n) with O(1) rotations. RB-DELETE — same asymptotic running time and number of rotations as RB-INSERT (see textbook). 3/5/09

CS 3343 Analysis of Algorithms

37

3/5/09

Case 1’ y

Recolor AA

x

CC DD

BB

y

Recurse

new x x

y

BB

39

BB

p[x] = right[p[p[x]] y = left[p[p[x]] color[y] = BLACK x = left[p[x]] 3/5/09

CC

y

AA

x

color[y] = RED CS 3343 Analysis of Algorithms

RIGHT-ROTATE(A)

CC

AA

(Or, A’s children are swapped.) Push C’s black onto A and D, and recurse, p[x] = right[p[p[x]] since C’s parent may be y = left[p[p[x]] red. 3/5/09

38

Case 2’

CC DD

CS 3343 Analysis of Algorithms

BB

x AA

Transform to Case 3’.

CS 3343 Analysis of Algorithms

40

10

Case 3’ CC

y

BB

x

LEFT-ROTATE(C) (and recolor)

BB CC

AA

AA

p[x] = right[p[p[x]] y = left[p[p[x]] color[y] = BLACK x = right[p[x]] 3/5/09

Done! No more violations of RB property 4 are possible. CS 3343 Analysis of Algorithms

41

11