Data Structure

C. –C. Yao
 Algorithm
• Definition: An algorithm is a finite set of
instructions that, if followed, accomplishes a
particular task. In addition, all algorithms must
satisfy the following criteria:
1) Input. Zero more quantities are externally supplied.
2) Output. At least one quantity is produced.
3) Definiteness. Each instruction is clear and
unambiguous.
4) Finiteness. If we trace out the instructions of an
algorithm, then for all cases, the algorithm terminates
after a finite number of steps.
5) Effectiveness. Every instruction must be basic enough
to be carried out, in principle, by a person using only
pencil and paper. It is not enough that each operation
be definite as in 3) it also must be feasible.

C. –C. Yao
 Example: Selection Sort
• Suppose we must devise a program that sorts a
collection of n ≥ 1 integers.

From those integers that are currently unsorted, find

the smallest and place it next in the sorted list.

• Problem in the above statement

– Does not describe where and how the integers are
initially sorted.
– Does not indicate where to place the result.

C. –C. Yao
 C++ Program for Selection
Sort
void sort (int *a, const int n)
// sort the n integers a[0] to a[n-1] into non-decreasing order
for (int i = 0; i < n; i++)
{
int j = i;
// find smallest integer in a[i] to a[n-1]
for (int k = i + 1; k < n; k++)
if (a[k] < a[j]) j = k;
// interchange
int temp = a[i]; a[i] = a[j]; a[j] = temp;
}
}

C. –C. Yao
 Selection Sort (Cont.)
• Theorem 1.1: sort(a, n) correctly sorts
a set of n ≥ 1 integers; the result
remains in a[0] … a[n-1] such that a[0]
≤ a[1] ≤ … ≤ a[n–1].

C. –C. Yao
 Example:Binary Search
• Assume that we have n ≥ 1 distinct integers that are already
sorted and stored in the array a[0] … a[n-1]. Our task is to
determine if the integer x is present and if so to return j such
that x = a[j]; otherwise return -1. By making use of the fact that
the set is sorted, we conceive the following efficient method:
Let left and right, respectively, denote the left and right ends of
the list to be searched. Initially, left = 0 and right = n – 1. Let
middle = (left + right) / 2 be the middle position in the list. If we
compare a[middle] with x, we obtain one of the three results:
(1) x < a[middle]. In this case, if x is present, it must be in the
positions between 0 and middle – 1. Therefore, we set right to
middle – 1.
(2) x == a[middle]. In this case, we return middle.
(3) x > a[middle]. In this case, if x is present, it must be in the
positions between middle+1 and n-1. So, we set left to middle+1.

C. –C. Yao
 Algorithm for Binary
Search
int BinarySearch (int *a, const int x, const int n)
// Search the sorted array a[0], … , a[n-1] for x
{
for (initialize left and right; while there are more elements;)
{
let middle be the middle element;
switch (compare (x, a[middle])) {
case ‘>’: set left to middle+1; break;
case ‘<‘: set right to middle -1; break;
case ‘=‘: found x;
} // end of switch
} // end of for
not found;
} // end of BinarySearch

C. –C. Yao
 C++ Program for Binary
Search

char compare (int x, int y)

{
if (x > y) return ‘>’;
else if ( x < y) return ‘<‘;
else return ‘=‘;
} // end of compare

C. –C. Yao
 Algorithm for Binary
Search (Cont.)
int BinarySearch (int *a, const int x, const int n)
// Search the sorted array a[0], … , a[n-1] for x
{
for (int left = 0, right = n - 1; left <= right;)
{
int middle = (left + right) / 2;
switch (compare (x, a[middle])) {
case ‘>’: left = middle+1; break; // x > a[middle]
case ‘<‘: right = middle -1; break; // x < a[middle]
case ‘=‘: return middle; // x == a[middle]
} // end of switch
} // end of for
return -1;
} // end of BinarySearch

C. –C. Yao
 Recursive Algorithms
int BinarySearch (int *a, const int x, const int left, const int right)
{
if (left <= right)
{
int middle = (left + right) / 2;
if (x < a[middle]) return BinarySearch(a,x,left,middle-1);
else if (x < a[middle]) return BinarySearch(a,x,left,middle-1);
return middle;
} // end if
return -1;
} // end of BinarySearch

C. –C. Yao
Recursive Algorithms(cont.)
Recursive program:

int main()
{
int n=10;
printf(“%d”, rfib(n));
}
int rfib(int n)
{
if (n==1 || n==2) return 1;
return rfib(n1)+rfib(n2);
}

C. –C. Yao
 Performance Analysis
• Space Complexity: The space
complexity of a program is the
amount of memory it needs to run to
completion.
• Time Complexity: The time
complexity of a program is the
amount of computer time it needs to
run to completion.

C. –C. Yao
 Space Complexity
• A fixed part that is independent of the
characteristics of the inputs and outputs. This
part typically includes the instruction space,
space for simple varialbes and fixed-size
component variables, space for constants, etc.
• A variable part that consists of the space needed
by component variables whose size is dependent
on the particular problem instance being solved,
the space needed by referenced variables, and
the recursion stack space.
• The space requirement S(P) of any program P is
written as S(P) = c +Sp where c is a constant

C. –C. Yao
 Time Complexity
• The time, T(P), taken by a program P
is the sum of the compile time and
the run (or execution) time. The
compile time does not depend on the
instance characteristics. We focus on
the run time of a program, denoted
by tp (instance characteristics).

C. –C. Yao
 Time Complexity in C++
• General statements in a C++ program
Step count
– Comments 0
– Declarative statements 0
– Expressions and assignment statements 1
– Iteration statements N
– Switch statement N
– If-else statement N
– Function invocation 1 or N
– Memory management statements 1 or N
– Function statements 0
– Jump statements 1 or N

C. –C. Yao
 Time Complexity (Cont.)
• Note that a step count does not
necessarily reflect the complexity of
the statement.
• Step per execution (s/e): The s/e of
a statement is the amount by which
count changes as a result of the
execution of that statement.

C. –C. Yao
 Time Complexity Iterative
Example

float sum (float *a, const int n)

{
float s = 0;
for (int i = 0; i < n; i++)
s += a[i];
return;
}

C. –C. Yao
 Step Count of Iterative
Example
float sum (float *a, const int n)
{
float s = 0;
count++; // count is global
for (int i = 0; i < n; i++)
{
count++; // for for
s += a[i];
count++; // for assignment
}
count++; // for last time of for
count++; // for return
return;
}

C. –C. Yao
 Step Count of Iterative
Example (Simplified)
void sum (float *a, const int n)
{
for (int i = 0; i < n; i++)
count += 2;
count +=3;
}

If initially count = 0, then the total of steps

is 2n + 3.

C. –C. Yao
 Time Complexity of
Recursive Example

float rsum (float *a, const int n)

{
if (n <= 0) return 0;
else return (rsum(a, n–1) + a[n-1]);
}

C. –C. Yao
 Step Count of Recursive
Example
float rsum (float *a, const int n)
{
count++; // for if conditional
if (n <= 0) {
count++; // for return
return 0;
}
else {
count++; // for return
return (rsum(a, n–1) + a[n-1]);
}
}

Assume trsum(0) = 2
trsum(n) = 2 + trsum(n-1)
= 2 + 2 + trsum(n-2)
= 2*2 + trsum(n-2)
= 2n + trsum(0)
= 2n + 2

C. –C. Yao
 Matrix Addition Example
line void add (matrix a, matrix b, matrix c, int m, int
n)
1 {
2 for (int i = 0; i < m; i++)
3 for (int j = 0; j < n; j++)
4 c[i][j] = a[i][j] + b[i][j];
5 }

C. –C. Yao
 Step Count of Matrix
Addition Example
void add (matrix a, matrix b, matrix c, int m, int n)
{
for (int i = 0; i < m; i++)
{
count++; // for for i
for (int j = 0; j < n; j++)
{
count++; // for for j
c[i][j] = a[i][j] + b[i][j];
count++; // for assigment
}
count++; // for last time of for j
}
count++; // for last time of for i
}

C. –C. Yao
 Step Count of Matrix Addition
Example (Simplified)

line void add (matrix a, matrix b, matrix c, int m, int n)

{
for (int i = 0; i < m; i++)
{
for (int j = 0; j < n; j++)
count += 2;
count+2;
}
count++;
}

C. –C. Yao
 Step Table of Matrix
Addition Example

Line s/e Frequency Total steps

1 0 1 0
2 1 m+1 m+1
3 1 m(n+1) mn+m
4 1 mn mn
5 0 1 0
Total number of steps 2mn+2m+1

C. –C. Yao
 Fibonacci Numbers
• The Fibonacci sequence of numbers starts
as 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, …
Each new term is obtained by taking the
sum of the two previous terms. If we call
the first term of the sequence F0 then F0 =
0, F1 = 1, and in general
Fn = Fn-1 + Fn-2 , n ≥ 2.

C. –C. Yao
 C++ Program of Fibonacci
Numbers
1 void fibonacci (int n)
2 // compute the Fibonacci number F n
3 {
4 if (n <= 1) cout << n << endl; // F0 = 0, and F1 = 1
5 else { // compute Fn
6 int fn; int fnm2 = 0; int fnm1 = 1;
7 for (int i = 2; i <= n; i++)
8 {
9 fn = fnm1 + fnm2;
10 fnm2 = fnm1;
11 fnm1 = fn;
12 } // end of for
13 cout << fn << endl;
14 } // end of else
15 } // end of fibonacci

C. –C. Yao
 Step Count of Fibonacci
Program
• Two cases:
– n = 0 or n = 1
• Line 4 regarded as two lines: 4(a) and 4(b), total step
count in this case is 2
– n>1
• Line 4(a), 6, and 13 are each executed once
• Line 7 gets executed n times.
• Lines 8 – 12 get executed n-1 times each.
• Line 6 has s/e of 2 and the remaining lines have an
s/e of 1.
• Total steps for the case n > 1 is 4n + 1

C. –C. Yao
 Asymptotic Notation
• Determining step counts help us to compare the
time complexities of two programs and to predict
the growth in run time as the instance
characteristics change.
• But determining exact step counts could be very
difficult. Since the notion of a step count is itself
inexact, it may be worth the effort to compute
the exact step counts.
• Definition [Big “oh”]: f(n) = O(g(n)) iff there exist
positive constants c and n0 such that f(n) ≤ cg(n)
for all n, n ≥ n0

C. –C. Yao
 Examples of Asymptotic
Notation

• 3n + 2 = O(n)
3n + 2 ≤ 4n for all n ≥ 3
• 100n + 6 = O(n)
100n + 6 ≤ 101n for all n ≥ 10
• 10n2 + 4n + 2 = O(n2)
10n2 + 4n + 2 ≤ 11n2 for all n ≥ 5

C. –C. Yao
 Asymptotic Notation (Cont.)
Theorem 1.2: If f(n) = amnm + … + a1n + a0, then f(n)
= O(nm).
Proof:
m
f ( n )≤∑ ∣ai∣ni
i= 0
m
n m ∑ ∣ai∣ni −m
0
m for n ≥ 1
n
m
∑ ∣ai∣
0

So, f(n) = O(nm)

C. –C. Yao
 Asymptotic Notation (Cont.)

• Definition: [Omega] f(n) = Ω(g(n)) iff

there exist positive constants c and
n0 such that f(n) ≥ cg(n) for all n, n ≥
n0.
• Example:
– 3n + 2 = Ω(n)
– 100n + 6 = Ω(n)
– 10n2 + 4n + 2 =Ω(n2)

C. –C. Yao
 Asymptotic Notation (Cont.)

Definition: f(n) = Θ(g(n)) iff there exist

positive constants c1, c2, and n0 such
that c1g(n) ≤ f(n) ≤ c2g(n) for all n, n ≥ n0.

C. –C. Yao
 Asymptotic Notation (Cont.)

Theorem 1.3: If f(n) = amnm + … + a1n +

a0 and am > 0, then f(n) = Ω(nm).

Theorem 1.4: If f(n) = amnm + … + a1n +

a0 and am > 0, then f(n) = Θ(nm).

C. –C. Yao
 Practical Complexities
• If a program P has complexities Θ(n)
and program Q has complexities
Θ(n2), then, in general, we can assume
program P is faster than Q for a
sufficient large n.
• However, caution needs to be used on
the assertion of “sufficiently large”.

C. –C. Yao
 Function Values
log n n n log n n2 n3 2n
0 1 0 1 1 2

1 2 2 4 8 4

2 4 8 16 64 16

3 8 24 64 512 256

4 16 64 256 4096 65536

5 32 160 1024 32768 4294967296

C. –C. Yao
Chap 2

Array
 ADT
• In C++, class consists four
components:
– class name
– data members : the data that makes up
the class
– member functions : the set of
operations that may be applied to the
objects of class
– levels of program access : public,
protected and private.
 Definition of Rectangle
• #ifndef RECTANGLE_H
• #define RECTANGLE_H
• Class Rectangle{
• Public:
• Rectangle();
• ~Rectangle();
• int GetHeight();
• int GetWidth();
• Private:
• int xLow, yLow, height, width;
• };
• #endif
 Array
char A[3][4]; // row-major
logical structure physical structure
0 1 2 3
A[0][0]
0 A[0][1]
1 A[0][2]
2 A[0][3]
A[1][0]
A[2][1] A[1][1]
A[1][2]
A[1][3]
Mapping:
A[i][j]=A[0][0]+i*4+j
 Array
The Operations on 1-dim Array
– Store
• 將新值寫入陣列中某個位置
• A[3] = 45
0 1 2 3
A 45 ...

• O(1)
 Polynomial
Representation 1
private:
Representations
int degree; // degree ≤ MaxDegree
float coef [MaxDegree + 1];

Representation 2
private:
int degree;
float *coef;

Polynomial::Polynomial(int d)
{
degree = d;
coef = new float [degree+1];
}

Representation 3
class Polynomial; // forward delcaration

class term {
friend Polynomial;
private:
float coef; // coefficient
int exp; // exponent
};

private:
static term termArray[MaxTerms];
static int free;
int Start, Finish;

term Polynomial:: termArray[MaxTerms];

Int Polynomial::free = 0; // location of next free location in temArray
 Figure 2.1 Array Representation of two
Polynomials
Represent the following two polynomials:
A(x) = 2x1000 + 1
B(x) = x4 + 10x3 + 3x2 + 1

A.Start A.Finish B.Start B.Finish free

coef 2 1 1 10 3 1

exp 1000 0 4 3 2 0

0 1 2 3 4 5 6
 Polynomial Addition
只存非零值 (non-zero) ：一元多項式
– Add the following two polynomials:
A(x) = 2x1000 + 1
B(x) = x4 + 10x3 + 3x2 + 1

0 1 2 0 1 2 3 4
A_coef 2 2 1 B_coef 4 1 10 3 1

A_exp 1000 0 B_exp 4 3 2 0

0 1 2 3 4 5
C_coef 5 2 1 10 3 2

C_exp 1000 4 3 2 0
 Polynomial Addition
Polynomial Polynomial:: Add(Polynomial B)
// return the sum of A(x) ( in *this) and B(x)
{
Polynomial C; int a = Start; int b = B.Start; C.Start = free; float c;
while ((a <= Finish) && (b <= B.Finish))
switch (compare(termArray[a].exp, termArray[b].exp)) {
case ‘=‘:
c = termArray[a].coef +termArray[b].coef;
if ( c ) NewTerm(c, termArray[a].exp);
a++; b++;
break;
case ‘<‘:
NewTerm(termArray[b].coef, termArray[b].exp);
b++;
case ‘>’:
NewTerm(termArray[a].coef, termArray[a].exp);
a++;
} // end of switch and while
// add in remaining terms of A(x)
for (; a<= Finish; a++)
NewTerm(termArray[a].coef, termArray[a].exp);
O(m+n)
// add in remaining terms of B(x)
for (; b<= B.Finish; b++)
NewTerm(termArray[b].coef, termArray[b].exp);
C.Finish = free – 1;
return C;
} // end of Add
 Program 2.9 Adding a new
Term

void Polynomial::NewTerm(float c, int e)

// Add a new term to C(x)
{
if (free >= MaxTerms) {
cerr << “Too many terms in polynomials”<< endl;
exit();
}
termArray[free].coef = c;
termArray[free].exp = e;
free++;
} // end of NewTerm
 Disadvantages of Representing
Polynomials by Arrays

• What should we do when free is going

to exceed MaxTerms?
– Either quit or reused the space of
unused polynomials. But costly.
• If use a single array of terms for
each polynomial, it may alleviate the
above issue but it penalizes the
performance of the program due to
the need of knowing the size of a
polynomial beforehand.
 Sparse Matrices

[ ]
15 0 0 22 0 −15

[ ]
−27 3 4 0 11 3 0 0 0
6 82 −2 0 0 0 −6 0 0
109 −64 11 0 0 0 0 0 0
12 8 9 91 0 0 0 0 0
48 27 47 0 0 28 0 0 0
 Sparse Matrix Representation

• Use triple <row, column, value>

• Store triples row by row
• For all triples within a row, their
column indices are in ascending order.
• Must know the number of rows and
columns and the number of nonzero
elements
 Sparse Matrix Representation
(Cont.)

class SparseMatrix; // forward declaration

class MatrixTerm {
friend class SparseMatrix
private:
int row, col, value;
};

In class SparseMatrix:
private:
int Rows, Cols, Terms;
MatrixTerm smArray[MaxTerms];
 Transposing A Matrix
• Intuitive way:
for (each row i)
take element (i, j, value) and store it in (j, i, value)
of the transpose
• More efficient way:
for (all elements in column j)
place element (i, j, value) in position (j, i, value)
 Transposing A Matrix
• The Operations on 2-dim Array
– Transpose

[ ] [ ]
1 2 3
1 4 7 10
4 5 6
2 5 8 11
7 8 9
3 6 9 12 3 ×4
10 11 12 4×3

for (i = 0; i < rows; i++ )

for (j = 0; j < columns; j++ )
B[j][i] = A[i][j];
 Program 2.10 Transposing a
Matrix

SparseMatrix SparseMatrix::Transpose()
// return the transpose of a (*this)
{
SparseMatrix b;
b.Rows = Cols; // rows in b = columns in a
b.Cols = Rows; // columns in b = rows in a
b.Terms = Terms; // terms in b = terms in a
if (Terms > 0) // nonzero matrix
{
int CurrentB = 0;
for (int c = 0; c < Cols; c++) // transpose by columns
for (int i = 0; i < Terms; i++)
// find elements in column c
if (smArray[i].col == c) {
b.smArray[CurrentB].row = c;
b.smArray[CurrentB].col = smArray[i].row;
b.smArray[CurrentB].value = smArray[i].value;
CurrentB++; O(terms*columns)
}
} // end of if (Terms > 0)
} // end of transpose
 Fast Matrix Transpose
• The O(terms*columns) time =>
O(rows*columns2) when terms is the order
of rows*columns
• A better transpose function in Program
2.11. It first computes how many terms in
each columns of matrix a before
transposing to matrix b. Then it
determines where is the starting point of
each row for matrix b. Finally it moves
each term from a to b.
 Program 2.11 Fast Matrix
Transposing

SparseMatrix SparseMatrix::Transpose()
// The transpose of a(*this) is placed in b and is found in Q(terms +
columns) time.
{
int *Rows = new int[Cols];
int *RowStart = new int[Rows];
SparseMatrix b;
b.Rows = Cols; b.Cols = Rows; b.Terms = Terms;
if (Terms > 0) // nonzero matrix O(columns)
{ O(terms)
// compute RowSize[i] = number of terms in row i of b
for (int i = 0; I < Cols; i++) RowSize[i] = 0; // Initialize
O(columns-1)
for ( I = 0; I < Terms; I++) RowSize[smArray[i].col]++;

// RowStart[i] = starting position of row i in b

RowStart[0] = 0;
for (i = 0; i < Cols; i++) RowStart[i] = RowStart[i-1] + RowSize[i-1];
 Program 2.11 Fast Matrix
Transposing (Cont.)

for (i = 1; I < Terms; i++) // move from a to b

{
int j = RowStart[smArray[i].col];
b.smArray[j].row = smArray[i].col; O(terms)
b.smArray[j].col = smArray[i].row;
b.smArray[j].value = smArray[i].value;
RowStart[smArray[i].col]++;
} // end of for
} // end of if

delete [] RowSize;
delete [] RowStart; O(row * column)
return b;
} // end of FastTranspose
 Matrix Multiplication
• Definition: Given A and B, where A is mxn
and B is nxp, the product matrix Result has
dimension mxp. Its [i][j] element is
n−1
result ij= ∑ a ij bkj
k=0

for 0 ≤ i < m and 0 ≤ j < p.

 Representation of Arrays
• Multidimensional arrays are usually
implemented by one dimensional array
via either row major order or column
major order.
• Example: One dimensional array
α α+1 α+2 α+3 α+4 α+5 α+6 α+7 α+8 α+9 α+10 α+12

A[0] A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8] A[9] A[10] A[11
]
 Two Dimensional Array Row
Major Order

Col 0 Col 1 Col 2 Col u2 - 1

Row 0 X X X X

Row 1 X X X X

Row u1 - 1 X X X X

u2 u2
elements elements

Row 0 Row 1 Row i Row u1 - 1

i * u2 element
 Memory mapping
• char A[3][4]; // row-major
• logical structure physical structure
0 1 2 3
A[0][0]
0 A[0][1]
1 A[0][2]
2 A[0][3]
A[1][0]
A[2][1] A[1][1]
A[1][2]
A[1][3]
Mapping:
A[i][j]=A[0][0]+i*4+j
The address of elements in 2-dim
array
– 二維陣列宣告為 A[r][c] ，每個陣列元素佔
len bytes ，且其元第 1 個元素 A[0][0] 的記
憶體位址為，並以列為主 (row major) 來
儲存此陣列，則
• A[0][3] 位址為 +3*len
• A[3][0] 位址為 +(3*r+0)*len
• ...
• A[m][n] 位址為 +(m* r + n)*len
– A[0][0] 位址為，目標元素 A[m][n] 位址的
算法
• Loc(A[m][n]) = Loc(A[0][0]) + [(m0)*r+
(n0)]* 元素大小
 String
• Usually string is represented as a
character array.
• General string operations include
comparison, string concatenation,
copy, insertion, string matching,
printing, etc.

H e l l o W o r l d \0
 String Matching The Knuth-
Morris-Pratt Algorithm
• Definition: If p = p0p1…pn-1 is a pattern,
then its failure function, f, is defined as

f ( j )=
{
largest k<j such that p 0 p1 . .. p k =p j −k p j− k+1 . . . p j
−1
if such a k ≥ 0
otherwise .
exists
}
• If a partial match is found such that si-j …
si-1 = p0p1…pj-1 and si ≠ pj then matching may
be resumed by comparing si and pf(j–1)+1 if j ≠
0. If j = 0, then we may continue by
comparing si+1 and p0.
 Fast Matching Example
Suppose exists a string s and a pattern pat =
‘abcabcacab’, let’s try to match pattern pat in
string s.

j 0 1 2 3 4 5 6 7 8 9
pat a b c a b c a c a b
f -1 -1 -1 0 1 2 3 -1 0 1

s= ‘- a b c a ?j = 4,?pf(j-1)+1
. =.p1 . ?’
pat = ‘a b c a b c a c a b’

New start matching point

‘a b c a b c a c a b’
 Chap 3

Stack and Queue

 Templates in C++
• Template function in C++ makes it
easier to reuse classes and functions.
• A template can be viewed as a
variable that can be instantiated to
any data type, irrespective of
whether this data type is a
fundamental C++ type or a user-
defined type.
 Selection Sort Template
Template <class KeyType>
void sort(KeyType *a, int n)
// sort the n KeyTypes a[0] to a[n-1] into nondecreasing order
{
for (int i = 0; i < n; i++)
{
int j = i;
// find smallest KeyType in a[i] to a[n-1]
for (int k = i+1; k < n; k++)
if (a[k] < a[j]) { j = k;}
// interchange
KeyType temp = a[i]; a[i] = a[j]; a[j] = temp;
}
}

float farray[100];
int intarray[200];
………..
sort(farray, 100);
sort(intarray, 200);
 Template (Cont.)
• Can we use the sort template for the
Rectangle class?
• Well, not directly. We’ll need to use
operator overloading to implement “>”
for Rectangle class.
 Stack
• What is a stack? A stack is an
ordered list in which insertions and
deletions are made at one end called
the top. It is also called a Last-In-
First-Out (LIFO) list.
 Stack (Cont.)
• Given a stack S = (a0, …, an-1), a0 is the bottom
element, an-1 is the top element, and ai is on top of
element ai-1, 0 < i < n.

a3
a3

a2 a2
a1 a1
a0 a0

Push (Add) Pop (Delete)

 System Stack
• Whenever a function is invoked, the program
creates a structure, referred to as an activation
record or a stack frame, and places it on top of
the system stack.

previous frame pointer fp

return address a1

local variables

previous frame pointer fp previous frame pointer

return address main return address main
ADT 3.1 Abstract Data Type
Stack
Template <class KeyType>
class Stack
{ // objects: A finite ordered list with zero or more elements
public:
Stack (int MaxStackSize = DefaultSize);
// Create an empty stack whose maximum size is MaxStackSize
Boolean IsFull();
// if number of elements in the stack is equal to the maximum size
// of the stack, return TRUE(1) else return FALSE(0)
void Add(const KeyType& item);
// if IsFull(), then StackFull(); else insert item into the top of the stack.
Boolean IsEmpty();
// if number of elements in the stack is 0, return TRUE(1) else return FALSE(0)
KeyType* Delete(KeyType& );
// if IsEmpty(), then StackEmpty() and return 0;
// else remove and return a pointer to the top element of the stack.
};
 Implementation of Stack by
Array

an-1

a2
a1
a0

a0 a1 a2 an-1
Array index 0 1 2 3 n-1
 Queue
• A queue is an ordered list in which all
insertions take place at one end and
all deletions take place at the
opposite end. It is also known as
First-In-First-Out (FIFO) lists.

a0 a1 a2 an-1

front rear
 ADT 3.2 Abstract Data Type Queue
Template <class KeyType>
class Queue
{
// objects: A finite ordered list with zero or more elements
public:
Queue(int MaxQueueSize = DefaultSize);
// Create an empty queue whose maximum size is MaxQueueSize
Boolean IsFull();
// if number of elements in the queue is equal to the maximum size of
// the queue, return TRUE(1); otherwise, return FALSE(0)
void Add(const KeyType& item);
// if IsFull(), then QueueFull(); else insert item at rear of the queue
Boolean IsEmpty();
// if number of elements in the queue is equal to 0, return TRUE(1)
// else return FALSE(0)
KeyType* Delete(KeyType&);
// if IsEmpty(), then QueueEmpty() and return 0;
// else remove the item at the front of the queue and return a pointer to it
};
 Queue Manipulation Issue
• It’s intuitive to use array for
implementing a queue. However,
queue manipulations (add and/or
delete) will require elements in the
array to move. In the worse case, the
complexity is of O(MaxSize).
Shifting Elements in Queue

front rear

front rear

front rear
 Circular Queue
• To resolve the issue of moving elements in
the queue, circular queue assigns next
element to q[0] when rear == MaxSize – 1.
• Pointer front will always point one position
counterclockwise from the first element in
the queue.
• Queue is empty when front == rear. But it
is also true when queue is full. This will be
a problem.
 Circular Queue (Cont.)

4 J4 4
J3 n-4 n-4
3 3 J1
J2
J1
n-3 J2 n-3
2 2 J4 J3
n-2 n-2
1 1
0 n-1 0 n-1

front = 0; rear = 4 front = n-4; rear = 0

 Circular Queue (Cont.)
• To resolve the issue when front == rear on whether
the queue is full or empty, one way is to use only
MaxSize – 1 elements in the queue at any time.
• Each time when adding an item to the queue,
newrear is calculated before adding the item. If
newrear == front, then the queue is full.
• Another way to resolve the issue is using a flag to
keep track of last operation. The drawback of the
method is it tends to slow down Add and Delete
function.
Subtyping and Inheritance in C++
• Inheritance is used to express
subtype relationships between two
ADTs.
• If B inherits from A, then B IS-A A.
Also, A is more general than B.
– VW Beetle IS-A Car; Eagle IS-A Bird
 Inheritance
• A derived class inherits all the non-private
members (data and functions) of the base class.
• Inherited members from public inheritance have
the same level of access in the derived class as
they did in the base class.
• The derived class can reuse the implementation of
a function in the base class or implement its own
function, with the exception of constructor and
destructor.
 Class Inheritance Example
class Bag
{ public:
Bag (int MaxSize = DefaultSize); // constructor
virtual ~Bag(); // destructor
virtual void Add(int); // insert element into bag
virtual int* Delete (int&); //delete element from bag
virtual Boolean IsFull(); // return TRUE if the bag is full; FALSE otherwise
virtual Boolean IsEmpty(); // return TRUE if the bag is empty; FALSE otherwise
protected:
virtual void Full(); // action when bag is full
virtual void Empty(); // action when bag is empty
int *array;
int MaxSize; // size of array
int top; // highest position in array that contains an element
}
 Class Inheritance
Example(Cont.)
class Stack : public Bag
{
public:
Stack(int MaxSize = DefaultSize); // constructor
~Stack(); // destructor
int* Delete(int&); // delete element from stack
};
Stack:: Stack (int MaxStackSize) : Bag(MaxStackSize) { }
// Constructor for Stack calls constructor for Bag

Stack::~Stack() { }
// Destructor for Bag is automatically called when Stack is destroyed. This ensures that array is deleted.

int* Stack::Delete(int& x)
{ if (IsEmpty()) {Empty(); return 0; }
x = array[top--];
return &x;
}
 Class Inheritance Example
Bag b(3); // uses Bag constructor to create array of size 3
Stack s(3); // uses Stack constructor to create array of size 3

b.Add(1); b.Add(2); b.Add(3);

// use Bag::Add. Bag::Add calls functions Bag::IsFull and Bag::Full

s.Add(1); s.Add(2); s.Add(3);

// Stack::Add not defined, so use Bag::Add. Bag::Add calls Bag::IsFull
// and Bag::Full because these have not been redefined in Stack

int x;
b.Delete(x); // uses Bag::Delete, which calls Bag::IsEmpty and Bag::Emtpy
s.Delete(x);
// uses Stack::Delete, which calls Bag::IsEmtpy and Bag::Emtpy because these
// have not been redefined in Stack.
 The Maze Problem

[ ]
Entrance 0 1 0 0 0 1 1 0 0 0 1 1 1 1 1
1 0 0 0 1 1 0 1 1 1 0 0 1 1 1
0 1 1 0 0 0 0 1 1 1 1 0 0 1 1
1 1 0 1 1 1 1 0 1 1 0 1 1 0 0
1 1 0 1 0 0 1 0 1 1 1 1 1 1 1
0 0 1 1 0 1 1 1 0 1 0 0 1 0 1
0 0 1 1 0 1 1 1 0 1 0 0 1 0 1
0 1 1 1 1 0 0 1 1 1 1 1 1 1 1
0 0 1 1 0 1 1 0 1 1 1 1 1 0 1
1 1 0 0 0 1 1 0 1 1 0 0 0 0 0
0 0 1 1 1 1 1 0 0 0 1 1 1 1 0
0 1 0 0 1 1 1 1 1 0 1 1 1 1 0
Exit
 Allowable Moves

N
[i-1][j-1] [i-1][j] [i-1][j+1]
NW NE

W [i][j-1] X [i][j+1] E
[i][j]

[i+1][j-1] [i+1][j] [i+1]

SW [j+1]
S SE
 The Maze Problem (Cont.)
• Stack is used in solving the maze problem
for storing the coordinates and direction.
• Use of another mxp array to mark any
position that has been visited before.
 Evaluation Expression in C++

• When evaluating Priority Operator

operations of the 1 Unary minus, !

same priorities, it 2 *, /, %

follows the 3 +, -
direction from left 4 <, <=, >=, >
to right. 5 ==, !=

6 &&

7 ||
 Postfix Notation
Expressions are converted into Postfix notation before
compiler can accept and process them.

X = A/B – C + D * E – A * C
Infix A/B–C+D*E–A*C
Postfix => AB/C-DE*+AC*-

Operation Postfix
T1 = A / B T1C-DE*+AC*-
T2 = T1 - C T2 DE*+AC*-
T3 = D * E T2T3+AC*-
T4 = T2 + T3 T4AC*-
T5 = A * C T4 T5 -
T6 = T4 - T5 T6
Postfix notation generation
X = A+ B* C / D – E * F 後置式 : ABC*D/+EF*-
輸出 A A AB AB ABC ABC* ABC* ABC*D

堆疊

* * / /
+ + + + + + +

輸出 ABC*D/ ABC*D/+ ABC*D/+E ABC*D/+E ABC*D/+EF

堆疊
* *

+ - - - -
Postfix notation generation
X = A+ (B+C*D) – E 後置式 : ABCD*++E–
輸出 A A A AB AB ABC ABC ABCD ABCD*
* *
堆疊
+ + + + +
( ( ( ( ( ( (
+ + + + + + + +

輸出 ABCD*+ ABCD*+ ABCD*++ ABCD*++E ABCD*++E-

堆疊
(
+ + - -
Postfix notation execution
後置式 : ABC*D/+EF*-

堆疊 C D
B B B*C B*C B*C/D
A A A A A A A+B*C/D

F
E E E*F
A+B*C/D A+B*C/D A+B*C/D A+B*C/D-E*F
 Multiple Stacks and Queues

• Two stacks case:

0 1 2 3 4 m-4 m-3 m-2 m-1
⌊ m/n ⌋ −1

Stack A Stack B
• More than two stacks case
0 ⌊ m/n ⌋ −1 2 ⌊ m / n ⌋ −1 m-1

b[0 b[1] b[2 b[n

] ] ]
t[0 t[1] t[2 t[n]
] ]
 Chap 4

Linked Lists
 Review of Sequential Representations
• Previously introduced data structures, including
array, queue, and stack, they all have the property
that successive nodes of data object were stored
a fixed distance apart.
• The drawback of sequential mapping for ordered
lists is that operations such as insertion and
deletion become expensive.
• Also sequential representation tends to have less
space efficiency when handling multiple various
sizes of ordered lists.
 Linked List
• A better solutions to resolve the aforementioned
issues of sequential representations is linked lists.
• Elements in a linked list are not stored in
sequential in memory. Instead, they are stored all
over the memory. They form a list by recording
the address of next element for each element in
the list. Therefore, the list is linked together.
• A linked list has a head pointer that points to the
first element of the list.
• By following the links, you can traverse the linked
list and visit each element in the list one by one.
 Linked List Insertion
• To insert an element into the three
letter linked list:
– Get a node that is currently unused; let
its address be x.
– Set the data field of this node to GAT.
– Set the link field of x to point to the
node after FAT, which contains HAT.
– Set the link field of the node cotaining
FAT to x.
 Linked List Insertion And Deletion

first

BAT CAT EAT FAT HAT

GAT

first

BAT CAT EAT FAT GAT HAT

 Designing a List in C++
• Design Attempt 1: Use a global variable first
which is a pointer of ThreeLetterNode.
– Unable to access to private data members: data and link.
• Design Attempt 2: Make member functions public.
– Defeat the purpose of data encapsulation.
• Design Attempt 3: Use of two classes. Create a
class that represents the linked list. The class
contains the items of another objects of another
class.
 Program 4.1 Composite
Classes
class ThreeLetterList; // forward delcarion
class ThreeLetterNode {
friend class ThreeLetterList;
private:
char data[3];
ThreeLetterNode * link;
};

class ThreeLetterList {
public:
// List Manipulation operations
.
.
private:
ThreeLetterNode *first;
};
 Nested Classes
• The Three Letter List problem can also use nested classes to
represent its structure.

class ThreeLetterList {
public:
// List Manipulation operations
.
.
private:
class ThreeLetterNode { // nested class
public:
char data[3];
ThreeLetterNode *link;
};
ThreeLetterNode *first;
};
 Pointer Manipulation in C++
• Addition of integers to pointer variable is
permitted in C++ but sometimes it has no logical
meaning.
• Two pointer variables of the same type can be
compared.
– x == y, x != y, x == 0

x a a x b

y b x b y b
y

x=y *x = * y
 Define A Linked List
Template
• A linked list is a container class, so its
implementation is a good template candidate.
• Member functions of a linked list should be
general that can be applied to all types of
objects.
• When some operations are missing in the original
linked list definition, users should not be forced
to add these into the original class design.
• Users should be shielded from the detailed
implementation of a linked list while be able to
traverse the linked list.
Solution => Use of ListIterator
 List Iterator
• A list iterator is an object that is used to traverse all
the elements of a container class.
• ListIterator<Type> is delcared as a friend of both
List<Type> and ListNode<Type>.
• A ListIterator<Type> object is initialized with the
name of a List<Type> object l with which it will be
associated.
• The ListIterator<Type> object contains a private data
member current of type ListNode<Type> *. At all
times, current points to a node of list l.
• The ListIterator<Type> object defines public member
functions NotNull(), NextNotNull(), First(), and Next()
to perform various tests on and to retrieve elements
of l.
 Template of Linked Lists
Enum Boolean { FALSE, TRUE};
template <class Type> class List;
template <class Type> class ListIterator;

template <class Type> class ListNode {

friend class List<Type>; friend class ListIterator <Type>;
private:
Type data;
ListNode *link;
};

Template <class Type> class List {

friend class ListIterator <Type>;
public:
List() {first = 0;};
// List manipulation operations
.
.
private:
ListNode <Type> *first;
};
 Template of Linked Lists
(Cont.)
template <class Type> class ListIterator {
public:
ListIterator(const List<Type> &l): list(l), current(l.first) {};
Boolean NotNull();
Boolean NextNotNull();
Type * First();
Type * Next();
Private:
const List<Type>& list; // refers to an existing list
ListNode<Type>* current; // points to a node in list
};
Attaching A Node To The End Of A List
Template <class Type>
Void List<Type>::InsertBack(Type k)
{
ListNode<Type>*newnode = new ListNode<Type>(k);
if (first == 0) first = last =newnode;
else {
last->link = newnode;
last = newnode;
}
};
 Concatenating Two Chains
Template <class Type>
void List<Type>:: Concatenate(List<Type> b)
// this = (a1, …, am) and b = (b1, …, bn) m, n ≥ ,
// produces the new chain z = (a1, …, am, b1, bn) in this.
{
if (!first) { first = b.first; return;}
if (b.first) {
for (ListNode<Type> *p = first; p->link; p = p->link); // no body
p->link = b.first;
}
}
 Reversing a list
Template <class Type>
void List<Type>:: Reverse( )
// List is reversed so that (a1, …, am) becomes (am, …, a1) .
{
ListNode <Type> *current = first;
*previous = 0; //previous trails current
while (current) {
ListNode <Type> *r = previous;
previous = current;
current = current -> link;
previous->link = r;
}
first = previous;
}
 When Not To Reuse A Class
• If efficiency becomes a problem
when reuse one class to implement
another class.
• If the operations required by the
application are complex and
specialized, and therefore not
offered by the class.
 Circular Lists
• By having the link of the last node points
to the first node, we have a circular list.
– Need to make sure when current is pointing to
the last node by checking for current->link ==
first.
– Insertion and deletion must make sure that the
circular structure is not broken, especially the
link between last node and first node.
Diagram of A Circular List

first

last
 Linked Stacks and Queues

top

front rear

Linked Queue
0

Linked Stack
 Revisit Polynomials

a.first 3 14 2 8 1 0 0

a= 3 x 14 + 2x 8 + 1

b.first 8 14 -3 10 10 6 0

b= 8 x 14 − 3x 10 + 10 x 6
 Polynomial Class Definition
struct Term
// all members of Terms are public by default
{
int coef; // coefficient
int exp; // exponent
void Init(int c, int e) {coef = c; exp = e;};
};

class Polynomial
{
friend Polynomial operator+(const Polynomial&, const Polynomial&);
private:
List<Term> poly;
};
 Operating On Polynomials
• With linked lists, it is much easier to
perform operations on polynomials
such as adding and deleting.
– E.g., adding two polynomials a and b

a.first 3 14 2 8 1 0 0

b.first 8 14 -3 10 10 6 0

q
(i) p->exp == q->exp
c.first 11 14 0
 Operating On Polynomials

a.first 3 14 2 8 1 0 0

b.first 8 14 -3 10 10 6 0

c.first 11 14 0 -3 10 0

(ii) p->exp < q->exp

 Operating On Polynomials

a.first 3 14 2 8 1 0 0

b.first 8 14 -3 10 10 6 0

q
c.first 11 14 0 -3 10 2 8 0

(iii) p->exp > q->exp

 Memory Leak
• When polynomials are created for computation and
then later on out of the program scope, all the memory
occupied by these polynomials is supposed to return to
system. But that is not the case. Since
ListNode<Term> objects are not physically contained in
List<Term> objects, the memory they occupy is lost to
the program and is not returned to the system. This is
called memory leak.
• Memory leak will eventually occupy all system memory
and causes system to crash.
• To handle the memory leak problem, a destructor is
needed to properly recycle the memory and return it
back to the system.
 List Destructor
Template <class Type>
List<Type>::~List()
// Free all nodes in the chain
{
ListNode<Type>* next;
for (; first; first = next) {
next = first->link;
delete first;
}
}
 Free Pool
• When items are created and deleted
constantly, it is more efficient to have a
circular list to contain all available items.
• When an item is needed, the free pool is
checked to see if there is any item
available. If yes, then an item is retrieved
and assigned for use.
• If the list is empty, then either we stop
allocating new items or use new to create
more items for use.
Using Circular Lists For Polynomials
• By using circular lists for polynomials
and free pool mechanism, the
deleting of a polynomial can be done
in a fixed amount of time
independent of the number of terms
in the polynomial.
 Deleting A Polynomial with a Circular
List Structure

av

3
a.first 3 14 2 8 1 0

1
2 second

av
 Equivalence Class
• For any polygon x, x ≡ x. Thus, ≡ is
reflexive.
• For any two polygons x and y, if x ≡ y,
then y ≡ x. Thus, the relation ≡ is
symetric.
• For any three polygons x, y, and z, if
x ≡ y and y ≡ z, then x ≡ z. The
relation ≡ is transitive.
 Equivalence
Definition: A relation ≡ over a set S, is said
to be an equivalence relation over S iff it
is symmetric, reflexive, and transitive over
S.
Example: Supposed 12 polygons 0 ≡ 4, 3 ≡ 1,
6 ≡ 10, 8 ≡ 9, 7 ≡ 4, 6 ≡ 8, 3 ≡ 5, 2 ≡ 11, and
11 ≡ 0. Then they are partitioned into
three equivalence classes:
{0, 2, 4, 7, 11}; {1 , 3, 5}; {6, 8, 9 , 10}
 Equivalence (Cont.)
• Two phases to determine equivalence
– In the first phase the equivalence pairs
(i, j) are read in and stored.
– In phase two, we begin at 0 and find all
pairs of the form (0, j). Continue until
the entire equivalence class containing 0
has been found, marked, and printed.
• Next find another object not yet
output, and repeat the above process.
 Equivalence Classes (Cont.)
• If a Boolean array pairs[n][n] is used
to hold the input pairs, then it might
waste a lot of space and its
initialization requires complexity
Θ(n2) .
• The use of linked list is more
efficient on the memory usage and
has less complexity, Θ(m+n) .
Linked List Representation

[0 [1] [2 [3 [4 [5 [6 [7 [8 [9 [10][11]
] ] ] ] ] ] ] ] ]

data 11 3 11 5 7 3 8 4 6 8 6 0
link 0 0 0 0 0 0

data 4 1 0 10 9 2
link 0 0 0 0 0 0
 Linked List for Sparse
Matrix
• Sequential representation of sparse matrix suffered from
the same inadequacies as the similar representation of
Polynomial.
• Circular linked list representation of a sparse matrix has
two types of nodes:
– head node: tag, down, right, and next
– entry node: tag, down, row, col, right, value
• Head node i is the head node for both row i and column i.
• Each head node is belonged to three lists: a row list, a
column list, and a head node list.
• For an n x m sparse matrix with r nonzero terms, the
number of nodes needed is max{n, m} + r + 1.
Node Structure for Sparse Matrices

down tag right down tag row col right f i j

next value

Head node Typical node Setup for aij

[ ]
0 0 11 0
12 0 0 0 A 4x4 sparse
0 −4 0 0 matrix
0 0 0 −15
 Linked Representation of A Sparse
Matrix

H0 H1 H2 H3
Matrix
4 4
head

0 2
H0
11

1 0
H1
12

2 1
H2
-4

3 3
H3
-15
 Reading In A Sparse Matrix
• Assume the first line consists of the
number of rows, the number of columns,
and the number of nonzero terms. Then
followed by num-terms lines of input, each
of which is of the form: row, column, and
value.
• Initially, the next field of head node i is to
keep track of the last node in column i.
Then the column field of head nodes are
linked together after all nodes has been
read in.
 Complexity Analysis
• Input complexity: O(max{n, m} + r) = O(n +
m + r)
• Complexity of ~Maxtrix(): Since each node
is in only one row list, it is sufficient to
return all the row lists of a matrix. Each
row is circularly linked, so they can be
erased in a constant amount of time. The
complexity is O(m+n).
 Doubly Linked Lists
• The problem of a singly linked list is that
supposed we want to find the node precedes a
node ptr, we have to start from the beginning of
the list and search until find the node whose link
field contains ptr.
• To efficiently delete a node, we need to know its
preceding node. Therefore, doubly linked list is
useful.
• A node in a doubly linked list has at least three
fields: left link field (llink), a data field (item),
and a right link field (rlink).
 Doubly Linked List
• A head node is also used in a doubly
linked list to allow us to implement
our operations more easily.

llink item rlink

Head Node

llink item rlink

Empty List
Deletion From A Doubly Linked Circular
List

llink item rlink

Head Node

x -> left -> right = x -> right

x -> right -> left = x -> left
Insertion Into An Empty Doubly Linked
Circular List
node node

x x

newnode p

p -> left = x; p -> right = x -> right;

x -> right -> left = p; x -> right = p;
 Generalized Lists
• Definition: A generalized list, A, is a finite
sequence of n ≥ 0 elements, α0, α1, α2, …, αn-1, where
αi, is either an atom or a list. The elements αi,0≤ i
≤ n – 1, that are not atoms are said to be the
sublists of A.
• A list A is written as A = (α0, …, αn-1 ), and the
length of the list is n.
• Conventionally, a capital letter is used to
represent a list and a lower case letter is to
represent an atom.
• The α0 is the head of list A and the rest (α1, …, αn-
1) is the tail of list A.
 Generalized List Examples
• D = ( ): the null, or empty, list; its length is
zero.
• A = (a, (b, c)): a list of length of two; its
first element is the atom a, and its second
element is the linear list (b, c).
• B = (A, A, ( )): A list of length of three
whose first two elements are the list A,
and the third element is the null list D.
• C = (a,C): is a recursive list of length two; C
corresponds to the infinite list C = (a, (a, (a,
…))).
 Generalized Lists
• head(A) = ‘a’ and tail(A) = (b, c),
head(tail(A) ) = (b, c) and tail(tail(A)) =
().
• Lists may be shared by other lists
• Lists may be recursive.
 Generalized List Application Example
10 3 2 8 3 2 8 2 2 4 4 3 4
p ( x,y,z ) =x y z + 2x y z +3x y z +x y z+ 6x y z+ 2 yz

• Consider the polynomial P(x, y, z) with

various variables. It is obvious the
sequential representation is not suitable to
this.
• What if a linear list is used?
– The size of the node will vary in size, causing
problems in storage management.
• Let’s try the generalized list.
 Generalized List Application Example
• P(x, y, z) can be rewritten as follows:
10 8 3 8 2 2 4 3 4
(( x +2x ) y + 3x y ) z +(( x +6x ) y +2y ) z
• The above can be written as Cz2 + Dz. Both C and D
are polynomials themselves but with variables x
and y only.
• If we look at polynomial C only, it is actually of
the form Ey3 + Fy2, where E and F are polynomial
of x only.
• Continuing this way, every polynomial consists of a
variable plus coefficient-exponent pairs. Each
coefficient is itself a polynomial.
 PolyNode Class in C++
enum Triple{ var, ptr, no };
class PolyNode
{
PolyNode *link;
int exp;
Triple trio;
union {
char vble;
PolyNode *dlink;
int coef;
};
};
 PolyNode in C++ (Cont.)
• trio == var: the node is a head node.
– vble indicates the name of the variable. Or it is
an integer point to the variable in a variable
table.
– exp is set to 0.
• trio == ptr: coefficient itself is a list and is
pointed by the field dlink. exp is the
exponent of the variable on which the list
is based on.
• trio == no, coefficient is an integer and is
stored in coef. exp is the exponent of the
variable on which the list is based on.
 Representing 3x2y

trio vble exp link trio vble exp link

var y 0 ptr 1 0

var x 0 no 3 2 0
P
 Representation of P(x, y, z)
10 8 3 8 2 2 4 3 4
(( x +2x ) y + 3x y ) z +(( x +6x ) y +2y ) z
P(x, y, z)

v z 0 p 2 p 1 0

v y 0 p 3 p 2 0 v y 0 p 4 p 1 0

v x 0 n 3 8 0 v x 0 n 2 0 0

v x 0 n 1 10 n 2 8 0 v x 0 n 1 4 n 6 3 0
 Recursive Algorithms For
Lists
• A recursive algorithm consists of two
components:
– The recursive function (the workhorse);
declared as a private function
– A second function that invokes the
recursive function at the top level (the
driver); declared as a public function.
 Program 4.6 Copying A List
// Driver
void GenList::Copy(const GenList& l)
{
first = Copy(l.first);
}

// Workhorse
GenListNode* GenList::Copy(GenListNode *p)
// Copy the nonrecursive list with no shared sublists pointed at by p
{
GenListNode *q = 0;
if (p) {
q = new GenListNode;
q->tag = p->tag;
if (!p->tag) q->data = p->data;
else q->dlink = Copy(p->dlink);
q->link = Copy(p->link);
}
return q;
}
Linked Representation for A
A=((a, b), ((c, d), e))
b r
t t 0

t u v
s
f a f b 0 t f e 0

w x
f c f d 0
 Generalized List Representation
Example

D=0 Empty list

A=(a, (b, c))
A f a t 0

f b t c 0

B t t 0 0 B=(A, A, ())

C f a t 0 C=(a, C)
Recursiveness GenList::Copy
Level of Value of p Continuing p Continuing p
recursion level level

1 b 2 r 3 u

2 s 3 u 4 v

3 t 4 w 5 0

4 0 5 x 4 v

3 t 6 0 3 u

2 s 5 x 2 r

1 b 4 w 3 0

2 r

1 b
 Important List Functions

• List Equality (Program 4.37)

• List Depth (Program 4.38)
– An empty list has depth 0.

depth ( s )=
{ 0  if s is an atom
1+max {depth ( x 1 ), ⋅¿ ,depth ( x n ) }  if s is the list ( x 1 ,⋅¿ ,x n ) , n≥ 1 }
Reference Counts, Shared and Recursive
Lists
• Lists may be shared by other lists for the
purpose of space saving.
• Lists that are shared by other lists create
problems when performing add or delete
functions. For example, let’s look at the previous
A, B, C, D example. When deleting the front node
of list A would requires List B to update its
pointers.
• The use of the data field of a head node to
record the reference count can resolve the
aforementioned problem. The list can not be
deleted unless the reference count is 0.
 Example of Reference Counts, Shared
and Recursive Lists

X f 1 0
A=(a, (b, c))
Y f 3 f a t 0

f 1 f b f c 0

Z f 1 t t t 0 B=(A, A, ())

W f 2 f a t 0 f 1 0

C=(a, C)
 Erasing A List Recursively
// Driver
GenList::~GenList()
// Each head node has a reference count. We assume first ≠ 0.
{
Delete(first);
first = 0;
}

// Workhorse
void GenList::Delete(GenListNode* x)
{
x->ref--; // decrement reference coutn of head node.
if (!x->ref)
{
GenListNode *y = x; // y traverses top-level of x.
while (y->link) { y= y->link; if (y->tag == 1) Delete (y->dlink);}
y->link = av; // Attach top-level nodes to av list
av = x;
}
}
 Issue In Erasing Recursive
Lists
• When erasing a recursive list (either
direct recursive or indirect
recursive), the reference count does
not become 0. Then the nodes are not
returned to available list. This will
cause memory leak issue.
Chap 5

Trees
 Trees
• Definition: A tree is a finite set of
one or more nodes such that:
– There is a specially designated node
called the root.
– The remaining nodes are partitioned into
n ≥ 0 disjoint sets T1, …, Tn, where each
of these sets is a tree. We call T1, …, Tn
the subtrees of the root.
 Pedigree Genealogical Chart

Cheryl

Kevin Rosemary

John Terry Richard Kelly

Jack Mary Anthony Karen Joe Michelle Mike Angela

Binary Tree
 Lineal Genealogical Chart

Proto Indo-European

Italic Hellenic Germanic

Osco-Umbrian Latin Greek North Germanic West Germanic

Osco Umbrian Spanish French Italian Icelandic Norwegian Swedish Low High Yiddish
 Tree Terminology
• Normally we draw a tree with the root at the top.
• The degree of a node is the number of subtrees
of the node.
• The degree of a tree is the maximum degree of
the nodes in the tree.
• A node with degree zero is a leaf or terminal
node.
• A node that has subtrees is the parent of the
roots of the subtrees, and the roots of the
subtrees are the children of the node.
• Children of the same parents are called siblings.
 Tree Terminology (Cont.)
• The ancestors of a node are all the nodes
along the path from the root to the node.
• The descendants of a node are all the
nodes that are in its subtrees.
• Assume the root is at level 1, then the
level of a node is the level of the node’s
parent plus one.
• The height or the depth of a tree is the
maximum level of any node in the tree.
 A Sample Tree

Level

A 1

B C D 2

E F G H I J 3

K L M 4
 List Representation of Trees
The tree in previous slide could be written as
(A (B (E (K, L), F), C(G), D(H (M), I, J)))

A 0

B F 0 C G 0 D I J 0

E K L 0 H M 0
Possible Node Structure For A Tree of
Degree
• Lemma 5.1: If T is a k-ary tree (i.e., a
tree of degree k) with n nodes, each
having a fixed size as in Figure 5.4,
then n(k-1) + 1 of the nk child fileds
are 0, n ≥ 1.

Data Child 1 Child 2 Child 3 Child 4 … Child k

Wasting memory!
 Representation of Trees
• Left Child-Right Sibling Representation
– Each node has two links (or pointers).
– Each node only has one leftmost child and one
closest sibling.
A

data

left child right sibling B C D

E F G H I J

K L M
Degree Two Tree Representation

E C

K F G D
Binary Tree!

L H

M I

J
 Tree Representations

A A A

B B B

A A A

B C B C B

C
Left child-right sibling
Binary tree
 Binary Tree
• Definition: A binary tree is a finite set of
nodes that is either empty or consists of a
root and two disjoint binary trees called
the left subtree and the right subtree.
• There is no tree with zero nodes. But
there is an empty binary tree.
• Binary tree distinguishes between the
order of the children while in a tree we do
not.
 Binary Tree Examples

A A
A
B
B C
B
C
D E F G
A D

H I
E
B
 The Properties of Binary
Trees
• Lemma 5.2 [Maximum number of nodes]
1) The maximum number of nodes on level i of a
binary tree is 2i-1, i ≥ 1.
2) The maximum number of nodes in a binary
tree of depth k is 2k – 1, k ≥ 1.
• Lemma 5.3 [Relation between number of
leaf nodes and nodes of degree 2]: For
any non-empty binary tree, T, if n0 is the
number of leaf nodes and n2 the number
of nodes of degree 2, then n0 = n2 + 1.
• Definition: A full binary tree of depth k
is a binary tree of depth k having 2k – 1
nodes, k ≥ 0.
 Binary Tree Definition
• Definition: A binary tree with n nodes and
depth k is complete iff its nodes correspond
to the nodes numbered from 1 to n in the full
binary tree of depth k.
level
1 1

2
2 3

4 5 6 7 3

8 9 10 11 12 13 14 15 4
 Array Representation of A Binary
Tree
• Lemma 5.4: If a complete binary tree with
n nodes is represented sequentially, then
for any node with index i, 1 ≤ i ≤ n, we have:
– parent(i) is at ⌊ i/2 ⌋ if i ≠1. If i = 1, i is at the
root and has no parent.
– left_child(i) is at 2i if 2i ≤ n. If 2i > n, then i has no
left child.
– right_child(i) is at 2i + 1 if 2i + 1 ≤ n. If 2i + 1 > n,
then i has no right child.
• Position zero of the array is not used.
 Proof of Lemma 5.4 (2)
Assume that for all j, 1 ≤ j ≤ i,
left_child(j) is at 2j. Then two nodes
immediately preceding left_child(i +
1) are the right and left children of i.
The left child is at 2i. Hence, the left
child of i + 1 is at 2i + 2 = 2(i + 1)
unless 2(i + 1) > n, in which case i +
1 has no left child.
Array Representation of Binary
Trees
[1] A
[2 B
] [1] A
[3
][4 [2] B
C
] [3] C
[5
] [4] D
[6
] [5] E
[7
][8 [6] F
D
] [7] G
[9
] [8] H
[9] I

[16] E
 Linked Representation
class Tree;
class TreeNode {
friend class Tree;
private:
TreeNode *LeftChild;
char data;
TreeNode *RightChild;
};

class Tree {
public:
// Tree operations
.
private:
TreeNode *root;
};
Node Representation

data

LeftChild RightChild

LeftChild data RightChild

 Linked List Representation For The
Binary Trees

root
root
A 0

A
B 0

B C
C 0

D 0 D E F 0 G 0

0 E 0 H 0 I 0
 Tree Traversal
• When visiting each node of a tree exactly
once, this produces a linear order for the
node of a tree.
• There are 3 traversals if we adopt the
convention that we traverse left before
right: LVR (inorder), LRV (postorder), and
VLR (preorder).
• When implementing the traversal, a
recursion is perfect for the task.
Binary Tree With Arithmetic Expression

* E

* D

/ C

A B
 Tree Traversal
• Inorder Traversal: A/B*C*D+E
=> Infix form
• Preorder Traversal: +**/ABCDE
=> Prefix form
• Postorder Traversal: AB/C*D*E+
=> Postfix form
 Inorder traversal
template <class T>
void Tree <T>::Inorder()
{//Driver calls workhorse for traversal of entire tree.
Inorder(root);
}

template <class T>

void Tree <T>::Inorder (TreeNode <T> *currentNode)
{//Workhouse traverses the subtree rooted at currentNode.
If (currentNode){
Inorder(currentNode->leftChild);
Visit(currentNode);
Inorder(currentNode->Child);
}
}
 Iterative Inorder Traversal
void Tree::NonrecInorder()
// nonrecursive inorder traversal using a stack
{
Stack<TreeNode *> s; // declare and initialize stack
TreeNode *CurrentNode = root;
while (1) {
while (CurrentNode) { // move down LeftChild fields
s.Add(CurrentNode); // add to stack
CurrentNode = CurrentNode->LeftChild;
}
if (!s.IsEmpty()) { // stack is not empty
CurrentNode = *s.Delete(CurrentNode);
cout << CurrentNode->data << endl;
CurrentNode = CurrentNode->RightChild;
}
else break;
}
}
 Level-Order Traversal
• All previous mentioned schemes use
stacks.
• Level-order traversal uses a queue.
• Level-order scheme visit the root
first, then the root’s left child,
followed by the root’s right child.
• All the node at a level are visited
before moving down to another level.
Level-Order Traversal of A Binary Tree
void Tree::LevelOrder()
// Traverse the binary tree in level order
{
Queue<TreeNode *> q;
TreeNode *CurrentNode = root;
while (CurrentNode) {
cout << CurrentNode->data<<endl;
if (CurrentNode->LeftChild) q.Add(CurrentNode->LeftChild);
if (CurrentNode->RightChild) q.Add(CurrentNode->RightChild);
CurrentNode = *q.Delete();
}
}

+*E*D/CAB
 Traversal Without A Stack
• Use of parent field to each node.
• Use of two bits per node to
represents binary trees as threaded
binary trees.
Some Other Binary Tree Functions
• With the inorder, postorder, or
preorder mechanisms, we can
implement all needed binary tree
functions. E.g.,
– Copying Binary Trees
– Testing Equality
• Two binary trees are equal if their
topologies are the same and the information
in corresponding nodes is identical.
 The Satisfiability
Problem
• Expression Rules
– A variable is an expression
– If x and y are expressions then
x ∧ y,x ∨ y,and ¬ x are expressions
– Parentheses can be used to alter the
normal order of evaluation, which is not
before and before or.
Propositional Formula In A Binary Tree

¬¿
¿

x3

x1 ¬¿
¿
¬¿
¿ x3
( x 1∧ ¬ x 2 )∨ ( ¬ x 1 ∧ x 3 )∨ ¬ x 3

x2 x1 O(g2n)
 Perform Formula Evaluation
• To evaluate an expression, we can
traverse its tree in postorder.
• To perform evaluation, we assume
each node has four fields
– LeftChild
– data
– value
– RightChild LeftChild data value RightChild
First Version of Satisfiability Algorithm
For all 2n possible truth value combinations for the n
variables
{
generate the next combination;
replace the variables by their values;
evaluate the formula by traversing the tree it points
to in postorder;
if (formula.rootvalue()) {cout << combination; return;}
}
Cout << “no satisfiable combination”;
 Evaluating A Formula
void SatTree::PostOrderEval() // Driver
{
PostOrderEval(root);
}

void SatTree::PostOrderEval(SatNode * s) // workhorse

{
if (s) {
PostOrderEval(s->LeftChild);
PostOrderEval(s->RightChild);
switch (s->data) {
case LogicalNot: s->value =!s->RightChild->value; break;
case LogicalAnd: s->value = s->LeftChild->value && s->RightChild->value;
break;
case LogicalOr: s->value = s->LeftChild->value || s->RightChild->value;
break;
case LogicalTrue: s->value = TRUE; break;
case LogicalFalse: s->value = FALSE;
}
}
}
 Threaded Binary Tree
• Threading Rules
– A 0 RightChild field at node p is replaced by a
pointer to the node that would be visited after
p when traversing the tree in inorder. That is,
it is replaced by the inorder successor of p.
– A 0 LeftChild link at node p is replaced by a
pointer to the node that immediately precedes
node p in inorder (i.e., it is replaced by the
inorder predecessor of p).
Threaded Tree Corresponding to Figure
5.10(b)

B C

D E F G

H I

Inorder sequence: H, D, I, B, E, A, F, C, G
 Threads
• To distinguish between normal
pointers and threads, two boolean
fields, LeftThread and RightThread,
are added to the record in memory
representation.
– t->LeftChild = TRUE
=> t->LeftChild is a thread
– t->LeftChild = FALSE
=> t->LeftChild is a pointer to the left child.
 Threads (Cont.)
• To avoid dangling threads, a head
node is used in representing a binary
tree.
• The original tree becomes the left
subtree of the head node.
• Empty Binary Tree

LeftThread LeftChild data RightChild RightThread

TRUE FALSE
 Memory Representation of Threaded
Tree of Figure 5.20

f - f

f A f

f B f f B f

f D f t E t f D f t E t

t H t t I t
 Finding the inorder successor
without stack
• By using the threads, we can perform an
inorder traversal without making use of a
stack.
T* ThreadedInorderIterator::Next()
{//Return the inorder successor of currentnode
ThreadedNode <T> *temp = currentNode -> rightChild;
if (!currentNode->rightThread)
while (!temp->leftThread) temp = temp -> leftChild;
currentNode = temp;
if (currentNode == root) return 0;
else return &currentNode -> data;
}
 Inserting A Node to AThreaded Binary
Tree
• Inserting a node r as the right child of a
node s.
– If s has an empty right subtree, then the
insertion is simple and diagram in Figure 5.23(a).
– If the right subtree of s is not empty, the this
right subtree is made the right subtree of r
after insertion. When thisis done, r becomes the
inorder predecessor of a node that has a
LdeftThread==TRUE field, and consequently
there is an thread which has to be updated to
point to r. The node containing this thread was
previously the inorder successor of s. Figure
5.23(b) illustrates the insertion for this case.
Insertion of r As A Right Child of s in A
Threaded Binary Tree

s s

r r

before after
Insertion of r As A Right Child of s in A
Threaded Binary Tree (Cont.)

s s

r r r

before after
 Program 5.16 Inserting r As The Right
Child of s
void ThreadedTree::InsertRight(ThreadNode *s, ThreadedNode *r)
// Insert r as the right child of s
{
r->RightChild = s->RightChild;
r->RightThread = s->RightThread;
r->LeftChild = s;
r->LeftThread = TRUE; // LeftChild is a thread
r->RightChild = r; // attach r to s
r->RightThread = FALSE;
if (!r->RightThread) {
ThreadedNode *temp = InorderSucc(r); // returns the inorder
successor of r
temp->LeftChild = r;
}
}
 Priority Queues
• In a priority queue, the element to be
deleted is the one with highest (or lowest)
priority.
• An element with arbitrary priority can be
inserted into the queue according to its
priority.
• A data structure supports the above two
operations is called max (min) priority
queue.
 Examples of Priority Queues
• Suppose a server that serve multiple
users. Each user may request different
amount of server time. A priority queue is
used to always select the request with the
smallest time. Hence, any new user’s
request is put into the priority queue. This
is the min priority queue.
• If each user needs the same amount of
time but willing to pay more money to
obtain the service quicker, then this is
max priority queue.
 Priority Queue
Representation
• Unorder Linear List
– Addition complexity: O(1)
– Deletion complexity: O(n)
• Chain
– Addition complexity: O(1)
– Deletion complexity: O(n)
• Ordered List
– Addition complexity: O(n)
– Deletion complexity: O(1)
 Max (Min) Heap
• Heaps are frequently used to implement
priority queues. The complexity is O(log n).
• Definition: A max (min) tree is a tree in
which the key value in each node is no
smaller (larger) than the key values in its
children (if any). A max heap is a complete
binary tree that is also a max tree. A min
heap is a complete binary tree that is also
a min tree.
 Max Heap Examples

14 9

12 7 6 3

10 8 6 5
 Insertion Into A Max Heap
(1)

20 20

15 2 15 2

14 10 14 10 1
 Insertion Into A Max Heap
(2)

20 20

15 2 15 2
5

14 10 14 10 2
 Insertion Into A Max Heap
(3)

20 20
21

15 2 15 20
2

14 10 14 10 2
 Deletion From A Max Heap
template <class Type>
void MaxHeap <T>::Pop()
{// Delete from the max heap
if (IsEmpty()) throw “Heap is empty.”;
heap[1].~T(); //delete max element
//remove last element from heap
T lastE = heap [heapSize--];
int currentNode = 1;
int child = 2;
while (child >= leapSize)
{
if (child < heapSize && heap[child] < heap[child+1]) child++;
if (lastE >= heap[child]) break;
heap[currentNode] = heap[child];
currentNode = child; child *= 2;
}
heap[currentNode] = lastE;
}
Deletion From A Max Heap (Cont.)

21 2

15 20 15 20

14 10 2 14 10 2
Deletion From A Max Heap (Cont.)

2
20

15 20
15 2

14 10
14 10
 Binary Search Tree
• Definition: A binary serach tree is a binary
tree. It may be empty. If it is not empty
then it satisfies the following properties:
– Every element has a key and no two elements
have the same key (i.e., the keys are distinct)
– The keys (if any) in the left subtree are
smaller than the key in the root.
– The keys (if any) in the right subtree are
larger than the key in the root.
– The left and right subtrees are also binary
search trees.
 Binary Trees

30 60
20

5 40 70
15 25

65 80
14 10 22 2

Not binary Binary search

search tree trees
 Searching A Binary Search
Tree
• If the root is 0, then this is an empty
tree. No search is needed.
• If the root is not 0, compare the x with
the key of root.
– If x equals to the key of the root, then it’s
done.
– If x is less than the key of the root, then no
elements in the right subtree can have key
value x. We only need to search the left tree.
– If x larger than the key of the root, only the
right subtree is to be searched.
Search Binary Search Tree by Rank
• To search a binary search tree by the
ranks of the elements in the tree, we need
additional field “LeftSize”.
• LeftSize is the number of the elements in
the left subtree of a node plus one.
• It is obvious that a binary search tree of
height h can be searched by key as well as
by rank in O(h) time.
Searching A Binary Search Tree by Rank
template <class Type>
BstNode <Type>* BST<Type>::Search(int k)
// Search the binary search tree for the kth smallest element
{
BstNode<Type> *t = root;
while(t)
{
if (k == t->LeftSize) return n;
if (k < t->LeftSize) t = t->LeftChild;
else {
k -= LeftSize;
t = t->RightChild;
}
}
return 0;
}
Insertion To A Binary Search Tree
• Before insertion is performed, a search
must be done to make sure that the value
to be inserted is not already in the tree.
• If the search fails, then we know the value
is not in the tree. So it can be inserted
into the tree.
• It takes O(h) to insert a node to a binary
search tree.
 Inserting Into A Binary Search Tree

30 30

5 40 5 40

2 80 2 35 80
 Insertion Into A Binary Search Tree
Template <class Type>
Boolean BST<Type>::Insert(const Element<Type>& x)
// insert x into the binary search tree
{
// Search for x.key, q is the parent of p
BstNode<Type> *p = root; BstNode<Type> *q = 0;
while(p) {
q = p;
if (x.key == p->data.key) return FALSE; // x.key is already in tree
if (x.key < p->data.key) p = p->LeftChild;
else p = p->RightChild;
} O(h)
// Perform insertion
p = new BstNode<Type>;
p->LeftChild = p->RightChild = 0; p->data = x;
if (!root) root = p;
else if (x.key < q->data.key) q->LeftChild = p;
else q->RightChild = p;
return TRUE;
}
Deletion From A Binary Search Tree
• Delete a leaf node
– A leaf node which is a right child of its parent
– A leaf node which is a left child of its parent
• Delete a non-leaf node
– A node that has one child
– A node that has two children
• Replaced by the largest element in its left subtree, or
• Replaced by the smallest element in its right subtree
• Again, the delete function has complexity of
O(h)
Deleting From A Binary Search Tree

30
30

5 40 2
5 40

2 35 80 2 80
 Deleting From A Binary Search Tree

30 5
30 5
30

2
5 40 2
5 40 2 40

2 80 2 80 80
 Joining and Splitting Binary Trees
• C.ThreeWayJoin(A, x, B): Creates a binary
search tree C that consists of binary
search tree A, B, and element x.
• C.TwoWayJoin(A, B): Joins two binary
search trees A and B to obtain a single
binary search tree C.
• A.Split(i, B, x, C): Binary search tree A
splits into three parts: B (a binary search
tree that contains all elements of A that
have key less than i); if A contains a key i
than this element is copied into x and a
pointer to x returned; C is a binary search
tree that contains all records of A that
have key larger than i.
 ThreeWayJoin(A, x, B)

81

30 90

5 40 81 85 94

2 35 80 84 92

A x B
 TwoWayJoin(A, B)

80
84

30 90

5 40 85 94

2 35 80 84 92

A B
A.Split(i, B, x, C)

30 x i = 30

5 40

2 35 80

B
75 81

A C
 A.Split(i, B, x, C)

i = 80 30 t Y L Z R

30 L 81
5 40 t R

5 40 L
C
2 35 80 t

2 35 75 80
75 81
x
A B
 Selection Trees
• When trying to merge k ordered sequences
(assume in non-decreasing order) into a
single sequence, the most intuitive way is
probably to perform k – 1 comparison each
time to select the smallest one among the
first number of each of the k ordered
sequences. This goes on until all numbers in
every sequences are visited.
• There should be a better way to do this.
 Winner Tree
• A winner tree is a complete binary tree in
which each node represents the smaller of
its two children. Thus the root represents
the smallest node in the tree.
• Each leaf node represents the first record
in the corresponding run.
• Each non-leaf node in the tree represents
the winner of its right and left subtrees.
 Winner Tree For k = 8

1
6
2 3
6 8
4 5 7
6
9 6 8 17
8 9 10 11 12 13 14 15
10 9 20 6 8 9 90 17

15 20 20 15 15 11 95 18
16 38 30 25 50 16 99 20
28

run1 run2 run3 run4 run5 run6 run7 run8

 Winner Tree For k = 8

1
8
2 3
9 8
4 5 7
6
9 15 8 17
8 9 10 11 12 13 14 15
10 9 20 15 8 9 90 17

15 20 20 25 15 11 95 18
16 38 30 28 50 16 99 20

run1 run2 run3 run4 run5 run6 run7 run8

 Analysis of Winner Tree

• The number of levels in the tree is ( log 2 ( k+ 1 ) )

– The time to restructure the winner tree is O(log2k).
• Since the tree has to be restructured each time a
number is output, the time to merge all n records is
O(n log2k).
• The time required to setup the selection tree for
the first time is O(k).
• Total time needed to merge the k runs is O(n log2k).
 Loser Tree
• A selection tree in which each
nonleaf node retains a pointer to the
loser is called a loser tree.
• Again, each leaf node represents the
first record of each run.
• An additional node, node 0, has been
added to represent the overall
winner of the tournament.
 Loser Tree

0
6 Overall
winner

1
8
2 3
9 17
4 5 7
6
10 20 9 90
9 10 11 12 13 14 15
10 9 20 6 8 9 90 17

run 1 2 3 4 5 6 7 8
 Loser Tree

0
8 Overall
winner

1
9
2 3
15 17
4 5 7
6
10 20 9 90
9 10 11 12 13 14 15
10 9 20 15 8 9 90 17

run 1 2 3 4 5 6 7 8
 Forests
• Definition: A forest is a set of n ≥ 0
disjoint trees.
• When we remove a root from a tree,
we’ll get a forest. E.g., Removing the
root of a binary tree will get a forest
of two trees.
 Transforming A Forest Into A Binary
Tree
• Definition: If T1, …, Tn is a forest of
trees, then the binary tree corresponding
to this forest, denoted by B(T1, …, Tn),
– is empty if n = 0
– has root equal to root (T1); has left subtree
equal to B(T11, T12,…, T1m), where T11, T12,…, T1m
are the subtrees of root (T1); and has right
subtree B(T2, …, Tn).
 Set Representation
• Trees can be used to represent sets.
• Disjoint set union: If Si and Sj are
two disjoint sets, then their union Si
∪Sj = {all elements x such that x is in
Si or Sj}.
• Find(i). Find the set containing
element i.
Possible Tree Representation of Sets

0 4 2

6 7 8 1 9 3 5

S1 S2 S3
 Possible Representations of Si ∪Sj

0 4

6
7 8 4 0 1 9

1 9 6 7 8
 Unions of Sets
• To obtain the union of two sets, just
set the parent field of one of the
roots to the other root.
• To figure out which set an element is
belonged to, just follow its parent
link to the root and then follow the
pointer in the root to the set name.
 Data Representation for S1, S2, S3

Set
Name Pointer

S1 0 4 2

S2
6 7 8 1 9 3 5
S3
 Array Representation of S1, S2, S3
• We could use an array for the set
name. Or the set name can be an
element at the root.
• Assume set elements are numbered 0
through n-1.

i [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
parent -1 4 -1 2 -1 2 0 0 0 4
 Union-Find Operations
• For a set of n elements each in a set of its
own, then the result of the union function
is a degenerate tree.
• The time complexity of the following
union-find operation is O(n2).
union(0, 1), union(1, 2), …, union(n-2, n-1)
find(0), find (1), …, find(n-1)
• The complexity can be improved by using
weighting rule for union.
 Degenerate Tree

Union operation n-1 union(0, 1), find(0)

O(n) n-1 union(1, 2), find(0)
n-2

Find operation
O(n2) union(n-2, n-1), find(0)

0
 Weighting Rule
• Definition [Weighting rule for
union(I, j)]: If the number of nodes
in the tree with root i is less than
the number in the tree with root j,
then make j the parent of i;
otherwise make i the parent of j.
 Trees Obtained Using The Weighting
Rule

0 1 n-1 0 2 n-1 0 3 n-1

1 1 2

0 4 n-1 0

1 2 3
1 2 3 n-1
 Weighted Union
• Lemma 5.5: Assume that we start with a
forest of trees, each having one node. Let T
be a tree with m nodes created as a result
of a sequence of unions each performed
using function WeightedUnion. The height
of T is no greater than ⌊ log 2 m ⌋ +1 .
• For the processing of an intermixed
sequence of u – 1 unions and f find
operations, the time complexity is O(u +
f*log u).
Trees Achieving Worst-Case Bound

[-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1]

0 1 2 3 4 5 6 7

(a) Initial height trees

[-2] [-2] [-2] [-2]

0 2 4 6

1 3 5 7

(b) Height-2 trees following union (0, 1), (2, 3), (4, 5), and (6, 7)
 Trees Achieving Worst-Case Bound
(Cont.)

[-4] [-4] [-8]

0 4 0

1 2 5 6 1 2 4

3 7 3 5 6

(c) Height-3 trees following union (0, 7

2), (4, 6)

(d) Height-4 trees following union (0, 4)

 Collapsing Rule
• Definition [Collapsing rule]: If j is a node
on the path from i to its root and
parent[i]≠ root(i), then set parent[j] to
root(i).
• The first run of find operation will collapse
the tree. Therefore, all following find
operation of the same element only goes up
one link to find the root.
 Collapsing Find

[-8] [-8]

0 0

1 2 4 1 2 4 6 7

3 5 6 3 5

7 After
Before collapsing
collapsing
 Analysis of WeightedUnion and
CollapsingFind
• The use of collapsing rule roughly double the
time for an individual find. However, it
reduces the worst-case time over a
sequence of finds.
• Lemma 5.6 [Tarjan and Van Leeuwen]:
Assume that we start with a forest of
trees, each having one node. Let T(f, u) be
the maximum time required to process any
intermixed sequence of f finds and u unions.
Assume that u ≥ n/2. Then
k1(n + fα(f + n, n)) ≤ T(f, u) ≤ k2(n + fα(f + n, n))
for some positive constants k1 and k2.
 Revisit Equivalence Class
• The aforementioned techniques can be applied to the equivalence
class problem.
• Assume initially all n polygons are in an equivalence class of their
own: parent[i] = -1, 0 ≤ i < n.
– Firstly, we must determine the sets that contains i and j.
– If the two are in different set, then the two sets are to be replaced
by their union.
– If the two are in the same set, then nothing need to be done since
they are already in the equivalence class.
– So we need to perform two finds and at most one union.
• If we have n polygons and m equivalence pairs, we need
– O(n) to set up the initial n-tree forest.
– 2m finds
– at most min{n-1, m} unions.
• If weightedUnion and CollapsingFind are used, the time complexity
is O(n + m (2m, min{n-1, m})).
– This seems to slightly worse than section 4.7 (O(m+n)). But this
scheme demands less space.
 Example 5.5
[-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1] [-1]
0 1 2 3 4 5 6 7 8 9 10 11

(a) Initial trees

[-2] [-2] [-2] [-2] [-1] [-1] [-1] [-1]

0 3 6 8 2 5 7 11

4 1 10 9

(b) Height-2 trees following 0 ≡4, 3 ≡1, 6 ≡10, and 8 ≡9

 Example 5.5 (Cont.)
[-3] [-4] [-3] [-2]

0 6 3 2

4 7 10 8 1 5 11

9
[-3] [-4] [-3]

0 6 3

4 7 2 10 8 1 5

11 9
Uniqueness of A Binary Tree
• In section 5.3 we introduced
preorder, inorder, and postorder
traversal of a binary tree. Now
suppose we are given a sequence (e.g.,
inorder sequence BCAEDGHFI), does
the sequence uniquely define a binary
tree?
 Constructing A Binary Tree From Its
Inorder Sequence

A A

B, C D, E, F, G, H, I B D, E, F, G, H, I

C
Constructing A Binary Tree From Its
Inorder Sequence (Cont.)

1
A

2 4
B D
3 5 6
C E F
7 9
G I
8
H
Preorder: 1, 2, 3, 4, 5, 6, 7, 8, 9

Inorder: 2, 3, 1, 5, 4, 7, 8, 6, 9
 Distinct Binary Trees

1 1 1 1 1

2 2 2 3 2 2

3 3 3 3

(1, 2, 3) (1, 3, 2) (2, 1, 3) (2, 3, 1) (3, 2, 1)

 Distinct Binary Trees
(Cont.)
• The number of distinct binary trees is equal to the number of distinct
inorder permutations obtainable from binary trees having the preorder
permutation, 1, 2, …, n.
• Computing the product of n matrices are related to the distinct binary
tree problem.
M1 * M2 * … * Mn

n=3 (M1 * M2) * M3 M1 * (M2 * M3 )

n=4 ((M1 * M2) * M3) * M4
(M1 * (M2 * M3)) * M4
M1 * ((M2 * M3) * M4 )
(M1 * (M2 * (M3 * M4 )))
((M1 * M2) * (M3 * M4 ))

Let bn be the number of different ways to compute the product of n

matrices. b2 = 1, b3 =n−12, and b4 = 5.
b n = ∑ b i bn−i , n> 1
i=1
 Distinct Binary Trees
(Cont.)

• The number of distinct binary trees of n

nodes is b n =∑ b i b n−i−1 , n≥1 ,and b0 =1

bn

bi bn-i-1
 Distinct Binary Trees
(Cont.)
B ( x )= ∑ b i x
i
• Assume we let i≥ 0 which is the generating
function for the number of binary trees.
• By the recurrence relation we get

2
xB ( x )=B ( x )−1
1−√ 1−4x
B( x )=
2x

B( x )=
1
2x ( ( )
1− ∑
n≥0
1/ 2
n
(−4x )n = ∑
) ( )
1/ 2
m≥0 m+ 1
(−1 )m 2 2m+ 1 x m

bn=
1 2n
n+1 n ( )
≈ b n =O (4 n /n3/2 )
Chap 6

Graph
 Konigsberg Bridge Problem
• A river Pregel flows around the island
Keniphof and then divides into two.
• Four land areas A, B, C, D have this river
on their borders.
• The four lands are connected by 7 bridges
a – g.
• Determine whether it’s possible to walk
across all the bridges exactly once in
returning back to the starting land area.
Konigsberg Bridge Problem (Cont.)

c C
d g

A
e D
Kneiphof

C
g
a f c d
B b e
A D
a b
f

B
 Euler’s Graph
• Define the degree of a vertex to be the
number of edges incident to it
• Euler showed that there is a walk starting
at any vertex, going through each edge
exactly once and terminating at the start
vertex iff the degree of each vertex is
even. This walk is called Eulerian.
• No Eulerian walk of the Konigsberg bridge
problem since all four vertices are of odd
edges.
 Application of Graphs
• Analysis of electrical circuits
• Finding shortest routes
• Project planning
• Identification of chemical compounds
• Statistical mechanics
• Genertics
• Cybernetics
• Linguistics
• Social Sciences, and so on …
 Definition of A Graph
• A graph, G, consists tof two sets, V and E.
– V is a finite, nonempty set of vertices.
– E is set of pairs of vertices called edges.
• The vertices of a graph G can be represented as
V(G).
• Likewise, the edges of a graph, G, can be
represented as E(G).
• Graphs can be either undirected graphs or
directed graphs.
• For a undirected graph, a pair of vertices (u, v) or
(v, u) represent the same edge.
• For a directed graph, a directed pair <u, v> has u
as the tail and the v as the head. Therefore, <u, v>
and <v, u> represent different edges.
 Three Sample Graphs

0 0
0

1 2 1 2
1

3
3 4 5 6
2
V(G1) = {0, 1, 2, 3}
V(G2) = {0, 1, 2, 3, 4, 5, 6}
V(G3) = {0, 1, 2}

E(G1) = {(0, 1), (0, 2), (0, 3), (1,

E(G2) = {(0, 1), (0, 2), (1, 3), (1,
2), (1, 3), (2, 3)} E(G3) = {<0, 1>, <1, 0>, <1, 2>}
4), (2, 5), (2, 6)}

(a) G1 (b) G2 (c) G3

 Graph Restrictions
• A graph may not have an edge from a
vertex back to itself.
– (v, v) or <v, v> are called self edge or
self loop. If a graph with self edges, it
is called a graph with self edges.
• A graph man not have multiple
occurrences of the same edge.
– If without this restriction, it is called a
multigraph.
 Complete Graph
• The number of distinct unordered pairs (u,
v) with u≠v in a graph with n vertices is n(n-
1)/2.
• A complete unordered graph is an unordered
graph with exactly n(n-1)/2 edges.
• A complete directed graph is a directed
graph with exactly n(n-1) edges.
 Examples of Graphlike Structures

0 1 1 3

2 2

(a) Graph with a self edge (b) Multigraph

 Graph Edges
• If (u, v) is an edge in E(G), vertices u
and v are adjacent and the edge (u, v)
is the incident on vertices u and v.
• For a directed graph, <u, v> indicates
u is adjacent to v and v is adjacent
from u.
 Subgraph and Path
• Subgraph: A subgraph of G is a graph G’ such that
V(G’) V(G) and E(G’) E(G).
• Path: A path from vertex u to vertex v in graph G is
a sequence of vertices u, i1, i2, …, ik, v, such that (u,
i1), (i1, i2), …, (ik, v) are edges in E(G).
– The length of a path is the number of edges on it.
– A simple path is a path in which all vertices except possibly
the first and last are distinct.
– A path (0, 1), (1, 3), (3, 2) can be written as 0, 1, 3, 2.
• Cycle: A cycle is a simple path in which the first and
last vertices are the same.
• Similar definitions of path and cycle can be applied
to directed graphs.
 G1 and G3 Subgraphs
0
0 0 1 2
1 2
1 2 3
(i) (ii)
(iii) 3
(iv)
(a) Some subgraphs of G1

0 0 0 0

1 1 1

(i) (ii) 2
2 2
(a) Some subgraphs of G3 (iii) (iv)
 Connected Graph
• Two vertices u and v are connected in an
undirected graph iff there is a path from
u to v (and v to u).
• An undirected graph is connected iff for
every pair of distinct vertices u and v in
V(G) there is a path from u to v in G.
• A connected component of an undirected is a
maximal connected subgraph.
• A tree is a connected acyclic graph.
 Strongly Connected Graph
• A directed graph G is strongly
connected iff for every pair of
distinct vertices u and v in V(G),
there is directed path from u to v
and also from v to u.
• A strongly connected component is a
maximal subgraph that is strongly
connected.
Graphs with Two Connected Components

H2
H1 0 0

1 2 1 2

3 3

G4
Strongly Connected Components of G3

1 2
 Degree of A Vertex
• Degree of a vertex: The degree of a vertex is the
number of edges incident to that vertex.
• If G is a directed graph, then we define
– in-degree of a vertex: is the number of edges for which
vertex is the head.
– out-degree of a vertex: is the number of edges for which the
vertex is the tail.
• For a graph G with n vertices and e edges, if di is
the degree of a vertex i in G, then the number of
edges of G is n−1
e=( ∑ d i )/2
i= 0
 Abstract of Data Type
Graphs
class Graph
{
// objects: A nonempty set of vertices and a set of
undirected edges
// where each edge is a pair of vertices
public:
Graph(); // Create an empty graph
void InsertVertex(Vertex v);
void InsertEdge(Vertex u, Vertex v);
void DeleteVertex(Vertex v);
void DeleteEdge(Vertex u, Vertex v);

Boolean IsEmpty(); // if graph has no vertices return TRUE

List<List> Adjacent(Vertex v);

// return a list of all vertices that are adjacent to v
};
 Adjacent Matrix
• Let G(V, E) be a graph with n vertices, n ≥ 1. The
adjacency matrix of G is a two-dimensional nxn
array, A.
– A[i][j] = 1 iff the edge (i, j) is in E(G).
– The adjacency matrix for a undirected graph is
symmetric, it may not be the case for a directed graph.
• For an undirected graph the degree of any vertex
i is its row sum.
• For a directed graph, the row sum is the out-
degree and the column sum is the in-degree.
 Adjacency Matrices

[ ]
0 1 2 3 4 5 6 7
0 0 1 1 0 0 0 0 0
1 1 0 0 1 0 0 0 0
2 1 0 0 1 0 0 0 0
0 1 2 3 3 0 1 1 0 0 0 0 0

[ ] [ ]
0 0 1 1 1 0 1 2 4 0 0 0 0 0 1 0 0
1 1 0 1 1 00 1 0 5 0 0 0 0 1 0 1 0
2 1 1 0 1 11 0 1 6 0 0 0 0 0 1 0 1
3 1 1 1 0 20 0 0 7 0 0 0 0 0 0 1 0

(a) G1 (b) G3 (c) G4

 Adjacency Lists

• Instead of using a matrix to represent the adjacency of a

graph, we can use n linked lists to represent the n rows of
the adjacency matrix.
• Each node in the linked list contains two fields: data and
link.
– data: contain the indices of vertices adjacent to a vertex i.
– Each list has a head node.
• For an undirected graph with n vertices and e edges, we
need n head nodes and 2e list nodes.
• The degree of any vertex may be determined by counting
the number nodes in its adjacency list.
• The number of edges in G can be determined in O(n + e).
• For a directed graph (also called digraph),
– the out-degree of any vertex can be determined by counting
the number of nodes in its adjacency list.
– the in-degree of any vertex can be obtained by keeping
another set of lists called inverse adjacency lists.
 Adjacent Lists

HeadNodes
[0 3 1 2 0
][1
2 3 0 0
]
[2 1 3 0 0
]
[3 0 1 2 0
]
(a) G1

HeadNodes

[0 1 0
][1
2 0 0
]
[2 0
]
(b) G3
Adjacent Lists (Cont.)

HeadNodes
[0 2 1 0
][1
3 0 0
]
[2 0 3 0
]
[3 1 1 0
]
[4 5 0
][5
6 4 0
]
[6 5 7 0
]
[7 6 0
]

(c) G4
 Sequential Representation of Graph G4

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

9 11 13 15 17 18 20 22 23 2 1 3 0 0 3 1 2 5 6 4 5 7 6
Inverse Adjacency Lists for
G3

[0] 1 0

[1] 0 0

[2] 1 0
 Multilists
• In the adjacency-list representation of an
undirected graph, each edge (u, v) is
represented by two entries.
• Multilists: To be able to determine the
second entry for a particular edge and
mark that edge as having been examined,
we use a structure called multilists.
– Each edge is represented by one node.
– Each node will be in two lists.
 Orthogonal List Representation for G3

head nodes
(shown twice) 0 1 2

0 0 1 0 0

1 1 0 0 1 2 0 0

2 0
 Adjacency Multilists for G1

HeadNodes
[0 N0 0 1 N1 N3 edge (0, 1)
][1
] N1 0 2 N2 N3 edge (0, 2
[2
]
[3 N2 0 3 0 N4 edge (0, 3)
]
N3 1 2 N4 N5 edge (1, 2)
The lists are
N4 1 3 0 N5 edge (1, 3)
Vertex 0: N0 -> N1 -> N2
Vertex 1: N0 -> N3 -> N4 2 3 0 0 edge (2, 3)
N5
Vertex 2: N1 -> N3 -> N5
Vertex 3: N2 -> N4 -> N5
 Weighted Edges
• Very often the edges of a graph have
weights associated with them.
– distance from one vertex to another
– cost of going from one vertex to an
adjacent vertex.
– To represent weight, we need additional
field, weight, in each entry.
– A graph with weighted edges is called a
network.
 Graph Operations
• A general operation on a graph G is to
visit all vertices in G that are
reachable from a vertex v.
– Depth-first search
– Breath-first search
 Depth-First Search
• Starting from vertex, an unvisited vertex w
adjacent to v is selected and a depth-first search
from w is initiated.
• When the search operation has reached a vertex
u such that all its adjacent vertices have been
visited, we back up to the last vertex visited that
has an unvisited vertex w adjacent to it and
initiate a depth-first search from w again.
• The above process repeats until no unvisited
vertex can be reached from any of the visited
vertices.
 Graph G and Its Adjacency
Lists
0
1 2

3 4 5 6
HeadNodes
[0] 7
1 2 0
[1] 0 3 4 0
[2] 0 5 6 0
[3] 1 7 0
[4] 1 7 0
[5] 2 7 0
[6] 2 7 0
[7 3 4 5 6 0
 Analysis of DFS
• If G is represented by its adjacency
lists, the DFS time complexity is
O(e).
• If G is represented by its adjacency
matrix, then the time complexity to
complete DFS is O(n2).
 Breath-First Search
• Starting from a vertex v, visit all unvisited
vertices adjacent to vertex v.
• Unvisited vertices adjacent to these newly
visited vertices are then visited, and so on.
• If an adjacency matrix is used, the BFS
complexity is O(n2).
• If adjacency lists are used, the time
complexity of BFS is O(e).
A Complete Graph and Three of Its
Spanning Trees
Depth-First and Breath-First Spanning
Trees

0 0

1 2 1 2

3 4 5 6 3 4 5 6

7 7

(a) DFS (0) spanning tree (b) BFS (0) spanning tree
 Spanning Tree

• Any tree consisting solely of edges in G and

including all vertices in G is called a spanning tree.
• Spanning tree can be obtained by using either a
depth-first or a breath-first search.
• When a nontree edge (v, w) is introduced into any
spanning tree T, a cycle is formed.
• A spanning tree is a minimal subgraph, G’, of G
such that V(G’) = V(G), and G’ is connected.
(Minimal subgraph is defined as one with the
fewest number of edges).
• Any connected graph with n vertices must have at
least n-1 edges, and all connected graphs with n –
1 edges are trees. Therefore, a spanning tree has
n – 1 edges.
 A Connected Graph and Its Biconnected
Components

0 8 9 0 8 9

1 7 1 7 7

2 3 5 1 7

4 6 2 3 3 5 5

4 6
(a) A connected graph
(b) Its biconnected components
 Biconnected Components
• Definition: A vertex v of G is an articulation point
iff the deletion of v, together with the deletion
of all edges incident to v, leaves behind a graph
that has at least two connected components.
• Definition: A biconnected graph is a connected
graph that has no articulation points.
• Definition: A biconnected component of a
connected graph G is a maximal biconnected
subgraph H of G. By maximal, we mean that G
contains no other subgraph that is both
biconnected and properly contains H.
 Biconnected Components
(Cont.)
• Two biconnected components of the same graph can have at
most one vertex in common.
• No edge can be in two or more biconnected components.
• The biconnected components of G partition the edges of G.
• The biconnected components of a connected, undirected
graph G can be found by using any depth-first spanning tree
of G.
• A nontree edge (u, v) is a back edge with respect to a
spanning tree T iff either u is an ancestor of v or v is an
ancestor of u.
• A nontree edge that is not back edge is called a cross edge.
• No graph can have cross edges with respect to any of its
depth-first spanning trees.
 Biconnected Components (Cont.)
• The root of the depth-first spanning tree is an
articulation point iff it has at least two children.
• Any other vertex u is an articulation point iff it
has at least one child, w, such that it is not
possible to reach an ancestor of u using apath
composed solely of w, descendants of w, and a
single back edge.
• Define low(w) as the lowest depth-first number
that can be reached fro w using a path of
descendants followed by, at most, one back edge.
dfn ( w ) ,min { low ( x )∣ x is a child of w } ,

min { dfn ( x )∣( w, x ) is a back edge }

low ( w )= min
• u is an articulation point iff u is
either the root of the spanning tree
and has two or more children or u is
not the root and u has a child w such
that low(w) ≥ dfn(u).
 Depth-First Spanning Tree

3
5 0 10 8 9 9
2 6
4 5

4 1 7 8
1 6 3 2 7 6
3 2 3 5
4
1 8 7
4 6
2 7
5 0 9 8 9 10
dfn and low values for the Spanning Tree

vertex 0 1 2 3 4 5 6 7 8 9
dfn 5 4 3 1 2 6 7 8 10 9
low 5 1 1 1 1 6 6 6 10 9
 Minimal Cost Spanning Tree
• The cost of a spanning tree of a weighted,
undirected graph is the sum of the costs
(weights) of the edges in the spanning tree.
• A minimum-cost spanning tree is a spanning tree of
least cost.
• Three greedy-method algorithms available to
obtain a minimum-cost spanning tree of a
connected, undirected graph.
– Kruskal’s algorithm
– Prim’s algorithm
– Sollin’s algorithm
 Kruskal’s Algorithm
• Kruskal’s algorithm builds a minimum-cost
spanning tree T by adding edges to T one at a
time.
• The algorithm selects the edges for inclusion in T
in nondecreasing order of their cost.
• An edge is added to T if it does not form a cycle
with the edges that are already in T.
• Theorem 6.1: Let G be any undirected, connected
graph. Kruskal’s algorithm generates a minimum-
cost spanning tree.
Stages in Kruskal’s Algorithm

0 28 0 0
10 1 1 1
10
14 16

5 6 2 5 6 2 5 6 2
24
25 18
12
4 4 4
22 3 3 3

(a) (b) (c)

 Stages in Kruskal’s Algorithm
(Cont.)

0 0 0
10 1 1 1
10 10
14 14 16
5 6 2 5 6 2 5 6 2

12 12 12
4 4 4
3 3 3

(d) (e) (f)

 Stages in Kruskal’s Algorithm
(Cont.)

0 0

1 10 1
10 14
14 16
16

5 6 2 5 6 2
25 12
12 4
4
3 22 3
22

(g) (g)
 Prim’s Algorithm

• Similar to Kruskal’s algorithm, Prim’s algorithm

constructs the minimum-cost spanning tree edge
by edge.
• The difference between Prim’s algorithm and
Kruskal’s algorithm is that the set of selected
edges forms a tree at all times when using Prim’s
algorithm while a forest is formed when using
Kruskal’s algorithm.
• In Prim’s algorithm, a least-cost edge (u, v) is
added to T such that T∪ {(u, v)} is also a tree.
This repeats until T contains n-1 edges.
• Prim’s algorithm in program 6.7 has a time
complexity O(n2).
 Stages in Prim’s Alogrithm

0 0 0
1 10
1 1
10 10

5 6 2 5 6 2 5 6 2
25 25
4 4 4
3 3 22 3

(a) (b) (c)

Stages in Prim’s Alogrithm (Cont.)

0 0 0
1 1 10
1
10 10
16 14 16

5 6 2 5 6 2 5 6 2

25 25 25 12
12 12
4 4 4
22 3 22 3 22 3

(d) (e) (f)

 Sollin’s Algorithm

• Contrast to Kruskal’s and Prim’s algorithms,

Sollin’s algorithm selects multiple edges at each
stage.
• At the beginning, the selected edges and all the n
vertices form a spanning forest.
• During each stage, an minimum-cost edge is
selected for each tree in the forest.
• It’s possible that two trees in the forest to
select the same edge. Only one should be used.
• Also, it’s possible that the graph has multiple
edges with the same cost. So, two trees may
select two different edges that connect them
together. Again, only one should be retained.
Stages in Sollin’s Algorithm

0 0

1 10 1
10
14 14 16

5 6 2 5 6 2
25
12 12
4 4
3 22 3
22

(a) (b)
 Shortest Paths
• Usually, the highway structure can be
represented by graphs with vertices
representing cities and edges representing
sections of highways.
• Edges may be assigned weights to
represent the distance or the average
driving time between two cities connected
by a highway.
• Often, for most drivers, it is desirable to
find the shortest path from the
originating city to the destination city.
 Single Source/All Destinations:
Nonnegative Edge Costs
• Let S denotes the set of vertices to which the
shortest paths have already been found.
1) If the next shortest path is to vertex u, then the path
begins at v, ends at u, and goes through only vertices
that are in S.
2) The destination of the next path generated must be
the vertex u that has the minimum distance among all
vertices not in S.
3) The vertex u selected in 2) becomes a member of S.
• The algorithm is first given by Edsger Dijkstra.
Therefore, it’s sometimes called Dijstra
Algorithm.
Single Source/All Destinations: General
Weights

• When negative edge lengths are permitted, the graph must not
have cycles of negative length.
• When there are no cycles of negative length, there is a shortest
path between any two vertices of an n-vertex graph that has at
most n-1 edges on it.
1. If the shortest paht from v to u with at most k, k > 1, edges has no
more than k – 1 edges, then distk[u] = distk-1[u].
2. If the shortest path from v to u with at most k, k > 1, edges has
exactly k edges, then it is comprised of a shortest path from v to
some vertex j followed by the edge <j, u>. The path from v to j has k –
1 edges, and its length is distk-1[j].
• The distance can be computed in recurrence by the following:

{ d is t k − 1 [ i ] + le n g th [ i ] [ u ] } ¿ ¿ ¿
• The algorithm is also referred to as the Bellman and Ford
Algorithm.
 Graph and Shortest Paths From Vertex
0 to all destinations

50 10
0 1 2 Path Length

35
15 1) 0, 3 10
10 20 20 30
2) 0, 3, 4 25

3 4 5 3) 0, 3, 4, 1 45
15 3
4) 0, 2 45

(a) Graph (b) Shortest paths from 0

 Diagram for Example 6.5
Chicago 1500 0 Boston

1200
0 1000 250
San Francisco
800 0 New York
0 0
Denver 1400
300 1000 900
0 1700
0 1000
Los Angeles New Orleans 0 Miami

[ ]
0 1 2 3 4 5 6 7
0 0
1 300 0
2 1000 800 0
3 1200 0
4 0 1500 0 250
5 0 1000 0 900 1400
6 0 0 1000
7 1700 0 0
 Action of Shortest Path

iteration S Vertex Distance

selected
LA SF DEN CHI BOST NY MIA NO
[0] [1] [2] [3] [4] [5] [6] [7]

Initial -- --- +∞ +∞ +∞ 1500 0 250 +∞ +∞

1 {4} 5 +∞ +∞ +∞ 1250 0 250 1150 1650

2 {4,5} 6 +∞ +∞ +∞ 1250 0 250 1150 1650

3 {4,5,6} 3 +∞ +∞ 2450 1250 0 250 1150 1650

4 {4,5,6,3} 7 3350 +∞ 2450 1250 0 250 1150 1650

5 {4,5,6,3,7} 2 3350 3250 2450 1250 0 250 1150 1650

6 {4,5,6,3,7, 1 3350 3250 2450 1250 0 250 1150 1650

2}
{4,5,6,3,7,
2,1}
 Directed Graphs

7 -5
0 1 2

(a) Directed graph with a negative-length edge

-2

0 1 2
1 1

(b) Directed graph with a cycle of negative length

 Shortest Paths with
Negative Edge Lengths
k distk[7]

0 1 2 3 4 5 6
1 4
1 0 6 5 5 ∞ ∞ ∞

2 0 3 3 5 5 4 ∞
0 2 6
3 0 1 3 5 2 4 7

3 5 4 0 1 3 5 0 4 5

5 0 1 3 5 0 4 3

(a) A directed graph 6 0 1 3 5 0 4 3

(b) distk
 All-Pairs Shortest Paths
• In all-pairs shortest-path problem, we are
to find the shortest paths between all
pairs of vertices u and v, u ≠ v.
– Use n independent single-source/all-destination
problems using each of the n vertices of G as a
source vertex. Its complexity is O(n3) (or O(n2
logn + ne) if Fibonacci heaps are used).
– On graphs with negative edges the run time will
be O(n4). if adjacency matrices are used and
O(n2e) if adjacency lists are used.
 All-Pairs Shortest Paths
(Cont.)
• A simpler algorithm with complexity O(n 3) is available. It works
faster when G has edges with negative length, as long as the
graphs have at least c*n edges for some suitable constant c.
– An-1[i][j]: the length of the shortest i-to-j path in G
– Ak[i][j]: the length of the shortest path from I to j going through no
intermediate vertex of index greater than k.
– A-1[i][j]: is just the length[i][j]
1. The shortest path from i to j going through no vertex with index
greater than k does not go through the vertex with index k. so its
length is Ak-1[i][j].
2. The shortest path goes through vertex k. The path consists of
subpath from i to k and another one from k to j.

Ak[i][j] = min{Ak-1[i][j], Ak-1[i][k]+ Ak-1[k][j] }, k ≥ 0

 Example for All-Pairs Shortest-Paths
Problem

A-1 0 1 2 A0 0 1 2
0 0 4 11 0 0 4 11
6
1 6 0 2 1 6 0 2

0 1 2 3 ∞ 0 2 3 7 0
4
(b) A-1 (c) A0
11
2
3
2 A1 0 1 2 A2 0 1 2
0 0 4 6 0 0 4 6
1 6 0 2 1 5 0 2
2 3 7 0 2 3 7 0

(d) A1 (e) A2
 Transitive Closure
• Definition: The transitive closure matrix,
denoted A+, of a graph G, is a matrix such
that A+[i][j] = 1 if there is a path of length
> 0 fromi to j; otherwise, A*[i][j] = 0.
• Definition: The reflexive transitive closure
matrix, denoted A*, of a graph G, is a
matrix such that A*[i][j] = 1 if there is a
path of length 0 from i to j; otherwise,
A*[i][j] = 0.
Graph G and Its Adjacency Matrix A, A+,
A*

0 1 2 3 4

[ ]
0 0 1 0 0 0
0 1 2 3 4 1 0 0 1 0 0
2 0 0 0 1 0
(a) Digraph G 3 0 0 0 0 1
4 0 0 0 0 0

(b) Adjacency matrix A

0 1 2 3 4

[ ]
0 1 2 3 4

[ ]
0 0 1 1 1 1
1 0 0 1 1 1 0 1 1 1 1 1
2 0 0 1 1 1 1 0 1 1 1 1
3 0 0 1 1 1 2 0 0 1 1 1
4 0 0 1 1 1 3 0 0 1 1 1
4 0 0 1 1 1
(c) A+
(d) A*
 Activity-on-Vertex (AOV) Networks

• Definition: A directed graph G in which the

vertices represent tasks or activities and the
edges represent precedence relations between
tasks is an activity-on-vertex network or AOV
network.
• Definition: Vertex i in an AOV network G is a
predecessor of vertex j iff there is a directed
path from vertex i to vertex j. i is an immediate
predecessor of j iff <i, j> is an edge in G. If i is a
predecessor of j, then j is an successor of i. If i is
an immediate predecessor of j, then j is an immediate
successor of i.
 Activity-on-Vertex (AOV) Networks
(Cont.)
• Definition: A relation · is transitive iff it is
the case that for all triples, i, j, k, i.j and
j·k => i·k. A relation · is irreflexive on a set S
if for no element x in S it is the case that
x·x. A precedence relation that is both
transitive and irreflexive is a partial order.
• Definition: A topological order is a linear
ordering of the vertices of a graph such
that, for any two vertices I and j, if I is a
predecessor of j in the network, then i
precedes j in the linear ordering.
 An Activity-on-Vertex (AOV) Network

Course number Course name Prerequisites

C1 Programming I None
C2 Discrete Mathematics None
C3 Data Structures C1, C2
C4 Calculus I None
C5 Calculus II C4
C6 Linear Algebra C5
C7 Analysis of Algorithms C3, C6
C8 Assembly Language C3
C9 Operating Systems C7, C8
C10 Programming Languages C7
C11 Compiler Design C10
C12 Artificial Intelligence C7
C13 Computational Theory C7
C14 Parallel Algorithms C13
C15 Numerical Analysis C5
An Activity-on-Vertex (AOV) Network
(Cont.)

C9

C10 C11
C1

C8 C12
C2

C3 C7 C13 C14

C4 C5 C6 C15
Figure 6.36 Action of Program 6.11
on an AOV network

1 1 1

0 2 4 2 4 2 4

3 5 3 5 5

(a) Initial (b) Vertex 0 deleted (c) Vertex 3 deleted

1 1

4 4
4

(d) Vertex 2 deleted (e) Vertex 5 deleted (f) Vertex 1 deleted

Figure 6.37 Internal representation
used by topological sorting algorithm

count first data link

[0 0 1 2 3 0
][1
1 4 0
]
[2 1 4 5 0
]
[3 1 5 4 0
]
[4 3 0
][5
2 0
]
 An AOE Network

a1 = 6
1 6 a10 = 2
a4 = 6
a7= 9
start 0 4 8 finish
a2 = 4
a5 = 1 a11 = 4
2 7
a3 = 5
a9 = 4

3 5
a6 = 2

event interpretation
0 Start of project
1 Completion of activity a1
4 Completion of activities a4 and a5
7 Completion of activities a8 and a9
8 Completion of project
 Adjacency lists for Figure 6.38 (a)

count first vertex dur link

[0 0 1 6 2 4 3 5 0
][1
1 4 1 0
]
[2 1 4 1 0
]
[3 1 5 2 0
]
[4 3 6 9 7 7 0
][5
2 7 4 0
]
[6 2 8 2 0
]
[7 2 8 4 0
]
[8 2 0
]
 Computation of ee

ee [0] [1] [2] [3] [4] [5] [6] [7] [8] Stack

Initial 0 0 0 0 0 0 0 0 0 [0]

output 0 0 6 4 5 0 0 0 0 0 [3,2,1]

output 3 0 6 4 5 0 7 0 0 0 [5,2,1]

output 5 0 6 4 5 0 7 0 11 0 [2,1]

output 2 0 6 4 5 0 7 0 11 0 [1]

output 1 0 6 4 5 5 7 0 11 0 [4]

output 4 0 6 4 5 7 7 0 14 0 [7,6]

output 7 0 6 4 5 7 7 16 14 18 [6]

output 6 0 6 4 5 7 7 16 14 18 [8]

output 8
Chap 7

Sorting
 Motivation of Sorting
• The term list here is a collection of records.
• Each record has one or more fields.
• Each record has a key to distinguish one
record with another.
• For example, the phone directory is a list.
Name, phone number, and even address can
be the key, depending on the application or
need.
 Sorting
• Two ways to store a collection of records
– Sequential
– Non-sequential
• Assume a sequential list f. To retrieve a
record with key f[i].key from such a list, we
can do search in the following order:
f[n].key, f[n-1].key, …, f[1].key => sequential search
Example of An Element of A Search List
class Element
{
public:
int getKey() const {return key;};
void setKey(int k) {key = k;};
private:
int key;
// other records
…
}
 Sequential Search

int SeqSearch (Element *f, const int n, const int k)

// Search a list f with key values
// f[1].key, …, f[n].key. Return I such
// that f[i].key == k. If there is no such record,
return 0
{
int i = n;
f[0].setKey(k);
while (f[i].getKey() != k) i--;
return i;
}
 Sequential Search
• The number of comparisons for a record key i
is n – i +1.
• The average number of comparisons for a
successful search is
∑ ( n −i+ 1 )/ n= ( n+ 1 )/ 2
1≤ i ≤ n

• For the phone directory lookup, there should

be a better way than this.
 Search
• A binary search only takes O(log n) time to search
a sequential list with n records.
• In fact, if we look up the name start with W in the
phone directory, we start search toward the end
of the directory rather than the middle. This
search method is based on interpolation scheme.
k−f [l ]. key
i=( )∗n
f [u ]. key −f [ l]. key

• An interpolation scheme relies on a ordered list.

 Verifying Two Lists With Sequential
Search

void Verify1(Element* F1, Element* F2, const int n, const int m)

// Compare two unordered lists F1 and F2 of size n and m, respectively
{
Boolean *marked = new Boolean[m];
for (int i = 1; i <= m; i++) marked[i] = FALSE;
O(mn)
for (i = 1; i<= n; i++)
{
int j = SeqSearch(F2, m, F1[i].getKey());
if (j == 0) cout << F1[i].getKey() <<“not in F2 “ << endl;
else
{
if (F1[i].other != F2[j].other)
cout << “Discrepancy in “<<F[i].getKey()<<“:”<<F1[i].other
<< “and “ << F2[j].other << endl;
marked[j] = TRUE; // marked the record in F2[j] as being seen
}
}
for (i = 1; i <= m; i++)
if (!marked[i]) cout << F2[i].getKey() <<“not in F1. “ << endl;
delete [ ] marked;
}
 Fast Verification of Two Lists

void Verify2(Element* F1, Element* F2, const int n, const int m)

// Same task as Verfy1. But sort F1 and F2 so that the keys are in
// increasing order. Assume the keys in each list are identical
{
sort(F1, n);
sort(F2, m); O(max{n log n, m log m})
int i = 1; int j = 1;
while ((i <= n) && (j <= m))
switch(compare(F1[i].getKey(), F2[j].getKey()))
{
case ‘<‘: cout<<F1[i].getKey() <<“not in F2”<< endl;
i++; break;
case ‘=‘: if (F1[i].other != F2[j].other)
cout << “Discrepancy in “ << F1[i].getKey()<<“:”
<<F1[i].other<<“ and “<<F2[j].other << endl;
i++; j++; break;
case ‘>’: cout <<F2[j].getKey()<<“ not in F1”<<endl;
j++;
}
if (i <= n)PrintRest(F1, i, n, 1); //print records I through n of F1
else if (j <= m) PrintRest(F2, j, m, 2); // print records j through m of F2
}
 Formal Description of Sorting
• Given a list of records (R1, R2, …, Rn).
Each record has a key Ki. The sorting
problem is then that of finding
permutation, σ, such that
Kσ(i) ≤ K σ(i+1) , 1 ≤ i ≤ n – 1. The desired
ordering is (Rσ(1), Rσ(2), Rσ(n)).
Formal Description of Sorting
(Cont.)
• If a list has several key values that are
identical, the permutation, σs, is not unique.
Let σs be the permutation of the following
properties:
(1) Kσ(i) ≤ K σ(i+1) , 1 ≤ i ≤ n – 1
(2) If i < j and Ki == Kj in the input list, then Ri
precedes Rj in the sorted list.
• The above sorting method that generates σs
is stable.
 Categories of Sorting Method
• Internal Method: Methods to be used when
the list to be sorted is small enough so that
the entire sort list can be carried out in the
main memory.
– Insertion sort, quick sort, merge sort, heap sort
and radix sort.
• External Method: Methods to be used on
larger lists.
 Insert Into A Sorted List
void insert(const Element e, Element* list, int i)
// Insert element e with key e.key into the ordered sequence list[0], …, list[i] such that the
// resulting sequence is also ordered. Assume that e.key ≥ list[0].key. The array list must
// have space allocated for at least i + 2 elements
{
while (e.getKey() < list[i].getKey())
{
list[i+1] = list[i];
i--;
}
list[i+1] = e;
} O(i)
 Insertion Sort

void InsertSort(Element* list, const int n)

// Sort list in nondecreasing order of key
{
list[0].setKey(MININT);
for (int j = 2; j <= n; j++)
insert(list[j], list, j-1);
}

n−1
O( ∑ (i+1 )=O(n 2 )
i=1
 Insertion Sort Example 1

• Record Ri is left out of order (LOO) iff Ri < { R j} ¿

• Example 7.1: Assume n = 5 and the input key sequence is 5,
4, 3, 2, 1

j [1] [2] [3] [4] [5]

- 5 4 3 2 1
2 4 5 3 2 1
3 3 4 5 2 1
4 2 3 4 5 1
5 1 2 3 4 5
 Insertion Sort Example 2

• Example 7.2: Assume n = 5 and the input key sequence is 2,

3, 4, 5, 1

j [1] [2] [3] [4] [5]

- 2 3 4 5 1
2 2 3 4 5 1 O(1)
3 2 3 4 5 1 O(1)
4 2 3 4 5 1 O(1)
5 1 2 3 4 5 O(n)
 Insertion Sort Ananlysis
• If there are k LOO records in a list,
the computing time for sorting the list
via insertion sort is O((k+1)n) = O(kn).
• Therefore, if k << n, then insertion sort
might be a good sorting choice.
 Insertion Sort Variations
• Binary insertion sort: the number of
comparisons in an insertion sort can be
reduced if we replace the sequential search
by binary search. The number of records
moves remains the same.
• List insertion sort: The elements of the list
are represented as a linked list rather than
an array. The number of record moves
becomes zero because only the link fields
require adjustment. However, we must retain
the sequential search.
 Quick Sort
• Quick sort is developed by C. A. R. Hoare.
The quick sort scheme has the best average
behavior among the sorting methods.
• Quick sort differs from insertion sort in
that the pivot key Ki is placed at the correct
spot with respect to the whole list. Kj ≤ Ks(i)
for j < s(i) and Kj ≥ s(i) for j > s(i).
• Therefore, the sublist to the left of S(i) and
to the right of s(i) can be sorted
independently.
 Quick Sort

void QuickSort(Element* list, const int left, const int right)

// Sort records list[left], …, list[right] into nondecreasing order on field key. Key pivot = list[left].key is
// arbitrarily chosen as the pivot key. Pointers I and j are used to partition the sublist so that at any time
// list[m].key pivot, m < I, and list[m].key pivot, m > j. It is assumed that list[left].key ≤ list[right+1].key.
{
if (left < right) {
int i = left, j = right + 1, pivot = list[left].getKey();
do {
do i++; while (list[i].getKey() < pivot);
do j--; while (list[j].getKey() > pivot);
if (i<j) InterChange(list, i, j);
} while (i < j);
InterChange(list, left, j);
QuickSort(list, left, j–1);
QuickSort(list, j+1, right);
}
}
 Quick Sort Example
• Example 7.3: The input list has 10 records with keys (26, 5,
37, 1, 61, 11, 59, 15, 48, 19).

R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Left Right

[26 5 37 1 61 11 59 15 48 19 1 10

[11 5 19 1 15] 26 [59 61 48 37] 1 5

[1 5] 11 [19 15] 26 [59 61 48 37] 1 2

1 5 11 [19 15] 26 [59 61 48 37] 4 5

1 5 11 15 19 26 [59 61 48 37] 7 10

1 5 11 15 19 26 [48 37] 59 [61] 7 8

1 5 11 15 19 26 37 48 59 [61] 10 10

1 5 11 15 19 26 37 48 59 61
 Quick Sort (Cont.)
• In QuickSort(), list[n+1] has been set to have a key
at least as large as the remaining keys.
• Analysis of QuickSort
– The worst case O(n2)
– If each time a record is correctly positioned, the sublist of
its left is of the same size of the sublist of its right.
Assume T(n) is the time taken to sort a list of size n:
T(n) ≤ cn + 2T(n/2), for some constant c
≤ ≤ cn + 2(cn/2 +2T(n/4))
≤ 2cn + 4T(n/4)
:
:
≤ cn log2n + T(1) = O(n logn)
 Lemma 7.1
• Lemma 7.1: Let Tavg(n) be the expected
time for function QuickSort to sort a
list with n records. Then there exists a
constant k such that Tavg(n) ≤ kn logen
for n ≥ 2.
 Analysis of Quick Sort
• Unlike insertion sort (which only needs additional
space for a record), quick sort needs stack space to
implement the recursion.
• If the lists split evenly, the maximum recursion
depth would be log n and the stack space is of O(log
n).
• The worst case is when the lists split into a left
sublist of size n – 1 and a right sublist of size 0 at
each level of recursion. In this case, the recursion
depth is n, the stack space of O(n).
• The worst case stack space can be reduced by a
factor of 4 by realizing that right sublists of size
less than 2 need not be stacked. Asymptotic
reduction in stack space can be achieved by sorting
smaller sublists first. In this case the additional
stack space is at most O(log n).
 Quick Sort Variations
• Quick sort using a median of three:
Pick the median of the first, middle,
and last keys in the current sublist as
the pivot. Thus, pivot = median{Kl,
K(l+r)/2, Kr}.
 Decision Tree
• So far both insertion sorting and quick
sorting have worst-case complexity of O(n2).
• If we restrict the question to sorting
algorithms in which the only operations
permitted on keys are comparisons and
interchanges, then O(n logn) is the best
possible time.
• This is done by using a tree that describes
the sorting process. Each vertex of the tree
represents a key comparison, and the
branches indicate the result. Such a tree is
called decision tree.
 Decision Tree for Insertion Sort

Yes K1 ≤ K 2 No

K2 ≤ K 3 K1 ≤ K 3

Yes No Yes No

stop K1 ≤ K 3 stop K2 ≤ K 2

Yes No IV Yes No
I

stop stop stop stop

II III V VI
 Decision Tree (Cont.)
• Theorem 7.1: Any decision tree that
sorts n distinct elements has a height
of at least log2(n!) + 1

• Corollary: Any algorithm that sorts only

by comparisons must have a worst-case
computing time of Ω(n log n)
 Simple Merge
void merge(Element* initList, Element* mergeList, const int l, const int m,
const int n)
{
for (int i1 =l,iResult = l, i2 = m+1; i1<=m && i2<=n; iResult++){
if (initList[i1].getKey() <= initList[i2].getKey()) {
mergeList[iResult] = initList[i1];
i1++;
}
else { O(n - l + 1)
mergeList[iResult] = initList[i2];
i2++;
}
}
if (i1 > m)
for (int t = i2; t <= n; t++)
mergeList[iResult + t - i2] = initList[t];
else
for (int t = i1; t <= m; t++)
mergeList[iResult + t - i1] = initList[t];
}
 Analysis of Simple Merge
• If an array is used, additional space for
n – l +1 records is needed.
• If linked list is used instead, then
additional space for n – l + 1 links is
needed.
 O(1) Space Merge

• A second merge algorithm only requires

O(1) additional space.
• Assume total of n records to be merged
into a list, where n is a perfect square. And
the numbers of records in the left sublist
and the right sublist are multiple of √ n
 O(1) Space Merge Steps
Step 1: Identify the √ n records with largest keys. This is
done by following right to left along the two lists to be
merged.
Step 2: Exchange the records of the second list that were
identified in Step 1 with those just to the left of those
identified from the first list so that the √ n records with
largest keys are contiguous.
Step 3: Swap the block of √ n largest with the leftmost
block (unless it is already the leftmost block). Sort the
rightmost block.
Step 4: Reorder the blocks, excluding the block of largest
records, into nondecreasing order of the last key in the
blocks.
Step 5: Perform as many merge substeps as needed to
merge the √ n−1 blocks, other than the block with the
largest keys.
Step 6: Sort the block with the largest keys.
 O(1) Space Merge Example (First 8
Lines)

0 2 4 6 8 a c e g i j k l m n t w z|1 3 5 7 9 b d f h o p q r s u v x y

0 2 4 6 8 a c e g i j k l m n t w z 1 3 5 7 9 b d f h o p q r s u v x y

0 2 4 6 8 a|c e g i j k|u v x y w z|1 3 5 7 9 b|d f h o p q|r s l m n t

u v x y w z|c e g i j k|0 2 4 6 8 a|1 3 5 7 9 b|d f h o p q|l m n r s t

u v x y w z 0 2 4 6 8 a|1 3 5 7 9 b|c e g I j k|d f h o p q|l m n r s t

0 v x y w z u 2 4 6 8 a|1 3 5 7 9 b|c e g I j k|d f h o p q|l m n r s t

0 1 x y w z u 2 4 6 8 a|v 3 5 7 9 b|c e g I j k|d f h o p q|l m n r s t

0 1 2 y w z u x 4 6 8 a|v 3 5 7 9 b|c e g I j k|d f h o p q|l m n r s t

O(1) Space Merge Example (Last 8 Lines)

0 1 2 3 4 5 u x w 6 8 a|v y z 7 9 b|c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 u w a|v y z x 9 b|c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a w|v y z x u b|c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a w v y z x u b c e g i j k|d f h o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k v z u|y x w o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k v z u y x w o p q|l m n r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q y x w|v z u r s t

0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t|v z u y x w
 Analysis of O(1) Space Merge
• Step 1 and 2 and the swapping of Step 3 each take
O( √ n ) time and O(1) space.
• The sort of Step 3 can be done in O(n) time and
O(1) space using an insertion sort.
• Step 4 can be done in O(n) time and O(1) space
using a selection sort. (Selection sort sorts m
records using O(m2) key comparisons and O(m)
record moves. So it needs O(n) comparisons and the
time to move blocks is O(n).
• If insertion sort is used in Step 4, then the time
becomes O(n1.5) since insertion sort needs O(m2)
record moves ( √ n records per block * n record
moves).
 Analysis of O(1) Space Merge
(Cont.)

• The total number of merge substeps is at

most √ n−1 . The total time for Step 5 is
O(n).
• The sort of Step 6 can be done in O(n) by
using either a selection sort or an
insertion sort.
• Therefore, the total time is O(n) and the
additional space used is O(1).
 Iterative Merge Sort
• Treat the input as n sorted lists, each
of length 1.
• Lists are merged by pairs to obtain n/2
lists, each of size 2 (if n is odd, the one
list is of length 1).
• The n/2 lists are then merged by pairs,
and so on until we are left with only one
list.
 Merge Tree
26 5 77 1 61 11 59 15 48 19

5 26 1 77 11 61 15 59 19 48

1 5 26 77 11 15 59 61 19 48

1 5 11 15 26 59 61 77 19 48

1 5 11 15 19 26 48 59 61 77
 Iterative Merge Sort

void MergeSort(Element* list, const int n)

// Sort list list into non-decreasing order of the keys list[1].key, …,list[n].key.
{
Element* tempList = new Element[n+1];
// l is the length of the sublist currently being merged.
for (int l = 1; l < n; l *= 2)
{
MergePass(list, tempList, n, l);
l *= 2;
MergePass(tempList, list, n, l); //interchange role of list and tempList
}
delete[ ] tempList;
}
 Merge Pass

void MergePass(Element* initList, Elemen* resultList, const int n,

const int l)
// One pass of merge sort. Adjacent pairs of sublists of length l are merged
// from list initList to list resultList. n is the number of records in initList
{
for (int i = 1;
i <= n – 2*l + 1; // Are enough elements remaining to form two sublists of length l?
i += 2*l)
merge(initList, resultList, i, i + l - 1, i + 2*l – 1);

// merge remaining list of length < 2 * l

if ((i + l – 1) < n) merge(initList, resultList, i, i+l–1, n);
else for (int t = i; t <= n; t++) resultList[t] = initList[t];
}
 Analysis of MergeSort

• Total of ( log2 n ) passes are made over the

data. Each pass of merge sort takes O(n)
time.
• The total of computing time is O(n log n)
 Recursive Merge Sort
• Recursive merge sort divides the list to be
sorted into two roughly equal parts:
– the left sublist [left : (left+right)/2]
– the right sublist [(left+right)/2 +1 : right]
• These sublists are sorted recursively, and
the sorted sublists are merged.
• To avoid copying, the use of a linked list
(integer instead of real link) for sublist is
desirable.
Sublist Partitioning For Recursive Merge
Sort

26 5 77 1 61 11 59 15 48 19

5 26 11 59 19 48

5 26 77 1 61 11 15 59 19 48

1 5 26 61 77 11 15 19 48 59

1 5 11 15 19 26 48 59 61 77
 Program 7.11 (Recursive
Merge Sort )

class Element
{
private:
int key;
Field other;
int link;
public:
Element() {link = 0;};
};

int rMergeSort(Element* list, const int left, const int right)

// List list = (list[left], …, list[right]) is to be sorted on the field key.
// link is a link field in each record that is initially 0
// list[0] is a record for intermediate results used only in ListMerge
{
if (left >= right) return left;
O(n log n)
int mid = (left + right)/2;
return ListMerge(list, rMergeSort(list, left, mid),
rMergeSort(list, mid+1, right));
}
Program 7.12 (Merging Linked
Lists)

int ListMerge(Element* list, const int start1, const int start2)

{
int iResult = 0;
for (int i1 = start1, i2 = start2; i1 && i2;){
if (list[i1].key <= list[i2].key) {
list[iResult].link = i1;
iResult = i1; i1 = list[i1].link;
}
else {
list[iResult].link = i2;
iResult = i2; i2 = list[i2].link;
}
}
// move remainder
if (i1 == 0) list[iResult].link = i2;
else list[iResult] = i1;
return list[0].link;
}
 Natural Merge Sort
• Natural merge sort takes advantage of
the prevailing order within the list
before performing merge sort.
• It runs an initial pass over the data to
determine the sublists of records that
are in order.
• Then it uses the sublists for the merge
sort.
 Natural Merge Sort Example

26 5 77 1 61 11 59 15 48 19

5 26 77 1 11 59 61 15 19 48

1 5 11 26 59 61 77 15 19 48

1 5 11 15 19 26 48 59 61 77
 Heap Sort
• Merge sort needs additional storage space
proportional to the number of records in the file
being sorted, even though its computing time is O(n
log n)
• O(1) merge only needs O(1) space but the sorting
algorithm is much slower.
• We will see that heap sort only requires a fixed
amount of additional storage and achieves worst-
case and average computing time O(n log n).
• Heap sort uses the max-heap structure.
 Heap Sort (Cont.)
• For heap sort, first of all, the n
records are inserted into an empty
heap.
• Next, the records are extracted from
the heap one at a time.
• With the use of a special function
adjust(), we can create a heap of n
records faster.
 Program 7.13 (Adjusting A
Max Heap)

void adjust (Element* tree, const int root, const int n)

// Adjust the binary tree with root root to satisfy the heap property. The left and right subtrees of root
// already satisfy the heap property. No node has index greater than n.
{
Element e = tree[root];
int k = e.getKey();
for (int j = 2*root; j <= n; j *= 2)
{
if (j < n)
if (tree[j].getKey() < tree[j+1].getKey()) j++;
// compare max child with k. If k is max, then done.
if (k >= tree[j].getKey()) break;
tree[j/2] = tree[j]; // move jth record up the tree
}
tree[j/2] = e;
}
 Program 7.14 (Heap Sort)

void HeapSort (Element* list, const int n)

// The list list = (list[1], …, list[n]) is sorted into nondecreasing order of the field key.
{
for (int i = n/2; i >= 1; i--) // convert list into a heap
adjust(list, i, n); ∑ 2 i− 1 ( k −i )= ∑ 2 k −i − 1 i≤n ∑ i / 2 i <2n =O ( n )
1≤ i ≤ k 1≤ i≤ k 1 ≤ i≤ k

for (i = n-1; i >= 1; i--) // sort list

{
Element t = list[i+1]; // interchange list1 and list i+1
list[i+1] = list[1];
list[1] = t;
adjust(list, 1, i); O(n log
} n)
}
 Converting An Array Into A Max Heap

[1] 26 [1] 77

[2 5 [3 77 [2 61 [3 59
] ] ] ]

[4] 1 [5] 61 11 59 48 [4] [5] 19 11 26

[6] [7] [6] [7]

15 48 19 15 1 5
[8] [9] [10 [8] [9] [10
] ]
(a) Input array (b) Initial heap
 Heap Sort Example
[1] 61 [1] 59

[2 48 [3 59 [2 48 [3 26
] ] ] ]

[4] 15 [5] 19 11 26 15 [5] 19 11 1

[6] [7] [6] [7]

5 1 5

[8] [9] [8]

Heap size = 9, Heap size = 8,

Sorted = [77] Sorted = [61, 77]
 Sorting Several Keys
• A list of records are said to be sorted with respect to
the keys K1, K2, …, Kr iff for every pair of records i and j,
i < j and (K1i, K2i, …, Kri) ≤ (K1j, K2j, …, Krj).
• The r-tuple (x1, x2, …, xr) is less than or equal to the r-
tuple (y1, y2, …, yr) iff either xi = yi, 1 ≤ i ≤ j, and xj+1 < yj+1
for some j < r or xi = yi , 1 ≤ i ≤ r.
• Two popular ways to sort on multiple keys
– sort on the most significant key into multiple piles. For each
pile, sort on the second significant key, and so on. Then
piles are combined. This method is called sort on most-
significant-digit-first (MSD).
– The other way is to sort on the least significant digit first
(LSD).
• Example, sorting a deck of cards: suite and face value.
– Spade > Heart > Diamond > Club
 Sorting Several Keys (Cont.)
• LSD and MSD only defines the order in which
the keys are to be sorted. But they do not
specify how each key is sorted.
• Bin sort can be used for sorting on each key.
The complexity of bin sort is O(n).
• LSD and MSD can be used even when there is
only one key.
– E.g., if the keys are numeric, then each decimal
digit may be regarded as a subkey. => Radix sort.
 Radix Sort
• In a radix sort, we decompose the sort key using
some radix r.
– The number of bins needed is r.
• Assume the records R1, R2, …, Rn to be sorted
based on a radix of r. Each key has d digits in the
range of 0 to r-1.
• Assume each record has a link field. Then the
records in the same bin are linked together into a
chain:
– f[i], 0 ≤ i ≤ r (the pointer to the first record in bin i)
– e[i], (the pointer to the end record in bin i)
– The chain will operate as a queue.
– Each record is assumed to have an array key[d], 0 ≤
key[i] ≤ r, 0 ≤ i ≤ d.
 Program 7.15 (LSD Radix Sort)
void RadixSort (Element* list, const int d, const int n)
{
int e[radix], f[radix]; // queue pointers
for (int i = 1; i <= n; i++) list[i].link = i + 1; // link into a chain starting at current
list[n].link = 0; int current = 1;
for (i = d – 1; i >= 0; i--) // sort on key key[i] d passes
{
for (int j = 0; j < radix; j++) f[j] = 0; // initialize bins to empty queues
for (; current; current = list[current].link) { // put records into queues
int k = list[current].key[i];
if (f[k] == 0) f[k] = current;
else list[e[k]].link = current; O(n)
e[k] = current;
}
for (j = 0; f[j] == 0; j++); // find the first non-empty queue
current = f[j]; int last = e[j];
for (int k = j+1; k < radix; k++){ // concatenate remaining queues
if (f[k]){
list[last].link = f[k]; O(r)
last = e[k];
}
}
list[last].link = 0;
}
O(d(n+r)
} )
 Radix Sort Example
list[1 list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10
] ]
179 208 306 93 859 984 55 9 271 33

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

33 859

271 93 984 55 306 208 179

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]
list[1 list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10
] ]
271 93 33 984 55 306 208 179 859 9
 Radix Sort Example (Cont.)
list[1 list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10
] ]
271 93 33 984 55 306 208 179 859 9

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

208 859 179

306 33 55 271 984 93

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]

list[1 list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10
] ]
306 208 9 33 55 859 271 179 984 93
 Radix Sort Example (Cont.)
list[1 list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10
] ]
306 208 9 33 55 859 271 179 984 93
e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

93

55

33 271

9 179 208 306 859 984

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]
list[1 list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10
] ]
9 33 55 93 179 208 271 306 859 948
 List And Table Sorts
• Apart from radix sort and recursive merge sort,
all the sorting methods we have looked at so far
require excessive data movement.
• When the amount of data to be sorted is large,
data movement tends to slow down the process.
• It is desirable to modify sorting algorithms to
minimize the data movement.
• Methods such as insertion sort or merge sort
can be modified to work with a linked list rather
than a sequential list. Instead of physical
movement, an additional link field is used to
reflect the change in the position of the record
in the list.
Program 7.16 (Rearranging Records Using
A Doubly Linked List

void list (Element* list, const int n, int first)

{
int prev = 0;
for (int current = first; current; current = list[current].link)
// convert chain into a doubly linked list
{ O(n)
list[current].linkb = prev;
prev = current;
}
for (int i = 1; i < n; i++) // move listfirst to position i while maintaining the list
{
if (first != i) {
if (list[i].link) list[list[i].link].linkb = first;
list[list[i].linkb].link = first;
Element a = list[first]; list[first] = list[i];list[i] = a;
}
first = list[i].link; O(nm Assume each record
} ) is m words
}
 Example 7.9

i
R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 26 5 77 1 61 11 59 15 48 19
link 9 6 0 2 3 8 5 10 7 1

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 26 5 77 1 61 11 59 15 48 19
link 9 6 0 2 3 8 5 10 7 1
linkb 10 4 5 0 7 2 9 6 1 8
 Example 7.9 (Cont.)
i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 77 26 61 11 59 15 48 19
link 2 6 0 9 3 8 5 10 7 4
linkb 0 4 5 10 7 2 9 6 4 8
Configuration after first iteration of the for loop of list1, first = 2

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 77 26 61 11 59 15 48 19
link 2 6 0 9 3 8 5 10 7 1
linkb 0 4 5 10 7 2 9 6 1 8
Configuration after second iteration of the for loop of list1, first = 6
 Example 7.9 (Cont.)
i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 11 26 61 77 59 15 48 19
link 2 6 8 9 6 0 5 10 7 4
linkb 0 4 2 10 7 5 9 6 4 8
Configuration after third iteration of the for loop of list1, first = 8

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 11 15 61 77 59 26 48 19
link 2 6 8 10 6 0 5 9 7 8
linkb 0 4 2 6 7 5 9 10 8 8
Configuration after fourth iteration of the for loop of list1, first = 10
Example 7.10 (Rearranging Records Using
Only One Link)

void list2(Element* list, const int n, int first)

// Same function as list1 except that a second link field linkb is not required
{
for (int i = 1; i < n; i++)
{
while (first < i ) first = list[first].link; O(n)
int q = list[first].link; // listq is next record in nondecreasing order
if (first != i)
// interchange listi and listfirst moving listfirst to its correct spot as listfirst has ith smallest key.
// Also set link from old position of list i to new one
{
Element t = list[i];
list[i] = list[first]; list[first] = t; list[i].link = first;
}
first = q;
} O(nm)
}
Example 7.10 (Rearranging Records Using
Only One Link)

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 77 26 61 11 59 15 48 19
link 4 6 0 9 3 8 5 10 7 1
Configuration after first iteration of the for loop of list1, first = 2

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 77 26 61 11 59 15 48 19
link 4 6 0 9 3 8 5 10 7 1
Configuration after second iteration of the for loop of list1, first = 6
Example 7.10 (Rearranging Records Using
Only One Link)

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 11 26 61 77 59 15 48 19
link 4 6 6 9 3 0 5 10 7 1
Configuration after third iteration of the for loop of list1, first = 8

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 11 15 61 77 59 26 48 19
link 4 6 6 8 3 0 5 9 7 1
Configuration after fourth iteration of the for loop of list1, first = 10
Example 7.10 (Rearranging Records Using
Only One Link)

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 11 15 19 77 59 26 48 61
link 4 6 6 8 10 0 5 9 7 3
Configuration after fifth iteration of the for loop of list1, first = 1

i R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
key 1 5 11 15 19 26 59 77 48 61
link 4 6 6 8 1 8 5 0 7 3
Configuration after sixth iteration of the for loop of list1, first = 9
 Table Sort
• The list-sort technique is not well suited for quick
sort and heap sort.
• One can maintain an auxiliary table, t, with one entry
per record. The entries serve as an indirect
reference to the records.
• Initially, t[i] = i. When interchanges are required,
only the table entries are exchanged.
• It may be necessary to physically rearrange the
records according to the permutation specified by t
sometimes.
 Table Sort (Cont.)
• The function to rearrange records
corresponding to the permutation t[1],
t[2], …, t[n] can be considered as an
application of a theorem from
mathematics:
– Every permutation is made up of disjoint
cycles. The cycle for any element i is made
up of i, t[i], t2[i], …, tk[i], where tj[i]=t[tj-
1
[i]], t0[i]=i, tk[i]=i.
 Program 7.18 (Table Sort)

void table(Element* list, const int n, int *t)

{
for (int i = 1; i < n; i++) {
if (t[i] != i) {
Element p = list[i]; int j = i;
do {
int k = t[j]; list[j] = list[k]; t[j] = j;
j = k;
} while (t[j] != i)
list[j] = p;
t[j] = j;
}
}
}
 Table Sort Example
4
5

R1 R2 R3 R4 R5 R6 R7 R8
key 35 14 12 42 26 50 31 18

t 3 2 8 5 7 1 4 6
Initial configuration
1 2 3

key 12 14 18 42 26 35 31 50

t 1 2 3 5 7 6 4 8

Configuration after rearrangement of first cycle

key 12 14 18 26 31 35 42 50

t 1 2 3 4 5 6 7 8

Configuration after rearrangement of second cycle

 Analysis of Table Sort
• To rearrange a nontrivial cycle including k distinct
records one needs k+1 moves. Total of record moves
n−1

∑ (k l +1)
l=0, kl ≠0

• Since the records on all nontrivial cycles must be

different, then ∑ k l ≤n
• The total number of record moves is maximum
when ∑ k l =n and there are ⌊ n/2 ⌋ cycles.
• Assume that one record move costs O(m) time, the
total computing time is O(mn).
 Summary Of Internal Sorting
• No one method is best for all conditions.
– Insertion sort is good when the list is already
partially ordered. And it is best for small number
of n.
– Merge sort has the best worst-case behavior but
needs more storage than heap sort.
– Quick sort has the best average behavior, but its
worst-case behavior is O(n2).
– The behavior of radix sort depends on the size of
the keys and the choice of r.
 External Sorting
• There are some lists that are too large to fit in the
memory of a computer. So internal sorting is not
possible.
• Some records are stored in the disk (tape, etc.).
System retrieves a block of data from a disk at a
time. A block contains multiple records.
• The most popular method for sorting on external
storage devices is merge sort.
– Segments of the input list are sorted.
– Sorted segments (called runs) are written onto external
storage.
– Runs are merged until only run is left.
 Example 7.12
• Consider a computer which is capable of sorting 750
records is used to sort 4500 records.
• Six runs are generated with each run sorting 750
records.
• Allocate three 250-record blocks of internal
memory for performing merging runs. Two for input
run 2 runs and the last one is for output.
• Three factors contributing to the read/write time
of disk:
– seek time
– latency time
– transmission time
 Example 7.12 (Cont.)

run2 (751 – run3 (1501 – run4 (2251 – run5 (3001 – run6 (3751 –
run1 (1 – 750) 2250) 3750) 4500)
1500) 3000)
 Example 7.12 (Cont.)
• tIO = ts + tl +trw
tIS = time to internal sort 750 records
ntm = time to merge n records from
input buffers to the output buffer
ts = maximum seek time
tl = maximum latency time
trw = time to read or write on block of 250 records
 Example 7.12 (Cont.)
Operation Time
(1) Read 18 blocks of input, 18tIO, 36tIO + 6tIS
internally sort, 6tIS , write 18
blocks, 18tIO

(2) Merge runs 1 to 6 in pairs 36tIO + 4500tm

(3) Merge two runs fo 1500 24tIO + 3000tm

records each, 12 blocks
(4) Merge one run of 3000 records 36tIO + 4500tm
with one run of 1500 records

Total time 132tIO + 12000tm+ 6tIS

 K-Way Merging
• To merge m runs via 2-way merging will need
passes.
• If we use higher order merge, the number of( log2 m ) +1
passes over would be reduced.
• With k-way merge on m runs, we need
passes over.
( logk mof
• But is it always true that the higher order )
merging, the less computing time we will have?
– Not necessary!
– k-1 comparisons are needed to determine the next
output.
– If loser tree is used to reduce the number of
comparisons, we can achieve complexity of O(n log2 m)
– The data block size reduced as k increases. Reduced
block size implies the increase of data passes over
 Buffer Handling for Parallel Operation
• To achieve better performance, multiple
input buffers and two output buffers are
used to avoid idle time.
• Evenly distributing input buffers among all
runs may still have idle time problem. Buffers
should be dynamically assigned to whoever
needs to retrieve more data to avoid halting
the computing process.
• We should take advantage of task
overlapping and keep computing process busy
and avoid idle time.
 Buffering Algorithm
Step 1: Input the first block of each of the k runs, setting up
k linked queues, each having one block of data.
Step 2: Let LastKey[i] be the last key input from run i. Let
NextRun be the run for which LastKey is minimum.
Step 3: Use a function kwaymerge to merge records from the
k input queues into the output buffer.
Step 4: Wait for any ongoing disk I/O to complete.
Step 5: If an input buffer has been read, add it to the queue
for the appropriate run.
Step 6: If LastKey[NextRun] != +infinity, then initiate reading
the next block from run NextRun into a free input buffer.
Step 7: Initiate the writing of output buffer.
Step 8: If a record with key +infinity has not been merged into
the output buffer, go back to Step 3. Otherwise, wait for
the ongoing write to complete and then terminate.
 Optimal Merging of Runs
26 26

11 15 6 20

6 5 2 4 5 15

2 4

weighted external path length weighted external path length

= 2*3 + 4*3 + 5*2 + 15*1 = 2*2 + 4*2 + 5*2 + 15*2
= 43 = 52
 Huffman Code
• Assume we want to obtain an optimal set of codes for
messages M1, M2, …, Mn+1. Each code is a binary string that
will be used for transmission of the corresponding message.
• At receiving end, a decode tree is used to decode the binary
string and get back the message.
• A zero is interpreted as a left branch and a one as a right
branch. These codes are called Huffman codes.
• The cost of decoding a code word is proportional to the
number bits in the code. This number is equal to the distance
of the corresponding external node from the root node.
• If qi is the relative frequency with which message Mi will be
transmitted, then the expected decoding time is
∑ qi di
1≤i≤n+1

where di is the distance of the external node for message Mi

from the root node.
 Huffman Codes (Cont.)
• The expected decoding time is minimized
by choosing code words resulting in a
decode tree with minimal weighted
external path length.

0 1

M4
0 1

M3
0 1

M1 M2
 Huffman Function
class BinaryTree {
public:
BinaryTree(BinaryTree bt1, BinaryTree bt2) {
root->LeftChild = bt1.root;
root->RightChild = bt2.root;
root->weight = bt1.root->weight + bt2.root->weight;
}
private:
BinaryTreeNode *root;
}

void huffman (List<BinaryTree> l)

// l is a list of single node binary trees as decribed above
{
int n = l.Size(); // number of binary trees in l
for (int i = 0; i < n-1; i++) { // loop n-1 times
BinaryTree first = l.DeleteMinWeight();
BinaryTree second = l.DeleteMinWeight();
BinaryTree *bt = new BinaryTree(first, second);
l.Insert(bt);
O(nlog
} n)
}
 Huffman Tree Example
5 10 16

2 3 5 5 9 7
(c)
2 3
(a)
39
(b)
23
16 23
10 13
9 7 10 13
5 5
5 5
2 3 (d) (e)
2 3
Chap 8

Hashing
 Symbol Table
• Symbol table is used widely in many applications.
– dictionary is a kind of symbol table
– data dictionary is database management
• In general, the following operations are performed
on a symbol table
– determine if a particular name is in the table
– retrieve the attribute of that name
– modify the attributes of that name
– insert a new name and its attributes
– delete a name and its attributes
 Symbol Table (Cont.)
• Popular operations on a symbol table include search,
insertion, and deletion
• A binary search tree could be used to represent a
symbol table.
– The complexities for the operations are O(n).
• A technique called hashing can provide a good
performance for search, insert, and delete.
• Instead of using comparisons to perform search,
hashing relies on a formula called the hash function.
• Hashing can be divided into static hashing and
dynamic hashing
 Static Hashing

• Identifiers are stored in a fixed-size table called hash table.

• The address of location of an identifier, x, is obtained by
computing some arithmetic function h(x).
• The memory available to maintain the symbol table (hash
table) is assumed to be sequential.
• The hash table consists of b buckets and each bucket
contains s records.
• h(x) maps the set of possible identifiers onto the integers 0
through b-1.
• If the identifier is 6 alpha-numerical long, with the first one
being a letter. Then, the total of distinct identifiers are
T= ∑ i
26∗36 > 1 . 6∗10
9

0 ≤ i≤ 5
 Hash Tables
• Definition: The identifier density of a
hash table is the ratio n/T, where n is
the number of identifiers in the table
and T is the total number of possible
identifiers. The loading density or
loading factor of a hash table is α=
n/(sb).
 Hash Tables (Cont.)
• Two identifiers, I1, and I2, are said to be
synonyms with respect to h if h(I1) = h(I2).
• An overflow occurs when a new identifier i is
mapped or hashed by h into a full bucket.
• A collision occurs when two non-identical
identifiers are hashed into the same bucket.
• If the bucket size is 1, collisions and
overflows occur at the same time.
 Example 8.1

Slot 1 Slot 2
0 A A2
If no overflow occur, the time
1 required for hashing depends
only on the time required to
2
compute the hash function h.
3 D
4
5

6 GA G Large number
of collisions and
overflows!

25
 Hash Function
• A hash function, h, transforms an identifier, x, into
a bucket address in the hash table.
• Ideally, the hashing function should be both easy to
compute and results in very few collisions.
• Also because the size of the identifier space, T, is
usually several orders of magnitude larger than the
number of buckets, b, and s is small, overflows
necessarily occur. Hence, a mechanism to handle
overflow is needed.
 Hash Function (Cont.)
• Generally, a hash function should not
have bias on the use of the hash table.
• A uniform hash function supports that
a random x has an equal chance of
hashing into any of the b buckets
 Mid-Square
• Mid-Square function, hm, is computed by
squaring the identifier and then using an
appropriate number of bits from the middle
of the square to obtain the bucket address.
• Since the middle bits of the square usually
depend on all the characters in the
identifier, different identifiers are
expected to result in different hash
addresses with high probability.
 Division
• Another simple hash function is using the
modulo (%) operator.
• An identifier x is divided by some number M
and the remainder is used as the hash
address for x.
• The bucket addresses are in the range of 0
through M-1.
• If M is a power of 2, then hD(x) depends
only on the least significant bits of x.
 Division (Cont.)

• If identifiers are stored right-justified with leading zeros

and M = 2i, i ≤ 6, the identifiers A1, B1, C1, etc., all have the
same bucket.
– Because programmers tend to use many variables with the same
suffix, the choice of M as a power of two would result in many
collisions.
– If left-justified is used, all one-character identifiers would
map to the same bucket, 0, for M = 2i, i ≤ 54; all two character
identifiers would map to the bucket 0 for M = 2 i, i ≤ 48.
• If a division function hD is used as the hash function, the
table size should not be a power of two.
• If M is divisible by two, the odd keys are mapped to odd
buckets and even keys are mapped to even buckets. Thus,
the hash table is biased.
 Division (Cont.)
• Let x=x1x2 and y = x2x1 be two identifiers. If
internal binary representation of x1 is C(x1) and
that for x2 has value C(x2), then if each character
is represented by six bits, the numeric value of x
is 26C(x1) +C(x2), and that for y is 26C(x2)+C(x1).
• If p is a prime number that
6
divides M, then
( f D ( x )− f D ( y )) %p= ( 2 C ( x 1 ) %p+C ( x 2 ) %p
−26 C ( x 2 )%p− C ( x 1 ) %p )%p

• If p =3, then
( f D ( x )− f D ( y )) %p= ( 643C ( x 1 )3 +C ( x 2 )3 )
−643C( x 2 ) 3 −C ( x 1 ) 3 ) 3
C ( x 1 ) 3 +C ( x 2 ) 3 −C ( x 2 )3 −C ( x 1 )3
03
 Division (Cont.)
• Program in which many variables are
permutations of each other would again
result in a biased use of the table and hence
result in many collisions.
– In the previous example, 64%3 =1 and 64%7=1.
• To avoid the above problem, M needs to be a
prime number. Then, the only factors of M
are M and 1.
 Folding
• The identifier x is partitioned into several
parts, all but the last being of the same
length.
• All partitions are added together to obtain
the hash address for x.
– Shift folding: different partitions are added
together to get h(x).
– Folding at the boundaries: identifier is folded
at the partition boundaries, and digits falling
into the same position are added together to
obtain h(x). This is similar to reversing every
other partition and then adding.
 Example 8.2
• x=12320324111220 are partitioned into three decimal digits long.
P1 = 123, P2 = 203, P3 = 241, P4 = 112, P5 = 20.
• Shift folding: 5
h( x )=∑ Pi =123+203 +241+112+20=699
i=1
• Folding at the boundaries: 123 203 241 112 20

Folding 1 time 123 203

211 142
Folding 2 times 123

302
h(x) = 123 + 302 + 241 + 211 + 20 = 897
211

241
 Digit Analysis
• This method is useful when a static file where all
the identifiers in the table are known in advance.
• Each identifier x is interpreted as a number using
some radix r.
• The digits of each identifier are examined.
• Digits having most skewed distributions are
deleted.
• Enough digits are deleted so that the number of
remaining digits is small enough to give an address
in the range of the hash table.
 Overflow Handling
• There are two ways to handle
overflow:
– Open addressing
– Chaining
 Open Addressing
• Assumes the hash table is an array
• The hash table is initialized so that
each slot contains the null identifier.
• When a new identifier is hashed into a
full bucket, find the closest unfilled
bucket. This is called linear probing or
linear open addressing
 Example 8.3
0 A
• Assume 26-bucket table with 1 A2
one slot per bucket and the 2 A1
following identifiers: GA, D, 3 D
A, G, L, A2, A1, A3, A4, Z, 4 A3
ZA, E. Let the hash function 5 A4
6 GA
h(x) = first character of x.
7 G
• When entering G, G collides 8 ZA
with GA and is entered at 9 E
ht[7] instead of ht[6].
25
 Open Addressing (Cont.)
• When linear open address is used to handle
overflows, a hash table search for identifier
x proceeds as follows:
– compute h(x)
– examine identifiers at positions ht[h(x)], ht[h(x)
+1], …, ht[h(x)+j], in this order until one of the
following condition happens:
• ht[h(x)+j]=x; in this case x is found
• ht[h(x)+j] is null; x is not in the table
• We return to the starting position h(x); the table is
full and x is not in the table
 Linear Probing

• In example 8.3, we found that when linear probing is 0 A

used to resolve overflows, identifiers tend to cluster 1 A2
together. Also adjacent clusters tend to coalesce.
This increases search time. 2 A1
– e.g., to find ZA, you need to examine ht[25], ht[0], …, 3 D
ht[8] (total of 10 comparisons).
– To retrieve each identifier once, a total of 39 buckets 4 A3
are examined (average 3.25 bucket per identifier). 5 A4
• The expected average number of identifier GA
comparisons, P, to look up an identifier is 6
approximately (2 -α)/(2-2α), whereαis the loading 7 G
density. 8 ZA
• Example 8.3 α=12/26=.47 and P = 1.5.
9 E
• Even though the average number of probes is small,
the worst case can be quite large.

25 Z
 Quadratic Probing
• One of the problems of linear open addressing is
that it tends to create clusters of identifiers.
• These clusters tend to merge as more identifiers
are entered, leading to bigger clusters.
• A quadratic probing scheme improves the growth of
clusters. A quadratic function of i is used as the
increment when searching through buckets.
• Perform search by examining bucket h(x), (h(x)
+i2)%b, (h(x)-i2)%b for 1 ≤ i ≤ (b-1)/2.
• When b is a prime number of the form 4j+3, for j
an integer, the quadratic search examine every
bucket in the table.
 Rehashing
• Another way to control the growth of
clusters is to use a series of hash
functions h1, h2, …, hm. This is called
rehashing.
• Buckets hi(x), 1 ≤ i ≤ m are examined in
that order.
 Chaining
• We have seen that linear probing perform poorly because the
search for an identifier involves comparisons with identifiers
that have different hash values.
– e.g., search of ZA involves comparisons with the buckets ht[0] –
ht[7] which are not possible of colliding with ZA.
• Unnecessary comparisons can be avoided if all the synonyms
are put in the same list, where one list per bucket.
• As the size of the list is unknown before hand, it is best to
use linked chain.
• Each chain has a head node. Head nodes are stored
sequentially.
 Hash Chain Example
ht
0 A4 A3 A1 A2 A 0
1 0
2 0
3 D 0
4 E 0
5 0
6 G GA 0
7 0
8 0
Average search length is (6*1+3*2+1*3+1*4+1*5)/12
9 0 =2
10 0
11 L 0

25 ZA Z 0
 Chaining (Cont.)
• The expected number of identifier comparisons can
be shown to be ~ 1 +α/2, where αis the loading
density n/b (b=number of head nodes). For α=0.5,
it’s 1.25. And if α=1, then it’s about 1.5.
• Another advantage of this scheme is that only the
b head nodes must be sequential and reserved at
the beginning.
• The scheme only allocates other nodes when they
are needed. This could reduce overall space
requirement for some load densities, despite of
links.
 Hash Functions
• Theoretically, the performance of a hash table
depends only on the method used to handle
overflows and is independent of the hash function
as long as an uniform hash function is used.
• In reality, there is a tendency to make a biased use
of identifiers.
• Many identifiers in use have a common suffix or
prefix or are simple permutations of other
identifiers.
– Therefore, different hash functions would give different
performance.
 Average Number of Bucket Accesses
Per Identifier Retrieved

a = n/b 0.50 0.75 0.90 0.95

Hash Function Chain Open Chain Chain Open Chain Open

Open

mid square 1.26 1.73 1.40 1.45 37.14 1.47 37.53

9.75

division 1.19 4.52 1.31 7.20 1.38 22.42 1.41 25.79

shift fold 1.33 21.75 1.48 65.10 1.40 77.01 1.51 118.57

bound fold 1.39 22.97 1.57 1.55 69.63 1.51 97.56

48.70

digit analysis 1.35 4.55 1.49 1.52 89.20 1.52 125.59

30.62

theoretical 1.25 1.50 1.37 1.45 5.50 1.48 10.50

2.50
 Theoretical Evaluation of Overflow
Techniques
• In general, hashing provides pretty good
performance over conventional techniques
such as balanced tree.
• However, the worst-case performance of
hashing can be O(n).
• Let ht[0..b-1] be a hash table with b
buckets. If n identifiers x1, x2, …, xn are
entered into the hash table, then there are
bn distinct hash sequences h(x1), h(x2), …,
h(xn).
 Theoretical Evaluation of Overflow
Techniques (Cont.)
• Sn is the average number of comparisons
neede to find the jth key xj, averaged over 1
≤ j ≤ n, with each j equally likely, and
averaged over all bn hash sequences,
assuming each of these also to be equally
likely.
• Un is the expected number of identifier
comparisons when a search is made for an
identifier not in the hash table.
 Theorem 8.1

Theorem 8.1: Let α=n/b be the loading density of

a hash table using a uniform hashing
function h. Then
(1) for linear open addressing
1 1 1 1
U n ≈ [ 1+ ] S n ≈ [ 1+ ]
2 ( 1− α )2 2 1−α
(1) for rehashing, random probing, and
quadratic probing
1
U n ≈ 1 /( 1− α ) S n ≈−[ ]log e (1−α )
α
(1) for chaining
U n≈ α S n ≈ 1+α / 2
 Dynamic Hashing
• The purpose of dynamic hashing is to retain the
fast retrieval time of conventional hashing while
extending the technique so that it can
accommodate dynamically increasing and
decreasing file size without penalty.
• Assume a file F is a collection of records R. Each
record has a key field K by which it is identified.
Records are stored in pages or buckets whose
capacity is p.
• The goal of dynamic hashing is to minimize access
to pages.
• The measure of space utilization is the ratio of
the number of records, n, divided by the total
space, mp, where m is the number of pages.
 Dynamic Hashing Using
Directories
• Given a list of identifiers in the following:
Identifiers Binary representation

A0 100 000

A1 100 001

B0 101 000

B1 101 001

C0 110 000

C1 110 001

C2 110 010

C3 110 011

C5 110 101

• Now put these identifiers into a table of four pages. Each page can hold
at most two identifiers, and each page is indexed by two-bit sequence
00, 01, 10, 11.
• Now place A0, B0, C2, A1, B1, and C3 in a binary tree, called Trie, which
is branching based on the last significant bit at root. If the bit is 0, the
upper branch is taken; otherwise, the lower branch is taken. Repeat this
for next least significant bit for the next level.
 A Trie To Hold Identifiers

0 A0, B0 0 A0, B0

0 1 0
C2 1 C2
0 A1, B1
1
0 A1, B1 1
0
1 C5
1 C3 1 C3
(a) two-level trie 0 A0, B0
on four pages (b) inserting C5
0
1 C2 0 A1, C1 with overflow
0

1 0 1 B1
1 C5 (c) inserting C1
with overflow
1 C3
 Issues of Trie Representation
• From the example, we find that two
major factors that affects the
retrieval time.
– Access time for a page depends on the
number of bits needed to distinguish the
identifiers.
– If identifiers have a skewed distribution,
the tree is also skewed.
 Extensible Hashing
• Fagin et al. present a method, called extensible
hashing, for solving the above issues.
– A hash function is used to avoid skewed distribution. The
function takes the key and produces a random set of
binary digits.
– To avoid long search down the trie, the trie is mapped to a
directory, where a directory is a table of pointers.
– If k bits are needed to distinguish the identifiers, the
directory has 2k entries indexed 0, 1, …, 2k-1
– Each entry contains a pointer to a page.
 Trie Collapsed Into Directories

a a a
0000 c A0, B0
00 A0, B0 000 A0, B0
c c 0001 A1, C1
001 A1, B1 b
01 A1, B1 0010 f C2
b b
10 C2 010 C2 0011 a C3
d e 0100
11 C3 011 C3 e
a 0101 C5
b
100 0110 f
d 0111
101 C5 a
b 1000 d
110 1001 B1
c b
111 1010
f
1011
a
1100 e
1101
b
1110 f
1111

(a) 2 bits (b) 3 bits © 4 bits

 Hashing With Directory

• Using a directory to represent a trie allows table

of identifiers to grow and shrink dynamically.
• Accessing any page only requires two steps:
– First step: use the hash function to find the address of
the directory entry.
– Second step: retrieve the page associated with the
address
• Since the keys are not uniformly divided among
the pages, the directory can grow quite large.
• To avoid the above issue, translate the bits into a
random sequence using a uniform hash function.
So identifiers can be distributed uniformly across
the entries of directory. In this case, multiple
hash functions are needed.
 Overflow Handling
• A simple hash function family is simply adding a leading zero
or one as the new leading bit of the result.
• When a page identified by i bits overflows, a new page needs
to be allocated and the identifiers are rehashed into those
two pages.
• Identifiers in both pages should have their low-order I bits in
common. These are referred as buddies.
• If the number of identifiers in buddy pages is no more than
the capacity of one page, then they can be coalesce into one
page.
• After adding a bit, if the number of bits used is greater than
the depth of the directory, the whole directory doubles in
size and its depth increases by 1.
Program 8.5 Extensible Hashing

const int WordSize = 5; // maximum no. of directory bits

const int PageSize = 10; // maximum size of a page
struct TwoChars { char str[2];};
struct page {
int LocalDepth; // no. of bits to distinguish ids
TwoChars names[PageSize]; //the actual identifiers
int NumIdents; // no. of identifiers in this page
};
typedef page* paddr;
struct record { // a sample record
TwoChars KeyField;
int IntData;
char CharData;
};
paddr rdirectory[MaxDir]; // will contain pointers to pages
int gdepth; // not to exceed WordSize
 Program 8.5 Extensible Hashing (Cont.)

paddr hash(const TwoChars& key, const int precision);

// key is hashed using a uniform hash function, and the low order precision bits are returned
//as the page address

paddr buddy(const paddr index);

// Take an address of a page and return the page’s buddy; i.e., the leading bit it complemented

int size(const paddr ptr);

// Return the number of identifiers in the page

paddr coalesce(const paddr ptr, const paddr buddy);

// Combine page ptr and buddy, buddy into a single page

Boolean PageSearch(const TwoChars& key, const paddr index);

// Search page index for key key. If found, return TRUE; otherwise, return FALSE.

int convert(const paddr p);

// Convert a pointer to a page to an equivalent integer
 Program 8.5 Extensible Hashing (Cont.)

void enter(const record r, const paddr p);

// Insert the new record r into the page pointed at by p

void PageDelete(const TwoChars& key, const paddr p);

// Remove the record with key key from the page pointed at by p

paddr find(const TwoChars& key);

// Search for a record with key key in the file. If found, return the address of the page in which it was
// found. If not return 0.
{
paddr index = hash(key, gdepth);
int IntIndex = convert(index);
padd ptr = rdirectory[IntIndex];
if (PageSearch(key, ptr) retrun ptr;
else return 0;
}
 Program 8.5 Extensible Hashing (Cont.)

void insert(const record& r, const TwoChars& key)

// Insert a new record into the file pointed at by the directory
{
paddr p = find(key); // check if key is present
if(p) return; // key already in
if(p->NumIdents != PageSize) { // page not full
enter(r, p); p->NumIdents ++;
}
else {
Split the page into two, insert the new key, and update gdepth if necessary; if this causes gdepth
to exceed WordSize, print an error and terminate.
}
}
void Delete(const TwoChars& key)
//Find and delete the record with key key
{
paddr p = find(key);
if(p) {
PageDelete(key, p);
if ((size(p) + size(buddy(p)) <= PageSize) coalesce(p, buddy(p));
}
}
 Analysis of Directory-Based Dynamic
Hashing

• The most important of the directory version of

extensible hashing is that it guarantees only two
disk accesses for any page retrieval.
• We get the performance at the expense of space.
This is because a lot of pointers could point to the
same page.
• One of the criteria to evaluate a hashing function
is the space utilization.
• Space utilization is defined as the ratio of the
number of records stored in the table divided by
the total number of space allocated.
• Research has shown that dynamic hashing has
69% of space utilization if no special overflow
mechanisms are used.
 Directoryless Dynamic Hashing
• Directoryless hashing (or linear hashing) assume a
continuous address space in the memory to hold all
the records. Therefore, the directory is not
needed.
• Thus, the hash function must produce the actual
address of a page containing the key.
• Contrasting to the directory scheme in which a
single page might be pointed at by several directory
entries, in the directoryless scheme one unique
page is assigned to every possible address.
A Trie Mapped To Directoryless,
Continuous Storage

0 A0, B0 00 A0
B0
0 1 01 C2
C2
-
10 A1
0 A1, B1
1 B1
11 C3
1 C3 -
 Directoryless Scheme Overflow
Handling

00 A0 000 A0 000 A0
B0 B0 B0
overflow
01 C2 001 C2 001 C2
page
- - -
10 A1 010 A1 010 A1
B1 B1 C5 B1 C1 C5
11 C3 011 C3 011 C3
- - -
100 - 100 -
- new page - new page
101 -
-
 Figure 8.14 The rth Phase of Expansion of
Directoryless Method

pages already split pages not yet split pages added so far

addresses by r+1 bits addresses by r bits addresses by r+1 bits

q r
2r pages at start
 Analysis of Directoryless
Hashing
• The advantage of this scheme is that for many
retrievals the time is one access for those
identifiers that are in the page directly addressed
by the hash function.
• The problem is that for others, substantially more
than two accesses might be required as one moves
along the overflow chain.
• Also when a new page is added and the identifiers
split across the two pages, all identifiers including
the overflows are rehashed.
• Hence, the space utilization is not good, about 60%.
(shown by Litwin).
 Chap 9

Priority Queues
Operations supported by priority queue

• The functions that have been supported:

– SP1: Return an element with minmum priority
– SP2: Insert an element with an arbitrary
priority
– SP2: Delete an element with minimum
priority
• Extended functions:
– Meld two priority queues
– delete an arbitrary element
– decrease the key/priority
 Double-ended priority
queue(DEPQ)
• Support the following operations
– DP1: Return an element with minimum
priority
– DP2: Return an element with maximum
priority
– DP3: Insert an element with an arbitrary
– DP4: Delete an element with minimum
priority
– DP5: Delete an element with maximum
priority
 Leftist Trees
• To combine two priority queues into a single
priority queue, the combine operation takes
O(n) time if heaps are used.
• The complexity of combine operation can be
reduced to O(log n) if a leftist tree is used.
• Leftist trees are defined using the concept
of an extended binary tree. An extended
binary tree is a binary tree in which all empty
binary subtrees have been replaced by a
square node.
• The square nodes in an extended binary tree
are called external nodes. The original nodes
of the binary tree are called internal nodes.
 Extended Binary Trees

A G

B C H I

D E F J
 Leftist Trees (Cont.)
• Let x be a node in an extended binary tree. Let
LeftChild(x) and RightChild(x), respectively,
denote the left and right children of the internal
node x.
• Define shortest(x) to be the length of a shortest path
from x to an external node. It is easy to see that
shortest(x) satisfies the following recurrence:
0 if x is an external node
¿
shortest(x) = ¿
1 + min{shortest(LeftChild(x)), RightChild(x))} otherwise
 Shortest(x) of Extended Binary Trees

2 2
A G

2 1 1 1
B C H I

1 1 1 1
D E F J
 Leftist Tree Definition
Definition: A leftist tree is a binary tree such
that if it is not empty, then
shortest(LeftChild(x)) ≥
shortest(RightChild(x)) for every internal
node x.
Lemma 9.1: Let x be the root of a leftist tree
that has n (internal) nodes
(a)n ≥ 2shortest(x) – 1
(b)The rightmost root to external node path is
the shortest root to external node path. Its
length is shortest(x).
Class Definition of A Leftist
Tree

template<class KeyType>class MinLeftistTree; // forward declaration

template<class KeyType>
class LeftistNode {
friend class MinLeftistTree<KeyType>;
private:
Element<KeyType>data;
LeftistNode *LeftChild, *RightChild;
int shortest;
}

template<class KeyType>
class MinLeftistTree:public MinPQ<KeyType> {
public:
// constructor
MinLeftistTree(LeftistNode<KeyType> *int = 0) root(int) {};
// the three min-leftist tree operations
void Insert(const Element<KeyType>&);
Element<KeyType>* DeleteMin(Element<KeyType>&);
void MinCombine(MinLeftistTree<KeyType>*);
private:
LeftistNode<KeyType>* MinUnion(LeftistNode<KeyType>*, LeftistNode<KeyType>*);
LeftistNode<KeyType>* root;
};
 Definition of A Min (Max) Leftist Tree

• Definition: A min leftist tree (max

leftist tree) is a leftist tree in which
the key value in each node is no larger
(smaller) than the key values in its
children (if any). In other words, a min
(max) leftist tree is a leftist tree that
is also a min (max) tree.
 Examples of Min Leftist Trees

2 2
2 2

1 1 1 1
7 50 9 8

1 1 2 1
11 80 12 10

1 1 1 1
13 20 18 15
 Min (Max) Leftist Tree (Cont.)
• Like any other tree structures, the popular
operations on the min (max) leftist trees are
insert, delete, and combine.
• The insert and delete-min operations can
both be done by using the combine operation.
– e.g., to insert an element x into a min leftist tree,
we first create a min leftist tree that contains
the single element x. Then we combine the two
min leftist trees.
– To delete the min element from a nonempty min
leftist tree, we combine the min leftist trees
root->LeftChild and root->RightChild and delete
the node root.
 Combine Leftist Trees
• To combine two leftist trees:
– First, a new binary tree containing all elements in
both trees is obtained by following the rightmost
paths in one or both trees.
– Next, the left and right subtrees of nodes are
interchanged as necessary to convert this binary
tree into a leftist tree.
• The complexity of combining two leftist
trees is O(log n)
 Combining The Min Leftist Tree
2
2 2
8
1 1 1 2
10 50 7 5
1 1 1 1 2 2
15 80 11 9 8 2
1 2 1 1 2 2
13 12 10 50 5 7
2 1 1 1
5 20 15 2
18 80 1 1
1 2 8 9 11
9 8 1 1 2 1
2 1 1 10 50 12 13
12 10 50 1
1 1 1
1 1 15 80 20 18
20 18 15 80
 Binomial Heaps
• A binomial heap is a data structure that
supports the same functions as those
supported by leftist trees.
• Unlike leftist trees, where an individual
operation can be performed in O(log n) time,
it is possible that certain individual
operations performed on a binomial heap may
take O(n) time.
• By amortizing part of the cost of expensive
operations over the inexpensive ones, then
the amortized complexity of an individual
operation is either O(1) or O(log n) depending
on the type of operations.
 Cost Amortization
• Given a sequence of operations I1, I2, D1, I3, I4, I5, I6, D2,
I7. Assume each insert operation costs one time unit and D1
and D2 operations take 8 and 10 time units, respectively.
• The total cost to perform the sequence of operations is 25.
• If we charge some actual cost of an operation to other
operations, this is called cost amortization.
• In this example, the amortized cost of I1 – I6 each has 2 time
units, I7 has one, and D1 and D2 each has 6.
• Now suppose we can prove that no matter what sequence of
insert and delete-min operations is performed, we can charge
costs in such a way that the amortized cost of each insertion is
no more than 2 and that of each deletion is no more than 6. We
can claim that the sequence of insert/delete-min operations
has cost no more than 2*i + 6*d.
• With the actual cost, we conclude that the sequence cost is no
more than i+10*d.
• Combining the above two bounds, we obtain min{2*i+6*d,
i+10*d}.
 Binomial Heaps
• Binomial heaps have min binomial heap and max binomial
heap.
• We refer to the min binomial heap as B-heap.
• B-heap can perform an insert and a combine operation
in O(1) actual and amortized time and a delete-min
operation with O(log n) amortized time.
• A node in a B-heap has the following data members:
– degree: is the number of children it has
– child: is a pointer points to any one of its children. All children
forms a circular list.
– link: is a singly link used to maintain a circular list with its
siblings.
– data
• The roots of the min trees that comprise a B-heap are
linked to form a singly linked circular list. The B-heap
is then pointed at by a single pointer min to the min
tree root with smallest key.
 B-Heap Example
1
8 3

12 7 16
10 5 4

15 30 9
6
min
20

8 3 1

10 12 7 16
5 4

6 15 30 9

20
 Insertion Into A B-Heap
• An element x can be inserted into a B-heap
by first putting x into a new node and then
inserting this node into the circular list
pointed at by min. The operation is done in
O(1) Time.
• To combine two nonempty B-heaps, combine
the top circular lists of each into a single
circular list.
• The new combined B-heap pointer is the min
pointer of one of the two trees, depending on
which has the smaller key.
• Again the combine operation can be done in
O(1) time.
The Tree Structure After Deletion of
Min From B-Heap

8 3 12 7 16

10 15 30 9
5 4

6 20
 Deletion of Min Element
• If min is 0, then the B-heap is empty. No
delete operations can be performed.
• If min is not 0, the node is pointed by min.
Delete-min operation deletes this node from
the circular list. The new B-heap consists of
the remaining min trees and the submin trees
of the delete root.
• To form the new B-heap, min trees with the
same degrees are joined in pairs. The min
tree whose root has the larger key becomes
the subtree of the other min tree.
Joining Two Degree-One Min
Trees

7 3 12 16

8 9 5 4 15 30

10 6 20
Joining Two Degree-Two Min
Trees

3 16
12

7 5 4
15 30

8 9 6
20

10 Since no two min trees have

the same degree, the min join
process stops.
 The New B-Heap

min

12 3 16

15 30 7 5 4

20 8 9 6

10
 Program 9.12 Steps In A Delete-Min
Operation

template<class KeyType>
Element<KeyType>*Binomial<KeyType>::DeleteMin(Element<KeyType>&
x)

Step 1: [Handle empty B-heap] if (!min){ DeletionError(); return();}

Step 2: [Deletion from nonempty B-heap] x=min->data; y=min->child;
delete min from its circular list; following this deletion, min points
to any remaining node in the resulting list; if there is no such node,
then min = 0;
Step 3: [Min-tree joining] Consider the min trees in the lists min and
y; join together pairs of min trees of the same degree until all
remaining min trees have different degrees;
Step 4: [Form min tree root list] Link the roots of the remaining min
trees (if any) together to form a circular list; set min to point to
 Binomial Tree Definition
• Definition: The binomial tree Bk, of degree k is a
tree such that if k = 0, then the tree has exactly
one node, and if k > 0, then the tree consists of a
root whose degree is k and whose subtrees are B0,
B1, …, Bk-1.
• Lemma 9.2: Let a be a B-heap with n elements
that results from a sequence of insert, combine,
and delete-min operations performed on a
collection of initially empty B-heaps. Each min
tree in a has degree ≤ log2n. Consequently,
MaxDegree ≤ ⌊ 2 ⌋ , and the actual cost of a
log n
delete-min operation is O(log n + s).
 B-Heap Costs
• Theorem 9.1: If a sequence of n insert,
combine, and delete-min operations is
performed on initially empty B-heaps,
then we can amortize costs such that
the amortized time complexity of each
insert and combine is O(1), and that of
each delete-min operation is O(log n).
 Fibonacci Heaps
• A Fibonacci heap is a data structure that
supports the three binomial heap
operations: insert, delete-min (or delete-
max), and combine, as well as the following
additional operations:
(1) decrease key: Decrease the key of a specified
node by a given positive amount
(2) delete: Delete the element in a specified node.
• The first of these can be done in O(1)
amortized time and the second in O(log n)
amortized time.
• The binomial heap operations can be
performed in the same asymptotic times
using a Fibonacci heap as they can be using
a binomial heap.
 Fibonacci Heaps (Cont.)
• Two varieties of Fibonacci heaps:
– Min Fibonacci heap: is a collection of min trees
– Max Fibonacci heap: is a collection of max trees.
• Refers to min Fibonacci heap as F-heaps.
• B-heaps are a special case of F-heaps.
• A node in F-heap data structure contains additional
data members other than those in B-heaps:
– parent: is used to point to the node’s parent (if any).
– ChildCut: to support cascading cut described later.
– LeftLink and RightLink: replace the link data member in B-
heap node. These links form a doubly linked circular list.
• In F-heaps, singly linked circular list is replaced by
doubly linked circular list.
 Deletion From An F-Heap
• The basic operations insert, delete-min, and
combine are performed exactly as for the
case of B-heaps.
• Follow the below steps to delete a node b
from an F-heap:
(1) If min = b, then do a delete-min; otherwise do
Steps 2, 3, and 4 below
(2) Delete b from its doubly linked list
(3) Combine the doubly linked list of b’s children
with the doubly linked list pointed at by min into
a single doubly linked list. Trees of equal degree
are not joined as in a delete-min operation.
(4) Dispose of node b.
 F-Heap After Deletion of 12

8 3 1 11 30

10 4 16 20
5 7

6 9
min

8 3 1 11 30

10 4 16 20
5 7

6 9
 Decrease Key
• To decrease the key in node b, do the
following:
(1) Reduce the key in b
(2) If b is not a min tree root and its key is smaller
than that in its parent, then delete b from its
doubly linked list and insert it into the doubly
linked list of min tree roots.
(3) Change min to point to b if the key in b is
smaller than that in min.
F-Heap After The Reduction of 15 by 4
min

8 3 1

10 4 12 7 16
5

6 15 30 9

20 min

8 3 1 11

10 4 12 7 16 20
5

6 30 9
 Cascading Cut
• Because the new delete and decrease-key
operations, the F-heap is not necessary a
Binomial tree. Therefore, the analysis of
theorem 9.1 is no longer true for F-heaps
if no restructuring is done.
• To ensure that each min tree of degree k
has at least ck nodes, for some c, c> 1, each
delete and decrease-key operations must
be followed by a particular step called
cascading cut.
• The data member ChildCut is used to
assist the cascading cut step.
• ChildCut data member is only used for non-
 Cascading Cut (Cont.)
• ChildCut of node x is TRUE iff one of the children of
node x was cut off after the most recent time x was
made the child of its current parent.
• Whenever a delete or decrease-key operation deletes
a node q that is not a min tree root from its doubly
linked list, then the cascading cut step is invoked.
• During the steps, we examine the nodes on the path
from the parent p of the deleted node q up the nearest
ancestor of the deleted node with ChildCut = FALSE.
• If there is no such ancestor, then the path goes from p
to the root of the min tree containing p.
• All nonroot nodes on this path with ChildCut data
member TRUE are deleted from their respective
doubly linked list and added to the doubly linked list of
min tree root nodes of the F-heap.
• If the path has a node with ChildCut set to FALSE,
then it is changed to TRUE.
 A Cascading Cut Example
2 10 12 10

4 5 16 15 18 30 11

6 60
8 6
20 8 7
20 7
10
2
30 12 11
4 5 ChildCut=TRUE
*
14 18
60
16 15
 F-Heap Analysis

• Lemma 9.3: Let a be an F-heap with n elements that

results from a sequence of insert, combine, delete-min,
delete, and decrease-key operations performed on
initially empty F-heaps.
(a) Let b be any node in any of the min trees of a. The degree of b
is at most logΦ m, where φ=(1+ √ 5 )/2 and, m is the number
elements in the subtree with root b.
(b) MaxDegree ≤ ⌊ log φ n ⌋ , and the actual cost of a delete-min
operation is O(log n + s).
 Theorem 9.2
Theorem 9.2: If a sequence of n insert, combine,
delete, delete-min, and decrease-key operations is
performed on an initially empty F-heap, then we can
amortize costs such that the amortized time
complexity of each insert, combine, and decrease-
key operation is O(1) and that of each delete-min
and delete operation is O(log n). The total time
complexity of the entire sequence is the sum of the
amortized complexities of the individual operations
in the sequence.
 Min-Max Heaps
• A double-ended priority queue is a data
structure that supports the following
operations:
– inserting an element with an arbitrary key
– deleting an element with the largest key
– deleting an element with the smallest key
• A Min-Max Heap supports all of the above
operations.
 Min-Max Heap (Cont.)
• Definition: A min-max heap is a complete
binary tree such that if it is not empty, each
element has a data member called key.
Alternating levels of this tree are min levels
and max levels, respectively. The root is on a
min level. Let x be any node in a min-max
heap. If x is on a min (max) level then the
element in x has the minimum (maximum) key
from among all elements in the subtree with
root x. A node on a min (max) level is called a
min (max) node.
Figure 9.1: A 12-element Min-Max Heap

7 min

70 40 max

30 9 10 15 min

45 50 30 20 12 max
 Min-Max Heap (Cont.)

• The min-max heap stores in a one-dimension array

h.
• Insert a key 5 into this min-max heap.
• Initially key 5 is inserted at j. Now since 5 < 10
(which is j’s parent), 5 is guaranteed to be smaller
than all keys in nodes that are both on max levels
and on the path from j to root. Only need to
check nodes on min levels.
• When inserting 80 into this min-max heap, since
80 > 10, and 10 is on the min level, we are assured
that 80 is larger than all keys in the nodes that
are both on min levels and on the path from j to
the root. Only need to check nodes on max levels.
 Insert to Min-Max Heap

7 min

70 40 max

30 9 10 15 min

45 50 30 20 12 j max
Min-Max Heap After Inserting Key 5

5 min

70 40 max

30 9 7 15 min

45 50 30 20 12 10 max
Min-Max Heap After Inserting Key 80

7 min

70 80 max

30 9 10 15 min

45 50 30 20 12 40 max
 Program 9.3 Insertion Into A Min-Max
Heap
template <class KeyType>
void MinMaxHeap<KeyType>::Insert(const Element<KeyType>& x)
// inset x into the min-max heap
{
if (n==MaxSize) {MinMaxFull(); return;}
n++;
int p =n/2; // p is the parent of the new node
if(!p) {h[1] = x; return;} // insert into an empty heap
switch(level(p)) {
case MIN:
if (x.key < h[p].key) { // follow min levels
h[n] = h[p];
VerifyMin(p, x);
}
else { VerifyMax(n, x); } // follow max levels
break;
case MAX:
if (x.key > h[p].key) { // follow max levels
h[n] = h[p];
VerifyMax(p, x);
}
else { VerifyMin(n, x); } // follow min levels
break;
}
}
 Program 9.4 Searching For The Correct Max
Node For Insertion

template <class KeyType>

void MinMaxHeap<KeyType>::VerifyMax(int i, const Elemetn<KeyType>&
x)
// Follow max nodes from the max node I to the root and insert x at proper place
{
for (int gp = i/4; // grandparent of i
gp && (x.key > h[gp].key); O(log n)
gp /= 4)
{ // move h[gp] to h[i]
h[i] = h[gp];
i = gp;
}
h[i] = x; // x is to be inserted into node i
}
 Deletion of the Min Element

12 min

70 40 max

30 9 10 15 min

45 50 30 20 max
 Deletion of the Min Element
(Cont.)
• When delete the smallest key from the min-max heap, the
root has the smallest key (key 7). So the root is deleted.
• The last element with key 12 is also deleted from the min-
max heap and then reinsert into the min-max heap. Two
steps to follow:
– The root has no children. In this case x is to be inserted into
the root.
– The root has at least one child. Now the smallest key in the
min-max heap is in one of the children or grandchildren of the
root. Assume node k has the smallest key, then following
conditions must be considered:
• x.key ≤ h[k].key. x may be inserted into the root.
• x.key >h[k].key and k is a child of the root. Since k is a max node, it
has not descendents with key larger than h[k].key. So, node k has
no descendents with key larger than x.key. So the element h[k]
may be moved to the root, and x can be inserted into node k.
• x.key> h[k] and k is a grandchild of the root. h[k] is moved to the
root. Let p the parent of k. If x.key > h[p].key, then h[p] and x are
to be interchanged.
 Min-Max Heap After Deleting Min
Element

9 min

70 40 max

30 12 10 15 min

45 50 30 20 max
 Deaps
• A deap is a double-ended heap that
supports the double-ended priority
operations of insert, delet-min, and
delete-max.
• Similar to min-max heap but deap is
faster on these operations by a
constant factor, and the algorithms are
simpler.
 Deaps (Cont.)
• Definition: A deap is a complete binary tree that
is either empty or satisfies the following
properties:
(1) The root contains no element
(2) The left subtree is a min heap.
(3) The right subtree is a max heap.
(4) If the right subtree is not empty, then let i be
any node in the left subtree. Let j be the
corresponding node in the right subtree. If such
a j does not exist, then let j be the node in the
right subtree that corresponds to the parent
of i. The key in node i is less than or equal to
that of j.
 A Deap Example

5 45

10 8 25 40

15 19 9 30 20
 Deap (Cont.)
• From Deap’s definition, it’s obvious that for an n-element
deap, the min element is the root of min heap and the max
element is the root of the max heap.
• If n = 1, then the min and max elements are the same and
are in the root of the min heap.
• Since deap is a complete binary tree, it may be stored as an
implicit data structure in a one-dimension array similar to
min, max, min-max heaps.
• In the case of deap, the position 1 of the array is not used.
For an n-element deap, it occupied n+1 element of an array.
• If i is a node in the min heap of the deap, its corresponding
node in the max heap is i+ 2⌊ log i ⌋−1 .
2

• Then j defined in property (4) of definition is given by

⌊ log 2 i ⌋−1
j=i+ 2 ;
if (j > n+1) j /= 2;
Figure 9.7 Insertion Into A Deap

5 45

10 8 25 40

15 19 9 30 20 j

i
 Figure 9.8: Deap Structure After
Insertion of 4

4 45

5 8 25 40

15 10 9 30 20 19
 Figure 9.8: Deap Structure After
Insertion of 30

5 45

10 8 30 40

15 19 9 30 20 25
Program 9.7: Inserting Into A
Deap

template <class KeyType>

void Deap<KeyType>::Insert(const Element<KeyType>& x) {
//Insert x into the deap
int I;
if (n==MaxSize) {DeapFull(); return;}
n++;
if (n==1) {d[2]=x; return;} // insert into an empty deap
int p = n+1; // p is the new last position of the deap
switch(MaxHead(p)) {
case TRUE: // p is a position in the max heap
i = MinPartner(p);
if (x.key < d[i].key) {
d[p] = d[i]; O(log n)
MinInsert(i, x);
}
else MaxInsert(p, x);
break;
case FALSE: // p is a position in the min heap
i = MaxPartner(p);
if (x.key > d[i].key) {
d[p] = d[i];
MaxInsert(i, x);
}
else MinInsert(p, x);
}
}
 Deletion of Min Element

• Suppose we want to remove the minimum element from the

deap.
– We first place the last element into a temporary element t.
– Vacancy created by the deletion of the min element is filled by
moving from position 2 to a leaf.
– Each move is preceded by a step that moves the smaller of the
elements in the children of the current node up.
– Then move to the position previously occupied by the moved
element.
– Repeat the process until we move the empty node to a leaf
position.
– Compare the key put in the temporary element with the max
partner.
– If <, no exchange is needed. The temporary element is inserted
into the empty leaf node.
– If >, exchange them.
Deap Structure After Deletion of Min
Element

8 45

10 9 25 40

15 19 20 30
Part I – AVL Trees
 Unbalanced Binary Search
Tree
Number of 5
comparisons
4 8
needed to search
for NOV: 6. 1 7 9

Average number
2 6
of comparisons: 12

3.5 3
11

10
 Skew Binary Search Tree
Consider the keys are entered in
lexicographic
1
2
order.
3

8
 Binary Search Tree
Consider a balanced
binary search tree as
illustrated. 6

Number of comparisons 4 9

needed to search for

2 5 8 11
NOV: 6.
Average number of 1 3 7 10 12

comparisons: 3.1
 Binary Search Trees:
Balanced vs. Unbalanced
The average and maximum search time
can be minimized if the binary search
tree is maintained as a complete binary
tree all the times.
The time becomes O(log n) for an n-node binary
search tree.
AVL tree (1962)
Balanced binary search tree with respect to the
heights of subtrees.
Any retrievals can be performed in O(log n)
 Height-Balanced
Definition
An empty tree is height-balanced.
If T is nonempty binary tree with TL and TR as
its left and right subtrees respectively.
T is height-balanced iff
• TL and TR are height-balanced, and
• |hL-hR|≦1 where hL and hR are heights of TL and
TR, respectively.
 Examples
5

6
4 8

4 9
1 7 9

2 5 8 11
2 6 12

11 1 3 7 10 12
3

10

Not height-balanced Height-balanced

 Balance Factor
Definition:
For every node T, define its balance factor,
BF(T), as
BF(T) = hL - hR
where hL and hR are the heights of the left
and right subtrees of T.
BF(T) for any node T in an AVL tree is –1, 0, or 1.
Balance Factors for an AVL
Tree
-1

1 1

0 1 -1
0

0
0 0 -1 0

0
 Construction of an AVL Tree
Consider to insert the following numbers:
8, 9, 10, 2, 1, 5, 3, 6, 4, 7, 11, 12
0 -1

8 8
0

Insert 8 Insert 9
 Consider the
-2 nearest parent
A with bf = ±2
8 0
-1
9
RR
9 0 0
0 8 10
insert in the
10 right subtree of
the right
Insert 10 subtree of A
 1

9
1 0

8 10
0

Insert 2
 2
Consider the
2 9 nearest parent 1
0
A with bf = ±2
0 9
8 10 0
1

LL 2 10
2 0
0 0

1 8
1
insert in the
left subtree of
the left subtree
Insert 1 of A
 Consider the
2 nearest parent 0

A with bf = ±2
0 8
9 -1
-1 0

LR 2 9
2 10 0 0
0 0
1

1 5 10
1 8 insert in the
0 right subtree of
the left subtree
5 of A

Insert 5
 1 1

-1 8 -1 8
-1 -1

2 9 2 9
0 0 0 0
1 0

1 5 10 1 5 10

0 0 0

3 3 6

Insert 3 Insert 6
 1
2 Consider the
nearest
8
-2 8 parent A 0 -1
-1
with bf = ±2
3 9
2 9 0
0 0 1 0
1

1 RL 2 5 10
5 10 0
0 0
-1 0
1
3 6 insert in the 4 6
0
left subtree
of the right
4
subtree of A

Insert 4
 2 0

5
88
-1 -1 1 0

3 9 3 8
1 -1 0
LR 1 0 -1 -1

2 5 10 2 4 6 9
0
0 0 0 0
-1

1 1 7 10
4 6
0

Insert 7
 -1

5
1 -1

3 8
1 0 -1 -2

9
0
2 4 6
0 -1
RR

1 7 10 0 0

11 5
0
Insert 11 1

3 8

1 0 -1 0

2 4 6 10
0 0 0 0

1 7 9 11
 -1

5
1 -1

3 8

1 0 -1 -1

2 10
0 4 6

0 0 -1

1 7 9 11
0

12

Insert 12
 Rotation Types (1)
Suppose Y is the new node.
LL: Y is inserted in the left subtree of the left
subtree of A.
A B
B LL C A
TA
C TB TB TA
 Rotation Types (2)
LR: Y is inserted in the right subtree of the left
subtree of A

A
C

B TA LR
B A

C
TCL TCR TA

TCL TCR
 Rotation Types (3)
RR: Y is inserted in the right subtree of the
right subtree of A.

A B

TA B RR A C

TB C
TA TB
 Rotation Types (4)
RL: Y is inserted in the left subtree of the right
subtree of A

C
A

RL A B
TA B

C
TA TCL TCR

TCL TCR
 The Class Definition of AVL
Tree
class AvlNode {
friend class AVL;
public:
AvlNode(int k) {data = k; bf = 0; leftChild = NULL; rightChild =
NULL; }
private: Store the value of
int data; balance factor of the
int bf; node
AvlNode *leftChild, *rightChild;
};
class AVL {
public:
AVL() : root(0) {};
bool Search(int key);
bool Insert(int key);
bool Delete(int key);
private:
AvlNode *root;
};
 Phase 1
bool AVL::Insert(int key)
{
if (!root)
{
root = new AvlNode(key);
return true;
}

//Phase 1: locate insertion point for key

AvlNode *a = 0, //most recent node with bf = ± 1
*pa = 0, //parent of a
*p = root, //p moves through the tree
*pp = 0; //parent of p
while (p)
{
if (p->bf != 0)
{
a = p;
pa = pp;
}
if (k > p->key)
{
pp = p;
p = p->rightChild;
}
else if (k < p->key)
{
pp = p;
p = p->leftChild;
}
else
return false;
}
 Phase 2

//Phase 2: Insert and rebalance

//k is not in the tree and may be inserted as the appropriate child of
pp.
AvlNode *y = new AvlNode(k);
if (k < pp->key)
pp->leftChild = y; //insert as left Child
else
pp->rightChild = y; //insert as right Child
 Adjust balance factors of nodes on path
from
int d; a to pp.
AvlNode *b, //child of a
*c; //child of b 1
if (a == NULL) { pa
p = root; d = (k > p->key)? -1 : 1; d=-1
} -1 8
a -1
else if (k > a->key) {
b = p = a->rightChild; d = -1;
2 b 9
} 0 0
0
else { p
b = p = a->leftChild; d = 1; 1
} 1 5 10
while (p != y) {
0 0
if (k > p->key) {
p->bf = -1; p = p->rightChild; -1
p 3 6
} y
else {
p->bf = 1; p = p->leftChild;
p 4
}
}
 Rotation - LL
if (a == NULL)
return true;
else if (a->bf==0 || a->bf+d == 0) {
a->bf += d;
return true;
}
if (d == 1) //left imbalance
{ a b
if (b->bf == 1) //rotation type LL
{ b A B a
a->leftChild = b->rightChild;
B C A
b->rightChild = a;
TA
a->bf = 0; C TB TB TA
b->bf = 0;
}
 Rotation - LR
else //rotation type LR
{
c = b->rightChild;
b->rightChild = c->leftChild; a
a->leftChild = c->rightChild;
A
c->leftChild = b;
c->rightChild = a; b
switch (c->bf) { B TA
case 1:
a->bf = -1; b->bf = 0; break; c
C
case -1: c
b->bf = 1; a->bf = 0; break;
C
case 0: TCL TCR
b->bf = 0; b->bf = 0; break; b a
} B A
c->bf = 0; b = c; //b is the new root
} TCL TCR TA
}
 else { //right imbalance. This is symmetric to left imbalance

}
if (pa == NULL)
root = b; pa
else if (a == pa->leftChild)
9 pa
pa->leftChild = b; a
else 9
b
pa->rightChild = b; b 8 10

return true; 2 10
} 2 a
LL
1 8
1
 Data Structures
Chapter 11: Multiway Search Trees

11-574
 m-way Search Trees
Definition: An m-way search tree, either is
empty or satisfies the following properties:
(1) The root has at most m subtrees and has
the following structures:
n, A0, (K1, A1), (K2, A2), …, (Kn, An)
where the Ai, 0 ≤ i ≤ n ≤ m, are pointers to
subtrees, and the Ki, 1 ≤ i ≤ n ≤ m, are key
values.
(2) Ki < Ki +1, 1  i  n
(3) Let K0 =- and Kn+1 =. All key values in
the subtree Ai are less than Ki +1 and greater
than Ki , 0  i  n 11-575
(4) The subtrees A , 0  i  n , are also m-way
 A Three-way Search Tree
a
T 20, 40
b c d
10, 15 25, 30 45, 50
e
28 node schematic format
a 2, b, (20, c), (40, d)
b 2, 0, (10, 0), (15, 0)
c 2, 0, (25, e), (30, 0)
d 2, 0, (45, 0), (50, 0)
e 1, 0, (28, 0)
11-576
Searching an m-way Search Tree
Suppose to search an m-way search tree T for
the key value x. By searching the keys of the
root, we determine i such that Ki  x < Ki+1.
– If x = Ki, the search is complete.
– If x  Ki, x must be in a subtree Ai if x is in
T.
We proceed to search x in subtree Ai and
continue the search until we find x or
determine that x is not in T.

 m i
Maximum number  ( h
of nodes
m
0i  h 1
1) /(m in
1) a tree of degree m
and height h:
11-577
 B-trees (1)
Definition: A B-tree of order m is an m-way
search tree that either is empty or satisfies
the following properties:
(1) 2  # children of root  m
(2) All nodes other than the root node and
external nodes:
m/2  # children  m
(3) All external nodes are at the same level.

• 2-3 tree is a B-tree of order 3 and 2-3-4 tree

is a B-tree of order 4.
• All B-trees of order 2 are full binary trees. 578
 B-trees (2)
N: # of keys in a B-tree of order m and
height h
h 1 h
2m / 2  1  N  m  1, h  1
h  1  log m / 2 [( N  1) / 2]
Reason: Root has at least 2 keys, each other
node has at least m/2 keys.
For example, a B-tree of order m=200
with # of keys N  21062 will have h ≤
3.
11-579
 A
2-3 tree: B-tree
40
of order 3

B C
2-3-4 tree: B-tree 10 20 80
of order 4 50

10 70 80

5 7 8 30 40 60 75 85 90 92

11-580
Insertion into a B-tree of
Order 5
320 540

430 480

380 395 406 412 451 472 493 506 511

(a) Initial portion of a B-tree

11-581
 395 430 480

380 382 406 412 451 472 493 506 511

(b) After inserting 382 (Split the full node)

395 430 480 508

380 382 406 412 451 472 493 506 511 518

(c) After inserting 518 and 508 11-582

 Insertion into a B-tree of Order 4 (2-3-4 Tree)

87 140

23 61 74 90 100 106 152 186 194

(a) An initial B-tree

97 102 140

23 61 74 90 100 106 152 186 194

(b) Inserting 102 with a left bias 11-583

 87 100 140

23 61 74 90 102 106 152 186 194

(c) Inserting 102 with a right bias

11-584
 Deletion from a B-tree of
Order 5
(i) Rotation: Shift a key from its parent and
its sibling.
80 120 150

A 90 113 126 135 142 B

Delete key 113

80 126 150

A 90 120 135 142 B

11-585
(ii) Combination: Take a key from its parent
and combine with its sibling.

80 126 150

68 73 90 120 135 142 B

Delete key 120

and
combine
80 150

68 73 90 126 135 142 B

11-586
 (iii) Combine, then rotate for its parent

60 170

30 50 80 150 180 220 280

A B C D E
65 72 87 96 153 162 173 178 187 202

Delete 65,
combine
60 180 and rotate

30 50 150 170 220 280

A B C D E
72 80 87 96 153 162 173 178 187 202
11-587
 (iv) Combine, then combine for its parent.

60 180 300
G

30 50 150 170 220 280

A B C D E F
153 162 173 178 187 202

Deleting 173
60 300 and a double
G combination
30 50 150 180 220 280
D E F
A B C
153 162 170 178 187 202
11-588
 Deletion in a B-tree
The combination process may be done up to the
root.
If the root has more than one key,
Done
If the root has only one key
remove the root
The height of the tree decreases by 1.

Insertion, deletion or searching in a B-

tree
requires O(logN) time, where N denotes the
11-589
number of elements in the B-tree.
 B+-trees
• Index node: internal node, storing keys (not
elements) and pointers
• Data node: external node, storing elements
(with their keys)
Data nodes are linked together to form a doubly
linked list. A
20 40 index

B C D
data
10 30 70 80

2 12 20 32 40 71 80
4 16 25 36 50 72 82
6 18 60 84
11-590
 Insertion into the B+-tree
A A
20 40 20 40

B C D B C D
10 30 70 80 10 16 30 70 80

2 12 20 32 40 71 80 2 12 16 20 32 40 71 80
4 16 25 36 50 72 82 4 14 18 25 36 50 72 82
6 18 60 84 6 60 84
(a) Initial B+-tree G (b) 14 inserted (split)
40
A F
20 80
(c) 86 inserted
B C D E
(split)
10 16 30 70 84

2 12 16 20 32 40 71 80 84
4 14 18 25 36 50 72 82 86 11-591
6 60
 Deletion from a B+-tree
A A
20 40 20 40

B C D B C D
10 30 70 80 10 16 30 70 82

2 12 20 32 40 71 80 2 12 16 20 32 40 72 82
4 16 25 36 50 72 82 4 14 18 25 36 50 80 84
6 18 60 84 6 60
(a) Initial B+-tree A (b) 71 deleted
20 40 (borrow)
B C D
10 16 30 70
(c) 80 deleted
2 12 16 20 32 40 72 (combine)
4 14 18 25 36 50 82
11-592
6 60 84
 Deleting 32 (1)
G
40
A F
20 80
B C D E
10 16 30 70 84

2 12 16 20 32 40 71 80 84
4 14 18 25 36 50 72 82 86
6 60
(a) Initial B+-tree
G
40
A F
20 80
B C D E
10 16 70 84

2 12 16 20 40 71 80 84 (b) C is deficient
4 14 18 25 50 72 82 86 11-593
6 36 60
 Deleting 32 (2)
G
40
A F
20 80
B C D E
10 16 70 84

2 12 16 20 40 71 80 84
4 14 18 25 50 72 82 86
6 36 60 (b) C is deficient
G
40
A F
16 80
B C D E
10 20 70 84
(c) After borrowing
2 12 16 20 40 71 80 84 from B
4 14 18 25 50 72 82 86
11-594
6 36 60
 Deleting 86 (1)
G
40
A F
20 80
B C D E
10 16 30 70 84

2 12 16 20 32 40 71 80 84
4 14 18 25 36 50 72 82 86
6 60
(a) Initial B+-tree
G
40
A F
20 80
B C D E
10 16 30 70
(b) E becomes
2 12 16 20 40 71 80 deficient (combine)
4 14 18 25 50 72 82 11-595
6 36 60 84
 Deleting 86 (2)
G
40
A F
20
B C D
10 16 30 70 80 (c) F becomes deficient
(combine)
2 12 16 20 32 40 71 80
4 14 18 25 36 50 72 82
6 60 84

A
20 40
B C D
10 16 30 70 80
(d) After combining
2 12 16 20 32 40 71 80 A,G,F
4 14 18 25 36 50 72 82 11-596
6 60 84