Sorting

(Insertion and Merge)

1/67
Insertion Sort
2/67
 Insertion Sort
 Idea: like sorting a hand of playing cards
 Start with an empty left hand and the cards facing
down on the table.
 Remove one card at a time from the table, and
insert it into the correct position in the left hand
compare it with each of the cards already in the
hand, from right to left
 The cards held in the left hand are sorted

these cards were originally the top cards of the

pile on the table

3/67
 Insertion Sort

To insert 12, we need to

make room for it by moving
first 36 and then 24.

Is 12>36 ?
NO
So move 36 to right to make space for one element

4/67
 Insertion Sort

Is 12>24 ?
NO
So move 24 right to the created space for new element

5/67
 Insertion Sort

Is 12>10 ?
YES
So move 12 to the space created for new element

6/67
Insertion Sort

7/67
Insertion Sort

8/67
 Insertion Sort
 Iteration i. Repeatedly swap element i with the one to its left if
smaller.

 After ith iteration, a[0] through a[i] contain first i+1

elements in ascending order.

Array index 0 1 2 3 4 5 6 7 8 9

Value 2.78 7.42 0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 0: step 0.
9/67
 Insertion Sort
 Iteration i. Repeatedly swap element i with the one to its left if
smaller.

 After ith iteration, a[0] through a[i] contain first i+1

elements in ascending order.

Array index 0 1 2 3 4 5 6 7 8 9

Value 2.78 7.42 0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 1: step 0.
10/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 2.78 0.56

7.42 7.42
0.56 1.12 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 2: step 0.
11/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 2.78 2.78

0.56 0.56 7.42 1.12 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 2: step 1.

12/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 2.78 7.42 1.12 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 2: step 2.

13/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 2.78 7.42

1.12 1.12
7.42 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 3: step 0.

14/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12

2.78 2.78
1.12 7.42 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 3: step 1.

15/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12 2.78 7.42 1.17 0.32 6.21 4.42 3.14 7.71

Iteration 3: step 2.

16/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12 2.78 1.17

7.42 7.42
1.17 0.32 6.21 4.42 3.14 7.71

Iteration 4: step 0.

17/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12 2.78

1.17 1.17
2.78 7.42 0.32 6.21 4.42 3.14 7.71

Iteration 4: step 1.

18/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12 1.17 2.78 7.42 0.32 6.21 4.42 3.14 7.71

Iteration 4: step 2.

19/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12 1.17 2.78 0.32

7.42 0.32
7.42 6.21 4.42 3.14 7.71

Iteration 5: step 0.

20/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12 1.17 2.78

0.32 0.32
2.78 7.42 6.21 4.42 3.14 7.71

Iteration 5: step 1.
21/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 1.12 1.17

0.32 0.32
1.17 2.78 7.42 6.21 4.42 3.14 7.71

Iteration 5: step 2.

22/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 0.32

1.12 0.32
1.12 1.17 2.78 7.42 6.21 4.42 3.14 7.71

Iteration 5: step 3.
23/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.56 0.32

0.32 0.56 1.12 1.17 2.78 7.42 6.21 4.42 3.14 7.71

Iteration 5: step 4.

24/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 7.42 6.21 4.42 3.14 7.71

Iteration 5: step 5.

25/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 7.42

6.21 6.21
7.42 4.42 3.14 7.71

Iteration 6: step 0.

26/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 6.21 7.42 4.42 3.14 7.71

Iteration 6: step 1.

27/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 6.21 7.42

4.42 4.42
7.42 3.14 7.71

Iteration 7: step 0.

28/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 6.21

4.42 4.42
6.21 7.42 3.14 7.71

Iteration 7: step 1.

29/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 4.42 6.21 7.42 3.14 7.71

Iteration 7: step 2.

30/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 4.42 6.21 7.42

3.14 3.14
7.42 7.71

Iteration 8: step 0.

31/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 4.42 3.14

6.21 3.14
6.21 7.42 7.71

Iteration 8: step 1.

32/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 4.42

3.14 3.14
4.42 6.21 7.42 7.71

Iteration 8: step 2.

33/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 3.14 4.42 6.21 7.42 7.71

Iteration 8: step 3.

34/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 3.14 4.42 6.21 7.42 7.71

Iteration 9: step 0.

35/67
 Insertion Sort

Array index 0 1 2 3 4 5 6 7 8 9

Value 0.32 0.56 1.12 1.17 2.78 3.14 4.42 6.21 7.42 7.71

Iteration 10: DONE.

36/67
 Insertion Sort

input array

5 2 4 6 1 3

at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted

37/79
Insertion Sort

38/79
 INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

for j ← 2 to n a1 a2 a3 a4 a5 a6 a7 a8

do key ← A[ j ] key
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
• Insertion sort – sorts the elements in place
39/79
 Is it right code for Insertion Sort?
for (i = 1 ; i < n ; i++)
{ key = A [ i ];
j = i - 1;
while(j >= 0 && A [ j ] > key)
{ A [ j+1 ] = A [ j ];
j--;
}
A [ j+1 ] = key;
}
40/67
#include <stdio.h>
main()
{ int i,n=12,j,key, A[12] = { 97,556,43,-5,6,404,55,3,22,-122,4,35};
for( i=0;i<n;i++) printf(" %d ",A[i]);
for (i = 1 ; i < n ; i++)
{ key = A [ i ];
j = i - 1;
while(j >= 0 && A [ j ] > key)
{ A [ j+1 ] = A [ j ];
j--;
}
A [ j+1 ] = key;
}
41/67
printf("\n");

for( i=0;i<n;i++)

printf(" %d ",A[i]);

42/67
 Analysis of Insertion Sort
INSERTION-SORT(A) cost times
for j ← 2 to n c1 n
do key ← A[ j ] c2 n-1
Insert A[ j ] into the sorted sequence A[1 . . j -1] 0 n-1
i←j-1 c4 n-1
∑
n
while i > 0 and A[i] > key c5 j =2 j
t
c6 ∑
n
do A[i + 1] ← A[i] j =2
(t j − 1)
c7 ∑
n
i←i–1 j =2
(t j − 1)
A[i + 1] ← key c8 n-1
tj: # of times the while statement is executed at iteration j

T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑ t j + c6 ∑ (t j − 1) + c7 ∑ (t j − 1) + c8 (n − 1)
n n n

j =2 j =2 j =2

43/79
 Best Case Analysis
• The array is already sorted “while i > 0 and A[i] > key”
– A[i] ≤ key upon the first time the while loop test is
run (when i = j -1)

– tj = 1

• T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) + c8(n-

1) = (c1 + c2 + c4 + c5 + c8)n + (c2 + c4 + c5 + c8)

= an + b = Θ(n)
T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑ t j + c6 ∑ (t j − 1) + c7 ∑ (t j − 1) + c8 (n − 1)
n n n

j =2 j =2 j =2

44/79
 Worst Case Analysis
• The array is in reverse sorted order “while i > 0 and A[i] > key”
– Always A[i] > key in while loop test
– Have to compare key with all elements to the left of the j-th
position ⇒ compare with j-1 elements ⇒ tj = j

n(n + 1)
n n
n(n + 1) n
n(n − 1)
using
=j 1 =
∑ j=
2
>∑ j=
=
j 2 = 2
> ∑ ( j − 1) =
−1 =
j 2 2
we have:

 n( n + 1)  n( n − 1) n( n − 1)
T ( n ) = c1n + c2 ( n − 1) + c4 ( n − 1) + c5  − 1 + c6 + c7 + c8 ( n − 1)
 2  2 2

= an 2 + bn + c a quadratic function of n

• T(n) = Θ(n2) order of growth in n2

T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑ t j + c6 ∑ (t j − 1) + c7 ∑ (t j − 1) + c8 (n − 1)
n n n

j =2 j =2 j =2
45/79
 Comparisons and Exchanges in
Insertion Sort
INSERTION-SORT(A) cost times
for j ← 2 to n c1 n
do key ← A[ j ] c2 n-1
Insert A[ j ] into the sorted sequence A[1 . . j -1] 0 n-1
i←j-1 ≈ n2/2 comparisons c4 n-1
c5 ∑
n
while i > 0 and A[i] > key j =2 j
t

c6 ∑
n
do A[i + 1] ← A[i] j =2
(t j − 1)

≈ n2/2 exchanges c7 ∑
n
i←i–1 j =2
(t j − 1)

A[i + 1] ← key c8 n-1

46/79
 Insertion Sort - Summary
 Advantages

 Good running time for “almost sorted” arrays Θ(n)

 Disadvantages

 Θ(n2) running time in worst and average case

 ≈ n2/2 comparisons and exchanges

47/67
Merge Sort

48/67
 Merge Sort
• It is an in-place sorting algorithm; It uses no auxiliary data
structures (extra space) while sorting.
• Partition the n > 1 elements into two smaller instances.
• First (n/2) elements define one of the smaller instances;
remaining (n/2) elements define the second smaller instance.
• Each of the two smaller instances is sorted recursively.
• The sorted smaller instances are combined using a process
called merge.
• Complexity is O(n log n).
49/67
 Merge Two Sorted Lists
A = (2, 5, 6)

B = (1, 3, 8, 9, 10)

C=()

Compare smallest elements of A and B and merge smaller into C.

A = (2, 5, 6)

B = (3, 8, 9, 10)

C = (1)

50/67
 Merge Two Sorted Lists
A = (5, 6)

B = (3, 8, 9, 10)

C = (1, 2)

A = (5, 6)

B = (8, 9, 10)

C = (1, 2, 3)

A = (6)

B = (8, 9, 10)

C = (1, 2, 3, 5)
51/67
 Merge Two Sorted Lists
A=()

B = (8, 9, 10)

C = (1, 2, 3, 5, 6)

When one of A and B becomes empty, append the other list to C.

O(1) time needed to move an element into C.

Total time is O(n + m), where n and m are, respectively, the

number of elements initially in A and B.

52/67
 Merge Sort

[8, 3, 13, 6, 2, 14, 5, 9, 10, 1, 7, 12, 4]

[8, 3, 13, 6, 2, 14, 5] [9, 10, 1, 7, 12, 4]

[8, 3, 13, 6] [2, 14, 5] [9, 10, 1] [7, 12, 4]

[8, 3] [13, 6] [2, 14] [5] [9, 10] [1] [7, 12] [4]

[8] [3] [13] [6] [2] [14] [9] [10] [7] [12]

53/67
 Merge Sort

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13,14]

[2, 3, 5, 6, 8, 13, 14] [1, 4, 7, 9, 10,12]

[3, 6, 8, 13] [2, 5, 14] [1, 9, 10] [4, 7, 12]

[3, 8] [6, 13] [2, 14] [5] [9, 10] [1] [7, 12] [4]
[8] [3] [13] [6] [2] [14] [9] [10] [7] [12]

54/67
 Time Complexity
 Let t(n) be the time required to sort n
elements.
 t(0) = t(1) = c, where c is a constant.
 When n > 1,
 t(n) = t(ceil(n/2)) + t(floor(n/2)) + dn,
 where d is a constant.
 To solve the recurrence, assume n is a power
of 2 and use repeated substitution.
 t(n) = O(n log n).
55/67
 Merge Sort
 Downward pass over the recursion tree.

 Divide large instances into small ones.

 Upward pass over the recursion tree.

 Merge pairs of sorted lists.

 Number of leaf nodes is n.

 Number of nonleaf nodes is n-1.

56/67
 Time Complexity
Downward pass.
O(1) time at each node.
O(n) total time at all nodes.
Upward pass.
O(n) time merging at each level that
has a nonleaf node.
Number of levels is O(log n).
Total time is O(n log n).

57/67
 Nonrecursive Version
 Eliminate downward pass.

 Start with sorted lists of size 1 and do

pairwise merging of these sorted lists as

in the upward pass.

58/67
 Nonrecursive Merge Sort

[8] [3] [13] [6] [2] [14] [5] [9] [10] [1] [7] [12] [4]

[3, 8] [6, 13] [2, 14] [5, 9] [1, 10] [7, 12] [4]

[3, 6, 8, 13] [2, 5, 9, 14] [1, 7, 10, 12] [4]

[2, 3, 5, 6, 8, 9, 13, 14] [1, 4, 7, 10, 12]

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14]

59/67
 Complexity
 Sorted segment size is 1, 2, 4, 8, …
 Number of merge passes is ceil(log2n).
 Each merge pass takes O(n) time.
 Total time is O(n log n).
 Need O(n) additional space for the merge.
 Merge sort is slower than insertion sort when n <=
15 (approximately). So define a small instance to be
an instance with n <= 15.
 Sort small instances using insertion sort.
 Start with segment size = 15.

60/67
 Natural Merge Sort
 Initial sorted segments are the naturally
ocurring sorted segments in the input.
 Input = [8, 9, 10, 2, 5, 7, 9, 11, 13, 15, 6, 12,
14].
 Initial segments are:
 [8, 9, 10] [2, 5, 7, 9, 11, 13, 15] [6, 12, 14]
 2 (instead of 4) merge passes suffice.
 Segment boundaries have a[i] > a[i+1].

61/67
// Merge Sort C Program
#include<stdlib.h>
#include<stdio.h>
int main()
{ //int arr[] = {85, 24, 63, 45, 17, 31, 96, 50};
int arr[] = {-5,406,3,425,317,31,6,50,55,54,66,78,99,43,72};
int arr_size = sizeof(arr)/sizeof(arr[0]);
printf("\n Given array is : ");
printArray(arr, arr_size);
mergeSort(arr, 0, arr_size - 1);
printf("\n Sorted array is : ");
printArray(arr, arr_size);
}
62/67
void printArray(int A[], int size)
{ int i;
for (i=0; i < size; i++)
printf("%d ", A[i]);
printf("\n");
}
void mergeSort(int arr[], int l, int r)
{ if (l < r)
{ int m = l+(r-l)/2;
mergeSort(arr, l, m);
mergeSort(arr, m+1, r);
merge(arr, l, m, r);
}
} 63/67
// Merge Function
void merge(int arr[], int l, int m, int r)
{ int i, j, k;
int n1 = m - l + 1;
int n2 = r - m;
int L[n1], R[n2]; // Create temp arrays
for (i = 0; i < n1; i++)
L[i] = arr[l + i]; // Copy data to temp array
for (j = 0; j < n2; j++)
R[j] = arr[m + 1 + j];

64/67
// Merge temp arrays
i = 0;
j = 0;
k = l;
while (i < n1 && j < n2)
{ if (L[i] <= R[j])
{ arr[k] = L[i];
i++;
}
else
{ arr[k] = R[j];
j++;
}
k++;
} 65/67
while (i < n1) // Copy the remaining elements of L[]
{ arr[k] = L[i];
i++;
k++;
}
while (j < n2)// Copy the remaining elements of R[]
{ arr[k] = R[j];
j++;
k++;
}
}
66/67
Given array is : 85 24 63 45 17 31 96 50

Sorted array is : 17 24 31 45 50 63 85 96

Given array is : -5 406 3 425 317 31 6 50 55 54 66 78 99 43 72

Sorted array is : -5 3 6 31 43 50 54 55 66 72 78 99 317 406 425

67/67
Given array is : -55 -44 -33 0 3 6 10 15 24 26 37 49 53 66 70 72

Sorted array is : -55 -44 -33 0 3 6 10 15 24 26 37 49 53 66 70 72

Given array is : 555 333 244 100 93 86 60 55 44 36 27 25 23 16

72

Sorted array is : 2 7 16 23 25 27 36 44 55 60 86 93 100 244 333

555

68/67