INTRODUCTION TO RANDOMIZATION &

APPROXIMATION ALGORITHMS
Agenda
Part I:

• Introduction to Randomized Algorithms and Approximation Algorithms

• Random Hiring Problem
• String Matching Algorithm
• RabinKarp Algorithm for String Matching
• Approximation Algorithm
• Vertex Covering

Part II:
• Introduction to Complexity Classes
• P Type Problems
• Introduction to NP Type Problems
• Hamiltonian Cycle Problem
• NP Complete Problem
• Satisfiability Problem
• NP Hard Problem
• Examples
 Randomized Algorithm

INPU ALGORITHM OUTPU

T T
Random No

An algorithm that uses random numbers to decide what

to do next anywhere in its logic is called Randomized
Algorithm.
Example of Randomized Algorithm and
calculation of running time

Problem input, x : 5 8 1 9 2
Sequence of random bits [1-5]: 3, 5, 1, 5,3,
….
 Two Types of Randomized Algorithms
and Some Complexity Classes

• A Las Vegas algorithm fails with some probability, but we can tell when it
fails. In particular, we can run it again until it succeeds, which means that
we can eventually succeed with probability 1 (but with a potentially
unbounded running time). Alternatively, we can think of a Las Vegas
algorithm as an algorithm that runs for an unpredictable amount of time
but always succeeds.
QuickSort is an example of a Las Vegas algorithm.
• A Monte Carlo algorithm fails with some probability, but we can’t tell
when it fails. If the algorithm produces a yes/no answer and the failure
probability is significantly less than 1/2, we can reduce the probability of
failure by running it many times and taking a majority of the answers.
The polynomial equality-testing algorithm in an example of a Monte Carlo
algorithm.
 Las Vegas vs. Monte Carlo

• Las Vegas • Monte Carlo

algorithms algorithms
– Always produces a – Allow a small
(correct/optimal) probability for
solution. outputting an
– Like RandQS. incorrect/non-
optimal solution.
– Like Polynomial
Equality testing.
6
 Las Vegas Algorithms
• For example, RandQS is a Las Vegas algorithm.

• A Las Vegas always gives the correct solution

• The only variation from one run to another is its

running time, whose distribution we study.

7
 Polynomial Equivalence Testing
• Given two polynomial functions f(x) and g(x),
where x= {x1, x2, x3...xn}. Test if f(x) = g(x)?
Example 1: f(x) = a3 – b3; g(x) = (a-b) (a2+ab+b2).
Example 2: f(x) = (t+5)2; g(x) = t2 + 25
Monte Carlo algorithms
Select random nos, x= 0, f(x) = 0; g(x)= 25. Allow a small
probability for
X=1, f(x) = 36; g(x) = 26, etc... outputting an
incorrect/non-
optimal solution.
 Randomized quicksort

algorithm RandQS(X ) f
if X is empty then randomization
return;
select x uniformly at random from X ;
let Y = f y 2 X j y < xg;
let Z = f z 2 X j z > xg;
call RandQS(Y );
print x;
call RandQS(Z);
g
9
Randomized QS - An illustration

4 2 7 8 1 9 3 6 5

4 2 1 3 5 7 8 9 6

1 2 4 3 6 7 8 9

1 3 4 6 8 9

3 9

10
 Randomized QS - An illustration

2 7

1 4 6 8

Algorithm Inorder(tree)
1. Traverse the left subtree, i.e., call Inorder(left-
subtree)
3 9
2. Visit the root.
3. Traverse the right subtree, i.e., call Inorder(right-
subtree)

1 2 3 4 5 6 7 89
 2 Questions for RandQS

• Is RandQS correct?
– That is, does RandQS “always” output a sorted
list of X?
• What is the time complexity of RandQS?
– Due to the randomization for selecting x, the
running time for RandQS becomes a random
variable.
– We are interested in the expected time
complexity for RandQS.
Randomized QuickSort always sorts an input array and 
expected worst case time complexity of QuickSort is E(O(nLogn))
12
 Monte Carlo algorithms
• For example, RandEC (the randomized minimum-cut
algorithm) is a Monte Carlo algorithm.
• A Monte Carlo algorithm may sometimes produce a
solution that is incorrect.
• For decision problems, there are two kinds of Monte
Carlo algorithms:
– those with one-sided error
– those with two-sided error

13
 Which is better?
• The answer depends on the application.
• A Las Vegas algorithm is by definition a Monte
Carlo algorithm with error probability 0.
• Actually, we can derive a Las Vegas algorithm
A from a Monte Carlo algorithm B by repeated
running B until we get a correct answer.

14
 What is Approximation algorithm
Introduction:
• An Approximate Algorithm is a way of approach NP-
COMPLETENESS for the optimization problem. This technique
does not guarantee the best solution. The goal of an
approximation algorithm is to come as close as possible to the
optimum value in a reasonable amount of time which is at the
most polynomial time. Such algorithms are called approximation
algorithm or heuristic algorithm.
• Example : For the vertex cover problem, the optimization
problem is to find the vertex cover with fewest vertices, and
the approximation problem is to find the vertex cover with few
vertices.
 5 * 3= 15? Is this verified? Yes!!

What is Approximation algorithm?

Non
Deterministic
Polynomial
Solution may be
found in
polynomial
time, but not
NP sure.
Non Deterministic Polynomial time
Ex: TSP, Knapsack, etc.

P = Polynomial
S = f(N)
Steps = f(Size)
 What is Approximation algorithm?

P = NP
NP

8%
9%

P NE NP
P = NP
Dont Know

83%
Vertex Cover Problem using Approximation
algorithm
• The vertex-cover problem is to find a vertex cover of
minimum size in a given undirected graph. We call such a
vertex cover an optimal vertex cover. This problem is NP-
hard, since the related decision problem is NP-complete.

4 4
2 2
5 5
1 1

6 6
3 3

C* = {1,2}
Vertex Cover Problem
E’={ab, bc, bd, de, ce, cf}
Select arbitary {u,v} belongs to E’
C=C U {u,v}
Return C;
--------------------------------------------
Case 1: Choose {ab}; A = {ab}
C = {a,b}
Remove edges: ab, bd, bc from E’
Choose {ce}; A = {ab, ce}
C = {a,b,c,e}
Remove edges: ab, bd, bc, de, ce, cf
-----------------------------------------
Case 2: Choose {bc}; A = {bc}
C = {b,c}
Remove Edges: ab, bc, bd, ce, cf
Choose {de}, A = {bc , de}
C = {b,c, d,e}
Remove edges: ab, bc, bd, ce, cf, de.
--------------------------------------------
2-approximation algorithm for Vertex Cover
4 • Let A be the set of edges
2
5 chosen by the algorithm ;
1 A = {(1,2)}
6 | A | = 1
3 C = {1,2}
C* = {1,2}
| C* | = 2
4
2 • | A |   ≤   | C* |
• | C |   =   2 | A |
5
1
Therefore: | C |   ≤   2 | C* |
6 That is, |C| cannot be larger than twice
3 the optimal, so is a 2-approximation
algorithm for Vertex Cover.
2-approximation algorithm for Vertex Cover
Let A be the set of edges chosen by the
algorithm ;
| A | = 2
The optimal solution C* must have at least as
many vertices as edges in A:
C = {b,c, d,e}
| C | = 4;

C* = {b,c, d,e}
| C* | = 4

• | A |   ≤   | C* |
• | C |   =   2 | A |

Therefore: | C |   ≤   2 | C* |

That is, |C| cannot be larger than twice the
optimal, so is a 2-approximation algorithm for
Vertex Cover.
 Example of Approx algorithm – Vertex cover

O(E)
O(V)

Line 1 initializes C to the empty set.

Line 2 sets E' to be a copy of the edge set E[G] of the graph.
The loop on lines 3-6 repeatedly picks an edge (u, v) from E',
adds its endpoints u and v to C, and deletes all edges in E' that
are covered by either u or v.
The running time of this algorithm is O(E+V)