Randomized Algorithms
Introduction
Types
- Las Vegas (Ex. Randomized Quick Sort)
- Monte Carlo (Ex. Karger's Min Cut)
Introduction
• A randomized algorithm is one that receives, in
addition to its input data, a stream of random
bits that it can use for the purpose of making
random choices.
• Even for a fixed input, different runs of a
randomized algorithm may give different results.
Example
• Search for an element 'x' in an array A.
1. search (int A[], int x)
2. for i = 0 to A.length
Simple Algorithm
3. check if element exists.
4. end for
search (int A[], int x) Randomized Algorithm
while(1)
Randomly select one element out of A.length elements.
until 'x' is found
end
Contd…
• An extremely important tool for the construction
of algorithms in a wide range of applications.
• Principal advantages:
– Relatively smaller execution time or space
requirement than that of the best known
deterministic algorithm for the same problem.
– Extremely simple to understand and to implement.
• Running time of randomized algorithms is given
as an expected values.
Types
• Las-Vegas
– Always gives correct results or it informs about the
failure.
– Running time is a random variable.
– Example: Randomized quicksort, where the pivot is
chosen randomly, but the result is always sorted.
• Monte Carlo
– May produce an incorrect result with some
probability.
– Running time is fixed.
– Example: Karger’s Min Cut, to get a min-cut for a given
graph G with some probability.
Example – Search (int A[], int x)
searchLasVegas(int A[], int x)
while(true)
searchMonteCarlo(int A[], int x)
Randomly select one
element out of A.length i = 0; flag = false
elements. while(i <= <some number>)
if (found) Randomly select one
return true element out of A.length
elements.
end
i++;
if (found)
flag = true
end
return flag
QUICKSORT(A, p, r)
• QUICKSORT(A, p, r) 1. PARTITION(A, p, r)
1. if p < r 2. x = A[r]
2. q = PARTITION(A, p, r) 3. i = p – 1
3. QUICKSORT(A, p, q – 1) 4. for j = p to r – 1
5. if A[j] ≤ x
4. QUICKSORT(A, q + 1, r)
6. i=i+1
• To sort an array A with n
elements, the first call to 7. Exchange A[i] with A[j]
QUICKSORT is made with 8. Exchange A[i + 1] with A[r]
p = 0 and r = n – 1. 9. return i + 1
Example: 2, 8, 7, 1, 3, 5, 6, 4
p r p r
2 8 7 1 3 5 6 4 2 1 3 8 7 5 6 4
i j i j
p r p r
2 8 7 1 3 5 6 4 2 1 3 8 7 5 6 4
i j i j
p r p r
2 8 7 1 3 5 6 4 2 1 3 8 7 5 6 4
i j i j
p r p r
2 8 7 1 3 5 6 4 2 1 3 4 7 5 6 8
i j i j
p r
2 1 7 8 3 5 6 4
i j
Randomized Quick Sort (Las Vegas)
• RANDOMIZED-QUICKSORT(A, p, r)
1. if p < r
2. q = RANDOMIZED-PARTITION(A, p, r)
3. RANDOMIZED-QUICKSORT(A, p, q – 1)
4. RANDOMIZED-QUICKSORT(A, q + 1, r)
• RANDOMIZED-PARTITION(A, p, r)
5. i = RANDOM(p, r)
6. Exchange A[r] with A[i]
7. return PARTITION(A, p, r)
Example
0 1 2 3 4 5 6 7
5 3 8 9 4 7 6 1
• Generate a random number in between 0 and 7.
– Let the number be 5.
– Exchange A[5] with A[7]
0 1 2 3 4 5 6 7
5 3 8 9 4 1 6 7
Contd…
5 3 8 9 4 1 6 7 5 3 4 9 8 1 6 7
i pj r p i j r
5 3 8 9 4 1 6 7 5 3 4 1 8 9 6 7
pi j r p i j r
5 3 8 9 4 1 6 7 5 3 4 1 6 9 8 7
p i j r p i jr
5 3 8 9 4 1 6 7 5 3 4 1 6 7 8 9
p i j r p i jr
5 3 8 9 4 1 6 7
p i j r
5 3 4 1 6 7 8 9
Contd… 5 6 4 1 3 7 8 9
1 3 4 5 6 7 8 9
1 3 4 5 6 7 8 9
1 3 6 5 4 7 8 9
1 3 4 5 6 7 8 9
1 3 4 5 6 7 8 9
1 3 4 5 6 7 8 9
Exchanged 1 3 4 5 6 7 8 9
1 3 4 5 6 7 9 8
Final 1 3 4 5 6 7 8 9
1 3 4 5 6 7 8 9
Karger’s Min-cut (Monte Carlo)
• Minimum cut of an undirected graph G =
(V,E) is a partition of the nodes into two
groups V1 and V2, so that the number of edges
between V1 and V2 is minimized.
– V1 ∩ V2 = Ø and V1 𝖴 V2 = V
Example:
Size of minimum cut is two
with node partitions as
V1 = {a, b, e, f} and
V2 = {c, d, g, h}.
Contd…
1. Repeat until just two nodes remain:
2. Pick an edge of G at random and collapse
its two endpoints into a single node.
• For the two remaining nodes u1 and u2, set
V1 = {nodes in u1} and V2 = {nodes in u2}
Karger's (basic algorithm)
• begin
• i=1
• repeat
• repeat
• Take a random edge (u,v) ∈ E in G
• replace u and v with the contraction u'
• until only 2 nodes remain
• obtain the corresponding cut result Ci
• i=i+1
• until i = m
• output the minimum cut among C1,C2,...,Cm.
• end
A, B D, A, B
Example Select D-AB with
D
– 1i = 1 probability 1/4
A C C
D B
ACD
C AC Select D-AC with
probability 2/4
i=2 B
D B
Ci Result is minimum Ci value
i Probability
(# of edges) (or Ci with higher
1 3 1/4 = 0.25 probability). Min cut is 2 with
2 2 2/4 = 0.5 vertex sets {A, C, D} and {B}.
Example – 2
a b c d
h g f e
i=1
1 a b c d 14 edges to choose from. Pick
b – c with probability 1/14.
h g f e
2 a bc d 13 edges to choose from. Pick
g – f with probability 1/13.
h g f e
3 a bc d 12 edges to choose from. Pick
bc – gf with probability 2/12.
h gf e
i=1
4 a bcgf d 10 edges to choose from. Pick
a – h with probability 1/10.
h e
5 ah bcgf d 9 edges to choose from. Pick
d – e with probability 1/9.
6 ah bcgf de 8 edges to choose from. Pick
bcgf – de with probability 4/8.
7 ah bcgfde Done. Min cut is 4 with
probability 0.5. Vertex sets
are {a, h} and {b, c, d, e, f, g}
i=2
1 a b c d 14 edges to choose from. Pick
d – e with probability 1/14.
h g f e
2 a b c de 13 edges to choose from. Pick
a – h with probability 1/13.
h g f
3 ah b c de 12 edges to choose from. Pick
b – g with probability 1/12.
g f
i=2
4 ah bg c de 11 edges to choose from. Pick
c – f with probability 1/11.
5 ah bg cf de 10 edges to choose from. Pick
cf – de with probability 4/10.
6 ah bg cfde 6 edges to choose from. Pick
ah – bg with probability 4/6.
7 Done. Min cut is 2 with
ahbg cfde probability 0.67. Vertex sets
are {a, b, g, h} and {c, d, e, f}
Contd…
a b c d
h g f e
i Ci (# of edges) Probability Vertex Sets
{a, h}
1 4 0.5
{b, c, d, e, f, g}
{a, b, g, h}
2 2 0.67
{c, d, e, f}
