DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS

OM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

since gcd (m, o) = m. {the last value of m is also the gcd of the initial m & n.} The third procedure is as follows:
Step 1: Find the prime factors of m.
The structured description of this algorithm is: Step 2: Find the prime factors of n.
Step 1: If n=0, return the value of m as the answer and stop; otherwise, proceed to step2. Step 3: Identify all the common factors in the two prime expansions found in step 1 & 2. (If p is a
Step 2: Divide m by n and assign the value of the remainder to r. common factor occurring pm & pn times is m and n, respectively,it should be repeated min { pm,
Step 3: Assign the value of n to m and the value of r to n. Go to step 1. pn } times.).
Step 4: Compute the product of all the common factors and return it as gcd of the numbers given.
1.1.1 Euclid”s algorithm :
Example: 60 = 2.2.3.5
ALGORITHM Euclid(m, n) 24 = 2.2.2.3
//Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers m and n gcd (60,24) = 2.2.3 = 12 .

P
//Output: Greatest common divisor of m and n
while n _= 0 do This procedure is more complex and ambiguity arises since the prime factorization is not defined.

AP

AP
r ←m So to make it as an efficient algorithm, incorporate the algorithm to find the prime factors.
mod n
m←n 1.2 FUNDAMENTALS OF ALGORITHMIC PROBLEM SOLVING
n←r
return m Algorithms can be considered to be procedural solutions to problems. There are certain
R

R
steps to be followed in designing and analyzing an algorithm. Code the algorithm
This algorithm comes to a stop, when the 2nd no becomes 0. The second number of the pair gets
smaller with each iteration and it cannot become negative. Indeed, the new value of n on the next
CO

CO
iteration is m mod n, which is always smaller than n. hence, the value of the second number in the Understand the problem
pair eventually becomes 0, and the algorithm stops.

Example: gcd (60, 24) = gcd (24,12) = gcd (12,0) = 12.

Decide on: Computational means,
U

U
The second method for the same problem is: obtained from the definition itself. i.e., gcd of m & n exact vs.approximate problem solving,
is the largest integer that divides both numbers evenly. Obviously, that number cannot be greater data structure,
than the second number (or) smaller of these two numbers,which we will denote by t = min {m, n algorithm design technique
ST

ST
}. So start checking whether t divides both m and n: if it does t is the answer ; if it doesn’t t is
decreased by 1 and try again. (Do this repeatedly till you reach 12 and then stop for the example
given below) Design an algorithm

1.1.2 Consecutive integer checking algorithm:

Step 1: Assign the value of min {m, n} to t. Prove the correctness
Step 2: Divide m by t. If the remainder of this division is 0, go to step 3; otherwise go to step 4.
Step 3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer
and stop; otherwise, proceed to step 4.
Step 4: Decrease the value of t by 1. Go to step 2. Analyze the algorithm
Note: this algorithm, will not work when one of its input is zero. So we have to specify the range
of input explicitly and carefully.
Code the algorithm

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

∗ Understanding the problem To specify algorithm flowchart is used which is a method of expressing an algorithm by a
collection of connected geometric shapes consisting descriptions of the algorithm’s steps.
An input to an algorithm specifies an instance of the problem the algorithm solves. It’s also
important to specify exactly the range of instances the algorithm needs to handle. Before this we have ∗ Proving an Algorithm’s correctness
to clearly understand the problem and clarify the doubts after leading the problems description.
Correct algorithm should work for all possible inputs. Correctness has to be proved for every algorithm. To prove that the algorithm gives the
required result for every legitimate input in a finite amount of time. For some algorithms, a proof of
∗Ascertaining the capabilities of a computational Device
correctness is quite easy; for others it can be quite complex. A technique used for proving correctness
s by mathematical induction because an algorithm’s iterations provide a natural sequence of steps
The second step is to ascertain the capabilities of a machine. The essence of von-Neumann
needed for such proofs. But we need one instance of its input for which the algorithm fails. If it is
machines architecture is captured by RAM, Here the instructions are executed one after another, one
incorrect, redesign the algorithm, with the same decisions of data structures design technique etc.
operation at a time, Algorithms designed to be executed on such machines are called sequential
The notion of correctness for approximation algorithms is less straightforward than it is for exact

P
algorithms. An algorithm which has the capability of executing the operations concurrently is called
algorithm. For example, in gcd (m,n) two observations are made. One is the second number gets
parallel algorithms. RAM model doesn’t support this.
smaller on every iteration and the algorithm stops when the second number becomes 0.

AP

AP
∗ Choosing between exact and approximate problem solving ∗ Analyzing an algorithm

The next decision is to choose between solving the problem exactly or solving it There are two kinds of algorithm efficiency: time and space efficiency. Time efficiency
approximately. Based on this, the algorithms are classified as exact and approximation algorithms. indicates how fast the algorithm runs; space efficiency indicates how much extra memory the
There are three issues to choose an approximation algorithm. First, there are certain problems like algorithm needs. Another desirable characteristic is simplicity. Simper algorithms are easier to
R

R
extracting square roots, solving non-linear equations which cannot be solved exactly. Secondly, if the understand and program, the resulting programs will be easier to debug. For e.g. Euclid’s algorithm to
problem is complicated it slows the operations. E.g. traveling salesman problem. Third, this algorithm fid gcd (m,n) is simple than the algorithm which uses the prime factorization. Another desirable
can be a part of a more sophisticated algorithm that solves a problem exactly.
CO

CO
characteristic is generality. Two issues here are generality of the problem the algorithm solves and the
∗Deciding on data structures range of inputs it accepts. The designing of algorithm in general terms is sometimes easier. For eg, the
general problem of computing the gcd of two integers and to solve the problem. But at times
Data structures play a vital role in designing and analyzing the algorithms. Some of the designing a general algorithm is unnecessary or difficult or even impossible. For eg, it is unnecessary
algorithm design techniques also depend on the structuring data specifying a problem’s instance. to sort a list of n numbers to find its median, which is its [n/2]th smallest element. As to the range of
U

U
Algorithm + Data structure = Programs inputs, we should aim at a range of inputs that is natural for the problem at hand.
∗ Coding an algorithm

∗Algorithm Design Techniques

ST

ST
Programming the algorithm by using some programming language. Formal verification is done
An algorithm design technique is a general approach to solving problems algorithmically that for small programs. Validity is done thru testing and debugging. Inputs should fall within a range and
is applicable to a variety of problems from different areas of computing. Learning these techniques are hence require no verification. Some compilers allow code optimization which can speed up a program
important for two reasons, First, they provide guidance for designing for new problems. Second, by a constant factor whereas a better algorithm can make a difference in their running time. The
algorithms are the cornerstones of computer science. Algorithm design techniques make it possible to analysis has to be done in various sets of inputs.
classify algorithms according to an underlying design idea; therefore, they can serve as a natural way A good algorithm is a result of repeated effort & work. The program’s stopping / terminating
to both categorize and study algorithms. condition has to be set. The optimality is an interesting issue which relies on the complexity of the
problem to be solved. Another important issue is the question of whether or not every problem can be
∗Methods of specifying an Algorithm solved by an algorithm. And the last, is to avoid the ambiguity which arises for a complicated
algorithm.
A psuedocode, which is a mixture of a natural language and programming language like
constructs. Its usage is similar to algorithm descriptions for writing psuedocode there are some
dialects which omits declarations of variables, use indentation to show the scope of the statements
such as if, for and while. Use → for assignment operations, (//) two slashes for comments.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

1.3 IMPORTANT PROBLEM TYPES 3. String processing

The two motivating forces for any problem is its practical importance and some specific A String is a sequence of characters. It is mainly used in string handling algorithms. Most
characteristics. common ones are text strings, which consists of letters, numbers and special characters. Bit strings
The different types are: consist of zeroes and ones. The most important problem is the string matching, which is used for
1. Sorting searching a given word in a text. For e.g. sequential searching and brute- force string matching
2. Searching algorithms.
3. String processing
4. Graph problems 4. Graph problems
5. Combinatorial problems
6. Geometric problems One of the interesting area in algorithmic is graph algorithms. A graph is a collection of points
7. Numerical problems. called vertices which are connected by line segments called edges. Graphs are used for modeling a

P
wide variety of real-life applications such as transportation and communication networks.
1. Sorting It includes graph traversal, shortest-path and topological sorting algorithms. Some graph

AP

AP
Sorting problem is one which rearranges the items of a given list in ascending order. We problems are very hard, only very small instances of the problems can be solved in realistic amount of
usually sort a list of numbers, characters, strings and records similar to college information about their time even with fastest computers.
students, library information and company information is chosen for guiding the sorting technique. There are two common problems: the traveling salesman problem, finding the shortest tour
For eg in student’s information, we can sort it either based on student’s register number or by their through n cities that visits every city exactly once
names. Such pieces of information are called a key. The graph-coloring problem is to assign the smallest number of colors to vertices of a graph so
R

R
The most important when we use the searching of records. There are different types of sorting that no two adjacent vertices are of the same color. It arises in event-scheduling problem, where the
algorithms. There are some algorithms that sort an arbitrary of size n using nlog2n comparisons, On events are represented by vertices that are connected by an edge if the corresponding events cannot be
the other hand, no algorithm that sorts by key comparisons can do better than that. Although some scheduled in the same time, a solution to this graph gives an optimal schedule.
CO

CO
algorithms are better than others, there is no algorithm that would be the best in all situations. Some
algorithms are simple but relatively slow while others are faster but more complex. Some are suitable 5. Combinatorial problems
only for lists residing in the fast memory while others can be adapted for sorting large files stored on a
disk, and so on. The traveling salesman problem and the graph-coloring problem are examples of combinatorial
There are two important properties. The first is called stable, if it preserves the relative order of problems. These are problems that ask us to find a combinatorial object such as permutation,
U

U
any two equal elements in its input. For example, if we sort the student list based on their GPA and if combination or a subset that satisfies certain constraints and has some desired (e.g. maximizes a value
two students GPA are the same, then the elements are stored or sorted based on its position. The or minimizes a cost).
second is said to be ‘in place’ if it does not require extra memory. There are some sorting algorithms These problems are difficult to solve for the following facts. First, the number of combinatorial
ST

ST
that are in place and those that are not. objects grows extremely fast with a problem’s size. Second, there are no known algorithms, which are
solved in acceptable amount of time.
2. Searching
6. Geometric problems
The searching problem deals with finding a given value, called a search key, in a given set.
The searching can be either a straightforward algorithm or binary search algorithm which is a different Geometric algorithms deal with geometric objects such as points, lines and polygons. It also
form. These algorithms play a important role in real-life applications because they are used for storing includes various geometric shapes such as triangles, circles etc. The applications for these algorithms
and retrieving information from large databases. Some algorithms work faster but require more are in computer graphic, robotics etc.The two problems most widely used are the closest-pair
memory, some are very fast but applicable only to sorted arrays. Searching, mainly deals with problems, given ‘n’ points in the plane, find the closest pair among them. The convex-hull problem is
addition and deletion of records. In such cases, the data structures and algorithms are chosen to to find the smallest convex polygon that would include all the points of a given set.
balance among the required set of operations.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

7. Numerical problems 1.4.1.2 Units for measuring Running time

This is another large special area of applications, where the problems involve mathematical objects We can use some standard unit of time to measure the running time of a program
of continuous nature: solving equations computing definite integrals and evaluating functions and so implementing the algorithm. The drawbacks to such an approach are: the dependence on the speed of
on. These problems can be solved only approximately. These require real numbers, which can be a particular computer, the quality of a program implementing the algorithm.
represented in a computer only approximately. If can also lead to an accumulation of round-off errors.
The drawback to such an approach are : the dependence on the speed of a particular computer,
The algorithms designed are mainly used in scientific and engineering applications.
the quality of a program implementing the algorithm, the compiler used to generate its machine code
and the difficulty in clocking the actual running time of the program. Here, we do not consider these
1.4 FUNDAMENTALS OF THE ANALYSIS OF ALGORITHM EFFICIENCY
extraneous factors for simplicity.
One possible approach is to count the number of times each of the algorithm’s operations is
The American Heritage Dictionary defines “analysis” as the “seperation of an intellectual or
executed. The simple way, is to identify the most important operation of the algorithm, called the

P
substantantial whole into its constituent parts for individual study”.
basic operation , the operation contributing the most to the total running time and compute the umber
Algorithm’s efficiency is determined with respect to two resources: running time and memory
of times the basic operation is executed.

AP

AP
space. Efficiency is studied first in quantitative terms unlike simplicity and generality. Second, give
The basic operation is usually the most time consuming operation in the algorithm’s inner
the speed and memory of today’s computers, the efficiency consideration is of practical importance.
most loop. For example, most sorting algorithm works by comparing elements (keys), of a list being
The algorithm’s efficiency is represented in three notations: 0 (“big oh”), Ω (“big omega”) and
sorted with each other; for such algorithms, the basic operation is the key comparison.
θ (“big theta”). The mathematical analysis shows the framework systematically applied to analyzing
the efficiency of nonrecursive algorithms. The main tool of such an analysis is setting up a sum Let Cop be the time of execution of an algorithm’s basic operation on a particular computer
representing the algorithm’s running time and then simplifying the sum by using standard sum and let c(n) be the number of times this operations needs to be executed for this algorithm. Then we
R

R
manipulation techniques. can estimate the running time, T (n) as: T (n) ∼ Cop c(n)
1.4.1 ANALYSIS FRAMEWORK
Here, the count c(n) does not contain any information about operations that are not basic and
CO

CO
For analyzing the efficiency of algorithms the two kinds are time efficiency and space in tact, the count itself is often computed only approximately. The constant Cop is also an
efficiency. Time efficiency indicates how fast an algorithm in question runs; space efficiency deals approximation whose reliability is not easy to assess. If this algorithm is executed in a machine which
with the extra space the algorithm requires. The space requirement is not of much concern, because is ten times faster than one we have, the running time is also ten times or assuming that C(n) = ½ n(n-
now we have the fast main memory, cache memory etc. so we concentrate more on time efficiency. 1), how much longer will the algorithm run if we doubt its input size? The answer is four times
longer. Indeed, for all but very small values of n,
U

U
1.4.1.1 Measuring an Input’s size C(n) = ½ n(n-1) = ½ n2- ½ n ≈ ½ n2
ST

ST
Almost all algorithms run longer on larger inputs. For example, it takes to sort larger arrays, and therefore,
multiply larger matrices and so on. It is important to investigate an algorithm’s efficiency as a function T(2n) Cop C(2n) ½(2n)2 = 4
of some parameter n indicating the algorithm’s input size. For example, it will be the size of the list
for problems of sorting, searching etc. For the problem of evaluating a polynomial p (x) = an xn+ ------ Cop
+ a0 of degree n, it will be the polynomial’s degree or the number of its coefficients, which is larger T(n) ≈ C(n) ≈ ½(2n)2
by one than its degree.
Here Cop is unknown, but still we got the result, the value is cancelled out in the ratio. Also, ½ the
The size also be influenced by the operations of the algorithm. For e.g., in a spell-check
multiplicative constant is also cancelled out. Therefore, the efficiency analysis framework ignores
algorithm, it examines individual characters of its input, then we measure the size by the number of
multiplicative constants and concentrates on the counts’ order of growth to within a constant multiple
characters or words.
for large size inputs.
Note: measuring size of inputs for algorithms involving properties of numbers. For such
algorithms, computer scientists prefer measuring size by the number b of bits in the n’s binary
representation.
b= log2n+1.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

1.4.1.3 Orders of Growth 1.4.1.4 Worst-case, Best-case and Average-case efficiencies:

This is mainly considered for large input size. On small inputs if there is difference in running The running time not only depends on the input size but also on the specifics of a particular
time it cannot be treated as efficient one. input. Consider the example, sequential search. It’s a straightforward algorithm that searches for a
Values of several functions important for analysis of algorithms: given item (search key K) in a list of n elements by checking successive elements of the list until
either a match with the search key is found or the list is exhausted.
log2
n n n n log2n n2 n3 2n n! The psuedocode is as follows.
10 10
1
10 3.3 3.3 x 101 2
103 103 3.6 x 106 Algorithm sequential search {A [0. . n-1] , k }
10 10 10 1.3 x 9.3 x // Searches for a given value in a given array by Sequential search
2 2
6.6 x 102 4
106 1030 10157 // Input: An array A[0..n-1] and a search key K

P
6.6
// Output: Returns the index of the first element of A that matches K or -1 if there is
10 10 10
3 3 // no match
1.0 x 104 6
109

AP

AP
10
i← o
10 10 10 101
4 4 while i< n and A [ i ] ≠ K do i←
13 1.3 x 105 8 2
i+1
10 10 101 101
5 5
17 1.7 x 106 0 5
if i< n return i else
10 10 101 101 return –1
R

R
6 6
20 2.0 x 107 2 8

Clearly, the running time of this algorithm can be quite different for the same list size n. In the
CO

CO
The function growing slowly is the logarithmic function, logarithmic basic-operation count to worst case, when there are no matching elements or the first matching element happens to be the last
run practically instantaneously on inputs of all realistic sizes. Although specific values of such a count one on the list, the algorithm makes the largest number of key comparisons among all possible inputs
depend, of course, in the logarithm’s base, the formula of size n; Cworst (n) = n.
logan = logab x logbn The worst-case efficiency of an algorithm is its efficiency for the worst-case input of size n,
Makes it possible to switch from one base to another, leaving the count logarithmic but with a new which is an input of size n for which the algorithm runs the longest among all possible inputs of that
U

U
multiplicative constant. size. The way to determine is, to analyze the algorithm to see what kind of inputs yield the largest
value of the basic operation’s count c(n) among all possible inputs of size n and then compute this
On the other end, the exponential function 2n and the factorial function n! grow so fast even worst-case value Cworst (n).
ST

ST
for small values of n. These two functions are required to as exponential-growth functions.
“Algorithms that require an exponential number of operations are practical for solving only problems The best- case efficiency of an algorithm is its efficiency for the best-case input of size n,
of very small sizes.”
which is an input of size n for which the algorithm runs the fastest among all inputs of that size. First,
determine the kind of inputs for which the count C(n) will be the smallest among all possible inputs of
Another way to appreciate the qualitative difference among the orders of growth of the size n. Then ascertain the value of C(n) on the most convenient inputs. For e.g., for the searching with
functions is to consider how they react to, say, a twofold increase in the value of their argument n. The
input size n, if the first element equals to a search key, Cbest(n) = 1.
function log2n increases in value by just 1 (since log22n = log22 + log2n = 1 + log2n); the linear
function increases twofold; the nlogn increases slightly more than two fold; the quadratic n 2 as
Neither the best-case nor the worst-case gives the necessary information about an algorithm’s
fourfold ( since (2n)2 = 4n 2) and the cubic function n3 as eight fold (since (2n)3 = 8n3); the value of behaviour on a typical or random input. This is the information that the average-case efficiency seeks
2n is squared (since 22n = (2n)2 and n! increases much more than that. to provide. To analyze the algorithm’s average-case efficiency, we must make some assumptions
about possible inputs of size n.
Let us consider again sequential search. The standard assumptions are that:

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

1. the probability of a successful search is equal to p (0 ≤ p ≤ 1), and, Informal Introduction:

2. the probability of the first match occurring in the ith position is same for every i.
O [g(n)] is the set of all functions with a smaller or same order of growth as g(n) Eg: n O (n2),
th
Accordingly, the probability of the first match occurring in the i position of the list is p/n for every i, 100n+5 O(n2), 1/2n(n-1) O(n2).
and the no of comparisons is i for a successful search. In case of unsuccessful search, the number of The first two are linear and have a smaller order of growth than g(n)=n2, while the last one is
comparisons is n with probability of such a search being (1-p). Therefore, quadratic and hence has the same order of growth as n2. on the other hand,
n3 (n2), 0.00001 n3 O(n2), n4+n+1 O(n 2 ).The function n3 and 0.00001 n3 are both cubic and
have a higher order of growth than n2 , and so has the fourth-degree polynomial n4 +n+1
Cavg(n) = [1 . p/n + 2 . p/n + ………. i . p/n + ……….. n . p/n] + n.(1-p) The second-notation, Ω [g(n)] stands for the set of all functions with a larger or same order of
= p/n [ 1+2+…….i+……..+n] + n.(1-p) growth as g(n). for eg, n3 Ω(n2), 1/2n(n-1) Ω(n2), 100n+5 Ω(n2)
= p/n . [n(n+1)]/2 + n.(1-p) [sum of 1st n natural number formula] Finally, θ [g(n)] is the set of all functions that have the same order of growth as g(n).
E.g, an2+bn+c with a>0 is in θ(n2)
= [p(n+1)]/2 + n.(1-p)

P
 O-notation:
This general formula yields the answers. For e.g, if p=1 (ie., successful), the average number

AP

AP
of key comparisons made by sequential search is (n+1)/2; ie, the algorithm will inspect, on an average, Definition: A function t(n) is said to be in 0[g(n)]. Denoted t(n) 0[g(n)], if t(n) is bounded above by
about half of the list’s elements. If p=0 (ie., unsuccessful), the average number of key comparisons some constant multiple of g(n) for all large n ie.., there exist some positive constant c and some non
negative integer no such that t(n) ≤ cg(n) for all n≥no.
will be ‘n’ because the algorithm will inspect all n elements on all such inputs. Eg. 100n+5 0 (n2)
The average-case is better than the worst-case, and it is not the average of both best and worst-cases. Proof: 100n+ 5 ≤ 100n+n (for all n ≥ 5) = 101n ≤ 101 n2
Another type of efficiency is called amortized efficiency. It applies not to a single run of an Thus, as values of the constants c and n0 required by the definition, we con take 101 and 5
algorithm but rather to a sequence of operations performed on the same data structure. In some respectively.
R

R
situations a single operation can be expensive, but the total time for an entire sequence of such n The definition says that the c and n0 can be any value. For eg we can also take. C = 105, and n0 = 1.
operations is always better than the worst-case efficiency of that single operation multiplied by n. It is i.e., 100n+ 5 ≤ 100n + 5n (for all n ≥ 1) = 105n
CO

CO
considered in algorithms for finding unions of disjoint sets.
Recaps of Analysis framework:  Ω-Notation:
1. Both time and space efficiencies are measured as functions of the algorithm’s i/p size. Definition: A fn t(n) is said to be in Ω[g(n)], denoted t(n) Ω[g(n)], if t(n) is bounded below by
2. Time efficiency is measured by counting the number of times the algorithm’s basic operation is some positive constant multiple of g(n) for all large n, ie., there exist some positive constant c and
some non negative integer n0 s.t.
executed. Space efficiency is measured by counting the number of extra memory units consumed
t(n) ≥ cg(n) for all n ≥ n0.
U

U
by the algorithm.
For example: n3 Ω(n2), Proof is n3 ≥ n2 for all n ≥ n0. i.e., we can select c=1 and n0=0.
3. The efficiencies of some algorithms may differ significantly for input of the same size. For such
algorithms, we need to distinguish between the worst-case, average-case and best-case
ST

ST
efficiencies.  θ - Notation:
4. The framework’s primary interest lies in the order of growth of the algorithm’s running tine as its
input size goes to infinity. Definition: A function t(n) is said to be in θ [g(n)], denoted t(n) θ (g(n)), if t(n) is bounded both
above and below by some positive constant multiples of g(n) for all large n, ie., if there exist some
positive constant c1 and c2 and some nonnegative integer n0 such that c2g(n) ≤ t(n) ≤ c1g(n) for all n
1.5 ASYMPTOTIC NOTATIONS AND ITS PROPERTIES ≥ n0.
Example: Let us prove that ½ n(n-1) θ( n2 ) .First, we prove the right inequality (the upper bound)
1. n(n-1) = ½ n2 – ½ n ≤ ½ n2 for all n ≥ n0.
The efficiency analysis framework concentrates on the order of growth of an algorithm’s basic Second, we prove the left inequality (the lower bound)
operation count as the principal indicator of the algorithm’s efficiency. To compare and rank such 2. n(n-1) = ½ n2 – ½ n ≥ ½ n2 – ½ n½ n for all n ≥ 2 = ¼
n2. Hence, we can select c2= ¼, c2= ½ and n0 = 2
orders of growth, we use three notations; 0 (big oh), Ω (big omega) and θ (big theta). First, we see the
informal definitions, in which t(n) and g(n) can be any non negative functions defined on the set of
natural numbers. t(n) is the running time of the basic operation, c(n) and g(n) is some function to
compare the count with.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Useful property involving these Notations: Using Limits for Comparing Orders of Growth:

The property is used in analyzing algorithms that consists of two consecutively executed parts: The convenient method for doing the comparison is based on computing the limit of the
ratio of two functions in question. Three principal cases may arise:
THEOREMIf t1(n) Є O(g1(n)) and t2(n) Є O(g2(n)) then t1 (n) + t2(n) Є O(max{g1(n),
t(n) 0 implies that t(n) has a smaller order of growth than g(n) lim= c implies that t(n) has the same
g2(n)}). order of growth as g(n) n ->∞ g(n) ∞ implies that t (n) has a larger order of growth than g(n).

PROOF (As we shall see, the proof will extend to orders of growth the following simple Note that the first two cases mean that t(n) Є O(g(n)), the last two mean that t(n) Є Ω(g(n)),
fact about four arbitrary real numbers a1 , b1 , a2, and b2: if a1 < b1 and a2 < b2 then a1 + a2 < 2 and the second case means that t(n) Є θ(g(n)).
max{ b1, b2}.) Since t1(n) Є O(g1(n)) , there exist some constant c and some nonnegative integer n EXAMPLE 1 Compare orders of growth of ½ n(n - 1) and n2. (This is one of the
1 such that examples we did above to illustrate the definitions.)

P
t1(n) < c1g1 (n) for all n > n1
lim ½ n(n-1) = ½ lim n2 – n = ½ lim (1- 1/n ) = ½
since t2(n) Є O(g2(n)),

AP

AP
n ->∞ n2 n ->∞ n2 n ->∞
t2(n) < c2g2(n) for all n > n2.
Since the limit is equal to a positive constant, the functions have the same order of growth
Let us denote c3 = maxfc1, c2} and consider n > max{ n1 , n2} so that we can or, symbolically, ½ n(n - 1) Є θ (n2)
use both inequalities. Adding the two inequalities above yields the following:
t1(n) + t2(n) < c1g1 (n) + c2g2(n) Basic Efficiency Classes:
R

R
< c3g1(n) + c3g2(n) = c3 [g1(n) + g2(n)] Even though the efficiency analysis framework puts together all the functions whose orders
of growth differ by a constant multiple, there are still infinitely many such classes. (For example,
< c32max{g1 (n),g2(n)}.
CO

CO
the exponential functions an have different orders of growth for different values of base a.)
Therefore, it may come as a surprise that the time efficiencies of a large number of algorithms fall
Hence, t1 (n) + t2(n) Є O(max {g1(n) , g2(n)}), with the constants c and n0 required by the O
into only a few classes. These classes are listed in Table in increasing order of their orders of
definition being 2c3 = 2 max{c1, c2} and max{n1, n2}, respectively.This implies that the algorithm's growth, along with their names and a few comments.
overall efficiency will be determined by the part with a larger order of growth, i.e., its least efficient
You could raise a concern that classifying algorithms according to their asymptotic
part:
U

U
efficiency classes has little practical value because the values of multiplicative constants are
usually left unspecified. This leaves open a possibility of an algorithm in a worse efficiency class
t1(n) Є O(g1(n)) running faster than an algorithm in a better efficiency class for inputs of realistic sizes. For
t2(n) Є O(g2(n)) then t1 (n) + t2(n) Є O(max{g1(n), g2(n)}).
ST

ST
example, if the running time of one algorithm is n3 while the running time of the other is 106n2,
the cubic algorithm will outperform the quadratic algorithm unless n exceeds 106. A few such
For example, we can check whether an array has identical elements by means of the
following two-part algorithm: first, sort the array by applying some known sorting algorithm; anomalies are indeed known. For example, there exist algorithms for matrix multiplication with a
second, scan the sorted array to check its consecutive elements for equality. If, for example, a better asymptotic efficiency than the cubic efficiency of the definition-based algorithm (see
Section 4.5). Because of their much larger multiplicative constants, however, the value of these
sorting algorithm used in the first part makes no more than 1/2n(n — 1) comparisons (and hence is
more sophisticated algorithms is mostly theoretical.
in O(n2)) while the second part makes no more than n — 1 comparisons (and hence is in O(n}), the
Fortunately, multiplicative constants usually do not differ that drastically. As a rule, you
efficiency of the entire algorithm will be in
should expect an algorithm from a better asymptotic efficiency class to outperform an algorithm
from a worse class even for moderately sized inputs. This observation is especially true for an
(O(max{n2, n}) = O(n2). algorithm with a better than exponential running time versus an exponential (or worse) algorithm.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

comparisons will be the same for all arrays of size n; therefore, in terms of this metric, there is no
Class Name Comments need to distinguish among the worst, average, and best cases here.)
1 Constant Short of best case efficiency when its input grows
the time also grows to infinity. Let us denote C(n) the number of times this comparison is executed and try to find a formula
expressing it as a function of size n. The algorithm makes one comparison on each execution of the
logn Logarithmic It cannot take into account all its input, any algorithm
loop, which is repeated for each value of the loop's variable i within the bounds between 1 and n —
that does so will have atleast linear running time.
1 (inclusively). Therefore, we get the following sum for C(n):
n Linear Algorithms that scan a list of size n, eg., sequential
search
n-1
nlogn nlogn Many divide & conquer algorithms including
C(n) = ∑ 1
mergersort quicksort fall into this class
i=1
n2 Quadratic Characterizes with two embedded loops, mostly

P
sorting and matrix operations. This is an easy sum to compute because it is nothing else but 1 repeated n — 1 times.
n3 Cubic Efficiency of algorithms with three embedded loops, Thus,

AP

AP
2n Exponential Algorithms that generate all subsets of an n-element n-1
set
C(n) = ∑ 1 = n-1 Є θ(n)
n! factorial Algorithms that generate all permutations of an n-
element set i=1
Here is a general plan to follow in analyzing nonrecursive algorithms.
R

R
1.6 MATHEMATICAL ANALYSIS OF NON RECURSIVE ALGORITHMS General Plan for Analyzing Efficiency of Nonrecursive Algorithms

1. Decide on a parameter (or parameters) indicating an input's size.

CO

CO
The general framework outlined is applied to analyze the efficiency of nonrecursive
algorithms. Let us start with a very simple example that demonstrates the entire principal steps 2. Identify the algorithm's basic operation. (As a rule, it is located in its innermost loop.)
typically taken in analyzing such algorithms.
3. Check whether the number of times the basic operation is executed depends only on the size
EXAMPLE 1 Consider the problem of finding the value of the largest element in a list of n of an input. If it also depends on some additional property, the worst-case, average-case, and, if
U

U
numbers. For simplicity, we assume that the list is implemented as an array. The following is a necessary, best-case efficiencies have to be investigated separately.
pseudocode of a standard algorithm for solving the problem. 4. Set up a sum expressing the number of times the algorithm's basic operation is executed.
ALGORITHM MaxElement(A[0,..n - 1]) 5. Using standard formulas and rules of sum manipulation either find a closed-form formula
ST

ST
//Determines the value of the largest element in a given array //Input: for the count or, at the very least, establish its order of growth.
An array A[0..n - 1] of real numbers
//Output: The value of the largest element in A
In particular, we use especially frequently two basic rules of sum manipulation
maxval <- A[0]
u u
for i <- 1 to n - 1 do
if A[i] > maxval ∑ c ai =c∑ ai ----(R1)
maxval <— A[i] i=1 i=1
return maxval
The obvious measure of an input's size here is the number of elements in the array, i.e., n. u u u
The operations that are going to be executed most often are in the algorithm's for loop. There are ∑ (ai ± bi) = ∑ ai ± ∑ bi ----(R2)
two operations in the loop's body: the comparison A[i] > maxval and the assignment maxval <- i=1 i=1 i=1
A[i]. Since the comparison is executed on each repetition of the loop and the assignment is not, we and two summation formulas
should consider the comparison to be the algorithm's basic operation. (Note that the number of

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

u n-2 n-2 n-2

∑ 1 = u- l + 1 where l ≤ u are some lower and upper integer limits --- (S1 )
= ∑ (n-1) - ∑ i = (n-1) ∑ 1 – [(n-2)(n-1)]/2
i= l
i=0 i=0 i=0
n
∑ i = 1+2+….+n = [n(n+1)]/2 ≈ ½ n2 Є θ(n2) ----(S2) = (n-1) 2 - [(n-2)(n-1)]/2 = [(n-1)n]/2 ≈ ½ n2 Є θ(n2)
Also it can be solved as (by using S2)
i=0 n-2
∑ (n-1-i) = (n-1) + (n-2) + ……+ 1 = [(n-1)n]/2 ≈ ½ n2 Є θ(n2)
(Note that the formula which we used in Example 1, is a special case of formula (S1) for l= i=0
= 0 and n = n - 1)
Note that this result was perfectly predictable: in the worst case, the algorithm needs to

P
EXAMPLE 2 Consider the element uniqueness problem: check whether all the elements in a given compare all n(n - 1)/2 distinct pairs of its n elements.
array are distinct. This problem can be solved by the following straightforward algorithm.

AP

AP
1.7 MATHEMATICAL ANALYSIS OF RECURSIVE ALGORITHMS
ALGORITHM UniqueElements (A [0..n - 1])
//Checks whether all the elements in a given array are distinct //Input: In this section, we systematically apply the general framework to analyze the efficiency of
An array A[0..n - 1] recursive algorithms. Let us start with a very simple example that demonstrates all the principal
//Output: Returns "true" if all the elements in A are distinct // and steps typically taken in analyzing recursive algorithms.
"false" otherwise.
R

R
for i «— 0 to n — 2 do for Example 1: Compute the factorial function F(n) = n! for an arbitrary non negative integer n. Since,
j' < - i: + 1 to n - 1 do if
CO

CO
A[i] = A[j] n! = 1 * 2 * ……. * (n-1) *n = n(n-1)! For n ≥ 1
return false and 0! = 1 by definition, we can compute F(n) = F(n-1).n with the following recursive algorithm.
return true
ALGORITHM F(n)
The natural input's size measure here is again the number of elements in the array, i.e., n.
// Computes n! recursively
U

U
Since the innermost loop contains a single operation (the comparison of two elements), we should
// Input: A nonnegative integer n
consider it as the algorithm's basic operation. Note, however, that the number of element
// Output: The value of n!
comparisons will depend not only on n but also on whether there are equal elements in the array
ifn =0 return 1
ST

ST
and, if there are, which array positions they occupy. We will limit our investigation to the worst
else return F(n — 1) * n
case only.
For simplicity, we consider n itself as an indicator of this algorithm's input size (rather than
By definition, the worst case input is an array for which the number of element comparisons the number of bits in its binary expansion). The basic operation of the algorithm is multiplication,
Cworst(n) is the largest among all arrays of size n. An inspection of the innermost loop reveals that
whose number of executions we denote M(n). Since the function F(n) is computed according to the
there ate two kinds of worst-case inputs (inputs for which the algorithmdoes not exit the loop
prematurely): arrays with no equal elements and arrays in which the last two elements are the only formula
pair of equal elements. For such inputs, one comparison is made for each repetition of the innermost F(n) = F ( n - 1 ) - n for n > 0,
loop, i.e., for each value of the loop's variable j between its limits i + 1 and n - 1; and this is repeated the number of multiplications M(n) needed to compute it must satisfy the equality
for each value of the outer loop, i.e., for each value of the loop's variable i between its limits 0 and n
- 2. Accordingly, we get M(n) = M(n - 1) +1 for n > 0.
n-2 n-1 n-2 n-2
to compute to multiply
C worst (n) = ∑ ∑ 1 = ∑ [(n-1) – (i+1) + 1] = ∑ (n-1-i)
F(n-1) F(n-1) by n
i=0 j=i+1 i=0 i=0

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

applies to solving our particular recurrence:

Indeed, M(n - 1) multiplications are spent to compute F(n - 1), and one more multiplication is M(n) = M(n - 1) + 1 substitute M(n - 1) = M(n - 2) + 1
needed to multiply the result by n. +
= [M(n - 2) + 1] 1 = M(n - 2) + 2 substitute M(n - 2) = M(n - 3) + 1
The last equation defines the sequence M(n) that we need to find. Note that the equation +
defines M(n) not explicitly, i.e., as a function of n, but implicitly as a function of its value at another = [M(n - 3) + 1] 2 = M (n - 3) + 3.
point, namely n — 1. Such equations are called recurrence relations or, for brevity, recurrences. After inspecting the first three lines, we see an emerging pattern, which makes it possible to
predict not only the next line (what would it be?) but also a general formula for the pattern: M(n) = M(n
Recurrence relations play an important role not only in analysis of algorithms but also in some areas
— i) + i. Strictly speaking, the correctness of this formula should be proved by mathematical induction,
of applied mathematics. Our goal now is to solve the recurrence relation M(n) = M(n — 1) + 1, i.e., but it is easier to get the solution as follows and then verify its correctness.
to find an explicit formula for the sequence M(n) in terms of n only.
What remains to be done is to take advantage of the initial condition given. Since it is
Note, however, that there is not one but infinitely many sequences that satisfy this
specified for n = 0, we have to substitute i = n in the pattern's formula to get the ultimate result of
recurrence. To determine a solution uniquely, we need an initial condition that tells us the value
our backward substitutions:
with which the sequence starts. We can obtain this value by inspecting the condition that makes the

P
algorithm stop its recursive calls:
M(n) = M(n - 1) + 1 = • • • = M(n - i) + i = - ---- = M(n -n) + n = n.
if n = 0 return 1.

AP

AP
The benefits of the method illustrated in this simple example will become clear very soon,
This tells us two things. First, since the calls stop when n = 0, the smallest value of n for
when we have to solve more difficult recurrences. Also note that the simple iterative algorithm that
which this algorithm is executed and hence M(n) defined is 0. Second,by inspecting the code's
accumulates the product of n consecutive integers requires the same number of multiplications, and
exiting line, we can see that when n = 0, the algorithm performs no multiplications. Thus, the initial
it does so without the overhead of time and space used for maintaining the recursion's stack.
condition we are after is
R

R
M (0) = 0.
The issue of time efficiency is actually not that important for the problem of computing n!,
the calls stop when n = 0, no multiplications when n = 0 Thus, we succeed in setting up the
however. The function's values get so large so fast that we can realistically compute its values only
recurrence relation and initial condition for the algorithm's number of multiplications M(n):
CO

CO
for very small n's. Again, we use this example just as a simple and convenient vehicle to introduce
M(n) = M(n - 1) + 1 for n > 0, (2.1) the standard approach to analyzing recursive algorithms.
M (0) = 0.
Before we embark on a discussion of how to solve this recurrence, let us pause to reiterate Generalizing our experience with investigating the recursive algorithm for computing n!, we
an important point. We are dealing here with two recursively defined functions. The first is the can now outline a general plan for investigating recursive algorithms.
factorial function F(n) itself; it is defined by the recurrence
U

U
A General Plan for Analyzing Efficiency of Recursive Algorithms
F(n) = F(n - 1) • n for every n > 0, F(0) = l. 1. Decide on a parameter (or parameters) indicating an input's size.
ST

ST
2. Identify the algorithm's basic operation.
The second is the number of multiplications M(n) needed to compute F(n) by the recursive 3. Check whether the number of times the basic operation is executed can vary on
algorithm whose pseudocode was given at the beginning of the section. As we just showed, M(n) is different inputs of the same size; if it can, the worst-case, average-case, and best-case
defined by recurrence (2.1). And it is recurrence (2.1) that we need to solve now. efficiencies must be investigated separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number of times
Though it is not difficult to "guess" the solution, it will be more useful to arrive at it in a the basic operation is executed.
systematic fashion. Among several techniques available for solving recurrence relations, we use 5. Solve the recurrence or at least ascertain the order of growth of its solution.
what can be called the method of backward substitutions. The method's idea (and the reason for the
name) is immediately clear from the way it Example 2: the algorithm to find the number of binary digits in the binary representation of a
positive decimal integer.

ALGORITHM BinRec(n) : - //Input: A

positive decimal integer n

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

//Output: The number of binary digits in n's binary representation if n F(n) = F(n -1) + F(n-2) for n > 1 ----(2.3)
= 1 return 1 and two initial conditions
else return BinRec(n/2) + 1
F(0) = 0, F(1) = 1 ----(2.4)
Let us set up a recurrence and an initial condition for the number of additions A(n) made by the The Fibonacci numbers were introduced by Leonardo Fibonacci in 1202 as a solution to a
algorithm. The number of additions made in computing BinRec(n/2) is A(n/2), plus one more problem about the size of a rabbit population. Many more examples of Fibonacci-like numbers have
addition is made by the algorithm to increase the returned value by 1. This leads to the recurrence since been discovered in the natural world, and they have even been used in predicting prices of
A(n) = A(n/2) + 1 for n > 1. (2.2) stocks and commodities. There are some interesting applications of the Fibonacci numbers in
Since the recursive calls end when n is equal to 1 and there are no additions made then, the computer science as well. For example, worst-case inputs for Euclid's algorithm happen to be
initial condition is consecutive elements of the Fibonacci sequence. Our discussion goals are quite limited here,
A(1) = 0 however. First, we find an explicit formula for the nth Fibonacci number F(n), and then we briefly
discuss algorithms for computing it.

P
The presence of [n/2] in the function's argument makes the method of backward
substitutions stumble on values of n that are not powers of 2. Therefore, the standard approach to
Explicit Formula for the nth Fibonacci Number
solving such a recurrence is to solve it only for n — 2k and then take advantage of the theorem

AP

AP
If we try to apply the method of backward substitutions to solve recurrence (2.6), we will
called the smoothness rule which claims that under very broad assumptions the order of growth
fail to get an easily discernible pattern. Instead, let us take advantage of a theorem that describes
observed for n = 2k gives a correct answer about the order of growth for all values of n. solutions to a homogeneous second-order linear recurrence with constant coefficients
(Alternatively, after getting a solution for powers of 2, we can sometimes finetune this solution to ax(n) + bx(n - 1) + cx(n - 2) = 0, -----(2.5)
get a formula valid for an arbitrary n.) So let us apply this recipe to our recurrence, which for n = 2k where a, b, and c are some fixed real numbers (a ≠ 0) called the coefficients of the recurrence
takes the form
R

R
and x(n) is an unknown sequence to be found. According to this theorem—see Theorem 1 in
A(2 k) = A(2 k -1 ) + 1 for k > 0, A(2 0 ) = 0 Appendix B—recurrence (2.5) has an infinite number of solutions that can be obtained by one of the
Now backward substitutions encounter no problems: three formulas. Which of the three formulas applies for a particular case depends on the number of
CO

CO
real roots of the quadratic equation with the same
A(2 k) = A(2 k -1 ) + 1 substitute A(2k-1) = A(2k -2) + 1 coefficients as recurrence (2.5):
k -2
= [A(2 ) + 1] + 1 = A(2 ) + 2 substitute A(2k -2)
k -2
ar 2 + br + c = 0. ------ (2.6)
= A(2 ) + 1 = [A(2 k -3) + 1] + 2 = A(2 k -3) + 3
k-3 Quite logically, equation (2.6) is called the characteristic equation for recurrence (2.5). Let us
…………… apply this theorem to the case of the Fibonacci numbers.
U

U
F(n) - F(n - 1) - F(n - 2) = 0. ---------(2.7)
= A(2 k -i) + i
Its characteristic equation is r 2 -
ST

ST
……………
r - 1 = 0,
= A(2 k -k) + k with the roots
r 1,2 = (1 ± √1-4(-1)) /2 = (1 ± √5)/2
Thus, we end up with
Algorithms for Computing Fibonacci Numbers
A(2k) = A ( 1 ) + k = k
or, after returning to the original variable n = 2k and, hence, k = log2 n, Though the Fibonacci numbers have many fascinating properties, we limit our discussion to a
A (n ) = log2 n Є θ (log n). few remarks about algorithms for computing them. Actually, the sequence grows so fast that it is the
size of the numbers rather than a time-efficient method for computing them that should be of
Example: Fibonacci numbers primary concern here. Also, for the sake of simplicity, we consider such operations as additions and
multiplications at unit cost in the algorithms that follow. Since the Fibonacci numbers grow
In this section, we consider the Fibonacci numbers, a sequence of numbers as 0, 1, 1, 2, infinitely large (and grow rapidly), a more detailed analysis than the one offered here is warranted.
3, 5, 8, …. That can be defined by the simple recurrence These caveats notwithstanding, the algorithms we outline and their analysis are useful examples for
a student of design and analysis of algorithms.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

To begin with, we can use recurrence (2.3) and initial condition (2.4) for the obvious extra array for storing all the preceding elements of the Fibonacci sequence can be avoided: storing
recursive algorithm for computing F(n). just two values is necessary to accomplish the task.
ALGORITHM F(n)
//Computes the nth Fibonacci number recursively by using its definition //Input: The third alternative for computing the nth Fibonacci number lies in using a formula. The
A nonnegative integer n efficiency of the algorithm will obviously be determined by the efficiency of an exponentiation
//Output: The nth Fibonacci number algorithm used for computing ø n. If it is done by simply multiplying ø by itself n - 1 times, the
if n < 1 return n
algorithm will be in θ (n) = θ (2b) . There are faster algorithms for the exponentiation problem. Note
else return F(n - 1) + F(n - 2)
also that special care should be exercised in implementing this approach to computing the nth
Fibonacci number. Since all its intermediate results are irrational numbers, we would have to make
Analysis: sure that their approximations in the computer are accurate enough so that the final round-off yields
The algorithm's basic operation is clearly addition, so let A(n) be the number of additions a correct result.
performed by the algorithm in computing F(n). Then the numbers of additions needed for

P
computing F(n — 1) and F(n — 2) are A(n — 1) and A(n — 2), respectively, and the algorithm Finally, there exists a θ (logn) algorithm for computing the nth Fibonacci number that
needs one more addition to compute their sum. Thus, manipulates only integers. It is based on the equality

AP

AP
we get the following recurrence for A(n):
A(n) = A(n - 1) + A(n - 2) + 1 for n > 1, (2.8) F(n-1) F(n) 0 1n
A(0)=0, A(1) = 0. F(n) F(n+1) 1 1 for n ≥ 1
The recurrence A(n) — A(n — 1) — A(n — 2) = 1 is quite similar to recurrence (2.7) but its
right-hand side is not equal to zero. Such recurrences are called inhomo-geneous recurrences. and an efficient way of computing matrix powers.
R

R
There are general techniques for solving inhomogeneous recurrences (see Appendix B or any
textbook on discrete mathematics), but for this particular recurrence, a special trick leads to a faster
solution. We can reduce our inhomogeneous recurrence to a homogeneous one by rewriting it as
CO

CO
[A(n) + 1] - [A(n -1) + 1]- [A(n - 2) + 1] = 0 and substituting B(n) = A(n) + 1:
B(n) - B(n - 1) - B(n - 2) = 0
B(0) = 1, B(1) = 1.
U

U
This homogeneous recurrence can be solved exactly in the same manner as recurrence
(2.7) was solved to find an explicit formula for F(n).
We can obtain a much faster algorithm by simply computing the successive elements of
ST

ST
the Fibonacci sequence iteratively, as is done in the following algorithm.

ALGORITHM Fib(n)
//Computes the nth Fibonacci number iteratively by using its definition
//Input: A nonnegative integer n
//Output: The nth Fibonacci number
F[0]<-0; F[1]<-1
for i <- 2 to n do
F[i]«-F[i-1]+F[i-2]
return F[n]

This algorithm clearly makes n - 1 additions. Hence, it is linear as a function of n and "only"
exponential as a function of the number of bits b in n's binary representation. Note that using an

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

UNIT II text and other nonnumerical data, metrics such as the Hamming distance are used. A bottom-up
algorithm begins with each element as a separate cluster and merges them into successively larger
BRUTE FORCE AND DIVIDE-AND-CONQUER clusters by combining the closest pair of clusters.
Brute Force – Computing an – String Matching - Closest-Pair and Convex-Hull Problems - Exhaustive For simplicity, we consider the two-dimensional case of the closest-pair problem. We assume that the
Search - Travelling Salesman Problem - Knapsack Problem - Assignment problem. Divide and points in question are specified in a standard fashion by their (x, y) Cartesian coordinates and that the
Conquer Methodology – Binary Search – Merge sort – Quick sort – Heap Sort - Multiplication of distance between two points pi(xi,yi) and pj(xj, yj ) is the standard Euclidean distance
Large Integers – Closest-Pair and Convex - Hull Problems. d(pi, pj ) = (xi− xj )2 + (yi− yj )2.
The brute-force approach to solving this problem leads to the following obvious algorithm: compute
2.1 BRUTE FORCE the distance between each pair of distinct points and find a pair with the smallest distance. Of course,
we do not want to compute the distance between the same pair of points twice. To avoid doing so, we
Brute force is a straightforward approach to solving a problem, usually directly based on the consider only the pairs of points (pi, pj ) for which i < j.
problem’s statement and definitions of the concepts involved. For e.g. the algorithm to find the gcd of Pseudocode below computes the distance between the two closest points; getting the closest points

P
two numbers. themselves requires just a trivial modification.
Brute force approach is not an important algorithm design strategy for the following reasons:

AP

AP
ALGORITHM BruteForceClosestPair(P )
• First, unlike some of the other strategies, brute force is applicable to a very wide variety //Finds distance between two closest points in the plane by brute force
of problems. Its used for many elementary but algorithmic tasks such as computing the //Input: A list P of n (n ≥ 2) points p1(x1, y1), . . . , pn(xn, yn)
sum of n numbers, finding the largest element in a list and so on. //Output: The distance between the closest pair of
• Second, for some problem it yields reasonable algorithms of at least some practical value points d←∞
with no limitation on instance size. for i ←1 to n − 1 do
R

R
• Third, the expense of designing a more efficient algorithm if few instances to be solved for j ←i + 1 to n do
and with acceptable speed for solving it. d ←min(d, sqrt((xi
CO

CO
• Fourth, even though it is inefficient, it can be used to solve small-instances of a problem. − xj )2 + (yi
• Last, it can serve as an important theoretical or educational propose. − yj )2)) //sqrt is square
root return d
2.1.1 CLOSEST PAIR AND CONVEX HULL PROBLEMS
The basic operation of the algorithm is computing the square root. In the age of electronic
U

U
2.1.1.1 CLOSEST-PAIR PROBLEM calculators with a square-root button, one might be led to believe that computing the square root is as
simple an operation as, say, addition or multiplication. Of course, it is not. For starters, even for most
The closest-pair problem calls for finding the two closest points in a set of n points. It is the integers, square roots are irrational numbers that therefore can be found only approximately. Moreover,
ST

ST
simplest of a variety of problems in computational geometry that deals with proximity of points in the computing such approximations is not a trivial matter. But, in fact, computing square roots in the loop
plane or higher-dimensional spaces. Points in question can represent such physical objects as airplanes can be avoided! (Can you think how?) The trick is to realize that we can simply ignore the square-root
or post offices as well as database records, statistical samples, DNA sequences, and so on. An air- function and compare the values (xi− xj )2 + (yi− yj )2 themselves.
traffic controller might be interested in two closest planes as the most probable collision
candidates.Aregional postal service manager might need a solution to the closestpair problem to find We can do this because the smaller a number of which we take the square root, the smaller its
candidate post-office locations to be closed. square root, or, as mathematicians say, the square-root function is strictly increasing. Then the basic
One of the important applications of the closest-pair problem is cluster analysis in statistics. operation of the algorithm will be squaring a number. The number of times it will be executed can be
Based on n data points, hierarchical cluster analysis seeks to organize them in a hierarchy of clusters computed as follows: Of course, speeding up the innermost loop of the algorithm could only decrease
based on some similarity metric. For numerical data, this metric is usually the Euclidean distance; for the algorithm’s running time by a constant but it cannot improve its asymptotic efficiency class.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

2.1.1.2 CONVEX-HULL PROBLEM 2.2.1TRAVELING SALESMAN PROBLEM

On to the other problem—that of computing the convex hull. Finding the convex hull for a
given set of points in the plane or a higher dimensional space is one of the most important—some The problem asks to find the shortest tour through a given set of n cities that visits each city
people believe the most important—problems in computational geometry. This prominence is due to a exactly once before returning to the city where it started. The problem can be stated as the problem of
variety of applications in which this problem needs to be solved, either by itself or as a part of a larger finding the shortest Hamiltonian circuit of the graph-which is a weighted graph, with the graph’s
task. Several such applications are based on the fact that convex hulls provide convenient vertices representing the cities and the edge weights specifying the distance.
approximations of object shapes and data sets given. For example, in computer animation, replacing
objects by their convex hulls speeds up collision detection; the same idea is used in path planning for Hamiltonian circuit is defined as a cycle that passes thru all the vertices of the graph exactly
Mars mission rovers. once.The Hamiltonian circuit can also be defined as a sequence of n+1 adjacent vertices vi0, vi1, ….
Convex hulls are used in computing accessibility maps produced from satellite images by Vin-1, vi0, where the first vertex of the sequence is the same as the last one while all other n-1 vertices
Geographic Information Systems. They are also used for detecting outliers by some statistical are distinct. Obtain the tours by generating all the permutations of n-1 intermediate cities, compute the
techniques. An efficient algorithm for computing a diameter of a set of points, which is the largest tour lengths, and find the shortest among them.

P
distance between two of the points, needs the set’s convex hull to find the largest distance between two
of its extreme points (see below). Finally,convex hulls are important for solving many optimization Consider the condition that the vertex B precedes C then,The total no of permutations will be

AP

AP
problems, because their extreme points provide a limited set of solution candidates. (n-1)! /2, which is impractical except for small values of n. On the other hand, if the starting vertex is
A set of points (finite or infinite) in the plane is called convex if for any two points p and q in not considered for a single vertex, the number of permutations will be even large for n values.
the set, the entire line segment with the endpoints at p and q belongs to the set.All the sets depicted are
convex, and so are a straight line,a triangle, a rectangle, and, more generally, any convex polygon,1 a 2.2.2 KNAPSACK PROBLEM
circle, and the entire plane. On the other hand, the sets, any finite set of two or more distinct points, the
The problem states that: given n items of known weights w1,w 2, … wn and values v1, v2,…, vn and a
R

R
boundary of any convex polygon, and a circumference are examples of sets that are not convex. Now
we are ready for the notion of the convex hull. knapsack of capacity w, find the most valuable subset of the items that fit into the knapsack. Eg
Intuitively, the convex hull of a set of n points in the plane is the smallest convex polygon that consider a transport plane that has to deliver the most valuable set of items to a remote location without
CO

CO
contains all of them either inside or on its boundary. If this formulation does not fire up your exceeding its capacity.
enthusiasm, consider the problem as one of barricading n sleeping tigers by a fence of the shortest Example:
length,it is somewhat lively, however, because the fenceposts have to be erected right at the spots W=10, w1,w2, w3, w4 = { 7,3,4,5 } and v1,v2, v3, v4 = { 42,12,40,25 }
where some of the tigers sleep! There is another, much tamer interpretation of this notion. Imagine that
the points in question are represented by nails driven into a large sheet of plywood representing the Subset Total weight Total value
U

U
plane. Take a rubber band and stretch it to include all the nails, then let it snap into place. The convex
Ø 0 0
hull is the area bounded by the snapped rubber band A formal definition of the convex hull that is
{1} 7 $42
applicable to arbitrary sets, including sets of points that happen to lie on the same line, follows.
ST

ST
The convex hull of a set S of points is the smallest convex set containing S. (The “smallest” {2} 3 $12
requirement means that the convex hull of S must be a subset of any convex set containing S.) If S is {3} 4 $40
convex, its convex hull is obviously S itself. If S is a set of two points, its convex hull is the line {4} 5 $25
segment connecting these points. If S is a set of three points not on the same line, its convex hull is the { 1,2 } 10 $54
triangle with the vertices at the three points given; if the three points do lie on the same line, the { 1,3 } 11 Not feasible
convex hull is the line segment with its endpoints at the two points that are farthest apart. { 1,4 } 12 Not feasible
{ 2,3 } 7 $52
2.2 EXHAUSTIVE SEARCH { 2,4 } 8 $37
{ 3,4 } 9 $65
It is a straightforward method used to solve problems of combinatorial problems. It generates { 1,2,3 } 14 Not feasible
each and every element of the problem’s domain, selecting based on satisfying the problem’s
{ 1,2,4 } 15 Not feasible
constraints and then finding a desired element (eg., maximization or minimization of desired
{ 1,3,4 } 16 Not feasible
characteristics). The 3 important problems are TSP, knapsack problem and assignment problem.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

{ 2,3,4 } 12 Not feasible ith component indicates the column n of the element selected in the i th row. i.e.,The job number
{ 1,2,3,4 } 19 Not feasible assigned to the ith person. For eg <2,3,4,1> indicates a feasible assignment of person 1 to job2, person
2 to job 3, person3 to job 4 and person 4 to job 1. There is a one-to-one correspondence between
feasible assignments and permutations of the first n integers. If requires generating all the permutations
This problem considers all the subsets of the set of n items given, computing the total weight of of integers 1,2,…n, computing the total cost of each assignment by summing up the corresponding
each subset in order to identify feasible subsets(i.e., the one with the total weight not exceeding the elements of the cost matrix, and finally selecting the one with the smallest sum.
knapsack capacity) and finding the largest among the values, which is an optimal solution. The number
Based on number of permutations, the general case for this problem is n!, which is impractical
of subsets of an n-element set is 2n the search leads to a Ω(2n) algorithm, which is not based on the except for small instances. There is an efficient algorithm for this problem called the Hungarian
generation of individual subsets. method. This problem has a exponential problem solving algorithm, which is also an efficient one. The
problem grows exponentially, so there cannot be any polynomial-time algorithm.
Thus, for both TSP and knapsack, exhaustive search leads to algorithms that are inefficient on

P
every input. These two problems are the best known examples of NP- hard problems. No polynomial-
2.3 DIVIDE AND CONQUER
time algorithm is known for any NP-hard problem. The two methods Backtracking and Branch &
bound enable us to solve this problem in less than exponential time.

AP

AP
Divide and Conquer is a best known design technique, it works according to the following plan:

2.2.3 ASSIGNMENT PROBLEM

a) A problem’s instance is divided into several smaller instances of the same problem of equal
size.
The problem is: given n people who need to be assigned to execute n jobs, one person per job. b) The smaller instances are solved recursively.
The cost if the ith person is assigned to the jth job is a known quantity c) The solutions of the smaller instances are combined to get a solution of the original
R

R
c[i,j] for each pair i,j=1,2,….,n. The problem is to find an assignment with the smallest total cost.
problem.

Example:
CO

CO
As an example, let us consider the problem of computing the sum of n numbers a0, a1, … an-1. If n>1,
Job1 Job2 Job3 Job4 we can divide the problem into two instances: to compare the sum of first n/2 numbers and the
Person1 9 2 7 8 remaining n/2 numbers, recursively. Once each subset is obtained add the two values to get the final
Person2 6 4 3 7 solution. If n=1, then return a0 as the solution.
Person3 5 8 1 8 i.e. a0 + a1….. + an-1 = (a0 + …..+ an/2– 1) + (an/2+ …. + a n-1)
U

U
Person4 7 6 9 4
This is not an efficient way, we can use the Brute – force algorithm here. Hence, all the problems are
not solved based on divide – and – conquer. It is best suited for parallel computations, in which each
ST

ST
sub problem can be solved simultaneously by its own processor.
9 2 7 8 < 1,2,3,4 > cost = 9 + 4 + 1 + 4 = 18
6 4 3 7 < 1,2,4,3 > cost = 9 + 4 + 8 + 9 = 30
Analysis:
C= 5 8 1 8 < 1,3,2,4 > cost = 9 + 3 + 8 + 4 = 24
7 6 9 4 < 1,3,4,2 > cost = 9 + 3 + 8 + 6 = 26 etc.
In general, for any problem, an instance of size n can be divided into several instances of size
n/b with a of them needing to be solved. Here, a & b are constants; a≥1 and b > 1. Assuming that size n
is a power of b; to simplify it, the recurrence relation for the running time T(n) is:
From the problem, we can obtain a cost matrix, C. The problem calls for a selection of one
element in each row of the matrix so that all selected elements are in different columns and the total T(n) = a T (n / b) + f (n)
sum of the selected elements is the smallest possible.
Where f (n) is a function, which is the time spent on dividing the problem into smaller ones and on
Describe feasible solutions to the assignment problem as n- tuples <j1 , j2, …., jn> in which the combining their solutions. This is called the general divide-and- conquer recurrence. The order of
growth of its solution T(n) depends on the values of the constants a and b and the order of growth of
the function f (n).

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Theorem: i = 0; j = 0; k = 0 while i
Є θ(nd) where d≥0 in the above recurrence equation, then θ(nd) if a < bd < p and j < q do
T(n) Є θ(ndlogn if a = bd θ(nlogba)
if a > bd if B[i] ≤ C[j]
A[k] = B[i]; i = i+1
For example, the recurrence equation for the number of additions A(n) made by divide-and-conquer else
on inputs of size n=2k is: A[k] = B[j]; j = j+1
A (n) = 2 A (n/2) + 1 K = k+1
if i=p
Thus for eg., a=2, b=2, and d=0 ; hence since a > bd copy C[j..q-1] to A[k..p+q-1]
A (n) Є θ(nlogba) else copy B[i..p-1] to A[k..p+q-1]
= θ(nlog22)

P
= θ(nl) Analysis:

AP

AP
2.3.1 MERGE SORT Assuming for simplicity that n is a power of 2, the recurrence relation for the number of key
comparisons C(n) is
It is a perfect example of divide-and-conquer. It sorts a given array A [0..n-1] by dividing it
into two halves A[0…(n/2-1)] and A[n/2…n-1], sorting each of them recursively and then merging C(n) = 2 C (n/2) + Cmerge (n) for n>1, c(1) = 0
the two smaller sorted arrays into a single sorted one.
R

R
At each step, exactly one comparison is made, after which the total number of elements in the two
Algorithm Merge Sort (A[0..n-1]) //Sorts array arrays still needed to be processed is reduced by one element. In the worst case, neither of the two
A by recursive merge sort arrays becomes empty before the other one contains just one element. Therefore, for the worst case,
CO

CO
Cmerge (n) = n-1 and the recurrence is:
//Input: An array A [0…n-1] of orderable elements //Output:
Array A [0..n-1] sorted in increasing order Cworst (n) = 2Cworst (n/2) + n-1 for n>1, Cworst (1) = 0 When n is a power of 2, n=
if n>1 2k, by successive substitution, we get,
Copy A[0…(n/2-1)] to B[0…(n/2-1)]
U

U
Copy A[n/2 …n-1] to C[0…(n/2-1)] C(n) = 2 C (n/2) + Cn
Merge sort (B[0..(n/2-1)] = 2 (2 C (n/4) + C n/2 ) + Cn
ST

ST
= 2 C (n/4) +2 Cn
Merge sort (C[0..(n/2-1)] = 4 (2 C (n/8) + C n/4 ) + 2Cn
Merge (B,C,A) = 8 C (n/8) +3 Cn
:
:
The merging of two sorted arrays can be done as follows: Two pointers are initialized to point to first el = 2kC(1) + kCn
Then the elements pointed to are compared and the smaller of them is added to a new array being const
smaller element is incremented to point to its immediate successor in the array it was copied from. This = an + Cnlog2n
the two given arrays is exhausted then the remaining elements of the other array are copied to the end of Since k = logn and n = 2k, we get, log2n = k(log22) = k * 1 It is easy to see that if 2k ≤ n ≤ 2k+1 ,
the new array. then

Algorithm Merge (B[0…P-1], C[0…q-1], A[0…p + q-1]) //Merge two sorted arrays into one sorted C(n) ≤ C(2k+1) Є θ(nlog2n)
array. //Input: Arrays B[0..p-1] and C[0…q-1] both sorted
There are 2 inefficiency in this algorithm:
//Output: Sorted Array A [0…p+q-1] of the elements of B & C

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

sub array, called as pivot. The pivot by default is considered to be the first element in the list. i.e. P =
1. It uses 2n locations. The additional n locations can be eliminated by introducing a key field which A (l)
is a linked field which consists of less space. i.e., LINK (1:n) which consists of [0:n]. These are The method which we use to rearrange is as follows which is an efficient method based on
pointers to elements A. It ends with zero. Consider Q&R, two scans of the sub array ; one is left to right and the other right to left comparing each element with
the pivot. The L R scan starts with the second element. Since we need elements smaller than the
Q=2 and R=5 denotes the start of each lists: pivot to be in the first part of the sub array, this scan skips over elements that are smaller than the
LINK: (1 (2) (3)
(4) (5) (6) (7) (8) pivot and stops on encountering the first element greater than or equal to the pivot. The R L scan
starts with last element of the sub array. Since we want elements larger than the pivot to be in the
1 3 0
second part of the sub array, this scan skips over elements that are larger than the pivot and stops on
Q = (2,4,1,6) and R = (5,3,7,8)
encountering the first smaller element than the pivot.

From this we conclude that A(2) < A(4) < A(1) < A (6) and A (5) < A (3) < A (7) < A(8).
Three situations may arise, depending on whether or not the scanning indices have

P
crossed. If scanning indices i and j have not crossed, i.e . i < j, exchange A [i] and A [j] and resume
2. The stack space used for recursion. The maximum depth of the stack is proportional to log2n. This
the scans by incrementing and decrementing j, respectively.
is developed in top-down manner. The need for stack space can be eliminated if we develop

AP

AP
algorithm in Bottom-up approach.
If the scanning indices have crossed over, i.e. i>j, we have partitioned the array after
exchanging the pivot with A [j].
It can be done as an in-place algorithm, which is more complicated and has a larger multiplicative
constant.
Finally, if the scanning indices stop while pointing to the same elements, i.e. i=j, the value
they are pointing to must be equal to p. Thus, the array is partitioned. The cases where, i>j and i=j
R

R
2.3.2 QUICK SORT
can be combined to have i ≥ j and do the exchanging with the pivot.
Quick sort is another sorting algorithm that is based on divide-and-conquer strategy. Quick
CO

CO
sort divides according to their values. It rearranges elements of a given array A[o…n-1] to achieve its Algorithm partition (A[l..r])
partition, a situation where all the elements before some position s are smaller than or equal to A [s] // Partitions a sub array by using its first elt as a pivot.
and all elements after s are greater than or equal to A [s]: // Input: A sub array A[l…r] of A[0…n-1] defined by its left and right indices l & r (l<r).
// O/P : A partition of A[l..r] with the split position returned as this function’s value. P←A[l]
A[o]….A[s-1] [s] A [s+1]…..A [n-1] i←l ; j←r + 1 repeat
U

U
All are < A [s] all are > A [s] repeat i←i+1 until A [i] ≥ P repeat ← j-1 until A [j]≤ swap
(A[i], A [j]
ST

ST
After this partition A [S] will be in its final position and this proceeds for the two sub arrays:
until i ≥ j
Swap (A[i], A[j] ) //undo the last swap when i ≥ j Swap
Algorithm Quicksort(A[l..r]) //Sorts sub array by quick sort
(A [l], A [j])
//I/P: A sub array A [l..r] of A [o..n-1], designed by its left and right indices l & r //O/P: A
return j
[l..r] is increasing order – sub array
if l < r Analysis:
S partition (A[l..r] // S is a split position Quick
sort A [l…s-1] The efficiency is based on the number of key comparisons. If all the splits happen in the
Quick sort A [s+1…r] middle of the sub arrays, we will have the best case. The no. of key comparisons will be:

The partition of A [0..n-1] and its sub arrays A [l..r] (0<l<r<n -1) can be achieved by the C best (n) = 2 C best (n/2) + n for n > 1 C
following algorithms. First, select an element with respect to whose value we are going to divide the best (1) = 0

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

According to theorem, C best (n) Є θ (n log 2 n); solving it exactly for n = 2 k yields C best operation is repeated recursively for the first half of the array if K < A (m) and for the second half if
(n) = n log 2 n. K > A (m).

In the worst case, all the splits will be skewed to the extreme : one of the two sub arrays Binary search can also implemented as a nonrecursive algorithm.
will be empty while the size of the other will be just one less than the size of a subarray being Algorithm Binarysearch(A[0..n-1], k)
partitioned. It happens for increasing arrays, i.e., the inputs which are already solved. If A [0…n-1] is
a strictly increasing array and we use A [0] as the pivot, the L→ R scan will stop on A[1] while the R // Implements nonrecursive binarysearch
→ L scan will go all the way to reach A[0], indicating the split at position 0: // Input: An array A[0..n-1] sorted in ascending order and a search key k
// Output: An index of the array’s element that is equal to k or -1 if there is no such
So, after making n+1 comparisons to get to this partition and exchanging the pivot A [0] with itself, // element
the algorithm will find itself with the strictly increasing array A[1..n-1] to sort. This sorting of l= 0; r= n-1
increasing arrays of diminishing sizes will continue until the last one A[n- 2..n-1] has been while l ≤ r do

P
processed. The total number of key comparisons made will be equal to: m= [(l+r)/2]
if k = A[m] return m

AP

AP
C worst (n) = (n+1) + n + ….+ 3 = (n+1) (n+2) -3 else if k < A[m] r = m-1
2 else l= m+1
Є θ (n2) return -1

Finally, the average case efficiency, let Cavg(n) be the average number of key comparisons made by
Analysis:
quick sort on a randomly ordered array of size n. Assuming that the partition split can happen in each
R

R
position s (o ≤ s ≤ n – 1) with the same probability 1/n, we get the following recurrence relation:
The efficiency of binary search is to count the number of times the search key is compared
n-1 with an element of the array. For simplicity, we consider three-way comparisons. This assumes that
CO

CO
after
Cavg(n) = ∑ [(n+1) + Cavg(s) + Cavg(n-1-s) one comparison of K with A [M], the algorithm can determine whether K is smaller, equal to, or
S=0 larger than A [M]. The comparisons not only depends on ‘n’ but also the particular instance of the
problem. The worst case comparison Cw (n) include all arrays that do not contain a search key, after
Cavg(0) = 0 , Cavg(1) = 0 one comparison the algorithm considers the half size of the array.
U

U
Therefore, Cavg(n) ≈ 2nln2 ≈ 1.38nlog2n Cw (n) = Cw (n/2) + 1 for n > 1, Cw (1) = 1 eqn (1)

Thus, on the average, quick sort makes only 38% more comparisons than in the best case. To refine To solve such recurrence equations, assume that n = 2k to obtain the solution.
ST

ST
this algorithm : efforts were taken for better pivot selection methods (such as the median – of – three Cw (2k) = k +1 = log2n+1
partitioning that uses as a pivot the median of the left most, right most and the middle element of the For any positive even number n, n = 2i, where I > 0. now the LHS of eqn (1) is:
array) ; switching to a simpler sort on smaller sub files ; and recursion elimination (so called non
recursive quick sort). These improvements can cut the running time of the algorithm by 20% to 25% Cw (n) = [ log2n]+1 = [ log22i]+1 = [ log 22 + log2i] + 1
Partitioning can be useful in applications other than sorting, which is used in selection = ( 1 + [log2i]) + 1 = [ log2i] + 2
problem also.
The R.H.S. of equation (1) for n = 2 i is

2.3.3 BINARY SEARCH Cw [n/2] + 1 = Cw [2i / 2] + 1

= Cw (i) + 1
It is an efficient algorithm for searching in a sorted array. It works by comparing a search = ([log2 i] + 1) + 1
key k with the array’s middle element A [m]. If they match, the algorithm stops; otherwise the same = ([log2 i] + 2

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Since both expressions are the same, we proved the assertion. We measure the problem’s instance size by the number of nodes n(T) is a given binary tree T.
The number of comparisons made to compute the maximum of two numbers and the number of
The worst – case efficiency is in θ (log n) since the algorithm reduces the size of the array additions A(n (T))made by the algorithm are the same.
remained as about half the size, the numbers of iterations needed to reduce the initial size n to the
The recurrence relation for A(n (T)) is:
final size 1 has to be about log2n. Also the logarithmic functions grows so slowly that its values
A(n(T)) = A(n(TL))+ A ((TR)) + 1 for n(T)>0,
remain small even for very large values of n.
A(0) = 0
The average-case efficiency is the number of key comparisons made which is slightly Analysis:
smaller than the worst case. The efficiency is based on the comparisons and addition operations, and also we should check
whether the tree is empty or not. For an empty tree, the comparison T = ∅ is executed once but these
are no additions and for a single node tree, the comparison and addition numbers are three and one
i.e.Cavg (n) ≈ log2n respectively.
The tree’s extension can be drawn by replacing the empty sub trees by special nodes which

P
More accurately, for successful search Cavg (n) ≈ log2n –1 and for unsuccessful search Cavg (n) ≈ helps in analysis. The extra nodes (square) are called external; the original nodes (circles) are called
log2n + 1. internal nodes. The extension of the empty binary tree is a single external node.

AP

AP
Though binary search is an optional algorithm there are certain algorithms like The algorithm height makes one addition for every internal node of the extended treed and
interpolation search which gives better average – case efficiency. The hashing technique does not one comparison to check whether the tree is empty for every internal and external node. The number
even require the of external nodes x is always one more than the number internal nodes n:
array to be sorted. The application of binary search is used for solving non-linear equations in one i.e. x = n + 1
R

R
unknown.
To prove this equality by induction in the no. of internal nodes n ≥ 0. The induction’s basis is
Binary search is sometimes presented as a quint essential example of divide-and-conquer. true because for n = 0 we have the empty tree with 1 external node by definition: In general, let us
CO

CO
Because, according to the general technique the problem has to be divided into several smaller assume thatx = K + 1 for any extended binary tree with 0 ≤ K ≤ n internal nodes. Let T be an
subproblems and then combine the solutions of smaller instances to obtain the original solution. But extended binary tree with n internal nodes and x external nodes, let n L and xL be the number of
here, we divide into two subproblems but only one of them need to be solved. So the binary search internal and
can be considered as a degenerative case of this technique and it will be more suited for decrease-by-
half algorithms. external nodes in the left sub tree of T and nR & nR is the internal & external nodes of right subtree
respectively. Since n > 0, T has a root which is its internal node and hence n = nL + nR + 1 using the
U

U
Binary Tree Traversals and Related Properties:
equality
A binary tree T is defined as a finite set of nodes that is either empty or consists of a
ST

ST
x = xL + xR = (nL + 1) + (nR + 1)
root and two disjoint binary trees TL and TR called the left and right sub tree of the root.
= (nL + nR + 1) +1 =
Since, here we consider the divide-and-conquer technique that is dividing a tree into left n+1 which completes the proof
subtree and right subtrees. As an example, we consider a recursive algorithm for computing the
height of a binary tree. Note, the height of a tree is defined as the length of the longest path from the In algorithm Height, C(n), the number of comparisons to check whether the tree is empty is:
root to a leaf. Hence, it can be computed as the maximum of the heights of the root’s left and right c (n) = n + x
sub trees plus 1. Also define, the height of the empty tree as – 1. Recursive algorithm is as follows:
Algorithm Height (T) = n + (n + 1)
// Computes recursively the height of a binary tree T = 2n + 1
// Input : A binary tree T
// Output : The height of T while the number of additions is:
if T = ∅ return – 1 A (n) = n
else return max {Height (TL), Height (TR)} + 1

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

The important example is the traversals of a binary tree: Pre order, Post order and In order. And, b = b1 bo implies that b = b1 10 n/2 + bo
All three traversals visit nodes of a binary tree recursively, i.e. by visiting the tree’s root and its left Therefore, C = a * b = (a1 10 n/2 + ao) * (b1 10 n/2 + bo)
and right sub trees. They differ by which the root is visited. = (a1 * b1) 10 n + (a1* b0 + ao * b1) 10 n/2 + (a0* b0)
In the Pre order traversal, the root is visited before the left and right subtrees are visited. = C210 n + C110 n/2 + Co
In the in order traversal, the root is visited after visiting the left and rightsubtrees. Where, C2 = a1 * b1 – Product of first half Co =
In the Post order traversal, the root is visited after visiting the left and right subtrees. ao * bo – Product of second half
a1 + ao) * (b1 + bo) – (C2 + Co) product of the sum of the a’s halves and the sum of the b’s halves
The algorithm can be designed based on recursive calls. Not all the operations require the minus the sum of C2 & Co.
traversals of a binary tree. For example, the find, insert and delete operations of a binary search tree If n/2 is even, we can apply the same method for computing the products C2, Co, and C1.
requires traversing only one of the two sub trees. Hence, they should be considered as the Thus, if n is power of 2, we can design a recursive algorithm, where the algorithm terminates when
applications of variable – size decrease technique (Decrease-and-Conquer) but not the divide-and- n=1 or when n is so small that we can multiply it directly.
conquer technique. Analysis:

P
2.3.4 MULTIPLICATION OF LARGE INTEGERS It is based on the number of multiplications. Since multiplication of n-digit numbers require

AP

AP
three multiplications of n/2 digit numbers, the recurrence for the number of multiplications M(n) will
Some applications like cryptology require manipulation of integers of more than 100 decimal be
digits. It is too long to fit into a word. Moreover, the multiplication of two n digit numbers, result
with n2 digit multiplications. Here by using divide-and-conquer concept we can decrease the number M(n) = 3M (n/2) for n>1, M(1) = 1
of multiplications by slightly increasing the number of additions. Solving it by backward substitutions for n=2k yields.
R

R
For example, say two numbers 23 and 14. It can be represented as follows: M (2K) = 3 M (2K-1) = 3 [3M (2K-2)] = 32M(2K-2)
1 0 1 0
23 = 2 x 10 + 3 x 10 and 14 = 1 x 10 + 4 x 10 = ….3i M (2k-i )= …..3KM (2K-K) = 3K
CO

CO
Now, let us multiply them: Since k = log2n,
23 * 14 = (2 x 101 + 3 x 100) * (1 x 101 + 4 x 100) M(n) = 3 log2n
= n log23 ≈ n 1.585
= (2*1) 102 + (3*1 + 2*4) 101 + (3*4) 100
= 322
n 1.585 , is much less than n 2
U

U
When done in straight forward it uses the four digit multiplications. It can be reduced to one digit
multiplication by taking the advantage of the products (2 * 1) and (3 * 4) that need to be computed
anyway. 2.3.5 STRASSEN’S MATRIX MULTIPLICATION
ST

ST
i.e. 3 * 1 + 2 * 4 + (2 + 3) * (1 +4) – (2 * 1) – (3 * 4)
In general, for any pair of two-digit numbers a = a1 ao and b = b1 bo their product c can be computed This is a problem which is used for multiplying two n x n matrixes. Volker Strassen in 1969
by the formula: introduced a set of formula with fewer number of multiplications by increasing the number of
additions.
C = a * b = C2 102 + C1 101 + Co
Where,
Based on straight forward or Brute-Force algorithm.
C2 = a1 * b1 – Product of 1st digits
Co = ao * bo – Product of their 2nd digits a00 * b01 + a01 *
a1 + ao) * (b1 + bo) – (C2 + Co) product of the sum of the a’s digits and the sum of the b’s digits a00 * b00 + a01 * b10 b11
minus the sum of C2 and Co.
a1 * b01 + a11 *
Now based on divide-and-conquer, if there are two n-digits integers a and b where n is a 0 * b00 + a11 * b10 a10 b11
positive even number. Divide both numbers in the middle; the first half and second half ‘a’ are a1
and ao respectively. Similarly, for b it is b1 and bo respectively.
Here, a = a1 ao implies that a = a1 10 n/2 + ao But according to strassen’s formula’s the product of two n x n matrixes are obtained as:

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Since this saving is obtained by increasing the number of additions, A (n) has to be checked for
obtaining the number of additions. To multiply two matrixes of order n>1, the algorithm needs to
m1 + m4 – m5 + m7 m3 + m5 multiply seven matrices of order n/2 and make 18 additions of matrices of size n/2; when n=1, no
m2 + m4 m1 + m3 – m2 + m6 additions are made since two numbers are simply multiplied. The recurrence relation is

A(n) = 7 A (n/2) + 18 (n/2)2 for n>1

Where,
A (1) = 0
m1 = ( a00 + a11 ) * (b00 + b11)
m2 = ( a10 + a11 )* b00
This can be deduced based on Master’s Theorem, as A(n) Є θ (nlog27). In other words, the
m3 = a00 * (b01 - b11)
number of additions has the same order of growth as the number of multiplications. Thus in
m4 = a11 * (b10 – b00)
Strassen’s algorithm it is θ (nlog27), which is better than θ (n3) of brute force.

P
m5 = ( a00 + a01 * b11
m6 = (a10 – a00) * (b00 + b01)
2.3.6 THE CLOSEST-PAIR AND CONVEX-HULL PROBLEMS BY DIVIDE-AND-
m7 = ( a01 - a11 ) * (b10 + b11)

AP

AP
CONQUER

Thus, to multiply two 2-by-2 matrixes, Strassen’s algorithm requires seven multiplications 2.3.6.1 THE CLOSEST-PAIR PROBLEM
and 18 additions / subtractions, where as the brute-force algorithm requires eight multiplications and
4 additions. Let A and B be two n-by-n matrixes when n is a power of two. (If not, pad the rows and Let P be a set of n > 1 points in the Cartesian plane. For the sake of simplicity, we assume
columns with zeroes). We can divide A, B and their product C into four n/2 by n/2 sub matrices as that the points are distinct. We can also assume that the points are ordered in nondecreasing order of
R

R
follows: their x coordinate. (If they were not, we could sort them first by an efficeint sorting algorithm such as
mergesort.) It will also be convenient to have the points sorted in a separate list in nondecreasing
CO

CO
Analysis: order of the y coordinate; we will denote such a list Q.
If 2 ≤ n ≤ 3, the problem can be solved by the obvious brute-force algorithm. If n > 3, we can
The efficiency of this algorithm, M(n) is the number of multiplications in multiplying two n divide the points into two subsets Pl and Pr of _n/2_ and _n/2_points, respectively, by drawing a
by n matrices according to Strassen’s algorithm. The recurrence relation is as follows: vertical line through the median m of their x coordinates so that _n/2_ points lie to the left of or on
the line itself, and _n/2_ points lie to the right of or on the line. Then we can solve the closest-pair
U

U
M(n) = 7M (n/2) for n>1, M(1) = 1 problem recursively for subsets Pl and Pr . Let dl and dr be the smallest distances between pairs of
points in Pl and Pr , respectively, and let d = min{dl, dr}.
Solving it by backward substitutions for n=2k yields. Note that d is not necessarily the smallest distance between all the point pairs because points
ST

ST
of a closer pair can lie on the opposite sides of the separating line. Therefore, as a step combining the
M (2K) = 7 M (2K-1) = 7 [7M (2K-2)] = 72M(2K-2) solutions to the smaller subproblems, we need to examine such points. Obviously, we can limit our
attention to the points inside the symmetric vertical strip of width 2d around the separating line, since
= ….7i M (2k-i )= …..7KM (2K-K) = 7K the distance between any other pair of points is at least d obtained from Q and hence ordered in
nondecreasing order of their y coordinate.
Since k = log2n, Scan this list, updating the information about dmin, the minimum distance seen so far, if we
M(n) = 7 log2n encounter a closer pair of points. Initially, dmin = d, and subsequently dmin ≤ d. Let p(x, y) be a
= n log27 point on this list. For a point p(x, y) to have a chance to be closer to p than dmin, the point must
≈ n 2.807 follow p on list S and the difference between their y coordinates must be less than dmin.

which is smaller than n3 required by Brute force algorithm. Geometrically,this means that p must belong to the rectangle. The principal insight exploited
by the algorithm is the observation that the rectangle can contain just a few such points, because the
points in each half (left and right) of the rectangle must be at least distance d apart. It is easy to prove
that the total number of such points in the rectangle, including p, does not exceed eight a more

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

careful analysis reduces this number to six .Thus, the algorithm can consider no more than five next Let −−−→p1pn be the straight line through points p1 and pn directed from p1 to pn. This line
points following p on the list S, before moving up to the next point. Here is pseudocode of the separates the points of S into two sets: S1 is the set of points to the left of this line, and S2 is the set of
algorithm. points to the right of this line. (We say that point q3 is to the left of the line −−−−→ q1q2 directed
from point q1 to point q2 if q1q2q3 forms a counterclockwise cycle. Later, we cite an analytical way
ALGORITHM EfficientClosestPair(P, Q) to check this condition, based on checking the sign of a determinant formed by the coordinates of the
//Solves the closest-pair problem by divide-and-conquer three points.) The points of S on the line −−−→p1pn, other than p1and pn, cannot be extreme points
//Input: An array P of n ≥ 2 points in the Cartesian plane sorted in of the convex hull and hence are excluded from further consideration.
// nondecreasing order of their x coordinates and an array Q of the The boundary of the convex hull of S is made up of two polygonal chains:an “upper” boundary
// same points sorted in nondecreasing order of the y coordinates //Output: Euclidean distance and a “lower” boundary. The “upper” boundary, called the upper hull, is a sequence of line segments
between the closest pair of points if n ≤ 3 with vertices at p1, some of the points in S1 (if S1 is not empty) and pn. The “lower” boundary,
return the minimal distance found by the brute-force algorithm else called
copy the first _n/2_ points of P to array Pl copy the same _n/2_ points from Q to array Ql copy the the lower hull, is a sequence of line segments with vertices at p1, some of the points in S2 (if S2 is

P
remaining _n/2_ points of P to array Pr copy the same _n/2_ points from Q to array Qr dl not empty) and pn. The fact that the convex hull of the entire set S is composed of the upper and
lower

AP

AP
←EfficientClosestPair(Pl, Ql) dr ←EfficientClosestPair(Pr, Qr) d ←min{dl, dr hulls, which can be constructed independently and in a similar fashion, is a very useful observation
exploited by several algorithms for this problem.
} For concreteness, let us discuss how quickhull proceeds to construct the upper hull; the lower
m←P[_n/2_ − 1].x hull can be constructed in the same manner. If S1 is empty, the empty, the algorithm identifies point
copy all the points of Q for which |x − m| < d into array S[0..num − 1] dminsq ←d2 pmax in S1, which is the farthest from the line −−−→ p1pn . If there is a tie, the point that maximizes
the angle _ pmaxppn can be selected. (Note that point pmax maximizes the area of the triangle with
R

R
for i ←0 to num − 2 do k←i + 1
two vertices at p1 and pn and the third one at some other point of S1.) Then the algorithm identifies
while k ≤ num − 1 and (S[k].y − S[i].y)2 < dminsq
all the points of set S1 that are to the left of the line −−−→ p1pmax;these are the points that will
dminsq ←min((S[k].x − S[i].x)2+ (S[k].y − S[i].y)2, dminsq) k←k + 1
CO

CO
make up the set S1,1. The points of S1 to the left of the line −−−−−−→ pmaxpn will make up the set
return sqrt(dminsq)
S1,2. It is not difficult to prove the following: pmax is a vertex of the upper hull.
The points inside _p1pmaxpn cannot be vertices of the upper hull (and hence can be
the following recurrence for the running time of the algorithm:
T (n) = 2T (n/2) + f (n), where f (n) _(n). eliminated from further consideration).There are no points to the left of both lines −−−→p1pmax and
−−−−−−→
U

U
Applying the Master Theorem (with a = 2, b = 2, and d = 1), we get T (n) _(n log n). The necessity pmaxpn. Therefore, the algorithm can continue constructing the upper hulls of p1 ∪ S1,1 ∪ pmax
to presort input points does not change the overall efficiency class if sorting is done by a O(n log n) and pmax ∪ S1,2 ∪ pn recursively and then simply concatenate them to get the upper hull of the
entire set
algorithm such as mergesort. In fact, this is the best efficiency class one can achieve, because it has p1 ∪ S1 ∪ pn. actually implemented. Fortunately, we can take advantage of the following very
ST

ST
been proved that any algorithm for this problem must be in _(n log n) under some natural useful fact from analytical geometry: if q1(x1, y1), q2(x2, y2), and q3(x3, y3) are three arbitrary
assumptions about operations an algorithm can perform. points in the Cartesian plane, then the area of the triangle _q1q2q3 is equal to one-half of the
magnitude of the determinant
$$$$$$
2.3.6.2 CONVEX-HULL PROBLEM x1 y1 1
x2 y2 1
Let us revisit the convex-hull problem,: find the smallest convex polygon that contains n x3 y3 1
$$$$$$
given points in the plane. We consider here a divide-and-conquer algorithm called quickhull because = x1y2 + x3y1 + x2y3 − x3y2 − x2y1 − x1y3,
of its resemblance to quicksort. while the sign of this expression is positive if and only if the point q3 = (x3, y3) is to the left of the
Let S be a set ofn> 1 points p1(x1, y1), . . . , pn(xn, yn) in the Cartesian plane.We assume that line −−−→ q1q2. Using this formula, we can check in constant time whether a point lies to the left of
the line determined by two other points as well as find the distance from the point to the line.
the points are sorted in nondecreasing order of their x coordinates,with ties resolved by increasing
order of the y coordinates of the points involved.It is not difficult to prove the geometrically obvious
fact that the leftmost point p1 and the rightmost point pn are two distinct extreme points of the set’s
convex hull

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

UNIT III n 1 c(n,k)

Algorithm Binomial (n,k)
DYNAMIC PROGRAMMING AND GREEDY TECHNIQUE // Computes c(n,k)
// Input:A pair of non –ive int n ≥ k ≥ 0
Dynamic programming – Principle of optimality - Coin changing problem, Computing a
Binomial Coefficient – Floyd‘s algorithm – Multi stage graph - Optimal Binary Search Trees – // Output:The value of c(n,k)
Knapsack Problem and Memory functions. Greedy Technique – Container loading problem - for i 0 to n do
Prim‘s algorithm and Kruskal's Algorithm – 0/1 Knapsack problem, Optimal Merge pattern - for j 0 to min(i,k) do if
Huffman Trees. j=0 or j=k
c[i,j] 1
3.1 DYNAMIC ROGRAMMING else c[i,j] c[i-1, j-1] + c[i-1, j]
return c[n,k]
Invented in 1950 by Richard Bellman an U.S mathematician as a general method for

P
optimizing multistage decision processes. It is a planning concept, which is a technique for
The time efficiency is based on the basic operation i.e., addition. Also the 1st k+1 rows of the
solving problems with overlapping subproblems. This arises from a recurrence relating a
table form a triangle, while the remaining n-k rows form a rectangle, we have to split the sum

AP

AP
solution to a given problem with solutions to its smaller subproblems of the same type. Rather expressing A(n,k) into 2 parts
than solving overlapping subproblems it suggests solving each of the smaller subproblems only
once and recording the results in a table from which we can then obtain a solution to the
original problem. It can also be used for avoiding using extra space. k i-1 n k k n
Bottom-up approach all the smaller subproblems to be solved. In top-down it avoids solving A(n,k) = ∑ ∑ 1 + ∑∑ 1 = ∑ (i-1) + ∑ k
the unnecessary subproblems, which is called memory-functions.
R

R
i= i=k i=k

3.1.1 COMPUTING A BINOMIAL COEFFICIENT 1 j=1 +1 j=1 i=1 +1

CO

CO
It’s used to apply in nonoptimization problem. Elementary combinatorics, binomial = [(k-1)k]/2 + k(n-k) Є θ(nk)
coefficient, denoted c(n,k), is the number of combinations k elements from an n-element set (0 ≤
k ≤ n). 3.2 WARSHALL’S AND FLOYD’S ALGORITHMS
The binomial formula is:
U

U
(a+b)n = c(n,0)an + …. + c(n,i)an-ibi + … + c(n,n)bn Warshall’s algorithm for computing the transitive closure of a directed graph and Floyd’s
c(n,k) = c(n-1,k-1) + c(n-1,k) for n>k>0 and c(n,0)= c(n,n) = 1 algorithm for the all pairs shortest-paths problem.
ST

ST
3.2.1 Warshall’s algorithm:
0 1 2………………………………k-1 k
The adjacency matrix A={aij} of a directed graph is the Boolean matrix that has 1 in its ith row
0 1
and jth column iff there is a directed edge from the ith vertex to jth vertex.
1 1 1
Definition:- The transitive closure of a directed graph with n vertices can be defined as the n x n
2 1 21
Boolean matrix T={tij}, in which the element in the ith row (1<=i<=n) and the jth column (1 ≤ j
. ≤ n) is 1 if there exists a nontrivial directed path (i.e., directed path of positive length) from the
. ith vertex to the jth vertex; otherwise ,tij is 0.
.
k 1 1 We can generate it with DFS or BFS .It constructs the transitive closure of a given
. diagraph with n vertices through a series of n x n matrices.
. R0, R (1), …… R (k-1) , R (k),……R(n) .

.
n-1 1 c(n-1,k-1) c(n-1,k)

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

ALGORITHMS ALGORITHMS
 DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS As to the space is considered we used separate matrices for recording intermediate
results, which is unnecessary.

The element rij(k) in the ith row and j th col of matrix R(k) = 1 iff there exists a directed path 3.2.1 Floyd’s algorithm for the All-pairs shortest paths problem:
from the ith vertex to jth vertex with each intermediate vertex, if any, not numbered higher than
k. It starts with R(0) which doesnot allow any intermediate vertices in its paths; R(1) use the 1st It is to find the distances (the length of the shortest paths) from each vertex to all other
vertex as intermediate; i.e., it may contain more ones than R(0). Thus, R(n) reflects paths that can vertices. Its convenient to record the lengths of shortest paths in an n x n matrix D called
use all n vertices of the digraph as intermediate and hence is nothing else but the digraph’s distance matrix. The element dij in the ith row and jth column of this matrix indicates the
transitive closure.
length of the shortest path from the ith vertex to jth vertex (1≤ i,j ≤ n).
(k-1).

R(k) is computed from its immediate predecessor R(k-1) .Let rij (k),the element in the ith row and Floyd’s algorithm is applicable to both undirected and directed weighted
jth col of matrix R(k) ,be equal to 1.This means that there exists a path from ith vertex vi to the jth graphs provided that they do not contain a cycle of negative length.
vertex vj with each intermediate vertex numbered not higher than k.
It computes the distance matrix of a weighted graph with n vertices thru a series of n x n
vi, a list of intermediate vertices each numbered not higher than k,vj.

P
P
matrices.
D0, D (1), …… D (k-1) , D (k),…… D(n)
There are 2 possibilities. First is, if there is a list the intermediate vertex doesnot contains the kth

AP
AP
vertex. The path from vi to vj has vertices not higher than k-1, and therefore rij(k -1)=1. The second
is, that path does contain the kth vertex vk among the intermediate vertices. vk occurs only once in D(0) with no intermediate vertices and D(n) consists all the vertices as intermediate
the list. Therefore vi,vertices numbered ≤k-1, vk, vertices numbered ≤k-1, vj. vertices. dij(k) is the shortest path from vi to vj with their intermediate vertices numbered not
Means there exists a path from vi to vk with each intermediate vertex numbered not higher than higher than k. D(k) is obtained from its immediate predecessor D(k-1).
k- 1 hence rik(k-1) =1 and the 2nd part is such that a path from vk to vj with each intermediate

R
R
vertex numbered not higher than i.e., vi a list of int. vertices each numbered not higher than k, vj.
k-1 hence, rkj(k-1) = 1 i.e., if rij(k) = 1,then either rij(k-1) = 1 or both rik(k-1) =1 and
rkj(k-1) =1. We can partition the path into 2 disjoint subsets: those that do not use the k th vertex as
CO

CO
intermediate and those that do. Since the paths of the 1st subset have their
Thus the formula is : rij(k) = rij(k -1) or rik(k-1) and rkj(k-1) This intermediate vertices numbered not higher than k-1,the shortest of them is dij
formula yields the Warshall’s algorithm which implies:
 If an element rij is 1 in R(k-1) ,it remains 1 in R(k) . The length of the shortest path in the 2nd subset is,if the graph doesnot contain a cycle of
 If an elt rij is 0 in R(k-1) , it has to be changed to 1 in R(k) iff the element in its row i negative length, the subset use vertex vk as intermediate vertex exactly once.
U

U
and column k and the element in its column j and row k are both 1’s in R(k-1) .
vi,vertices numbered ≤ k-1, vk, vertices numbered ≤ k-1, vj .
ST

Algorithm Warshall (A[1..n,1..n])

ST
R(0) A The path vi to vk is equal to dik(k-1) and vk to vij is dkj(k-1) ,the length of the shortest
for k 1 to n do for i 1
to n do path among the paths that use the kth vertex is equal to dik(k-1) + dkj(k-1).
for j 1 to n do
R(k)[i,j] R R(k-1) [i,j] or R R(k-1) [i,k] and R R(k-1) [k,j] dij(k) = min{ dij(k-1), dik(k-1) + dkj(k-1) } for k ≥ 1,
(n)
Return R . dij(0) = wij.
3
The time efficiency is in θ(n ).This can be speed up by restricting its innermost loop.
Algorithm Floyd(w[1..n,1..n])
Another way to make the algorithm run faster is to treat matrix rows as bit strings and apply
bitwise OR operations. //Implemnt Floyd’s algorithm for the all-pairs shortest-paths problem
// Input: The weight matrix W of a graph
// Output: The distance matrix of the shortest paths lengths

Dw
For k 1 to n do
DOWNLOADED FROM STUCOR APP
ALGORITHMS

DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

For i 1 to n do EXAMPLE Let us illustrate the algorithm by applying it to the four-key set we
For j 1 to n do used at the beginning of this section:
D[i,j] min {D[i,j], D[i,k] + D[k,j]} Key A B C D
Probability 0.1 0.2 0.4 0.3
Return D.
C(i, j) = mini≤k≤j {C(i, k − 1) + C(k + 1, j)} +s=ito j ps for 1≤ i ≤ j ≤ n.
The time efficiency is θ (n3).The principle of optimality holds for these problems
where there is an optimization. Thus, out of two possible binary trees containing the first two keys, A and B, the root of the
optimal tree has index 2 (i.e., it contains B), and the average number of comparisons in a
successful search in this tree is 0.4.
3.3 OPTIMAL BINARY SEARCH TREES
ALGORITHM OptimalBST(P [1..n])
A binary search tree is one of the most important data structures in computer science.
One of its principal applications is to implement a dictionary, a set of elements with the //Finds an optimal binary search tree by dynamic programming
operations of searching, insertion, and deletion. If probabilities of searching for elements of a //Input: An array P[1..n] of search probabilities for a sorted list of n keys
set are known—e.g., from accumulated data about past searches—it is natural to pose a //Output: Average number of comparisons in successful searches in the
question about an optimal binary search tree for which the average number of comparisons in a // optimal BST and table R of subtrees’ roots in the optimal BST

P
search is the smallest possible. For simplicity, we limit our discussion to minimizing the
average number of comparisons in a successful search. The method can be extended to include for i ←1 to n do
unsuccessful searches as well.As an example, consider four keys A, B, C, and D to be searched C[i, i − 1]←0

AP

AP
for with probabilities 0.1, 0.2, 0.4, and 0.3, respectively. The average number of comparisons C[i, i]←P[i]
in a successful search in the first of these trees is 0.1 . 1+ 0.2 . 2 + 0.4 .3+ 0.3 . 4 = 2.9, and for R[i, i]←i
the second one it is 0.1 . 2 C[n + 1, n]←0
+ 0.2 . 1+ 0.4 . 2 + 0.3 . 3= 2.1.Neither of these two trees is, in fact, optimal. (Can you tell for d ←1 to n − 1 do //diagonal count
which binary tree is optimal?) For our tiny example, we could find the optimal tree by for i ←1 to n − d do
generating all 14 binary search trees with these keys. As a general algorithm, this exhaustive- j ←i + d
R

R
search approach is unrealistic: the total number of binary search trees with n keys is equal to minval←∞
the nth Catalan number, for k←i to j do
c(n) = 1/(n + 1)(2n/n) for n > 0, c(0) = 1, if C[i, k − 1]+ C[k + 1, j]< minval
CO

CO
which grows to infinity as fast as 4n/n1.5 (see Problem 7 in this section’s exercises). So let a1, minval←C[i, k − 1]+ C[k + 1, j]; kmin←k
. . . , an be distinct keys ordered from the smallest to the largest and let p1, . . . , pn be the R[i, j ]←kmin
probabilities of searching for them. Let C(i, j) be the smallest average number of comparisons sum←P[i]; for s ←i + 1 to j do sum←sum + P[s] C[i,
made in a successful search in a binary search tree Tji made up of keys ai, . . j ]←minval + sum
. , aj , where i, j are some integer indices, 1≤ i ≤ j ≤ n.Following the classic dynamic return C[1, n], R
U

U
programming approach, we will find values of C(i, j) for all smaller instances of the problem,
although we are interested just in C(1, n). To derive a recurrence underlying a dynamic The algorithm’s space efficiency is clearly quadratic; the time efficiency of this version of the
programming algorithm, we will consider all possible ways to choose a root ak among the algorithm is cubic (why?).Amore careful analysis shows that entries in the root table are
ST

ST
keys ai, . . . , aj . For such a binary search tree (Figure 8.8), the root contains key ak, the left always nondecreasing along each row and column. This limits values for R(i, j) to the range
subtree T k−1 i contains keys ai, . . . , ak−1 optimally arranged, and the right subtree Tjk+1 R(i, j − 1), . . . , R(i + 1, j) and makes it possible to reduce the running time of the algorithm to
contains keys ak+1, . . . , aj also optimally arranged. (Note how we are taking advantage of the _(n2).
principle of optimality here.). 3.4 THE KNAPSACK PROBLEM AND MEMORY FUNCTIONS
The two-dimensional table has the values needed for computing C(i, j) by formula
,they are in row i and the columns to the left of column j and in column j and the rows below The knapsack problem states that given n items of known weights w1, w2 ,…, wn and values
row i. The arrows point to the pairs of entries whose sums are computed in order to find the v1, v2 ,….., vn and a knapsack of capacity W,find the most valuable subset of the
smallest one to be recorded as the value of C(i, j). This suggests filling the table along its items that fit into the knapsack. In dynamic programming we need to obtain the solution by
diagonals, starting with all zeros on the main diagonal and given probabilities pi, 1≤ i ≤ n, right
solving the smaller subinstances.
above it and moving toward the upper right corner.
The algorithm we just sketched computes C(1, n)—the average number of comparisons
for successful searches in the optimal binary tree. If we also want to get the optimal tree itself, Let v[i,j] be the value of an optimal soln to the instance, i.e., the value of the most
we need to maintain another two-dimensional table to record the value of k for which the valuable subset of the 1st i items that fit into the knapsack of capacity j. We can divide the
minimum is achieved. The table has the same shape as the table in and is filled in the same subsets into 2: those that do not include the ith item and those that do.
manner, starting with entries R(i, i) = i for 1≤ i ≤ n. When the table is filled, its entries indicate
indices of the roots of the optimal subtrees, which makes it possible to reconstruct an optimal
1. Among the subsets that do not include the ith item, the value of an optimal subset
tree for the entire set given.
is,by definition: v[i-1,j]

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

2. Among the subsets that do not include the ith item, an optimal subset is made up 3.5 GREEDY TECHNIQUE
of this item and an optimal subset of the 1st i-1 items that fit into the knapsack of
capacity j-wi. The value of such an optimal subset is vi+v[i-1,j-wi]. The change making problem is a good example of greedy concept. That is to give change
for a specific amount ‘n’ with least number of coins of the denominations d1 >d2……dm. For
Thus the value of an optimal solution among all feasible subsets of the 1 st i items is the example: d1
maximum of these 2 values. Of course, if the ith item does not fit into the knapsack,the value =25, d1=10, d3=5, d4=1. To get a change for 48 cents. First step is to give 1 d1, 2 d2 and 3 d4’s
of an optimal subset selected from the 1st i items is the same as the value of an optimal subset which gives a optimal solution with the lesser number of coins. The greedy technique is applicable
selected from the first i-1 items. to optimization problems. It suggests constructing a solution through a sequence of steps, each
expanding a partially constructed solution obtained so far, until a complete solution to the problem
Therefore, v[i,j] = { max{ v[i-1,j], vi + v[i-1,j-wi] },if j-wi ≥ 0 v[i-1,j], if j-wi< is reached. On each step the choice made must be –
0.  Feasible - i.e., it has to satisfy the problem constraints.
The initial conditions are:  Locally optimal - i.e., it has to be the best local choice among all feasible
v[0,j] = 0 for j ≥ 0 and v[i,0] = 0 for i ≥ 0. choices available on that step.
Our goal is to find v[n,w],the maximal value of a subset of n items which fit into W.  Irrevocable – i.e., once made, it cannot be changed on subsequent steps of the

P
algorithm.
The maximal value is v[4,5]=$37. Since v[4,5] ≠ v[3,5] item 4 was included in an
optimal solution along with an optimal subset for filling 5-2 = 3 remaining units of the knapsack
PRIM’S ALGORITHM

AP

AP
3.5.1
capacity. Next v[3,3] = v[2,3], item 3 is not a part of an optimal subset. Since v[2,3] ≠ v[1,3] item
2 is a part of an optimal selection, which leaves elt.v[1,3-1] to specify the remaining composition.
A spanning tree of connected graph is its connected acyclic sub graph that contains all the
v[1,2] ≠ v[0,2] item 1 is part of the solution. Therefore {item1,item2,item4} is the subset with
vertices of the graph. A minimum spanning tree of a weighted connected graph is its spanning tree
value 37.
of the smallest weight, where the weight of the tree is defined as the sum of the weight on all its
edges. The minimum spanning tree problem is the problem of finding a minimum spanning tree
R

R
Memory Functions:
for a given weighted connected graph.
Two serious obstacles are: first, the number of spanning tree grows exponentially with
The direct top-down approach to finding a solution to such a recurrence leads to an algorithm that
CO

CO
the graph size. Second, generating all spanning trees for the given graph is not easy; in fact, it’s
solves common subproblems more than once and hence is very inefficient. The other approach is
more difficult than finding a minimum spanning for a weighted graph by using one of several
bottom-up, it fills a table with solutions to all smaller subproblems but each of them is solved only
efficient algorithms available for this problem.
once. The drawback of bottom-up is to solve all smaller subproblems even if its not necessary for
getting the solution we use to combine the top-down and bottom-up, where the goal is to get a Graph and its spanning tree ; T1 is the min spanning tree .
method that solves only subproblems that are necessary and does it only once. Such a method is
U

U
by using memory functions. Prim’s algorithm constructs a minimum spanning tree thru a sequence of expanding sub
trees. The initial sub tree in such a sequence consists of a single vertex selected arbitrarily from
the set V of the graph’s vertices. On each iteration, we expand the current tree in the greedy
ST

ST
This method solves by top-down; but, creates a table to be filled in by bottom-up. Intially, all the
table’s entries are initialized with a special “null” symbol to indicate that they have not yet been manner by simply attaching to it the nearest vertex not in that tree. The algorithm stops after being
calculated called virtual initialization. Therefore, whenever a new value needs to be calculated, the constructed. The total number of iterations will be n-1, if there are ‘n’ number of edges.
method checks the corresponding entry in the table first: if this entry is not null its retrieved from
the table; otherwise, its computed by the recursive call whose result is then recorded in the table. Algorithm Prim(G)
// Input: A weighted connected graph G = {V, E}
Algorithm MF Knapsack(i,j) // Output: ET, the set of edges composing a minimum spanning tree of G.
// Uses a global variables input arrays w[1..n],v[1..n] and VT <-{V0}
// Table v[0..n,0..w] whose entries are initialized ET <- ø
// with -1’s except for row 0 and column 0 initialized with 0’s. if For i <- 1 to |V|-1 do
v[i,j] < 0 if j< weights[i] Find a minimum weight edge e*=(v*,u*) among all the edges (v,u)
value MF knapsack(i-1, j) else such that v is in VT & u is in V-VT.
value max (MF knapsack(i-1,j),values[i]+MF knapsack(i-1,j-weights[i]) v[i,j] value
return VT <- VT U
v[i,j].
{u*} ET <- ET
U {e*}

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Return ET.
3.5.2 KRUSKAL’S ALGORITHM
The algorithm makes to provide each vertex not in the current tree with the information
about the shortest edge connecting the vertex to a tree vertex. Vertices that are not adjacent is It looks at Minimum Spanning Tree for a weighted connected graph G=<V,E> as
labeled “∞ “ (infinity). The Vertices not in the tree are divided into 2 sets : the fringe and the an acyclic sub graph with |V|-1 edges for which the sum of the edges weights is the smallest. The
unseen. The fringe contains only the vertices that are not in the tree but are adjacent to atleast one algorithm constructs a Minimum Spanning Tree as an expanding sequence of sub graphs, which
tree vertex. From these next vertices is selected. The unseen vertices are all the other vertices of are always acyclic but are not necessarily connected on the intermediate stages of the algorithm.
the graph, (they are yet to be seen). This can be broken arbitrarily.
After we identify a vertex u* to be added to the tree, we need to perform 2 operations: The algorithm begins by sorting the graphs edges in increasing order of their
 Move u* from the set V-VT to the set of tree vertices VT. weights. Then, starting with the empty subgraph , it scans this sorted list adding the next edge on
the list to the current sub graph if such an inclusion does not create a cycle and simply skips the
 For each remaining vertex u in V-VT that is connected to u* by a shorter edge than the edge otherwise.
u’s
current distance label, update its label by u* and the weight of the edge between u* & u, Algorithm Kruskal(G)

P
respectively. // Input: A weighted graph G=<V,E>
Prim’s algorithm yields always a Minimum Spanning Tree. The proof is by induction. // Output: ET,the set of edges composing a Minimum Spanning

AP

AP
Since T0 consists of a single vertex and hence must be a part of any Minimum Spanning Tree. Tree of G Sort E in increasing order of edge weights
Assume that Ti-1 is part of some Minimum Spanning Tree. We need to prove that Ti generated ET<- ø ; ecounter <- 0 //initialize the set of tree edges and its size.
from Ti-1 is also a part of Minimum Spanning Tree, by contradiction. Let us assume that no
Minimum Spanning Tree of the graph can contain Ti. let ei = (v,u) be the minimum weight edge
from a vertex in Ti-1 to a vertex not in Ti-1 used by Prim’s algorithm to expand T i-1 to Ti, e i
cannot belong to any Minimum Spanning Tree including T. Therefore if we add ei to T, a cycle K <- 0 //initialize the no of processed edges
R

R
must be formed. While ecounter < |V|-1
K <- K+1
In addition to edge ei = (v,u), this cycle must contain another edge (v’,u’)
CO

CO
if ET U{eik} is acyclic
connecting a vertex v’ Є Ti-1 to a vertex u’ which is not in Ti-1. If we now delete the edge (v’,u’) ET <- ET U {eik};
from this cycle we obtain another spanning tree of the entire graph whose weight is less than or ecounter<-ecounter+1
equal to the weight of T. Since the weight of ei is less than or equal to the weight of (v’,u’). Hence
Return ET.
this is a Minimum Spanning Tree and contradicts that no Minimum Spanning Tree contains Ti.
Kruskal’s algorithm is not simpler than prim’s. Because, on each iteration it has to check
U

U
The efficieny of this algorithm depends on the data structure chosen for the graph itself and for the whether the edge added forms a cycle. Each connected component of a sub graph generated is a
p vertex priorities are the distance to the nearest tree vertices. For eg, if the graph is represented by tree because it has no cycles.
weigh matrix and the priority queue is implemented as an unordered array the algorithm’s running
ST

ST
time will be in θ(|V|2). There are efficient algorithms for doing these observations, called union_ find algorithms.
With this the time needed for sorting the edge weights of a given graph and it will be O(|E|log|E|) .
Priority queue can be implemented by a min_heap, which is a complete binary tree
in which every element is less than or equal to its children. Deletion of smallest element from and Disjoint sub sets and union_find algorithm.
insertion of a new element into a min_heap of size n is O(logn) operations.
Kruskal’s algorithm requires a dynamic partition of some n-element set S into a collection
If a graph is represented by its adjacency linked lists and the priority queue is of disjoint subsets S1,S2,S3….. Sk. After initializing each consist of different elements of S,
implemented as a min_heap, the running time of the algorithm’s is in O(|E|log|V|). This is because which is the
the algorithm performs |V|-1 deletions of the smallest element and makes |E| verifications , and sequence of intermixed union and find algorithms or operations . Here we deal with an abstract
changes of an element’s priority in a min_heap of size not greater than |V|. Each of these data type of collection of disjoint subsets of a finite set with the following operations:
operations is a O(log|V|) operations.Hence , the running time is in:
 Makeset(x): Creates a 1-elt set{x}. Its assumed that this operations
(|V|-1+|E|) O(log|V|) = O(|E|log|V|) can be applied to each of the element of set S only once.
Because , in a connected graph ,|V|-1 ≤ |E|.  Find(x): Returns a subset containing x.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

 Union(x,y): Constructs the union of the disjoint subsets Sx & Sy containing x & y
respectively and adds it to the collection to replace Sx & Sy, which are deleted from it. For the makeset(x) time is in θ(1) and for ‘n’ elements it will be in θ(n). The efficiency of find(x)
is also in θ (1): union(x,y) takes longer of all the operations like update & delete.
For example, Let S={1,2,3,4,5,6}. Then make(i) Creates the set{i} and apply this operation to
create single sets:
Union (2,1), union(3,2),….. , union(i+1,i),….union(n,n-1)
{1},{2},{3},{4},{5},{6}.
Performing union (1,4) & union(5,2) yields Which runs in θ (n2) times , which is slow compared to other ways: there is an operation called
{1,4},{5,2},{3},{6} union-by-size which unites based on the length of the list. i.e., shorter of the two list will be
attached to longer one, which takes θ(n) time. Therefore, the total time needed will be θ(nlog n).
Then union(4,5) & union(3,6) gives
{1,4,5,2},{3,6} Each ai is updated Ai times, the resulting set will have 2Ai elements. Since the entire set
has n elements, 2Ai ≤ n and hence A i ≤ log2n. Therefore, the total number of possible updates of
It uses one element from each of the disjoint sub sets in a collection as that subset’s representative. the representatives for all n elements is S will not exceed nlog2n.
There are two alternatives for implementing this data structure, called the quick find, optimizes the Thus for union by size, the time efficiency of a sequence of at most n-1 unions and m finds
time efficiency of the find operation, the second one, called the quick union, optimizes the union is in O(n log n+m).

P
operation. The quick union-represents by a rooted tree. The nodes of the tree contain the subset
elements , with the roots element considered the subsets representatives, the tree’s edges are

AP

AP
directed from children to their parents.
Size Last First
Makeset(x) requires θ (1) time and for ‘n’ elements it is θ (n), a union (x,y) is θ (1) and
find(x) is in θ (n) because a tree representing a subset can degenerate into linked list with n nodes.
L List 1 4 1 4 5 2 The union operation is to attach a smaller tree to the root of a larger one. The tree size can
be measured either by the number of nodes or by its height(union-by-rank). To execute each find it
R

R
takes O(log n) time. Thus, for quick union , the time efficiency of at most n-1 unions and m finds
Null Null is in O(n+ m log n).
L List 2 0l n
CO

CO
Forest representation of subsets,{1,4,5,2} & {3,6}.
Resultant of union(5,6).
L List 3 2 3 6 n Null
An even better efficiency can be obtained by combining either variety of quick union with path
compression. This modification makes every node encountered during he execution of a find
U

U
operation point to the tree’s root.
n Null
List 4 0l n Null
3.5.3 DIJKSTRA’S ALGORITHM
ST

ST
n Null
The single-source shortest paths problem: for a given vertex called the source in a
List 5 0l n Null
weighted connected graph, find shortest paths to all its other vertices is considered. It is a set pf
paths, each leading from the source to a different vertex in the graph, though some paths may. of
n Null course, have edges in common.
L List 6 0l Null
This algorithm is applicable to graphs with nonnegative weights only. It finds shortest
paths to a graph’s vertices in order of their distance from a given source. First, it finds the shortest
Subset representatives
path from the source to a vertex nearest to it, then to a second nearest and so on. In general ,
Element Index Representation
before its ith iteration commences, the algorithm has already, identified its shortest path to i -1
1 1
other vertices nearest to the source. This forms a sub tree Ti and the next vertex chosen to be
2 1
should be vertices adjacent to the vertices of Ti, fringe vertices. To identify , the i th nearest vertex,
3 3
the algorithm computes for every fringe vertex u, the sum of the distance to the nearest tree vertex
4 1 v and then select the smallest such sum. Finding the next nearest vertex u* becomes the simple
5 1 task of finding a fringe vertex with the smallest d value. Ties can be broken arbitrarily.
6 3 After we have identified a vertex u* to be added to the tree, we need to perform two operations:

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

 Move u* from the fringe to the set of the tree vertices. Example : consider the 5 char alphabet {A,B,C,D,-} with the following occurrence probabilities:
 For each remaining fringe vertex u that is connected to u* by an edge of weight w(u*,u)
s.t. du* + w(u*,u) < du, update the labels of u by u* and , du* + w(u*,u) respectively. Char A B C D __

Algorithm Dijkstra(a)
Probability 0 0.35 0 0.1 0 0.2 0 0.2 0 0.15
// Input: A weighted connected graph G=<V,E> and its vertex s
// Output: The length dv of a shortest path from s to v and its penultimate vertex pv //
// for every vertex v in V.
The resulting code words are as follows:
Initialize(Q) //initialize vertex priority queue to empty for every vertex v in V do for
every vertex v in V do Char A B C D _ _
dv <- ∞; pv <-null
Insert(Q,v, dv) //initialize vertex priority in the priority queue ds Probability 0 0.35 0 0.1 0 0.2 0 0.2 0 0.15
<- 0; decrease(Q,s, ds) //update priority of s with ds VT <- ø
Codeword 1 11 1 100 0 00 0 01 1 101

P
for i<-0 to |V|-1 do
u* <- deleteMin(Q) //delete the minimum priority element
VT <- VT U {u*} Hence DAD is encoded as 011101, and 10011011011101 is decoded as BAD-AD.

AP

AP
for every vertex u in V-VT that is adjacent to u* do
if du* + w(u*,u) < du With the occurrence probabilities given and the codeword lengths obtained, the expected
du <- du* + w(u*,u); pu <- number of bits per char in this code is:
u* Decrease(Q,u, du)
2*0.35 + 3*0.1 + 2*0.2 + 2*0.2 + 3*0.15 = 2.25
R

R
The time efficiency depends on the data structure used for implementing the priority queue The compression ratio, is a measure of the compression algorithms effectiveness of (3-
and for representing the input graph. It is in θ(|V|)2 for graphs represented by their weight matrix 2.25)/3*100%=25%. The coding will use 25% less memory than its fixed length encoding.
and the priority queue implemented as an unordered array. For graphs represented by their
CO

CO
adjacency linked lists and the priority queue implemented as a min_heap it is in O(|E|log|V|). Huffman’s encoding is one of the most important file compression methods. It is simple and
yields an optimal.
3.6 HUFFMAN TREES
The draw back can be overcome by the so called dynamic Huffman encoding, in which the
Suppose we have to encode a text that comprises n characters from some alphabet by coding tree is updated each time a new char is read from source text.
U

U
assigning to each of the text’s characters some sequence of bits called the code word. Two types
are there :fixed length encoding that assigns to each character a bit string of the same length m Huffman’s code is not only limited to data compression. The sum ∑ liwi where i=1
variable-length encoding, which assigns codewords of different lengths to different characters , li is the length of the simple path from the root to ith leaf, it is weighted path length. From this
ST

ST
decision trees, can be obtained which is used for game applications.
introduces a problem that fixed length encoding does not have.

To avoid complication, we called prefix-free code. In a prefix code, no codeword is a

prefix of a code word of another character. Hence, with such an encoding we can simplify scan a
bit string until we get the first group of bits that is a code word for some character, replace these
bits by this character, and repeat this operation until the bit string’s end is reached.
Huffman’s algorithm

Step 1: Initialize n one-node trees and label them with the characters of the alphabet.
Record the frequency of each character in its tree’s root to indicate the tree’s weight.
Step 2: Repeat the following operation until a single tree is obtained. Find two trees with the
smallest weight. Make them the left and right sub tree of a new tree and record the sum of their
weights in the root of the new tree as its weight. A tree constructed by the above algm is
called a Huffman tree. It defines – a Huffman code.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

UNIT IV Step 4 [Form the next tableau] Divide all the entries in the pivot row by its entry in the pivot
column. Subtract from each of the other rows, including the objective row, the new pivot row
ITERATIVE IMPROVEMENT multiplied by the entry in the pivot column of the row in question. Replace the label of the pivot
row by the variable's name of the pivot column and go back to Step 1.
The Simplex Method-The Maximum-Flow Problem – Maximm Matching in Bipartite Graphs-
maximize z = 3x + 5y + 0u + 0v
The Stable marriage Problem.
subject to y+u =4
4.1 THE SIMPLEX METHOD 3y v =6
≥0, y≥0, u≥0, ≥0
Variables u and v, transforming inequality constraints into equality constrains, are called slack
variables 4.2 THE MAXIMUM-FLOW PROBLEM
A basic solution to a system of m linear equations in n unknowns (n ≥ m) is obtained by setting n –
m variables to 0 and solving the resulting system to get the values of the other m variables. The Maximum Flow Problem
variables set to 0 are called nonbasic; the variables obtained by solving the system are called basic. Problem of maximizing the flow of a material through a transportation network (e.g., pipeline
A basic solution is called feasible if all its (basic) variables are nonnegative. system, communications or transportation networks)

P
Example x+ y+ u =4 Formally represented by a connected weighted digraph with n vertices numbered from 1 to n with
x + 3y + v=6 the following properties:

AP

AP
(0, 0, 4, 6) is basic feasible solution  contains exactly one vertex with no entering edges, called the source (numbered 1)
(x, y are nonbasic; u, v are basic)  contains exactly one vertex with no leaving edges, called the sink (numbered n)
There is a 1-1 correspondence between extreme points of LP’s feasible region  has positive integer weight uij on each directed edge (i.j), called the edge capacity,
and its basic feasible solutions. indicating the upper bound on the amount of the material that can be sent from i to j
Maximize z = 3x + 5y + 0u + 0v through this edge
R

R
subject to x +y + u= 4
x + 3y+v= 6 Definition of flow:
x≥0, y≥0, u≥0, v≥0 A flow is an assignment of real numbers xij to edges (i,j) of a given network that satisfy the
CO

CO
basic feasible solution following:
(0, 0, 4, 6) flow-conservation requirements: The total amount of material entering an intermediate vertex
must be equal to the total amount of the material leaving the vertex

capacity constraints
0 ≤ xij ≤ uij for every edge (i,j) E Flow
U

U
value and Maximum Flow Problem
Since no material can be lost or added to by going through intermediate vertices of the
ST

ST
network, the total amount of the material leaving the source must end up at the sink:
∑ x1j = ∑ xjn
j: (1,j) є E j: (j,n) є E
The value of the flow is defined as the total outflow from the source (= the total inflow
into the sink).
Simplex method:
Step 0 [Initialization] Present a given LP problem in standard form and set up initial tableau. Step Augmenting Path (Ford-Fulkerson) Method
1 [Optimality test] If all entries in the objective row are nonnegative — stop: the tableau Start with the zero flow (xij = 0 for every edge)
represents an optimal solution.  On each iteration, try to find a flow-augmenting path from source to sink, which a
Step 2 [Find entering variable] Select (the most) negative entry in the objective row. Mark its path along which some additional flow can be sent
column to indicate the entering variable and the pivot column.  If a flow-augmenting path is found, adjust the flow along the edges of this path to get
Step 3 [Find departing variable] For each positive entry in the pivot column, calculate the θ-ratio a flow of increased value and try again
by dividing that row's entry in the rightmost column by its entry in the pivot column. (If there are  If no flow-augmenting path is found, the current flow is maximum
no positive entries in the pivot column — stop: the problem is unbounded.) Find the row with the
smallest θ-ratio, mark this row to indicate the departing variable and the pivot row.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Definition of a Cut:
Finding a flow-augmenting path Let X be a set of vertices in a network that includes its source but does not include its sink, and let
X, the complement of X, be the rest of the vertices including the sink. The cut induced by this
To find a flow-augmenting path for a flow x, consider paths from source to sink in the underlying partition of the vertices is the set of all the edges with a tail in X and a head in X.
undirected graph in which any two consecutive vertices i,j are either: Capacity of a cut is defined as the sum of capacities of the edges that compose the cut.
 connected by a directed edge (i to j) with some positive unused capacity rij We’ll denote a cut and its capacity by C(X,X) and c(X,X)
= uij – xij Note that if all the edges of a cut were deleted from the network, there would be no directed path
– known as forward edge ( → ) from source to sink
OR Minimum cut is a cut of the smallest capacity in a given network
 connected by a directed edge (j to i) with positive flow xji
– known as backward edge ( ← ) Max-Flow Min-Cut Theorem
If a flow-augmenting path is found, the current flow can be increased by r units by increasing xij The value of maximum flow in a network is equal to the capacity of its minimum cut.
by r on each forward edge and decreasing xji by r on each backward edge, where The shortest augmenting path algorithm yields both a maximum flow and a minimum cut:
r = min {rij on all forward edges, xji on all backward edges} o maximum flow is the final flow produced by the algorithm

P
 Assuming the edge capacities are integers, r is a positive integer o minimum cut is formed by all the edges from the labeled vertices to unlabeled vertices
 On each iteration, the flow value increases by at least 1 on the last iteration of the algorithm all the edges from the labeled to unlabeled vertices are full, i.e.,
their flow amounts are equal to the edge capacities, while all the edges from the unlabeled to labeled
 Maximum value is bounded by the sum of the capacities of the edges leaving the source;

AP

AP
vertices, if any, have zero flow amounts on them.
hence the augmenting-path method has to stop after a finite number of iterations
 The final flow is always maximum, its value doesn’t depend on a sequence of Time Efficiency
augmenting paths used The number of augmenting paths needed by the shortest-augmenting-path algorithm never
 The augmenting-path method doesn’t prescribe a specific way for generating flow- exceeds nm/2, where n and m are the number of vertices and edges, respectively
augmenting paths Since the time required to find shortest augmenting path by breadth-first search is in
R

R
 Selecting a bad sequence of augmenting paths could impact the method’s Efficiency. O(n+m)=O(m) for networks represented by their adjacency lists, the time efficiency of the shortest-
augmenting-path algorithm is in O(nm2) for this representation
CO

CO
Shortest-Augmenting-Path Algorithm More efficient algorithms have been found that can run in close to O(nm) time, but these
Generate augmenting path with the least number of edges by BFS as follows. algorithms don’t fall into the iterative-improvement paradigm.
Starting at the source, perform BFS traversal by marking new (unlabeled) vertices with two
4.3 MAXIMUM MATCHING IN BIPARTITE GRAPHS
labels:
first label – indicates the amount of additional flow that can be brought from the source to the vertex Bipartite Graphs
being labeled
U

U
Bipartite graph: a graph whose vertices can be partitioned into two disjoint sets V and U, not
second label – indicates the vertex from which the vertex being labeled was reached, with “+” or necessarily of the same size, so that every edge connects a vertex in V to a vertex in U.
“–” added to the second label to indicate whether the vertex was reached via a forward or A graph is bipartite if and only if it does not have a cycle of an odd length
ST

ST
backward edge A bipartite graph is 2-colorable: the vertices can be colored in two colors so that every edge has
The source is always labeled with ∞,- its vertices colored differently
All other vertices are labeled as follows: Matching in a Graph
If unlabeled vertex j is connected to the front vertex i of the traversal queue by a directed A matching in a graph is a subset of its edges with the property that no two edges share a vertex a
edge from i to j with positive unused capacity rij = uij –xij (forward edge), vertex j is labeled with matching in this graph.
lj,i+, where lj = min{li, rij}
If unlabeled vertex j is connected to the front vertex i of the traversal queue by a directed A maximum (or maximum cardinality) matching is a matching with the largest number of edges
edge from j to i with positive flow xji (backward edge), vertex j is labeled lj,i-, where lj = min{li, xji} 1) always exists
If the sink ends up being labeled, the current flow can be augmented by the amount 2) not always unique
indicated by the sink’s first label Free Vertices and Maximum Matching
The augmentation of the current flow is performed along the augmenting path traced by following the A matching in this graph (M)
vertex second labels from sink to source; the current flow quantities are increased on the forward For a given matching M, a vertex is called free (or unmatched) if it is not an endpoint of any
edges and decreased on the backward edges of this path edge in M; otherwise, a vertex is said to be matched
If the sink remains unlabeled after the traversal queue becomes empty, the algorithm returns the If every vertex is matched, then M is a maximum matching
current flow as maximum and stops If there are unmatched or free vertices, then M may be able to be improved
We can immediately increase a matching by adding an edge connecting two free vertices

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

An instance of the stable marriage problem can be specified either by two sets of preference lists
Augmenting Paths and Augmentation or by a ranking matrix, as in the example below.
An augmenting path for a matching M is a path from a free vertex in V to a free vertex in U
whose edges alternate between edges not in M and edges in M Step 0 Start with all the men and women being free
The length of an augmenting path is always odd
Adding to M the odd numbered path edges and deleting from it the even numbered path edges Step 1 While there are free men, arbitrarily select one of them and do the following:
increases the matching size by 1 (augmentation) One-edge path between two free vertices is special Proposal The selected free man m proposes to w, the next woman on his preference list
case of augmenting path Response if w is free, she accepts the proposal to be matched with m. If she is not free,
Matching on the right is maximum (perfect matching)
she compares m with her current mate. If she prefers m to him, she accepts m’s proposal, making
Theorem A matching M is maximum if and only if there exists no augmenting path with respect to M her former mate free; otherwise, she simply rejects m’s proposal, leaving m free
Step 2 Return the set of n matched pairs
Augmenting Path Method (template)
Start with some initial matching. e.g., the empty set The algorithm terminates after no more than n2 iterations with a stable marriage output
Find an augmenting path and augment the current matching along that path o e.g., using breadth-first The stable matching produced by the algorithm is always man-optimal: each man gets the highest
search like method rank woman on his list under any stable marriage. One can obtain the woman-optimal matching by

P
When no augmenting path can be found, terminate and return the last matching, which is maximum? making women propose to men
A man (woman) optimal matching is unique for a given set of participant preferences

AP

AP
BFS-based Augmenting Path Algorithm The stable marriage problem has practical applications such as Matching medical-school graduates
Initialize queue Q with all free vertices in one of the sets (say V) with hospitals for residency training.
While Q is not empty, delete front vertex w and label every unlabeled vertex u adjacent to w as
follows:
Case 1 (w is in V): If u is free, augment the matching along the path ending at u by moving
R

R
backwards until a free vertex in V is reached. After that, erase all labels and reinitialize Q with all the
vertices in V that are still free
If u is matched (not with w), label u with w and enqueue u
CO

CO
Case 2 (w is in U) Label its matching mate v with w and enqueue v
After Q becomes empty, return the last matching, which is maximum
Each vertex is labeled with the vertex it was reached from. Queue deletions are indicated by
arrows. The free vertex found in U is shaded and labeled for clarity; the new matching obtained
by the augmentation is shown on the next slide.
U

U
This matching is maximum since there are no remaining free vertices in V (the queue is
empty)
Note that this matching differs from the maximum matching found earlier.
ST

ST
4.4 THE STABLE MARRIAGE PROBLEM

Stable Marriage Problem

There is a set Y = {m1,…,mn} of n men and a set X = {w1,…,wn} of n women. Each man has a
ranking list of the women, and each woman has a ranking list of the men (with no ties in these
lists).
A marriage matching M is a set of n pairs (mi, wj).
A pair (m, w) is said to be a blocking pair for matching M if man m and woman w are not
matched in M but prefer each other to their mates in M.

A marriage matching M is called stable if there is no blocking pair for it; otherwise, it’s called
unstable.
The stable marriage problem is to find a stable marriage matching for men’s and women’s given
preferences.

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Many important algorithms, especially those for sorting and searching, work by
UNIT V comparing items of their inputs.We can study the performance of such algorithms with a device
called a decision tree. As an example, a decision tree of an algorithm for finding a minimum of
COPING WITH THE LIMITATIONS OF ALGORITHM POWER three numbers. Each internal node of a binary decision tree represents a key comparison
indicated in the node, e.g., k < k . The node’s left subtree contains the information about
Limitations of Algorithm Power-Lower-Bound Arguments-Decision Trees-P, NP and NP-
Complete Problems--Coping with the Limitations - Backtracking – n-Queens problem – subsequent comparisons made if k < k , and its right subtree does the same for the case of k > k.
Hamiltonian Circuit Problem – Subset Sum Problem-Branch and Bound – Assignment problem – (For the sake of implicity, we assume throughout this section that all input items are distinct.)
Knapsack Problem – Travelling Salesman Problem- Approximation Algorithms for NP – Hard Each leaf represents a possible outcome of the algorithm’s run on some input of size n. Note that
Problems – Travelling Salesman problem – Knapsack problem. the number of leaves can be greater than the number of outcomes because, for some algorithms,
the same outcome can be arrived at through a different chain of comparisons. An important point
is that the number of leaves must be at least as large as the number of possible outcomes.The
5.1 LIMITATIONS OF ALGORITHM POWER algorithm’s work on a particular input of size n can be traced by a path from the root to a leaf in
its decision tree, and the number of comparisons made by the algorithm on such a run is equal to
A fair assessment of algorithms as problem-solving tools is inescapable: they are very
the length of this path. Hence, the number of comparisons in the worst case is equal to the height
powerful instruments, especially when they are executed by modern computers. But the power

P
of algorithms is not unlimited, and its limits are the subject of this chapter. As we shall see, of the algorithm’s decision tree.
some problems cannot be solved by any algorithm. Other problems can be solved The central idea behind this model lies in the observation that a tree with a given number

AP

AP
algorithmically but not in polynomial time. And even when a problem can be solved in of leaves, which is dictated by the number of possible outcomes, has to be tall enough to have
polynomial time by some algorithms, there are usually lower bounds on their efficiency. that many leaves. Specifically, it is not difficult to prove that for any binary tree with l leaves and
height h, h ≥ _log2 l_) Indeed, a binary tree of height h with the largest number of leaves has all
5.1.1 LOWER-BOUND ARGUMENTS its leaves on the last level (why?). Hence, the largest number of leaves in such a tree is 2h. In
other words, 2h ≥ l, which immediately implies. Inequality puts a lower bound on the heights of
Two ways to determine the efficiency of an algorithm is to know the asymptotic binary decision trees and hence the worst-case number of comparisons made by any comparison-
R

R
efficiency class and the class of hierarchy. The alternative is to find how efficient a particular based algorithm for the problem in question. Such a bound is called the
algorithm is with respect to other algorithms for the same problem. When we want to ascertain Information theoretic lower bound.
the efficiency of an algorithm with respect to other algorithms for the same problem, it is desired
CO

CO
We illustrate this technique below on two important problems: sorting and searching in a sorted
to know the best possible efficiency of any algorithm solving the problem may have. Knowing array.
such a lower bound can give us the improvement to achieve an efficient algorithm. If such a
bound is tight, i.e., if we already know an algorithm in the same efficiency class as the lower Decision Trees for Searching a Sorted Array
bound, we can hope for a constant-factor improvement at best.
We shall see how decision trees can be used for establishing lower bounds on the number
U

U
Trivial Lower Bounds is used to yield the bound best option is to count the number of of key comparisons in searching a sorted array of n keys: A[0]< A[1]< . . .< A[n − 1]. The
items in the problem’s input that must be processed and the number of output items that need principal algorithm for this problem is binary search. The number of comparisons made by
to be produced. binary search in the worst case, Cbsworst(n), is given by the formula Cbsworst(n) = _log2 n_ +
ST

ST
Information theoretic Arguments seeks to establish a lower bound based on the amount of 1= _log2(n + 1)_.
information it has to produce. It is called so because of its connection to information theory. The
decision trees are the mechanism used to solve the problem. Since we are dealing here with three-way comparisons in which search keyK is compared
with some element A[i] to see whetherK < A[i],K = A[i], orK > A[i], it is natural to try using
Adversary arguments establish the lower bounds. Problem reduction is used based on the other ternary decision trees. Figure 11.4 presents such a tree for the case of n = 4.The internal nodes of
problem where an efficient algorithm is solved. that tree indicate the array’s elements being compared with the search key. The leaves indicate
either a matching element in the case of a successful search or a found interval that the search
5.1.2 DECISION TREES key belongs to in the case of an unsuccessful search.
For an array of n elements, all such decision trees will have 2n + 1 leaves (n for
The number of leaves must be at least as large as the number of possible outcomes. The successful searches and n + 1 for unsuccessful ones). Since the minimum height h of a ternary
algorithm’s work on a particular input of size n can be traced by a path from the root to a leaf in tree with l leaves is _log3 l_, we get the following lower bound on the number of worst-case
comparisons:
its decision tree, and the number of comparisons made by the algorithm. On such a run is equal Cworst(n) ≥ _log3(2n + 1).
to the number of edges in this path. Hence, the number of comparisons in the worst case is equal
to the height of the algorithm’s decision tree. It is used for comparison-based sorting algorithm. This lower bound is smaller than _log2(n + 1)_, the number of worst-case comparisons
Decision trees are also used for searching a sorted array, which can be either a 2-node or 3-node for binary search, at least for large values of n (and smaller than or equal to _log2(n + 1)_ for
trees. every positive integer n—see Problem 7 in this section’s exercises). Can we prove a better lower
bound, or is binary search far from being optimal? The answer turns out to be the former. To

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

obtain a better lower bound, we should consider binary rather than ternary decision trees, such as NP-Complete Problems
the one in Figure 11.5. Internal nodes in such a tree correspond to the same threeway Informally, an NP-complete problem is a problem in NP that is as difficult as any other
comparisons as before, but they also serve as terminal nodes for successful searches. Leaves problem in this class because, by definition, any other problem in NP can be reduced to it in
therefore represent only unsuccessful searches, and there are n + 1 of them for searching an n- polynomial time Here are more formal definitions of these concepts.
element array.
decision problem D1 is said to be polynomially reducible to a decision problem D2, if there
As comparison of the decision trees in, the binary decision tree is simply the ternary exists a function t that transforms instances of D1 to instances of D2 such that:
decision tree with all the middle subtrees eliminated. Applying inequality to such binary decision 1. t maps all yes instances of D1 to yes instances of D2 and all no instances of D1 to
trees immediately yields Cworst(n) ≥ _log2(n + 1)). This inequality closes the gap between the no instances of D2
lower bound and the number of worstcase comparisons made by binary search, which is also 2. t is computable by a polynomial time algorithm This definition immediately
_log2(n + 1)_. A much more sophisticated analysis shows that under the standard assumptions implies that if a problem D1 is polynomially reducible to some problemD2 that can be solved in
about searches, binary search makes the smallest number of comparisons on the average, as well. polynomial time, then problem D1 can also be solved in polynomial time .
The average number of comparisons made by this algorithm turns out to be about log2 n − 1 and
log2(n + 1) for successful and unsuccessful searches, respectively. A decision problem D is said to be NP-complete if:
1. it belongs to class NP
5.1.3 P, NP, AND NP-COMPLETE PROBLEMS 2. every problem in NP is polynomially reducible to D

P
Tractable problems are that can be solved in polynomial time and nontractable are those A nondeterministic algorithm is a two-stage procedure that takes as its input an instance I
of a decision problem and does the following. Nondeterministic (“guessing”) stage: An arbitrary

AP

AP
which cannot be solved in polynomial time. Decision problems are also a p problems where a
string S is generated that can be thought of as a candidate solution to the given instance I.
decision can be taken.
A decision problem that can be solved in polynomial time is called polynomial. e.g., m- Class NP is the class of decision problems that can be solved by nondeterministic
coloring problem. Undesired problems cannot be solved by any algorithm. Halting problem will polynomial algorithms. This class of problems is called nondeterministic polynomial. Most
halt on that on that input or continue working indefinitely on it. Nondeterministic polynomial is decision problems are in NP. First of all, this class includes all the problems in P: P ⊆ NP.
the class of decision problems that can be solved by nondeterministic algorithms.
R

R
P≤NP
5.2 COPING WITH THE LIMITATIONS OF ALGORITHM POWER
Non-deterministic consists of 2 stages. First is the guessing stage, an arbitrary string S is
generated that can be thought of as a candidate solution to the given instance I. Second stage is
CO

CO
verification stage, a deterministic algorithm takes both I & S as its input and output yes if S Backtracking & Branch-and-Bound are the two algorithm design techniques for solving
represents a solution to instance I. NP-complete problem is a problem in NP that is as difficult as problems in which the number of choices grows at least exponentially with their instance size.
any other problem in this class because, by definition, any other problem in NP can be reduced to Both techniques construct a solution one component at a time trying to terminate the process as
it in polynomial time. soon as one can ascertain that no solution can be obtained as a result of the choices already
A decision problem D 1 is said to be polynomially reducible to a decision problem D2 if made. This approach makes it possible to solve many large instances of NP-hard problems in an
U

U
there exists a function t that transforms instances of D1 to instances of D2 such that acceptable amount of time.
1) t maps all yes instances of D1 to yes instances of D2 and all no instances of D1 to no
instances of D2; Both Backtracking and branch-and-bound uses a state-space tree-a rooted tree whose nodes
ST

ST
2) t is computed by a polynomial time algorithm. represent partially constructed solutions to the problem. Both techniques terminate a node as
A decision problem is said to be NP-complete if soon as it can be guaranteed that no solution to the problem can be obtained by considering
1) it belongs to class NP; choices that correspond to the node’s descendants.
2) Every problem in n NP is polynomially reducible to D.
CNF-satisfiability is NP-complete. It deals with Boolean expressions. eg: (x1 v x2 v x3) & (x1 v 5.3 BACKTRACKING
x2)
Class P is a class of decision problems that can be solved in polynomial time by (deterministic) Backtracking constructs its state-space tree in the depth-first search fashion in the
algorithms. This class of problems is called polynomial. majority of its applications. If the sequence of choices represented by a current node of state-
It is natural to wonder whether every decision problem can be solved in polynomial time. space tree can be developed further without violating the problem’s constraints, it is done by
The answer to this question turns out to be no. In fact, some decision problems cannot be solved considering the first remaining legitimate option for the next component. Otherwise, the method
at all by any algorithm. Such problems are called undecidable, as opposed to decidable problems backtracks by undoing the last component of the partially built solution and replaces it by the
that can be solved by an algorithm. A famous example of an undecidable problem was given by next alternative.
Alan Turing in 1936. The problem in question is called the halting problem: given a computer
program and an input to it, determine whether the program will halt on that input or continue
A node in a state-space tree is said to be promising if it corresponds to a partially
working indefinitely on it.
constructed solution that may still lead to a complete solution; otherwise it is called non

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

promising. Leaves represent either nonpromising dead ends or complete solutions found by the else
algorithms. for each element xЄsi+1 consistent with x[1…i] and the constraints do x[i+1] x Backtrack
(x[1…i+1])
5.3.1 N-QUEENS PROBLEM
In conclusion, the backtracking first is typically applied to difficult combinatorial
The problem is to place n Queens on an nXn chessboard so that no two queens attack each other problems for which no efficient algorithms for finding exact solutions possibly exist. Second,
by being in the same row or in the same column or on the same diagonal. unlike the exhaustive search approach, which is extremely slow, backtracking atleast for some
problem it gives in a acceptable amount of time, which is usually in the cases of optimization
Place n queens on an n-by-n chess board so that no two of them are in the same row, problems. Third, in backtracking doesn’t eliminate any elements of a problem’s state space and
column, or diagonal ends up generating all its elements, it provides a specific technique for doing so, which can be of
value in its own right.
1 2 3 4
1 queen 1 5.4 BRANCH-AND-BOUND
2 queen 2

P
It is an algorithm design technique that enhances the idea of generating a state-space tree
3 queen 3 with the idea of estimating the best value obtainable from a current node of the decision tree: if

AP

AP
such an estimate is not superior to the best solution seen up to that point in the processing, the
4 queen 4 node is eliminated from further consideration.

A feasible solution is a point in the problem’s search space that satisfies all the problem’s
5.3.2 HAMILTONIAN CIRCUIT PROBLEM constraints, while an optimal solution is a feasiblesolution with the best value of the objective
function compared to backtracking branch-and-bound requires 2 additional items:
R

R
Assume that if a Hamiltonian circuit exists, it starts at vertex a. Then vertex a will be the root of
the state-space tree, and, then from vertex a traverse thru all vertices without forming a cycle. 1) A way to provide, for every node of a state-space tree a bound on the best value of the
CO

CO
objective function on any solution that can be obtained by adding further components to the
5.3.3 SUBSET-SUM PROBLEM partial solution represented by the node.
2) The best value of the best solution seen so far.
Find a subset of a given set s={s1,………sn} of n positive integers whose sum is
equal to a given positive integer d. For ex, s={1,2,5,6,8} and d=9, there are 2 solutions, {1,2,6} If this information is available, we can compare a node’s bound with the value of
U

U
& {1,8 }. It is convenient to sort the elements in increasing order: the best solution seen so far: if the bound value is not better than the best solution seen so far-
i.e., not smaller for a minimization problem and not larger for a maximization problem- the node
s1≤ s2≤ s3…………sn is nonpromising and can be terminated because no solution obtained from it can yield a better
ST

ST
solution than the one already available.
We record the value of s′, the sum of these numbers in the node. If s′ is equal to d, we have a
solution to the problem. We can either report this result and stop or, if all the solutions need to be In general, we terminate a search path at the current node in a state-space tree of a branch &
found, continue by backtracking to the node’s parent. If s′ is not equal to d, we can terminate the bound algorithm for any one of the following three reasons:
node as nonpromising if either of the two inequalities holds:
s′+si+1>d (the sum s′ is too large) 1) The value of the node’s bound is not better than the value of the best solution seen so
far.
2) The node represents no feasible solutions because the constraints of the problem are
n
already violated.
s′+∑ sj<d (the sum s′ is too small)
j=j+1
3) The subset of feasible solutions represented by the node consists of a simple point.
Compare the value of the objective function for this feasible solution with that of the best
It is generally used as ‘n’ tuples to generate the tree as x1, x2,x3…………xn
solution seen so far and update the latter with the former if the new solution is better.
Algorithm Backtrack ( x[1…i])
//Output: All the tuples representing the problem’s solutions if x[1…i] is a solution write x[1…i]

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

5.4.1 ASSIGNMENT PROBLEM

5.4.3 TRAVELING SALESPERSONS PROBLEM
It is a problem where each job will be assigned to each person. And no 2 jobs can be assigned to
same person and no 2 person should be assigned with the same job. Visit all the vertices with a low cost yields the optimal solution. Weighted graph

Select one element in each row of the cost matrix C so that: State-Space tree with the list of vertices in a node specifies a beginning part of the Hamiltonian
circuits represented by the node
 no two selected elements are in the same column
 the sum is minimized
Example The lower bound is obtained as lb = s/2 ; where s is the sum of the distance of the n cities.
Job 1 Job 2 Job 3 Job 4
5.5 APPROXIMATION ALGORITHMS FOR NP-HARD PROBLEMS
Person a 9 2 7 8
Person b 6 4 3 7
It is the combinatorial problems which fall under NP-Hard problems. Accuracy ratio and
Person c 5 8 1 8
performance ratio has to be calculated. Nearest-neighbour algorithm and Twice-around-the-tree
Person d 7 6 9 4

P
algorithm. Greedy algorithm is used for the continuous knapsack problem for the fractional
version.
Lower bound: Any solution to this problem will have total cost

AP

AP
at least: 2 + 3 + 1 + 4 (or 5 + 2 + 1 + 4) Approximation algorithms are often used to find approximation solutions to difficult problems of
combinatorial optimization. The performance ratio is the principal metric for measuring the
5.4.2 KNAPSACK PROBLEM accuracy of such approximation algorithms.

Given a items of known weights Wi and values Vi, i=1,2,........n and a knapsack of
Apply a fast (i.e., a polynomial-time) approximation algorithm to get a solution that is not
R

R
capacity W, find the most valuable subset of the items that items that fit in the knapsack. necessarily optimal but hopefully close to it
Let us now discuss how we can apply the branch-and-bound technique to solving the
knapsack problem. This problem was introduced in Section 3.4: given n items of known weights
CO

CO
wi and values vi , i = 1, 2, . . . , n, and a knapsack of capacity W, find the most valuable subset of Accuracy measures:
the items that fit in the knapsack. It is convenient to order the items of a given instance in accuracy ratio of an approximate solution sa
descending order by their value-to-weight ratios. Then the first item gives the best payoff per r(sa) = f(sa) / f(s*) for minimization problems r(sa) = f(s*) / f(sa) for maximization problems
weight unit and the last one gives the worst payoff per weight unit, with ties resolved arbitrarily:
v1/w1 ≥ v2/w2 ≥ . . . ≥ vn/wn. where f(sa) and f(s*) are values of the objective function f for the approximate solution sa and
U

U
It is natural to structure the state-space tree for this problem as a binary tree constructed actual optimal solution s*,performance ratio of the algorithm A the lowest upper bound of r(sa)
as follows (see Figure 12.8 for an example). Each node on the ith level of this tree, 0 ≤ i ≤ n,
on all instances.
represents all the subsets of n items that include a particular selection made from the first i
ST

ST
ordered items. This particular selection is uniquely determined by the path from the root to the
node: a branch going to the left indicates the inclusion of the next item, and a branch going to the 5.5.1 APPROXIMATION ALGORITHMS FOR THE TRAVELING SALESMAN
right indicates its exclusion. We record the total weight w and the total value v of this selection in PROBLEM
the node, along with some upper bound ub on the value of any subset that can be obtained by
adding zero or more items to this selection. A simple way to compute the upper bound ub is to The nearest-neighbor is a greedy method for approximating a solution to the traveling salesman
add to v, the total value of the items already selected, the product of the remaining capacity of problem. The performance ratio is unbounded above, even for the important subset of Euclidean
the knapsack W − w and the best per unit payoff among the remaining items, which is vi+1/wi+1: graphs.
ub = v + (W − w)(vi+1/wi+1).
As a specific example, let us apply the branch-and-bound algorithm to the instance of the
Nearest-neighbor algorithm
knapsack problem (We reorder the items in descending order of their value-to-weight ratios,
though.)
Starting at some city, always go to the nearest unvisited city, and, after visiting all the cities,
item weight value v/w return to the starting one.
Note: Nearest-neighbor tour may depend on the starting city
1 4 $40 10
Accuracy: RA = ∞ (unbounded above) – make the length of AD arbitrarily large in the above
2 7 $42 6
example
3 15 $20 5
4 3 $12 4

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP

DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS DOWNLOADED FROM STUCOR APP CS8451/DESIGN AND ANALYSIS OF ALGORITHMS

Multifragment-heuristic algorithm
Step 1 Sort the edges in increasing order of their weights. (Ties can be broken arbitrarily.) Example: The knapsack’s capacity is
Initialize the set of tour edges to be constructed to the empty set. 16
Step 2 Repeat this step n times, where n is the number of cities in the instance being solved: add item weight value v/w
the next edge on the sorted edge list to the set of tour edges, provided this addition does not
1 2 $40 20
create a vertex of degree 3 or a cycle of length less than n; otherwise, skip the edge.
Step 3 Return the set of tour edges. 2 5 $30 6
3 10 $50 5
There is, however, a very important subset of instances, called Euclidean, for which we can 4 5 $10 2
make a nontrivial assertion about the accuracy of both the nearestneighbor and multifragment-
heuristic algorithms. These are the instances in which intercity distances satisfy the following Accuracy
natural conditions: RA is unbounded (e.g., n = 2, C = m, w1=1, v1=2, w2=m, v2=m)
 triangle inequality d[i, j ]≤ d[i, k]+ d[k, j] for any triple of cities i, j, and k (the yields exact solutions for the continuous version
distance between cities i and j cannot exceed the length of a two-leg path from i to some
intermediate city k to j ) Approximation Scheme for Knapsack Problem


P
 symmetry d[i, j ]= d[j, i] for any pair of cities i and j (the distance from ito j is the Step 1: Order the items in decreasing order of relative values: v1/w1≥… ≥ vn/wn
same as the distance from j to i) Step 2: For a given integer parameter k, 0 ≤ k ≤ n, generate all subsets of k items or less and for

AP

AP
Minimum-Spanning-Tree–Based Algorithms There are approximation algorithms for the each of those knapsack, add the remaining items in that fit the decreasing.
traveling salesman problem that exploit a connection between Hamiltonian circuits and spanning Step 3: Find the most valuable subset among the subsets generated in Step 2 and return it as the
trees of the same graph. Since removing an edge from a Hamiltonian circuit yields a spanning algorithm’s output
tree, we can expect that the structure of a minimum spanning tree provides a good basis for
constructing a shortest tour approximation. Here is an algorithm that implements this idea in a
 Accuracy: f(s*) / f(sa) ≤ 1 + 1/k for any instance of size n
rather straightforward fashion.
 Time efficiency: O(knk+ 1)
R

R
Twice-around-the-tree algorithm  There are fully polynomial schemes: algorithms with polynomial running time as
functions of both n and k
CO

CO
Twice-around-the-tree is an approximation algorithm for TSP with the performance ratio of 2 for
Euclidean graph. The algorithm is based on modifying a walk around a MST by shortcuts. Polynomial approximation schemes for the knapsack problem are polynomial time parametric
algorithms that approximation solutions with any predefined accuracy level.
Stage 1: Construct a minimum spanning tree of the graph
Stage 2: Starting at an arbitrary vertex, create a path that twice around the tree and returns to the
Greedy algorithm for the discrete knapsack problem
same vertex
U

U
Stage 3: Create a tour from the circuit constructed in Stage 2 by making shortcuts to avoid Step 1 Compute the value-to-weight ratios ri= vi/wi, i = 1, . . . , n, for the items given.
visiting intermediate vertices more than once Step 2 Sort the items in nonincreasing order of the ratios computed in Step 1. (Ties can be
broken arbitrarily.)
ST

ST
Note: RA = ∞ for general instances, but this algorithm tends to produce better tours than the
Step 3 Repeat the following operation until no item is left in the sorted list: if the current item on
nearest-neighbor algorithm
the list fits into the knapsack, place it in the knapsack and proceed to the next item; otherwise,
just proceed to the next item.
5.5.2 APPROXIMATION ALGORITHMS FOR THE KNAPSACK PROBLEM
Greedy algorithm for the continuous knapsack problem
A sensible greedy algorithm for the knapsack problem is based on processing an input’s items in
descending order of their value-to-weight ratios. For continuous version, the algorithm always Step 1 Compute the value-to-weight ratios vi/wi, i = 1, . . . , n, for the items given.
yields an exact optimal solution. Step 2 Sort the items in nonincreasing order of the ratios computed in Step 1. (Ties can be
broken arbitrarily.)
Greedy Algorithm for Knapsack Problem Step 3 Repeat the following operation until the knapsack is filled to its full capacity or no item is
left in the sorted list: if the current item on the list fits into the knapsack in its entirety, take it and
Step 1: Order the items in decreasing order of relative values: v1/w1≥… ≥ vn/wn proceed to the next item; otherwise, take its largest fraction to fill the knapsack to its full
capacity and stop.
Step 2: Select the items in this order skipping those that don’t fit into the knapsack

DOWNLOADED FROM STUCOR APP DOWNLOADED FROM STUCOR APP