HAWASSA UNIVERSITY

INSTITUTE OF TECHNOLOGY
DEPARTMENT OF COMPUTER SCIENCE

DAA: Design and Analysis of Algorithms [CoSc3042]

Handout

Compiled by Gezahegn Gaje.

10/15/2024
Chapter I-Introduction to DAA 2024

0.1. Review of elementary Data Structures

Data structures

A data structure is a way to store and organize data in order to facilitate access and modifications. No
single data structure works well for all purposes, and so it is important to know the strengths and
limitations of several of them.

Recalling data structures from Data structure class!!

Arrays, Sorting algorithms, Linked List, Stack, Queue, trees, graphs

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-1

Chapter I-Introduction to DAA 2024

Chapter I

1. Introduction to Design and Analysis of Algorithm

1.1. What is algorithm?

Informally, an algorithm is any well-defined computational procedure that takes some value, or set of
values, as input and produces some value, or set of values, as output. An algorithm is thus a sequence
of computational steps that transform the input into the output. We can also view an algorithm as a
tool for solving a well-specified computational problem. The statement of the problem specifies in
general terms the desired input/output relationship. The algorithm describes a specific computational
procedure for achieving that input/output relationship.

An Algorithm is a finite sequence of instructions, each of which has a clear meaning and can be
performed with a finite amount of effort in a finite length of time. No matter what the input values
may be, an algorithm terminates after executing a finite number of instructions. An algorithm can be
specified in English, as a computer program, or even as a hardware design. The only requirement is
that the specification must provide a precise description of the computational procedure to be
followed. In addition every algorithm must satisfy the following criteria:

1. Input: there are zero or more quantities, which are externally supplied;
2. Output: at least one quantity is produced;
3. Definiteness: each instruction must be clear and unambiguous;
4. Finiteness: if we trace out the instructions of an algorithm, then for all cases the algorithm
will terminate after a finite number of steps;
5. Effectiveness: every instruction must be sufficiently basic that it can in principle be carried
out by a person using only pencil and paper. It is not enough that each operation be definite,
but it must also be feasible.

In formal computer science, one distinguishes between an algorithm, and a program. A program does
not necessarily satisfy the fourth condition. One important example of such a program for a computer
is its operating system, which never terminates (except for system crashes) but continues in a wait
loop until more jobs are entered. We represent algorithm using a pseudo language that is a
combination of the constructs of a programming language together with informal English statements.

1.2. Problems solved by algorithms

Practical applications of algorithms are ubiquitous and include the following examples:

 The Human Genome Project has made great progress toward the goals of identifying all the
100,000 genes in human DNA, determining the sequences of the 3 billion chemical base pairs

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-2

Chapter I-Introduction to DAA 2024

that make up human DNA, storing this information in databases, and developing tools for data
analysis. Each of these steps requires sophisticated algorithms.
 The Internet enables people all around the world to quickly access and retrieve large amounts
of information. With the aid of clever algorithms, sites on the Internet are able to manage and
manipulate this large volume of data. Examples of problems that make essential use of
algorithms include finding good routes on which the data will travel and using a search engine
to quickly find pages on which particular information resides.
 Electronic commerce enables goods and services to be negotiated and exchanged
electronically, and it depends on the privacy of personal information such as credit card
numbers, passwords, and bank statements. The core technologies used in electronic commerce
include public-key cryptography and digital signatures, which are based on numerical
algorithms and number theory.
 Manufacturing and other commercial enterprises often need to allocate scarce resources in the
most beneficial way. An oil company may wish to know where to place its wells in order to
maximize its expected profit. A political candidate may want to determine where to spend
money buying campaign advertising in order to maximize the chances of winning an election.
An airline may wish to assign crews to flights in the least expensive way possible, making
sure that each flight is covered and that government regulations regarding crew scheduling are
met. An Internet service provider may wish to determine where to place additional resources
in order to serve its customers more effectively. All of these are examples of problems that
can be solved using linear programming.

Algorithmic problems have two common characteristics: firstly they have many candidate solutions,
the overwhelming majority of which do not solve the problem at hand. Finding one that does, or one
that is “best,” can present quite a challenge. Second, they have practical applications. A transportation
firm, such as a trucking or railroad company, has a financial interest in finding shortest paths through
a road or rail network because taking shorter paths results in lower labor and fuel costs. Or a routing
node on the Internet may need to find the shortest path through the network in order to route a
message quickly.

Having a solid base of algorithmic knowledge and technique is one characteristic that separates the
truly skilled programmers from the novices. With modern computing technology, you can accomplish
some tasks without knowing much about algorithms, but with a good background in algorithms, you
can do much, much more.

1.3.Analyzing algorithms

Analyzing an algorithm has come to mean predicting the resources that the algorithm requires. Occasionally,
resources such as memory, communication bandwidth, or computer hardware are of primary concern, but most
often it is computational time that we want to measure. Generally, by analyzing several candidate algorithms
for a problem, we can identify a most efficient one. Such analysis may indicate more than one viable
candidate, but we can often discard several inferior algorithms in the process.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-3

Chapter I-Introduction to DAA 2024

1.3.1. Complexity of Algorithms

Time Complexity: The time needed by an algorithm expressed as a function of the size of a problem
is called the time complexity of the algorithm. The time complexity of a program is the amount of
computer time it needs to run to completion. The limiting behavior of the complexity as size increases
is called the asymptotic time complexity. It is the asymptotic complexity of an algorithm, which
ultimately determines the size of problems that can be solved by the algorithm.

Space Complexity: The space complexity of a program is the amount of memory it needs to run to
completion. The space need by a program has the following components:

1. Instruction space: Instruction space is the space needed to store the compiled version of the
program instructions.
2. Data space: Data space is the space needed to store all constant and variable values.
Data space has two components:

a. Space needed by constants and simple variables in program.

b. Space needed by dynamically allocated objects such as arrays and class instances.
3. Environment stack space: The environment stack is used to save information needed to
resume execution of partially completed functions.
4. Instruction Space: The amount of instructions space that is needed depends on factors such as:
a. The compiler used to complete the program into machine code.
b. The compiler options in effect at the time of compilation
c. The target computer.

The complexity of an algorithm M is the function f(n) which gives the running time and/or storage
space requirement of the algorithm in terms of the size “n” of the input data. Mostly, the storage
space required by an algorithm is simply a multiple of the data size “n‟. Complexity shall refer to the
running time of the algorithm. The function f(n), gives the running time of an algorithm, depends not
only on the size “n‟ of the input data but also on the particular data. The complexity function f(n)
for certain cases are:

1. Best Case: The minimum possible value of f(n) is called the best case.
2. Average Case: The expected value of f(n).
3. Worst Case: The maximum value of f(n) for any possible input.

The field of computer science, which studies efficiency of algorithms, is known as

analysis of algorithms. Algorithms can be evaluated by a variety of criteria. Most often we shall be
interested in the rate of growth of the time or space required to solve larger and larger
instances of a problem. We will associate with the problem an integer, called the size
of the problem, which is a measure of the quantity of input data.

1.4.Asymptotic Notations

The following notations are commonly use notations in performance analysis and used to
characterize the complexity of an algorithm:
a. Big–OH(O),
DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-4
Chapter I-Introduction to DAA 2024

b. Big–OMEGA(),
c. Big–THETA() and
a. Big–Oh O (Upper Bound): f(n) = O(g(n)), (pronounced order of or big oh), says that the growth
rate of f(n) is less than or equal that of g(n).

b. Big–Omega  (Lower Bound): f(n) = (g(n)) (pronounced omega), says that the growth rate
of f(n) is greater than or equal to that of g(n).

c. Big–Theta  (Same order): f(n) = (g(n)) (pronounced theta), says that the growth rate of f(n)
equals (=) the growth rate of g(n) [if f(n) = O(g(n)) and T(n) =  (g(n)].

Suppose “M‟ is an algorithm, and suppose “n‟ is the size of the input data. Clearly the complexity
f(n) of M increases as n increases. It is usually the rate of increase of f(n) we want to examine. This is
usually done by comparing f(n) with some standard functions. The most common computing times
n n
are: O(1), O(logn), O(n), O(nlogn), O(n2), O(n3), O(2 ), n! and n

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-5

Chapter I-Introduction to DAA 2024

Numerical Comparison of Different Algorithms

The execution time for six of the typical functions is given below:

O(log n) does not depend on the base of the logarithm. To simplify the analysis, the convention will
not have any particular units of time. Thus we throw away leading constants. We will also throw
away low–order terms while computing a Big–Oh running time. Since Big-Oh is an upper bound, the
answer provided is a guarantee that the program will terminate within a certain time period. The
program may stop earlier than this, but never later.

One way to compare the function f(n) with these standard function is to use the functional “O‟
notation, suppose f(n) and g(n) are functions defined on the positive integers with the property that
f(n) is bounded by some multiple g(n) for almost all “n‟. Then, f(n)=O(g(n)) which is read as “f(n) is
of order g(n)”. For example, the order of complexity for:

1. Linear search is O (n)

2. Binary search is O (logn)
3. Bubble sort is O (n2)
4. Merge sort is O (nlogn)

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-6

Chapter I-Introduction to DAA 2024

The rule of sums

Suppose that T1(n) and T2(n) are the running times of two programs fragments P1 and P2, and that
T1(n) is O(f(n)) and T2(n) is O(g(n)). Then T1(n) + T2(n), the running time of P1 followed by P2 is
O(max(f(n), g(n))), this is called as rule of sums. For example, suppose that we have three steps
whose running times are respectively O(n2), O(n3) and O(nlogn). Then the running time of the first
two steps executed sequentially is O(max(n2, n3)) which is O(n3). The running time of all three
together is O(max (n3, nlogn)) which is O(n3).

The rule of products

If T1(n) and T2(n) are O(f(n) and O(g(n)) respectively, then T1(n)*T2(n) is O(f(n)g(n)). It follows term
the product rule that O(cf(n)) means the same thing as O(f(n)) if “c” is any positive constant. For
example, O(n2/2) is same as O(n2).

The Running time of a program

When solving a problem we are faced with a choice among algorithms. The basis for this can be any
one of the following:
i. We would like an algorithm that is easy to understand, code and debug.
ii. We would like an algorithm that makes efficient use of the computer‟s resources, especially,
one that runs as fast as possible.

1.5. Measuring the running time of a program

The running time of a program depends on factors such as:

1. The input to the program.
2. The quality of code generated by the compiler used to create the object program.
3. The nature and speed of the instructions on the machine used to execute the program, and
4. The time complexity of the algorithm underlying the program.

The running time depends not on the exact input but only the size of the input. For many
programs, the running time is really a function of the particular input, and not just of the input
size. In that case we define T(n) to be the worst case running time, i.e. the maximum overall input
of size “n‟, of the running time on that input. We also consider Tavg(n) the average, over all input
of size “n‟ of the running time on that input. In practice, the average running time is often much
harder to determine than the worst-case running time. Thus, we will use worst–case running time
as the principal measure of time complexity.

Seeing the remarks (2) and (3) above, we cannot express the running time T(n) in standard time
units such as seconds. Rather we can only make remarks like the running time of such and such
algorithm is proportional to n2. The constant of proportionality will remain un-specified, since it
depends so heavily on the compiler, the machine and other factors.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-7

 Chapter I-Introduction to DAA 2024

Asymptotic Analysis of Algorithms: Our approach is based on the asymptotic complexity

measure. This means that we don‟t try to count the exact number of steps of a program, but how
that number grows with the size of the input to the program. That gives us a measure that will
work for different operating systems, compilers and CPUs. The asymptotic complexity is written
using big-O notation.

Rules for using big-O: The most important property is that big-O gives an upper bound only. If
an algorithm is O(n2), it doesn‟t have to take n2 steps (or a constant multiple of n2). But it can‟t
take more than n2.

1. Ignoring constant factors: O(c f(n))=O(f(n)), where c is a constant; e.g. O(20n3) = O(n3).
2. Ignoring smaller terms: If a<b then O(a+b) = O(b), for example O(n2+n)= O(n2)
3. Upper bound only: If a<b then an O(a) algorithm is also an O(b) algorithm. For example, an
O(n) algorithm is also an O(n2) algorithm (but not vice versa).
4. n and log n are "bigger" than any constant, from an asymptotic view (that means for large
enough n). So if k is a constant, an O(n+k) algorithm is also O(n), by ignoring smaller terms.
Similarly, an O(logn + k) algorithm is also O(logn).
5. Another consequence of the last item is that an O(nlogn+n) algorithm, which is O(n(logn+1)),
can be simplified to O(nlogn).

Calculating the running time of a program:

Let us now look into how big-O bounds can be computed for some common algorithms.

Example 1:

Let‟s consider a short piece of source code:

x = 3*y + 2;
z = z + 1; If y, z are scalars, this piece of code takes a constant amount of time, which we write as O(1).
In terms of actual computer instructions or clock ticks, it‟s difficult to say exactly how long it takes.
But whatever it is, it should be the same whenever this piece of code is executed. O(1) means some
constant, it might be 5, or 1 or 1000.

Example 2:

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-8

Chapter I-Introduction to DAA 2024

Example 2:
Analysis of simple for loop
Now let‟s consider a simple for loop: for (i = 1; i<=n; i++)
v[i] = v[i] + 1;

This loop will run exactly n times, and because the inside of the loop takes constant time, the total
running time is proportional to n. We write it as O(n). The actual number of instructions might be
50n, while the running time might be 17n microseconds. It might even be 17n+3 microseconds
because the loop needs some time to start up. The big-O notation allows a multiplication factor (like
17) as well as an additive factor (like 3). As long as it‟s a linear function which is proportional to n,
the correct notation is O(n) and the code is said to have linear running time.

Example 3
Analysis for nested for loop:
Now let‟s look at a more complicated example, a nested for loop:
for (i = 1; i<=n; i++)
for (j = 1; j<=n; j++)
a[i,j] = b[i,j] * x;

The outer for loop executes n times, while the inner loop executes n times for every execution of the
outer loop. That is, the inner loop executes nxn = n2 times. The assignment statement in the inner
loop takes constant time, so the running time of the code is O(n2) steps. This piece of code is said to
have quadratic running time.

1.6.General rules for the analysis of programs

In general the running time of a statement or group of statements may be parameterized by the input
size and/or by one or more variables. The only permissible parameter for the running time of the
whole program is “n‟ the input size.

1. The running time of each assignment read and write statement can usually be taken to be O(1).
2. The running time of a sequence of statements is determined by the sum rule. I.e. the running time
of the sequence is, to within a constant factor, the largest running time of any statement in the
sequence.
3. The running time of an if–statement is the cost of conditionally executed statements, plus the
time for evaluating the condition. The time to evaluate the condition is normally O(1) the time
for an if–then–else construct is the time to evaluate the condition plus the larger of the time
needed for the statements executed when the condition is true and the time for the statements
executed when the condition is false.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-9

Chapter I-Introduction to DAA 2024

4. The time to execute a loop is the time to execute the body and the time to evaluate the condition
for termination (usually the latter is O(1)). Often this time is, neglected constant factors, the
product of the number of times around the loop and the largest possible time for one execution of
the body, but we must consider each loop separately to make sure.

1.7. Algorithms Design and its goals

The three basic design goals that one should strive for in a program are: trying to save time, space and
face. A program that runs faster is a better program, so saving time is an obvious goal. Likewise, a
program that saves space over a competing program is considered desirable. We can choose from a
wide range of algorithm design techniques like incremental (insertion sort), Divide and Conquer
(Quick-sort), Greedy methods (0/1 knapsack), Dynamic programming (fractional Knapsack) , Brute
force, Backtracking, Brach and bounce etc.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 1-10

Chapter II- Divide and Conquer 2024

Chapter-II

2. Divide and Conquer (DAC)

2.1. Introduction to DAC

Divide and conquer is a design strategy which is well known to breaking down efficiency barriers. When
the method applies, it often leads to a large improvement in time complexity.

Divide and conquer strategy is as follows: divide the problem instance into two or more smaller instances
of the same problem, solve the smaller instances recursively, and assemble the solutions to form a solution
of the original instance. The recursion stops when an instance is reached which is too small to divide.
When dividing the instance, one can either use whatever division comes most easily to hand or invest time
in making the division carefully so that the assembly is simplified.

Divide and conquer algorithm consists of two parts:

Divide : Divide the problem into a number of sub problems. The sub problems are solved recursively.
Conquer : The solution to the original problem is then formed from the solutions to the sub problems
(patching together the answers).

Traditionally, routines in which the text contains at least two recursive calls are called divide and conquer
algorithms, while routines whose text contains only one recursive call are not. Divide–and–conquer is a
very powerful use of recursion.

2.2. Control Abstraction of Divide and Conquer

A control abstraction is a procedure whose flow of control is clear but whose primary operations are
specified by other procedures whose precise meanings are left undefined. The control abstraction for divide
and conquer technique is DAC(P), where P is the problem to be solved.

DAC (P)
{
if SMALL(P) then return S(p);
else
{
divide p into smaller instances p1, p2, …. Pk, k  1;
apply DAC to each of these sub problems;
return (COMBINE(DAC(p1) , DAC(p2),…., DAC(pk));
}
}

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 2-1

Chapter II- Divide and Conquer 2024

SMALL(P) is a Boolean valued function which determines whether the input size is small enough so
that the answer can be computed without splitting. If this is so, function S is invoked otherwise, the
problem p is divided into smaller sub problems. These sub problems p1, p2…, pk are solved by
recursive application of DAC. If the sizes of the two sub problems are approximately equal then the
computing time of DAC is:

 g(n) n small
T (n) = aT(n/b)f(n) otherwise
 


Where, T(n) is the time for DAC on n inputs g(n) is the time to complete the answer directly for small
inputs and f(n) is the time for divide and combine. a and b are constants. This is called the general
divide-and-conquer recurrence. Example for GENERAL METHOD: As an example, let us consider the
problem of computing the sum of n numbers a0... an-1. If n>1, we can divide the problem into two
instances of the same problem. They are sum of the first |n/2|numbers. Compute the sum of the 1st [n/2]
numbers, and then compute the sum of another n/2 numbers. Combine the answers of two n/2 numbers
sum. i.e., a0 + . . . + an-1 = (a0+....+an/2) +(an/2+...+an-1).

Assuming that size n is a power of b, to simplify our analysis, we get the following recurrence for the
running time T(n). T(n)=aT(n/b)+f(n). This is called the general divide and-conquer recurrence. f(n) is
a function that accounts for the time spent on dividing the problem into smaller ones and on combining
their solutions.

Advantages of DAC:
 The time spent on executing the problem using DAC is smaller than other method.
 This technique is ideally suited for parallel computation.
 This approach provides an efficient algorithm in computer science.
However, in DAC approach, most of the algorithms are designed using recursion; hence memory
management is very high. For recursive function, stack is used, where function state needs to be stored.

2.3. Recurrence

Recurrence Relation for a sequence of numbers S is a formula that relates all but a finite number of terms
of S to previous terms of the sequence, namely, {a0, a1, a2, .. . . . , an-1}, for all integers n with n ≥ n0, where
n0 is a nonnegative integer.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 2-2

Chapter II- Divide and Conquer 2024

Recurrence relations are also called as difference equations.

Sequences are often most easily defined with a recurrence relation; however the calculation of terms by
directly applying a recurrence relation can be time consuming. The process of determining a closed form
expression for the terms of a sequence from its recurrence relation is called solving the relation. Some
guess and check with respect to solving recurrence relation are as follows:
 Make simplifying assumptions about inputs
 Tabulate the first few values of the recurrence
 Look for patterns, guess a solution
 Generalize the result to remove the assumptions
Examples: Factorial, Fibonacci, Quick sort, Binary search etc.
Recurrence relation is an equation, which is defined in terms of itself. There is no single technique or
algorithm that can be used to solve all recurrence relations. In fact, some recurrence relations cannot be
solved. Most of the recurrence relations that we encounter are linear recurrence relations with constant
coefficients. Several techniques like substitution, induction, characteristic roots and generating function are
available to solve recurrence relations.

2.3.1. The Substitution Method

The substitution method for solving recurrences comprises two steps:

1. Guess the form of the solution.
2. Use mathematical induction to find the constants and show that the solution works.
We substitute the guessed solution for the function when applying the inductive hypothesis to smaller values; hence
the name “substitution method.” This method is powerful, but we must be able to guess the form of the answer in
order to apply it. We can use the substitution method to establish either upper or lower bounds on a recurrence.

One way to solve a divide-and-conquer recurrence equation is to use the iterative method. This is a “plug-
and-chug” method. In using this method, we assume that the problem size n is fairly large and we than
substitute the general form of the recurrence for each occurrence of the function T on the right-hand side.
For example, performing such a substitution with the merge sort recurrence equation yields the equation.

2.3.2. The Iteration method

The method of iterating the recurrence doesn‟t require us to guess the answer; it may require more algebra
than the substitution method. The idea is to iterate (expand) the recurrence and express it as a summation of
terms dependent only on n and the initial conditions. (Check the class lecture for examples).

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 2-3

Chapter II- Divide and Conquer 2024

2.3.3. The Master Method

The master method provides a “cook-book” method for solving recurrences of the form:
T(n) =aT(n/b)f(n),
where a>=1 and b>1 are constants and f(n) is an asymptotic positive function. The master
method requires the memoization of three cases which we are going to discuss down.

In all efficient divide and conquer algorithms we will divide the problem into sub-problems, each of which
is some part of the original problem, and then perform some additional work to compute the final answer.
As an example, if we consider merge sort, it operates on two problems, each of which is half the size of the
original, and then uses O(n) additional work for merging. This gives the running time equation:
The following theorem can be used to determine the running time of divide and conquer algorithms. For a
given program or algorithm, first we try to find the recurrence relation for the problem. If the recurrence is
of below form then we directly give the answer without fully solving it.
If the recurrence is of the form:

NB: To use the master method, we simply determine which case (if any) of the master theorem applies and
write down the answer.  Refer your lecture sessions!

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 2-4

Chapter II- Divide and Conquer 2024

Questions: Solve the following problems by the above master theorem.

a) T(n)=3T(n/2) +n2 g) T(n)=2T(n/2) +n/lgn

b) T(n)=4T(n/2) +n2 h) T(n)=(1/2)T(n/2) +1/n
c) T(n)=T(n/2) +n2 i) T(n)=6T(n/3) +n2lgn
d) T(n)=2nT(n/2) +nn j) T(n)=64T(n/8) -n2lgn
e) T(n)=16T(n/4) +n k) T(n)=9T(n/3)+n
f) T(n)=2T(n/2) +nlgn l) T(n)=T(2n/3)+1

2.4. Applications of Divide and conquer rule or algorithm:

 Binary search,
 Merge sort,
 Quick sort,
 Strassen’s matrix multiplication.

2.4.1. Binary Search

A binary search algorithm is a technique for finding a particular value in a sorted list. The binary
search consists of the following steps:

 Search a sorted array by repetitively dividing the search interval by half

 Begin with an interval covering the array
 If the value of search element is less than the middle element of the interval, narrow the
interval to the lower half. Otherwise narrow it to the upper half.
 Repetitively check until the value is found or the interval is empty.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 2-5

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

Figure 2-1 Trace of binary search

2.4.2. Merge sort

Merge sort is an O(nlogn) comparison based sorting algorithm. Sorting by merging is recursive, devide-
and-conquer strategy. In the base case we have a sequence with exactly one element in it. Since such a
sequence is already sorted, there is nothing to be done. The steps to solve the sequences are as follows:

 Divide the sequences into two sequences of length [n/2] and [n/2]
 Recursively sort each of subsequences
 Merge the sorted subsequences to obtain the final result.

Figure 2-2 Merge sort procedures

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 2-6

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

Figure 2-3. Bottom-up view of merge sort.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 2-7

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

CHAPTER III

3. GREEDY METHODS

3.1. Introduction

Greedy is the most straight forward design technique. Most of the problems have n inputs and require us to
obtain a subset that satisfies some constraints. Any subset that satisfies these constraints is called a
feasible solution. We need to find a feasible solution that either maximizes or minimizes the objective
function. A feasible solution that does this is called an optimal solution.

The greedy method is a simple strategy of progressively building up a solution, one element at a time, by
choosing the best possible element at each stage. At each stage, a decision is made regarding whether or not
a particular input is in an optimal solution. This is done by considering the inputs in an order determined by
some selection procedure. If the inclusion of the next input, into the partially constructed optimal solution
will result in an infeasible solution then this input is not added to the partial solution. The selection
procedure itself is based on some optimization measure. Several optimization measures are plausible for a
given problem. Most of them, however, will result in algorithms that generate sub-optimal solutions. This
version of greedy technique is called subset paradigm.

Some problems like Knapsack, Job sequencing with deadlines and minimum cost spanning trees are
based on subset paradigm. For the problems that make decisions by considering the inputs in some order,
each decision is made using an optimization criterion that can be computed using decisions already made.
This version of greedy method is ordering paradigm. Some problems like optimal storage on tapes,
optimal merge patterns and single source shortest path are based on ordering paradigm.
Description: Procedure Greedy describes
Greedy Algorithm in General :
the essential way that a greedy based
Algorithm Greedy (a, n)// a(1 : n) contains the n inputs algorithm will look, once a particular
{ problem is chosen and the functions select,
solution := NULL; // initialize the solution to empty feasible and union are properly
for i:=1 to n do implemented. The function select selects
{ x := select (a); an input from „a‟, removes it and assigns
if feasible (solution, x) then its value to „x‟. Feasible is a Boolean
solution := Union (Solution, x); valued function, which determines if „x‟
} can be included into the solution vector.
return solution; The function Union combines „x‟ with
} solution and updates the objective
function.
DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-1
DAA: Design and Analysis of Algorithms [CoSc3042] 2024

3.2. Fractional Knapsack Problem

Let us apply the greedy method to solve the knapsack problem. We are given n objects and a knapsack.
The object i has a weight wi and the knapsack has a capacity m. If a fraction xi, 0<xi<1 of object i is
placed into the knapsack then a profit of pixi is earned. The objective is to fill the knapsack that
maximizes the total profit earned. Since the knapsack capacity is m, we require the total weight of all
chosen objects to be at most m. The problem is stated as: maximize subject to

Where, 0 < xi < 1 and 1 < i < n

The profits and weights are positive numbers.

Algorithm
If the objects are already been sorted into non-increasing order of p[i]/w[i] then the algorithm given
below obtains solutions corresponding to this strategy.

Algorithm GreedyKnapsack (m, n)

{
for i := 1 to n do x[i] := 0.0 // initialize x
U := m;
 p[1:n] and w[1:n] contain the profits and weights respectively.
for i := 1 to n do
 Objects ordered so that p[i]/w[i]> p[i+1]/w[i+1].
{
 m is the knapsack size and x[1:n] is the solution vector.
if (w(i) > U) then break;
x [i] := 1.0; U := U – w[i];
}
if (i < n) then x[i] := U/w[i];
}
Running time:
The objects are to be sorted into non-decreasing order of pi/wi ratio. But if we disregard the time to
initially sort the objects, the algorithm requires only O(n) time. For unsorted list, it takes O(nlogn).

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-2

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

Example:
Consider the following instance of the knapsack problem: n = 3, m = 20, (p1, p2, p3) =(25, 24, 15)
and (w1, w2, w3) = (18, 15, 10).
1. First, we try to fill the knapsack by selecting the objects in some order:

2. Select the object with the maximum profit first (p = 25). So, x1 = 1 and profit earned is 25. Now,
only 2 units of space is left, select the object with next largest profit (p = 24). So, x2=2/15

3. Considering the objects in the order of non-decreasing weights wi.

4. Considered the objects in the order of the ratio pi/wi.

5. Sort the objects in order of the non-increasing order of the ratio pi/xi. Select the object with the
maximum pi/xi ratio, so, x2 = 1 and profit earned is 24. Now, only 5 units of space is left, select
the object with next largest pi/xi ratio, so x3 = ½ and the profit earned is 7.5.

This solution is the optimal solution.

Solve this:

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-3

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

3.3. Minimum Spanning Trees(MST)

The MST problem is to find a free tree T of a graph G that contains all the vertices of G and has the
minimum total weight of the edge of G over all such trees.

Problem formulation

Let G=(V, E, W) be a weighted connected undirected graph. Find a tree T that contains all the vertices in
G and minimize the sum of the weights of the edges (u, v) of T that is,

 Tree that contains every vertex of a connected graph is called a spanning tree. The problem
of constructing MST is computing spanning tree T with smallest total weight.
 A tree is a connected graph with no cycles. A spanning tree is a sub graph of G which has
the same set of vertices of G and is a tree.
 A minimum spanning tree of a weighted graph G is the spanning tree of G whose edges
sum to minimum weight.
 A graph may have many spanning trees, for instance the complete graph on four vertices.

The above graph has 16 spanning trees as shown below:

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-4

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

Let us discuss Kruskal‟s and Prim‟s algorithm which are classic applications of the greedy strategy.

3.3.1. Prim’s Algorithm

This algorithm repeatedly chooses the smallest-weight edge from the tree so far to the other vertices.

If a spanning tree has a weightier edge between Vt and v-VT, it can be improved by replacing it with e.

We can put Vt edges into a priority queue, and then dequeue edges until one goes b/n VT and V-VT.

Prim/s algorithm using a binary heap to implement a priority queue is O(ElogE)=O(ElogV)

Algorithm:

Example: Elicitation

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-5

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

3.3.2. Kruskal’s Algorithm

 This algorithm repeatedly chooses the smallest-weight edge that doesn‟t form a cycle.
 Kruskal‟s algorithm is O(ElogV) using efficient cycle detection.
 Disjoint set implementation for detecting cycles:
 put each vertex into a singleton set
 e is chosen if its vertices are different sets
 when e is chosen, the two sets are unioned

Algorithm:

Example: Elicitation

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-6

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

3.4.Single source shortest paths

3.5. Dijkstra’s Algorithm

Dijkstra‟s algorithm solves the single source shortest path when all edges have none negative weights. It is
a greedy algorithm and similar to prim‟s algorithm. Algorithm starts at the source vertex S it grows a tree T
that ultimately spans all vertices reachable from S. Vertices are added to T in order of distance i.e., first S,
then the vertex closest to S, then the next closest and so on. The algorithm is as given below.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-7

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

Example: Elicitation

(Check your lecture board)

3.5.1. Bellman-Ford Algorithm

The Bellman-Ford algorithm solves the single-source shortest-paths problem in the general case in which edge
weights may be negative. Given a weighted, directed graph G = (V, E) with source s and weight function w: E->R,
the Bellman-Ford algorithm returns a boolean value indicating whether or not there is a negative-weight cycle that is
reachable from the source. If there is such a cycle, the algorithm indicates that no solution exists. If there is no such
cycle, the algorithm produces the shortest paths and their weights.

The algorithm relaxes edges, progressively decreasing an estimate v and d on the weight of a shortest path from the
source s to each vertex v €V until it achieves the actual shortest-path weight ⸹(s, v) . The algorithm returns TRUE if
and only if the graph contains no negative-weight cycles that are reachable from the source.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-8

DAA: Design and Analysis of Algorithms [CoSc3042] 2024

Example: elicitation

The source is vertex s. The d values appear within the vertices, and shaded edges indicate predecessor
values: if edge (u, v) is shaded, then v.π=u. In this particular example, each pass relaxes the edges in the
order (t, x); (t, y); (t, z); (x, t); (x, y); (y, z); (z, x); (z, s); (s, t); (s, y). (a) The situation just before the first
pass over the edges. (b)–(e) The situation after each successive pass over the edges. The d and π values in
part (e) are the final values. The Bellman-Ford algorithm returns TRUE in this example.

DAA: Design and Analysis of Algorithms | Compiled by Gezahegn G. Page 3-9