Scribd
Upload
23 views

2-CST3011 - Algorithms and Data Structures-Algorithm Analysis

Lecture 2 for algorithms course

Uploaded by

Ben Aiah
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
23 views

2-CST3011 - Algorithms and Data Structures-Algorithm Analysis

Lecture 2 for algorithms course

Uploaded by

Ben Aiah
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 34

CST3011 – Algorithms and Data

Structures
Lecture II: Algorithm Analysis
Objectives
 At the end of this lesson, students are
expected to be able to
 Use the mathematical framework that more
rigorously describe the running time of an
algorithm

 Estimate the time required for a program for a


given input size N

 Apply techniques that drastically reduce the


running time of an algorithm
Outline
 Introduction
 Why Analyse Algorithms?
 Experimental studies
 Asymptotic Analysis
 Model
 Expressing Algorithms
 Running Time Calculations
 Asymptotic Notation
Introduction
 An algorithm is a clearly specified set of simple
instructions to be followed to solve a problem

 Criteria for judging an algorithm are


 Correctness: testing, verification
 Efficiency: usage of resources, the main subject of the
course
 Solves the problem but requires a year
 Solves the problem takes over the entire resources
 Simplicity: design and implementation

 Once the algorithm is determined to be correct, we


determine the amount of resources required:
Algorithm Analysis
Algorithm Analysis
 The running time of an algorithm, is a function of the
input
 Sorting 10000 elements require more time than sorting 10

 The exact values of these functions depend on


several factors
 Speed of the host machine
 Quality of the compiler
 Quality of the program etc.

 Analysis can be done by


 experimental studies
 Theoretical analysis
Experimental Studies
 Write a program implementing the algorithm

 Run the program with inputs of varying size

 Get the actual running time


 Use System.currentTimeMillis() for instance

 Plot the results

 Limitations
 The necessity to implement the algorithm
 Results may not be indicative of the running time on other
inputs not included in the experiment
 To compare two algorithms, the same hardware must be used
Theoretical Analysis
 Use a high level description of the algorithm
instead of implementation

 Takes into account all possible inputs

 Allows evaluation of the speed of the


algorithm independent of the
hardware/software environment
Why Analyse?
 To discover characteristics for evaluation or comparing
 In this course, we care most about asymptotic performance
 How does the algorithm behave as the problem size gets very large?
 *Running time
 Other characteristics
 Memory/storage requirements
 Bandwidth/power requirements/logic gates/etc.

 We define two functions


 Tavg(N) as average running time and
 Tworst(N) as the worst running time
 Clearly Tavg(N) ≤ Tworst(N)

 We focus on the worst case


 Easier to analyse
 Provides an upper bound
 An absolute guarantee, crucial to applications like games, finance and robotics
Model
 Analysis is performed with respect to a
computational model

 We will usually use a generic uniprocessor


random-access machine (RAM)
 All memory equally expensive to access
 No concurrent operations
 All reasonable instructions take unit time
 Except, of course, function calls
Algorithm description
 Algorithms can be expressed in one of the following medium,
depending on the level of expressiveness that is intended
 Plain English: most natural but least precise language
 Pseudocode: programming language which never complains about
syntax errors
 Programming language e.g. C, Java: most precise but difficult

 Pseudocode is a high-level description of an algorithm


 More structure than English prose
 Less detailed than programming language
 Preferred notation for describing algorithms, hides program design
issues
 There is no single standard for Pseudocode.

 We, for consistency sake, are going to define our own


pseudocode, CSC3011CODE based on Pascal/Basic syntax
CSC3011CODE Definition in BNF
<algorithm> ::= Algorithm <id> (<arglist>) <body> End.
<arg_list>::= <letter> , <arg_list> | <empty>
<body> ::= Input : <input_desc>
Output: <output desc>
<stmt_list>

<stmt_list> ::= <var_decl>; <stmt_list>


|<assign>; <stmt_list>
|<empty>
<var_decl>::=<letter> <letters>:<type>;
<letters>::=, <letter><letters>|<empty>
<assign>::=<letter> := <exp>
<exp>::=<letter>+<exp>|<letter>*<exp>|<empty> ,
(<exp>)etc
CST3011CODE Definition
 Loops
 For . . Do . . Next statement
 Repeat . . Until . .
 Selection
 If . . Then . . Else statement
 Case statement
 Array indexing starts at 0
 Indentation represents the block structures
 N2 superscripts and other mathematical
formatting allowed
 Other operators
  for assignment,  not equal,  comparison etc
Example
Algorithm sumOfSquares(M, N)
Input: integers M and N
Output: the sum of M2 and N2

P := M*M; 2

Q:= N*N; 2
Return P + Q; 2
End.
6 operations
Primitive operations
 Basic computation performed by an algorithm
 Identifiable in pseudocode
 Largely independent from the programming
language
 Exact definition not important
 Examples:
 Evaluating an expression
 Assigning a value to a variable
 Indexing into an array
 *Calling a method
 Return from a method
Rules for counting operators
 Rule 1 – For loops:
 The running time of the for loop is at most the running time of the
statements inside the for loop (including tests) times the number of
iterations. E.g:
For i  1 to n – 1 Do 1+ n – 1 = n
s  s + A[i] 3(n – 1) = 3n – 3,
Next n-1 total 5n – 4 ops
 Rule 2 – Nested loops:
 Analyse these inside out. The total running time of a statement
inside a group of nested loops is the running time of the
statement multiplied by the product of the sizes of all the
loops: E.g.
For i  0 to n – 1 1+n
For j  0 to m – 1 (1 + m)(n)
A[i][j]  i * j 4 (m)(n)
Next (m)(n)
Next (n)
Rules for counting operators cont’d
 Rule 3: Consecutive statements
 These just add

 Rule 4: If . . Then . . Else


 Consider If <cond> Then <S1> Else <S2>
 The running time of this statement is never more than
the running time of the test plus the larger of the
running times of S1 and s2: E.g.
Algorithm f(n)
If n  1 Then
Return 1
Else
Return n * f(n-1)
End.
Running time is O(n) (Recurrence problem)
Counting Primitive Operations
 Inspect the Pseudocode to determine the
maximum number of primitive operations executed
by an algorithm, as a function of the input size
 Example: # of operations
Algorithm arrayMax(A, n)
Input: A, an array of size n
Output: x, the largest value in A
current Max  A[0] 2
For i  1 to n – 1 do n-1+1 = n
If A[i]> currentMax then 2(n – 1)
currentMax  A[i] 2(n – 1)
End For
Return currentMax 1
End.
Total operations = 5n-1
Estimating Running Time
 Algorithm arrayMax executes 5n-1 primitive
operations in the worst case scenario

 Define:
 T(N) worst case time function dependant on N the
input
 A, Time taken by the fastest operation
 B ,Time taken by the slowest operation

 We have
 A(6N – 3) T(N)B(6N – 3)
 Therefore, T(N) is bound by two linear
functions
Growth Rate of Running Time
 Changing the hardware/software environment
 Affects T(N) by a constant factor
 Does not alter the growth rate of T(N)

 The Linear growth rate of running time T(N) is


a intrinsic property of the algorithm arrayMax
Growth Rates
 Popular growth rates
of functions in order: 5000

 Logarithmic 4000
 Linear f(n) = n
f(n) = log(n)
3000
 Quadratic f(n) = n log(n)
f(n) = n^2
 Cubic 2000 f(n) = n^3
f(n) = 2^n

1000
 Others include
0
 LogLogN 1 3 5 7 9 11 13 15 17 19

 Nlog(N)
 Exponential
Asymptotic Notation
 The analysis required to estimate resource
use is a theoretical issue

 We use the asymptotic (big-O) notation:


 What does O(n) running time mean? O(n 2)? O(n lg
n)?

 Our first task is to define this notation more


formally and completely
Asymptotic Notation
 We will use the following four definitions

 T(N) =  (f(N)) if  c,n0  +,  T(N)  cf(N)  N  n0

 T(N) =  (f(N)) if  c,n0  +,  T(N)  cf(N)  N  n0

 T(N) =  (f(N)) T(N) =  (f(N)) & T(N) =  (f(N))

 T(N) =  (f(N)) if  c,n0  +,  T(N)  cf(N)  N  n0

 These definitions establish a relation order among


functions: i.e. the relative rates of growth of
functions
Example
 1000N = (N2) i.e. N2 grows more rapidly than 1000N and
will eventually be larger than 1000N.

 If g(N) = 100N2 + 20N + 200 then g(N) = (N2) = (N3) =


(N4)

 2N + 10 = (N)

 Since:
 2N + 10  cN
 (c – 2)N  10
 N  10/(c – 2) for c = 3 and n0 = 10, it is true
 Exercise:
 Show that g(N) =  (f(N))  f(N) =  (g(N))
Big-Oh Rules and Growth Rates
 The Big-Oh notation gives a upper bound on
the growth rate of the function

 The statement “f(n) is O(g(n))”, simply means


that the growth rate of f(n) is no more than
the growth rate of g(n)

 We can use the Big-Oh notation to rank


functions according to their growth rate
Big-Oh Rules
 Rule 1:
 If T1(n) = O(f(n)) and T2(N) = O(g(n)), then
 T1(n) + T2(n) = max(O(f(n), O(g(n))

 Rule II
 If f(n) is a polynomial of order k, then f(n) is (nk) i.e.
 Rule III
 Logk(n) = O(n) for any constant k

 Rule of Thumb: Drop the lower-order terms


 Drop constant factors
 Say 2n + 4 is O(n) not (O(2n) or O(2n + 4)

 Use the smallest possible class of functions


 Say “2n is O(n)” instead of “2n is O(n 2)
Asymptotic Algorithm Analysis
 The Asymptotic analysis of an algorithm determines
the running time in Big-Oh notation
 To perform the Asymptotic analysis:
 We find the worst-case number of primitive operations
executed as a function of the input size
 We express this function with the Big-Oh notation
 Example:
 We determine that arrayMax executes at most n – 1
operations
 We say arrayMax runs O(n) time
 Since the constant factors and lower-order terms are
eventually dropped anyway, we can disregard them
when counting primitive operations – but don’t!!!
Example: Prefix averages
 We illustrate asymptotic analysis with two
algorithms for prefix averages

 The i-th prefix average is the average of the


array X is the average of the first i elements of
X
 A[i] = [X[0]+X[1]+ . . +X[i]]/(i+1)

 Computing the array A of prefix averages of


another array X has applications to financial
analysis
Solution I
 Pseudo code
Algorithm prefixAverages(A, n)
Input: an array A of size n
Output: X, an array of prefix averages of X

X  new array of size n n


For i  0 to n – 1 do 1+n
S  A[0] 2n
For j = 1 to i do 1 + 2 + .. + (n-1)
S  S + A[j] 3(1 + 2 + .. + (n-1))
End For
X[i]  S/(i + 1) 4n
End For
Return X 1
End.
Solution I: Digest
 Arithmetic Progression
 Quadratic order
Solution II
 Pseudocode
Algorithm prefixAverages2(A, n)
Input
Output

X  new array of size n


s0
For i  0 to n – 1 do
s  s + A[i]
X[i]  s/(i + 1)
Next
Return A
End.
Growth rate by limit
 To determine the growth rate of two function
f(N) and g(N), compute

 This limit has four possible values


 Limit = 0  f(N) = (g(N)
 Limit = c  0  f(N) = (g(N))
 Limit =   g(N) = (f(N))
 Limit oscillates: There is no relation
L’Hôpitals Rule
 In determining the limit of a quotient of f(N)
and g(N) as n tends to infinity, you can use
the L’Hôpitals Rule, which states

 Exercise: Show that log(N) = o(N), using the


L’Hôpitals Rule.
Example
 Show that log(n) = o(N)

 Soln:
??QUESTIONS??

You might also like