Chapter 1 – Introduction to

Data Structure and

Algorithm
By: Kibret Z.
Department of Computer Science
College of Informatics
University of Gondar
Outline
• In this chapter, you be able to learn about :
• Data and information
• Data types and data representation in computer memory
• Data structure and classification of data structure
• Algorithm and its properties
• Algorithm analysis
• Complexity analysis
• Asymptotic notations
• Compute the complexity of algorithm

March 2023 DATA STRUCTURE AND ALGORITHM 2

Data and Information
• What is data? What is the difference between data and Information?
• “Data is a collection of discrete values that convey information, describing quantity,
quality, fact, statistics, other basic units of meaning, or simply sequences of symbols
that may be further interpreted.” [-Wikipedia]
• A datum is an individual value in a collection of data.
• Definition:
• Data is the basic entity or fact that is used in a calculation or manipulation process.
• Data is commonly processed by some stages.
• Unprocessed data or raw data is a collection of numbers, characters, that may be
considered as an input of a stage and processed data is the output of the stage.

March 2023 DATA STRUCTURE AND ALGORITHM 3

Data and Information Continue…

• Data may be a single value or a set of values and it is to be organized in a

particular fashion.
• This organization or structuring of data will have a profound impact on the efficiency of
the program.
• Data are raw facts without context, whereas Information is data with context.
• Data requires interpretation to become an Information.
• Data can be any character, text, words, number, pictures, sound, or video and
if not put into context means nothing to a human or computer.
• For example, 1896 is a data, whereas information may be April 1, 2006 -the date of
Initial release of Apache Hadoop.
Data Information

Fig 1.1: Data and Information

March 2023 DATA STRUCTURE AND ALGORITHM 4

Representation of Data
• Mainly two types data to be represented in computer memory of :
1. Numerical Data
2. Alphanumerical Data
• Numerical data are classified into two types:
1. Integer
2. Floating-point
• Alphanumerical data are classified into two types:
1. Character- may be alphabets, digits, special characters and white spaces.
2. String - is a sequence of characters may consist of any number and any combination of
characters.
• The string can be defined as an array of character terminated with a null character.

March 2023 DATA STRUCTURE AND ALGORITHM 5

Data Type
• Data types specify the size and types of values to be stored.
• C++ supports a wide variety of data types and the programmer can select the
data type appropriate to the needs of the application.
• C++ supports the following data types:
1. Primary or Built-in or Fundamental data type – are language built-in
2. Derived data types - are derived from the primitive or built-in datatypes
3. Abstract or User-defined data types - are defined by the user itself

March 2023 DATA STRUCTURE AND ALGORITHM 6

 Data Types in C++

Built-in Derived User Defined

• char • Class
• Function
• int •
• Array Structure
• boolean • Pointer • Union
• float • Enumeration
• double • Typedef
• void
• wide character
(wchar_t)

Fig 1.2: Data Types in C++

March 2023 DATA STRUCTURE AND ALGORITHM 7

Data Structure
• Data Structure is the representation of the logical relationship between individual
elements of data.
• In Computer Science, Data Structure is defined as a mathematical or logical model
of organizing the data items into computer memory in such a way that they can
be used efficiently.
• Why we need to study Data Structure?
• It is to learn how to organize data into memory so that it can be accessed quickly and conveniently.
• Data Structure refers to the study of data and representation of data objects within
the program;
• i.e., the implementation of structured relationships among different data objects.
• Example: lists, stacks, queues, heaps, search trees, hash tables, etc.

March 2023 DATA STRUCTURE AND ALGORITHM 8

Data Structure Continue…

• The data structures are the building blocks of a program and hence the
selection of a particular data structure stresses on
• The data structures must be rich enough in structure to reflect the relationship existing
between the data, and
• The structure should be simple so that we can process data effectively whenever
required.
• Data Structure can be used for the following purpose:
• Organizing the data – How data items are organized in the main memory?
• Accessing methods – How data items can be accessed?
• Specifying the degree of associativity – How data items are interrelated?
• Processing alternatives for data – How many different ways are there in which these
data items can be processed?

March 2023 DATA STRUCTURE AND ALGORITHM 9

Data Structure Continue…

• A program is a set of instructions, which involve a computer performing

some kind of computation or algorithm.
• Data Structure affects the design of both structural and functional aspects of
a program.
• To implement a program of an algorithm, we should select an appropriate
data structure for that algorithm.
• Therefore, the programs are inherited by an algorithm and its associated data
structure.
Algorithm + Data Structure = Program

March 2023 DATA STRUCTURE AND ALGORITHM 10

Classifications of Data Structure
• Data Structure can be classified into two categories:
i. Primitive data structure
ii. Non-Primitive data structure
• Primitive Data Structure:
• are defined that can be manipulated or operated by the machine instruction.
• generally built upon simpler primitive data types, called Primitive data structures,
which are represented in computer memory.
• Example: Integer, floating point, characters, pointer, boolean, etc. are some of the
different primitive data structure.

March 2023 DATA STRUCTURE AND ALGORITHM 11

 Fig 1.3: Types of Data Structure
March 2023 DATA STRUCTURE AND ALGORITHM 12
Classifications of Data Structure Continue…

• Non-primitive Data Structure

• are a data structure that cannot be manipulated or operated directly by the machine
instructions.
• are more sophisticated data structures.
• are derived from the primitive data structure.
• Example: Arrays, structure, stack, queues, linked lists etc.
• The Non-primitive data structures are classified into two categories:
i. Linear data structure
ii. Non-linear data structure

March 2023 DATA STRUCTURE AND ALGORITHM 13

Classifications of Data Structure Continue…
• Linear Data Structures
• The data structure is linear if every data item is related to its previous and next data items.
• data items are arranged in memory in a linear sequence and data items are accessed linearly.
• The traversing of the linear data structure is exactly once.
• Linear data structures are two types:
i. Sequential
ii. Linked
• Sequential Data Structures are based on arrays where objects are stored in a sequence of
consecutive memory locations.
• Example: Arrays, Stacks, Queues
• Linked Data Structure is a data structure, which consists of a set of nodes linked together and
organized with links.
• Example: Linked Lists, Linked Stacks, Linked Queues

March 2023 DATA STRUCTURE AND ALGORITHM 14

Classifications of Data Structure Continue…
• Non-Linear Data Structure
• A data structure is non-linear if every data item attaches to many other data items in
specific ways to reflect relationships (e.g., tree).
• In non-linear data structures, the data elements are not in sequence,
• i.e., insertion and deletion are not possible in a linear manner.
• The traversing of the non-linear data structure is always more than one.
• Tree, graph, table, and sets are the examples of non-linear data structure.

March 2023 DATA STRUCTURE AND ALGORITHM 15

Static and Dynamic Data Structure
• Static data structure
• once memory space is allocated it cannot extend,
• i.e. the memory allocation for the data structure takes place at compile-time that
cannot be changed afterwards.
• Example: Array
• Dynamic Data Structure
• Memory space can be extended or shrink during the execution,
• i.e., the memory allocation as well as memory de-allocation for the data structure takes
place at run-time and allocates memory as required amount at any time.
• Example: linked list, stack, queue, tree

March 2023 DATA STRUCTURE AND ALGORITHM 16

Applications of Data Structure
• Data Structure is used in various fields of Computer Science, such as:
• Compiler Design
• Operating System
• Database Management System
• Statistical Analysis Package
• Numerical Analysis
• Graphics
• Artificial Intelligence
• Simulation

March 2023 DATA STRUCTURE AND ALGORITHM 17

Applications of Data Structure Continue…

• Different kinds of data structures are suitable for different kinds of

applications.
• Some data structures are highly specialized for specific tasks
• For example,
• Databases use B-tree indexes for small percentages of data retrieval, and
• Compilers and databases use dynamic hash tables as lookup tables,
• Operating systems use queues for process management, I/O request handling.

March 2023 DATA STRUCTURE AND ALGORITHM 18

Abstract Data Type (ADT)
• A data type refers to the type of data that variables hold.
• Data Object represents an object having a data.
• The study of classes of objects whose logical behavior is defined by a set of
values and a set of operations.
• An Abstract Data Type (ADT) describes the data objects, which constitute
• the data structure and
• the fundamental operations supported on them.

March 2023 DATA STRUCTURE AND ALGORITHM 19

Abstract Data Type (ADT) Continue…

• An ADT promotes data abstraction and focuses on what a data structure

does, rather than how it implements (does).
• An ADT is a conceptual model of information structure.
• An ADT specifies the components, their structuring relationships and a list of
operations that are allowed to perform.
• It is just a specification, no design or implementation information is included.
• Specification involves the ―what the operations, not the ―how.
• ADT‘s are generalizations of primitive data types.
• A data structure is the design representation of an ADT.
• The same ADT may be represented by several data structures.
• There are many data structures corresponding to the ADT ―set‖.

March 2023 DATA STRUCTURE AND ALGORITHM 20

Operations Performed in Data Structure
• Different sets of operations can be performed on different data structures.
Operations Description

Creation Allocation of memory for the data structure, the creation of data structure may take
place either during compile-time or during run-time.

Insertion Insert a data item in the data structure.

Deletion Delete a data item from the data structure.
Traversing Accessing and processing each data item of the data structure exactly once.

Searching Find the location of the key value within the data structure.
Sorting Arranging all the data items in a data structure either in ascending or in descending
order or in lexicographical order (for Strings).
Merging
March 2023 Combining the data items
DATAof two different
STRUCTURE sorted lists into a single sorted list.
AND ALGORITHM 21
Overview of Different Data Structures
• Array
• An array is a collection of the same type of data items, which are stored in consecutive
memory locations under a common name.
• In arrays, there is always a fixed relationship between the addresses of two consecutive
elements as all the items of an array must be stored contiguously.
• Stack
• A stack is a collection of elements into which new elements may be inserted and from
which elements may be deleted only at one end called the top of the stack.
• All the insertion and deletion in a stack is done from the top of the stack, the last added
element will be first to be removed from the stack.
• Stack is also called Last-In-First-Out (LIFO) data structure.

March 2023 DATA STRUCTURE AND ALGORITHM 22

Overview of Different Data Structures Continue…
• Queue
• A queue is a homogeneous collection of elements in which deletions can take place only at the
front end, known as dequeue and insertions can take place only at the rear end, known as
enqueue.
• The element, which inserts in the queue first, will delete the queue first.
• In this order, a queue is called First-In-First-Out (FIFO) system.
• Linked List
• A linked list is an ordered collection of finite homogeneous data elements called node where
the linear order is maintained by means of links or pointers.
• Data items may be scattered arbitrarily all over the memory.
• There is no relationship between the addresses of elements;
• Each element of a linked list must store explicitly the address of the element, next to it.

March 2023 DATA STRUCTURE AND ALGORITHM 23

Overview of Different Data Structures Continue…

• Tree
• The tree is a non-linear data structure.
• A Tree may be defined as a non-empty finite set of nodes, such that
i. There is a specially designated node called the root.
ii. The remaining nodes are partitioned into zero or more disjoint trees T1, T2, … Tn are called the
subtrees of the root R.
• Graph
• The graph is another non-linear data structure.
• A Graph G is defined as an ordered set G = (V, E), consists of finite non-empty set of
objects V, where V (G) = {v1, v2, v3, … vn} called vertices (or nodes or points) and
another set E where E (G) = {e1, e2, e3, …em} whose elements are called edges, that
connects these vertices.

March 2023 DATA STRUCTURE AND ALGORITHM 24

Algorithm
• An algorithm is a well-defined computational procedure that transforms
inputs into outputs achieving the desired input-output relationship.
• An algorithm can perform the calculation, data processing and automated
reasoning task in a finite time.
• The word ―algorithm is a Persian word algorism derived from the name of
the Persian author and great mathematician ―Abu Abd Allh Jafar
Mohammad bin Musba al Khowarzimi‖ (Born in 780 A.D. in Baghdad).

March 2023 DATA STRUCTURE AND ALGORITHM 25

Algorithm Continue…
• Algorithm is a finite sequence of instructions/steps, each of which is very elementary that
must be followed to solve a problem.
• All algorithms must satisfy the following criteria:
i. Input: There are some inputs (zero or more quantities), which are externally supplied.
ii. Output: The result (at least one quantity) should be produced after completion of the job to the
user.
iii. Definiteness: The instructions should be unique, concise, clear and unambiguous.
iv. Finiteness: The instructions should be relative in nature and should not be of an infinite type.
Should be terminated after a finite number of steps.
v. Effectiveness: Every instruction must be elementary, so that it can be carried out, in principle,
by a person using only pencil and paper.
Repetition of same steps should be avoided. It must be feasible.

March 2023 DATA STRUCTURE AND ALGORITHM 26

Algorithm Development Life Cycle
• In the life cycle of an algorithm for the development of an algorithm, the
following phases are involved:
i. Algorithm design / Design Phase
ii. Writing Phase
iii. Testing / Experiment Phase
iv. Analysis of Algorithm / Analysing Phase

March 2023 DATA STRUCTURE AND ALGORITHM 27

Algorithm Design
• Algorithmic design patterns, which provide general techniques for designing
algorithms, including
• divide-and-conquer,
• dynamic programming,
• greedy method,
• backtracking, and
• branch-and-bound.
• In the design phase of an algorithm, one of the algorithmic design techniques
and the data structure is used.

March 2023 DATA STRUCTURE AND ALGORITHM 28

Writing Phase
• An algorithm generally is written in a modular approach.
• An algorithm may have the following structures or steps:
• Input Step, Initialization Step, Assignment Step, Decision Step, Repetition Step and
Output Step.
• To make each and every step clear in the algorithm, comments are written
whenever necessary.
• An algorithm is written using the following basic methods:
i. Sequential: Steps are performed sequentially.
ii. Selection/Decision: One of the several alternative actions is selected.
iii. Repetition: One or more than one step(s) are performed repeatedly.

March 2023 DATA STRUCTURE AND ALGORITHM 29

Example1: Algorithm to convert temperature from Celsius to Fahrenheit

• Algorithm: Convert(C)
[The variable C is used as Celsius value and F is used as Fahrenheit value]
1. Read C
2. Set F = 1.8 * C + 32
3. Print F
4. Return

March 2023 DATA STRUCTURE AND ALGORITHM 30

Example 2: Algorithm to calculate the sum of N natural numbers
Algorithm: Sum (N)
[The variable N is represented the number of natural numbers.]
1. Set S = 0 and I = 1
2. Read N
3. S = S + I
4. I = I + 1
5. If I <= N Goto Step 3
6. Print S
7. Return

March 2023 DATA STRUCTURE AND ALGORITHM 31

Testing Phase
• After writing an algorithm, it is necessary to check whether the algorithm is
correct or not.
• One must perform each step using paper and pencil by some required valid
input, use the algorithm and get the required output in a finite time.
• Testing or Experiment is another step in the development of the algorithm.
• When any error is found, then go back to the algorithm design phase and
redesign the algorithm.

March 2023 DATA STRUCTURE AND ALGORITHM 32

Analysis of Algorithms
• Algorithms are to be analysed to compute the objective criteria for different
input size before actual implementation.
• Suppose P is a problem and A & B are two different algorithms to solve the
problem P.
• There must have some performance measurement system to decide that which
algorithm is better than the other.
• Suppose M is an algorithm and n is the size of the input data.
• The time and space used by the algorithm M are two main measures for the
efficiency of M.
• The complexity of an algorithm M is the function f(n) which gives the running
time and/or storage space requirement of the algorithm in terms of the size n of the
input data.

March 2023 DATA STRUCTURE AND ALGORITHM 33

Analysis of Algorithms Continue…

• The performance of the algorithm can be measured on the scales of time and
space.
• Time means, we are looking for the fastest algorithm for the problem.
• Space means we are looking for an algorithm that consumes or needs minimum
memory space for its execution.
• When performance is measured in terms of space, it is known as space
complexity
• when performance is measured in terms of time, it is termed as time
complexity.

March 2023 DATA STRUCTURE AND ALGORITHM 34

Kinds of Analysis of Algorithm
• There are two kinds of analysis of algorithm may be possible:
1. Apriori Analysis
2. Posteriori Analysis
• Theoretical / Apriori Analysis
• Apriori meaning "from the earlier".
• It is on a theoretical basis, independent of programming languages and machine
structures.
• Do analysis (space and time) of an algorithm prior to running it on a specific system.
• The stress is laid on the frequency of execution of statements in the algorithm.

March 2023 DATA STRUCTURE AND ALGORITHM 35

Kinds of Analysis of Algorithm Continue…
• Empirical / Posteriori Analysis
• Posteriori means "from the latter".
• The actual amount of space and time taken by the algorithms are recorded during execution.
• It is dependent on the programming language used and machine structure.
• Do an analysis of the algorithm only after running it on the system.
• It directly depends on the system and changes from system to system.
• In real-life, we cannot do posteriori analysis as software is generally made for an
anonymous user, which runs it on a system different from those present in the
industry.
• Therefore, in Apriori analysis, we use asymptotic notations to determine time and space
complexity as they change from computer to computer but asymptotically they are same

March 2023 DATA STRUCTURE AND ALGORITHM 36

Space Complexity
• The space complexity is the amount of space (memory) is needed for an
algorithm to solve the problem.
• An efficient algorithm takes space as small as possible.
• Space needed by a program is the sum of the following components:
1. Fixed Space Requirement: Includes space for code, space for simple variables,
constants.
2. Variable Space Requirement: Includes space needed by component variables,
structured variable and dynamically allocated space and the recursive stack space.

March 2023 DATA STRUCTURE AND ALGORITHM 37

Time Complexity
• The time complexity of a program is the amount of computer time it needs
to run to completion.
• The time complexity is of two types such as
1. Compilation time
2. Runtime
• The amount of time taken by the compiler to compile an algorithm is known
as compilation time.
• During compilation time it calculates only the declaration statements and
checks for any syntax and semantic errors.

March 2023 DATA STRUCTURE AND ALGORITHM 38

Time Complexity Continue…
• The run time depends on the size of an algorithm.
o If the number of instructions in an algorithm is large, then the run time is also large,
o If the number of instructions in an algorithm is small, then the time for executing the program is also
small.
• The runtime is calculated for executable statements and not for declaration statements.
• Different factors affecting the execution time, e.g.
• programmer skills,
• compiler options,
• hardware characteristics (instruction set, clock speed),
• the algorithm used, input size, etc.
• Generally, an estimate of the growth rate of execution time with input size is sufficient for
comparison.

March 2023 DATA STRUCTURE AND ALGORITHM 39

Time Complexity Continue…
• The rules for computing running time
1. Sequence: Add the time of the individual statements. The maximum is the one that counts.
2. Alternative structures: Time for testing the condition plus the maximum time taken by any of the
alternative paths.
3. Loops: Execution time of a loop is at most the execution time of the statements of the body (including
the condition tests) multiplied by the number of iterations.
4. Nested loops: Analyze them inside out.
5. Subprograms: Analyze them as separate algorithms and substitute the time wherever necessary.
6. Recursive Subprograms: Generally, the running time can be expressed as a recurrence relation.
• The solution of the recurrence relation yields the expression for the growth rate of execution time.
• While measuring the time complexity of an algorithm, we concentrate on the frequency count
of all key statements (important statement).

March 2023 DATA STRUCTURE AND ALGORITHM 40

Time Complexity Continue…
• Example 1:
Algorithm
a = a + 1
• In this algorithm, the frequency count of key statement a = a + 1, is one.
• Example 2:
Algorithm
For I = 1 to n
a = a + 1
[End of loop]
• In this algorithm, the frequency count of key statement a = a + 1, is n.

March 2023 DATA STRUCTURE AND ALGORITHM 41

Time Complexity Continue…
• Example 3:
Algorithm
For I = 1 to n
For J = 1 to n
a = a + 1
End of loop]
End loop]
• In this algorithm, the frequency count of key statement a = a + 1, is n2.
• If an algorithm performs f(n) basic operations when the size of input is n, then its total running
time will be t × f (n), where t is the time required to perform a basic operation.
• Note:
• Different operations consume different times to execute,
• For simplicity, we are assuming that each basic operation consumes same time to execute, which is say t in the
above case.

March 2023 DATA STRUCTURE AND ALGORITHM 42

Type of Time Complexity
• There are three types of time complexities, which can be found in the analysis
of an algorithm:
1. Best-case time complexity
2. Average-case time complexity
3. Worst-case time complexity
1. Best-case time complexity
• The best-case time complexity of an algorithm is a measure of the minimum time that
the algorithm will require.
• For example, the best case for a simple linear search on a list occurs when the desired
element is the first element of the list.

March 2023 DATA STRUCTURE AND ALGORITHM 43

Type of Time Complexity Continue…
2. Worst-case time complexity
• The worst-case time complexity of an algorithm is a measure of the maximum time that the algorithm will
require.
• A worst-case estimate is normally computed because it provides an upper bound for all inputs including the
extreme slowest case also.
• For example, the worst case for a simple linear search on a list occurs when the desired element is found at
the last position of the list or not on the list.
3. Average-case time complexity
• The average-case time complexity of an algorithm is a measure of average time of all instances taken by an
algorithm.
• Average case analysis does not provide the upper bound and sometimes it is difficult to compute.
• Average-case time complexity and worst-case time complexity are the most used in algorithm
analysis.
• Best-case time complexity is rarely found but is does not have any uses.

March 2023 DATA STRUCTURE AND ALGORITHM 44

AsymptoAsymtic Notations
• Commonly used notations in performance analysis to characterize the
complexity of an algorithm.
• Big Oh (O),
• Little oh (o),
• Omega (Ω),
• Theta (Θ) and
• Little omega (ω)
• They are related notations to describe different kinds of bounds on
asymptotic growth rates.

March 2023 DATA STRUCTURE AND ALGORITHM 45

Big-O (“big oh”) Notation (Upper Bound)
• The big-O notation is the formal method of expressing the upper bound of an
algorithm's running time.
• It is a measure of the longest amount of time it could possibly take for the
algorithm to complete.

Fig 1.4 : Big-O Notation

March 2023 DATA STRUCTURE AND ALGORITHM 46

Big-O (“big oh”) Notation (Upper Bound) Continue…

• If f(n) and g(n) are functions defined for positive integers, then we write
f(n) = O(g(n)) if there exist positive constants n0 and c such that
|f(n)|≤ c| g(n |, for all n ≥ n0.
• In general, O(g(n)) = {f(n): there exist positive constants c and n0 such that
0 ≤ f(n) ≤ cg(n) for all n, where n ≥ n0}.
• That is the right of n0, the value of f (n) always lies on or below cg (n).
• This notation gives an upper bound for a function to within a constant factor.

March 2023 DATA STRUCTURE AND ALGORITHM 47

Big-O (“big oh”) Notation (Upper Bound) Continue…
• Example 1:
• Suppose f (n) = 3n + 2
• Now we can write f (n) = 3n + 2 ≤ 4n, for all n ≥ 2, here g (n) = n, c = 4, n0 = 2
• Hence, we can write f (n) = O (g (n)) = O (n)
• Example 2:
• Suppose f (n) = 10n2 + 4n + 2
• We can write f (n) = 10n2 +4n+2 ≤ 11n2 , for all n ≥ 5, here g(n) = n2, c = 11, n0 = 5
• Hence, we can write f (n) = O (g (n)) = O (n2)
• Example 3:
• Suppose f (n) = 2n3 + n2+2n
• Now we can write f(n)=2n 3+n2+2n ≤ 3n3, for all n ≥ 2, here g(n)= n3, c= 3, n0 = 2
• Hence, we can write f (n) = O (g (n)) = O (n3)

March 2023 DATA STRUCTURE AND ALGORITHM 48

Big-Ω (Omega) Notation (Lower Bound)
• Big-Ω notation is the formal method of expressing the lower bound on an algorithm's
running time.
• It describes the best that can happen for a given data size.
• If f(n) and g(n) are functions defined for positive integers, then we write f(n) = Ω (g
(n)) if there exist positive constants n0 and c such that
|f(n)| ≥ c|g(n)|, for all n≥ n0.
• In general, Ω (g (n)) = {f(n): there exist positive constants c and n0 such that
0 ≤ cg(n) ≤ f(n) for all n, where n≥ n0}.
• That is the right of n0, the value of the function f(n) always lies on or above cg (n).
• This notation gives a lower bound for a function to within a constant factor.

March 2023 DATA STRUCTURE AND ALGORITHM 49

Big-Ω (Omega) Notation (Lower Bound) Continue…

Fig 1.5 : Omega Notation

March 2023 DATA STRUCTURE AND ALGORITHM 50

Big-Ω (Omega) Notation (Lower Bound) Continue…

• Example 1:
• Suppose f (n) = 3n + 2
• We can write f (n) = 3n + 2 ≥ 3n, for all n ≥ 1, here g (n) = n, c = 3, n0 = 1
• Hence, we can write f (n) = Ω (g (n)) = Ω (n)
• Example 2:
• Suppose f (n) = 10n2 + 4n + 2
• We can write f(n)=10n2 +4n+2≥10n2, for all n≥1, here g(n)=n2, c=10, n0=5
• Hence, we can write f (n) = Ω (g (n)) = Ω (n2)
• Example 3:
• Suppose f (n) = 2n3 + n2 + 2n
• We can write f (n) = 2n3 + n2 +2n ≥ 2n3, for all n ≥ 1, here c = 2, n0 = 1
• Hence, we can write f (n) = Ω (g (n)) = Ω (n3)

March 2023 DATA STRUCTURE AND ALGORITHM 51

Θ (Theta) -Notation (Tight Bound)
• If f(n) & g (n) are functions defined for positive integers, then we say
f(n)=Θ(g(n)) if there exists positive constants n0, c1 and c2 such that
c1|g(n)| ≤ |f(n)| ≤ c2|g(n)|, for all n ≥ n0
• In general, Θ (g (n)) = {f (n): there exist positive constants c1, c2 and n0 such
that c1g(n) ≤ f(n) ≤ c2g(n) for all n, where n ≥ n0}.
• That is the right of n0 the value of the function f(n) always lies between c1g(n) and
c2g(n) inclusive.
• This notation bounds a function to within constant factors.
• This is saying that the function, f(n) is bounded from both the top and bottom by the
same function, g(n).

March 2023 DATA STRUCTURE AND ALGORITHM 52

Θ (Theta) -Notation (Tight Bound) Continue…

Fig 1.6 : Theta Notation

March 2023 DATA STRUCTURE AND ALGORITHM 53

Θ (Theta) -Notation (Tight Bound) Continue…
• Example 1:
• Suppose f (n) = 3n + 2
• We can write f(n) = 3n + 2 ≤ 4n, for all n ≥ 2, here g (n) = n, c1 = 3, c2 = 4, n0 = 2
• Hence, we can write f (n) = Θ (g (n)) = Θ(n)
• Example 2:
• Suppose f (n) = 10n2 + 4n + 2
• We can write f(n) = 10n2 +4n+2 ≤ 11n2 , for all n ≥ 5, here g(n) = n2, c1 = 10, c2=11, n0 =
5
• Hence, we can write f (n) = Θ (g (n)) = Θ (n2)
• Example 3:
• Suppose f (n) = 2n3 + n2+2n
• Now we can write f(n)=2n 3+n2+2n ≤ 3n3, for all n ≥ 2, here g(n)= n3, c1= , c2 = 3, n0 = 2
• Hence, we can write f (n) = Θ (g (n)) = Θ (n3)
March 2023 DATA STRUCTURE AND ALGORITHM 54
o (“little oh”) - Notation
• For non-negative functions, f(n) and g(n), f(n) is little o of g(n) if and only if
f(n) = O(g(n)), but f(n) ≠ Θ(g(n)).
• It is denoted as "f(n) = o(g(n))".
• Little-o notation represents a loose bounding version of Big O, g (n) bounds from
the top, but it does not bound the bottom.
• If f(n) & g(n) are functions defined for positive integers, then we say
f(n) = o(g(n)) iff
➢f (n) = O (g (n)) and f (n) ≠ Θ (g (n)) OR

• Example:
Suppose f (n) =2n3 + 3n2 + n + 10
we can write f (n) = o (g (n)) = o (n4)
March 2023 DATA STRUCTURE AND ALGORITHM 55
ω (little omega)-Notation
• Little-ω notation represents a loose bounding version of Big-Ω;
• g (n) is a loose lower boundary of the function f (n);
• it bounds from the bottom, but not from the top.
• For non-negative functions, f(n) and g(n), f(n) is little omega of g(n) if and only if
f(n) = Ω(g(n)), but f(n) ≠ Θ(g(n)).
• It is denoted as "f(n) = ω(g(n))".
• We can write f (n) = ω (g (n)) iff

• Example:
• Suppose f (n) = 4n3 + 2n + 3
• we can write f (n) = ω (g (n)) = ω (n2)

March 2023 DATA STRUCTURE AND ALGORITHM 56

Growth Functions of Algorithm
• The big-O notation is used to classify algorithms by how they respond in
their processing time or working space requirements to changes in the input
size.
• Order of growth in algorithm means how the time or space for computation
increases with input size,
• i.e. how the growth function grows with the input size.
• Based on growth functions, the algorithm can be categorized in to 7 type
shown on the next slides

March 2023 DATA STRUCTURE AND ALGORITHM 57

Growth Functions of Algorithm Continue…
1. Constant time algorithms [ O(1) ]:
• This means that the algorithm requires the same fixed number of steps regardless of the size of the task.
• Example: Array access, Push and pop operations for a stack, enqueue and dequeue operations for a
queue, other atomic operations.
2. Logarithmic time algorithms [ O () ]:
• Logarithm time algorithms mean that T(n) is upper bounded by logarithmic of the input size, i.e. T(n) = .
• Example: Binary search in a sorted list of n elements, Insert and Find operations for a binary search tree
with n nodes, Insert and Remove operations for a heap with n nodes, etc.
3. Linear time algorithms [ O (n) ]:
• Linear time algorithm means that for large enough input sizes the running time increases linearly with
the size of the input.
• Example: Array traversal

March 2023 DATA STRUCTURE AND ALGORITHM 58

Growth Functions of Algorithm Continue…

4. Linearithmic time algorithm [ O (n) ]:

• An algorithm is said to run in linearithmic time when T(n) = O (n).
• A linearithmic term grows faster than a linear term, but slower than any polynomial in
n with degree greater than 1.
• Example: Merge sort, Heap sort
5. Polynomial time algorithms [O (nk), for k > 1 ]:
i. Quadratic time (O (n2)) algorithm: The number of operations is proportional to the
size of the task squared.
• Example: Slow sorting algorithms, for example, selection sort of n elements.
ii. Cubic time (O (n3)) algorithm: The number of operations is proportional to the size
of the task cubed.
• Example: Matrix multiplication

March 2023 DATA STRUCTURE AND ALGORITHM 59

Growth Functions of Algorithm Continue…

6. Exponential time algorithms [ O (kn), for k>1 ]:

• Exponential time algorithm means that T(n) is upper bounded by exponential
expression of input size, i.e. T(n) = kn
• Example: Towers of Hanoi, implementation of recursive Fibonacci, generating all
permutations of n symbols.
7. Factorial time algorithms [ O (n!) ]:
• Polynomial time algorithm means that T (n) is upper bounded by the factorial of the
input size, i.e. T(n) = n!.
• Example: Non-attacking n queen problem, traveling salesman problem by brute-force
search

March 2023 DATA STRUCTURE AND ALGORITHM 60

Growth Functions of Algorithm Continue…

• The best time in the above list is obviously constant time,

• The worst is exponential time, which, as we have seen, quickly overwhelms
even the fastest computers even for relatively small n.
• Polynomial growth (linear, quadratic, cubic, etc.) is considered manageable as
compared to exponential growth.

March 2023 DATA STRUCTURE AND ALGORITHM 61

Growth Functions of Algorithm Continue…

n g n g 2
1 0 0 1 1 2
2 1 2 4 8 4
4 2 8 16 64 16
8 3 24 64 512 256
16 4 64 256 4096 65536
32 5 160 1024 32768 4294967296
Table 1.1: Values for different growth functions

• Order of asymptotic behavior of the functions from the above list, we can compare
among time complexities:
O (1) < O () < O (n) < O (n) < O (nk) < O (kn) < O (n!)

March 2023 DATA STRUCTURE AND ALGORITHM 62

Big-o Complexities for Growth Functions

Figure 1.7: Big-O Complexity

March 2023 DATA STRUCTURE AND ALGORITHM 63
Manipulation of Asymptotic Notations
• A number of rules to manipulate asymptotic notations are followed:
1. O(cf(n)) = O(f(n))
2. O(O(f(n)) = O(f(n))
3. O(f(n) g (n)) = f(n) O(g(n))
4. O(f (n) O(g (n)) = O(f(n) g(n))
5. O(f(n) + g(n)) = O(max (f(n), g(n))
6. If f(n) = O(g(n)) and g(n) = O(h(n)), then f(n) = O(h(n)). [Transitivity Rule]
7. f(n) = O(g(n)) iff g(n) = Ω(f(n)). [Symmetry Rule]

March 2023 DATA STRUCTURE AND ALGORITHM 64

Manipulation of Asymptotic Notations Continue…

• Suppose, in an algorithm consists of two independent parts-A1 and A2.

• These parts may be considered as a sub-algorithm or a single instruction.
• Let us assume that A1 and A2 take time t1 and t2 respectively for computation.
• Now, whenever these two parts are in a sequential structure that means first A1 is
computed, and then A2 is computed, denoted by A1: A2, the total computation time
is t1 + t2.
• According to maximum rule, the computation time denoted by max(t1, t2).

March 2023 DATA STRUCTURE AND ALGORITHM 65

How to Determine Complexities?
1. Sequence of statements
statement 1;
statement 2;
...
statement k;
• This is code that really is exactly k statements
• The total time is found by adding the times for all statements:
total time = time(statement 1) + time(statement 2) + ... + time(statement k)
• If each statement is “simple” (only involves basic operations) then the time for each
statement is constant and the total time is also constant: O(1).

March 2023 DATA STRUCTURE AND ALGORITHM 66

How to Determine Complexities?
2. if-then-else statements
if (cond) {
sequence of statements 1
}
else {
sequence of statements 2
}

• Here, either sequence 1 will execute, or sequence 2 will execute.

• Therefore, the worst-case time is the slowest of the two possibilities:
max(time(sequence 1), time(sequence 2)).
• For example, if sequence 1 is O(N) and sequence 2 is O(1) , then worst-case time
for the whole if-then-else statement would be O(N).

March 2023 DATA STRUCTURE AND ALGORITHM 67

How to Determine Complexities? Continue…

3. for loops

for (i = 0; i < N; i++) {

sequence of statements
}

• The loop executes N times, so the sequence of statements also executes N

times.
• Since we assume the statements are O(1), the total time for the for loop is N * O(1),
which is O(N) overall.

March 2023 DATA STRUCTURE AND ALGORITHM 68

How to Determine Complexities? Continue…

4. Nested loops
for (i = 0; i < N; i++) {
for (j = 0; j < M; j++) {
sequence of statements
}
}

• The outer loop executes N times. Every time the outer loop executes, the inner loop
executes M times. As a result, the statements in the inner loop execute a total of N *
M times.
• Thus, the complexity is O(N * M).

March 2023 DATA STRUCTURE AND ALGORITHM 69

How to Determine Complexities? Continue…

5. Statements with method calls:

• When a statement involves a method call, the complexity of the statement
includes the complexity of the method call.
• Assume that you know that method f takes constant time, and that method g
takes time proportional to (linear in) the value of its parameter k.
• Then the statements below have the time complexities indicated.
f(k); // O(1)
g(k); // O(k)
• When a loop is involved, the same rule applies. For example:
for (j = 0; j < N; j++) g(N);
has complexity (N2).
• The loop executes N times and each method call g(N) is complexity O(N).

March 2023 DATA STRUCTURE AND ALGORITHM 70

Examples - How to Determine Complexities?
1. What is the worst-case complexity of the each of the following code
fragments? Two loops in a row:

for (i = 0; i < N; i++) { Answer:

• The first loop is O(N) and the second loop is O(M).
sequence of statements
• Since you do not know which is bigger, you say this is
} O(N+M).
• This can also be written as O(max(N,M)).
for (j = 0; j < M; j++) { • In the case where the second loop goes to N instead
sequence of statements of M the complexity is O(N).
• You can see this from either expression above.
}
O(N+M) becomes O(2N) and when you drop the
constant it is O(N).
• O(max(N,M)) becomes O(max(N,N)) which is O(N).

March 2023 DATA STRUCTURE AND ALGORITHM 71

Examples - How to Determine Complexities? Continue…

2. How would the complexity change if the second loop went to N instead of
M? A nested loop followed by a non-nested loop:
for (i = 0; i < N; i++) {
for (j = 0; j < N; j++) { Answer:
sequence of statements 1. The first set of nested loops is O(N2)
} 2. The second loop is O(N).
} 3. This is O(max(N2,N)) which is O(N2).

for (k = 0; k < N; k++) {

sequence of statements
}

March 2023 DATA STRUCTURE AND ALGORITHM 72

Examples - How to Determine Complexities? Continue…

3. A nested loop in which the number of times the inner loop executes
depends on the value of the outer loop index:
Answer:
for (i = 0; i < N; i++) { • When i is 0 the inner loop executes N times.
for (j = i; j < N; j++) { • When i is 1 the inner loop executes N-1
sequence of statements times.
} • In the last iteration of the outer loop when i
} is N-1 the inner loop executes 1 time.
• The number of times the inner loop
statements execute is N + N-1 + ... + 2 +1.
• This sum is N(N+1)/2 and gives O(N2)

March 2023 DATA STRUCTURE AND ALGORITHM 73

Examples - How to Determine Complexities? Continue…

4. For each of the following loops with a method call, determine the overall
complexity. As above, assume that method f takes constant time, and
that method g takes time linear in the value of its parameter.
a) for (j = 0; j < N; j++) f(j);
b) for (j = 0; j < N; j++) g(j);
c) for (j = 0; j < N; j++) g(k);
Answer:
a) Each call to f(j) is O(1). The loop executes N times so it is N x O(1) or O(N).
b) The first time the loop executes j is 0 and g(0) takes “no operations.” The next time j is 1
and g(1) takes 1 operations. The last time the loop executes j is N-1 and g(N-1) takes
N-1 operations. The total work is the sum of the first N-1 numbers and is O(N2).
c) Each time through the loop g(k) takes k operations and the loop executes N times. Since
you do not know the relative size of k and N, the overall complexity is O(N x k).
March 2023 DATA STRUCTURE AND ALGORITHM 74
March 2023 DATA STRUCTURE AND ALGORITHM 75
Exercise
• Compute the complexity of the following fragment of algorithms?
a b sum = 0;
sum = 0;
for (i=0; i<3; i++) for (i=0; i<n*n; i++)
for (j=0; j<n; j++) sum++;
sum++;

C sum = 0; d for(i=1;i<=n;i++){
for (i = 0; i < n; i++){ for(j=1;j<=2*i;j++){
if (is_even(i)) { k=j;
for (j = 0; j < n; j++) while(k>=0)
sum++; k=k-1;
} }
else }
sum = sum + n;
}

March 2023 DATA STRUCTURE AND ALGORITHM 76