Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

JSPM University Pune

School of Computational Sciences/School of Basic and Applied Sciences
Master of Computer Science First Year
Sem I 2024-25
Data Structure and Algorithms
Notes of UNIT-I
 Introduction to Data Structures and Algorithms

1. What is Data?

- Definition: Data refers to raw facts and figures that can be processed or analyzed. It can be in
various forms, such as numbers, text, images, etc.

- Example: The number "42," the word "hello," or an image of a cat are all examples of data.

2. What is an Entity?

- Definition: An entity is an object that exists and is distinguishable from other objects. It can be
a person, place, thing, or concept.

- Example: A student, a book, or an employee can be considered entities in a system.

3. What is Information?

- Definition: Information is processed data that provides meaning or context. When data is
organized or structured in a way that makes it useful, it becomes information.

- Example: A student’s name and grades combined (like "John Doe: A, B, A") provide
information about the student's academic performance.

4. Importance of Data Structures

- Data structures are ways to organize and store data efficiently.

- They allow for effective data management and manipulation.

- Good data structures improve the performance of algorithms, making data retrieval and
processing faster.

5. Types of Data Structures

- Primitive Data Structures: Basic types that are built into programming languages (e.g., integers,
floats, characters).

Dr. A.A. Khan (Assistant Professor) 1|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

 Non-Primitive Data Structures: More complex types that are derived from primitive
types.

- Linear Data Structures:

- Arrays: A collection of elements identified by index or key.

- Linked Lists: A sequence of nodes, each containing data and a reference to the next
node.

- Stacks: A collection of elements that follows Last In First Out (LIFO) principle.

- Queues: A collection of elements that follows First In First Out (FIFO) principle.

 Non-linear Data Structures:

- Trees: A hierarchical structure with nodes connected by edges. Examples include

binary trees and binary search trees.

- Graphs: A collection of nodes (vertices) connected by edges. Useful for representing

networks.

6. Introduction to Algorithms

- Definition: An algorithm is a step-by-step procedure or formula for solving a problem.

- Importance: Efficient algorithms can significantly reduce the time complexity of data
processing tasks.

7. Common Algorithm Types

- Sorting Algorithms: Organize data in a specific order (e.g., Quick Sort, Merge Sort).

- Searching Algorithms: Find specific data within a data structure (e.g., Binary Search, Linear
Search).

- Recursive Algorithms: Solve problems by breaking them down into smaller subproblems (e.g.,
calculating factorial).

8. Complexity Analysis

- Time Complexity: Analyzes how the time to execute an algorithm changes with the input size.
Common notations include Big O (O(n)), which describes the worst-case scenario.

- Space Complexity: Analyzes how the memory requirements of an algorithm grow with the
input size.

Dr. A.A. Khan (Assistant Professor) 2|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

 Data vs. Information

1. Definition

- Data: Raw, unprocessed facts and figures without context. It can be numbers, text, symbols,
images, etc. Data alone doesn't convey meaning.

- Information: Processed data that has been organized, structured, or presented in a context that
makes it meaningful and useful. Information provides insights or knowledge.

2. Characteristics

- Data:
- Unorganized: Data is often unstructured and lacks context. For example, a list of
numbers (e.g., 10, 20, 30) is simply data.

- Raw Facts: It can be considered as the building blocks for creating information.

- Types: Can be qualitative (descriptive) or quantitative (numerical).

- Information:
- Organized: Information is organized and processed in a way that gives it meaning. For
example, "The scores of students in a class: John - 85, Mary - 90, Alex - 78."

- Contextualized: Information is data that has been analyzed and interpreted.

- Purposeful: Information is created for a specific purpose or decision-making process.

3. Examples

- Data:
- A collection of numbers: 100, 200, 300.

- The word "apple."

- A set of dates: 01/01/2020, 02/02/2021.

- Information:
- "The total sales for the first quarter are $600."

- "Apple is a type of fruit that is rich in vitamins."

- "The meeting is scheduled for February 2, 2021."

4. Processing

- Data: Requires processing to become information. This can involve:

Dr. A.A. Khan (Assistant Professor) 3|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Organization: Structuring data in a database or spreadsheet.

- Calculation: Performing mathematical operations to summarize or analyze.

- Interpretation: Drawing conclusions based on data analysis.

- Information: Often leads to further actions or decisions. It can be used to:

- Inform decisions: E.g., choosing a product based on reviews.

- Drive actions: E.g., creating a report based on sales data.

 Data Types

1. Definition of Data Type

A data type is a classification that specifies which type of value a variable can hold and what
kind of operations can be performed on it. Data types determine the size and layout of the
variable's memory, the range of values that can be stored, and the operations that can be
performed on the data.

2. Importance of Data Types

- Memory Management: Helps the compiler allocate the appropriate amount of memory.

- Type Safety: Ensures that operations are performed on compatible data types, reducing runtime
errors.

- Code Clarity: Makes code easier to understand and maintain by clearly indicating what kind of
data is being handled.

3. Types of Data Types

Data types can be broadly categorized into two main types:

A. Primitive Data Types

These are the basic building blocks of data manipulation in a programming language. They are
usually predefined and are not composed of other data types.

1. Integer: Represents whole numbers, both positive and negative (e.g., -3, 0, 42).

2. Float: Represents decimal numbers or floating-point numbers (e.g., 3.14, -0.001).

3. Character: Represents a single character (e.g., 'A', 'b', '7').

4. Boolean: Represents truth values, either `true` or `false`.

Dr. A.A. Khan (Assistant Professor) 4|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

B. Non-Primitive Data Types

These are more complex data types that are derived from primitive data types. They can be
constructed using primitive types and can hold multiple values.

1. Arrays: A collection of elements of the same data type, stored in contiguous memory
locations. E.g., `[1, 2, 3]` or `['a', 'b', 'c']`.

2. Strings: A sequence of characters (e.g., "Hello, World!"). In many languages, strings

are considered an array of characters.

3. Lists: An ordered collection that can contain elements of different data types (e.g., `[1,
"two", 3.0]`).

4. Dictionaries (or Maps): A collection of key-value pairs, where each key is unique (e.g.,
`{"name": "Alice", "age": 25}`).

5. Sets: A collection of unique elements, which can be unordered (e.g., `{1, 2, 3}`).

 Built-in Data Types

Built-in data types are those that are provided by a programming language as a fundamental part
of the language itself. Here are some commonly used built-in data types in various programming
languages:

C, C++

- Integers (`int`): E.g., `int x = 5;`

- Floating Point Numbers (`float`, `double`): E.g., `double y = 3.14;`

- Characters (`char`): E.g., `char initial = 'A';`

- Booleans (`bool`): E.g., `bool isActive = true;`

- Arrays: E.g., `int numbers[] = {1, 2, 3};`

- Strings: Provided by the C++ Standard Library as `std::string`.

- Vectors, Maps, and Sets: Part of the Standard Template Library (STL).

 Abstract Data Types (ADTs)

1. Definition of Abstract Data Type (ADT)

An Abstract Data Type (ADT) is a theoretical concept that defines a data type purely in terms of
its behavior (operations) and the properties of the data, rather than how it is implemented. It

Dr. A.A. Khan (Assistant Professor) 5|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

provides a high-level view of the data structure, allowing programmers to focus on what the data
type does instead of how it does it.

2. Characteristics of ADTs

- Encapsulation: ADTs encapsulate the data and the operations that can be performed on that
data, providing a clear separation between the interface and implementation.

- Operations: An ADT specifies a set of operations (methods) that can be performed on its data,
such as adding, removing, or accessing elements.

- Data Hiding: The implementation details of an ADT are hidden from the user, allowing
changes to be made to the implementation without affecting code that uses the ADT.

3. Benefits of Using ADTs

- Modularity: ADTs promote modularity in code, making it easier to develop and maintain
complex systems.

- Reusability: Once an ADT is defined and implemented, it can be reused in different programs
without needing to understand its implementation details.

- Abstraction: ADTs help simplify complex data structures by providing a clean interface for
users to interact with.

4. Common Examples of ADTs

Several standard data structures can be classified as ADTs, including:

 Stack

Definition: A stack is a collection of elements that follows the Last In First Out (LIFO)
principle.

Operations:

- `push(item)`: Add an item to the top of the stack.

- `pop()`: Remove and return the item from the top of the stack.

- `peek()`: Return the top item without removing it.

- `isEmpty()`: Check if the stack is empty.

 Queue

Dr. A.A. Khan (Assistant Professor) 6|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Definition: A queue is a collection of elements that follows the First In First Out (FIFO)
principle.

Operations:

- `enqueue(item)`: Add an item to the end of the queue.

- `dequeue()`: Remove and return the item from the front of the queue.

- `front()`: Return the front item without removing it.

- `isEmpty()`: Check if the queue is empty.

 List

- Definition: A list is an ordered collection of items that can be accessed by their position
(index).

Operations:

- `add(item)`: Add an item to the list.

- `remove(item)`: Remove a specified item from the list.

- `get(index)`: Retrieve the item at a specified index.

- `size()`: Return the number of items in the list.

- Dictionary (or Map)

- Definition: A dictionary is a collection of key-value pairs, where each key is unique.

Operations:

- `put(key, value)`: Add a key-value pair to the dictionary.

- `get(key)`: Retrieve the value associated with the specified key.

- `remove(key)`: Remove the key-value pair associated with the specified key.

- `containsKey(key)`: Check if a key exists in the dictionary.

5. Implementation of ADTs

While ADTs define what operations are available, the actual implementation can vary. For
example:

- A stack can be implemented using an array, a linked list, or other data structures.

Dr. A.A. Khan (Assistant Professor) 7|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- A queue can be implemented using arrays, linked lists, or circular buffers.

6. Real-World Analogy

Think of an ADT like a TV remote control:

- Interface: The buttons on the remote represent the operations (e.g., power on/off, volume
up/down).

- Implementation: The internal circuitry of the remote (how it sends signals to the TV) is hidden
from the user.

 Definition of Data Structures

1. What is a Data Structure?

A data structure is a specialized format for organizing, processing, storing, and retrieving data. It
defines a systematic way to manage data so that it can be accessed and modified efficiently. Data
structures provide a way to store multiple items of the same type together, enabling operations to
be performed on these items.

2. Importance of Data Structures

- Efficient Data Management: Data structures are essential for managing large amounts of data
and ensuring efficient access and modification.

- Performance Optimization: Choosing the right data structure can greatly impact the
performance of algorithms, including speed and memory usage.

- Facilitating Algorithms: Many algorithms depend on specific data structures to function

properly, making them crucial for effective problem-solving.

3. Key Characteristics of Data Structures

- Organization: Data structures organize data in a way that reflects the relationships and
operations needed.

- Access Methods: Different data structures allow various methods to access and manipulate
data.

- Memory Efficiency: Data structures can be designed to use memory more efficiently depending
on the application.

- Operations: Each data structure supports specific operations, such as insertion, deletion,
traversal, and searching.

4. Types of Data Structures

Dr. A.A. Khan (Assistant Professor) 8|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

Data structures can be broadly categorized into two main types: primitive and non-primitive data
structures.

A. Primitive Data Structures

- These are the basic building blocks provided by programming languages. They include:

- Integers: Whole numbers (e.g., 1, -5).

- Floats: Decimal numbers (e.g., 3.14).

- Characters: Single letters or symbols (e.g., 'A', 'b').

- Booleans: True or false values.

B. Non-Primitive Data Structures

- These are more complex data structures built from primitive types. They include:

- Arrays: A collection of elements of the same type, accessed by index. (e.g., `[1, 2, 3]`)

- Linked Lists: A sequence of nodes, each containing data and a reference to the next
node.

- Stacks: A collection of elements following the Last In First Out (LIFO) principle. (e.g.,
push and pop operations)

- Queues: A collection of elements following the First In First Out (FIFO) principle.
(e.g., enqueue and dequeue operations)

- Trees: Hierarchical structures consisting of nodes connected by edges. (e.g., binary

trees, AVL trees)

- Graphs: Collections of nodes connected by edges, useful for representing networks.

(e.g., social networks, transportation systems)

- Hash Tables: Data structures that use a hash function to map keys to values for
efficient data retrieval.

5. Applications of Data Structures

- Databases: Data structures like trees and hash tables are used for indexing and querying data
efficiently.

- Web Development: Stacks and queues can manage requests and user sessions.

Dr. A.A. Khan (Assistant Professor) 9|Page

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Artificial Intelligence: Graphs represent relationships in data, such as social networks or routing
paths.

- Game Development: Data structures help manage game state and entities efficiently.

Sure! Here’s a detailed overview of the types of data structures, specifically focusing on linear
and non-linear data structures, designed for beginners in data structures and algorithms.

 Types of Data Structures

1. Linear Data Structures

Definition

Linear data structures are collections of data elements arranged in a sequential manner, where
each element is connected to its previous and next element. They allow for a single level of data
organization.

Characteristics

- Sequential Access: Elements can be accessed sequentially.

- Fixed Size: The size of linear data structures is often predetermined (like arrays).

- Memory Allocation: Linear data structures can be stored in contiguous memory

locations (like arrays) or non-contiguous locations (like linked lists).

Common Types of Linear Data Structures

A. Arrays

- Description: A collection of elements of the same type, stored in contiguous memory locations.
Elements can be accessed using indices.

- Operations:

- Access: O(1) time complexity.

- Insertion/Deletion: O(n) time complexity (in the worst case).

- Example: `int arr[5] = {1, 2, 3, 4, 5};`

B. Linked Lists

- Description: A sequence of nodes, where each node contains data and a reference (or pointer)
to the next node. Linked lists can be singly linked (one direction) or doubly linked (two
directions).

Dr. A.A. Khan (Assistant Professor) 10 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Operations:

- Access: O(n) time complexity.

- Insertion/Deletion: O(1) time complexity (if the position is known).

- Example: A linked list of integers: `1 -> 2 -> 3 -> 4 -> null`.

C. Stacks

- Description: A collection of elements that follows the Last In First Out (LIFO) principle. The
last element added is the first to be removed.

- Operations:

- Push: Add an element to the top.

- Pop: Remove the top element.

- Peek: View the top element.

- Example: Consider a stack with elements: `top -> 3, 2, 1`.

D. Queues

- Description: A collection of elements that follows the First In First Out (FIFO) principle. The
first element added is the first to be removed.

- Operations:

- Enqueue: Add an element to the end.

- Dequeue: Remove the front element.

- Front: View the front element.

- Example: Consider a queue with elements: `front -> 1, 2, 3 -> back`.

2. Non-Linear Data Structures

Definition

Non-linear data structures are collections of data elements that do not have a sequential
arrangement. Instead, the data can be connected in a hierarchical or interconnected way.

Characteristics

- Hierarchical Structure: Data can be organized in a tree-like or graph-like structure.

Dr. A.A. Khan (Assistant Professor) 11 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Complex Relationships: Elements can have multiple connections or relationships, making

traversal more complex.

- Dynamic Size: Non-linear data structures can grow or shrink as needed.

Common Types of Non-Linear Data Structures

A. Trees

- Description: A hierarchical structure made up of nodes, where each node contains data and
references to its child nodes. The top node is called the root, and nodes without children are
called leaves.

- Types:

- Binary Tree: Each node has at most two children.

- Binary Search Tree (BST): A binary tree where the left child is less than the parent
node, and the right child is greater.

- AVL Tree: A self-balancing binary search tree.

- Operations:

- Insertion, deletion, and searching can vary in time complexity (average case O(log n)
for BST).

B. Graphs

- Description: A collection of nodes (vertices) connected by edges. Graphs can represent

complex relationships like social networks or pathways.

Types:

- Directed Graph: Edges have a direction (one-way).

- Undirected Graph: Edges have no direction (two-way).

- Weighted Graph: Edges have weights (costs) associated with them.

Operations:

- Traversing (using algorithms like Depth-First Search or Breadth-First Search).

- Finding the shortest path (using algorithms like Dijkstra's or Bellman-Ford).

 Definition of Algorithms

Dr. A.A. Khan (Assistant Professor) 12 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

1. What is an Algorithm?

An algorithm is a finite sequence of well-defined, step-by-step instructions or rules designed to

perform a specific task or solve a particular problem. Algorithms are fundamental to
programming and computer science, as they provide a systematic approach to problem-solving.

Sure! Here’s a detailed overview of the differences between algorithms and programs, tailored
for beginners in data structures and algorithms.

 Difference Between Algorithms and Programs

1. Definition

- Algorithm:

- An algorithm is a finite set of well-defined steps or rules designed to perform a specific task or
solve a particular problem.

- It focuses on the what and how of problem-solving without being tied to any specific
programming language.

- Program:

- A program is a collection of instructions written in a programming language that tells a

computer how to perform a specific task.

- It is the implementation of one or more algorithms in code that a computer can execute.

2. Nature

- Algorithm:

- Abstract: An algorithm is a conceptual framework that describes the logic and steps required
to solve a problem.

- Language-Independent: Algorithms can be expressed in natural language, pseudocode, or

flowcharts without regard to syntax of programming languages.

- Program:

- Concrete: A program is a specific realization of an algorithm that can be executed by a

computer.

- Language-Specific: Programs must be written in a specific programming language (e.g.,

Python, Java, C++), adhering to the syntax and semantics of that language.

3. Purpose

Dr. A.A. Khan (Assistant Professor) 13 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Algorithm:

- Designed to solve a problem or perform a task efficiently.

- Focuses on the logic, structure, and methodology for problem-solving.

- Program:

- Designed to run on a computer to perform the tasks specified by the algorithm.

- Includes not only the algorithm but also the necessary elements like data input/output, error
handling, and user interaction.

4. Components

- Algorithm:

- Comprised of a sequence of steps or rules that dictate the process of solving a problem.

- Examples of components include operations, conditions, and loops.

- Program:

- Comprised of variables, data types, control structures (if statements, loops), functions, and
libraries or APIs.

- It includes not only the logic of algorithms but also code structure and syntax.

5. Execution

- Algorithm:

- An algorithm itself cannot be executed; it is a plan or a set of instructions that must be

implemented in code to be used.

- Program:

- A program is executed by the computer's processor, performing the tasks defined by the
underlying algorithms.

6. Examples

- Algorithm:

- Example: A simple algorithm for finding the maximum number in an array might include:

1. Initialize a variable `max` to the first element of the array.

2. Loop through the array.

Dr. A.A. Khan (Assistant Professor) 14 | P a g e
Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

3. If the current element is greater than `max`, update `max`.

4. At the end of the loop, `max` contains the maximum value.

- Program:

- Example: The same maximum-finding algorithm implemented in Python:

```python

def find_max(arr):

max = arr[0]

for num in arr:

if num > max:

max = num

return max

numbers = [3, 5, 2, 8, 1]

print(find_max(numbers)) Output: 8

7. Reusability

- Algorithm:

- Algorithms can often be reused in different programs and can be adapted for various
problems.

- They provide a blueprint for solving similar types of problems.

- Program:

- Programs may contain specific implementations that are not reusable across different contexts
without modification.

- While components of a program can be modularized (e.g., functions), the overall program is
typically designed for a specific task.

 Characteristics/Properties of Algorithms

An algorithm is a well-defined procedure that takes some input and produces an output after a
finite number of steps. To effectively analyze and design algorithms, it's crucial to understand
their core properties:

Dr. A.A. Khan (Assistant Professor) 15 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

1. Correctness
- Definition: An algorithm is correct if it produces the expected result for all valid inputs.
- Verification: It involves proving that the algorithm terminates and that its output meets the
specification.
- Types of Correctness:
- Partial Correctness: The algorithm produces correct output if it terminates.
- Total Correctness: The algorithm is both partially correct and guarantees termination.
2. Efficiency
- Definition: Efficiency measures how well an algorithm utilizes resources, such as time and
space.
- Time Complexity: How the running time of the algorithm scales with the input size.
- Space Complexity: How the memory usage of the algorithm scales with the input size.
3. Finiteness
- Definition: An algorithm must terminate after a finite number of steps.
- Implication: Infinite loops or unbounded recursion indicate that an algorithm is not finite.
4. Definiteness
- Definition: Each step of an algorithm must be precisely defined, leaving no ambiguity.
- Implication: Every operation must be clear and unambiguous, ensuring that the algorithm can
be executed as intended.
5. Input and Output
- Input: An algorithm should take zero or more inputs.
- Output: It should produce at least one output based on the inputs.
6. Generalizability
- Definition: An algorithm should be applicable to a wide range of problems within its scope.
- Implication: It should be designed to handle different inputs of varying sizes and types.

 Importance of Algorithms

- Problem Solving: Algorithms provide a clear methodology for solving problems efficiently.

- Performance Optimization: A well-designed algorithm can significantly improve the

performance of a program, affecting speed and resource usage.

- Reusability: Once an algorithm is developed, it can be reused in different applications or

systems.

 Types of Algorithms

Algorithms can be categorized based on their characteristics or the problems they address:

- Sorting Algorithms: Used to arrange data in a particular order (e.g., Quick Sort, Merge Sort,
Bubble Sort).

Dr. A.A. Khan (Assistant Professor) 16 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Searching Algorithms: Used to find specific data within a data structure (e.g., Linear Search,
Binary Search).

- Graph Algorithms: Used to solve problems related to graph structures (e.g., Dijkstra’s
Algorithm for shortest paths, Depth-First Search).

- Dynamic Programming: A method for solving complex problems by breaking them down into
simpler subproblems (e.g., Fibonacci sequence, Knapsack problem).

- Greedy Algorithms: Algorithms that make the locally optimal choice at each step with the hope
of finding a global optimum (e.g., Prim’s and Kruskal’s algorithms for minimum spanning trees).

 Algorithm Design Techniques

Various design techniques help in creating efficient and effective algorithms. Here are some
fundamental techniques:
1. Divide and Conquer
- Definition: Divide and conquer is a powerful algorithm design paradigm that involves breaking
a problem into smaller subproblems, solving each subproblem independently, and then
combining the solutions to solve the original problem.
- Steps:
- Divide: Split the problem into smaller subproblems.
- Conquer: Solve the subproblems recursively. If they are small enough, solve them
directly.
- Combine: Merge the solutions of subproblems.
- Examples:

 Merge Sort: Sorts an array by dividing it into two halves, recursively sorting each half, and
merging the sorted halves.

 Quick Sort: Selects a pivot element, partitions the array into elements less than and greater
than the pivot, and recursively sorts the partitions.

 Binary Search: Searches for an element in a sorted array by dividing the array in half and
eliminating one half. - Advantages: Often leads to efficient algorithms with good time
complexity.

2. Dynamic Programming (DP)

- Definition: Dynamic programming (DP) is a method for solving complex problems by breaking
them down into simpler overlapping subproblems and storing the results to avoid redundant
computations.
- Key Concepts:
- Overlapping Subproblems: The same subproblems are solved multiple times.

Dr. A.A. Khan (Assistant Professor) 17 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Optimal Substructure: The solution to a problem can be constructed from solutions to

its subproblems.
Steps

1. Identify the subproblems and their relationships.

2. Define the state: Represent the solution of subproblems with states and create a recursive
relation.
3. Memoization or Tabulation: Store the results of computed states to avoid
recomputation.

- Approaches:
- Top-Down (Memoization): Recursively solve subproblems and store results.
- Bottom-Up (Tabulation): Solve subproblems iteratively and build up to the solution.
- Examples:

 Fibonacci Sequence: Instead of recalculating Fibonacci numbers repeatedly, store previously

calculated values.

 Knapsack Problem: Determines the maximum value that can be carried in a knapsack given
weight constraints.
 Longest Common Subsequence: Finds the longest sequence that can appear in the same
order in two sequences.
3. Greedy Algorithms
- Definition: Greedy algorithms build up a solution piece by piece, always choosing the next
piece that offers the most immediate benefit (the locally optimal choice) without regard for the
global situation.
- Characteristics:
- Greedy Choice Property: A local optimum choice leads to a global optimum.
- Optimal Substructure: The global optimum can be obtained by combining local
optima.
Steps

1. Choose: Select the best option available at that moment.

2. Feasibility: Check if the choice is valid and maintains the constraints of the problem.
3. Solution: Repeat until a complete solution is constructed.

- Examples:

 Kruskal’s Algorithm: Finds the minimum spanning tree for a connected graph by selecting
the edges with the smallest weights.

Dr. A.A. Khan (Assistant Professor) 18 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

 Prim’s Algorithm: Also finds the minimum spanning tree by growing the tree one edge at a
time, starting from an arbitrary vertex.

 Huffman Coding: Constructs a variable-length code based on character frequencies to

minimize data storage. - Advantages: Often simpler and faster but may not always find the
optimal solution for every problem.
4. Backtracking
- Definition: Backtracking is a systematic way to explore all potential solutions to a problem by
incrementally building candidates and abandoning those that fail to satisfy the constraints of the
problem.
Steps

1. Choose: Make a choice that leads to a possible solution.

2. Explore: Move forward to see if this choice leads to a valid solution.
3. Backtrack: If the choice does not lead to a solution, undo the last choice and try the next
option.

- Examples:

 N-Queens Problem: Places N queens on an N×N chessboard so that no two queens threaten
each other.

 Sudoku Solver: Fills in the grid according to Sudoku rules, backtracking when an invalid state
is reached.

 Subset Sum Problem: Determines if a subset of numbers can sum up to a specific target. -
Advantages: Useful for problems involving constraints and solutions with multiple possible
configurations.
5. Branch and Bound
- Definition: Branch and bound is an algorithm design technique for solving optimization
problems by systematically exploring branches of a solution space. It is often used for problems
that involve finding the best solution among many possibilities.
Characteristics

 Involves exploring branches, pruning those that cannot yield a better solution than the
best found so far.
 Uses bounds to eliminate branches that cannot possibly yield an optimal solution.

- Components:
- Branching: Divide the problem into smaller subproblems.
- Bounding: Use bounds to determine if a subproblem can lead to a better solution.
- Pruning: Eliminate subproblems that do not need to be explored further.

Dr. A.A. Khan (Assistant Professor) 19 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Examples:

 Traveling Salesman Problem: Determines the shortest possible route that visits each city
exactly once and returns to the origin city.

 Integer Linear Programming: Solves problems that require finding integer solutions to
linear programming problems.
- Advantages: Can reduce the computational complexity by eliminating non-promising solutions
early.
6. Iterative Improvement
- Definition: Start with an initial solution and iteratively improve it based on a heuristic.
- Techniques:
- Local Search: Explore neighboring solutions to find a better one.
- Simulated Annealing: Use a probabilistic approach to avoid getting stuck in local
optima.
- Examples: Hill Climbing, Genetic Algorithms, Simulated Annealing.
- Advantages: Useful for optimization problems where finding an exact solution is
computationally infeasible.
Certainly! Here’s a detailed overview of performance analysis of algorithms, designed for
beginners in data structures and algorithms.
 Performance Analysis of Algorithms
1. Analyze Algorithm Performance
- Efficiency: To compare different algorithms and choose the most efficient one for a specific
task.
- Resource Management: To understand how much time and memory the algorithm requires,
which is crucial in resource-constrained environments.
- Scalability: To assess how an algorithm's performance changes with increasing input sizes.

2. Key Metrics for Performance Analysis

A. Time Complexity
-Time complexity is a measure of the time an algorithm takes to complete as a function of the
size of the input. It helps in estimating the execution time of an algorithm.
- Notation: Time complexity is typically expressed using Big O notation (O), which describes the
upper limit of the running time.
- Classes of Time Complexity:
- Constant Time: O(1) - The execution time does not change regardless of input size.
- Logarithmic Time: O(log n) - The execution time grows logarithmically as input size
increases (e.g., binary search).
- Linear Time: O(n) - The execution time grows linearly with the input size (e.g.,
traversing an array).

Dr. A.A. Khan (Assistant Professor) 20 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Linearithmic Time: O(n log n) - Common in efficient sorting algorithms like

mergesort and heapsort.
- Quadratic Time: O(n²) - The execution time grows quadratically (e.g., bubble sort,
selection sort).
- Exponential Time: O(2^n) - The execution time grows exponentially (e.g., solving the
traveling salesman problem with brute force).

B. Space Complexity
Space complexity measures the amount of memory an algorithm uses relative to the size of the
input. It is also expressed in Big O notation.
- Components of Space Complexity:
- Fixed Part: The space required for constants, simple variables, fixed-size variables, etc.
- Variable Part: The space required for dynamically allocated memory, recursive stack
space, etc.
- Classes of Space Complexity:
- O(1): Constant space usage.
- O(n): Linear space usage (e.g., using an array to store data).
- O(n²): Quadratic space usage (e.g., a two-dimensional array).

3. Analyzing Time Complexity

A. Best Case, Average Case, Worst Case
- Best Case: The minimum time required for the algorithm to complete (e.g., finding an
element at the beginning).
- Average Case: The expected time for all possible inputs.
- Worst Case: The maximum time required for the algorithm to complete (e.g., finding an
element at the end or not found).
B. Examples
1. Linear Search:
- Best Case: O(1) (element found at the first position)
- Average Case: O(n) (element found in the middle)
- Worst Case: O(n) (element not found)

2. Binary Search:
- Best Case: O(1) (element found in the middle)
- Average Case: O(log n)
- Worst Case: O(log n)

4. Empirical Analysis
In addition to theoretical analysis, empirical analysis involves measuring the actual running time
of algorithms for given inputs. This can be done by:

Dr. A.A. Khan (Assistant Professor) 21 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Running the algorithm with different input sizes.

- Timing how long the algorithm takes to execute.
- Plotting the results to visualize performance.

5. Trade-offs
Sometimes, an algorithm may be faster but use more memory, or vice versa. It's important to
consider the trade-offs based on the constraints of the specific application.
- Time vs. Space Trade-off: Using more memory to store precomputed values can reduce
execution time (e.g., memoization in dynamic programming).

Pseudocode

- Pseudocode is an informal way to express an algorithm using a combination of natural

language and programming constructs. It allows programmers to plan and communicate their
ideas without worrying about syntax. For example:

Algorithm: FindMax

Input: Array of numbers

Output: Maximum number in the array

1. Initialize max to the first element of the array

2. For each number in the array

a. If the number is greater than max

i. Set max to the number

3. Return max

 Complexity of Various Code Structures

1. Basic Code Structures

A. Constant Time Complexity: O(1)

- Description: The execution time is constant and does not change with the size of the input.

- Example: Accessing an element in an array by index.

``` C Program

include <stdio.h>

int get_first_element(int arr[], int size) {

Dr. A.A. Khan (Assistant Professor) 22 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

return arr[0]; // O(1)

int main() {

int arr[] = {1, 2, 3, 4, 5};

printf("First element: %d\n", get_first_element(arr, 5));

return 0;

} ```

B. Linear Time Complexity: O(n)

- Description: The execution time grows linearly with the input size.

- Example: Traversing through an array to find a specific element.

C Program

include <stdio.h>

include <stdbool.h>

bool find_element(int arr[], int size, int target) {

for (int i = 0; i < size; i++) {

if (arr[i] == target) {

return true; // O(n)

return false;

int main() {

int arr[] = {1, 2, 3, 4, 5};

int target = 3;

if (find_element(arr, 5, target)) {

Dr. A.A. Khan (Assistant Professor) 23 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

printf("Element found.\n");

} else {

printf("Element not found.\n");

return 0;

C. Quadratic Time Complexity: O(n²)

- Description: The execution time grows quadratically as the input size increases, often seen in
nested loops.

- Example: Bubble sort or selection sort.

C Program

include <stdio.h>

void bubble_sort(int arr[], int size) {

for (int i = 0; i < size - 1; i++) {

for (int j = 0; j < size - i - 1; j++) {

if (arr[j] > arr[j + 1]) {

// Swap arr[j] and arr[j + 1]

int temp = arr[j];

arr[j] = arr[j + 1];

arr[j + 1] = temp; // O(n²)

int main() {

Dr. A.A. Khan (Assistant Professor) 24 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

int arr[] = {5, 2, 9, 1, 5, 6};

int size = sizeof(arr) / sizeof(arr[0]);

bubble_sort(arr, size);

printf("Sorted array: ");

for (int i = 0; i < size; i++) {

printf("%d ", arr[i]);

return 0;

4. Logarithmic Time Complexity: O(log n)

- Description: The execution time increases logarithmically as the input size increases. This is
common in algorithms that divide the input size in each step, such as binary search.

- Example: Binary Search

C Program

include <stdio.h>

int binary_search(int arr[], int size, int target) {

int left = 0, right = size - 1;

while (left <= right) {

int mid = (left + right) / 2;

if (arr[mid] == target) {

return mid; // O(log n)

} else if (arr[mid] < target) {

left = mid + 1;

} else {

right = mid - 1;

Dr. A.A. Khan (Assistant Professor) 25 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

return -1; // Not found

int main() {

int arr[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

int size = sizeof(arr) / sizeof(arr[0]);

int target = 7;

int result = binary_search(arr, size, target);

if (result != -1) {

printf("Element found at index: %d\n", result);

} else {

printf("Element not found.\n");

return 0;

3. Linearithmic Time Complexity: O(n log n)

- Description: The execution time grows in proportion to \( n \) times the logarithm of \( n \).
Common in efficient sorting algorithms.

- Example: Merge sort

include <stdio.h>

void merge(int arr[], int left, int mid, int right) {

int n1 = mid - left + 1;

int n2 = right - mid;

int L[n1], R[n2];

Dr. A.A. Khan (Assistant Professor) 26 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

for (int i = 0; i < n1; i++)

L[i] = arr[left + i];

for (int j = 0; j < n2; j++)

R[j] = arr[mid + 1 + j];

int i = 0, j = 0, k = left;

while (i < n1 && j < n2) {

if (L[i] <= R[j]) {

arr[k] = L[i];

i++;

} else {

arr[k] = R[j];

j++;

k++;

while (i < n1) {

arr[k] = L[i];

i++;

k++;

while (j < n2) {

arr[k] = R[j];

j++;

k++;

Dr. A.A. Khan (Assistant Professor) 27 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

void merge_sort(int arr[], int left, int right) {

if (left < right) {

int mid = left + (right - left) / 2;

merge_sort(arr, left, mid);

merge_sort(arr, mid + 1, right);

merge(arr, left, mid, right); // O(n log n)

int main() {

int arr[] = {12, 11, 13, 5, 6, 7};

int size = sizeof(arr) / sizeof(arr[0]);

merge_sort(arr, 0, size - 1);

printf("Sorted array: ");

for (int i = 0; i < size; i++) {

printf("%d ", arr[i]);

return 0;

4. Exponential Time Complexity: O(2^n)

- Description: The execution time grows exponentially with the input size, making algorithms
impractical for large inputs.

- Example: Recursive algorithms that solve problems by exploring all possible combinations
(e.g., Fibonacci sequence without memoization).

include <stdio.h>

int fibonacci(int n) {

Dr. A.A. Khan (Assistant Professor) 28 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

if (n <= 1) {

return n; // O(2^n)

return fibonacci(n - 1) + fibonacci(n - 2);

int main() {

int n = 5;

printf("Fibonacci of %d is: %d\n", n, fibonacci(n));

return 0;

5. Space Complexity

A. Constant Space Complexity: O(1)

- Description: The algorithm requires a fixed amount of space regardless of the input size.

- Example: Swapping two variables.

include <stdio.h>

void swap(int *a, int *b) {

int temp = *a;

*a = *b; // O(1)

*b = temp;

int main() {

int x = 5, y = 10;

swap(&x, &y);

printf("x: %d, y: %d\n", x, y);

return 0;

Dr. A.A. Khan (Assistant Professor) 29 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

B. Linear Space Complexity: O(n)

- Description: The space required grows linearly with the input size.

- Example: Creating a copy of an array.

include <stdio.h>

void create_copy(int arr[], int size) {

int new_arr[size]; // O(n)

for (int i = 0; i < size; i++) {

new_arr[i] = arr[i];

printf("Copy created.\n");

int main() {

int arr[] = {1, 2, 3, 4, 5};

create_copy(arr, 5);

return 0;

C. Recursive Space Complexity

- Description: Recursive algorithms consume stack space for each recursive call.

- Example: Recursive factorial function.

include <stdio.h>

int factorial(int n) {

if (n == 0) {

return 1; // O(n) space due to recursive calls

Dr. A.A. Khan (Assistant Professor) 30 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

return n * factorial(n - 1);

int main() {

int n = 5;

printf("Factorial of %d is: %d\n", n, factorial(n));

return 0;

 Order of Growth

1. Importance of Order of Growth

- Performance Prediction: Helps in predicting how the algorithm will perform with larger
datasets.

- Comparison: Enables comparison between different algorithms, guiding the selection of

the most efficient one for a particular problem.

- Simplicity: Provides a simplified way to express the complexity without getting bogged
down in constant factors and lower-order terms.

2. Common Orders of Growth

The growth rates can be expressed in Big O notation, which classifies algorithms based
on their worst-case scenario performance. Here are some common orders of growth:

A. Constant Time: O(1)

- Description: The running time remains constant regardless of the input size.

- Example: Accessing an element in an array by index.

```c

int get_first_element(int arr[], int size) {

return arr[0]; // O(1)

```

B. Logarithmic Time: O(log n)

Dr. A.A. Khan (Assistant Professor) 31 | P a g e
Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Description: The running time increases logarithmically as the input size increases. This
often occurs in algorithms that halve the input size at each step, like binary search.

- Example: Binary search in a sorted array.

```c

int binary_search(int arr[], int size, int target) {

int left = 0, right = size - 1;

while (left <= right) {

int mid = (left + right) / 2;

if (arr[mid] == target) return mid; // O(log n)

else if (arr[mid] < target) left = mid + 1;

else right = mid - 1;

return -1;

```

C. Linear Time: O(n)

- Description: The running time grows linearly with the size of the input. This is common
in simple iterations over data structures like arrays or lists.

- Example: Finding an element in an unsorted array.

```c

bool find_element(int arr[], int size, int target) {

for (int i = 0; i < size; i++) {

if (arr[i] == target) return true; // O(n)

return false;

Dr. A.A. Khan (Assistant Professor) 32 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

```

D. Linearithmic Time: O(n log n)

- Description: The running time grows in proportion to \( n \) times the logarithm of \( n

\). This is typical of efficient sorting algorithms.

- Example: Merge sort.

```c

void merge_sort(int arr[], int left, int right) {

// Implementation of merge sort

// O(n log n)

```

E. Quadratic Time: O(n²)

- Description: The running time grows quadratically with the size of the input, often due
to nested loops.

- Example: Bubble sort.

```c

void bubble_sort(int arr[], int size) {

for (int i = 0; i < size - 1; i++) {

for (int j = 0; j < size - i - 1; j++) {

if (arr[j] > arr[j + 1]) {

// Swap

} // O(n²)

```

Dr. A.A. Khan (Assistant Professor) 33 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

F. Exponential Time: O(2^n)

- Description: The running time doubles with each additional input element, making it
impractical for large inputs.

- Example: Recursive algorithms like the naive Fibonacci function.

```c

int fibonacci(int n) {

if (n <= 1) return n; // O(2^n)

return fibonacci(n - 1) + fibonacci(n - 2);

```

3. Comparing Orders of Growth

When comparing algorithms, it's crucial to focus on the dominant term in their growth
rates, especially as the input size becomes large. Here’s a summary of growth rates from
fastest to slowest:

1. O(1) - Constant time

2. O(log n) - Logarithmic time

3. O(n) - Linear time

4. O(n log n) - Linearithmic time

5. O(n²) - Quadratic time

6. O(2^n) - Exponential time

4. Practical Considerations

- Constant Factors:

While Big O notation abstracts away constant factors, they can significantly affect
performance in practice.

 Asymptotic Notations

1. Purpose of Asymptotic Notations

Dr. A.A. Khan (Assistant Professor) 34 | P a g e
Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

- Simplification: Focuses on the growth rate of functions, ignoring constant factors and
lower-order terms.

- Comparison: Allows comparison of algorithms based on their efficiency in handling

large inputs.

- Generalization: Provides a standard way to analyze performance across different

contexts and implementations.

2. Types of Asymptotic Notations

A. Big O Notation (O)

Purpose: Big O notation helps to analyze the efficiency of algorithms, allowing

developers to predict how they will perform as the data set size increases.

Notation: It expresses the upper bound of an algorithm's growth rate, focusing on the
term that grows the fastest as input size increases, while ignoring constants and lower-
order terms.

or

- Definition: Represents the upper bound of an algorithm’s running time or space

complexity. It describes the worst-case scenario.

- Mathematical Representation:

- T(n) = O(f(n)) means there exist constants c > 0 and n_0 such that T(n) eq c.f(n) for
all \( n \geq n_0 \).

- Example:

- For a linear search, the worst-case time complexity is O(n) since you may need to
check each element.

B. Big Omega Notation (Ω)

- Definition: Represents the lower bound of an algorithm’s running time or space

complexity. It describes the best-case scenario.

- Mathematical Representation:

- T(n)=Ω(f(n))means there exist constants c > 0 and n0 such that T(n)≥c⋅f(n) for all
n≥n0.- Example:

- In a linear search, the best-case time complexity is Ω(1), which occurs when the target
element is the first element in the array.

Dr. A.A. Khan (Assistant Professor) 35 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

C. Big Theta Notation (Θ)

- Definition: Represents a tight bound on an algorithm’s running time or space

complexity. It provides both upper and lower bounds.

- Mathematical Representation:

- T(n)=Θ(f(n)) means there exist constants c1,c2>0 and n0 such that c1⋅f(n)≤T(n)≤c2⋅f(n) for
all n≥n0.

- Example:

- For an algorithm that runs in linear time, such as finding the maximum element in an
array, we can say T(n)=Θ(n).

3. Common Functions in Asymptotic Analysis

Function Big O Notation Big Omega Big Theta Notation

Notation
Constant O(1) Ω(1) Θ(1)

Logarithmic O(log n) Ω(log n) Θ(log n)

Linear O(n) Ω(n) Θ(n)

Linearithmic O(n log n) Ω(n log n) Θ(n log n)

Quadratic O(n²) Ω(n²) Θ(n²)

Cubic O(n³) Ω(n³) Θ(n³)

Exponential O(2^n) Ω(2^n) Θ(2^n)

4. Example Analysis

Let’s analyze a simple algorithm using asymptotic notations.

Example: Finding Maximum in an Array

```c

include <stdio.h>

int find_max(int arr[], int size) {

Dr. A.A. Khan (Assistant Professor) 36 | P a g e

Data Structure and Algorithms FY MCA/MSc (Sem I) JSPM University Pune

int max = arr[0]; // O(1)

for (int i = 1; i < size; i++) {

if (arr[i] > max) {

max = arr[i]; // O(n)

return max; // Overall: O(n)

```

- Best Case: If the maximum element is the first element, the time complexity is Ω(1)

-Worst Case: The time complexity is O(n) since we might have to check all elements.

- Tight Bound: The time complexity is Θ(n).

Dr. A.A. Khan (Assistant Professor) 37 | P a g e