Knapsack Problem

The knapsack problem states that − given a set of items, holding weights and profit values,
one must determine the subset of the items to be added in a knapsack such that, the total
weight of the items must not exceed the limit of the knapsack and its total profit value is
maximum.

It is one of the most popular problems that take greedy approach to be solved. It is called
as the Fractional Knapsack Problem.

To explain this problem a little easier, consider a test with 12 questions, 10 marks each, out
of which only 10 should be attempted to get the maximum mark of 100. The test taker now
must calculate the highest profitable questions the one that hes confident in to achieve the
maximum mark. However, he cannot attempt all the 12 questions since there will not be
any extra marks awarded for those attempted answers. This is the most basic real-world
application of the knapsack problem.

Knapsack Algorithm

The weights (Wi) and profit values (Pi) of the items to be added in the knapsack are taken as
an input for the fractional knapsack algorithm and the subset of the items added in the
knapsack without exceeding the limit and with maximum profit is achieved as the output.

Algorithm

• Consider all the items with their weights and profits mentioned respectively.

• Calculate Pi/Wi of all the items and sort the items in descending order based on
their Pi/Wi values.

• Without exceeding the limit, add the items into the knapsack.

• If the knapsack can still store some weight, but the weights of other items exceed
the limit, the fractional part of the next time can be added.

• Hence, giving it the name fractional knapsack problem.

Examples

• For the given set of items and the knapsack capacity of 10 kg, find the subset of the
items to be added in the knapsack such that the profit is maximum.
 Items 1 2 3 4 5

Weights (in kg) 3 3 2 5 1

Profits 10 15 10 12 8

Solution

Step 1

Given, n = 5

Wi = {3, 3, 2, 5, 1}

Pi = {10, 15, 10, 12, 8}

Calculate Pi/Wi for all the items

Items 1 2 3 4 5

Weights (in kg) 3 3 2 5 1

Profits 10 15 10 20 8

Pi/Wi 3.3 5 5 4 8

Step 2

Arrange all the items in descending order based on Pi/Wi

Items 5 2 3 4 1

Weights (in kg) 1 3 2 5 3

Profits 8 15 10 20 10

Pi/Wi 8 5 5 4 3.3

Step 3
Without exceeding the knapsack capacity, insert the items in the knapsack with maximum
profit.

Knapsack = {5, 2, 3}

However, the knapsack can still hold 4 kg weight, but the next item having 5 kg weight will
exceed the capacity. Therefore, only 4 kg weight of the 5 kg will be added in the knapsack.

Items 5 2 3 4 1

Weights (in kg) 1 3 2 5 3

Profits 8 15 10 20 10

Knapsack 1 1 1 4/5 0

Hence, the knapsack holds the weights = [(1 * 1) + (1 * 3) + (1 * 2) + (4/5 * 5)] = 10, with
maximum profit of [(1 * 8) + (1 * 15) + (1 * 10) + (4/5 * 20)] = 37.

Optimal Binary Search Tree

A set of integers are given in the sorted order and another array freq to frequency count.
Our task is to create a binary search tree with those data to find the minimum cost for all
searches.

An auxiliary array cost[n, n] is created to solve and store the solution of subproblems. Cost
matrix will hold the data to solve the problem in a bottom-up manner.

Input and Output

Input:

The key values as node and the frequency.

Keys = {10, 12, 20}

Frequency = {34, 8, 50}

Output:

The minimum cost is 142.

These are possible BST from the given values.

 For
case 1, the cost is: (34*1) + (8*2) + (50*3) = 200

For case 2, the cost is: (8*1) + (34*2) + (50*2) = 176.

Similarly for case 5, the cost is: (50*1) + (34 * 2) + (8 * 3) = 142 (Minimum)

Algorithm

optCostBst(keys, freq, n)

Input: Keys to insert in BST, the frequency for each key, number of keys.

Output: Minimum cost to make optimal BST.

Begin

define cost matrix of size n x n

for i in range 0 to n-1, do

cost[i, i] := freq[i]

done

for length in range 2 to n, do

for i in range 0 to (n-length+1), do

j := i + length – 1

cost[i, j] := ∞

for r in range i to j, done

if r > i, then

c := cost[i, r-1]
 else

c := 0

if r < j, then

c := c + cost[r+1, j]

c := c + sum of frequency from i to j

if c < cost[i, j], then

cost[i, j] := c

done

done

done

return cost[0, n-1]

End

Floyd-Warshall

The Floyd-Warshall algorithm is a graph algorithm that is deployed to find the shortest path
between all the vertices present in a weighted graph. This algorithm is different from other
shortest path algorithms; to describe it simply, this algorithm uses each vertex in the graph
as a pivot to check if it provides the shortest way to travel from one point to another.

Floyd-Warshall algorithm works on both directed and undirected weighted graphs unless
these graphs do not contain any negative cycles in them. By negative cycles, it is meant
that the sum of all the edges in the graph must not lead to a negative number.

Since, the algorithm deals with overlapping sub-problems the path found by the vertices
acting as pivot are stored for solving the next steps it uses the dynamic programming
approach.

Floyd-Warshall algorithm is one of the methods in All-pairs shortest path algorithms and it
is solved using the Adjacency Matrix representation of graphs.
Floyd-Warshall Algorithm

Consider a graph, G = {V, E} where V is the set of all vertices present in the graph and E is
the set of all the edges in the graph. The graph, G, is represented in the form of an
adjacency matrix, A, that contains all the weights of every edge connecting two vertices.

Algorithm

Step 1 − Construct an adjacency matrix A with all the costs of edges present in the graph. If
there is no path between two vertices, mark the value as ∞.

Step 2 − Derive another adjacency matrix A1 from A keeping the first row and first column of
the original adjacency matrix intact in A1. And for the remaining values, say A1[i,j],
if A[i,j]>A[i,k]+A[k,j] then replace A1[i,j] with A[i,k]+A[k,j]. Otherwise, do not change the
values. Here, in this step, k = 1 (first vertex acting as pivot).

Step 3 − Repeat Step 2 for all the vertices in the graph by changing the k value for every
pivot vertex until the final matrix is achieved.

Step 4 − The final adjacency matrix obtained is the final solution with all the shortest paths.

Pseudocode

Floyd-Warshall(w, n){ // w: weights, n: number of vertices

for i = 1 to n do // initialize, D (0) = [wij]

for j = 1 to n do{

d[i, j] = w[i, j];

for k = 1 to n do // Compute D (k) from D (k-1)

for i = 1 to n do

for j = 1 to n do

if (d[i, k] + d[k, j] < d[i, j]){

d[i, j] = d[i, k] + d[k, j];

return d[1..n, 1..n];

}

Example

Consider the following directed weighted graph G = {V, E}. Find the shortest paths between
all the vertices of the graphs using the Floyd-Warshall algorithm.

Solution

Initial Matrix D(0)D^{(0)}D(0) (k = 0)

1 2 3 4 5

1 [ 0, 5, ∞, 6, ∞ ]

2 [ ∞, 0, 1, ∞, 7 ]

3 [ 3, ∞, 0, 4, ∞ ]

4 [ ∞, ∞, 2, 0, 3 ]

5 [ 2, ∞, ∞, 5, 0 ]

We will now iterate through k=1k = 1k=1 to 555 and update distances.
Step k=1k = 1k=1

We check if going through vertex 1 improves any paths:

• Row 2: nothing improves because D[2][1]=∞D[2][1] = ∞D[2][1]=∞

• Row 3: nothing improves

• Row 4: nothing improves

• Row 5: nothing improves

Matrix remains the same after k=1k = 1k=1.

Step k=2k = 2k=2

We check if going through vertex 2 improves any paths:

• D[1][3]=min(∞,5+1)=6D[1][3] = min(∞, 5+1) = 6D[1][3]=min(∞,5+1)=6

• D[1][5]=min(∞,5+7)=12D[1][5] = min(∞, 5+7) = 12D[1][5]=min(∞,5+7)=12

• D[3][2]=min(∞,3+5)=8D[3][2] = min(∞, 3+5) = 8D[3][2]=min(∞,3+5)=8

• D[3][5]=min(∞,3+12)=15D[3][5] = min(∞, 3+12) = 15D[3][5]=min(∞,3+12)=15 → but

actually no improvement here since D[3][5]=∞D[3][5]=∞D[3][5]=∞

Updated matrix after k=2k = 2k=2:

1 2 3 4 5

1 [ 0, 5, 6, 6, 12 ]

2 [ ∞, 0, 1, ∞, 7 ]

3 [ 3, 8, 0, 4, 15 ]

4 [ ∞, ∞, 2, 0, 3 ]

5 [ 2, ∞, ∞, 5, 0 ]

Step k=3k = 3k=3

We check if going through vertex 3 improves paths:

• D[4][1]=min(∞,2+3)=5D[4][1] = min(∞, 2+3) = 5D[4][1]=min(∞,2+3)=5

 • D[4][2]=min(∞,2+8)=10D[4][2] = min(∞, 2+8) = 10D[4][2]=min(∞,2+8)=10

• D[4][4]=min(0,2+4)=0D[4][4] = min(0, 2+4) = 0D[4][4]=min(0,2+4)=0 (already 0)

Updated matrix after k=3k = 3k=3:

1 2 3 4 5

1 [ 0, 5, 6, 6, 12 ]

2 [ ∞, 0, 1, ∞, 7 ]

3 [ 3, 8, 0, 4, 15 ]

4 [ 5, 10, 2, 0, 3 ]

5 [ 2, ∞, ∞, 5, 0 ]

Step k=4k = 4k=4

We check if going through vertex 4 improves paths:

• D[1][5]=min(12,6+3)=9D[1][5] = min(12, 6+3) = 9D[1][5]=min(12,6+3)=9

• D[2][5]=min(7,∞)=7D[2][5] = min(7, ∞) = 7D[2][5]=min(7,∞)=7 → already minimum

• D[3][5]=min(15,4+3)=7D[3][5] = min(15, 4+3) = 7D[3][5]=min(15,4+3)=7

• D[5][1]=min(2,5+5)=2D[5][1] = min(2, 5+5) = 2D[5][1]=min(2,5+5)=2 → already

minimum

Updated matrix after k=4k = 4k=4:

1 2 3 4 5

1 [ 0, 5, 6, 6, 9 ]

2 [ ∞, 0, 1, ∞, 7 ]

3 [ 3, 8, 0, 4, 7 ]

4 [ 5, 10, 2, 0, 3 ]

5 [ 2, ∞, ∞, 5, 0 ]

Step k=5k = 5k=5

We check if going through vertex 5 improves paths:

• D[2][1]=min(∞,7+2)=9D[2][1] = min(∞, 7+2) = 9D[2][1]=min(∞,7+2)=9

• D[2][4]=min(∞,7+5)=12D[2][4] = min(∞, 7+5) = 12D[2][4]=min(∞,7+5)=12

• D[5][2]=min(∞,2+5)=7D[5][2] = min(∞, 2+5) = 7D[5][2]=min(∞,2+5)=7

Final matrix after k=5k = 5k=5:

1 2 3 4 5

1 [ 0, 5, 6, 6, 9 ]

2 [ 9, 0, 1, 12, 7 ]

3 [ 3, 8, 0, 4, 7 ]

4 [ 5, 10, 2, 0, 3 ]

5 [ 2, 7, ∞, 5, 0 ]

Analysis

From the pseudocode above, the Floyd-Warshall algorithm operates using three for loops
to find the shortest distance between all pairs of vertices within a graph. Therefore, the time
complexity of the Floyd-Warshall algorithm is O(n3), where n is the number of vertices in the
graph. The space complexity of the algorithm is O(n2).

Prim's Algorithm

To execute the prim's algorithm, the inputs taken by the algorithm are the graph G {V, E},
where V is the set of vertices and E is the set of edges, and the source vertex S. A minimum
spanning tree of graph G is obtained as an output.

Algorithm

• Declare an array visited[] to store the visited vertices and firstly, add the arbitrary
root, say S, to the visited array.

• Check whether the adjacent vertices of the last visited vertex are present in
the visited[] array or not.
 • If the vertices are not in the visited[] array, compare the cost of edges and add the
least cost edge to the output spanning tree.

• The adjacent unvisited vertex with the least cost edge is added into the visited[]
array and the least cost edge is added to the minimum spanning tree output.

• Steps 2 and 4 are repeated for all the unvisited vertices in the graph to obtain the full
minimum spanning tree output for the given graph.

• Calculate the cost of the minimum spanning tree obtained.

Examples

• Find the minimum spanning tree using prims method (greedy approach) for the
graph given below with S as the arbitrary root.

Solution

Step 1

Create a visited array to store all the visited vertices into it.

V={}

The arbitrary root is mentioned to be S, so among all the edges that are connected to S we
need to find the least cost edge.

S→B=8
V = {S, B}

Step 2

Since B is the last visited, check for the least cost edge that is connected to the vertex B.

B→A=9

B → C = 16

B → E = 14

Hence, B → A is the edge added to the spanning tree.

V = {S, B, A}

Step 3

Since A is the last visited, check for the least cost edge that is connected to the vertex A.

A → C = 22

A→B=9

A → E = 11
But A → B is already in the spanning tree, check for the next least cost edge. Hence, A → E is
added to the spanning tree.

V = {S, B, A, E}

Step 4

Since E is the last visited, check for the least cost edge that is connected to the vertex E.

E → C = 18

E→D=3

Therefore, E → D is added to the spanning tree.

V = {S, B, A, E, D}

Step 5

Since D is the last visited, check for the least cost edge that is connected to the vertex D.

D → C = 15

E→D=3
Therefore, D → C is added to the spanning tree.

V = {S, B, A, E, D, C}

The minimum spanning tree is obtained with the minimum cost = 46

Kruskal's Algorithm

The inputs taken by the kruskals algorithm are the graph G {V, E}, where V is the set of
vertices and E is the set of edges, and the source vertex S and the minimum spanning tree
of graph G is obtained as an output.

Algorithm

• Sort all the edges in the graph in an ascending order and store it in an array edge[].

• Construct the forest of the graph on a plane with all the vertices in it.

• Select the least cost edge from the edge[] array and add it into the forest of the
graph. Mark the vertices visited by adding them into the visited[] array.

• Repeat the steps 2 and 3 until all the vertices are visited without having any cycles
forming in the graph

• When all the vertices are visited, the minimum spanning tree is formed.

• Calculate the minimum cost of the output spanning tree formed.

Examples

Construct a minimum spanning tree using kruskals algorithm for the graph given below −
Solution

As the first step, sort all the edges in the given graph in an ascending order and store the
values in an array.

Edge B→D A→B C→F F→E B→C G→F A→G C→D D→E C→G

Cost 5 6 9 10 11 12 15 17 22 25

Then, construct a forest of the given graph on a single plane.

From the list of sorted edge costs, select the least cost edge and add it onto the forest in
output graph.

B→D=5

Minimum cost = 5

Visited array, v = {B, D}

Similarly, the next least cost edge is B → A = 6; so we add it onto the output graph.

Minimum cost = 5 + 6 = 11

Visited array, v = {B, D, A}

The next least cost edge is C → F = 9; add it onto the output graph.

Minimum Cost = 5 + 6 + 9 = 20

Visited array, v = {B, D, A, C, F}

The next edge to be added onto the output graph is F → E = 10.

Minimum Cost = 5 + 6 + 9 + 10 = 30

Visited array, v = {B, D, A, C, F, E}

The next edge from the least cost array is B → C = 11, hence we add it in the output graph.

Minimum cost = 5 + 6 + 9 + 10 + 11 = 41

Visited array, v = {B, D, A, C, F, E}

The last edge from the least cost array to be added in the output graph is F → G = 12.

Minimum cost = 5 + 6 + 9 + 10 + 11 + 12 = 53

Visited array, v = {B, D, A, C, F, E, G}

The obtained result is the minimum spanning tree of the given graph with cost = 53.
Dijkstras Algorithm

The dijkstras algorithm is designed to find the shortest path between two vertices of a
graph. These two vertices could either be adjacent or the farthest points in the graph. The
algorithm starts from the source. The inputs taken by the algorithm are the graph G {V, E},
where V is the set of vertices and E is the set of edges, and the source vertex S. And the
output is the shortest path spanning tree.

Algorithm

• Declare two arrays − distance[] to store the distances from the source vertex to the
other vertices in graph and visited[] to store the visited vertices.

• Set distance[S] to 0 and distance[v] = ∞, where v represents all the other vertices in
the graph.

• Add S to the visited[] array and find the adjacent vertices of S with the minimum
distance.

• The adjacent vertex to S, say A, has the minimum distance and is not in the visited
array yet. A is picked and added to the visited array and the distance of A is changed
from ∞ to the assigned distance of A, say d1, where d1 < ∞.

• Repeat the process for the adjacent vertices of the visited vertices until the shortest
path spanning tree is formed.

Examples

To understand the dijkstras concept better, let us analyze the algorithm with the help of an
example graph −
Step 1

Initialize the distances of all the vertices as ∞, except the source node S.

Vertex S A B C D E

Distance 0 ∞ ∞ ∞ ∞ ∞

Now that the source vertex S is visited, add it into the visited array.

visited = {S}

Step 2

The vertex S has three adjacent vertices with various distances and the vertex with
minimum distance among them all is A. Hence, A is visited and the dist[A] is changed from
∞ to 6.

S→A=6

S→D=8

S→E=7

Vertex S A B C D E

Distance 0 6 ∞ ∞ 8 7

Visited = {S, A}
Step 3

There are two vertices visited in the visited array, therefore, the adjacent vertices must be
checked for both the visited vertices.

Vertex S has two more adjacent vertices to be visited yet: D and E. Vertex A has one
adjacent vertex B.

Calculate the distances from S to D, E, B and select the minimum distance −

S → D = 8 and S → E = 7.

S → B = S → A + A → B = 6 + 9 = 15

Vertex S A B C D E

Distance 0 6 15 ∞ 8 7

Visited = {S, A, E}

Step 4

Calculate the distances of the adjacent vertices S, A, E of all the visited arrays and select
the vertex with minimum distance.

S→D=8

S → B = 15

S → C = S → E + E → C = 7 + 5 = 12
 Vertex S A B C D E

Distance 0 6 15 12 8 7

Visited = {S, A, E, D}

Step 5

Recalculate the distances of unvisited vertices and if the distances minimum than existing
distance is found, replace the value in the distance array.

S → C = S → E + E → C = 7 + 5 = 12

S → C = S → D + D → C = 8 + 3 = 11

dist[C] = minimum (12, 11) = 11

S → B = S → A + A → B = 6 + 9 = 15

S → B = S → D + D → C + C → B = 8 + 3 + 12 = 23

dist[B] = minimum (15,23) = 15

Vertex S A B C D E
 Distance 0 6 15 11 8 7

Visited = { S, A, E, D, C}

Step 6

The remaining unvisited vertex in the graph is B with the minimum distance 15, is added to
the output spanning tree.

Visited = {S, A, E, D, C, B}

The shortest path spanning tree is obtained as an output using the dijkstras algorithm.
Huffman Trees and Codes.

Definition

Huffman coding provides codes to characters such that the length of the code depends on
the relative frequency or weight of the corresponding character. Huffman codes are of
variable-length, and without any prefix (that means no code is a prefix of any other). Any
prefix-free binary code can be displayed or visualized as a binary tree with the encoded
characters stored at the leaves.

Huffman tree or Huffman coding tree defines as a full binary tree in which each leaf of the
tree corresponds to a letter in the given alphabet.

The Huffman tree is treated as the binary tree associated with minimum external path
weight that means, the one associated with the minimum sum of weighted path lengths for
the given set of leaves. So the goal is to construct a tree with the minimum external path
weight.

An example is given below-

Letter frequency table

Letter z k m c u d l e

Frequency 2 7 24 32 37 42 42 120

Huffman code

Letter Freq Code Bits

e 120 0 1

d 42 101 3

l 42 110 3

u 37 100 3

c 32 1110 4
 Letter Freq Code Bits

m 24 11111 5

k 7 111101 6

z 2 111100 6

The Huffman tree (for the above example) is given below -

The Maximum Flow Problem

The Maximum Flow problem is about finding the maximum flow through a directed graph,
from one place in the graph to another.

More specifically, the flow comes from a source vertex s, and ends up in a sink vertex t, and
each edge in the graph is defined with a flow and a capacity, where the capacity is the
maximum flow that edge can have.

More details look here

https://www.geeksforgeeks.org/dsa/max-flow-problem-introduction/

https://www.w3schools.com/dsa/dsa_theory_graphs_maxflow.php