BackTracking

Backtracking
● Exhaustive search is required for some problems.
● Backtracking is a more intelligent variation of this
approach.
● Main idea is to construct solutions one component at a
time and evaluate each partially constructed candidates in
following way:
– If a partially constructed solution can be developed,
continue.
– If there is no legitimate option for the next component,
backtrack to replace the last component of the partially
constructed solution with its next option
 Backtracking
● We build a state-space tree for a backtracking algorithm
which is usually constructed in the depth-first search way.
● If our algorithm reaches a complete solution to the
problem, we stop (if just one solution is required) or we
continue searching for other possible solutions (if there
are).
● The advantage of this method is that if it is reallized that
the partial vector in the state-space tree can no longer lead
to the optimal solution then all previous steps can be
ignored and we backtrack to pervious step.
– It drastically reduces the number of searches compared to
exhaustive or brute force strategy. (We will look in examples)
 Sum of Subsets Problem
● Find a subset of a given set A = {a1, . . . , an }
of n positive integers whose sum is equal to a
given positive integer D.
● For example A = {11, 13, 24, 7} and SUM = 31,
there are two solutions: {11, 13, 7} and {24, 7}.
– We can represent this solution in terms of indices
of these items as well e.g., {1,2,4} and {3,4}.
● Some instances of this problem may have no
solutions.
 Sum of Subsets Problem
● It is convenient to sort the set’s elements in
increasing order.
● In the state space tree at a node, If sum is not
equal to D, we can terminate the node as non-
promising if either of the following two
inequalities holds:
– s+ai+1 > D (the sum is too large)
– s+Σ(j=i+1 to n) (aj) < D (the sum is too small)
 Sum of Subsets Problem
● Example: For A = {3,5,6,7} and D = 15, the
solution: {3, 5, 7}.
 N-queens Problem
● We are given a chessboard of size NxN, and we need to place
N queens so that no queen attacks the other queens. Queen
can attack in three directions: vertical, horizontal and diagonal.
● Lets us look at the statespace tree for a 4-Queen problem (N=4)
 N-queens Problem
● We are given a chessboard of size NxN, and we need to place N
queens so that no queen attacks the other queens. Queen can attack
in three directions: vertical, horizontal and diagonal.
● We can see the enormous state-space for the 4-queen problem can
lead to a very intractable brute-force or exhaustive search strategy.
● Let us see how backtracking works on a 4 queen problem.
● We start with the root node and the path in the search space is ().
● Now we generate children in ascending order starting with 2, and path
is now (1)
– This signify placing queen 1 on column 1.
● Now node 3 is generated and immediately killed (why? Because on
second column we cannot place a queen.)
● Now we generate node 8 and path becomes (1,3) as you can see on
next slide.
 N-queens Problem
● Node 8 won't lead to a solution as all child will
get killed as evident from the next slide and we
backtrack.
● We now move to path (1,4) representing first
st
queen is placed on 1 column and second
queen is placed on 4th column.
● In this manner we proceed to find the
backtracking solution to 4-queen problem and
in turn the general n-queen problem.
N-Queens Problem
 Exercise Problem: Graph Coloring
● Given an undirected graph and a number m, determine if the graph
can be colored with at most m colors such that no two adjacent
vertices of the graph are colored with same color.
● Coloring of graph means assignment of colors to all vertices.
● This Problem is also called as m-colorability decision problem
● Applications:
– Making exam schedule for a university
● a graph where every vertex is a subject and an edge between two
vertices mean there is a common student.
– Mobile Radio Frequency Assignment
● How to assign frequencies to mobile towers, such that frequencies
assigned to all towers at the same location must be different.
(mobile towers are vertices and edge betweem them represent that
they are in range of each other)
● What is the minimum number of frequencies needed?
– Map Coloring
● Geographical maps of countries or states where no two adjacent
cities cannot be assigned same color.
 Branch and Bound
● In Backtracking, we cut off a branch of the problem’s state-
space tree as soon as we can deduce that it cannot lead to a
solution
● This idea can be strengthened further if we deal with an
optimization problem.
● An optimization problem seeks to minimize or maximize some
objective function like
– a tour length in TSP,
– Value of items selected in knapsack subject to some constraints.
● An optimal solution is a feasible solution with the best value of
the objective function
 Branch and Bound
● Compared to backtracking, branch-and-bound requires two
additional items:
1. A way to provide, for every node of a state-space tree, a
bound on the best value of the objective function on any
solution, that can be obtained by adding further
components to the partially constructed solution
represented by the node
2. the value of the best solution seen so far
 Branch and Bound
● In general, we terminate a search path for any one of the
following three reasons:

1) The value of the node’s bound is not better than the

value of the best solution seen so far.
2) The node represents no feasible solutions because the
constraints of the problem are already violated.
3) The subset of feasible solutions represented by the node
consists of a single point (and hence no further choices
can be made)
 0/1 Knapsack Problem
● We will look into the Branch and Bound solution to 0/1
Knapsack problem
● Given n items of known weights wi and profits vi , i = 1, 2, .
. . , n, and a knapsack of capacity W, find the most
valuable subset of the items that fit in the knapsack.
● Let us order the elements in their decreasing order of
profit/weight ratio (vi/wi , for all i)
● Each node on the ith level of this tree, 0 ≤ i ≤ n, represents
all the subsets of n items that include a particular selection
made from the first i ordered items
● We record the total weight w and the total value v of this
selection in the node, along with some upper bound ub
 0/1 Knapsack Problem
● A simple way to compute the upper bound ub
is to
– add to v, the total value of the items already
selected,
– the product of the remaining capacity of the
knapsack W − w and the best per unit payoff
among the remaining items, which is vi+1/wi+1
ub = v + (W − w)(vi+1/wi+1)
 0/1 Knapsack Problem
● Example: The items are already ordered in descending
order of their value-to-weight ratios.

● Let us apply the branch-and-bound algorithm.

0/1 Knapsack Problem
0/1 Knapsack Problem
Some Basic Concepts
 0/1 Knapsack problem
● For the knapsack problem, every node of the tree represents a subset, and
also a feasible solution of the items given.
● We can use this fact to update the information about the best subset seen so
far after generating each new node in the tree.
● If we had done this for the instance investigated above, we could have
terminated nodes 2 and 6 before node 8 was generated because they both are
inferior to the subset of value 65 of node 5
● The technique discussed here is applicable for minimization problems, So
convert the knapsack problem (maximizing the profit) into minimization
problem by negating the objective function
Branch and Bound – 0/1 Knapsack
Branch and Bound – 0/1 Knapsack
Computation
 References
● Narasimha Karumanchi, Data Structures and Algorithms Made
Easy”, 2nd Ed., (2014)
● Sahni, Sartaj, Data Structures, Algorithms and Applications in
C++, Universities Press, 2nd ed., (2005)
● Corman, Leiserson & Rivest, Introduction to Algorithms, MIT
Press, 3rd Ed., (2009)
● Computer Algorithms, by Ellis Horowitz, Sartaj Sahni, and
Sanguthevar Rajasekaran
● https://www.mathsisfun.com
● Geeksforgeeks, wikipedia