Scribd
Upload
21 views

U4 - Limitations of Algo Power

limitations of algorithm power

Uploaded by

BhaskarHosur
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
21 views

U4 - Limitations of Algo Power

limitations of algorithm power

Uploaded by

BhaskarHosur
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

LIMITATIONS OF

ALGORITHM
POWER
-BHASKAR H S
LOWER BOUNDS

Lower bound: It is an estimate on a minimum amount of work


needed to solve a given problem
Examples:
• number of comparisons needed to find the largest element in
a set of n numbers
• number of comparisons needed to sort an array of size n
• number of comparisons necessary for searching in a sorted
array
• number of multiplications needed to multiply two n-by-n
matrices
LOWER BOUNDS (CONT.)

• Lower bound can be


• an exact count
• an efficiency class ()
Tight lower bound: there exists an algorithm with the same efficiency as the lower
bound
Problem Lower bound Tightness

sorting (nlog n) yes


searching in a sorted array (log n) yes
element uniqueness (nlog n) yes
n-digit integer multiplication (n) unknown
multiplication of n-by-n matrices (n2) unknown
METHODS FOR ESTABLISHING LOWER
BOUNDS

• trivial lower bounds

• information-theoretic arguments
(decision trees)

• adversary arguments

• problem reduction
TRIVIAL LOWER BOUNDS

Trivial lower bounds: based on counting the number of items that


must be processed in input and generated as output

Examples
• finding max element

• sorting

• Hamiltonian circuit existence

Conclusions
• may and may not be useful

• be careful in deciding how many elements must be processed


(min-max)
DECISION TREES
Decision tree —model algorithms that use comparisons:
• internal nodes represent comparisons
• leaves represent outcomes
• How many orderings of 2 elements? Of 3? Of n?
• Tree for 3-element insertion sort [insert b in a, then c into ab]
abc
yes no
a<b

abc bac
yes no yes no
b< c a<c

a<b<c acb bca


b<a<c
yes no yes no
a<c b<c

a<c<b c<a<b b<c<a c<b<a


DECISION TREES AND SORTING
ALGORITHMS
• Any comparison-based sorting algorithm can be represented by
a decision tree

• Number of leaves (outcomes)  n!

• Height of binary tree with n! leaves  log2n!

• Minimum number of comparisons in the worst case  log2n!


for any comparison-based sorting algorithm

• log23! = log2 6 = 3

• log2n!  n log2n, for large n

• This lower bound is tight (mergesort)


BASIC CONCEPTS

- Classifying Problems
- Tractable and non-tractable
- Optimization and Decision Problems

- Problem Classes
-P
- NP
- NP Complete
CLASSIFYING PROBLEM COMPLEXITY

Is the problem tractable, i.e., is there a polynomial-time (O(p(n))


algorithm that solves it?
Possible answers:
• yes (give examples)
• no
• because it’s been proved that no algorithm exists at all

(e.g., Turing’s halting problem)


• because it’s been be proved that any algorithm takes
exponential time
• unknown
PROBLEM TYPES: OPTIMIZATION AND DECISION

• Optimization problem: find a solution that maximizes or


minimizes some objective function

• Decision problem: answer yes/no to a question

Many problems have decision and optimization versions.

E.g.: traveling salesman problem


• optimization: find Hamiltonian cycle of minimum length
• decision: find Hamiltonian cycle of length  m
Decision problems are more convenient for formal investigation
of their complexity.
P, NP, NP COMPLETE

P and NP are important classes of


problems.

Question: Are they the same? Is P = NP ?


CLASS P

P: the class of decision problems that are solvable in O(p(n))


time, where p(n) is a polynomial of problem’s input size n

Examples:
• searching

• graph connectivity

• graph acyclicity
CLASS NP

NP (nondeterministic polynomial): class of decision problems


whose proposed solutions can be verified in polynomial time =
solvable by a nondeterministic polynomial algorithm
A nondeterministic polynomial algorithm is an abstract two-stage
procedure that:
• generates a random string that may solve the problem
• checks whether this solution is correct, in polynomial time
By definition, it solves the problem if it will generate and verify a
solution on one of its tries
Why this definition?
• led to development of the rich theory called “computational
complexity”
EXAMPLE: CNF SATISFIABILITY

Problem: Is a boolean expression in its conjunctive normal form (CNF)


satisfiable, i.e., are there values of its variables that makes it true?
This problem is in NP. Nondeterministic algorithm:
• Guess truth assignment
• Substitute the values into the CNF formula to see if it evaluates to true

Example: (A | ¬B | ¬C) & (A | B) & (¬B | ¬D | E) & (¬D | ¬E)


Truth assignments:
ABCDE
0 0 0 0 0
. . .
1 1 1 1 1
Checking phase: O(n)
NP-COMPLETE : THE HARDEST NP
PROBLEMS
A decision problem D in NP is NP-complete if it’s as hard as (ie.,
no easier than) any and every problem in NP. That is:
• D is in NP
• every problem in NP is polynomial-time
NP problems reducible to D

NP-complete
problem

Think: Which problems can be easier?


COOK: THE FIRST NP-
COMPLETE PROBLEM
A decision problem D that is in NP is NP-complete if
• D is in NP
• Every problem in NP is polynomial-time reducible to D
NP problems

NP-complete
problem

How to prove every NP problem is reducible to D? Not easy!


Cook’s theorem (1971): CNF-sat is NP-complete
P = NP ? DILEMMA REVISITED

• If P = NP then every problem in NP (including all NP-complete


problems) could be solved in polynomial time
• If a polynomial-time algorithm for just one NP-complete
problem is discovered, then every problem in NP can be solved
in polynomial time, i.e., P NP=problems
NP

NP-complete
problem

• Most but not all researchers believe that P  NP , i.e. P is a


proper subset of NP

You might also like