15.

093: Optimization Methods

Lecture 12: Discrete Optimization

1 Todays Lecture
Slide 1
• Modeling with integer variables
• What is a good formulation?
• Theme: The Power of Formulations

2 Integer Optimization
2.1 Mixed IO
Slide 2
(MIO) max c� x + h� y
s.t. Ax + By ≤ b
n
x ∈ Z+ (x ≥ 0, x integer)
m
y ∈ R+ (y ≥ 0)

2.2 Pure IO
Slide 3
(IO) max c� x
s.t. Ax ≤ b
n
x ∈ Z+
Important special case: Binary Optimization
(BO) max c� x
s.t. Ax ≤ b
x ∈ {0, 1}n

2.3 LO
Slide 4
(LO) max c� x
s.t. By ≤ b
n
y ∈ R+

3 Modeling with Binary Variables

3.1 Binary Choice
� Slide 5
1, if event occurs
x∈
0, otherwise
Example 1: IO formulation of the knapsack problem
n : projects, total budget b

aj : cost of project j

cj : value
� of project j
Slide 6
1, if project j is selected.
xj =
0, otherwise.

1
 n
�
max cj xj
j=1
�
s.t. aj xj ≤ b
xj ∈ {0, 1}

3.2 Modeling relations

Slide 7
• At most one event occurs �
xj ≤ 1
j

• Neither or both events occur

x2 − x1 = 0

• If one event occurs then, another occurs

0 ≤ x2 ≤ x1

• If x = 0, then y = 0; if x = 1, then y is uncontrained

0 ≤ y ≤ U x, x ∈ {0, 1}

3.3 The assignment problem

Slide 8
n people

m jobs

cij : cost
� of assigning person j to job i.

1 person jis assigned to job i

xij =
� 0
min cij xij

�
s.t. xij = 1 each job is assigned
j=1

�m

xij ≤ 1 each person can do at most one job.

i=1

xij ∈ {0, 1}

3.4 Multiple optimal solutions

Slide 9
• Generate all optimal solutions to a BOP.

max c� x
s.t. Ax ≤ b
x ∈ {0, 1}n

• x∗ optimal solution: I0 = {j : x∗j = 0}, I1 = {j : x∗j = 1}.

2
 • Add constraint � �
xj + (1 − xj ) ≥ 1.
j∈I0 j∈I1

• Generate third best?

• Extensions to MIO?

4 What is a good formulation?

4.1 Facility Location
Slide 10
• Data

N = {1 . . . n} potential facility locations

I = {1 . . . m} set of clients

cj : cost of facility placed at j

hij : cost of satisfying client i from facility j.

• Decision variables
�
1, a facility is placed at location j
xj =
0, otherwise
yij = fraction of demand of client i
satisfied by facility j.
Slide 11
n
� m �
� n
IZ1 = min cj xj + hij yij
j=1 i=1 j=1
�n
s.t. yij = 1
j=1
yij ≤ xj
xj ∈ {0, 1}, 0 ≤ yij ≤ 1.
Slide 12
Consider an alternative formulation.
n
� m �
� n
IZ2 = min cj xj + hij yij
j=1 i=1 j=1
�n
s.t. yij = 1
j=1
�m
yij ≤ m · xj
i=1
xj ∈ {0, 1}, 0 ≤ yij ≤ 1.

Are both valid?

Which one is preferable?

3
4.2 Observations
Slide 13
• IZ1 = IZ2 , since the integer points both formulations define are the same.

n �
� 0 ≤ xj ≤ 1
P1 = {(x, y) : yij = 1, yij ≤ xj ,
0 ≤ yij ≤ 1
j=1
n
� m
�
P2 = {(x, y) : yij = 1, yij ≤ m · xj ,
j=1 i=1
�
0 ≤ xj ≤ 1
0 ≤ yij ≤ 1
Slide 14
• Let
Z1 = min cx + hy, Z2 = min cx + hy
(x, y) ∈ P1 (x, y) ∈ P2

• Z2 ≤ Z1 ≤ IZ1 = IZ2

4.3 Implications
Slide 15
• Finding IZ1 (= IZ2 ) is difficult.
• Solving to find Z1 , Z2 is a LOP. Since Z1 is closer to IZ1 several methods

(branch and bound) would work better (actually much better).

• Suppose that if we solve min cx + hy, (x, y) ∈ P1 we find an integral

solution. Have we solved the facility location problem?

Slide 16

• Formulation 1 is better than Formulation 2. (Despite the fact that 1 has

a larger number of constraints than 2.)

• What is then the criterion?

4.4 Ideal Formulations

Slide 17
• Let P be a linear relaxation for a problem
• Let

H = {(x, y) : x ∈ {0, 1}n} ∩ P

• Consider Convex Hull (H)

� �
= {x : x = λi xi , λi = 1, λi ≥ 0, xi ∈ H}
i i

Slide 18

4
 • The extreme points of CH(H) have {0, 1} coordinates.
• So, if we know CH(H) explicitly, then by solving min cx + hy, (x, y) ∈

CH(H) we solve the problem.

• Message: Quality of formulation is judged by closeness to CH(H).

CH(H) ⊆ P1 ⊆ P2

5 Minimum Spanning
Tree (MST)
Slide 19
• How do telephone companies bill you?
• It used to be that rate/minute: Boston → LA proportional to distance in

MST

• Other applications: Telecommunications, Transportation (good lower bound

for TSP)

Slide 20
• Given a graph G = (V, E) undirected and Costs ce , e ∈ E.
• Find a tree of minimum cost spanning all the nodes.
�
1, if edge e is included in the tree
• Decision variables xe =
0, otherwise
Slide 21
• The tree should be connected. How can you model this requirement?
• Let S be a set of vertices. Then S and V \ S should be connected
�
i∈S
• Let δ(S) = {e = (i, j) ∈ E :
j ∈V \S
• Then, �
xe ≥ 1
e∈δ(S)

• What is the number of edges in a tree?

�
• Then, xe = n − 1
e∈E

5
5.1 Formulation
� Slide 22
IZMST = min ce xe

⎧ e∈E

�
⎪ xe ≥ 1 ∀ S ⊆ V, S �= ∅, V
⎨ e∈δ(S)
⎪
�
H xe = n − 1

⎩ e∈E

⎪
⎪
xe ∈ {0, 1}.
Is this a good formulation? Slide 23

Pcut = {x ∈ R|E| : 0 ≤ x ≤ e,
�
xe = n − 1
e∈E
�
xe ≥ 1 ∀ S ⊆ V, S �= ∅, V }
e∈δ(S)

Is Pcut the CH(H)?

5.2 What is CH(H)?

Slide 24
Let �
Psub = {x ∈ R|E| : xe = n − 1
e∈E
�
xe ≤ |S| − 1 ∀ S ⊆ V, S =
� ∅, V }
e∈E(S)
� �
i∈S
E(S) = e = (i, j) :
j∈S
Why is this a valid IO formulation? Slide 25

• Theorem: Psub = CH(H).

• ⇒ Psub is the best possible formulation.
• MESSAGE: Good formulations can have an exponential number of con­

straints.

6 The Traveling Salesman

Problem
Slide 26
Given G = (V, E) an undirected graph. V = {1, . . . , n}, costs ce ∀ e ∈ E. Find
a tour that minimizes total length.

6
6.1 Formulation I
� Slide 27
1, if edge e is included in the tour.
xe =
0, otherwise.
�
min ce xe

e∈E

�
s.t. xe ≥ 2, S⊆E
e∈δ(S)
�
xe = 2, i∈V
e∈δ(i)
xe ∈ {0, 1}

6.2 Formulation II
� Slide 28
min �ce xe
s.t. xe ≤ |S| − 1, S ⊆ E
e∈E(S)
�
xe = 2, i ∈ V
e∈δ(i)
xe ∈ {0, 1}
Slide 29
T SP
= {x ∈ R|E| ;
� �
Pcut xe ≥ 2, xe
= 2
e∈δ(S) e∈δ(i)

0 ≤ xe ≤ 1}
�

T SP
Psub
= {x ∈ R|E| ; xe = 2

e∈δ(i)

�
xe ≤ |S| − 1
e∈δ(S)
0 ≤ xe ≤ 1}
Slide 30
T SP T SP
• Theorem: Pcut = Psub �⊇ CH(H)
• Nobody knows CH(H) for the TSP

7 Minimum Matching
Slide 31
• Given G = (V, E); ce costs on e ∈ E. Find a matching of minimum cost.
• Formulation: �
min �ce xe
s.t. xe = 1, i∈V
e∈δ(i)
xe ∈ {0, 1}

• Is the linear relaxation CH(H)?

Slide 32

7
Let
PMAT = {x ∈ R|E| :
�
xe = 1
e∈δ(i)
�
xe ≥ 1 |S| = 2k + 1, S �= ∅
e∈δ(S)

xe ≥ 0}

Theorem: PMAT = CH(H)

8 Observations
Slide 33
• For MST, Matching there are efficient algorithms. CH(H) is known.
• For TSP � ∃ efficient algorithm. TSP is an N P − hard problem. CH(H)

is not known.

• Conjuecture: The convex hull of problems that are polynomially solvable

are explicitly known.

9 Summary
Slide 34
1. An IO formulation is better than another one if the polyhedra of their

linear relaxations are closer to the convex hull of the IO.

2. A good formulation may have an exponential number of constraints.

3. Conjecture: Formulations characterize the complexity of problems. If a

problem is solvable in polynomial time, then the convex hull of solutions

is known.

8
MIT OpenCourseWare
http://ocw.mit.edu

15.093J / 6.255J Optimization Methods

Fall 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.