Solution for assignment 1

1. Consider the following unconstrained problem:

Minimize 2𝑥12 − 𝑥1 𝑥2 + 𝑥22 − 3𝑥1 + 𝑒 2𝑥1+𝑥2
a. Write the first-order necessary optimality conditions. Is this condition also
sufficient for optimality? Why?
b. Is 𝑋̂ = (0, 𝑂)′ an optimal solution? If not, identify a direction d along which the
function would decrease.
c. Minimize the function starting from (0, 0) along the direction d obtained in Part
b.
d. Dropping the last term in the objective function, use a classical direct
optimization technique to solve this problem.

Solution:

Let 𝑓(𝑥) = 2𝑥12 − 𝑥1 𝑥2 + 𝑥22 − 3𝑥1 + 𝑒 2𝑥1 +𝑥2

a. The first-order necessary condition is 𝑓 (𝑥 ) = 0, that is:

4𝑥1 − 𝑥2 − 3 + 2𝑒 2𝑥1+𝑥2 𝟎
[ ]=[ ]
−𝑥1 + 2𝑥2 + 𝑒 2𝑥1 +𝑥2 𝟎
The Hessian 𝐻( 𝑥) 𝑜𝑓 𝑓 (𝑥 ) is
2𝑥1 +𝑥2
𝑯(𝒙) = [4 + 4𝑒 2𝑒 2𝑥1+𝑥2 − 1] and as can be easily verified
2𝑥1 +𝑥2
2𝑒 −1 2 + 𝑒 2𝑥1+𝑥2
𝐻( 𝑥) is a positive definite matrix for all x. Therefore, the first-order necessary condition
is sufficient in this case.
b. 𝑥 = (0,0) is not an optimal solution. 𝑓(𝑥)=[−1,1]𝑡 , and any direction 𝑑 = (𝑑1 , 𝑑2 )
such that
−𝑑1 + 𝑑2 +  0 (𝑒. 𝑔. , 𝑑 = (1,0)) is a descent direction of 𝑓 ( 𝑥) 𝑎𝑡 𝑥 .
c. Consider 𝑑 = (1,0). Then 𝑓(𝑥 + 𝑑) = 22 − 3 +𝑒 2
The minimum value of 𝑓(𝑥 + 𝑑) over the interval [0, ) is 0.94 and is attained at
∗ = 0.1175

1
 d. If the last term is dropped,𝑓(𝑥) = 2𝑥 2 − 𝑥1 𝑥2 + 𝑥22 − 3𝑥1 . Then the first-order
necessary condition yields a unique solution 𝑥1 = 6/7 and 𝑥2 = 3/7. Again, the Hessian
of 𝑓 (𝑥 ) is positive definite for all x, and so the foregoing values of 𝑥1 and 𝑥2 are
optimal. The minimum value of 𝑓 ( 𝑥)x is given by – 63/49.

2. Consider the following problem:

Minimize 𝑥14 + 𝑥24 + 12𝑥12 + 6𝑥22 − 𝑥1 𝑥2 − 𝑥1 − 𝑥2
subject to
𝑥1 + 𝑥2 ≥ 6
2𝑥1 - 𝑥2 ≥ 3
𝑥1 ≥ 0, 𝑥2 ≥ 0
Write out the KKT conditions and show that (𝑥1 , 𝑥2 ) = (3, 3) is the unique optimal
solution

Solution:

The KKT system is given by:

If 𝑥 = (3, 3), then denoting the Lagrange multipliers by 𝑢 , we have that 𝑢3 + 𝑢4 = 0.

Consequently, the first two equations give 𝑢1 = 152 and 𝑢2 = 12. Thus, all the KKT
conditions are satisfied at𝑥 = (3, 3). The Hessian of the objective function is positive
definite, and so the problem involves minimizing a strictly convex function over a convex
set. Thus, 𝑥 = (3, 3) is the unique global optimum.

2
 3. Consider the following problem:
9
Minimize ( 𝑥1 − 4)2 + (𝑥2 − 2)2

subject to 𝑥2 − 𝑥12 ≥ 0
𝑥1 + 𝑥2 ≤ 6
𝑥1 , 𝑥2 ≥ 0.
a. Write the KKT optimality conditions and verify that these conditions hold true at the
point
𝑋̅ = (3/2,9/4)′.
b. Interpret the KKT conditions at 𝑋̅ graphically.
c. Show that 𝑋̅ is indeed the unique global optimal solution

Solution:

a. The KKT system for the given problem is:

At 𝑥 = (3/2, 9/4)𝑡 , denoting the Lagrange multipliers by 𝑢 , we necessarily have 𝑢2 =

𝑢3 = 𝑢4 = 0, which yields a unique value for 𝑢1 𝑛amely, 𝑢1 = 1/2. The above values for
𝑥1 , 𝑥2 , and 𝑢𝑖 for 𝑖 = 1, 2, 3, 4 satisfy the KKT system, and therefore 𝑥 is a KKT point.

3
 b. Graphical illustration:
From the graph, it follows that at 𝑥, the gradient of 𝑓 ( 𝑥) is a negative multiple of the
gradient of 𝑔1 (𝑥) = 𝑥12 − 𝑥2 where 𝑔1 (𝑥) ≤ 0 is the only active constraint at 𝑥.

c. It can be easily verified that the objective function is strictly convex, and that the active
constraint function is also convex (in fact, the entire feasible region is convex in this
case). Hence, 𝑥 is the unique (global) optimal solution to this problem.

4
 4. Consider the following problem:

𝑥 +3𝑥2 +3
Minimize 2𝑥1
1 +𝑥2 +6

Subject to 2𝑥1 + 𝑥2 ≤ 12
−𝑥1 + 2𝑥2 ≤ 4
𝑥1 , 𝑥2 ≥ 0
a. Show that the KKT conditions are sufficient for this problem.
b. Show that any point on the line segment joining the points (0, 0) and (6, 0) is an
optimal solution.

Solution:

𝑥1 +3𝑥2 +3
a. The objective function 𝑓(𝑥1 , 𝑥2 ) = is pseudo convex over the feasible
2𝑥1 +𝑥2 +6

region. The constraint functions are linear, and are therefore quasi convex and
quasi concave. Therefore, if 𝑥 is a KKT point for this problem, then 𝑥 is a global
optimal solution.

b. First note that 𝑓 (0,0) = 𝑓(6,0) = 1/2, and moreover, 𝑓 [ (0,0) + (1 −

 )(6,0)] = 1/2 for any   [0,1]. Since (0, 0) and (6, 0) are feasible solutions,
and the feasible region is a polyhedral set, any convex combination of (0, 0) and
(6, 0) is also a feasible solution. It is thus sufficient to verify that one of these two
points is a KKT point. Consider (6, 0).

5
The KKT system for this problem is as follows:

Substituting (6, 0) for (𝑥1 , 𝑥2 ) into this KKT system yields the following unqiue values
for the Lagrangian multipliers: 𝑢1 = 𝑢2 = 𝑢3 = 0, and 𝑢4 = 5/36. Since
𝑢𝑖  0 , 𝑖 = 1,2,3,4, we conclude that (6, 0) is indeed a KKT point, and therefore, by Part
(a), it solves the given problem. Hence, by the above argument, any point on the line
segment joining (0, 0) and (6, 0) is an optimal solution.

6
 5. Let c be an n-vector, b an m-vector, A an m x n matrix, and H a symmetric n x n positive
definite matrix. Consider the following two problems:
1
Minimize 𝑐′𝑥 + 2 𝑥′𝐻𝑥

subject to 𝐴𝑥 ≤ 𝑏
1
Minimize ℎ′𝑣 + 2 𝑣′𝐺𝑣

subject to 𝑣 ≥ 0,
where 𝐺 = 𝐴𝐻 −1 𝐴′ and ℎ = 𝐴𝐻 −1 ∗ 𝑐 + 𝑏.
Investigate the relationship between the KKT conditions of these two problems.

Solution:

In the first problem, the KKT system is given by:

Since the matrix H is invertible, Equation (1) yields 𝐻 −1 𝑐 + 𝑥 + 𝐻 −1 𝐴𝑡 𝑢 = 0

By premultiplying this equation by A, we obtain , 𝐴 𝐻 −1 𝑐 + 𝐴𝑥 + 𝐴𝐻 −1 𝐴𝑡 𝑢 = 0
which can be rewritten as
𝐴 𝐻 −1 𝑐 + 𝑏 + 𝐴𝑥 − 𝑏 + 𝐴𝐻 −1 𝐴𝑡 𝑢 = 0 … … … … (3)
Next, note that from Equation (2), we have 𝑦 = 𝑏 − 𝐴𝑥, so that Equation (3) can be
further rewritten as
ℎ + 𝐺𝑢 − 𝑦 = 0, where 𝑤ℎ𝑒𝑟𝑒 𝑢𝑡 𝑦 = 0, 𝑢 ≥ 0, 𝑦 ≥ 0 … … … … … … … . (4)
In the second given problem, the KKT system is given by
ℎ + 𝐺𝑣 − 𝑧 = 0, 𝑣 𝑡 𝑧 = 0, 𝑣 ≥ 0, 𝑧 ≥ 0 … … … … … … . . (5)
where z is the vector of Lagrange multipliers. By comparing (4) and (5), we see that
the two problems essentially have identical KKT systems, where 𝑢  𝑣 and  𝑧 , that
is, the Lagrange multipliers in the first problem are decision variables in the second
problem, while the Lagrange multipliers in the second problem are slack variables in
the first problem.

7
 6. Consider the following problem:
Maximize 𝑥12 + 4𝑥1 𝑥2 + 𝑥22
subject to 𝑥12 + 𝑥22 = 1.
a. Using the KKT conditions, find an optimal solution to the problem.
b. Test for the second-order optimality conditions.
c. Does the problem have a unique optimal solution?

Solution:

We switch to minimizing the function 𝑓(𝑥1 , 𝑥2 ) = 𝑥12 + 4𝑥1 𝑥2 + 𝑥22

a. The KKT system is as follows:

There are four solutions to this system:

The objective function 𝑓(𝑥1 , 𝑥2 ) takes on the value of –3 for the first two points, and the
value of 1 at the remaining two. Since the linear independence constraint qualification
(CQ) holds, the KKT conditions are necessary for optimality. Hence, there are two
optimal solutions:
𝑥1 = (1/ √2, 1/ √2) and 𝑥2 =( −1/ √2, −1/ √2).
To support this statement, one can use a graphical display, or use the second-order
sufficiency condition given in Part (b) below

8
 b. 𝑳(𝒙) = −𝑥12 − 4𝑥1 𝑥2 + 𝑣(𝑥12 + 𝑥22 − 1) . Therefore
𝒗−𝟏 −𝟐
𝟐 𝑳(𝒙) = 𝟐 [ ]
−𝟐 𝒗−𝟏
For v = 3, 𝟐 𝑳(𝒙) is a positive definite matrix and therefore,
√2 √2
𝑥1 = ( √2/2, √2/2) and 𝑥2 = (− ,− 2 ) are both strict local optima.
2

c. See answers to Parts (a) and (b).