10182D
10182D
A. Schrijver
Department of Econometrics Ti/burg University
P.O. Box 90153, 5000 LE Ti/burg, The Netherlands
and
Centre for Mathematics and Computer Science
P.O. Box 4079, 1009 AB Amsterdam, The Netherlands
1. INTRODUCTION
In April 1984, at the 16th Annual ACM Symposium on Theory of Computing,
NARENDRA KARMARKAR of AT&T Bell Laboratories presented a new algo-
rithm for linear programming. The algorithm was not only shown to be
theoretically efficient (i.e., its running time is bounded by a polynomial in the
input size}, but was also claimed to be very fast in practice - about 50 times
faster than Dantzig's classical simple~ method, for the largest problems
evaluated.
This news created much excitement among computer scientists and
mathematical programmers, and subsequent reports, inter alia in Science maga-
zine and on the front page of the New York Times, contributed to a further
propagation of the sensation. Linear programming is one of the mathematical
fields most applied in practice. Linear programming problems occur in such
diverse areas as engineering, transportation, agriculture, distribution, schedul-
ing, nutrition, management, and a reduction of the computer time needed
would not only speed up solving linear programming problems, but also would
allow one to solve larger LP-problems than before. In situations like oil pro-
cessing and automatic control, quick, almost forthwith, solution of LP-
problems is essential.
In 1979, L.G. KHACHIYAN published the first polynomial-time method for
linear programming, the ellipsoid method. This method, though theoretically
efficient, turned out to behave rather disappointingly in practice. So
Karmarkar's claim that he now has found a method which is both theoretically
and practically efficient, was much welcomed. Karmarkar's paper was pub-
lished in the December issue of Combinatorica [5]. However, no computational
details were given.
Next, KARMARKAR was invited to give plenary lectures at two international
conferences, the ORSA/TIMS-meeting in November 1984 in Dallas, and the
12th International Symposium on Mathematical Programming in August 1985
at MIT. KARMARKAR described his method and variants, explaining some of
the tricks used in practice, claiming superiority of his method over the simplex
method, and giving a few comparisons, but he refused to give full disclosure of
test problems, computer programs and running times. This has led to much
uncertainty and discussion among mathematical programmers with respect to
the practical value of the new method. It led to a report 'Founding father of
just a footnote?' in the Boston Globe of August 9, 1985:
3
In this account of the new method I will restrict myself to the theoretical
aspects.
In Section 2 and 3 we briefly discuss the simplex method and Khachiyan's
ellipsoid method. In Section 4 and 5 we describe Karmarkars method, while
in Sections 6 and 7 we show that the method has polynomially bounded run-
ning time.
4
more dimensions. DE LA VALLEE PoussIN [9) gave a similar method.
DANTZIG [2] algebraized the method, obtaining an attractive compact
scheme (simplex tableau) and iterative procedure (pivoting), which facilitates
computer implementation. This simplex method turns out to be very efficient
in practice and enables one to solve LP-problems in several thousands of vari-
ables.
However, it could not be proved theoretically that the simplex method is
efficient. That is, no proof has been found that the running time of the simplex
method is bounded by a polynomial in the size of the problem, i.e. in
2: log( Iaij I+ 1) + 2: log (I b; I + l) + 2: log (I c1 I + 1). (2)
0 j
In fact, KLEE and MINTY [7] showed, by giving a bad class of LP-problems,
that with Dantzig's pivoting rule, the simplex method can require exponential
running time. Their examples have as feasible regions a deformation of the n-
dimensional cube (described by 2n inequalities), for which Dantzig's rule leads
to a trip along all 2n vertices. Several alternative pivot selection rules have
been proposed, but none of them could be proved to yield a polynomial-time
method.
On the other hand, BORGWARDT [l] recently gave a pivoting rule which he
showed to yield a polynomial-time algorithm on the average, in a certain
natural probabilistic model. His result very much agrees with practical experi-
ence, where data seem to be more 'random' than structural.
5
I
I
I
I
I
I
I
I
I
I
I
I
\
\
\
\
\
\
\
\
\
\
\
i
A deformation of the n-dimensional cube, with a simplex path along 2n vertices
(n =3)
6
around the origin of radius R : = n n+I rn.
E o is the first ellipsoid. Next ellipsoids E 1> E 2 , ••• are determined with the
following rule. If Ek has been found, with center say zk> check if Azk';;;;,b holds.
If so, we have found a solution of Ax ..;;,b as required. If not, we can choose an
inequality, say aTx..;;,b; in Ax:s;;;;.b violated by zk. Let Ek+I be the ellipsoid
such that
Ek+1~Ekn(xlafx:s;;;;.aTzk} (5)
and such that Ek+ 1 has smallest volume (there exist simple updating formulas
for obtaining the parameters describing Ek+ 1 from those describing Ek and
from a;). Since (xlAx..;;,b}~(xlaTx..;;,aTzk}, it follows by induction on k
from (5) that
Ek~ (x IAx..;;,b }. (6)
Moreover, it can be proved that
volume Ek+J <e-l/ 4n ·volume Ek. (7)
Since one easily sees that volume E 0 ..;;,(2Rf<n 2n'rn', inductively from (7) we
have:
(8)
On the other hand, with Cramer's rule, using the boundedness and fulldi-
mensionality of {x !Ax ..;;,b }, we know:
volume {x IAx..;;,b};;;;i.n- 2n'r-n'. (9)
(6), (8) and (9) imply:
(10)
i.e.,
(11)
So after a polynomially bounded number of iterations we will have found a
solution of Ax :s;;;;.b . Updating the ellipsoid parameters can be done in 0(n 2 )
arithmetic operations, while all calculations have to be done with a precision
of 0(n 3 log T) digits. Altogether this amounts to 0(n 8 log2 T) bit operations
(excluding data-handling, which takes 0(loglog T · logloglog T) for each bit
operation).
Although KHACHIYAN showed the polynomial solvability of the linear pro-
gramming problem, his method turned out to perform badly in practice. This
is caused, among others, by the facts that the upper bound (11) of iterative
steps, though polynomial in the input size, can be rather big also for moderate
problems, and that the precision required to describe the successive ellipsoids
is huge. (The ellipsoid method has implications in combinatorial optimization
- see [4].)
7
Thus the question remained if there is a method for linear programming
which is both practically and theoretically efficient. K.ARMARKAR claims that
the following method is so.
8
5.KARMARKAR's METHOD
Karmarkar's method consists of constructing a sequence of vectors x 0 ,x 1 ,x 2 , ...
converging to a solution of (12) (provided (12) has a solution). The essence of
the method is to replace the condition x ;;;;.o by a stronger condition x EE, for
some ellipsoid E contained in ~"+-. As we shall see, minimizing c T x over
{xlxEE,Ax=O,lTx=l} is easy, while minimizing c7 x over
{x Ix EIR "+-,Ax =O, 1T x = l} is the original problem ( 12).
Let A and c as in (12) be given. We may assume without loss of generality
that the rows of A are linearly independent, and that n ;;;;.2. Throughout we use
r:=~ (18)
Let
1
XO : =-·1. (19)
n
So Ax 0 =0, 1Tx 0 =I, x 0 >0. Next a sequence of vectors x 0 ,x 1 ,x 2 , ... such
that Axk =O, 1Txk=I, xk>O is determined, with the following recursion:
denote xk = :(x\k>, ... , x~k))T, and let D be the diagonal matrix:
D : = diag(x\kl, ... , x~k >). (20)
xk+l : = (lTDzk+1)-1·Dzk+1.
I
Note that if we replace in the minimization problem the condition z EB(l, 2r)
by the weaker condition z ;;;;.o, then we would obtain a minimization problem
with optimum value at most 0 if and only if min
{ c Tx IAx =O, 1Tx=1, x ;;;;.Q} ,,;;;;O, which is our original problem (12).
As z k + 1 minimizes (c TD )z over the intersection of a ball with an affine
space, we can write down a formula for z k + 1 :
PROPOSITION 1.
(J-DA T(AD 2 A T)- 1AD-n - 1·1 · lT)Dc
zk+1 - 1 - -I r .~-=----=-:...::_~=-.::...:.......L-_.:;_=------'--
- 2 11(1-DA T(AD 2 AT)- 1AD-n - 1·l · l7)Dcll.
9
c TD -p T is a linear combination of rows of AD and of the row vector 1T.)
Then zk+I is the vector reached from 1 by going over a distance ; r in the
direction - p, i.e.,
zk+I =1- l.r_E_ 0 (23)
2 llpll"
6. A LEMMA IN CALCULUS
In order to show correctness and convergence of the algorithm, we use the fol-
lowing lemma in elementary calculus. For x=(xJ. ... ,xnf, we denote:
(24)
10
x' ~O~v' ~='. Therefore .
"' " "' .
.
.. . r --+-·::. ~ _1!:_ \---.
x r :: ~x 1-----.--r (26)
18
Thi'.' !irsl inequality here follows fmm
quality (note that x • and the that if
x--rr+;; O.x 2 ..... r 2 +::==µ..
On the other hand. (x. satisfies
x ~.1· +:: =: O..x2 + +::: =.u. . = - Ll8 \
:c:: Hence \Ve haH~ equal it\.
throughout in (2.6t Therefore. r' '"':: •. proving (25 ).
mm l Ilx Ix EH B(l. 0
Without k)SS of generality. \ i ~x: ~ ... ~xn. Then for all l ~1 < </... ~n.
vector ) attains
nun{xr::ix+_r-r;; (281
(otherwise we could replace the componenb .t, ..\.X1. of x by better 1values).
Hence bv (25). x,-='x,. Therefore x 2 =x1 = ... -=x,.. A:-. xEHi"IB(l.-::;r). this
. 1· . l , .I . . l Th' h ,-.-
imp !CS X1 =2· anu X: = ... =xn =(I +2 (n ·-- l l). lS s OWS LJ
I er D::' •I n
1
t'rDI 11.:p\1-"l.
II
1/2
-- -
n-1"
(31)
Indeed, if(12) has a solution then ADz=O, z;;;.O, cTDzo;;;,O for some z=f:O. We
may assume lrz=n. Hence,
Q;;a.min{(crD)zlzER'!i.,ADz=O, lrz=n} (32)
;;a.min{(crD)z lz EB(l,(n - l)r),ADz =O, lr z =n}
(the last inequality follows from (i) of the Lemma).
The last minimum in (32) is attained by the vector 1-(n - l)r~, as
cTxk,,;;;, c
Txk
< [2]k/n
- · CTXO
:s;::::
[2]k/n ·nT
- (35)
(IIxk)l/n e (IIxO)l/n"""' e '
where T denotes the maximum absolute value of the entries in A and c (w.
I.o.g.T;;a.n).
This gives, if we take
12
N : = I [ l -2/n2] n2ln(nT)l, (36)
THEOREM. If (12) has a solution, then CT xk <n --n T-n for some k =O, ... , N.
So supfose (12) has a solution. Then with Karmarkar's method we find a vec-
tor x satisfying xk~O,Ax=O, lTx=I, cTxk<n-nT-n. By elementary
linear algebra, it is easy to find a vertex x • of the polytope
{x~OIAx=O, lTx=l} with cTx·~cTxk. Hence, cTx*<n-nT-n. By
Cramer's rule, the entries in x • have a common denominator at most nn T".
As c is integral, this implies c T x * ~O. So x • is a solution of ( l 2).
Parts of the description above are taken from the forthcoming book [8].
REFERENCES
1. K.-H. BoRGW ARDT ( 1982). The average number of pivot steps required
by the simplex method is polynomial. Zeitschrift fur Operations Research
26, 157-177.
2. G.B. DANTZIG (1951 ). Maximization of a linear function of variables sub-
ject to linear inequalities. TJ.C. KOOPMANS (ed.). Activity Analysis of Pro-
duction and Allocation, John Wiley & Sons, New York, 339-347.
3. J.B.J. FOURIER (1826). Analyse des travaux de l'Academie Royale des
Sciences, pendant J'annee 1823, Partie mathematique. Histoire de
f'Academie Royale des Sciences de l'lnstitut de France 6, xxix-xli.
4. M. GR6TSCHEL, L. LovAsz, A. SCHRIJVER (1986). The Ellipsoid Method
and Combinatorial Optimization, Springer-Verlag, Berlin.
5. N. KARMARKAR (1984). A new polynomial-time algorithm for linear pro-
gramming. Combinatorica 4, 373-395.
6. LG. KHACHIYAN (1979). A polynomial algorithm in linear programming
(in Russian). Doklady Akademii Nauk SSSR 244, 1093-1096.
7. V. KLEE, G.J. MINTY (1972). How good is the simplex algorithm?
0. SHISHA (ed.). Inequalities, Ill. Academic Press, New York, 159-175.
8. A. ScHRIJVER (1986). Theory of Linear and Integer Programming, John
13
& Sons. Chichester.
9. Cu. Dr LA VAUEr Pot:SSIN ( 1910). Sur la methode de l'approximatwn
Scient~fique de Bruxelles 35 (.!}, 1-16.
rmrmnum. Annales de 111 ."io;:i<'tE;
14