LARGE-SCALE LINEAR PROGRAMMING

by GEORGE B. DANTZIG

TECHNICAL REPORT NO. 67-8 NOVEMBER 1967

Department of Operations Research Stanford University Stanford, California

Research partially supported by Office of Naval Research, Contract ONR-N-00014-67-A-O112-0011; U.S. Atomic Energy Commission, Contract AT(04-3)-326 PA #18; National Science Foundation Grant GP 6431; and U.S. Army Research Office, Contract No. DAHC04-67-C-0028. Reproduction In whole or In part for any purpose of the United States Government Is permitted.

LARGE-SCALE LINEAR PROGRAMMING by George B. Dantzlg*

Large-Scale Systems and the Computer Revolution;

From its very Inception, it was envisioned that linear programming would be applied to very large, detailed models of economic and military systems.

Kantorovitch's 1939 proposals, which were before

the advent of the electronic computer, mentioned such possibilities, [78]. Linear programming evolved out of the U.S» Air Force interest in 1947 in finding optimal time-staged deployment plans in case of war, [126];

a

problem whose mathematical structure is similar to that of finding an optimal growth pattern of a developing economy and similar to other control problems, [41], [58], [123].

Structurally the dynamic problems

are characterized in discrete form by staircase matrices representing the inputs and outputs from one time period to the next.

Treated as an

ordinary linear program, the number of rows and columns grows in proportion to tne number of time periods T

3

and possibly higher.

T

and the computational effort grows by

This fact has limited the use of linear

programming as a tool for planning over many time periods.

*Department8 of Operations Research and Computer Science, Stanford University, Stanford, California. Research of G B Dantzlg partially supported by Office of Naval Research, Contract ONR-N-00014-67-A-0112-0011; U.S. Atomic Energy Commission, Contract AT(04-3)-326 PA #18; National Science Foundation Grant GP 6431; and U.S. Army Research Office, Contract No. DAhC04-67-C-0028. Reproduction in whole or in part for any purpose of the United States Government is permitted.

At the present

1967

stage of the computer revolution,

there Is growing Interest on the part of practical users of linear programming models to solve larger and larger systems [40].

Such

applications Imply that eventually automated systems will obtain Information from counters and sensing devices, process data Into the proper form for optimization and finally Implement the results by control devices.

There has been steady progress In this mechanization

of flow to and from the computer.

Hitherto, this has been one of the

obstacles encountered In setting-up and solving large-scale systems, [113].

The second obstacle has been the cost and the time required to

successfully solve large problems, [74].

It Is difficult to measure the potential of large-scale planning.

Certain developing countries appear, according to optimal

calculations

on simplified models to be able to grow at the rate of

15% per year

Implying a doubling of their Industrial base In 5 years.

But administrators apparently Ignore plans and make decisions based on political expediency which restrict growth to 2 or 3X and sometimes -2%, It is the belief of the author that the mechanization of data flow (at least in advanced countries) in the next decade will provide pathways for constructing large models and the effective implementation of the results of optimization.

This application of mathematics to decision

processes will eventually become as Important as the classical applications to physics and will, in time, change the emphasis in pure mathematics.

Three Approaches to Solving Large-Scale Systems;

There have been a great number of papers on this subject as evidenced from the list of references attached.

I have broadly classified them

into:

I

Decomposition Principle (Sub-optimization using interior path)

II

Compact Inverse (Using a simplex variant) Parametric Variation (Sub-optimization using simplex variant)

III

The aim is to say a little about each, citing some references and some structures to which they are applicable.

I;

We shall begin with

The Decomposition Principle. [47]:

Consider the non-linear programming problem:

Find

x ■ (x.t...9x ) such that

g(x)

(1)

= Min

f^x)
4000 and L

An important

is large relative to

M

most

or more) of the diagonal equations have exactly one

basic variable among the set of its variables.

The fact that most

basic variables are at their upper-bound value can be used to advantage. The first code along these lines was developed by M. Kasatkln and J. Bigelow for a problem of Crown Zellerbach paper corporation. Running time on an example was about 1/10 the time that was required by a general purpose code.

(lid)

See also [65].

Bordered Angular Systems; This consists of blocks along the

diagonal of non-zero coefficients and a border of non-zeros along the top and left.

|

l l

(12)

. •«a

This structure is sufficiently genera] yet specialized to usefully cover

11

I

a majority of current applications except the staircase type. Generalization (of the procedures Just discussed) have been made by heesterman, [72].

Ritter [99] has generalized Rosen's parametric

scheme, [103].

III.

Parametric Variation;

The third and last approach depends on the system being weakly linked I.e. on the existence of a few rows and columns which, If removed, makes the solution of the remaining system trivial.

For

example, a network-flow problem with an extra budget constraint.

By

assigning a Lagrange Multiplier to the latter, the constraint could be removed and the objective equation modified by adding to it the multiple of the removed constraint. then be easily solved.

The resulting pure network could

If the solution does not satisfy the

constraint and complementary slackness conditions, then the Lagrange Multiplier is varied until it does.

This is also the idea behind the

decomposition principle but the proposed methods of variation (such as those below) are more direct: Rosen:

"Partition Programming" [103], Ritter [99].

Krön:

"Diakoptics" [83].

Balas:

"Infeasibillty Pricing" [10].

Beale:

"Pseudo Basic Variables" [17].

Abadie & Williams: [3]. Gass: (Ilia)

"Dualplex Method" [59].

Dualolex Method: As representative of the parametric approaches I have selected 12

Gas«' "Dualplex Method" which is related to Rosen's "Partition Programming" In dual form.

It Is clear If we had a transposed block-

ansular structure

E

l

(13)

E

2

D

- b

E

3

v

-*S

1

y

N

Plvo tiiig allowed here

Not here

X > 0

Y > 0

that pivoting in the right hand interconnecting part would destroy the angular structure but pivoting anywhere in We assume that for a given

Y - Y

a feasible solution

^ 0

X » X

_^ 0

E.,

E2,

E.

would not.

(variables associated with

exists and is optimal.

D )

Let the system

be reduced to optimal canonical form restricting pivots to only columns of

E, 'i *

(14)

IXg + AXj^ + DY

-

b

+ dY

-

z - Zo (Max)

X^,Y

non-basic.

CXJJ

where

X

are basic variables and

for the moment, we solve the subproblem

(15)

DY ^ b ,

Y >. 0 ,

13

Max

dY

Holding

^ " 0

I The dual of this subproblem Is (16)

nD .> d ,

Since

xT D

n .> 0 ,

Mln

lib .

Is presumed to consist of few rows and many columns» It is

suitable for solution by the standard simplex method. an optimal solution and by

D.

the

the basic components non-basic

Y - Y

1-th row of X

D

^ 0

x. - 0 X»

or

x. > 0

6. - c. - IIA. > 0 J J J

or

A. the j-th column of ■ 0

X

and

X.. > 0

x. + D.Y' - h. ;

where

be partitioned Into

11-n

be optimal to Its dual.

and by

be partitioned Into

Let

X-

and

XIV

be

Denote A .

Let

according as Let the

according as

< 0 .

_

Xj-O

XJJ

> 0

«in'0

«IV«o

X

X

III "

0

IV "

0

Y

' -

0

Block

1 1

Pivot

(17)

1 A 1 c

z-z (Max) o

.

The block pivot: The next step Is to find the block pivot of highest rank that switches the role of as many basic and nonbaslc variables in X

as possible.

XT

and

Since both sets are at zero value this does not

effect the current feasible solution.

If there Is a choice of block

pivot Its columns are selected from those with highest

14

6.

values.

After the pivot the new dual ■ubproblen Is solved using as starting basis, the one corresponding to the final basis of the previous subproblem.

Y' _> 0

subproblem but

11'

Is still a feasible price vector of the dual no longer satisfies It.

However,

Theorem (Gass):

If after the block pivot those components corresponding to n

6. > 0

are replaced by the value

n' -6. ,

(tf

11'

the new

constitutes an Infeaslble basic solution to the new subproblem;

Y' >_ 0 remains as a feasible vector of dual simplex multipliers.

Because of Infeaslblllty, the new subproblem can be Improved (using the dual simplex method). 6.^0

or

z -*• + oo .

This Is repeated Iteratlvely until all

Associated with each Iteration Is a basic feasible

to the full problem so that usual proof of a flnlte-number-of-lteratlons applies.

The parametric methods should be regarded as Important variants of the standard simplex process.

Concluding Remarks;

This completes the survey of the three types of approaches to solving large-scale systems:

Decomposition, Compact Inverse, and

Parametric Variation, and of the type of matrix structures that each are best suited.

Little has been said about how different proposals

compare on test problems.

At present, there does not appear to be a

15

!

practical way Co do this.

The program of Instructions for the computer

are often an order of magnitude more complex than a good commercial linear program system and the latter can cost two to five hundred thousand dollars to develop.

The author feels that better computer

languages have to be developed to facilitate the experimental coding and comparlslon of large-scale system proposals, [74].

16

i

BIBLIOGRAPHY ON LARGE-SCALE SYSTEMS

1.

ABADIE, J.M., "Dual Decomposition Method for Linear Programs'*, Comp. Center Case Institute of Technology, July 1962.

2.

ABADIE, J.M., "On Decomposition Principle", Operations Research Center, University of California, Berkeley, ORC 63-20, 1963.

3.

ABADIE, J.M. and WILLIAMS, A.C., "Dual and Parametric Methods In Decomposition", In Recent Advances In Math. Prog., edited by R. Graves and P. Wolfe, McGraw-Hill, 1963.

4.

ACZEL, M.A. and RÜSSEL, A.H.,"New Methods of Solving Linear Programs", Q.R. Ou. Vol. 8 No. 4, Dec. 1957.

5.

ADIN, B. Thomas, "Optimizing a Multistage Production Process", O.R. Ou. Vol. 14, No. 2, June 1963.

6.

AGGARWAL, S.P., "A Simplex Technique for a Particular Convex Programming Problem", Canadian Ooeratloqftl Research Journal. Vol. 4, No. 2 July 1966.

7.

ALTMAN, M., "An Elimination Method for L.P. with Application to the Decomposition Problem", Bull. Acad. Polon. Sei. Ser. Scl. Math. Astron. Phvslcs.

8.

ALWAY, G.G., "A Trlangularlzatlon Method for Computations In Linear Programming", Naval Research Logistics Quarterly. Vol. 9, pp. 163-180.

9.

BAKES, M.D., "Solution of Special Linear Programming Problem with Additional Constraints", O.R. Ou. Vol. 17, No. 4, Dec. 1966.

10.

BALAS, Egon, "An Infeaslblllty - Pricing Decomposition Method for Linear Program", July 1966, Operations Research 14 (1966) 843-873.

11.

BALAS, Egon, :Solution of Large Scale Transportation Problems Through Aggregation", Operations Research. 13 (1965) 82-93.

12.

BALINSKI, M.L., "On Some Decomposition Approaches In Linear Programming", and "Integer Programming", The University of Michigan Engineering Summer Conferences, 1966.

13.

BARNETT, S., "Stability of the Solution to a Linear Programming Problem", O.R. Ou.. Vol. 13, No. 3, September 1962.

14.

BAUMOL, W.J. and FABIAN, T., "Decomposition, Pricing for Decentralization and External Economics", Management Science. Vol. 11 No. 1, September 1964. 17

15.

BEALE, E.M.L., "Survey of Integer Programming", O.R. Qu. Vol. 16, No. 2, June 1965.

16.

BEALE, E.M.L., "Decomposition and Partitioning Methods for Nonlinear Programming", in Non-Linear Programming. J. Abadie, Ed., Northholland publishing Company, also Wiley.

17.

BEALE, E.M.L., "The Simplex Method Using Pseudo-Basic Variables for Structured Linear Programming Problems", from Recent Advances in Math. Prog.. edited by R. Graves and P. Wolfe, McGraw-Hill, 1963.

18.

BELL, E.J., "Primal-Dual Decomposition Programming", Unpublished Ph.D. Thesis, Industrial Engineering Department, University of California, Berkeley, 1964.

19.

BELLAR, F.J, , "Iterative Solution of Large-Scale Systems of Simultaneous Linear Equations", SIAM Journal. Vol. 9, No. 2, June 1961.

20.

BENDERS, J.F., "Partitioning Procedures for Solving Mixed Variables Programming Problems", Num. Math. 4, 1962.

21.

BENNETT, J.M., "An Approach to Some Structured Linear Programming Problems" Operations Research 14 (1966) 4 (July-August) pp. 636-645.

22.

BESSIERE, F. et SAUTER, £., "Optimisation et Enviornment Economique: La Methode Des Modeles Flargis", Revue Francaise de Recherche Ooerationnelle No. 40, 1966.

23.

BOOT, J.C.G., "On Trivial and Binding Constraints in Programming Problems", Management Sei. Vol. 8, 1962, pp. 419-441.

24.

BRADLEY, S.P., "Solution Techniques for the Traffic Assignment Problem", ORC 65-35, University of California, Berkeley, 1965.

25.

BRASILOW, C.B., LASDON, L.S., PEARENS, J.D. , MACKO, 0., TAKAKORA, Y., "Papers on Multilevel Control Systems", DTV 70-A-65, Case Institute of Technology, 1965.

26.

CATChPOLE, A.R., "The Application of Linear Programming to Integrated Supply Problems in the Oil Industry", O.R. Ou.. Vol. 13, No. 2, June, 1962.

27.

CHARNES, A. and COOPER, W.W., "Generalizations of the Warehousing Model", O.R. Qu. Vol. 6, No. 4, Dec. 1955.

28.

CHARNES, A. and COOPER, W.W., "Management Models and Industrial Applications in Linear Programming", Management Science, Vol. 4, No. 1 October 1957, pp. 38-91.

18

■

29.

CHURCHMAN, C.W., "On the Ethics of Large-Scale Systems, Part I", Internal Working Paper, No. 37, SSL, University of California, Berkeley, September 1965.

30.

CRAVEN, B.D., "A Generalization of the Transportation Method of Linear Programming", Q.R> On. Vol. 14, No. 2, June 1963.

31.

CURTES, H.A., "Use of Decomposition Theory in the Solution of the State Assignment Problem of Sequential Machines", Journal of A.C.M.. July 1963, p. 386,

32.

DANTZIG, G.B», "Upper Bounds, Secondary Constraints and Block Triangularity in Linear Programming", Econometrica, Vol. 23, No. 2 April, 1955.

33.

DANTZIG, G.B., "Optimal Solution of a Dynamic Leontief Model with Substitution", Econometrica. Vol. 23, No. 3, July 1955.

34.

DANTZIG, G»B., "Linear Programming Under Unvertainty", Management Science. Vol. 1 (1