Mixed-Integer Linear Programming Integer Programs (IP) An optimization model is an Integer Program if any of its decision variables is discrete

Mixed-Integer Linear Programming (MILP): Model Formulation

If all variables are discrete, the model is a pure integer program Otherwise, the model is a mixed-integer program Integer variables appear in many problems:

Benoˆıt Chachuat

Trays in a distillation column

McMaster University Department of Chemical Engineering

Can be solved continuous, then rounded to nearest integer

Number of employees (1000’s)

ChE 4G03: Optimization in Chemical Engineering

Number of parallel chemical reactors Whether or not to operate boiler#2 on Monday

Not appropriate to solve continuous and round after

Scheduling people and equipment to tasks over time Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

1 / 26

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

Mixed-Integer Linear Programming

Mixed-Integer Linear Programming

Class Exercise: Give more examples of integer decisions in the field of Chemical Engineering:

Linear vs. Nonlinear Integer Programs

1

4G03

2 / 26

An IP model is an integer linear program (ILP) if its (single) objective function and all its constraints are linear

Fluid flow:

Otherwise, it is an integer nonlinear program (INLP)

◮ ◮

2

Standard Mixed-Integer Linear Programming (MILP) Formulation:

Heat transfer: ◮

3

Mass transfer: ◮ ◮

4

xmin ≤ x ≤ xmax ,

Reactor design:

y ∈ {0, 1}ny

Mixed-Integer Nonlinear Programs (MINLPs) are very difficult to solve

◮ ◮

Benoˆıt Chachuat (McMaster University)

∆

z = cT x + dT y x,y    ≤  s.t. Ax + Ey = b   ≥

min

◮

This is a great motivation for the continued use of LP methods! MILP: Model Formulation

4G03

3 / 26

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

4 / 26

Outline

“Lumpy” Linear Programs One broad class of “lumpy” LPs is when either/or decisions are added to what is otherwise an LP model

Modeling MILP requires knowledge and ingenuity — We will learn some typical MILP formulations here: 1

2

Expressing All-or-Nothing Requirements

Native MILP Formulations “Lumpy” Linear Programs Knapsack Models Assignment and Matching Models Scheduling Models

All-or-nothing variable requirements of the form x j = 0 ∨ x j = uj

Approximations of Nonlinear Models as MILPs Separable Programming

Example: In a blending model, use none or all of a given ingredient

can be modeled by substituting x j ← uj y j ,

s.t. yj ∈ {0, 1}

min 18x1 + 3x2 + 9x3

x1 ,x2 ,x3

s.t. 2x1 + x2 + 7x3 ≤ 150 0 ≤ x1 ≤ 60

For additional examples, see Rardin (1998), Chapter 11

⇐⇒

0 ≤ x2 ≤ 30 x3 = 0 ∨ x3 = 20 Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

5 / 26

“Lumpy” Linear Programs (cont’d)

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

6 / 26

4G03

8 / 26

Formulating Models for Fixed-Charged Objectives Class Exercise: Consider a fixed-charge objective function

Θ(xj )

min Θ1 (x1 ) + Θ2 (x2 )

Another source of “lumpy” LP models is when the objective involves fixed charges (i.e., start-up cost) fj

x1 ,x2

cj

where

s.t. xj ≤ uj yj

∆



110 + 9x2 , if x2 > 0 0, otherwise

Θ2 (x2 ) =



fj + cj xj , if xj > 0 0, otherwise with nonnegative fixed charge fj > 0 and variable xj ≤ uj , can be modeled by substituting Θ(xj ) ← fj yj + cj xj ,

150 + 7x1 , if x1 > 0 0, otherwise

xj

Expressing Fixed-Charge Requirements Fixed-cost requirements of the form: Θ(xj ) =



Θ1 (x1 ) = uj

∆

∆

and

and x1 , x2 satisfy x1 + x2 ≥ 8 0 ≤ x1 ≤ 3 0 ≤ x2 ≤ 8.

yj ∈ {0, 1}

Formulate a corresponding MILP model. Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

7 / 26

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

“Lumpy” Linear Programs (cont’d)

Formulating Models for Semi-Continuous Variables

cj xj

Class Exercise: Consider the following blending problem, where the ingredient x3 is a semi-continuous variable:

Yet another source of “lumpy” LP models is when a decision variable is semi-continuous

min 18x1 + 3x2 + 9x3

cj

x1 ,x2 ,x3

s.t. 2x1 + x2 + 7x3 ≤ 150 0 ≤ x1 ≤ 60 lj

xj

uj

0 ≤ x2 ≤ 30 x3 = 0 ∨ 10 ≤ x3 ≤ 20

Expressing Semi-Continuous Requirements Semi-continuous variables of the form: xj = 0 ∨ lj ≤ xj ≤ uj , with lj > 0 xjl , xju

can be modeled by introducing 2 new continuous variables new binary variable yj ∈ {0, 1}, and 2 additional constraints:

Formulate a corresponding MILP model.

≥ 0, 1

xj = lj xjl + uj xju 1 = yj + xjl + xju Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

Knapsack Models

4G03

Value

The problem is to select a maximum value collection of items subject to limitations on resources consumed Knapsack models are the simplest of all (pure) integer linear programs (ILPs)

9 / 26

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

10 / 26

Knapsack Models (cont’d)

000 111 1111 0000 000 0000 1111 I4 111 0000 1111 000 111 000 111

I3 000 111 000 111 000 111 1111 0000 0000 1111 I2 0000 1111 0000 1111 00 11 0000 1111 00 11 00 11 00 11 I1 00 11 00 11

max.

1

Dependence of choice j on choice i can be enforced on corresponding binary variables by adding constraints of the form:

2

Mutually exclusiveness conditions allowing at most one of a set of choices J can be enforced by adding constraints of the form:

Capacity

Each element is either all in or all out of the selection:  1, if item j selected ∆ yj = 0, otherwise

Similarly, at most p of a set of choices J can be enforced as:

Example: Selection of projects subject to limitations on budgets Class Exercise: Readily available Canadian coins are 1¢, 5¢, 10¢ and 25¢. Formulate an ILP model to minimize the number of coins needed to provide change amount for q¢. Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

11 / 26

3

Constraints can also enforce selection of exactly p or at least p of a set of choices J

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

12 / 26

Formulating ILP Models

Assignment Models

Class Exercise: The Department of Chemical Engineering at McMaster is acquiring optimization software for use in ChE 4G03. The 4 codes available and the types of algorithms they provide are indicated by ×’s in the following table: Algorithm Type LP IP NLP Cost

1 × — — 3

Code, j 2 3 × × × — — × 4 6

Set I

Set J

The issue here is optimal matching or pairing of objects of two distinct types It is standard to model all assignment forms with the decision variables:

4 × × × 14

∆

yi ,j =



1, if i of the 1st set is matched with j of the 2nd 0, otherwise

Examples: Assign sales personnel to customers, jobs to machine, etc. 1

2

Formulate a MILP model to acquire a minimum cost collection of codes providing LP, IP and NLP capability Formulate a MILP model to acquire a minimum cost collection of codes with exactly one providing LP, one providing IP, and one providing NLP capability

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

Assignment constraints forcing every i and/or every j to be assigned are of the form:

13 / 26

Assignment Models (cont’d)

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

14 / 26

Matching Models Set I

Generalized assignment constraints encompass cases where allocation of i to j requires fixed space si ,j within j capacity bj :

Decision variables of matching models typically are:  1, if i is paired with i ′ ∆ ′ yi ,i = 0, otherwise

Linear assignment models minimize/maximize a linear objective function of the form: XX ci ,j yi ,j i ∈I j∈J

where i ′ > i , by convention, to avoid double counting

where ci ,j is the cost of assigning i to j Class Exercise: Items i = 1, . . . , 100 of volume ci are being stored in an automated warehouse. Storage location j = 1, . . . , 20 are at a distance dj from the input/output station, and all have capacity b. Formulate a MILP model to store all items at minimum total travel distance. Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

Unlike assignment models, matching models eliminate the distinction between the sets

15 / 26

Matching constraints forcing some i ∈ I to pair with and only with some other i ′ ∈ I are of the form:

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

16 / 26

Matching Models (cont’d)

Process Scheduling Models

Class Exercise: The instructor of ChE 4G03 wants to assign his students to 2-person teams for a graded tutorial. each student s has scored his/her preference ps,s ′ for working with each other student s ′ . 1

Formulate a MILP model to form teams in a way that maximizes total preference.

Scheduling is the allocation of resources over time Single-process scheduling problems seek an optimal sequence in which to complete a given collection of tasks on a single process that can accommodate only one job at a time Feed 1

Product 1

111111 000000 p1 000000 111111 000000 111111

r1

Feed 2

Product 2 r2

Feed 3

Time

d1

Product 3 r3

p2 11111 00000 00000 11111

d2

11111 00000 p3 00000 11111 00000 11111

d3

Typical Data:

Time Time

∆

pj = estimated process time for task j ∆

rj = release time at which task j becomes available for processing ∆

dj = due date by which task j should be completed Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

17 / 26

Process Scheduling Models (cont’d) Product 1

x2

x3 d1

Product 2 r2

Feed 3

Product 3 r3

11111 00000 p2 00000 11111 00000 11111

11111 00000 00000 11111 p3 00000 11111 00000 11111

d2

d3

Time

Feed 1

Time

Feed 2

Time

Feed 3

Time Decision Variables and Related Constraints: ∆ xj = start time for task j resource availability: xj ≥ rj , ∀j

Product 1

x2

x3

111111 000000 p1 000000 111111 000000 111111

r1

Time

d1

Product 2 r2 Product 3 r3

Max. completion time (makespan) Mean completion time

Due dates are usually handled as goals to be reflected in the objective function rather than explicit constraints This avoids the situation where there is no feasible schedule that meets all due dates Dates that must be met are termed deadlines to distinguish and can be easily enforced as: xj + pj ≤ dj , ∀j MILP: Model Formulation

18 / 26

p2 11111 00000 00000 11111

d2

11111 00000 p3 00000 11111 00000 11111

d3

Time Time

Objective Functions — Often minimizes one of the following:

Handling of Due Dates:

Benoˆıt Chachuat (McMaster University)

4G03

x1

111111 000000 000000 111111 p1 000000 111111 000000 111111

r1

Feed 2

MILP: Model Formulation

Process Scheduling Models (cont’d) x1

Feed 1

Benoˆıt Chachuat (McMaster University)

4G03

19 / 26

Max. lateness Mean lateness Max. tardiness Mean tardiness

Benoˆıt Chachuat (McMaster University)

max{xj + pj : j = 1, . . . , n} n 1X (xj + pj ) n j=1 max{xj + pj − dj : j = 1, . . . , n} n 1X (xj + pj − dj ) n j=1 max{0, max{xj + pj − dj : j = 1, . . . , n}} n 1X max{0, xj + pj − dj } n j=1

MILP: Model Formulation

4G03

20 / 26

Process Scheduling Models (cont’d)

Process Scheduling Models (cont’d)

Class Exercise: The following table shows the process time, release times, due dates, and scheduled starts for three jobs. Compute the corresponding value of each of the six objective functions listed previously. Process time, pi Release time, ri Due Date, di Scheduled start, xi 1

Job 1 15 5 20 9

Job 2 6 10 25 24

Feed 1

Product 1

Feed 2

Job 3 9 0 36 0

Feed 3

Product 3

Time

∨ xj ′ + pj ′ ≤ xj ,

∀j, j ′ = 1, . . . , n,

j 6= j ′

Illustrative Example: y1,2 = y1,3 = 1, y2,3 = 0

MILP: Model Formulation

4G03

21 / 26

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

22 / 26

Separable Programming x2

x3

Consider the following mathematical program:

111111 000000 p1 000000 111111 000000 111111

r1

Time

d1

Product 2 r2 Product 3 r3

p2 11111 00000 00000 11111

d2

11111 00000 p3 00000 11111 00000 11111

d3

min

Time

x

Time

s.t.

Big-M Method

∆

z=

n X

fj (xj ) = f1 (x1 ) + · · · + fn (xn )

j=1

n X

ai ,j xj ≤ bi ,

i = 1, . . . , m

j=1

Conflicts between tasks j and j ′ can be prevented with disjunctive constraint pairs:

xj ≥ 0,

xj + pj ≤ xj ′ + M(1 − yj,j ′ )

◮

xj ′ + pj ′ ≤ xj + Myj,j ′

◮

where M is a large positive constant (e.g., the max. makespan) Class Exercise: Formulate integer linear constraints for feasible schedules on a single process with 3 tasks having process times 14, 3, and 7. Benoˆıt Chachuat (McMaster University)

d3

0 + 9 − 36 = − 27

x1

Feed 3

11111 00000 p3 00000 11111 00000 11111

Time

Reformulation Using Disjunctive Variables:  1, if j binding comes before j ′ ∆ yj,j ′ = 0, if j ′ binding comes before j

Process Scheduling Models (cont’d)

Feed 2

d2

0+9= 9

24 + 6 − 25 = 5,

Product 1

p2 11111 00000 00000 11111

Conflict Constraints:

Maximum lateness is max{4, 5, −27} = 5 Mean completion time is 31 (4 + 5 − 27) = −6

Feed 1

Time

d1

r3

Lateness of the three jobs (xj + pj − dj ) is:

Benoˆıt Chachuat (McMaster University)

000000 111111 p1 111111 000000 000000 111111

r2

Maximum completion time is max{24, 30, 9} = 30 Mean completion time is 13 (24 + 30 + 9) = 21 9 + 15 − 20 = 4,

x3

Product 2

xj + pj ≤ xj ′ 24 + 6 = 30,

x2

r1

Completion times (xi + pi ) are: 9 + 15 = 24,

2

x1

MILP: Model Formulation

4G03

23 / 26

j = 1, . . . , n

The objective consists of n nonlinear, separable terms fj (xj ), each a function of a single variable only The m constraints are linear

When each fj is convex or concave on the feasible region, the separable program can be approximated with an LP Otherwise, the separable program can be approximated with an MILP Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

24 / 26

Approximating General Separable Programs as MILPs

Approximating General Separable Programs as MILPs

Piecewise affine approximation: (Nj intervals)

Piecewise affine approximation: (Nj intervals)

(2)

fj (xj )

fj fj

fj

fj

c2 =

fj

Define: (k) cj

(1)

−fj

(2) (1) xj −xj

=

0≤

(1) (3)

fj

(k)

− fj

(k−1)

(k−1) − xj (k) (k−1) ωj,k ≤ xj − xj

Substitute each variable xj and function fj by:

(2)

0 ≤ ωj,2 ≤ (0) xj

(2) xj

−

xj ←

(1) xj

(1) xj

(2) xj

(3) xj

(2)

fj (xj )

c2 =

(k) xj

(0)

(0) xj

+

Nj X

fj (xj ) ← fj

(0)

+

X

fj fj

fj

fj

ωj,k

k=1 Nj

xj

Disjunctive Variables:  0, if ωj,k = 0 ∆ yj,k = 1, if ωj,k > 0 Disjunctive Constraints: h i (k) (k−1) ωj,k ≤ xj − xj yj,k ,

(1) (3)

(2)

(0)

ωj,k 0 ≤ ωj,2 ≤ (0) xj

(k)

(1) −fj (2) (1) xj −xj

fj

(2) xj

(1) xj

−

(1) xj

(2) xj

(3) xj

xj

cj ωj,k

∀k = 1, . . . , Nj i (k) (k−1) ≥ xj − xj yj,k+1 , h

∀k = 1, . . . , Nj − 1

Do you see why this works?

k=1 Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

25 / 26

Approximating Separable Programs as MILPs (cont’d) Approximate Mixed-Integer Linear Program (on Nj Intervals): min ω,y

n X

fj

(0)

j=1

s.t.

Nj n X X

+

Nj n X X

(k)

cj ωjk

j=1 k=1

ai ,j ωj,k ≤ bi −

j=1 k=1

n X

(0)

ai ,j xj ,

i = 1, . . . , m

j=1

h i (k) (k−1) ωj,k ≤ xj − xj yj,k , k = 1, . . . , Nj h i (k) (k−1) ωj,k ≥ xj − xj yj,k+1 , k = 1, . . . , Nj − 1 yj,k ∈ {0, 1},

j = 1, . . . , n,

k = 1, . . . , Nj

The solution can be made as accurate as desired by using enough intervals — One pays the price in terms of increased problem size! Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

26 / 26

Benoˆıt Chachuat (McMaster University)

MILP: Model Formulation

4G03

25 / 26