IMA Journal of Mathematical Control and Information (2001) 18, 189–205

A maximum principle for optimal control problems with mixed constraints M. D . R. DE P INHO Departamento de Engenharia Electrot´ecnica e de Computadores, Faculdade de Engenharia da Universidade do Porto, 4099 Porto Codex, Portugal R. B. V INTER Centre for Process Systems Engineering and Department of Electrical and Electronic Engineering, Imperial College of Science, Technology and Medicine, London SW7 2BY, UK AND

H. Z HENG Department of Business Studies, Edinburgh University, 50 George Street, Edinburgh EH8 9JY, UK [Received on 15 March 1999; accepted on 10 September 1999] Necessary conditions in the form of maximum principles are derived for optimal control problems with mixed control and state constraints. Traditionally, necessary condtions for problems with mixed constraints have been proved under hypothesis which include the requirement that the Jacobian of the mixed constraint functional, with respect to the control variable, have full rank. We show that it can be replaced by a weaker ‘interiority’ hypothesis. This refinement broadens the scope of the optimality conditions, to cover some optimal control problems involving differential algebraic constraints, with index greater than unity. Keywords: optimal control; maximum principle; mixed constraints; differential algebraic equations.

1. Introduction Consider the optimal control problem  Minimize h(x(a), x(b))      subject to    x(t) ˙ = f (t, x(t), u(t)) (P)  0 = b(t, x(t), u(t))      u(t) ∈ U (t)    (x(a), x(b)) ∈ C

a.e. t ∈ [a, b] a.e. t ∈ [a, b] a.e. t ∈ [a, b]

where h : n × n → , f : [a, b] × n × m → n , b : [a, b] × n × m → k , U (t) ⊂ m for each t ∈ [a, b] C ⊂ n × n . c The Institute of Mathematics and its Applications 2001 

190

M . D . R . DE PINHO ET AL.

A (Lebesgue) measurable function u : [a, b] → m such that u(t) ∈ U (t) a.e. t ∈ [a, b] is called a control function. A function x ∈ AC([a, b]; n ) satisfying x(t) ˙ = f (t, x(t), u(t)) and 0 = b(t, x(t), u(t)) a.e. is called a state trajectory (corresponding to u). A couple (x, u) comprising a state trajectory x and the control u with which it is associated is called a process. It is a minimizer if it minimizes the cost over processes satisfying the endpoint constraints. Take a minimizer (x, ¯ u) ¯ In this paper we seek necessary conditions of optimality in the form of a maximum principle for (P). A notable feature of the class of problems which we consider is that the differential and algebraic constraints cannot automatically be reduced by state transformations to a more tractable set of pure differential constraints. By considering (P) as a limiting case as  → 0 of an optimal control problem with additional dynamic constraint  y˙ = b, we are led to expect that a maximum principle applies which asserts the existence of an absolutely continuous function p, a measurable function q and a scalar λ
0 such that  p + λ = 1, − p(t) ˙ = p(t) f x (t, x(t), ¯ u(t)) ¯ + q(t)bx (t, x(t), ¯ u(t)), ¯ u → p(t) f (t, x(t), ¯ u) + q(t)b(t, x(t), ¯ u) is maximized over U (t) at u = u(t) ¯ and ( p(a), − p(b)) ∈ NC (x(a), ¯ x(b)) ¯ + λ∂h(x(a), ¯ x(b)). ¯ Here NC and ∂h denote the limiting normal cone and limiting subdifferential (see the definitions below). For simplicity we treat the case of smooth f and b. We focus on the question of whether optimality conditions of this type are valid. (P) is a problem with ‘mixed’ state and control functional equality constraints. Such problems have been addressed by a number of authors (Hestenes, 1966; Stefani & Zezza, 1996, etc). A common hypothesis under which necessary conditions have previously been derived is that the Jacobi matrix ∇u b(t, x(t), ¯ u(t)) ¯ has full rank. An exception is to be found in an earlier paper by Schwarzkopf (1976), who briefly sketches, in the case of a scalar state variable, a derivation of necessary conditions when the full rank condition is replaced by the requirement that δ B ⊂ b(t, x(t), ¯ U (t))

(1.1)

for some δ > 0. (Notice that this ‘interiority’ condition does not guarantee that ∇u b(t, x(t), ¯ u(t)) ¯ has full rank.) This hypothesis is of interest because (in contrast to the full-rankness condition) it is satisfied in certain cases when (P) is a reformulation of an optimal control problem involving differential algebraic systems of index higher than unity (or indeed of systems for which the index is not even defined). Different variational techniques are used in Devdaryani & Ledyaev (in press) to derive a maximum principle, under a less restrictive constraint qualification than (1.1), for problems reducible to free endpoint problems. By contrast, we treat general constrained endpoint constraints. In this paper we provide a simple derivation of necessary conditions for optimal control problems with mixed constraints, under the above interiority hypothesis. Additionally we

PROBLEMS WITH MIXED CONSTRAINTS

191

assume that the generalized velocity set {( f (t, x, u), b(t, x, u)): u ∈ U (t)} is convex. Unlike the results of Schwarzkopf (1976) we allow nonsmooth data. The nature of the hypotheses is explored in a number of examples. The proof technique is to associate with (P) an ‘auxiliary’ control problem, to which we apply a recent sharpened form of the Euler–Lagrange inclusion, and then show that the resulting conditions can be transformed into conditions of the required nature. 2. Preliminaries | · | will denote the Euclidean norm. B is the closed unit ball. A vector a is called nonnegative if all its components are nonnegative, written as a
0. Similarly a matrix A is nonnegative if all its entries are nonnegative, written as A
0. We shall make use of some properties of M-matrices, which we list here for convenience. D EFINITION 2.1 Let A be a m × m real matrix. A is an M-matrix if ai j  0, i  = j and the real parts of the eigenvalues of A are positive. L EMMA 2.2 (Berman & Plemmons, 1994) The following statements are equivalent: (i) A is an M-matrix. −1 (ii) There exists A−1 and  A
0. (iii) aii > 0 and aii > j=i |ai j | for i = 1, . . . , m. We shall also require the following constructs from nonsmooth analysis. D EFINITION 2.3 Given a closed set A ⊂ k and a point x ∈ A, the limiting normal cone to A at x, written N A (x), is the set of all vectors p ∈ k satisfying the following condition: there exist pi → p and xi → x in A such that, for each i pi · (x − xi )  o(|x − xi |) for all x ∈ A, in which o(α)/α → 0 as α ↓ 0.  D EFINITION 2.4 Given a lower semicontinuous function f : k →  {+∞} and a point x ∈ k such that f (x) < +∞, the limiting subdifferential of f at x, written ∂ f (x), is the set ∂ f (x) := {ζ : (ζ, −1) ∈ Nepi{ f } (x, f (x))} in which epi{ f } denotes the epigraph set {(x, α) ∈ k × : α
f (x)}. Properties of limiting normal cones, limiting subdifferentials and associated calculus rules can be found in Clarke (1983), Loewen (1993) and Mordukhovich (1994). Finally, we state an earlier derived weak maximum principle for optimal control problems (see de Pinho & Vinter, 1995) which will be quoted in the proofs.

192

M . D . R . DE PINHO ET AL.

T HEOREM 2.5 Take  > 0. Let (x, ¯ u) ¯ be a minimizer for the problem   Minimize h(x(a), x(b))      subject to x(t) ˙ = f (t, x(t), u(t)) a.e. t ∈ [a, b]    u(t) ∈ U (t) a.e. t ∈ [a, b]    (x(a), x(b)) ∈ C over elements (x, u) comprising an absolutely continuous function x and a measurable function u satisfying the constraints of the above problem and for which |x(t) − x(t)| ¯ 

and

|u(t) − u(t)| ¯ 

a.e.

Assume that (i) f (·, x, u) is measurable, and there exists k ∈ L 1 such that | f (t, x, u) − f (t, x  , u  )|  k(t)|(x, u) − (x  , u  )| ¯ u, u  ∈ u(t) for all x, x  ∈ x(t) ¯ +  B, ¯ +  B¯ and almost all t ∈ [a, b]. (ii) Graph U is a Borel measurable set and (u(t) ¯ +  B) ∩ U (t) is closed for almost all t ∈ [a, b]. (iii) h is locally Lipschitz continuous and C is closed. There exists λ
0, p(·) ∈ AC([a, b]; n ) and ζ (·) ∈ L 1 ([a, b]; n ) such that λ +  p(·) L ∞  = 0 ˙¯ (− p(t), ˙ x(t), ζ (t)) ∈ co ∂ p(t) · f (t, x(t), ¯ u(t)) ¯ ζ (t) ∈ co NU (t) (u(t)) ¯

a.e.

a.e.

¯ x(b)) ¯ + λ∂h(x(a), ¯ x(b)) ¯ ( p(a), − p(b)) ∈ NC (x(a), where ∂ p · f (t, x, u) denotes the limiting subdifferential of (x, p, u) → p · f (t, x, u) for fixed t. 3. Main results We shall invoke the following hypotheses on the data for (P), which make reference to some process (x(·), ¯ u(·)) ¯ and parameter  > 0: H1. f (·, x, ·) and b(·, x, ·) are L × B measurable for each x. H2. There exists an integrable function K F (·) such that, for almost every t ∈ [a, b], f (t, ·, u) is Lipschitz continuous on x(t) ¯ +  B for u ∈ U (t). H3. There exists a constant K b > 0 such that, for almost every t ∈ [a, b], b(t, ·, u) is a continuously differentiable function with Lipschitz constant K b on x(t) ¯ +  B for all for u ∈ U (t) and almost every t ∈ [a, b].

PROBLEMS WITH MIXED CONSTRAINTS

193

H4. There exists a positive number δ such that δ B ⊂ b(t, x(t), ¯ U (t)) for almost every t ∈ [a, b]. H5. h is Lipschitz continuous on a neighbourhood of (x(a), ¯ x(b)) ¯ and C is closed. H6. Graph U is a Borel measurable set. H7. {( f (t, x, u), b(t, x, u)) : u ∈ U (t)} is convex for each t ∈ [a, b]. Define the Hamiltonian: H (t, x, p, q, u) = p · f (t, x, u) + q · b(t, x, u).

(3.1)

T HEOREM 3.1 (Maximum Principle) Let (x(·), ¯ u(·)) ¯ be a minimizing process of (P). Assume that, for some  > 0, hypotheses H1–H7 are satisfied. Then there exist p ∈ AC([a, b]; n ), q(·) ∈ L 1 ([a, b]; k ) and λ
0 such that: (i) (ii) (iii) (iv)

 p∞ + λ = 1. − p(t) ˙ ∈ co ∂x H (t, x(t), ¯ p(t), q(t), u(t)) ¯ a.e. u → H (t, x(t), ¯ p(t), q(t), u) is maximized over U (t) at u = u(t) ¯ a.e. ( p(a), − p(b)) ∈ NC (x(a), ¯ x(b)) ¯ + λ∂h(x(a), ¯ x(b)). ¯

We now consider a number of examples which provide some insights into the nature of the optimal control problems to which the above maximum principle is applicable. E XAMPLE 1 One area of application of optimality conditions for optimal control problems with mixed constraints is to the control of devices modelled by differential algebraic systems of equations (DAE systems). DAE models are widespread in the chemical processing industry. Such problems can take the form  Minimize h(x(a), x(b))      subject to      ˙ = f˜(x(t), y(t), v(t)) a.e. t ∈ [a, b] x(t)  ˜ (P) 0 = b(x(t), y(t)) a.e. t ∈ [a, b]   v(t) ∈ V (t) a.e. t ∈ [a, b]      y(t) ∈ A(t) a.e. t ∈ [a, b]    (x(a), x(b)) ∈ C, where h : n × n → , f˜ : n × l × m → n , b˜ : n × l → l , V (t) ⊂ m , A(t) ⊂ l , C ⊂ n × n . ˜ describes the evolution of the In this problem, the dynamic constraint (x, ˙ 0) = ( f˜, b) ‘differential’ and ‘algebraic’ variables x and y, respectively. In certain cases, a DAE model can be reduced to a system of ordinary differential equations by repeated differentiation. The number of differentiations required to accomplish this task is referred to as the index of the DAE model. Regarding the derivation of necessary conditions, the most studied case is that when the index is unity. Here, it is assumed that ˜ ∇ y b(x, y) has full rank, for all (x, y).

194

M . D . R . DE PINHO ET AL.

˜ One differentiation of t → b(x(t), y(t) then yields a system of differential equations for (x, y). (See de Pinho & Vinter, 1995). The analysis in this paper provides optimality conditions for DAE optimal control problems, involving DAE dynamic constraints for which the index is, possibly, greater than unity. Consider the case of (P) in which n = l = 1, m = 1 and h(x0 , x1 ) = x1 , ˜ f (x, y, v) = d(x)y 3 + v, b˜ = y 3 , C = {0} × R n . Here d(·) :  →  is some Lipschitz continuous function, V (t) = [0, 1] and A(t) = [−1, +1]. The minimizer is clearly (x, ¯ y¯ , v) ¯ = (0, 0, 0). This problem can be fitted to the framework of problem (P). To do this, we regard (y, v) as the control variable u = (u 1 , u 2 ) and set f (t, x, u) = f˜(x, u 1 , u 2 ), b(t, x, u) = ˜ u 1 ). The data for this problem satisfy all the hypotheses of Theorem 3.1 (including b(x, the convexity and interiority conditions H4 and H7), yet the DAE dynamic constraint has infinite index (in the sense that the constraint cannot be reduced to a system of first order differential equations by a finite number of differentiations). The purpose of the next two examples is to illustrate that neither the convexity nor the interiority hypotheses can be dispensed with. E XAMPLE 2 Consider the case of (P) in which n = k = 1, m = 2 and h(x0 , x1 ) = x1 , f (x, y, v) = u 1 , b = x − u 22 , U (t) = [−1, +1] × [−1, +1], C = {0} × R. (x, ¯ u¯ 1 , u¯ 2 ) = (0, 0, 0) is a minimizer for this problem. If the assertions of Theorem 3.1 were valid, there would exist multipliers ( p(·), q(·), λ) such that − p(t) ˙ = q(t) 0=

max

(u 1 ,u 2)∈U (t)

{ p · u 1 − q · u 22 },

− p(1) = λ. The only solution is ( p, q, λ) ≡ (0, 0, 0). This is not possible, in view of the nontriviality property of the multipliers. It follows that the maximum principle, as stated above, is not valid for this problem. In this example, the interiority hypothesis H4 is violated.

PROBLEMS WITH MIXED CONSTRAINTS

195

E XAMPLE 3 Consider finally the case of (P) in which n = k = 1, m = 2 and h(x0 , x1 ) = −x1 , f (x, y, v) = u 31 , b = u1 + u2, U (t) = [−1, +1] × [0, +1]}, C = {0} × R. (x, ¯ u¯ 1 , u¯ 2 ) = (0, 0, 0) is a minimizer for this problem. If the assertions of Theorem 3.1 were valid, there would exist multipliers ( p(·), q(·), λ) such that − p(t) ˙ = 0, 0=

max

(u 1 ,u 2 )∈U (t)

{ p · u 31 + q · (u 1 + u 2 )},

p(1) = λ. If λ = 0, then p(t) ≡ 0. This is not possible, by nontriviality of the multipliers. Suppose, however, that λ > 0. By scaling, we can arrange that λ = 1. It follows that p(t) = −q(t) = 1 and consequently 0
u 31 − u 1 − u 2 for all (u 1 , u 2 ) ∈ U (t). This is impossible. Indeed, if u 1 = − 12 and u 2 = 0, we are led to the contradiction − 18 + 12 = 38 > 0. The maximum principle, as stated above, does not apply to this problem. The pathological aspect of this problem is that the convexity hypothesis H7 is violated. Before proving the Theorem 3.1 we briefly discuss a problem where, in addition to the equality constraint b(t, x, u) = 0, a ‘unilateral’ constraint g(t, x, u)  0 is imposed:  Minimize h(x(a), x(b))      subject to      ˙ = f (t, x(t), u(t)) x(t) (Q) 0 = b(t, x(t), u(t))    0
g(t, x(t), u(t))     u(t) ∈ u(t)    (x(a), x(b)) ∈ C,

a.e. a.e. a.e. a.e.

t t t t

∈ [a, b] ∈ [a, b] ∈ [a, b] ∈ [a, b]

where f , h, b and U are defined as before and g : [a, b] × n × m → l . Assume that (x(·), ¯ u(·)) ¯ is an optimal solution of (Q) and that basic hypotheses H1–H3 and H5–H7 are satisfied when b is replaced by b˜ : [a, b] × n × m →  L (L = k + l) defined by ˜ x, u) := (b(t, x, u), g(t, x, u)). b(t, Furthermore, assume that the following hypothesis is also verified:

196

M . D . R . DE PINHO ET AL.

H4 . There exists a positive scalar δ such that: δ B ⊂ {(b(t, x(t), ¯ U (t)), g(t, x(t), ¯ U (t)) + v) : v ∈ l v
0} for almost all t ∈ [a, b]. Define the Hamiltonian H Q (t, x, p, q, r, u) = p · f (t, x, u) + q · b(t, x, u) + r · g(t, x, u). For problem (Q) the following result holds: C OROLLARY 3.2 Let (x(·), ¯ u(·)) ¯ be a minimizing process for problem (Q). Assume that ˜ together with H4 are satisfied. Then hypotheses H1–H3, H5–H7 (when b is replaced by b) there exist an absolutely continuous function p, functions q ∈ L 1 ([a, b]; k ) and r ∈ L 1 ([a, b]; l ) and a scalar λ
0 such that: (i) (ii) (iii) (iv) (v)

 p∞ + λ = 1. − p(t) ˙ ∈ co∂x H Q (t, x(t), ¯ p(t), q(t), r (t), u(t)) ¯ a.e. w → H Q (t, x(t), ¯ p(t), q(t), r (t), u) is maximized over U (t) at u = u(t) ¯ a.e. ¯ x(b)) ¯ + λ∂h(x(a), ¯ x(b)). ¯ ( p(a), − p(b)) ∈ NC (x(a), r (t)  0, r (t) · g(t, x(t), ¯ u(t)) ¯ = 0 a.e. t ∈ [a, b].

Proof of Corollary 3.2. We start by pointing out that the introduction of a new control v allows us to rewrite (Q) as  Minimize h(x(a), x(b))      subject to      ˙ = f (t, x(t), u(t)) a.e. t ∈ [a, b] x(t)  (Q ) 0 = b(t, x(t), u(t)) a.e. t ∈ [a, b]   0 = g(t, x(t), u(t)) + v(t) a.e. t ∈ [a, b]     l  (u(t), v(t)) ∈ U (t) × {v ∈  : v
0} a.e. t ∈ [a, b]    (x(a), x(b)) ∈ C. This new problem is in the form of P where w = (u, v) is the new control variable and W (t) = U (t) × {v ∈ l : v
0} the control set. Obviously (x, ¯ u, ¯ v) ¯ is feasible for (Q  ) where v(t) ¯ = −g(t, x, ¯ w). ¯ It is easy to see that, in fact, (x, ¯ u, ¯ v) ¯ is optimal for (Q  ).  Additionally observe that H4 implies that ˆ x(t), Bδ (0, 0) ⊂ b(t, ¯ W (t)) for almost all t ∈ [a, b], ˆ x, w) = (b(t, x, u), g(t, x, u) + v). where b(t, We now apply Theorem 3.1 to (Q  ) to obtain functions p, q and r and a scalar λ
0 satisfying (i), (ii) and (iv) of the theorem and p(t) · f (t, x, ¯ u) ¯ =

max { p · f (t, x, ¯ u) + q · b(t, x, ¯ u) + r · g(t, x, ¯ u) + r · v} a.e.

(u,v)∈W (t)

(3.2)

197

PROBLEMS WITH MIXED CONSTRAINTS

Let us fix v = v¯ in (3.2). As v¯ = −g(t, x, ¯ u), ¯ it follows that p(t) · f (t, x, ¯ u) ¯ + r · g(t, x(t), ¯ u(t)) ¯ = max { p · f (t, x, ¯ u) + q · b(t, x, ¯ u) + r · g(t, x(t), ¯ u)} u∈U (t)

a.e.

(3.3)

i.e. we recover (iii). Going back to (3.2) and fixing now u = u¯ we obtain 0 = max{r · (g(t, x, ¯ u) ¯ + v)}.

(3.4)

v
0

If g(t, x(t), ¯ u(t)) ¯ < 0, then (3.4) implies r = 0. If g(t, x(t), ¯ u(t)) ¯ = 0, then (3.4) implies r (t)  0. We then deduce the complementary slackness condition r (t)  0,

r (t) · g(t, x(t), ¯ u(t)) ¯ =0

a.e. t ∈ [a, b]. 

The proof is complete. 4. Proof of Theorem 3.1

The proof technique is to associate with (P) an ‘auxiliary’ control problem, to which we apply an Euler–Lagrange inclusion previously obtained (see Theorem 2.5), and then show that the resulting conditions can be transformed into conditions of the required nature. The auxiliary problem is a ‘nonsmooth’ optimal control problem. Proof. We verify the assertions of the theorem initially under the additional hypothesis (H*). There exists a function c f ∈ L 1 and a number cb such that | f (t, x(t), ¯ u)|  c f (t)

and

|b(t, x(t), ¯ u)|  cb ,

for all u ∈ U (t) and almost every t ∈ [a, b]. This hypothesis will be eliminated in the final stages of the proof. S TEP 1 Define ei = (ei1 , ei2 , . . . , eik ) ∈ k as  if i = j δ1 ei j = −δ1 /k if i  = j where δ1 ∈ (0, δ) (δ given by H4). Then by H4 and a measurable selection theorem, there exist control functions {u 1 (·), . . . , u k (·)} and {v1 (·), . . . , vk (·)} satisfying b(t, x(t), ¯ u i (t)) = ei , b(t, x(t), ¯ vi (t)) = −ei . Fix i. For almost every t ∈ [a, b] we have bi (t, x, u i (t))
bi (t, x(t), ¯ u i (t)) − K b |x − x(t)| ¯ = δ1 − K b |x − x(t)|, ¯ bi (t, x, u(t)) ¯  K b |x − x(t)|, ¯

198

M . D . R . DE PINHO ET AL.

and for j  = i b j (t, x, u i (t))  b j (t, x(t), ¯ u i (t)) + K b |x − x(t)| ¯ = −δ1 /k + K b |x − x(t)|, ¯ b j (t, x, u(t)) ¯
−K b |x − x(t)|. ¯ Define

Setting 1 = min we have

∆b(t, x, u) := b(t, x, u) − b(t, x, u(t)). ¯

δ1 2k K b , 

we deduce that for all x ∈ x(t) ¯ + 1 B, almost every t ∈ [a, b]

∆bi (t, x, u i (t))
δ1 − K b |x − x(t)| ¯ − K b |x − x(t)| ¯ = δ1 − 2K b |x − x(t)| ¯ > 0.

(4.1)

∆b j (t, x, u i (t))  −δ1 /k + K b |x − x(t)| ¯ + K b |x − x(t)| ¯ = −δ1 /k + 2K b |x − x(t)| ¯ < 0.

(4.2)

and, for j  = i,

Similarly, we can show that for all x ∈ x(t) ¯ + 1 B and almost every t ∈ [a, b] we obtain ∆b j (t, x, u i (t)) > −δ1 /k − 2K b |x − x(t)|, ¯

(4.3)

and also that ∆bi (t, x, vi (t)) < 0, ∆b j (t, x, vi (t)) > 0

if j  = i.

Define the k × k matrices: B1 (t, x) := (∆b(t, x, u 1 (t)), . . . , ∆b(t, x, u k (t))) and B2 (t, x) := (∆b(t, x, v1 (t)), . . . , ∆b(t, x, vk (t))). L EMMA 4.1 There exists an   > 0 such that for all x ∈ x(t) ¯ +   B and almost every t ∈ [a, b] both matrices B1 (t, x) and −B2 (t, x) are M-matrices. Proof. We prove that B1 (t, x) is an M-matrix. From (4.1), (4.2) and (4.3) it follows that for   ∈ (0, 1 ), x ∈ x(t) ¯ +   B and for almost all t ∈ [a, b],  δ1 (k − 1) + 2K b (k − 1)|x − x(t)| |∆b j (t, x, u i (t))|  ¯ k j=i < δ1 − 2K b |x − x(t)| ¯  ∆bi (t, x, u i (t)). This follows from Lemma 2.2 that for all x ∈ x(t) ¯ +   B and almost all t ∈ [a, b], B1 (t, x) is an M-matrix. Similar arguments show that −B2 (t, x) is an M-matrix. This completes the proof. 

199

PROBLEMS WITH MIXED CONSTRAINTS

Lemma 4.1 and Lemma 2.2 assure us that B1−1 (t, x)
0 and B2−1 (t, x)  0 for all x ∈ x(t) ¯ +   B and almost all t ∈ [a, b]. It can be easily checked that there exists a scalar m b > 0, which depends on δ and m, such that |B1−1 (t, x(t))| ¯  mb

and

|B2−1 (t, x(t))| ¯  mb.

(4.4)

M (which includes S TEP 2 We now choose a finite collection of controls functions {wi (·)}i=1 u(·)). ¯ Let β(t) = (β1 (t), . . . , β M (t)) be any measurable function taking values in S where  M  S = (β1 , . . . , β M ) ∈  M : βi  1, βi
0, i = 1, . . . , M . (4.5) i=1

Define the vector-valued functions γ 1 , γ 2 : [a, b] × n × S → k as  M  γ 1 (t, x, β) := max −b(t, x, u(t)) ¯ − βl ∆b(t, x, wl (t)), 0 , l=1

 γ (t, x, β) := min −b(t, x, u(t)) ¯ − 2

M 

(4.6)

βl ∆b(t, x, wl (t)), 0 ,

(4.7)

l=1

where the max and min are taken componentwise. Observe that γ 1 (t, x, β)
0 and γ 2 (t, x, β)  0 for all (t, x, β) ∈ [a, b] × n × S. The functions γ i (·, x, β) are L measurable, γ i (t, ·, ·) are Lipschitz continuous on (x(t) ¯ +   B) × S, for i = 1, 2, and 1 2 ¯ 0) = γ (t, x(t), ¯ 0) = 0 for almost every t ∈ [a, b]. γ (t, x(t), Define the vector-valued functions α 1 , α 2 : [a, b] × n × S → k as α 1 (t, x, β) := B1−1 (t, x)γ 1 (t, x, β),

(4.8)

B2−1 (t, x)γ 2 (t, x, β).

(4.9)

α (t, x, β) := 2

Because γ 1 (t, x, β)
0, B1−1 (t, x)
0, γ 2 (t, x, β)  0 and B2−1 (t, x)  0 we have, for all x ∈ x(t) ¯ +   B, all β ∈ S and almost every t ∈ [a, b]: α 1 (t, x, β)
0,

(4.10)

α (t, x, β)
0.

(4.11)

2

By definition of α 1 , α 2 and γ 1 , γ 2 we have b(t, x, u(t)) ¯ + B1 (t, x)α 1 (t, x, β) + B2 (t, x)α 2 (t, x, β) +

M 

βl ∆b(t, x, wl (t)) ≡ 0.

l=1

Finally we define the n × k matrices: F1 (t, x) := (∆ f (t, x, u 1 (t)), . . . , ∆ f (t, x, u k (t))), F2 (t, x) := (∆ f (t, x, v1 (t)), . . . , ∆ f (t, x, vk (t))),

(4.12)

200

M . D . R . DE PINHO ET AL.

and a function φ(t, x, β) := f (t, x, u(t)) ¯ + F1 (t, x)α 1 (t, x, β) + F2 (t, x)α 2 (t, x, β) +

M 

βl ∆ f (t, x, wl (t)).

l=1

Obviously φ(·, x, β) is L measurable and φ(t, ·, ·) is Lipschitz continuous near (x(t), ¯ 0).

S TEP 3 For some η > 0, set Sη := S (ηB) and consider the following problem:   Minimize h(x(a), x(b))      subject to (R) x(t) ˙ = φ(t, x(t), β(t)) a.e. t ∈ [a, b]    β(t) ∈ S a.e. t ∈ [a, b]  η   (x(a), x(b)) ∈ C. Observe that (x, ¯ β¯ ≡ 0)) is a feasible process for (R). Suppose there exists another feasible process (x, β) for (R) with lesser cost, i.e. h(x(a), x(b)) < h(x(a), ¯ x(b)). ¯ By H7 and (4.12) we deduce that (x, ˙ 0) ∈ {( f (t, x, u), b(t, x, u)): u ∈ U (t)}. Measurable selection theorems assert the existence of a control function u˜ such that x˙ = f (t, x, u) ˜ and 0 = b(t, x, u). ˜ So (x, u) ˜ is a feasible process for (P), contradicting the optimality of (x, ¯ u). ¯ So (x, ¯ β¯ ≡ 0) is an optimal solution for (R). The Hamiltonian for (R) is H R (t, x, p, β) : = p · φ(t, x, β) = p · f (t, x, u) ¯ + p · F1 (t, x)α 1 (t, x, β) + p · F2 (t, x)α 2 (t, x, β) + p·

M 

βl ∆ f (t, x, wl (t)).

l=1

Problem (R) satisfies the conditions under which Theorem 2.5 applies. This result asserts the existence of λ
0, p(·) ∈ A ⊂ ([a, b]; n ) and ζ (·) ∈ L 1 ([a, b];  M ) such that λ +  p(·) L ∞ = 1, ˙¯ (− p(t), ˙ x(t), ζ (t)) ∈ co ∂x, p,β H R (t, x(t), ¯ p(t), 0) a.e. ζ (t) ∈ co N Sη (0) a.e. ( p(a), − p(b)) ∈ NC (x(a), ¯ x(b)) ¯ + λ∂h(x(a), ¯ x(b)). ¯

(4.13) (4.14) (4.15) (4.16)

We deduce from the max rule (Clarke, 1983) and a selection theorem the following estimates for the generalized Jacobians: ∂x,β γ 1 (t, x, ¯ 0) ⊂ {−M(t)(∇x b(t, x, ¯ u), ¯ ∆b(t, x, ¯ w1 ), . . . , ∆b(t, x, ¯ w M )) : M(t) = diag(µi (t), . . . , µk (t)), µi (t) ∈ [0, 1] measurable}

201

PROBLEMS WITH MIXED CONSTRAINTS

and ∂x,β γ 2 (t, x, ¯ 0) ⊂ {−N (t)(∇x b(t, x, ¯ u), ¯ ∆b(t, x, ¯ w1 ), . . . , ∆b(t, x, ¯ w M )) : N (t) = diag(ν1 (t), . . . , νk (t)), νi (t) ∈ [0, 1] measurable}. We then conclude that (− p(t), ˙ ζ (t)) ∈ {co ∂x p · f (t, x(t), ¯ u(t))} ¯ × { p · ∆ f (t, x(t), ¯ w1 (t)), . . . , p · ∆ f (t, x(t), ¯ w M (t))} + {q · ∇x b(t, x(t), ¯ u(t))} ¯ × {q · ∆b(t, x(t), ¯ w1 (t)), . . . , q · ∆b(t, x(t), ¯ w M (t))}

(4.17)

for almost all t ∈ [a, b], where (for some L ∞ functions M and N ) q(t) := −M(t)(B1−1 )T (t)F1T (t) p(t) − N (t)(B2−1 )T (t)F2T (t) p(t)

(4.18)

and F1 , F2 , B1−1 and B2−1 are evaluated at (t, x(t)). ¯ In view of (4.4) and (H ∗ ), we deduce the existence of an integrable function K q such that |q(t)|  K q (t)| p(t)|.

(4.19)

If we write H (t, x, p, q, u) := p · f (t, x, u) + q · b(t, x, u) and also note that ζ (t) ∈ N Sη (0) = {(η1 , . . . , η M ) ∈  M : ηi  0,

i ∈ {1, . . . , M}},

we obtain from (4.17) that − p(t) ˙ ∈ co ∂x H (t, x(t), ¯ p(t), q(t), u(t)) ¯

(4.20)

and p · ∆ f (t, x, ¯ wi ) + q · ∆b(t, x, ¯ wi )  0

for

i = 1, . . . , M,

(4.21)

that is, Maxu∈Uˆ (t) {H (x(t), ¯ p(t), q(t), u)} = H (x(t), ¯ p(t), q(t), u(t)) ¯

a.e.

in which Uˆ (t) =

M  {wi (t)}. i

S TEP 4 Our next task is to show that this last relationship remains true when we replace Uˆ (t) by U (t). This is achieved by reasoning along the lines of the proof of Theorem 5.1.1 of Clarke (1983).

202

M . D . R . DE PINHO ET AL.

It is convenient to introduce the following scaled version q(·) ˜ of q(·): q(t) ˜ = K q (t)−1 q(t).

(4.22)

In view of (4.20), (4.13) and the Lipschitz continuity and differentiability hypotheses H2 and H3, we deduce that  p L ∞ is bounded by a constant (which does not depend on our choice of wi s). In view of (4.21) and (4.19) we can find a constant L (again independent of the wi ) such that r  L ∞  L, where r (·) := ( p(·), q(·)). ˜ −1 B ⊃ L B Let {Z j }∞ j=1 be an increasing family of finite subsets of L B such that Z j + j for each j. Fix j and consider z ∈ Z j . Appealing to an appropriate selection theorem we may find a measurable function u z (·), u z (t) ∈ U (t) a.e., such that

sup {z · β(t, u)}  z · β(t, u z (t)) + j −1

u∈U (t)

a.e.

(4.23)

Here β(t, u) := ( f (t, x(t), ¯ u), c f (t)b(t, x(t), ¯ u)). For each j and t ∈ [a, b], define the measurable multifunction U j (t) = {u z (t): z ∈ Z j }. Let us now apply the results of the earlier analysis for the choice of ‘finitely generated’ control constraint set Uˆ (t) = U j (t). This tells us that there exist p j ∈ AC, q˜ j ∈ L ∞ and λ j
0 such that − p˙ j (t) ∈ co ∂x p j · f + {c f q˜ j · ∇x b}, a.e., ( p j (a), − p j (b)) ∈ NC (x(a), ¯ x(b)) ¯ + λ j ∂h(x(a), ¯ x(b)), ¯ ( p j (t), q˜ j (t)) · β(t, u)  ( p j (t), q˜ j (t)) · β(t, u(t)) ¯ for all u ∈ U j (t), a.e., λ j +  p j (·) L ∞ = 1.

(4.24) (4.25) (4.26) (4.27)

Now {( p j (·) L ∞ }, {(q˜ j (·) L ∞ } and {λ j } are uniformly bounded sequences and { p˙ j } is a uniformly integrably bounded sequence. Following extraction of subsequences we have then that p j (·) → p(·) uniformly,

q˜ j (·) → q(·) ˜ weakly∗ in L ∞

for some p ∈ AC and q˜ ∈ L ∞ , and { p˙ j } has a weak limit in L 1 . A straightforward modification of the proof of Clarke (1983, Theorem 3.1.7) justifies passing to the limit in (4.24) to obtain − p(t) ˙ ∈ co ∂x p · f + q · ∇x b a.e., where q := K q q˜ and in which the generalized gradient is evaluated at (t, x(t), ¯ p(t), q(t)). We deduce from (4.25) and the upper semicontinuity property of the limiting normal cones that ( p(a), − p(b)) ∈ NC (x(a), ¯ x(b)) ¯ + λ∂h(x(a), ¯ x(b)). ¯ Also, from (4.27), λ +  p(·) L ∞ = 1.

PROBLEMS WITH MIXED CONSTRAINTS

203

We turn now to the ‘maximization of the Hamiltonian’ condition. Let u(·) ˜ be an arbitrary control function. Fix j. For almost every t ∈ [a, b], z ∈ Z j can be chosen such that |( p j (t), q˜ j (t))−z|  j −1 . It follows that, for almost every t, ( p j (t), q˜ j (t)) · β(t, u(t)) ˜  z · β(t, u(t)) ˜ + j −1 (c f (t) + cb c f (t))  z · β(t, u z (t)) + j −1 (1 + c f (t) + cb c f (t)) by (4.23)  ( p j (t), q˜ j (t)) · β(t, u z (t)) + j −1 (1 + 2[c f (t) + cb c f (t)])  ( p j (t), q˜ j (t)) · β(t, u(t)) ¯ + j −1 (1 + 2[c f (t) + c f (t)cb ]) by (4.26). We conclude that 

b

{ p j · [ f (u) ¯ − f (u)] ˜ + q˜ j · c f [b(u) ¯ − b(u)]} ˜ dt
−α j ,

a

in which f (u) ¯ denotes f (t, x(t), ¯ u(t)), ¯ etc., and α j := j −1 [|b − a| + 2c f  L 1 + 2cg (c f  L 1 )]. Because p j (·) → p(·) uniformly, q˜ j (·) → q(·) ˜ weakly∗ in L ∞ and q := kφ q, ˜ we obtain in the limit  b {[ p · f (u) ¯ + q · b(u)] ¯ − [ p · f (u) ˜ + q · b(u)]} ˜ dt
0. (4.28) a

This inequality, which is valid for any control u, ˜ is an ‘integral’ form of the maximization of the Hamiltonian condition. S TEP 5 A version of the maximum principle has been proved under the additional hypothesis (H*). Suppose now that (H*) is possibly not satisfied. We set up another problem for which (H*) is satisfied, which involves restricting U (·) and convexifying the resulting velocity set. (This last modification is carried out to ensure continuing satisfaction of hypotheses H1–H7). The special case of the maximum principle already proved is applied to this problem, leading to the desired optimality condition for the original problem. Take U˜ (t) := {u ∈ U (t): | f (t, x(t), ¯ u) − f (t, x(t), ¯ u(t))| ¯ 1 and |b(t, x(t), ¯ u)|  1}, and consider a variant of (P) in which we replace the ‘differential-algebraic’ constraint by   n+k n+k   (x, ˙ 0) = αi f (t, x, vi ), αi b(t, x, vi ) , i=0

i=0

204

M . D . R . DE PINHO ET AL.

in which α = (α0 , . . . , αn+k ) and v = (v0 , . . . , vn+k ) are regarded as control variables, subject to the constraints α(t) ∈ P n+k Here,

and

 



P n+k : = (α0 , . . . αn+k ) :

v(t) ∈ U˜ (t) × · · · × U˜ (t)·

n+k 





αi = 1, αi
0, for all i .

i=0

The new problem satisfies H1–H7, and also (H*). Besides, (x, ¯ α(t) ≡ (1, 0, . . . , 0), v(t) = (u(t), ¯ . . . , u(t))) ¯ is a minimizer. The preceding analysis provides elements p ∈ AC, q ∈ L 1 and λ
0, satisfying (4.24), (4.25) and (4.27). Also, the ‘integral’ condition (4.28) is satisfied for every u˜ in U˜ (t). We show that this last condition implies max { p · f (u) + q·b(u)} = p(t) · f (u(t)) ¯ + q · b(u(t)) ¯

u∈U (t)

a.e.

(4.29)

(where f (u) = f (t, x(t), ¯ u), etc.). If this were not true, we would be able to find a control u  (·) ∈ U (·) such that p(t) · [ f (u  (t)) − f (u(t))] ¯ + q(t) · [b(u  (t)) − b(u(t))] ¯
0

a.e.,

and the inequality would be strict on a subset of positive measure. Define the measurable function γ : [a, b] → [0, 1) −1 γ (t) := (1 + | f (u  (t)) − f (u(t))| ¯ + |b(u  (t)) − b(u(t))|) ¯ .

Then p(t) · [ f (u(t)) ¯ + γ (t)( f (u  (t)) − f (u(t)))] ¯ + q(t) · [b(u(t)) ¯ + γ (t)(b(u  (t)) − b(u(t)))] ¯ −[ p(t) · f (u(t)) ¯ + q(t) · b(u(t))] ¯
0, and the inequality is, once again, strict on a subset of positive measure. Appealing to the convexity of {( f (t, x, u), b(t, x, u)): u ∈ U (t)} and a measurable selection theorem, we can find u(·) ˜ ∈ U (·) such that f (u(t)) ˜ = f (u(t)) ¯ + γ (t)[ f (u  (t)) − f (u(t))] ¯ 

b(u(t)) ˜ = b(u(t)) ¯ + γ (t)[b(u (t)) − b(u(t))] ¯

a.e., a.e.

It follows that  b ([ p(t) · f (u(t)) ˜ + q(t) · b(u(t)))] ˜ − [ p(t) · f (u(t)) ¯ + q(t) · b(u(t))]) ¯ dt > 0. a

But for our choice of γ , u˜ is actually a member of U˜ (t), which means that (4.28) is satisfied. (4.29) must therefore be true. The theorem is proved. 

PROBLEMS WITH MIXED CONSTRAINTS

205

Acknowledgement M. d. R. d. P. was supported by F. C. T. Project 2438/95, Portugal. R EFERENCES B ERMAN , A. & P LEMMONS , R. J. 1994 Nonnegative Matrices in the Mathematical Sciences. Philadelphia: SIAM Classics in Applied Mathematical. C LARKE , F. H. 1983 Optimization and Nonsmooth Analysis. New York: Wiley. D EVDARYANI , E. N. & L EDYAEV , Y. S. 1999 Maximum principle for implicit control systems. Appl. Math. Optim., 40, 79–103. H ESTENES , M. R. 1966 Calculus of Variations and Optimal Control Theory. New York: Wiley. L OEWEN , P. D. 1993 Optimal control via nonsmooth analysis. CRM Procedings and Lecture Notes. Providence: American Mathematical Society. M ORDUKHOVICH , B. S. 1994 Generalized differential calculus for nonsmooth and set-valued mappings. J. Math. Anal. Appl., 183, 250–288. DE P INHO , M. D . R. & V INTER , R. B. 1995 An Euler–Lagrange inclusion for optimal control problems. IEEE Trans. Autom. Control, AC 40, 1191–1198. S TEFANI , G. & Z EZZA , P. L. 1996 Optimality conditions for constrained control problems. SIAM J. Control Optim., 34, 635–659. S CHWARZKOPF , A. B. 1976 Relaxed control problems with equality state constraints. SIAM J. Optim. Theor. Appl., 19, 455–468.