2009 American Control Conference Hyatt Regency Riverfront, St. Louis, MO, USA June 10-12, 2009

ThB20.1

Krasovskii’s Method in the Stability of Network Control Diego Feijer

and

Abstract— We consider network resource allocation problems based on convex optimization, and their decentralized solutions by means of primal, dual, or primal-dual subgradient control. We show how Krasovskii’s method, that seeks Lyapunov functions which are quadratic forms of the vector field, provides new global stability proofs for various problems of this kind. Applications include congestion control, cross-layer congestion and contention control, and other general network utility maximization problems. We show more generally how this proof method applies to concave-convex saddle point problems solved by subgradient methods.

I. I NTRODUCTION Since the seminal work of Kelly in [4], it has become standard to study network resource allocation in the following terms: (i) a global, convex optimization objective, usually in terms of network utility maximization; (ii) decentralized, subgradient algorithms for its solution over a network, based on primal and/or dual decompositions; (iii) global convergence proofs of these algorithms to the optimum, usually based on Lyapunov techniques. For the congestion control problem, an excellent summary of many contributions with the above features is [14]. More recently, generalizations of this strategy to multiple layers of network protocols have been pursued, as discussed in the surveys [3], [9]. Finding a Lyapunov function to establish global stability is sometimes straightforward, when the dynamics are a gradient law for a single set of variables: this happens for primal congestion control laws [4], where rates follow the gradient of a modified objective function (utility minus a congestion cost), for dual laws that follow the gradient of the dual [10], and also for routing problems that follow the negative gradient of a congestion cost (see [13]). In these cases, the objective itself must be monotonic along trajectories. When multiple sets of variables are simultaneously controlled (primal-dual, or cross layer problems), finding a Lyapunov function is less obvious: for instance, primal-dual laws are seeking a saddle point of the Lagrangian, around which it is not sign definite; the Lagrangian cannot be a Lyapunov function. Proofs in these cases use quadratic Lyapunov functions (see [14]), motivated in some cases by a passivity decomposition [15]. However, such decompositions become less obvious for cross-layer problems with multiple optimization variables. In this paper we pursue another source of Lyapunov proofs for such problems: Krasovskii’s method [6], [5], that applies to the system z˙ = F (z) a Lyapunov function V (z) = z˙ T Qz˙ = F (z)T QF (z). Diego Feijer and Fernando Paganini are with the Department of Electrical Engineering, Universidad ORT Uruguay, Cuareim 1451, 11.100 Montevideo (e-mails: {feijer,paganini}@ort.edu.uy). Research supported by AFOSR-US, and by ANII-Uruguay, grant FCE 2007 265.

978-1-4244-4524-0/09/$25.00 ©2009 AACC

Fernando Paganini
T
Q + Q ∂F is negative If Q > 0 is found such that ∂F ∂z ∂z semidefinite at every point z, then V˙ ≤ 0, which provides the essential step for a Lyapunov proof, possibly supplemented by the use of LaSalle’s invariance principle (see [5]). In Section II we apply this method to give new stability proofs in primal, dual, and primal-dual congestion control algorithms. It is in the last case where the method is most interesting, exploiting a particular symmetry in the equations to lead to a natural candidate for Q. This motivates in Section III the generalization to primal-dual methods for cross-layer problems. One such problem is joint congestion and contention control for wireless networks, introduced in [7], [8], where convexity is achieved by a logarithmic change of variables. [8] proposes primal algorithms for this problem, and [16] studied a dual version. We are mostly interested in the primal-dual approach, first studied in [17] with stochastic models. Here, using fluid models, we embed the primal-dual case in a class of problems for which Krasovskii’s method provides a Lyapunov stability proof. In Section IV, we see that the symmetry principle in our primal-dual construction appears more generally in any gradient method for a saddle point problem, as studied classically in [1]; thus, the Krasovskii method also covers this situation. Conclusions and future work are discussed in Section V. The Appendix contains some technical proofs. II. N ETWORK C ONGESTION C ONTROL We work here with the standard setup for congestion control as in [14]. The network is made of a set of sources S and a set of links L, of cardinality n and m, respectively. Each source s injects packets into the network at a rate xs , and has an associated strictly concave utility function Us (xs ). These packets use a subset L(s) ⊆ L of links. On the other hand, each link l transports traffic of a subset of sources P S(l) ⊆ S, with total rate yl = s∈S(l) xs . The link capacity is cl . We can summarize the relationship between the vectors of source and link rates through y = Rx, where the routing matrix R of dimension m × n, is defined by: ( 1 if s ∈ S(l), Rls = 0 otherwise. The resource allocation proposed by Kelly [4] is the solution of the following network utility maximization. P Problem 1: Maximize s∈S Us (xs ), over xs ≥ 0, subject to the capacity constraints

3292

y l ≤ cl ,

l ∈ L.

(1)

D(λ) = max L(x, λ).

A. Primal Congestion Control

x≥0

A primal algorithm for the resource allocation involves a dynamic law for the source rates xs . In order to enforce (approximately) the capacity constraints, a price signal λl = ϕl (yl ) is introduced, through an increasing penalty function ϕl (·) for the constraint yl ≤ cl . The total price per source X qs = λl (2)

The condition for x to be the maximizer of (7) is xs = Us′−1 (qs ),

lim fs (qs ) = 0.

qs →∞

acts as a penalty term for the control of the source rates,

Here κs > 0, and the positive projection is defined by  w, if w > 0 or z > 0; + [w]z := 0 otherwise.

(3)

(4)

The projection is said to be active if the second case applies. The above algorithm does not solve Problem 1 exactly, but rather the barrier " approximation Z # X X yl max Us (xs ) − ϕl (y)dy . (5) x≥0

s∈S

l∈L

0

In fact, x˙ in (3) is a subgradient of the modified objective in (5), which makes it a natural Lyapunov function for proving convergence, as indeed was done in [4], [14]. Here, as a first simple example of the use of Krasovskii’s method, we wish to consider the alternate function X x˙ 2 1 s V (x) = x˙ T K −1 x˙ = , (6) 2 2κs s∈S

where K = diag(κs ). This function can be discontinuous at switches of the projection operation; we will show later how these can be treated. For now, we avoid this issue, differentiating V at a point with xs > 0 to get X x˙ s x ¨s V˙ (x) = κs s∈S   X X = x˙ s Us′′ (xs )x˙ s − ϕ′l (yl )y˙ l  s∈S

Since Us is strictly concave for all s ∈ S, and ϕl is non decreasing for all l ∈ L, we see that V˙ (x) ≤ 0. A LaSalle argument can complete the asymptotic stability proof, analogous to others covered later in this paper. B. Dual Congestion Control Problem 1 can be solved exactly by using duality: let λ = (λl )l∈L be the Lagrange multipliers associated with the capacity constraints (1). The Lagrangian is X X L(x, λ) = Us (xs ) − λl (yl − cl ). s∈S

l∈L

Strong duality implies that Problem 1 is equivalent to the dual optimization minλ≥0 D(λ), where the Lagrange dual function D : Rm + → R is

(8)

(9)

ˆ the primal-dual optimal points, that We denote by x ˆ and λ satisfy the Karush-Kuhn-Tucker conditions for this problem: (8) and the complementary slackness condition ( = 0 yˆl < cl , ˆ (10) λl ≥ 0 yˆl = cl . Dual flow control consists of a static update of source rates following (8), and a dynamic control of the prices, λ˙l = γl [yl − cl ]+ λl .

(11)

Here γl > 0, and the positive projection is defined as in (4). The above price dynamics amounts to a subgradient algorithm for the dual function, indeed ∂D . ∂λ This observation makes D(λ) a natural Lyapunov function for these dynamics, see e.g. [13]. Again, we would like to look at the alternative of Krasovskii’s method, so consider the Lyapunov candidate (here Γ = diag(γl )) y−c=−

V (λ) =

1 ˙ T −1 ˙ X λ˙ 2l λ Γ λ= . 2 2γl

(12)

l∈L

We again ignore initially the projections and compute the derivative of V at a point λ > 0: V˙ =

X λ˙ l λ ¨l l∈L T

γl

= λ˙ T Rx˙ = λ˙ T R diag {fs′ (qs )} q˙

= q˙ diag {fs′ (qs )} q˙ ≤ 0,

l∈L(s)

= x˙ T diag{Us′′ (xs )}x˙ − x˙ T RT diag{ϕ′l (yl )}Rx. ˙

or xs = 0 and Us′ (0) < qs ,

where qs is defined as in (2), for these new prices λ. The above condition defines a decreasing “demand curve” xs = fs (qs ); we make the standing assumption

l∈L(s)

x˙s = κs [Us′ (xs ) − qs ]+ xs .

(7)

(13)

since the demand curve fs is decreasing. We now treat more carefully the projections, that make the dynamics (11) a hybrid system of the type considered in [2], [11]. Specifically, let the discrete state σ denote the subset of links for which the positive projection is active; σ has a finite set of alternatives. (11) is equivalent to ( γl (yl − cl ), l 6∈ σ(t); ˙ λl = (14) 0 l ∈ σ(t). The Lyapunov function (12) takes the form X γl V (σ, λ) = (yl − cl )2 . 2

(15)

l6∈σ

For an interval of time in which σ(t) ≡ σ0 constant, the calculations in (13) are still valid: the saturated links do not appear since λl ≡ 0 for l ∈ σ0 . Therefore V will still

3293

decrease in this interval. It remains to study the behavior of V at times when the set σ of active projections changes; here, V may be discontinuous. Notice, however: − • The set σ will be enlarged if a link l with yl (t ) ≤ cl reaches λl (t) = 0. In that case, the sum in (15) loses a term, and so V (t− ) ≥ V (t+ ). V is discontinuous but in the decreasing direction. • Consider now a link l which had an active projection at t− , and no longer has it at t+ . This happens only if yl − cl went through zero, from negative to positive, at t. An extra term is added to (15), but this term is initially at zero. Hence there is no discontinuity of V . Therefore, V satisfies the monotonicity conditions of a Lyapunov function. To prove asymptotic stability, a LaSalle argument is given in the Appendix. C. Primal-Dual Congestion Control The primal-dual control scheme uses subgradient dynamics in both primal and dual variables; i.e. , rates are updated as in (3), where q satisfies (2) with link prices generated by (11). In other words, we have  + ∂L x˙ = K = K[U ′ (x) − RT λ]+ (16) x, ∂x x  + ˙λ = Γ − ∂L = Γ[Rx − cl ]+ (17) λ. ∂λ λ Here K = diag(κs ), Γ = diag(γl ) and U ′ (x) is the column vector of Us′ (xs ). In this case the Lyapunov choice is less obvious, since the dynamics seek a saddle point, not an extremum, of the Lagrangian. The first global stability proof for this law in the network control literature was the passivity argument in [15], which results in a quadratic Lyapunov function not directly related to the primal or dual objectives1. However, when using Krasovskii’s method, the primaldual case is a natural combination of the previous cases. Superimposing the functions in (6) and (12), introduce the following Lyapunov function in the state z = (x, λ)T :   1 K −1 0 V (z) = z˙ T Qz, ˙ with Q = . (18) 0 Γ−1 2 Consider a point with x > 0 and λ > 0, so projections are inactive. The dynamics (16)-(17) have the form z˙ = F (z), with the derivative of the vector field F (z) satisfying     ∂F 1 diag{Us′′ (xs )} −RT Q = . R 0 ∂z 2 The antisymmetric structure of the above matrix has its roots in the fact that (16)-(17) are derived from gradients of the same Lagrangian: as we will see later in more generality, we are exploiting the identity ∂2L ∂2L = . ∂xs ∂λl ∂λl ∂xs 1 In Section IV we see that the quadratic Lyapunov proof also follows as a special case of the classical study [1].

As a consequence of this symmetry we have  T    ∂F ∂F diag{Us′′ (xs )} Q+Q = 0 ∂z ∂z

0 0



,

negative semidefinite; so again we have V˙ ≤ 0, the basis for a Lyapunov argument. The details of the proof are relayed to Theorem 1 below, which generalizes primal-dual laws to a larger class of problems. III. G ENERALIZATIONS AND

CROSS - LAYER

OPTIMIZATION

As a natural continuation of the congestion control studies, recent research has incorporated into the optimization framework other resources to be allocated in networks, involving other layers of protocols. In particular, many researchers have tackled the control of routing, medium access control (MAC), or physical layer control, most prominently for the case of wireless networks. Recent surveys which cover part of the substantial literature are [9], [3]. In particular, [3] advocates the network utility maximization problem (NUM), and its decompositions, as the unifying paradigm over which network architectures can be designed and controlled. We now formulate one such NUM problem, which generalizes Problem 1. P Problem 2: Maximize s∈S Us (xs ), subject to hl (x) ≤ cl (p),

∀l ∈ L,

where {hl } and {cl } are twice continuously differentiable convex and concave functions, respectively. Through the variable p ∈ P ⊆ Rk above, we can represent the dependence of link capacity on parameters of the lower layers, e.g. the MAC protocols or physical layer parameters. Letting x enter the constraint in a possibly nonlinear (convex) way, provides more generality for the choice of primal variables, other than link rates themselves. An example is described in Section III-A. We introduce the Lagrangian for this problem, X X L(x, p, λ) = Us (xs ) − λl (hl (x) − cl (p)). (19) s∈S

l∈L

The dual problem will be minλ≥0 D(λ), where the Lagrange dual function can be written as ( ) X T D(λ) = max Us (xs ) − λ h(x) + max λT c(p). x

s∈S

p

(20) Note the “dual decomposition” (between variables x and p) in the solution of (20). Below, we will study two control algorithms to solve the dual problem, with stability proofs using the Krasovskii method. A. Cross-Layer Congestion and Contention Control In this section we motivate the formulation in Problem 2 through the following application, which originates in [7], [8]. Consider an ad-hoc wireless network made up of a set of nodes, which use a random MAC. Each node n accesses the medium with probability P n , and when transmitting it

3294

chooses one of its outgoing links with probability pl . Thus, we have the following convex constraints (which define the set P) for every node: X pl ≥ 0, pl = P n , P n ≤ 1. (21) l∈Lout (n)

The transmission of a link is interfered by another link, if the receiver of the former is within range of the transmitter of the latter; when such a collision occurs, we assume no useful information is transmitted. So the capacity of a link l depends on the probability of accessing the channel without presence of the interfering nodes; we write Y Cl (p) := cl pl (1 − P k ), k∈NI (l)

where NI (l) is the set of nodes that interfere with link l. Under this additional degree of freedom, we wish to allocate network resources (rates and transmission probabilities) to optimize an overall network utility. The optimal congestion and contention control problem is thus X Ξ : maximize Us (xs ) s∈S

subject to (21) and

X

xs ≤ Cl (p),

∀l. (22)

s∈S(l)

The difficulty with the above problem is that the last constraint is non-convex; nevertheless, a change of variables was proposed in [7] that helps tackle this problem. Take the logarithm on both sides of inequality (22), and define
˜s (˜ x ˜s := log xs , U xs ) := Us ex˜s . Then problem Ξ is equivalent to X ˜ : maximize ˜s (˜ Ξ U xs )

decentralization through additional variables per source and link, not a very scalable proposition. The separability concerns disappear if one uses a primal-dual approach; this was first studied in [17], where the focus is on stochastic issues associated with estimation of subgradients; a stability result was given involving quadratic Lyapunov functions in discrete time. In this paper we will obtain, as a special case of Theorem ˜ 1 below, a global stability result for primal-dual laws for Ξ, based on deterministic fluid models in continuous time, and Krasovksii’s method. B. Primal-Dual control for Problem 2 and its stability: version with dual update for p. We discuss here a solution of Problem 2 based on a primaldual approach. Lagrange multipliers λ will be dynamically updated to follow a subgradient of the dual function, and the primal variables x will also use gradient control laws. The question arises as to how to deal with the primal variables p, for which in principle we could use either choice: solve directly for the optimal p(λ) in (20), or a gradient approach. In this section we study the former option, attractive for problems where this calculation can be done in a separable way, like the one in Section III-A. Specifically, define ψ(λ) := maxp∈P λT c(p), and denote by p¯(λ) the maximizing p. The function ψ(λ) is convex in λ (maximum of linear functions). In the particular case of Section III-A, we have   X X X  ψ(λ) = λl log p¯l + λk log(1 − P¯ n ) , n∈N

where

s∈S

subject to (21) and



log 

X

s∈S(l)



ex˜s  ≤ C˜l (p),

∀l (23)

where C˜l (p) := log cl + log pl + k∈NI (l) log(1 − P k ). Note that C˜l is concave in the variables pl , P k , and the left hand side of (23) is convex in x ˜s . Therefore, the new formulation falls in the class of Problem 2, provided that the new objective function is concave in x ˜: this happens (see [7]) for utility functions satisfying P

d2 Us (xs ) dUs (xs ) xs + ≤ 0. 2 dxs dxs

(24)

˜ is to find a The remaining challenge for formulation Ξ distributed network solution. In this regard, note that the left hand side of (23) is not separable between primal variables. In a dual approach that would solve the problem exactly, the optimization over x in (20) is not easy to decentralize. [8] tried for this purpose a primal-based approach, where the gradient direction of x can be found in a distributed way, ˜ by a barrier function. but is forced then to approximate Ξ Recently, [16] formulated a dual approach which achieves

k∈LI (n)

l∈L(n)

p¯l (λ) := P

λl P . λ l∈Lout (n) l + l∈LI (n) λl

Note that in this case the optimal p¯l (λ) depends only Lagrange multipliers of nodes neighboring link l; for this reason the calculation is amenable for distributed computation with message passing between nodes. Returning now to the general case, define ¯ λ) = max L(x, p, λ) L(x, p X X = Us (xs ) − λl hl (x) + ψ(λ). s∈S

l∈L

Our primal-dual computation for Problem 2 will be based on gradients of this function: "  ¯ +  T #+ ∂L ∂h ′ x˙ = K = K U (x) − λ , (25) ∂x x ∂x x  ¯ +  + ∂L λ˙ = Γ − = Γ h(x) − ∇ψ T (λ) λ ∂λ λ = Γ [h(x) − c(¯ p(λ))]+ λ ,

(26)

where the second equality in (26) follows from the Envelope Theorem with p¯(λ) = arg maxp λT c(p). We state the following result.

3295

Theorem 1: The optimum of Problem 2 is a globally asymptotically stable equilibrium of the dynamics (25)-(26). Proof: Consider the Lyapunov function defined in (18). Operating again initially at a point z > 0 where all projections are inactive, we have now " 2
#   ∂ L ∂h T − ∂F 1 2 ∂x ∂x = Q ,
∂h ∂z 2 −∇2 ψ(λ) ∂x

where

X ∂2L ′′ = diag{U (x )} − λl ∇2 hl (x). s s ∂x2

(27)

l∈L

Note from the strict concavity of Us (xs ) and the convexity 2 of hl (x), that ∂∂xL2 is negative definite. Also, −∇2 ψ(λ) is negative semidefinite because of the convexity of ψ(λ). Therefore #  T   " ∂2 L 0 ∂F ∂F ∂x2 ≤ 0, Q+Q = ∂z ∂z 0 −∇2 ψ(λ) (28) hence V˙ ≤ 0 along trajectories, as desired. The remainder of the argument, which considers the projections and LaSalle invariance, is covered in the Appendix. C. Primal-Dual control for Problem 2 and its stability: version with primal update for p. An alternative primal-dual law for Problem 2 involves a gradient update of the primal variable p: this could be suitable for a situation where explicit formulas for p¯(λ) are not available. Returning to the original Lagrangian (19), the control law would be " +  T #+  ∂L ∂h ′ x˙ = K = K U (x) − λ , (29) ∂x x ∂x x     ∂L T ∂c p˙ = Υ =Υ λ , (30) ∂p P ∂p P  + ∂L λ˙ = Γ − = Γ [h(x) − c(p)]+ (31) λ , ∂λ λ where Υ = diag{ǫk }, ǫk > 0, and where [·]P denotes the projection on to P. Theorem 2: Suppose that c(p) is a strictly concave function. Then the optimum of Problem 2 is a globally asymptotically stable equilibrium of the dynamics (29)-(31). The proof is omitted due to space limitations; we will only indicate the relevant Lyapunov function based on Krasovskii’s method,

V (z) = z˙ T Qz, ˙

K −1 1 Q=  0 2 0 

0 Υ−1 0

 0 0 . Γ−1

IV. K RASOVSKII ’ S

METHOD AND GRADIENT LAWS FOR SADDLE POINTS

All primal-dual laws considered have the general form     ∂L ∂L x˙ = K , λ˙ = Γ − ; (32) ∂x ∂λ they are thus gradient laws which seek a saddle point (maximum in x, minimum in λ), of a certain function L(x, λ). Here for simplicity we have removed projections. Such algorithms were studied in the classical work of Arrow, Hurwicz and Uzawa [1]; here it was proved, using a quadratic Lyapunov function and in essence an invariance argument (before LaSalle!), that the saddle is achieved for L(x, λ) strictly concave in x and convex in λ. It is noteworthy that in the congestion control literature, such proofs were essentially rediscovered based on passivity arguments [15]. In this paper we have found that the Krasovskii method provides an alternate proof for these problems. Indeed, the Lyapunov function z˙ T Qz˙ in (18) with z = (x, λ)T , satisfies for the dynamics (32) the condition " ∂2 L #   ∂2L ∂F 1 ∂x2 ∂x∂λ Q = . ∂z 2 − ∂ 2 L − ∂ 2 L2 ∂λ∂x ∂λ As before, when computing the Lyapunov derivative, the terms off the block diagonal will cancel, rendering # " T  # " ∂ 2 L ∂F ∂F 0 2 ∂x ≤ 0, Q+Q = 2 ∂z ∂z 0 − ∂∂λL2 for L(x, λ) concave in x and convex in λ. Thus V˙ ≤ 0, and a Lasalle argument follows if there is strict concavity in x. V. C ONCLUSIONS We studied network optimization problems from congestion control and extensions to cross-layer optimization. We found a new technique for proving global stability results in a variety of such problems, invoking Krasovskii’s method for the choice of Lyapunov functions. This covers primal and dual algorithms, but is particularly suited for the primal-dual case, in which a certain symmetry of the gradient equations allows for a simplification in the Lyapunov derivative. Furthermore, we have seen that the method extends to gradient dynamics for saddle-point problems as studied classically in [1]. A natural future question is the applicability of this proof method to situations where strict concavity is relaxed. A PPENDIX : STABILITY PROOFS BASED ON L A S ALLE I NVARIANCE IN SWITCHED S YSTEMS Dual congestion control We have shown that the function V (σ, λ) in (15) is decreasing along trajectories of the system. We wish to invoke Theorem IV.1 in [11], which establishes a LaSalle invariance principle for switched systems of this kind. Invoking [11], the dynamics must converge to an invariant set inside the set of trajectories that at all times satisfy either d (i): σ fixed and dt V (σ, λ(t)) ≡ 0; or (ii): σ switches at time − t between σ and σ + , but V (σ − , λ(t)) = V (σ + , λ(t)). This implies V (t) is constant for all time.

3296

Focusing on case (i), we see from (13) that q˙s ≡ 0 for all s where fs′ (qs ) < 0; this always happens unless the projection in (8) occurs, in which case xs ≡ 0. In any case the rates xs are constant while (i) holds, and so are the link rates yl . If there is a switch as in case (ii), referring to expression (15) and the discussion after it, we see that the rate yl cannot change either during this switch. Therefore our invariant trajectory satisfies yl ≡ yl0 , constant for all time. If yl0 < cl for some l, the corresponding λl must be at zero with the projection active; otherwise, it would fall linearly and lead to a discontinuous switch in V , violating (ii) above. If yl0 > cl , λl would tend to infinity, but this means through assumption (9) that all sources using this link would have rate going to zero, contradicting the constancy of yl . Therefore, the only other option is yl0 = cl for some link with λl ≥ 0. We see that these are precisely the KKT conditions for the problem, thus the invariant trajectory is at the optimum. Proof of Theorem 1 The primal-dual dynamics (25)-(26) have projections in x and in λ; let these be represented by the discrete state σ = (σx , σλ ). The Lyapunov function from (18) is " #2 X ∂hl X κs ′ V (σ, z) = λl Us (xs ) − 2 ∂xs l∈L s6∈σx X γl [hl (x) − cl (¯ p(λ))]2 + 2 l6∈σλ

A key observation is that the expression V˙ = x˙ T

diag{Us′′ (xs )} −

X

!

˙ λl ∇2 hl (x) x˙ − λ˙ T ∇2 ψ λ,

l∈L

implicit in (28), still applies on any interval of constant σ; the applicable projections in x or λ simply provide zero terms. Therefore, V is decreasing in this situation. In the case of a projection switch, an analogous argument as the one in Section II-B applies: the only discontinuities that can arise in V are in the decreasing direction, when a new projection gets activated. Therefore we are still under the conditions of the LaSalle theory in [11]. It remains to characterize an invariant set within the conditions (i) or (ii) as discussed in the preceding proof for the dual. For an interval in case (i) (V˙ ≡ 0 without switching), the strict concavity assumed in Us (xs ), implies that again we will have that x˙s ≡ 0 for all s ∈ / σx , and trivially for the rest. Since the state variable x has no discontinuities at switching, x is constant over all time for the invariant trajectory, x ≡ x ˆ = arg maxx L(x, p, λ). Also, from the second term in V˙ we see that cl (¯ p(λ)) = ∂ψ must remain constant as well. So the right-hand side of ∂λl λ˙ l = γl [hl (ˆ x) − cl (¯ p(λ))] is constant in time for an invariant trajectory. Again, if it were negative it would lead to λl saturating at zero after a finite time, and thus a discontinuous switch in V . So we can assume hl (ˆ x) − cl (¯ p(λ)) ≥ 0 for all l; it remains to rule out

the case where it is strictly positive. For this, we note that the trajectory is moving within the optimum of (20), X X D(λ(t)) = Us (ˆ xs ) − λl (t)[hl (ˆ x) − cl (¯ p(λ))]. (33) s∈S

l∈L

Hence, if hl (ˆ x)− cl (¯ p(λ)) > 0 for some l, the corresponding λl grows linearly in time, and D(λ(t)) is strictly decreasing. To continue we invoke the following lemma proved in [12]. ¯ be a vector such that the set M ¯ = {λ ≥ Lemma: Let λ λ ¯ is nonempty. Then M ¯ is bounded. 0 : D(λ) ≤ D(λ)} λ From the above Lemma we observe that λ(t) ∈ Mλ(0) for all t ≥ 0. But since Mλ(0) is a bounded set, this contradicts the linear growth of λl (t). Therefore, the invariant set has λ also at an equilibrium ˆ which by the previous argument must satisfy the compleλ, mentary slackness conditions: either λl = 0, or hl (ˆ x) − cl (¯ p(λ)) = 0. ˆ p also converges to Note also that once λ converges to λ, ˆ = arg maxp λ ˆ T c(p). Therefore, the invariant trajectory pˆ(λ) is an optimal point of Problem 2. R EFERENCES [1] K. J. Arrow, L. Hurwicz and H. Uzawa, Studies in Linear and NonLinear Programming, Stanford University Press, Stanford, 1958. [2] M.S. Branicky, “Multiple Lyapunov functions and other analysis tools for switched and hybrid systems”, IEEE Transactions on Automatic Control, April 1998, pp. 475–482. [3] M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle. “Layering as optimization decomposition: A mathematical theory of network architectures”. Proc. of the IEEE, vol. 95, pages 255–312, Jan 2007. [4] F. P. Kelly, A. Maulloo, and D. Tan, “Rate control for communication networks: Shadow prices, proportional fairness and stability”, Jour. Oper. Res. Society, vol. 49(3), pp 237-252, 1998. [5] H. K. Khalil, Nonlinear Systems, Prentice-Hall, 1996. [6] N. N. Krasovskii, Problems of the Theory of Stability of Motion, (Russian), 1959. English translation: Stanford University Press, 1963. [7] J-W. Lee, M. Chiang and A. R. Calderbank, “Utility-optimal randomaccess control”, IEEE Transactions on Wireless Communication, vol. 6, no. 7, pp. 2741–2751, July 2007. [8] J-W. Lee, M. Chiang and A. R. Calderbank, “Jointly optimal congestion and contention control based on network utility maximization”, IEEE Communications Letters, vol. 10, no. 3, pp. 216-218, 2006. [9] X. Lin, N. Shroff and R. Srikant, “A Tutorial on Cross-Layer Optimization in Wireless Networks,” IEEE Journal on Selected Areas in Communications, August 2006, pp. 1452-1463. [10] S. H. Low and D. E. Lapsley, “Optimization flow control, I: basic algorithm and convergence”, IEEE/ACM Transactions on Networking, vol.7, no.6,pp861-874, December 1999. [11] J. Lygeros, K. H. Johansson, S. N. Simic, J. Zhang and S. Sastry, “Dynamical properties of hybrid automata”, IEEE Transactions on Automatic Control, 2003, vol. 48, pp. 2–17. [12] A. Nedic and A. Ozdaglar, “Approximate primal solutions and rate analysis for dual subgradient methods”, LIDS Technical Report 2753, MIT, March 2007. [13] F. Paganini, “Control and economics in networking, via optimization: perspectives from an emerging discipline”, European Control Conference, Kos, Greece, July 2007. [14] R. Srikant, The Mathematics of Internet Congestion Control, Birkhauser, 2004. [15] J. T. Wen and M. Arcak. “A unifying passivity framework for network flow control”, IEEE INFOCOM, August 2003. [16] Y. Yu and G. B. Giannakis, “Cross-layer congestion and contention control for wireless ad hoc networks”, IEEE Transactions on Wireless Communication, vol. 7, no. 1, January 2008. [17] J, Zhang, D. Zheng, and M. Chiang, ”The impact of stochastic noisy feedback on distributed network utility maximization”, IEEE INFOCOM 2007.

3297