Optimal Control - Theory and Applications

18879 C: Special Topics in Systems and Control: Optimal Control Optimal Control - Theory and Applications Course #: 18879C/96840SV Semester: Fall 201...
Author: Joleen Hubbard
2 downloads 1 Views 1MB Size
18879 C: Special Topics in Systems and Control: Optimal Control

Optimal Control - Theory and Applications Course #: 18879C/96840SV Semester: Fall 2013 Breadth Area: Artificial Intelligence, Robotics and Control Instructor Dr. Abraham Ishihara Phone: 650 335-2818 Email: [email protected]; Office hours: TBD Location: Room 209 Lecture Information: Time: Mon/Fri 01:30 PM - 02:20 PM (PST) Location: SV - Room 118; PGH - HH 1107 Time: Wed 01:30 PM - 02:20 PM (PST) Location: SV - Room 212; PGH - HH 1107 Lectures will be available online via adobe connect: http://cmusv.adobeconnect.com/optimalcontrol/ Teaching Assistant: TBD Email: Chaitanya Poolla ([email protected]) Office Hours: TBD Location: PGH TBD Course Description This course will cover the fundamentals of optimal control theory including applications from current research in aeronautics and robotics. Specific topics include: extrema of functions and functionals, Lagrange multipliers, calculus of variations, du Bois-Reymond equation, corner conditions, Legendre/Jacobi necessary conditions, isoperimetric problems and constrained optimization, variational approach to optimal control, bang-bang control, LQR, Pontryagin Maximum Principle (PMP), Dynamic programming and the HamiltonJacobi-Bellman (HJB) equations, relationship between PMP and Dynamic Programming, singular optimal control, and stochastic optimal control. Guest Speakers We will have guest speakers who use optimal control techniques in

2013

18879 C: Special Topics in Systems and Control: Optimal Control

2013

practice for problems including trajectory planning and scheduling for Next Gen Airspace, intelligent control, mechatronics servo-control, optics, and intelligent path planning for autonomous robotic systems. Recitation: There will be a weekly recitation section on Wednesdays where the TA will review topics from the course and go over problems in detail that are related to the HW. Prerequisites: This course is intended for advanced undergraduate and beginning graduate students. The prerequisites are ordinary differential equations and 18-470 – Fundamentals of Control. It is helpful, but not required, to have taken or to take concurrently: 18-771 – Linear Systems. Course Content Overview 1. Introduction to Optimal Control (2 Lectures) – Applications in aeronautics and robotics: trajectory optimization/path planning – Historical perspectives: Calculus of Variations and Optimal Control  References: (Goldstine, 1980; McShane, 1989) – Further introduction and notions of calculus of variations; Brachistochrone problem; shortest optical path; numerical examples  References: (Gelfand, Fomin, & Silverman, 2000; Mesterton-Gibbons, 2009; Pinch, 1995) – Optimal Control Problem Statement; 5 examples including geometric solution of the rocket railroad problem  References: (Lawrence C Evans, 2005), chapter 1; ( gin & Neustadt, 1962), chapter 1. 2. Review of relevant Calculus Facts and Optimization in Finite Dimensional Spaces (2 Lectures) – Fundamental theorem of calculus; mean-value theorem; T yl ’s he em; ch ule; integration by parts; Leibnitz’s d ffe e rule – Definition of global and local maxima, minima in one and several variables; necessary and sufficient conditions for optimality; critical, end, and discontinuous points  References: (Pinch, 1995) – Feasible directions and global minima; compactness and Weirerstrass theorem  Reference: (Liberzon, 2012) – Extremum problems with constraints; Lagrange Multipliers involving one and several variables.

18879 C: Special Topics in Systems and Control: Optimal Control  Reference: (Amazigo & Rubenfeld, 1980)1 3. Some Elements of Functional Analysis (1 Lecture) – Metric Spaces; open sets; closed sets; neighborhoods; convergence; Cauchy sequences; completeness – Vector spaces; normed spaces; continuity; uniform continuity; convergence; limits Banach spaces; linear operators; functionals; compactness; examples of important function spaces  Reference: (Kreyszig, 1989), chapters 1,2 4. Calculus of Variation (2 Lectures) – Basic problem; Weak and strong extrema – Euler-Lagrange equations – Variable end-point problem; integral constraints; second order conditions – Brachistochrone and shortest optical path revisited  Reference: (Born & Wolf, 1999; Liberzon, 2012) 5. Variational approach to optimal control (2 Lectures) – Weierstrass-Erdmann corner condition – Basic problem formulation – Variational Approach to (i) fixed-time, free endpoint and (ii) free-time, free end-point problems – Weakness of the variational approach – Statement of Pontryagin of Maximum Principle  Reference: (Fleming & Rishel, 1975; Liberzon, 2012; Mesterton-Gibbons, 2009; Pinch, 1995) 6. LQR , Bang-Bang control, singular control and other optimal control problems (5 Lectures)  Reference: (Bryson & Ho, 1975; Lewis, Vrabie, & Syrmos, 2012) 7. Elements of Dynamical Systems (1 Lecture) – Definition of a solution; semi-group properties – Heine-Borel thm; Existence and uniqueness – Continuous dependence on initial data and parameters – ODE theorems needed for the proof of PMP  Reference: (Murray & Miller, 1954)

1

Most advanced calculus texts should have this material.

2013

18879 C: Special Topics in Systems and Control: Optimal Control

2013

8. Proof of Pontryagin Maximum Principle (4 Lectures) – Admissible controls, adjoint system, transversality conditions  Reference: (Lawrence C Evans, 2005; Liberzon, 2012; gin & Neustadt, 1962) 9. Examples from Trajectory Optimization (2-4 Lectures) 10. Dynamic Programming – Derivation of HJB equations  Reference: (Lawrence C Evans, 2005; Liberzon, 2012) – Examples: rocket railroad car; general Linear Quadratic Regulator – Method of Characteristics and relationship to Pontryagin Maximum Principle  Reference: (Lawrence C Evans, 2005; Liberzon, 2012) 11. Stochastic Optimal Control – Review of relevant probability theory  Reference: (Billingsley, 1986) – I ’s Lemm  Reference: (Mao, 1997) – Stochastic dynamic programming  Reference: (Lawrence C Evans, 2001, 2005)

Grading Grading will be determined by the best four of five HW assignments (60%) and one course project (40%). It is highly encouraged to work together on the HW assignments. Each student, however, must turn in his or her HW individually. Some HWs may involve programming in Matlab. Course Project: Students will form teams and propose a project or choose from several options proposed by the instructor in the area of aeronautics or robotics.

Recommended References This course involves a number of topics and we have not been able to find a single text book that covers all the material we intend to discuss. We therefore do not require any specific text book. However, we note that a

18879 C: Special Topics in Systems and Control: Optimal Control

2013

significant component of our course notes will follow (Mesterton-Gibbons, 2009)2and (Liberzon, 2012)3 both of which are available on amazon.com: – http://www.amazon.com/Calculus-Variations-Optimal-ControlMathematical/dp/0821847724/ref=sr_1_fkmr0_1?ie=UTF8&qid=1377474157&sr=8-1fkmr0&keywords=mest+gibbons+optimal+control – http://www.amazon.com/Calculus-Variations-Optimal-ControlTheory/dp/0691151873/ref=sr_1_1?ie=UTF8&qid=1377474106&sr=81&keywords=liberzon+optimal+control The first reference (Mesterton-Gibbons, 2009) is highly recommended since much of the HW will be based on this material. Notes on the References: Applications of Optimal Control The following references are more application oriented and less bent on rigorous theory. In particular, to get a sense of potential applications of optimal control methods we recommend (Lewis et al., 2012) and (Bryson & Ho, 1975)  (Anderson & Moore, 2007; Athans & Falb, 2006; Bryson & Ho, 1975; Kirk, 2012; Lewis et al., 2012; Naidu, 2003) Background Mathematics Optimal control spans a diverse set of mathematical disciplines including advanced calculus, ordinary differential equations, variational analysis, functional analysis, and probability. The recommended references are  (Amazigo & Rubenfeld, 1980; Billingsley, 1986; Lawrence C Evans, 2001; Kreyszig, 1989; Mao, 1997; Miller & Michel, 1982; Murray & Miller, 1954) Variational Analysis Variational analysis is at the heart of optimal control theory and the calculus of variation. This course is primarily focused on finite dimensional dynamic systems. However, there are a number of textbooks that discuss the extension to distributed or infinite dimensional systems including:  (Berdichevsky, 2009; L.C. Evans, 2010; Leipholz, 1976) Variational Approach to Optimal Control There are number books that provide more emphasis on the calculus of variations and the variational approach to optimal control.  (Athans & Falb, 2006; Bryson & Ho, 1975; Gelfand et al., 2000; Kirk, 2012; Lewis et al., 2012; MestertonGibbons, 2009; Naidu, 2003; Pinch, 1995) Rigorous Treatment of Optimal Control These are the textbooks that present a complete treatment of the Pontryagin Maximum Principle and often include a discussion on the relationship between the PMP and Dynamic Programing. They include:  (Clarke, 2013; Lawrence C Evans, 2005; Fleming & Rishel, 1975; gin & Neustadt, 1962; Troutman, 2012)

2 3

Current price is $32.52 and is eligible for free shipping with Amazon Prime. Current price is $52.77 and is eligible for free shipping with Amazon Prime.

18879 C: Special Topics in Systems and Control: Optimal Control

Optimal Control Schedule for: ECE 18879K: Special Topics in Systems and Control; Fall 2012 Class Meets Topic Tues

Thurs

Total Classes: 28 Reading

8/28

8/30

Wk 2

9/4

9/6

Wk 3

9/11

9/13

9/14 Some Elements of Functional Analysis; Calculus of Variation

Notes;

Wk 4

9/18

9/20

9/21 Calculus of Variation; Variational approach to optimal control

Notes; Ch 2 (Lib)

Wk 5

9/25

Wk 7

8/31 Historical and motivating examples Review ofperspectives, some Calculusbasic Factsnotions and Optimization in Finite 9/7 Dimensional Spaces

Notes; Ch 1 (Lib) Notes; Ch 1 (Lib)

9/28 Variational approach to optimal control LQR , Bang-Bang control, singular control and other optimal control 10/2 10/4 10/5 problems LQR , Bang-Bang control, singular control and other optimal control 10/9 10/11 10/12 problems 9/27

Notes; Ch 3 (Lewis) Notes;

Wk 9

10/23 10/25 10/26 Proof of Pontryagin Maximum Principle

Notes; Ch 4 (Lib) Notes; Ch 4 (Lib)

Wk 10

10/30

11/1

11/2 Proof of Pontryagin Maximum Principle

Wk 11

11/6

11/8

11/9 Examples from Trajectory Optimization

Notes;

Wk 12

11/13 11/15 11/16 Examples from Trajectory Optimization

Notes;

Wk 13

11/20 11/22 11/23 Dynamic Programming

Notes; Ch 5 (Lib)

Wk 14

11/27 11/29 11/30 Derivation of HJB

Notes; Ch 5 (Lib)

12/6

12/7 Stochastic Optimal Control; Ito Calculus

Wk 16

12/11 12/13 12/14

Wk 17

12/18 12/20

Notes;

Breakdown:

No Class 15%

HW 2

15%

HW 3

15%

HW 4

15%

HW 5

15%

Total HW

60%

Final

40%

Total

100%

HW #3 Due 10/12

HW #4 Due 11/2

Thanksgiving HW #5 Due 12/7 Exam Week Grades due on Dec. 20

8/31

HW 1

HW #2 Due 9/21

Notes; Ch 3 (Lewis)

10/16 10/18 10/19 Elements of Dynamical Systems

12/4

HW #1 Due 9/7

Notes; Ch 3 (Lib)

Wk 8

Wk 15

Deadlines

Fri

Wk 1

Wk 6

2013

Due Date You can drop one HW; best 4 of 5 (total 60%) Final - 40%

18879 C: Special Topics in Systems and Control: Optimal Control

2013

References Amazigo, J. C., & Rubenfeld, L. A. (1980). Advanced Calculus and Its Applications to the Engineering and Physical Sciences: John Wiley and Sons. Anderson, B. D. O., & Moore, J. B. (2007). Optimal Control: Linear Quadratic Methods: Dover Publications, Incorporated. Athans, M., & Falb, P. L. (2006). Optimal Control: An Introduction to the Theory and Its Applications: Dover Publications. Berdichevsky, V. L. (2009). Variational Principles of Continuum Mechanics: II. Applications: Springer. Billingsley, P. (1986). Measure and Probability. John Willey. Born, M., & Wolf, E. (1999). Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light: Cambridge University Press. Bryson, A. E., & Ho, Y. C. (1975). Applied Optimal Control: Optimization, Estimation, and Control: Taylor & Francis. Clarke, F. (2013). Functional Analysis, Calculus of Variations and Optimal Control: Springer London, Limited. Evans, L. C. (2001). An introduction to stochastic differential equations version 1.2. Department of Mathematics UC Berkeley, in internet. Evans, L. C. (2005). An introduction to mathematical optimal control theory. Lecture Notes, University of California, Department of Mathematics, Berkeley. Evans, L. C. (2010). Partial Differential Equations: American Mathematical Society. Fleming, W. H., & Rishel, R. W. (1975). Deterministic and stochastic optimal control: Springer-Verlag. Gelfand, I. M., Fomin, S. V., & Silverman, R. A. (2000). Calculus of Variations: Dover Publications. Goldstine, H. H. (1980). A History of the Calculus of Variations from the 17th through the 19th Century. Sources in the History of Mathematics and Physical Sciences, 5. Kirk, D. E. (2012). Optimal Control Theory: An Introduction: Dover Publications, Incorporated. Kreyszig, E. (1989). Introductory Functional Analysis with Applications: Wiley. Leipholz, U. (1976). Direct Variational Methods and Eigenvalue Problems in Engineering: Springer. Lewis, F. L., Vrabie, D., & Syrmos, V. L. (2012). Optimal Control: Wiley. Liberzon, D. (2012). Calculus of variations and optimal control theory: a concise introduction: Princeton University Press. Mao, X. (1997). Stochastic Differential Equations and Their Applications: Horwood Pub. McShane, E. (1989). The calculus of variations from the beginning through optimal control theory. SIAM journal on control and optimization, 27(5), 916-939. Mesterton-Gibbons, M. (2009). A Primer on the Calculus of Variations and Optimal Control Theory: American Mathematical Society. Miller, R. K., & Michel, A. N. (1982). Ordinary Differential Equations: Academic Press, Incorporated. Murray, F. J., & Miller, K. S. (1954). Existence Theorems for Ordinary Differential Equations: University Press. Naidu, D. S. (2003). Optimal Control Systems: CRC Press. Pinch, E. R. (1995). Optimal Control and the Calculus of Variations: Oxford University Press, Incorporated. gin, L. S., & Neustadt, L. W. (1962). The Mathematical Theory of Optimal Processes: Gordon and Breach Science Publishers. Troutman, J. L. (2012). Variational Calculus and Optimal Control: Optimization With Elementary Convexity: Springer Verlag.