Optimal Control and Estimation
MAE 546
Princeton University
School of Engineering and Applied Science
Spring 2010
Tuesday and Thursday, 3-4:20 pm
J-201, Engineering Quadrangle
Designing control logic that commands a dynamic system to a desired output or that augments the system's stability is a common objective in many technical fields, ranging from decision-making for economic and social systems through trajectory control for robots and vehicles to the development of optimal therapeutic protocols for treating disease. If the control objective can be expressed as a quantitative criterion, then optimization of this criterion establishes a feasible design for control logic. This course addresses the theory and application of optimal control, including the effects of uncertain inputs (i.e., disturbances) and measurement error.
INTRODUCTION & OVERVIEW
Optimal control theory governs strategies for maximizing a performance measure or minimizing a cost function as the state of a dynamic system evolves. If the information that the control system must use is uncertain or if the dynamic system is forced by random disturbances, it may not be possible to optimize this criterion with certainty. The best one can hope to do is to maximize or minimize the expected value of the criterion, given assumptions about the statistics of the uncertain factors. This leads to the concept of stochastic optimal control, control logic that recognizes the random behavior of the system and that attempts to optimize response or stability on the average rather than with assured precision. Finding optimal control and state histories for dynamic systems is an extension of static optimization (e.g., finding control parameters that define ordinary maxima or minima of algebraic functions).
OPTIMAL TRAJECTORIES
The general problem is to find a history of the control [i.e., u(t) from start to stop times] that forces the state from its initial value to its final value (along a desirable trajectory) and, at the same time, that optimizes the design cost function. The resulting state history [x(t) from start to stop times] is an optimal trajectory. Examples of optimal trajectories include the minimum-time-to-climb flight path of an aircraft, the minimum-fuel translunar orbit of a spacecraft, the minimum-distance path of an assembly robot's end effector, the minimum-energy control of a continuous-stirred-tank reactor, or an optimal financial investment strategy.
NEIGHBORING-OPTIMAL CONTROL
Once an optimal trajectory has been defined, small perturbations in its initial value, final value, system parameters, or external disturbances could destroy the optimality of the trajectory. Nevertheless, there may be a neighboring-optimal trajectory in the vicinity of the original path. Neighboring-optimal solutions could be computed by perturbing the system and reapplying the optimal control principles presented earlier. If perturbations are small, linearizing assumptions allow a family of neighboring-optimal trajectories to be defined by a single feedback control law.
STOCHASTIC OPTIMAL CONTROL
When a dynamic system is subjected to disturbances or parameter variations that cannot be specified ahead of time, a deterministic cost function cannot be minimized. If, however, the statistics of these uncertain quantities are known, a cost function that is the expected value of the previous cost function can be minimized. As time passes, the future becomes the present, and uncertainties become certain. An open-loop control strategy cannot account for the actual variations, but a closed-loop (i.e., "feedback") control law can. Feedback entails measurements, which may contain errors; thus, it becomes necessary to estimate the state for closed-loop control.
We consider the control of nonlinear systems with random inputs and perfect measurements, nonlinear systems with random inputs and imperfect measurements, and the circumstances for which the combined optimal control-and-estimation strategy is specified by separate control and estimation laws. The certainty-equivalence property of linear-quadratic-Gaussian controllers is described, and linear, time-invariant systems with random inputs and imperfect measurements are treated in detail.
LINEAR MULTIVARIABLE CONTROL
Virtually all dynamic systems are nonlinear, yet an overwhelming majority of operating control laws have been designed as if the systems to be controlled were linear and time-invariant. If the nominal trajectory is actually a fixed set point or a slowly varying sequence of set points, the locally linear model can be considered essentially time-invariant. As long as the qualitative differences in response are minimal, the linear, time-invariant model facilitates control system design.
Stochastic optimal control theory provides a comprehensive, consistent, and flexible design approach for such systems. Its principal virtue for linear multivariable control is that it provides a method of computing control and estimation gains explicitly, with certain guarantees about the stability of the closed-loop system. Control and estimation gains derive from steady-state solutions of matrix Riccati equations. Control structures, asymptotic properties, and stability margins can be examined in considerable detail.
We review steady-state solutions of matrix Riccati equations for discrete-and continuous-time problems. The system's nominal steady-state response to command inputs is combined with feedback control and estimation gains to calculate forward gains that provide the desired reponse (i.e., non-zero set-point regulation). A variety of controller structures can be specified by alternative cost functions, introducing model-following, output regulation, proportional-integral compensation, low-pass filtering of control commands, and disturbance rejection. Modal properties and spectral characteristics are reviewed. Stability margins are specified, and methods of robust control system design are presented.
TITLES OF PREVIOUS TERM PAPERS
- Approximate Output Tracking Using Nonlinear Cost Minimization for Non-Minumum Phase CTOL Aircraft Model
- Estimation and Control of a Low-Order Model of Transitional Channel Flow
- Estimation of Foot Reaction Forces of a Running Cockroach
- Study of Optimal Control for Nuclear Reactors
- Stochastic Optimal Control of Resistive Wall Mode in a Tokamak
- Optimal Control and Estimation of a Firing Neuron
- Kalman Filter Applications for Multiconjugate Adaptive Optics
- Time-Optimal Controller for Multiple Vehicle Velocity and Position Placement in the Phase Plane
- Optimal Estimation of the Communication Graph in Multi-Agent Consensus Systems
- Optimal Video Tracking of Multiple Fish with Kalman Filters
- Optimal Control and Estimation of a Deformable Mirror using Two-Actuators
- Trajectory Optimization for Multi-Agent Collision Avoidance
- Optimal Control of a Two-Strain Tuberculosis Epidemic
The primary reference for this course is OPTIMAL CONTROL AND ESTIMATION.
Additional books that are useful for reference include:
- Anderson, B., and Moore, J., Optimal Control: Linear-Quadratic Methods, Prentice Hall, 1990.
- Brogan, W., Modern Control Theory, Prentice-Hall, 1991.
- Bryson , A. E., Jr., and Ho, Y. C., Applied Optimal Control, Hemisphere, 1975.
- Dickinson, B., Systems: Analysis, Design, and Computation, Prentice Hall, 1991.
- Gelb, A., ed., Applied Optimal Estimation, MIT Press, 1974.
- Graham, A., Kronecker Products and Matrix Calculus: with Applications, J. Wiley, 1981.
- Grantham, W., and Vincent, T., Modern Control Systems Analysis and Design, Wiley, 1993.
- Hull, D., Optimal Control Theory for Applications, Springer-Verlag, 2003.
- Kwakernaak, H., and Sivan, R., Linear Optimal Control Systems, Wiley, 1972.
- Maciejowski, J., Multivariable Feedback Design, Addison-Wesley, 1989.
- Maybeck, P., Stochastic Models, Estimation, and Control, Academic Press, 1982.
- Skelton, R., Dynamic Systems Control, Wiley, 1988.
- Zhou, Z., Doyle, J., and Glover, K., Robust and Optimal Control, Prentice Hall, 1996.
MAE 546 Syllabus