Optimal control

From Wikipedia, the free encyclopedia

Optimal control theory is a mathematical field that is concerned with control policies that can be deduced using optimization algorithms.

The control that minimizes a certain cost functional is called the optimal control. It can be derived using Pontryagin's maximum principle (a necessary condition), or by solving the Hamilton-Jacobi-Bellman equation (a sufficient condition).

Optimal control deals with the problem of finding a control law for a given system such that a certain optimality criterion is achieved. A simple example should clarify the issue: consider a car traveling on a straight line through a hilly road. The question is, how should the driver press the accelerator pedal in order to minimize the total traveling time? Clearly in this example, the term control law refers specifically to the way in which the driver presses the accelerator and shifts the gears. The system is intended to be both the car and the hilly road, and the optimality criterion is the minimization of the total traveling time. The problem formulation usually also contains constraints. For example the amount of available fuel might be limited, the accelerator pedal cannot be pushed through the floor of the car, etc.

On a more general framework, given a dynamical system with time-varying input $u (t)$ , time-varying output $y (t)$ and time-varying state $x (t)$ , a so-called cost functional can be defined that is a measure that the control designer aims to minimize. The cost functional is the sum of the path costs, which usually take the form of an integral over time, and the terminal costs, which is a function only of the terminal (i.e., final) state, $x (T)$ . Thus, this cost functional typically takes the form

$J=\phi(x(T)) + \int_0^T L(x,u,t)\,\mathrm{d}t.$

where $T$ is the terminal time of the system; of course, the initial (i.e., starting) time of the system need not be 0 in general. Minimization of this functional is related to the minimization of action in Lagrangian mechanics, and so $L (x, u, t)$ is also called the Lagrangian here.

In the previous example, a proper cost functional would be a mathematical expression giving the traveling time as a function of the speed, geometrical considerations, and initial conditions of the system.

It is often the case the constraints are interchangeable with the cost functional. Another optimal control problem would be to minimize the fuel consumption, given that the car must complete the course in a given time. Yet another problem is obtained if both time and fuel are translated into some kind of monetary cost that is then minimized.

1 Linear quadratic control
2 Continuous and Discrete Time Control
3 Reference books
4 External links

[edit] Linear quadratic control

It is very common, when designing proper control systems, to model reality as a linear system, such as

$\dot{x}(t)=A x(t) + B u(t)$

$y(t) = C x(t) \,$

One common cost functional used together with this system description is

$J=\int_0^\infty ( x^T(t)Qx(t) + u^T(t)Ru(t) )\,\mathrm{d}t$

where the matrices Q and R are positive-semidefinite and positive-definite, respectively. Note that this cost functional is thought in terms of penalizing the control energy (measured as a quadratic form) and the time it takes the system to reach zero-state. This functional could seem rather useless since it assumes that the operator is driving the system to zero-state, and hence driving the output of the system to zero. This is indeed right, however the problem of driving the output to the desired level can be solved after the zero output one is. In fact, it can be proved that this secondary problem can be solved in a very straightforward manner. The optimal control problem defined with the previous functional is usually called the state regulator problem and its solution the linear quadratic regulator (LQR) which is no more than a feedback matrix gain of the form

$u(t)=-K(t)\cdot x(t)$

where K is a properly dimensioned matrix and solution of the continuous time dynamic Riccati equation. This problem was elegantly solved by Rudolf Kalman (1960).

[edit] Continuous and Discrete Time Control

The examples thus far have shown continuous time systems and control solutions. However, optimal control theory also focuses on control of discrete time systems. In fact, as optimal control solutions are now often implemented digitally, modern control is becoming chiefly interested in discrete time systems and solutions.

[edit] Reference books

Pontryagin, L.S.: The Mathematical Theory of optimal processes, 1962
Kirk, D.E.: Optimal Control Theory: An Introduction, 2004
Stengel, R. F., Optimal Control and Estimation Dover, 1994
Lebedev, L.P. and Cloud, M.J.: The Calculus of Variations and Functional Analysis with Optimal Control and Applications in Mechanics, World Scientific, 2003, Ch.2
Lewis, F.L. and Syrmos, V. L.: Optimal Control, John Wiley & Sons, 2nd edition