Hamiltonian (control theory)

From Wikipedia, the free encyclopedia

You have new messages (last change).

The Hamiltonian of optimal control theory was developed by L. S. Pontryagin as part of his minimum principle. It was inspired by, but is distinct from, the Hamiltonian of classical mechanics. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control should be chosen so as to minimize the Hamiltonian.

Contents

[edit] Problem statement

A control u(t) is to be chosen so as to minimize the objective function:

J(u)=\Psi(x(T))+\int^T_0 L(x,u,t) dt

The system state x(t) evolves according to the state equations

\dot{x}=f(x,u,t) \qquad x(0)=x_0 \quad t \in [0,T]

the control must satisfy the constraints

a \le u(t) \le b \quad t \in [0,T]

(in later work it was shown that constraints on the state could also be imposed, these will be left out of our presentation)

[edit] Definition of the Hamiltonian

For the problem above the Hamiltonian is

H(x,\lambda,u,t)=\lambda^T(t)f(x,u,t)+L(x,u,t) \,

where λ(t) is a vector of costate variables of the same dimension as the state variables x(t).

[edit] The minimum principle

The state variables satisfy

\dot{x}= \frac{\partial H}{\partial \lambda}

subject to the initial condition x(0) = x0. (It can be easily verified that this is exactly equivalent to the state equations given above).

The costate variables satisfy the costate equations

\dot{\lambda}=-\frac{\partial H}{\partial x}

subject to the terminal condition

\lambda(T)=\nabla_x \Psi(x^*(T))

Let x * (t) and λ * (t) denote the optimal state and costate trajectories respectively, then at each point in time t the optimal control u * (t) minimizes the Hamiltonian as compared to any other control u(t) satifying the control constraints:

H(x^*(t),u^*(t),\lambda^*(t),t) \leq H(x^*(t),u(t),\lambda^*(t),t), \quad \forall u \quad a \le u(t) \le b

(Note that, although at first sight the minimum principle gives a recipe for finding u * (t), that recipe is somewhat circular since the above expression uses x * (t) and λ * (t), the optimal state and costate, which presupposes a knowledge of the optimal solution! One way around this problem is to iterate the three equations above: starting from a trial u(t) compute the associated state via the first equation, then compute the costate via the second equation, then find an improved u(t) by applying the minimum principle with the current state and costate (rather than the optimal ones). One can then repeat the process until convergence).

[edit] The Hamiltonian in discrete time

When the problem is formulated in discrete time, the Hamiltonian is defined as:

H(x,\lambda,u,t)=\lambda^T(t+1)f(x,u,t)+L(x,u,t) \,

and the costate equations are

\lambda(t)=-\frac{\partial H}{\partial x}

(Note that the d.t. Hamiltonian at time t involves the costate variable at time t + 1. This small detail is essential so that when we differentiate with respect to x we get a term involving λ(t + 1) on the right hand side of the costate equations. Using a wrong convention here can lead to incorrect results, i.e. a costate equation which is not a backwards difference equation).

Retrieved from "http://en.wikipedia.org../../../h/a/m/Hamiltonian_%28control_theory%29.html"