Maximum Principle: Proof and Applications

The PMP (Pontryagin Maximum Principle) is a nontrivial result: its proof does not reduce to simple integration by parts as with the Euler–Lagrange (EL) equation. Pontryagin and his students (Boltyansky, Gamkrelidze, Mishchenko) constructed the proof via “needle variations” and the theorem of separation of convex sets. Applications cover economics, engineering, biomedicine, and management—everywhere dynamic optimization is involved.

Idea of the Proof

Needle variations. Instead of smooth perturbations (as in EL), we consider a "needle": on a short interval $[t_0, t_0 + \varepsilon]$, we replace $u^(t)$ with any admissible $v \in U$, leaving $u^$ for the remainder of the time. This causes a jump in the state $x(t)$, and, accordingly, in the objective function $J$.

Cone of variations. The collection of all such perturbations in the limit as $\varepsilon \to 0$ yields the attainable cone in the state space. If $u^*$ is optimal, the attainable cone and the direction of improvement of $J$ must be on “opposite sides” of some hyperplane—otherwise, there would exist a perturbation that improves $J$.

Separation of cones. The normal to the separating hyperplane is the vector $\psi(t_0)$. The condition that a needle variation does not improve $J$ is equivalent to $H(x^(t_0), v, \psi(t_0), t_0) \leq H(x^(t_0), u^*(t_0), \psi(t_0), t_0)$ for all $v \in U$. This is precisely the maximum condition.

Euler Condition as a Special Case

If $U = \mathbb{R}^m$ (no constraints), the maximum of $H$ is attained in the interior of $U \rightarrow \frac{\partial H}{\partial u} = 0$. This is the classical Euler equation in Hamiltonian form.

If $U = [a, b]$ and $H$ is convex in $u$, then $u^* = \mathrm{clip}(u_{\text{free}}, a, b)$, where $u_{\text{free}}$ is the root of $\frac{\partial H}{\partial u} = 0$.

If $H$ is linear in $u$, as often happens in problems with control via thrust, velocity, or investment, the optimum lies on the boundary of $U \rightarrow$ bang-bang.

Economic Applications

1. The Optimal Growth Problem (Ramsey–Cass–Koopmans).
$\max \int_0^\infty e^{-\rho t} \cdot u(c(t)),dt$ subject to $\dot{k} = f(k) - c$, $k(0) = k_0$.

Here $k$ is capital, $c$ is consumption, $f(k)$ is the production function, $\rho$ is the discount rate, $u$ is the utility function.

$H = e^{-\rho t} \cdot u(c) + \psi \cdot (f(k) - c)$.

From $\frac{\partial H}{\partial c} = 0$: $e^{-\rho t} \cdot u'(c) = \psi$.
From $\dot{\psi} = -\frac{\partial H}{\partial k} = -\psi \cdot f'(k)$.

Logarithmizing and differentiating the first condition: $\dot{c}/c = (f'(k) - \rho)/\sigma$, where $\sigma = -c \cdot u''(c)/u'(c)$ is the elasticity of substitution. This is the Euler consumption equation—the fundamental equation of modern macroeconomics.

Numerical example. For $f(k) = k^{0.36}$, $\rho = 0.04$, $\sigma = 1$ (log-utility): steady state $k^* = \left(\frac{\alpha}{\rho+\delta}\right)^{1/(1-\alpha)}$, for $\delta = 0.1 \rightarrow k^* \approx 1.79$, $c^* \approx 0.99$.

2. Exhaustion of Natural Resources (Hotelling’s Rule).
$\max \int_0^T e^{-rt} \cdot p(q) \cdot q, dt$ subject to $\dot{s} = -q$, $s(T) \geq 0$.

PMP gives: the marginal rent of the resource ($p + p' \cdot q - c'(q)$) grows at rate $r$. Therefore, the price of oil, gas, and minerals “must” grow at roughly the discount rate. Empirically, the rule holds only approximately, due to technological breakthroughs and the discovery of new deposits.

3. Optimal Epidemic Control. SEIR model with control $u$ (fraction vaccinated): minimize $\int(I + \alpha \cdot u^2), dt$—a balance between number of infections and cost of vaccination. PMP gives the optimal mass vaccination schedule.

Engineering Applications

• Optimal Landing. Falcon 9: the problem is to minimize fuel expenditure subject to constraints on angle of attack, thrust, final velocity. Solved via convex relaxation of PMP in real time.
• HVAC Systems. Optimum building air conditioning schedule (minimization of energy use while maintaining comfort temperature)—a classic optimal control problem.

Exercise. Fisheries:
$\max \int_0^T e^{-rt} \cdot [p - c(x)] \cdot h, dt$ subject to $\dot{x} = g(x) - h$, $x(0) = x_0$, $0 \leq h \leq h_{max}$.
$g(x) = r_b \cdot x \cdot (1 - x/K)$—the logistic population model.
(a) Write down the PMP.
(b) Find the biological equilibrium (without control): $g(x) = 0 \rightarrow x = K$.
(c) Find the bio-economic steady state $x^$: $f'(x^) = r + c'(x^*)\cdot\ldots$ (write out exactly).
(d) Numerically (for $p = 10$, $r = 0.05$, $K = 1$, $r_b = 0.5$, $c(x) = 1/x$) plot the optimal trajectory $x(t)$, $h(t)$.