Functionals and the Euler-Lagrange Equation

What is Calculus of Variations?

Ordinary mathematical analysis seeks the extremum of a function of a number or vector: find $x$ that minimizes $f(x)$. Calculus of variations solves a problem of another level: find a function $y(x)$ that minimizes some “function of a function”—a functional. Such problems arise in physics (“along which trajectory does a particle move?”), engineering (“what shape should a bridge have?”), and economics (“what is the optimal capital accumulation trajectory?”).

The history of the discipline begins in 1696, when Johann Bernoulli proposed the brachistochrone problem: find the shape of a slide along which a ball rolls between two points in the shortest time. The problem astonished contemporaries: the answer is not a straight line (the shortest distance), but a cycloid. This discovery gave impetus to the development of an entire mathematical theory.

What is a Functional?

Functional—a mapping from a space of functions to $\mathbb{R}$: $J: {y : [a,b] \rightarrow \mathbb{R}} \rightarrow \mathbb{R}$.

The most important type of functional:

$ J[y] = \int_{x_0}^{x_1} F(x, y(x), y'(x)),dx $

Here, $F$ is a given “Lagrangian function” of three arguments: $F(x, y, p)$, where $x$ is the independent variable, $y$ is the value of the function, $p = y'$ is the derivative. The functional “sums” the contribution of $F$ along the curve.

Examples of functionals:

Length of a curve: $F(x, y, y') = \sqrt{1 + (y')^2}$. Functional = $\int \sqrt{1 + (y')^2},dx$ — arc length.
Time of descent along a slide: a ball rolls down from height $y(x)$. Speed = $\sqrt{2gy}$, time element = $ds/v = \frac{\sqrt{1+(y')^2}}{\sqrt{2gy}},dx$. Functional: $J[y] = \int \frac{\sqrt{1+(y')^2}}{\sqrt{2gy}},dx$.
Potential energy of an elastic beam: $F = \frac{EI}{2} (y'')^2$—square of the curvature, multiplied by stiffness.

First Variation and the Idea of Deriving the EL Equation

Let $y^(x)$ be the supposed extremum. Consider a “perturbation”: $y_\epsilon(x) = y^(x) + \epsilon\eta(x)$, where $\eta$ is an arbitrary smooth function satisfying the boundary conditions $\eta(x_0) = \eta(x_1) = 0$ (we do not shift the ends), and $\epsilon$ is a small numerical parameter.

First variation: $\delta J[y^; \eta] = \frac{d}{d\epsilon}\Big|_{\epsilon=0} J[y^ + \epsilon\eta]$

For an extremum, it is necessary: $\delta J[y^*; \eta] = 0$ for all admissible $\eta$.

Computation:

$ \frac{d}{d\epsilon} J[y^* + \epsilon\eta] = \int (F_y \eta + F_{y'} \eta'),dx $

at $\epsilon=0$ (here $F_y = \partial F/\partial y$, $F_{y'} = \partial F/\partial p$).

The second term is integrated by parts: $\int F_{y'} \eta' ,dx = [F_{y'} \eta]{x_0}^{x_1} - \int \frac{d}{dx} F{y'} , \eta,dx$.

With zero boundary conditions $\eta(x_0)=\eta(x_1)=0$, the boundary term vanishes:

$ \delta J = \int (F_y - \frac{d}{dx} F_{y'}) \eta,dx = 0 \quad \text{for all } \eta. $

Main lemma: if a continuous $g$ is such that $\int g(x)\eta(x),dx = 0$ for all smooth $\eta$ with zero endpoints, then $g \equiv 0$.

Consequently:

Euler-Lagrange equation: $F_y - \frac{d}{dx}(F_{y'}) = 0$

This is a second-order ordinary differential equation (in expanded form): $F_{yy'} y'' + F_{y'x} + F_{y'y} y' - F_y = 0$.

Full Analysis: The Problem of the Shortest Distance

Problem: Find the shortest curve in $\mathbb{R}^2$ between points $A = (0, 0)$ and $B = (1, 1)$.

Functional: $J[y] = \int_0^1 \sqrt{1 + (y')^2},dx$. Lagrangian: $F(x, y, p) = \sqrt{1 + p^2}$.

EL Equation:

$F_y = \partial F/\partial y = 0$ (F does not explicitly depend on $y$!)
$F_{y'} = \partial F/\partial p = p/\sqrt{1 + p^2} = y'/\sqrt{1 + (y')^2}$

EL equation: $0 - \frac{d}{dx}\left(\frac{y'}{\sqrt{1 + (y')^2}}\right) = 0 \rightarrow \frac{d}{dx}\left(\frac{y'}{\sqrt{1 + (y')^2}}\right) = 0$.

Integrate: $\frac{y'}{\sqrt{1 + (y')^2}} = C$ (constant). Solve: $y'^2 = C^2(1 + y'^2) \rightarrow y'^2(1 - C^2) = C^2 \rightarrow y' = \frac{C}{\sqrt{1 - C^2}} = \text{const}$.

Conclusion: $y' = \text{constant} \rightarrow y = a x + b$—a straight line! With boundary conditions $y(0) = 0$, $y(1) = 1$: $y = x$.

The Brachistochrone Problem

Problem: Minimize the time $J[y] = \int_0^a \frac{\sqrt{1 + (y')^2}}{\sqrt{2gy}},dx$.

Lagrangian: $F = \frac{\sqrt{1 + (y')^2}}{\sqrt{2gy}}$. Does not explicitly depend on $x$ $\rightarrow$ first integral: $F - y' F_{y'} = C$ (Beltrami integral).

$F - y'\cdot\left(\frac{y'}{\sqrt{1 + (y')^2}}\right)/\sqrt{2gy} = \frac{1}{\sqrt{2gy}\sqrt{1 + (y')^2}} = C$.

This equation is solved parametrically: $x = R(t - \sin t)$, $y = R(1 - \cos t)$—a cycloid! The ball rolls not along a straight line, nor along a circular arc, but along a cycloid—a curve traced by a point on the rim of a rolling wheel.

Extensions of the EL Equation

Several functions $y_1(x), \ldots, y_n(x)$: $J = \int F(x, y_1, \ldots, y_n, y_1', \ldots, y_n'),dx$. A system of $n$ EL equations: $F_{y_i} - \frac{d}{dx} F_{y_i'} = 0$.

Higher derivatives: $J = \int F(x, y, y', y''),dx$. EL: $F_y - \frac{d}{dx} F_{y'} + \frac{d^2}{dx^2} F_{y''} = 0$. A fourth-order equation. Example: beam, $J = \int (EI,y'')^2,dx \rightarrow EI,y'''' = 0$.

Applications

In mechanics, the variational principle formulates equations of motion via the minimum of action. In optics—the Fermat principle (light travels along the path of least time). In economics—optimal consumption trajectory (Ramsey model). In machine learning—regularization can be understood as minimization of a functional with smoothness constraints.