Linear Functionals and Dual Spaces

Motivation: Linear "Measurements" of a Space

A linear functional $f: X \to \mathbb{R}$ is a "measurement" of elements: it assigns a number to each element in a linear way. The dual space $X^*$ contains all bounded (continuous) linear functionals. The Hahn–Banach theorem is an "extension tool": any partial measurement can be extended to the whole space without loss of norm. This is foundational for convex optimization and duality theory.

The Hahn–Banach Theorem

Analytic form: Let $M$ be a subspace of a normed space $X$, $f$ a bounded linear functional on $M$. Then there exists $F \in X^$ with $F|M = f$ and $|F|{X^} = |f|_M$.

Geometric form: Two disjoint convex sets (one open) are separated by a hyperplane ${ x : F(x) = c }$ for some $F \in X^*$, $c \in \mathbb{R}$.

Corollaries:

1. For $x_0 \ne 0$: $\exists F \in X^*: F(x_0) = |x_0|$, $|F| = 1$.
2. $|x| = \sup_{F \in X^*,, |F|=1} |F(x)|$ — dual representation of the norm.
3. $X$ separates functionals: $x \ne y \implies \exists F: F(x) \ne F(y)$.

The Riesz Representation Theorem

In a Hilbert space (Riesz, 1907): For every $F \in H^$ there exists a unique $y \in H$ such that $F(x) = \langle x, y \rangle$ for all $x \in H$, and $|F| = |y|$. Corollary: $H^$ is isometrically isomorphic to $H$. Hilbert spaces are "self-dual".

In $L^p$ spaces: $(L^p)^* \cong L^{p'}$ where $1/p + 1/p' = 1$ ($1 < p < \infty$). $(L^1)^* \cong L^\infty$. $(L^\infty)^*$ strictly contains $L^1$.

Weak Topologies

Weak convergence: $x_n \rightharpoonup x$ if $F(x_n) \to F(x)$ for all $F \in X^*$. Norm convergence implies weak convergence, but not vice versa.

The Alaoglu–Banach theorem: The unit ball of $X^$ is weak compact. A key tool of the calculus of variations.

Numerical Example

Problem: In $l^2$ find the element $y$ corresponding to the functional $F(x) = \sum_{n=1}^\infty x_n / (n \cdot 2^n)$ via the Riesz theorem, and compute $|F|$.

Step 1. $F(x) = \langle x, y \rangle = \sum_n x_n y_n$. Thus $y_n = 1/(n \cdot 2^n)$.

Step 2. Check $y \in l^2$: $|y|^2_{l^2} = \sum_n 1/(n^2 \cdot 4^n)$. Estimate: $\sum 1/(n^2 \cdot 4^n) \leq \sum 1/4^n = 1/(1-1/4) - 1 = 1/3$. Thus $y \in l^2$ ✓.

Step 3. Norm $|F| = |y|{l^2} = \left( \sum{n=1}^\infty 1/(n^2 \cdot 4^n) \right)^{1/2}$. Numerically: $n=1$: $1/4$, $n=2$: $1/(4 \cdot 16) = 1/64$, $n=3$: $1/(9 \cdot 64) \approx 0.00174$. Sum $\approx 0.266 \to |F| \approx 0.515$.

Step 4. Weak convergence $e_n \rightharpoonup 0$: for $F(e_n) = y_n = 1/(n \cdot 2^n) \to 0$. For any $y \in l^2$ the functional $F_y(e_n) = y_n \to 0$ (since $y \in l^2 \implies y_n \to 0$). Thus $e_n \rightharpoonup 0$ ✓. But $|e_n| = 1 \neq 0 \to$ norm convergence is absent ✓.

Step 5. Application of the Hahn–Banach theorem: on the subspace $M = {x : x_1 = 0}$ define the zero functional $f \equiv 0$. An extension to all of $l^2$ is any functional of the form $F_y$ with arbitrary $y_1$, $y_2 = y_3 = \ldots = 0$. A non-zero functional annihilated on $M$: $F(x) = x_1$ ✓.

Real-World Application

Support vector machine (SVM) method: the classification task reduces to finding a separating hyperplane with maximal margin — the geometric form of the Hahn–Banach theorem. The dual SVM problem is solved more efficiently than the primal: $O(n^2)$ instead of $O(d)$ when $n$ is the small number of support vectors.

Additional Aspects

The dual space $X^* = {$continuous linear functionals $X \to \mathbb{R}$ or $\mathbb{C}}$ is often studied independently because it is often more convenient or richer than the original. For Hilbert spaces: $H^* \cong H$ (Riesz theorem). For $L^p$ with $1 \leq p < \infty$: $(L^p)^* \cong L^q$, $1/p + 1/q = 1$; but $(L^\infty)^$ is a much more complicated "big" space. The weak topology (generated by all functionals) and the weak topology on $X^$ are powerful tools for compactification: the Banach–Alaoglu theorem states that the closed unit ball of $X^$ is weak* compact, which ensures existence of solutions in many optimization, game theory, and PDE problems.

Connection with Other Areas of Mathematics

In differential equations theory, linear functionals already arise in formulating initial and boundary conditions. In weak statements of PDE problems, solutions are sought as elements of Sobolev spaces, while functionals specify the equation's action on test functions. The Lax–Milgram theorem, based on representing continuous linear functionals in a Hilbert space, guarantees uniqueness of weak solutions to elliptic problems.

In algebra, dual spaces underlie the notion of the dual module and the contravariant functor $\operatorname{Hom}( \cdot, F )$. In representation theory, linear functionals generate adjoint representations; in Pontryagin and Toponogov’s works this is closely linked to harmonic analysis on groups and decompositions into eigenfunctions.

In topology, weak and weak* topologies yield important compactness results. The Gelfand–Naimark–Segal construction provides a correspondence between positive linear functionals on a $C^*$-algebra and representations of this algebra in a Hilbert space, which essentially ties functionals to the measure and topology of the algebra's spectrum.

In probability theory, linear functionals realize the expectation operator: the distribution of a random variable is uniquely determined by expectation functional values on the class of bounded measurable functions. The Riesz–Markov–Kakutani theorem shows how positive linear functionals on the space of continuous functions on a compact set generate Borel measures, providing a bridge from functional analysis to Kolmogorov’s classical measure theory.

In numerical methods, duality is applied in finite element theory, where a posteriori error estimates are constructed through dual (adjoint) problems. In convex optimization, the classical works of Rockafellar reveal the link among linear functionals, subdifferentials, and dual minimization problems.

Historical Background and Development of the Idea

The first systematic discussions of linear functionals are found in works by Fredholm and Hilbert in the early 20th century, where functionals appeared as integral expressions in Fredholm equation theory. The formal concept of the dual space was fixed in the works of Riesz and Fischer (1907–1910) in the context of orthogonal decompositions and the Hilbert space of square-summable functions. The Riesz theorem on the representation of continuous functionals in Hilbert space was formulated and proved by F. Riesz in 1907 in connection with spectral theory problems. In the 1920s–1930s, S. Banach in the book "Theory of Linear Operations" (Warsaw–Lviv, 1932) finally formalized the concepts of normed space, its dual, and weak topology. In the same period, S. Banach and H. Hahn proved the functional extension theorem, which quickly became a cornerstone of functional analysis. The geometric interpretation via separating hyperplanes was actively developed in the Mazur–Orlicz school, preparing the groundwork for further development of convex analysis in the works of Fenchel and Rockafellar in the mid-20th century. Alaoglu’s work (1940) on the weak* compactness of the unit ball of the dual space became foundational for the calculus of variations and optimal control theory.