Hilbert Spaces and Orthogonality

Motivation: Geometry in Infinite Dimensions

A Hilbert space is an “infinite-dimensional analogue of Euclidean space.” Here, there is an inner product, angle between elements, orthogonality, and orthogonal projections. Quantum mechanics: states of a system are elements $H = L^2(\mathbb{R}^3)$; operators are observables. Signal analysis: decomposition in an orthogonal basis generalizes the Fourier series.

Inner Product

Pre-Hilbert Space $(H, \langle\cdot, \cdot\rangle)$: a vector space with a function $\langle\cdot, \cdot\rangle : H \times H \to \mathbb{R}$:

1. Linearity in the first argument: $\langle\alpha x + \beta y, z\rangle = \alpha\langle x, z\rangle + \beta\langle y, z\rangle$.
2. Symmetry: $\langle x, y \rangle = \langle y, x \rangle$.
3. Positive definiteness: $\langle x, x \rangle \geq 0$; $\langle x, x \rangle = 0 \iff x = 0$.

Norm from the inner product: $|x| = \sqrt{\langle x, x\rangle}$.

Cauchy–Schwarz Inequality: $|\langle x, y \rangle| \leq |x|\cdot|y|$. Equality $\iff x$ and $y$ are linearly dependent.

Proof: consider $|x - t\langle x, y\rangle/|y|^2 \cdot y|^2 \geq 0$ and expand.

Parallelogram Law: $|x + y|^2 + |x - y|^2 = 2(|x|^2 + |y|^2)$. Characterizes spaces with $\langle\cdot, \cdot\rangle$.

Hilbert space: pre-Hilbert, complete with respect to the norm $|\cdot| = \sqrt{\langle \cdot, \cdot \rangle}$.

Examples: $\mathbb{R}^n$ ($\langle x, y \rangle = \sum x_i y_i$), $l^2$ ($\langle x, y \rangle = \sum x_n y_n$), $L^2[a, b]$ ($\langle f, g \rangle = \int f(x) g(x) dx$).

Orthogonality and Projections

Orthogonality: $x \perp y \iff \langle x, y \rangle = 0$. Pythagorean theorem: if $x \perp y$, then $|x + y|^2 = |x|^2 + |y|^2$.

Orthogonal complement: $M^\perp = {x : \langle x, y \rangle = 0\ \text{for all}\ y \in M}$. $M^\perp$ is a closed subspace.

Projection theorem: For a closed subspace $M \subset H$ and $x \in H$: there exists a unique nearest element $P x \in M$: $|x - Px| = \inf_{y\in M} |x - y|$. The difference $x - Px \perp M$. The operator $P$ is the orthogonal projector: $P^2 = P$, $P^* = P$.

Direct decomposition: $H = M \oplus M^\perp$ — every $x = Px + (x - Px)$ uniquely.

Numerical Example

Problem: In $L^2[0,1]$, find the best linear approximation of $f(x) = e^x$ from the subspace $V = \operatorname{span}{1, x}$, minimizing $|f - p|_{L^2}$.

Step 1. Normalize the basis using the Gram–Schmidt process. $\varphi_1 = 1$ ($|1|^2 = \int_0^1 1,dx = 1 \rightarrow \varphi_1 = 1$). $\widetilde{\varphi}_2 = x - \langle x, 1 \rangle \cdot 1 = x - 1/2$. $|x - 1/2|^2 = \int_0^1 (x - 1/2)^2 dx = 1/12$. $\varphi_2 = (x - 1/2)/\sqrt{1/12} = 2\sqrt{3}(x - 1/2)$.

Step 2. Compute projection coefficients: $\langle e^x, 1 \rangle = \int_0^1 e^x dx = e - 1 \approx 1.718$. $\langle e^x, x - 1/2 \rangle = \int_0^1 e^x(x - 1/2),dx$. Integrate by parts: $= [e^x(x - 1/2)]_0^1 - \int_0^1 e^x dx = e \cdot (1/2) - (-1/2) - (e - 1) = 3/2 - e/2 \approx 0.141$.

Step 3. Projection: $Pf = \langle e^x, 1 \rangle\cdot 1 + \langle e^x, x - 1/2\rangle \cdot (x - 1/2)/|x - 1/2|^2 = (e - 1) + (3/2 - e/2)\cdot 12 \cdot (x - 1/2) = (e - 1) + (18 - 6e)(x - 1/2)$.

Step 4. Check at $x=0$: $(e-1) + (18-6e)(-1/2) = e - 1 - 9 + 3e = 4e - 10 \approx 0.873$. Exact: $e^0 = 1$. At $x=0.5$: $(e - 1) + 0 = e - 1 \approx 1.718$. Exact $e^{0.5} \approx 1.649$. The linear approximation is quite good in the $L^2$ sense.

Step 5. Error: $|e^x - Pf|_{L^2}^2 = |e^x|^2 - |Pf|^2 = \int_0^1 e^{2x}dx - [(e-1)^2 + (3/2-e/2)^2 \cdot 12] = (e^2 - 1)/2 - [...] \approx 3.195 - 3.190 \approx 0.005$. Relative error $\approx 0.4%$.

Real-World Application

Quantum mechanics: the probability of measuring the state $\psi$ in the eigenstate $\varphi_n$ is $|\langle \psi, \varphi_n \rangle|^2$ (Born rule). Orthogonality of eigenfunctions guarantees that different energy levels “do not mix” during measurement.

Additional Aspects

A Hilbert space is a Banach space with a norm induced by an inner product: $|x|^2 = \langle x, x \rangle$. This grants full geometry (angles, orthogonality, projections) and makes $H$ the working model of quantum mechanics. Riesz–Fréchet theorem: every continuous linear functional can be represented as $\varphi(x) = \langle x, a \rangle$ for a unique $a \in H$. This means $H \cong H^*$ (self-duality). Orthogonal decompositions $H = M \oplus M^\perp$ for closed subspaces $M$ always exist; projection onto $M$ underpins the least squares method and numerical approximation techniques. Hilbert spaces are the natural stage for spectral theory of self-adjoint operators, which is foundational for quantum mechanics and signal processing.

Hilbert spaces provide a unifying language for quantum mechanics, signal processing, and numerical methods for partial differential equations, linking finite-dimensional geometric intuition with the rich spectral theory of infinite dimensions.

Connection with Other Branches of Mathematics

The theory of Hilbert spaces is closely intertwined with the study of linear operators in differential equations. The classic results of Jacques Hadamard and Maurice Fréchet on well-posed problems are formulated via closed self-adjoint operators in $L^2$ spaces. The spectral theorem of von Neumann and Riesz makes possible the expansion of solutions of Sturm–Liouville-type equations in orthonormal systems of eigenfunctions.

In functional analysis, Hilbert spaces serve as a model for abstract linear algebra in infinite dimensions. The notions of normal, unitary, and self-adjoint operators generalize diagonalization of matrices. The work of John von Neumann and Marshall Stone showed that every continuous group of unitary operators is generated by a self-adjoint operator, forming the foundation of representations of Lie groups.

With topology, Hilbert spaces are connected through the Riesz theorem characterizing reflexive Banach spaces and through A. N. Kolmogorov’s results on entropy and the compactness of sets in $L^2$. The Hilbert cube and Hilbert space play the role of universal objects in infinite-dimensional topology: the works of Keldysh and Anderson describe them as universal separable spaces.

In probability theory, $L^2$ spaces of random variables are at the heart of martingale theory and the Itô stochastic integral. The orthogonal projection theorem is interpreted as conditional expectation: projection onto the subspace generated by a $\sigma$-algebra gives $E(X \mid \mathfrak{G})$. In numerical methods, Galerkin and finite element methods use orthogonal projections in Hilbert spaces of functions, justifying convergence and stability of approximations to solutions of partial differential equations.

Historical Background and Development of the Idea

The roots of the idea go back to David Hilbert, who studied integral equations in the early 20th century. In works from 1904–1912, he investigated spaces of quadratically summable coefficients and eigenfunctions of integral operators, thereby laying the intuition for “linear algebra in the infinite-dimensional case.” Karl Munk and Erhard Schmidt developed the concepts of completeness and orthonormal systems; in 1907, Schmidt introduced the decomposition now known as the Hilbert–Schmidt decomposition. The formal definition of a Hilbert space as a complete pre-Hilbert space was established in the 1920s in works by von Neumann and Riesz. The classic treatise by Riesz and Nagy, “Lectures on Functional Analysis” (1929, Hungarian original; later German and English editions), systematized the theory. The motivation simultaneously arose from integral equations, variational calculus, and mathematical physics problems. After the work of Schrödinger and Heisenberg, von Neumann in the book “Mathematical Foundations of Quantum Mechanics” (1932) showed that the state space of a quantum system is naturally modeled by a Hilbert space, and observables by self-adjoint operators.