Orthonormal Bases and Fourier Series

Motivation: Decomposition into “Components”

In $\mathbb{R}^n$, any vector can be decomposed over an orthonormal basis: $x = \sum \langle x, e_i \rangle e_i$. In infinite-dimensional space, the analogue is decomposition over a complete orthonormal system. This generalizes Fourier series and forms the foundation of wavelet analysis and spectral methods.

Orthonormal Systems (ONS)

ONS: A family ${e_n}$ with $\langle e_n, e_m \rangle = \delta_{nm}$.

Fourier coefficients: $c_n = \langle x, e_n \rangle$. Partial sums $S_n x = \sum_{k=1}^n c_k e_k$ give the best approximation of $x$ in $\operatorname{span}{e_1, ..., e_n}$.

Bessel’s Inequality: $\sum_n |c_n|^2 \leq |x|^2$ for any ONS. Consequence: $c_n \to 0$.

Complete ONS (Orthonormal Basis):

• $\langle x, e_n \rangle = 0$ for all $n \implies x = 0$ (there is no nonzero vector orthogonal to all $e_n$).
• Equivalently: $x = \sum_n c_n e_n$ (the series converges in norm) for all $x \in H$.
• Parseval’s Identity: $|x|^2 = \sum_n |c_n|^2$.

All separable Hilbert spaces have a countable orthonormal basis.

Fourier Series in $L^2[-\pi, \pi]$

Trigonometric basis: $e_n(x) = (1/\sqrt{2\pi})\cdot e^{inx}$, $n \in \mathbb{Z}$. This is a complete orthonormal basis of $L^2[-\pi, \pi]$.

Fourier coefficients: $\hat{c}n = \frac{1}{2\pi} \int{-\pi}^{\pi} f(x) e^{-inx}, dx$.

Real form: $a_n = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x) \cos(nx), dx$, $b_n = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x) \sin(nx), dx$.

Convergence: In $L^2$: $|f - S_n f|_{L^2} \to 0$. Pointwise, if $f$ is piecewise smooth, then $S_nf(x) \to (f(x+0) + f(x-0))/2$.

Gibbs Phenomenon: Near a discontinuity, partial sums “overshoot” by about 9% of the jump height—a fundamental property of Fourier series.

Numerical Example

Problem: Find the Fourier series for $f(x) = x$ on $[-\pi, \pi]$ and derive the identity $\sum 1/n^2 = \pi^2/6$.

Step 1. $f(x) = x$ is an odd function, so $a_n = 0$ for all $n$.

Step 2. $b_n = \frac{1}{\pi} \int_{-\pi}^{\pi} x \sin(nx) dx$. Integrate by parts: $ \int x \sin(nx) dx = -\frac{x \cos(nx)}{n} + \frac{\sin(nx)}{n^2}. $ $ b_n = \frac{1}{\pi}\left[ -\frac{x \cos(nx)}{n} + \frac{\sin(nx)}{n^2}\right]_{-\pi}^{\pi}. $

At $\pi$: $-\pi \cdot \cos(n\pi)/n + 0 = -\pi \cdot (-1)^n/n$.

At $-\pi$: $\pi \cdot \cos(n\pi)/n + 0 = \pi \cdot (-1)^n / n$.

$ b_n = \frac{1}{\pi}\left(-\pi \cdot (-1)^n/n - \pi \cdot (-1)^n/n\right) = -2(-1)^n/n = 2(-1)^{n+1}/n. $

Step 3. Fourier series: $ x = 2 \sum_{n=1}^{\infty} \frac{(-1)^{n+1}}{n} \sin(nx) = 2\left( \sin x - \frac{\sin 2x}{2} + \frac{\sin 3x}{3} - \cdots \right). $

Step 4. Parseval’s Identity: $ |f|{L^2}^2 = \int{-\pi}^\pi x^2 dx = \frac{2\pi^3}{3}. $ Right side: $\sum_{n=1}^\infty |b_n|^2 \cdot \pi = \pi \sum 4/n^2$. By Parseval ($|f|^2 = \pi\sum b_n^2$): $ \frac{2\pi^3}{3} = \pi \sum \frac{4}{n^2} \implies \sum_{n=1}^\infty \frac{1}{n^2} = \frac{\pi^2}{6} \quad \checkmark. $

(The famous Euler identity of 1734, derivable from Hilbert space analysis.)

Step 5. At $x = \pi/2$: $\pi/2 = 2(1 - 1/3 + 1/5 - \ldots ) \implies \pi/4 = 1 - 1/3 + 1/5 - \ldots$ (Leibniz–Madhava series, $\sim$ 1400 AD.) $\checkmark$.

Real-life Application

Sound synthesis and audio processing: any sound is decomposed into a Fourier series. MP3 stores the Fourier coefficients, discarding “inaudible” components (below the psychoacoustic threshold). This enables reducing file size tenfold with no perceptible loss of quality.

Additional Aspects

An orthonormal basis ${e_n}$ in a Hilbert space allows any vector to be decomposed as $x = \sum \langle x, e_n \rangle \cdot e_n$ (Parseval’s equality: $|x|^2 = \sum |\langle x, e_n \rangle|^2$). In $L^2[-\pi, \pi]$, the basis ${e^{inx}/\sqrt{2\pi}}$ yields classical Fourier series. In $L^2(\mathbb{R})$ the analogue is the Fourier transform with continuous “index” $\omega$. Haar bases and wavelet bases provide alternative decompositions, better adapted to local features of the signal—the foundation of JPEG-2000, MP3, and many modern compression formats. In numerical linear algebra, Gram–Schmidt orthogonalization and QR decomposition construct orthonormal bases for efficient least squares and eigenvalue problem solving.

Connection with Other Fields of Mathematics

The theory of orthonormal bases is directly part of the spectral theory of self-adjoint operators, which underlies Fourier methods for solving differential equations. A classic example: decomposition over eigenfunctions of the Laplacian when solving the heat equation and vibrating string problems. Von Neumann’s spectral theorem for self-adjoint operators in Hilbert space formulates this as decomposition over a spectral measure, where orthonormal families of eigenfunctions play the role of a basis.

In functional analysis, orthonormal systems serve as a convenient tool in studying Banach spaces through their Hilbert models. The Riesz–Fischer theorem describes an isometric isomorphism between the space of square-summable sequences and the $L^2$-modules of Fourier coefficients, linking abstract geometry and the structure of functions. In operator theory, Schauder bases and orthonormal bases are compared when studying compact and nuclear operators, for example in the works of Hilbert and Schmidt.

In probability theory, orthonormal systems of functions are used to construct chaos decompositions. Hermite series describe Gaussian processes, and the Karhunen–Loève expansion is the analogue of Fourier series for stochastic processes with finite covariance measure. Orthonormal polynomial bases (Legendre, Chebyshev) arise in the theory of orthogonal polynomials and Gaussian quadratures, connecting Fourier analysis with numerical methods of integration and Fredholm equation solving.

In topology and geometric analysis, orthonormal systems of eigenforms of the Laplace–de Rham operator are used in the Hodge theorem, which decomposes the space of differential forms into harmonic, exact, and closed components. In algebra, an analogous role is played by orthonormal bases of representations of compact groups: by the Peter–Weyl theorem, the matrices of irreducible unitary representations form an orthonormal system in $L^2(G)$, generalizing the trigonometric Fourier series to the case of compact Lie groups.

Historical Note and Development of the Idea

The roots of expansions by orthogonal systems go back to the debates between Fourier, Lagrange, and d’Alembert in the early 19th century, regarding the representability of arbitrary functions as a series of sines and cosines in the context of the heat conduction problem for a rod. Fourier's book "Théorie analytique de la chaleur" (1822) became the starting point of systematic use of trigonometric series. Legendre polynomials were studied by Legendre in "Exercices de Calcul Intégral" (1811–1817); later, Jacobi and Sturm connected them with eigenvalue problems for differential operators. Hilbert and Riesz at the beginning of the 20th century (works of 1907–1913 in "Mathematische Annalen") formulated the notion of an abstract Hilbert space, where orthonormal systems received their modern definition, and Bessel’s inequality and Parseval’s identity became unified in a comprehensive theory. In 1915, Riesz proved the Riesz–Fischer theorem relating $L^2$ and $\ell^2$ via Fourier coefficients. In the mid-20th century, von Neumann, Stone, and Gelfand developed spectral theory, where orthonormal bases of eigenfunctions described quantum-mechanical operators. At the same time, Hardy, Littlewood, and Carleson studied delicate convergence of Fourier series; this culminated in Carleson’s theorem of 1966 on almost-everywhere convergence of the Fourier series for functions in $L^2$.