Functional Analysis

Formulas

The concept of “distance” is ubiquitous: the distance between points on a plane, between functions (how similar are two curves?), between strings (how many substitutions are needed to turn one word into another?). A metric space is the minimal abstraction that encapsulates the axioms of distance ...

Metric space (X, d): a set X with a function metric d: X×X → ℝ₊, satisfying: 1. Positivity: d(x,y) ≥ 0; d(x,y) = 0 ⟺ x = y. 2. Symmetry: d(x,y) = d(y,x). 3. Triangle inequality: d(x,z) ≤ d(x,y) + d(y,z).

The axioms are minimal: everything needed for analysis—convergence, continuity, compactness—follows from them. The concepts of “neighborhood”, “open set”, “closed set” are transferred literally.

C[a,b]: continuous functions on [a,b]. Supremum metric: d(f,g) = max|f(x)−g(x)|.

Formulas

A metric space is "bare" distance. A normed space adds linear structure: you can add elements and multiply them by scalars, and the norm is consistent with these operations. Banach spaces—complete normed spaces—are the main object of functional analysis and numerical methods. Here live linear ope...

A norm ‖·‖: V → ℝ on a vector space V (over ℝ or ℂ): 1. ‖x‖ ≥ 0; ‖x‖ = 0 ⟺ x = 0. 2. ‖αx‖ = |α|·‖x‖ (homogeneity). 3. ‖x+y‖ ≤ ‖x‖ + ‖y‖ (triangle inequality).

Equivalence of norms: ‖·‖_a ≈ ‖·‖_b if ∃C₁,C₂: C₁‖x‖_a ≤ ‖x‖_b ≤ C₂‖x‖_a. In finite-dimensional spaces, all norms are equivalent. In infinite-dimensional spaces—not so.

Examples: (ℝⁿ, ‖·‖_p), (lᵖ, ‖·‖_p), (C[a,b], ‖·‖_∞), (Lᵖ[a,b], ‖·‖_p) for p ∈ [1,∞].

Formulas

In finite-dimensional spaces, a closed and bounded ball is compact. In infinite-dimensional spaces — it is not: an orthonormal sequence $e_n$ in $l^2$ does not have a convergent subsequence. Compact operators are an "intermediate class": they map infinite-dimensional balls into precompact sets. T...

A compact set $K$: from any sequence in $K$ one can extract a subsequence converging to a point in $K$ (sequential compactness).

Hausdorff criterion: $K$ is compact $\iff$ $K$ is complete and totally bounded (for every $\epsilon > 0$ there exists a finite $\epsilon$-net: $K \subseteq \bigcup_{a\in A} B(a,\epsilon)$, $|A| < \infty$).

Heine–Borel theorem: In $\mathbb{R}^n$: $K$ is compact $\iff$ $K$ is closed and bounded. In infinite-dimensional spaces — false: the closed unit ball of $l^2$ is closed and bounded, but not compact.

Formulas

A Hilbert space is an “infinite-dimensional analogue of Euclidean space.” Here, there is an inner product, angle between elements, orthogonality, and orthogonal projections. Quantum mechanics: states of a system are elements $H = L^2(\mathbb{R}^3)$; operators are observables. Signal analysis: dec...

Pre-Hilbert Space $(H, \langle\cdot, \cdot\rangle)$: a vector space with a function $\langle\cdot, \cdot\rangle : H \times H \to \mathbb{R}$: 1. Linearity in the first argument: $\langle\alpha x + \beta y, z\rangle = \alpha\langle x, z\rangle + \beta\langle y, z\rangle$. 2. Symmetry: $\langle x, ...

Cauchy–Schwarz Inequality: $|\langle x, y \rangle| \leq \|x\|\cdot\|y\|$. Equality $\iff x$ and $y$ are linearly dependent.

Proof: consider $\|x - t\langle x, y\rangle/\|y\|^2 \cdot y\|^2 \geq 0$ and expand.

Formulas

In $\mathbb{R}^n$, any vector can be decomposed over an orthonormal basis: $x = \sum \langle x, e_i \rangle e_i$. In infinite-dimensional space, the analogue is decomposition over a complete orthonormal system. This generalizes Fourier series and forms the foundation of wavelet analysis and spect...

Fourier coefficients: $c_n = \langle x, e_n \rangle$. Partial sums $S_n x = \sum_{k=1}^n c_k e_k$ give the best approximation of $x$ in $\operatorname{span}\{e_1, ..., e_n\}$.

Bessel’s Inequality: $\sum_n |c_n|^2 \leq \|x\|^2$ for any ONS. Consequence: $c_n \to 0$.

Trigonometric basis: $e_n(x) = (1/\sqrt{2\pi})\cdot e^{inx}$, $n \in \mathbb{Z}$. This is a complete orthonormal basis of $L^2[-\pi, \pi]$.

Formulas

A linear functional $f: X \to \mathbb{R}$ is a "measurement" of elements: it assigns a number to each element in a linear way. The dual space $X^*$ contains all bounded (continuous) linear functionals. The Hahn–Banach theorem is an "extension tool": any partial measurement can be extended to the ...

Analytic form: Let $M$ be a subspace of a normed space $X$, $f$ a bounded linear functional on $M$. Then there exists $F \in X^*$ with $F|_M = f$ and $\|F\|_{X^*} = \|f\|_M$.

Geometric form: Two disjoint convex sets (one open) are separated by a hyperplane $\{ x : F(x) = c \}$ for some $F \in X^*$, $c \in \mathbb{R}$.

Corollaries: 1. For $x_0 \ne 0$: $\exists F \in X^*: F(x_0) = \|x_0\|$, $\|F\| = 1$. 2. $\|x\| = \sup_{F \in X^*,\, \|F\|=1} |F(x)|$ — dual representation of the norm. 3. $X$ separates functionals: $x \ne y \implies \exists F: F(x) \ne F(y)$.

Formulas

In matrix algebra, the spectrum of a matrix is its set of eigenvalues. In infinite-dimensional spaces, the concept is richer: the spectrum can be continuous (multiplication operator), point (eigenvalues), or residual. Spectral theory is the language of quantum mechanics: observables correspond to...

Resolvent set ρ(T): λ ∈ ρ(T) if (T−λI)⁻¹ exists as a bounded operator on all of X.

Multiplication operator: T: L²[a,b]→L²[a,b], Tf = g·f. σ(T) = closure of the image of g. σₚ(T) = ∅ (no eigenvalues) — continuous spectrum.

Shift operator: T: l²→l², T(x₁,x₂,...) = (0,x₁,x₂,...). σₚ = ∅, σ_r = open unit disk, σ_c = unit circle.

Formulas

Many problems in physics—wave scattering, heat transfer, electrostatics—reduce to integral equations of the form $x(t) - \lambda \int K(t, s) x(s) ds = f(t)$. The integral operator $K$ is compact. Fredholm theory is a direct generalization of the theory of systems of linear equations to infinite-...

Compact operator $K: X \to Y$: bounded and maps bounded sets into precompact ones. Equivalently: from any bounded sequence $\{x_n\}$ one can extract a subsequence whose image converges.

Classes: Operators of finite rank ($\dim \operatorname{Im} K < \infty$); Hilbert–Schmidt operators: $Kf(t) = \int K(t, s) f(s) ds$ with $\int \int |K(t, s)|^2 ds\, dt < \infty$.

Properties: The composition of a bounded and a compact operator is compact; the norm limit of compact operators is compact; a compact $K$ maps weakly convergent sequences into strongly convergent sequences.

Formulas

The heat conduction equation $du/dt = \Delta u$, the Schrödinger equation $i\hbar\partial\psi/\partial t = H\psi$, diffusion equations—all have the form $du/dt = Au$. The solution—“exponential of an operator”: $u(t) = e^{tA}u_0$. Semigroup theory makes this idea rigorous and gives criteria for th...

C₀-semigroup: a family $\{T(t)\}_{t\geq0}$ of bounded operators $X\rightarrow X$: 1. $T(0) = I$. 2. $T(t+s) = T(t)T(s)$ — semigroup property. 3. $\lim_{t\to0^+} \|T(t)x - x\| = 0$ for all $x \in X$ — strong continuity.

Infinitesimal generator: $Ax = \lim_{h\to0^+} (T(h)x - x)/h$. $D(A) = \{x : \text{limit exists}\}$ — domain of the generator.

Theorem: $A$ is the generator of a $C_0$-semigroup with $\|T(t)\| \leq Me^{\omega t}$ $\iff$: 1. $A$ is closed, $D(A)$ is dense in $X$. 2. For $\lambda > \omega$: $(\lambda I - A)^{-1}$ exists and $\|(\lambda I - A)^{-n}\| \leq M/(\lambda - \omega)^n$.

Formulas

Classical derivatives require smoothness. But problems in mechanics and physics have solutions with discontinuities: a load with a jump creates a "kink" in the deflection of the beam. Sobolev spaces generalize the notion of derivative to $L^2$-functions via the integration by parts formula. This ...

Motivation: For $u \in C^1$ and $\varphi \in C_0^\infty$: $\int u' \cdot \varphi\, dx = -\int u \cdot \varphi'\, dx$. If $u$ is not smooth, we define the weak derivative $v$ as a function from $L^1$ such that:

$ \int v \cdot \varphi\, dx = -\int u \cdot \varphi'\, dx\quad \text{for all } \varphi \in C_0^\infty(\Omega). $

$W^{k,p}(\Omega)$: Functions $u \in L^p(\Omega)$, all weak derivatives $D^\alpha u$ ($|\alpha| \leq k$) are also in $L^p$. Norm: $ \|u\|_{W^{k,p}} = \left(\sum_{|\alpha|\leq k} \|D^\alpha u\|_p^p \right)^{1/p}. $

Formulas

The Lax–Milgram theorem guarantees the existence and uniqueness of a weak solution. The Galerkin method approximates the infinite-dimensional problem by a finite-dimensional one: a finite-dimensional subspace $V_h \subset V$ is chosen. The finite element method (FEM) is a concrete implementation:...

Galerkin problem: Find $u_h \in V_h$ such that $a(u_h, v_h) = F(v_h)$ for all $v_h \in V_h$.

Basis $\{\varphi_1, \ldots, \varphi_n\}$. Decomposition $u_h = \sum_j u_j \varphi_j$. Substitution yields the linear system $Ku = f$:

Céa's Theorem: $\|u - u_h\|_V \leq (M/\alpha) \cdot \inf_{v_h \in V_h} \|u - v_h\|_V$. The Galerkin approximation is optimal up to the constant $M/\alpha$.

Formulas

Classical optimization minimizes a function on ℝⁿ. Infinite-dimensional optimization minimizes a functional J: H → ℝ — a “function of a function.” Finding an optimal trajectory, minimizing the length of a curve, inverse signal reconstruction problems — all these are infinite-dimensional optimizat...

Convex functional J: H → ℝ: J(λu + (1−λ)v) ≤ λJ(u) + (1−λ)J(v) for λ ∈ [0,1].

Existence theorem: If J is convex, lower semi-continuous (LSC: lim inf J(uₙ) ≥ J(u) for uₙ ⇀ u), and coercive (J(u) → +∞ as ‖u‖ → ∞), then the minimum is attained. For strict convexity — uniqueness.

Fréchet derivative: J'(u) ∈ H* such that J(u+h) = J(u) + J'(u)(h) + o(‖h‖). In a Hilbert space: J'(u) ≡ ∇J(u) ∈ H via the Riesz theorem.

Formulas

The Fourier transform turns any signal into a superposition of sines and cosines of various frequencies. For differential equations this is gold: differentiation $\mapsto$ multiplication by frequency; convolution $\mapsto$ pointwise multiplication. Plancherel's theorem: Fourier is a unitary (isom...

Fourier transform in $L^1(\mathbb{R})$: $ \hat{f}(\xi) = \int_\mathbb{R} f(x) e^{-2\pi i x \xi} dx. $

Boundedness: $|\hat{f}(\xi)| \leq \|f\|_{L^1}$. Riemann–Lebesgue lemma: $\hat{f}(\xi) \to 0$ as $|\xi| \to \infty$.

Inverse: $f(x) = \int \hat{f}(\xi) e^{2\pi i x \xi} d\xi$ (inversion formula, when $\hat{f} \in L^1$).

Formulas

Physicists have long used the Dirac “delta function” δ(x): infinite at the point 0, zero everywhere else, with ∫δ(x)dx = 1. Classically, such an object does not exist. Between 1945–50, Schwartz constructed the theory of distributions—making δ(x) a rigorous object. The key property: every distribu...

D(Ω) = C₀^∞(Ω): infinitely differentiable functions with compact support. Topology: φₙ → φ—supports lie within a common K, D^α φₙ ⇒ D^α φ uniformly for all α.

Schwartz space S(ℝⁿ): rapidly decaying functions: sup_x |x^β D^α φ| < ∞ for all α,β. D ⊂ S.

Heaviside function: H(x) = 0 for x < 0, 1 for x ≥ 0. ⟨H', φ⟩ = −⟨H, φ'⟩ = −∫₀^∞ φ'(x)dx = −[φ]₀^∞ = φ(0) = ⟨δ, φ⟩. H' = δ ✓.

Formulas

Fourier analysis "sees" frequencies globally: a discontinuity at a single point is "smeared" across all coefficients. Wavelets are a "microscope with adjustable magnification": they analyze the signal locally in both time and scale. High frequencies correspond to short wavelets (good time resolut...

Heisenberg's Uncertainty Principle: $\sigma_t \cdot \sigma_\omega \geq \frac{1}{4\pi}$, where $\sigma_t, \sigma_\omega$ are the root mean square deviations of $f(t)$ and $\hat{f}(\omega)$. It is impossible to have both good time and good frequency resolution simultaneously.

Windowed Fourier (STFT): $f(t) \cdot g(t-\tau) \to$ Fourier. With a fixed window $g \to$ same resolution for all frequencies. This is suboptimal for signals with variable frequency.

Wavelets: adapt the window size to the scale. High frequencies correspond to a narrow wavelet (precise localization in time). Low frequencies correspond to a wide wavelet (precise localization in frequency).