Stochastic Differential Games: Problem Statement

Uncertainty in Strategic Interaction

In deterministic games, the future is predetermined: knowing the initial state and the strategies of both players, one can accurately predict the trajectory. In real systems, there is always noise: market volatility, turbulence, thermal fluctuations. Stochastic differential games (SDG) generalize deterministic theory to the case when the dynamics includes a random perturbation in the form of Brownian motion. This opens up new analytical tools: stochastic HJI, BSDE, connections with financial mathematics.

Stochastic Dynamics

Stochastic differential equation (SDE):

dX = f(X, u, v, t) dt + σ(X, t) dW

Here:

• X ∈ ℝⁿ — state
• u ∈ U, v ∈ V — controls of the players
• σ(X, t) ∈ ℝ^{n×m} — volatility matrix
• W — m-dimensional standard Brownian process (Wiener process)

Functional: J = E[g(X(T)) + ∫₀ᵀ F(X, u, v, t) dt] — expected cost.

Goal: min_u max_v J (or vice versa).

Bellman's Principle in the Stochastic Case

For V(x,t) = min_u max_v E[...| X(t) = x]:

V(x,t) = min_u max_v E_{dW}[F dt + V(X+dX, t+dt)]

We use Ito's formula: dV = V_t dt + ∇V·dX + (1/2)tr(σσᵀ ∇²V) dt.

Substituting and taking the limit:

Stochastic HJI equation:

∂V/∂t + (1/2)tr(σσᵀ ∇²V) + H*(x, t, ∇V) = 0

The additional term (1/2)tr(σσᵀ ∇²V) — the “Itô term” (a consequence of Ito’s formula for a stochastic differential).

Comparison:

• Deterministic HJI: ∂V/∂t + H* = 0
• Stochastic HJI: ∂V/∂t + (σ²/2)∆V + H* = 0

Stochasticity adds the Laplacian ∆V = tr(∇²V), which “smooths” the value function.

Optimal Strategies and the Minimax Principle

Optimal game Hamiltonian: H*(x,t,p,Q) = min_{u∈U} max_{v∈V} [F + pᵀf + (1/2)tr(σσᵀQ)]

Note: (1/2)tr(σσᵀQ) does not depend on u and v → the “noise” term does not affect the optimal strategies!

Consequence: the optimal u* and v* in a stochastic game are the same as in the deterministic game (with the same H). Noise changes the value of the game V but not the optimal strategies.

Related Backward Stochastic Differential Equations

For problems with horizon T and a terminal condition, V can be represented via a BSDE (backward SDE):

Forward SDE (for X): dX = f(X, u*, v*, t)dt + σ dW, X(0) = x₀.

Backward SDE (for Y ≈ V(X,t), Z ≈ σᵀ∇V):

dY = −H(X, u*, v*, ∇V, t) dt + Zᵀ dW, Y(T) = g(X(T)).

The pair (Y, Z) — “adjoint processes”. Z is the martingale component, analogous to the “volatility” of the adjoint process.

Connection between BSDE and the Pontryagin principle: the adjoint variable p(t) in the stochastic case is not just an ODE, but an SDE for (p, q), where q is an additional martingale term.

Complete Analysis: Stochastic LQ-Game

Problem: dX = (u+v)dt + σ dW, X(0) = 1, J = E[X(T)² + ∫₀ᵀ (u² − v²)dt].

Stochastic Riccati-Isaacs equation (same form as in the deterministic case):

−Ṗ = −P² + P² = 0 → P = const = 1.

Optimal strategies: u* = −X, v* = X (the same as in the deterministic problem!).

Closed system: dX = (−X+X)dt + σ dW = σ dW → X(t) = 1 + σW(t).

Value of the game: V = E[X(T)²] + ∫₀ᵀ E[u² − v²]dt = E[(1+σW(T))²] = 1 + σ²T.

Deterministic case (σ=0): V = 1. Stochastic: V = 1 + σ²T.

Physical meaning: noise “increases” the game’s value. This is intuitive: under random walks of X, the quadratic functional is larger due to the “spread” of X.

Connection with Stochastic Control and Finance

With a single player (no v): stochastic optimal control (Fleming-Rishel). HJB equation: ∂V/∂t + (σ²/2)∆V + min_u{F + p·f} = 0.

Black-Scholes equation is a particular case! dS = μS dt + σS dW (stock price). The option price V satisfies: ∂V/∂t + (1/2)σ²S²∂²V/∂S² + rS ∂V/∂S − rV = 0. This is HJB without control (u = 0) + boundary condition. All of financial option theory is a particular case of stochastic optimal control!

Stochastic Differential Games

When the dynamics is subject to random perturbations (market noise, wind, measurement uncertainty), the classical HJI is generalized to the stochastic HJI:

V_t + min_u max_v {F + ∇V · f + (1/2) tr(σσᵀ ∇²V)} = 0

The extra term tr(σσᵀ ∇²V) is the “diffusion” of the value function due to noise. The equation is parabolic (not hyperbolic, as in the deterministic case), which simplifies analysis: viscosity solutions exist under broad conditions.

Connection with Financial Mathematics

The Black-Scholes equation for option pricing is a particular case of a stochastic differential game (with one player—the option seller hedging against market movements). Extensions—games between traders (market making), optimal execution of large orders accounting for price impact.

Numerical Methods

• Stochastic grids: extension of finite differences
• Symmetric splitting: decomposition into deterministic and noise parts
• Backward SDE (BSDE): representation of V via stochastic equations, numerical solution via the Longstaff-Schwartz method
• Deep BSDE (E-Han-Jentzen, 2017): neural network approximation for high dimensions

Modern Applications

• Portfolio management under risk-uncertainty: robust Markowitz-Shanning optimization
• Distributed energy systems: coordination of millions of consumers with noisy loads
• Autonomous vehicles: accounting for uncertainty in the behavior of other participants
• Robotics under difficult conditions: quadcopters in wind