Differential Games

Definitions

Dynamics

— $\dot{x} = f(x, u, v)$, $x \in \mathbb{R}^n$ (state), $u \in U$ (control of player P, minimizer), $v \in V$ (control of player E, maximizer).

Functional

— $J = g(x(T)) + \int_0^T F(x(t), u(t), v(t)) \,dt$.

Open-loop strategy

— $u = u(t)$—control as a function of time only. Planned in advance, does not react to the system state. Mathematically simpler, but unrealistic in practice.

Feedback-form strategy

— $u = \alpha(x, t)$—control as a function of the current state. Each player "sees" $x$ and reacts. This is a realistic model for real systems.

Output feedback strategy

— $u = \alpha(y, t)$, where $y = h(x)$—incomplete observation. The most complex case.

Important Fact

— for zero-sum games under Isaacs' condition, the value of the game is the same for open-loop and feedback strategies!

By sum

— zero-sum ($J_1 + J_2 = 0$—interests are completely opposed), nonzero-sum (each has their own $J$), cooperative (players can make agreements).

By information

— full information (both see $x$), incomplete (part of $x$ is hidden).

By horizon

— finite ($T < \infty$), infinite ($T = \infty$, pursuit until capture problem).

By dynamics

— linear ($f = Ax + Bu + Cv$), nonlinear, stochastic (with noise).

Aviation

— interceptor (P) and target (E). The interceptor wants to minimize the distance to the target. The target wants to maximize it. Optimal strategy for P: fly toward the "anticipation point," not directly.

Economics

— two companies set prices for competing products. The price of the first affects the demand for the second and vice versa. Dynamic model $\to$ differential game.

Autonomous vehicles

— two cars at an intersection. Each wants to pass without crashing. This is a Stackelberg or Nash equilibrium problem in differential games.

Biology

— predator and prey in three-dimensional space. Strategically, optimal "chase" is not always "in a straight line."

Formulas

In the 1950s, the Cold War confronted military analysts with a new challenge: how to intercept a highly maneuverable missile? How to evade an interceptor? This is not an optimal control problem in the usual sense—the "target" has its own will and actively resists. Rufus Isaacs, working at RAND Co...

In the optimal control problem: one player controls the system, minimizing the cost. Nature is "not against"—there is no opponent. In a differential game: two (or more) players control a shared system, and their goals conflict.

This makes the problem fundamentally more complex: the optimal strategy of one player depends on the strategy of the other, and vice versa. It's a "loop": you need to find strategies that are simultaneously optimal given the opponent's strategies.

Dynamics: $\dot{x} = f(x, u, v)$, $x \in \mathbb{R}^n$ (state), $u \in U$ (control of player P, minimizer), $v \in V$ (control of player E, maximizer).

Definitions

Definition

— V(x, t) = min_{u(·)} max_{v(·)} J(x, t; u(·), v(·)) is the value of the game started from state x at time t.

Terminal condition

— V(x, T) = g(x), the cost of the terminal state.

HJI equation

— ∂V/∂t + H*(x, t, ∇V) = 0

Isaacs' condition

— min_{u∈U} max_{v∈V} H(x,u,v,p) = max_{v∈V} min_{u∈U} H(x,u,v,p)

Meaning

— the "game Hamiltonian" is the same upon swapping min and max. This is analogous to the "equilibrium" condition in static games.

When satisfied

— with compact U, V and continuous H — always (Neumann minimax theorem). With disconnected controls there may be problems.

If violated

— the value of the game may depend on the order of moves (who "announces" their strategy first). One must distinguish the "lower value" V⁻ = max_v min_u J and the "upper value" V⁺ = min_u max_v J.

Numerical methods

— finite differences with monotone schemes (Osher–Shu upwind), level-set methods (Osher–Sethian). These methods closely approximate the viscosity solution.

Dynamics

— ẋ = u − v, x ∈ ℝ, u ∈ [−1,1] (P), v ∈ [−a,a] (E, a < 1).

Cost functional

— J = x(T)² (minimize the distance at T).

HJI

— ∂V/∂t + min_{|u|≤1} max_{|v|≤a} [(u−v) ∂V/∂x] = 0.

Formulas

In optimal control theory, the value function V(x, t) satisfies the Hamilton–Jacobi–Bellman (HJB) equation. When a second player with opposing interests is introduced, the equation is modified: instead of a min over u, there is simultaneously a min over u and a max over v. This is the Hamilton–Ja...

Definition: V(x, t) = min_{u(·)} max_{v(·)} J(x, t; u(·), v(·)) is the value of the game started from state x at time t.

where the game Hamiltonian: H*(x, t, p) = min_{u∈U} max_{v∈V} {F(x, u, v, t) + pᵀ f(x, u, v, t)}

Isaacs' condition: min_{u∈U} max_{v∈V} H(x,u,v,p) = max_{v∈V} min_{u∈U} H(x,u,v,p)

Definitions

Distance

— $r(t) = |x_P(t) - x_E(t)|$.

Capture

— $r(t) \leq l$ (capture radius). $P$ wants to achieve capture, $E$ wants to avoid it.

Barrier surface

— Isaacs found the geometry of the region of initial conditions from which capture is guaranteed (Capture Zone) and from which evasion is guaranteed (Escape Zone). The boundary — the "barrier" — is a special solution of the HJI.

Optimality

— with linear dynamics and small angle, PN with $N \rightarrow \infty$ is optimal. With $N = 4-5$ it is practically close to optimal.

Why PN works

— the aim of interception is a future point (“lead point”), not the target’s current position. PN automatically computes lead via angular rate.

Parameters

— $P$ and $E$ in $\mathbb{R}^2$, $\alpha = 2$, $\beta = 1$, $l = 0$ (point capture). Initial positions: $x_P = (0,0)$, $x_E = (10, 0)$.

Drones

— the problem of safe collision avoidance is the reverse of the pursuit-evasion problem. Car $A$ wants to “escape” from $B$ to avoid collision. The optimal "evading" algorithm minimizes risk.

Multi-UAV capture

— a group of pursuers versus one evader. Result: two coordinated pursuers are enough to guarantee capture even of a faster $E$ in a bounded space.

Robots

— patrol-evader problem for autonomous security robots. The SafeRL library uses HJI to compute “safe” controls in real time.

Formulas

A hunter pursues a hare. A fox chases a rabbit. A military interceptor — pursues a target. The Pursuit-Evasion (PE) problem is one of the oldest applied mathematical problems. Isaacs gave it a rigorous mathematical form and discovered some stunning results: the optimal strategy is not always “fly...

Two players in $\mathbb{R}^2$. Pursuer $P$ with position $x_P \in \mathbb{R}^2$, speed $|u| \leq \alpha$. Evader $E$ with position $x_E \in \mathbb{R}^2$, speed $|v| \leq \beta$.

Dynamics: $\dot{x}_P = u$, $\dot{x}_E = v$, $|u| \leq \alpha$, $|v| \leq \beta$.

Capture: $r(t) \leq l$ (capture radius). $P$ wants to achieve capture, $E$ wants to avoid it.

Definitions

Meaning

— the maximum "guaranteed" payoff for player 1 = the minimum "guaranteed" loss for player 2. This value is called the value of the game.

Proof via LP

— the task $\max_x \min_y x^\top A y$ is a linear program (one can introduce $v = \min_y x^\top A y$). Strong duality of LP $\rightarrow$ equality min = max.

Algorithm for finding

— we write as an LP. Both problems are LPs, strong duality.

Important fact

— for zero-sum games with fixed strategies — there is a difference!

Open-loop strategies

— $u = u(t)$, $v = v(t)$ (fixed before the game). The "open" order of moves:

Feedback strategies

— $u = \alpha(x, t)$, $v = \beta(x, t)$. Under Isaacs' condition: $\min \max = \max \min = V(x_0)$. Feedback strategies "bypass" the issue of move order.

Solution

— optimal strategies are linear:

Problem

— $\dot{x} = u + v$, $x(0) = 1$, $J = x(T)^2 + \int_0^T [u^2 - v^2] dt$ ($T = 1$).

Formulas

In 1928, John von Neumann proved a remarkable fact: in any zero-sum matrix game there exists an "equilibrium." The minimizer cannot "worsen" the result below some value V, and the maximizer cannot raise it above the same V. This value V is the "value of the game." The minimax theorem is the mathe...

For a payoff matrix $A \in \mathbb{R}^{m \times n}$: $ \max_{x \in \Delta_m} \min_{y \in \Delta_n} x^\top A y = \min_{y \in \Delta_n} \max_{x \in \Delta_m} x^\top A y $ where $\Delta_k$ is the standard simplex of mixed strategies (a probability distribution over $k$ pure strategies).

Meaning: the maximum "guaranteed" payoff for player 1 = the minimum "guaranteed" loss for player 2. This value is called the value of the game.

Proof via LP: the task $\max_x \min_y x^\top A y$ is a linear program (one can introduce $v = \min_y x^\top A y$). Strong duality of LP $\rightarrow$ equality min = max.

Definitions

System

— ẋ = Ax + Bu + Dw (w is an unknown disturbance), output z = Cx + Eu.

H∞ Problem

— find control u = K(x) minimizing the "gain coefficient" from w to z in the worst case:

As a differential game

— control u is the minimizer, disturbance w is the maximizer:

Connection with stability

— if the H∞ norm is finite—the system is stable. The minimal γ for which the problem is solvable is the system's "robustness margin."

Model

— ẋ₁ = x₂ (body velocity), mẋ₂ = −k(x₁−x₃) − c(x₂−x₄) + u (damper), Mẋ₃ = x₄, Mẋ₄ = k(x₁−x₃) + c(x₂−x₄) − kₜ(x₃−w). Here w is the road roughness, u is active suspension.

Goal

— minimize ‖body acceleration ẍ₁‖ under limited ‖u‖ and worst-case ‖w‖.

Result

— An H∞ regulator reduces body acceleration by 30-40% compared to passive suspension under the same constraints on suspension travel.

Formulas

In real control systems—aircraft, power systems, chemical reactors—there are always unknown disturbances: turbulence, load changes, parametric uncertainty. Classical control (LQR) is optimal for a known model, but can "fail" in the presence of disturbances. H∞ control builds a regulator that is g...

H∞ Problem: find control u = K(x) minimizing the "gain coefficient" from w to z in the worst case:

‖T_{zw}‖_∞ is the H∞ norm of the transfer function from w to z. This is the "worst-case signal gain."

As a differential game: control u is the minimizer, disturbance w is the maximizer:

Definitions

For LQ games

— V(x,t) = xᵀP(t)x (quadratic in x), P(t) is the Riccati-Isaacs matrix satisfying an ODE.

Solvability condition

— the Riccati equation must have a solution over all [0, T]. For large T (or large C), the solution may "blow up"—the value of the game is unbounded.

Reach-Avoid set

— initial conditions x₀ from which P can guarantee capture. The boundary = barrier surface.

Computation

— level-set method (Hamilton-Jacobi toolbox). V(x,t) = 0 — boundary. V(x,t) < 0 — P’s zone, V(x,t) > 0 — E’s zone.

System

— ẋP = uP, ẋE = uE, |uP| ≤ αP, |uE| ≤ αE. Capture: |xP − xE| ≤ l.

Change of variable

— r = xP − xE (relative position). ṙ = uP − uE.

Finite difference method

— discretization in x on a grid, integration of HJI backwards in t.

Curse of dimensionality

— for n=4 and step h=0.1: 10⁴ = 10,000 points—feasible. For n=6: 10⁶—hard. For n=10: impossible.

Formulas

In problems with a finite horizon [0, T], the system state x(T) occupies a special status: the function g(x(T)) defines the "terminal payoff". This determines the "boundaries"—the sets of initial states x₀ from which P or E can guarantee the desired outcome. Mathematically: the value function is ...

The principle of optimality gives: V(x,t) = min_u max_v {F(x,u,v,t)dt + V(x+f·dt, t+dt)}.

This is the "Bellman equation" in reverse time: knowing V at t+dt, we compute V at t. We start from T (where V(x,T) = g(x)) and "move backward".

For LQ games: V(x,t) = xᵀP(t)x (quadratic in x), P(t) is the Riccati-Isaacs matrix satisfying an ODE.

Definitions

Dynamics

— $\dot{x} = f(x, u_1, u_2, ..., u_N, t).$

Player $i

#x27;s functional

— $J_i(u_1, ..., u_N) = g_i(x(T)) + \int_0^T F_i(x, u_1, ..., u_N, t) dt.$

Key difference from zero-sum

— $\sum_i J_i \neq const$ in general. "Collective" win is possible, or conversely — everyone can lose.

Meaning

— No player can improve their outcome by unilaterally changing their strategy (with the strategies of others fixed).

Existence

— Under reasonable conditions (compact $U_i$, continuous $J_i$), an NE in mixed strategies exists (Nash's theorem, 1950).

Uniqueness

— In general, NE may be multiple or unique. For LQ games with "diagonal" structure — unique.

Two-level game

— the leader knows the follower's reaction function $R(u_L) = \arg\min_{u_F} J_F(u_L, u_F).$ The leader minimizes $J_L(u_L, R(u_L))$ — "solves first".

Stackelberg equilibrium

— set $(u_L^*, u_F^*)$ with $u_F^* = R(u_L^*)$, $u_L^* = \arg\min_{u_L} J_L(u_L, R(u_L)).$

Property

— The leader is always no worse than in NE (has a "first-move advantage").

Static NE

— $\frac{\partial \pi_1}{\partial p_1} = 0$ and $\frac{\partial \pi_2}{\partial p_2} = 0 \rightarrow p_1^* = p_2^* = \frac{a + b k}{2b - c}.$

Nash equilibrium

— via Riccati equations. Price trajectories $p_1(t)$, $p_2(t)$ converge to static NE — "dynamics of price competition."

Best Response Dynamics

— each player alternately optimizes their strategy given others' fixed. Convergence to NE isn't guaranteed, but often works.

Deep Nash

— explicit formulation of Nash conditions as a system of equations + neural network for joint solution.

Formulas

Real-world competition is rarely a duel. Three companies share a market. Five countries manage fisheries. A hundred traders operate in a market. In such situations, the structure of strategic interaction is fundamentally more complex: each player optimizes against "all the others at once," and no...

N players, system state $x \in \mathbb{R}^n$, player $i$ controls $u_i \in U_i$.

Player $i

Key difference from zero-sum: $\sum_i J_i \neq const$ in general. "Collective" win is possible, or conversely — everyone can lose.

Definitions

Superadditivity

— $v(S \cup T) \geq v(S) + v(T)$ for $S \cap T = \varnothing$. If there is "synergy" from cooperation, it is advantageous to unite.

Axiomatization

— the unique allocation satisfying:

Core

— the set of payoff allocations $(x_1, \ldots, x_N)$ with:

Example

— countries agreed at the beginning of the game on CO₂ restrictions. After 10 years, country A discovers it is better off "leaving" the agreement. The arrangement is "unstable" — not dynamically consistent.

Model

— $N$ countries fish in a shared ocean. Fish biomass: $\dot{x} = r x(1 - x/K) - \sum_i u_i$ (logistic growth minus total catch).

Payoff for $i$

— $J_i = \int_0^\infty e^{-\rho t} \left( u_i - c u_i^2 / (2x) \right) dt$ (profit from catch accounting for costs).

Nash (non-cooperative)

— each country overfishes $\rightarrow$ "tragedy of the commons" (Hardin). Resource is depleted.

Cooperative solution

— maximize $\sum_i J_i \rightarrow$ lower total catch $\rightarrow$ higher total payoff! The Shapley value distributes it fairly.

Dynamic consistency

— IDP ensures that each country receives payments such that "leaving" the agreement is never profitable.

Problem

— $v(\{1\}) = 2$, $v(\{2\}) = 3$, $v(\{3\}) = 4$, $v(\{1,2\}) = 7$, $v(\{1,3\}) = 8$, $v(\{2,3\}) = 9$, $v(\{1,2,3\}) = 12$.

Formulas

Nash equilibrium describes a "no-agreement situation": every player for themselves. But in reality, players often can agree on joint strategies and share the payoff. Fishing nations agree to limit catches. Countries create climate agreements. Companies unite into consortia. Cooperative games stud...

$v(S)$ = the maximum total payoff that $S$ can guarantee itself through joint actions

Superadditivity: $v(S \cup T) \geq v(S) + v(T)$ for $S \cap T = \varnothing$. If there is "synergy" from cooperation, it is advantageous to unite.

With superadditivity: $v(N) \geq \sum_i v(\{i\})$ — it is more profitable for everyone to unite than to act individually.

Definitions

Homogeneity assumption

— All agents are identical (i.i.d. — independently and identically distributed initial states).

Nash Equilibrium in the limit $N\to\infty$

— A typical agent optimizes their strategy, treating $m(x,t)$ as “given” (not dependent on their own actions). In equilibrium, $m(x,t)$ is generated by exactly this optimal strategy of the typical agent — self-consistency!

HJB (backward in time)

— Optimal control $u^*(x, t)$ for the typical agent given $m$:

FPK (forward in time)

— Evolution of the distribution $m$ under the optimal strategy $u^*(x, t)$:

Self-consistency

— $V$ depends on $m$ (the agent adapts to the crowd), and $m$ depends on $V$ (the crowd moves optimally). This is a nonlinear system!

Existence of a solution

— Under reasonable conditions on $H$ there exists a solution $(V, m)$ to the MFG system. For monotonic games (where “more agents $\to$ less attractive”), uniqueness holds.

Crowd dynamics

— $m(x,t)$ is the density of pedestrians. Each one chooses the path, minimizing time plus discomfort from crowding. $H = |u|^2/2 + \alpha m$ (penalty for dense crowds). The MFG solution reproduces the observed physical effect of “formation of lanes”...

Financial markets

— “Trading with price impact.” $N$ traders sell an asset, each sale lowers the price. MFG gives the “optimal” trading algorithm for each trader, accounting for the aggregate influence of the entire group.

Telecommunications

— Users of a distributed network select resources (channels, servers). MFG describes the “distributional equilibrium" for a large number of users.

Epidemiology

— Agents choose their level of “social distancing.” MFG describes “rational” behavior during a pandemic — which may not coincide with the “social optimum”!

Deep MFG (Carmona-Lauriere, 2021)

— Approximate $V(x, t)$ and $m(x, t)$ with neural networks. Scales to high dimensions (50+ variables).

Problem

— $N\to\infty$ particles seek to reach a target at $x=0$. Dynamics: $dX = u\,dt + \sigma\,dW$. Cost: $J = E\left[\int \left( \frac{|u|^2}{2} + \alpha m(X, t) \right)dt + |X(T)|^2\right]$.

MFG system

— HJB: $-\frac{\partial V}{\partial t} - \frac{\sigma^2}{2} \Delta V + \frac{|\nabla V|^2}{2} + \alpha m = 0$. FPK: $\frac{\partial m}{\partial t} + \mathrm{div}(m \nabla V) - \frac{\sigma^2}{2} \Delta m = 0$.

Imagine thousands of traders on a financial market. Each one is rational and influences the price, but each is “small” compared to the market as a whole. Or thousands of pedestrians in a narrow corridor — each one optimally chooses their path, but interacts with the “average density” of the crowd...

With $N\to\infty$, a “typical” agent interacts not with specific other agents, but with the “distribution” of the entire population $m(x, t)$ — the density of agents in state $x$ at time $t$.

Homogeneity assumption: All agents are identical (i.i.d. — independently and identically distributed initial states).

Nash Equilibrium in the limit $N\to\infty$: A typical agent optimizes their strategy, treating $m(x,t)$ as “given” (not dependent on their own actions). In equilibrium, $m(x,t)$ is generated by exactly this optimal strategy of the typical agent — self-consistency!

Definitions

Connection between BSDE and the Pontryagin principle

— the adjoint variable p(t) in the stochastic case is not just an ODE, but an SDE for (p, q), where q is an additional martingale term.

Formulas

In deterministic games, the future is predetermined: knowing the initial state and the strategies of both players, one can accurately predict the trajectory. In real systems, there is always noise: market volatility, turbulence, thermal fluctuations. Stochastic differential games (SDG) generalize...

The additional term (1/2)tr(σσᵀ ∇²V) — the “Itô term” (a consequence of Ito’s formula for a stochastic differential).

Stochasticity adds the Laplacian ∆V = tr(∇²V), which “smooths” the value function.

Optimal game Hamiltonian: H*(x,t,p,Q) = min_{u∈U} max_{v∈V} [F + pᵀf + (1/2)tr(σσᵀQ)]

Definitions

Non-stationarity problem

— if agent 1 updates its policy, the environment from the viewpoint of agent 2 changes. The "target" moves—Q-learning convergence is not guaranteed!

Advantages

— simplicity, scalability to many agents.

Drawbacks

— no theoretical guarantees of convergence. The environment is non-stationary. Often works in practice!

During training

— critic $Q_i(x, a_1,...,a_N)$ sees all states and actions. Actor $\mu_i(o_i)$ sees only its own observation. The critic provides a "stable" quality estimate.

During execution

— each agent acts only based on its own $o_i$ — decentralized.

Critic update

— $L_i = E[(Q_i(x,a) - y_i)^2]$, $y_i = r_i + \gamma Q_i'(x', a_1',...,a_N')|_{a_j' = \mu_j'(o_j')}$.

Actor update

— $\nabla_{\theta_i} J = E[\nabla_{a_i} Q_i \cdot \nabla_{\theta_i} \mu_i(o_i)]$.

Advantage

— the critic "sees" the full picture → stable learning. In NE: $Q_i(x, a^*)$ accurately estimates the equilibrium value.

Self-play

— the agent plays against a copy of itself. With proper implementation, it converges to NE for two-player zero-sum games (e.g., chess, go).

AlphaGo/AlphaZero

— pure self-play + MCTS (Monte Carlo Tree Search) + deep neural networks. Achieves superhuman level in go and chess—essentially solving a huge discrete "differential" game.

League Training (AlphaStar)

— for StarCraft II—a game with significant uncertainty. The "league" is a set of heterogeneous past agent versions, avoiding "cyclicality" ($A$ beats $B$, $B$ beats $C$, $C$ beats $A$).

Policy gradient for games

— $\nabla_\theta E[J] = E\left[\sum_t \nabla_\theta \log \pi_\theta(a_{i,t}|o_{i,t}) \cdot A_{i,t}\right]$, where $A_{i,t} = Q_i(s, a) - V_i(s)$—advantage.

MAPPO (Multi-Agent PPO)

— extension of PPO to $N$ agents with separate critics. The standard in modern MARL tasks (StarCraft, Google Football, traffic optimization).

Task

— 2 agents, goal—to meet at point (5, 5). Joint reward: $r = -|x_1 - goal| - |x_2 - goal|$.

Nash interpretation

— in NE both agents move directly to (5,5)—neither can improve their result unilaterally. MADDPG finds this NE through interaction.

Formulas

Differential games provide an elegant theory, but in practice, analytic solutions to the HJI are possible only in a few special cases (LQ-games, problems with simple dynamics). In real tasks there is nonlinear dynamics, high dimensionality, and the system model is unknown. Here, reinforcement lea...

Agent $i$ wants to maximize the discounted sum of rewards: $\sum_t \gamma^t r_{i,t}$.

Non-stationarity problem: if agent 1 updates its policy, the environment from the viewpoint of agent 2 changes. The "target" moves—Q-learning convergence is not guaranteed!

Each agent $i$ learns its own Q-function $Q_i(s, a_i)$ independently, ignoring the actions of others.

Definitions

Model

— Two producers release $q_i(t)$ units of goods. Price depends on total output: $P(t) = a − b(q_1(t) + q_2(t))$.

Capital dynamics

— $\dot{K}_i = u_i − δK_i$ (investment $u_i$, depreciation $δ$). $q_i = αK_i$.

Functional

— $J_i = \int_0^\infty e^{−ρt} [P(t)q_i(t) − c u_i(t)] dt$.

Nash equilibrium

— via Hamiltonian equations or DP. Optimal investments $u^*_i(t)$ depend on own capital and the competitor’s capital.

Comparison with cooperative

— Cooperative players produce less (monopoly output), earn more in total. Nash—higher total output, lower price, “social losses” are smaller. Prisoner's dilemma in dynamics!

Equilibrium

— $\dot{x} = \dot{y} = 0 → x^* = (an + g_2b + g_1n)/(mn − ab)$, similarly $y^*$.

Stability

— $mn > ab$ → stable equilibrium. $mn < ab$ → unstable arms race (armaments grow until war or disarmament). This is the classic model of the Cold War!

Game generalization

— Each country optimally chooses $u_i$ (rate of arms increase). $J_i = \int e^{−ρt} [−c u_i^2 + π_i(x, y)] dt$. Nash equilibrium via Riccati equations—gives the "optimal" arms race.

Conclusion

— Nash equilibrium of the "race" is inefficient—total arms expenditures in NE are higher than with a cooperative agreement. This is a mathematical justification for why arms control is beneficial to both sides.

Effect on society

— Nash investments in R&D vs optimal? With spillovers ($α > 0$—one firm "learns" from another): Nash investments may be lower or higher than socially optimal. Patent system influences this balance.

Task

— $N$ traders want to sell their positions $x_i(0)$ within time $T$. Sales affect price: $P(t) = P_0 − λ \sum_i \dot{q}_i(t)$ (aggregate “price impact”).

MFG equations

— HJB: $−\frac{\partial V}{\partial t} + λQ \frac{\partial V}{\partial x} + \frac{(\frac{\partial V}{\partial x})^2}{2λ} = 0$. FPK: $\frac{\partial m}{\partial t} + \frac{\partial[m \cdot H_pV]}{\partial x} = 0$.

Nash without agreement

— Each country maximizes $\int e^{−ρt}[b_i(e_i) − d_i(S)] dt$. The external effect of emissions (damage to others) is not considered → "tragedy of the atmosphere". Emissions are higher than optimum.

Cooperative agreement

— Maximize $\sum_i J_i$ → reduce emissions to the social optimum. The "gain" from cooperation needs to be distributed through transfer payments.

Optimal execution

— Sell a large block of shares in time $T$, minimizing costs and market impact. Basic Almgren–Chriss model (2001): Minimizing expected costs plus penalty for variance. Extensions for trader competition—a game between several executors.

Formulas

Financial markets, oligopolistic competition, climate negotiations—all of these are tasks of strategic interaction in a dynamic environment. Differential games provide a rigorous mathematical apparatus for their analysis. The key advantage over static game theories is the consideration of dynamic...

Model: Two producers release $q_i(t)$ units of goods. Price depends on total output: $P(t) = a − b(q_1(t) + q_2(t))$.

Capital dynamics: $\dot{K}_i = u_i − δK_i$ (investment $u_i$, depreciation $δ$). $q_i = αK_i$.

Nash equilibrium: via Hamiltonian equations or DP. Optimal investments $u^*_i(t)$ depend on own capital and the competitor’s capital.