Cooperative Differential Games and Payoff Allocation

When Cooperation is More Profitable

Nash equilibrium describes a "no-agreement situation": every player for themselves. But in reality, players often can agree on joint strategies and share the payoff. Fishing nations agree to limit catches. Countries create climate agreements. Companies unite into consortia. Cooperative games study when cooperation is beneficial and how to fairly distribute the joint payoff.

Characteristic Function

For each subset of players $S \subseteq N$ (a coalition) is defined:

$v(S)$ = the maximum total payoff that $S$ can guarantee itself through joint actions

Superadditivity: $v(S \cup T) \geq v(S) + v(T)$ for $S \cap T = \varnothing$. If there is "synergy" from cooperation, it is advantageous to unite.

With superadditivity: $v(N) \geq \sum_i v({i})$ — it is more profitable for everyone to unite than to act individually.

Shapley Value

Question: how to fairly divide $v(N)$ among the players?

Shapley value (Shapley, 1953): $\varphi_i$ — the "fair" share of player $i$:

$ \varphi_i(v) = \sum_{S \subseteq N \setminus {i}} \frac{|S|! (N - |S| - 1)!}{N!} \cdot [v(S \cup {i}) - v(S)] $

Meaning: averaging the "marginal contribution" of $i$ when joining coalition $S$ over all possible joining orders.

Axiomatization: the unique allocation satisfying:

1. Efficiency: $\sum_i \varphi_i = v(N)$
2. Symmetry: if $i$ and $j$ are interchangeable in $v$ — $\varphi_i = \varphi_j$
3. Null player: if $v(S \cup {i}) = v(S)$ for all $S$ — $\varphi_i = 0$
4. Additivity: $\varphi_i(v+w) = \varphi_i(v) + \varphi_i(w)$

Game Core

Core — the set of payoff allocations $(x_1, \ldots, x_N)$ with:

• Efficiency: $\sum_i x_i = v(N)$
• Group rationality: $\sum_{i \in S} x_i \geq v(S)$ for all $S$

No coalition would want to "exit" the grand coalition. Stable allocation.

The core can be empty! For example, a three-player game where $v(1,2) = v(1,3) = v(2,3) = 1$, $v(1,2,3) = 1$, $v(i) = 0$. The core is empty: any allocation $(x_1, x_2, x_3)$ with $\sum x_i = 1$ violates some restriction.

The Shapley value always exists, the core — not necessarily.

Dynamic Consistency

In dynamic games, a problem arises: a coalition agreement, optimal "at the start" of the game, may turn out to be unprofitable "in the middle".

Example: countries agreed at the beginning of the game on CO₂ restrictions. After 10 years, country A discovers it is better off "leaving" the agreement. The arrangement is "unstable" — not dynamically consistent.

Dynamically consistent allocation (Yeung-Petrosyan, 2001): a payment trajectory $\beta(t)$ such that at each moment $t$ players still prefer to adhere to the agreement.

This imposes a restriction: the "payment mechanism" (IDP — Incremental Distribution Procedure) must pay each player their share so that the value of the "remaining game" always matches the agreement.

Application: Fisheries Management

Model: $N$ countries fish in a shared ocean. Fish biomass: $\dot{x} = r x(1 - x/K) - \sum_i u_i$ (logistic growth minus total catch).

Payoff for $i$: $J_i = \int_0^\infty e^{-\rho t} \left( u_i - c u_i^2 / (2x) \right) dt$ (profit from catch accounting for costs).

Nash (non-cooperative): each country overfishes $\rightarrow$ "tragedy of the commons" (Hardin). Resource is depleted.

Cooperative solution: maximize $\sum_i J_i \rightarrow$ lower total catch $\rightarrow$ higher total payoff! The Shapley value distributes it fairly.

Dynamic consistency: IDP ensures that each country receives payments such that "leaving" the agreement is never profitable.

Full Example: Three-player Model

Problem: $v({1}) = 2$, $v({2}) = 3$, $v({3}) = 4$, $v({1,2}) = 7$, $v({1,3}) = 8$, $v({2,3}) = 9$, $v({1,2,3}) = 12$.

Shapley value:

$\varphi_1$: consider the orders: (1,2,3): contribution = $2-0=2$; (1,3,2): contribution = $2$; (2,1,3): contribution = $7-3=4$; (2,3,1): contribution = $12-9=3$; (3,1,2): contribution = $8-4=4$; (3,2,1): contribution = $12-9=3$. Average: $(2+2+4+3+4+3)/6 = 18/6 = 3$.

$\varphi_2$: similarly = $(5+3+3+5+3+3)/6 = 22/6 \approx 3.67$.

$\varphi_3$: $12 - 3 - 3.67 = 5.33$ (since $\sum \varphi_i = v(N) = 12$).

Core: seeking $(x_1, x_2, x_3)$ with $\sum = 12$, $x_1 \geq 2$, $x_2 \geq 3$, $x_3 \geq 4$, $x_1 + x_2 \geq 7$, $x_1 + x_3 \geq 8$, $x_2 + x_3 \geq 9$. For example, (3, 4, 5): check all constraints: $3 \geq 2,✓$, $4 \geq 3,✓$, $5 \geq 4,✓$, $7 \geq 7,✓$, $8 \geq 8,✓$, $9 \geq 9,✓$. The core is nonempty!

Cooperative Games: Coalition Formation

In cooperative theory, players can conclude binding agreements. The central concept is the characteristic function $v(S)$, which gives the value of a coalition $S \subseteq N$. For differential games, $v(S)$ is the value of the game when $S$ members cooperate and $N \setminus S$ play against them (or solve their own subproblem).

Properties of the characteristic function:

• Superadditivity: $v(S \cup T) \geq v(S) + v(T)$ for disjoint $S, T$ — cooperation is more profitable
• Convexity: $v(S \cup T) + v(S \cap T) \geq v(S) + v(T)$ — guarantees stability

Payoff Concepts

• Core: the set of allocations where no coalition can improve its position by leaving the agreement
• Shapley vector: $\varphi_i = (1/n!) \sum_\pi [v(S_{\pi,i} \cup {i}) - v(S_{\pi,i})]$ — average contribution of player $i$ across all joining orders
• Nucleolus (Schmeidler): minimizes the "dissatisfaction" of the least satisfied coalition
• $\tau$-value (Tijs): a compromise between minimal rights and maximal claims

Dynamic Stability

A critical problem of cooperative differential games is time inconsistency: an allocation optimal at the start can cease to be optimal partway through the game. Players dissatisfied with the current arrangement may exit the agreement. The solution is the Petrosyan-Zenkevich construction: dynamic allocations ensuring stability at any time.

Applications

Cooperative differential games are applied in common resource management (fishing, water resources), climate agreements (Kyoto, Paris), supply chain management, patent pools, infrastructure unification (shared communication networks).