DP in Economics: Accumulation, Resources, Growth

Dynamic programming is the standard language of modern macroeconomics and business strategy. Ramsey, Stokey-Lucas, RBC, Bewley, Aiyagari models—all are formulated through a recursive problem. Numerical methods (value function iteration, policy function iteration, Endogenous Grid Method) allow for quantitative predictions, testable on data. Without DP one cannot imagine either the modern Sargent-Ljungqvist textbook, or articles by Nobel laureates Prescott, Sargent, Lucas.

Basic Capital Accumulation Model (Stochastic Ramsey)

Recursive problem: V(k) = max_{c ∈ [0, f(k)]} [u(c) + β·V(f(k) − c + k·(1 − δ))],

where k is capital, c is consumption, f(k) = k^α is production (Cobb-Douglas function), δ is the depreciation rate, β = 1/(1 + ρ) is the discount factor (ρ is the rate of time preference), u(c) = c^{1−σ}/(1 − σ) is the utility function with elasticity of substitution 1/σ.

Value Function Iteration method (VFI):

1. Discretize k onto a grid {k_1, ..., k_N}, for example N = 500 points.
2. Initialization: V_0(k_i) = 0 for all i.
3. In iteration n+1: V_{n+1}(k_i) = max_{c} [u(c) + β·V_n(k')], where k' = f(k_i) − c + k_i·(1 − δ). Interpolate V_n between grid points (linearly).
4. Repeat until ||V_{n+1} − V_n||_∞ < ε.

Convergence: β-contraction. ||V_{n+1} − V_n||∞ ≤ β·||V_n − V{n−1}||_∞. Number of iterations for accuracy ε: n ≈ log(ε(1−β)/||V_0 − V*||) / log(β). For β = 0.96, ε = 10⁻⁶ — ~300 iterations.

Policy Function Iteration (PFI). Alternative to VFI:

1. Initial policy g₀(k) (for example, 10% of income—consumption).
2. Policy evaluation: solve the linear system for V^g (fixed g).
3. Policy improvement: g_{n+1}(k) = argmax_c [u(c) + β·V^{g_n}(k')].
4. Final convergence in a finite number of iterations (faster than VFI).

Endogenous Grid Method (EGM, Carroll 2006). The fastest: instead of a grid on k, create a grid on future capital k', compute consumption via the FOC and reconstruct k. Acceleration by 10-100 times.

Numerical Example: Ramsey Model

Parameters: α = 0.36, β = 0.96, δ = 0.10, σ = 1 (logarithmic utility u = ln c).

Steady state from FOC: f'(k*) = 1/β − 1 + δ → α·k*^{α−1} = 1/0.96 − 1 + 0.1 → k* = (α·β/(1 − β·(1 − δ)))^{1/(1−α)} ≈ 3.16. c* = f(k*) − δ·k* ≈ 1.10.

Numerically, VFI (Python, N = 500, ε = 10⁻⁶): convergence in ~280 iterations. Policy c*(k) is a smooth increasing function, for k > k* consumption exceeds production, for k < k*—vice versa, accumulation occurs.

Exhaustible Resource Problem (Hotelling Rule)

Problem: max Σ_{t=0}^T β^t·u(q_t) with s_{t+1} = s_t − q_t, s_0 = S, s_t ≥ 0.

Stock s is a non-renewable resource (oil, gas). Extraction q_t = consumption.

Solution through DP. V(s) = max_{q ≤ s} [u(q) + β·V(s − q)]. For u(c) = ln c, trial form: V(s) = A + B·ln(s).

Substituting: A + B·ln(s) = max_q [ln(q) + β·(A + B·ln(s − q))]. FOC by q: 1/q = β·B/(s − q) → q = s/(1 + β·B). Substituting back and comparing coefficients at ln(s): B = 1/(1 − β). Hence q*(s) = s·(1 − β).

Hotelling Rule. The marginal value of the resource (royalty) grows at rate 1/β − 1 = ρ. This explains the long-term rise in raw material prices: the owner of the subsoil holds the resource until it becomes sufficiently valuable.

Empirically the rule holds for silver, copper (after adjusting for technology), but poorly for oil (the discovery of new deposits and OPEC distort the picture).

Neoclassical Growth Model: Quantitative Analysis (RBC)

Kydland and Prescott (1982) added productivity shocks z_t to Ramsey: f(k_t, z_t) = z_t·k_t^α·n_t^{1−α}, plus endogenous labor n.

Calibration: α = 0.36, β = 0.99 (quarterly), δ = 0.025, σ = 1, shock z_{t+1} = ρ·z_t + ε_t, ρ = 0.95, σ_ε = 0.007.

Solution via VFI or linearization around steady state.

Predictions vs. Data (USA 1947-2007):

• Log-GDP dispersion: model 1.7%, data 1.7%. ✓
• Investment dispersion / consumption dispersion: ~10, data ~6. Close.
• Output-labor correlation: 0.97, data 0.86. Overestimated.

"Success" of RBC: the model reproduces the first and second moments of the business cycle from a single source—productivity shocks. This generated massive literature (DSGE, New Keynesian models).

Real Applications

• Central banks. The Fed, ECB, Bank of Russia use DSGE models (medium-scale—30-100 variables), solved via VFI or linearization, for forecasting and policy evaluation (Smets-Wouters, FRB/US, R-Quest).
• Insurance companies and pension funds. Long-term asset-liability management models—a stochastic DP with economic scenarios. Analogous to the Merton problem, over 20-50 years.
• Corporate finance. Capital investment decisions with irreversibility—the optimal stopping problem, a classical DP example (Dixit-Pindyck "Investment Under Uncertainty").
• Tax policy. Optimal taxation in Aiyagari models with heterogeneous agents—the gigantic DP with wealth distribution, solved by approximate methods (perturbation, EGM).

Assignment. Numerically solve the basic Ramsey problem: u(c) = ln c, f(k) = k^0.36, β = 0.96, δ = 0.10. Grid k ∈ [0.05, 5] with N = 500 points. (a) Implement VFI to ε = 10⁻⁶, record number of iterations. (b) Plot the value function V(k). (c) Find the consumption policy c*(k), compare with the analytical solution for log-utility (exact known: c*(k) = (1 − αβ)·f(k)). (d) Simulate 100 periods from k_0 = 1: how does it converge to steady state k*? (e) Experiment: change β = 0.90—how do k* and convergence speed change?