文档介绍：Math Review for . Students
ECO 1011H (Fall, 2002)
by Shouyong Shi
Lecture IV: Dynamic Programming
Reading materials:
. Dixit (2nd edition), chapter 11.
Stokey, Lucas with Prescott, -55, chapters 4 and 5.
1. Examples
(1) Solve a two-period problem with the Lagrangian method:
8
<> max ln c1 + ¯ ln c2
(E1) . c + k Ak®, t = 1; 2
> t t+1 t
: k = k¹ given·(and k 0).
1 1 3 ¸
The Lagrangian:
²
®
L = ln c1 + ¸1 [Ak1 c1 k2]
+¯ [ln c + ¸ (Ak¡® c¡ k )] :
2 2 2 ¡ 2 ¡ 3
The ¯rst-order conditions are:
²
for c : 1 = ¸
1 c1 1
® 1
for k2: ¸1 = ¯¸2A®k2 ¡
for c : 1 = ¸
2 c2 2
for k3: ¯¸2 0, \=" if k3 > 0:
for ¸ : c¡ + k· Ak®, \=" if ¸ > 0
1 1 2 · 1 1
for ¸ : c + k Ak®, \=" if ¸ > 0.
2 2 3 · 2 2
Because c > 0 and c > 0, the conditions for c and c imply ¸ > 0 and ¸ > 0. So,
² 1 2 1 2 1 2
k3 = 0 and the budget constraints hold with equality.
The solutions for (c ; c ; k ; k ) are
² 1 2 2 3
1 ® ¯® ®
c1 = 1+¯®Ak1 , k2 = 1+¯® Ak1 ,
³ ´®
® ¯® ®
c2 = Ak2 = A 1+¯® Ak1 ; k3 = 0:
Notice that these are the optimal plan, made at time 1 for the two periods
²
(2) A conceptually di®erent problem:
Suppose that the optimal plan made at time 1 is (c¤; c¤; k¤; k¤) (as the above).
² 1 2 2 3
Now period 1 has passed, and the agent is at the beginning of period 2, with a given
² ¹
capital stock k2 that may or may not be the same as the originally planned level (k2¤).
The agent faces the following problem:
¹®
max ln c2 : c2 + k3 Ak2 ; k3 0 .
(c2;k3)f · ¸ g
The solution is given as
^ ¹®
k3 = 0 and c^2 = Ak2 :
Note:
²
a. c^ = c¤ if k¹ = k¤.
2 6 2 2 6 2
b. This procedure requires the choices in period 2 to be optimal for any given capital
stock at the beginning of period 2. (No regret.)
c. Formally, the choices in period 2 are functions of the \state variable" in period 2
{ the capital stock at the beginning of period 2. These functions are called policy
functions. Suppressing the