Continuation Value Methods for Sequential Decisions: A General Theory

Abstract

After the introductory chapter, this thesis comprises four main chapters before concluding in chapter 6. The thesis undertakes a systematic analysis of the con- tinuation value based method for sequential decision problems originally due to Jovanovic (1982). Although recently this technique is widely employed in a va- riety of economic applications, its theoretical connections to the traditional value function based method, relative efficiency, and optimality/analytical properties have hitherto received no general investigation. The thesis fills this gap.

On the one hand, the thesis shows that the operator employed by this method (referred to below as the Jovanovic operator) is semiconjugate to the traditional Bellman operator and has essentially equivalent dynamic properties. In particu- lar, under general assumptions, any fixed point of one of the operators is a direct translation of a fixed point of the other. Iterative sequences generated by the operators are also simple translations. After adding topological structure to the generic setting, the thesis shows that the Bellman and Jovanovic operators are both contraction mappings under identical assumptions, and that convergence to the respective fixed points occurs at the same rate.

To ensure sufficient generality for economic applications, the optimality and sym- metry analysis has been embedded separately in (a) spaces of potentially un- bounded functions endowed with generic weighted supremum norm distances, and (b) spaces of integrable functions with divergence measured by Lp norms. Unbounded rewards are allowed provided that they do not cause continuation values to diverge. Moreover, the theory mentioned above is established for im- portant classes of sequential decision problems, including:

• standard optimal stopping problems (chapter 2), • repeated optimal stopping problems (chapter 3), and • dynamic discrete choice problems (chapter 4).

On the other hand, despite these similarities, the thesis shows that there do re- main important differences between the continuation value based method and the traditional value function based method in terms of efficiency and analytical convenience.

One of these differences concerns the dimensionality of the effective state spaces associated with the Bellman and Jovanovic operators. First, aside from a class of problems for which the continuation dynamics are trivial, the effective state space of the continuation value function is never larger than that of the value function. Second, for a broad class of sequential problems, the effective state space of the continuation value function is strictly lower dimensional than that of the value function. Another key difference is that continuation value functions are typi- cally smoother than value functions. The relative smoothness comes from taking expectations over stochastic transitions. In each scenario, it is highly advanta- geous to work with the continuation value method rather than the traditional value function method.

The thesis systematically characterizes these hidden advantages in terms of model primitives and provides a range of important applications (chapters 2 and 5). Moreover, by exploiting these advantages, the thesis develops a general theory for sequential decision problems based around continuation values and obtains a range of new results on optimality, optimal behavior and efficient computation (chapter 5). Show more