Topics in Dynamic programming and Economic Networks

Abstract

Dynamic programming is a recursive method for finding optimal decision rules for sequential decision-making problems under uncertainty. It is the most fundamental tool for modern economics, addressing topics that include economic growth, optimal saving, asset pricing, business cycles, and fiscal and monetary policies. Complementing this, economic networks stand as another critical tool, recognising the dynamic interactions among a multitude of diverse agents. The network models provide a comprehensive analysis of how individual agent behaviours, incentives, and strategic interactions contribute to the larger systemic forces or shocks within aggregate economics.

This thesis addresses problems in dynamic programming and static economic networks. A key focus is on dynamic programming theory, specifically integrating both state-dependent and action-dependent discounting, as opposed to conventional constant discounting. The core contribution involves establishing a comprehensive optimality theory in dynamic programming in the context of eventual discounting.

Moreover, the research expands the concept of state-action-dependent discounting in dynamic programming to model-free reinforcement learning. The thesis demonstrates the convergence of Q-learning when subjected to the condition of eventual discounting.

In addition to dynamic programming, the thesis investigates a unified static network that includes production networks, network games, and financial networks. Our results show the existence and (almost sure) uniqueness of equilibrium in both eventually contracting and non-expansive frameworks.

Finally, employing dynamic programming techniques, the thesis constructs an endogenous production network characterised by a unique equilibrium, featuring robust characteristics such as a power-law distribution of firm sizes. This multifaceted exploration contributes insights to the understanding of the existence and uniqueness of equilibrium in dynamic programming and economic networks. Show more