Closed-form Solutions to Sequential Decision Making within Markets

Abstract

Sequential decision making is a pervasive and inescapable requirement of every day life. Deciding upon which sequence of actions to take is complicated by incomplete information about the environment, the effects of each decision upon the future state of the environment, ill-defined objectives and our own cognitive limitations. These challenges are exacerbated in financial markets which are in a constant state of flux, with prices adjusting to new information, winning traders replacing losing traders and the introduction of new technologies. Decision theoretic planning provides powerful and flexible frameworks for many sequential decision making problems within financial markets. Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs) are two such frameworks, which can model financial decision making problems with either or both discrete and continuous variables. Traditional methods to solve MDPs and POMDPs use Monte-Carlo based methods or discretise the continuous state space. This is often to the detriment of the quality of the solution, which in many financial domains, such as those involving asset prices, are required to be exact and closed-form. Closed-form solutions, by virtue of being declarative and symbolic, allow the decision maker to undertake further analysis and optimisation. In this thesis we present novel techniques to calculate exact and closed-form solutions to new classes of MDPs and POMDPs by leveraging Symbolic Dynamic Programming (SDP). Our specific contributions include: (i) closed-form solutions to a subclass of Continuous Stochastic Games, which we use to encapsulate and solve a zero-sum game between a Binary Option trader and an adversarial market; (ii) closed-form solutions to Sequential Market Making with Inventory, which extends seminal works from market microstructure theory and algorithmic market-making; and (iii) Analytic Decision Analysis for Parameterised Hybrid MDPs, which we use to examine the sensitivity of a model for optimal trans- action execution to its parameters. Our experimental evaluations confirm the efficacy of our novel solutions for a range of new classes of MDPs and POMDPs. Show more