Consistency of Feature Markov Processes
Date
2010-10
Authors
Sunehag, Peter
Hutter, Marcus
Journal Title
Journal ISSN
Volume Title
Publisher
Springer Verlag
Abstract
We are studying long term sequence prediction (forecasting).
We approach this by investigating criteria for choosing a compact
useful state representation. The state is supposed to summarize useful
information from the history. We want a method that is asymptotically
consistent in the sense it will provably eventually only choose between
alternatives that satisfy an optimality property related to the used criterion.
We extend our work to the case where there is side information that
one can take advantage of and, furthermore, we briefly discuss the active
setting where an agent takes actions to achieve desirable outcomes.
Description
Keywords
Markov Process (MP), Hidden Markov Model (HMM), Finite State Machine (FSM), Probabilistic Deterministic Finite State Automata (PDFA), Penalized Maximum Likelihood (PML), ergodicity, asymptotic consistency, suffix trees, model selection, reinforcement learning
Citation
Collections
Source
Type
Conference paper
Book Title
Entity type
Access Statement
Open Access