Hutter, Marcus2015-08-142015-08-1412 June 201938-72281532-4435http://hdl.handle.net/1885/14721Online estimation and modelling of i.i.d. data for short sequences over large or complex ''alphabets'' is a ubiquitous (sub)problem in machine learning, information theory, data compression, statistical language processing, and document analysis. The Dirichlet-Multinomial distribution (also called Polya urn scheme) and extensions thereof are widely applied for online i.i.d. estimation. Good a-priori choices for the parameters in this regime are difficult to obtain though. I derive an optimal adaptive choice for the main parameter via tight, data-dependent redundancy bounds for a related model. The 1-line recommendation is to set the 'total mass' = 'precision' = 'concentration' parameter to m/2ln[(n+1)/m], where n is the (past) sample size and m the number of different symbols observed (so far). The resulting estimator is simple, online, fast, and experimental performance is superb.© 2013 M. Hutter. Author can archive publisher’s version/PDF. http://www.sherpa.ac.uk/romeo/issn/1532-4435/ as at 14/8/15sparse codingadaptive parametersDirichlet-MultinomialPolya urndata-dependent redundancy boundsmall/large alphabetdata compression2013-062018-11-29