MDL convergence speed for Bernoulli sequences
-
Altmetric Citations
Description
The Minimum Description Length principle for online sequence estimation/prediction in a proper learning setup is studied. If the underlying model class is discrete, then the total expected square loss is a particularly interesting performance measure: (a) this quantity is finitely bounded, implying convergence with probability one, and (b) it additionally specifies the convergence speed. For MDL, in general one can only have loss bounds which are finite but exponentially larger than those for...[Show more]
dc.contributor.author | Poland, Jan | |
---|---|---|
dc.contributor.author | Hutter, Marcus | |
dc.date.accessioned | 2015-08-31T02:19:17Z | |
dc.date.available | 2015-08-31T02:19:17Z | |
dc.identifier.issn | 0960-3174 | |
dc.identifier.uri | http://hdl.handle.net/1885/15030 | |
dc.description.abstract | The Minimum Description Length principle for online sequence estimation/prediction in a proper learning setup is studied. If the underlying model class is discrete, then the total expected square loss is a particularly interesting performance measure: (a) this quantity is finitely bounded, implying convergence with probability one, and (b) it additionally specifies the convergence speed. For MDL, in general one can only have loss bounds which are finite but exponentially larger than those for Bayes mixtures. We show that this is even the case if the model class contains only Bernoulli distributions. We derive a new upper bound on the prediction error for countable Bernoulli classes. This implies a small bound (comparable to the one for Bayes mixtures) for certain important model classes. We discuss the application to Machine Learning tasks such as classification and hypothesis testing, and generalization to countable classes of i.i.d. models. | |
dc.description.sponsorship | This work was supported by SNF grant 2100-67712.02. | |
dc.publisher | Springer Verlag | |
dc.rights | © Springer Science + Business Media, LLC 2006 http://www.sherpa.ac.uk/romeo/issn/0960-3174/..."Author's post-print on any open access repository after 12 months after publication" from SHERPA/RoMEO site (as at 31/08/15). | |
dc.source | Statistics and Computing | |
dc.title | MDL convergence speed for Bernoulli sequences | |
dc.type | Journal article | |
local.identifier.citationvolume | 16 | |
dc.date.issued | 2006-06 | |
local.publisher.url | http://link.springer.com/ | |
local.type.status | Accepted Version | |
local.contributor.affiliation | Hutter, M., Research School of Computer Science, The Australian National University | |
local.bibliographicCitation.issue | 2 | |
local.bibliographicCitation.startpage | 161 | |
local.bibliographicCitation.lastpage | 175 | |
local.identifier.doi | 10.1007/s11222-006-6746-3 | |
dcterms.accessRights | Open Access | |
Collections | ANU Research Publications |
Download
File | Description | Size | Format | Image |
---|---|---|---|---|
Poland and Hutter MDL Convergence Speed 2006.pdf | 308.92 kB | Adobe PDF |
Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.
Updated: 17 November 2022/ Responsible Officer: University Librarian/ Page Contact: Library Systems & Web Coordinator