Sparse adaptive dirichlet-multinomial-like processes
| dc.contributor.author | Hutter, Marcus | en |
| dc.date.accessioned | 2025-12-31T18:41:37Z | |
| dc.date.available | 2025-12-31T18:41:37Z | |
| dc.date.issued | 2013 | en |
| dc.description.abstract | Online estimation and modelling of i.i.d. data for short sequences over large or complex "alphabets" is a ubiquitous (sub)problem in machine learning, information theory, data compression, statistical language processing, and document analysis. The Dirichlet-Multinomial distribution (also called Polya urn scheme) and extensions there of are widely applied for online i.i.d. estimation. Good a-priori choices for the parameters in this regime are difficult to obtain though. I derive an optimal adaptive choice for the main parameter via tight, data-dependent redundancy bounds for a related model. The 1-line recommendation is to set the 'total mass' = 'precision' = 'concentration' parameter to m/[2 ln n+1/m], where n is the (past) sample size and m the number of different symbols observed (so far). The resulting estimator is simple, online, fast, and experimental performance is superb. | en |
| dc.description.status | Peer-reviewed | en |
| dc.format.extent | 28 | en |
| dc.identifier.issn | 1532-4435 | en |
| dc.identifier.scopus | 84898021490 | en |
| dc.identifier.uri | https://hdl.handle.net/1885/733797761 | |
| dc.language.iso | en | en |
| dc.relation.ispartofseries | 26th Conference on Learning Theory, COLT 2013 | en |
| dc.source | Journal of Machine Learning Research | en |
| dc.subject | Adaptive parameters | en |
| dc.subject | Data compression | en |
| dc.subject | Data-dependent redundancy bound | en |
| dc.subject | Dirichlet-Multinomial | en |
| dc.subject | Polya urn | en |
| dc.subject | Small/large alphabet | en |
| dc.subject | Sparse coding | en |
| dc.title | Sparse adaptive dirichlet-multinomial-like processes | en |
| dc.type | Conference paper | en |
| dspace.entity.type | Publication | en |
| local.bibliographicCitation.lastpage | 459 | en |
| local.bibliographicCitation.startpage | 432 | en |
| local.contributor.affiliation | Hutter, Marcus; School of Computing, ANU College of Systems and Society, The Australian National University | en |
| local.identifier.ariespublication | u4334215xPUB1166 | en |
| local.identifier.citationvolume | 30 | en |
| local.identifier.pure | 9c45ccef-8abd-4c99-9f9d-09e9fc8d0549 | en |
| local.identifier.url | https://www.scopus.com/pages/publications/84898021490 | en |
| local.type.status | Published | en |