Skip navigation
Skip navigation

Bayesian Treatment of Incomplete Discrete Data applied to Mutual Information and Feature Selection

Hutter, Marcus; Zaffalon, Marco

Description

Given the joint chances of a pair of random variables one can compute quantities of interest, like the mutual information. The Bayesian treatment of unknown chances involves computing, from a second order prior distribution and the data likelihood, a posterior distribution of the chances. A common treatment of incomplete data is to assume ignorability and determine the chances by the expectation maximization (EM) algorithm. The two different methods above are well established but typically...[Show more]

dc.contributor.authorHutter, Marcus
dc.contributor.authorZaffalon, Marco
dc.coverage.spatialHamburg Germany
dc.date.accessioned2015-12-08T22:16:47Z
dc.date.createdSeptember 15-18 2003
dc.identifier.isbn3540001689
dc.identifier.urihttp://hdl.handle.net/1885/30828
dc.description.abstractGiven the joint chances of a pair of random variables one can compute quantities of interest, like the mutual information. The Bayesian treatment of unknown chances involves computing, from a second order prior distribution and the data likelihood, a posterior distribution of the chances. A common treatment of incomplete data is to assume ignorability and determine the chances by the expectation maximization (EM) algorithm. The two different methods above are well established but typically separated. This paper joins the two approaches in the case of Dirichlet priors, and derives efficient approximations for the mean, mode and the (co)variance of the chances and the mutual information. Furthermore, we prove the unimodality of the posterior distribution, whence the important property of convergence of EM to the global maximum in the chosen framework. These results are applied to the problem of selecting features for incremental learning and naive Bayes classification. A fast filter based on the distribution of mutual information is shown to outperform the traditional filter based on empirical mutual information on a number of incomplete real data sets.
dc.publisherSpringer
dc.relation.ispartofseriesGerman Conference on Artificial Intelligence (KI 2003)
dc.rightsCopyright Information: © Springer-Verlag Berlin Heidelberg 2003. http://www.sherpa.ac.uk/romeo/issn/0302-9743/..."Author's post-print on any open access repository after 12 months after publication" from SHERPA/RoMEO site (as at 2/09/15)
dc.sourceProceedings of the 26th German Conference on Artificial Intelligence (KI-2003)
dc.source.urihttp://www.hitec-hh.de/ki2003/
dc.titleBayesian Treatment of Incomplete Discrete Data applied to Mutual Information and Feature Selection
dc.typeConference paper
local.description.notesImported from ARIES
local.description.refereedYes
dc.date.issued2003
local.identifier.absfor080109 - Pattern Recognition and Data Mining
local.identifier.ariespublicationu4708487xPUB77
local.type.statusPublished Version
local.contributor.affiliationHutter, Marcus, College of Engineering and Computer Science, ANU
local.contributor.affiliationZaffalon, Marco, IDSIA-Istituto Dalle Molle di Studi sull Intelligenza Artificiale
local.description.embargo2037-12-31
local.bibliographicCitation.startpage396
local.bibliographicCitation.lastpage406
local.identifier.doi10.1007/b13477
dc.date.updated2016-02-24T11:20:49Z
local.identifier.scopusID2-s2.0-9444271605
CollectionsANU Research Publications

Download

File Description SizeFormat Image
01_Hutter_Bayesian_Treatment_of_2003.pdf698.86 kBAdobe PDF    Request a copy


Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  19 May 2020/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator