Skip navigation
Skip navigation

Sampling Table Configurations for the Hierarchical Poisson-Dirichlet Process

Chen, Changyou; Du, Lan; Buntine, Wray

Description

Hierarchical modeling and reasoning are fundamental in machine intelligence, and for this the two-parameter Poisson-Dirichlet Process (PDP) plays an important role. The most popular MCMC sampling algorithm for the hierarchical PDP and hierarchical Dirichlet Process is to conduct an incremental sampling based on the Chinese restaurant metaphor, which originates from the Chinese restaurant process (CRP). In this paper, with the same metaphor, we propose a new table representation for the...[Show more]

dc.contributor.authorChen, Changyou
dc.contributor.authorDu, Lan
dc.contributor.authorBuntine, Wray
dc.coverage.spatialAthens Greece
dc.date.accessioned2015-12-08T22:38:48Z
dc.date.createdSeptember 5-9 2011
dc.identifier.isbn9783642237799
dc.identifier.urihttp://hdl.handle.net/1885/35957
dc.description.abstractHierarchical modeling and reasoning are fundamental in machine intelligence, and for this the two-parameter Poisson-Dirichlet Process (PDP) plays an important role. The most popular MCMC sampling algorithm for the hierarchical PDP and hierarchical Dirichlet Process is to conduct an incremental sampling based on the Chinese restaurant metaphor, which originates from the Chinese restaurant process (CRP). In this paper, with the same metaphor, we propose a new table representation for the hierarchical PDPs by introducing an auxiliary latent variable, called table indicator, to record which customer takes responsibility for starting a new table. In this way, the new representation allows full exchangeability that is an essential condition for a correct Gibbs sampling algorithm. Based on this representation, we develop a block Gibbs sampling algorithm, which can jointly sample the data item and its table contribution. We test this out on the hierarchical Dirichlet process variant of latent Dirichlet allocation (HDP-LDA) developed by Teh, Jordan, Beal and Blei. Experiment results show that the proposed algorithm outperforms their "posterior sampling by direct assignment" algorithm in both out-of-sample perplexity and convergence speed. The representation can be used with many other hierarchical PDP models.
dc.publisherSpringer
dc.relation.ispartofseriesMachine Learning and Knowledge Discovery in Databases European Conference (ECML PKDD 2011)
dc.sourceProceedings of Machine Learning and Knowledge Discovery in Databases - European Conference (ECML PKDD 2011)
dc.subjectKeywords: Convergence speed; Data items; Dirichlet process; Gibbs samplers; Gibbs sampling; HDP-LDA; Hierarchical Dirichlet process; Hierarchical modeling; Hierarchical Poisson-Dirichlet Processes; Latent Dirichlet allocation; Latent variable; Machine intelligence; block Gibbs sampler; Dirichlet Processes; HDP-LDA; Hierarchical Poisson-Dirichlet Processes
dc.titleSampling Table Configurations for the Hierarchical Poisson-Dirichlet Process
dc.typeConference paper
local.description.notesImported from ARIES
local.description.refereedYes
dc.date.issued2011
local.identifier.absfor010405 - Statistical Theory
local.identifier.ariespublicationu4963866xPUB130
local.type.statusPublished Version
local.contributor.affiliationChen, Changyou, College of Engineering and Computer Science, ANU
local.contributor.affiliationDu, Lan, College of Engineering and Computer Science, ANU
local.contributor.affiliationBuntine, Wray, College of Engineering and Computer Science, ANU
local.description.embargo2037-12-31
local.bibliographicCitation.startpage296
local.bibliographicCitation.lastpage311
local.identifier.doi10.1007/978-3-642-23780-5_29
local.identifier.absseo890299 - Computer Software and Services not elsewhere classified
dc.date.updated2016-02-24T11:29:51Z
local.identifier.scopusID2-s2.0-80052420115
CollectionsANU Research Publications

Download

File Description SizeFormat Image
01_Chen_Sampling_Table_Configurations_2011.pdf327.61 kBAdobe PDF    Request a copy


Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  19 May 2020/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator