Modeling Topic Hierarchies with the Recursive Chinese Restaurant Process

Date

2012

Authors

Kim, Joon Hee
Kim, Dongwoo
Kim, Suin
Oh, Alice

Journal Title

Journal ISSN

Volume Title

Publisher

ACM

Abstract

Topic models such as latent Dirichlet allocation (LDA) and hierarchical Dirichlet processes (HDP) are simple solutions to discover topics from a set of unannotated documents. While they are simple and popular, a major shortcoming of LDA and HDP is that they do not organize the topics into a hierarchical structure which is naturally found in many datasets. We introduce the recursive Chinese restaurant process (rCRP) and a nonparametric topic model with rCRP as a prior for discovering a hierarchical topic structure with unbounded depth and width. Unlike previous models for discovering topic hierarchies, rCRP allows the documents to be generated from a mixture over the entire set of topics in the hierarchy. We apply rCRP to a corpus of New York Times articles, a dataset of MovieLens ratings, and a set of Wikipedia articles and show the discovered topic hierarchies. We compare the predictive power of rCRP with LDA, HDP, and nested Chinese restaurant process (nCRP) using heldout likelihood to show that rCRP outperforms the others. We suggest two metrics that quantify the characteristics of a topic hierarchy to compare the discovered topic hierarchies of rCRP and nCRP. The results show that rCRP discovers a hierarchy in which the topics become more specialized toward the leaves, and topics in the immediate family exhibit more affinity than topics beyond the immediate family.

Description

Keywords

Hierarchical Topic Modeling, Bayesian Nonparametric models

Citation

Source

Proceedings of the 21st ACM International Conference on Information and Knowledge Management

Type

Conference paper

Book Title

Entity type

Access Statement

License Rights

DOI

10.1145/2396761.2396861

Restricted until

2099-12-31