Probabilistic Visual Concept Trees




Xie, Lexing
Yan, Rong
Tesic, Jelena
Natsev, Apostol
Smith, John R

Journal Title

Journal ISSN

Volume Title


Association for Computing Machinery Inc (ACM)


This paper presents probabilistic visual concept trees, a model for large visual semantic taxonomy structures and its use in visual concept detection. Organizing visual semantic knowledge systematically is one of the key challenges towards large-scale concept detection, and one that is complementary to optimizing visual classification for individual concepts. Semantic concepts have traditionally been treated as isolated nodes, a densely-connected web, or a tree. Our analysis shows that none of these models are sufficient in modeling the typical relationships on a real-world visual taxonomy, and these relationships belong to three broad categories - semantic, appearance and statistics. We propose probabilistic visual concept trees for modeling a taxonomy forest with observation uncertainty. As a Bayesian network with parameter constraints, this model is flexible enough to account for the key assumptions in all three types of taxonomy relations, yet it is robust enough to accommodate expansion or deletion in a taxonomy. Our evaluation results on a large web image dataset show that the classification accuracy has considerably improved upon baselines without, or with only a subset of concept relationships



Keywords: Bayes networks; Belong to; Classification accuracy; Concept detection; Evaluation results; Isolated nodes; Parameter constraints; probalistic classification; Real-world; Semantic concept; Visual classification; Visual concept; Visual semantics; visual tax bayes networks; probalistic classification; visual taxonomy



Proceedings ACM Multimedia 2010


Conference paper

Book Title

Entity type

Access Statement

License Rights



Restricted until