Semantic Hierarchical Document Signature For Determining Sentence Similarity
Date
2010
Authors
Manna, Sukanya
Gedeon, Tamas (Tom)
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers (IEEE Inc)
Abstract
In this paper, we present a new approach that incorporates semantic information from a document, in the form of Hierarchical Document Signature (HDS), to measure semantic similarity between sentences. Due to variability of expressions of natural language, it is very essential to exploit the semantic properties of a document to accurately identify semantically similar sentences since sentences conveying the same fact or concept may be composed lexically and syntactically different. Inversely, sentences which are lexically common may not necessarily convey the same meaning. This poses a significant impact on many text mining applications performance where sentence-level judgment is involved. Our HDS uses the natural hierarchy of the document and represents it in a modularized form of document level to sentence level, sentence to word level; aggregating similarity components at the lower levels and propagating them to the next higher level to produce the final similarity between sentences. The evaluation of our HDS model has shown that it resembles the decision making process as done by human to a greater extent than different vector space models which only uses 'bag of words' concept.
Description
Keywords
Keywords: Bag of words; Decision making process; Hierarchical document; Modularized; Natural languages; New approaches; Semantic information; Semantic properties; Semantic similarity; Sentence level; Sentence similarity; Significant impacts; Text mining; Vector spa
Citation
Collections
Source
Proceedings of the 19th international conference on Fuzzy Systems
Type
Conference paper