Tensor Term Indexing: An application of HOSVD for Document Summarization

Abstract

In this paper, a new method for text summarization is proposed by using an extended version of the Tensor Term Importance (TTI) model. This method summarizes documents by extracting important sentences from a document. It improves the per document summarization efficiency by incorporating additional information of the whole document set referring to the same topic (or coherent documents). The basic idea of this approach is to represent the whole document set in a uniform form, in the term-sentence-document tensor, and to use higher-order singular value decomposition (HOSVD) to highlight the important terms in each document. Here, we present two different methods of summarization. In the first method, the sentences having the highly weighted terms are extracted as the important sentences representing the document. The important sentences identified by selecting those that contains more from the important terms. The second model uses a so-called super sentence and uses that to extract other sentences having high similarity with it. Unlike in Latent Semantic Analysis (LSA) where SVD is applied for compressing the sparse term-document matrix and defining latent semantic links between terms, in TTI SVD is used to reduce noise and to highlight the important term-document relations in the document. Our evaluation results show that our TTI based methods are more similar to human generated summaries than other automated summarizers which work on single documents at a time. Show more