Pattern Trees: An effective Machine Learning Approach

Abstract

Fuzzy classification is one of the most important applications of fuzzy logic. Its goal is to find a set of fuzzy rules which describe classification problems. Most of the existing fuzzy rule induction methods (e.g., the fuzzy decision trees induction method) focus on searching rules consisting of t-norms (i.e., AND) only, but not t-conorms (OR) explicitly. This may lead to the omission of generating important rules which involve t-conorms explicitly. This paper proposes a type of tree termed pattern trees which make use of different aggregations including both t-norms and t-conorms. Like decision trees, pattern trees are an effective machine learning tool for classification applications. This paper discusses the difference between decision trees and pattern trees, and also shows that the subsethood based method (SBM) and the weighted subsethood based method (WSBM) are two specific cases of pattern trees, with each having a fixed pattern tree structure. A novel pattern tree induction method is proposed. The comparison to other classification methods including SBM, WSBM and fuzzy decision tree induction over datasets obtained from UCI dataset repository shows that pattern trees can obtain higher accuracy rates in classifications. In addition, pattern trees are capable of generating classifiers with good generality, while decision trees can easily fall into the trap of over-fitting. According to two different configurations, simple pattern trees and pattern trees have been distinguished. The former not only produce high prediction accuracy, but also preserve compact tree structures, while the latter can produce even better accuracy, but as a compromise produce more complex tree structures. Subject to the particular demands (comprehensibility or performance), simple pattern trees and pattern trees provide an effective methodology for real world applications. Weighted pattern trees have been proposed in which certain weights are assigned to different trees, to reflect the nature that different trees may have different confidences. The experiments on British Telecom (BT) customer satisfaction dataset show that weighted pattern trees can slightly outperform pattern trees, and both are slightly better than fuzzy decision trees in terms of prediction accuracy. In addition, the experiments show that both weighted and unweighted pattern trees are robust to over-fitting. Finally, a limitation of pattern trees as revealed via BT dataset analysis is discussed and the research direction is outlined.