Boosting through optimization of margin distributions
Date
2010
Authors
Shen, Chunhua
Li, Hanxi
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers (IEEE Inc)
Abstract
Boosting has been of great interest recently in the machine learning community because of the impressive performance for classifi- cation and regression problems. The success of boosting algorithms may be interpreted in terms of the margin theory. Recently, it has been shown that generalization error of classifiers can be obtained by explicitly taking the margin distribution of the training data into account. Most of the current boosting algorithms in practice usually optimize a convex loss function and do not make use of the margin distribution. In this brief, we design a new boosting algorithm, termed margin-distribution boosting (MDBoost), which directly maximizes the average margin and minimizes the margin variance at the same time. This way the margin distribution is optimized. A totally corrective optimization algorithm based on column generation is proposed to implement MDBoost. Experiments on various data sets show that MDBoost outperforms AdaBoost and LPBoost in most cases.
Description
Keywords
Keywords: AdaBoost; Boosting algorithm; Column generation; Data sets; Generalization Error; Loss functions; Machine learning communities; Margin theory; Optimization algorithms; Regression problem; Training data; Algorithms; Optimization; Adaptive boosting; algorit AdaBoost; Boosting; Column generation; Margin distribution
Citation
Collections
Source
IEEE Transactions on Neural Networks
Type
Journal article
Book Title
Entity type
Access Statement
License Rights
Restricted until
2037-12-31
Downloads
File
Description