Outlier Robust Model Selection in Linear Regression

Abstract

We propose a new approach to the selection of regression models based on combining a robust penalized criterion and a robust conditional expected prediction loss function that is estimated using a stratified bootstrap. Both components of the procedure use robust criteria (i.e., robust p-functions) rather than squared error loss to reduce the effects of large residuals and poor bootstrap samples. A key idea is to separate estimation from model selection by choosing estimators separately from the p-function. Using the stratified bootstrap further reduces the likelihood of obtaining poor bootstrap samples. We show that the model selection procedure is consistent under some conditions and works well in our simulations. In particular, we find that simultaneous minimization of prediction error and conditional expected prediction loss is better than separate minimization of the prediction error or the conditional expected prediction loss.