Robust estimation : limit theorems and their applications

Abstract

This thesis is concerned with the asymptotic theory of general M-estimators and some minimal distance estimators. Particular attention is paid to uniform convergence theory which is used to prove limit theorems for statistics that are usually implicitly defined as solutions of estimating equations. The thesis is divided into eight chapters and into three main sections. In Section A the theory of convergence is studied as a prelude to validating the use of the particular M-estimators given in Section B and C. Section B initially covers the view of robustness of Hampel (1968) but places more emphasis on the application of the notions of differentiability of functionals and on M-estimators of a general parameter that are robust against "tail" contamination. Sections A and B establish a base for a comparison of robustness and application aspects of minimal distance estimators, particularly with regard to their application to estimating mixtures of normal distributions. An important application of this is illustrated for the analysis of seismic data. This constitutes Section C. Chapter 1 is devoted to the study of uniform convergence theorems over classes of functions and sets allowing also the possibility that the underlying probability mechanism may be from a specified family. A new Glivenko-Cantelli type theorem is proved which has applications later to weakening differentiability requirements for the convergence of loss functions used in this thesis. For implicitly defined estimators it is important to clearly identify the estimator. By uniform convergence, asymptotic uniqueness in regions of the parameter space of solutions to estimating equations can be established. This then justifies the selection of solutions through appropriate statistics, thus defining estimators uniquely for all samples. This comes under the discussion of existence and consistency in Chapter 2. Chapter 3 includes central limit theorems and the law of the iterated logarithm for the general M-estimator, established under various conditions, both on the loss function and on the underlying distribution. Uniform convergence plays a central role in showing the validity of approximating expansions. Results are shown for both univariate and multivariate parameters. Arguments for the univariate parameter are often simpler or require weaker conditions. Our study of robustness is both of a theoretical and quantitative nature. Weak continuity and also Frechet differentiability with respect to Prokhorov, Levy and Kolmogorov distance functions are established for multivariate M-functionals under similar but necessarily stronger conditions than those required for asymptotic normality. Relationships between the conditions imposed on the class of loss functions in order to attain Frechet differentiability and those necessary and sufficient conditions placed on classes of functions for which uniform convergence of measures hold can be shown. Much weaker conditions exist for almost sure uniform convergence and this goes part way to explaining the restrictive nature of this functional derivative approach to showing asymptotic normality. In Chapter 5 the notion of a set of null influence is emphasized. This can be used to construct M-functionals robust (in terms of asymptotic bias and variance) against contamination in the "tails" of a distribution. This set can depend on the parameter being estimated and in this sense the resulting estimator is adaptive. Its construction is illustrated in Chapter 6 for the estimation of scale. Robustness against "tail" contamination is illustrated by numerical comparison with other M-estimators. Particular applications are given to inference in the joint estimation of location and scale where it is important to identify the root to the M-estimating equations. Techniques justified by uniform convergence are used here. Uniform convergence also lends itself to the use of a graphical method of plotting "expectation curves". It can be used for either identifying the M-estimator from multiple solutions of the defining equations or in large samples (e.g. > 50) as a visual indication of whether the fitted model is a good approximation for the underlying mechanism. Theorems based on uniform convergence are given that show a domain of convergence (numerical analysis interpretation) for the Newton-Raphson iteration method applied to M-estimating equations for the location parameter when redescending loss functions are used. The M-estimator theory provides a common framework whereby some minimal distance methods can be compared. Two established L₂ minimal distance estimators are shown to be general M-estimators. In particular a Cramer-Von Mises type distance estimator is shown to be qualitatively robust and have good small sample properties. Its applicability to some new mixture data from geological recordings, which clearly requires robust methods of analysis is demonstrated in Chapters 7 and 8.