On the Optimality of Sample-Based Estimates of the Expectation of the Empirical Minimizer

Date

2010-10

Authors

Bartlett, Peter L.
Mendelson, Shahar
Philips, Petra

Journal Title

Journal ISSN

Volume Title

Publisher

EDP Sciences

Abstract

We study sample-based estimates of the expectation of the function produced by the empirical minimization algorithm. We investigate the extent to which one can estimate the rate of convergence of the empirical minimizer in a data dependent manner. We establish three main results. First, we provide an algorithm that upper bounds the expectation of the empirical minimizer in a completely data-dependent manner. This bound is based on a structural result due to Bartlett and Mendelson, which relates expectations to sample averages. Second, we show that these structural upper bounds can be loose, compared to previous bounds. In particular, we demonstrate a class for which the expectation of the empirical minimizer decreases as O(1/n) for sample size n, although the upper bound based on structural properties is Ω(1). Third, we show that this looseness of the bound is inevitable: we present an example that shows that a sharp bound cannot be universally recovered from empirical data.

Description

Keywords

Error bounds, empirical minimization, data-dependent complexity

Citation

Source

ESAIM: Probability and Statistics

Type

Journal article

Book Title

Entity type

Access Statement

License Rights

DOI

10.1051/ps:2008036

Restricted until