On the limitations of embedding methods

Abstract

We show that for any class of functions H which has a reasonable combinatorial dimension, the vast majority of small subsets of the combinatorial cube can not be represented as a Lipschitz image of a subset of H, unless the Lipschitz constant is very large. We apply this result to the case when H consists of linear functionals of norm at most one on a Hilbert space, and thus show that "most" classification problems can not be represented as a reasonable Lipschitz loss of a kernel class. Show more