Pushing the Limits Against the No Free Lunch Theorem: Towards Building General-Purpose (GenP) Classification Systems

In this chapter, we provide an overview of the tools our research group is exploiting to build general-purpose (GenP) classification systems. Although the “no free lunch” (NFL) theorem claims, in effect, that generating a universal classifier is impossible, the goals of GenP systems are more modest in requiring little to no parameter tuning for performing competitively across a range of tasks within a domain or with specific data types, such as images, that span across several fields. The tools outlined here for building GenP systems include methods for building ensembles, matrix representations of data treated as images, deep learning approaches, data augmentation, and classification within dissimilarity spaces. Each of these tools is explained in detail and illustrated with a few examples taken from our work building GenP systems, which spans nearly fifteen years. We note both our successes and some of our limitations. This chapter ends by pointing out some developments in quantum computing and quantum-inspired algorithms that may allow researchers to push the limits hypothesized by the NFL theorem even further.