An algorithm aiming at unimodal density-based clustering using Gaussian mixture models

We study the problem of unimodal density-based clustering based on Gaussian mixture models. In the proposed approach, clusters are thought of as regions of high probability density separated from other regions of low probability density, encouraging the creation of unimodal clusters. The problem of estimation of unimodal Gaussian mixtures has been solved only in the univariate case, while in this work we try to provide a solution for the multivariate setting. The unimodal density-based clustering works in two stages. First, a new merging algorithm based on the density definition of a cluster is used. This algorithm identifies which components should be merged in order to obtain a number of clusters less than or equal to the initial number of mixture components, on the basis of density similarities. Second, a penalized likelihood approach is adopted to induce unimodality in the merged set of components. We evaluate the performance of both methods on the basis of simulated samples and empirical applications.