An essential prerequisite for random generation of good quality samples in Variational Autoencoders (VAE) is that the distribution of variables in the latent space has a known distribution, typically a normal distribution N(0, 1). This should be induced by a regularization term in the loss function, minimizing for each data X, the Kullback-Leibler distance between the posterior inference distribution of latent variables Q(z|X) and N(0, 1). In this article, we investigate the marginal inference distribution Q(z) as a Gaussian Mixture Model, proving, under a few reasonable assumptions, that although the first and second moment of Q(z) might indeed be coherent with those of a normal distribution, there is no reason to believe the same for other moments; in particular, its Kurtosis is likely to be different from 3. The actual distribution of Q(z) is possibly very far from a Normal, raising doubts on the effectiveness of generative sampling according to the vanilla VAE framework.
Titolo: | About Generative Aspects of Variational Autoencoders |
Autore/i: | Asperti, Andrea |
Autore/i Unibo: | |
Anno: | 2019 |
Serie: | |
Titolo del libro: | Machine Learning, Optimization, and Data Science - 5th International Conference, LOD 2019, Siena, Italy, September 10-13, 2019, Proceedings |
Pagina iniziale: | 71 |
Pagina finale: | 82 |
Digital Object Identifier (DOI): | http://dx.doi.org/10.1007/978-3-030-37599-7_7 |
Abstract: | An essential prerequisite for random generation of good quality samples in Variational Autoencoders (VAE) is that the distribution of variables in the latent space has a known distribution, typically a normal distribution N(0, 1). This should be induced by a regularization term in the loss function, minimizing for each data X, the Kullback-Leibler distance between the posterior inference distribution of latent variables Q(z|X) and N(0, 1). In this article, we investigate the marginal inference distribution Q(z) as a Gaussian Mixture Model, proving, under a few reasonable assumptions, that although the first and second moment of Q(z) might indeed be coherent with those of a normal distribution, there is no reason to believe the same for other moments; in particular, its Kurtosis is likely to be different from 3. The actual distribution of Q(z) is possibly very far from a Normal, raising doubts on the effectiveness of generative sampling according to the vanilla VAE framework. |
Data stato definitivo: | 22-gen-2020 |
Appare nelle tipologie: | 4.01 Contributo in Atti di convegno |