Comparing processes or models is of interest in various applications. Among the existing approaches, one of the most popular methods is to use the Kullback-Leibler (KL) divergence which is related to Shannon's entropy. Similarly, the Rényi divergence of order α can be deduced from the Rényi entropy. When α tends to 1, it leads to the KL divergence. In this paper, our purpose is to derive the expression of the Rényi divergence between the probability density functions of k consecutive samples of two real first-order moving average (MA) processes by using the eigen-decompositions of their Toeplitz correlation matrices. The resulting expression is compared with the expressions of the Rao distance and the Jeffrey's divergence (JD) based on the eigenvalues. The way these quantities evolve when k increases is then presented. When dealing with unit-zero MA processes, the derivate is infinite for the JD and finite for the others. The influence of α is also studied.

Rényi Divergence to Compare Moving-Average Processes

Diversi, Roberto
2018

Abstract

Comparing processes or models is of interest in various applications. Among the existing approaches, one of the most popular methods is to use the Kullback-Leibler (KL) divergence which is related to Shannon's entropy. Similarly, the Rényi divergence of order α can be deduced from the Rényi entropy. When α tends to 1, it leads to the KL divergence. In this paper, our purpose is to derive the expression of the Rényi divergence between the probability density functions of k consecutive samples of two real first-order moving average (MA) processes by using the eigen-decompositions of their Toeplitz correlation matrices. The resulting expression is compared with the expressions of the Rao distance and the Jeffrey's divergence (JD) based on the eigenvalues. The way these quantities evolve when k increases is then presented. When dealing with unit-zero MA processes, the derivate is infinite for the JD and finite for the others. The influence of α is also studied.
2018 IEEE Statistical Signal Processing Workshop, SSP 2018
149
153
Merchan, Fernando; Grivel, Eric; Diversi, Roberto
File in questo prodotto:
Eventuali allegati, non sono esposti

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/668667
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? ND
social impact