A class of deep Boltzmann machines is considered in the simplified framework of a quenched system with Gaussian noise and independent entries. The quenched pressure of a K-layers spin glass model is studied allowing interactions only among consecutive layers. A lower bound for the pressure is found in terms of a convex combination of K Sherrington-Kirkpatrick models and used to study the annealed and replica symmetric regimes of the system. A map with a one dimensional monomer-dimer system is identified and used to rigorously control the annealed region at arbitrary depth K with the methods introduced by Heilmann and Lieb. The compression of this high noise region displays a remarkable phenomenon of localisation of the processing layers. Furthermore a replica symmetric lower bound for the limiting quenched pressure of the model is obtained in a suitable region of the parameters and the replica symmetric pressure is proved to have a unique stationary point.
Deep Boltzmann Machines: Rigorous Results at Arbitrary Depth / Alberici, Diego; Contucci, Pierluigi; Mingione, Emanuele. - In: ANNALES HENRI POINCARE'. - ISSN 1424-0637. - STAMPA. - 40:(2021), pp. 1-24. [10.1007/s00023-021-01027-2]
Deep Boltzmann Machines: Rigorous Results at Arbitrary Depth
Alberici, Diego
;Contucci, Pierluigi;Mingione, Emanuele
2021
Abstract
A class of deep Boltzmann machines is considered in the simplified framework of a quenched system with Gaussian noise and independent entries. The quenched pressure of a K-layers spin glass model is studied allowing interactions only among consecutive layers. A lower bound for the pressure is found in terms of a convex combination of K Sherrington-Kirkpatrick models and used to study the annealed and replica symmetric regimes of the system. A map with a one dimensional monomer-dimer system is identified and used to rigorously control the annealed region at arbitrary depth K with the methods introduced by Heilmann and Lieb. The compression of this high noise region displays a remarkable phenomenon of localisation of the processing layers. Furthermore a replica symmetric lower bound for the limiting quenched pressure of the model is obtained in a suitable region of the parameters and the replica symmetric pressure is proved to have a unique stationary point.File | Dimensione | Formato | |
---|---|---|---|
Pacm3.pdf
accesso aperto
Tipo:
Versione (PDF) editoriale
Licenza:
Licenza per Accesso Aperto. Creative Commons Attribuzione (CCBY)
Dimensione
491.24 kB
Formato
Adobe PDF
|
491.24 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.