While Hopfield networks are known as paradigmatic models for memory storage and retrieval, modern artificial intelligence systems mainly stand on the machine learning paradigm. We show that it is possible to formulate a teacher-student self-supervised learning problem with Boltzmann machines in terms of a suitable generalization of the Hopfield model with structured patterns, where the spin variables are the machine weights and patterns correspond to the training set’s examples. We analyze the learning performance by studying the phase diagram in terms of the training set size, the dataset noise and the inference temperature (i.e. the weight regularization). With a small but informative dataset the machine can learn by memorization. With a noisy dataset, an extensive number of examples above a critical threshold is needed. In this regime the memory storage limits become an opportunity for the occurrence of a learning regime in which the system can generalize.

Francesco Alemanno, Luca Camanzi, Gianluca Manzan, Daniele Tantari (2023). Hopfield model with planted patterns: A teacher-student self-supervised learning model. APPLIED MATHEMATICS AND COMPUTATION, 458, 1-23 [10.1016/j.amc.2023.128253].

Hopfield model with planted patterns: A teacher-student self-supervised learning model

Francesco Alemanno;Gianluca Manzan;Daniele Tantari
2023

Abstract

While Hopfield networks are known as paradigmatic models for memory storage and retrieval, modern artificial intelligence systems mainly stand on the machine learning paradigm. We show that it is possible to formulate a teacher-student self-supervised learning problem with Boltzmann machines in terms of a suitable generalization of the Hopfield model with structured patterns, where the spin variables are the machine weights and patterns correspond to the training set’s examples. We analyze the learning performance by studying the phase diagram in terms of the training set size, the dataset noise and the inference temperature (i.e. the weight regularization). With a small but informative dataset the machine can learn by memorization. With a noisy dataset, an extensive number of examples above a critical threshold is needed. In this regime the memory storage limits become an opportunity for the occurrence of a learning regime in which the system can generalize.
2023
Francesco Alemanno, Luca Camanzi, Gianluca Manzan, Daniele Tantari (2023). Hopfield model with planted patterns: A teacher-student self-supervised learning model. APPLIED MATHEMATICS AND COMPUTATION, 458, 1-23 [10.1016/j.amc.2023.128253].
Francesco Alemanno; Luca Camanzi; Gianluca Manzan; Daniele Tantari
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S0096300323004228-main.pdf

accesso aperto

Tipo: Versione (PDF) editoriale
Licenza: Licenza per Accesso Aperto. Creative Commons Attribuzione - Non commerciale - Non opere derivate (CCBYNCND)
Dimensione 856.7 kB
Formato Adobe PDF
856.7 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/951634
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 2
social impact