Hierarchical Temporal Memory (HTM) is a biologically-inspired framework that can be used to learn invariant representations of patterns in a wide range of applications. Classical HTM learning is mainly unsupervised and once training is completed the network structure is frozen, thus making further training (i.e. incremental learning) quite critical. In this paper we develop a novel technique for HTM (incremental) supervised learning based on error minimization. We prove that error backpropagation can be naturally and elegantly implemented through native HTM message passing based on Belief Propagation. Our experimental results show that a two stage training composed by unsupervised pre-training + supervised refinement is very effective (both accurate and efficient). This is in line with recent findings on other deep architectures.

D. Maltoni, E.M. Rehn (2012). Incremental Learning by Message Passing in Hierarchical Temporal Memory. Berlin : Springer Verlag [10.1007/978-3-642-33212-8].

Incremental Learning by Message Passing in Hierarchical Temporal Memory

MALTONI, DAVIDE;
2012

Abstract

Hierarchical Temporal Memory (HTM) is a biologically-inspired framework that can be used to learn invariant representations of patterns in a wide range of applications. Classical HTM learning is mainly unsupervised and once training is completed the network structure is frozen, thus making further training (i.e. incremental learning) quite critical. In this paper we develop a novel technique for HTM (incremental) supervised learning based on error minimization. We prove that error backpropagation can be naturally and elegantly implemented through native HTM message passing based on Belief Propagation. Our experimental results show that a two stage training composed by unsupervised pre-training + supervised refinement is very effective (both accurate and efficient). This is in line with recent findings on other deep architectures.
2012
Artificial Neural Networks in Pattern Recognition
24
35
D. Maltoni, E.M. Rehn (2012). Incremental Learning by Message Passing in Hierarchical Temporal Memory. Berlin : Springer Verlag [10.1007/978-3-642-33212-8].
D. Maltoni; E.M. Rehn
File in questo prodotto:
Eventuali allegati, non sono esposti

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/143876
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? ND
social impact