The term GreenAI refers to a novel approach to Deep Learning, that is more aware of the ecological impact and the computational efficiency of its methods. The promoters of GreenAI suggested the use of Floating Point Operations (FLOPs) as a measure of the computational cost of Neural Networks; however, that measure does not correlate well with the energy consumption of hardware equipped with massively parallel processing units like GPUs or TPUs. In this article, we propose a simple refinement of the formula used to compute floating point operations for convolutional layers, called alpha-FLOPs, explaining and correcting the traditional discrepancy with respect to different layers, and closer to reality. The notion of alpha-FLOPs relies on the crucial insight that, in case of inputs with multiple dimensions, there is no reason to believe that the speedup offered by parallelism will be uniform along all different axes.

Dissecting FLOPs Along Input Dimensions for GreenAI Cost Estimations / Asperti, Andrea; Evangelista, Davide; Marzolla, Moreno. - STAMPA. - 13164:(2021), pp. 86-100. (Intervento presentato al convegno The 7th International Conference on Machine Learning Optimization Data Science, LOD 2021 tenutosi a Grasmere, United Kingdom nel 4-8 october 2021) [10.1007/978-3-030-95470-3_7].

Dissecting FLOPs Along Input Dimensions for GreenAI Cost Estimations

Asperti, Andrea
;
Evangelista, Davide;Marzolla, Moreno
2021

Abstract

The term GreenAI refers to a novel approach to Deep Learning, that is more aware of the ecological impact and the computational efficiency of its methods. The promoters of GreenAI suggested the use of Floating Point Operations (FLOPs) as a measure of the computational cost of Neural Networks; however, that measure does not correlate well with the energy consumption of hardware equipped with massively parallel processing units like GPUs or TPUs. In this article, we propose a simple refinement of the formula used to compute floating point operations for convolutional layers, called alpha-FLOPs, explaining and correcting the traditional discrepancy with respect to different layers, and closer to reality. The notion of alpha-FLOPs relies on the crucial insight that, in case of inputs with multiple dimensions, there is no reason to believe that the speedup offered by parallelism will be uniform along all different axes.
2021
International Conference on Machine Learning, Optimization, and Data Science
86
100
Dissecting FLOPs Along Input Dimensions for GreenAI Cost Estimations / Asperti, Andrea; Evangelista, Davide; Marzolla, Moreno. - STAMPA. - 13164:(2021), pp. 86-100. (Intervento presentato al convegno The 7th International Conference on Machine Learning Optimization Data Science, LOD 2021 tenutosi a Grasmere, United Kingdom nel 4-8 october 2021) [10.1007/978-3-030-95470-3_7].
Asperti, Andrea; Evangelista, Davide; Marzolla, Moreno
File in questo prodotto:
File Dimensione Formato  
2107.11949.pdf

accesso aperto

Tipo: Postprint
Licenza: Licenza per accesso libero gratuito
Dimensione 1.56 MB
Formato Adobe PDF
1.56 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/857696
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 2
social impact