A common practice in modern explainable AI is to post-hoc explain black-box machine learning (ML) predictors – such as neural networks – by extracting symbolic knowledge out of them, in the form of either rule lists or decision trees. By acting as a surrogate model, the extracted knowledge aims at revealing the inner working of the black box, thus enabling its inspection, representation, and explanation. Various knowledge-extraction algorithms have been presented in the literature so far. Unfortunately, running implementations of most of them are currently either proof of concepts or unavailable. In any case, a unified, coherent software framework supporting them all – as well as their interchange, comparison, and exploitation in arbitrary ML workflows – is currently missing. Accordingly, in this paper we present the design of PSyKE, a platform providing general-purpose support to symbolic knowledge extraction from different sorts of black-box predictors via many extraction algorithms. Notably, PSyKE targets the extraction of symbolic knowledge in logic form, making it possible to extract first-order logic clauses as output. The extracted knowledge is thus both machine- and human- interpretable, and it can be used as a starting point for further symbolic processing—e.g. automated reasoning.

On the Design of PSyKE: A Platform for Symbolic Knowledge Extraction

Federico Sabbatini;Giovanni Ciatto;Roberta Calegari;Andrea Omicini
2021

Abstract

A common practice in modern explainable AI is to post-hoc explain black-box machine learning (ML) predictors – such as neural networks – by extracting symbolic knowledge out of them, in the form of either rule lists or decision trees. By acting as a surrogate model, the extracted knowledge aims at revealing the inner working of the black box, thus enabling its inspection, representation, and explanation. Various knowledge-extraction algorithms have been presented in the literature so far. Unfortunately, running implementations of most of them are currently either proof of concepts or unavailable. In any case, a unified, coherent software framework supporting them all – as well as their interchange, comparison, and exploitation in arbitrary ML workflows – is currently missing. Accordingly, in this paper we present the design of PSyKE, a platform providing general-purpose support to symbolic knowledge extraction from different sorts of black-box predictors via many extraction algorithms. Notably, PSyKE targets the extraction of symbolic knowledge in logic form, making it possible to extract first-order logic clauses as output. The extracted knowledge is thus both machine- and human- interpretable, and it can be used as a starting point for further symbolic processing—e.g. automated reasoning.
WOA 2021 – 22nd Workshop “From Objects to Agents”
29
48
Federico Sabbatini, Giovanni Ciatto, Roberta Calegari, Andrea Omicini
File in questo prodotto:
File Dimensione Formato  
paper14.pdf

accesso aperto

Descrizione: PDF editoriale
Tipo: Versione (PDF) editoriale
Licenza: Licenza per Accesso Aperto. Creative Commons Attribuzione (CCBY)
Dimensione 2.72 MB
Formato Adobe PDF
2.72 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/834364
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? ND
social impact