Streaming applications often involve several sources that continuously produce data in diverse formats-for instance, IoT applications may receive heterogeneous messages from different devices. In such a scenario, technicians are interested in characterizing the incoming traffic to understand which kinds of data are flowing into the stream; this activity is called schema profiling and consists of producing key insights about the schema of data in a high-variety context. In this paper, we present a streaming approach to schema profiling called DSC+ (Dynamic Stream Clustering). The approach works under the overlapping sliding window paradigm and produces profiles by clustering the schemas extracted from the data according to the k-means algorithm. DSC+ is a two-phase algorithm: schemas are pre-aggregated into a coreset using a compact data structure called schema features and then clustered. Unlike previous approaches, DSC+ is designed to work in a dynamic and unknown domain, where the number of schema attributes is variable and is not known a priori. DSC+ implements rules to dynamically and efficiently update the profile in real-time, minimizing the computations at every slide of the window while maintaining consistency with the principles of k-means. Experiments are carried out on both real and synthetic datasets to demonstrate the efficiency and effectiveness of our proposal against related state-of-the-art algorithms.

Forresi, C., Francia, M., Gallinucci, E., Golfarelli, M. (2026). Dynamic stream clustering for real-time schema profiling with DSC+. KNOWLEDGE AND INFORMATION SYSTEMS, 68(1), 1-36 [10.1007/s10115-026-02889-w].

Dynamic stream clustering for real-time schema profiling with DSC+

Forresi C.;Francia M.;Gallinucci E.;Golfarelli M.
2026

Abstract

Streaming applications often involve several sources that continuously produce data in diverse formats-for instance, IoT applications may receive heterogeneous messages from different devices. In such a scenario, technicians are interested in characterizing the incoming traffic to understand which kinds of data are flowing into the stream; this activity is called schema profiling and consists of producing key insights about the schema of data in a high-variety context. In this paper, we present a streaming approach to schema profiling called DSC+ (Dynamic Stream Clustering). The approach works under the overlapping sliding window paradigm and produces profiles by clustering the schemas extracted from the data according to the k-means algorithm. DSC+ is a two-phase algorithm: schemas are pre-aggregated into a coreset using a compact data structure called schema features and then clustered. Unlike previous approaches, DSC+ is designed to work in a dynamic and unknown domain, where the number of schema attributes is variable and is not known a priori. DSC+ implements rules to dynamically and efficiently update the profile in real-time, minimizing the computations at every slide of the window while maintaining consistency with the principles of k-means. Experiments are carried out on both real and synthetic datasets to demonstrate the efficiency and effectiveness of our proposal against related state-of-the-art algorithms.
2026
Forresi, C., Francia, M., Gallinucci, E., Golfarelli, M. (2026). Dynamic stream clustering for real-time schema profiling with DSC+. KNOWLEDGE AND INFORMATION SYSTEMS, 68(1), 1-36 [10.1007/s10115-026-02889-w].
Forresi, C.; Francia, M.; Gallinucci, E.; Golfarelli, M.
File in questo prodotto:
File Dimensione Formato  
unpaywall-bitstream--86479284.pdf

accesso aperto

Tipo: Versione (PDF) editoriale / Version Of Record
Licenza: Creative commons
Dimensione 2.21 MB
Formato Adobe PDF
2.21 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/1088481
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
  • OpenAlex ND
social impact