Streaming applications often involve several sources that continuously produce data in diverse formats-for instance, IoT applications may receive heterogeneous messages from different devices. In such a scenario, technicians are interested in characterizing the incoming traffic to understand which kinds of data are flowing into the stream; this activity is called schema profiling and consists of producing key insights about the schema of data in a high-variety context. In this paper, we present a streaming approach to schema profiling called DSC+ (Dynamic Stream Clustering). The approach works under the overlapping sliding window paradigm and produces profiles by clustering the schemas extracted from the data according to the k-means algorithm. DSC+ is a two-phase algorithm: schemas are pre-aggregated into a coreset using a compact data structure called schema features and then clustered. Unlike previous approaches, DSC+ is designed to work in a dynamic and unknown domain, where the number of schema attributes is variable and is not known a priori. DSC+ implements rules to dynamically and efficiently update the profile in real-time, minimizing the computations at every slide of the window while maintaining consistency with the principles of k-means. Experiments are carried out on both real and synthetic datasets to demonstrate the efficiency and effectiveness of our proposal against related state-of-the-art algorithms.
Forresi, C., Francia, M., Gallinucci, E., Golfarelli, M. (2026). Dynamic stream clustering for real-time schema profiling with DSC+. KNOWLEDGE AND INFORMATION SYSTEMS, 68(1), 1-36 [10.1007/s10115-026-02889-w].
Dynamic stream clustering for real-time schema profiling with DSC+
Forresi C.;Francia M.;Gallinucci E.;Golfarelli M.
2026
Abstract
Streaming applications often involve several sources that continuously produce data in diverse formats-for instance, IoT applications may receive heterogeneous messages from different devices. In such a scenario, technicians are interested in characterizing the incoming traffic to understand which kinds of data are flowing into the stream; this activity is called schema profiling and consists of producing key insights about the schema of data in a high-variety context. In this paper, we present a streaming approach to schema profiling called DSC+ (Dynamic Stream Clustering). The approach works under the overlapping sliding window paradigm and produces profiles by clustering the schemas extracted from the data according to the k-means algorithm. DSC+ is a two-phase algorithm: schemas are pre-aggregated into a coreset using a compact data structure called schema features and then clustered. Unlike previous approaches, DSC+ is designed to work in a dynamic and unknown domain, where the number of schema attributes is variable and is not known a priori. DSC+ implements rules to dynamically and efficiently update the profile in real-time, minimizing the computations at every slide of the window while maintaining consistency with the principles of k-means. Experiments are carried out on both real and synthetic datasets to demonstrate the efficiency and effectiveness of our proposal against related state-of-the-art algorithms.| File | Dimensione | Formato | |
|---|---|---|---|
|
unpaywall-bitstream--86479284.pdf
accesso aperto
Tipo:
Versione (PDF) editoriale / Version Of Record
Licenza:
Creative commons
Dimensione
2.21 MB
Formato
Adobe PDF
|
2.21 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.



