Towards a Process-Driven Design of Data Platforms

Data platforms are state-of-the-art solutions to implement data-driven applications and analytics, since they facilitate the ingestion, storage, management, and exploitation of big data. Data platforms are built on top of complex ecosystems of services answering different data needs and requirements; such ecosystems are offered by different providers (e.g., Amazon AWS and Apache). However, when it comes to engineering data platforms, no unifying strategy and methodology is there yet, and the design is mainly left to the expertise of practitioners in the field. In particular, service providers simply expose a long list of interoperable and alternative engines, making it hard to select the optimal subset without a deep knowledge of the ecosystem. A more effective approach to the design starts from the knowledge of the data transformation and exploitation processes that should be supported by the platform. In this paper, we sketch a computer-aided design methodology and then focus on the selection of the optimal services needed to implement such processes. We believe that our approach lightens the design of data platforms and enables an unbiased selection and comparison of solutions even through different service ecosystems.