MLCOps: a Platform to Support Cloud Continuum Machine Learning Operations

The increasing reliance on Machine Learning (ML) to extract insights and value from data is driving researchers and businesses to seamlessly integrate it throughout the entire Cloud Continuum (CC), spanning from large-scale cloud infrastructures to edge computing and end devices. However, traditional service orchestration frameworks are fine-tuned to manage single-site general-purpose applications, struggling with the complexity of CC ML deployments, which seek customized strategies for enhanced performance and resiliency. To this end, we propose Machine Learning cloud Continuum Operations (MLCOps), an overlay solution that supports ML applications in CC deployments. MLCOps implements an innovative multisite orchestration layer and provides a new service runtime support, allowing one to react quickly to changing conditions. MLCOps advocates a layered approach to carry out specialized strategies, implementing global and local actions on infrastructure and environment to ensure the performance and availability of the deployed ML services. Without loss of generality, we show how the framework could be instantiated to support an anomaly detection task. In this context, we assess various (re)configuration strategies that can take place in actual deployments, assessing the ability of MLCOps to preserve the reliability of CC ML application deployments under dynamic workload conditions.