Hybrid Offline/Online Optimization for Energy Management via Reinforcement Learning

Home Integration of Constraint Programming, Artificial Intelligence, and Operations Research Conference paper Hybrid Offline/Online Optimization for Energy Management via Reinforcement Learning Download book PDF Download book EPUB Hybrid Offline/Online Optimization for Energy Management via Reinforcement Learning Mattia Silvestri, Allegra De Filippo, Federico Ruggeri & Michele Lombardi Conference paper First Online: 10 June 2022 681 Accesses Part of the Lecture Notes in Computer Science book series (LNCS,volume 13292) Abstract Constrained decision problems in the real world are subject to uncertainty. If predictive information about the stochastic elements is available offline, recent works have shown that it is possible to rely on an (expensive) parameter tuning phase to improve the behavior of a simple online solver so that it roughly matches the solution quality of an anticipative approach but maintains its original efficiency. Here, we start from a state-of-the-art offline/online optimization method that relies on optimality conditions to inject knowledge of a (convex) online approach into an offline solver used for parameter tuning. We then propose to replace the offline step with (Deep) Reinforcement Learning (RL) approaches, which results in a simpler integration scheme with a higher potential for generalization. We introduce two hybrid methods that combine both learning and optimization: the first optimizes all the parameters at once, whereas the second exploits the sequential nature of the online problem via the Markov Decision Process framework. In a case study in energy management, we show the effectiveness of our hybrid approaches, w.r.t. the state-of-the-art and pure RL methods. The combination proves capable of faster convergence and naturally handles constraint satisfaction