DRL Path Planning for UAV-Aided V2X Networks: Comparing Discrete to Continuous Action Spaces

It is expected that future 6G vehicular networks will rely on Unmanned Aerial Vehicles (UAVs) acting as flying Base Stations, namely Unmanned Aerial Base Stations (UABSs), in order to provide a wide range of services that current terrestrial networks cannot manage. Vehicles may exploit strong links with the UABS, enabling applications such as advanced driving and extended sensing. In this context, if vehicular users are satisfied with an appropriate Quality of Experience (QoE), they are able to upload a given amount of data for a given time window, continuously. To allow this, an efficient path planning is fundamental. This paper presents a Deep Reinforcement Learning (DRL)-based solution, where a novel reward function is proposed with the aim of offering a continuous service to vehicles. Results are presented in terms of the percentage of satisfied users and both, a continuous action space as well as a discrete action space, are considered exploiting two different DRL algorithms (i.e., Double Dueling Deep Q Network (3DQN) and Deep Deterministic Policy Gradient (DDPG)) in order to compare the two and select the best one according to the described scenario of interest.