TY - JOUR
T1 - Distributed Resource Scheduling for Large-Scale MEC Systems
T2 - A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
AU - Jiang, Feibo
AU - Dong, Li
AU - Wang, Kezhi
AU - Yang, Kun
AU - Pan, Cunhua
N1 - Funding information: This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant no. 41604117, 41904127, 61620106011 and U1705263; in part by the Hunan Provincial Natural Science Foundation of China under Grant no. 2020JJ4428, 2020JJ5105, 2021JJ30455; in part by the Hunan Provincial Science Technology Project Foundation under Grant 2018TP1018 and 2018RS3065.
PY - 2022/5/1
Y1 - 2022/5/1
N2 - In large-scale mobile edge computing (MEC) systems, the task latency and energy consumption are important for massive resource-consuming and delay-sensitive Internet of things devices (IoTDs). Against this background, we propose a distributed intelligent resource scheduling (DIRS) framework to minimize the sum of task latency and energy consumption for all IoTDs, which can be formulated as a mixed integer nonlinear programming. The DIRS framework includes centralized training relying on the global information and distributed decision making by each agent deployed in each MEC server. Specifically, we first introduce a novel multi-agent ensemble-assisted distributed deep reinforcement learning (DRL) architecture, which can simplify the overall neural network structure of each agent by partitioning the state space and also improve the performance of a single agent by combining decisions of all the agents. Secondly, we apply action refinement to enhance the exploration ability of the proposed DIRS framework, where the near-optimal state-action pairs are obtained by a novel Levy flight search. Finally, an imitation acceleration scheme is presented to pre-train all the agents, which can significantly accelerate the learning process of the proposed framework through learning the professional experience from a small amount of demonstration data. The simulation results in three typical scenarios demonstrate that the proposed DIRS framework is efficient and outperforms the existing benchmark schemes.
AB - In large-scale mobile edge computing (MEC) systems, the task latency and energy consumption are important for massive resource-consuming and delay-sensitive Internet of things devices (IoTDs). Against this background, we propose a distributed intelligent resource scheduling (DIRS) framework to minimize the sum of task latency and energy consumption for all IoTDs, which can be formulated as a mixed integer nonlinear programming. The DIRS framework includes centralized training relying on the global information and distributed decision making by each agent deployed in each MEC server. Specifically, we first introduce a novel multi-agent ensemble-assisted distributed deep reinforcement learning (DRL) architecture, which can simplify the overall neural network structure of each agent by partitioning the state space and also improve the performance of a single agent by combining decisions of all the agents. Secondly, we apply action refinement to enhance the exploration ability of the proposed DIRS framework, where the near-optimal state-action pairs are obtained by a novel Levy flight search. Finally, an imitation acceleration scheme is presented to pre-train all the agents, which can significantly accelerate the learning process of the proposed framework through learning the professional experience from a small amount of demonstration data. The simulation results in three typical scenarios demonstrate that the proposed DIRS framework is efficient and outperforms the existing benchmark schemes.
KW - Multi-agent reinforcement learning
KW - Distributed deep reinforcement learning
KW - Imitation learning
KW - Resource scheduling
KW - Levy flight
UR - http://www.scopus.com/inward/record.url?scp=85115674476&partnerID=8YFLogxK
U2 - 10.1109/JIOT.2021.3113872
DO - 10.1109/JIOT.2021.3113872
M3 - Article
SN - 2327-4662
VL - 9
SP - 6597
EP - 6610
JO - IEEE Internet of Things Journal
JF - IEEE Internet of Things Journal
IS - 9
ER -