Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks

Yousef Emami, Bo Wei, Kai Li*, Wei Ni, Eduardo Tovar

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

8 Citations (Scopus)

Abstract

Unmanned Aerial Vehicles (UAVs) can collaborate to collect and relay data for ground sensors in remote and hostile areas. In multi-UAV-assisted wireless sensor networks (MA-WSN), the UAVs' movements impact on channel condition and can fail data transmission, this situation along with newly arrived data give rise to buffer overflows at the ground sensors. Thus, scheduling data transmission is of utmost importance in MA-WSN to reduce data packet losses resulting from buffer overflows and channel fading. In this paper, we investigate the optimal ground sensor selection at the UAVs to minimize data packet losses. The optimization problem is formulated as a multi-agent Markov decision process, where network states consist of battery levels and data buffer lengths of the ground sensor, channel conditions, and waypoints of the UAV along the trajectory. In practice, an MA-WSN contains a large number of network states, while the up-to-date knowledge of the network states and other UAVs' sensor selection decisions is not available at each agent. We propose a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MUAIS) to minimize the data packet loss, where the UAVs learn the underlying patterns of the data and energy arrivals at all the ground sensors. Numerical results show that the proposed MUAIS achieves at least 46% and 35% lower packet loss than an optimal solution with single-UAV and an existing non-learning greedy algorithm, respectively.

Original languageEnglish
Title of host publication2021 International Wireless Communications and Mobile Computing, IWCMC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages669-674
Number of pages6
ISBN (Electronic)9781728186160
DOIs
Publication statusPublished - 2021
Event17th IEEE International Wireless Communications and Mobile Computing, IWCMC 2021 - Virtual, Online, China
Duration: 28 Jun 20212 Jul 2021

Publication series

Name2021 International Wireless Communications and Mobile Computing, IWCMC 2021

Conference

Conference17th IEEE International Wireless Communications and Mobile Computing, IWCMC 2021
Country/TerritoryChina
CityVirtual, Online
Period28/06/212/07/21

Keywords

  • Communication scheduling
  • Deep Q-Network
  • Multi-UAV deep reinforcement learning
  • Unmanned aerial vehicles

Fingerprint

Dive into the research topics of 'Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks'. Together they form a unique fingerprint.

Cite this