NAIR: An Efficient Distributed Deep Learning Architecture for Resource Constrained IoT System

Yucong Xiao, Daobing Zhang*, Yunsheng Wang, Xuewu Dai, Zhipei Huang, Wuxiong Zhang, Yang Yang, Ashiq Anjum, Fei Qin

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    1 Downloads (Pure)


    The distributed deep learning architecture can support the front-deployment of deep learning systems in resource constrained IoT devices and is attracting increasing interest. However, most ready-to-use deep models are designed for centralized deployment without considering the transmission loss of the intermediate representation inside the distributed architecture. This oversight significantly affects the inference performance of distributed deployed deep models. To alleviate this problem, a state-of-the-art work chooses to retrain the original model to form an intermediate representation with ordered importance and yields better inference accuracy under constrained transmission bandwidth. This paper first reveals that this solution is essentially a pruning-like solution, where unimportant information is adaptively pruned to fit within the limited bandwidth. With this understanding, a novel scheme named Naturally Aggregated Intermediate Representation (NAIR) has been proposed, which aims to naturally amplify the difference of importance embedded in the intermediate representation from a mature deep model and reassemble the intermediate representation into a hierarchy of importance from high-to-low to accommodate the transmission loss. As a result, this method shows further improved performance in various scenarios, avoids compromising the overall inference performance of the system, and saves astronomical retraining and storage costs. The effectiveness of NAIR has been validated through extensive experiments, achieving a 112% improvement in performance compared to the state-of-the-art work.
    Original languageEnglish
    Pages (from-to)1-14
    Number of pages14
    JournalIEEE Internet of Things Journal
    Early online date18 Mar 2024
    Publication statusE-pub ahead of print - 18 Mar 2024

    Cite this