Noise tolerant drift detection method for data stream mining

Pingfan Wang, Nanlin Jin*, Wai Lok Woo, John R. Woodward, Duncan Davies

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Drift detection methods identify changes in data streams. Such changes are called concept drifts. Existing drift detection methods often assume that the input is a noise-free data stream. However, in real world applications, for example, data streams generating from internet of things are normally contaminated with noise. (noise, i.e. class noise and/or attribute noise). In this paper, we propose a Noise Tolerant Drift Detection Method (NTDDM), which is based on two-step detection and validation function to detect drifts, and filters out the false drifts caused by the noise. The NTDDM is compared with six well-known drift detection methods and tested on four benchmarks having different levels. Three performance indicators are proposed to determine whether the drift detection is made within a reasonable time, and the length of time to the known drift starting point. The comparative studies demonstrate that NTDDM outperforms the existing methods, over these performance indicators. Our proposed method has achieved a statistically significant improvement on drift detection compared to the methods in experiment. The proposed NTDDM makes it possible to efficiently and effectively detect drift in a noisy data stream.

Original languageEnglish
Pages (from-to)1318-1333
Number of pages16
JournalInformation Sciences
Volume609
Early online date25 Aug 2022
DOIs
Publication statusPublished - 1 Sep 2022

Fingerprint

Dive into the research topics of 'Noise tolerant drift detection method for data stream mining'. Together they form a unique fingerprint.

Cite this