QuadCDD: A Quadruple-based Approach for Understanding Concept Drift in Data Streams

Pingfan Wang*, Hang Yu, Nanlin Jin, Duncan Davies, Wai lok Woo

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
56 Downloads (Pure)

Abstract

Concept drift is a prevalent phenomenon in data streams that necessitates detection and in-depth understanding, as it signifies that the statistical properties of a target variable, which the model aims to predict, change over time in unforeseen ways. Existing detection methods predominantly aim to identify the drift start time, which lack comprehensive understanding of data streams, leading to a loss of drift information. In this paper, we present a novel Quadruple-based Approach for Understanding Concept Drift in Data Streams (QuadCDD) framework that not only detects and predicts the concept drift start point but also offers a more detailed analysis of concept drift through the use of quadruples, encompassing drift start, drift end, drift severity, and drift type. Our framework employs quadruples to enable informed decision-making and adopt appropriate actions to handle various concept drifts, effectively maintaining high and stable performance in data streams with concept drift. Experimental results validate the effectiveness of our QuadCDD framework in accurately detecting and understanding concept drifts, as well as in preserving the stability and performance of models in the presence of these drifts.
Original languageEnglish
Article number122114
Number of pages14
JournalExpert Systems with Applications
Volume238
Early online date25 Oct 2023
DOIs
Publication statusPublished - 15 Mar 2024

Keywords

  • Concept drift
  • Deep neural network
  • Detection and understanding
  • Performance stability

Cite this