Concept Drift Detection by Tracking Weighted Prediction Confidence of Incremental Learning

Pingfan Wang, Wai Lok Woo*, Nanlin Jin, Duncan Davies

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Data stream mining is great significant in many real-world scenarios, especially in the big data area. However, conventional machine learning algorithms are incapable to process because of its two characteristics (1) potential unlimited number of data is generated in real-time way, it is impossible to store all the data (2) evolving over time, namely, concept drift, will influence the performance of predictor trained on previous data. Concept drift detection method could detect and locate the concept drift in data stream. However, existing methods only utilize the prediction result as indicator. In this article, we propose a weighted concept drift indicator based on incremental ensemble learning to detect the concept. The indicator not only considers the prediction result, but the change of prediction stability of predictor with occurs of concept drift. Also, an incremental ensemble learning based on vote mechanism is especially used to get constantly updated value of indicator. Based on the experiment result on both benchmark and real-world dataset, our method could effectively detect concept drift and outperform other existing methods.

Original languageEnglish
Title of host publicationIVSP 2022
Subtitle of host publication2022 4th International Conference on Image, Video and Signal Processing
Place of PublicationNew York, US
PublisherACM
Pages218-223
Number of pages6
ISBN (Electronic)9781450387415
DOIs
Publication statusPublished - 18 Mar 2022
EventIVSP 2022: 2022 4th International Conference on Image, Video and Signal Processing - Virtual
Duration: 18 Mar 202220 Mar 2022
http://www.ivsp.net/IVSP2022.html

Publication series

NameACM International Conference Proceeding Series

Conference

ConferenceIVSP 2022
CityVirtual
Period18/03/2220/03/22
Internet address

Fingerprint

Dive into the research topics of 'Concept Drift Detection by Tracking Weighted Prediction Confidence of Incremental Learning'. Together they form a unique fingerprint.

Cite this