现有的概念漂移算法大多建立在数据流的分类模型上,忽略了特征空间与样本空间的分布特点,以及特征选择和加权的重要性.针对此问题提出了一种基于特征项分布的信息熵及特征动态加权算法,从概念漂移的动态演化性出发,根据样本和特征空间的拟合程度,运用特征信息熵理论对数据流中的概念漂移现象进行捕捉,以实现新旧概念的过渡.利用改进的隐含Dirichlet模型特征动态加权算法,以解决当前特征与历史特征的权重确定和无效特征的裁剪问题.在公开的语料库CCERT和Trec06上的测试实验证明了所提出算法的有效性.
Most of the existing concept drift algorithm focuses on the classification model data streams,some of which overlook the distribution of the feature space and sample space,and the importance of feature selection and weighting.To solve this problem,we propose a dynamic information entropy and feature weighting algorithm based on the distribution of feature items from the dynamic evolution of the concept drift departure. To realize the concept transition,we capture the concept drifting of the data stream by the information entropy,according to the fitness degree between the sample and feature space. We improve the feature dynamic weighting latent dirichlet model,to overcome the problem of the current and historical feature weight assignment,as well as cropping the invalid features. Furthermore,the validity of the proposed algorithm was confirmed by the test in open corpus CCERT and Trec06.