TF-IDF是一种常用的文本特征选择方法。基于该模型的特征选择思想,以特征项的类内分布、类间分布信息为依据,通过引入类内分布及类间分布权重因子对模型的TF及IDF部分进行加权,提出一种基于类别分布信息的文本特征选择模型。新模型使得TF部分含有类内文本频数信息,同时1DF部分含有特征项的类间频数信息。随后的文本分类试验表明,平均查全率、查准率分别提高6.4%、7.8%,FI值提高约7%,验证了本研究提出的基于类别分布的文本特征选择模型的有效性。
TF-IDF is a commonly used text feature selection method. Based on the characteristics of the model selection ideas and using the feature within class distribution and the distribution between class information as the foundations, we propose a model of text feature selection based on the category distribution information through the introduction of weighting factor distribution within classes and between classes. The new model makes the TF part contains the within class text frequency information. At the same time, the IDF part contains the between class frequency information. The subsequent text classification experiments proved that the average recall rate, precision rate increased 6. 4%, 7.8% respectively. At the same time, the F1 value increased about 7%. We demonstrate the effectiveness of the text feature selection model proposed in this paper.