文本特征表示是在文本自动分类中最重要的一个环节。在基于向量空间模型(VSM)的文本表示中特征单元粒度的选择直接影响到文本分类的效果。在维吾尔文文本分类中,对于单词特征不能更好地表征文本内容特征的问题,在分析了维吾尔文Bigram对文本分类作用的基础上,构造了一个新的统计量CHIMI,并在此基础上提出了一种维吾尔语Bigram特征提取算法。将抽取到的Bigram作为文本特征,采用支持向量机(SVM)算法对维吾尔文文本进行了分类实验。实验结果表明,与以词为特征的文本分类相比,Bigram作为文本特征能够提高维吾尔文文本分类的准确率和召回率并且通过实验验证了该算法的有效性。
Text representation is the most important phase in automatic text categorization. In the vector space model based text representation, the selection of feature granularity has the direct impact on the text categorization performance.The word features don't have the good representative power to represent the Uyghur texts in text categorization. To solve this problem, the CHIMI based Uyghur Bigram extraction method is proposed and the Uyghur text categorization experiments are conducted using support vector machine algorithm based on the extracted Bigrams as text features. The experimental results show that the Bigram based Uyghur text categorization achieves higher classification precision and recall compared to the word based categorization and experiments demonstrate the effectiveness of the proposed algorithm.