近几年的中文分词研究中,基于条件随机场(CRF)模型的中文分词方法得到了广泛的关注。但是这种分词方法在处理歧义切分方面存在一定的问题。CRF虽然可以消除大部分原有的分词歧义,却会带来更多新的错误切分。该文尝试找到一种简单的、基于"固结词串"实例的机器学习方法解决分词歧义问题。实验结果表明,该方法可以简单有效的解决原有的分词歧义问题,并且不会产生更多新的歧义切分。
Chinese word segmentation based on CRF(Conditional Random Field) has attracted the most attention in recent research.But this method has certain defects in handling the ambiguity of word segmentation: eliminating most original ambiguity errors at the cost of more new errors.In this paper,we attempt on a simple and example-based machine learning method to deal with the problem of word segmentation ambiguity: the method based on stable string.The experiment results indicate that stable string based method can solve the ambiguity simple and effective.And it will not introduce more new errors.