针对专利文献的特点,该文提出了一种基于统计和规则相结合的多策略分词方法。该方法利用文献中潜在的切分标记,结合切分文本的上下文信息进行最大概率分词,并利用术语前后缀规律进行后处理。该方法充分利用了从大规模语料中获取的全局信息和切分文本的上下文信息,有效地解决了专利分词中未登录词难以识别问题。实验结果表明,该文方法在封闭和开放测试下分别取得了较好的结果,对未登录词的识别也有很好的效果。
According to the characteristics of the patent documents, this paper presents a multi-strategy approach for word segmentation based on statistics and rules. Our method takes advantage of the latent segmentation-marks in the document and employs the context information of the text in the a maximum probabilistic model of segmentation. Meanwhile, the term affix rules are applied in the post-processing. Making full use of the global information from a large scale corpus and the specific context information, this method effectively solves the problem of the out-of-vo- cabulary words difficult to identify in the patent segmentation. The experimental results indicate that this method achieves good results in the close and opening test, with improves on unknown words recognition as well.