针对作为中文信息处理基础的抽词问题,本文在作者提出的正向串频最大匹配法(MMFS)的基础上,提出了逆向串频最大匹配法(RMMFS)及双向串频最大匹配法(BMMFS).这两种方法分别采用逆向和双向长串优先与串频统计的思路,并引进规则和支持度指标筛选,不需要词典,不需要事先进行语料库学习,不需要建立字索引,通过串匹配获取中文文本中的汉字共现模式,实时地抽取出包含专业术语及专有名词等未登录词在内的专指语义串、短语和词.实验研究了抽词准确率受规则的影响及随文本大小和词频变化的分布,结果表明BMMFS可以取得更好的抽词效果.
To solve the problem of automatic word extraction which is the basis of Chinese information processing, this paper presents two new methods reverse maximum matching and frequency statistics(RMMFS) and bidirectional maximum matching and frequency statistics( BMMFS). RMMFS and BMMFS count string frequency by reverse and bidirectional matching longer strings first respectively and obtain co-occurrence patterns of Chinese characters. Refined and filtered by introducing rules and support criteria, special semantic strings, phrases and words, including unknown words like proper nouns and terms, can be extracted in real time without using dictionary, without previous study and without constructing Chinese characters index. The effect of rules and the accuracy distribution of word extraction of RMMFS and BMMFS in different text size and word frequency are studied. Experiments show that BMMFS can get better result.