为了克服现有文档相似性模型对文档特性拟合的不完全性和缺乏理论根据的弱点,本文在统计语言模型的基础上,提出了一种基于混合语言模型(Mixture Language Model,MLM)文档相似性计算模型。MLM利用统计语言模型描述文档特征,将相关影响因素作为模型的潜在子模型,文档语言模型由各子模型混合构成,从而准确和全面地反映文档特征。由于MLM根据具体应用确定相关影响因素,并以此构建相应文档描述模型,因此具有很强的灵活性和扩展性。在MLM的基础上,本文给出了一个基于文档主题内容相似性的实例,在TREC9数据集上的实验表明MLM优于向量空间模型(VSM)。
To overcome the incompleteness of modeling document characteristics and the lack of theory for current document similarity models, this paper puts forward to utilize mixture language model (MLM) to evaluate document-to- document similarity. In MLM, the characteristic of a document is described based on statistic language model, and the factors of influencing its characteristic are viewed as the latent models, and then the document language model is a mixture model combined with each latent models. MLM not only models document characteristics more perfectly, but it is flexible and scalable to be implemented with respect to applications. Within the framework of MLM, a document similarity method is presented from the viewpoint of document content. The experimental results over the TREC9 dataset indicate that MLM outperforms VSM.