为了研究在线百科全书维基百科在语义相关词抽取、语义相关度计算等方面的应用,从中文维基百科下载并整理了5万余篇解释页面文档,抽取了反映语义关系的站内超级链接网络,通过挖掘文档间相互引用的链接关系抽取了近40万对语义相关词,并研究了语义相关词之间的主题聚类特性,选择两两相关的多个词得到相关词集合并计算了平均语义相关度.相关度计算综合利用了链接在文档中的位置、频率等信息,通过对比经典算法在同一测试集上的计算结果对算法进行了分析评估,证明了维基百科语义相关词抽取方法的有效性.
To find how to collect semantic related words and calculate semantic relatedness, an experiment is done to download about 50 thousand documents from the web site of Chinese Wikipedia and extract hyperlinks between lines which contains semantic information. By mining hyperlinked references in documents, about 400 thousand semantic related word pairs are collected. With more experi- ments on topic groups of related words, tightly related words are grouped into smaller sets with an average semantic relatedness calculated. Semantic relatedness is calculated using information of hyperlink positions and frequencies in documents. Comparing with the result by classic algorithms, the reliability of the new measures is analyzed.