中文摘要：

[目的／意义]从大规模已知文本集中检测出与待检测论文的相似文本并计算相似度大小，用于满足在线论文相似性检测秒级响应需求。[方法／过程]采用分治法策略，对已知文本句集进行基于正交基的软聚类预处理，并对软聚类后的每个簇建立倒排索引。接着在快数据处理平台Spark上执行相似性检测，采用字符结合词组形式计算出待检测论文与已知文本的相似度大小。[结影结论]通过200万规模的已知文本集实验结果显示，综合4种类型的待检测论文，所提出的倒排索引结合软聚类算法准确率P为100．0％，召回率R为93．6％，调和平均值F为96．7％。调和平均值F比相似性检测算法LCS高10％左右，比Simhash算法高约23％。在检测速度上，对于一篇字数为5000左右的待检测论文，检测时间约为6．5秒，比Simhash算法快近300倍，比LCS算法快约4000倍。此外，实验结果还表明基于Spark的分布式并行相似性检测算法具有较好的可扩展性。

英文摘要：

[ Purpose/significance ] This paper detects the texts similar with papers to be detected from the large scale known texts and computes their similarities, to meet the second response requirement of online paper similarity de- tection. [ Method/process ] It uses divide and conquer strategy to softly cluster known text sentence set, and establishes inverted index for each cluster after soft clustering. Then it performs the similarity computing between papers to be detec- ted and known texts on the fast data processing platform - Spark, using the method of character combined with phrase. [ Result/conclusion ] Through the experiment of two million known texts set, the results show that the proposed inverted index algorithm combined with soft clustering has precision rate P 100.0% , recall rate R 93.6% and harmonic mean F value 96.7%, integrating four types of papers to be detected. The harmonic mean F is about t0% higher than LCS algo- rithm and 23 % higher than Simhash algorithm. In the detection of the paper with 5 000 words, the proposed algorithm has the detection time of about 6.5 seconds, nearly 300 times faster than the Simhash algorithm, and approximately 4 000 times faster than LCS algorithm. In addition, the results also show that the Spark based distributed parallel similarity de- tection algorithm has better scalability.