针对传统的索引和过滤算法处理在线相似连接时的不足,提出新的索引方法和过滤算法.在采用倒排索引的基础上,将索引按照位置和长度的相关信息进行划分,以减少查询空间,加强倒排索引的执行效率.此外,设计加权签名过滤算法,用来估计2个集合交的长度的上限,提高过滤的效率.集合的相似连接通常应用于过滤-验证的工作框架里,主要采用2个步骤:先产生候选结果集合;再对候选集合进行验证.通过对真实数据集的实验,结果表明,该过滤算法可以和其他过滤算法一起协同应用于过滤-验证的工作框架里,对数据进行在线相似连接处理,同时在计算效率上也有显著的提升.
To address the deficiency of similarity join online when using traditional indexing and filtering algorithm,we proposed several novel filtering approaches by improving the inverted based and signature based schemes.Enhancing the inverted index to reduce the search spaces,which partition the index according to the information of item's position and the record's length.In addition,we designed a novel weighted signature filtering scheme,where the upper bound of the overlap between two sets can be estimated to improve the effectiveness of filtering.Typically,the processing of set similarity join often adopts the filtering-refinement framework,which generates candidates by some filtering schemes and then produces the final results by refining the candidates.The proposed schemes can be seamlessly integrated into the filtering-refinement framework with other filtering schemes to process set similarity join online.Extensive experiments are conducted using real datasets.The experiments results show the efficiency of the proposed schemes.