传统的信息挖掘技术已经无法满足大数据环境下日益复杂的应用需求,而分布式数据挖掘技术是解决这个难题的一种手段,因此提出了基于改进型频繁模式树(FP-Tree)的分布式关联分类算法。首先,在各局部节点优化FP-Tree。生成局部条件模式树(CFP-Tree),再通过各节点间传送CFP-Tree构建全局CFP-Tree;其次,在挖掘全局CFP-Tree时通过计算显著度来获取初始的全局显著分类规则;最后,利用剪枝策略选取一个较小规则集来构造全局的关联分类器。实验结果表明该算法能够有效降低网络通信量,提高信息挖掘效率,同时保证剪枝的质量和规则的统计显著性,提高分类的精确性。
Traditional information mining technology has been unable to meet the increasingly complex application requirements in the big data environment. The distributed data mining technique is a means to solve this problem. An improved distributed associative classification algorithm based on improved FP-tree was presented. First, FP-Tree was optimized in each local node to generate local conditional pattern tree ( CFP-Tree), and then a global CFP-Tree was constructed through the inter-site transmission of each CFP-Tree. Second, the initial global significant classification rules were obtained by calculating significant degree in the process of global CFP-Tree mining. Final, the pruning strate- gies were used to get a small set of rules to construct the overall associative classifier. Experimental results show that this algorithm can not only effectively reduce network traffic and improve mining efficiency, but also ensure ensuring statistical significance of rules and improve the ability for the discovery of implicit rules.