位置:成果数据库 > 期刊 > 期刊详情页
基于MapReduce的多机并行DP算法与实验分析
  • ISSN号:1560-8999
  • 期刊名称:地球信息科学学报
  • 时间:2013.2.2
  • 页码:55-60
  • 分类:TP393.03[自动化与计算机技术—计算机应用技术;自动化与计算机技术—计算机科学与技术]
  • 作者机构:[1]武汉大学卫星导航技术研究中心,武汉430079, [2]武汉大学资源与环境科学学院,武汉430079, [3]武汉大学地理信息系统教育部重点实验室,武汉430079
  • 相关基金:国家自然科学基金项目(41101448,51008138); 中央高校自主科研项目(274737); 中国博士后科学基金项目(2011M501230)
  • 相关项目:基于区位适应的城市用地空间结构优化决策模型研究
中文摘要:

随着网络地图不断发展,个性化网络地图也得到快速发展。个性化网络地图需要以矢量数据为数据基础,以满足人们对地图色彩、符号等个性化要求,所以需要实时、快速进行大量数据化简。本文以经典Douglas—Peucker算法作为曲线化简算法,利用开源云计算平台Hadoop建立多机协作的曲线并行化简服务框架,设计和实现了多机并行Douglas—Peucker算法,并在集群上进行实验分析,验证算法的效率和适用性。算法核心是设计数据的逻辑分片,利用MapReduce计算原理,将分片分配到集群中,实现并行运算。实验分别分为两个方面:(1)比较在固定阈值不同数据量情况下,传统DP算法与多机并行DP算法效率;(2)比较在相同数据量不同阈值情况下,传统DP算法与多机并行DP算法效率。实验表明,在大数据量和高复杂度情况下,多机并行DP算法的效率更高。

英文摘要:

Real time and rapid simplification of large-scale data,required by personalized WebGIS service which is based on vector data,becomes more and more important.The study was based on Douglas-Peucker,one of classical curve simplification algorithms,but in the view of its low performance,it can hardly simplify large-scale data in real time and rapidly.At the same time,the development of cloud-computing offers new storage technologies and computational methods for real time and rapid simplification of large-scale data.So this study made use of hadoop,one of the open source cloud computing platforms,to design and realize multi-machine parallel Douglas-Peucker algorithm.In the algorithm,we deigned the logic slices of data,and assigned the slices to the clusters by MapReduce computing model,achieved parallel simplification.In order to verify the efficiency of the algorithm,we designed the experiments and compared the efficiency of traditional DP algorithm and multi-machine parallel DP algorithm in tow aspects: 1) the same threshold and different amount of data;and 2) the fixed amount of data and different thresholds.The(result of the experiments showed: the multi-machine parallel DP algorithm was more efficient than tradition DP algorithm for large-scale data and high-complexity computing.In this case,the data processing time was much longer than the data allocated in the inter-cluster and the transmission time,and every node was involved in a certain operation,improved the efficiency of operations.But for small scale data and low-complexity computing,the advantage of multi-machine parallel DP algorithm was non-obvious.Mainly due to a part of the nodes didn't participate in the operation,the computing potential of the cluster was not full play,while the data processing required time was very short,so the data allocation and transmission time impacted obviously.And,in order to meet the real time and rapid simplification of large-scale data,the multi-machine parallel DP algorithm should choose the appropriate si

同期刊论文项目
同项目期刊论文
期刊信息
  • 《地球信息科学学报》
  • 中国科技核心期刊
  • 主管单位:中国科学院
  • 主办单位:中国科学院地理科学与资源研究所 中国地理学会
  • 主编:徐冠华
  • 地址:北京大屯路甲11号
  • 邮编:100101
  • 邮箱:sxfu@lreis.ac.cn
  • 电话:010-64888891
  • 国际标准刊号:ISSN:1560-8999
  • 国内统一刊号:ISSN:11-5809/P
  • 邮发代号:82-919
  • 获奖情况:
  • 国内外数据库收录:
  • 中国中国科技核心期刊,中国北大核心期刊(2008版),中国北大核心期刊(2011版),中国北大核心期刊(2014版)
  • 被引量:3181