| 数字海洋云计算服务流中数据预部署研究 |
| |
| 作者姓名： | SHI Suixiang XU Lingyu DONG Han WANG Lei WU Shaochun QIAO Baiyou WANG Guoren |
| |
| 作者单位： | National Marine Data and Information Service, State Oceanic Administration, Tianjin 300171, China;Key Laboratory of Digital Ocean, State Oceanic Administration, Tianjin 300171, China;College of Computer Engineering and Science, Shanghai University, Shanghai 200072, China;National Marine Data and Information Service, State Oceanic Administration, Tianjin 300171, China;Key Laboratory of Digital Ocean, State Oceanic Administration, Tianjin 300171, China;College of Computer Engineering and Science, Shanghai University, Shanghai 200072, China;College of Computer Engineering and Science, Shanghai University, Shanghai 200072, China;College of Information Science and Engineering, Northeastern University, Shenyang 110819, China;College of Information Science and Engineering, Northeastern University, Shenyang 110819, China |
| |
| 基金项目： | The Ocean Public Welfare Scientific Research Project of State Oceanic Administration of China under contract No. 20110533. |
| |
| 摘 要： | Data pre-deployment in the HDFS （Hadoop distributed file systems） is more complicated than that in traditional file systems. There are many key issues need to be addressed, such as determining the target location of the data prefetching, the amount of data to be prefetched, the balance between data prefetching services and normal data accesses. Aiming to solve these problems, we employ the characteristics of digital ocean information service flows and propose a deployment scheme which combines input data prefetching with output data oriented storage strategies. The method achieves the parallelism of data preparation and data processing, thereby massively reducing I/O time cost of digital ocean cloud computing platforms when processing multi-source information synergistic tasks. The experimental results show that the scheme has a higher degree of parallelism than traditional Hadoop mechanisms, shortens the waiting time of a running service node, and significantly reduces data access conflicts.
|
| 关 键 词： | 服务流程 部署方案 数字海洋 海洋信息 计算平台 分布式文件系统 数据预取 目标位置 |
| 收稿时间： | 4/4/2014 12:00:00 AM |
| 修稿时间： | 2014/5/27 0:00:00 |
Research on data pre-deployment in information service flow of digital ocean cloud computing |
| |
| SHI Suixiang,XU Lingyu,DONG Han,WANG Lei,WU Shaochun,QIAO Baiyou,WANG Guoren.Research on data pre-deployment in information service flow of digital ocean cloud computing[J].Acta Oceanologica Sinica,2014,33(9):82-92. |
| |
| Authors: | SHI Suixiang XU Lingyu DONG Han WANG Lei WU Shaochun QIAO Baiyou and WANG Guoren |
| |
| Institution: | 1.National Marine Data and Information Service, State Oceanic Administration, Tianjin 300171, China;Key Laboratory of Digital Ocean, State Oceanic Administration, Tianjin 300171, China2.College of Computer Engineering and Science, Shanghai University, Shanghai 200072, China3.College of Information Science and Engineering, Northeastern University, Shenyang 110819, China |
| |
| Abstract: | Data pre-deployment in the HDFS (Hadoop distributed file systems) is more complicated than that in traditional file systems. There are many key issues need to be addressed, such as determining the target location of the data prefetching, the amount of data to be prefetched, the balance between data prefetching services and normal data accesses. Aiming to solve these problems, we employ the characteristics of digital ocean information service flows and propose a deployment scheme which combines input data prefetching with output data oriented storage strategies. The method achieves the parallelism of data preparation and data processing, thereby massively reducing I/O time cost of digital ocean cloud computing platforms when processing multi-source information synergistic tasks. The experimental results show that the scheme has a higher degree of parallelism than traditional Hadoop mechanisms, shortens the waiting time of a running service node, and significantly reduces data access conflicts. |
| |
| Keywords: | HDFS data prefetching cloud computing service flow digital ocean |
| 本文献已被 CNKI 维普 SpringerLink 等数据库收录！ |
| 点击此处可从《海洋学报(英文版)》浏览原始摘要信息 |
| 点击此处可从《海洋学报(英文版)》下载免费的PDF全文 |
|
