Hadoop环境下基于SparkSQL海量自动站数据查询统计初探

在Hadoop分布式计算和存储架构下,自定义ETL数据清洗规则将海量自动站小时单站文件按所属年和站号合并为大文件流转存储至HDFS中,并运用SparkSQL并行计算框架进行统计处理生成常用气象要素日统计值。结果表明,数据处理和获取时效较关系型数据库方式有显著提升。采用SparkSQL并行计算框架对多气象要素多站点和长时间序列进行数据统计处理查询均能达到秒级别响应,并随着统计站点数的不断增加和时间跨度的延长其优势更为明显,能更高效地支撑此类气象数据服务,为海量气象数据处理从关系型数据库到大数据分布式架构的转换处理提供了新思路。     

新一代天气雷达资料传输说明及设置技巧

山西省人工降雨防雹办公室在山西省布设了雷电监测定位系统,在全省7个子站配备了ADTD雷电探测仪,并通过CDMA无线传输和气象系统局域网传输的方式将所探测的闪电数据传输到山西省人工降雨防雹办公室,以达到对雷电天气的实时监控．在数据上传国家探测中心的同时,也为我们飞行作业计划的制定提供了非常重要的指导作用。本文主要介绍了雷电定位系统的构成以及故障检查和处理。     

南极伊利莎白公主地250年来雪芯累积、氧同位素与化学研究

1Introduction In the ongoing discussion of climate change,the mass balance of Antarctica has received increasing attention during recent decades,since its reaction to global warming will strongly influence sea-level change(Schlosser and Oerter,2002).Many …     

A hybrid integrated deep learning model for the prediction of citywide spatio-temporal flow volumes

ABSTRACT

The spatio-temporal residual network (ST-ResNet) leverages the power of deep learning (DL) for predicting the volume of citywide spatio-temporal flows. However, this model, neglects the dynamic dependency of the input flows in the temporal dimension, which affects what spatio-temporal features may be captured in the result. This study introduces a long short-term memory (LSTM) neural network into the ST-ResNet to form a hybrid integrated-DL model to predict the volumes of citywide spatio-temporal flows (called HIDLST). The new model can dynamically learn the temporal dependency among flows via the feedback connection in the LSTM to improve accurate captures of spatio-temporal features in the flows. We test the HIDLST model by predicting the volumes of citywide taxi flows in Beijing, China. We tune the hyperparameters of the HIDLST model to optimize the prediction accuracy. A comparative study shows that the proposed model consistently outperforms ST-ResNet and several other typical DL-based models on prediction accuracy. Furthermore, we discuss the distribution of prediction errors and the contributions of the different spatio-temporal patterns.     

黄河长江珠江三角洲近30年海平面上升趋势及2030年上升量预测

黄河、长江和珠江三角洲由于地面沉降等原因,过去30年的相对海平面上升率远大于全球或全国海平面上升率（约1.5mm/a）。下一世纪,根据IPCC的最佳估计,至2030年全球海平面将上升18cm。我国三大三角洲,根据目前地面沉降情况、发展趋势及政府的控制措施,估计2030年相对海平面上升量老黄河三角洲（天津地区）为60cm,现代黄河三角洲（山东省东营市地区）为30—35cm,长江三角洲（上海地区）为30—40cm,珠江三角洲20—25cm,以上估计数可供沿海有关决策部门制订今后长远发展规划及拟订政策时的参考。     

Connecting Local to Global: Geographic Information Systems and Ecological Footprints as Tools for Sustainability

Tools that support public engagement with sustainability are essential for local sustainability planning. This research investigates the ability of two geographic information system (GIS)-based tools to promote discussion of sustainability in a suburban context. A local ecological footprint tool and a community environmental atlas (an environmentally themed online mapping system) were created for seven suburban boroughs of Montreal. Variations of both tools have been used to support sustainability efforts, but their use has not been widely evaluated. Working from a public participation GIS (PPGIS) framework that recognizes the powerful influence of data representation, this research uses focus groups to evaluate how well these tools address three criteria that have emerged from the literature on public engagement in sustainability: interdependency across systems, reflexivity about personal and social decision making, and interactions across spatial scales. Whereas the atlas remains advantageous for discussing local spatial specifics, it was found that the ecological footprint helped people see the interconnections among systems, integrate local and global aspects of sustainability, and reflect on the values and assumptions underlying current social and economic structures.     

ENVIRONMENTAL CONSEQUENCES OF HUMAN ACTIVITY ON THE YELLOW RIVER AND ITS DELTA,CHINA

The Yellow River is unique among the major rivers of the world because of its extremely heavy sediment load in relation to its small discharge. The Yellow River has changed courses numerous times during the past 5000 to 6000 years. With its large sediment load, most of which comes from the Loess Plateau of central China, the river has created a number of deltas along the coast of northern China. The abandoned deltas are eroding, whereas the present delta being formed in Bohai Bay is growing rapidly. Human activity, including extensive agriculture, heavy water consumption, the use of river-control structures, and war, has been responsible for many of the changes occurring in the Yellow River and its delta. [Key words: hydrology, loess, floods, Yellow River, Bohai Bay, China.]     

1996-2005年壮族传统民居的期刊文献分析

通过对中国期刊全文数据库检索,选择在1996-2005十年间,有关壮族传统民居——干栏式建筑的期刊文献,从论文发表的时间及刊物、资助的课题、作者、理论与案例、学科角度等方面对文章进行了归类、分析,并对研究的现状做了一些评述,得出了当前研究角度广泛但是深度不够、课题资助的力度有待加强、开始关注对传统民居及其文化的保护、以旅游开发进行民居保护成为研究的新方向等结论。     

地方嵌入对在华外资企业出口市场多元化的影响

外资企业通过直接出口和间接溢出效应对中国出口增长做出了巨大贡献.现有文献大多基于外资企业知识溢出研究其对本地出口和创新等方面的促进作用,较少关注外资企业自身的贸易活动.利用2000-2016年中国海关贸易数据库,在全球-地方互动视角下,搭建本地、目的国(地区)和本地-目的国(地区)联系三维度框架,重点关注外资企业地方嵌...     

载体桩在陕南粉质黏土地区的适用性研究

以陕南粉质黏土地区某住宅楼施工现场为试验场地,开展了灌注桩和载体桩复合地基的现场静载试验,结合经验法理论计算,从承载力和建设成本两个方面,对比分析了两种地基基础方案的优劣。结果显示:灌注桩和载体桩复合地基荷载-沉降曲线均为缓变型,两种方案均能提供较高的抗压承载力;当载体桩桩长超过10 m,且作为复合地基增强体时,经验法计算过程未考虑桩端阻力的扩大头效益,计算结果明显低于现场试验结果;在相同桩身用料投入条件下,载体桩复合地基较灌注桩能够提供更高承载力。在陕南的类似粉质黏土地区,建议优先考虑选用载体桩复合地基方案。