高性能并行分布式水文模型研究进展

doi:10.18306/dlkxjz.2022.04.016

地理科学进展 ›› 2022, Vol. 41 ›› Issue (4): 731-740.doi: 10.18306/dlkxjz.2022.04.016

• 研究综述 • 上一篇

高性能并行分布式水文模型研究进展

叶翔宇¹^,²(), 李强¹^,³, 郭禹含¹^,², 梁廖逢¹^,², 王中根¹^,⁴^,^*()

1.中国科学院地理科学与资源研究所陆地水循环及地表过程重点实验室,北京 100101
2.中国科学院大学资源与环境学院,北京 100049
3.青岛大学,山东青岛 266071
4.应急管理部国家自然灾害防治研究院,北京 100085

收稿日期:2021-08-09 修回日期:2021-09-27 出版日期:2022-04-28 发布日期:2022-06-28
通讯作者: *王中根(1973— ),男,河南潢川人,研究员,主要从事水文水资源研究。E-mail: wangzg@igsnrr.ac.cn
作者简介:叶翔宇(1996— ),男,浙江松阳人,博士生,研究方向为高性能水文模型、水文模型参数不确定性分析、山洪风险评估等。E-mail: yexy.18b@igsnrr.ac.cn
基金资助:
第二次青藏高原综合科学考察研究项目(2019QZKK0903);国家重点研发计划项目(2017YFB0203101)

Progress of research on high-performance parallel distributed hydrological model

YE Xiangyu¹^,²(), LI Qiang¹^,³, GUO Yuhan¹^,², LIANG Liaofeng¹^,², WANG Zhonggen¹^,⁴^,^*()

1. Key Laboratory of Water Cycle and Related Land Surface Processes, Institute of Geographic Sciences and Natural Resources Research, CAS, Beijing 100101, China
2. College of Resources and Environment, University of Chinese Academy of Sciences, Beijing 100049, China
3. Qingdao University, Qingdao 266071, Shandong, China
4. National Institute of Natural Hazards, Ministry of Emergency Management of the People's Republic of China, Beijing 100085, China

Received:2021-08-09 Revised:2021-09-27 Online:2022-04-28 Published:2022-06-28
Supported by:
The Second Tibetan Plateau Scientific Expedition and Research Program(2019QZKK0903);National Key Research and Development Program of China(2017YFB0203101)

摘要/Abstract

摘要：

传统分布式水文模型采用串行计算模式,其计算能力无法满足大规模水文精细化、多要素、多过程耦合模拟的需求,亟需并行计算的支持。进入21世纪后,计算机技术的飞速发展和并行环境的逐步完善,为分布式水文模型并行计算提供了软硬件支撑。论文从并行环境、并行算法2个方面对已有研究进行总结概括,分析了不同并行环境和并行算法的优势与不足,并提出提高模型并行效率的手段,即合理分配进程/线程数缩减通信开销,采用混合并行环境增强模型可扩展性,空间或时空离散化提高模型的可并行性及动态分配计算任务、平衡工作负载等。最后,论文对高性能并行分布式模型的未来研究方向进行展望。

关键词: 分布式水文模型, 并行计算, 高性能水文模型, 并行环境, 并行算法

Abstract:

Traditional distributed hydrological models adopt the serial computing mode and the computing power cannot meet the requirements of large-scale refined, multi-element, and multi-process coupling hydrological simulation, thus the support of parallel computing is urgently needed. In the 21st century, the rapid development of computer technology and the gradual improvement of the parallel environment have provided hardware and software support for the parallel computing of distributed hydrological models. This article summarized the existing research from the two aspects of parallel environment and parallel algorithm, analyzed the advantages and disadvantages of different parallel environments and parallel algorithms, and proposed several methods to improve the parallel efficiency of the models, such as rationally allocating the number of processes/threads to reduce communication overhead, adopting a hybrid parallel environment to enhance model scalability, spatial or spatiotemporal discretization to improve model parallelism, and dynamically allocating computing tasks to balance workloads, and so on. Finally, this article examined future research directions of high-performance parallel distributed models.

Key words: distributed hydrological models, parallel computing, high-performance hydrological model, parallel environment, parallel algorithm

叶翔宇, 李强, 郭禹含, 梁廖逢, 王中根. 高性能并行分布式水文模型研究进展[J]. 地理科学进展, 2022, 41(4): 731-740.

YE Xiangyu, LI Qiang, GUO Yuhan, LIANG Liaofeng, WANG Zhonggen. Progress of research on high-performance parallel distributed hydrological model[J]. PROGRESS IN GEOGRAPHY, 2022, 41(4): 731-740.

图/表 4

图1

图2

表1

图3

参考文献 43

[1]	王中根, 刘昌明, 黄友波. SWAT模型的原理、结构及应用研究[J]. 地理科学进展, 2003, 22(1): 79-86.
	[ Wang Zhonggen, Liu Changming, Huang Youbo. The theory of SWAT model and its application in Heihe Basin. Progress in Geography, 2003, 22(1): 79-86. ]
[2]	王中根, 刘昌明, 吴险峰. 基于DEM的分布式水文模型研究综述[J]. 自然资源学报, 2003, 18(2): 168-173.
	[ Wang Zhonggen, Liu Changming, Wu Xianfeng. A review of the studies on distributed hydrological model based on DEM. Journal of Natural Resources, 2003, 18(2): 168-173. ]
[3]	徐宗学, 程磊. 分布式水文模型研究与应用进展[J]. 水利学报, 2010, 41(9): 1009-1017.
	[ Xu Zongxue, Cheng Lei. Progress on studies and applications of the Distributed Hydrological Models. Journal of Hydraulic Engineering, 2010, 41(9): 1009-1017. ]
[4]	Freeze R A, Harlan R L. Blueprint for a physically-based, digitally-simulated hydrologic response model[J]. Journal of Hydrology, 1969, 9(3): 237-258. doi: 10.1016/0022-1694(69)90020-1
[5]	Arnold J G, Srinivasan R, Muttiah R S, et al. Large area hydrologic modeling and assessment, Part 1: Model development[J]. Journal of the American Water Resources Association, 1998, 34(1): 73-89. doi: 10.1111/j.1752-1688.1998.tb05961.x
[6]	Liang X, Lettenmaier D P, Wood E F, et al. A simple hydrologically based model of land-surface water and energy fluxes for General-Circulation Models[J]. Journal of Geophysical Research: Atmospheres, 1994, 99(D7): 14415-14428. doi: 10.1029/94JD00483
[7]	郭方, 刘新仁, 任立良. 以地形为基础的流域水文模型: TOPMODEL及其拓宽应用[J]. 水科学进展, 2000, 11(3): 296-301.
	[ Guo Fang, Liu Xinren, Ren Liliang. A topography based hydrological model: TOPMODEL and its widened application. Advances in Water Science, 2000, 11(3): 296-301. ]
[8]	刘昌明, 王中根, 郑红星, 等. HIMS系统及其定制模型的开发与应用[J]. 中国科学(技术科学), 2008, 38(3): 350-360.
	[ Liu Changming, Wang Zhonggen, Zheng Hongxing, et al. Development and applications of HIMS modeling system. Scientia Sinica (Technologica), 2008, 38(3): 350-360. ]
[9]	刘军志, 朱阿兴, 秦承志, 等. 分布式水文模型的并行计算研究进展[J]. 地理科学进展, 2013, 32(4): 538-547. doi: 10.11820/dlkxjz.2013.04.006
	[ Liu Junzhi, Zhu Axing, Qin Chengzhi, et al. Review on parallel computing of distributed hydrological models. Progress in Geography, 2013, 32(4): 538-547. ] doi: 10.11820/dlkxjz.2013.04.006
[10]	向东, 周祖昊, 袁胜, 等. 基于MPI的分布式水循环模型并行计算性能研究[J]. 水文, 2020, 40(5): 36-40, 27.
	[ Xiang Dong, Zhou Zuhao, Yuan Sheng, et al. Study on the parallel calculating performance of the distributed water cycle model based on MPI. Journal of China Hydrology, 2020, 40(5): 36-40, 27. ]
[11]	Cui Z T, Vieux B E, Neeman H, et al. Parallelisation of a distributed hydrologic model[J]. International Journal of Computer Applications in Technology, 2005, 22(1): 42-52. doi: 10.1504/IJCAT.2005.006802
[12]	李铁键, 刘家宏, 和杨, 等. 集群计算在数字流域模型中的应用[J]. 水科学进展, 2006, 17(6): 841-846.
	[ Li Tie-jian, Liu Jiahong, He Yang, et al. Application of cluster computing in the digital watershed model. Advances in Water Science, 2006, 17(6): 841-846. ]
[13]	Li T J, Wang G Q, Chen J, et al. Dynamic parallelization of hydrological model simulations[J]. Environmental Modelling & Software, 2011, 26(12): 1736-1746.
[14]	Vivoni E R, Mascaro G, Mniszewski S, et al. Real-world hydrologic assessment of a fully-distributed hydrological model in a parallel computing environment[J]. Journal of Hydrology, 2011, 409(1/2): 483-496. doi: 10.1016/j.jhydrol.2011.08.053
[15]	Wang H, Fu X D, Wang G Q, et al. A common parallel computing framework for modeling hydrological processes of river basins[J]. Parallel Computing, 2011, 37(6/7): 302-315. doi: 10.1016/j.parco.2011.05.003
[16]	胡鹏辉. 面向栅格化分布式水文模拟的并行调度方法研究[D]. 南昌: 南昌大学, 2017.
	[ Hu Penghui. Research on parallel scheduling method for rasterized distributed hydrological simulation. Nanchang, China: Nanchang University, 2017. ]
[17]	卢浩, 王少华, 李绍俊, 等. 基于OpenMP的并行化水文分析算法研究与实现[J]. 测绘与空间地理信息, 2013, 36(S1): 7-10.
	[ Lu Hao, Wang Shaohua, Li Shaojun, et al. Research and implementation of hydrologic analysis parallel algorithm based on OpenMP. Geomatics & Spatial Information Technology, 2013, 36(S1): 7-10. ]
[18]	Xu R, Huang X X, Luo L, et al. A new grid-associated algorithm in the distributed hydrological model simulations[J]. Science in China Series E: Technological Sciences, 2010, 53(1): 235-241. doi: 10.1007/s11431-009-0426-4
[19]	Liu J Z, Zhu A X, Liu Y B, et al. A layered approach to parallel computing for spatially distributed hydrological modeling[J]. Environmental Modelling & Software, 2014, 51: 221-227.
[20]	秦泽宁, 黎曙, 周祖昊, 等. 分布式水文模型区域分解并行计算方法及其应用[J]. 水电能源科学, 2020, 38(10): 1-4, 12.
	[ Qin Zening, Li Shu, Zhou Zuhao, et al. Domain decomposition parallel computing method of distributed hydrological model and its application. Water Resources and Power, 2020, 38(10): 1-4, 12. ]
[21]	秦泽宁, 周祖昊, 刘明堂, 等. 分布式水文模型时空离散化并行计算方法研究[J]. 人民黄河, 2020, 42(8): 15-20.
	[ Qin Zening, Zhou Zuhao, Liu Mingtang, et al. Investigation on temporal-spatial discretization parallel computing method for distributed hydrological model. Yellow River, 2020, 42(8): 15-20. ]
[22]	Wang H, Fu X D, Wang Y J, et al. A High-performance temporal-spatial discretization method for the parallel computing of river basins[J]. Computers & Geosciences, 2013, 58: 62-68. doi: 10.1016/j.cageo.2013.04.026
[23]	Le P V V, Kumar P, Valocchi A J, et al. GPU-based high-performance computing for integrated surface-sub-surface flow modeling[J]. Environmental Modelling & Software, 2015, 73: 1-13.
[24]	张涛. 基于CUDA的水文模型并行算法研究[D]. 成都: 电子科技大学, 2015.
	[ Zhang Tao. The research of parallel algorithm based on CUDA for hydrological model. Chengdu, China: University of Electronic Science and Technology of China, 2015. ]
[25]	刘永和, 冯锦明, 徐文鹏. 分布式水文模型的GPU并行化及快速模拟技术[J]. 水文, 2015, 35(4): 20-26.
	[ Liu Yonghe, Feng Jinming, Xu Wenpeng. GPU parallel computing and fast simulation of distributed hydrological models. Journal of China Hydrology, 2015, 35(4): 20-26. ]
[26]	Liu J Z, Zhu A X, Qin C Z, et al. A two-level parallelization method for distributed hydrological models[J]. Environmental Modelling & Software, 2016, 80: 175-184.
[27]	Zhu L J, Liu J Z, Qin C Z, et al. A modular and parallelized watershed modeling framework[J]. Environmental Modelling & Software, 2019, 122: 104526. doi: 10.1016/j.envsoft.2019.104526.
[28]	Apostolopoulos T K, Georgakakos K P. Parallel computation for streamflow prediction with distributed hydrologic models[J]. Journal of Hydrology, 1997, 197: 1-24. doi: 10.1016/S0022-1694(96)03281-7
[29]	Zhang Y, Hou J L, Cao Y P, et al. OpenMP parallelization of a gridded SWAT (SWATG)[J]. Computers & Geosciences, 2017, 109: 228-237. doi: 10.1016/j.cageo.2017.08.002
[30]	向东. 基于WEP-L水循环模型的产流并行计算研究[D]. 郑州: 华北水利水电大学, 2020.
	[ Xiang Dong. Research on parallel computing of runoff generation based on WEP-L water cycle model. Zhengzhou, China: North China University of Water Resources and Electric Power, 2020. ]
[31]	Kollet S J, Maxwell R M. Integrated surface-groundwater flow modeling: A free-surface overland flow boundary condition in a parallel groundwater flow model[J]. Advances in Water Resources, 2006, 29(7): 945-958. doi: 10.1016/j.advwatres.2005.08.006
[32]	苏丹阳. 基于物理概念的水文模型InHM机群并行计算研究[D]. 杭州: 浙江大学, 2012.
	[ Su Danyang. Study on the parallehization of a physics-based hydrologic model InHM utilizing computer cluster. Hangzhou, China: Zhejiang University, 2012. ]
[33]	Hwang H T, Park Y J, Sudicky E A, et al. A parallel computational framework to solve flow and transport in integrated surface-subsurface hydrologic systems[J]. Environmental Modelling & Software, 2014, 61: 39-58.
[34]	Wu R J, Chen X Y, Hammond G, et al. Coupling surface flow with high-performance subsurface reactive flow and transport code PFLOTRAN[J]. Environmental Modelling & Software, 2021, 137: 104959. doi: 10.1016/j.envsoft.2021.104959.
[35]	Li T J, Wang G Q, Chen J. A modified binary tree codification of drainage networks to support complex hydrological models[J]. Computers & Geosciences, 2010, 36(11): 1427-1435. doi: 10.1016/j.cageo.2010.04.009
[36]	Liu J Z, Zhu A X, Qin C Z. Estimation of theoretical maximum speedup ratio for parallel computing of grid-based distributed hydrological models[J]. Computers & Geosciences, 2013, 60: 58-62. doi: 10.1016/j.cageo.2013.04.030
[37]	Wang H, Zhou Y, Fu X D, et al. Maximum speedup ratio curve (MSC) in parallel computing of the binary-tree-based drainage network[J]. Computers & Geosciences, 2012, 38(1): 127-135. doi: 10.1016/j.cageo.2011.05.015
[38]	秦泽宁. 基于WEP-L水文模型的汇流过程并行计算方法研究[D]. 郑州: 华北水利水电大学, 2020.
	[ Qin Zening. Research on parallel computing method of flow routing process based on WEP-L hydrological model[D]. Zhengzhou, China: North China University of Water Resources and Electric Power, 2020. ]
[39]	朱吉生, 黄诗峰, 李纪人, 等. 水文模型尺度问题的若干探讨[J]. 人民黄河, 2015, 37(5): 31-37.
	[ Zhu Jisheng, Huang Shifeng, Li Jiren, et al. Some scale issues in hydrological modes. Yellow River, 2015, 37(5): 31-37. ]
[40]	王纲胜, 夏军, 朱一中, 等. 基于非线性系统理论的分布式水文模型[J]. 水科学进展, 2004, 15(4): 521-525.
	[ Wang Gangsheng, Xia Jun, Zhu Yizhong, et al. Distributed hydrological modeling based on nonlinear system approach. Advances in Water Science, 2004, 15(4): 521-525. ]
[41]	夏军, 王纲胜, 谈戈, 等. 水文非线性系统与分布式时变增益模型[J]. 中国科学(地球科学), 2004, 34(11): 1062-1071.
	[ Xia Jun, Wang Gangsheng, Tan Ge, et al. Hydrological nonlinear system and distributed time-varying gain model. Scientia Sinica (Terrae), 2004, 34(11): 1062-1071. ]
[42]	朱阿兴, 朱良君, 史亚星, 等. 流域系统综合模拟与情景分析: 自然地理综合研究的新范式?[J]. 地理科学进展, 2019, 38(8): 1111-1122. doi: 10.18306/dlkxjz.2019.08.001
	[ Zhu Axing, Zhu Liangjun, Shi Yaxing, et al. Integrated watershed modeling and scenario analysis: A new paradigm for integrated study of physical geography? Progress in Geography, 2019, 38(8): 1111-1122. ] doi: 10.18306/dlkxjz.2019.08.001
[43]	Guo Y H, Zhang Y Q, Zhang L, et al. Regionalization of hydrological modeling for predicting streamflow in ungauged catchments: A comprehensive review[J]. Wiley Interdisciplinary Reviews: Water, 2021, 8(1): e1487. doi: 10.1002/wat2.1487.

并行算法离散化方式	模型类型	文献来源	并行环境	模拟单元	单元数	最大加速比/进程(线程)
空间离散化	计算独立型	Cui等, 2005^[11]	MPI	网格	533	7/15进程
	顺序依赖型	Apostolopoulos 等, 1997^[28]	Encore	子流域	9	1.5/9线程
		Xu 等, 2010^[18]	OpenMP	子流域	288	2.42/4 线程
		Vivoni 等, 2011^[14]	MPI	子流域	5707	60~80/ 128~512 进程
		Li 等, 2011^[13]	MPI	子流域	4912	5.34/13 进程
		Wang等, 2011^[15]	MPI	子流域	501	7.8/10 进程
		Liu 等, 2014^[19]	OpenMP	子流域/栅格	115个子流域, 2381602个栅格	12.49/24线程
		Liu 等, 2016^[26]	Hybrid	子流域/栅格	67个子流域, 2381602个栅格	35/8进程 (48线程)
		Zhang 等, 2017^[29]	OpenMP	子流域	97240	8.6/15线程
		Zhu 等, 2019^[27]	Hybrid	子流域/栅格	17个子流域,53900 个栅格	20/10进程 (40线程)
		秦泽宁等, 2020^[20]	OpenMP	子流域	8485	2.3/4线程
		向东, 2020^[30]	MPI/ OpenMP/CUDA	子流域	8485	MPI: 4.8/10进程 OpenMP: 7.8/10线程 CUDA: ≈3/256线程
	紧密耦合型	Kollet 等, 2006^[31]	MPI	有限差分单元	5000000	82/100 进程
		苏丹阳, 2012^[32]	MPI	子流域/栅格	110个子流域, 270816个栅格	≈4/4进程
		Hwang 等, 2014^[33]	OpenMP	有限元	25000	7.1/8进程
		Le 等, 2015^[23]	CUDA	有限元	3.5亿	-
		Wu 等, 2021^[34]	MPI	有限体积	10008005	地面径流: ≈64/64进程地下径流: ≈512/512进程
时空离散化	顺序依赖型	Wang等, 2013^[22]	MPI	子流域	1869	15.04/24进程
时空离散化	顺序依赖型	秦泽宁等, 2020^[21]	OpenMP	子流域	8485	8.17/20线程

高性能并行分布式水文模型研究进展

Progress of research on high-performance parallel distributed hydrological model

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 43

相关文章 13

Metrics

本文评价

推荐阅读 0

[1]	郭禹含, 王中根, 伍玉良. 多源再分析降水数据在拉萨河流域应用对比研究[J]. 地理科学进展, 2017, 36(8): 1033-1039.
[2]	王晓蕾, 孙林, 张宜清, 罗毅. 中亚阿姆河上游产流过程特征研究[J]. 地理科学进展, 2015, 34(3): 364-372.
[3]	江净超, 朱阿兴, 秦承志, 刘军志, 陈腊娇, 吴辉. 分布式水文模型软件系统研究综述[J]. 地理科学进展, 2014, 33(8): 1090-1100.
[4]	刘军志, 朱阿兴, 秦承志, 陈腊娇, 吴辉, 江净超. 分布式水文模型的并行计算研究进展[J]. 地理科学进展, 2013, 32(4): 538-547.
[5]	赵斯思, 周成虎. GPU加速的多边形叠加分析[J]. 地理科学进展, 2013, 32(1): 114-120.
[6]	王中根,朱新军,夏军,李建新. 海河流域分布式SWAT模型的构建[J]. 地理科学进展, 2008, 27(4): 1-6.
[7]	张奇. 湖泊集水域地表—地下径流联合模拟[J]. 地理科学进展, 2007, 26(5): 1-10.
[8]	池宸星,郝振纯, 王玲, 胡健伟. 黄土区人类活动影响下的产汇流模拟研究[J]. 地理科学进展, 2005, 24(3): 101-108.
[9]	刘昌明, 李道峰, 田英, 郝芳华, 杨桂莲. 基于DEM的分布式水文模型在大尺度流域应用研究[J]. 地理科学进展, 2003, 22(5): 437-445.
[10]	王中根, 刘昌明, 黄友波. SWAT模型的原理、结构及应用研究[J]. 地理科学进展, 2003, 22(1): 79-86.
[11]	王中根, 刘昌明, 左其亭, 刘青娥. 基于DEM的分布式水文模型构建方法[J]. 地理科学进展, 2002, 21(5): 430-439.
[12]	万洪涛, 周成虎, 万庆. 流域水文模型计算域离散方法[J]. 地理科学进展, 2001, 20(4): 347-354.
[13]	王中根,朱新军夏军,李建新. 海河流域分布式SWAT 模型的构建[J]. 地理科学进展, 0, 0(): 1-6.