一种适用于有限差分模式的负载平衡区域分解方法

分布式内存并行处理在数值天气预报等超大规模科学计算中已经得到了广泛的应用。中尺度模式由于分辨率高 ,计算量大 ,需使用更多的处理机进行并行运算。另一方面 ,由于复杂的物理过程的采用 ,增加了不同天气的计算量的不平衡。但是 ,目前所广泛使用的并行处理方法在处理机数量较多时不能很好地均衡计算负载 ,引起并行计算效率的降低。本文提出了一种新的非规则区域分解负载分配方法。并与已有的负载分配方法进行了分析试验对比 ,该方法能更有效地平衡负载 ,取得更好的加速效果

A LOAD BALANCING DOMAIN DECOMPOSITION METHOD FOR FINITE DIFFERENCE NUMERICAL WEATHER PREDICTION MODELS

Hundreds to thousands of nodes, which can reach T flops, compose modern parallel computers. But programming on such kind of system is difficult. A very important issue is load balancing. The more the nodes in the system, the more difficult to balance the load. Domain decomposition is common technique in parallel processing of mesoscale weather prediction models. The different columns of the model are distributed on different nodes. One can expect to increase the speedup the model by increase the resolution of model. However, as the resolution of the model is increased, the grids of the model and the steps of iteration are increase. More nodes are needed if we want the model can be finished in the same periods of time. As results, less columns running on each node. A little of unbalance of the load can be a serious problem on highly paralleled models. At the same time, the physical process of higher resolution model can be more complex, which results more unbalance among processors. Many models use regular east west north south domain decomposition technique use n by m nodes, ignoring the load balancing problem completely, The advantage is its simple and the communication between processors is low. It is success if the grids and the number of the nodes is highly compatible and the physics is not very complex. However, when the grid points of the model is not highly compatible with the number of the nodes, which is often the case in very dynamic environments. For example, one processor has one role and column of grid points than others, the processor with more grids slow down very other processors as though the load on each grid point is the same, and it can be more serious if the load of each grid points is very different becaus the physics of the model is very different under different weather, or on different land surface. Some researches show that the speedup of the model goes down rapidly after the model runs several hours when the microphysics is turn on. The solution is using adjustable domain to catch up the variation of the load. Some researchers use adjustable rectangle domain, which is better than the fixed domain. But our results shows a nearly rectangle domain, adding a few steps on some sides of the rectangle domain, can balance the load more better than the rectangle domain with only a little increase of communication due to the steps. The result shows the algorithm to partition the grid points of the forecast area and compare it with three other methods, one is fixed rectangles domain method and others are two adjustable rectangles domain method. In order to analysis the effects of the method three other method was also tested. In the experiment, a distribution of the computation load for each grid point is given, four methods were applied and the load unbalance can be calculated for each method, which shows that our method is the best one in balancing load. This method is applied to a three dimension diffusion equation model to test the feasibility in and real model. The experiment was on IBM SP machine. In order to test its ability to balance the load with sharp different of computation between grid points, a simulated physics process was added in the model. The computation time was measured for every grid point and it was used as the load of the grid points. The result is identical with previous one with only the difference that the speedup is lower due to the communication.