Interpolation oriented parallel communication to optimize coupling in earth system modeling

Abstract

Complicated global climate problems trigger researchers from different scientific disciplines to link multiphysics simulations called models for integrated modeling of climate changes by using a software framework called earth system modeling (ESM). As its critical component, coupler is in charge of connections and interactions among models. With the advance of next-generation models, greater data transfer volume and higher coupling frequency are expected to put heavy performance burden on coupler. High efficient coupling techniques are required. In this paper, we propose the sub-domain mapping method to improve the parallel coupling consisted of data transfer and data transformation. By using one specific interpolation oriented communication routing, the communication operations that are originally decentralized in various steps can be combined together for execution. This can reduce the redundant communications and the entailed synchronization costs. The tests on the Tianhe-1A (TH-1A) supercomputer show that our method can achieve 1.1 to 4.9 fold performance improvements. We also present further optimization solution for the multi-interpolation cases. The test results show that our method can achieve up to 3.4 fold speedup over the original coupling execution of the current climate system.

Keywords

coupler / communication optimization / coupling performance / ESM

References

[1] Dunlap R, Rugaber S, Mark L. A feature model of coupling technologies for earth system models. Computers & Geosciences, 2013, 53: 13-20

[2] Valcke S, Balaji V, Craig A, Deluca C, Dunlap R, Ford R W, Jacob R, Larson J, O’Kuinghttons R, Riley G D, Vertenstein M. Coupling technologies for earth system modelling. Geoscientific Model Development Discussions, 2012, 5(3): 1987-2006

[3] Bao Q, Lin P, Zhou T, Liu Y, Yu Y, Wu G, He B, He J, Li L, Li J, Li Y, Liu H, Qiao F, Song Z, Wang B, Wang J, Wang P, Wang X, Wang Z, Wu B, Wu T, Xu Y, Yu H, Zhao W, Zheng W, Zhou L. The flexible global ocean-atmosphere-land system model, spectral version 2: FGOALS-s2. Advances in Atmospheric Sciences, 2013, 30: 561-576

[4] Craig A P, Jacob R, Kauffman B, Bettge T, Larson J, Ong E, Ding C, He Y. CPL6: the new extensible, high performance parallel coupler for the community climate system model. International Journal of High Performance Computing Applications, 2005, 19(3): 309-327

[5] Redler R, Valcke S, Ritzdorf H. OASIS4-a coupling software for next generation earth system modelling. Geoscientific Model Development, 2010, 3(1): 87-104

[6] Jacob R, Larson J, Ong E. M × N communication and parallel interpolation in community climate system model version 3 using the model coupling toolkit. International Journal of High Performance Computing Applications, 2005, 19(3): 293-307

[7] Collins W D, Bitz C M, Blackmon M L, Bonan G B, Bretherton C S, Carton J A, Chang P, Doney S C, Hack J J, Henderson T B, Kiehl J T, Large W G, McKenna D S, Santer B D, Smith R D. The community climate system model version 3. Journal of Climate, 2006, 19(11): 2122-2143

[8] Yang X J, Liao X K, Lu K, Hu Q F, Song J Q, Su J S. The TianHe-1A supercomputer: its hardware and software. Journal of Computer Science and Technology, 2011, 26(3): 344-351

[9] Gent P R, Danabasoglu G, Donner L J, Holland M M, Hunke M M, Hunke E C, Jayne S R, Lawrence D M, Neale R B, Rasch P J, Vertenstein M, Worley P H, Yang Z L, Zhang M. The community climate system model version 4. Journal of Climate, 2011, 24(19): 4973-4991

[10] Dennis J M, Vertenstein M, Worley P H, Mirin A A, Craig A P, Jacob R, Mickelson S. Computational performance of ultra-high-resolution capability in the community earth system model. International Journal of High Performance Computing Applications, 2012, 26(1): 5-16

[11] Vertenstein M, Craig T, Middleton A, Feddema D, Fischer C. Community earth system model 1.0.4 User’s Guide. 2011, 1-145

[12] Craig A P, Vertenstein M, Jacob R. A new flexible coupler for earth system modeling developed for community climate system model 4 and community earth system model 1. International Journal of High Performance Computing Applications, 2012, 26(1): 31-42

[13] Valcke S. The OASIS3 coupler: a European climate modelling community software, Geosci. Model Dev. Discuss, 5: 2139-2178

[14] Hinton A, Kwiatkowska M, Norman G, Parker D. PRISM: a tool for automatic verification of probabilistic systems. Tools and Algorithms for the Construction and Analysis of Systems. Springer Berlin Heidelberg, 2006: 441-444

[15] Larson J, Jacob R, Ong E. Themodel coupling toolkit: a new Fortran90 toolkit for building multiphysics parallel coupled models. International Journal of High Performance Computing Applications, 2005, 19(3): 277-292

[16] Balaji V. The FMS manual: a developer’s guide to the GFDL flexible modeling system. 2002

[17] Balaji V, Anderson J, Held I, Winton M, Malyshev S, Stouffer R. The exchange grid: a mechanism for data exchange between earth system components on independent grids. In: Proceedings of the 2005 International Conference on Parallel Computational Fluid Dynamics. 2006, 1-18

[18] Hill C, DeLuca C, Suarez M, Silva A D. The architecture of the earth system modeling framework. Computing in Science & Engineering, 2004, 6(1): 18-28

[19] Collins N, Theurich G, Deluca C, Suarez M, Trayanov A, Balaji V, Li P, Yang W, Hill C, Silva A D. Design and implementation of components in the earth system modeling framework. International Journal of High Performance Computing Applications, 2005, 19(3): 341-350

[20] Yoshimura H, Yukimoto S. Development of a simple coupler (Scup) for earth system modeling. Pap Meteor Geophys, 2008, 59: 19-29

[21] Arakawa T, Yoshimura H, Saito F, Ogochi K. Data exchange algorithm and software design of KAKUSHIN coupler Jcup. Procedia Computer Science, 2011, 4: 1516-1525

[22] Ford R W, Riley G D, Bane M K, Armstrong C W, Freeman T L. GCF: a general coupling framework. Concurrency and Computation: Practice and Experience, 2006, 18(2): 163-181

[23] Armstrong C W, Ford R W, Riley G D. Coupling integrated earth system model components with bfg2. Concurrency and Computation: Practice and Experience, 2009, 21(6): 767-791

[24] Balaji V, Boville B, Cheung S, Clune T, Collins N, Craig T, Cruz C, Silva A D, Deluca C, Fainchtein R D, Eaton B, Hallberg B, Handerson T, Hill C, Iredell M, Jacob R, Jones P, Kluzek E, Kauffman B, Larson J, Li P, Liu F, Michalakes J, Murphy S, Neckels D, Kuinghttons R O, Oehmke B, Panaccione C, Rosinski J, Sawyer W, Sch wab E, Smithline S, Spector W, Stark D, Suarez M, Swift S, Theurich G, Trayanoy A, Vasquez S, Wolfe J, Yang W, Young M, Zaslavsky L. ESMF User Guide Version 3.1. Earth System Modeling Framework, 2009

RIGHTS & PERMISSIONS

2014 Higher Education Press and Springer-Verlag Berlin Heidelberg