Fine-grained MPI+OpenMP plasma simulations: communication overlap with dependent tasks

Jérôme Richard (1, 2) , Guillaume Latu (1) , Julien Bigot (2) , Thierry Gautier (3)

1 IRFM - Institut de Recherche sur la Fusion par confinement Magnétique
2 MDLS - Maison de la Simulation
3 AVALON - Algorithms and Software Architectures for Distributed and HPC Platforms

This paper demonstrates how OpenMP 4.5 tasks can be used to efficiently overlap computations and MPI communications based on a case-study conducted on multi-core and many-core architectures. It focuses on task granularity, dependencies and priorities, and also identifies some limitations of OpenMP. Results on 64 Skylake nodes show that while 64% of the wall-clock time is spent in MPI communications, 60% of the cores are busy in computations, which is a good result. Indeed, the chosen dataset is small enough to be a challenging case in terms of overlap and thus useful to assess worst-case scenarios in future simulations. Two key features were identified: by using task priority we improved the performance by 5.7% (mainly due to an improved overlap), and with recursive tasks we shortened the execution time by 9.7%. We also illustrate the need to have access to tools for task tracing and task visualization. These tools allowed a fine understanding and a performance increase for this task-based OpenMP+MPI code.

Keywords: Keywords: Dependent tasks; OpenMP 4.5; MPI; Many-core

Domains: Computer Science [cs]; Distributed, Parallel, and Cluster Computing [cs.DC]; Computer Science [cs]; Performance [cs.PF]; Physics [physics]; Physics [physics]; Plasma Physics [physics.plasm-ph]

Jérôme Richard, Guillaume Latu, Julien Bigot, Thierry Gautier. Fine-grained MPI+OpenMP plasma simulations: communication overlap with dependent tasks. Lecture Notes in Computer Science, 2019, Euro-Par 2019: Euro-Par 2019: Parallel Processing, 11725, pp.419-433. ⟨10.1007/978-3-030-29400-7_30⟩. ⟨cea-02404825⟩