Skip to Main content Skip to Navigation
Journal articles

A generalized massively parallel ultra-high order FFT-based Maxwell solver

Haithem Kallala 1 Jean-Luc Vay 2 Henri Vincenti 3
3 PHI - Physique à Haute Intensité
IRAMIS - Institut Rayonnement Matière de Saclay, LIDyl - Laboratoire Interactions, Dynamiques et Lasers (ex SPAM)
Abstract : Dispersion-free ultra-high order FFT-based Maxwell solvers have recently proven to be paramountto a large range of applications, including the high-fidelity modeling of high-intensity laser–matterinteractions with Particle-In-Cell (PIC) codes. To enable a massively parallel scaling of these solvers,a novel parallelization technique was recently proposed, which consists in splitting the simulationdomain into several processor sub-domains, with guard regions appended at each sub-domain bound-ary. Maxwell’s equations are advanced independently on each sub-domain using local shared-memoryFFTs (instead of a single distributed global FFT). This implies small truncation errors at sub-domainboundaries, the amplitude of which depends on guard regions sizes and order of the Maxwell solver.For moderate guard region sizes, this ’local’ technique proved to be highly scalable on up to a millioncores and notably enabled the 3D modeling of so-called plasma mirrors, for which 8 guard cells onlywere enough to prevent truncation error growth. Yet, for other applications, the required number ofguard cells might be much higher, which would severely limit the parallel efficiency of this techniquedue to the large volume of guard cells to be exchanged between sub-domains. In this context, wepropose a novel parallelization technique that ensures very good scaling of FFT-based solvers withan arbitrarily high number of guard cells. Our ’hybrid’ technique consists in performing distributedFFTs on local groups of processors with guard regions now appended to boundaries of each group ofprocessors. It uses a dual domain decomposition method for the Maxwell solver and other parts ofthe PIC cycle to keep the simulation load-balanced. This ’hybrid’ technique was implemented in theopen source exascale library PICSAR. Benchmarks show that for a large number of guard cells (>16),the ’hybrid’ technique offers up to×3 speed-up and×8 memory savings compared to the ’local’ one.
Complete list of metadatas
Contributor : Caroline Lebe <>
Submitted on : Wednesday, September 4, 2019 - 2:17:19 PM
Last modification on : Friday, June 5, 2020 - 4:03:10 AM

Links full text



Haithem Kallala, Jean-Luc Vay, Henri Vincenti. A generalized massively parallel ultra-high order FFT-based Maxwell solver. Computer Physics Communications, Elsevier, 2019, 244, pp.25-34. ⟨10.1016/j.cpc.2019.07.009⟩. ⟨cea-02278490⟩



Record views