The Gaia AVU-GSR solver: a CPU + GPU parallel code toward Exascale systems

27 Sept 2024

Speaker: Valentina Cesare (Istituto Nazionale di Astrofisica (INAF) - Osservatorio Astrofisico di Catania)

Abstract:

The solver module of the Astrometric Verification Unit-Global Sphere Reconstruction (AVU-GSR) pipeline aims to find the astrometric parameters of ∼108 stars in the Milky Way, besides the attitude and instrumental settings of the Gaia satellite and the parametrized post Newtonian parameter gamma with a resolution of 10-100 micro-arcseconds. To perform this task, the code solves a system of linear equations with the iterative LSQR algorithm, where the coefficient matrix is large (10-50 TB) and sparse and the iterations stop when convergence is reached in the least square sense. The two matrix-by-vector products performed at each LSQR step were GPU-ported, firstly with OpenACC and then with CUDA, resulting in a 1.5x and 14x speedup, respectively, over an original code version entirely parallelized on the CPU with MPI + OpenMP. The CUDA code was further optimized and then ported with portable programming frameworks, obtaining a further 2x acceleration factor. One critical section of the code consists in the computation of covariances, whose total number is Nunk x (Nunk - 1)/2 and occupy ~1 EB, being Nunk ∼5×108 the total number of unknowns. This “Big Data” issue cannot be faced with standard approaches: we defined an I/O based pipeline made of two concurrently launched jobs, where one job, i.e. the LSQR, writes the files and the second job reads them, iteratively computes the covariances and deletes them. The pipeline does not present significant bottlenecks until a number of covariances elements equal to ∼8×106. The code currently runs in production on Leonardo CINECA infrastructure.