Push it to the Limit - Researching Thermal Convection Using Record Simulations
Knowledge of the mechanisms and processes of heat transfer is a prerequisite for solving many current challenges, e.g. optimizing cooling systems or understanding heat exchange processes in the atmosphere. Simulations and visualisations help to understand these processes, identify previously unknown characteristics, and optimise applications. The complexity and data volume of these simulations present a unique demand for High-Performance Computing (HPC), pushing systems and codes to new performance peaks.
Turbulent heat transfer processes
Our colleagues Mathis Bode and Jens Henrik Göbbert, together with Jörg Schumacher and Roshan Samuel from Technische Universität Ilmenau, Katepalli Sreenivasan from New York University, and Janet Scheel from Occidental College, have taken up this challenge and investigated how data on turbulent heat transfer processes can be generated and analysed. Their work focuses on a Rayleigh-Bénard Convection (RBC) setup which consists of a horizontal layer of fluid confined between two plates. While the lower plate is kept at a higher temperature, the upper plate is colder, creating a thermal exchange (see Fig. 1).
The type of heat transfer is described by the Rayleigh number. If it exceeds a certain point, the classic heat conduction dominated by a convective flow. The resulting turbulence is extremely dynamic and requires high-frequency and complex data collection, analysis, and visualisation.
A challenge for simulations
Traditional flow applications are extremely computationally intensive, as the coupled scales in turbulent flows, potential multiphysics effects, and numerical difficulties must be handled computationally. One of the top 25 supercomputers worldwide, the JUWELS Booster at the Jülich Supercomputing Centre (JSC), was pushed to its absolute peak performance for this simulation. 840 nodes with 3360 GPUs, which is about 90% of all booster nodes of JUWELS, were used in parallel with the NekRS code over a total period of about 24 hours to accomplish this task, already starting with the converged flow fields from smaller Rayleigh numbers as an initial solution.
Record simulations as a model for exascale
The simulations with Rayleigh numbers up to 1012 not only provided a deeper understanding and new insights into the turbulent heat transfer in the RBC simulation field but also made some properties and mechanisms perceptible and visualisable in the first place (see Fig. 2). Among other results, the simulation data have shown that fluctuations in the boundary layers play a key role in these dynamics. Regarding the similarities between the systems of the JEWELS Booster and the expected JUPITER Booster module, the performed work flow can be seen as a model for NekRS runs on exascale level with JUPITER.
Further information:
Paper: R. J. Samuel, M. Bode, J. D. Scheel, K. R. Sreenivasan and J. Schumacher, Boundary regions in plane thermal convection layers are fluctuation-dominated, accepted for publication in Journal of Fluid Mechanics, available on arXiv as https://arxiv.org/pdf/2403.12877.