HPST Enters Full Production

The High Performance Storage Tier (HPST) has entered full production and is opening access to more projects via JUDOOR. It is available as $CSCRATCH. Acting as an NVMe-based cache layer on top of JUST, it offers faster access to data stored in the SCRATCH file system. This added storage sub-system further closes the gap between the GPFS file system on one hand and the compute systems on the other. Data analysis and I/O-intensive applications, in particular read-intensive AI workloads, can greatly benefit from staging data to a faster storage tier.

To achieve high bandwidth and low latency, the HPST is directly integrated into the InfiniBand fabric of all three main systems – JUWELS, JURECA, and JUSUF – allowing for a peak bandwidth of up to 2 TB/s. To minimize the effort required by applications to utilize the system, the underlying 2 PB of SSD storage space can be directly accessed using POSIX. However, for maximum performance we recommend using the ParaStationMPI MPI-IO or the native IME interface.

Prior to launching their respective jobs on a compute system, users can use the available IME tools to stage data from the back-end SCRATCH file system to the HPST and then synchronize it back once the job has finished. Using this strategy, some applications have observed an I/O time reduction of up to 5x during job execution. The support team checks the I/O pattern of applications requesting access to the HPST to maximize the overall system benefit. Respective candidates are assigned a quota on the HPST; when this quota is exceeded, data is automatically evicted, allowing for parallel use by multiple applications without interference between users. The HPST system was supplied by HPE and DDN, and runs DDN’s Infinite Memory Engine (IME). The procurement is part of the Fenix e-infrastructure within the scope of the EU project ICEI.

Contact: Dr. Salem El Sayed Mohammed

from JSC News No. 288, 29 April 2022