Heat
Heat provides highly optimized algorithms and data structures for tensor computations using CPUs, GPUs and distributed cluster systems on top of MPI. Heat builds on PyTorch and mpi4py to provide high-performance computing infrastructure for memory-intensive applications within the NumPy/SciPy ecosystem, e.g. to run NumPy/SciPy code on GPUs (CUDA, ROCm, coming up: Apple MPS). Heat’s generic Python-first programming interface integrates seamlessly with the existing data science ecosystem and makes it as effortless as using NumPy to write scalable scientific and data science applications.
![Figure taken from [1]: Distribution of a 3D array across three processes: (a), Array is not distributed, i.e., split=None, each process has access to the full data; split=0, split=1, and split=2: Array is distributed along axis 0, 1 or 2.](/en/ias/jsc/about-us/structure/simulation-and-data-labs/sdl-neuroscience/software/heat/heat_1.png/@@images/image/great)
In addition, Heat offers deep learning tools specifically designed for HPC systems, including parallel data loading, in-memory data redistribution, parallelization strategies, and adjoint MPI operations to enable automatic differentiation across multiple HPC nodes.
More information
Website: https://heat.readthedocs.io/en/stable/
RSD: https://helmholtz.software/software/heat
GitHub: https://github.com/helmholtz-analytics/heat
References
[1] Götz, M., Debus, C., Coquelin, D., Krajsek, K., Comito, C., Knechtges, P., Hagemeier, B., Tarnawa, M., Hanselmann, S., Siggel, S., Basermann, A. & Streit, A. (2020). HeAT - a Distributed and GPU-accelerated Tensor Framework for Data Analytics. In 2020 IEEE International Conference on Big Data (Big Data) (pp. 276-287). IEEE, DOI: 10.1109/BigData50022.2020.9378050.
Our collaboration partners
ATML Data Management and Analytics@JSC
High-Performance Computing Department@DLR
Department Data Analytics, Access and Applications@KIT
