ONLINE -- MSA Seminar: More science with fewer bits – Mixed-Precision Computing with Tensor Cores

Modular Supercomputing Architectures Seminar

Speakers: Julien Demouth and Mathias Wagner, NVIDIA

Scientific HPC applications often require high accuracy and hence are carried out in fp64. However, this comes at the cost of higher memory traffic, and often the compute throughput is also reduced compared to lower precision. Using clever error correction methods it can be shown that using mixed precision approaches accelerates HPC simulations significantly and still yields fp64 accuracy. We discuss these methods for bandwidth and compute-bound problems. In the latter case, modern GPUs such NVIDIA V100 and A100 can often exploit their Tensor Core units which provide a performance boost beyond the reduction of the data type width. This also drives the use of lower precision in Deep Learning which is getting more and more popular. We present a few recent results obtained on NVIDIA V100 and A100 with techniques that take advantage of it. Those examples make extensive use of the Tensor Core units, as well.

This seminar is offered online via BigBlueButton at https://webconf.fz-juelich.de/b/and-2jh-qj4 .

In case of difficulties with BigBlueButton, the seminar will be switched to DFNconf streaming at https://www.conf.dfn.de/stream/nr5hm44jym4c4