Supercomputer Made in Jülich Setting New Benchmarks
Forschungszentrum Jülich and its partners have built the most powerful supercomputer in Europe with unparalleled flexibility and energy efficiency.
Jülich, 16 November 2020 – Forschungszentrum Jülich’s upgraded supercomputer JUWELS is now capable of 85 petaflops. This is equivalent to 85 quadrillion computing operations per second or the computing power of more than 300,000 modern PCs. Thanks to its new booster module, JUWELS is able to massively expand the application limits of simulations and also offers the strongest platform in Europe for the use of artificial intelligence (AI). Developed by Forschungszentrum Jülich, Atos, a global leader in digital transformation headquartered in France, the Munich-based supercomputing specialist ParTec, and the accelerated computing platform company NVIDIA, JUWELS is the fastest computer in Europe.
It currently ranks number 7 on the TOP500 list of the world's fastest computers published today. The Jülich supercomputer financed by the national Gauss Centre for Supercomputing takes third place in the current Green500 and is the world’s most energy-efficient supercomputer in the highest performance class.
Video: Flexible supercomputer made in Jülich (Duration: 4:01 min)
“We see supercomputing not only as the subject of our research, but above all as a powerful tool that helps us to address complex research topics together with our partners from science and industry,” said Prof. Wolfgang Marquardt, Chairman of the Board of Directors of Forschungszentrum Jülich.
“With its fully expanded JUWELS system, Forschungszentrum Jülich provides scientists from a wide range of institutions and scientific disciplines with access to supercomputing capacities of the highest level. At the same time, however, the system also demonstrates our responsible action with respect to the ever increasing energy demand for the provision of computing power.”
Bridging the gap between simulations and reality
“A very recent example in the current COVID-19 crisis is supporting simulations for drug development," said Prof. Thomas Lippert, head of the Jülich Supercomputing Centre (JSC). "Only the computing power of the booster enables our researchers to simulate the processes before, during and after a potential drug meets a receptor or protein realistically enough."
Another current project is the detailed simulation of surface, earth, and groundwater movements. With the new JUWELS booster module, researchers are for the first time able to perform simulations with the required fine resolution, depicting details such as individual slopes or river corridors.
Early Access Applications
Computing time on the JUWELS booster is precious. Already during the first test phase in late summer and autumn of this year, the first applications ran on the system. The so-called "Early Access Applications" provide useful information for code optimization and first scientific results.
Intelligent task sharing – highest energy efficiency
JUWELS is based on a highly flexible modular architecture developed by Forschungszentrum Jülich together with European and international partners. “With its powerful, highly efficient graphics processors, the booster module is designed for extremely computationally intensive applications that can be easily processed in parallel on a large number of computing cores,” said Dr. Dorian Krause, who is responsible for setting up and operating the extraordinarily complex system at Jülich. “JUWELS is also the leader in terms of energy efficiency among the top 10 fastest computers in the world.”
JUWELS is one of the first supercomputers worldwide using NVIDIA A100 Tensor Core GPUs, based on the NVIDIA Ampere architecture. The booster unites around 12 million CUDA cores (FP64) across its more than 3,700 graphics processors, connected via an NVIDIA Mellanox HDR 200Gb/s InfiniBand high-performance network. The booster alone reaches a peak performance of 73 petaflops. With nearly 2.5 exaflops of peak AI performance, i.e. 2.5 trillion computing operations per second, it offers the strongest platform in Europe for the use of artificial intelligence (AI).
“The key highlight of JUWELS is that both modules – the previous “cluster module”, which works with fast processors (CPUs), and the booster module with its GPUs – are very tightly interconnected,” says Bernhard Frohwitter, CEO of the Munich-based supercomputing specialist ParTec. The interaction of the modules controls ParTec's modular software system ParaStation Modulo, a world-leading development from Germany. “With ParaStation Modulo, JUWELS can dynamically access CPUs and GPUs within a code at will and thus optimize the calculation.”
“Both modules come from Atos, based on our BullSequana X infrastructure, whose 100% highly-efficient water-cooled patented DLC (Direct Liquid Cooling) solution contributes significantly to the low energy consumption of the system.” explains Agnès Boudot, Senior Vice President, Head of HPC & Quantum at Atos. “Atos’ design ensures that the fullest computational power issued from CPU and GPU blades is translated into users’ applications.”
Prepared for future technologies
Prof. Thomas Lippert sees the JUWELS system also as a milestone on the path towards the European exascale computer, which is set to be launched in 2023. The construction and operation of such a supercomputer is regarded throughout the world as the next major step in supercomputing. With a computing power of at least one exaflops, i.e. 1 trillion double precision floating point operations per second, it would be at least 12 times faster than the JUWELS booster.
“JUWELS' modular architecture, the design of its computing nodes, its network, infrastructure, and cooling system as well as its software architecture can be transferred to an exascale computer in the next years, while keeping costs and energy consumption at a reasonable level,” said Thomas Lippert. As far as the future of supercomputing is concerned, thanks to its modular design, JUWELS is perfectly prepared to integrate future technologies being researched at Forschungszentrum Jülich — for example quantum computer modules or neuromorphic modules that work on the model of the human brain.
Funded by federal and state governments
The acquisition of the booster is being funded by the federal government and the state of North Rhine-Westphalia. JSC operates JUWELS as a member of the Gauss Centre for Supercomputing (GCS), the association of Germany’s national supercomputing centers: three computing centers of Forschungszentrum Jülich (JSC), the Bavarian Academy of Sciences (LRZ) and the University of Stuttgart (HLRS).
Computing time on JUWELS is allocated on a national and European level after application and scientific evaluation. GCS and Forschungszentrum Jülich are supported by the Federal Ministry of Education and Research (BMBF) and the Ministry of Culture and Science of the State of North Rhine-Westphalia as well as the Ministry of Science, Research and the Arts of Baden-Württemberg and the Bavarian State Ministry of Education, Science and the Arts.
The modular concept realized at the Jülich Supercomputing Centre (JSC) was developed as part of a long-term cooperation with the Munich software company ParTec. It provides a platform for a supercomputer consisting of several specialized modules that can be dynamically combined as required using uniform software. Since 2011, European partners from industry and research have been developing and testing the first modular design systems under Jülich leadership, and the concept has been continuously expanded in the EU-funded Deep research projects.
The booster is the result of a collaboration between the experts at JSC and the supercomputer manufacturer Atos (France), the supercomputing specialist ParTec (Germany), and the accelerated computing and networking platform company, NVIDIA (USA).
12 petaflops (12 quadrillion computing operations per second)
12 petaflops (12 quadrillion computing operations per second)
2511 CPU nodes + 56 GPU nodes
936 GPU nodes
total of 5134 CPUs (Intel Xeon Skylake) + total of 224 GPUs (NVIDIA V100)
total of 1872 CPUs (AMD EPYC Rome) + total of 3744 GPUs (NVIDIA A100)
122,768 CPU cores + 71.680 FP64 CUDA cores (GPUs in total)
44,928 CPU cores + 12,939,264 FP64 CUDA cores (GPUs in total)
total of 275 TB
total of 479 TB + total of 150 TB High Bandwidth Memory
100 Gb/s (Mellanox InfiniBand EDR)
200 Gb/s (NVIDIA Mellanox HDR InfiniBand)
Supercomputer for Highly Complex Calculations and AI
JUWELS made huge gains in computing performance thanks to a new booster module. Researchers from all over Europe have been adapting and developing programs over recent months so that they run massively parallel on the more than 3,700 graphics processors (GPUs) in the booster.
Images may be used for editorial purposes provided credit is given.
Prof. Dr. Dr. Thomas Lippert
Director of the Jülich Supercomputing Centre
Tel: +49 2461 61-6402
Dr. Dorian Krause
Jülich Supercomputing Centre
Tel: 02461 61-3631
Phone: +49 2461 61-4771