Neuromorphic Architectures for Accelerated Neural Network Simulations
Background and motivation
The critical biological ingredients and underlying principles of brain functions are still widely unknown. Research on neural network dynamics and function is to a large extent based on simulations on traditional von Neumann computer architectures, in which memory and computation are separated. The resultant performance limitation of such systems is known as the von Neumann bottleneck. Even with today's advanced supercomputers, simulations of large-scale neural networks cannot be significantly accelerated, hindering the study of slow biological processes such as learning and development. Accelerated processing at a lower cost while maintaining flexibility, accuracy, and reproducibility of simulation results is the current challenge that a neuromorphic computing system has to cope with to serve as a research platform for neuroscience. This trade-off between flexibility and efficiency requires novel system architecture approaches beyond the von Neumann model.
Previous work
Strong impulses have been given by the project "Advanced Computing Architectures (ACA): towards multi-scale natural-density neuromorphic computing". Funded by the Helmholtz Association, ACA was a pilot project with the long-term goal of building a computer system to enable the simulation of biological learning processes in time-lapse in order to better understand learning and brain development. During the four-year project period (2018 - 2022), the foundation was laid for future developments of architectures specifically tailored to neuroscience simulations. A major achievement was the development of a concept for a digital neuromorphic hardware platform that supports fast, deterministic, and reproducible simulations of biologically plausible spiking neural networks with realistic connection densities. The highly interdisciplinary project was carried out in cooperation across FZJ institutes (INM-6/IAS-6, JSC, PGI-7, PGI-10, and ZEA-2) and external partners (RWTH Aachen (IDS), University of Manchester, and Heidelberg University). More than 30 peer-reviewed articles have resulted from the ACA project [https://www.fz-juelich.de/en/aca].
Our approach
We are exploring novel approaches to neuromorphic system architectures, such as hybrid soft- and hardware designs that combine a traditional von Neumann architecture with application-specific hardware units. These units transform existing simulation codes and algorithms into highly parallel operational digital designs. This can be a computational primitive or special function, a neuron model, an entire neural network, or even a complete simulation platform. The ability to take advantage of state-of-the-art reconfigurable hardware, such as field-programmable gate arrays (FPGAs) and System-on-Chip (SoC) devices, facilitates access to design space exploration and prototyping to identify opportunities for hardware acceleration.
One of the key activities within the research group is the development of innovative strategies to transform various scientific simulations into spiking neural networks compatible with neuromorphic hardware. By transforming traditional simulation codes from different scientific domains, we aim to optimize energy efficiency and explore the potential for computational acceleration. A running project involves reengineering a molecular dynamics (MD) simulation code into a spiking neural network, with the goal of enabling it to run on neuromorphic hardware within a high-performance computing (HPC) cluster. This work includes preparing for the integration of neuromorphic modules into the existing HPC infrastructure, leveraging the potential benefits of a heterogeneous architecture.
The objective of our work is to push the boundaries of computational science, making advanced simulations more accessible, sustainable and energy-efficient.
Publications
Heittmann, A., Psychou, G., Trensch, G., Cox, C. E., Wilcke, W. W., Diesmann, M., et al. (2022). Simulating the Cortical Microcircuit Significantly Faster Than Real Time on the IBM INC-3000 Neural Supercomputer. Frontiers in Neuroscience doi:10.3389/fnins.2021.728460
Trensch G., Morrison A. (2022). A System-on-Chip Based Hybrid Neuromorphic Compute Node Architecture for Reproducible Hyper-Real-Time Simulations of Spiking Neural Networks. Frontiers in Neuroinformatics 16. doi:10.3389/fninf.2022.884033