Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


The Project DeepRain

Efficient Local Precipitation Prediction through Machine Learning

The DeepRain project is a collaboration among the Jülich Supercomputing Centre (JSC) at Forschungszentrum Jülich, the German Weather Service (DWD), the Universities of Osnabrück and Bonn and the Jacobs University in Bremen. During the three years of funding from BMBF, the partners investigate how modern methods of machine learning can be applied to improve precipitation forecasts in Germany. Precise predictions of rain and snow with a reliable indication of the expected amount of precipitation are still an extreme challenge for weather modelling, especially at the local scales where they are most relevant.

The project makes use of radar data, high-resolution topographic data, and ensemble forecast data from a numerical weather model and employs state-of-the-art information processing technology as well as advanced statistics for the evaluation of the new forecasts. DeepRain is funded by the German ministry of education and research (BMBF) under grant number 01 IS 18047 and started on Oct 1st, 2018.

DeepRain WorkflowCopyright: Forschungszentrum Jülich

The graphics shows the workflow as employed by DeepRain. The various input data are first imported into high-performance database systems and processed there so that they are available as training, test and evaluation data for the neural network. The topographic data is static, all other data sets contain temporal changes up to the recent past. Predictions are also available from the weather model, which the neural network can use for downscaling. With the help of machine learning, a spatial mask is first generated for each prediction time step in order to identify those points where precipitation is likely to occur for further analysis. Then the predictions are divided into three classes for strong, medium and weak precipitation and for each grid point the probability for the occurrence of precipitation of the respective classes is calculated, whereby the coupling between adjacent grid boxes must be taken into account. The prediction results are made available via a web interface where they can be evaluated with independent data from station measurements.

Goals

  • Innovative Fusion of Earth System data
  • Groundbreaking application of deep learning for weather forecasting
  • 24-hour predictions of precipitation on a 1 km scale
  • Knowledge transfer among scientific disciplines and to end users

Approach

DeepRain explores new ways in the four areas: Data fusion, application of AI, validation methods, and big data IT technology.

Data FusionCopyright: Forschungszentrum Jülich

Spatially resolved (grid point) data from different data sources are imported into the array database rasdaman, where they are then available for flexible high-performance extraction. The project will lead to enhanced query options for the rasdaman database.
Modern deep learning technologies and neural network architectures are adapted to the specific requirements of meteorological data sets. In contrast to typical image recognition or segmentation problems, DeepRain faces the challenge of limited input data (large fields, but small number of samples). Complex and time-evolving model errors make it a challenging task to select appropriate data structures and neural network architectures. Furthermore, precipitation is far from following a normal distribution, and heavy rainfall events are rare cases.

AI ApplicationCopyright: Forschungszentrum Jülich

ValidationCopyright: Forschungszentrum Jülich

Advanced statistical techniques from climate model validation and postprocessing will be adapted to short-range weather forecast results on high resolution. Careful and robust evaluation of the deep learning results shall build confidence in this novel technique, or – if the results are not convincing – establish critical benchmarks against which future attempts to supersede classical downscaling methods have to be measured.
The number and size of the meteorological fields (about 600 TBytes total) and the complexity of the problem call for sophisticated data and work flows and provide the opportunity to test new data architectures and data staging concepts on the Jülich supercomputers JURECA and JUWELS. The DeepRain project will thus contribute to the design choices of future exascale supercomputing architectures.

IT TechnologyCopyright: Forschungszentrum Jülich

Publications

Partners

DeepRain is funded by the Bundesministerium fuer Bildung und Forschung (BMBF), the grant period is October 2018 until September 2021.


Servicemeu

Homepage