Looking Back at the Workshop "Transformers for Environmental Science"

JSC, the Otto von Guericke University of Magdeburg, and the Technical University of Munich jointly organized a workshop on "Transformers for Environmental Science" from 22 to 23 September 2022. The workshop was co-sponsored by the ERC grant IntelliAQ and brought together about 40 participants in Magdeburg and up to 20 additional online attendees, who discussed the potential of this new AI technology for environmental applications. The programme included lectures on recent advances in transformer architectures and transfer learning as well as on prototype developments focusing largely on atmospheric research and remote sensing. Keynote presentations were given by Peter Düben (ECMWF), Pedram Hassanzadeh (Rice University, Houston), Duncan Watson-Parris (Oxford University), Lucas Beyer (Google Brain), and Jonathan Godwin (Google Deepmind). A poster session and panel discussion provided opportunities for an exchange of ideas. For more details, please visit https://sites.google.com/view/esstransformers/.

Large transformer models require huge amounts of data and constitute an attractive application for accelerated supercomputers such as JUWELS Booster. Within the “atmorep” compute time project, first steps towards training such a model for atmospheric research are being taken.

Contact: Dr. Martin Schultz

from JSC News No. 292, 2 November 2022