New Embed2Scale Project: Earth Observation and Weather Data Federation with AI Embeddings
On 15 January 2024, the EU project Embed2Scale kicked off at EUSPA, the EU Agency for the Space Programme, in Prague. This project is coordinated by MARTEL INNOVATE BV and includes ten partners, one of them being the Jülich Supercomputing Center. The true potential of the European Union's Earth Observation Copernicus Programme is unlocked when integrated with additional geo-information, such as weather models and GNSS data. However, managing hundreds of petabytes of geospatial data is challenging, as no single platform can store it all. Service providers often download data from various archives, but the vast amount of data makes some applications economically unfeasible.
Embed2Scale aims to address these challenges by facilitating efficient data exchange through AI-driven data compression. It will develop methods to convert raw geo-information into highly compressed embeddings, potentially reducing the size by up to 1000 times. The envisaged key breakthroughs include enabling decentralized applications through significant data volume reduction, enhancing the portability of geospatial analytics by reducing computational demands, minimizing data labeling through few-shot learning, and enabling near-real-time, petabyte-scale searches of earth observation and climate data. The plan is to demonstrate these advancements with practical applications in the Copernicus Programme and to foster community growth through open-source contributions and standardization. The embedding technology will be tested in four areas: maritime monitoring, aboveground biomass measurement, climate and air pollution forecasting, and early detection of crop stress and yield.
More information can be found on the project website.