Climate AI Nordics

Climate AI Nordics

Nordic network for collaborations around tackling climate change using AI and machine learning.

Featured paper: Continuous Urban Change Detection From Satellite Image Time Series With Temporal Feature Refinement and Multitask Integration

Authors: Sebastian Hafner, Heng Fang, Hossein Azizpour, and Yifang Ban

Paper: arxiv.org/abs/2406.17458, published in IEEE Transactions on Geoscience and Remote Sensing (Volume: 63).

Monitoring how cities grow over time is essential for managing urban sprawl and planning sustainable development. Most existing methods compare satellite image pairs acquired at two points in time, typically multiple years apart (i.e., bi-temporal change detection). Due to rapid ongoing urbanization, however, monitoring urban growth at short time intervals is becoming increasingly important. To that end, this paper presents a continuous urban change detection framework that captures changes between consecutive images in high-resolution satellite image time series (SITS).

The proposed model introduces two innovations: the Temporal Feature Refinement (TFR) and Multitask Integration (MTI) modules. The TFR module applies a transformer-based self-attention mechanism to model temporal dependencies in building representations extracted from SITS. Importantly, the TFR module preserves the time dimension, unlike existing SITS change detection methods that are limited to detecting changes between the first and last image of a time series. On the other hand, the MTI module uses Markov networks to fuse the outputs of building segmentation and change detection tasks. By applying belief propagation, the MTI module ensures temporal consistency and removes contradictions between change maps and building footprints.

The framework was tested on three challenging datasets, namely SpaceNet 7 (PlanetScope, 4 m), Wuhan Urban Semantic Understanding (Gaofen-2, 1 m), and Time Series Change Detection (WorldView-2, 0.5 m), and consistently outperformed bi-temporal change detection methods, as well as SITS change detection methods.

Ablation studies confirmed that modeling full temporal context through self-attention significantly improves performance over recurrent alternatives, and that dense temporal connections in the MTI module lead to more consistent and accurate urban change maps.

You May Also Enjoy

Self-supervised pre-training for glacier calving front extraction from synthetic aperture radar imagery

Event date:

Webinar with Nora Gourmelon, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). The factors influencing melt at glacier fronts facing the ocean remain an active area of research. To better understand these factors, it is essential to monitor changes in glacier calving fronts. Due to the importance of weather-dependent and seasonal variations, Synthetic Aperture Radar (SAR) is the preferred imaging method for such monitoring. In recent years, deep learning models have been developed to automate the extraction of calving front positions, however, their performance on SAR data remains suboptimal. Limited labeled data and high variability in SAR images hinder traditional supervised learning. Foundation models pre-trained on large, diverse datasets could provide robust feature representations that require minimal labeled data for fine-tuning on specific SAR tasks. However, in preliminary experiments, we found that the domain gap is too large for the task of extracting calving fronts from SAR imagery, and foundation models fine-tuned on the "CAlving Fronts and where to Find thEm" (CaFFe) benchmark dataset performed subpar. Therefore, we compiled an unlabeled dataset of Sentinel-1 SAR image sequences of Arctic glaciers, each associated with a single Sentinel-2 optical reference image. Using this dataset, we developed a novel multi-modal self-supervised pre-training strategy and applied it to pre-train a hybrid CNN-transformer model. Fine-tuning on the CaFFe benchmark showed that the pre-trained model outperforms its non-pre-trained counterpart, enabling more robust calving front segmentation and demonstrating the potential of data-efficient learning for SAR imagery.

Earth observation and deep learning for urban applications

Event date:

Webinar with Sebastian Hafner, RISE. Urbanization is progressing at an unprecedented rate in many places around the world, making Earth observation (EO) a vital tool for monitoring its dynamics on a global scale. Modern satellite missions provide new opportunities for urban mapping and change detection (CD) through high-resolution imagery. At the same time, EO data analysis has advanced from traditional machine learning approaches to deep learning (DL), particularly Convolutional Neural Networks (ConvNets). Yet, current DL methods for urban mapping and CD face key challenges, including effectively integrating multi-modal EO data, reducing reliance on large labeled datasets, and improving transferability across geographic regions. This talk will present methods to address these challenges in urban mapping and CD, with applications such as multi-hazard building damage detection. It will also highlight the IDEAMAPS Data Ecosystem project, showing how its community-centric approach advances our understanding of urban deprivation and enables DL-based deprivation mapping.

Self-supervised pre-training for glacier calving front extraction from synthetic aperture radar imagery

Event date:

Webinar with Nora Gourmelon, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). The factors influencing melt at glacier fronts facing the ocean remain an active area of research. To better understand these factors, it is essential to monitor changes in glacier calving fronts. Due to the importance of weather-dependent and seasonal variations, Synthetic Aperture Radar (SAR) is the preferred imaging method for such monitoring. In recent years, deep learning models have been developed to automate the extraction of calving front positions, however, their performance on SAR data remains suboptimal. Limited labeled data and high variability in SAR images hinder traditional supervised learning. Foundation models pre-trained on large, diverse datasets could provide robust feature representations that require minimal labeled data for fine-tuning on specific SAR tasks. However, in preliminary experiments, we found that the domain gap is too large for the task of extracting calving fronts from SAR imagery, and foundation models fine-tuned on the "CAlving Fronts and where to Find thEm" (CaFFe) benchmark dataset performed subpar. Therefore, we compiled an unlabeled dataset of Sentinel-1 SAR image sequences of Arctic glaciers, each associated with a single Sentinel-2 optical reference image. Using this dataset, we developed a novel multi-modal self-supervised pre-training strategy and applied it to pre-train a hybrid CNN-transformer model. Fine-tuning on the CaFFe benchmark showed that the pre-trained model outperforms its non-pre-trained counterpart, enabling more robust calving front segmentation and demonstrating the potential of data-efficient learning for SAR imagery.

Celebrating one year with Climate AI Nordics!

Published:

From pioneering AI models for weather and biodiversity to our first Nordic Workshop on AI for Climate Change, the Climate AI Nordics community has grown and thrived throughout its first year of existance. Here's a look back at key events and milestones from the first year.