Special Sessions

Deep learning approaches for multi-temporal and multi-modal data processing and analysis for urban areas

Germain Forestier, Universite de Haute-Alsace, IRIMAS, France
Jonathan Weber, Universite de Haute-Alsace, IRIMAS, France
Camille Kurtz, Universite Paris Cite, LIPADE, France
Sylvain Lobry, Universite Paris Cite, LIPADE, France
Charlotte Pelletier, Universite Bretagne Sud, IRISA, France


Current remote sensing missions produce a large amount of data on a daily basis, leading to significant challenges in designing processing chains adapted to their analysis. Modern sensors are now able to offer images of the same scene with improved revisit time frequencies (e.g. Sentinel, Venμs), and with increasing spatial and spectral resolutions, sensed and acquired from different types of signals, such as radar, optical, LiDAR and DSM. This data also provides complementary information (e.g. observation scales, physical properties of soils) for the analysis of imaged territories and their temporal evolution, particularly in an urban context.

Computer vision, image processing and artificial intelligence techniques therefore play an important role to process such an amount of multi-temporal and multi-modal data. Deep neural network approaches are currently revolutionizing the field of machine learning with techniques such as unsupervised, self-supervised and supervised approaches. Along with strategies like domain adaptation, transfer learning, time series analysis, image registration and fusion, these approaches are increasingly used nowadays to tackle challenging remote sensing tasks, especially for ever-growing urban environments at the landscape level, such as object detection / extraction, semantic segmentation, classification, land cover mapping and object tracking for Earth observation and urban monitoring.

The aim of this special session is to popularize the use of these promising approaches. Indeed, although these methods have become very popular in computer vision over the last ten years, they must be adapted to take advantage of the multi-modal and multi-temporal specificity of the remote sensing data and the underlying tasks, especially in the context of urban environment analysis. Our objectives are to gather both computer scientists (academics and professionals) familiar with multi-temporal and multi-modal heterogeneous data analysis and end-users close to geographical and environmental sciences who could give sense to raw data.We are also targeting potential new users interested in these approaches in order to stimulate deeper multidisciplinary interactions on this topic.