Automatic remote sensing tools can help inform many large-scale challenges such as disaster management, climate change, etc. While a vast amount of spatio-temporal satellite image data is readily available, most of it remains unlabelled. Without labels, this data is not very useful for supervised learning algorithms. Self-supervised learning instead provides a way to learn effective representations for various downstream tasks without labels. In this work, we leverage characteristics unique to satellite images to learn better self-supervised features. Specifically, we use the temporal signal to contrast images with long-term and short-term differences, and we leverage the fact that satellite images do not change frequently. Using these characteristics, we formulate a new loss contrastive loss called Change-Aware Contrastive (CACo) Loss. Further, we also present a novel method of sampling different geographical regions. We show that leveraging these properties leads to better performance on diverse downstream tasks. For example, we see a 6.5% relative improvement for semantic segmentation and an 8.5% relative improvement for change detection over the best-performing baseline with our method.
This research is based upon work
supported in part by the ODNI (IARPA) via 2021-
20111000006, NSF 1900783, NSF 2144117, and NSF
2212084. The views and conclusions contained herein are
those of the authors and should not be interpreted as necessarily representing the official policies, either expressed
or implied, of ODNI, IARPA, or the US Government. The
US Government is authorized to reproduce and distribute
reprints for governmental purposes notwithstanding any
copyright annotation therein.
We also thank Aditya Chetan, Cheng Perng Phoo, Yihong Sun, and Luming Tang for useful feedback.