Mutual Information Measure for Image Segmentation Using Few Labels - IRT Saint Exupéry - Institut de Recherche Technologique Access content directly
Preprints, Working Papers, ... Year : 2023

Mutual Information Measure for Image Segmentation Using Few Labels


Recently several models have been developed to reduce the annotation effort which is required to perform semantic segmentation. Instead of learning from pixel-level annotations, these models learn from cheaper annotations, e.g. image-level labels, scribbles or bounding boxes. However, most of these models cannot easily be adapted to new annotations e.g. new classes since it requires retraining the model. In this paper, we propose a similarity measure between pixels based on a mutual information objective to determine whether these pixels belong to the same class. The mutual information objective is learned in a fully unsupervised manner while the annotations (e.g. points or scribbles) are only used during test time. For a given image, the unlabeled pixels are classified by computing their nearest-neighbors in terms of mutual information from the set of labeled pixels. Experimental results are reported on the Potsdam dataset and Sentinel-2 data is used to provide a real world use case where a large amount of unlabeled satellite images is available but only a few pixels can be labeled. On the Potsdam dataset, our model achieves 70.22% mIoU and 87.17% accuracy outperforming the state-of-the-art weakly-supervised methods.
Fichier principal
Vignette du fichier
main.pdf (6.87 Mo) Télécharger le fichier
ImageSegmentationExamplesPotsdam.pdf (753.65 Ko) Télécharger le fichier
ModelComparison.pdf (426.85 Ko) Télécharger le fichier
NeuralNetwork.pdf (65.28 Ko) Télécharger le fichier
PixelSelectionAndSegmentationPotsdam.pdf (357.77 Ko) Télécharger le fichier
PixelSelectionAndSegmentationSentinel2.pdf (2.62 Mo) Télécharger le fichier
PixelSelectionAndSegmentationSentinelTimeSeries2.pdf (1.49 Mo) Télécharger le fichier
SegmentingMoreImages.pdf (962.22 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02892004 , version 1 (17-05-2023)




  • HAL Id : hal-02892004 , version 1


Eduardo Hugo Sanchez, Mathieu Serrurier, Mathias Ortner. Mutual Information Measure for Image Segmentation Using Few Labels. 2023. ⟨hal-02892004⟩
38 View
20 Download


Gmail Facebook X LinkedIn More