Guided flow matching for multimodal remote sensing image clustering

Abstract

Abstract

Subject to the constraints of different imaging mechanisms, multimodal remote sensing (RS) images form distinct manifolds in observation spaces of varying dimensions. Leveraging the complementarity of multimodal images is therefore key to improving clustering performance. To this end, we devise a guided flow matching (GFM) model that constructs distribution paths for multimodal images. This approach is based on the perspective that these paths can serve as a means of modality fusion, leveraging the powerful representational capacity of diffusion models. Specifically, the GFM model incorporates causal velocity networks associated with the paths. These velocity networks are guided by coarse labels derived from one modality. Since this guidance embeds preliminary cluster information, the guided network tends to capture discriminative fusion features from the multimodal data. Accordingly, the velocity estimator takes the distribution paths and the coarse labels as inputs and predicts the corresponding velocities. These velocities encapsulate rich feature representations, which are then fed into the K-means algorithm to generate the final clusters. Experimental results demonstrate that the proposed method achieves competitive clustering performance compared to existing approaches and produces smooth clustering maps, as verified through visualization. An ablation study on the guidance mechanism further confirms its effectiveness in generating discriminative fusion features for multimodal clustering.