TiP-MCE: Multimodal Cross-encoder Re-ranking for Indoor Visual Place Recognition

Abstract

Abstract

Indoor visual place recognition (VPR) is challenged by repetitive architec tural layouts where appearancebased retrieval confuses visually similar floors and corridors.This limitation reduces reliability for robotic navigation in large indoor buildings, including practical deployments in Vietnam.Scene text provides high-level semantic cues, yet prior text-aware re-ranking commonly relies on non-learnable string overlap and remains sensitive to optical character recognition (OCR) noise while exploiting spatial visual and textual interactions only weakly.TiP-MCE is introduced as an enhancement over TextInPlace by replacing heuristic text matching with a learnable multimodal cross-encoder (MCE) re-ranker.For each query and each candidate in the top-K list, visual tokens and filtered text instances with content embeddings, bounding boxes, and confidence scores are jointly processed by Transformer self attention to estimate the probability that both images depict the same place.A two-phase training protocol is adopted.Metric learning is used to train the global retrieval branch, then MCE is trained with binary crossentropy on hard negatives mined from top-M retrieval results using pose distance supervision.Experiments on Maze-with-Text show the best overall Recall@1 of 86.8%, surpassing TextInPlace at 85.4% and TextInPlace-I at 86.1%, with larger gains on harder floors and competitive Recall@5.