Optimized design of a multimodal perception system for sports robots based on YOLOv5 and KCF

Abstract

Abstract

Introduction For motion robots that use dynamic perception, state-of-the-art systems still struggle to simultaneously tackle various challenges, including high-speed motion blur, strong interactive occlusion, and drastic changes in scene lighting, which limit the robustness and real-time performance of tracking. Methods This study proposes a multimodal perception optimization model that integrates multiple algorithms. The model first uses YOLOv5 to achieve rapid detection and localization of multiple object categories. Then, a nuclear correlation filter was integrated to achieve robust tracking of consecutive frames. Furthermore, spatiotemporal graph convolutional networks and dual channel attention mechanisms were introduced to enhance the extraction of motion patterns and key features. In addition, it also combines convolutional neural networks based on faster regions and deep simple online real-time tracking to support high-precision detection and identity preservation. Finally, apply genetic algorithm to dynamic parameter optimization and adaptive adjustment. Results The proposed model achieved a maximum multi-target tracking accuracy of 96.23% on the SportsMOT dataset and AthletePose3D dataset, with a tracking accuracy improvement of 50.44%, consistently outperforming the baseline method, and a minimum end-to-end perception delay of 49.6 ms. Discussion The model demonstrates excellent tracking stability and real-time response capability in various dynamic scenarios, achieving high accuracy and robustness in target perception and trajectory prediction, and is expected to support intelligent sports applications, including training assistance, real-time opponent simulation, and autonomous interception decision-making.