Object Embedding-Based Knowledge Distillation for Enhanced Visual Question Answering

Abstract

Abstract

With the rapid advancement of AI, multi-modal tasks have become key components in enhancing machine intelligence. We can observe their presence in everyday technology. A prominent example is Visual Question Answering (VQA), where users interact with AI systems to receive contextual responses based on visual input. Applications such as real-time translation, image captioning, and information retrieval through smartphone cameras highlight the growing impact of multi-modal AI in our daily lives. While these innovations make AI indispensable for modern problem-solving, their implementation often requires significant computational resources. To address this, model compression techniques such as Knowledge Distillation (KD) have been highly effective. KD aims to improve the performance of compact models by transferring knowledge from larger, more cumbersome models. However, traditional KD methods typically transfer only the probability distribution of the final layer, which may not capture the full complexity of the knowledge in tasks like VQA. In this paper, we propose a novel approach where, instead of transferring probabilities, we use object embeddings as the source of rich knowledge. These embeddings, learned from a high-performing teacher model, provide a deeper level of knowledge transfer. This rich presentation enables the compact model to better understand visual and contextual relationships, ultimately improving its performance on complex VQA tasks.