Combining generic and specific information for cross-modal retrieval

Thi Quynh Nhi Tran (1) , Hervé Le Borgne (1) , M. Crucianu (2)

1 DIASI - Département Intelligence Ambiante et Systèmes Interactifs
2 CEDRIC - VERTIGO - CEDRIC. Données complexes, apprentissage et représentations

Cross-modal retrieval increasingly relies on joint statistical models built from large amounts of data represented according to several modalities. However, some information that is poorly represented by these models can be very significant for a retrieval task. We show that, by appropriately identifying and taking such information into account, the results of cross-modal retrieval can be strongly improved. We apply our model to three benchmarks for the text illustration task and find that the more data has misrepresented information, the more our model is comparatively effective.

Keywords: cross-modal retrieval; text illustration; canonical correlation analysis

Domains: Computer Science [cs]

Thi Quynh Nhi Tran, Hervé Le Borgne, M. Crucianu. Combining generic and specific information for cross-modal retrieval. ICMR '15 Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, Jun 2015, Shangai, China. pp.551-554, ⟨10.1145/2671188.2749348⟩. ⟨cea-01813724⟩