Skip to Main content Skip to Navigation
Conference papers

Detecting Human-to-Human-or-Object (H²O) Interactions with DIABOLO

Abstract : Detecting human interactions is crucial for human behavior analysis. Many methods have been proposed to deal with Human-to-Object Interaction ($HOI$) detection, i.e., detecting in an image which person and object interact together and classifying the type of interaction. However, Human-to-Human Interactions, such as social and violent interactions, are generally not considered in available $HOI$ training datasets. As we think these types of interactions cannot be ignored and decorrelated from $HOI$ when nalyzing human behavior, we propose a new interaction dataset to deal with both types of human interactions: Human-to-Human-or-Object ($H²O$). In addition, we introduce a novel taxonomy of verbs, intended to be closer to a description of human body attitude in relation to the surrounding targets of interaction, and more independent of the environment. Unlike some existing datasets, we strive to avoid defining synonymous verbs when their use highly depends on the target type or requires a high level of semantic interpretation. As $H²O$ dataset includes V-COCO images annotated with this new taxonomy, images obviously contain more interactions. This can be an issue for HOI detection methods whose complexity depends on the number of people, targets or interactions. Thus, we propose DIABOLO (Detecting InterActions By Only Looking Once), an efficient subject-centric single-shot method to detect all interactions in one forward pass, with constant inference time independent of image content. In addition, this multi-task network simultaneously detects all people and objects. We show how sharing a network for these tasks does not only save computation resource but also improves performance collaboratively. Finally, DIABOLO is a strong baseline for the new proposed challenge of $H²O$-Interaction detection, as it outperforms all state-of-the-art methods when trained and evaluated on $HOI$ dataset V-COCO. We hope that this new dataset and new baseline will foster future research. $H²O$ is available on
Complete list of metadata
Contributor : Contributeur MAP CEA Connect in order to contact the contributor
Submitted on : Friday, April 8, 2022 - 4:30:05 PM
Last modification on : Tuesday, April 12, 2022 - 3:46:04 AM
Long-term archiving on: : Saturday, July 9, 2022 - 7:07:00 PM


 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2022-10-05

Please log in to resquest access to the document



Astrid Orcesi, Romaric Audigier, Fritz Poka Toukam, Bertrand Luvison. Detecting Human-to-Human-or-Object (H²O) Interactions with DIABOLO. FG 2021 - 16th IEEE International Conference on Automatic Face and Gesture Recognition, Dec 2021, Jodhpur (virtual event), India. pp.1-8, ⟨10.1109/FG52635.2021.9667005⟩. ⟨cea-03635726⟩



Record views