Towards Identifying Interactions Between Human and Object Using Transformer-Based Models

Abdurashidova Kamola Turgunbaevna, Abdukhakimov Fayzulla Kudratulla ugli, Chorshanbiyeva Sevinch Akramovna
Bulletin of TUIT: Management and Communication Technologies
№ 1(9)2024 DOI: 10.61663/241tuitmct9
Open article file

Abstract

Understanding the subtle interactions between individuals and objects is critical for a variety of applications, including robotics, computer vision, and human-computer interaction. This paper looks at the research landscape for using transformer-based models to discover and understand these relationships. Existing approaches based on their methodologies, datasets used, and application fields have been explored thoroughly. The paper identifies several existing SOTA (state-of-the-art) methodologies that use transformer-based models to detect interactions among people, and objects as well.

References

[1] Jitendra Gupta, Saurabh Malik. Visual semantic role labeling. arXiv preprint arXiv:1505.04474, 2015.
[2] Chen Gao, Yuliang Zou, and Jia-Bin Huang. ican: Instancecentric attention network for human-object interaction detection. arXiv preprint arXiv:1808.10437, 2018.
[3] Yong-Lu Li, Siyuan Zhou, Xijie Huang, Liang Xu, Ze Ma, Hao-Shu Fang, Yanfeng Wang, and Cewu Lu. Transferable interactiveness knowledge for human-object interaction detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3585–3594, 2019.
[4] Yong-Lu Li, Liang Xu, Xinpeng Liu, Xijie Huang, Yue Xu, Shiyi Wang, Hao-Shu Fang, Ze Ma, Mingyang Chen, and Cewu Lu. Pastanet: Toward human activity knowledge engine. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 382–391, 2020.
[5] Oytun Ulutan, ASM Iftekhar, and Bangalore S Manjunath. Vsgnet: Spatial attention network for detecting human object interactions using graph convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13617–13626, 2020.
[6] TiancaiWang, Tong Yang, Martin Danelljan, Fahad Shahbaz Khan, Xiangyu Zhang, and Jian Sun. Learning human-object interaction detection using interaction points. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4116–4125, 2020.
[7] Yue Liao, Si Liu, FeiWang, Yanjie Chen, Chen Qian, and Jiashi Feng. Ppdm: Parallel point detection and matching for real-time human-object interaction detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 482–490, 2020.
[8] Bumsoo Kim, Taeho Choi, Jaewoo Kang, and Hyunwoo Kim. Uniondet: Union-level detection towards real-time human-object interaction detection. In Proceedings of the European conference on computer vision (ECCV), 2020.
[9] Kim, B., Lee, J., Kang, J., Kim, E. S., & Kim, H. J. (2021). Hotr: End-to-end human-object interaction detection with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 74-83).
[10] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. arXiv preprint arXiv:2005.12872, 2020.
[11] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, December 2017.
[12] Tamura, M., Ohashi, H., & Yoshinaga, T. (2021). Qpic: Query-based pairwise human-object interaction detection with image-wide contextual information. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10410-10419).
[13] N Nasimova, B Muminov, R Nasimov, K Abdurashidova, M Abdullaev. Comparative Analysis of the Results of Algorithms for Dilated Cardiomyopathy and Hypertrophic Cardiomyopathy Using Deep Learning. 2021 International Conference on Information Science and Communications Technologies (ICISCT) (2021): pp. 1-5.
[14] Abdurashidova, K., Abdukhakimov, F., & Chorshanbiyeva, S. (2024). UNDERSTANDING HUMAN INTERACTION USING GRAPH NEURAL NETWORKS: AN OVERVIEW. Talqin va tadqiqotlar, (4 (41)).