METHODS FOR EVALUATING THE EFFECTIVENESS OF OBJECT DETECTION MODELS IN COMPUTER VISION
DOI:
https://doi.org/10.35546/kntu2078-4481.2023.2.25Keywords:
computer vision, evaluation metrics, IoU, Recall, F1-Score, Precision, confusion matrix, mAPAbstract
The main tasks of computer vision are recognition, object detection, and segmentation. Image recognition is used in a variety of industries from security systems to medical diagnostics. Object detection is a technique for locating objects and then recognizing them in real time. Segmentation is the process of dividing an image into many segments. The process of building a model can be complex, and in order for the model to fully meet the task, it is necessary to determine its effectiveness. The purpose of the study is to review the performance, accuracy, and productivity of computer vision models. The classical version of the intersection of union (IoU) indicator is described. Various modifications and improvements of the IoU such as the multi-scale IoU (MSIoU), the bounding IoU (BaIoU), and the BhIoU are presented. Particular attention is paid to the generalized intersection through union (GIoU) to eliminate the disadvantages of IOU loss. That is, the IOU loss will always be zero when two blocks do not interact – do not intersect. The performance of computer vision algorithms for object detection and segmentation is usually tested using the mean of the mean (mAP). And since mAP is based on various submetrics, we considered the confusion matrix, Intersection through Merging, Recall, and Precision. For a better understanding of the metrics, an example was demonstrated with the calculation of accuracy, precision, recall, harmonic mean of accuracy, and model sensitivity (F1-Score). Finally, it is obvious that this study has shown that these metrics can be used to check how accurate a trained model is.
References
C.-d. Yang, R.-c. Xie, S.-b. Shi and Z. Tang, “Multi-Target Tracking and Segmentation Method for Missile-Borne Image Based on IoU Association,” 2019 12th International Conference on Intelligent Computation Technology and Automation (ICICTA), Xiangtan, China, 2019, pp. 207–211, doi: 10.1109/ICICTA49267.2019.00051.
C. Ma, L. Zhuo, J. Li, Y. Zhang and J. Zhang, “Prohibited Object Detection in X-ray Images with Dynamic Deformable Convolution and Adaptive IoU,” 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France, 2022, pp. 3001–3005, doi: 10.1109/ICIP46576.2022.9897684.
M. Sugang, L. Ningbo, P. Guansheng, C. Yanping, W. Ying and H. Zhiqiang, “Object detection algorithm based on cosine similarity IoU,” 2022 International Conference on Networking and Network Applications (NaNA), Urumqi, China, 2022, pp. 1–6, doi: 10.1109/NaNA56854.2022.00077.
V. Levkivskyi, D. Marchuk, N. Lobanchykova et al, . “Available parking places recognition system”, 2022 CEUR Workshop Proceedings 4th Workshop for Young Scientists in Computer Science & Software Engineering, Vol. 3077, рр. 123–134, [Online], available at: http://ceur-ws.org/Vol-3077/paper07.pdf
L. Janos Lance, T. Edwin Sybingco and J. A. C. Jose, “Efficient Vehicle Counting Algorithm using Gaussian Mixing Models and IOU Based Tracker for LPWAN Based Intelligent Traffic Management Systems,” 2022 IEEE 14th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Boracay Island, Philippines, 2022, pp. 1–4, doi: 10.1109/HNICEM57413.2022.10109498.
Zhang, H., Wang, Y., Dayoub, F., & Sunderhauf, N. (2021). Varifocalnet: An iou-aware dense object detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8514–8523).
A. Ahmadzadeh, D.J. Kempton, Y. Chen and R.A. Angryk, “Multiscale IOU: A Metric for Evaluation of Salient Object Detection with Fine Structures,” 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA, 2021, pp. 684–688, doi: 10.1109/ICIP42928.2021.9506337.
D. Zhou, J. Fang, X. Song, C. Guan, J. Yin, Y. Dai, and R. Yang, “Iou loss for 2d/3d object detection,” In 2019 International Conference on 3D Vision (3DV), pp. 85–94. IEEE, 2019.
H. Rezatofighi, N.Tsoi, J. Gwak, A. Sadeghian, I. Reid and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 658–666. 2019.
Z. Liu, J. Cheng, Q. Wang and L. Xian, “Improved Design Based on IoU Loss Functions for Bounding Box Regression,” 2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC ), Beijing, China, 2022, pp. 452–458, doi: 10.1109/IAEAC54830.2022.9929938.
