UNCERTAINTY CALIBRATION IN COMPUTER VISION: COMPARISON OF BNN, DEEP ENSEMBLES, MC DROPOUT AND EVIDENTIAL DL

Abstract

The article is devoted to a comparative analysis of modern methods of uncertainty calibration in computer vision tasks.The problem of miscalibration of deep neural networks becomes critical when used in systems with a high cost of error: medical diagnostics, autonomous driving, industrial control. Modern architectures often demonstrate overconfidence in incorrect predictions, which makes effective decision-making impossible. Four approaches are investigated: Bayesian Neural Networks (BNN) with variational inference; Deep Ensembles that aggregate predictions of independent models; Monte Carlo Dropout for approximating Bayesian inference; Evidential Deep Learning with modeling of Dirichlet distributions. Experiments were conducted on CIFAR-10/100 for classification, CIFAR-10-C for robustness assessment, COCO val2017 for object detection, Cityscapes val for segmentation. All methods were tested on identical architectures: ResNet-50, WideResNet-28-10, Faster R-CNN, DeepLabV3+. The results show that Deep Ensembles provide the best balance: accuracy improvement by 0.8–1.8 % at ECE < 2 %. Temperature scaling reduces Expected Calibration Error by 60–75 % without additional costs, making it mandatory for production systems. BNNs demonstrate the best OOD detection (AUROC 0.813), but are inferior in accuracy by 0.5–1.2 %. MC Dropout has a 20-fold increase in inference time with a moderate improvement in calibration. Evidential DL shows instability on OOD data. Practical recommendations are formulated: for critical applications – an ensemble of 5 models with temperature scaling; for real-time systems – one model with calibration; for edge devices – knowledge distillation from the ensemble. Optimal configuration: 3–5 models with temperature scaling provides 90 % improvement with 3–5x increase in inference time.