MODELING AND EVALUATION OF THE PROPERTIES OF AN ADAPTIVE ENSEMBLE INDICATOR FOR DIAGNOSING THE STRATEGIC RESOURCE SECURITY OF A DISTRIBUTED ORGANIZATIONAL SYSTEM
DOI:
https://doi.org/10.35546/kntu2078-4481.2026.1.42Keywords:
machine learning-based classification, heterogeneous ensemble of classifiers, adaptive quality metric, class imbalance, specificity (risk of hypodiagnosis), soft voting, strategic decision supportAbstract
The paper presents the results of a study of the properties of an adaptive ensemble indicator for diagnosing the strategic resource security (DSRS) of a distributed organizational system, intended to support decision-making under conditions of high uncertainty, incomplete information, and class imbalance. The indicator is implemented as a heterogeneous ensemble of binary classifiers combining machine learning models of different paradigms (Naive Bayes, Support Vector Machine, Random Forest, k-Nearest Neighbors, Logistic Regression) and is supplemented with specialized mechanisms for feature scaling, probability calibration, and adaptive weighting of model outputs. The relevance of the study is determined by the specific nature of resource data in strategic management systems, which are characterized by complex and asymmetric feature distributions, temporal lags, high heterogeneity of observation conditions, dominance of satisfactory states, and the critical importance of minimizing the risk of hypodiagnosis of unfavorable situations. Taking these characteristics into account, the paper proposes a composite quality metric KQ that integrates the F1neg score, the Matthews correlation coefficient, and Cohen’s kappa, while considering their mutual correlations and domain-specific managerial priorities. The research is conducted through a series of computational experiments using specially generated synthetic datasets that reproduce domain-driven statistical properties of resource observations, including heavy-tailed distributions, label noise, severe class imbalance, and the influence of historical resource dynamics. For this purpose, a unified framework for managing computational experiments is developed, ensuring reproducibility, comparability of results, and statistical validation of observed effects. Within three experimental studies, the algorithm is subjected to baseline verification, a stepwise analysis of the contribution of individual mechanisms to classification quality, and an evaluation of ensemble performance under stress conditions in comparison with the best-performing individual classifier. The results demonstrate a stable advantage of the proposed ensemble DSRS indicator in terms of the composite quality metric, specificity, and F1neg score, which is confirmed by statistically significant paired tests and effect size estimates. It is shown that the proposed approach provides more reliable detection of potentially dangerous resource states and forms an interpretable basis for supporting strategic decision-making in distributed organizational systems.
References
Skybyk S., Doroshenko A., Ilyina O., Sinitsyn I. Machine-learning-based model for indicators of the resource-based security of interests in high-level organizational systems. In: V. Snytyuk, V. Pleskach, I. Javorskyj, M. Dyvak, N. Goranin, V. Levashenko, E. Zaitseva, & O. Pursky (Eds.). Proc. 9th International Scientific and Practical Conference “Applied Information Systems and Technologies in the Digital Society” (AISTDS 2025) (CEUR Workshop Proceedings, Vol. 4133, pp. 153–169). 2025. CEUR-WS.
Ільїна О., Сініцин І., Слабоспицька О. Принципи та моделі експертно-аналітичної методології підтримки формування адаптивних організаційних рішень за умов глибокої невизначеності. Проблеми програмування. 2022. № 3–4. С. 364–375. DOI: https://doi.org/10.15407/pp2022.03-04.364
Сініцин І. П., Шевченко В. Л., Дорошенко А. Ю., Федоренко Р. М. Моделі та програмні системи управління оборонними ресурсами: монографія. Київ : ІПС НАНУ, 2024.
Dietterich T. G. Ensemble methods in machine learning. Multiple Classifier Systems. MCS 2000. Lecture Notes in Computer Science. 2000. Vol. 1857. P. 1–15. DOI: https://doi.org/10.1007/3-540-45014-9_1
Kuncheva L. I. Combining pattern classifiers: methods and algorithms. New York : Wiley-Interscience, 2004.
Polikar R. Ensemble based systems in decision making. IEEE Circuits and Systems Magazine. 2006. Vol. 6, N 3. P. 21–45. DOI: https://doi.org/10.1109/MCAS.2006.1688199
Hnatchuk Y., Lebedovska M. Decision support system for project resource planning based on the random forest method. Computer Systems and Information Technologies. 2025. Vol. 4. P. 35–42. DOI: https://doi.org/10.31891/csit-2025-4-4
Sommer R., Paxson V. Outside the closed world: on using machine learning for network intrusion detection. Proc. of the 2010 IEEE Symposium on Security and Privacy (SP’10). Washington : IEEE, 2010. P. 305–316. DOI: https://doi.org/10.1109/SP.2010.25
Fawcett T., Provost F. Combining data mining and machine learning for effective user profiling. Proc. 2nd International Conference on Knowledge Discovery and Data Mining (KDD’96). 1996. P. 8–13.
Fawcett T. An introduction to ROC analysis. Pattern Recognition Letters. 2006. Vol. 27, N. 8. P. 861–874. DOI: https://doi.org/10.1016/j.patrec.2005.10.010
Hastie T., Tibshirani R., Friedman J. The elements of statistical learning. New York : Springer, 2009.
Gentleman R., Lang D. T. Statistical analyses and reproducible research. Journal of Computational and Graphical Statistics. 2007. Vol. 16, N 1. P. 1–23. DOI: https://doi.org/10.1198/106186007X178663
Santner T. J., Williams B. J., Notz W. I. The design and analysis of computer experiments. New York : Springer, 2018.
Kress M. Operational logistics: the art and science of sustaining military operations. 2nd ed. New York : Springer International Publishing, 2016.
Степанюк М. Ю., Сініцин І. П., Котеля О. В. Проблема створення інформаційної системи логістики в Збройних силах України, що відповідає стандартам НАТО. Проблеми програмування. 2018. № 4. С. 101–110. DOI: https://doi.org/10.15407/pp2018.04.101
Large J., Lines J., Bagnall A. A probabilistic classifier ensemble weighting scheme based on exponentially weighting the probability estimates. Data Mining and Knowledge Discovery. 2019. Vol. 33. P. 1674–1709. DOI: https://doi.org/10.1007/s10618-019-00638-y
Sokolova M., Lapalme G. A systematic analysis of performance measures for classification tasks. Information Processing & Management. 2009. Vol. 45, N 4. P. 427–437. DOI: https://doi.org/10.1016/j.ipm.2009.03.002
Nadeau C., Bengio Y. Inference for the generalization error. Machine Learning. 2003. Vol. 52, N 3. P. 239–281. DOI: https://doi.org/10.1023/A:1024068626366
Ojala M., Garriga G. C. Permutation tests for studying classifier performance. Journal of Machine Learning Research. 2010. Vol. 11. P. 1833–1863. DOI: https://doi.org/10.1109/ICDM.2009.108
Baldi P., Brunak S., Chauvin Y., Andersen C. A. F., Nielsen H. Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics. 2000. Vol. 16, N 5. P. 412–424. DOI: https://doi.org/10.1093/bioinformatics/16.5.412
Witten I. H., Frank E., Hall M. A., Pal C. J. Data Mining: Practical Machine Learning Tools and Techniques. 4th ed. Burlington : Morgan Kaufmann, 2016.





