CRITERIA FOR SIMILARITY OF LINGUISTIC MODELS

Abstract

Anomaly detection, as a form of data analytics, is finding more and more applications in various fields of human activity every year. Thus, with the growth of the IT- sector and global integration, there is an increasing need for tools to monitor cybernetic systems, respond to and remedy disruptions and failures. Intrusion detection, unauthorized access, malfunctioning in critical security systems and infrastructure management systems are among the priorities in the modern information technology world. Anomaly detection is the task of finding patterns in data that do not match expected behavior. Anomalies are inappropriate observations, emissions that are contrary to the nature of the process under study. The purpose of this work is to increase the speed of anomaly detection, in comparison with the algorithms used in automated control systems, through the use of linguistic modeling and a syntax approach to time series analysis. The presence of an anomaly is established by comparing two models - for a training dataset in which anomalies are not guaranteed and for the actual data in which we look for an anomaly. Various metrics and peer review methods are used to assess similarity. The first approach is to create a model that characterizes the normal course of the process. For this purpose, all anomalous sections are excluded from the series, or one is selected from a section of the series that does not contain deviations from the norma − thus the so-called reference row is formed. For him the construction of a linguistic model is performed. An important component of the linguistic approach to detecting anomalies in time series is the criterion by which the similarity of the two models is evaluated. The choice of criteria depends on the possibility of applying a linguistic approach to the analysis of time series of different nature. The basic metrics of the similarity of the texts by Hamming, Lowenstein, Jaro-Winkler and others are considered. During the development of the algorithm, 4 different metrics for evaluating linguistic models were considered. Given their strengths and weaknesses, root mean square was chosen as the most appropriate approach.