ASSESSMENT OF THE ACCURACY OF ARTIFICIAL LANGUAGE TRANSLATION METHODS

Abstract

The article is devoted to evaluating the accuracy of methods for translating artificial languages in an intelligent information system for translating artificial languages using artificial intelligence methods and statistical algorithms, proposing an innovative approach to improving the effectiveness of machine translation in the context of the dynamic development of modern technologies. The analysis showed insufficient translation accuracy, which was proposed to be improved by integrating two methods into one system.The aim of the work is to evaluate the accuracy of translation methods in the proposed intellectual information system for translating artificial languages. The work examines such translation evaluation metrics as BLEU, ChrF, BLEURT, COMMET, TER, METEOR and human evaluation (as a benchmark). Each of the algorithms (mathematical models, neural network algorithm and combined algorithm) was tested on 12 tests, each of which involved translating one of four languages into another (English, Ukrainian, French and Esperanto) and vice versa. The results show an increase in average translation accuracy of 0.5 % compared to the method based on mathematical models and 0.2 % compared to the method based on neural networks. In some pairs, the result was worse than the individual algorithm, but this was due to the coefficients used when combining the results. Since individual settings are possible for each language pair, the correct result will appear in the worst case with the same accuracy as the best of the two results. However, this requires separate configuration. Based on the results obtained, it can be concluded that the combination of these two methods improves the accuracy of artificial language translation. The results confirm the effectiveness of the developed system, which allows for highly efficient translation of natural and artificial languages.