MATHEMATICAL MODEL FOR EVALUATION OF TRILINGUAL ELECTRONIC TRANSLATION
The paper considers the issue of improving the mathematical model for a hybrid method of machine translation evaluation. Today, electronic translation remains a popular tool. The development of technologies and adaptation to linguistic and cultural features makes it indispensable in education, business and everyday life. The scientific article is devoted to the study of improving the model for evaluating electronic translation. The authors paid special attention to the analysis of existing methods for evaluating electronic translation, such as BLEU, METEOR, ROUGE was improved on the basis of which a new mixed model for evaluating electronic translation. An exact, also given example of calculating the assessment with an incoming text in the Uzbek language, which is translated into the Kyrgyz and Karakalpak languages. As a result, assessments were obtained using the METEOR, ROUGE methods. In addition, digital results of evaluating electronic translation based on the hybrid method were obtained. An analysis of the comparison of the translation platforms GOOGLE Translate, PROMT, Yandex and DEEPL after the implementation of the above methods is given.
[1] Papineni K., Roukos S, Ward T., Zhu W. J., BLEU: a method for automatic evaluation of machine translation. ACL-2002: 40th Annual meeting of the Association for Computational Linguistics. pp. 311–318.
[2] Coughlin D., "Correlating Automated and Human Assessments of Machine Translation Quality" in MT Summit IX, New Orleans, USA pp. 23–27. 2003
[3] Callison-Burch C., Osborne M., Koehn P. "Re-evaluating the Role of BLEU in Machine Translation Research" in 11th Conference of the European Chapter of the Association for Computational Linguistics: EACL 2006. pp. 249–256.
[4] Lin C., Oching F. "Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics" in Proceedings of the 42nd Annual Meeting of the Association of Computational Linguistics. 2004.
[5] Banerjee S., Lavie A. "METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments" in Proceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43rd Annual Meeting of the Association of Computational Linguistics, Ann Arbor, Michigan, June 2005.
[6] Переходько И.В., Мячин Д.А., «Оценка качества машинного перевода», 2017. 260с.
[7] Kishore Papineni"BLEU: a Method for Automatic Evaluation of Machine Translation" (2002).
[8] Alon Lavie "METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments" (2004).
[9] Matthew Snover"A Study of Translation Edit Rate with Targeted Human Annotation" (2006).
[10] Chin-Yew Lin "ROUGE: A Package for Automatic Evaluation of Summaries" (2004).
[11] Thibault Sellam, Dipanjan Das, Ankur Parikh "BLEU might be Guilty: A Study of Bias in Evaluation Metrics for Machine Translation" (2019).
[12] Banerjee S., Lavie A. «METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments» in Proceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43rd Annual Meeting of the Association of Computational Linguistics (ACL-2005), Ann Arbor, Michigan, June 2005
[13] Lavie, A., Sagae, K. and Jayaraman, S. (2004) «The Significance of Recall in Automatic Metrics for MT Evaluation» in Proceedings of AMTA 2004, Washington DC. September 2004 Banerjee, S. and Lavie, A. (2005)
[14] Alon Lavie, Abhaya Agarwal. Meteor: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments // Proceedings of the Second Workshop on Statistical Machine Translation. — 2007. — С. 228—231. 26 октября 2020 года.
[15] Michael Denkowski, Alon Lavie.
[https://aclanthology.org/N10-1031.pdf Extending the METEOR Machine Translation Evaluation Metric to the Phrase Level] // Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL. — С. 250–253. 2022.
[16] Elmira Nazirova, Shaxnoza Abidova, Shaxnoza Obidova Mathematical model and algorithm of simple words and simple sentences in the Uzbek language // 2021 International Conference on Information Science and Communications Technologies (ICISCT) (Scopus). Date Added to IEEE Xplore: 17 January 2022. DOI: 10.1109/ICISCT52966.2021.9670202
[17] Nazirova E.Sh., Abidova Sh.B., Hojiev S.A., Jamolov N.F. Construction of a mathematical model of complex word formation in the Uzbek-Karakalpak language. // Scientific horizon in the context of social crises. Proceedings of the 7th International Scientific and Practical Conference. Tokyo-Japan, 6-8.02.2021 y. pp.1117-1123.
[18] Пулатов А.К., Джураева Н. Разработка формальной модели грамматики узбекского языка // Узбекский математический журнал. – Ташкент, 2002. – №1. – С. 47-54.
[19] Коваль С.А. О сравнимости и эквивалентности компьютерных представлений морфологии // Компьютерная лингвистика и интеллектуальные технологии. Тр. междунар. конференции Диалог'2003 (Протвино, 11–16 июня 2003 г.) / Под ред. И.М. Кобозевой, Н.И. Лауфер, В.П. Селегея. М.: Наука, 2003. С. 305–311.
[20] Venkata Sireesha Nagineni, Neerudi Bhoopal, DSNMRAO, Kasireddy Idamakanti, Devineni Gireesh Kumar, Bhanu Teja Bacha “Quality Evaluation and Assessment of Machine Translations for English to Turkish (Google Translations Vs DeepL Translations)” Global Conference on Information Technologies and Communications (GCITC) Karnataka, India. Dec 1-3, 2023