MODELING THE DEVELOPMENT OF LEMMATIZATION ALGORITHMS FOR THE UZBEK LANGUAGE

Maksud S. Sharipov, Ogabek O. Sobirov
Bulletin of TUIT: Management and Communication Technologies
№ 1(8)2025 DOI:10.61663/251tuitmct8
Open article file

Abstract

Lemmatization is a vital task in Natural Language Processing (NLP), essential for various applications. However, the Uzbek language, categorized as a low-resource language in NLP, lacks dedicated lemmatization systems. This deficiency hinders the development of advanced NLP tools for Uzbek texts. To address this, we present a structured approach using IDEF0 and IDEF1X models to design an Uzbek lemmatization system. The IDEF0 model details the system's functionality, outlining the lemmatization process, while the IDEF1X model illustrates the relationships between data components. Furthermore, we developed a rule-based system that leverages Uzbek morphology to accurately determine word lemmas. This research contributes a well-defined architectural framework and a practical rule-based solution for lemmatization, aiding in the advancement of NLP for Uzbek and similar low-resource, morphologically rich languages.

References

[1]“https://en.wikipedia.org/wiki/Uzbek_language,” Wikipedia.
[2] M. Sharipov and O. Sobirov, “Development of a Rule-Based Lemmatization Algorithm Through Finite State Machine for Uzbek Language,” in CEUR Workshop Proceedings, V. J. and K. B., Eds., CEUR-WS, 2022, pp. 154 – 159.
[Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85146112590&partnerID=40&md5=e1080c39d101c0e351cfed1a8228d391
[3] O. Sobirov, “Lemmatization algorithms and their application to uzbek: approaches based on Artificial Intelligence, Machine Learning, and Deep Learning,” Electronic journal of actual problems of modern science, education and training. ISSN 2181-9750, pp. 4–13, Dec. 2024.
[4] M. Abjalova, E. Adalı, and M. Adilova, “The Process of Lemmatization and Stemming in the Automatic Morphological Analysis of Uzbek Texts,” in 2024 9th International Conference on Computer Science and Engineering (UBMK), 2024, pp. 1–6. doi: 10.1109/UBMK63289.2024.10773506.
[5] National Institute of Standards and Technology, “Integration Definition for Function Modeling (IDEF0),” 1993.
[6] Y. Kuratov and M. Arkhipov, “Adaptation of deep bidirectional multilingual transformers for Russian language,” in Komp’juternaja Lingvistika i Intellektual’nye Tehnologii, 2019.
[7] N. Arefyev, B. Sheludko, A. Podolskiy, and A. Panchenko, “Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution,” in COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference, 2020. doi: 10.18653/v1/2020.coling-main.107.
[8] Ç. Çöltekin, “TRmorph: A morphological analyzer for Turkish,” Let.Rug.Nl, 2013.
[9] M. S. Sharipov, H. S. Adinaev, and E. R. Kuriyozov, “Rule-Based Punctuation Algorithm for the Uzbek Language,” in 2024 IEEE 25th International Conference of Young Professionals in Electron Devices and Materials (EDM), 2024, pp. 2410–2414. doi: 10.1109/EDM61683.2024.10615061.
[10] M. Sharipov, E. Kuriyozov, O. Yuldashev, and O. Sobirov, “UzbekTagger: The rule-based POS tagger for Uzbek language,” arXiv preprint arXiv:2301.12711, 2023.
[11] S. Maksud, K. Elmurod, Y. Ollabergan, and S. Ogabek, “UzbekVerbDetection: Rule-based Detection of Verbs in Uzbek Texts,” in 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, 2024, pp. 17343 – 17347.
[Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85195974039&partnerID=40&md5=ff3003ea2644833f3dc45277429459f7
[12] X. Madatov, S. Xajibaeva, and E. Xujamov, “‘O ‘ZBEK BOSHLANG ‘ICH SINF MAKTAB KORPUSI’ DAN BAZIS SO ‘ZLARNI AVTOMATIK AJRATIB OLISH,” COMPUTER LINGUISTICS: PROBLEMS, SOLUTIONS, PROSPECTS, vol. 1, no. 1, 2025.
[13] U. Salaev, “Modeling Morphological Analysis based on Word-ending for Uzbek Language,” vol. 2, pp. 29–34, Jun. 2023, doi: 10.5281/zenodo.10155225.