kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университет10.55452/1998-6688-2026-23-2-326-339kaz29-2916Research ArticleКОМПЬЮТЕРНЫЕ НАУКИCOMPUTER SCIENCEЯЗЫКОВАЯ МОДЕЛЬ, ОБЪЕДИНЯЮЩАЯ ОНТОЛОГИЧЕСКИЕ И КОРПУСНЫЕ ДАННЫЕA LANGUAGE MODEL INTEGRATING ONTOLOGICAL AND CORPUS-DATAhttps://orcid.org/0000-0001-9358-1614СамбетбаеваМ.SambetbayevaM.

PhD, ассоциированный профессор, ведущий научный сотрудник.

Алматы, Астана

PhD, Association Professor, leading researcher.

Almaty, Astana

madina_jgtu@mail.ruhttps://orcid.org/0000-0002-2013-1513ЕримбетоваА.YerimbetovaA.

PhD, ассоциированный профессор, ведущий научный сотрудник.

Алматы

PhD, Association Professor, leading researcher.

Almaty

aigerian8888@gmail.comhttps://orcid.org/0009-0001-8872-7428АбдығалымБ.AbdygalymB.

Магистр, младший научный сотрудник.

Алматы, Астана

Doctoral student, junior researcher.

Almaty, Astana

bayangali.abd@gmail.comhttps://orcid.org/0000-0002-4255-5456ДайырбаеваЭ.DaiyrbayevaE.

Магистр, научный сотрудник, старший преподаватель.

Алматы

Master, researcher, senior lecturer.

Almaty

nurbekkyzyelmira@gmail.comhttps://orcid.org/0000-0001-8989-7827ТұрғанбаевА.TurganbayevA.

Инженер-программист.

Алматы

Software engineer.

Almaty

m1challenge@inbox.ruИнститут информационных и вычислительных технологий КН МНВО РК; Евразийский университет им. Л.Н. ГумилеваКазахстанInstitute of Information and Computational Technologies of the Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan; L.N. Gumilyov Eurasian National UniversityKazakhstanИнститут информационных и вычислительных технологий КН МНВО РК; META UniversityКазахстанInstitute of Information and Computational Technologies of the Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan; META UniversityKazakhstanИнститут информационных и вычислительных технологий КН МНВО РК; Казахский национальный исследовательский технический университет им. К.И. СатпаеваРоссияInstitute of Information and Computational Technologies of the Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan; Satbayev UniversityRussian FederationИнститут информационных и вычислительных технологий КН МНВО РКРоссияInstitute of Information and Computational Technologies of the Committee of Science of the Ministry of Science and Higher Education of the Republic of KazakhstanRussian Federation202627062026232326339Copyright © Самбетбаева М., Еримбетова А., Абдығалым Б., Дайырбаева Э., Тұрғанбаев А., 20262026Самбетбаева М., Еримбетова А., Абдығалым Б., Дайырбаева Э., Тұрғанбаев А.Sambetbayeva M., Yerimbetova A., Abdygalym B., Daiyrbayeva E., Turganbayev A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/2916

Настоящее исследование направлено на анализ цифровых угроз в повседневном дискурсе через создание аннотированного корпуса из 1000 текстов, собранных из соцсетей и новостных платформ, связанных с военными и геополитическими событиями. Целью предлагаемого исследования является удовлетворение насущной потребности в анализе цифровых угроз в повседневном дискурсе путем создания аннотированного корпуса текстов с элементами информационных операций. Разработана многоуровневая схема аннотации, охватывающая тип воздействия, эмоциональную окраску, признаки дезинформации и авторские намерения (провокация, запугивание и др.). Аннотирование выполнено в Label Studio с контролем качества и учетом контекста; надежность подтверждена коэффициентом Каппа = 0,82. Модель Onto-IO-BERT в пилотном эксперименте показала F1 = 0,81, превзойдя базовые модели. Практическая применимость корпуса подтверждена анализом реальных Telegram-сообщений. В настоящей работе представлена многоуровневая модель аннотации, адаптированная для анализа военных информационных операций в пространстве Казахстанского МВД, и созданный корпус является новым ресурсом для анализа военных информационных операций, заполнив важный пробел в существующих наборах данных. Представленный корпус содержит тексты на казахском, русском и английском языках. Корпус доступен по ссылке: https://github.com/baiangali/ multi_mil.

This study addresses the urgent need to analyze digital threats in everyday discourse by constructing a 1,000text annotated corpus from social media and news platforms covering military and geopolitical events. The purpose of the proposed study is to address the urgent need to analyze digital threats in everyday discourse by creating an annotated corpus of texts with elements of information operations. A multi-layered annotation scheme captures semantic actors and pragmatic features – including impact type, emotional tone, disinformation markers, and intent (e.g., provocation, intimidation). Annotation via Label Studio ensured flexibility, quality control, and context sensitivity, with inter-annotator reliability (Cohen’s Kappa = 0.82) confirming consistency. In pilot experiments, the Onto-IO-BERT model achieved an F1-score of 0.81, outperforming baseline classifiers. Practical utility was validated through analysis of real Telegram messages. The framework is tailored for studying military information operations within Kazakhstan’s Ministry of Internal Affairs and the created corps is a new resource for analyzing military information operations, filling a significant gap in the existing data set. The presented corpus contains texts in Kazakh, Russian and English. The corpus is openly accessible at: https://github.com/baiangali/multi_mil

военные терминыаннотированный корпусинформационные операцииNLPанализ социальных сетейmilitary termsannotated corpusinformation operationsNLPsocial media analysisReferencesYerimbetova, A., Sakenov, B., Sambetbayeva, M., Daiyrbayeva, E., Berzhanova, U., & Othman, M. Creating a Parallel Corpus for the Kazakh Sign Language and Learning. Applied Sciences, 15(5), 2808 (2025). https://doi.org/10.3390/app15052808Yerimbetova, A., Sakenov, B., Sambetbayeva, M., Daiyrbayeva, E., Berzhanova, U., & Othman, M. Creating a Parallel Corpus for the Kazakh Sign Language and Learning. Applied Sciences, 15(5), 2808 (2025). https://doi.org/10.3390/app15052808Sadirmekova, Zh., Daiyrbayeva, E., Karbozova, I., & Bekbolatov, S. Development of a method for automatic extraction of ontology entity names from natural language texts. In UBMK 2024 Proceedings: 9th International Conference on Computer Science and Engineering, IEEE, 74–79 (2024). https://doi.org/10.1109/UBMK63289.2024.10773503Sadirmekova, Zh., Daiyrbayeva, E., Karbozova, I., & Bekbolatov, S. Development of a method for automatic extraction of ontology entity names from natural language texts. In UBMK 2024 Proceedings: 9th International Conference on Computer Science and Engineering, IEEE, 74–79 (2024). https://doi.org/10.1109/UBMK63289.2024.10773503Mussiraliyeva, S., Bolatbek, M., Zhumakhanova, A., Medetbek, Z., & Sagynay, M. Comparative analysis of machine learning algorithms to identify extremist texts in the Kazakh language. Scientific Journal of Astana IT University, 14(14), 71–90 (2023). https://doi.org/10.37943/14DKRN4681Mussiraliyeva, S., Bolatbek, M., Zhumakhanova, A., Medetbek, Z., & Sagynay, M. Comparative analysis of machine learning algorithms to identify extremist texts in the Kazakh language. Scientific Journal of Astana IT University, 14(14), 71–90 (2023). https://doi.org/10.37943/14DKRN4681Mussiraliyeva, S., Bolatbek, M., Zhumakhanova, A., Baispay, G., & Medetbek, Z. Creating a model of semantic analysis of extremist texts in the Kazakh language. Journal of Mathematics, Mechanics and Computer Science, 121(1), 110–121 (2024). https://doi.org/10.26577/JMMCS2024121111Mussiraliyeva, S., Bolatbek, M., Zhumakhanova, A., Baispay, G., & Medetbek, Z. Creating a model of semantic analysis of extremist texts in the Kazakh language. Journal of Mathematics, Mechanics and Computer Science, 121(1), 110–121 (2024). https://doi.org/10.26577/JMMCS2024121111Rzayeva, L., Pogolovkin, D., & Myrzatay, A. Development of a modular messaging analysis service based on NLP for digital forensics. News of the NAS RK. Series of Physics and Mathematics, 2, 212–233 (2025). https://doi.org/10.32014/2025.2518-1726.354 (in Russian).Rzayeva, L., Pogolovkin, D., & Myrzatay, A. Development of a modular messaging analysis service based on NLP for digital forensics. News of the NAS RK. Series of Physics and Mathematics, 2, 212–233 (2025). https://doi.org/10.32014/2025.2518-1726.354 (in Russian).Modzelewski, A., Da San Martino, G., Savov, P., Wilczyńska, M. A., & Wierzbicki, A. MIPD: Exploring manipulation and intention in a novel corpus of disinformation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 1234–1245 (2024). ACL.Modzelewski, A., Da San Martino, G., Savov, P., Wilczyńska, M. A., & Wierzbicki, A. MIPD: Exploring manipulation and intention in a novel corpus of disinformation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 1234–1245 (2024). ACL.Klein, I. Information operations and influence campaigns. IEEE Phoenix Computer Society, February 2024.Klein, I. Information operations and influence campaigns. IEEE Phoenix Computer Society, February 2024.Sambetbayeva, M., Nekessova, A., Yerimbetova, A., Bayangali, A., Kaldarova, M., Telman, D., & Smailov, N. A Multi-Level Annotation Model for Fake News Detection: Implementing Kazakh-Russian Corpus via Label Studio. Big Data and Cognitive Computing, 9(8), 215 (2025). https://doi.org/10.3390/bdcc9080215Sambetbayeva, M., Nekessova, A., Yerimbetova, A., Bayangali, A., Kaldarova, M., Telman, D., & Smailov, N. A Multi-Level Annotation Model for Fake News Detection: Implementing Kazakh-Russian Corpus via Label Studio. Big Data and Cognitive Computing, 9(8), 215 (2025). https://doi.org/10.3390/bdcc9080215Lin, Y., Wang, H., & Celikyilmaz, A. A survey on recent advances in named entity recognition from deep learning models. ACM Computing Surveys, 55(2), 1–40 (2023). https://doi.org/10.1145/3514221Lin, Y., Wang, H., & Celikyilmaz, A. A survey on recent advances in named entity recognition from deep learning models. ACM Computing Surveys, 55(2), 1–40 (2023). https://doi.org/10.1145/3514221Derczynski, R., Bontcheva, L., & Roberts, K. Broad Twitter corpus: A diverse named entity recognition dataset. In Proceedings of COLING 2016, 1169–1179 (2016).Derczynski, R., Bontcheva, L., & Roberts, K. Broad Twitter corpus: A diverse named entity recognition dataset. In Proceedings of COLING 2016, 1169–1179 (2016).Barrón-Cedeño, A., Rosso, P., & Yu, S. Fine-grained analysis of propaganda in news articles. In Proceedings of EMNLP 2019 , 564–573 (2019).Barrón-Cedeño, A., Rosso, P., & Yu, S. Fine-grained analysis of propaganda in news articles. In Proceedings of EMNLP 2019 , 564–573 (2019).Rashkin, H., Choi, E., Jang, J. Y., Volkova, S., & Choi, Y. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of EMNLP 2017, 2921–2927 (2017).Rashkin, H., Choi, E., Jang, J. Y., Volkova, S., & Choi, Y. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of EMNLP 2017, 2921–2927 (2017).Horn, C., Wiegand, M., & Klakow, D. Towards a multidimensional model of media bias in news articles. In Proceedings of COLING 2018, 498–509 (2018).Horn, C., Wiegand, M., & Klakow, D. Towards a multidimensional model of media bias in news articles. In Proceedings of COLING 2018, 498–509 (2018).Reisigl, M., & Wodak, R. The discourse-historical approach. In Methods of Critical Discourse Studies, 3rd ed., 23–61 (2015).Reisigl, M., & Wodak, R. The discourse-historical approach. In Methods of Critical Discourse Studies, 3rd ed., 23–61 (2015).Van Dijk, T. Critical discourse studies: A sociocognitive approach. In Methods of Critical Discourse Analysis , 62–86 (2011).Van Dijk, T. Critical discourse studies: A sociocognitive approach. In Methods of Critical Discourse Analysis , 62–86 (2011).Zeldes, A. The GUM corpus: Creating multilayer resources in a university setting. Language Resources and Evaluation, 51, 581–612 (2017). https://doi.org/10.1007/s10579-016-9343-4Zeldes, A. The GUM corpus: Creating multilayer resources in a university setting. Language Resources and Evaluation, 51, 581–612 (2017). https://doi.org/10.1007/s10579-016-9343-4Al-Rawi, A. Framing the Syrian conflict on Twitter. Global Media and Communication, 10 (2), 153– 170 (2014). https://doi.org/10.1177/1742766514536022Al-Rawi, A. Framing the Syrian conflict on Twitter. Global Media and Communication, 10 (2), 153– 170 (2014). https://doi.org/10.1177/1742766514536022Vego, E. Effects-based operations: A critique. Joint Forces Quarterly, 41, 51–57 (2006).Vego, E. Effects-based operations: A critique. Joint Forces Quarterly, 41, 51–57 (2006).Dandeker, C. Military language and strategic doctrine. Sociological Review, 54(3), 581–598 (2006). https://doi.org/10.1111/j.1467-954X.2006.00628.xDandeker, C. Military language and strategic doctrine. Sociological Review, 54(3), 581–598 (2006). https://doi.org/10.1111/j.1467-954X.2006.00628.xRöttger, P., Schröder, M., Grotov, A., & Augenstein, I. Harmful but legal: Cognitive framing and large language models in online information warfare. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). ACL.Röttger, P., Schröder, M., Grotov, A., & Augenstein, I. Harmful but legal: Cognitive framing and large language models in online information warfare. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). ACL.Walker, C., Strassel, S., Medero, J., & Maeda, K. ACE 2005 multilingual training corpus. Linguistic Data Consortium. https://catalog.ldc.upenn.edu/LDC2006T06Walker, C., Strassel, S., Medero, J., & Maeda, K. ACE 2005 multilingual training corpus. Linguistic Data Consortium. https://catalog.ldc.upenn.edu/LDC2006T06Wang, X., Wang, Z., Han, X., Jiang, W., Han, R., Liu, Z., Li, J., Li, P., Lin, Y., & Zhou, J. MAVEN: A massive general domain event detection dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),1652–1671 (2020).Wang, X., Wang, Z., Han, X., Jiang, W., Han, R., Liu, Z., Li, J., Li, P., Lin, Y., & Zhou, J. MAVEN: A massive general domain event detection dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),1652–1671 (2020).Li, X., Li, F., Pan, L., Chen, Y., Peng, W., Wang, Q., Lyu, Y., & Zhu, Y. DuEE: A large-scale dataset for Chinese event extraction in real-world scenarios. In X. Zhu, M. Zhang, Y. Hong, & R. He (Eds.), Natural Language Processing and Chinese Computing, 534–545 (2020).Li, X., Li, F., Pan, L., Chen, Y., Peng, W., Wang, Q., Lyu, Y., & Zhu, Y. DuEE: A large-scale dataset for Chinese event extraction in real-world scenarios. In X. Zhu, M. Zhang, Y. Hong, & R. He (Eds.), Natural Language Processing and Chinese Computing, 534–545 (2020).Ebner, S., Xia, P., Culkin, R., Rawlins, K., & Van Durme, B. Multi-sentence argument linking. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8057–8077 (2020).Ebner, S., Xia, P., Culkin, R., Rawlins, K., & Van Durme, B. Multi-sentence argument linking. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8057–8077 (2020).Li, S., Ji, H., & Han, J. Document-level event argument extraction by conditional generation. In Proceedings of NAACL-HLT 2021, 894–908 (2021).Li, S., Ji, H., & Han, J. Document-level event argument extraction by conditional generation. In Proceedings of NAACL-HLT 2021, 894–908 (2021).Tong, M., Xu, B., Wang, S., Han, M., Cao, Y., Zhu, J., Chen, S., Hou, L., & Li, J. DocEE: A largescale and fine-grained benchmark for document-level event extraction. In Proceedings of NAACL-HLT 2022, 3970–3982 (2022).Tong, M., Xu, B., Wang, S., Han, M., Cao, Y., Zhu, J., Chen, S., Hou, L., & Li, J. DocEE: A largescale and fine-grained benchmark for document-level event extraction. In Proceedings of NAACL-HLT 2022, 3970–3982 (2022).Huang, H., Sun, J., Wei, H., Xiao, K., Wang, M., & Li, X. A dataset of domain events based on opensource military news. China Scientific Data, 8(1), 30 (2023). https://doi.org/10.11922/sciencedata.2023.0005.zhHuang, H., Sun, J., Wei, H., Xiao, K., Wang, M., & Li, X. A dataset of domain events based on opensource military news. China Scientific Data, 8(1), 30 (2023). https://doi.org/10.11922/sciencedata.2023.0005.zhZhu, M., Xu, Z., Zeng, K., Xiao, K., Wang, M., Ke, W., & Huang, H. CMNEE: A large-scale documentlevel event extraction dataset based on open-source Chinese military news. In Proceedings of LREC-COLING 2024, 3367–3379 (2024).Zhu, M., Xu, Z., Zeng, K., Xiao, K., Wang, M., Ke, W., & Huang, H. CMNEE: A large-scale documentlevel event extraction dataset based on open-source Chinese military news. In Proceedings of LREC-COLING 2024, 3367–3379 (2024).Chen, L.-C., Chang, K.-H., & Yang, S.-C. Integrating corpus-based and NLP approach to extract terminology and domain-oriented information: An example of US military corpus. Acta Scientiarum. Technology, 44, 2022. https://doi.org/10.4025/actascitechnol.v44i1.60486Chen, L.-C., Chang, K.-H., & Yang, S.-C. Integrating corpus-based and NLP approach to extract terminology and domain-oriented information: An example of US military corpus. Acta Scientiarum. Technology, 44, 2022. https://doi.org/10.4025/actascitechnol.v44i1.60486Li, H., Zhu, S.-C., & Zheng, Z. (2023). DiPlomat: A dialogue dataset for situated pragmatic reasoning. arXiv preprint. https://arxiv.org/abs/2306.09030Li, H., Zhu, S.-C., & Zheng, Z. (2023). DiPlomat: A dialogue dataset for situated pragmatic reasoning. arXiv preprint. https://arxiv.org/abs/2306.09030Label Studio Documentation. Label Studio annotator guide, 2023. https://labelstud.io/guideLabel Studio Documentation. Label Studio annotator guide, 2023. https://labelstud.io/guideInternational Organization for Standardization. ISO 24617-1:2012 – Language resource management – Semantic annotation framework (SemAF) – Part 1: Time and events (ISO-TimeML). https://www.iso.org/standard/37331.htmlInternational Organization for Standardization. ISO 24617-1:2012 – Language resource management – Semantic annotation framework (SemAF) – Part 1: Time and events (ISO-TimeML). https://www.iso.org/standard/37331.htmlInternational Organization for Standardization. ISO 24617-6:2016 – Language resource management – Semantic annotation framework – Part 6: Principles of semantic annotation (SemAF Principles). https://www.iso.org/standard/60581.htmlInternational Organization for Standardization. ISO 24617-6:2016 – Language resource management – Semantic annotation framework – Part 6: Principles of semantic annotation (SemAF Principles). https://www.iso.org/standard/60581.htmlWardle, C., & Derakhshan, H. Information disorder: Toward an interdisciplinary framework for research and policymaking. Council of Europe, 2017. https://shorensteincenter.org/information-disorderframework-for-research-and-policymaking/Wardle, C., & Derakhshan, H. Information disorder: Toward an interdisciplinary framework for research and policymaking. Council of Europe, 2017. https://shorensteincenter.org/information-disorderframework-for-research-and-policymaking/Joint Chiefs of Staff. (2012). Joint publication 3-13: Information operations. U.S. Department of Defense. https://irp.fas.org/doddir/dod/jp3_13.pdfJoint Chiefs of Staff. (2012). Joint publication 3-13: Information operations. U.S. Department of Defense. https://irp.fas.org/doddir/dod/jp3_13.pdfPresident of the Republic of Kazakhstan. (2023, March 20). On the approval of the information doctrine of the Republic of Kazakhstan (Decree No. 145). https://adilet.zan.kz/rus/docs/U2300000145President of the Republic of Kazakhstan. (2023, March 20). On the approval of the information doctrine of the Republic of Kazakhstan (Decree No. 145). https://adilet.zan.kz/rus/docs/U2300000145Schmitt, M. N. (Ed.). (2017). Tallinn Manual 2.0 on the international law applicable to cyber operations. Cambridge University Press.Schmitt, M. N. (Ed.). (2017). Tallinn Manual 2.0 on the international law applicable to cyber operations. Cambridge University Press.Artstein, R., & Poesio, M. Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596 (2008). https://doi.org/10.1162/coli.07-034-R2Artstein, R., & Poesio, M. Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596 (2008). https://doi.org/10.1162/coli.07-034-R2

The authors declare that there are no conflicts of interest present.