ARTIFICIAL INTELLIGENCE TO DEVELOP AN ADAPTIVE LEARNING SYSTEM TO DEVELOP CRITICAL THINKING SKILLS IN STUDENTS

The article presents the results of the development and pilot testing of an adaptive learning system designed to develop and assess students’ critical thinking using elements of artificial intelligence. The scientific novelty of the study, in comparison with existing research on automated essay scoring and LLM-based assessment, lies in the multi-component assessment of critical thinking, the cascade mechanism of AI-based assessment, and the appeal mechanism. These features make it possible to more accurately identify students’ weak points in the development of critical thinking, ensure transparent assessment of students’ critical thinking, increase the system’s resilience to failures, and provide students with the opportunity to challenge the assessment in controversial situations. The system evaluates essays and problem-solving tasks according to six components of critical thinking: analysis, interpretation, evaluation, argumentation, logical coherence, and reflection. For each component, calculation formulas were proposed; these are computed using language models, semantic embeddings based on sentence-transformers, and a backup TF-IDF mechanism. The pilot testing was conducted on a sample of 20 students from Satbayev University. Three instructors, represented by the authors, participated in the expert assessment. The experimental data included 40 student works: 20 essays and 20 problem-solving tasks. Each work was independently assessed by three instructors as well as by the learning system. The reference score was determined as the average score of the three instructors. The correlation between the automated assessment and the averaged expert assessment was r = 0.84, while the mean deviation was 1.0 point on a ten-point scale. The results obtained indicate that the system can be used as a tool to support instructors; however, further research on a larger sample is required. Gamification elements, progress visualization, and the appeal mechanism increase students’ engagement and ensure a balance between automated and expert assessment.

1. Batdı, V., Elaldı, Ş., Özçelik, C. et al. Evaluation of the effectiveness of critical thinking training on critical thinking skills and academic achievement by using mixed-meta method. Review of Education, 12, e70001 (2024). https://doi.org/10.1002/rev3.70001

2. Almulla, M.A. Constructivism Learning Theory: A Paradigm for Students' Critical Thinking, Creativity, and Problem Solving to Affect Academic Performance in Higher Education. Sustainability, 15 (17), 12903 (2023). https://doi.org/10.1080/2331186X.2023.2172929

3. Pack, A., Barrett, A., and Escalante, J. Large language models and automated essay scoring of English language learner writing: Insights into validity and reliability. Computers and Education: Artificial Intelligence, 6, 100234 (2024). https://doi.org/10.1016/j.caeai.2024.100234

4. Facione, P.A. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction (The Delphi Report) (Millbrae, CA: California Academic Press, 1990).

5. Uto, M. A Review of Deep-Neural Automated Essay Scoring Models. Behaviormetrika, 48 (2), 459– 484 (2021). https://doi.org/10.1007/s41237-021-00142-y

6. Lee, S., Cai, Y., Meng, D., Wang, Z., and Wu, Y. Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoring. arXiv preprint, arXiv:2404.04941 (2024). https://arxiv.org/abs/2404.04941

7. Stahl, M., Biermann, L., Nehring, A., and Wachsmuth, H. Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation. Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) (Mexico City: Association for Computational Linguistics, 2024), pp. 283–298. https://aclanthology.org/2024.bea-1.23/

8. Chu, S.Y., Kim, J.W., Wong, B., and Yi, M.Y. Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs. Findings of the Association for Computational Linguistics: NAACL 2025 (Albuquerque, New Mexico: Association for Computational Linguistics, 2025), pp. 5811–5829. https://doi.org/10.18653/v1/2025.findings-naacl.322

9. Bloom, B.S. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain (New York: David McKay Company, 1956).

10. Reimers, N., and Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2019). https://doi.org/10.18653/v1/D19-1410

11. Klebanov, B.B., and Madnani, N. Automated Essay Scoring. Synthesis Lectures on Human Language Technologies (Morgan & Claypool Publishers, 2024).