A Priority-ordered Decision Model for Educational LLM Agents
DOI:
https://doi.org/10.18372/1990-5548.89.21471Keywords:
adaptive learning systems, divergence detection, educational decision support, learning analytics, reference architecture, tool-augmented LLM agentsAbstract
This work is devoted to the design of a decision-support architecture for artificial intelligence agents used in higher-education. The paper proposes a priority-ordered decision model that ranks three evaluation signals – divergence between self-assessment and teacher assessment, performance trend, and peer comparison – before any call to a language model, so that the same input data always yields the same intervention priority. The model is based on a deterministic progress-weighting service defined by equations (1) – (9), with a course-configurable divergence threshold (τ = 0.15 in the reference configuration) and four weight coefficients that sum to 1. The paper develops a reference implementation with a typed evidence schema, a FastAPI backend, and a tool-augmented LLM agent restricted to four read-only tools. A worked example shows that a flagged divergence (d = 0.25 > τ) overrides an improving trend (L: 0.58 → 0.64 → 0.67), correctly prioritizing a calibration alert over a routine update. The results can be used to design auditable, LLM-based recommendation systems for education and other domains where several evidence signals must be reconciled before a generative model is invoked.
References
R. Sajja, Y. Sermet, M. Cikmaz, D. Cwiertny, and I. Demir, “Artificial intelligence-enabled intelligent assistant for personalized and adaptive learning in higher education,” Information, vol. 15, no. 10, Article 596, 2024. https://doi.org/10.3390/info15100596
[2] R. K. Jangale, S. B. Wankhade, S. N. Raut, and Y. S. Kute, “The impact of AI assistants on students' academic performance and learning behavior,” International Journal on Science and Technology, 2025. URL: https://www.ijsat.org/papers/2025/4/9222.pdf
[3] Amplify Learn. AI, AI as a teaching partner: project report, Colleague AI Classroom. 2025. URL: https://www.amplifylearn.ai/wp-content/uploads/2025/12/AI_as_teaching_partner.pdf
[4] R. Sajja, Y. Sermet, B. Fodale, and I. Demir, “Evaluating AI-powered learning assistants in engineering higher education: student engagement, ethical challenges, and policy implications,” arXiv. 2025. https://doi.org/10.48550/arXiv.2506.05699
[5] Z. Xu, F. H. Choudhury, S. Ma, T. P. Murphrey, and K. E. Dooley, “Leveraging learning analytics to model student engagement in graduate statistics: a problem-based learning approach in agricultural education,” Behavioral Sciences, vol. 15, no. 10, Article 1360, 2025. https://doi.org/10.3390/bs15101360
[6] H. Xue and Y. Niu, “Multi-output based hybrid integrated models for student performance prediction,” Applied Sciences, vol. 13, no. 9, Article 5384, 2023. https://doi.org/10.3390/app13095384
[7] L. Loble and J. Lodge, “Artificial intelligence, cognitive offloading and implications for education,” University of Technology Sydney / Australian Network for Quality Digital Education. 2026. URL: https://www.uts.edu.au/news/2026/03/experts-warn-unstructured-ai-use-in-schools-risks-cognitive-atrophy
[8] C. Herlihy, K. Truong, A. Chouldechova, and M. Dudík, “A structured regression approach for evaluating model performance across intersectional subgroups,” arXiv. 2024. https://doi.org/10.48550/arXiv.2401.14893
[9] E.-M. Schön, M. Neumann, C. Hofmann-Stölting, R. Baeza-Yates, and M. Rauschenberger, “How are AI assistants changing higher education?” Frontiers in Computer Science, vol. 5, Article 1208550. 2023. https://doi.org/10.3389/fcomp.2023.1208550
[10] C. Cohn, S. Rayala, N. Srivastava, J. H. Fonteles, S. Jain,et al., “A theory of adaptive scaffolding for LLM-based pedagogical agents,” arXiv. 2025. https://doi.org/10.48550/arXiv.2508.01503
[11] C. Cohn, S. Guo, S. Rayala, H. D. Wang, N. Mohammed, et al., Evidence-decision-feedback: theory-driven adaptive scaffolding for LLM agents. arXiv. 2026. https://doi.org/10.48550/arXiv.2602.01415
[12] Y. Fan, L. Tang, H. Le, K. Shen, S. Tan, et al., Beware of metacognitive laziness: effects of generative artificial intelligence on learning motivation, processes, and performance, British Journal of Educational Technology. 2024. https://doi.org/10.1111/bjet.13544
[13] A. C. Graesser, S. D'Mello, and W. Cade, “Instruction based on tutoring. Handbook of research on learning and instruction,” ed. by R. E. Mayer, P. A. Alexander. Routledge, 2011, pp. 408–426.
[14] A. Mavroudi and G. Torgersen, “A large language model implementing an AI assistant in a higher education setting,” CEUR Workshop Proceedings, vol. 3685, 2024. URL: https://ceur-ws.org/Vol-3685/short13.pdf
[15] H. Lee and N. Bosch, “Calibration discrepancy predicts students' subsequent metacognitive strategy use in computer-based learning environments,” International Journal of Artificial Intelligence in Education, vol. 35, no. 6, pp. 3746–3779. 2025. https://doi.org/10.1007/s40593-025-00514-5
[16] M. D. Johnson, Learners' self-assessment and metacognition when using an open learner model with drill down: PhD thesis. University of Birmingham, 2018. URL: https://etheses.bham.ac.uk/id/eprint/8590/
[17] C. Milton, V. Lokesh, and G. Thiruvengadam, “Examining the impact of AI-powered writing tools on independent writing skills of health science graduates,” Advanced Education, vol. 12, no. 25, pp. 143–161, 2024. https://doi.org/10.20535/2410-8286.315068
[18] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, et al., “ReAct: synergizing reasoning and acting in language models,” Proceedings of the 11th International Conference on Learning Representations (ICLR 2023), 2023. https://doi.org/10.48550/arXiv.2210.03629
[19] T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, et al., “Toolformer: language models can teach themselves to use tools,” Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 68539–68551. https://doi.org/10.52202/075280-2997
[20] E. Schluntz and B. Zhang, Building effective agents. Anthropic Engineering Blog. 2024. URL: https://www.anthropic.com/engineering/building-effective-agents.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
The scientific journal “Electronics and control systems” adheres to the principles of Open Access and provides free, immediate, and permanent access to all published materials without financial, technical, or legal barriers for readers.
All articles are published in Open Access under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.
Copyright
Authors who publish their works in the journal “Electronics and control systems”:
-
retain the copyright to their publications;
-
grant the journal the right of first publication of the article;
-
agree to the distribution of their materials under the CC BY 4.0 license;
-
have the right to reuse, archive, and distribute their works (including in institutional and subject repositories), provided that proper reference is made to the original publication in the journal.
