Uncertainty-Aware LLM-Guided Cloud Resource Scheduling with Budgeted Robust Optimization
DOI:
https://doi.org/10.66525/ijictt132Keywords:
cloud resource scheduling, robust optimization, conformal prediction, confidence calibration, large language models, service level agreementsAbstract
Hybrid schedulers that let a large language model propose placements and a solver enforce feasibility have brought semantic reasoning into cluster resource management. Such pipelines still treat two quantities as exact: the model proposal itself, and the resource demands and runtimes attached to each task. Both are estimates, and a scheduler that ignores their error either over-trusts a wrong suggestion or packs a node to a capacity it does not have. This paper presents URSA, a scheduler that carries uncertainty through the whole decision path. Confidence in each model proposal is obtained from the agreement of temperature-diverse samples and mapped to a hit probability by post-hoc calibration, which then sets a per-task trust weight in the objective. Runtime prediction error is turned into finite-sample upper bounds by split conformal prediction at a level selected from each job's remaining deadline slack. Declared-versus-actual demand deviation is absorbed by a budgeted robust assignment program whose conservatism is controlled by a single parameter. Evaluation on real Spark TPC-H DAGs with measured task-duration profiles shows that URSA removes 95-96% of node memory-pressure and eviction events relative to a deterministic guided solver at equal job completion time, shortens makespan by 4-13%, and raises CPU utilization by up to 11%. Its behaviour stays flat as guidance reliability falls from 0.90 to 0.30 and under over-confident guidance, whereas the deterministic pipeline swings by up to 47% in utilization.
Downloads
References
[1] Xing, S., Wang, Y., & Liu, W. (2025). Self-adapting CPU scheduling for mixed database workloads via hierarchical deep reinforcement learning. Symmetry, 17(7), 1109.
[2] Ji, E., Wang, Y., Xing, S., & Jin, J. (2025). Hierarchical reinforcement learning for energy-efficient API traffic optimization in large-scale advertising systems. IEEE Access.
[3] Chen, T., Shang, W., & Guo, Z. (2024). Xsla: An SLA-aware heterogeneous XPU scheduling framework for edge AI inference. World Journal of Information Technology, 2(2), 48–59.
[4] Zou, J., Qin, Y., & Rhee, M. (2024). Beyond model scale: A task-aware reliability benchmark for large language models in clinical decision support. Innovation and Technology Studies, 1(1), 39–47.
[5] Fang, Q. (2024). Bias-aware large language models for equitable educational feedback and assessment. Trends in Social Sciences and Humanities Research, 2(1), 41–50.
[6] Qin, Y., Zou, J., & Rhee, M. (2022). An AI-driven intelligent transaction execution framework for internet finance platforms: Cost-sensitive real-time integrity interdiction. Eurasia Journal of Science and Technology, 4(2), 36–44.
[7] Wang, Y., & Zhang, C. (2023). AERIS: Adaptive executor scoped resource management for privacy preserving data processing in Apache Spark. Eurasia Journal of Science and Technology, 5(2), 25-33.
[8] Zhao, W., & Wang, B. (2022). CAPA: A prediction‑driven autoscaling framework for SLO‑aware cloud‑native machine learning inference. Journal of Computer Science and Electrical Engineering, 4(1), 8-16.
[9] Guo, Z., Chen, T., & Shang, W. (2022). Class‑conditional intermediate‑domain adversarial adaptation for cross‑device acoustic scene classification. Journal of Computer Science and Electrical Engineering, 4(1), 26-35
[10] Shen, A. (2023). Compute-efficient scaling of large sequential recommendation models under dynamic user behavior. World Journal of Information Technology, 1(1), 62–71.
[11] Fang, Q. (2024). True-EWS: Trustworthy early warning with equalized conformal guarantees for equitable student support in U.S. education. Educational Research and Human Development, 1(2), 44–54.
[12] Shen, A., & Huang, S. (2022). A systems-aware framework for intelligent large-scale recommendation models. Journal of Computer Science and Electrical Engineering, 4(1), 36–44.Mai, N. T., Cao, W., & Fang, Q. (2025). A study on how LLMs (eg GPT-4, chatbots) are being integrated to support tutoring, essay feedback and content generation. Journal of Computing and Electronic Information Management, 18(3), 43-52.
[13] Fang, Q., & Liu, W. (2026). Neuro-symbolic pedagogical alignment (NSPA) for long-horizon classroom discourse analysis: Mitigating dialect bias via counterfactual preference optimization. Computers and Education: Artificial Intelligence, 100664.
[14] Fang, Q. (2024). TRUE-EWS: Trustworthy early warning with equalized conformal guarantees for equitable student support in U.S. education. Educational Research and Human Development.
[15] Shen, A. (2023). Compute-efficient scaling of large sequential recommendation models under dynamic user behavior. World Journal of Information Technology..
[16] D. Narayanan, K. Santhanam, F. Kazhamiaka, A. Phanishayee, and M. Zaharia, "Heterogeneity-aware cluster scheduling policies for deep learning workloads," in Proc. USENIX OSDI, 2020, pp. 481–498.
[17] A. Qiao et al., "Pollux: co-adaptive cluster scheduling for goodput-optimized deep learning," in Proc. USENIX OSDI, 2021, pp. 1–18.
[18] S. Kambhampati et al., "Position: LLMs can't plan, but can help planning in LLM-modulo frameworks," in Proc. Int. Conf. Mach. Learn. (ICML), 2024.
[19] K. Valmeekam, A. Olmo, S. Sreedharan, and S. Kambhampati, "On the planning abilities of large language models—a critical investigation," in Proc. NeurIPS, 2023, pp. 75993–76005.
[20] T. Brown et al., "Language models are few-shot learners," in Proc. NeurIPS, 2020, pp. 1877–1901.
[21] J. Wei et al., "Chain-of-thought prompting elicits reasoning in large language models," in Proc. NeurIPS, 2022, pp. 24824–24837.
[22] X. Wang et al., "Self-consistency improves chain of thought reasoning in language models," in Proc. Int. Conf. Learn. Represent. (ICLR), 2023.
[23] OpenAI, "GPT-4 technical report," 2023, arXiv:2303.08774.
[24] H. Touvron et al., "Llama 2: open foundation and fine-tuned chat models," 2023, arXiv:2307.09288.
[25] M. Chen et al., "Evaluating large language models trained on code," 2021, arXiv:2107.03374.
[26] S. Kadavath et al., "Language models (mostly) know what they know," 2022, arXiv:2207.05221.
[27] G. Ding, S. Yang, H. Lin, Z. Chen, and J. S. Yang, "LLM-driven adaptive cloud resource scheduling: bridging reasoning intelligence with optimization guarantees," IEEE Open J. Comput. Soc., vol. 7, pp. 560–573, 2026.
[28] M. Xiong, Z. Hu, X. Lu, Y. Li, J. Fu, J. He, and B. Hooi, "Can LLMs express their uncertainty? An empirical evaluation of confidence elicitation in LLMs," in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
[29] S. Lin, J. Hilton, and O. Evans, "Teaching models to express their uncertainty in words," Trans. Mach. Learn. Res., 2022.
[30] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, "On calibration of modern neural networks," in Proc. Int. Conf. Mach. Learn. (ICML), 2017, pp. 1321–1330.
[31] B. Zadrozny and C. Elkan, "Transforming classifier scores into accurate multiclass probability estimates," in Proc. ACM SIGKDD, 2002, pp. 694–699.
[32] D. Bertsimas and M. Sim, "The price of robustness," Oper. Res., vol. 52, no. 1, pp. 35–53, 2004.
[33] A. Ben-Tal and A. Nemirovski, "Robust solutions of uncertain linear programs," Oper. Res. Lett., vol. 25, no. 1, pp. 1–13, 1999.
[34] D. Bertsimas, D. B. Brown, and C. Caramanis, "Theory and applications of robust optimization," SIAM Rev., vol. 53, no. 3, pp. 464–501, 2011.
[35] E. Delage and Y. Ye, "Distributionally robust optimization under moment uncertainty with application to data-driven problems," Oper. Res., vol. 58, no. 3, pp. 595–612, 2010.
[36] V. Vovk, A. Gammerman, and G. Shafer, Algorithmic Learning in a Random World. New York, NY, USA: Springer, 2005.
[37] Wang, Y., & Xing, S. (2025). AI-driven CPU resource management in cloud operating systems. Journal of Computer and Communications, 13(06), 135-149.
[38] Xing, S., & Wang, Y. (2025). Cross-modal attention networks for multi-modal anomaly detection in system software. IEEE Open Journal of the Computer Society.
[39] J. D. Ullman, "NP-complete scheduling problems," J. Comput. Syst. Sci., vol. 10, no. 3, pp. 384–393, 1975.
[40] Yue, X., & Zhu, R. (2024). An explainable hybrid machine learning framework for cash-flow-conditioned multi-horizon liquidity risk early warning: An SME-oriented benchmark study. World Journal of Management Science, 2(1), 63-72.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Amelia Turner (Author)

This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in the International Journal of Information and Communication Technology Trends (IJICTT) are distributed under a Creative Commons license. This allows readers to freely access, share, and adapt the content for non-commercial purposes, provided proper attribution is given to the original authors. Authors retain copyright of their work.
