IMPLEMENTATION OF MULTI-AGENT FRAMEWORK OF IMPALA FOR A SINGLE ZONE TEMPERATURE CONTROL OF A SIMULATED THERMAL ZONE

Buildings account for a significant portion of global energy consumption, with HVAC and humidity-control systems representing the majority of their operational demand. Traditional rule-based strategies often fail to adapt to dynamic indoor-outdoor conditions, motivating the use of data-driven control methods. This study presents a multiagent reinforcement learning (MARL) framework for simultaneous temperature and humidity control in a singlezone building modeled in EnergyPlus. The proposed approach employs the distributed Importance Weighted ActorLearner Architecture (IMPALA) algorithm with centralized training and decentralized execution (CTDE), enabling two agents: temperature and humidity to learn coordinated policies directly from high-fidelity simulation feedback. The results demonstrate strong learning performance: both agents improved their per-step rewards substantially (temperature +18.9%, humidity +33.7%), indicating effective convergence and cooperative behavior. The learned controller maintained thermal comfort comparable to the rule-based baseline (mean occupied temperature difference ≈ 0.04 °C; occupied PMV ≈ 0.45) while achieving notable energy savings. Total annual HVAC energy consumption decreased by 8.9%, with the most significant improvement observed in humidification energy, which was reduced by 34.4%. Heating and cooling loads remained nearly unchanged, confirming that energy reductions were achieved without compromising comfort.

1. Dean, B., et al. Towards zero-emission efficient and resilient buildings. Global Status Report (2016).

2. Razmara, M., et al. Optimal exergy control of building HVAC system. Applied Energy, 156, 555–565 (2015). https://doi.org/10.1016/j.apenergy.2015.07.051

3. Chua, K.J., et al. Achieving better energy-efficient air conditioning – a review of technologies and strategies. Applied Energy, 104, 87–104 (2013). https://doi.org/10.1016/j.apenergy.2012.10.037

4. Maasoumy, M., et al. Handling model uncertainty in model predictive control for energy efficient buildings. Energy and Buildings, 77, 377–392 (2014). https://doi.org/10.1016/j.enbuild.2014.03.057

5. Salakij, S., et al. Model-Based Predictive Control for building energy management. I: Energy modeling and optimal control. Energy and Buildings, 133, 345–358 (2016). https://doi.org/10.1016/j.enbuild.2016.09.044

6. Yang, S., et al. Experiment study of machine-learning-based approximate model predictive control for energy-efficient building control. Applied Energy, 288, 116648 (2021). https://doi.org/10.1016/j.apenergy.2021.116648

7. Crawley, D.B., et al. EnergyPlus: creating a new-generation building energy simulation program. Energy and Buildings, 33(4), 319–331 (2001). https://doi.org/10.1016/S0378-7788(00)00114-6

8. Strachan, P.A., Kokogiannakis, G., Macdonald, I.A. History and development of validation with the ESP-r simulation program. Building and Environment, 43(4), 601–609 (2008). https://doi.org/10.1016/j.buildenv.2006.06.025

9. Salvalai, G. Implementation and validation of simplified heat pump model in IDA-ICE energy simulation environment. Energy and Buildings, 49, 132–141 (2012). https://doi.org/10.1016/j.enbuild.2012.01.038

10. Shrivastava, R.L., Kumar, V., Untawale, S.P. Modeling and simulation of solar water heater: A TRNSYS perspective. Renewable and Sustainable Energy Reviews, 67, 126–143 (2017). https://doi.org/10.1016/j.rser.2016.09.005

11. Koeln, J., et al. Multi-zone temperature modeling and control. In: Intelligent Building Control Systems: A Survey of Modern Building Control and Sensing Strategies. Springer, Cham, 139–166 (2017). https://doi.org/10.1007/978-3-319-68462-8_6

12. Perera, D.W.U., Pfeiffer, C.F., Skeie, N.-O. Control of temperature and energy consumption in buildings – a review. International Journal of Energy & Environment, 5(4) (2014).

13. Sutton, R.S., Barto, A.G. Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998). https://doi.org/10.1017/S0263574799271172

14. Li, F., Du, Y. Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning. In: Deep Learning for Power System Applications. Springer, Cham, 71–96 (2023). https://doi.org/10.1007/978-3-031-45357-1_4

15. Barrett, E., Linder, S. Autonomous HVAC control, a reinforcement learning approach. In: ECML PKDD. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23461-8_1

16. Mocanu, E., et al. On-line building energy optimization using deep reinforcement learning. IEEE Transactions on Smart Grid, 10(4), 3698–3708 (2018). https://doi.org/10.1109/TSG.2018.2834219

17. Li, Y., et al. Transforming cooling optimization for green data center via deep reinforcement learning. IEEE Transactions on Cybernetics, 50(5), 2002–2013 (2019). https://doi.org/10.1109/TCYB.2019.2927410

18. Liu, B., Akcakaya, M., McDermott, T.E. Automated control of transactive HVACs in energy distribution systems. IEEE Transactions on Smart Grid, 12(3), 2462–2471 (2020). https://doi.org/10.1109/TSG.2020.3042498

19. Kazmi, H., et al. Multi-agent reinforcement learning for modeling and control of thermostatically controlled loads. Applied Energy, 238, 1022–1035 (2019). https://doi.org/10.1016/j.apenergy.2019.01.140

20. Blad, C., Bøgh, S., Kallesøe, C. A multi-agent reinforcement learning approach to price and comfort optimization in HVAC-systems. Energies, 14(22), 7491 (2021). https://doi.org/10.3390/en14227491

21. Espeholt, L., et al. IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures. Proceedings of ICML (2018).

22. EnergyPlus Weather Data. URL: https://energyplus.net/weather (accessed: 2026.06.01).