Communication Mechanism in Multi-Agent Reinforcement Learning (MARL) for Cooling Water System Control

Document Type

Conference Proceeding

Publication Date

1-1-2025

Abstract

Reducing the operating frequency of cooling water pumps and towers in a cooling water system lowers their power consumption, but leads to an increase in the return water temperature to chillers, potentially increasing chiller power consumption and creating a trade-off in total system energy use. Achieving optimal energy efficiency involves finding a delicate balance between supply and demand, presenting a complex control challenge that requires real-time monitoring of various operational parameters and informed decision-making for multiple equipment. Recently, reinforcement learning (RL) algorithms have shown promise in addressing this challenge. However, the large action space faced by agent controllers complicates the learning process, as RL agents must explore vast action spaces to make informed decisions. This paper introduces a multi-agent RL control approach to optimize the cooling water system by segmenting the original action space to enable efficient learning. Specifically, the proposed approach includes: (1) a data-driven system model trained on in-situ data; (2)an agent-based control strategy, where the operational states and frequencies of cooling towers and water pumps serve as actions of two distinct agents; (3) a shared state information framework for communication, where agents exchange system feedback (e.g., Coefficient of Performance, or COP) and actions in a structured sequence. Simulation results indicate a noticeable improvement when employing two RL agents, as the reduced action space allows for more efficient exploration. However, the performance comparison between scenarios with and without communication among agents shows no significant difference, suggesting a need to incorporate physics-informed modeling for further research. Additionally, the results imply that cooling towers and cooling water pumps can be treated more independently without compromising system performance.

Publication Source (Journal or Book title)

ASHRAE Transactions

First Page

211

Last Page

219

This document is currently not available here.

Share

COinS