Optimal Control of Combined Sewer Systems to Minimize Sewer Overflows by Using Reinforcement Learning
Conference
Yin, Z, Leon, AS, Sharifi, A et al. (2023). Optimal Control of Combined Sewer Systems to Minimize Sewer Overflows by Using Reinforcement Learning
. 711-722. 10.1061/9780784484852.067
Yin, Z, Leon, AS, Sharifi, A et al. (2023). Optimal Control of Combined Sewer Systems to Minimize Sewer Overflows by Using Reinforcement Learning
. 711-722. 10.1061/9780784484852.067
A combined sewer system (CSS) collects rainwater runoff, domestic sewage, and industrial wastewater in the same pipe. The volume of wastewater can sometimes exceed the system capacity during heavy rainfall events. When this occurs, untreated stormwater and wastewater discharge directly to nearby streams, rivers, and other water bodies. This would threaten public health and the environment, contributing to drinking water contamination and other concerns. Minimizing sewer overflows requires an optimization method that can provide an optimal sequence of decision variables at control gates. Conventional strategies use classical optimization algorithms, such as genetic algorithms and pattern search, to find the optimal sequence of decision variables. However, these conventional frameworks are very time-consuming, and it is almost impossible to achieve near real-time optimal control. This paper presents a faster optimization framework by using a new optimal control tool: reinforcement learning. The environment (flow modeler) used in this paper is the numerical model: Environmental Protection Agency's Storm Water Management Model (EPA SWMM) to ensure the accuracy of environment response. The reward function is constructed based on the calculated water depth and overflow rate from SWMM. The process keeps minimizing the reward function to obtain the optimal flow release sequence at each controlled orifice gate. The combined sewer system (CSS) of the Puritan-Fenkell 7-mile facility in Detroit, MI, is chosen as the case study.
