Ruiyu Li

We present a heat load forecasting method for district heating systems that aims to facilitate its regulation. Unlike traditional time-series prediction-based approaches, we address the problem with the Transformer, a recent breakthrough in deep learning that leverages an attention mechanism to identify recurring patterns in the input sequence regardless of their distance. Because the heat load state at a time point is highly dependent on its preceding states, we adopt a simple but effective locality enhancement method to boost the local context information. Using a simulated heating system modeled by MATLAB/Simulink software, we demonstrate that the Transformer-based heat load forecasting approach can achieve higher accuracy than classic RNN models such as LSTM. Once equipped with a locality enhancement mechanism, it can adapt to short-term heat load fluctuations swiftly and precisely.

Online multi-task learning (OMTL) enhances streaming data processing by leveraging the inherent relations among multiple tasks. It can be described as an optimization problem in which a single loss function is defined for multiple tasks. Existing gradient-descent-based methods for this problem might suffer from gradient vanishing and poor conditioning issues. Furthermore, the centralized setting hinders their application to online parallel optimization, which is vital to big data analytics. Therefore, this study proposes a novel OMTL framework based on the alternating direction multiplier method (ADMM), a recent breakthrough in optimization suitable for the distributed computing environment because of its decomposable and easy-to-implement nature. The relations among multiple tasks are modeled dynamically to fit the constant changes in an online scenario. In a classical distributed computing architecture with a central server, the proposed OMTL algorithm with the ADMM optimizer outperforms SGD-based approaches in terms of accuracy and efficiency. Because the central server might become a bottleneck when the data scale grows, we further tailor the algorithm to a decentralized setting, so that each node can work by only exchanging information with local neighbors. Experimental results on a synthetic and several real-world datasets demonstrate the efficiency of our methods.