Transfer learning on transformers for building energy consumption forecasting—A comparative study

dc.citation.volume336
dc.contributor.authorSpencer R
dc.contributor.authorRanathunga S
dc.contributor.authorBoulic M
dc.contributor.authorvan Heerden AH
dc.contributor.authorSusnjak T
dc.date.accessioned2025-04-02T01:48:51Z
dc.date.available2025-04-02T01:48:51Z
dc.date.issued2025-06-01
dc.description.abstractEnergy consumption in buildings is steadily increasing, leading to higher carbon emissions. Predicting energy consumption is a key factor in addressing climate change. There has been a significant shift from traditional statistical models to advanced deep learning (DL) techniques for predicting energy use in buildings. However, data scarcity in newly constructed or poorly instrumented buildings limits the effectiveness of standard DL approaches. In this study, we investigate the application of six data-centric Transfer Learning (TL) strategies on three Transformer architectures—vanilla Transformer, Informer, and PatchTST—to enhance building energy consumption forecasting. Transformers, a relatively new DL framework, have demonstrated significant promise in various domains; yet, prior TL research has often focused on either a single data-centric strategy or older models such as Recurrent Neural Networks. Using 16 diverse datasets from the Building Data Genome Project 2, we conduct an extensive empirical analysis under varying feature spaces (e.g., recorded ambient weather) and building characteristics (e.g., dataset volume). Our experiments show that combining multiple source datasets under a zero-shot setup reduces the Mean Absolute Error (MAE) of the vanilla Transformer model by an average of 15.9 % for 24 h forecasts, compared to single-source baselines. Further fine-tuning these multi-source models with target-domain data yields an additional 3–5 % improvement. Notably, PatchTST outperforms the vanilla Transformer and Informer models. Overall, our results underscore the potential of combining Transformer architectures with TL techniques to enhance building energy consumption forecasting accuracy. However, careful selection of the TL strategy and attention to feature space compatibility are needed to maximize forecasting gains.
dc.description.confidentialfalse
dc.description.noteskeywords: Building energy consumption forecasting, Transfer learning for time series, Transformer models for time series forecasting, Data-centric transfer learning strategies, PatchTST, Informer, Zero-shot learning, Model fine-tuning, Data scarcity
dc.identifier.citationSpencer R, Ranathunga S, Boulic M, van Heerden AH, Susnjak T. (2025). Transfer learning on transformers for building energy consumption forecasting—A comparative study. Energy and Buildings. 336.
dc.identifier.doi10.1016/j.enbuild.2025.115632
dc.identifier.eissn1872-6178
dc.identifier.elements-typejournal-article
dc.identifier.issn0378-7788
dc.identifier.number115632
dc.identifier.piiS0378778825003627
dc.identifier.urihttps://mro.massey.ac.nz/handle/10179/72720
dc.languageEnglish
dc.publisherElsevier B V
dc.publisher.urihttps://www.sciencedirect.com/science/article/pii/S0378778825003627
dc.relation.isPartOfEnergy and Buildings
dc.rights(c) The author/sen
dc.rights.licenseCC BYen
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/en
dc.subjectBuilding energy consumption forecasting
dc.subjectTransfer learning for time series
dc.subjectTransformer models for time series forecasting
dc.subjectData-centric transfer learning strategies
dc.subjectPatchTST
dc.subjectInformer
dc.subjectZero-shot learning
dc.subjectModel fine-tuning
dc.subjectData scarcity
dc.titleTransfer learning on transformers for building energy consumption forecasting—A comparative study
dc.typeJournal article
pubs.elements-id500233
pubs.organisational-groupOther
Files
Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
500233 PDF.pdf
Size:
4.54 MB
Format:
Adobe Portable Document Format
Description:
Published version.pdf
License bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
9.22 KB
Format:
Plain Text
Description:
Collections