When forecasting time series data, exogenous covariates can provide the model with additional information that cannot be obtained from the time series alone. Since the number of multivariate foundation models for time series is limited, this thesis focuses on extending the Time-MoE model with exogenous covariates. On the dataset of groundwater level measurements, we tested several approaches to augment the Time-MoE model: sequential and parallel ensembles, using the last hidden layer of the foundation model and directly inputting exogenous covariates into the prediction head. Among the methods tested, the best-performing approaches were ensembles that combined statistical models with the Time-MoE model. The most successful among them achieved an average R2 value of 0.672, while the best approach using the last layer achieved an average value of 0.607, and the approach with direct input 0.500. Compared to existing univariate methods, the developed methods achieved better results, primarily due to access to weather data. Compared to existing multivariate methods, however, the differences looked negligible, indicating that Time-MoE combined with weather data does not provide additional advantages on the dataset considered. Nevertheless, the advantage of the developed approaches is their modularity, as the foundational model can be replaced if needed.
|
