Nonstationary multi-output gaussian processes via harmonizable spectral mixtures

Abstract

In several disciplines, Gaussian Processes (GPs)\cite{williams2006gaussian} are the gold standard for modelling time series or functions in general, especially in cases where modelling uncertainty is required. The extension of GP to handle multiple outputs is known as multi-output Gaussian processes (MOGP)\cite{williams2007multi}, which, by modelling the outputs as jointly Gaussian, is able to share information across outputs, potentially improving the estimation. Both, the single output and multiple output GP are entirely determined by a covariance function. Kernel design MOGP has received increased attention recently, in particular, the Multi-Output Spectral Mixture kernel (MOSM) \cite{parra2017spectral} approach has been praised as a general model in the sense that it extends other approaches such as Linear Model of Coregionalization \cite{goovaerts1997geostatistics}, Intrinsic Coregionalization Model \cite{goovaerts1997geostatistics} and Cross-Spectral Mixture \cite{ulrich2015gp}. MOSM relies on Cramér s theorem \cite{cramer1940theory} to parametrize the power spectral densities (PSD) as a Gaussian mixture, thus, having a structural restriction: by assuming the existence of a PSD, the method is only suited for stationary processes. The main purpose of this thesis is to extend the MOSM model to make it suitable for non-stationary processes. To address this, we propose the multi-output harmonizable spectral mixture (MOHSM) kernel, an expressive and flexible family of MOGP kernels to model non-stationary processes as a natural extension of the MOSM. which relies on the concept of harmonizability, a term introduced by Loève \cite{loeve1978probability} which generalizes the spectral representations to non-stationary processes. The proposed MOHSM is able to model both stationary and non-stationary processes while maintaining the desires properties of the MOSM: a clear interpretation of the parameters, from a spectral viewpoint, and flexibility in each channel. We also propose a data-driven heuristics for the initial point in the optimization, in order to improve the model training. In addition, we also present a variation of the model, which open the door to consider more more general spectral densities. The proposed method is first validated on synthetic data with the purpose of illustrating the key properties of our approach, and then compared to existing MOGP methods on two real-world settings from finance and electroencephalography.