Bandlimited functions in Machine Learning

Abstract

There has been an increasing interest in the study of Gaussian Processes in the machine learning literature, both for the increase in computational power in the recent years and also the development of new algorithms that reduce the computational complexity to train a Gaussian Process model. In particular, there has been an special interest in its connections with the classical signal processing literature, and its applications in studying the spectrum of time series, more specifically band-limited time series, as it is the main object of interest in signal processing. One of the advantages of using Gaussian process in signal processing is that its probabilistic nature can naturally handle problems regarding irregular sampled points and, not only that, it allow us to study the phenomenon of irregular sampling to our advantage, as, theoretically, when sampling irregularly it is possible to obtain better results by reducing the alias error that regular sampling is often associated with. In the first chapter we will review the basic theory regarding Fourier analysis, classical sampling theory and Gaussian processes, as well as the connections between the two, including some results which will be useful in order to understand said connections. In the second chapter, we will discuss the Nyquist-Shannon frequency and irregular sampling in order to understand the possible problems as well as the advantages given by it, in particular, we will study some known sampling schemes as well as some theoretic results. Lastly, we will review some applications regarding Gaussian Processes and band-limited kernels. First we will develop a filtering model by developing a generative model, which can handle irregular sampled points. Next, it will be shown how different irregular sampling techniques actually work on practice using Gaussian Process regression, and what can be gained by them. Following this, we present an algorithm for finding an optimal sampling scheme of a signal based on Bayesian optimization. At the end, an application using band-limited functions for a compressed sensing problem is presented, which is what will be, hopefully, the next step of this research.