Continual Multi-task Gaussian Processes

Seminar on "Continual Multi-task Gaussian Processes", by Mauricio Alvarez, University of Sheffield.

Title: Continual Multi-task Gaussian Processes

Abstract: We address the problem of continual learning in multi-task Gaussian process (GP) models for handling sequential input-output observations. Our approach extends the existing prior-posterior recursion of online Bayesian inference, i.e. past posterior discoveries become future prior beliefs, to the infinite functional space setting of GP. For a reason of scalability, we introduce variational inference together with a sparse approximation based on inducing inputs. As a consequence, we obtain tractable continual lower-bounds where two novel Kullback-Leibler (KL) divergences intervene in a natural way. The key technical property of our method is the recursive reconstruction of conditional GP priors based on the variational posterior parameters learned so far. To achieve this goal, we introduce a novel factorization of past variational distributions, where the predictive GP equation propagates the posterior uncertainty forward. We then demonstrate that it is possible to derive GP models over many types of sequential observations, either discrete or continuous and that it is also amenable to stochastic optimization. The continual inference approach is applicable to scenarios where potential multi-channel or heterogeneous observations might appear. Extensive experiments demonstrate that the method is fully scalable, shows a reliable performance and is robust to uncertainty error propagation over plenty of synthetic and real-world datasets. This work is based on the arXiv submission https://arxiv.org/abs/1911.00002 [1911.00002] Continual Multi-task Gaussian Processes If you have a disability and are having trouble accessing information on this website or need materials in an alternate format, contact web-accessibility@cornell.edu for assistance.web-accessibility@cornell.edu for assistance. arxiv.org and it is joint work with Pablo Moreno-Muñoz and Antonio Artés-Rodríguez.

Short-bio: Mauricio Álvarez has a PhD degree in Computer Science from The University of Manchester, UK, in 2011. After finishing his Ph.D., Mauricio joined the Department of Electrical Engineering at Universidad Tecnológica de Pereira, Colombia, where he was appointed as a Faculty member until Dec 2016. From Jan 2017, Mauricio joined the Department of Computer Science at the University of Sheffield, where he is currently a Senior Lecturer in Machine Learning. Mauricio's research interests include probabilistic models, kernel methods, and stochastic processes. He works on the development of new approaches and the application of Machine Learning in areas that include neural engineering, systems biology, and humanoid robotics.