I am interested in Bayesian statistics, machine learning and deep learning. In particular I am working on large scale Bayesian machine learning. Most of my research so far has been on distributed Bayesian learning using stochastic natural gradient expectation propagation. I am also interested in stochastic gradient MCMC methods and variational inference. Occasionally I blog about my work.

X Lu,
V Perrone,
L Hasenclever,
Y W Teh,
S J Vollmer,
Relativistic Monte Carlo, in AISTATS, 2017.

Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCMC) algorithm that generates proposals for a Metropolis-Hastings algorithm by simulating the dynamics of a Hamiltonian system. However, HMC is sensitive to large time discretizations and performs poorly if there is a mismatch between the spatial geometry of the target distribution and the scales of the momentum distribution. In particular the mass matrix of HMC is hard to tune well. In order to alleviate these problems we propose relativistic Hamiltonian Monte Carlo, a version of HMC based on relativistic dynamics that introduce a maximum velocity on particles. We also derive stochastic gradient versions of the algorithm and show that the resulting algorithms bear interesting relationships to gradient clipping, RMSprop, Adagrad and Adam, popular optimisation methods in deep learning. Based on this, we develop relativistic stochastic gradient descent by taking the zero-temperature limit of relativistic stochastic gradient Hamiltonian Monte Carlo. In experiments we show that the relativistic algorithms perform better than classical Newtonian variants and Adam.

@inproceedings{LuPerHas2016a,
annote = {ArXiv e-prints: 1609.04388},
author = {Lu, X and Perrone, V and Hasenclever, L and Teh, Y W and Vollmer, S J},
booktitle = {AISTATS},
title = {{Relativistic Monte Carlo}},
year = {2017}
}

2016

L. Hasenclever,
S. Webb,
T. Lienart,
S. Vollmer,
B. Lakshminarayanan,
C. Blundell,
Y. W. Teh,
Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server, 2016.

This paper makes two contributions to Bayesian machine learning algorithms. Firstly, we propose stochastic natural gradient expectation propagation (SNEP), a novel alternative to expectation propagation (EP), a popular variational inference algorithm. SNEP is a black box variational algorithm, in that it does not require any simplifying assumptions on the distribution of interest, beyond the existence of some Monte Carlo sampler for estimating the moments of the EP tilted distributions. Further, as opposed to EP which has no guarantee of convergence, SNEP can be shown to be convergent, even when using Monte Carlo moment estimates. Secondly, we propose a novel architecture for distributed Bayesian learning which we call the posterior server. The posterior server allows scalable and robust Bayesian learning in cases where a dataset is stored in a distributed manner across a cluster, with each compute node containing a disjoint subset of data. An independent Monte Carlo sampler is run on each compute node, with direct access only to the local data subset, but which targets an approximation to the global posterior distribution given all data across the whole cluster. This is achieved by using a distributed asynchronous implementation of SNEP to pass messages across the cluster. We demonstrate SNEP and the posterior server on distributed Bayesian learning of logistic regression and neural networks.

@unpublished{HasWebLie2015a,
author = {Hasenclever, L. and Webb, S. and Lienart, T. and Vollmer, S. and Lakshminarayanan, B. and Blundell, C. and Teh, Y. W.},
note = {ArXiv e-prints: 1512.09327},
title = {Distributed {B}ayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server},
year = {2016}
}

2014

W. Hordijk,
L. Hasenclever,
J. Gao,
D. Mincheva,
J. Hein,
An investigation into irreducible autocatalytic sets and power law distributed catalysis, Natural Computing, vol. 13, no. 3, pp. 287–296, 2014.

RAF theory has been established as a useful and formal framework for studying the emergence and evolution of autocatalytic sets. Here, we present several new and additional results on RAF sets. In particular, we investigate in more detail the existence, expected sizes, and composition of the smallest possible, or irreducible, RAF sets. Furthermore, we study a more realistic variant of the well-known binary polymer model in which the catalysis events are assigned according to a power law distribution. Together, these results provide further insights into the existence and structure of autocatalytic sets in simple models of chemical reaction systems, with possible implications for theories on the origin of life.

@article{Hordijk2014,
author = {Hordijk, W. and Hasenclever, L. and Gao, J. and Mincheva, D. and Hein, J.},
doi = {10.1007/s11047-014-9429-6},
issn = {1572-9796},
journal = {Natural Computing},
number = {3},
pages = {287--296},
title = {{An investigation into irreducible autocatalytic sets and power law distributed catalysis}},
volume = {13},
year = {2014}
}

2013

S. S. Pegler,
K. N. Kowal,
L. Hasenclever,
M. G. Worster,
Lateral controls on grounding-line dynamics, Journal of Fluid Mechanics, vol. 722, no. 5929, p. R1, May 2013.

We present a theoretical and experimental study of viscous gravity currents introduced at the surface of a denser inviscid fluid layer of finite depth inside a vertical Hele-Shaw cell. Initially, the viscous fluid floats on the inviscid fluid, forming a self-similar, buoyancy-driven current resisted predominantly by the viscous stresses due to shear across the width of the cell. Once the viscous current contacts the base of the cell, the flow can be considered in two regions: a grounded region in which the current lies in full contact with the base; and a floating region. The subsequent advance of the grounding line separating these regions is shown to be controlled by the thickening of the current associated with balancing the local shear stresses. An understanding of the flow transitions is developed using asymptotic and numerical analysis of a model based on lubrication theory.

@article{Pegler2013,
author = {Pegler, S. S. and Kowal, K. N. and Hasenclever, L. and Worster, M. G.},
doi = {10.1017/jfm.2013.140},
file = {:home/leonard/.local/share/data/Mendeley Ltd./Mendeley Desktop/Downloaded/Pegler et al. - 2013 - Lateral controls on grounding-line dynamics.pdf:pdf},
issn = {0022-1120},
journal = {Journal of Fluid Mechanics},
keywords = {Hele-Shaw flows,geophysical and geological flows,ice sheets},
month = may,
number = {5929},
pages = {R1},
publisher = {Cambridge University Press},
title = {{Lateral controls on grounding-line dynamics}},
volume = {722},
year = {2013}
}