Papers

Amortising Variational Bayesian Inference over prior hyperparameters with a Normalising Flow

L Battaglia, G Nicholls - arXiv preprint arXiv:2412.16419, 2024 - arxiv.org
Statistics paper stat.CO Suggest

… Simulation Based Inference has been used to amortise … However, there is no Simulation Based Inference for … They are interested in simulation-based inference, where the …

Link to paper

BibTeX

@article{2412.16419v2,
Author        = {Laura Battaglia and Chris U. Carmona and Ross A. Haines and Max Anderson Loake and Michael Benskin and Geoff K. Nicholls},
Title         = {Amortising over hyperparameters in Generalised Bayesian Inference},
Eprint        = {2412.16419v2},
ArchivePrefix = {arXiv},
PrimaryClass  = {stat.CO},
Abstract      = {In Bayesian inference prior hyperparameters are chosen subjectively or estimated using empirical Bayes methods. Generalised Bayesian Inference (GBI) also has a learning rate hyperparameter. This is compounded in Semi-Modular Inference (SMI), a GBI framework for multiple datasets (multi-modular problems). As part of any GBI workflow it is necessary to check sensitivity to the choice of hyperparameters, but running MCMC or fitting a variational approximation at each of the hyperparameter values of interest is impractical. Simulation-based Inference has been used by previous authors to amortise over data and hyperparameters, fitting a posterior approximation targeting the forward-KL divergence. However, for GBI and SMI posteriors, it is not possible to amortise over data, as there is no generative model. Working with a variational family parameterised by a conditional normalising flow, we give a direct variational approximation for GBI and SMI posteriors, targeting the reverse-KL divergence, and amortised over prior and loss hyperparameters at fixed data. This can be sampled efficiently at different hyperparameter values without refitting, and supports efficient robustness checks and hyperparameter selection. We show that there exist amortised conditional normalising-flow architectures which are universal approximators. We illustrate our methods with an epidemiological example well known in SMI work and then give the motivating application, a spatial location-prediction task for linguistic-profile data. SMI gives improved prediction with hyperparameters chosen using our amortised framework. The code is available online.},
Year          = {2024},
Month         = {Dec},
Url           = {http://arxiv.org/abs/2412.16419v2},
File          = {2412.16419v2.pdf}
}

Share