Papers

Fuse It or Lose It Deep Fusion for Multimodal Simulation-Based Inference

M Schmitt, ST Radev, PC Bürkner - arXiv preprint arXiv:2311.10671, 2023 - arxiv.org
Education paper cs.LG Suggest

… the flexibility of MultiNPE as a novel simulationbased inference tool for real-world … it pushes the boundaries of modern simulationbased inference with neural networks and …

Link to paper

BibTeX

@article{2311.10671v3,
Author        = {Marvin Schmitt and Leona Odole and Stefan T. Radev and Paul-Christian Bürkner},
Title         = {Fuse It or Lose It: Deep Fusion for Multimodal Simulation-Based
  Inference},
Eprint        = {2311.10671v3},
ArchivePrefix = {arXiv},
PrimaryClass  = {cs.LG},
Abstract      = {We present multimodal neural posterior estimation (MultiNPE), a method to
integrate heterogeneous data from different sources in simulation-based
inference with neural networks. Inspired by advances in deep fusion, it allows
researchers to analyze data from different domains and infer the parameters of
complex mathematical models with increased accuracy. We consider three fusion
approaches for MultiNPE (early, late, hybrid) and evaluate their performance in
three challenging experiments. MultiNPE not only outperforms single-source
baselines on a reference task, but also achieves superior inference on
scientific models from cognitive neuroscience and cardiology. We systematically
investigate the impact of partially missing data on the different fusion
strategies. Across our experiments, late and hybrid fusion techniques emerge as
the methods of choice for practical applications of multimodal simulation-based
inference.},
Year          = {2023},
Month         = {Nov},
Url           = {http://arxiv.org/abs/2311.10671v3},
File          = {2311.10671v3.pdf}
}

Share