Papers

The Importance of Being Adaptable An Exploration of the Power and Limitations of Domain Adaptation for Simulation-Based Inference with Galaxy Clusters

M Ntampaka, A Ciprijanovic, AM Delgado… - arXiv preprint arXiv …, 2025 - arxiv.org
Astrophysics paper astro-ph.IM Suggest

… challenge for simulation-based inference, where models … scientific case: simulation-based inference for estimating … sets to mimic simulation-based inference: a Training Set …

Link to paper

BibTeX

@article{2510.09748v1,
Author        = {Michelle Ntampaka and A. Ciprijanovic and Ana Maria Delgado and John Soltis and John F. Wu and Mikaeel Yunus and John ZuHone},
Title         = {The Importance of Being Adaptable: An Exploration of the Power and
  Limitations of Domain Adaptation for Simulation-Based Inference with Galaxy
  Clusters},
Eprint        = {2510.09748v1},
ArchivePrefix = {arXiv},
PrimaryClass  = {astro-ph.IM},
Abstract      = {The application of deep machine learning methods in astronomy has exploded in
the last decade, with new models showing remarkably improved performance on
benchmark tasks. Not nearly enough attention is given to understanding the
models' robustness, especially when the test data are systematically different
from the training data, or "out of domain." Domain shift poses a significant
challenge for simulation-based inference, where models are trained on simulated
data but applied to real observational data. In this paper, we explore domain
shift and test domain adaptation methods for a specific scientific case:
simulation-based inference for estimating galaxy cluster masses from X-ray
profiles. We build datasets to mimic simulation-based inference: a training set
from the Magneticum simulation, a scatter-augmented training set to capture
uncertainties in scaling relations, and a test set derived from the
IllustrisTNG simulation. We demonstrate that the Test Set is out of domain in
subtle ways that would be difficult to detect without careful analysis. We
apply three deep learning methods: a standard neural network (NN), a neural
network trained on the scatter-augmented input catalogs, and a Deep
Reconstruction-Regression Network (DRRN), a semi-supervised deep model
engineered to address domain shift. Although the NN improves results by 17% in
the Training Data, it performs 40% worse on the out-of-domain Test Set.
Surprisingly, the Scatter-Augmented Neural Network (SANN) performs similarly.
While the DRRN is successful in mapping the training and Test Data onto the
same latent space, it consistently underperforms compared to a straightforward
Yx scaling relation. These results serve as a warning that simulation-based
inference must be handled with extreme care, as subtle differences between
training simulations and observational data can lead to unforeseen biases
creeping into the results.},
Year          = {2025},
Month         = {Oct},
Url           = {http://arxiv.org/abs/2510.09748v1},
File          = {2510.09748v1.pdf}
}

Share