Adapting Multiverse Analysis for Prediction: A Decision-Maker’s Dashboard

This paper presents a novel framework that adapts multiverse analysis, traditionally used for robustness in statistical inference to the predictive modeling domain, focusing on the problem of predictive inconsistency. The authors develop an interactive dashboard that allows decision-makers to visualize and interpret how different modeling pipeline choices (e.g., preprocessing, feature selection, resampling, model type) influence predicted outcomes for the same individual. The system integrates three components: (1) interactive specification curves for probabilistic predictions, (2) a regression-tree-based explainer that identifies which modeling decisions drive outcome variability, and (3) counterfactual comparison tools to assess fairness implications across demographic profiles. A recidivism prediction case study illustrates how such visualization can expose methodological arbitrariness and support more informed decision-making